Appraising 'evidence-based' books?

The recent post by Tracy Allison Altman showed that the title of a book is often misleading. She showed that the word ‘evidence-based’ becomes meaningless if three books with this term in the title deal with totally different subjects. It appears that the ‘evidence-based approach’ starts to become the flavor of the month. Just yesterday I discovered a book called Management: an evidence-based approach. But how can you tell what it really is about? Especially if these books are quite expensive. Well, if you an editor you can order a copy for appraisal from the publisher, but that is not enough for the rest of us. What are actually some agreed upon metrics?

Evidence-based management, what is the current definition?
Before you  start measuring things, you need to explain what you mean by evidence-based management. On this site, we use the following definition by Rousseau, Briner and Denyer, 2009: “Evidence-based management is about making decisions through the conscientious, explicit, and judicious use of four sources of information: practitioner expertise and judgment, evidence from the local context, a critical evaluation of the best available research evidence, and the perspectives of those people who might be affected by the decision.”

Levels of evidence
Just reading the definition, the pyramid with the levels of evidence (like the one in use by medicine) is not sufficient. Just putting a symbol D or level III evidence (indicating expert opinion) won’t help us. Most people don’t know what that means.

Rigor and relevance
If you look for systematic reviews in management in ABI/INFORM, you probably won’t find much. So, methodological rigor is very important, but is it still relevant to practice? I currently have the impression that the terms are mutually exclusive. But, that will hopefully change in the future. Even the fact if management is a profession is an active discussion. But there is also good news. At the recent evidence-based management caucus at the Academy of Management in Montreal, 5 people announced ‘systematic reviews’. Maybe we can follow a few of these projects on this website.

A suggestion for appraisal

What type(s) of evidence is/are pre-dominantly used in the book?

Practitioner judgement and expertise (practitioner evidence)

Evidence from a local context| (situational evidence)

Critical appraisal of the best available research evidence (C.A.T.) (maybe indicating the level of evidence with a C or a D)

Perspectives from people affected by the decision (group ‘evidence’)

Target audience: beginner, intermediate or expert.

And probably a review of the book on this blog, but that will take some time and effort.

Note: this is probably not enough, but I would like to hear some suggestions for improving the appraisal.