Meta-analyses and systematic reviews

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

chicoborja

Clinical Audiologist
15+ Year Member
20+ Year Member
Joined
Nov 13, 2002
Messages
203
Reaction score
0
How are meta-analyses and systematic reviews viewed compared to original research publications within academia? On the one hand, it seems that they are highly valued in terms of evidence based practice. However, on the other hand I can see researchers as viewing them as weak since there is no raw data collection in the traditional sense. Could someone get tenure if they simply publish all meta-analysts and systematic reviews with very little original research?

Members don't see this ad.
 
Things like meta-analyses are not seen as being as good as basic science or clinical research because there's no hypothesis. You're not testing anything, you're distilling others' work. I mean, what are your controls?

All of the ones I've read are written by residents fufilling their "scholarly activity" requirement (not to knock on them, god willing, I'll be one some day). You can't base an academic career on them. They are not bad if you're learning how to write articles and finding out about the submission process.
 
Things like meta-analyses are not seen as being as good as basic science or clinical research because there's no hypothesis.

Personal opinion or where did you hear this? I think it really depends on the field and the topic. Yeah, an MSTP is going to expect something more from you like a bench research or a "from scratch" clinical project, but I think if you're doing a Cochrane review or a large scale meta-analysis of an important topic, it would looked upon positively. Having done both clinical research from ground up and meta-analysis, in some ways the meta-analyses are more difficult.

In the interest of full disclosure, a fellow SDN member and myself (I am the lead author) are currently working on a Cochrane review so my opinion may be skewed by that fact.
 
Members don't see this ad :)
A lot of meta-analyses are done by residents. The quality of resident research is highly variable. I've read a bunch of it, I've conducted it, and I've co-authored it. Often they're too busy to care (which is OK - their job is a little something called saving lives!). I see "review" and I think resident. This is also seen as "data mining," or re-using data in a way that may, or may not be, similar to how it was in the original study. It's a method of analysis that's easy to abuse. My experience with the generally poor quality of these studies is the basis of my thoughts.

Systematic definitely reviews have purpose. Each chapter of the Mont Reid Surgical Handbook details a specific area of general surgery and is written by residents. Residents summarize all of the relevant literature and become an expert on their chapter. They make great tools for learning and help the field. Maybe if you're the chief of surgery, and you've published the last 5 editions of a text, yeah it'd help your career in academia. But how did you become chief in the first place?

On hypotheses and meta analyses: What are you testing DKM? You're figuring out the "gestalt" that sums up what a bunch of people found out about a particular topic. But there's no testing. No controls. You might have exclusion criteria (e.g. "If a study was found to have insufficient power, the authors did not pool that study's data into the final analysis"), but there's no such thing as a positive or negative control in a meta analysis.

I have no idea what you're writing about DKM, and I have not read your work. I cannot judge what you've done, and my comments were not directed at your work. The OP asked for general opinions on relative stature of metas and whether it was possible to build an academic career solely upon them. I've never seen or heard of anyone doing this, and I would think it highly unlikely to do so.
 
A lot of meta-analyses are done by residents. The quality of resident research is highly variable. I've read a bunch of it, I've conducted it, and I've co-authored it. Often they're too busy to care (which is OK - their job is a little something called saving lives!). I see "review" and I think resident. This is also seen as "data mining," or re-using data in a way that may, or may not be, similar to how it was in the original study. It's a method of analysis that's easy to abuse. My experience with the generally poor quality of these studies is the basis of my thoughts.

Systematic definitely reviews have purpose. Each chapter of the Mont Reid Surgical Handbook details a specific area of general surgery and is written by residents. Residents summarize all of the relevant literature and become an expert on their chapter. They make great tools for learning and help the field. Maybe if you're the chief of surgery, and you've published the last 5 editions of a text, yeah it'd help your career in academia. But how did you become chief in the first place?

On hypotheses and meta analyses: What are you testing DKM? You're figuring out the "gestalt" that sums up what a bunch of people found out about a particular topic. But there's no testing. No controls. You might have exclusion criteria (e.g. "If a study was found to have insufficient power, the authors did not pool that study's data into the final analysis"), but there's no such thing as a positive or negative control in a meta analysis.

I have no idea what you're writing about DKM, and I have not read your work. I cannot judge what you've done, and my comments were not directed at your work. The OP asked for general opinions on relative stature of metas and whether it was possible to build an academic career solely upon them. I've never seen or heard of anyone doing this, and I would think it highly unlikely to do so.
Points taken....my comment stemmed solely from the fact that it took less time for us to build a database of 521 subjects (as of 15 minutes ago) with about 290 data points each based on autopsy findings and related documentation than it did to conduct a meta-analysis based on literature review of different studies in other journals for another project. To me (and this is just my opinion) a meta-analysis can be a huge pain in the ass to conduct if done properly. In fact, it almost should be....if it isn't, you might want to question how thorough and productive you are being.

I will admit though that it is easily (and frequently) abused and a lot of the meta-analyses out there are simply the product of someone trying to fulfill the requirements of their program (be that a residency or some manner of grad school).
 
In the UK, I believe they require a systematic review to accompany any research proposal.

Our IRB chair jokes they only do that to give them the chance to increase their publication count.......but we both think the real reasoning behind it is a good idea.

In fact, it is really boring work (I have written two Cochrane reviews).
I was just chosen to do a Cochrane review......you're right, this is going to be mind numbing. Oh well, that's why I get for being crazy enough to volunteer for it.
 
Top