Hi all. I wanted to share a few things that struck me about this article, but was on very loud screechy BART/muni trains. So I'll share here. Thank you to Laura for doing an amazing job facilitating :)
It took me a while to get my head around their study aim. Their aim was to look at predictors of drug-placebo response instead of those of the placebo response only (hence the meta-regression). I don't quite understand why the previous meta-analysis they refer to (reference 9) only looked at the predictors of the placebo response. Can anyone explain that to me?
In thinking about Chinese study exclusion, my initial thought was the same as Matt. Unfortunately, I am seeing this more and more as a general practice, and it's unfortunate that some fraudulent studies are causing us to doubt the rigor of all Chinese studies. I looked at this group's references for this exclusion practice, and they seem legit -- one from BMJ and one from the Cochrane colloquium. see below
- Wu TX, Li YP, Liu GJ, et al: Investigation of authenticity of ‘claimed’ randomized controlled trials (RCTs) and quality assessment of RCT reports published in China. Abstracts of the 14th Cochrane Colloquium, Dublin, Ireland, 2006. https://abstracts.cochrane.org/2006-dublin/investigation-authenticity-claimed-randomizedcontrolled-trials-rcts-and-quality
- Woodhead M: 80% of China’s clinical trial data are fraudulent, investigation finds. BMJ 2016; 355:i5396
I was as surprised as they were about their finding that studies pharma companies produce smaller effect sizes, but they make a good point in discussing the nature of the standardized mean difference (SMD=mean difference/standard deviation). As the sample sizes increase the standard deviation gets smaller (because it is a function of n, the sample size). Even if you had a very small difference between placebo and controls, with a small enough standard deviation you can achieve a small effect size. As they say, the companies have the resources to recruit large sample sizes, which necessarily gives them a higher likelihood of getting a significant result. They could basically recruit and recruit until a publishable result was assured. I wonder if there are enough unsponsored studies to reproduce this meta-analysis and exclude the pharma studies?
As a (future)provider, these considerations are an important part assessing evidence, especially when making changes to practice based on research. I really liked how this article spelled it out for us.
Lastly, I was pretty happy they discussed outcomes related to social integration, because I think its a very important part of recovery and quality of life for people with both positive and negative sx. Only 10 trials out of 167 trials had this as an outcome! I was glad they called for more studies measuring social integration and social experience.