Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Sarah Dobbins -
Number of replies: 12

Hi all. I wanted to share a few things that struck me about this article, but was on very loud screechy BART/muni trains. So I'll share here. Thank you to Laura for doing an amazing job facilitating :)

It took me a while to get my head around their study aim. Their aim was to look at predictors of drug-placebo response instead of those of the placebo response only (hence the meta-regression). I don't quite understand why the previous meta-analysis they refer to (reference 9) only looked at the predictors of the placebo response. Can anyone explain that to me?

In thinking about Chinese study exclusion, my initial thought was the same as Matt. Unfortunately, I am seeing this more and more as a general practice, and it's unfortunate that some fraudulent studies are causing us to doubt the rigor of all Chinese studies. I looked at this group's references for this exclusion practice, and they seem legit -- one from BMJ and one from the Cochrane colloquium. see below

  • Woodhead M: 80% of China’s clinical trial data are fraudulent, investigation finds. BMJ 2016; 355:i5396

I was as surprised as they were about their finding that studies pharma companies produce smaller effect sizes, but they make a good point in discussing the nature of the standardized mean difference (SMD=mean difference/standard deviation). As the sample sizes increase the standard deviation gets smaller (because it is a function of n, the sample size). Even if you had a very small difference between placebo and controls, with a small enough standard deviation you can achieve a small effect size. As they say, the companies have the resources to recruit large sample sizes, which necessarily gives them a higher likelihood of getting a significant result. They could basically recruit and recruit until a publishable result was assured. I wonder if there are enough unsponsored studies to reproduce this meta-analysis and exclude the pharma studies?

As a (future)provider, these considerations are an important part assessing evidence, especially when making changes to practice based on research. I really liked how this article spelled it out for us.

Lastly, I was pretty happy they discussed outcomes related to social integration, because I think its a very important part of recovery and quality of life for people with both positive and negative sx. Only 10 trials out of 167 trials had this as an outcome! I was glad they called for more studies measuring social integration and social experience.
In reply to Sarah Dobbins

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Sareen -

I was surprised to find that this was the first meta-analysis of its kind, looking at the efficacy of antispychotics in patients with schizophrenia (including schizoaffective, schizophreniform, and delusional disorder).

Sarah, thanks for pointing out that fact that few studies measured social functioning and quality of life (versus decreasing positive symptoms) since that is what patients are most concerned about. This is a good reminder for us to look at the illness from the patient’s perspective. As we know, few patients I see with schizophrenia are fixated on the hallucinations, for example, but many more are concerned about wanting to do well in their job or maintaining their relationships.

Something that journal club is constantly reminding me of is to pay attention to the population sampled to understand generalizability. These subjects were rarely first time antipsychotic responders, but are more representative of the chronically mentally ill population. It’s another reminder of the slow and often unsatisfying results we get when working with this chronically ill population. We often see some improvement with the use of antipsychotic medication, but not “good” improvement, which can be frustrating to both the patient and the prescriber. 

It was interesting to me that the results showed antipsychotics to help with “any” response but less so with a “good” response in short periods of time. This reminds me that most patients must endure a plethora of side effects for some response, but must wait much longer for a “good” response if it comes at all. It’s also a good reminder to me in terms of managing my expectations. I recently converted a patient from oral risperidone to Invega Sustenna, thinking that poor adherence was the cause of her minimal response to treatment. But even after months of knowing that the patient received her medication, she is still so symptomatic. She is representative of the results of this meta-analysis.

(I wasn’t there for the Journal Club discussion re: the China studies being excluded, but… wow!)

And maybe someone who was there for the journal club can speak to the study aim if you all happened to discuss it?

 

Reference:

Leucht, S., Leucht, C., Huhn, M., Chaimani, A., Mavridis, D. Helfer, B., . . . & Davis, J. M. (2017). Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: Systematic review, Bayesian meta-analysis, and meta-regression of efficacy predictors. American Journal of Psychiatry, 174: 10.


In reply to Sareen

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Jamie Sanders -

Sarah and Sareen,

Thank you both for your comments. I did my comp on interventions for first-episode psychosis so this was an interesting article to look at. The first thing that struck me was that they only looked at placebo-controlled trials which, while a higher level of evidence, does make me question the ethics of the studies. The studies I used in my lit review didn't compare treatment to placebo citing the unethical nature of a study that denies treatment to a subset of subjects so I wonder here what research groups are publishing studies where they're denying medication to a portion of their subjects. If they are randomly assigning patients to either receive or not receive medication that seems highly unethical to me. If they're doing it based on symptomology, indication, patient preference, country of publication or some other factor that seems inevitably to introduce serious bias. From their reference list, some of the placebo studies compare continued use of antipsychotics after remission, so it's not that an antipsychotic is effective or not, it's whether it continues to be effective after someone has been stabilized (Leucht et al., 2017). Ultimately most are meta-analyses of studies conducted from the 1950's to 1980's so when they're citing these results I'm sure they're making conclusions based on redundant results, i.e., multiple meta-analyses looking at the same or overlapping data does not bolster a conclusion, it just makes it appear that there is more evidence than there actually is.

Another point I think that's worth mentioning is that studies included publications from 1955 to 2016, a time period in which antipsychotic medications and treatments have drastically changed so to lump together efficacy of treatment interventions from the 50's until now seems fraught with confounding variables. I think it'd be much more informative to look at more recent studies in which treatments are more in line with clinical practice today.

Another point worth making is that the mean duration of illness was 13.4 years (SD 4.7 years). From the research I've done, early interventions are generally more effective than later ones, newer studies differentiating between long duration of untreated psychosis and short duration of untreated psychosis by measures of weeks and up to 2 years. This has led to, in recent years, a significant effort by clinicians, governments, treatment groups and facilities to identify psychosis early and get individuals into treatment early. If you compared studies from 10, 15, 20 years ago to recent ones you're likely looking at patients with longer durations of untreated illness and poorer outcomes which obscures Leucht et al.'s (2017) results and conclusions. It's also interesting that they conclude that smaller effect size from 1970 (0.74) compared to 2015 (0.38) suggests that antipsychotics are not efficacious, a conclusion I think is unsupported given all the uncontrolled variables.

They comment on pressures of increasing sample size in order to obtain significant results and more publishable results, but I found plenty of studies with relatively small sample sizes that chose them exactly because they wanted to control for extraneous variables and better determine efficacy of interventions in a specific location/geographical area, clinical site, for a specific diagnosis, intervention type, etc, so it's not clear to me what research they're referring to or if they're commenting more generally.  (Ironically their own study has similar challenges with inclusion of many studies with multiple changing and uncontrolled variables despite their recommendation to be more selective to produce more reliable results).

There are also a number of problems with their definitions of outcomes. Like you also pointed out Sareen, they distinguish "any" response from "good" response without defining what that means or differentiating between different types of symptom/measures of response. Negative symptoms are notoriously more difficult and less responsive to treatment, especially antipsychotics. I would say their conclusion that only twice as many patients improved with antipsychotics and of those only a minority had a good response are dubious at best and potentially dangerous.

Finally, like others have said, I think perhaps the most clinically and patient-relevant variable is quality of life. Anecdotally, I've worked with many patients who do not experience full remission from their symptoms and some who experience fairly minimal relief. That said, sometimes medication has helped them enough to become more functional, better engaged socially, less depressed/suicidal and more engaged in life in general. That's, of course, not ideal but sometimes small differences in symptoms make a big difference in quality of life so having something that is more reflective of an individual's QOL would help translate the results into clinically meaningful information.

One last general comment, I think it's interesting to consider expectations. In general, I think people's expectations of medicine have increased significantly over time. From the 1950's until now I'd say there's been a huge increase in what we expect from medical interventions so what was considered a "good" response historically may not be considered a "good" response today.

Leucht, S., Leucht, C., Huhn, M., Chaimani, A., Mavridis, D. Helfer, B., . . . & Davis, J. M. (2017). Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: Systematic review, Bayesian meta-analysis, and meta-regression of efficacy predictors. American Journal of Psychiatry, 174: 10.

In reply to Sarah Dobbins

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Marlene Thompson -

Sarah,

Thanks for kicking off this discussion! And Laura, thank you for stepping up for leading our session--you did a fantastic job!

I really appreciated all of your thoughtful insights, Sarah. I too believe that social integration is an important outcome that deserves more research attention. I also was surprised that only 10 out of the 167 trials focused on this topic.

During our conversation, we discussed how this study had pretty broad inclusion criteria for subjects ("Adults with acute exacerbations of schizophre- via or related disorders (following the Cochrane Schizophrenic Group, we accepted all diagnostic criteria and we also included schizoaffective, schizophreniform, and delusional disorder, because these do not require generally different treatment") and that this may have artificially screwed results. Does it really make sense to talk about antipsychotic drug efficacy when we're discussing so many different conditions? In my opinion, the results of this study would have been more convincing had they focused on a narrower range of conditions (i.e. just schizophrenia).

What does everyone else think about these inclusion criteria? 

In reply to Marlene Thompson

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Julia -

Great question, Marlene :) I was thinking about that too when reading this study. Here are my thoughts: 

I feel that the inclusion of participants with acute psychotic symptoms due to schizophrenia, schizoaffective, schizophreniform and delusional disorder was actually a strength of this review. Too often, research focuses on such specific and limited diagnostic criteria for study inclusion, and while this makes the study design strong it fails to address what we see in real life clinical practice.

Because acute psychotic episodes associated with these various diagnoses are treated similarly with the antipsychotic medications, I feel that including this broader range of diagnoses was not only acceptable but perhaps even more clinically relevant than if the researchers had focused exclusively on schizophrenia. Furthermore, it is often the case that diagnosis of these different psychotic disorders is difficult and individual clients may have multiple diagnoses over their lifetimes. Given that the acute exacerbations under study and the related treatments are pretty much the same, I feel that this review presents information that is clinically relevant and applicable to real world settings. 

Do others agree/disagree? 

In reply to Sarah Dobbins

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Evelyn Cunningham -

Just going to throw in another big THANK YOU to Laura for leading the group discussion this week.  It is a tough task to begin with, made even more daunting by the sheer weight of this article!

I was equally shocked that this analysis was the first of it's kind - just echoes that this field is still right in the middle of figuring out how to treat DSM disorders.  As far as why these studies focused on positive symptoms and not so much on quality of life/ social functioning (in my opinion), I would guess that it has to do with the focus on acute presentation.  I also wish that there were a more objective way to assess Quality of Life....though I guess objective measures wouldn't reflect the client's internal reality, but I digress...  I wonder about the fact that they chose to do a meta-analysis of acute treatment rather than maintenance, as well as including such a broad range of diagnoses, but I wonder if it is more indicative of the availability of the research?  Anyone have any thoughts on this or read the article differently than I did?

Sarah - you also bring up a good point about pharma creating such large sample sizes that their results become smaller, as well as creating such a sheer volume of studies as to find a publishable result.  I have found myself struggling to figure out how I want reps and the pharma companies to play a role in my life.  On one hand, I worry about the influence they can exert over a provider's choice of medication, possibly to the detriment of the patient, but then I also know that their ability to research and provide us with the most up to date research about outcomes on their med could potentially very positively impact practice.  I guess just using what we are learning here in terms of how to critically appraise an article is going to be our best tool - and remembering sample sizes and funding sources....  Also keeping in mind that they can provide us with educational tools, like the ones that Chelsea used in our last class!!

Looking forward to reading everyone else's thoughts on this!  Thanks for your input on the article, even though you weren't able to share in the moment!

In reply to Evelyn Cunningham

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Michelle -

Hi everyone. Just following up here after the journal club. I, too, was listening from my noisy commute home from clinical so I am glad to get a chance to discuss the article in greater detail here. I am very interested in all of your posts so far.

I had never heard of excluding Chinese research and this came as quite a shock to me. When I read the article, I had a negative reaction to the exclusion not understanding the full context. I agree with most of you that it seems very dramatic to exclude all Chinese research instead of just controlling for higher quality research.  

I wanted to ask everyone their thoughts on this research not differentiating between different medications. They mention in their paper that they categorized them all together since efficacy rates seem to be about the same across all medications, except clozaril. Is this something you all have seen before in other studies? Is this something that makes you think twice about this research and the applicability? Thanks again for the great conversation so far. 


In reply to Sarah Dobbins

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Nana Efua Adabie -

Great discussion so far. And yes, special thanks to Laura for facilitating. I had poor service in Maui so I couldn't hear some of the discussion, but I think I got enough to benefit from it. This study was very interesting to read through as I haven't read something like this before with a meta-regression. I was also surprised with their findings from pharma company studies. And you are right, Sara, They have the funds and resources to have a publishable result from whatever study they aim to conduct, which is questionable sometimes because in the end, they aim to benefit for themselves. However, Evelyn also makes a good point with getting the latest data from these pharm companies. We just need to thread lightly along those lines and make sure that we are as unbiased as we can be with the focus on what is best for our patients. 

Another thing I thought about with antipsychotic responses was participants genetic response to these medications, their adherence, and number of medication trials they've had. We know that it gets harder and harder to treat psychosis with every psychotic episode. So someone may be experiencing an acute psychotic episode but may still have a poor response to some of these antipsychotics due to their history. They may not have a good response but could still experience side effects. 

In regards to the exclusion of Chinese studies, I agree with most of you on here. I have come across and cited some very impressive Chinese studies, but there is always that questioning of if it is legitimate to use. It's almost become like the story of the man who cried wolf. 

In reply to Nana Efua Adabie

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Marlene Thompson -

Colleagues and future NPs, 

Great points thus far. One aspect of this study that I still wonder about (as I do for most drug efficacy studies) is their assessment of publication bias. They mention that they used a statistical software (called "contour-enhanced funnel plots") in order to detect whether or not small study sizes were indicative of publication bias. I wish they had explained this fact more--what do they mean by this? They do not touch on this notion later. Despite this vagueness, it seems likely that many studies within this meta-analysis may have suffered from publication bias since many were funded by big pharma (that has no incentive to report ineffective outcomes).

I appreciated the authors' discussion about the difference in reported efficacy for the same drugs between 1970 (before the BPRS assessment tool and standardized diagnostic tools were available) and 2015. This points to the flawed reality of pharmaceutical research--it is biased toward reporting greater effects. In other words, researchers may be biased to advertise their drug in the literature as a panacea. This highlights the need for psychiatric providers to be diligent in consuming research and making informed treatment decisions (rather than being swooned by drug reps).

Similar to Evelyn and Nana, I look forward to future studies that focus directly on measuring antipsychotics' effects on individuals' social reintegration. Indeed, schizophrenia is a complex and devastating disease. This study serves to underscore the limitations of our current meds and the importance of supporting those that do not experience adequate symptom relief.

In reply to Marlene Thompson

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Matthew Settle -

Hi All,


Just getting to respond to this post now. Thanks all for a great discussion and thanks Laura for facilitating. One thing that I wanted to discuss more during our meeting, but had difficulties with reception while I was walking, was the fact that Chinese studies were excluded. As unfortunate as this is, China has developed a reputation as a hotbed of dishonest published research. My partner used to work at PLOS (Public Library of Science), and the inside scoop there is that PLOS One (the open access journal available to anyone) is constantly barraged with research from China that is completely bogus, including listing well known researchers as authors who have had nothing to do with the study. This may be an unfortunate side effect of the need to “publish or perish,” combined with a culture that has a different take on the counterfeiting of intellectual property. Please keep in mind that this in no way reflects the whole culture, but is a problem that is well known in the research community. 

 

I really appreciate having been exposed to this article. Antipsychotics are something that we use a lot, and, as we all know, are not something to be taken lightly. I’m thankful for the opportunity given to us to practice critical appraisal in this journal club, and for the input from all of you bright, crictically thinking individuals, as there have been many perspectives brought up that I would not have considered otherwise.


In reply to Sarah Dobbins

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Julia -

Hi All, 

Thank you for another good discussion and thank you, Laura, for facilitating so well! 

As I read this article I had a few thoughts that I wanted to share but was unable to since I was driving back from clinical during the session. 

First of all, I wanted to highlight some of the strengths and weaknesses  that I noticed. 

One major strength was that that this review looked at "any" response versus "good" response to the antipsychotic medications under study. As the authors explained, far too often individual studies only evaluate "any" response to a medication which is VERY different than a "good" response, especially in light of the serious side effects associated with antipsychotics. Similarly, this review's focus on quality of life and functioning was a strength, as these are incredibly important outcomes that are often overlooked in medication trials, as Sarah and Sareen mentioned in their posts. "Good" vs "any" response and the quality of life outcomes are some of the most clinically important pieces to consider when prescribing medications, so the fact that this review examined them is a real strength. 

 Another strength of this study was that the authors were able to consistently use the PANSS or the BPRS to measure primary outcomes for all but one study within their meta-analysis, adding to the consistency of their study. I also liked that the authors analyzed bias and other factors that may have affected individual study outcomes  study duration, randomization/blinding, and sponsorship by drug companies. 

A third strength that I found was this review's broad inclusion criteria that spanned multiple diagnoses. I discussed this further in a response post to Marlene, but I did feel that this inclusion made the review more clinical relevant to real life settings. 

A weakness that I thought about when reading this study was that the authors did not look at studies of clozapine or IM formulations of antipsychotics. Given how efficacious these can be, I felt that it would have been interesting and clinically useful to see how these matched up to the rest of the oral antipsychotics under review. I also found it strange that the authors included both published and unpublished studies, and worry that this may have led to the inclusion of some research that was not peer reviewed. 

In addition to strengths and weaknesses, I was also intrigued by the authors' discussion of the large 1964 NIMH trial, which showed that antipsychotics were more effective than placebos. Initially, the authors presented this study as lending support to antipsychotics, in contrast to other later "failed" antipsychotic trials. Later, however, the authors explained that half of the participants in the early NIMH study were suffering from their first episode of psychosis and were drug naive, whereas the later "failed studies" and the studies in their meta-analysis were not limited to first onset psychosis. Instead, participants were mostly chronic patients who tend to respond less to antipsychotics. This is a very useful comparison to draw and important for clinical practice. I feel that it makes much clinical sense to differentiate between first episode and chronic psychosis when evaluating treatment efficacy. I appreciated that the authors brought this point to our attention but wish that they had gone deeper into exploring this as a variable in the different studies that they reviewed. 

Overall, like others, I found this review to be interesting, despite the study aim being somewhat confusing. 



In reply to Julia

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Shararah Aziz -

Hey Julia,

Thank you for the breakdown of the strengths and weaknesses of the article. I also found it interesting that the half of the participants in the NIMH study had their first psychotic break and were drug naive, whereas other later studies used participants who had chronic mental illness and used antipsychotics before. This comparison is important to know and really impacts the study. I was hoping the author would dive in deeper as well into exploring the differences. It made me realize to look at the author and question their motive in producing a study. I find it useful to be able to analyze an article and then make an informed decision.


In reply to Sarah Dobbins

Re: Sixty Years of Placebo-Controlled Antipsychotic Drug Trials in Acute Schizophrenia

by Shararah Aziz -

Thank you for the posts everyone. Although this post is a little delayed, I agree with the responses.

The author points out that the aim of the study is trying to figure out if antipsychotic drug efficacy may have decreased over the years. I found it interesting that most placebo-controlled studies were conducted by pharmaceutical companies, although I am not completely surprised since pharmaceutical companies have their own motives and agendas to profit.

After reading this article about antipsychotic effectiveness I learned how the rate of efficacy was manipulated. I assumed that antipsychotics are generally pretty effective if taken continuously and help decrease most positive and negative symptoms. As discussed in the article about the 1964 meta-analysis sponsored NIMH revealed that there is a significant difference between antipsychotics and placebos. Taking a deeper look, the author reveals much of the discrepancies  in the study such as increasing placebo response rate and analyzing only predictors of placebo response, which artificially inflated the statistics.

The most significant part of this study that struck out to me was that antipsychotic drugs increased the patients with “any” response (51% vs 30%) to patients that had “good “ response (23% vs 14%), which is different than the 1964 NIMH benchmark study used for antipsychotic efficacy. 51% of patients reported at least a minimal response meaning a 20% significant change in positive or negative symptoms. Also, I think it is important to note that participants receiving placebo were likely to discontinue the study prematurely due to various reasons. I also found it interesting that the antipsychotic effect size for positive symptoms were similar to overall symptoms and that negative symptoms and depression were even smaller. It is important for me to remember as a provider that many symptoms may not be alleviated by antipsychotics and can cause the patient to be deterred from treatment. Having a better understanding of the effectiveness of these drugs gives the provider a better understanding of how patient may be feeling.

This article raises the question of generalizability in studies and encourages me as future provider to take a closer look at research.