Week 9 Responses

Week 9 Responses

by Matthew -

Paper Title: Effects of delayed compared with early umbilical cord clamping on maternal postpartum hemorrhage and cord blood gas sampling: a randomized trial

https://www.ncbi.nlm.nih.gov/pubmed/22913332

 

Brief Topic background:

Delayed cord clamping (DCC) is when you delay the clamping of the umbilical cord after birth typically by at least 60 seconds. The idea is that during this delay you are allowing blood to continue to flow from the placenta to the infant. DCC has been shown to decrease mortality, necrotizing enterocolitis, hypoxic ischemic encephalitis, and increase hematocrit, blood volume, and iron stores. There used to be concern that DCC increases the risk of maternal blood loss during delivery and how it affects umbilical blood gas values. For the sake of this assignment I will focus on just the maternal blood loss.

 

Brief Study background: 

This study was a secondary analysis of an RCT (parallel-group study with 1:1 randomization) with the primary objective to investigate whether iron stores in term-born infants differ at four months of age as a result of DCC compared with early cord clamping (ECC). In this CURRENT study, they are evaluating the effects of DCC on post-partum hemorrhage and cord blood gas sampling.

 

 Describe:

1) What was the exposure and outcome being evaluated?

            Exposure:

Control Group = Early cord clamping (ECC) which is clamping at < 10 s

Treatment Group = DCC which was clamping at >=180 s 

            Outcomes:

                        Post partum hemorrhage and umbilical artery blood gas

 

2) What was the adherence to randomly assigned treatment (and how was it measured)?

            Overall the adherence was ~85% in both groups. They simply timed how long until the cord was clamped so they could easily tell who did and didn’t have appropriate clamping times. Here is a snap shot from their flow diagram that nicely details who received the intended treatment and who didn’t.

 

            

3) What was the primary intent-to-treat effect estimate?  

Maternal blood loss is difficult measure directly so its commonly reported categorically as either >500 or 1000 mL of blood loss. For comparison between the proportions they used Chi-square and they reported a difference of proportions of 1.2(-6.5 to 8.8).

 

4) Did they report an IV effect estimate?  

no

 

5) Would an IV effect estimate have been of interest in this study?  

I don’t think so given this was an RCT and they already performed an ITT analysis. I’m also not too concerned about there being any unmeasured confounders biasing their ITT results. If there was concern for that type of bias then perhaps I could imaging an IV analysis being of interest.

 

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

n/a. They did present the ITT estimate which I believe was appropriate. Had they presented an “As-treated” analysis it would have potentially messed up the randomization.

 

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

No, I don’t think they provide any information about a possible IV. However, I can easily imagine one for a potential study. Because DCC has become the standard of care so I could imagine using policy change as an IV. For example, you could do a historical comparison study at an institution. At my hospital we began performing >60s of DCC in 2011 and then >180s in 2017. We could use that change in policy as an IV I think.

Thank you!

Matthew

Re: Week 9 Responses

by Adrienne Epstein -

I am skipping this week.

Re: Week 9 Responses

by Sandeep Brar -

Weisbord SD et al. Outcomes after Angiography with Sodium bicarbonate and acetylcysteine. N Engl J Med. (2018).

Patient at high risk for renal complications who were scheduled for angiography were randomized to receive intravenous 1.26% sodium bicarbonate or intravenous 0.9% sodium chloride and 5 days of oral acetylcysteine or oral placebo. The primary end point was a composite of death, the need for dialysis, or a persistent increase of at least 50% from baseline in the serum creatinine level at 90 days.

Overall, 4050 of 4993 patients (81.1%) adhered to the prescribed regimen of acetylcysteine and placebo capsules, with similar rates of adherence in the two trial groups. Compliance was assessed by patient self-report. 

The OR for the primary composite endpoint in the sodium bicarbonate group compared to the sodium chloride group was 0.93 (95% CI: 0.72, 1.22; p=0.62), and OR 1.02 (95% CI: 0.78, 1.33; p=0.88) in the acetylcysteine group as compared to the placebo group. These are the intent-to-treat effect estimates.

The authors did not report an IV effect estimate. An IV effect estimate would have been of interest in this study to see whether those who actually took the acetylcysteine vs. placebo had a decreased risk of the primary composite endpoint. 

The authors gave enough information to calculate an IV effect estimate on the risk difference scale.

Primary endpoint events

Acetylcysteine: 114/2495

Placebo: 112/2498

Compliance day 5

Acetylcysteine: 2005/2495

Placebo: 2045/2498

[(114/2495) – (112/2498)]/[(2005/2495) – (2045/2498)]

(4.57% - 4.48%)/(80.36% - 81.87%)  

Risk difference = 0.09%

IV estimate = 5.69%

The ITT risk difference is almost 0, but the IV risk difference is 5.7%.

Re: Week 9 Responses

by Teresa Kortz -

Title: Mortality after Fluid Bolus in African Children with Severe Infection

 Link: https://www.ncbi.nlm.nih.gov/pubmed/21615299

Population: Children (60d to 12 yrs) with severe febrile illness and impaired perfusion in East Africa (N=3141)

 Exposure: 2 intervention groups compared to control: boluses of 20-40 ml/kg of 5% albumin solution (albumin-bolus group) or 0.9% saline solution (saline-bolus group)

 Control: No bolus, maintenance IV fluids only

 Outcome: The primary outcome was 48-hour mortality. Secondary outcomes: pulmonary edema, increased intracranial pressure, and mortality or neurologic sequelae at 4 weeks.

 Adherence: Adherence was high within each arm: of the 1050 assigned to the albumin-bolus group, 1045 received it (99.5%); of the 1047 assigned to the saline-bolus group, 1041 received it (99.4%); of the 1044 assigned to the control, 1 received a bolus of saline (99.9%). The authors do not specifically mention how adherence was measured, but a structured clinical case-report form was completed for each enrolled patient and presumably that captured elements of compliance. Given the short time window for treatment, the clearly defined intervention, and physician/nurse administration, I think it would be relatively easy to confirm adherence.

Primary ITT effect estimate: The 48-hour mortality was 10.6% (111 of 1050 children), 10.5% (110 of 1047 children), and 7.3% (76 of 1044 children) in the albumin-bolus, saline-bolus, and control groups, respectively. The RR for mortality in the saline bolus vs. control was 1.44 (95% CI 1.09-1.90; P = 0.01); in the albumin bolus vs. saline bolus was 1.01 (95% CI 0.78-1.29; P = 0.96); and for any bolus vs. control was 1.45 (95% CI 1.13-1.86; P = 0.003).

IV effect estimate: none

Do you think the IV estimate would be of more interest than the ITT estimate? Why or why not? The research question of biggest interest is: what is the effect of fluid bolus resuscitation (not treatment allocation) on 48-hour mortality? In this case, adherence was so high that I think the ITT estimate reflects the effect of treatment and an additional IV estimate would not add additional information or answer a different research question.

Can you calculate the IV effect estimate based on the information provided? If so, what is it? If not, why not? Yes. I will focus on the main comparison of interest: any bolus vs. control and the effect on 48-hour mortality. I have the proportion who experienced mortality at 48 hours (outcome) for each group, the relative risk of mortality (1.45), the number and proportion who adhered in each group, number who received the assigned treatment, and the number randomized to each group.  With this information, I completed a 2X2 table and calculated the ITT RD (3.25% difference). Then using the adherence information, I was able to calculate the IV RD (3.28% difference). Again, because the adherence was so high, it is no surprise that the two estimates are nearly identical (0.6% difference between ITT and IV RD estimates). Please see attached excel sheet for details.

Re: Week 9 Responses

by Marta San Luciano Palenzuela -

“Randomized delayed-start trial of levodopa in Parkinson’s disease” Verschuur et al, NEJM 2019

1.     What was the exposure and outcome being evaluated?

Exposure --> carbidopa/levodopa 25/100mg TID vs. Placebo in a delayed start fashion [1st 40 weeks carbidopa/levodopa TID vs. equally looking placebo TID, then both groups take carbidopa/levodopa TID for another 40 weeks; the purpose of the delayed start was to look for a disease-modifying effect where those taking the medication during the first 40 weeks had a persistent benefit over those who started taking the medication at a later time (delayed start group) at final follow up]

Outcome -->Primary outcome was the between group difference in the mean change from baseline to week 80 in the Unified Parkinson’s Disease Rating Scale (UPDRS) total score (0-176, higher scores represent greater symptom severity in Parkinson’s disease).

 2.     What was the adherence to randomly assigned treatment (and how was it measured)?

 It was reported that 24/222 (10.8%)  in the active treatment group moved to phase 2 (unblinded treatment for 40 weeks), and 87/223 (39%) had early transfer to phase 2 (started carbidopa/levodopa treatment). Therefore, adherence in the active treatment group was 89% and 61% in the placebo group.

3.     What was the primary intent-to-treat effect estimate?  

The intent-to-treat estimate was a difference in change in UPDRS score from baseline to week 80 between the two groups of 1.0 points (95%CI: -1.5, 3.5, p=0.44). A one-point difference is not a clinically significant change (usually 3-5 points).

 4.     Did they report an IV effect estimate? 

 No, but the authors reported a per-protocol analysis in their supplementary material. In the per-protocol analysis, the primary outcome of the difference between the two groups was -0.9 points (95%CI: -3.8 to 2.1), very similar to the ITT analysis.   

 5.     Would an IV effect estimate have been of interest in this study? 

 Yes. While the per protocol estimate is informative on the effect of the treatment on the treated, since those individuals who do not follow protocol are removed from the analysis, precision tends to be lower and 95%CI wide. The per-protocol analysis also results in non-random omission bias. Therefore, an IV analysis would probably give a more precise estimate of the effect of treatment on the treated (In this case, do people with Parkinson’s disease who actually start levodopa treatment early as opposed to those who start 40 weeks later, have better outcomes at 80 weeks?). Especially since in this case, there was considerable cross-over. 

 6.     If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

 Yes. After all, for many specific clinical questions, a very interesting question is what happens in those individuals who actually take the treatment (does levodopa indeed slow down disease progression for those individuals). This is in many ways, a more interesting question than the probably more public health related issue of the experience of the treatment as a whole in Parkinson’s disease in a real world setting when adherence is incomplete.

 7.     Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

 The instrumental variable estimate would be equal to the local average treatment effect (LATE)= ITT/proportion of compliers.

 In this case, in the treatment group, 89% of treatment assigned individuals took the treatment (the tricky thing of this trial is that the other 11% also took the treatment, but this was unblinded). In the control group, 39% took the active treatment even though they were assigned placebo (61% took the assigned placebo).

If we first assume that the 11% of people in the active group are ‘non-compliers’, then the proportion of compliers is 50% (100%- 11% - 39%=50%), and 50% were non-compliers.

 LATE =-1.0/0.5= -2.0  ; this is larger difference, although it is still not clinically relevant (but may suggest an effect)

 If we assume that the 11% in the treatment group are compliers (because they took the treatment), then 100-39= 61% is the proportion of compliers, then:

 LATE = -1.0/0.61=-1.64; still not clinically relevant, but certainly larger than the ITT and the per-protocol estimates


Re: Week 9 Responses

by Sarah Raifman -

Randomized Trial of Peanut Consumption in Infants at risk of peanut allergy:  https://www.nejm.org/doi/full/10.1056/NEJMoa1414850

1) What was the exposure and outcome being evaluated?

Exposure: consumption vs avoidance of peanuts until 60 months of age. Infants assigned to consumption group and had negative skin prick tests were given 2 g of peanut protein in a single dose (in the form of Bamba) and those who had positive test results were given incremental doses up to a total of 3.9g. (There was no placebo)

Outcome: proportion of participants with peanut allergy at 60 months of age 

2)     What was the adherence to randomly assigned treatment (and how was it measured)?

The overall rate of adherence was 92% (among the 319 assigned to consumption, 7 were instructed not to consume peanut because they had a positive result at baseline to the oral food challenge, and 9 stopped consumption because they began to have allergic symptoms to peanuts).

In the negative result on skin test group, 500/530 in the ITT group (94.3%) adhered to assignment and were included in the per protocol analysis. In the positive skin test group, 89/98 in the ITT group (90.8%) adhered to assignment.

Adherence was monitored with the use of food-frequency questionnaires during the study and corroborated at the end of the study through the measurement of peanut in bed dust, an objective and validated surrogate for consumption (dust samples were measured for 423 of 640 or 66.1% of the study population).

 Adherence was defined in peanut-avoidance group as consumption of less than 0.2 g of peanut protein (one peanut) on any occasion and less than 0.5 g over a single week in the first 2 years of life. In the peanut-consumption group, adequate adherence was defined as consumption of at least 2 g of peanut protein on at least one occasion in both the first and second years of life and of at least 3 g of peanut protein (25 g of Bamba or 12 g of peanut butter) per week for at least 50% of the weeks during which data were recorded.

3)     What was the primary intent-to-treat effect estimate?  

The study was done in two separate cohorts – those who tested positive vs negative on an initial skin prick test:

·       In the neg result skin prick group, 530 were included in the ITT analysis: 13.7% of the avoidance group and 1.9% of the consumption group were allergic to peanuts at 60 months, for an absolute difference in risk of 11.8 percentage points (CI: 3.4 to 20.3; p<0.001) and an 86.1% relative reduction in prevalence of peanut allergy.

·       In the pos skin prick group, 98 were included in the ITT analysis: 35.3% of the avoidance group and 10.6% of the consumption group were allergic, with an absolute difference in risk of 24.7 percentage points (95% CI: 4.9 to 43.3, p=0.004) and a 70% relative reduction in prevalence of peanut allergy.

·       Pooled estimates from the cohorts: Among the 628 in the ITT group, the prevalence of peanut allergy at 60 months of age was 17.2% in the avoidance group and 3.2% in the consumption group. This is an absolute difference in risk of 14 percentage points and an 81.4% relative reduction in prevalence of peanut allergy.   

4)     Did they report an IV effect estimate?  

No

5)     Would an IV effect estimate have been of interest in this study?  

An IV estimate is of interest particularly when the causes of noncompliance are independent risk factors for the outcome (ie. association between actual treatment received and the outcome is confounded).

Here there is minimal noncompliance (the random assignment is a strong predictor of the actual exposure) and the expected unmeasured confounding of the ITT effect is minimal. So I would say there’s less need for an IV. It may be interesting to use the IV in a secondary or sensitivity analysis.

There was no placebo for the peanut consumption intervention, therefore I don’t see how blinding would have been possible. In this way, it’s possible that the assignment may have an independent effect on the outcome if those assigned the peanut do not want to eat it. But again we see high adherence in both groups so the bias would be toward the null and minimal.

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not? no i don't, see above

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

P(X=1|Z=1), Adherence among those assigned to peanut consumption (pooled):  294/314 = 93.63%

P(X=1|Z=0), Adherence among those assigned to peanut avoidance (pooled): 19/314 = 6.1%

P(Y=1|Z=1), risk of outcome among those assigned to peanut consumption (pooled) = 3.2%

P(Y=1|Z=0), risk of outcome among those assigned to peanut avoidance (pooled) =  17.2%

IV estimator (risk difference scale) = (P(Y=1|Z=1)-P(Y=1|Z=0))/(P(X=1|Z=1)-P(X=1|Z=0))

= (3.2% - 17.2%) /(93.63% - 6.1%)  or ITT/compliance

= -0.1599

Diff in treatment rates between levels of instrument: 93.63% - 6.1% = 87.58%

ITT = 14%

IV estimator = 14% / 87.58% = 0.1599 or 16%

The IV estimator is 16% and the ITT is 14%.

 


Re: Week 9 Responses

by Maria Glymour -

Sandeep

I think you've made a very very helpful mistake in your calculation above.  The denominator of the IV estimate is not supposed to be compliance to the group you were assigned to, but rather compliance to the TREATMENT, i.e., it is the difference between the probability of receiving treatment in one group versus the other group. So in your calculation above, did 2045 people int he placebo group receive the placebo, or did 2045 receive the acetylcysteine (despite being assigned to placebo)?

I realize the way I have explained this is very confusing - but this is a big difference.  

Maria


Re: Week 9 Responses

by Sandeep Brar -
Hi Maria,

Maybe I am still a bit confused.
2045 people were adherent to placebo.
2498 individuals were assigned to the placebo arm.

Sandeep

Re: Week 9 Responses

by Monica Ospina Romero -

Title: Two Phase 3 Trials of Bapineuzumab in Mild-to-Moderate Alzheimer's Disease

DOI: 10.1056/NEJMoa1304839

Population: Adults with mild to moderate Alzheimer’s Disease, carriers and non-carriers of ApoE4 allele.

Exposure: Intravenous Bapineuzumab every 13 weeks up to 6 IV infusions compared to placebo

Outcome: The change from baseline to week 78 on the 11-item cognitive subscale of the Alzheimer's Disease Assessment Scale (ADAS-cog11, with scores ranging from 0 to 70 and higher scores indicating greater impairment) and the Disability Assessment for Dementia (DAD, with scores ranging from 0 to 100 and higher scores indicating less impairment). 

Adherence: the authors measured the total infusion received out of 6 possible infusions. They reported the percentage of people who received the 6 IV infusions in the placebo group (~75 %) and the treatment group (~64%).

Intention-to-treat estimate: They described a modified ITT population that included participants who received at least one dose of a study drug and underwent evaluation of the coprimary efficacy end points at baseline and at least once after baseline.

 

Effect estimates:

ADAS-cog11 total score change:

ApoE4 carrier study: Placebo 8.7±0.5, Bapineuzumab (0.5mg/kg) 8.5 ± 0.4, Difference -0.2 (95% CI: -1.4, 1.0)

Non-carrier study: Placebo 7.4 ± 0.5, Bapineuzumab (0.5mg/kg) 7.1 ± 0.6, Difference -0.3 (95% CI: -1.8, 1.1)

Bapineuzumab (1 mg/kg) 7.8 ± 0.6, Difference 0.4 (95% CI: -1.1, 1.8)

 

DAD total score change:

 

ApoE4 carrier study: Placebo -16.2 ± 1.0, Bapineuzumab (0.5mg/kg) -17.4 ± 0.8 Difference -1.2 95% CI: -3.8,1.3)

Non-carrier study: Placebo -15.5 ± 1.0, Bapineuzumab (0.5mg/kg) -12.7 ± 1.2 Difference 2.8 (95%CI: -0.2, 5.8)

Bapineuzumab (1mg/kg) -14.6 ± 1.2, Difference 0.9 (95% CI: -2.1, 4.0)

 

IV effect report: No, the authors didn’t report an IV analysis

 

Do you think the IV estimate would be of more interest than the ITT estimate?  The research question is whether treatment with Bapineuzumab decreases rate of cognitive change in patients with mild to moderate AD. In the ITT analysis we are estimating the effect of treatment assignment on cognitive function, with good adherence to the treatment this is the right analysis for this question. In the IV analysis we would estimate the effect of the treatment on cognition in compliers to the treatment. Since the adherence to Bapineuzumab was around 65% and the placebo group never had access to Bapineuzumab, it would be interesting to see if the null findings are the result of the non-compliance in the treatment group. Knowing the IV effect is compelling because if this medication improves cognition through decreasing amyloid beta products, we could demonstrate that the amyloid beta accumulation is an important etiologic factor in AD progression.  However, I still think the best analysis for the research question of this trial is ITT analysis.

 

Can you calculate the IV effect estimate based on the information provided?

Yes, I think I can estimate the IV effect using the Wald estimator.

I used the ITT effect estimate from the ApoE4 carrier study (ADAS-cog11 outcome)

Probability of receiving 6 doses of Bapineuzumab in the treatment group was 63.8%

Probability of receiving 6 doses of Bapineuzumab in the placebo group was 0%

 

IV effect =  (E[Y| Z=1] – E[Y| Z=0])/ (E[X| Z=1] – E[X| Z=0]) = -0.2/0.638 = -0.313


Re: Week 9 Responses

by Maria Glymour -

Nice example Monica.

One caveat: the IV estimate will change the point estimate compared to the ITT point estimate, but it will almost never be statistically significant when the ITT is not significant.

Maria


Re: Week 9 Responses

by Sarah Dobbins -

RCT: A Multiple-City RCT of Housing First With Assertive Community Treatment for Homeless Canadians With Serious Mental Illness. Link

1) What was the exposure and outcome being evaluated?

Exposure: Housing First with Assertive Community Treatment
Outcome: Many, but primarily housing stability and community functioning. Secondary outcomes were self-rated health status, mental health symptoms, physical and psychological integration, substance use, quality of life, arrests, time hospitalized, and emergency department visits.

2) What was the adherence to randomly assigned treatment (and how was it measured)?

Randomized: 
Housing First with ACT (N=469)
Treatment as usual (N=481)

Accepted tx: 
Housing First with ACT n=369 
Treatment as usual control n=337

Adherence, in this case, can be thought of as having two components: First accepting housing and then residing in Housing First units. In this study people assigned to "care as usual" had access to the existing programs available in their communities. This means they could receive any housing and/or community support services other than from the Housing First program. 

At the end of the study period, 273 of the 369 Housing First participants (74%) and 138 of 337 treatment-as-usual participants (41%) were in stable housing. The mean length of stay in housing  was 401.9 days for Housing First participants and 281.2 days for treatment-as-usual participants.

3) What was the primary intent-to-treat effect estimate?  

Housing First participants reported higher quality of life (adjusted standardized mean difference=.15, p<.01) and were assessed as having better community functioning (adjusted standardized mean difference=.18, p<.01) over the two-year period. 

4) Did they report an IV effect estimate?  

No; they analyzed the data as randomized. 

5) Would an IV effect estimate have been of interest in this study?  

I think so! 

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

Because it is unethical to withhold housing from an unhoused person, the study had "contamination" i.e. people in the control group got housing. Likewise, it seems that only 79% of the randomized people got treatment. The results of this study that seem most relevant to healthcare are the effects of treatment, not of randomization.  Moreover, when it matters how large the effect estimate is, for example with cost as an outcome, the IV approach would be more useful. 

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

I don't think so? I'm not totally grasping how this would be done. But i'd like to follow-up with this question after the lecture...

Re: Week 9 Responses

by Jean Digitale -

D'acremont, Valérie, et al. “Reduction of Anti-Malarial Consumption after Rapid Diagnostic Tests Implementation in Dar Es Salaam: a before-after and Cluster Randomized Controlled Study.” Malaria Journal, vol. 10, no. 1, 2011, doi:10.1186/1475-2875-10-107.

1) What was the exposure and outcome being evaluated?

Exposure: Training providers on guidelines for malaria rapid diagnostic tests: 1) only patients with fever should be tested, 2) only patients with positive test results should be treated, 3) Integrated Management of Childhood Illness (ICMI) guidelines should be followed for non-malaria illnesses in children under 5 years old

Outcome: anti-malarial prescriptions recorded during observation consultations

2) What was the adherence to randomly assigned treatment (and how was it measured)?

The study team measured adherence to guidelines by observing clinic visits. They reported that 57% of patients not complaining of fever were tested for malaria in intervention facilities and 64% were in control facilities. They reported that 7% of those with negative test results were treated in intervention facilities compared to 25% in control facilities.

3) What was the primary intent-to-treat effect estimate?  

22% of all patients were treated with anti-malarials in intervention facilities vs. 60% in control facilities; RR: 0.30 (95% CI: 0.14-0.70, p=0.007)

4) Did they report an IV effect estimate?  

No

5) Would an IV effect estimate have been of interest in this study? 

Yes, I think it is useful to know if adhered to properly, how much the treatment algorithm would decrease anti-malarial prescription. This would then represent, theoretically, how much overtreatment could be reduced (assuming the treatment algorithm is 100% sensitive and specific, such that it correctly differentiated who should and should not be treated, which it likely is not).

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

From a management perspective, if one is looking at cost of malaria treatment, the ITT estimate is likely more useful for planning. It is hard to have 100% adherence among multiple human providers. Thus, the ITT is probably a better estimate of the effect one could expect if one wanted to implement such a program in one’s own clinic. However, if one is looking at how much overtreatment could be reduced, it would be more useful to have an IV estimate.

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

We have most of the pieces we need for adherence, but it is unclear how to classify the febrile patients who were not tested for malaria. Presumably, there was a clinical reason for them not to be tested, but it may be overly generous to assume that guidelines were followed for anyone complaining of fever who was not tested. We would need to know for each patient if they were tested “correctly”: tested when they should have been if febrile, and not tested if afebrile. Then, we could determine whether each patient was correctly treated based on their test results. Using a measure of adherence created from these two steps, we could calculate the IV estimate by dividing the ITT effect by the effect of randomization on adherence.

(We also don’t know if ICMI guidelines were followed for each eligible patient. However, one could argue this is irrelevant for calculating whether a provider complied with anti-malarial prescribing.)

Re: Week 9 Responses

by Maria Glymour -

Very interesting example Jean. I'm having a hard time understanding the endogenous (exposure) variable here- is it the training or adherence to the guidelines? If adherence to the guidelines, do you need a trial to estimate this or would you know it just from description of patients?

Maria


Re: Week 9 Responses

by Jean Digitale -

As I was writing my post I had the same thought - the intervention is technically the training at the facility level. Thus, there would be 100% adherence (unless there was a different research question such that one wanted to look at adherence at the provider level, for example.) However, throughout the article they talk about adherence to the guidelines. Thus, I (perhaps erroneously) focused on that. I guess my explanation was if you randomized patients to guidelines? 

Re: Week 9 Responses

by Zahra Izadi -

I am skipping this week. 

Re: Week 9 Responses

by Eduardo Santiago-Rodriguez -

I am skipping.

Re: Week 9 Responses

by Laura Koth -

Lazarus SC et al. N Engl J Med 2019 (a colleague in my division)

https://www.ncbi.nlm.nih.gov/pubmed/31112384

1) What was the exposure and outcome being evaluated?

Exposure: sputum eosinophil level at baseline in asthmas >=12 years old

Outcome: hierarchical composite outcome that incorporated treatment failure, asthma control days, and the forced expiratory volume in 1 second

2) What was the adherence to randomly assigned treatment (and how was it measured)?

Design: randomized, double-blind, placebo-controlled crossover trial

How it was first assessed: during a 6-week, single-blind placebo run-in period they established adherence to the trial agent and daily completion of an electronic diary and the participants had to have more than 75% adherence. All the patients used an electronic diary (Spirotel, Medical International Research) to record symptoms, medication use, nighttime awakenings, and morning and evening peak expiratory flow.

The authors presented a 3 page supplement regarding adherence. In this supplement they described how the medication device measured drug use. They reported about 83% adherence using this criteria. The reported no significant difference among the three trial groups in the rate of adherence to the blinded medications. The adherence of diary completion was lower, reported to be ~65% for the treatment groups.

3) What was the primary intent-to-treat effect estimate?  

The primary analysis found no significant difference between the percentage who had a better “response” to mometasone (57%; 95% confidence interval [CI], 48 to 66) and those who had a better response to placebo (43%; 95% CI, 34 to 52; P=0.14); where “response” is their composite endpoint

4) Did they report an IV effect estimate?  

No they did not. I asked the author and he told me that the concept did not come up in their planning committee

5) Would an IV effect estimate have been of interest in this study?  

Perhaps the study I chose was not the best to illustrate the reasons to perform IV analysis in an RCT. In reading more about IV in RCT, I now understand better that this technique is a way to try to get at the “truth” if there is concern about adherence to the intervention. Perhaps they measured “adherence” in this study was good enough so they did not use an IV; or perhaps there is less concern about adherence when you have a cross over trial design.

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

In the additional reading on IV estimates in RCT’s, it seems as if an IV estimate might always be of more interest than an ITT estimate when there is concern over adherence.

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

Good question and I am not sure how to approach answering this question.


Re: Week 9 Responses

by Scott Lu -

Find an RCT of interest to you.   Describe:


Shaik F, Uldrick TS, Esterhuizen T, Mosam A.  Health-Related Quality of Life in Patients Treated With Antiretroviral Therapy Only Versus Chemotherapy and Antiretroviral Therapy for HIV-Associated Kaposi Sarcoma: A Randomized Control Trial.  J Glob Oncol. 2018 Oct;4:1-9. doi: 10.1200/JGO.18.00105.

 

1) What was the exposure and outcome being evaluated?

This RCT evaluated the effect of ART vs. ART + chemotherapy on an outcome of quality of life in African patients in Kwa-Zulu Natal with HIV-associated KS.

 

2) What was the adherence to randomly assigned treatment (and how was it measured)?

Adherence appeared to be a particular concern to investigators.  A study nurse assessed ART adherence using a self-report questionnaire at week 2 and 4 then every month after.  Adherence was reported as a percentage and mean adherence over the course of the study and categorized as excellent (>95%), good (80-94%), and poor (<80%).  Adherence was reported as excellent in 67 participants (60%), good in 33 (30%), and poor in 11 (10%) with no difference between arms over 12 months.  Notably in this case non-adherence to one arm of therapy would not necessarily put a participant in the other treatment arm.

 

3) What was the primary intent-to-treat effect estimate?  

An ITT analysis was conducted for the outcome of (median) quality of life (measured by the QORTC QLQ-30) between- and within- arms.  Investigators determined statistically significant improvements were seen from baseline to completion of treatment in both treatment arms in general health (ART: +13; IQR: 0-17 | ART + chemo: +17; IQR: 0-50), emotional functioning (ART: +17; IQR: 0-33| ART + chemo: +17; IQR: 0-33), cognitive functioning(ART: +17; IQR: 0-33| ART + chemo: +17; IQR: 0-33), fatigue (ART: -22; IQR: -56 - 0| ART + chemo: -33; IQR: -44 - -11), pain(ART: -17; IQR: -50-0| ART + chemo: -33; IQR: -83 - 0), and financial problems(ART: -33; IQR: -67-0| ART + chemo: -33; IQR: -100-0).  The ART-alone arm demonstrated improvement in social functioning, insomnia, constipation, and diarrhea as well.  Of note, role functioning improved in the chemotherapy arm but decreased in the ART arm.

 

4) Did they report an IV effect estimate?  

No, they did not.

 

5) Would an IV effect estimate have been of interest in this study?  

 

Yes: an IV effect estimate would inform on the causal effect of ART vs. ART + chemotherapy rather than being assigned to that therapy.  This study occurred in a resource-limited setting which may influence the extent of unmeasured confounders.  IV analyses may circumvent this bias to provide information of the effect of both treatment arms on quality of life. 

 

 

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

 

An argument could be made that this information is useful only as far as chemotherapy is available to individuals in the region.  In many parts of sub-Saharan Africa chemotherapy is difficulty for the average person to access, owing partially to monetary expense and limited chemotherapy infrastructure.  Access to chemotherapies is increasing, however, and thus I stand by an IV effect estimate would have been of interest here.

 

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

 

No, for multiple reasons pertaining to adherence.

First, adherence was assessed via questionnaires that spanned 7 days.  These questionnaires were applied by a study nurse at every study visit, which all occurred at intervals > 7 days.  This could be a usable estimator for calculating an IV, however there are other issues:

Authors reported compliance for the entire study population, not separated by treatment arm (ie. The 60% of subjects who reported “excellent” adherence noted above include those in both arms).  The authors did note that there was “no difference between arms”, which I believe could be interpreted as a denominator of 0 and prevent and IV effect estimate.  Another issue is the categorical aspect of adherence rather than giving the proportion itself.  Without those proportions a 2x2 table could not be made.

If they gave information on the proportion of subjects to receive every intended dose of the medication throughout the study period the Wald estimator could be used with a denominator of “difference in probability of receiving each dose of study medication over the study period (48 weeks) by treatment arm”, however that information was not provided.

I’d be very interested in discussing ways this could be calculated!

 


Re: Week 9 Responses

by Erika Meza-Luman -

Cladder-Micus, M. B., Speckens, A., Vrijsen, J. N., T Donders, A. R., Becker, E. S., & Spijker, J. (2018). Mindfulness-based cognitive therapy for patients with chronic, treatment-resistant depression: A pragmatic randomized controlled trial. Depression and anxiety, 35(10), 914–924. doi:10.1002/da.22788

Describe:

A multi-center, randomized-controlled trial analyzed compared the effect of mindfulness-based cognitive therapy (MBCT) with treatment-as-usual (TAU) in 106 chronically depressed outpatients that had previously received pharmacotherapy (>=4 weeks) and psychological treatment (>=10 sessions). 

1) What was the exposure and outcome being evaluated?

Exposure: mindfulness-based cognitive therapy (MBCT) + Treatment as usual (TAU) compared to Treatment as usual

Primary outcome: level of depressive symptoms assessed with IDS-SR

Secondary outcomes: remission (defined as the absence of depressive symptoms during the last 2 weeks) and rumination, mindfulness skills and self-compassion.

2) What was the adherence to randomly assigned treatment (and how was it measured)?

Adherence in the MBCT+TAU group was 75.5% measured according to completion.

3) What was the primary intent-to-treat effect estimate?  

Depression scores were lower in the MBCT+ TAU than the TAU group, difference in scores was -3.23 (95% CI: -7.02, 0.56; p=0.09)

4) Did they report an IV effect estimate?  

No, they did not report an IV effect estimate

5) Would an IV effect estimate have been of interest in this study?

I think that ITT analysis is appropriate for this study because it estimates the effect of treatment assignment on depressive symptoms. However, given that 24.5% of the intervention arm were non-compliers, perhaps an IV analysis would provide a more accurate estimate of the true effect of the treatment on depressive symptoms.

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

Not necessarily. The authors conducted sensitivity analyses imputing missing data according to the last observation carried forward technique and yielded similar findings. Additionally, authors state that completers and non-completers did not differ in the severity of depression symptoms, number of previous episodes or age of onset, therefore the ITT estimate may provide a realistic estimate of the effect that would be expected in a population that is not 100% adherent.  

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

Maybe? We could divide the ITT effect estimate by the compliance assuming that the treatment as usual group were all adherent to get: (-3.23) / 24.5 = - 0.13


Re: Week 9 Responses

by Crystal Langlais -

ARTICLE: Holmberg et al. A randomized trail comparing radical prostatectomy with watchful waiting in early prostate cancer. NEJM. 2002: 347 (11).

1) Treatment: Radical prostatectomy (surgical removal of prostate for the treatment of prostate cancer) 

Control: watchful waiting (control; clinical monitoring but no treatment other than transurethral resection).

Primary outcome: death due to prostate cancer (determined from independent ‘endpoint committee’ comprised of one pathologist and two urologists blinded to the group assignment and primary treatment).

2) Following exclusion (after randomization) of 2 men wrongly diagnosed and 1 man for not meeting inclusion criteria (history of prior cancer detected), 327/348 (94%) subjects randomized to watchful weight and 292/347 (84%) randomized to RP were adherent to treatment assignment. The authors did not explicitly state how adherence was measured.

3) Risk of death was lower in the radical prostatectomy group compared to watchful waiting group:        

Absolute difference at five year: 2.0 % (95%CI: -0.8, 4.8)

Relative hazard at five years: 0.50 (95%CI: 0.27, 0.91)

4-6) The authors did not provide an adherence analysis, only ITT. The IV (adherence) estimate would have estimated the effect of radical prostatectomy in those who actually underwent prostatectomy. I think this can always be of interest in an RCT and is worth stating, in addition to the ITT estimate.  The assigned Angrist et al artcle states the ITT effects will predict the effect of the treatment when compliance rates in the population are expected to be similar to those observed in the study.  Because this was a group of men with no preference to treatment at time of diagnosis, I think that is reasonable here. Reporting both is still a good idea.

7) From Brookhart et al., the Wald (IV) estimator is given by

(ITT estimate)/(E[X|Z=1] – E[X|Z=0])

= 2.0 / (0.84-(1-0.94)) = 2.56 


Re: Week 9 Responses

by Dan Kelly -

Article: Azithromycin to reduce childhood mortality in sub-Saharan Africa

https://www.ncbi.nlm.nih.gov/pubmed/?term=29694816

Describe: I selected a cluster-randomized trial among communities in Malawi, Niger, and Tanzania to evaluate if azithromycin reduces childhood mortality in sub-Saharan Africa. The study population was children 1 to 59 months of age identified in twice-yearly censuses. 


What was the exposure and outcome being evaluated?

Exposure was receipt of oral azithromycin during four twice-yearly mass distributions.

Outcome was all-cause mortality. 


What was the adherence to randomly assigned treatment (and how was it measured)? 

Adherence was measured on two levels. At the individual level, the children were given oral azithromycin under directly observed therapy. Then at the community level, there was a census so the investigator knew the coverage of the treatment in each randomly assigned community; they attempted to achieve at least 80% coverage of the treatment in each community. 

Azithromycin was administered to a mean of 90.3% (SD 10.6%) of the targeted population.

Placebo was administered to a mean of 90.4% (SD 10.1%) of the targeted population. 

 

What was the primary intent-to-treat effect estimate?  

Community-level, intention-to-treat analysis showed that over all four intercensal periods, mortality was 13.5% lower overall (95% CI: 6.7, 19.8) in the azithromycin group than in the placebo group (p<0.001). 

 

Did they report an IV effect estimate?  

No, they did not report an IV effect estimate. 

 

Would an IV effect estimate have been of interest in this study?  

I think that if you consider those who were randomly assigned as the IV and those would adhered to the treatment as the exposure, then they could have estimated an IV effect. This likely would have been of interest in this study because the IV effect estimate would have given a closer estimate of mortality if children took azithromycin. Then the IV effect would be answering an efficacy research question.

I also considered another IV that could be of interest based on the results of this study. Many countries in Africa have considered implementing mass distributions of azithromycin among their childhood populations. If some countries make a policy change in favor of mass azithromycin use while other countries do not, then I think the mass azithromycin policy among countries could be operationalized as an instrumental variable. Thus, an IV effect estimate would be an interesting study to build upon these results. 

 

If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

In this case, I see the IV and ITT estimates answering different questions. The ITT estimate gets at the real-world effect, which can be thought of more as an estimate of effectiveness, when a country decides to implement mass azithromycin distributions. I think this is more of interest than the alternative, which is the IV estimate answering the question of whether azithromycin reduces childhood mortality. If the ITT estimate is found to be statistically significant, then the ITT analysis, to some extent, answers both questions of whether the drug works as well as whether it works in the study context. 

  

Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not? 

I don’t think I can calculate the IV effect estimate because the authors would need to have disclosed community-level adherence. They only disclosed individual-level adherence. I’m not sure how to do this in the context of a cluster-randomized trial but I think that you probably need both sets of data to calculate the IV effect estimate. 

 


Re: Week 9 Responses

by Sepehr Hashemi -

Find an RCT of interest to you.   

 

McNamara Z, Findlay G, O'Rourke P, Batstone M: Removal versus retention of asymptomatic third molars in mandibular angle fractures: a randomized controlled trial. Int J Oral Maxillofac Surg 45:571, 2016

 

Describe:

1) What was the exposure and outcome being evaluated?

- Exposure: removal of asymptomatic third molar during surgery for reducing/fixing mandibular fractures where the fracture is very close to a third molar tooth.

- Outcome: Primary outcome: loosely defined as “uncomplicated fracture healing” (e.g. bony nonunion, infection etc.). Secondary outcomes: soft tissue healing complications, operation duration, malocclusion, and nerve injury.

2) What was the adherence to randomly assigned treatment (and how was it measured)?

- The surgeon nonblindly performed the treatment, therefore adherence to treatment was perfect.

3) What was the primary intent-to-treat effect estimate?  

- There were no cases of complicated healing post-operation, however, perhaps we can focus on nerve damage, which was statistically significantly different between two groups. They did not provide an effect estimate, however, an ITT (=per protocol here) risk ratio can be calculated → 0.394/0.161 = 2.45 (95% CI 0.9856466 - 6.052307, although p-value=0.386)

4) Did they report an IV effect estimate?  

- They did not.

5) Would an IV effect estimate have been of interest in this study?  

- IV is powerful since it eliminates the association of unmeasured confounders with the treatment received. However, if the IV (the random treatment assignment in an RCT) does not correlate strongly with the treatment received, the unmeasured confounders between actual treatment received and outcome may explain poor compliance, and the ITT estimate becomes less accurate. Since exposure adherence was perfect, an IV estimate would not be necessary, and the ITT estimate which coincidentally is the per protocol estimate is adequate.

6) If so, do you think the IV estimate would be of more interest than the ITT estimate?  Why/why not?

- See #5 please.

7) Can you calculate the IV effect estimate based on the information provided?  If so, what is it?  If not, why not?

- We can use Wald estimator = (ITT estimate) / (average treatment assignment compliance)

= ( E[Y|Z=1]-E[Y|Z=0] ) / (E[X|Z=1]-E[X|Z=0])

= 2.45 / 1

= 2.45