Please post your week 2 reading response in reply to this post, so they are all in one thread. Thank you.
Reference: Coldiron ME, Assao B, Page AL, et al. Single-dose oral ciprofloxacin prophylaxis as a response to a meningococcal meningitis epidemic in the African meningitis belt: A 3-arm, open-label, cluster-randomized trial. PLoS Med 2018; 15(6): e1002593. PMC6019097 following competing interests: RFG is a member of the Editorial Board of PLOS Medicine.
1. Unit of clustering: Villages with a suspected case meningococcal meningitis in the Madarounfa District of Niger were randomly assigned (1:1:1) to standard care (the control arm), single-dose oral ciprofloxacin for house- hold contacts only or village-wide distribution of ciprofloxacin.
2. The hypothesized effects and the level at which the exposure is measured (is it a characteristic of the cluster or the observation within the cluster): The hypothesis was that ciprofloxacin prophylaxis would decrease overall meningitis attack rates (ARs) per 100,000 people. The study exposure (no prophylaxis, household contact prophylaxis, or village-wide prophylaxis) was at the village (cluster) level, and the primary outcome was overall AR of suspected meningitis, also assessed at the village (cluster) level from the day of the village’s inclusion until the end of the epidemic. Secondary outcomes included AR by age and sex, which were assessed at the village (cluster) level, and the individual protective effectiveness conferred by single-dose oral ciprofloxacin, assessed at the individual level.
3. Statistical model used to estimate the effect: 1) To estimate the difference in the number of meningitis cases between the 3 arms, the authors log-transformed the ARs and used a cluster-level t test, with inverse variance weights to account for different cluster sizes and numbers of cases. The authors compared each treatment arm (cipro prophylaxis for household contacts only and village-wide cipro prophylaxis) pairwise to the control arm (significance level 5%). To control for potential confounders that were imbalanced between the control and the treatment arms, the authors estimated the AR ratio and 95%CI for each treatment arm compared to the control arm with a Poisson regression.
4. Other statistical models that might be appropriate and/or preferable (e.g., GEE vs mixed): A GEE model would directly account for the clustered study design and works well if the cluster size varies or if one wants to adjust for covariates. A downside to a GEE model is using a large-sample method when there are only k=17 units of randomization per arm. GEE works best when there are a large number of clusters (>38 or >50 depending on the source) and when the number of clusters is much greater than the number of observations per cluster. Compared to a mixed effects model, in this case an ANOVA because there was only one time point in the analysis, a GEE model only works for one level of clustering, which is present in this study (villages), and GEE is more prone to bias from missing data. In this case, there are an (almost) equal number of villages in each arm (17, 17, and 15), resulting in a relatively balanced study design. Therefore, a simple ttest that takes into account the unequal variance across groups is a straightforward and valid approach to an unadjusted analysis. I am not sure why the authors chose to perform an adjusted Poisson regression instead of a GEE model for the adjusted analyses, however.
Article: Efevbera Y, Bhabha J, Farmer P, Fink G. Girl child marriage, socioeconomic status, and undernutrition: evidence from 35 countries in Sub-Saharan Africa. BMC Med. 2019 Mar 8;17(1):55. doi: 10.1186/s12916-019-1279-8.
This article uses Demographic and Health Survey (DHS) data from 103 surveys (35 countries from 1991-2014). DHS data are collected using a stratified two-stage cluster sampling design, randomly sampling a number of enumeration areas (EAs) and a number of households within those EAs. The primary exposure and outcomes in this article were at the individual-level. The exposure, girl child marriage, was defined as self-reported union before the age of 18, and the primary outcome was underweight. The authors presented a conceptual framework including a number of hypothesized mechanisms through which early marriage could impact nutrition, but many of the mechanisms represent different effect directions. The authors therefore hoped to elucidate this relationship through this analysis.
To address clustering, the authors included fixed effects at the EA level and clustered standard errors at this level. The authors chose to include fixed effects, which evaluate only within-cluster variation and not between-cluster variation, out of concern for unmeasured confounding. For example, the authors do not control for cluster-level population density, poverty, food security, and other variables which may confound the relationship between child marriage and underweight, but do not need to include these variables because they explore only within-cluster variation.
Instead of fixed effects, the authors may have considered a mixed effects model (including both fixed and random effects), which is more efficient and evaluates both within and between variation. Random effects may be appropriate here because clusters were randomly selected; therefore, we can assume that the group-level error should be normally distributed. However, due to the authors’ stated concern regarding residual confounding at the cluster level, I understand their conservative decision to use fixed effects.
Article: Jeannette R. Ickovics, Valerie Earnshaw, Jessica B. Lewis, Trace S. Kershaw, Urania Magriples, Emily Stasko, Sharon Schindler Rising, Andrea Cassells, Shayna Cunningham, Peter Bernstein, and Jonathan N. Tobin. Cluster Randomized Controlled Trial of Group Prenatal Care: Perinatal Outcomes Among Adolescents in New York City Health Centers. Am J Public Health. 2016 February; 106(2): 359–365.
This cluster RCT enrolled 1233 pregnant adolescents from 14 community health centers and hospitals in New York City (serving predominately low-resource women) from 2008 to 2012. The 14 health facilities were randomized to provide prenatal care in a group setting (bundled with reproductive health promotion) or standard individual prenatal care. Groups included 8-12 women of the same gestational age, and a credentialed clinician and cofacilitator. There were 10 sessions lasting 120 minutes that followed the clinical guidelines. Eligible participants were 14–21 years old, fluent in English or Spanish, entered prenatal care before 24-weeks gestation, not considered medical high-risk pregnancy, and willing to participate in group prenatal care.
The unit of clustering was the community health centers/hospitals. Pregnant adolescents were clustered into prenatal care settings.
The level at which the exposure is measured is the facility level - clinic facilities were randomly assigned to deliver group prenatal care so the exposure is characteristic of the cluster not the observation within the cluster. The outcome was measured at the individual level.
- To minimize selection bias, they identified health centers and recruited before randomization. They randomized sites using a computer-generated sequence in stratified blocks to account for lags in recruitment and the time required for training and implementation.
- They calculated sample size using Optimal Design for Multilevel and Longitudinal Research, using previous results (d=0.21-0.39) and assuming power=0.80, alpha=0.05, and ICC=0.001 they estimated needing 14 sites with 90 patients at each site to detect a small or medium effect size of d=0.25.
The hypothesized effect is that women at clinical sites randomly assigned to deliver group prenatal care will have better reproductive and sexual health outcomes than those of women at sites randomly assigned to individual care and that greater exposure to group prenatal care would be associated with better outcomes. Outcomes included gestational age at delivery, infant birth weight, small for gestational age, among others.
The authors used multilevel mixed models to control for interdependence due to site clustering (e.g. gestational age ICC=0.002; condom use ICC=0.013), with effect of site modeled as a random effect.
- They conducted linear models for continuous variables, logistic models for dichotomous variables and models with Poisson distribution for count data and skewed distributions (e.g. gestational age in days, days in NICU).
- Adjusted models included number of group visits, controlling for characteristics associated with group attendance (born outside of US, parity, living with family of origin, began intervention earlier in pregnancy, number of individual care visits).
- Conducted post hoc analyses to evaluate impact of receiving at least the minimal intervention dose (50% of group prenatal visits scheduled).
Article: Rousseau, Cécile et al. “A cluster randomized-controlled trial of a classroom-based drama workshop program to improve mental health outcomes among immigrant and refugee youth in special classes.” PloS one vol. 9,8 e104704. 15 Aug. 2014, doi:10.1371/journal.pone.0104704
This cluster randomized trial evaluates the effectiveness of a school-based theatre intervention program in Canada for immigrant and refugee youth in special classes (due to emotional, learning or behavioral problems) for improving mental health and academic outcomes. 29 special classrooms in five multiethnic high schools were randomly assigned to one of three study arms: 1) theater intervention (n = 10); 2) tutoring (n = 10); or 3) no intervention/control status (n = 9), for a total of 477 participants.
the unit of clustering: students were clustered at the special classroom level
the hypothesized effects and the level at which the exposure is measured (is it a characteristic of the cluster or the observation within the cluster): The authors’ primary hypothesis was that students in the theatre intervention group would report a greater reduction in impairment from symptoms (emotional and behavioral) compared to students in the control and tutoring groups from pre-intervention (T0) to the end of the 12-week intervention program (T1). The exposure was measured at the cluster levels since the classrooms were assigned to a study arm as a unit.
the statistical model used to estimate the effect: authors used linear mixed-effects models to account for the correlation between students in the same class given the randomization by classroom. Covariates included gender, age, socio-economic status (family income and parents’ employment status), country of birth of youth and of parents, ethnicity, years in Canada, language proficiency (English and French), migratory status (immigrants, refugee or citizen). Authors also included the treatment group as a covariate to measure the size of the effect of a particular treatment on the outcome (impairment from emotional and behavioral symptoms) score. The authors then fit models including 1) an interaction between baseline and treatment group; 2) models with covariates gender and interaction between gender and treatment group, and 3) models with covariates baseline, treatment group, gender, baseline-treatment interaction, and gender-treatment interaction. The impact scores as the outcome were modeled under a Poisson family assumption since these can only take positive values.
Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed):
Although, I am still unclear as to the differences between GEE and mixed models, I think that if the authors wanted to estimate the average effect over the entire sample, then they could have used GEE. I think that they used mixed methods given the longitudinal nature of the study and because the outcome was measured using individual level questionnaires for impairment from emotional and behavioral symptoms at baseline and then at 12-month follow-up, but maybe GEE would have been more appropriate.
Victor, R.G., et al., A Cluster-Randomized Trial of Blood-Pressure Reduction in Black Barbershops. N Engl J Med, 2018. 378(14): p. 1291-1301.
This was a cluster-randomized trial published in the New England Journal of Medicine which examined the effect of pharmacist led intervention among black male barbershop patrons on reduction in systolic blood pressure at 6 months.
The primary hypothesis was that the reduction in systolic blood pressure after 6 months would be greater among participants at barbershops with the pharmacist-led intervention.
The unit of clustering was the barbershop. The exposure was assigned at the level of the barbershop and participant group was determined according to barbershop. Barbershops were randomly assigned to the intervention or to the active control approach in a 1:1 ratio in equally balanced blocks of four with the use of a prespecified random-number sequence. Barbers in shops assigned to the intervention were trained to encourage pharmacist follow-up and measure blood pressure. Pharmacists met regularly with participants in barbershops assigned to the intervention; the pharmacists prescribed an antihypertensive drug regimen, measured blood pressure and encouraged lifestyle changes. Barbers in the control group encouraged lifestyle modification and doctor appointments. The outcome was measured at the individual level and was a characteristic of the observation within the cluster. At each visit, five sequential blood-pressure readings were obtained; the first two readings were discarded, and the last three readings were averaged
The intervention effect was estimated using a linear mixed-effect model, which included a random cluster effect. The primary predictor was an indicator for intervention group versus control group. The model included 3 baseline covariates: baseline blood pressure, a doctor for routine medical care (yes vs. no), and high cholesterol level (yes vs. no). For change in systolic blood pressure, the actual calculated intra-class correlation coefficient was 0.05. For the binary secondary outcomes, the authors used GEE controlling for the same covariates.
𝑌𝑖𝑗 = 𝛽0 + 𝛽1𝑔𝑟𝑜𝑢𝑝𝑖 + 𝛽2𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒_𝐵𝑃𝑖𝑗 + 𝛽3𝑟𝑜𝑢𝑡𝑖𝑛𝑒_𝑑𝑜𝑐𝑡𝑜𝑟𝑖𝑗 + 𝛽4ℎ𝑖𝑔ℎ_𝑐ℎ𝑜𝑙𝑒𝑠𝑡𝑒𝑟𝑜𝑙𝑖𝑗 + 𝛽5𝑔𝑟𝑜𝑢𝑝𝑖 × 𝑟𝑜𝑢𝑡𝑖𝑛𝑒 𝑑𝑜𝑐𝑡𝑜𝑟𝑖𝑗 + 𝛽6𝑔𝑟𝑜𝑢𝑝𝑖 × ℎ𝑖𝑔ℎ_𝑐ℎ𝑜𝑙𝑒𝑠𝑡𝑒𝑟𝑜𝑙𝑖𝑗 + 𝑏𝑖 + 𝜀𝑖𝑗
𝑌𝑖𝑗 was the change in systolic blood pressure from baseline to 6 months for patient j in cluster i. 𝛽1 was the main intervention effect with 𝑔𝑟𝑜𝑢𝑝𝑖 = 1 if the i-th cluster was in the intervention group and 𝑔𝑟𝑜𝑢𝑝𝑖 = 0 if in the control group. 𝛽2 to 𝛽4 were fixed effects of the baseline participant level covariates. An interaction between intervention and each of the covariates, routine doctor and high cholesterol were included in the model. 𝑏𝑖 represents the random cluster effect and 𝜀𝑖𝑗 represents the measurement error for the j-th individual in the i-th cluster. 𝑏𝑖 was assumed to be independent of the measurement error 𝜀𝑖𝑗and that 𝜀𝑖𝑗’s were assumed to be mutually independent.
A GEE would also be appropriate for the primary outcome. This study has greater than 38 degrees of freedom, therefore no small-sample correction would be required. The advantage of using a GEE is that the variance estimate is more robust. Also, mixed models require more modeling assumptions but allow one to calculate the intra-class correlation coefficient.
Hi Sandeep,
I'm excited to see your discussion of Ron Victor's 2018 barber study paper. The effect sizes observed in this study were fantastic, and his work targeting hypertension in Black men by embedding pharmacists in community barbershops is being continued here in the department. It will be very interesting to find out if (and how) this approach can be realized more broadly outside of cities like Los Angeles (and Dallas), where the density of Black barbershops facilitated the efficient implementation of the intervention.
-Shelley
Reference (With link attached, if it works)
MacPherson P, Lalloo DG, Webb EL, et al. Effect of Optional Home Initiation of HIV Care Following HIV Self-testing on Antiretroviral Therapy Initiation Among Adults in Malawi: A Randomized Clinical Trial. JAMA. 2014;312:372-379.
https://ucsf.summon.serialssolutions.com/#!/search?bookMark=ePnHCXMwTZ3NasMwDIDNyFj39wI7DMHOLnHc2uluYay0hzJou12Nk9gQ6JrSZhT2OHvSSXHS9urIIkSKZMXxpzsWYd3qWMdHPAN-xGioRhN51fJLU07kuOiCZzk4M5-kvmGDhDglcqxu2V_g-ELt4WMXPpAB9RCHOf1e094_XZvNv4DO7MAUH2B9xMDfDq3cxvOGiBU4gJLZtsFY0lC9vkdF63B2_1JXRl1-ICP8xQGqLSzsxh6rV8hgicV9_V39uhI6aicqID95YNcevcU9sqjZ_2Bc_5y-r99mvOtnwAVldT7xsXACS1lXxHkZC2-FzpNC4itAeVcXMh37UllpVaq0z1WZ6DTOcfnrlBTWynv2EvT2WcN0DncwtPKhTJNQA_hhkOrzrNkFcIUJiGJc8KNFDFnEkEVwwnOntk3BJ_HeBCjwFARo3klZSjucWv4DJxSRpQ
Background
MacPherson et al. conducted this cluster randomized trial in sub-Saharan Africa for the purpose of investigating a new potential avenue for increasing HIV-status testing and ART uptake for the purpose of improving HIV prevention and care.
Unit of clustering
This study randomized 14 community health worker catchment areas (identified in a previous study) in a 1:1 ratio to two groups: HIV self-testing followed by either optional home initiation of HIV care or by facility-based HIV care.
Hypothesized Effect
Authors hypothesized optional home initiation of HIV care after HIV self-testing could increase population-level uptake of ART (as well as willingness to test and report a positive result) compared to self-testing followed by facility-based services alone.
Exposure Measurement
The exposure (self-care vs. facility-based services for HIV individuals identified by self-test) was measured at the health worker catchment level. 14 catchment areas were identified and volunteers from each cluster were selected and trained to promote the availability of HIV-self testing by door-to-door visits and lefleting. After this, self-test kits were distributed to residents requesting self-testing kits. Participants were given the option to report self-test results to the counselor, if a positive result was reported, participants had the option for self-referral or referral to study clinics. In both arms a nurse would visit identified participants.
Statistical Model
For the primary outcome t-tests were used following an intention to treat analysis on the cluster-level proportion of residents who initiated ART, took an HIV self-test kit, and reported a positive test result between groups. Authors adjusted for reported mortality in the year preceding enumeration.
Other Potential Models
The primary outcome was cumulative incidence of ART during the first 6 months of self-testing availability. Thus their use of a t-test seems appropriate. The authors report no missing data due to the cluster-level outcome assessment. If the study continued such that some mean longitudinal data could be analyzed a GEE could be used without robust standard errors. A GEE model is more non-parametric than a mixed model, which is effective in handling multiple layers of clustering and adapting to missing data, two factors that are not a major issue with the data the authors present.
Hi Scott,
Interesting study! Did the authors also comment on the size of the clusters? I think the t-test works best when the clusters are approximately equal in size.
-Teresa
Paper:
Patten, C. A., Lando, H. A., Desnoyers, C. A., Barrows, Y., Klejka, J., Decker, P. A., . . . Burhansstipanov, L. (2019). The Healthy Pregnancies Project: Study protocol and baseline characteristics for a cluster-randomized controlled trial of a community intervention to reduce tobacco use among Alaska Native pregnant women. Contemporary Clinical Trials, 78, 116-125. doi:https://ucsf.idm.oclc.org/login?url=https://doi.org/10.1016/j.cct.2019.01.012
Background:
Tobacco use prevalence is high among pregnant Alaska Native (AN) women but few interventions have been evaluated for this group. The Healthy Pregnancies Project aims to evaluate a multicomponent intervention for reducing tobacco use during pregnancy and the postpartum period among AN women.
Unit of clustering:
In this cluster-randomized, controlled trial village was the unit of assignment. Sixteen villages were randomly assigned by the study statistician to receive the intervention (8 villages) or the control condition (8 villages).
Hypothesized effect:
The authors hypothesized that the multilevel (individual as well as community level) tobacco cessation intervention will reduce tobacco use in AN women compared to usual care.
Exposure measurement:
The exposure measurement occurred at both individual as well as village level: intervention villages receive a community-wide social marketing campaign and, for enrolled pregnant women, individual phone counseling delivered by Native Sisters (Health Aides in village clinics). The primary outcome was 6 month post-partum biochemically verified abstinence from tobacco.
Statistical model:
GEE with a logit link function to account for clustering of outcomes within village was used to examine condition differences on the point prevalence tobacco use rates at each study group level at 6 months postpartum. Because only 14 df were available for the test of the intervention, they employed a small sample correction to the standard GEE. Since they did account for small sample size (<38 df) GEE seems like an appropriate model to use for analysis.
Tol WA, Komproe IH, Susanty D, Jordans MJD, Macy RD, De Jong JTVM. School-Based Mental Health Intervention for Children Affected by Political Violence in Indonesia: A Cluster Randomized Trial. JAMA. 2008;300(6):655–662. doi:10.1001/jama.300.6.655
This trial involved 495 children who were attending randomly selected schools in political violence-affected areas in Poso, Indonesia – the unit of clustering is schools. The authors used the clustered randomized trial design (versus individual randomized trial design) to avoid contamination with schools.
The hypothesized effect is that the secondary
prevention school-based group mental health intervention would reduce PTS and
other traumatic stress-related symptoms. In other words, those who received the
intervention would have a greater reduction in psychosocial stress compared to those
in the waitlisted control group. The outcomes of interest, measured at the
individual level, included changes in self-reported levels of complaint in the
following categories: PTS symptoms, trauma idiom, depressive symptoms, anxiety
symptoms, functional impairment, and hope. Secondarily, the authors were
interested assessing potential three way interaction (time*intervention*sex). The
level at which the exposure is measured is at the school level, since randomization
occurred at the school level.
To estimate the effect, the authors used mixed-effect
regression models. They included random effects to account for clustering of
scores between children attending the same school. They compared the
intervention and control groups by testing a random intercept model that
included the fixed and random effects of time and intervention.
Other statistical models that might be appropriate and/or preferable: It seems that the authors could have also used GEE to estimate the average effect of the intervention. Perhaps they chose not to use GEE because of the small sample size and relatively small number of clusters (14 schools total were randomly selected, 7 assigned to intervention).
Parshuram CS, Dryden-Palmer K, Farrell C, et al. Effect of a Pediatric Early Warning System on All-Cause Mortality in Hospitalized Pediatric Patients: The EPOCH Randomized Clinical Trial. JAMA.2018;319(10):1002–1012. doi:10.1001/jama.2018.0948
This cluster-randomized trial evaluated whether a pediatric early warning system could decrease all-cause mortality and significant clinical deterioration events at 21 hospitals in 7 countries. 10 hospitals were randomized to the intervention, and 11 were randomized to usual care. The Bedside Paediatric Early Warning System (BedsidePEWS) was implemented for one year in each intervention hospital between 2011-2015 in inpatient, non-critical care units. The intervention involved calculating a score for each patient and recommending appropriate action if the patient’s condition was at risk of deteriorating.
The unit of clustering was the hospital.
The hypothesis was that the BedsidePEWS would decrease all-cause mortality, clinical deterioration events, and other secondary outcomes.
The exposure was at the hospital level, as the intervention was randomized at the hospital level. Hospital-level data were used for most outcomes as well; however, individual-level data was used for some secondary outcomes (e.g. urgent ICU admission outcomes), accounting for clustering at the hospital level.
The authors used generalized estimating equations with an exchangeable correlation structure and grouping by hospital to assess the effect of BedsidePEWS. They used logistic models for binary outcomes, Poisson models for count outcomes with patient-days as an offset, and Gaussian models for continuous outcomes. They adjusted for each center’s baseline summary value in the models. They adjusted for multiple comparisons of the many secondary outcomes using Holm’s method. They found that the intervention did not reduce all-cause mortality (adjusted odds ratio, 1.01 [95% CI, 0.61 to 1.69]; P = .96), but did decrease significant clinical deterioration events (adjusted rate ratio, 0.77 [95% CI, 0.61 to 0.97]; P = .03).
They could have used a repeated measures ANOVA/ANCOVA to evaluate the effect given that they had two time points: both a baseline and follow-up measure for each outcome. I’m am not sure the relative advantages of GEE vs. repeated measures ANOVA/ANCOVA, and am keen to learn more about the different ways to analyze such data.
Article: Caselli RJ, Langlais BT, Dueck AC, Locke DEC, Woodruff BK. Subjective cognitive impairment and the broad autism phenotype. Alzheimer Dis Assoc Disord 2018;32:284-290.
Brief background: Many individuals without a clinically recognized autism spectrum disorder possess broad autism phenotypes (BAP) and are thus likely to have been included in most/all existing studies of cognitive aging and dementia. Characteristics of BAP include stress-prone personalities and increased anxiety and depression. Subsequently, individuals with BAP may have substantially impacted findings from any study looking at subjective cognitive impairment, as prior research has shown increased levels of psychological distress may result in overreport of subjective cognitive decline.
Unit of clustering: Individuals (longitudinal data on individuals)
Hypothesized effect: Subjective reporting of cognitive decline would be greater in individuals with BAP. Differences in subjective scores would differ before and after age 60.
Level at which the exposure is measured: individual visit (observation within the cluster)
Statistical approach: linear mixed effects regression
Alternative approach: GEE is commonly used in longitudinal analysis as it works well with one level of clustering and when the number of clusters (here, number of individuals) is much larger than the number of observations. With 419 individuals in this analysis, I think these criteria were likely met. (The authors did perform contrast analysis which I believe motivated their use of mixed model).
Article:
Bluthenthal, R. N., Do, D. P., Finch, B., Martinez, A., Edlin, B. R., & Kral, A. H. (2007). Community Characteristics Associated with HIV Risk among Injection Drug Users in the San Francisco Bay Area: A Multilevel Analysis. Journal of Urban Health, 84, 653–666. doi:10.1007/s11524-007-9213-3
The unit of clustering: This study uses several census-tract-level measures of demographic and socioeconomic disadvantage. Four community-level census measures are used: 1) percent African American, 2) percent male unemployment, 3) percent of households that receive public assistance, and 4) median household income. A composite measure was also examined: The average of four deprivation indicators (proportion of 16- to 19-year-old high school dropouts, male unemployment rate, households reporting receipt of public assistance, and female-headed households).
The hypothesized effects and the level at which the exposure is measured: Community characteristics that have been previously identified as possibly associated with HIV health risks include residential segregation, poverty, income inequality, high unemployment, and various other indices of social and economic disadvantage. The hypothesized effect is that community-level characteristics representing deprivation, inequality, or disadvantage will impact individual HIV risk-behaviors and HIV infection i.e. individuals within tracts are non-independent. The census-level characteristics were measured at the level of the census (however, they are interpreted as community characteristics.) Repeated observations within a census tract are hypothesized to correlated because residents share certain socioeconomic disadvantages.
The statistical model used to estimate the effect: They use a two-level random intercept logistic regression model. They conduct separate analyses to estimate the log-odds for the probability of an individual engaging four different risky behaviors of interest while simultaneously controlling for individual-level and census tract-level characteristics. These four behaviors (outcomes) are: 1) receptive syringe sharing, 2) distributive syringe sharing, 3) engaging in unprotected sex, and 4) having multiple sex partners
Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed). I am not exactly clear on the differences between GEE and mixed effects methods, but it seems that GEE would not be appropriate for this analysis because of the data structure. Also, they did not report on missing data in their study.Article: Jemal A., Kulldorff M., Devesa S. S., Hayes B. R., Fraumeni F. J. A geographic analysis of prostate cancer mortality in the United States, 1970-89. Int. J. Cancer: 101, 168–174, 2002.
Cluster study to examine if a reported increase in European American cancer morality in north west/central region and an increase in African American I south Atlantic region was significantly different from other regions in the united states or can and can this difference be explained by demographic and socioeconomic differences.
Unit of clustering: Geographic region were the unit of clustering but used layer clusters of counties to establish significant cluster areas. Comparing regional changes in prostate cancer mortality among European American and African American men.
Hypothesized effects and the level at which the exposure is measured: The atlas regional differences in prostate cancer morality by race are different from other regions in the U.S and as an additional hypothesis these differences may be explained by demographic and socioeconomic factors. The exposure is measured with observations within the geographic location using countries.
Statistical model used to estimate the effect: The authors use spatial scan statistic to estimate cluster significance, deaths in counties are calculated using Poisson distribution. Using this calculation, a maximum likelihood of morality is produced which is found in the primary cluster (least likely to be by chance.) This maximum is compared to the actual from the data set and a simulated p value is estimated.
I did not understand the primary model of analysis but seeing what they are trying to accomplish I believe a GEE model would work here as there is a large number of clusters.
Article: PMID: 20723142, MING LU et al Chronic obstructive pulmonary disease in the absence of chronic bronchitis in China, Respirology (2010) 15, 1072–1078
the unit of clustering: after reviewing the lecture several times, I am not sure I understand the meaning of clustering. Does this term only apply to multi-stage sampling? If so, then I don’t understand the figure on slide 29. Is the meaning of the figure that both are examples of clustering or only two-stage sample creates clustering?
Unfortunately, when I set out to find a paper in pulmonary medicine related to sampling, I didn’t appreciate your instructions that you wanted “clustering” and looked for papers related to the search words “Stratified Random Sampling lung disease” and found a paper that was interesting to me, but after reading it, I think it’s not the sampling example you wanted us to find.
I went back and tried to find a multi-stage study instead and found one that described itself as “an epidemiological survey of COPD in China in 2003….[that] was conducted in seven provinces/cities in China.” I think this means the unit of clustering is cities/provinces in China.
the hypothesized effects and the level at which the exposure is measured (is it a characteristic of the cluster or the observation within the cluster); I think the exposure was measured is an observation within the cluster which used individual interviews by trained interviewers using a standardized questionnaire and a portable spirometer to measure basic lung function.
and the statistical model used to estimate the effect. They performed a descriptive and analytic study. They described the disease prevalence (cross-sectional study). Then they dichotomized their subjects into those with lung disease and those without and compared demographic variables and presented data from a t test.
Then they assess for potential determinants of lung disease using multivariate logistic regression such as age, smoking, occupation, FH, lung disease as child.
Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed). This study doesn’t seem to be very sophisticated, (i.e., I could repeat all the analysis they did without knowing ANYTHING from the sampling lecture….) therefore, it is not much of a leap for me to say that they probably should have used other methods of analysis with were described in the first lecture, but I really don’t understand them enough to describe them.
Article: Clouston K, Katz A, Martens PJ, et al. Does access to a colorectal cancer screening website and/or a nurse-managed telephone help line provided to patients by their family physician increase fecal occult blood test uptake?: results from a pragmatic cluster randomized controlled trial. BMC Cancer. 2014;14(1):263. doi:10.1186/1471-2407-14-263
In this article authors describe a cluster randomized controlled trial in which they evaluated the effect of an intervention aimed to increase colorectal cancer screening uptake. All physicians in the study followed their usual practice regarding recommending CRC screening, but physicians in the intervention group also provided a fridge magnet that had information for accessing a nurse-managed telephone support line and/or CRC screening website. The unit of clustering were 39 medical clinics; of these 19 were assigned to be controls (and included 39 physicians and 1,174 patients) and 20 were assigned to the intervention arm (with 40 physicians and 1,221 patients). The exposure and the main outcome (FOBT completion within four months) were measured at the individual level. Authors hypothesized that the study intervention would increase patient fecal occult blood test uptake in 15%, representing an effect estimate of OR=1.83. To estimate the effect authors used multilevel mixed models and considered age, gender, SES, use of EMR and participation in primary care reform as fixed effects and clinic and physician as random effects. ICC’s for each level (cluster, physician and patients) were also calculated. I think that for the purposes of this study the approach adopted by the researchers was appropriate and although GEE would allow to calculate average effects it would not be as useful as the mixed models.
hello Eduardo!
ever since learning about ICC's in 203, I have been interested in finding examples of their use.
your summary caught my eye for that reason. I will try to look at the paper you reported, but did you happen to understand exactly what they did in their ICC analysis? I'm coming from the basic science perspective so I understand using ICC to assess cor between say blood levels of a marker, but practically, how did they employ the ICC analysis for "cluster" or "physician"? I'm just not conceptualizing the parameters used in the ICC formula?
no worries if you are unsure as this wasn't the point of the exercise necessarily
Laura
Article selected: J Neurol 2011; 258: 826-34. “Does the clinical practice guideline on Parkinson's disease change health outcomes? A cluster randomized controlled trial” Larisch A1, Reuss A, Oertel WH, Eggert K.
Unit of cluster: neurologist (n=39)
Randomization was stratified by federal state (two federal states in Germany were included) and structure of center (specialized practice with >40 PD patients/quarter)
Intervention=published practice guidelines for treatment of Parkinson’s disease (PD) were mailed with specific instructions PLUS 4h training seminar (in person or video)
Control=published guidelines were sent by mail without instructions and without seminar/video
Primary outcome= change in PD patients’ QoL measured by the PDQ-39 summary index (questionnaire) from baseline to 6 months
Hypothesized effects at the level of which exposure in measured: From their power/sample size calculation: “a difference in the PDQ-39 summary
index of 4.4 points between intervention and controls, at 6 months would result
in 90% power for 800 patients, accounting for a drop-out rate of 15% and
inflation factor due to clustering at of 1.38“.
Statistical model used: For primary outcome, the authors used a “Proc Mixed” procedure from SAS to estimate random and fixed effects (a hierarchical linear model) with intervention, time and interaction intervention*time as fixed effects, and neurologists (cluster) and patients as random effects.
For secondary dichotomous outcomes, a GEE using SAS Proc Genmod was used.
Other more appropriate models that could have been used: I think that the authors used an appropriate method to account for the correlation in outcomes in members of the same group/cluster, and appropriately assigned the clustering and individual members as random effects. The authors could have used a GEE also for the primary outcome (which was a continuous variable). Since they had 39 clusters (39 neurologists) and based on Murray et al 2018, the GEE models did not need a correction for having less than 38df (this analysis had 38df).
Munneke et al. Efficacy of community-based physiotherapy networks for patients with Parkinson’s disease: a cluster-randomised trial. Lancet Neurol. 2010 Jan;9(1):46-54. doi: 10.1016/S1474-4422(09)70327-8
This is a cluster-randomized trial that examined the effect of community-based physiotherapy networks in the Netherlands on health-care outcomes of patients with Parkinson’s disease.
The unit of clustering was community hospitals near three universities in the Netherlands and their catchment area. There 16 units, 8 for each arm.
The authors hypothesized that physiotherapists with specific training of Parkinson’s disease, a structured referral process to the participating physiotherapists, and optimization of communication between health professionals will decrease the primary outcome patient-specific index of Parkinson’s disease. The standardized effect size used for power calculation was 0.4.
Since this is a cluster-randomized trial, the exposure variable was the allocation to receive a community-based physiotherapy network or standard of care (usual referral process). This intervention was the same to all the patients in a cluster, so the “exposure” was a characteristic of the cluster.
The main statistical analysis was with “random effects model with random factor (cluster) and fixed variables (baseline value) and cluster size (number of citizens).” Since participants had multiple measurements at different times, participants were added as an additional random factor and time was added as a fixed factor.
The model used seems appropriate since it takes into account that this is hierarchical data. Since this study was a randomized study, a simpler statistical test such as a 2-sample t-test with unequal variance could have been used but they wanted to adjust for baseline covariates and cluster size, so the mixed model was appropriate. The advantage of using GEE would be that we don’t need to make strong assumptions about the correlation structure, but mixed models are better when we are working with different lengths of follow-up.
Dr. Glymour and classmates,
any chance we will be reviewing models such as that mentioned in Monica's summary:
The main statistical analysis was with “random effects model with random factor (cluster) and fixed variables (baseline value) and cluster size (number of citizens).” Since participants had multiple measurements at different times, participants were added as an additional random factor and time was added as a fixed factor.
I have only the background offered by epi203, 207 and biostats 200, 208 and just have this one quarter left to get familiar with these more sophisticated approaches. Would appreciate having a "general" understanding of them, recognizing that I may not be able to "perform" them myself. Perhaps, the name of the model is more intimidating than appreciating the general mathematical structure used for this modeling??.
if anyone out there can suggest any basic references for me to read, or a quick blurb on the mathematical structure? I would greatly appreciate it!!
Hi Laura,
You can review chapter 7 of the book Regression Methods in Biostatistics of Eric Vittinghoff et al.
Lean MEJ, Leslie WS, Barnes AC, et al. Primary care-led weight management for remission of type 2 diabetes (DiRECT): an open-label, cluster-randomised trial.Lancet. 2018; 391(10120):541-551.
It is a cluster-randomized clinical trial with the aim to evaluate whether intensive weight management within routine primary care would achieve remission of type 2 diabetes in adults 20-65 years old with diabetes diagnosis and BMI of 27-45 kg/m2.
The unit of clustering was primary care practices.
The hypothesized effects were a diabetes remission in 22% of the participants in the intervention group at 1 year vs. 5% in the control group,
The level at which the exposure is measured was primary care clinics, in which the randomization was at that level. Therefore, clinics were assigned to the intensive weight management (exposure group) or to the standard of care management (non-expose group).
The statistical model used to estimate the effect was mixed-effects regression models, with adjustment for primary care practice as a random effect.
Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed).
I think they used an adequate model. However, GEE could also be appropriate, since the number of clinics was 49, with 10 participants on each clinic. GEE fits better when there is at least 38 df, and there are more clusters than observations per cluster. One advantage of mixed models is that those are more robust due to missing data and loss of follow-up, but GEE has more robust SEs.
I read through the responses of my classmates, and they provided me with introspection of two publications in which I used clustered data. In retrospect, I may have used clustered data well in one article but sub-optimally in the other article. One article used data from 2013 Demographic and Health Survey (DHS) of Sierra Leone to assess HIV stigma as a driver of HIV transmission risk behaviors. The other article used cross-sectional data from a village in Sierra Leone to assess food insecurity as a driver of Ebola virus infection and death.
In the HIV article, we used DHS data, and the survey employed a stratified, two-stage cluster sampling design. The sample was designed to select a representative probability sample of households. The unit of clustering was an enumerated area in the master sampling frame of the country. We hypothesized that higher levels of HIV stigma would be associated with increased HIV transmission risk behaviors. We assessed HIV stigma as a community-level and individual-level exposure. Our statistical model used a logistic regression that accounted for fixed effects and probability weights of the sample. We did not include random effects and I thought the rationale had to do with the representative sampling technique. It’s possible that there were other reasons but I’m not sure of the full rationale. At the time, I used the vce command in STATA but have since learned other ways of creating fixed effects models, including OLS and GEE.
In the Ebola article, we collected data that over-represented Ebola-affected households and then randomly sampled unaffected households. Households were the unit of clustering in this sample. We hypothesized that higher levels of food insecurity would be associated with increased Ebola virus infection and death. We assessed food insecurity as an individual-level exposure. In our statistical model, we only considered fixed effects, particularly the within variation of food insecurity among households. There were over 30 households and our sampling technique across households varied. It’s possible that we could have created probability weights to make findings representative across the village, but we didn’t, so that’s one limitation of the inference. Regardless, there were differences across households, and random effects could have accounted for between-group variation. We could have used GEE or mixed effects models but instead we used a logistic regression that included the clustered command (vce). In retrospect, by focusing on fixed effects, I may have pursued a less efficient way to analyze the data whereas GEE or mixed effects would have been preferable.
References:
Kelly JD, Reid MJ, Lahiff M, Tsai AC, Weiser SD. Community-Level HIV Stigma as a Driver for HIV Transmission Risk Behaviors and Sexually Transmitted Diseases in Sierra Leone: A Population-Based Study. JAIDS. 2017 Aug;75(4):399-407. PMID: 28406807.
Kelly JD, Richardson ET, Drasher M, Barrie MB, Karku S, Kamara M, Hann K, Dierberg K, Hubbard A, Lindan CP, Farmer PE, Rutherford GW, Weiser SD. Food insecurity as a risk factor for outcomes related to Ebola virus disease in Kono District, Sierra Leone: a cross-sectional study. Am J Trop Med Hyg. 2018 May;98(5):1484-1488. doi: 10.4269/ajtmh.17-0820. PMID: 29557329.
Rebecca Wyse, Gnel Gabrielyan, Luke Wolfenden, Serene Yoong, Jeffrey Swigert, Tessa Delaney, Christophe Lecathelinais, Jia Ying Ooi, Jess Pinfold, David Just, Can changing the position of online menu items increase selection of fruit and vegetable snacks? A cluster randomized trial within an online canteen ordering system in Australian primary schools, The American Journal of Clinical Nutrition, , nqy351, https://doi-org.ucsf.idm.oclc.org/10.1093/ajcn/nqy351
This is an 8 week-long non-blinded clustered RCT assessing whether putting fruit/vegetable menu options first and last in school’s online lunch menus would increase the proportion of lunch orders of fruits/vegetables ordered by parents or students.
The unit of clustering: Six different primary schools, all of which have been using online lunch orders for at least 6 months and receive a minimum of 50 orders per month, were randomly assigned to control or intervention clusters at 1:1 ratio.
The hypothesized effects and the level at which the exposure is measured: The exposure (changing the online menu as the intervention) was characteristic of each cluster as a whole. The hypothesized effect was increase in the primary outcome of interest (the proportion of orders that contained at least one fruit/vegetable), but no specific level of increase was predicted.
The statistical model used to estimate the effect: Intend to treat approach was used in a linear mixed model, adjusting for school clusters and also number of orders placed by the same student.
Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed): I think this mixed model is appropriate, specially given the low number of clusters and high number of observations (GEE may not have been appropriate for this reason).Dan,
I can try to respond to some of your questions as best as I can, although if anyone has a different understanding of these topics feel free to chime in!
For your DHS analysis: you say you included FEs and clustered the standard errors as well. I am assuming these were enumeration area-level fixed effects? It's my understanding that your rationale for not using random effects doesn't hold here. Random effects require the assumption that the enumeration areas' error term should be normal around the intercept. This should hold here, since enumeration areas are randomly selected (after stratification -- which leads me to assume you should include FEs for the variable you stratified on, like urban/rural or region -- I know each DHS uses a different stratification method. But I'm not sure about that.) A huge benefit to using FEs here is that, by only looking at within-EA variation, you're essentially controlling for all potential confounders that exist between EAs. For example, if population density differs between EAs and is a confounder for the relationship between stigma and transmission, you're basically controlling for it even though you're not including it in the model. I am guessing that is why you also included fixed effects in your second paper. Perhaps there were factors that differed between households and may confound the relationship between food insecurity and EVD?
In any case, it's interesting to me that you assess food insecurity as an individual-level factor and not at the household level! I'd be curious to learn more. I will read the paper :)
Adrienne
Adrienne, thanks for breaking some of this down for me as I try to better understand how FEs and random effects work...
Maybe the assessment of food insecurity at the individual level have to do with assumptions about allocation of resources within the household in this population? and on individual members' access to additional resources outside the household? Regardless... I'm curious if you could spell out how the individual vs household assessment of food insecurity would change how it should be handled in the model. Should they have accounted for clustering at the household level, if so how?
Nice post, Andrea!
It sounds like this study actually had patients clustered inside physicians clustered inside clinics. Because you mentioned there are only 10 patients per clinic, I can understand why the authors may have chose not to account for another clustering level for physicians. Maria - it would be nice to discuss when you would want to adjust for this additional level vs when it is okay to ignore (as done here). If anyone else has thoughts, I'd love to hear them.
It sounds like this study actually had patients clustered inside physicians clustered inside clinics. Because you mentioned there are only 10 patients per clinic, I can understand why the authors may have chose not to account for another clustering level for physicians. Maria - it would be nice to discuss when you would want to adjust for this additional level vs when it is okay to ignore (as done here). If anyone else has thoughts, I'd love to hear them.
Hi Andrea,
Given the advantages of a mixed effects model in regards to missing data and loss to follow-up, did the authors address either of these issues in the manuscript?
-Teresa
Solomon DH,
Losina E, Lu B, Zak A, Corrigan C, Lee SB, et al. Implementation of
Treat-to-Target in Rheumatoid Arthritis Through a Learning Collaborative:
Results of a Randomized Controlled Trial. Arthritis & rheumatology
(Hoboken, NJ). 2017;69(7):1374-80.
Available at: https://onlinelibrary.wiley.com/doi/full/10.1002/art.40111
Aim: to test the effectiveness of a learning collaborative for improving implementation of treat-to-target (TTT) principles in rheumatoid arthritis (RA).
Exposure: a learning collaborative delivered during a period of 9 months
Outcome: change in composite TTT implementation score from baseline. Composite TTT implementation score has 4 components: 1) specifying a disease activity target, 2) recording RA disease activity, 3) documenting shared decision making, and 4) basing treatment decisions on target and disease activity or describing why TTT was not adhered to.
Unit of clustering: US rheumatology practice sites (11 recruited). They were diverse in geography, patient populations and organization.
Hypothesized effects: Authors hypothesized that adherence to the TTT would improve to a greater extent in patients seen at rheumatology sites randomized to the learning collaborative intervention as compared with patients seen at the control sites. Authors assumed that the control group would have no or only a small change (5%) in the TTT implementation score during the 9-month period compared with a change in the intervention group of about 20%; i.e. a between-arm effect size of 15% on the absolute scale. These assumptions were based on the improvement level observed in a similar trial using a learning collaborative. Authors also knew that the average number of providers at each of the 11 practice sites would be 4 and that there would be moderate intra-cluster correlation among patients within a given provider and site; they assumed an intra-cluster correlation of about 0.2 based on previous work. The statistically significant alpha level was set at a 2-sided P value of 0.05. Based on these assumptions, authors estimated that to ensure 80% power, the required number of patients per provider would be 5.
Level at which the exposure is measured (characteristics of the cluster or the observation within the cluster): while the experimental unit (level at which exposure is assigned) was site, the unit of analysis (level at which the outcome is measured) was patient. The composite TTT implementation score was calculated for each patient at baseline and at follow up visits. The change in composite TTT implementation score (primary endpoint) was then calculated for each patient. It is a characteristic of the observations within the cluster.
Statistical model used to estimate the effect: authors state “primary analysis compared the mean change in the TTT implementation score over 9 months across patients in the intervention sites as compared with the mean change for the patients in the control sites after accounting for intra-site and intra-provider correlation using linear mixed models. Any patient covariates that were imbalanced at the baseline visit were considered for the model, including age, sex, baseline disease activity, baseline RA drugs, and disease duration”. My interpretation of their description is that they used repeated measure mixed-effects model with two time points for each patient (at baseline and at 9 months), composite TTT implementation score as the outcome and an interaction term between timepoint and exposure to determine the effect of exposure in change in TTT score over the study period. Their model was adjusted to control for potential confounders that were appropriately selected.
They also had secondary outcomes which were as follows: 1) percentage of patients with any positive change in implementation score between baseline and follow-up, and 2) proportion of patients with full implementation of all TTT items at follow-up. For these authors used generalized linear mixed models for binary outcomes that accounted for clustering within sites and within providers.
Analysis of adverse events and resource utilization was performed using Poisson regressions for resource use and adverse events as the outcome variables. Similar to the primary analyses, they adjusted for intra-site and intra-provider correlations.
Any other statistical models that may be appropriate: GEE would have been appropriate with clustering at site level. Per Murray’s paper they may have had to use small-sample correction if they had <38 degrees of freedom. I think authors may have chosen mixed-effects over GEE because it allowed them to account for both intra-site and intra-provider correlation. Another reason why they may have chosen the mixed effects model is that they had missing data for a number of covariates that they used in the model.
Any other statistical models that would be preferred: GEE produces robust estimates and relaxes modelling assumption but can be less efficient than mixed effects if the correct model can be specified. I think authors choice is an appropriate choice for this study given the relatively small sample and presumably having two time-points in the analysis. It may also be the preferred choice based on the reasons above.
Denny S, Robinson E, Lawler C, et al. Association between availability and quality of health services in schools and reproductive health outcomes among students: a multilevel observational study. Am J Public Health. 2012;102:e14–e20. (doi:10.2105/AJPH.2012.300775)
https://ajph.aphapublications.org/doi/pdf/10.2105/AJPH.2012.300775
Unit of clustering: Schools. The data for this study were extracted from a large health survey of high-school students in New Zealand in 2007. This survey used a two-stage clustered sampling design in which 115 schools were first randomly selected from among all 389 schools with high-school students in the country. In the second stage, over 12,000 students were randomly selected from among all high-school students enrolled in each of these 115 schools.
Hypothesized effects: Study investigators hypothesized that increased access to school-based health services improves sexual and reproductive health among high-school students.
Level at which the exposure is measured: The exposure of interest in this study is the availability and quality of school-based health services. This exposure was measured in several ways, including 1) the number of nursing and doctor hours per week per 100 students, 2) whether a school offered doctor visits, 3) whether a school’s health practitioners held team-based meetings weekly or more often, 4) whether the health team met with the school counselor or health teacher each term or more often, and 5) whether a school performed a routine comprehensive health screening on all year-9 students. Each of these five measurements of the exposure was measured at the school level.
Statistical model used to estimate the effect: The study investigators used generalized linear mixed models with school treated as a random effect to estimate the effects of each of these cluster-level exposure variables on individual student sexual and reproductive health outcomes, adjusting for relevant cluster-level and individual-level covariates. The two outcomes they evaluated were consistent condom and/or contraception use among sexually active students and “being involved with a pregnancy” (that is, having ever been pregnant or having ever gotten someone pregnant) among sexually active students.
Other statistical model that may be appropriate or preferable: A generalized estimating equation (GEE) model may be preferable for this analysis since it fits a marginal model, allowing us to target inference at the population-level (school-level). We are more interested in the school-wise average rates of consistent condom use and pregnancy than in the differences among individual students.