Assignment: Find any article using clustered data and describe: the unit of clustering; the hypothesized effects and the level at which the exposure is measured (is it a characteristic of the cluster or the observation within the cluster); and the statistical model used to estimate the effect. Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed). It is helpful if you post the reference for the article or a link.
http://www-ncbi-nlm-nih-gov.ucsf.idm.oclc.org/pubmed/19624565
Geographic Variation in Organ Availability Is Responsible for Disparities in Liver Transplantation between Hispanics and Caucasians
Am J Transplant. 2009 Sep;9(9):2113-8. doi: 10.1111/j.1600-6143.2009.02744.x. Epub 2009 Jul 16.
File is attached.
This is an example of clustering from the field that I'm currently performing research in - liver transplantation.
Here clustering was used to investigate if Donor Service Area (DSA; Regional unit of transplant allocation) influenced hazards of transplantation and death/removal of hispanics using caucasians as a comparator. The hypothesized effect was that geographic variability may contribute to disparities.
The unit of clustering was DSA. The primary outcomes were the hazard of receiving a transplant and death/removal. This is a characteristic of the observation within the cluster.
The statistical model used to estimate the effect was a multilevel model, which seems appropriate. I'm not very familiar with other models that would be appropriate or preferable. Perhaps of relevance, the author reported the following "
The Hausman specification test for appropriateness of a random-effects estimator was statistically significant, so results are presented using a fixed-effects model (stratifiedbyDSA) with robust standard errors" - which I would like help interpreting if possible.
By adjusting for DSA-level effects in the multilevel model, the hazards of transplantation and death/removal for hispanics were no longer different from those of Caucasains. This suggested, as the title presents, that geographic variation in organ availability is responsible for disparities.
Nice example. Usually people use fixed effects models instead of multilevel models, so I am not sure what they actually did here, since they describe the model as both fixed effects and multilevel. How many DSAs are there?
58.
I'm a little unclear on the distinction of fixed effects and multilevel and their use in clustered data.
Is a multilevel model simply one that takes into consideration both fixed and random effects? How is a fixed effects model applied to cluster data/how do you account for the random subject level or cluster level impact?
There is a flurry of confusing language here, I believe in part because the same statistical tool was developed or popularized in different disciplines and called things to contrast it to the standard in the discipline at that time.
Mixed models are so called because they include both fixed effects and random effects. Sometimes people just say "random effects models" and also sometimes they say "hierarchical models" or "hierarchical linear models".
Distinguish the data (which is multilevel and includes information on cluster and individual level variables) and the statistical tool to analyze the data, which may be called a multilevel model.
There are two perfectly common uses of "fixed effects" in this setting, and you have to figure out which is which based on context. One use (which I believe they were invoking in that paper) is to include an indicator variable for each cluster (e.g., a binary variable for whether you were in cluster 1, another binary variable for whether you were in cluster 2... up to 1-# of clusters). This is another way to account for clustering, but not as common as mixed models because it has some disadvantages.
See article by Schwartz et al 2006
"UV, latitude, and spatial trends in prostate cancer mortality: All sunlight is not the same (United States)"
*I actually would like some clarification around ecological studies versus clustering data around location. Are they the same sort of method?
- Unit of clustering-individuals were grouped by counties and state economic areas (SEAs)
-Hypothesis: Those counties that receive less sunlight (and therefore less vitamin D synthesis) will have a higher rate of mortality due to prostate cancer.
- Exposure: The exposure is amount of UV radiation supplied from the National Oceanic and Atmospheric Administration; which is measured county by county basis in this paper
-Statistical model: They used basic linear regression to determine the relationship between latitude of the county and prostate cancer mortality, which was adjusted for age. Only white men were included in the study, as they say "Data for Black males are too sparse for national-level analyses." There was no trend below 40 degrees latitude but was a significant trend above 40 degrees latitude where vitamin D synthesis is limited to non-winter months. I think the models used in this paper were appropriate.
Very interesting example. Note that the exposure of interest here - UV radiation - is really an ecological/place level feature, not an individual level feature that has been aggregated up. Usually when people critique ecological studies it is because the study uses information on the association between aggregated variables (e.g., % immigrant in a state and % illiterate in a state, to select a famous example) to draw incorrect/spurious inferences about individual level associations, e..g, "immigrants tend to be illiterate". In the case such as this one, where the predictor of interest is a place level variable, a multilevel model is appropriate. Although you could say this was an ecological study, it does not have the same inference problems as classic (and widely critiqued) ecological studies.
Bermedo-Carrasco et al. - Inequities in cervical cancer screening among Colombian women: A multilevel analysis of a nationwide survey
http://www-ncbi-nlm-nih-gov.ucsf.idm.oclc.org/pubmed/25707752
(Enclosed in attached zip file)
The study enclosed sought to understand the factors associated with whether women in Colombia have had a Pap test. Cervical cancer is the second most incident cancer in Colombia, and the second highest cause of cancer-related mortality among women between ages 15 and 44.
Researchers identified three levels: 1) women; 2) neighborhoods; and 3) municipalities to measure the dichotomous outcome of ever having a Pap test. They hypothesized that there would be differences in Pap testing based on urbanicity, geographic region of residence, and sociodemographic factors, including education level, wealth quintile, and type of health insurance. Another major predictor measured was neighborhood prevalence of "no education", which was measured at the neighborhood (second) level.
In this study, a three-level mixed effects model was used. At the municipality-level and neighborhood-level, effects were classified as random effects in this study to account for potential variability in Pap test uptake at these higher levels. By clustering at the neighborhood and municipal levels, researchers could account for potential effects of neighborhood-based influences and geographic region of residence on Pap test uptake. For the purposes of this study, a mixed-effects model was most appropriate.
The findings from the study indicated that women in lower socioeconomic groups were significantly less likely to have had a pap test when accounting for other variables. Additionally, the lack of education at the neighborhood level led to a decrease in Pap test uptake, a finding which was pronounced in rural areas. The findings indicated that more defined strategies should focus on individual and contextual factors to improve Pap test uptake in women, to improve upon current cervical cancer screening programs.
Maskarinec et al. A longitudinal investigation of mammographic density: the Multiethnic Cohort. 2006. http://cebp.aacrjournals.org/content/15/4/732
This study investigated the effects of established breast cancer risk factors on changes in breast density over time. Changes in breast density over time have been shown to predict changes in breast cancer risk, cumulative exposure to breast density over time has been hypothesized to be important for breast cancer risk.
The unit of clustering here is the woman: repeated measures of breast density are taken for each woman in the study. This study attempted to determine if age, ethnicity, hormone replacement therapy, reproductive and dietary factors affected changes in breast density over time. All exposures were measured at the individual level. The hypothesized effect of age, menopause and BMI are to decrease breast density; other factors have been shown to affect cross-sectional measures of breast density but were not given a specific hypothesis in terms of the direction of the longitudinal change in breast density.
A mixed model was used to assess the effect of the exposures on percent density. Age, centered and scaled to a 10-year increase, was used as the time variable in this study. Exposures / covariates were included in the model as well as an interaction term for each variable with age (time variable) to assess if any of the variables caused changes in density with increasing age / over time. A mixed model was chosen to model changes in percent density; GEE could have been used as well but because the mammograms for each woman were not taken at standard intervals (e.g., women could have anywhere from 1-5 mammograms taken at different intervals) mixed may give more robust results than GEE. Further, I think the random intercept and woman-specific interpretation of the mixed model may be more appropriate in this setting because there is substantial variation in baseline percent density between women.
The findings indicate that age and menopause caused significant declines in breast density, whereas HRT use (combined E+P) delayed the decline in density over time. High BMI also delayed decline in density over time. Despite strong associations between other factors and baseline density, no other predictors were significant associated with changes in breast density over time.
Nice example Natalie. So they didn't use any level 1 (time-varying, in this case) covariates?
Josh-So they found both an individual and neighborhood level effect of education?
Nice example.
" Pancreatic Enzyme Replacement Therapy Dosing and Nutritional Outcomes in Children with Cystic Fibrosis" Haupt et al the Journal of Pediatrics May 2014 PMID: 24560182
File is attached.
This is an example of clustering in cystic fibrosis research
Clustering is used to investigate whether pancreatic enzyme dosing might be a determinant of better nutritional status (BMI). The CF center was used as the unit of clustering. Although analyses were completed at the patient level as well as at the CF center level.
Hypothesis: that CF centers with larger patients (higher average BMI percentile) will use larger doses of pancreatic enzymes.
Exposure: BMI quartile. The centers were grouped into BMI quartiles according to the mean adjusted BMI percentile from each center in 2008
Statistical Model: They used t testing and chi square analysis to compare between bottom and top BMI performances quartiles along static characteristics. For characteristics that varied over time, they used mixed models with a fixed effect of year and a covariance structures nesting patients within programs. They also used a mixed model to assess differences in enzyme dose by BMI quartile and adjusted for patient level characteristics such as patient BMI, age, race/ethnicity, FEV1% predicted, acid-blocker use, presences of pseudomonas, supplemental feeds, growth hormones and diagnosis of CF related diabetes. Finally they looked at the trend of enzyme dose within and between all quartiles over time using wilcoxon rank sums.
I think the choices of model were appropriate but I am unsure of whether they might have over adjusted. I would have liked to have heard some refinement on the choice of variables in their adjusted model as it just seems like a lot.
Danielle, This is such an unusual study! We should discuss in class because if I understand correctly, what they have done is quite odd. First, it seems that their fundamental question is whether enzyme dose improves nutritional status, as measured by BMI, for cystic fibrosis patients. Yet, they are using enzyme dose as the outcome in most of their analyses, and predicting by an ecological (center-level) measure of BMI.
Further, they are explicitly trying to get around confounding by indication by using ecological measures: "We attempted the approach of using average program practice and outcomes as a type of ecologic study to address the problem of indication bias, as used in 2 previous studies". This is potentially useful, and they actually motivate it as an instrumental variable, but it is unclear whether this is a valid IV - we can discuss more.
Very interesting article!
Wong, H. J., Sistrom, C. L., Benzer, T. I., Halpern, E. F., & Morra, D. J. (2013). Use of Imaging in the Emergency Department: Physicians Have Limited Effect on Variation. Radiology. doi:10.1148/radiol.13130972/-/DC1
http://www.ncbi.nlm.nih.gov/pubmed/23801769
RQ: How is variation in ED imaging use attributable to physicians?
Unit of clustering: Patient-level, Visit-level, Physician-Level
Hypothesized effects: Unclear, literature suggests there is wide variation in physician ordering behavior, but these models do not adequately take into account patient- and vist-level covariates
Level at which the exposure is measured: Characteristic of the cluster of individual physicians
Statistical model: 1) A two-level hierarchical logistic regression model in which patient, visit, and physician-level covariates are simultaneous fixed effects. The multilevel model was the log odds form
log (P(Yij)/1-P(Yij)) = β00 + β0i + γZi + θXij
P(Yij) is the overall probability of imaging use, β00 is the population average of the physician means of imaging use (as an intercept), β0i is the mean image use for the ith physician (as a random intercept), γ is the vector of fixed effects physician coefficients, Zi is the vector of physician covariates, θ is the vector of fixed effects patient and visit coefficients, and Xij is the vector of patient and visit covariates. This analysis was performed in parallel for high-cost imaging (MRI, CT, nuclear imaging), and low-cost imaging (x-ray, ultrasound).
2) Intraclass correlation coefficient was then calculated based on the above multilevel model estimated variance, representing the variation in imaging that is attributable to the physician level after accounting for patient/visit/physician covariates.
The authors found that most of the variation was attributable to patient and visit covariates. The ICC was low for physicians of approximately 1%, indicating that only 1% of the total imaging use variation was related to variation between physicians themselves. Although certain physicians could be identified as “high users” based on being outliers, they still only contributed little to the overall variation.
Critique: I believe this model seems appropriate in using a mixed model to calculate an ICC to measure the relative variance of individual physicians. I do disagree with their outcome measure as a clustered outcome of the probability of imaging use. The authors dichotomized between low and high-cost imaging and could have removed precision in detecting differences or magnitude of variance. I would propose using a continuous outcome such as cost per study ordered (e.g. relative value unit or RVU) in a linear multilevel regression model.
Interesting that their ICC was so low. Was this surprising to you?
The equation seems not quite right, in that it seems to conflate the equations for two different levels. Typically, one would include beta_not_i in one level and then separately decompose beta_not_i into beta_not_not and mu_not_i. As written, you add the overall average (beta not not) and the physician specific average (beta not i) together to get any individual's predicted value, which cannot be correct.
Associations across spatial patterns of disease incidences, socio-demographics and land use in Finland 1991-2010 Ari Voutilainen, Sirpa Hartikainen, Paula R. Sherwood
Unit of clustering: Municipality
Hypothesized effects: Spatial patterns in disease incidences in Finland are associated with socio-demographics and geometric location
Diseases described: 15 non-infectious diseases categorized into cancer versus non-cancer chronic diseases - coronary heart disease (CHD), hypertension, diabetes, asthma/chronic obstructive pulmonary disease, and serious mental illnesses.
Risk factor variables: 13 socio-demographic variables, and four environmental variables
Level at which exposure was measured: Municipality registries describing summarized population and enviromental characteristics
Description of method of collection and verification of registries a boost to internal validity
Used five country-level registers: Finnish Cancer Registry (FCR), Kelasto, maintained by the Social Insurance Institution (SII), SOTKAnet, maintained by the National Institute for Health and Welfare, Hertta, maintained by the authorities of Finland’s environmental administration, and Statistics Finland
statistical model used to estimate the effect
- Data clustering
- Spatial analysis
- Variation partioning
Other statistical models/Appropriateness: Not sure, maybe a time series analysis that would compare incidence rates across municipalities for the same period of time
Find any article using clustered data and describe: the unit of clustering; the hypothesized effects and the level at which the exposure is measured (is it a characteristic of the cluster or the observation within the cluster); and the statistical model used to estimate the effect. Describe whether there are any other statistical models that might be appropriate and whether they would be preferable (e.g., GEE vs mixed). It is helpful if you post the reference for the article or a link.
Unit of clustering: Individuals were clustered at the neighborhood-level; neighborhood cluster
Hypotheses: 1) Concentrated disadvantage decreases collective efficacy and residential stability increases collective efficacy 2) Collective efficacy explains the relationship between neighborhood disadvantage, residential instability, and rates of interpersonal violence 3) neighborhood concentrated disadvantage, immigration concentration, and residential stability explain a substantial portion of the variation in neighborhood collective efficacy.
Exposure: Informal social control and social cohesion were measured at the individual-level using 5-point Lykert scales and then aggregated to the neighborhood-level. The two scales were then combined into a summary measure labeled collective efficacy.
Statistical model used to estimate the effect: Multilevel(three-level) random effects model was used to estimate the effects of collective efficacy on neighborhood violence. First level were the individuals, second level was within neighborhood, third level was across neighborhoods.
It doesn't appear that a mixed effects model would be appropriate here since the exposure (collective efficacy) is not being assigned (do not know which factors would be fixed in this scenario) and I'm similarly unfamiliar with GEE to determine if it would be inappropriate to use in this particular study.
http://science.sciencemag.org.ucsf.idm.oclc.org/content/277/5328/918.full.pdf+html
Sampson, R. J., Raudenbush, S. W., & Earls, F. (1997). Neighborhoods and violent crime: A multilevel study of collective efficacy. Science, 277(5328), 918-924.
This study re-used a cohort of patients and serum samples previously used in another study of IBD patients. This study focused on 148 patients from the Karmiris cohort of Belgian adults with Crohn’s disease. It included 536 serum samples of the 148 patients. Thus, the unit of clustering was at the patient level, with each patient having multiple serum samples (i.e. 0, 4, 12 weeks then q3months). The hypothesis of the study was that low drug levels lead to drug-antibody formation, leading to accelerated drug clearance, progression of disease and finally loss of response to the medication. In order to evaluate this hypothesis, the authors performed a Cox proportional hazards survival analysis, censoring patients at the last time point that they were documented to be antibody free. The authors then went onto look at effects of drug and anti-drug antibody on inflammation (which would be marker of disease activity). In this case CRP levels acting as the outcome with either drug level or antibody level as the predictor. They chose to control for gender, age, and disease duration. It is interesting they did not control for concomitant therapy, as one of the other prevailing hypotheses it that use of additional therapy decreases likelihood of antibody formation. They go onto to describe their decision to use two models, one with CRP from the same serum sample and a second model using a future CRP level. I am not familiar with the reasoning or theory behind this two model approach. I think the authors used an appropriate statistical model and approach. I am curious about the thought process behind the multiple models they came up with an how they made those choices.
Melissa
Interesting paper- they used both a survival model (to evaluate the time-to-event binary outcome) and a mixed/random effects model (to evaluate dynamic changes in continuous outcomes, e.g., CRP levels). I don't understand the biological motivation for examining both current and future CRP. I think this is based on some theory about the underlying physiology - we can discuss in class but I think you'll have to explain the alternative hypothesized processes to us.
Ekland,
I am really glad you chose this paper because it is one of the most conceptually and methodologically influential papers in this area. I had forgotten what a lovely paper it is. I encourage everyone to look at it.
One really nice thing about this paper is that it makes explicit the link between latent variable models and multilevel models. They specify a 3 level outcome Yijk, where the k indexes neighborhoods (within Chicago), the j indicates people within neighborhoods, and the i indexes specific questions answered by each person.
They then note that Chicago has an overall average value of the latent trait (e.g., collective effficacy) and each neighborhood varies around that overall average, and each person varies around their personal neighborhood average, and each item of their measurement scale has a differential difficulty that influences an individual's response to that item.
Very nice example. Classic motivation of random effects/multilevel modeling. People often use mixed effects and multilevel models as synonyms, so you could call this either.
Nelson: this is an unusual paper. The primary focus here are ecological associations among variables (disease rates and municipality characteristics). They do not use the approach of either GEE or multilevel models because they really only aim to describe the patterns.
They end their article with a very nice note about the ecological fallacy: "The observed associations between diseases and their determinants are spatial associations based on municipal level data; they do not reveal actual relationships at the individual level. Although group-level associations arise from individual-level actions and qualities, and they can be supported by individual-level associations, conclusions drawn from the group-level associations cannot be returned to individuals as this leads to ecological fallacy."
Martin
That's a very entertaining study and a very clever approach to addressing a difficult set of questions about the physiologic changes in response to environmental stressors.
I found their description of independent and dependent variables confusing and I couldn't tell that they were always keeping the hypothesized dependent variables on the left hand side. They could have done this with mixed models (although their GEE approach might be preferable w/ only 17 traders).
Within the educational framework and associated theory, I am interested in investigations of different approaches to teaching and learning. For example whether training should be individualized versus group-based, lecture-based versus problem-based, etc.
To this end, I read the paper (attached) by Mergendoler et al., 2001, comparing problem-based learning (PBL) to lecture-based learning among high school students in the US in 1998. The students were being taught concepts and principles in the subject of Economics. The study was a non-randomized multi-level clustered intervention study, i.e., 3 teachers (level 1) teaching 9 classes (level 2) with a total of 186 students (level 3). The group-level variables were thus “teacher”, and “class”.
The primary predictor was a PBL curriculum, and the comparison treatment was “a traditional instructional context”. Knowledge attainment and “attitudes towards the class” were the main outcomes. The authors also assessed some individual-level covariates including: overall “academic ability”, “knowledge of economics at class entry”, “attitude towards economics”, and “academic risk-taking”.
Each of the 3 teachers selected some of their assigned classes at the beginning of the semester to be the treatment classes and others to be the comparison classes before they knew student assignment. They then taught the same concepts in all the classes but using PBL in the treatment classes and “traditional instruction” in the comparison classes. Two of the teachers ended up having 2 treatment classes and 1 comparison class each, while the 3rd teacher had all his 3 classes as treatment classes with no comparison.
Analytic methods included: 1) estimating correlation coefficients among the individual-level variables (the authors’ rationale for this was “to eliminate those [variables] that appeared redundant”), 2) estimating means of the individual-level variables by treatment group, and 3) ANOVA, “to examine relationships among treatment, outcome, and student characteristics. In this analysis, they compared outcome means by treatment group, first unadjusted, and then adjusted for covariates, grouped by teacher, and they tested for treatment-covariate interactions.
Analyses by teacher, showed that students in the traditional instruction arm “showed greater positive pre-post change in general economic knowledge”, but pooled analyses showed no significant difference between the two instructional approaches, suggesting that PBL is either inferior or, at best, equivalent to traditional instruction on the studied outcomes in this population.
Comment on the statistical models: ANOVA seems appropriate here given the preliminary nature of the study, but perhaps GEE could be a better alternative. I am not sure that the 2nd clustering level, i.e., class was considered during the analyses. I would expect that group-level variation in this case would occur both at the level of the teacher and at the level of the class to which the student is assigned, e.g., if Teacher A is teaching 2 classes, students in a specific class may show some variation that is explained by the specific class to which they were assigned.
Link to article: http://www.tandfonline.com/doi/pdf/10.1080/00220670009598732
Interesting example stephen. I agree they should have considered clustering at the classroom level. Their data are such that it would be very difficult to disentangle the teaching approach from effects of the individual teachers - with only 3 teachers one of whom used the same approach. Three teachers would not be enough to specify a mixed model, and 9 classrooms would also be challenging, because you need enough clustering units to estimate the distribution of the random effects.
Did they evaluate whether there was substantial clustering within teachers or within classrooms after conditioning on known covariates? This could be informative about whether they have a serious problem with this design or not.
Article: Zip code correlates of HIV-testing: A multi-level analysis in Los Angeles (article attached)
Unit of clustering: residential regions based on zip codes. To obtain a higher-risk subset, smaller, contiguous zip codes were merged to create larger regions that had at least 30 respondents. In the end, the study analyzed a total of 123 regions.
Hypothesized effect: The study hypothesized that HIV testing is a function of both individual behavior and the social/structural context of the area in which they reside. The authors believed that the cumulative number of AIDS cases, the number of publicly funded HIV test sites, and the racial/ethnic composition of the neighborhood affect regional testing rates, over and above the characteristics of the individuals living in the area.
Exposures of interest: 1) cumulative number of AIDS cases; 2) number of publicly funded HIV test sites; 3) racial/ethnic composition of the region
Statistical model: They built hierarchical models using multilevel logistic regression (they used MLwiN). The first model included no covariates and was used to detect differences in HIV testing by region. The second model added the individual-level covariates. The third model added the region-level exposures of interest. The final model added region-level covariates (e.g. percentage of single adults, median household income). The authors further add that they used second-order penalized or predictive quasi-likelihoodx (PQL) methods to obtain estimates (which I understand means that the model uses the current values of the fixed part plus the residuals versus just using the current values?).
From a superficial understanding of the model, I think the statistical method selected was appropriate. I’m curious to know, though, how do you graphically represent the causal network and identify confounders in a multilevel model -- Do you create a DAG for each level (e.g. individual then region) and do you come across the same complexities in controlling for confounders as you do in ecological studies?
Very nice example Carlo. See attached article re DAGs in neighborhoods research and also you might enjoy
Subramanian, S. V., M. Maria Glymour, and Ichiro Kawachi. "Identifying causal ecologic effects on health: a methodological assessment."Macrosocial determinants of population health. Springer New York, 2007. 301-331.
which I cannot upload but is available digitally from UCSF library ezproxy.