Hi All,
Below are some further general notes on the identifiability criteria, which I said I would post. Please let Dr. Graff and me know next week in class, if you still have questions.
Below these general notes are also a question and response about q3 from the HW3 key, which I fielded via email, but thought to share broadly with others, in case it is more broadly clarifying, as I recognize we did not go over q3 in the HW review. It is relating to the identifiability criteria. Please note, answers for q3 may vary, as long as the rationale you apply is based on correct definitions/reasoning. What we put down was one possible stance and rationale in the Key. We have made a note in the key for next year to clarify "answers may vary". The TA's already know this for their grading purposes.
Happy Sunday!
-June
Some additional General Comments about Consistency & Positivity in Observational Studies:
- To me, consistency is about knowing if the exposure you are studying is really the “causal factor”…or does it at least correspond to predictable outcomes for your question of interest. E.g. - Some challenge that BMI is not a good exposure for studying causal inference because a specific BMI level may be achieved through many different ways – diet, exercise, genetics, surgery, etc. It doesn’t correspond to a “well-defined” intervention, per the Hernan & Robins definition, therefore a question involving BMI and an outcome, say death, may not meet the “consistency” assumption, and one is less sure what they are making inferences about when they study BMI as an exposure… is it really the BMI number that is relevant, or it is how one got to that number?
- Nevertheless, some would argue that it doesn’t matter, and for specific diseases or outcomes, there may be data that indicate that BMI is a well-defined entity with it’s own biologic importance, and that whether someone lost weight and attained a normal BMI via diet or exercise or both is not relevant…and that just “having a normal BMI” is in fact the thing that is associated with reduced death.
- Note, there are different camps on this. The Hernan definition of having a well-defined intervention is one way to assess consistency. However, there are others who think that a study can have consistency w/o necessarily having a “well-defined intervention” (e.g. BMI).
- For conditional exchangeability, positivity and consistency, if you do not think these are met as in an ideal randomized experiment, a next appropriate question is “how close might you be?” How big an impact might the bias be? There are various sensitivity analyses that can be considered, depending on the situation. I think there is a spectrum of “not meeting” those criteria, and It reflects how well an observational study mimics an RCT, and thus how much one might suggest that the data support an association that should be followed up further, with a next study that may have better abilities to inform causal inference.
- Accordingly, even if one is hesitant to claim ‘cause’, observational data are very important to bring associations and risk factors to light, for further study.
Question via email: " After reviewing the answer key, I have a question about the concept of consistency. The authors assumed the monovalent and trivalent MMR vaccines equally but did not specify why their decision should be justified.
The effect of difference of monovalent and trivalent vaccines seems to be controversial, but we do not know clearly yet. Meanwhile, Merck decided not to resume production of monovalent vaccines in 2009. https://www.cdc.gov/vaccines/hcp/clinical-resources/mmr-faq-12-17-08.html I am wondering if the authors should have mentioned the reason why stratifying the vaccine type is not needed to hold consistency for the exposure. There were some studies to divide them into the trivalent and monovalent vaccines to see the different effects on the ASD risk as well.Response from JMC: "That is a good question. I think part of why the authors didn’t distinguish these is bc their hypothesis was about “vaccination status” broadly. I agree, that one could say this does not translate directly in a target trial, or that a target trial would probably choose one dose and frequency. Of note, the authors also are cautious about their interpretation I think. If you discussed your reasoning for whether you thought consistency holds or not, with an explanation like this, I think it’s fine. I’ve ccd the TAs so they know and we can update the key to clarify that answers may vary. I do believe the grading rubric (that the TAs follow) allows for varying answers, as long as reasoned using correct definitions of the term."