1. Ques during class – does randomization help with selection bias?
JMC comment in class -I said I didn’t think so bc I usually think of RCT’s having selection bias due to self-selection… the type of person signing up for a RCT is a special type of person, which can actually introduce selection bias…or limit generalizeability.
Further thoughts after class:
There are several different ways selection bias can occur... Randomization DOES help with limiting the potential of "selection bias" happening with the allocation of the intervention. If it was a simple alternating process (Tx, placebo, Tx, placebo).. then selection bias could occur if the Doctor or person enrolling could "guess" the next allocation in the sequence, and this influenced their enrollment practices. In this case, having a less guessable "randomization sequence" does limit selection bias.
Also, there is selection bias that can happen due to loss to follow-up in a RCT... I believe in class this was what was alluded to when it was stated that randomization doesn't help so much with selection bias that happens "at the end of a study."
2. Ques after class - If you have exchangeability do you also have consistency? Or vice versa?
JMC comment: Initial answer in class - No, as these are 2 separate concepts and criteria, one focused on whether the exposure groups can be compared appropriately, the other about the outcome behavior.
Further thoughts after class: I think I misunderstood the question.
I think what the person was asking about relates to the following – to have exchangeability, you assume that the consistency criteria is met. However, just because you have consistency, does not automatically mean you have exchangeability. Similarly, just because you argue that your population is exchangeable, doesn’t make consistency happen… it is more that you have to assume consistency is met, to be able to argue that your population is exchangeable. If you don’t know exactly what exposure you are testing (no consistency), then you can’t be sure that you have addressed all confounding factors. However, having a well-defined intervention that triggers an outcome in the counterfactual world that is the same as what would happen in the real world if that intervention was given, does not guarantee that in your observational data you are free of confounding.
Does that make more sense? We can also discuss further in class today, as there are HW questions on this topic.