Reading Response Discussions

1) Provide an example of 4 threats to validity that you have encountered in your research, drawing one from each of the domains Cook and Campbell delineate (statistical conclusion validity, internal validity, construct validity, and external validity).

Statistical conclusion validity
Power is a critical issue in clinical research.  In assessing the feasibility of an interventional study at a small free clinic I work at, we calculated a sample size that was beyond what the research team thought would be feasible to accumulate in the time/funds we had available.  Without a high enough sample size the study won't be powered to detect the smaller effects we hope to identify.

Internal validity
In a study by Osoba et a. (Effect of Treatment on Health-Related Quality of Life in Acquired Immunodeficiency Syndrome (AIDS)-Related Kaposi's Sarcoma) investigators used the MOS-HIV to measure quality of life as a secondary outcome of treatment of Kaposi sarcoma.  This tool measures 11 domains of quality of life, however the version of the tool they used featured 2 domains (physical and role functioning) whose question asked about quality of life in overlapping time periods such that a period of time during each evaluation/follow-up period included time measured by another follow-up visit.  They adapted their analysis to preserve valid measurements of quality of life (by excluding these two domains from their analysis and thus their conclusions/discussion).

Construct validity
Construct validity is a particular concern in quality of life research- the relationship between a group of items (questions) to each other and how well they related to a given scale (eg. how well questions grouped within mental quality of life relate to mental quality of life following the QOL definition the authors outline at outset.  Cook and Campbell include descriptions of how a casual relationship generalizes to and across populations of persons and settings as well as among treatments and observations.  In a comprehensive literature review of quality of life measurement in Kaposi sarcoma research, we found there is no well developed disease-specific module for Kaposi sarcoma.  Some researchers have attempted to overcome this by developing their own modules.  The process for developing a tool with high construct validity follows generation of topics and issues critical to Kaposi sarcoma.  Such a list is generated by interviewing providers and researchers into Kaposi sarcoma as well as individuals with Kaposi sarcoma.  This list can be evaluated to identify the highest yield topics and can be piloted against similar experts to evaluate construct validity.  Part of our comprehensive literature review involves determining how well these researchers established construct validity among their items.  (eg. Some authors made 3-item tools evaluating specific issues and describing response along a 1-5 ordinal categorical scale)

External validity
In an RCT of two different ART regimens in treatment of Kaposi sarcoma 224 subjects were identified and randomized to either NNRTI-based ART or PI-based ART.  Part of the eligibility criteria described individuals with an indication for immediate chemortherapy (ie. advanced disease) would be excluded.  It was found a majority of the subjects identified and evaluated for inclusion were ineligible based on this criteria, however the overall goal of the study was to determine relative effectiveness of two different ART regimens in Kaposi sarcoma under the theory that PI-based therapy may carry additional benefit in Kaposi sarcoma treatment owing to an anti-angiogenic effect from the protease inhibitor.  To answer this question and be able to influence therapy for Kaposi sarcoma (in this case non-advanced Kaposi sarcoma requiring immediate chemotherapy) such an exclusion criteria had to be set to preserve external validity (as the intervention does not include a measure of chemotherapy).  This is particularly important in the resource-limited setting where chemotherapy is not easily attainable for the average person.


 2) For any data set you frequently use, look up the sample design and describe it.

I have been using a data set from a randomized clinical trial of NNRTI-based vs. PI-based ART in treatment

naive adults with Kaposi sarcoma in rural Uganda.  Recruitment occurred at a single site and was complicated by widespread availability of ART in the region, requiring researchers to adjust expected participant recruitment.  While not all details are clear it appears that subjects were approached based on diagnosis of Kaposi sarcoma at the site.  Diagnosis was based on pathology however some individuals were identified on clinical presentation (which required two different providers to concur the likely diagnosis was Kaposi sarcoma).  Potential participants were identified further assessed at up to two pre-trial visits.  Ultimately 224 subjects were eligible and randomized of an initial 1,568.