Section outline

  • Lecture: Evaluating Uncertainty

    We consider confidence intervals, the limits of p-values, the intuition of bootstrapping to estimate confidence intervals, and estimating and interpreting subgroup effects. 

    Faculty:  Maria Glymour


    Location:      Mission Hall 1406  

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL

    • Required Reading:

      1. Sterne JAC, Smith GD. Sifting the evidence - what's wrong with significance tests? British Medical Journal. Jan 27 2001;322(7280):226-+.
      2.  Ioannidis JPA. Why most published research findings are false. PLoS Med. 2006;2(8):e124: 0696-0701
      3. Grunkemeier, G. and Y. Wu (2004). "Bootstrap resampling methods: something for nothing?" The Annals of thoracic surgery 77(4): 1142.
      4. Ertel et al., Frailty modifies effectiveness of psychosocial intervention in recovery from stroke. Clin Rehabil. 2007. 21:511.

    • Optional Reading:

    • Sterne on p-values File
      Not available unless: Your ID number contains 02
    • On (Ir)Reproducible Science File
      Not available unless: Your ID number contains 02
    • Introduction to the bootstrap File
      Not available unless: Your ID number contains 02
    • Ertel FIRST Secondary File
      Not available unless: Your ID number contains 02

    • Assignment: 

      Specify a hypothesis regarding a particular exposure and outcome and a binary effect modifier including specific measures of association (specify the magnitudes of that association you anticipate: I suggest making everything cross-sectional). Using the software of your choice, generate a population with 1000 people under a causal structure consistent with this hypothesis. Draw a simple random sample 100 individuals from this population and estimate the population average exposure-outcome association and the association stratified by your modifier of interest within this subset.  Repeat this 10 times and write the parameter estimates and CI each time.

      Repeat the data set construction, setting the causal effect to the null.  Again repeat this 10 times and write the parameter estimate and CI each time (if you figure out how to automate it, run it 1000 times and post the histogram of the parameter estimates and p-values).

      Use your code above and also a canned software command to estimate statistical power to detect the difference in means under the settings below:

      *n=100, μ0=.02, μ1=.12, SD=1, α=.05

      *n=100,μ0=.02, μ1=.12, SD=2, α=.05

      *n=500, μ0=.3, μ1=.3, SD=1, α=.05

       For each of the 3 settings above, what is the power to detect whether the ratio of the means=1?

    • Uncertainty Slides File