Section outline

  • Lecture 1: Course overview, the diagnostic process, and measures of interobserver agreement

    Introduction to the course and how we will teach it.

    Evidence-Based Medicine: 1) refining disease probabilities; 2) quantifying  treatment effects. 

    Diagnosis: assigning the right name to a patient's illness versus using test results to guide treatment decisions.

    Inter-rater agreement: concordance, unweighted Kappa, linear and quadratic-weighted Kappa.

    Faculty:  Michael Kohn

    Location:  Mission Hall 1400

    • Prospectus:

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to Registered students only
    • Lecture 1 from 2017 URL

      1:09:30 Weighted kappa.

      1:27:00 What's a good kappa?

      On a laptop or desktop you should be able to play back at 2X speed.  This does not appear to work on a mobile device.  If you use 2X, you can watch from 1:09 to the end of the lecture in 10 minutes.

      Not available unless: You belong to Registered students only
    • Required Reading from Evidence-Based Diagnosis, 2nd Edition (EBD-2), which hasn't been published.

      Chapter 0 (Preface), Chapter 1, & Chapter 5 (posted below as Word documents or pdf proofs)

    • Skip from "Reiliability of Continuous Measurements" on Page 12 up to "Using studies of reliability from the literature" on Page 21.

      This section covers Bland-Altman plots (which you will cover later this quarter in Epi 203) and calibration (which we will cover in Session 5 on "Evaluating Predictions").


    • Problem Set # 0:  Think about answers to the Ch 1 exercises posted below.  (You do not need to turn them in.)

       

    • In section on 9/19, we will discuss these questions.  Try to think about them and jot down some notes for yourself, but we won't be collecting or grading your answers.

    • Assignment Answer Key (Access restricted to registered students):

    • Optional Reading:

    • Kohn (2013) Understanding Evidence-Based Diagnosis File
      Not available unless: Your ID number contains 02
    • Resources:

  • Small Group Discussion Section 1

    Review of Chapter 1 (Reason for a Diagnosis) problems and "Kappa Game"

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Resources:

    • Problem Set #1, Inter-rater Agreement, due 9/26/19 at 1:00 PM

    • Due 9/26 at 1 pm.  Upload to CLE.

    • Use the link above to upload Problem Set #1 by 1 pm on 9/27 (this should have said 9/26, but we will accept late homework).

      You must be logged in to upload your assignment.

    • Problem Set #1 (M.Laker Section) Assignment
      Not available unless: You belong to M Laker
    • Assignment Answer Key (Access restricted to registered students):

    • PSet 1 Answer Key File
      Not available unless: You belong to Registered students only
  • Lecture 2: Dichotomous Tests
    Sensitivity, specificity, predictive value, prior and posterior probability; 2x2 table method for getting posterior probability; probability and odds;  likelihood ratios, the likelihood ratio slide rule, false-positive and false negative confusion; use of Regret graphs to understand testing and treatment thresholds

    Faculty:  Thomas Newman

    Location:  Mission Hall 1400

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .MP4 URL
      Not available unless: You belong to M Laker
    • Required Reading for EBD-2, which hasn't been published yet:

      Chapter 2, Dichotomous Tests (posted below as a Word document)

    • EBD-2 Ch 2 -- Dichotomous tests File
      Not available unless: Your ID number contains 02
    • Optional Reading:

    • Testing Threshold Derivations File
      Not available unless: Your ID number contains 02
    • Cost-Benefit X Graph and calcs MAK File

      This is the Excel version of the Regret Graph Calculator available at https://ebd-2.net/regret-calculator/


      Not available unless: Your ID number contains 02
  • Small Group Discussion Section 2

    Review HW-1 (Kappa) Problems

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Problem Set #2, Dichotomous Tests, due 10/3/2019 at 1:00 PM

    • Upload Problem Set #2 to the above link by 1 pm on Thursday 10/3.

    • Assignment Answer Key (Access restricted to registered students):

    • PSet2 Answer Key File
      Not available unless: You belong to Registered students only
  • Lecture 3: Multilevel and continuous tests
    ROC Curves, interval likelihood ratios, and the relationship between them; how making multilevel tests dichotomous wastes information; choosing cutoffs, the Walking Man approach to the "non-parametric" ROC curve.

    Faculty:  Michael Kohn

    Location:  Mission Hall 1400

    • Session Slides:

    • This version has one sheet for demo and one with the formulas

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading EBD-2 (which hasn't been published yet):

      EBD-2, Chapter 3 (posted below as a Word document)

  • Small Group Discussion Section 3

    Review HW-2 (Dichotomous Tests)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Problem Set #3, Multi-level Tests, due 10/10/19 at 1:00 PM

    • Due 10/10/2019, 1:00 pm.

    • Assignment Answer Key (Access restricted to registered students):

    • PSet #3 Answer Key File
      Not available unless: You belong to Registered students only
  • Office Hours

    Location: MH-2500

  • Lecture 4: Studies of Diagnostic Tests
    Common biases in test accuracy studies: incorporation bias, verification bias, double gold-standard bias, spectrum bias;  Prevalence, spectrum bias, and nonindependence; going beyond checklists and estimating the direction of bias.

    Faculty:  Tom Newman

    Location:  Mission Hall 1400

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading EBD-2 (which hasn't been published yet):

      EBD-2, Chapter 4 (posted below as a Word document)

    • Optional Reading:

    • Kohn Carpenter Newman -- Understanding the Direction of Bias in Studies of Diagnostic Test Accuracy File
      Not available unless: Your ID number contains 02
    • Mehta 2009 J Hepatol Exceeding the limits of liver histology markers File
      Not available unless: Your ID number contains 02
    • Moyer 2000 APAM AccuracyJaundice File
      Not available unless: Your ID number contains 02
  • Small Group Discussion Section 4

    Review HW-3 (Multi-level Tests)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Homework assignment #4, Bias in Test Accuracy Studies, due 10/17/19 at 1:00 PM

    • Assignment Answer Key (Access restricted to registered students):

    • Pset 4 Answer Key File
      Not available unless: You belong to Registered students only
  • Office Hours

    Location: online or +1 669 900 6833 (Meeting ID: 758 106 231)

      

     

  • Lecture 5: Evaluating Risk Predictions
    Risk predictions: differences from diagnostic tests; calibration and discrimination; comparing predictions; net benefit calculation; decision curves.

    Faculty:  Michael Kohn

    Location:  Mission Hall 1400

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading EBD-2:

      EBD-2, Chapter 6 (Posted below as a Word document)

    • Vickers_Net_Benefit_BMJ_2016.pdf File
      Not available unless: Your ID number contains 02
    • Optional Reading:

    • Kerr 2015 Decision Curves File
      Not available unless: Your ID number contains 02
  • Small Group Discussion Section 5

    Review HW-4 (Bias in Test Accuracy Studies)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

  • Office Hours

    Location: MH-2500

    https://ucsf.zoom.us/j/186412472

    Meeting ID: 186 412 472

    Telephone:
    US: +1 669 900 6833
     or +1 646 558 8656

  • Lecture 6: Combining Tests and Multivariable Risk Models
    Combining tests/diagnostic models:  Test non-independence conditional on D+/D-; recursive partitioning; logistic regression; importance of validation separate from derivation. 

    Faculty:  Michael Kohn

    Location: Mission Hall 1400

    • Session Audio/Video Recording (Access restricted to registered students):

    • Session Slides:

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading:

      EBD-2, Ch 7 Combining Tests and Multivariable Decision Rules (posted below)

    • Optional Reading:

  • Small Group Discussion Section 6

    Review HW-5 (Evaluating Risk Predictions)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

  • Office Hours

    Location: online or +1 669 900 6833 (Meeting ID: 758 106 231)

      

     

  • Lecture 7: Quantifying the benefits and harms of treatments using RCTS
    RCT checklist; importance of baseline incidence; measures of effect size: Relative risk (RR), relative risk reduction (RRR), Odds ratio (OR), Absolute risk reduction (ARR), Number needed to treat (NNT);   "Back of the Envelope Cost Effectiveness Analysis" 

    Faculty: Martina Steurer

    Location:  Mission Hall 1400

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading from Evidence-Based Diagnosis, 2nd Edition (EBD-2), which hasn't been published.

      Chapter 8 (posted below as a Word document)

    • Optional Reading:

    • LeNoury 2015 BMJ Restoring Study 329 File
      Not available unless: You belong to a group in Registered Students Only
    • Newman 2004 NEJM Antidepressant Perspective File
      Not available unless: You belong to a group in Registered Students Only
    • Stempel 2016 NEJM AUSTRI study Advair v fluticasone alone File
      Not available unless: You belong to a group in Registered Students Only
    • Stolker 2014 Circulation rethinking composite endpoints in clinical trials File
      Not available unless: Your ID number contains 02
    • Ciani 2013 BMJ Surrogate vs patient relevant outcomes File
      Not available unless: Your ID number contains 02
    • Miner 2017 Am J Gastro Pecanatide for chronic constipation File
      Not available unless: Your ID number contains 02
    • Resources:

  • Small Group Discussion Section 7

    Review HW-6 (Combining Tests)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Optional Reading:

    • Homework Assignment #7 (RCTs) Due 1 pm on 11/7/2019

    • Assignment Answer Key (Access restricted to registered students):

  • Office Hours

    Location: MH 2500 and online or +1 669 900 6833 (Meeting ID: 758 106 231)

      

     

  • Lecture 8: P-Values and Confidence Intervals / Alternatives to RCTs

    Faculty:  Thomas Newman

    Location: 
     Mission Hall 1400

     

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading:

      EBD-2, Ch 9 and 11

    • Goodman S 2008 Seminars Hematol Dirty Dozen P Value fallacies File
      Not available unless: Your ID number contains 02
    • Optional Reading:

    • Laptook 2017 JAMA Effect of therapeutic hypothermia after 6 hours Bayesian example File
      Not available unless: Your ID number contains 02
    • Quintana 2017 JAMA Bayesian analysis. Using prior information to interpret results of clinical trials File
      Not available unless: Your ID number contains 02
    • Browner WS, Newman TB. Are all significant P-values created equal? The analogy between diagnostic tests and clinical research. JAMA 1987;257:2459-63 File
      Not available unless: You belong to a group in Registered Students Only
    • Goodman, S. Toward evidence-based medical statistics. 1: The P value fallacy. Annals of Internal Medicine 1999; 130: 995-1004 File
      Not available unless: You belong to a group in Registered Students Only
    • Goodman, S. Toward evidence-based medical statistics. 2: The Bayes factor. Annals of Internal Medicine 1999; 130: 1005-1013 File
      Not available unless: You belong to a group in Registered Students Only
    • Newman, T. If almost nothing goes wrong, is almost everything all right? JAMA 1995; 274: 1013 File
      Not available unless: You belong to a group in Registered Students Only
    • Goodman S 2013 Circulation Are we there yet File
      Not available unless: Your ID number contains 02
    • Ray 2006 American Journal of Epidemiology New User DEsigns File
      Not available unless: Your ID number contains 02
  • Small Group Discussion Section 8

    Review HW-7 (RCTs)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

  • Office Hours

    Location:  online or +1 669 900 6833 (Meeting ID: 758 106 231)

      

     

  • Reminder: Problem-writing assignment due at midnight.  See top of Syllabus.

    Lecture 9: Screening Tests

    Need for a more critical approach for screening tests, risk factor vs disease screening; biases in observational studies of screening: lead time bias, length time bias, volunteer bias, stage migration bias, pseudodisease; randomized trials of screening

    Faculty:  Martina Steurer-Muller
     
    Location:  Mission Hall 1400

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2014 Lecture (Watch) URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Required Reading:


    • Optional Reading:

    • Merenstein D. Winners & Losers. JAMA 2004;291:242-5 File
      Not available unless: You belong to a group in Registered Students Only
    • Heath 2014 BMJ Role of fear in overdiagnosis File
      Not available unless: You belong to a group in Registered Students Only
    • Heath 2013 BMJ Overdiagnosis When good intentions meet vested interests File
      Not available unless: You belong to a group in Registered Students Only
    • Newman 2016 JAMA IM Lipid screening in children-low value care File
      Not available unless: You belong to a group in Registered Students Only
    • Annas 1995 NEJM Reframing the Debate on Healthcare by replacing Metaphors File
      Not available unless: Your ID number contains 02
    • Tilman 2014 Nature Global diets link environmental sustainability and human health File
      Not available unless: You belong to a group in Registered Students Only
  • Small Group Discussion Section 9

    Review HW-8 (P-Values, Confidence Intervals)

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Homework Assignment #9, Screening (Ch-10) Due 11/21/2019 1 pm: 

    • The Homework #9 upload line is located in Small Group Discussion Section #10.   Sorry for the confusion.

  • Office Hours

    Location: MH 2500 and online or +1 669 900 6833 (Meeting ID: 758 106 231)

      

     

  • Lecture 10: Cognitive Biases / Course Review

    Faculty:  Michael Kohn

    Location:  Mission Hall 1400

    • Session Slides:

    • Session Audio/Video Recording (Access restricted to registered students):

    • 2018 Lecture Watch (2019 students should watch this) URL
      Not available unless: You belong to a group in Registered Students Only
    • Required Reading:

      EBD, Chapter 12

    • Optional Reading:

    • Newman, TB. The Power of Stories over Statistics.BMJ. 2003;327(7429):1424-7 File
      Not available unless: Your ID number contains 02
  • Small Group Discussion Section 10

    Review HW-9 Screening

    Faculty:  Charles Murphy, Adrian Whelan, Sandeep Brar, Marta San Luciano Palenzuela, Miriam Laker, Kerstin Kolodzie, Martina Steurer, Tom Newman, Michael Kohn
     
    Location:  Murphy MH-2105, Whelan MH-2106, Brar MH-2107, San Luciano Palenzuela MH-2108

    • Homework Assignment #9:  Due 11/21/2019 at 1pm

    • Problem Set #9 (M.Laker Section) Assignment
      Not available unless: You belong to M Laker
    • Assignment Answer Key (Access restricted to registered students):

  • Lecture 11: Final Exam Review
     

    Faculty:  Michael Kohn

    Location: Mission Hall 1400

    • Session Audio/Video Recording (Access restricted to registered students):

    • Watch URL
      Not available unless: You belong to a group in Registered Students Only
    • 2019 Lecture .mp4 URL
      Not available unless: You belong to M Laker
    • Assignment: TAKE-HOME FINAL EXAM DUE THURS 12/5 8:45 AM

    • Note: We may post clarifications in response to forum posts!

    • Please use this option if you are unable to attend this session.  Please upload as a .pdf such that all your formatting will be preserved.

      Due 8:30 am.

    • Final Examination Answer Key (Access restricted to registered students):