Section outline

  • Course introduction; Diagnostic process and measures of interobserver agreement

    Introduction to the course and how we will teach it; testing to refine probabilities and guide decisions; concordance, unweighted Kappa, linear and quadratic weighted Kappa

    Faculty:  Michael Kohn

    No Large-Group Discussion (LGD).  LGDs start on Tuesday 9/21.


    The first small group will be Thursday 9/16 at 1:15 pm.  In that small group, we will review Problem Set #0 (not to turn in) and play the Kappa Game.  Prior to that small group meeting, please watch Lecture 1, read Chapters 1 and 5 (pages 110 - 121), and do Problem Set #0, which is not to submit. 

    All subsequent problem sets (#1 - #9) should be uploaded on the CLE prior by 1:15 pm on Thursday, starting with Problem Set #1 on 9/23.



    • Lecture Slides:

    • Lec 1 Part 1 Overview diagnosis 2021-0725 File
    • Lec 1 Part 2 Kappa 2021 File

    • Lecture Audio/Video Recording (Access restricted to registered students)

      Note: The live recordings sometimes omit the on-screen pointer/cursor or display it with a lag.  From the audio, it is clear that I am pointing at something with my mouse, but you can't see it on the video.  This is especially problematic during the Excel and Stata demonstrations.  I tried to make it a bit better by adding annotations to the video.

    • Lecture 1 Segment 1 -- Ch 1 material on diagnosis and testing (12:55) Media Resource
      Not available unless: You belong to Registered students only
    • Lecture 1 Segment 2 -- Ch 5 material on kappa (unweighted) (22:45) Media Resource
      Not available unless: You belong to Registered students only
    • Lecture 1 Segment 3 -- Ch 5 material on weighted kappa (27:15) Media Resource
      Not available unless: You belong to Registered students only
    • Lecture 1 Segment 4 Ch 5 What affects kappa? What's a good kappa? (4:41) Media Resource
      Not available unless: You belong to Registered students only

    • Required Reading:

    • Evidence-Based Diagnosis, 2nd Edition (EBD-2): Chapter 1, & Chapter 5 (pp 121-130 on reliability of continuous measurements is optional)

    • Optional video and material on kappa for more than 2 raters:

    • Melanoma pathology: example of agreement among multiple raters (2:35) Media Resource
      Not available unless: You belong to Registered students only

    • This video briefly introduces our example of multi-rater kappa, a study of the inter-rater reliability of histopathologists reviewing 37 slides for melanoma vs. benign melanocytic nevus.  See EBD-2 Appendix 5A -Multirater Kappa.

    • Study Using Multirater Kappa (Melanoma vs. Melanocytic Nevus) File
    • MultiraterKappa Datasets File

    • Optional Reading:

    • Kohn (2013) Understanding Evidence-Based Diagnosis File
      Not available unless: Your ID number contains 02

    • Resources:

    • Kappa Game Data Instructions and Collection Form File
    • Excel File for Calculating Kappa
    • Calculating kappa with Stata File
    • GraphPad Online Kappa Calculator URL

    • Assignments:

    • We will play the "Kappa Game" at the first Small Group Discussion on September 16.  Please try to view the lecture videos and and do the required reading in preparation for the Sept. 16 session.

    • Problem Set #0 -- Understanding Diagnosis (will not be graded) File

      Unlike Problem Sets #1-10, Problem Set #0 will not be graded.  We will discuss it in the Sept. 16 Small Group Discussion sections.

    • Problem Set #0 with Answers File

    • Assignment due date: 9/23/21 at 1:15 PM

    • Problem set #1 (Due 9/23/2021 at 1:15 PM) File
    • Upload Problem Set #1 Assignment

      Use the link above to upload Problem Set #1 by 1:15 PM on September 23. 

      You must be logged in to upload your assignment.

    • Problem Set #1 (M.Laker Section) Assignment
      Not available unless: You belong to M. Laker

    • Assignment Answer Key (Access restricted to registered students):

    • PSet #1 Answer Key File
      Not available unless: You belong to Registered students only