EPI 242: Program Evaluation in Clinical and Public Health Settings (Fall 2013)

Section outline

• Select section General

  Collapse Expand

  General

  Collapse all Expand all

  • Select activity Home Overall Schedule Cour...

    		Home Overall Schedule Courses Programs Rosters
    Program Evaluation in Clinical and Public Health Setting EPI 242 Fall 2013 (1.5 units) . Course Director: Janet Myers, PhD, MPH . Associate Professor, Department of Medicine

• Select section Thursday, September, 19, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, September, 19, 2013; 8:45 AM - 10:15 AM

  Lecture: Introduction to Evaluation/Purpose of Evaluation/Describing the Program, Logic Models, part 1

  1. Define program evaluation and understand how evaluation can support program planning, implementation and accountability. 2. Explore the key terms and concepts associated with program evaluation. 3. Describe program theory and program logic 4. Identify elements of program theory/logic in selected case studies.

  Faculty:  Janet Myers
  
  Location:  China Basin 5759

  • Select activity Prospectus:

    Prospectus:

  • Select activity Session 1 Overview

    

    Session 1 Overview File
  • Select activity Session Slides:

    Session Slides:

  • Select activity Session 1 Slides

    

    Session 1 Slides File
  • Select activity Required Reading: Weiss, Carol. Evaluation. Chapt...

    Required Reading:

    Weiss, Carol. Evaluation. Chapter 1-3

  • Select activity AEA Guiding Principles

    

    AEA Guiding Principles URL
  • Select activity Required Readings

    

    Required Readings Folder
  • Select activity Optional Reading: Rossi, Chapter 1

    Optional Reading:

    Rossi, Chapter 1

  • Select activity Resources:

    Resources:

  • Select activity Assignment: For students who are using a known or...

    Assignment:

    For students who are using a known or active project as their “case” for discussion, identify the goal/s, objective/s and activities associated with the program’s plan.  If you want to use a logic model to do this, that’s fine.  Identify the goals, assumptions, target population, inputs and resources, activities and outputs of the program or project. 

     For students who are not using their own project, identify for either MARC, IMPAACT or Kiosks, the program logic or theory, goals, assumptions, target population, inputs and resources, activities and outputs of the program or project. 

  • Select activity Assignment Due Date:  Thursday, September 26. 2013...

    Assignment Due Date:  Thursday, September 26. 2013 at the beginning of class.

     

• Select section Thursday, September 26, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, September 26, 2013; 8:45 AM - 10:15 AM

  Lecture: Evaluation Planning and Design, Logic Models, part 2
  

  1. Describe summative and formative evaluations; 2. Explore the strength of different types of evaluation designs; 3. Attribute outcomes to the program/understand the role of causality in rigorous evaluation; 4. Plan an evaluation/developing good evaluation questions.

  Faculty:  Janet Myers
  
  Location:  China Basin 5759

  • Select activity Prospectus:

    Prospectus:

  • Select activity Session 2 Overview

    

    Session 2 Overview File
  • Select activity Session Slides:

    Session Slides:
    

  • Select activity Session 2 Slides

    

    Session 2 Slides File
  • Select activity Required Reading: Weiss, Carol. Evaluation. Chapt...

    Required Reading:

    Weiss, Carol. Evaluation. Chapter 4, 6 & 8
    Daponte, Osborne. Evaluation Essentials. Chapter 4 & 5
    Bradford Hill criteria for causality

  • Select activity CDC-Evaluation-Workbook-508(3)

    

    CDC-Evaluation-Workbook-508(3) File
  • Select activity Optional Reading: Weiss, Carol. Evaluation. Chapt...

    Optional Reading:

    Weiss, Carol. Evaluation. Chapter 5 & 7

  • Select activity Assignment: For students who are using a known or...

    Assignment:

    For students who are using a known or active project as their “case,” finish your “logic model” or program plan by identifying the outcomes associated with your program or project.   Pay particular attention to causality and the evidence you plan to use to justify your conclusions (validity).  You can begin describing each of the logic model components in your evaluation plan narrative, the first section of which will be due next week. 

     

    For students who are not using their own project:

    1. Choose two of eight of Weiss’s criteria for resolving which questions to pursue (see page 77) and describe how each impacted the evaluation of EITHER MARC, IMPAACT or Kiosks (whichever you chose for the homework due this week). 
    2. Finally, also for the case you chose for the first assignment, describe whether you think the evaluation design was the most rigorous and if not, which design you would recommend (from Weiss Chapter 8).

  • Select activity Assignment Due Date:  Homework due next week

    Assignment Due Date:  Homework due next week

• Select section Thursday, October 3, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, October 3, 2013; 8:45 AM - 10:15 AM

  Lecture: Frameworks/Approaches used in Clinical Settings, Part 1 (Re-Aim and CFIR)
  

   

  Faculty:  Margaret Handley
  
  Location:  China Basin 5759

  • Select activity Required Reading:

    Required Reading:

  • Select activity The RE-AIM Framework: Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999 Sep;89(9):1322-7.

    

    The RE-AIM Framework: Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999 Sep;89(9):1322-7. File
  • Select activity The Consolidated Framework: Damschroder LJ, Aron DC, Keith, Kirsh SR, Alexander JA and Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sc

    

    The Consolidated Framework: Damschroder LJ, Aron DC, Keith, Kirsh SR, Alexander JA and Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sc File
  • Select activity Optional Reading:

    Optional Reading:

  • Select activity Case Study: Schillinger D, Hammer H, Wang F, Palacios J, McLean I, Tang A, Youmans S, Handley M. Seeing in 3-D: examining the reach of diabetes self-management support strategies in a public health care system. Health Educ Behav. 2008 Oct;35(5):664-82. Ep

    

    Case Study: Schillinger D, Hammer H, Wang F, Palacios J, McLean I, Tang A, Youmans S, Handley M. Seeing in 3-D: examining the reach of diabetes self-management support strategies in a public health care system. Health Educ Behav. 2008 Oct;35(5):664-82. Ep File
  • Select activity Case Study: Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, Curran G. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Case study: Implement Sci. 2012 Apr 11;7(1):30.

    

    Case Study: Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, Curran G. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Case study: Implement Sci. 2012 Apr 11;7(1):30. File
  • Select activity Case Study: Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: How can the RE-AIM framework help? J Am Med Inform Assoc . 2009. 16( 6):889-897.

    

    Case Study: Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: How can the RE-AIM framework help? J Am Med Inform Assoc . 2009. 16( 6):889-897. File
  • Select activity Assignment: For Projects: Submit the first secti...

    Assignment:

    For Projects: Submit the first section of your evaluation plan including: the title page or overview, Intended use and users, program description and logic model or framework that will guide your evaluation.
    
    For students not using their own projects, for both the Forney and Schillinger cases, in your view, make a case for how the framework enhanced or weakened the evaluation of the program in both settings.

  • Select activity Assignment Due Date: October 10, 2013

    Assignment Due Date: October 10, 2013
    

• Select section Thursday, October 10, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, October 10, 2013; 8:45 AM - 10:15 AM

  Lecture: Frameworks/Approaches used in Clinical Settings, Part 2 (FOI, PARIHS & ORCA)
  

   

  Faculty:  Janet Myers
  
  Location:  China Basin 5759

  • Select activity Required Reading:

    Required Reading:

  • Select activity PARiHS: Kitson AL et al. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implementation Science 2008, 3:1.

    

    PARiHS: Kitson AL et al. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implementation Science 2008, 3:1. File
  • Select activity ORCA: Helfrich CD et al. Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Science 2009, 4:38

    

    ORCA: Helfrich CD et al. Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Science 2009, 4:38 File
  • Select activity PARiHS: Helfrich CD, Damschroder LJ, Hagedorn HJ et al. A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implementation Science 2010, 5:82.

    

    PARiHS: Helfrich CD, Damschroder LJ, Hagedorn HJ et al. A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implementation Science 2010, 5:82. File
  • Select activity The Fidelity of Implementation Framework: Keith, RE et al. Fidelity of implementation: development and testing of a measure. Implementation Science 2010; 5:99.

    

    The Fidelity of Implementation Framework: Keith, RE et al. Fidelity of implementation: development and testing of a measure. Implementation Science 2010; 5:99. File
  • Select activity Assignment: For everyone: Select one of the implem...

    Assignment: For everyone: Select one of the implementation science models we’ve covered in class and briefly outline why it would support evaluation of either the project you are working on in class or on the case study you chose for the first week’s homework.

• Select section Thursday, October 17, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, October 17, 2013; 8:45 AM - 10:15 AM

  Lecture: Frameworks and Approaches used in Public Health Settings
  

  Faculty:  Alice Gandelman
  
  Location:  China Basin 5759

  • Select activity Required Reading:

    Required Reading:

  • Select activity MMWR Article

    

    MMWR Article URL
  • Select activity Program Evaluation: READ Framework, Steps, Standards, and Conducting Strong Evaluations. SKIM Framework-Based Materials and Other Evaluation Resources

    

    Program Evaluation: READ Framework, Steps, Standards, and Conducting Strong Evaluations. SKIM Framework-Based Materials and Other Evaluation Resources URL
  • Select activity Questions Approach: Organizing Framework for a Functional National HIV Monitoring and Evaluation System. 2008. UNAIDS. Available at:

    

    Questions Approach: Organizing Framework for a Functional National HIV Monitoring and Evaluation System. 2008. UNAIDS. Available at: URL
  • Select activity Precede Proceed Intro

    

    Precede Proceed Intro File
  • Select activity Gandelman Soc Det Health

    

    Gandelman Soc Det Health File
  • Select activity Watson Oral Heatlh Promotion

    

    Watson Oral Heatlh Promotion File
  • Select activity Berkowitz VERB

    

    Berkowitz VERB File
  • Select activity Li Precede Proceed Case Study

    

    Li Precede Proceed Case Study File
• Select section Thursday, October 24, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, October 24, 2013; 8:45 AM - 10:15 AM

  Lecture: Evaluation in the Global Context
  

  1. Identify outcomes that can be appropriately attributed to the program; 2. Choosing an evaluation design; 3. Selecting methods to answer the research questions.

  Faculty:  Lambdin
  
  Location:  China Basin 5759

  • Select activity Rugg et al

    

    Rugg et al File
  • Select activity Patton

    

    Patton File
  • Select activity 1471-2458-10-447

    

    1471-2458-10-447 File
  • Select activity Beyond Indicators Advances in Global HIV.11[1]

    

    Beyond Indicators Advances in Global HIV.11[1] File
  • Select activity Assignment: Due October 31stFor project people: Su...

    Assignment: Due October 31st
    For project people: Submit a revised/close to final draft of your evaluation methods section including the evaluation scope, questions and indicators.
    
    For non-projects people: Write a brief summary of the value of the findings of Bendavid and colleagues for policymakers making decisions about resource allocation. Prepare a response to Patton based on this perspective.

• Select section Thursday, October 31, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, October 31, 2013; 8:45 AM - 10:15 AM

  Lecture: Overview of Evaluation Methods/Implementing the Evaluation/Dissemination - think about it before you design methods
  

  Faculty:  Janet Myers
  
  Location:  China Basin 5759

  • Select activity Required Reading:

    Required Reading:

  • Select activity Collecting Evaluation Data Overview

    

    Collecting Evaluation Data Overview File
  • Select activity NW Center Data Collection Overview

    

    NW Center Data Collection Overview File
  • Select activity Collecting Evaluation Data - Surveys

    

    Collecting Evaluation Data - Surveys File
  • Select activity Power of the Ancedote

    

    Power of the Ancedote File
  • Select activity HIV Testing in EDs

    

    HIV Testing in EDs File
  • Select activity Norma Ware what has love

    

    Norma Ware what has love File
  • Select activity Optional Reading:

    Optional Reading:

  • Select activity How many interviews are enough

    

    How many interviews are enough File
  • Select activity Identifying Themes

    

    Identifying Themes File
  • Select activity Qualitative Data Analysis Lacey Luff

    

    Qualitative Data Analysis Lacey Luff File
  • Select activity Three Approaches to Content Analysis

    

    Three Approaches to Content Analysis File
  • Select activity within and across case analysis

    

    within and across case analysis File
  • Select activity The art of open-ended interviewing

    

    The art of open-ended interviewing File
  • Select activity Assignment: Due November 7thProject people: Contin...

    Assignment: Due November 7th
    
    Project people: Continue to refine your methods section; submit drafts of the tools/instruments/ interview guides that you will use to collect data.
    
    Non-project people: For either the HIV testing in EDs article or the article on adherence, comment on whether the methods used to explore the case were adequate to the task. Was the evidence provided strong enough to justify the investigators’ conclusions? If not, why not and what would you have done differently?

• Select section Thursday, November 7, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, November 7, 2013; 8:45 AM - 10:15 AM

  Lecture: Qualitative Methods in Evaluation
  

   

  Faculty:  Kimberly Koester
  
  Location:  China Basin 5759

  • Select activity A guide to quant analysis GAO

    

    A guide to quant analysis GAO File
  • Select activity How to represent qual data

    

    How to represent qual data File
  • Select activity Survey Guide

    

    Survey Guide File
  • Select activity Quick reference guide: Activities requiring or not requiring CHR review and approval prior to initiation.

    

    Quick reference guide: Activities requiring or not requiring CHR review and approval prior to initiation. URL
• Select section Thursday, November 14, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, November 14, 2013; 8:45 AM - 10:15 AM

  Lecture: A case of community-level program evaluation: Evaluating Patient Centered Medical Homes
  

   

  Faculty:  Reena Gupta
  
  Location:  China Basin 5759
• Select section Thursday, November 21, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, November 21, 2013; 8:45 AM - 10:15 AM

  Lecture: Multi-level Interventions and Measurement

  Faculty:  Kang Dufour
  
  Location:  China Basin 5759

  • Select activity Session Audio/Video Recording (Access restricted t...

    Session Audio/Video Recording (Access restricted to registered students):

• Select section Thursday, November 28, 2013; Thanksgiving - No Class

  Collapse Expand

  Thursday, November 28, 2013; Thanksgiving - No Class

   
• Select section Thursday, December 5, 2013; 8:45 AM - 10:15 AM

  Collapse Expand

  Thursday, December 5, 2013; 8:45 AM - 10:15 AM

  Lecture: Wrap Up

  Faculty:  Janet Myers
  
  Location:  China Basin 5759