Section outline

  • General

    Collapse all Expand all

  • Thursday, September, 17, 2015; 8:45 AM - 10:15 AM

    Lecture: Introduction to Evaluation/Purpose of Evaluation/Describing the Program, Logic Models, part 1

    1. Define program evaluation and understand how evaluation can support program planning, implementation and accountability. 2. Explore the key terms and concepts associated with program evaluation. 3. Describe program theory and program logic 4. Identify elements of program theory/logic in selected case studies.

    Faculty:  Janet Myers

    Location:  Mission Hall 1108

    • Prospectus:

    • Session Slides:

    • Session 1 slides File
    • Session 1 Overview 2015 File

    • Required Reading:

      Weiss, Carol. Evaluation. Chapter 1-3

    • AEA Guiding Principles URL
    • Kiosk Case Study Folder
      Not available unless: Your ID number contains 02
    • MARC Case study Folder
      Not available unless: Your ID number contains 02

    • Optional Reading:

    • Hatry HP. (2013). Sorting the Relationships among Performance Measurement, Program Evaluation and Performance management. New Directions for Evaluation, 137:19-32 File
      Not available unless: Your ID number contains 02
    • Rossi, Chapter 1. File
      Not available unless: Your ID number contains 02
    • Liket KC, Rey-Garcia M, Maas KEH. (2014). Why aren’t evaluations working and what to do about it: A framework for negotiating meaningful evaluation in nonprofits. American Journal of Evaluation, 35(2):171-188. File
      Not available unless: Your ID number contains 02

    • Assignment:

      Decide whether you will be doing an evaluation plan from your own experience or doing the “problem sets” I will assign in class.

      Plan people:  If you are doing an evaluation plan, submit a title and a brief description of the program you will be considering for the evaluation plan. Identify the goal/s, objective/s and activities associated with the program.  If you want to use a logic model to do this, that’s fine. 

      Problem Set People:  Choose one of the “cases” (MARC, IMPAACT or Kiosks) above and in about two pages, identify the program logic or theory, goals, assumptions, target population, inputs and resources, activities and outputs of the program or project.  Also describe whether you think the evaluation design was the most rigorous and if not, which design you would recommend, referencing Weiss Chapter 8) and why.

    • Assignment Due Date:  Thursday, October 1, 2015 at the beginning of class.

       

  • Thursday, September 24, 2015; 8:45 AM - 10:15 AM

    Lecture: Evaluation Planning and Design, Logic Models, part 2

    1. Describe summative and formative evaluations; 2. Explore the strength of different types of evaluation designs; 3. Attribute outcomes to the program/understand the role of causality in rigorous evaluation; 4. Plan an evaluation/developing good evaluation questions.

    Faculty:  Janet Myers

    Location:  Mission Hall 1108

    • Session Slides:

    • Session 2 Overview 2015 File

    • Required Reading:

      Weiss, Carol. Evaluation. Chapter 4, 6 & 8
      Daponte, Osborne. Evaluation Essentials. Chapter 4 & 5
      Bradford Hill criteria for causality

    • CDC-Evaluation-Workbook-508(3) File
      Not available unless: Your ID number contains 02
    • W.K. Kellogg Foundation Evaluation Handbook, W.K. Kellogg Foundation, 2004 File
      Not available unless: Your ID number contains 02
    • Kellogg Logic Model Development Guide File
      Not available unless: Your ID number contains 02
    • Osborne Duponte Chapters 4 5 File
      Not available unless: Your ID number contains 02
    • Lucas RM, McMichael AJ. (2005). Association or causation: evaluating links between “environment and disease.” Bulletin of the World Health Organization, 83(10):792-795 File
      Not available unless: Your ID number contains 02

    • Optional Reading:

      Weiss, Carol. Evaluation. Chapter 5 & 7

    • Assignment:

      Decide whether you will be doing an evaluation plan from your own experience or doing the “problem sets” I will assign in class.

      Plan people:  If you are doing an evaluation plan, submit a title and a brief description of the program you will be considering for the evaluation plan. Identify the goal/s, objective/s and activities associated with the program.  If you want to use a logic model to do this, that’s fine. 

      Problem Set People:  Choose one of the “cases” (MARC, IMPAACT or Kiosks) above and in about two pages, identify the program logic or theory, goals, assumptions, target population, inputs and resources, activities and outputs of the program or project.  Also describe whether you think the evaluation design was the most rigorous and if not, which design you would recommend, referencing Weiss Chapter 8) and why.

    • Assignment Due Date:  Homework due October 1, 2015

  • Thursday, October 1, 2015; 8:45 AM - 10:15 AM

    Lecture: Frameworks/Approaches used in Clinical Settings: Precede-Proceed, example

    Lecturer: Sarah Imershein

    Location:  Mission Hall 1108

    • Prospectus:

    • Session 3 Overview 2015 File

    • Session Slides:

    • Evaluation Clinical Frameworks 10 1 15 File

    • Required Reading:

    • Damschroder LJ, Aron DC, Keith, Kirsh SR, Alexander JA and Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 2009;4:50 File
      Not available unless: Your ID number contains 02
    • CFIR-DetailedRationaleForConstructs File
      Not available unless: Your ID number contains 02
    • CFIR - Figure and Explanatory Text File
      Not available unless: Your ID number contains 02
    • CFIR-MatrixOfConstructs File
      Not available unless: Your ID number contains 02
    • CFIR-ShortDefinitions File
      Not available unless: Your ID number contains 02
    • re-aim framework constructs checklist File
      Not available unless: Your ID number contains 02
    • REAIM Review Gaglio 2013 File
      Not available unless: Your ID number contains 02
    • English Kenya CFIR File
      Not available unless: Your ID number contains 02
    • PCMH Evaluation Guide File
    • Bakken RE-AIM File
      Not available unless: Your ID number is 02
    • Glasgow RE-AIM File
      Not available unless: Your ID number is 02
    • Fortney File
      Not available unless: Your ID number is 02

    • Optional Reading:

    • Schillinger D, Hammer H, Wang F, Palacios J, McLean I, Tang A, Youmans S, Handley M. Seeing in 3-D: examining the reach of diabetes self-management support strategies in a public health care system. Health Educ Behav. 2008 Oct;35(5):664-82. Epub 2007 May File
      Not available unless: Your ID number contains 02

    • Assignment:

      For project people: Apply any of the public health or clinical frameworks discussed in class (logic models, Precede-Proceed, REAIM, CFIR) to the program you plan to evaluate in a figure or in narrative form.

      For students not using their own problem set people: For both the Fortney and Schillinger cases, in your view, make a case for how the framework enhanced or weakened the evaluation of the program in both settings.

    • Assignment Due Date:  Due October 22nd

  • Thursday, October 8, 2015; 8:45 AM - 10:15 AM

    Lecture: Frameworks and Approaches used in Public Health Settings

    Faculty:  Alice Gandelman

    Location:  Missiion Hall 1108

    • Session Slides:

    • Janet Myers - TICR Oct 2015 File

    • Required Reading:

    • CDC Framework for PE in PH 1999 File
      Not available unless: Your ID number contains 02
    • MMWR Article URL
    • Program Evaluation: READ Framework, Steps, Standards, and Conducting Strong Evaluations. SKIM Framework-Based Materials and Other Evaluation Resources URL
    • Questions Approach: Organizing Framework for a Functional National HIV Monitoring and Evaluation System. 2008. UNAIDS. Available at: URL
      Not available unless: Your ID number contains 02
    • Precede Proceed Intro File
      Not available unless: Your ID number contains 02
    • Gandelman Soc Det Health File
      Not available unless: Your ID number contains 02
    • Berkowitz VERB File
      Not available unless: Your ID number contains 02

    • Optional Reading:

    • Li Precede Proceed Case Study File
      Not available unless: Your ID number contains 02
    • Watson Oral Heatlh Promotion File
      Not available unless: Your ID number contains 02

    • Assignment: Homework Due October 8th

      For project people: Submit a first draft of your evaluation methods section including the evaluation scope, questions and indicators.

      For non-projects people: For EITHER the Berkowitz, Li or Watson cases, in your view, make a case for how the framework enhanced or weakened the evaluation of the program or intervention.

  • Thursday, October 15, 2015; 8:45 AM - 10:15 AM

    Lecture: Frameworks example:  Knowledge Transfer and Exchange in Pre-Term Birth

     

    Faculty:  Michael Reyes

    Location:  Mission Hall 1108

    • Prospectus:

    • Session Slides:

    • Required Reading:

    • Helfrich CD et al. Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Science 2009, 4:38 File
      Not available unless: Your ID number contains 02

    • Assignment Due Date: October 8, 2015

  • Thursday, October 22, 2015; 8:45 AM - 10:15 AM

    Lecture: Frameworks/Approaches used in Clinical Settings: REAIM and CFIR

    Faculty:  Janet Myers

    Location:  Mission Hall 1108

    • Session Slides:

    • Required Reading:

      1. Weiss, Carol.  Evaluation:                                                                                                                          
        1. Chapter 7, “Collecting data”
        2. Chapter 9, “The Randomized Experiment”
        3. Chapter 11, “Qualitative Methods”
        4. Chapter 12, “Analyzing and Interpreting Data”
      2. Identifying Collecting Evaluation Data: An Overview of Sources and Methods (available at: http://learningstore.uwex.edu/Assets/pdfs/G3658-04.pdf)

    • Optional Reading:

    • data collection toolkit File
      Not available unless: Your ID number contains 02
    • Handley - Quasi-experimental design in practice-based settings File
      Not available unless: Your ID number contains 02
    • Meissner-SPR-Mixed Methods File
      Not available unless: Your ID number contains 02
    • Michie - Behavioral change wheel File
      Not available unless: Your ID number contains 02
    • Plano-Clark-SPR-Mixed Methods File
      Not available unless: Your ID number contains 02
    • Collecting Evaluation Data File
      Not available unless: Your ID number contains 02
    • Creswell - Mixed methods in primary care File
      Not available unless: Your ID number contains 02
    • Creswell- Designing and Conducting Mixed Methods Research Chapter 4 File
      Not available unless: Your ID number contains 02

    • Assignment: Due October 29

      Evaluation Plan, part 2.

      Continue to refine your methods section; submit drafts of the tools/instruments/interview guides that you will use to collect data.

  • Thursday, October 29, 2015; 8:45 AM - 10:15 AM

    Lecture: Qualitative Methods in Evaluation

     

    Faculty:  Kimberly Koester

    Location:  Mission Hall 1108

    • Session Slides:

    • Required Reading:

    • A guide to quant analysis GAO File
      Not available unless: Your ID number contains 02
    • How to represent qual data File
      Not available unless: Your ID number contains 02
    • Survey Guide File
      Not available unless: Your ID number contains 02
    • Quick reference guide: Activities requiring or not requiring CHR review and approval prior to initiation. URL

  • Thursday, November 5, 2015; 8:45 AM - 10:15 AM

    Lecture: Multi-level Interventions and Measurement

    Faculty:  Kang Dufour

    Location:  Mission Hall 1108

    • Session Slides:

  • Thursday, November 12, 2015; 8:45 AM - 10:15 AM

    Lecture: Evaluation in the Global Context

    1. Identify outcomes that can be appropriately attributed to the program; 2. Choosing an evaluation design; 3. Selecting methods to answer the research questions.

    Faculty:  Barrot Lambdin

    Location:  Mission Bay 1108

    • Session Slides:

    • Required Reading:

    • Rugg et al File
      Not available unless: Your ID number contains 02
    • Patton File
      Not available unless: Your ID number contains 02
    • 1471-2458-10-447 File
      Not available unless: Your ID number contains 02
    • Beyond Indicators Advances in Global HIV.11[1] File
      Not available unless: Your ID number contains 02

  • Thursday, November 10, 2015; 8:45 AM - 10:15 AM

    Lecture: Evaluating the implementation of Patient Centered Medical Home

     

    Faculty:  Starley Shade

    Location:  Mission Hall 1108

    • Session Slides:

  • Thursday, November 26, 2015; Thanksgiving - No Class

     

  • Thursday, December 3, 2015; 8:45 AM - 10:15 AM

    Lecture: Student Presentations

    Faculty:  Janet Myers

    Location:  Mission Hall 1108

    • Session Slides: