scroll all the way to page 9! looking forward to discussing as i'm a little fuzzy on whether i'm envisioning this the right way.
Your protocol is very helpful for me. Thanks for "modeling" good protocol completion behavior! For your quantitative data in the mixed methods section, I think I would just use the outcome indicators you identified already, because survey data would most likely be qualitative, right?
"Proportion of veterans prescribed non-biologic DMARDs with abnormal laboratory values (or other adverse events) - data currently collected/monitored by pharmacy"
new thoughts during section:
(early) physician process measure: number of patients with labs ordered prior to next visit (to get at whether physicians are ordering labs in a timely way for patients to complete them
Hi Elizabeth,
Great job. I like how you included on CLE your thoughts from section about your new process measure! Way to use technology!
Q1: Consider making into a table, with indicators and measurement tool being separate columns. It’ll be easier to digest (and is very handy for grants). I’d actually move your two process indicators into outcome measures: proportion of veterans prescribed a non-biologic DMARDs who have labs every 3 months; proportion who are seen in f/u. This is (from your 1st page) really the outcome of interest for you. The abnormalities in these labs might be the carrot that helps providers do the screening. For your own sanity, I’d also choose a frequency of testing or f/u that is easy to collect for you—I’d choose 1 year—a low bar, but easier to collect and very powerful.
Re: process indicators—It all depends on your intervention. If you have education, you might measure the #of trainings and/or attendance at trainings. If you use incentivization, you could look at proportion of providers who received incentives.
Q2: If you chose education, a good intermediate indicator here would be a change in provider attitudes/knowledge about laboratory testing. The proportion of people having abnormal labs is not an intermediate indicator, but actually is a finding further downstream after you’ve gotten folks to get labs.
Q3: I like your idea of focus groups or semi-structured interviews. Focus groups are a little easier to do b/c it’s one-shot and you get a lot of opinions. I’d decide who you want to focus your intervention on (patients vs providers)—as you may initially want to make sure the physicians are writing the labs and THEN look at whether the patients actually get the labs done. The quantitative analysis is really having you tell the reader what outcome measures you are going to collect, and how, over what period. So, you might describe looking at the whole patient population in the clinic taking these medications and see the # who had any labs in 2014 and then after your intervention in 2015.
Thanks Christina, that is helpful feedback. Regarding the frequency of monitoring - this is really monitoring not screening, so the frequency really needs to be every 3 months - and our clinic providers agree/know that is what needs to happen. I think the gap in practice is less related to knowledge and more related to needing to feed-back information to providers in a time-sensitive way, so that without too much effort, providers can know which of their patients don't have labs ordered or completed, or a follow-up appointment scheduled. I'll try to build this piece into the process measures.
Hi Elizabeth,
You may have already made changes to your program evaluation section, but if not here are some ideas:
I think the outcome you are looking for is consistent q3mth monitoring of labs and appropriate clinic FU for patients on DMARDs right?
The outcome indicator you have listed now seems like a secondary outcome that might happen if you meet the outcome #1. I think process indicators could be like my protocol - proportion of staff completed training etc.
For the mixed method evaluation - since providers/clinic staff are your target, would it make sense to target them in the qualitative analysis rather than the patients in order to keep it simple? I suppose patients could be a secondary study.
See you tomorrow!
Emily