Imershein #8 Debrief - Cluster and Time Series Analysis

Imershein #8 Debrief - Cluster and Time Series Analysis

by Sarah Imershein -
Number of replies: 2

#8 Clustering, Time Series

 

Describe the study design you will employ in order to determine if your intervention has had an effect on the outcome variable of interest.

 

The primary outcome variable of interest is change in OR safety culture and attitudes of surgical team members as assessed by the Safety Attitudes Questionnaire OR (SAQ-OR).  This is a pre-post assessment of culture immediately before launch of the Debrief and 1 year post-launch.

 

Secondary outcomes include patient indicators such as surgical site infection rates (SSIs), post-op cerebral vascular events (CVAs), venous thromboembolism (VTE), mortality, and cost index; and the process indicator of expected/observed procedure time.

 

Interrupted time series analysis will be employed to determine if the change in rate of these patient and process indicators is due to the intervention or not, and whether changes are correlated with degree of compliance with the intervention.

 

 

Define the unit-of-analysis for your main outcome evaluation, the minimum meaningful effect size, and the sample size necessary to detect this effect size.

 

The SAQ-OR is a 5-point Lickert Scale (1-5) survey.  If considered a continuous variable, the minimum detectable effect size change is 0.742 Lickert points, at α=0.05, and the pre-determined sample size of 113.  

Secondary outcomes:  For mortality, the t-test estimates 1630 cases at 80% power, α=0.05 to detect a 0.05 change in the mortality index. If the change in index is reduced to 0.03, 4524 are required.

If the autocorrelation is assumed to be 0.5 with 5 measurements before and 5 after implementation, the ratio of required sample sizes is 0.54, indicating a minimum of approximately 3019 cases will need to be analyzed for detecting 0.05 change in the mortality index.  8378 are needed to detect 0.03 change in the mortality index.  With an estimated 2800 neurosurgical cases each year, there will be approximately 11,200 cases to analyze by the end of the study period, this would predict a minimum detectable effect of 0.026.

 

 

In reply to Sarah Imershein

Re: Imershein #8 Debrief - Cluster and Time Series Analysis

by Heidi Moseson -

This is really interesting to see how the sample size changes as you hypothesize different effect estimates - and is also sort of sobering how quickly it jumps! I am curious where your estimate of the autocorrelation of 0.5 comes from...is this from prior literature? This seems moderate to fairly high, and perhaps if there is evidence to support the autocorrelation being lower, a smaller sample size might suffice. But if there is evidence for an autocorrelation of 0.5, then obviously very important to account for it with sufficient sample size. I only ask because of Chuck's comment that often people assume a high degree of autocorrelation, when in fact, it isn't there...

In reply to Heidi Moseson

Re: Imershein #8 Debrief - Cluster and Time Series Analysis

by Sarah Imershein -

I had a hard time finding autocorrelation published for mortality, so I used a conservative estimate.  But there is some literature showing mortality highly autocorrelated, as much as .8-.95 depending on diagnosis.  I ran a grid of more numbers for the exercise

80% power effect size t-test sample size Autocorrelation ratio (indep design/ITS) ITS sample size
0.03 4524 0.95 1.38 3278
0.03 4524 0.8 0.55 8225
0.03 4524 0.5 0.54 8378
0.03 4524 0.25 0.71 6372
0.03 4524 0.05 0.93 4865
0.05 1630 0.95 1.38 1181
0.05 1630 0.8 0.55 2964
0.05 1630 0.5 0.54 3019
0.05 1630 0.25 0.71 2296
0.05 1630 0.05 0.93 1753