Class Update and Final Project Dataset Guidance

Class Update and Final Project Dataset Guidance

by John -
Number of replies: 1

Hi Everyone,

A couple of people were having difficulties with the NSQIP dataset with regards to setting up missing data. While setting the -99's to missing is possible through the Type node, this does not appear to be working appropriately (likely a bug with SPSS). Alternatively, this can be solved by using a Select node to remove any records with -99, but this may remove some observations that were desired. Hence, I have uploaded a cleaned dataset for the NSQIP dataset 1% sample on the CLE Website that contains all of the -99s converted to a missing value code (which you may impute as you see fit in SPSS). If you are interested in using the larger full NSQIP file, e-mail me, and I can send you the Stata do-file that I used to convert the entire dataset.

With respect to those of you doing the Genomics dataset, it appears that neither the clustering or classification algorithms work very well for this dataset given the large number of variables (may even crash your computer). I recommend selecting about 20-30 genes that you think may be of high-medium importance and remove the remaining variables. You can easily do this with the Filter node in SPSS.

Please be reminded that there are several deadlines within the coming 2 weeks. While there will not be a Lab assignment next week, we will be available for consultation for final projects during the lab session next Thursday, August 31. We encourage everyone to read the project guidance uploaded to the CLE website before submitting their final projects. By following a standard protocol, this ensures that we can provide you with the maximum number of points possible for your hard work! We look forward to reading all of your final projects!

As usual, please feel free to e-mail us with questions!

John & Zara

In reply to John

Re: Class Update and Final Project Dataset Guidance

by John Kornak -

Hi All,

I just want to add to John & Zara's comments, for those working on the genomics dataset, that (at least) 2 algorithms that do work with the full set of data are support vector machines and C5.0. As some of you noticed the C5.0 tree is very simple... but it does work really well on the test data. Simple and accurate is very nice!

Cheers,

John