Homework 1 Help

Homework 1 Help

by Nicholas Arger -

Hi Dr. Gidden and helpful TA's,

I have been doing homework 1 and have a few questions:

Question 2. To fix the data, do we need to fix things in excel before importing it or do we do everything to fix the data in SPSS? 

 - For example, I changed CPSC case number to "ID" and told the stream to "ignore" the "narrative columns." The ones I am having the most difficulty with is the "other race" and "other diagnosis" columns - should we try to recode using the "reclassify" nodes or just ignore them since they are present in such as small amount of the data?

- The other major problem I am having is with the age category. The data coded children under 2 years old starting with 200 and adding one integer for every full month old they were. This greatly skews the data so I added a "derive" and put in the formula:

if Age>200 then (Age-200)/12 else Age endif

This at changed the ages to non-integers that were between 0 and 2, however I did not know what to do with data that were already classified as "0" because the original code used this to designate ages that were missing. I tried to do the "filter" node but couldn't figure out how to get SPSS to ignore 0 values so I ended up deleting the 0 values from the excel sheet and then re-uploading it into SPSS. I assume there is a better way to do this.

Question 5. Data quality issues - when I use the "data audit" node on my finished stream, in the "quailty" tab shows a lot of missing values for things like "sknewness." I am assuming this is because the dataset is large but I was wondering if there is a better way to see the data quality output to answer the question.

Question 5. To answer which variables were related to being admitted or dying, I tried running a "means" node and found that there were differences in ordinal/continuous variables, but could not interpret the specific diagnosis codes since they are nominal data. I also tried running a logistic regression but it was taking a long time, which I figured was because the dataset is moderately sized, but wanted to make sure that was how could answer the question before taking a long time on it.


Thanks!
Nikko

Re: Homework 1 Help

by Chuck Mcculloch -

You should be able to fix the issues within SPSS Modeler.  

More specifics:

other_race and other_diagnosis  Have you looked at the actual values in other_race? Are they likely to be useful?  Is there a high percentage of missing data for the variables?  If so, and you include that variable, all the cases with missing data will be dropped.  Is it worth it?

Yes - you should first deal with the zero values that should be missing.   There are a couple of ways to do this.  You can use a Derive node and set the value of age to missing if age is equal to zero.  The trick here is to insert "undef" as the new value (without the quotes) when age is 0.   You can also do it in the Type node.  If you open up the Type node and double-click on age you can check the "Define blanks" and set the range for missing values as 0 to 0.  

For data quality analysis you need to change the tab in the Data Audit Node.  At the top are tabs for "Audit", "Quality" and "Annotations".  Are you on the "Quality" tab?

Logistic is certainly one method for predicting a binary outcome.   But yes - including an input with many levels makes the models run longer. 

Re: Homework 1 Help

by Teresa Kortz -

Hi Nikko,

I used the Matrix node (https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.modeler.help/matrixnode_general.htm) to explore differences in dispo between categorical variables. If you do a cross-tab of fields it will generate a table, perform a chi-squared test, and give you a probability.

Re: Homework 1 Help

by Pingyang Liu -

Hello Nikko,


Thanks for bringing up issues in the homework. I also have a hard time figuring out some of the answers. I guess we could bring these up during Monday's class.

For me, the biggest problem is when I audit the data, there are a lot of missing treatment dates. It seems like the software only reads up to 12 (for the number of days in a month) so there are approximately 60% of the dates were missing from the analysis. 

For the analysis part, I am not sure what model to use since we just had the first class. I was hoping the lectures could cover it before we dive in. My way to this question was: since the admitted/dying rate of 8.08% out of all records, any variable that can show/predict admitted/dying >8.08% could be considered as related, which could be obtained by looking at the figures generated from audit node. Since the instruction ends with creating an auditing node, I am not sure the homework is intended to let us figure out which model to use at the moment.

For the missing values in age group, not sure if 0 was put there specifically to indicate missing age as it is for the sex when they recorded. Therefore I do not consider them as missing information from the data set.  I consider the 200+ an issue of inconsistency (which served well for me to answer one of the questions ) as I saw they also put 2 to indicate 2 year old. Appreciated your effort to calculate and share the formula to convert the month to year.


Best,

Pingyang

Re: Homework 1 Help

by Chuck Mcculloch -

You mentioned lots of missing date data.  Have you looked at the original data with, say Excel?  Or even Word?  Are the dates actually missing?

If not, the likely culprit is some failure at reading in the data. 

We aren't expecting you build models to answer the question about what appears to be associated.  Is that where you were stuck?   We expected you would do it using graphical methods or descriptive statistics.  Though you are welcome to try models to see which variables are selected.