Team Ravenpuff

Team Ravenpuff

by Elizabeth Black -

Hi all,

I know we didn't quite finish the write up during the lab section.  Some of the questions for the write up are pretty hard to do without having the Orange workflow.  The questions we need to answer are as follows:

a.     Did your group make any general observations about the data structure or distribution?

b.     What models did you consider and why?

c.     What validation scheme did you settle on?

d.     How did you make your modeling approaches and evaluation reproducible?

e.     Was there variability in AUC among candidate models?

f.       What was your winning model and its performance for AUC?

g.     Describe EXACTLY the hyperparameter values you used for your winning model. If possible, take a screen shot in Orange.

h.     Use a confusion matrix to characterize how your winning model got predictions wrong.


Re: Team Ravenpuff

by Elizabeth Black -
So that we don't put the entire burden of answering questions on one person, I've tried my best to answer some of these questions:
a. Did your group make any general observations about the data structure or distribution?
We used the summary statistic feature in Orange to visually inspect our data before selecting variables for inclusion and building our model. We found that there was very little missingness in the data, so chose not to exclude variables form our model for missingness or to impute missing variables. Based on our initial assessment of the data, we chose to include the following variables in our model: ***
b. What models did you consider and why?
We considered different models to examine our dichotomous outcome including logistic regression (both regular and lasso), random forest, KNN, SVM, Adaboost, and Naive Bayes. We selected the best performing of these models and stacked them.
d. How did you make your modeling approaches and evaluation reproducible?
We ensured that our model would be reproducible by using deterministic (replicable) sampling when dividing our data into test and train subsets.
e. There was wide variability among our candidate model. The worse had an AUC of 0.5 (no better than chance) and the best had an AUC of 0.75.