---
title: 'Homework #3:  Support Vector Machines and Tree-based methods'
output: html_document
---

## Maximum margin classifiers

1. Run the following code to create a simple dataset. 
```{r simulation}
set.seed(0)
X1 <- c(3,1,3,1,2,4,4)
X2 <- c(4,2,3,4,1,3,1)
Y <- as.factor(c(rep("Red",4),rep("Blue",3)))
dat <- data.frame(X1,X2,Y)
print(dat)
```

2. Plot the data. Color-code the points by their outcome value `Y`.
```{r}
```

3. (3 points) Fit a maximal margin classifier with a linear decision boundary to this dataset using the `svm` function from `e1071`. Use the option `scale=FALSE` so that the model does not do post-processing of the training data. Plot the fitted model.
```{r}
```

4. (1 point) Use the `coef` function to extract model coefficients from the maximum margin classifier.
```{r}
```

5. (2 points) Suppose we are going to add another observation to the dataset and refit the maximum margin classifier. Provide a new datapoint that, when added to this dataset, will not change the decision boundary. Refit the model to double check. Call this new datapoint `point1`.
```{r}
```

6. (2 points) Now provide a new datapoint that, when added to dataset `dat`, WILL change the decision boundary. Call this new datapoint `point2`. Pick an example point where the new dataset remains linearly separable.
```{r}
```

## Support vector machines
 
7. Read in the dementia data "dementia_full.csv" into a data frame called `dementia_dat`. This data is from the UCSF Memory and Aging Center. The goal was to predict the type of dementia based on patterns of brain loss as measured through structural MRI.
```{r}
```

8. (1 point) We will now predict the dementia diagnosis based on the available predictors. The diagnosis is given in the multiclass outcome of `MacCohort_kr`. To start, generate a table of the elements in the `MacCohort_kr` variable.
```{r}
```

9. (1 point) Set the random seed to 7 and then split the data into 2 sets (440 train, and 220 test)
```{r}
```

10. (3 points) Fit a support vector machine to predict `MacCohort_kr` using predictors `Left_MTG_middle_temporal_gyrus`, `Left_Amygdala` and `Left_AIns_anterior_insula`,  with a radial basis kernel. Use 3-fold CV to tune the value of the `C` and `sigma` hyperparameters. Use the `caret` package with `method=svmRadial` and tune over the values `C = 1e-2,1e-1,1,10,100,1000,10000` and `sigma=1e-4,1e-3,1e-2,1e-1, 1,10`. (You may need the `kernlab` package to run this.)
```{r}
```

11. (1 points) Determine the predictions of the SVM fit on the test dataset. What is the test accuracy?
```{r}
```

12. (3 points) What is the accuracy of a model that randomly guesses the class label simply based on the fraction of observations in each class? Is the SVM doing better than random guessing?
```{r}
```

## Random Forests

13. (2 points) Read in the prostate cancer dataset "Prostate_GSE6919_U95C.csv". Split 70% of the data for training and 30% for testing.
```{r}
```

14. (2 points) Fit a random forest with cancer outcome (`type`) as outcome and all other gene expression values as candidate predictors. Only consider 10 candidate predictors per node. (You may need to change the variable names to get this to work.) Use the "impurity" option to measure variable importance.
```{r} 
```

15. (1 point) Determine the AUC of the fitted model on the test dataset.
```{r}
```

16. (2 points) Based on the variable importance measures in the random forest, which gene was most important? What proportion of genes have a nonzero variable importance?
```{r}
```

17. (1 point) If you were going to fit a bagged model, how would you modify the call to the random forest code? Hint: How many candidate variables would you consider at each split? Note that you don't have to fit the model.
```{r}
```

18. (3 points) Tune `mtry` for the random forest model using 3-fold CV on all the prostate cancer data over the values `10,20,40,80,160,320`. Set `min.node.size` to 1. Which mtry leads to the best cross-validated AUC? What is the cross-validated AUC of the selected model?
```{r}
```

## Gradient boosted trees

Let's now fit a gradient boosted tree to see how variable importance can change.

19. (2 points) Fit a gradient boosted tree for this data using `xgboost`. Use 1000 trees, 0.1 eta, and max depth of 1. Make sure to use an appropriate loss function for the binary prediction task.
```{r}
```

20. (1 point) What is the AUC of this fitted model?
```{r}
```


21. (3 points) What is the most important gene selected by xgb? To get this, take the absolute SHAP value of the test observations and find the gene with the highest magnitude SHAP value on average. Is it the same gene selected by the random forest? What is the rank for the top gene selected by the random forest?
```{r}
```