---
title: 'Lab #6: Boosting'
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
set.seed(24)
```

## Gradient boosted trees

1. Install the `gbm` package

```{r}
```

2. Load the `gbm` package
```{r}
```

3. Read in the dementia data "dementia2.csv" into a data frame called `dat` 
```{r}
```

4. We will fit a gradient boosted tree to predict `Dementia`. To use `gbm`, convert the `Dementia` variable to have 0/1 values.
```{r}
```

5. Select 330 rows at random to be the training set and call the set of indices `train`. The other 330 observations will be the `test` set. 
```{r}
```

6. Fit a logistic generalized boosted regression model with `Dementia` as response and all other variables as predictors except for `MacCohort_kr` on the training data. Run the model for 10000 trees, a shrinkage parameter (lambda) of 0.01, and tree (interaction) depth of 2. Run the `summary` command on the fitted model.
```{r}
```

7. Plots the marginal distribution of the outcome with respect to the predictor `Left_MTG_middle_temporal_gyrus`. Do this using the `plot` function. To learn how to use it, type `?plot.gbm`.
```{r}
```

8. Do the same but for the weaker predictor `Left_MFC_medial_frontal_cortex`. How much does the distribution of the `Dementia` outcome vary with respect to `Left_MFC_medial_frontal_cortex` versus `Left_MTG_middle_temporal_gyrus`?
```{r}
```

9. Rerun gbm with the "adaboost" distribution instead of "bernoulli". Then run the `summary` command on the fitted model.

```{r}
```


10. Do you get the same most important predictor as in the logistic version?
```{r}
```

11. Comparing the two ordered list of variables ranked by importance, what is the first index where they differ?

```{r}
```


12. In boosting, the number of trees is a tuning parameter. Tune the number of trees using 3-fold cross-validation. Use the `caret` package and set `method=gbm`. Pick the tree with the smallest AUC by passing in `summaryFunction=twoClassSummary` to the `trainControl` method, and setting `metric=ROC` in the `caret::train` function. Search over values of `n.trees` from 100 to 10000 in steps of 100. Use the same values as before for the other hyperparameters `shrinkage=0.01` and `interaction.depth=2`.
```{r}
```

13. How many trees did you end up picking?
```{r}
```

14. What is the AUC of the tuned model on the test data?
```{r}
```

15. Examine the variable importance in the tune model. Is it similar to before?
```{r}
```