---
title: 'Lab #6: Boosting'
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
set.seed(24)
```

We will now analyze the dementia dataset using gradient boosted trees.

## Gradient boosted trees

1. Install the `xgboost` package

```{r}
```

2. Load the `xgboost` package
```{r}
```

3. Read in the dementia data "dementia.csv" into a data frame called `dat` 
```{r}
```

4. We will fit a gradient boosted tree to predict `Dementia`. To use `xgboost`, convert the `Dementia` variable to have 0/1 values.
```{r}
```

5. Set the random seed to 0. Select 260 rows at random for training, 200 for validation, and the remaining 200 for test.
```{r}
```

6. Fit a logistic generalized boosted regression model with `Dementia` as response and all other variables as predictors on the training data. Run the model for 4000 trees, a learning rate (eta) of 0.1, and tree (max_depth) depth of 2. Train using the "binary:logistic" objective. Add a `watchlist` so we can track the performance on the training and validation dataset.
```{r}
```

7. Plot how the training and test loss change as we run the training procedure. If we could stop the training early, which training iteration should we have stopped to minimize the validation error?
```{r}
```
8. Implement an early stop solution that stops training when the validation performance doesn't improve for 5 rounds. How many iterations did this train for?
```{r}
```

9. What is the test performance of the model with early stopping versus the model that trained for a fixed number of rounds? Did early stopping improve performance?
```{r}
```

10. You can investigate the contributions of each feature to individual predictions by setting `predcontrib = TRUE` in the `predict` function. Try this for the first test observation using the tree with early stopping. The outputted values are SHAP values that sum to the difference between the expected output of the model and the current prediction, in terms of the log odds. The feature contributions are given in order. The last value is the bias. Plot the SHAP values.
```{r}
```