---
title: 'Lab #3:  Penalized Regression'
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
set.seed(24)

library(MASS)

```

## Subset Selection

1. Install the `leaps` package

```{r}
```

2. Load the `leaps` package
```{r}
```

3. Read in the breast cancer imaging data "ispy1doctored.csv" into a data frame called `dat` 
```{r}
```

4. How many predictors do you have available in the dataset? Are there categorical variables?
```{r}
```

5. Perform best subset selection using the `regsubsets` function. Use `MRI_LD_Tfinal` as the outcome (the longest diameter of the breast cancer tumor at the final visit pre-surgery) and all other variables as predictors. Look at a summary of the fitted models. Allow the method to fit models that use all the predictors.
```{r}
```

6. You can grab the coefficients from the best subset selection procedure using the `coef` function. Print out the coefficients from the model with the best subset of size 8.
```{r}
```

## Ridge regression

7. Install the package `glmnet`
```{r}
```

8. Load the `glmnet` package
```{r}
```

9. Convert the data into components `x` and `y` that are suitable for use in the `glmnet` package (we will continue to consider `MRI_LD_Tfinal` as the outcome)
```{r}
```

10. Generate a set of values to try for lambda from $\lambda = 10^{5}$ to $\lambda = 10^{-2} = 0.01$. Even space out these values on the log10 scale.
```{r}
```

11. Fit ridge regression to the dataset with `MRI_LD_Tfinal` as outcome and all other variables as candidate predictors for all of the lambda values you have created
```{r}
```

12. Look at the value of `lambda` associated with `ridgeMod[75]`
```{r}
```

13. Display the fitted coefficients associated with this value of `lambda`
```{r}
```

14. And then the $l_2$ norm associated with this value of `lambda`
```{r}
```

15. Repeat steps 17 to 19 for `lambda` associated with index 55. Is the difference in the $l_2$ norm in the direction that you would expect? Why/why not?
```{r}
```
## Lasso

16. Fit lasso regression to the dataset with `MRI_LD_Tfinal` as outcome and all other variables as candidate predictors for the same set of lambda values you considered in fitting ridge regression 
```{r}
```

17. What are the coefficients for the lasso fit for the 75th lambda value? How many coefficients are nonzero?
```{r}
```

## Cross-validation

18. Install and load the `caret` package. Let's tune the lasso penalty parameter `lambda` using the `caret` package.
```{r}
```

19. To run 5-fold cross-validation, use the `trainControl` method to specify that we'll be running 5-fold cv.
```{r}
```

20. To perform 5-fold cross-validation using the `train` method in `caret`, we need to specify the grid of hyper-parameter values. Create a data.frame called `caret_grid` with two columns: `lambda` and `alpha`. We'll keep the value of the `alpha` column fixed at 1 because we want to use the lasso. Use the values in `grid` for the `lambda` column.
```{r}
```

21. Perform 5-fold cross-validation using the `train` method in `caret` over the values in `caret_grid`. Which value of `lambda` did 5-fold CV pick?
```{r}
```

22. The final model selected by `caret` is given by `cv_model$finalmodel`. What are the coefficients in the selected model?
```{r}
```