---
title: 'Lab #2:  Classification in R'
output:
  html_document:
    df_print: paged
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
set.seed(24)

library(MASS)

# install.packages("scatterplot3d") 
library(scatterplot3d) 

# install.packages("ROCit")
library(ROCit)
```

The purpose of this lab is to get you familiar with fitting classification models in R. We will start with binary classification and then multiclass classification.

## Simulating data:

1. Simulate a multivariate (3 variables) predictor with 3 classes using the following.
```{r sim_mvr}
nsamp <- 100 
SMat <- matrix(c(1.0, 0.6, 0.3, 0.4, 1.0, 0.7, 0.1, 0.1, 1.0), nrow=3, ncol=3) 
datmat1 <- mvrnorm(n = nsamp, mu = rep(-1.0, 3), Sigma = SMat) 
datmat2 <- mvrnorm(n = nsamp, mu = c(-1.0, 0.0,1.0), Sigma = SMat) 
datmat3 <- mvrnorm(n = nsamp, mu = c(0.0,0.5,-1.0), Sigma = SMat) 
```

2. Stack `datamat1`, `datamat2`, and `datamat3` to generate a single 300x3 matrix called `datamat`. 
```{r}
```

3. Generate a groups factor such that the first 100 data points are labeled `outcome1`, the second 100 `outcome2`, and the third 100 `outcome3`. 
```{r}
```

4. Combine groups and datamat into a data frame called `dat`. Give the dataframe column names `groups`, `predictor1`, `predictor2`, `predictor3`.
```{r}
```

5. Make pairwise plots of the predictors in the data using the command `pairs`.
```{r}
```

6. Re-do the scatterplot so that get each data-point is color coded by groups using the `col` argument.
```{r}
```

7. Use it to plot the data in 3D using `scatterplot3d`. Color-code the points by group.
```{r}
```

8. Split 1/3 of this data for training and 2/3 of it for testing. Name the data frames `train_dat` and `test_dat`, respectively. Make sure to shuffle before you split the data!
```{r}
```

## Logistic Regression

We will start with binary classification. To do this, we will need to subset the data to only contain observations with labels `outcome1` and `outcome2`.

9. Subset the dataset so that there are only the groups: `outcome1` and `outcome2`. Call the new data `dat2`. You should run `droplevels` to remove the unused level from the factor. Otherwise R will still think you have a `outcome3` group.
```{r}
```

10. Fit a logistic regression with groups as the outcome/response variable and all 3 predictors (additive).
```{r}
```

11. Run summary on your model.
```{r}
```

12. Generate confidence intervals for your model parameters.
```{r}
```

13. Generate odds-ratios given the coefficients of your model. 
```{r}
```

14. Generate odds-ratio confidence intervals for the fitted parameters. 
```{r}
```

15. Subset test dataset `test_dat` so that it only has observations from `outcome1` and `outcome2`. Using the logistic regression model to predict the probability each observation in this filtered test dataset.
```{r}
```

16. Use the `ROCit` package to fit and plot an ROC curve for this model using the test dataset using the `rocit` function.
```{r}
```

17. What is the AUC and its 95\% confidence interval (see `ciAUC`)?
```{r}
```

## Linear discriminant analysis

Now we will perform multiclass classification with all 3 outcome types.

18. Fit a linear discriminant analysis to the full training dataset with all 3 groups. 
```{r}
```

19. Compare the estimated group means with the values used in the simulation. 
```{r}
```

20. Examine the 2x2 table of predicted vs. actual observations in the training data. 
```{r}
```

21. Calculate prediction accuracy in the training data.
```{r}
```

22. Examine the 2x2 table of predicted vs. actual observations in the test dataset `test_dat`. 
```{r}
```

23. Examine the predictive accuracy in the test dataset `test_dat`. How different is it from the performance on the training data?
```{r}
```