---
title: 'Lab #4:  Support Vector Machines'
output:
  html_document:
    df_print: paged
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
set.seed(24)

library(e1071)  ### library containing svm function
library(ggplot2)
library(ROCit)
```

1. Simulate data for a binary classification problem using the following code. The training data will contain 10 observations from each class.
```{r sim}
create_data <- function(nsamp1, nsamp2) {
  nsamp <- nsamp1 + nsamp2
  x <- matrix(rnorm(nsamp*2), ncol=2)
  y <- c(rep(-1,nsamp1), rep(1,nsamp2))
  x[y==1,] <- x[y==1,] + 1
  dat <- data.frame(x=x, y=as.factor(y))  
}

nsamp1 <- 10
nsamp2 <- 10
dat <- create_data(nsamp1, nsamp2)
```

2. Plot the data using `qplot` from the `ggplot2` library. Is the data perfectly separable? Where do you think the decision boundary should be?
```{r}
```

## Support Vector Classifier

3. Fit a support vector classifier on the data using the `linear` kernel. Set the `cost` parameter (the cost of violating the decision boundary) to 10. Name the model `svmFit`. Which points are the support vectors? How many of them are there?
```{r}
```

4. Use the in-build SVM `plot` function to visualize the fitted model with the training data.
```{r}
```

5. What happens if you decrease the cost to 0.1 when fitting the support vector classifier? Refit the model and plot it. How do the number of support vectors change?
```{r}
```

## Support Vector Machine

6. Let's consider a `radial` basis kernel now. Let's set `cost=10` and `gamma=0.1`. The `gamma` parameter controls how "wiggly" our decision boundary is. A smaller gamma means you are fitting a less wiggly decision boundary. Plot the fitted model.
```{r}
```

7. Now let's use the `radial` basis kernel with the same cost but set `gamma=100`. Compare the refitted model to the one above. Why do you think most of the plotted region is now classified to one class?
```{r}
```

## ROC curves for SVM

8. Create a test dataset `dat_test` with 100 samples from both classes.
```{r}
```

9. To get an ROC curve, we need the decision values from a model. Classify the points from `dat_test` using the support vector classifier `svmFit` by running `predict` and pass in the argument `decision.values=T`.
```{r}
```

10. The decision values from `svmFit` are stored in the attribute `decision.values`. Use this to plot an ROC curve.
```{r}
```
## SVM with Multiple Classes

11. Use the following code to create a third class. We'll append this new class to the previously simulated data.
```{r sim2}
set.seed(1)
nsamp3 <- 30
x <- matrix(rnorm(nsamp3*2), ncol=2)
y <- rep(0,nsamp3)
x[,2] <- x[,2] + 2
dat_class0 <- data.frame(x=x, y=as.factor(y))
dat_3class <- rbind(dat, dat_class0)
```

12. Plot the data using `qplot`.
```{r}
```

13. Fit a support vector classifier for the 3 classes with `cost=10`. Go through the documentation for `svm`. How is this 3-way classifier constructed?
```{r}
```

14. Fit a support vector machine for the 3 classes using the radial basis kernel with `cost=10` and `gamma=1`. Which kernel do you think is most appropriate for this data?
```{r}
```