First, we read the data into RStudio and load the caret package. There are 160 variables in the dataset. Upon initial investigation, it is found that many variables have very few rows with any sort of data i.e. most of the observations for those variables are NA. Hence, we remove those variables from consideration leaving us with 60 variables.
For cross validation we use the k folds method and using the first fold we create our first training and test sets.
pml<-read.csv("pml-training.csv",na.strings = c("NA",""," "))
library("caret")
pmltemp<-pml[ , colSums(is.na(pml)) == 0]
tr<-createFolds(pmltemp$classe,returnTrain = TRUE)
pml1<-pmltemp[tr[[1]],]
pml1t<-pmltemp[-tr[[1]],]We’ll use the 1st K-fold for this training. Then, we train the model using the boosting method with the following code:
modtemp<-train(classe~.,data=pml1,method="gbm",verbose="FALSE")
pre<-predict(modtemp,pml1t)
pmltest<-read.csv("pml-testing.csv",na.strings = c("NA",""," "))
pretest<-predict(modtemp,pmltest)table(pre,pml1t$classe)##
## pre A B C D E
## A 558 0 0 0 0
## B 0 379 0 0 0
## C 0 1 343 0 0
## D 0 0 0 322 0
## E 0 0 0 0 361As can be seen in the table, there is 100% accuracy. But in the testing set, it predicted all cases to have classe ‘A’. So, we decided to remove some more variables from our data. Thus, we create our new model with 59 variables. Also, we use the boosting method.
pmltemp<-pml[ , colSums(is.na(pml)) == 0]
pmltemp<-pmltemp[,-1]
pml1<-pmltemp[tr[[2]],]
pml1t<-pmltemp[-tr[[2]],]
modtemp<-train(classe~.,data=pml1,method="gbm",verbose="FALSE")
pre<-predict(modtemp,pml1t)
pretest<-predict(modtemp,pmltest)In the above model training, we use the second k-fold.
confusionMatrix(pre,pml1t$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 557 1 0 0 0
## B 1 378 0 0 0
## C 0 0 337 0 0
## D 0 0 5 320 1
## E 0 0 0 2 360
##
## Overall Statistics
##
## Accuracy : 0.9949
## 95% CI : (0.9906, 0.9976)
## No Information Rate : 0.2844
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9936
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9982 0.9974 0.9854 0.9938 0.9972
## Specificity 0.9993 0.9994 1.0000 0.9963 0.9988
## Pos Pred Value 0.9982 0.9974 1.0000 0.9816 0.9945
## Neg Pred Value 0.9993 0.9994 0.9969 0.9988 0.9994
## Prevalence 0.2844 0.1932 0.1743 0.1641 0.1840
## Detection Rate 0.2839 0.1927 0.1718 0.1631 0.1835
## Detection Prevalence 0.2844 0.1932 0.1718 0.1662 0.1845
## Balanced Accuracy 0.9987 0.9984 0.9927 0.9951 0.9980On our subtesting set, we have more than 99% accuracy i.e. the out-of-sample error. Also, it gave us a very good result on our testing set as confirmed by the final quiz of the course as shown below.
pretest## [1] B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E