Module4Assignment - Copy

Module 4 Assignment College of Professional Studies, Northeastern University ALY6015, 21626 Harpreet Sharma February 5 th , 2024 Table of Contents Introduction ................................................................................................................................ 3

Analysis ...................................................................................................................................... 3 Ridge Regression .................................................................................................................... 3 Figure 1 .............................................................................................................................. 3 Ridge Regression with Cross-validation ............................................................................. 3 Figure 2 .............................................................................................................................. 4 The plot of Cross-validation result of Ridge Regression .................................................... 4 Figure 3 .............................................................................................................................. 5 Coefficients of the lambda min model ................................................................................. 5 Figure 4 .............................................................................................................................. 5 Coefficients of the lambda 1se model ................................................................................. 5 Lasso Regression .................................................................................................................... 6 Figure 5 .............................................................................................................................. 6 Lasso Regression with Cross-validation ............................................................................. 6 Figure 6 .............................................................................................................................. 7 The plot of the Cross-validation result of the Lasso Regression ......................................... 7 Figure 7 .............................................................................................................................. 8 Coefficients of the lambda min model ................................................................................. 8 Figure 8 .............................................................................................................................. 8 Coefficients of the lambda 1se model ................................................................................. 8 Conclusion/Interpretation ......................................................................................................... 10 References ................................................................................................................................ 11 Appendices ............................................................................................................................... 12 2

Introduction This report focuses on building regularization models using Ridge and Lasso regression techniques on the College dataset from the ISLR library which comprises 777 records and 18 variables. To address the problem of multicollinearity and overfitting in predictive modeling, regularization methods such as Ridge and Lasso are used. The objective is to use different predictor variables in the data set to predict graduation rates. Analysis 1. The dataset is split into a training set and a testing set (see Appendix A). This splitting is crucial for evaluating the performance of the models on unseen data. Ridge Regression 2. Ridge regression with cross-validation is performed on the training data to find the optimal regularization parameter (see Appendix B). Figure 1 Ridge Regression with Cross-validation As shown in Figure 1, Lambda min (1.775) minimizes MSE for better predictive accuracy but can lead to a more complex model. Lambda 1se (16.558) offers a slightly more regularized model within one standard error, striking a balance between simplicity and accuracy. 3

3. Plot of the Results Figure 2 The plot of Cross-validation result of Ridge Regression As shown in Figure 2, The x-axis displays the log of λ, and the y-axis represents the mean-squared error. The figures above the plot indicate the number of variables (with non-zero coefficients). The two dashed lines represent two lambda values: lambda min on the far left and lambda 1se on the right. 4. Fitting a Ridge Regression Model A regression model is fit against the training set and the following coefficients are obtained (see Appendix C). 4

Figure 3 Coefficients of the lambda min model Figure 4 Coefficients of the lambda 1se model As shown in Figures 3 and 4, what seems interesting is that the ridge regression models illustrate the balance between reducing coefficients towards zero and retaining all features, highlighting the compromise between model complexity and predictive accuracy. 5. Performance of Fit Model against the Ridge Training Set by RMSE The Ridge regression model with lambda min has an RMSE of approximately 12.54 on the training set, while the model with lambda 1se has an RMSE of 5