Machine Learning 101 P5: Support Vector Regression with Python

Introduction

Linear Support Vector Regression (SVR) and Simple Linear Regression are both supervised learning models used for regression tasks. Since both belong to Regression family, both models aim ultimately at predicting a continuous variable. However, the differences between them are vast:

Looks like Linear Regression is more striaghtforward to use. Then why Support Vector Regression model? Here are those valid reasons:

• It can capture non-linear relationship. It works pretty well with complex relationship between input(s) and output.
• Outliers won’t have a huge influence on the model, unlike linear regression model.
• It works well with limited data (small- to medium-sized dataset).
• It introduces a margin of tolerance (ϵ), allowing you to define a range where predictions are considered “good enough” and not penalized.
• It performs well with high-dimensional feature spaces, especially when paired with kernels. It can efficiently handle problems with many features, such as text or image regression tasks.

Mathematical concepts behind SVR