Linear regression

lastmod 2024-04-26created 2015-12-2333 edits

In statistics, linear regression is a regression analysis technique that models the linear correlation between a dependent variable y and one or more independent variables (or explanatory variables) X. When based on one explanatory variable it is called simple linear regression, and when based on two or more explanatory variables it is called multiple linear regression.

Linear regression models the regression equation using a linear predictor function, and the unknown parameters are estimated from the data. A regression equation built this way is called a linear model.

Linear regression was the first regression analysis technique to be studied in depth and used widely. This is because building a model that is linear in its unknown parameters is easier than building one that is nonlinear.

Linear regression has many use cases, but they can usually be summarized into one of the following two categories.

• If the goal is to predict values, linear regression is used to develop a predictive model that fits the data. The developed linear regression equation can then be used to predict y for x values that have no y.
• When there is a dependent variable y and associated independent variables X1, ..., Xp, linear regression analysis can be used to quantify the relationship between Xj and y. Xj may have no relationship to y at all, or it may be a variable that provides additional information.

Linear regression models are generally built using the least square method. Linear regression models can also be built with techniques other than the least square method. A linear regression model can also be built by minimizing a loss fuction. The least square method can be applied not only to linear regression models but also to nonlinear regression models. The least square method and linear regression are closely related, but that does not make them synonyms.

Simply

Put most simply, linear regression is a concept almost the same as a trendline in Excel. That is, it infers what Linear function the dataset I currently have follows.

Category

• Easy statistics with examples : Naver Blog - linear regression analysis ¹

• Univariate simple linear regression model: univariate simple linear regression model
• Univariate multiple linear regression model: univariate multiple linear regression model
• Multivariate simple linear regression model: multivariate simple linear regression model
• Multivariate multiple linear regression model: multivariate multiple linear regression model

Dependent and independent variables

• An independent variable represents an input or a cause,
• a dependent variable represents an output or an effect.

Cost function

A simple Cost function is as follows.

\[ W := W - \alpha\frac{1}{2m}\sum_{i=1}^{m} 2(Wx^{(i)} - y^{(i)})2x^{(i)} \] See this page for a detailed explanation of the formula.

Random linear datas

To sample random data in the form of a linear function, see the NumPy#Numpy generate data from linear function section.

sklearn APIs

• sklearn.linear_model.LinearRegression

DNN Regression

• (Deep Learning) DNN Regression

See also

• Machine learning
• Loss function
• Regression analysis
• Statistics
  • Dependent and independent variables
  • Errors and residuals
• Normal distribution
• GradientDescent:Example: Python Example using Linear regression & Gradient descent.

Favorite site

• Wikipedia (en) Description of linear regression
• Machine Learning Week2 - Linear Regression
• [Recommended] ML 01: Linear Regression
• [Recommended] 1) Univariate Linear Regression - Machine Learning lecture notes
• 2. TensorFlow linear regression analysis – First Contact with TensorFlow
• [Recommended] Linear regression analysis using TensorFlow
• [Recommended] What is 'regression' in a regression model? ²
• Machine Learning Chapter 3 - Cost minimization algorithm for Linear Regression -
• 06. Implementing the cost function directly in pure Python
• [Recommended] Linear Regression with NumPy - Davi Frossard ³
• 3.1. Linear Regression — Dive into Deep Learning documentation

References