MachineLearning:Course:LinearRegression
lastmod 2022-11-14created 2018-07-2016 edits
Explains the representative machine learning model and the simplest one, linear regression.
Model Representation
ML_-Model_Representation-_genericmlatwork.png
(He raised his middle finger, training began, and a model was born) 1
A typical machine learning concept diagram looks like the above.
- Input Features: input variables
- Training Data-Set: training variables
- Learning Algorithm
- Cost function: the method for finding the Parameter(θ).
What is modeling?
- [Recommended] What is 'regression' in a regression model? 2
When modeling a regression equation, you first estimate the relationship between X and Y and then estimate the specific parameters. For example, in the regression equation for an object's volume and temperature given earlier, we first assumed that there is a linear relationship between X (temperature) and Y (volume) and that its weight is w. Then, within the relationship estimated this way, we estimate the coefficient, that is, what w specifically is.
But in the example above, if in reality there is no relationship at all between the object's volume and temperature, or volume is linearly related to the square of the temperature (volume = w * temperature^2), or it is closely related not only to temperature but also to humidity (volume = w1temperature + w2humidity), then even if you estimate w so that the sum of errors is minimized, that regression equation will not be an accurate regression model. So if you predict volume using this regression equation, there will inevitably be a difference from the actual volume.
On the other hand, if volume really is linearly related to temperature, this regression model estimates the relationship between volume and temperature well, so estimating the parameter w using this relationship yields an accurate regression equation. Yet even if you build such an accurate model, as mentioned earlier, various errors (noise) in the real world still cause a slight difference between the volume predicted by this model and the actual volume.
In the end, whether or not you build an accurate regression model, there is always a difference between the model's predictions and the measured values. However, the difference caused by an inaccurate regression model and the difference arising in an accurate regression model have fundamentally different properties. That is, if you built an accurate model, the residuals regress to the mean of 0, but in an inaccurate model they do not. This is the property of residuals mentioned above. So when judging whether the relationship you estimated is a proper regression model, you should check whether the expected value (mean) of the residuals between your model's predictions and the actual values is 0 and whether they regress to the mean.
In other words, to know whether the error between the model you estimated and the actual data is simply noise that exists in the real world or is due to some attribute you failed to consider, you must check whether the residuals are i.i.d. To do this, you need to check that all of the following conditions are satisfied.
- The distribution of the residuals must be a normal distribution.
- There must be no correlation between the residuals and the independent variable X, and none with themselves (the residuals must be independent).
- The distribution of the residuals must be constant (the residuals must satisfy homoscedasticity).
If all the above conditions are satisfied, the residuals, the differences between the model's predictions and the actual data, regress to the mean. In other words, because of noise that inevitably exists in the real world, predictions and measured values never match perfectly, and for some data the difference may be large, but looking at the overall trend, they converge to the mean of the residuals (usually 0).
Categories
Linear regression
Infer which Linear function the dataset I currently have is.