What is multicollinearity and why is it important to recognize when applying AI?

Multicollinearity is a statistical concept that appears regularly in AI and machine learning models and can lead to biased results and interpretations. In this blog, I explain what multicollinearity is, why it can be a problem in predictive models, and how to address it.

What is multicollinearity?

Multicollinearity occurs when two or more independent variables in a statistical model correlate strongly with each other. This means that these variables contain similar information, making it harder to determine what effect each variable individually has on the dependent variable you're trying to predict. As a result, the estimates of the model parameters can become unreliable, which negatively affects the model's predictions.

A simple example

Let's say you want to predict an employee's salary and you use the following attributes as input variables:

• Age
• Number of years of work experience
• The sector in which someone works

These variables are called independent variables, because they can all influence the dependent variable, in this case the salary. But in this example, age and years of work experience can be strongly related. After all, the older someone is, the more years of work experience that person is likely to have. This ensures a high correlation between these two variables, which is a typical form of multicollinearity.

Why is multicollinearity a problem?

If variables are highly interrelated, they can cause problems in your model. This is because it becomes difficult to determine which of the variables really influences the outcome. As a result, the model can provide very distorted predictions. In our example, it may happen that the AI model unfairly overestimates the influence of age and underestimates the influence of work experience, or vice versa. This leads to a reduced accuracy and reliability of the model.

How do you recognize multicollinearity?

You can detect multicollinearity by using the Variance Inflation Factor (VIF). This measure shows how much the variance of a model parameter increases due to the presence of correlation between the independent variables. If the VIF value of a variable is greater than 5, you are probably dealing with multicollinearity.

How do you solve multicollinearity?

1. Removing one of the highly correlated variables
   If two variables contain almost the same information, consider deleting one. For example, in our example, you can choose to extract either age or number of years of work experience from the model.
2. Using PCA (Principal Component Analysis)
   PCA is a technique that converts highly correlated variables into new, unrelated variables. This way, you retain the information but minimize the effect of multicollinearity.
3. Combining variables
   In some cases, you can combine the variables. For example, instead of using age and years of work experience separately, create a new variable that shows the ratio between the two.

Conclusion

Multicollinearity can significantly affect the performance of your AI models. By being aware of this problem and solving it with techniques such as removing redundant variables, PCA, or combining variables, you can make your models more robust and reliable.

Want to learn more about how to optimize your AI models? Then watch the video.