Overfitting vs Underfitting Explained.

Introduction

One of the most important goals in machine learning is building models that perform well on both training data and new, unseen data. However, models often face two common problems:

• Underfitting — the model fails to learn important patterns
• Overfitting — the model memorizes the training data instead of generalizing

Understanding these concepts is essential for building reliable and accurate machine learning systems.

In this article, we will explain both concepts and demonstrate them using a practical Python example.

What Is Underfitting?

Underfitting occurs when a model is too simple to capture the relationships in the data.

An underfit model:

• Performs poorly on training data
• Performs poorly on testing data
• Fails to learn meaningful patterns

Example

Trying to predict complex house prices using only one feature may result in underfitting because the model lacks enough information.

What Is Overfitting?

Overfitting happens when a model learns the training data too well, including noise and unnecessary details.

An overfit model:

• Performs extremely well on training data
• Performs poorly on testing data
• Memorizes instead of generalizing

Example

A student memorizing exam answers without understanding concepts may struggle with new questions.

Practical Example in Python

In this example, we use polynomial regression to demonstrate:

• Underfitting
• Good fitting
• Overfitting

Step 1: Import Libraries

import numpy as np

import matplotlib.pyplot as plt

from sklearn.preprocessing import PolynomialFeatures

from sklearn.linear_model import LinearRegression

from sklearn.pipeline import make_pipeline

Step 2: Create Sample Data

# Generate random data

np.random.seed(42)

X = np.sort(np.random.rand(30, 1) * 10, axis=0)

y = np.sin(X).ravel() + np.random.randn(30) * 0.2

This creates a dataset with a non-linear relationship.

Step 3: Create Models

We will create:

A simple model (underfitting)

A balanced model (good fit)

A highly complex model (overfitting)

degrees = [1, 4, 15]

plt.figure(figsize=(15, 5))

for i, degree in enumerate(degrees):

model = make_pipeline(

PolynomialFeatures(degree),

LinearRegression()

)

model.fit(X, y)

X_test = np.linspace(0, 10, 100).reshape(-1, 1)

y_pred = model.predict(X_test)

plt.subplot(1, 3, i + 1)

plt.scatter(X, y, color='black')

plt.plot(X_test, y_pred)

plt.title(f"Degree {degree}")

plt.show()

Understanding the Results

Degree 1 — Underfitting

The model is too simple and cannot capture the curve in the data.

Degree 4 — Good Fit

The model captures the main pattern while still generalizing well.

Degree 15 — Overfitting

The model becomes overly complex and starts fitting noise instead of meaningful patterns.

How to Prevent Underfitting

• Use more complex models
• Add useful features
• Train for longer
• Reduce excessive regularization

How to Prevent Overfitting

• Use more training data
• Apply regularization techniques
• Use dropout in neural networks
• Apply early stopping
• Reduce model complexity

Real-World Importance

Avoiding overfitting and underfitting is critical in:

• Fraud detection
• Medical diagnosis
• Recommendation systems
• Financial forecasting
• Autonomous vehicles

Models that fail to generalize can produce inaccurate or unreliable predictions.

Conclusion

Overfitting and underfitting are two of the most important concepts in machine learning. Underfitting occurs when a model is too simple to learn patterns, while overfitting happens when the model memorizes training data instead of generalizing.

The key objective is to find the right balance between simplicity and complexity so that the model performs well on both training and unseen data.