Part 25: Loss Function Explained: How Does a Neural Network Know It Is Wrong?

Imagine you are learning to bake a cake for the first time. You follow a recipe, but the cake turns out too sweet. The next time, you reduce the sugar. You taste it again and make further adjustments until you get the result you want.

But how do you know whether your next attempt is better than the previous one?

You compare the result with what you wanted.

A loss function works in a similar way in Machine Learning and Deep Learning. It helps a model measure how far its predictions are from the actual answers.

Let's understand how!

🎯What Is a Loss Function?

A loss function is a mathematical function that measures the difference between a model's prediction and the actual result.

In simple words, it tells us how wrong a model's prediction is.

Let's take an example.

Imagine you are building a Machine Learning model to predict house prices.

• Actual house price: ₹50 lakh

• Model's prediction: ₹40 lakh

The model has made a mistake of ₹10 lakh.

A loss function measures this prediction error and assigns a numerical value to it. The model can then use this information to improve its predictions.

Generally, a lower loss means better predictions (predictions closer to actual answers), while a higher loss indicates larger errors (predictions farther from the actual answers). However, the meaning of the loss depends on the loss function being used.

🤔Why Do We Need a Loss Function?

Let's take an example.

Imagine you are preparing for an exam. You answered 50 questions out of which 35 were correct. After checking your answers, you identify your mistakes and work on the topics you struggled with.

A machine learning model follows a similar learning process.

Here's how it works:

Step 1: Make a Prediction

The model receives input data and predicts an output. For example, it predicts that a house will cost 60 lakh.

Step 2: Compare the prediction with the actual answer

The actual house price is ₹50 lakh, but the model predicts ₹60 lakh. The prediction is ₹10 lakh higher than the actual price.

Step 3: Calculate the loss

The loss function measures the prediction error and converts it into a numerical value based on the chosen loss function.

Step 4: Update the model

An optimization algorithm, such as Gradient Descent, uses information from the loss to help adjust the model's weights and reduce the loss.

Step 5:  Repeat

The model continues learning from the data.

Over multiple training steps, the model aims to reduce its loss and make better predictions.

Remember, the loss function measures the error, while the optimizer helps the model reduce it.

📊Types of Loss Functions

Different problems require different ways of measuring errors. Let's explore three commonly used loss functions.

📐1. Mean Squared Error (MSE)

MSE is commonly used for regression problems, where the model predicts numerical values.

It calculates the average of the squared differences between actual and predicted values, making errors count more heavily.

Formula:

Where,

n = Number of predictions

yi = actual values

y^i = predicted values

📏2. Mean Absolute Error (MAE)

MAE is another loss function used for regression.

Unlike MSE, it takes the absolute value of the error instead of squaring it.

Formula:

Where,

n = number of predictions

yi = actual values

y^i = predicted values

MAE is easier to interpret because it measures the average size of prediction errors in the original unit.

⚖️MSE vs MAE

Example: Two house-price predictions

Suppose the actual prices are ₹50 lakh and ₹60 lakh, while the model predicts ₹40 lakh and ₹65 lakh.

	House 1	House 2
Actual price	₹50 lakh	₹60 lakh
Predicted price	₹40 lakh	₹65 lakh
Error magnitude	₹10 lakh	₹5 lakh

MAE calculation

MAE = (10 + 5) / 2 = 7.5

The average absolute error is ₹7.5 lakh.

MSE calculation

MSE = (10^2 + 5^2 ) / 2 = (100 + 25) / 2 = 62.5

The MSE is 62.5 in squared-lakh units.

What does this tell us? MAE measures the average error directly, while MSE squares the errors, making larger errors count more heavily.

🎲3. Cross-Entropy Loss

Cross-entropy loss is commonly used for classification problems, where a model predicts the probability of an input belonging to a particular class.

Imagine an email model trying to classify a message as spam or not spam.

If the email is actually spam and the model predicts a 90% probability that it is spam, the loss is relatively low.

However, if the model predicts only a 10% probability that the email is spam, the loss is much higher.

Cross-entropy loss penalizes incorrect predictions, especially when the model is highly confident in the wrong answer. It encourages the model to assign higher probabilities to the correct classes.

💡Final Thoughts

A loss function helps a model understand how far its predictions are from the correct answers. By measuring errors and using them to guide training, models gradually learn to make better predictions.

🚀Coming Up Next

In the next blog, we'll explore Optimizers and discover how techniques like Momentum, RMSProp, and Adam help neural networks minimize loss and improve their learning process.