Error, loss functions, and why they are needed

Any machine learning model boils down to a simple idea: it tries to approximate reality with a function. This means there will always be a gap between what is really happening and what the model says. We call this gap the error. It is important to understand one thing right away: the model does not know what “good” or “bad” means. It does not understand the meaning of the task. All it can do is reduce the number we give it – this number is the loss. Formally, the error is the deviation between $y$ and $\hat{y}$, and the loss is a function that turns this deviation into a scalar that is convenient to optimize.

Below we will go through several short but illustrative PHP cases that step by step connect the formulas with real‑world tasks:

• Case 1. MSE and the cost of a big miss
• Case 2. Choosing a model via the loss function
• Case 3. Log loss and classifier confidence
• Case 4. Same accuracy – different log loss
• Case 5. Training a model as minimizing error