Visualizing the Cost Function for the Model with Bias Term

Model Definition

Hypothesis (Model): $f_{w,b}(x) = wx + b$
Parameters: $w$ (weight), $b$ (bias)
Cost Function: $J(w, b) = \frac{1}{2m} \sum_{i=1}^{m} \left( f_{w,b}(x^{(i)}) - y^{(i)} \right)^2$
Objective: Minimize $J(w,b)$ with respect to $w$ and $b$
Training Dataset:

Size (sq ft)	Price ($1000s)
1000	230.0
1500	330.0
2000	430.0
2500	530.0
3000	630.0

Case-1

Consider the model: $f_{w,b}(x) = 0.06x + 50$

This model consistently underestimates housing prices when above training dataset considered Figure 1:

Red x-shaped observations representing housing prices (in $1000s) vs size (sq ft)
A bold black line for the model $f_{w,b}(x) = 0.06x + 50$ , which does not fit the data well

Figure 2 shows the cost function $J(w, b)$ as a 3D surface:

The surface shows how cost varies with different values of $w$ and $b$
The purple dot marks the cost at $w = 0.06$ , $b = 50$ , with dashed lines indicating the height (cost value)

Figure 3 presents a contour plot of the cost function $J(w, b)$ :

Each contour line represents combinations of $w$ and $b$ that yield the same cost
The red dot marks the location of the current model parameters $w = 0.06$ , $b = 50$

Case-2

Consider the model: $f_{w,b}(x) = -0.15x + 800$

This model significantly overestimates housing prices, especially for smaller homes. Figure 4 shows:

Red x-shaped observations representing housing prices vs home size
A bold black line for the model $f_{w,b}(x) = -0.15x + 800$ , which does not fit the data well

Figure 5 shows the cost function $J(w, b)$ as a 3D surface:

The surface illustrates how cost varies with different values of $w$ and $b$
The purple dot marks the cost at $w = -0.15$ , $b = 800$

Figure 6 presents a contour plot of the cost function $J(w, b)$ :

Each contour line represents combinations of $w$ and $b$ that yield the same cost
The red dot marks the location of the current model parameters

Case-3: Best Fit Model

$f(x) = 0.20x + 30.00$

This model provides the lowest cost and aligns closely with the observed data. Figure 7 shows:

Red x-shaped observations representing housing prices vs home size
A bold black line for the model $f(x) = 0.20x + 30.00$ , which fits the data very well

Figure 8 shows the cost function $J(w, b)$ as a 3D surface:

The surface illustrates how cost varies with different values of $w$ and $b$
The purple dot marks the minimum cost at $w = 0.20$ , $b = 30.00$

Figure 9 presents a contour plot of the cost function $J(w, b)$ :

Each contour line represents combinations of $w$ and $b$ that yield the same cost
The red dot marks the location of the optimal model parameters

Cenk Yildiran