Tutorial 4 （Access Code: r@inBow246）

Can Regression Recover the Truth?

DSA3361 · Inferential Data Analytics
AY 2026/27 Semester 1
Zhu Xuelin

Can we trust a fitted line?

data

	x	y
1	x₁	y₁
2	x₂	y₂
⋮	⋮	⋮
100	x₁₀₀	y₁₀₀

We fit a line

sm.OLS(y,X).fit()

Can we trust a fitted line?

Same relationship:

+

Different measurements

Slightly different each time.

A world where we know the truth

We generate 100 sample points from

True .

import numpy as np

np.random.seed(123)
x = np.random.normal(
    loc=0, scale=1, size=100
)
eps = np.random.normal(
    loc=0, scale=np.sqrt(0.25), size=100
)
y = -1 + 0.5 * x + eps

yAssign toxeps

Fit the model and see what we get

We generate 100 sample points from

True .

We now fit the model :

import statsmodels.api as sm

Click each line to see what happens

import matplotlib.pyplot as plt

plt.scatter(x, y, color="black", alpha=0.75, label="Observed data")y_pred = model.fittedvalues # Compute predicted yorder = np.argsort(x)x_sorted, y_pred_sorted = x[order], y_pred[order]plt.plot(x_sorted, y_pred_sorted, color="red", linestyle="--", label="Fitted line")

plt.plot(x_sorted, -1 + 0.5 * x_sorted, color="blue", label="Population line")

plt.xlabel("x")plt.ylabel("y")plt.legend()plt.show()

Fit the model and see what we get

We generate 100 sample points from

True .

We now fit the model :

import statsmodels.api as sm

Click each line to see what happens

model.summary()

OLS Regression Results

Dep. Variable	y	R-squared	0.567
Model	OLS	Adj. R-squared	0.562
Method	Least Squares	F-statistic	128.3
Prob (F-statistic)	1.66e-19	Log-Likelihood	-69.520
No. Observations	100	Df Residuals	98
Df Model	1	Covariance Type	nonrobust
AIC	143.0	BIC	148.3

	coef	std err	t	P>\|t\|	[0.025	0.975]
const	-1.0095	0.049	-20.603	<0.001	-1.107	-0.912
x1	0.4917	0.043	11.325	<0.001	0.406	0.578

What if we add a quadratic term?

We generate 100 sample points from

True .

We now fit the model :

import statsmodels.api as sm

Click each line to see what happens

What if we add a quadratic term?

We generate 100 sample points from

True .

We now fit the model : Use the same code as before to plot.