Skip to learning content

DATA SCIENCE PYTHON PLAYGROUND

Machine Learning · Learn / Refresh

← ML Foundations lessonsQUESTIONS · MODELS · EVIDENCE
Honest evaluation · ML-F06 · 12–18 MIN

Seen is not unseen

Explain why training performance is insufficient.

Exercises within this concept

  1. FollowRead and run the PythonCurrent exercise
  2. ChangeExplain the Python result
  3. TransferExplain the Python result

Understand the idea

Training examples influenced fitting. Evaluation on separate rows asks whether the learned relationship transfers. A low training error can coexist with a large unseen error.

Explain why training performance is insufficient.ObservationsTraining → fit / validateFinal testProtect final rows from preparation and selection.
Schematic · Explain why training performance is insufficient.Scroll the diagram horizontally if needed.

Python skill: Subtract aligned pandas columns and distinguish training error from validation error.

Meet the syntax

pd.DataFrame
errors['validation_rmse'] - errors['train_rmse']
pd.DataFrame
Builds a small table from named columns and lists of their values.
errors['validation_rmse'] - errors['train_rmse']
Subtracts row by row using matching model labels; a positive gap means larger validation error.

Follow the code

Use the numbered comments to connect each Python block to the workflow above.

errors = pd.DataFrame({'train_rmse': [1, 4], 'validation_rmse': [8, 5]}, index=['A', 'B'])
answer = errors['validation_rmse'] - errors['train_rmse']

This practice: Read and run the Python. Next: Change · Seen is not unseen.

Given data · LINE24

24 observations. Deterministic teaching observations; values illustrate the concept rather than a real population claim. The dataframe df is supplied afresh for each Run.

LINE24 · first 8 prepared rows
distanceduration
15.30501
1.4782611.2361
1.9565211.8488
2.4347814.809
2.9130410.7589
3.391319.4972
3.8695717.89
4.3478315.531

Column meanings and units

Column names describe the supplied features and target. Keep the stated units and row identities when making comparisons.

Synthetic data are deliberately small and reproducible. Their patterns illustrate an idea; they are not evidence about a real population.

Input schema
ColumnStored type
distancefloat64
durationfloat64

Your task · Follow

Model A has training/validation RMSE 1/8; model B has 4/5. Store validation minus training RMSE for each model in a Series named answer, indexed A then B.

Hint 1 — Think

Subtract aligned pandas columns and distinguish training error from validation error.

Hint 2 — Tools

Use pd.DataFrame, errors['validation_rmse'] - errors['train_rmse']. Read the visible syntax meanings before editing.

Hint 3 — Approach

Model A has training/validation RMSE 1/8; model B has 4/5. Store validation minus training RMSE for each model in a Series named answer, indexed A then B. Keep the supplied row order and inspect the named output after running.

Explained solution
errors = pd.DataFrame({'train_rmse': [1, 4], 'validation_rmse': [8, 5]}, index=['A', 'B'])
answer = errors['validation_rmse'] - errors['train_rmse']

A has a gap of 7 and B a gap of 1. B also has the lower validation RMSE in this example. Training rows taught the model; validation rows ask about new examples. A gap alone does not select a model: compare the validation error itself on the same rows.

Helpful prior knowledge: Measuring prediction error These links are guidance, not locks.

Sources and API context

Examples run with this Playground’s scikit-learn 1.4.2 / Pyodide 0.26.4 runtime.

Your task · Follow

Model A has training/validation RMSE 1/8; model B has 4/5. Store validation minus training RMSE for each model in a Series named answer, indexed A then B.

answer

Ctrl/⌘+Enter: Run · Tab: indent · Esc then Tab: leave editor

Python loads when you run. Code and results stay in this activity only.

Run your code to inspect its output. Check uses that same run.