Skip to learning content

DATA SCIENCE PYTHON PLAYGROUND

Machine Learning · Learn / Refresh

← Choose and Explain Models lessonsQUESTIONS · MODELS · EVIDENCE
Compare fairly · ML-M02 · 12–18 MIN

Compare candidates on common evidence

Use matching evaluation designs and practical constraints.

Exercises within this concept

  1. FollowRead and run the PythonCurrent exercise
  2. ChangeReason about the Python
  3. TransferExplain the Python result

Understand the idea

Comparisons need the same population, outcome definition, split and folds when the aim is a paired predictive comparison. Nominate before the final test. Discovery and PCA use different evidence and must not be ranked by supervised accuracy.

Use matching evaluation designs and practical constraints.Candidate ACandidate BFitValidateCostCompare matching evidence; smaller error can cost more.
Schematic · Use matching evaluation designs and practical constraints.Scroll the diagram horizontally if needed.

Python skill: Compare aligned fold-score columns before summarising candidate performance.

Meet the syntax

fold_errors['tree'] - fold_errors['line']
fold_errors['tree'] - fold_errors['line']
Subtracts errors on matching folds. A negative value favours the tree on that fold.

Follow the code

Use the numbered comments to connect each Python block to the workflow above.

fold_errors = pd.DataFrame({'line': [4.0, 5.0, 4.5, 5.5, 4.0], 'tree': [3.5, 4.8, 5.0, 4.9, 3.8]})
answer = fold_errors['tree'] - fold_errors['line']

This practice: Read and run the Python. Next: Change · Compare candidates on common evidence.

Given data · LINE24

24 observations. Deterministic teaching observations; values illustrate the concept rather than a real population claim. The dataframe df is supplied afresh for each Run.

LINE24 · first 8 prepared rows
distanceduration
15.30501
1.4782611.2361
1.9565211.8488
2.4347814.809
2.9130410.7589
3.391319.4972
3.8695717.89
4.3478315.531

Column meanings and units

Column names describe the supplied features and target. Keep the stated units and row identities when making comparisons.

Synthetic data are deliberately small and reproducible. Their patterns illustrate an idea; they are not evidence about a real population.

Input schema
ColumnStored type
distancefloat64
durationfloat64

Your task · Follow

Across five matching folds, line RMSE is [4.0, 5.0, 4.5, 5.5, 4.0] and tree RMSE is [3.5, 4.8, 5.0, 4.9, 3.8]. Store tree minus line RMSE for each fold in answer.

Hint 1 — Think

Compare aligned fold-score columns before summarising candidate performance.

Hint 2 — Tools

Use fold_errors['tree'] - fold_errors['line']. Read the visible syntax meanings before editing.

Hint 3 — Approach

Across five matching folds, line RMSE is [4.0, 5.0, 4.5, 5.5, 4.0] and tree RMSE is [3.5, 4.8, 5.0, 4.9, 3.8]. Store tree minus line RMSE for each fold in answer. Keep the supplied row order and inspect the named output after running.

Explained solution
fold_errors = pd.DataFrame({'line': [4.0, 5.0, 4.5, 5.5, 4.0], 'tree': [3.5, 4.8, 5.0, 4.9, 3.8]})
answer = fold_errors['tree'] - fold_errors['line']

The tree has smaller RMSE in four of these illustrative folds but larger error in one. Pairing the evidence preserves the common evaluation population. The mean difference alone is not a universal ranking or proof of statistical significance.

Helpful prior knowledge: Choose the task before the family These links are guidance, not locks.

Sources and API context

Examples run with this Playground’s scikit-learn 1.4.2 / Pyodide 0.26.4 runtime.

Your task · Follow

Across five matching folds, line RMSE is [4.0, 5.0, 4.5, 5.5, 4.0] and tree RMSE is [3.5, 4.8, 5.0, 4.9, 3.8]. Store tree minus line RMSE for each fold in answer.

answer

Ctrl/⌘+Enter: Run · Tab: indent · Esc then Tab: leave editor

Python loads when you run. Code and results stay in this activity only.

Run your code to inspect its output. Check uses that same run.