Retain information without confusing it with a picture

The question

Reduce all 30 Breast measurements with PCA. Retain at least 90% variance and distinguish the reduced representation from a separate two-dimensional view.

data/ml-learning/ML-X19.csv Complete workflow and evidence

Your inputs

Challenge input · ML-X19

569 observations. One tumour sample represented by cell-nucleus image measurements. Load the CSV file yourself.

Download challenge input CSV · Original source dataset · Source and original dictionary

Ready to load: comma-delimited UTF-8, header row, no saved index. No filtering or target construction is required. Fitting input contains measurements only; species/diagnosis reference labels are not supplied or required.

ML-X19 · first 8 prepared rows
`radius_mean`	`texture_mean`	`perimeter_mean`	`area_mean`	`smoothness_mean`	`compactness_mean`	`concavity_mean`	`concave_points_mean`	`symmetry_mean`	`fractal_dimension_mean`	`radius_se`	`texture_se`	`perimeter_se`	`area_se`	`smoothness_se`	`compactness_se`	`concavity_se`	`concave_points_se`	`symmetry_se`	`fractal_dimension_se`	`radius_worst`	`texture_worst`	`perimeter_worst`	`area_worst`	`smoothness_worst`	`compactness_worst`	`concavity_worst`	`concave_points_worst`	`symmetry_worst`	`fractal_dimension_worst`
17.99	10.38	122.8	1001	0.1184	0.2776	0.3001	0.1471	0.2419	0.07871	1.095	0.9053	8.589	153.4	0.006399	0.04904	0.05373	0.01587	0.03003	0.006193	25.38	17.33	184.6	2019	0.1622	0.6656	0.7119	0.2654	0.4601	0.1189
20.57	17.77	132.9	1326	0.08474	0.07864	0.0869	0.07017	0.1812	0.05667	0.5435	0.7339	3.398	74.08	0.005225	0.01308	0.0186	0.0134	0.01389	0.003532	24.99	23.41	158.8	1956	0.1238	0.1866	0.2416	0.186	0.275	0.08902
19.69	21.25	130	1203	0.1096	0.1599	0.1974	0.1279	0.2069	0.05999	0.7456	0.7869	4.585	94.03	0.00615	0.04006	0.03832	0.02058	0.0225	0.004571	23.57	25.53	152.5	1709	0.1444	0.4245	0.4504	0.243	0.3613	0.08758
11.42	20.38	77.58	386.1	0.1425	0.2839	0.2414	0.1052	0.2597	0.09744	0.4956	1.156	3.445	27.23	0.00911	0.07458	0.05661	0.01867	0.05963	0.009208	14.91	26.5	98.87	567.7	0.2098	0.8663	0.6869	0.2575	0.6638	0.173
20.29	14.34	135.1	1297	0.1003	0.1328	0.198	0.1043	0.1809	0.05883	0.7572	0.7813	5.438	94.44	0.01149	0.02461	0.05688	0.01885	0.01756	0.005115	22.54	16.67	152.2	1575	0.1374	0.205	0.4	0.1625	0.2364	0.07678
12.45	15.7	82.57	477.1	0.1278	0.17	0.1578	0.08089	0.2087	0.07613	0.3345	0.8902	2.217	27.19	0.00751	0.03345	0.03672	0.01137	0.02165	0.005082	15.47	23.75	103.4	741.6	0.1791	0.5249	0.5355	0.1741	0.3985	0.1244
18.25	19.98	119.6	1040	0.09463	0.109	0.1127	0.074	0.1794	0.05742	0.4467	0.7732	3.18	53.91	0.004314	0.01382	0.02254	0.01039	0.01369	0.002179	22.88	27.66	153.2	1606	0.1442	0.2576	0.3784	0.1932	0.3063	0.08368
13.71	20.83	90.2	577.9	0.1189	0.1645	0.09366	0.05985	0.2196	0.07451	0.5835	1.377	3.856	50.96	0.008805	0.03029	0.02488	0.01448	0.01486	0.005412	17.06	28.14	110.6	897	0.1654	0.3682	0.2678	0.1556	0.3196	0.1151

Column meanings and units

Computed image features, with mean, standard-error and worst summaries as defined by the source. Diagnosis is a category.

This is a historical teaching classification dataset, not a clinical decision system. Measurements are assumed available before prediction.

Input schema
Column	Stored type
`radius_mean`	float64
`texture_mean`	float64
`perimeter_mean`	float64
`area_mean`	float64
`smoothness_mean`	float64
`compactness_mean`	float64
`concavity_mean`	float64
`concave_points_mean`	float64
`symmetry_mean`	float64
`fractal_dimension_mean`	float64
`radius_se`	float64
`texture_se`	float64
`perimeter_se`	float64
`area_se`	float64
`smoothness_se`	float64
`compactness_se`	float64
`concavity_se`	float64
`concave_points_se`	float64
`symmetry_se`	float64
`fractal_dimension_se`	float64
`radius_worst`	float64
`texture_worst`	float64
`perimeter_worst`	float64
`area_worst`	float64
`smoothness_worst`	float64
`compactness_worst`	float64
`concavity_worst`	float64
`concave_points_worst`	float64
`symmetry_worst`	float64
`fractal_dimension_worst`	float64

The file is available in the Python working folder. Load it yourself.

Your task

Reduce all 30 Breast measurements with PCA.
Keep at least 90% variance and distinguish the reduced representation from a separate two-dimensional view.
Plot the separate two-axis scores with PC1 and PC2 axis labels.

ratioscumulativeretainedreducedweightsscores2

Outputs you create

ratios: Explained variance ratios in descending component order.
cumulative: Cumulative sum of ratios.
retained: Smallest number of components reaching at least 90% variance.
reduced: Scores for the retained component prefix.
weights: Feature-by-two dataframe of PC1/PC2 axis weights, labelled by feature and component.
scores2: Row-by-two component scores, with signs consistent with weights.

Deliverables

These conditions describe a complete workflow. Use the declared Python variables beside the editor; Check inspects evidence from your Run.

Python name key

Teal outlines name table columns. Violet left rules name Python objects. Dotted rules mark syntax, functions and parameters. Use each name exactly as shown.

Measurement population
1. Load all 30 supplied measurements and standardise them without diagnosis labels.
Variance & retention
1. Fit PCA and find the smallest component count keeping at least 90% variance
2. Report ratios and cumulative variance.
Representation & view
1. Return the retained scores, consistent feature weights and a separate two-component score array
2. Plot that separate PC1/PC2 view with labelled axes.
Interpretation
1. Distinguish retained information from the two-axis picture.
2. Explain axis weights without claiming causal effects.

The playground workflow

Use the ML → Workflow discovery route: define the observations and measurements, explore and prepare them, fit the grouping or representation, inspect its evidence and explain what the result can support. There is no prediction target or reserved final test in this workflow.

Choose what to discoverHow can I summarise variation in Breast Cancer Wisconsin (Diagnostic) · All features continuous rows with fewer coordinates from radius_mean, texture_mean, and 28 more selected inputs?
Explore redundancyWhich selected inputs may overlap in Breast Cancer Wisconsin (Diagnostic) · All features continuous before they are replaced by principal-component coordinates?
Scale the inputsWhy should radius_mean, texture_mean, and 28 more selected inputs share a common scale before PCA compares their variation?
Fit PCA and inspect explained varianceHow much variation does each component represent for Breast Cancer Wisconsin (Diagnostic) · All features continuous, and where does adding another component become less useful?
Select componentsHow many coordinates should represent Breast Cancer Wisconsin (Diagnostic) · All features continuous under the chosen variance-retention rule?
Understand the component loadingsWhich original inputs shape the first axes for Breast Cancer Wisconsin (Diagnostic) · All features continuous, and how should their signs be read?
Project the rowsHow many coordinates did you keep for Breast Cancer Wisconsin (Diagnostic) · All features continuous, how much variation do they preserve, and what do the first two axes describe?

Data policy Use only the supplied measurements for fitting. Keep group labels, scores and profiles aligned with the rows they describe.

Help Hints & explained solution

Choose the amount of help you need. Opening one hint does not reveal the others.

Hint 1 — Think

The picture and retained representation may have different dimensions.

Hint 2 — Tools

PCA, explained_variance_ratio_, components_ and transform.

Revisit: PCA checkpoint

Hint 3 — Approach

Choose retention from cumulative variance, label weights/scores and report the two-axis view separately.

Explained solution

This complete script replaces the editor contents. It includes the input setup and the workflow.

import numpy as np
import pandas as pd
df=pd.read_csv('data/ml-learning/ML-X19.csv')
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
import seaborn as sns
X = df[[c for c in df.columns if c.endswith(('_mean','_se','_worst'))]]
scaler = StandardScaler()
scaled = scaler.fit_transform(X)
pca = PCA().fit(scaled)
scores = pca.transform(scaled)
ratios = pca.explained_variance_ratio_
cumulative = np.cumsum(ratios)
retained = int(np.searchsorted(cumulative,0.9)+1)
reduced = scores[:,:retained]
weights = pd.DataFrame(pca.components_[:2].T.copy(),index=X.columns,columns=['PC1','PC2'])
scores2 = scores[:,:2].copy()
fig, ax = plt.subplots(figsize=(6,4))
sns.scatterplot(x=scores2[:,0],y=scores2[:,1],s=12,ax=ax)
ax.set(title='Two-component view of measurements',xlabel='PC1 score',ylabel='PC2 score')
print('Retained dimensions:',retained,'; variance visible in 2D:',ratios[:2].sum())

View input setup in this script

These opening lines are included in the complete script above. They load the challenge input.

import numpy as np
import pandas as pd
df=pd.read_csv('data/ml-learning/ML-X19.csv')

Why these steps exist

The primary input excludes diagnosis and contains all 30 measurements. PCA is fitted on their standardised representation; cumulative variance determines the smallest retained prefix. Feature-labelled weights and row-aligned scores support interpretation, while the separate two-axis view does not claim to show the full retained information.

Another valid approach

Equivalent ordinary Python is welcome. A defensible candidate or grouping need not match the solution’s choice; objective evidence must remain consistent.