Threshold choices depend on costs

Understand the idea

Threshold changes trade false positives against false negatives. The default need not match a particular use case. Choose with training-only predictions, and assess uncertainty and consequences.

Choose thresholds using training-only evidence and explicit requirements.Scroll the diagram horizontally if needed.

What the syntax does

method='predict_proba': Requests held-out probability vectors rather than class labels from cross_val_predict.
method: "predict", "predict_proba", "decision_function"
cv=folds: Keeps threshold investigation inside the training population.
cv: 5, folds, list_of_index_pairs

Supplied for this practice

These objects are prepared afresh before your editor runs.

X: Feature dataframe for the declared population, preserving row indices.
X = df[['length', 'width']]
y: Target series, aligned with X.
y = df.label
X_train / X_test / y_train / y_test: Training feature rows from the declared split. Final-test feature rows from the declared split. Training targets, aligned with X_train. Final-test targets, aligned with X_test.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
model: Already fitted on X_train, y_train. Fitted estimator or pipeline for the requested model family. Fitted on X_train, y_train; reuse without refitting.

Outputs you create

answer: The requested result, with the values, labels and row order described by this task.

CLASS180 · first 8 rows
`length`	`width`	`label`
236.033	-0.805568	A
581.297	0.728558	A
-1511.27	-1.00866	A
99.0248	-0.24496	A
-13.0141	-0.660765	A
681.179	0.602475	A
51.1472	0.873157	A
362.131	-0.665605	A

Your task

Compute OOF probabilities for the supplied class model.

answer

Hints

Think: Threshold development needs probabilities for rows not used in their own fit.

Tools: cross_val_predict, StratifiedKFold and method='predict_proba'.

Approach: Generate class-probability vectors for every training row using the specified stratified folds.

Explained solution

from sklearn.model_selection import cross_val_predict,StratifiedKFold
answer=cross_val_predict(model,X_train,y_train,cv=StratifiedKFold(5,shuffle=True,random_state=42),method='predict_proba')

OOF probabilities permit training-only threshold analysis without opening the reserved final population.