Classification evidence retrieval

What the syntax does

f1_score(actual,predicted,labels=class_names,average=None,zero_division=0): Use the declared class list for comparable per-class F1; zero_division=0 assigns zero when a class has no correct positive evidence. idxmax() returns the row label of the largest score.
average: "binary", "micro", "macro", "weighted", None

Question and available inputs

Independent synthetic inspection cases with unequal class frequencies. Both policies are already recorded; this round compares their errors on the same cases rather than fitting a new model.

What is known at prediction time
Column	Meaning and availability
`actual`	Confirmed clear, inspect or urgent outcome.
`predicted`	First recorded policy; it never predicts urgent.
`alternative`	Second recorded policy; it redistributes errors across the classes.

Outputs you create

scores: Requested result described in the task and deliverables.
chosen_policy: Requested result described in the task and deliverables.

ERROR12_REVIEW · first 8 rows
`actual`	`predicted`	`alternative`
clear	clear	clear
clear	clear	clear
clear	clear	clear
clear	clear	clear
clear	clear	clear
clear	clear	clear
clear	clear	clear
clear	clear	clear

Your task

The inspection review compares policies predicted and alternative on the same clear/inspect/urgent cases.
Build scores with those policy names as rows and columns clear, inspect, urgent, macro_f1, accuracy.
The first three columns hold each class F1.
Store the higher-macro-F1 policy name in chosen_policy and explain why equal accuracy does not settle this choice.

scoreschosen_policy

Hints

Think: Equal numbers of correct cases can hide very different errors in the rare urgent class.

Tools: f1_score with labels and average=None or macro; accuracy_score; a labelled dataframe and idxmax.

Approach: Score each policy using the same actual rows; compare per-class contributions before choosing by macro F1.

Explained solution

from sklearn.metrics import f1_score,accuracy_score
labels=['clear','inspect','urgent']
rows=[]
for policy in ['predicted','alternative']:
    class_scores=f1_score(df.actual,df[policy],labels=labels,average=None,zero_division=0)
    rows.append([*class_scores,f1_score(df.actual,df[policy],labels=labels,average='macro',zero_division=0),accuracy_score(df.actual,df[policy])])
scores=pd.DataFrame(rows,index=['predicted','alternative'],columns=labels+['macro_f1','accuracy'])
chosen_policy=scores.macro_f1.idxmax()

Both policies classify 22 of 27 cases correctly. The first never predicts urgent, while the alternative recovers two of its three cases. Their macro F1 values differ because each class receives equal weight.