Bernoulli Naive Bayes for binary characteristics

The question

Classify Candy popular using only the nine presence/absence flags. Inspect smoothing and minority errors.

data/ml-learning/ML-X12.csv Complete workflow and evidence

Your inputs

Challenge input · ML-X12

85 observations. One candy product in the survey. Load the CSV file yourself.

Download challenge input CSV · Original source dataset · Source and original dictionary

Ready to load: comma-delimited UTF-8, header row, no saved index. No filtering or target construction is required. popular is 50% or above when original winpercent >= 50, otherwise below 50%. The target-derived winpercent column has been removed; use the nine flags as X.

ML-X12 · first 8 prepared rows
`chocolate`	`fruity`	`caramel`	`peanutyalmondy`	`nougat`	`crispedricewafer`	`bar`	`pluribus`	`popular`
1	0	1	0	0	1	1	0	50% or above
1	0	0	0	1	0	1	0	50% or above
0	0	0	0	0	0	0	0	below 50%
0	0	0	0	0	0	0	0	below 50%
0	1	0	0	0	0	0	0	50% or above
1	0	0	1	0	0	1	0	50% or above
1	0	1	1	1	0	1	0	50% or above
0	0	0	1	0	0	0	1	below 50%

Column meanings and units

sugarpercent: sugar percentile. pricepercent: price percentile. winpercent: percentage of survey matchups won. Ingredient, bar and multipack fields are 0/1 flags.

Percentile ranks are neither physical sugar percentages nor currency prices. Ratios of percentiles do not measure economic value. Chocolate and fruit flags can overlap; non-chocolate is not synonymous with fruit. The ML class target is defined by winpercent ≥ 50.

Input schema
Column	Stored type
`chocolate`	int64
`fruity`	int64
`caramel`	int64
`peanutyalmondy`	int64
`nougat`	int64
`crispedricewafer`	int64
`hard`	int64
`bar`	int64
`pluribus`	int64
`popular`	object

How we validate this model

Reserve 20% of the declared population for the final test using split seed 42. Keep class proportions with a stratified split.

Use the same five stratified training folds, shuffled with seed 42, for reference comparison, candidates and selection. Use out-of-fold training predictions for diagnosis. Select with macro F1 and compare with a most-frequent-class reference. Accuracy is supplementary. Open final-test evidence after selection and diagnosis.

The file is available in the Python working folder. Load it yourself.

Your task

Classify Candy popular using only the nine presence/absence flags.
Inspect smoothing and minority errors.

cv_resultsreference_resultschosen_namematrixfinal_f1final_accuracyprobabilities

Outputs you create

cv_results: Dataframe indexed by the requested model IDs, with initial_score, selected_score and settings columns.
Scores are observed mean training-fold scores; settings describes the chosen recipe.
reference_results: cross_validate result for the dummy reference; test_score contains five scores.
chosen_name: Model ID nominated from training evidence.
matrix: Confusion matrix of training-only diagnostic predictions.
final_f1: Macro F1 on final-test class predictions.
final_accuracy: Final-test accuracy, supplementary to macro F1.
probabilities: Dataframe of final probabilities, indexed by test rows and labelled by fitted classes.

Deliverables

These conditions describe a complete workflow. Use the declared Python variables beside the editor; Check inspects evidence from your Run.

Python name key

Teal outlines name table columns. Violet left rules name Python objects. Dotted rules mark syntax, functions and parameters. Use each name exactly as shown.

Words in this task

DataFrame: a table with rows and columns
numeric: a number, rather than text
dummy: a simple reference model used for comparison
folds: training-data groups used to compare models on the same held-out rows
out-of-fold: predictions made for training rows by models that did not fit those rows
reserved rows: rows kept aside for one final evaluation
aligned: matched by row or category identity, rather than guessed position
residuals: observed values minus predicted values

Data boundary
1. Load the supplied challenge CSV, define the declared X/y and keep aligned row identities through the split.
2. Keep learned preparation inside each training fit.
Validation & selection
1. Compare the requested initial model families with a dummy reference on matching training folds.
2. Evaluate the declared settings where applicable and choose from training evidence.
Diagnosis
1. Use out-of-fold training predictions to inspect residuals or class errors before opening the final test.
Final evidence
1. Fit the nominated model on training rows and evaluate the reserved rows once.
2. Report saved predictions, the requested metrics and labelled evidence
3. Explain limitations without using the final test for further selection.

The playground workflow

Use the same data boundary as ML → Workflow: frame, split, explore, prepare, validate against a reference, diagnose, then make the final evaluation. Each step answers one question.

Choose what to predictClassify whether a candy reaches at least 50% wins using 9 selected inputs; each row is one candy.
Split data and save the test setWhich rows are available for learning, and which are being saved for the final check?
Explore training dataDoes chocolate show a training pattern related to popularity class?
Define the preparation recipeWhat preparation does this particular model need?
Build the model pipelineHow do class priors and feature evidence combine under the independence assumption?
Validate the initial modelDo validation results stay reasonably similar across folds?
Compare with a simple referenceDo the selected features beat a simple guess on the same training folds?
Tune the modelHow much smoothing should class evidence use?
Inspect validation errorsWhich classes are most often confused in training-only validation?
Final testHow does the selected model's saved final popularity class result compare with the initial-model CV and any selected-setting CV for Candy Popularity · All features binary? Use the training-only diagnostic or optional final confusion view to identify a class pattern, then name one limitation.

Data policy Keep target-derived inputs out of X. Use training-only selection for prediction.

Help Hints & explained solution

Choose the amount of help you need. Opening one hint does not reveal the others.

Hint 1 — Think

Zeros carry evidence too.

Hint 2 — Tools

BernoulliNB, alpha, stratified CV and macro F1.

Revisit: Match Naive Bayes to feature types · Finish once, then report

Hint 3 — Approach

Compare smoothing using binary inputs and training evidence, then inspect errors and finish.

Explained solution

This complete script replaces the editor contents. It includes the input setup and the workflow.

import numpy as np
import pandas as pd
df=pd.read_csv('data/ml-learning/ML-X12.csv')
from sklearn.base import clone
from sklearn.model_selection import train_test_split, StratifiedKFold, cross_validate, cross_val_predict, GridSearchCV
from sklearn.dummy import DummyClassifier
from sklearn.metrics import f1_score, accuracy_score, confusion_matrix

import matplotlib.pyplot as plt
import seaborn as sns
X=df[['chocolate', 'fruity', 'caramel', 'peanutyalmondy', 'nougat', 'crispedricewafer', 'hard', 'bar', 'pluribus']]
y=df['popular']
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=.2,random_state=42,stratify=y)
folds=StratifiedKFold(n_splits=5,shuffle=True,random_state=42)
candidates={}
grids={}
# Prepare inputs and construct the candidate pipeline.
preprocessor = 'passthrough'
from sklearn.pipeline import Pipeline
from sklearn.naive_bayes import BernoulliNB
model = BernoulliNB()
pipeline = Pipeline([('prepare', preprocessor), ('model', model)])
model = pipeline
candidates['naive_bayes']=model
grids['naive_bayes']={'model__alpha': [0.1, 1.0, 5.0]}
# Validate initial candidates on matching training folds.
initial_results={name:cross_validate(candidate,X_train,y_train,cv=folds,scoring='f1_macro') for name,candidate in candidates.items()}
# Compare with a simple reference before tuning.
reference_results=cross_validate(DummyClassifier(strategy='most_frequent'),X_train,y_train,cv=folds,scoring='f1_macro')
rows=[]
selected={}
for name,candidate in candidates.items():
    initial=initial_results[name]
    grid=grids[name]
    if grid:
        search=GridSearchCV(candidate,grid,cv=folds,scoring='f1_macro',n_jobs=1)
        search.fit(X_train,y_train)
        selected[name]=search.best_estimator_
        validation_score=float(search.best_score_)
        settings=search.best_params_
    else:
        selected[name]=candidate
        validation_score=float(np.mean(initial['test_score']))
        settings='Keep defaults: no production search'
    rows.append({'model':name,'initial_score':float(np.mean(initial['test_score'])),'selected_score':validation_score,'settings':str(settings)})
cv_results=pd.DataFrame(rows).set_index('model')
# This solution nominates by mean training-fold score. Other evidence-based choices can be defensible.
chosen_name=cv_results.selected_score.idxmax()
chosen=selected[chosen_name]
oof_predictions=cross_val_predict(chosen,X_train,y_train,cv=folds)
diagnostic_y=y_train
class_labels=sorted(y_train.unique())
matrix=confusion_matrix(diagnostic_y,oof_predictions,labels=class_labels)
fig,ax=plt.subplots(figsize=(6,4))
sns.heatmap(matrix,annot=True,fmt='d',cmap='Blues',ax=ax,xticklabels=class_labels,yticklabels=class_labels)
ax.set(xlabel='Predicted class',ylabel='Actual class',title='Training-only confusion matrix')
fig.tight_layout()
fig.savefig('diagnostic.png',dpi=150,bbox_inches='tight')
final_model=clone(chosen).fit(X_train,y_train)
final_predictions=final_model.predict(X_test)
final_f1=f1_score(y_test,final_predictions,average='macro')
final_accuracy=accuracy_score(y_test,final_predictions)
print('Final macro F1:',final_f1,'; accuracy:',final_accuracy)
probabilities=pd.DataFrame(final_model.predict_proba(X_test),columns=final_model.classes_,index=X_test.index)

View input setup in this script

These opening lines are included in the complete script above. They load the challenge input.

import numpy as np
import pandas as pd
df=pd.read_csv('data/ml-learning/ML-X12.csv')

Why these steps exist

The input contains only the nine binary flags and the declared popular target. Bernoulli evidence includes absence as well as presence; alpha is compared on stratified training folds. Macro F1 and class diagnostics expose behaviour that overall accuracy can hide.

Another valid approach

Equivalent ordinary Python is welcome. A defensible candidate or grouping need not match the solution’s choice; objective evidence must remain consistent.