You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Multiclass weighted feature-importance baseline is not 0.5 #10
The weighted composite feature-importance plot used a hard-coded balanced accuracy no-skill cutoff of 0.5 for all classification tasks.
This is correct for binary classification, where random/no-skill balanced accuracy is 0.5, but it is not correct for multiclass classification. For multiclass datasets, the expected no-skill balanced accuracy should be 1 / number_of_classes.
Impact:
In multiclass runs, models with balanced accuracy above random chance but below 0.5 were incorrectly given a weight of 0 in the normalized and weighted composite FI plot. This could make useful model-specific feature-importance contributions disappear from the weighted FI visualization.
Example:
For a 3-class dataset, the no-skill baseline should be 0.333, not 0.5. A model with balanced accuracy 0.45 is better than random for that dataset, but the old weighting logic treated it as no-skill and zeroed out its FI contribution.
Fix:
Updated the P8 FI weighting logic so the balanced-accuracy baseline is task-aware:
Binary classification: baseline remains 0.5.
Multiclass classification: baseline is now 1 / number_of_classes.
Regression explained-variance weighting is unchanged.
The number of classes is inferred from the dataset outputs, including ClassCounts.csv and CV datasets.
Result:
Weighted composite FI plots now use the correct no-skill threshold for multiclass classification and no longer incorrectly zero out above-chance multiclass models below 0.5 balanced accuracy.
Activity
raptor419 commentedon Sep 23, 2026
Fixed in PR #11