Repository navigation
Reject classification models in conformal models - #3282
Open
shivenaggarwal wants to merge 2 commits into
Open
shivenaggarwal wants to merge 2 commits into
shivenaggarwal wants to merge 2 commits into
Conversation
Conformal models only support regression models (forecasting continuous numerical values). Classification models such as `SKLearnClassifierModel` were silently accepted, which resulted in meaningless intervals around the predicted class labels. `ConformalModel` now raises a `ValueError` for classification models.
shivenaggarwal
force-pushed
the
fix/conformal-reject-classifiers
branch
from
October 18, 2026 16:21
404012d to
5dde7e6
Compare
3 tasks done
…t-classifiers # Conflicts: # CHANGELOG.md
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist before merging this PR:
Related to #2884.
Summary
ConformalNaiveModelandConformalQRModelcurrently accept fitted classification models (SKLearnClassifierModel,LightGBMClassifierModel,XGBClassifierModel,CatBoostClassifierModel) without any error. The resulting intervals are built around class labels, so they don't mean anything. For example, with class labels{0, 1, 2}the calibrated upper quantiles come out as3or4.As mentioned in #2884:
This adds a check to
ConformalModel.__init__, so it applies to both conformal models:Classifiers are detected with the existing
_model_typeproperty (ModelType.FORECASTING_CLASSIFIER, set by_ClassifierMixin). Models that don't define it (torch, foundation, ensemble models, ...) are not affected, and the conformal module doesn't need to importsklearn_model.The sentence "Conformal models can be applied to any of Darts' global forecasting model..." in the three conformal docstrings now also states that classification models are not supported.
This PR only adds the guard. It does not implement conformal prediction for classifiers, which is what #2884 requests, so it should not close the issue.
Other Information
test_model_construction_classifier, parametrized over both conformal models. It checks that a fittedSKLearnClassifierModelis rejected with the message above and that a regression model fitted on the same series is still accepted. Onmasterit fails withDID NOT RAISE ValueError.pytest darts/tests/models/forecasting/test_conformal_model.py: 740 passed. The conformal tests in the serialization, probabilistic and historical forecasts modules also pass.CatBoostClassifierModelis rejected andCatBoostModelis still accepted.pre-commit runpasses on the changed files.