Skip to content

Fix automatic MeanEncoder smoothing for constant targets - #1094

Open
AHMETHAKANBEZIR1 wants to merge 1 commit into
feature-engine:mainfrom
AHMETHAKANBEZIR1:fix/mean-encoder-constant-target
Open

AHMETHAKANBEZIR1 wants to merge 1 commit into
feature-engine:mainfrom
AHMETHAKANBEZIR1:fix/mean-encoder-constant-target

Conversation

@AHMETHAKANBEZIR1

Copy link
Copy Markdown

Fixes #1093.

Problem

For a constant target, automatic damping divides zero category variance by zero global variance. This makes every observed category encode to NaN. Every category mean equals the global mean in this case, so the expected encoding is simply that constant.

Change

Use zero damping when the global variance is zero. Nonconstant automatic smoothing, numeric smoothing and unseen-category policies retain their existing paths. The change is three source lines plus regression tests for zero/nonzero targets, object/string/categorical variables, singleton categories and unseen="encode".

Validation

  • New tests on unmodified main: 9 failed, 3 passed (the three unseen="encode" cases are controls).
  • Python 3.14.6 / pandas 3.0.5 / NumPy 2.5.1 / scikit-learn 1.9.0: all 2,129 tests passed; 100% statement and branch coverage for mean_encoding.py (55 statements, 12 branches).
  • Python 3.12.14 / pandas 2.2.3 / NumPy 1.26.4 / scikit-learn 1.4.2: all 38 MeanEncoder tests passed.
  • Full flake8 --max-line-length=88 feature_engine tests, changed-file Black (Python 3.9 target) and isort (Black profile), focused mypy and git diff --check: passed.
  • Full documentation build python -m sphinx -W -b html docs ...: passed.
  • Whole-module mypy reports one existing NumPy timedelta64 overload error in datetime_subtraction.py:356. The clean base and this branch produce identical diagnostics; no type error was introduced in the changed module.

Validated from the source checkout at main d866312 on Windows CPU. Other Python/pandas matrix environments were not run.

#1027 merged into narwhals-migration; code inspection shows its variance division still needs the same guard. This PR targets main as specified in the contribution guide and does not duplicate the migration work.

AI disclosure

OpenAI Codex autonomously reproduced, implemented and tested this contribution at the account owner's request. The reasoning and validation limits are recorded above; no independent human code review is claimed. The commit includes Codex co-authorship.

Ready for review. Thank you.

Co-authored-by: Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MeanEncoder automatic smoothing produces NaN for a constant target

1 participant