Skip to content

Fix RareLabelEncoder for an existing replacement category - #1096

Open
AHMETHAKANBEZIR1 wants to merge 1 commit into
feature-engine:mainfrom
AHMETHAKANBEZIR1:fix/rare-label-existing-category
Open

AHMETHAKANBEZIR1 wants to merge 1 commit into
feature-engine:mainfrom
AHMETHAKANBEZIR1:fix/rare-label-existing-category

Conversation

@AHMETHAKANBEZIR1

Copy link
Copy Markdown

Fixes #1095.

Problem and change

RareLabelEncoder.transform unconditionally calls pandas Categorical.add_categories(replace_with), even when that label is already present. It raises for observed or unused existing replacement categories and when transforming its own categorical output again.

Add the category only when absent. Existing category order and ordered flag are preserved; new labels still follow the original add_categories path. Document that replace_with can already be a category.

Validation

  • On clean main d866312, 18 new cases fail and 6 new-label controls pass. The 24 cases cover string, integer and float replacement labels; ordered/unordered categoricals; absent, unused and observed replacement categories; repeat transformation; category ordering, nondefault index, unrelated columns and input preservation. Expected frames are constructed directly.
  • Python 3.14 / pandas 3.0.5 / NumPy 2.5.1 / sklearn 1.9.0: all 2,141 package tests passed, with 142 existing warnings.
  • Python 3.12 / pandas 2.2.3 / NumPy 1.26.4 / sklearn 1.4.2: all 53 RareLabelEncoder tests passed.
  • Coverage.py: 100% statements and branches of rare_label.py (65 statements, 26 branches). Its configuration emits an existing include-ignored warning because source and include are both set; measured module coverage is complete.
  • Full flake8 --max-line-length=88 feature_engine tests, changed-file Black with Python 3.9 target, isort with Black profile, focused mypy and git diff --check: passed.
  • Full Sphinx 5.3.0 HTML build with -W passed using docutils 0.16 and sphinx-autodoc-typehints 1.21.3. An earlier Sphinx 9.1 build failed with configuration warnings; that trial is not claimed as successful. The final pinned build used Python 3.12 and did not suppress warnings.

The merged #1030 targets narwhals-migration, not main; its pandas categorical path still calls add_categories unconditionally. It was inspected for overlap. This PR is a focused fix on the documented main contribution target; no migration runtime or Polars validation is claimed. Whole-package mypy was not rerun; the focused changed module check passed.

AI disclosure

OpenAI Codex autonomously investigated, reproduced, implemented and validated this contribution at AHMETHAKANBEZIR1's request. No independent human code review is claimed. The commit includes Codex co-authorship.

Co-authored-by: Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RareLabelEncoder fails when replacement label already exists in categorical dtype

1 participant