FIX: Warn when setencoding settings cannot be applied (#825)

[Jahnvi Thakkar (jahnvi480)]

Work Item / Issue Reference

AB#48882

---

Summary

Make unsupported setencoding() requests non-silent while preserving the existing UTF-16LE / SQL_C_WCHAR parameter binding.

• Emit a caller-visible UserWarning for valid requests that cannot be applied, including explicit or automatic SQL_CHAR and UTF-16BE. Keep supported defaults warning-free and retain existing validation errors.
• Preserve getencoding() compatibility and leave settings unchanged when warnings are treated as errors.
• Align native execute() and executemany() encoding checks.
• Clarify requested versus effective settings in README/docstrings and document the legacy internal C-type alias without changing its value.
• Replace permissive encoding tests with strict warning and data-preservation assertions across execute, executemany, setinputsizes, and streaming.

This is a warning-based fix for ignored configuration, not an implementation of configurable narrow binding. The public wiki has not been edited and still needs corresponding clarification if this approach is adopted.

Validation

• Reproduced the silent no-op on Windows x64 / Python 3.13.15 against live SQL Server LocalDB, including native binding diagnostics. New warning assertions fail on the baseline.
• Rebuilt the native extension; 408 focused encoding, CP1252-boundary, and execute-parity tests passed.
• Black passed for all 101 Python files.
• Full non-stress suite: 5295 passed, 173 skipped, 42 deselected, 2 failures. Both failures also reproduce with the unchanged HEAD connection implementation: a logging test assumes a password exists on a passwordless LocalDB connection, and a long-path test exceeds Windows MAX_PATH.
• Linux/macOS validation remains for CI.

[github-actions]

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task	Before	After	Paired change	Result
Connection opening	10.214 ms	10.248 ms	+0.3%	no signal
SELECT queries	1.088 ms	1.149 ms	+8.9%	no signal
Row insertion	34.925 ms	36.140 ms	+3.5%	no signal
Executemany inserts	155.390 ms	154.516 ms	-0.5%	no signal
Fetch-all queries	119.349 ms	120.463 ms	+0.3%	no signal
Row-by-row fetching	14.263 ms	14.436 ms	+1.1%	no signal
Batched row fetching	116.944 ms	117.815 ms	+0.8%	no signal
Transaction commit and rollback	117.066 ms	114.976 ms	-2.3%	no signal
Arrow row fetching	93.414 ms	95.093 ms	+2.0%	no signal
100,000-row insertion	447.968 ms	454.933 ms	+1.4%	no signal
Row fetching in batches of 100	133.320 ms	125.836 ms	+1.7%	no signal
Row fetching in batches of 10,000	136.925 ms	127.063 ms	-4.9%	no signal
Repeated positional queries	34.243 ms	34.065 ms	-2.9%	no signal
Repeated named-parameter queries	36.625 ms	35.978 ms	-0.7%	no signal
Legacy 100,000-row insertion	345.235 ms	350.876 ms	+2.4%	no signal
Insertion with explicit input sizes	480.925 ms	494.188 ms	+3.5%	no signal
Joined aggregation queries	179.403 ms	178.489 ms	-1.1%	no signal
Large joined-result fetching	172.771 ms	176.936 ms	+2.4%	no signal
1.2-million-row fetching	3456.377 ms	3475.779 ms	-0.0%	no signal
Common table expression queries	5.360 ms	5.393 ms	+0.3%	no signal
256 KiB VARCHAR(MAX) / fetchall()	1.266 ms	1.249 ms	+3.9%	no signal

Unix / SQL Server 2025

Database task	Before	After	Paired change	Result
Connection opening	97.039 ms	97.347 ms	+0.3%	no signal
SELECT queries	1.082 ms	1.066 ms	-2.3%	no signal
Row insertion	34.826 ms	34.764 ms	-0.2%	no signal
Executemany inserts	151.814 ms	154.088 ms	+0.8%	no signal
Fetch-all queries	123.438 ms	122.373 ms	+0.1%	no signal
Row-by-row fetching	14.519 ms	14.875 ms	+0.6%	no signal
Batched row fetching	117.936 ms	122.660 ms	+1.0%	no signal
Transaction commit and rollback	119.464 ms	115.578 ms	-3.3%	no signal
Arrow row fetching	94.970 ms	94.693 ms	-0.9%	no signal
100,000-row insertion	456.724 ms	442.955 ms	-2.8%	no signal
Row fetching in batches of 100	125.644 ms	125.692 ms	+0.0%	no signal
Row fetching in batches of 10,000	140.555 ms	142.038 ms	+10.9%	no signal
Repeated positional queries	33.925 ms	33.827 ms	-0.2%	no signal
Repeated named-parameter queries	37.101 ms	36.422 ms	-1.8%	no signal
Legacy 100,000-row insertion	356.388 ms	353.856 ms	-0.9%	no signal
Insertion with explicit input sizes	514.568 ms	493.852 ms	-0.2%	no signal
Joined aggregation queries	161.074 ms	160.684 ms	-0.8%	no signal
Large joined-result fetching	190.096 ms	194.885 ms	+3.9%	no signal
1.2-million-row fetching	3510.944 ms	3483.308 ms	+0.4%	no signal
Common table expression queries	5.240 ms	5.236 ms	+0.2%	no signal
256 KiB VARCHAR(MAX) / fetchall()	1.472 ms	1.504 ms	+1.5%	no signal

Build and measurement details

ADO build 179719

PR head: 8ed67c558b1f76e1b7ad0133c2c29c2d747618b8
Base: c5831908977f0b8b355fda6f71d86629855d46aa
Measured merge: 57970e0ec5979e84a6b729dd4f0c6740ac714d7d

• Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4295.3; 5 paired comparisons and 1 warmup.
• Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5005.3; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation consistently preserves wide-character binding and provides comprehensive regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Adds explicit warnings when unsupported setencoding() requests cannot affect UTF-16LE wide-character binding.

Changes:

• Warns while preserving requested settings and existing validation.
• Aligns native execute() and executemany() encoding gates.
• Documents and tests effective binding behavior.

File	Description
connection.py	Adds warnings and clarifies encoding semantics.
constants.py	Documents the legacy C-type alias.
ddbc_bindings.cpp	Aligns native encoding handling.
test_013_encoding_decoding.py	Adds strict warning and round-trip coverage.
README.md	Documents requested versus effective encoding.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 81%

🎯 Overall Coverage 85%

📈 Total Lines Covered: 9441 out of 11090 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/connection.py (100%)
• mssql_python/pybind/ddbc_bindings.cpp (77.8%): Missing lines 2140-2141

Summary

• Total: 11 lines
• Missing: 2 lines
• Coverage: 81%

mssql_python/pybind/ddbc_bindings.cpp

Lines 2136-2145

  2136                            (SQLPOINTER)SQL_CONCUR_READ_ONLY, 0);
  2137     }
  2138 
  2139     // This codec only applies to parameters already typed as real SQL_C_CHAR (1).
! 2140     // Public text parameter detection uses SQL_C_WCHAR (-8), including the
! 2141     // Python layer's legacy SQL_C_CHAR alias. setencoding() does not change
  2142     // paramCType and warns when the requested settings cannot be applied.
  2143     std::string charEncoding = "utf-8";
  2144     if (encoding_settings.contains("ctype") && encoding_settings.contains("encoding")) {
  2145         int ctype = encoding_settings["ctype"].cast<int>();

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%
mssql_python.cursor.py: 92.5%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Copilot review overview

🟡 Changes recommended

The executemany() streaming path still ignores the selected encoding and parameter C type.

Review effort: Balanced
Findings: 1 · 2

Open (3)

• Apply encoding conversion to executemany DAE streaming parameters · New
• Clarify that validation errors precede warnings for invalid requests · New
• Document validation errors for invalid codecs and combinations · New

[Copilot]

Copilot review overview

🔵 Needs a closer look

Valid UTF-16LE codec aliases can incorrectly select SQL_CHAR and emit an unsupported-setting warning.

Review effort: Balanced
Findings: None

Resolved since last review (3)

• Apply encoding conversion to executemany DAE streaming parameters
• Document validation errors for invalid codecs and combinations
• Clarify that validation errors precede warnings for invalid requests

Previously missed (1)

In code that hasn't changed since last review

Canonicalize codec aliases before ctype validation

mssql_python/​connection.py:1293

A valid alias for the supported default can still trigger this warning. For example, Python accepts UTF-16-LE, but casefold() produces utf-16-le, which is not in UTF16_ENCODINGS; the preceding auto-selection therefore chooses SQL_CHAR, and this condition reports an unsupported request even though the codec is UTF-16LE. Canonicalize Python codec aliases before ctype selection/validation and add a warning-free alias case.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Native ODBC encoding behavior still requires the pending Linux and macOS CI validation.

Review effort: Balanced
Findings: None