Skip to content

Fixes and improvements - #428

Open
Chenglong Wang (Chenglong-MS) wants to merge 38 commits into
mainfrom
dev
Open

Chenglong Wang (Chenglong-MS) wants to merge 38 commits into
mainfrom
dev

Conversation

@Chenglong-MS

Copy link
Copy Markdown
Collaborator

This pull request introduces several improvements and refinements to the Data Formulator agent and context handling, focusing on clearer terminology, enhanced connector and skill management, and dependency updates. The most important changes are summarized below.

Agent and Context Terminology Improvements:

  • All references to "primary table(s)" and "other available tables" have been replaced with "primary analysis inputs" and "other analysis inputs" throughout agent summaries, context, and system prompts for greater clarity and consistency. The [AVAILABLE TABLES] section is now [ANALYSIS INPUT TABLES]. [1] [2] [3] [4] [5] [6]

Connector Availability and Summary Enhancements:

  • The connector summary block now displays all currently loadable connectors, including those not yet cached, and distinguishes between connected/disconnected sources and catalog availability. Disconnected sources are hidden from the agent, and error handling/logging is improved. [1] [2] [3]

Skill Preloading and System Prompt Improvements:

  • The agent now preloads the "data-loading" skill when no analysis input tables are present, exposing its tools and actions immediately. System prompts include all preloaded skills with clear banners. Skill state is properly rehydrated from both loaded and preloaded markers in the trajectory. [1] [2] [3] [4] [5]

Dependency Updates:

  • Several frontend dependencies have been updated for security and compatibility, including vite, dompurify, echarts, js-yaml, and new resolutions for postcss, esbuild, and tmp. [1] [2] [3]

Documentation Cleanup:

  • The obsolete docs/desktop-portable.md file has been removed.
  • The old model evaluation plan in loops/model-evaluation/plan.md has been deleted.

Comment thread py-src/data_formulator/analyst/agent.py Fixed
Add orcarouter to the built-in providers so Data Formulator users can
connect the OrcaRouter AI gateway from the model picker and via
ORCAROUTER_* environment variables, mirroring the existing ollama/openai
wiring. The client routes through LiteLLM's openai provider against the
OrcaRouter base URL, preserving the orcarouter/ model namespace.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…ovider

feat: add OrcaRouter as a built-in model provider
Adds full Hindi translations for all 10 i18n domain files, following
the translation guide's rules (chart type identifiers, encoding channel
keys, and other computation-bound values stay in English). Registers
hi in the locale index/i18next resources and language switcher labels,
and documents AVAILABLE_LANGUAGES=en,zh,hi as opt-in in .env.template.

Relates to #337.
folderPathPlaceholder illustrates filesystem path syntax; translating
the segment words broke the example and was inconsistent with how
every other path/URL placeholder in the app is left untranslated.
Add Hindi (hi) locale for frontend i18n
Comment thread py-src/data_formulator/datalake/workspace.py Fixed
Comment on lines +823 to +826
'<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width">'
'<title>OpenRouter</title></head><body><p>' + message + '</p>'
'<a href="/">Return to Data Formulator</a>'
f'<script nonce="{nonce}">{script}</script></body></html>',
creator_id = key_info.get("creator_user_id")
connection = {
"creator_user_id": creator_id if isinstance(creator_id, str) else None,
"settings_url": "https://openrouter.ai/keys/" + hashlib.sha256(config["api_key"].encode()).hexdigest(),
The wrapper embeds user-supplied code, so on a system whose preferred
encoding is not UTF-8 (cp1252 on Windows) writing it raised
UnicodeEncodeError for any non-ASCII character, and silently transcoded
the rest -- the container reads run.py as UTF-8 either way.

Matches the explicit encoding already used elsewhere in the package,
e.g. reasoning_log.py and analyst/skills/__init__.py.
The constant table listed DEFAULT_ROW_LIMIT_EPHEMERAL in dfSlice.tsx.
That identifier does not exist anywhere in the repository, and there is
no separate ephemeral row limit in the frontend.
…hemeral-row

docs(dev-guides): drop the nonexistent DEFAULT_ROW_LIMIT_EPHEMERAL row
fix(sandbox): write the docker wrapper script as utf-8
Comment thread py-src/data_formulator/workflows/instances.py Fixed
Nick H (nickkhg) and others added 3 commits September 18, 2026 18:44
Two problems stopped the SQL Server connector from working.

Concurrent queries failed. The loader keeps one connection and builds a
new cursor for each query. mssql-python does not support MARS, so the
connection accepts only one active statement. Flask serves preview
requests on parallel threads, so the requests overlapped and failed with
"Connection is busy with results for another command". An RLock around
_read_sql, the only place that makes a cursor, makes the requests take
turns. This follows clickhouse_data_loader and mysql_data_loader.

Blank connection parameters did not get their defaults. The loader read
them with params.get(name, default). A saved connector spec holds an
empty string, not a missing key, so the default did not apply. An empty
port produced "SERVER=host,;" and the driver refused it with error 87.
The loader now reads these parameters with `or`.
…-blank-port

fix(mssql): serialize queries and default blank connection params
def _write_discovery(path: Path, state: dict[str, Any]) -> None:
temporary = path.with_suffix(f".{uuid4().hex}.tmp")
try:
temporary.write_text(json.dumps(state), encoding="utf-8")
temporary = path.with_suffix(f".{uuid4().hex}.tmp")
try:
temporary.write_text(json.dumps(state), encoding="utf-8")
os.replace(temporary, path)
temporary = path.with_suffix(f".{uuid4().hex}.tmp")
try:
temporary.write_text(json.dumps(state), encoding="utf-8")
os.replace(temporary, path)
temporary.write_text(json.dumps(state), encoding="utf-8")
os.replace(temporary, path)
finally:
temporary.unlink(missing_ok=True)
actor, saved['revision'], ','.join(changed_sections))
return json_ok(snapshot())
except ConfigurationConflict as exc:
return {'status': 'error', 'error': {'code': 'INVALID_REQUEST', 'message': str(exc), 'retry': False}}, 409

@staticmethod
def validate_name(name: str) -> None:
if not isinstance(name, str) or not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_-]*(?:\.workflow)?\.ya?ml", name):

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants