Skip to content

Japanese TN fix and improvement (#444) - #488

Open
folivoramanh wants to merge 2 commits into
mainfrom
staging/ja_tn_v1
Open

folivoramanh wants to merge 2 commits into
mainfrom
staging/ja_tn_v1

Conversation

@folivoramanh

Copy link
Copy Markdown
Collaborator
  • Japanese TN fix and improvement

  • [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

  • update ja tn coding convention

  • [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

  • update tsv formating

  • modify based on feedback


What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Before your PR is "Ready for review"

Pre checks:

  • Have you signed your commits? Use git commit -s to sign.
  • Do all unittests finish successfully before sending PR?
    1. pytest or (if your machine does not have GPU) pytest --cpu from the root folder (given you marked your test cases accordingly @pytest.mark.run_only_on('CPU')).
    2. Sparrowhawk tests bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...
  • If you are adding a new feature: Have you added test cases for both pytest and Sparrowhawk here.
  • Have you added __init__.py for every folder and subfolder, including data folder which has .TSV files?
  • Have you followed codeQL results and removed unused variables and imports (report is at the bottom of the PR in github review box) ?
  • Have you added the correct license header Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. to all newly added Python files?
  • If you copied nemo_text_processing/text_normalization/en/graph_utils.py your header's second line should be Copyright 2015 and onwards Google, Inc.. See an example here.
  • Remove import guards (try import: ... except: ...) if not already done.
  • If you added a new language or a new feature please update the NeMo documentation (lives in different repo).
  • Have you added your language support to tools/text_processing_deployment/pynini_export.py.

PR Type:

  • New Feature
  • Bugfix
  • Documentation
  • Test

If you haven't finished some of the above items you can still open "Draft" PR.

folivoramanh and others added 2 commits September 12, 2026 09:48
* Japanese TN fix and improvement

Signed-off-by: Mai Anh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update ja tn coding convention

Signed-off-by: Mai Anh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update tsv formating

Signed-off-by: Mai Anh <palasek182@gmail.com>

* modify based on feedback

Signed-off-by: Mai Anh <palasek182@gmail.com>

---------

Signed-off-by: Mai Anh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Mai Anh <palasek182@gmail.com>
Comment thread Jenkinsfile
Comment thread Jenkinsfile
JA_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/07-17-26-0'
HI_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/06-04-26-5'
KO_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/06-04-25-6'
KO_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/07-29-26-1'

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplication KO_TN_CACHE in main too
@mgrafu

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you change it here and keep only latest cache date?

@folivoramanh
folivoramanh requested a review from mgrafu September 12, 2026 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants