[xubenqiang]

Problem

is_document_indexed (workers/executor/executors/index.py:125) caught every exception from the "already indexed?" query, logged "proceeding to index" and returned False. A failed check therefore indexed a second full copy of the document next to the first one — and, because reindex=False never deletes, the run still reported success. sdk1's Index.index (unstract/sdk1/src/unstract/sdk1/index.py:313) had the same handler.

A query failure is not an answer to "is this indexed?": query_index raises VectorDBError for the same query.

Change

• New is_vector_store_not_found() in unstract/sdk1/src/unstract/sdk1/exceptions.py walks an exception (and its __cause__ / __context__ chain) looking for an HTTP 404, checking status_code, then response.status_code.
• Both handlers keep the 404 as "nothing indexed yet" — llama-index creates the Qdrant collection on the first write, so the first index into a new collection legitimately answers 404 — and re-raise everything else, so the run fails instead of duplicating nodes.

The log line is now No index yet for <instance>: <error>, proceeding to index at INFO for the 404 case, since that is the expected first-index path.

Known gap, matching the report: a 404 for some other reason (e.g. the bundled-server mismatch in #2014) still proceeds to index. Closing that needs adapter-specific code (collection_exists), which seemed a separate call.

Tests

unstract/sdk1/tests/test_vector_store_not_found.py covers the classification: status_code=404, response.status_code=404, a 404 wrapped as __cause__, a 500, a timeout, a plain ValueError, and a cyclic cause chain (must terminate, must not count as missing).

python -m pytest tests/test_vector_store_not_found.py -q
# 7 passed

Red check with the helper forced to the previous swallow-everything behaviour:

# 4 failed, 3 passed
FAILED test_other_status_code_is_not_treated_as_missing_index
FAILED test_timeout_is_not_treated_as_missing_index
FAILED test_plain_error_is_not_missing_index
FAILED test_cyclic_cause_chain_terminates

Honest scope note: these tests exercise the classifier, and the two call sites are one-line guards around it. I did not reproduce the duplicate-write scenario end to end (real Qdrant + injected timeout) in this environment; the worker test suite was not run because its dependency set is not installed here.

ruff check and ruff format --check pass on the changed files.

Fixes #2348

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[github-actions]

Thank you for your contribution! Before we can merge it, every committer needs to sign our Contributor License Agreement.

To sign, read the CLA and then post a comment on this pull request with exactly this text:

I have read the CLA Document and I hereby sign the CLA

Still to sign:

• @xubenqiang

_{Already signed? Comment recheck to run the check again.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
3.3% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

[greptile-apps]

via Greptile

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

The SDK and worker now stop indexing when the existence check fails, except when the error chain contains HTTP 404.

• Indexing continues only when a missing index returns HTTP 404.

Acknowledged scope:

• xubenqiang acknowledged that the tests cover only the classifier and that an end-to-end reproduction was not run.
• xubenqiang acknowledged and deferred unrelated 404 errors still allowing indexing because closing that known gap needs adapter-specific checks.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Check whether document is indexed] --> B{Query succeeds?}
    B -->|Yes| C{Document found?}
    C -->|Yes, no reindex| D[Skip indexing]
    C -->|No, or reindex requested| E[Continue indexing]
    B -->|No| F{Error chain contains HTTP 404?}
    F -->|Yes| E
    F -->|No| G[Raise error and fail the run]

Loading

_{Reviews (1) · Last reviewed commit: "[FIX] Annotate the existence-check tests" · Reviewed by Greptile}