Conversation
- add StoreEntityParam.verify_write, default True - query the edited titles after the upload, one API request per 50 titles - an absent page goes to StoreEntityResult.failed as PageNotCreatedError instead of being reported in StoreEntityResult.pages - the query bypasses the page cache, which would answer pre-write - correct the comment claiming created_pages only records successes Closes #175
Contributor
Release previewMerging this PR would release v2.3.4 (current: Changelog preview (truncated)## v2.3.4 (2026-09-18)
### Bug Fixes
- **core**: Confirm a missing page before reporting a failed store
([`5756436`](https://github.com/OpenSemanticLab/osw-python/commit/5756436e08ba95be3e177c6a78c424bfdba756c8))
- **core**: Verify that store_entity actually created the page
([`f84a529`](https://github.com/OpenSemanticLab/osw-python/commit/f84a5297c605fc458783e5497edeb1f97113053b))
### Chores
- Relicense to Apache-2.0 ([#173](https://github.com/OpenSemanticLab/osw-python/pull/173),
[`848615c`](https://github.com/OpenSemanticLab/osw-python/commit/848615cbd1e06401922eda88231a51028440a242))
Preview via python-semantic-release and conventional commits. |
- query a title reported as missing a second time, 5 s later, so a read from a lagging database replica cannot fail a store that applied - a failing verification query no longer discards the results of the whole store_entity call: the pages stay reported and the error is logged - add tests for both paths
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #175.
Changes
StoreEntityParam.verify_write, defaultTrue.store_entitycollects the titles it sent an edit for and queries theirexistence after the upload. A page that is absent is removed from
StoreEntityResult.pagesand added toStoreEntityResult.failedas a newOSW.PageNotCreatedError, which makesstore_entityraiseStoreEntityPartialErrorthrough the existing path.OSW._query_missing_page_titles(titles): oneaction=query&prop=inforequest per 50 titles, mapping normalized titles back to the requested form.
OSW._get_missing_page_titles(titles, confirm_delay_s=5): queries asecond time, 5 seconds later, for the titles the first query reported as
missing.
StoreEntityResult.pages.store_entity_only recordscreated_pageson successnow says what actually happens.
Rationale
created_pages[page.title] = pageruns for every page that reaches the laststatement of
store_entity_. That only means no exception was raised. Itdoes not mean the page exists.
page.changedis no help: it wasTruein the reported failing case aswell.
WtSite.get_page, because the page cache would answer with the state frombefore the write.
pages rather than one per page.
replica that does not have the write yet would otherwise fail a store that
did apply. A page still absent seconds later is not explained by that lag.
letting it propagate would discard the results the call had already
collected. Reporting the pages and logging the error keeps more information
than either alternative.
failedandStoreEntityPartialErrormeans callers that alreadyhandle partial failures need no change, and
result.storedstays truthful.offlineisTrueand for pages the overwritepolicy kept unedited, since no write was sent for those.
Behaviour change
StoreEntityPartialErrorinstead of returning normally. Callers that want the previous behaviour can
pass
verify_write=False.store_entitycall that edits at least one page makes oneadditional API request per 50 edited pages, and waits 5 seconds plus one
further request if a page is reported as missing.
Verification
tests/test_store_entity_verify.pycover the absent page, themixed batch, both skip conditions, the confirmation query, a page that
appears only on the second query, a failing query, the batching by 50 and the
title normalization.
tests/test_store_entity_failure.pyandtests/test_overwrite_policy.pyconstruct anOSWwithout a real site, sothey now stub
_get_missing_page_titlesto report every page as existing.Their assertions are unchanged.
ceil(edited_pages / 50)per call by construction.This change makes the library report the outcome correctly, it does not fix
the write.