fix(plugins/bitwarden_item): prevent duplicate items from parallel runs - #389
Merged
Merged
Conversation
Parallel Ansible workers each synced bw serve, found no matching item and created one, even for items that had existed for months. Two causes: - No coordination between processes: every worker synced, searched and created on its own, and the last one to write the cache file won. - bw serve answers list/object/items with success and no items for a few seconds after a sync, and 400 for an existing item in the same window (bitwarden/clients#23283, reproduced with bw 2026.8.0 and 2026.9.0). The lookup and the module now hold a mutex (flock on a file next to the cache) while they sync, search and create, and re-read the cache once they hold it. It is not called a lock to keep it apart from locking the vault. An empty item list is asked for again 5 times, 2s apart, and then aborts instead of being taken as an empty vault. The API fallback of get_item_by_id() retries the same way.
module_utils/bitwarden.py imports ansible.utils.display and falls back to
a no-op display on ImportError, for module runs on a managed node without
the controller's ansible package. Under Mitogen the import is served from
the controller and succeeds, but loading ansible.constants then raises
AnsibleError ("Missing base YAML definition file"), because the data files
are not served along. That escaped the except clause, so every run of the
bitwarden_item module failed with MODULE FAILURE, for example the grafana
role's "store tokens to bitwarden" task.
Catch any exception for the fallback. Verified with Mitogen 0.3.53 and
ansible-core 2.16 on Rocky Linux 9: 8 hosts calling the module three times
each run through, and the mutex from the previous commit is released
between the calls.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
bitwarden_itemlookup created duplicates of Bitwarden items when it ran for several hosts in parallel, also of items that had existed for months, and the next run then aborted with "Found multiple Bitwarden items". Two causes: the worker processes did not coordinate, andbw servebriefly reports an empty vault right after a sync (bitwarden/clients#23283, filed with a minimal reproduction). While testing under Mitogen, it turned out that thebitwarden_itemmodule does not run there at all; that is fixed in a second commit.What is in here
Duplicates (
443a6283)flockonlfops_bitwarden_cache.json.mutexnext to the cache file, waiting at most 300 s. Once a process holds it, it re-reads the cache, so it sees what the previous holder synced or created. Only one process per controller syncs, instead of one per host.bw lock,status: locked).list/object/itemsis not taken as an empty vault: it is asked for again 5 times, 2 s apart, and then the run aborts with a pointer to #23283.get_item_by_id()retries its API fallback the same way, sincebw serveanswers 400 for an existing item in the same window.Mitogen (
e9ac6aee)module_utils/bitwarden.pyfell back to a no-op display only onImportError. Under Mitogen the import ofansible.utils.displayis served from the controller, and loadingansible.constantsthen raisesAnsibleError("Missing base YAML definition file"), so every module run failed withMODULE FAILURE, for example thegrafanarole's "store tokens to bitwarden" task. The fallback now catches any exception.Points worth a close look
bw servecannot be told apart from an empty vault in that window, and duplicate secrets are the worse outcome. The error message says to create one item by hand. This is underChangedin the CHANGELOG.flockbelongs to the open file, not to the process, so holding it longer would make the next evaluation in the same worker, or the next module run in a Mitogen interpreter, wait for itself.Testing
bw2026.9.0, 8 hosts in parallel, both versions patched to force a full sync on every sync:mainbw servemitogen_linear): the lookup behaves the same; the module runs three times per host on 8 hosts without errors or mutex timeouts, wheremainfails every call.