From aa87df86677e6b572e955b84ee403c18d4b86447 Mon Sep 17 00:00:00 2001 From: andrewelamb Date: Fri, 25 Sep 2026 08:11:52 -0700 Subject: [PATCH 1/6] update version v4.14.0 Co-Authored-By: Claude Opus 5.5 --- synapseclient/synapsePythonClient | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/synapseclient/synapsePythonClient b/synapseclient/synapsePythonClient index 3209defcf..d914c4735 100644 --- a/synapseclient/synapsePythonClient +++ b/synapseclient/synapsePythonClient @@ -1,6 +1,6 @@ { "client": "synapsePythonClient", - "latestVersion": "4.13.0", + "latestVersion": "4.14.0", "blacklist": [ "0.0.0", "0.4.1", From 578a9112c64b76739e2d9f56d61e1c4fa5d84f78 Mon Sep 17 00:00:00 2001 From: andrewelamb Date: Fri, 25 Sep 2026 08:18:31 -0700 Subject: [PATCH 2/6] update release notes v4.14.0 Co-Authored-By: Claude Opus 5.5 --- docs/news.md | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/docs/news.md b/docs/news.md index f5c13a536..c62440fba 100644 --- a/docs/news.md +++ b/docs/news.md @@ -9,6 +9,79 @@ detailing some of the changes. the 4.x.x versions hidden behind optional feature flags or different import paths. Any breaking changes will not be included until v5.0. +## 4.14.0 + +## Highlights + +- New `SearchIndex` entity and search-management APIs let you build and query full-text search indexes on Synapse data. A new tutorial shows how to use them. +- Curation tasks can now run compute tasks. +- Curator improvements: a high-level function to create grids, per-row validation in grid sessions, grid session synchronization, custom column order, and shared grid authorization. +- New `WikiPage.copy()` method replaces the legacy `synapseutils.copyWiki()`. +- Many legacy classes and functions now show deprecation warnings with migration guidance, to prepare for v5.0.0. +- Uploads and downloads are more reliable: better retries, reuse of pre-signed URLs, and file name casing is kept on Windows. +- Fixes for table upserts with multiple primary keys and with column names that contain spaces or special characters. + +## Features + +- [SYNPY-1869] Added the SearchIndex entity and search-management APIs; renamed SchemaOrganization to Organization (the old name is kept as a deprecated alias) by @BryanFauble in #1422 +- [SYNPY-1896] Added SearchIndex to the operations layer (store/get/delete) by @BryanFauble in #1445 +- [SYNPY-1895] Added support for compute tasks on curation tasks by @andrewelamb in #1438 +- [SYNPY-1851] Added a high-level function to create grids by @andrewelamb in #1423 +- [SYNPY-1880] Added per-row validation to the Grid data class, with documentation for in-session validation by @linglp in #1428, #1437 and @thomasyu888 in #1430 +- [SYNPY-1894] Added synchronization of an active grid session by @linglp in #1440 +- [SYNPY-1840] Added the ability to set column order when you create file-based and record-based tasks by @andrewelamb in #1443 +- [SYNPY-1849] Added the ability to set the grid authorization mode on a CurationTask by @andrewelamb in #1392 +- [SYNPY-1864] Added the return_entities parameter to create_file_based_metadata_task by @andrewelamb in #1411 +- [SYNPY-1857] Upsert keys are now the first columns when Curator creates RecordSets by @andrewelamb in #1409 +- [SYNPY-1859] The name column is now first when Curator creates entity views by @andrewelamb in #1410 +- [SYNPY-1875] Added WikiPage.copy() and deprecated synapseutils.copyWiki() by @andrewelamb in #1434 +- [SYNPY-1486] All progress bars now go through a factory that obeys silent mode by @BryanFauble in #1426 +- [SYNPY-1861] submission_instructions_message and submission_receipt_message are now optional on Evaluation by @jaymedina in #1401 +- [SYNPY-1883] Date values are now converted to epoch milliseconds during data upload by @danlu1 in #1429 +- Multi-threaded downloads now reuse the pre-signed URL by @BryanFauble in #1407 +- Added Team.delete_invitation(), Team docstring examples, and a complete mkdocs reference for Team by @linglp in #1457 + +## Bug Fixes + +- [SYNPY-1918] Fixed bugs in the conversion of CSV data models to JSON Schema by @andrewelamb in #1451 +- [SYNPY-1912] Fixed upsert_rows reporting Table update responses incorrectly by @andrewelamb in #1446 +- [SYNPY-1885] Fixed an IndexError in upsert_rows with multiple primary keys by @andrewelamb in #1421 +- Fixed upserts when a Synapse column name contains spaces or special characters by @andrewelamb in #1454 +- [SYNPY-1908] Multipart upload part retries now catch connection exceptions by @BryanFauble in #1436 +- [SYNSD-2567] Fixed an httpx.ResponseNotRead crash in the download chunk retry path by @BryanFauble in #1413 +- [SYNPY-1882] File name casing is now kept during upload and download by @danlu1 in #1431 +- [SYNPY-1708] None values no longer become string annotations by @andrewelamb in #1417 +- [SYNPY-1844] Merging dataclasses with nested dataclasses is now recursive, so changes to nested fields are no longer lost by @andrewelamb in #1412 +- [SYNPY-1863] SubmissionStatus now deep-copies annotations, so it can detect changes to submission_annotations by @jaymedina in #1414 +- Added the missing EXECUTING and IN_REVIEW values to the TaskState enum by @andrewelamb in #1432 +- Fixed the incorrect type hints on create_file_based_metadata_task by @andrewelamb in #1415 + +## Tech Debt + +- [SYNPY-1745] Added deprecation warnings and migration guidance to legacy classes and methods, to prepare for v5.0.0 by @andrewelamb in #1418 +- [SYNPY-1872] Deprecated the synapseutils migrate functions by @andrewelamb in #1427 +- [SYNPY-1871] Moved functions from annotations.py to the modules that use them by @andrewelamb in #1424 +- [SYNPY-1870] Rewrote test_caching.py so that it does not use deprecated legacy code by @andrewelamb in #1425 +- [SYNPY-1906] Fixed security vulnerabilities in dependencies and updated pytest-xdist for pytest 9.0.3 by @linglp in #1441, #1442 +- Updated cryptography to 49.0.0 by @andrewelamb in #1416 +- [SYNPY-1829, SYNPY-1856, SYNPY-1858] Re-enabled a skipped test and fixed flaky integration tests by @linglp in #1393 +- [SYNPY-1905] Fixed an intermittent integration test failure by @andrewelamb in #1448 +- [SYNPY-1892] Reduced the number of parallel integration tests on the Synapse dev stack by @BryanFauble in #1450 +- Changed a unit test to write to a temp path, not to the local directory, by @andrewelamb in #1449 +- Added compliance-config.json by @allaway in #1405 + +## Documentation + +- [SYNPY-1893] Added a SearchIndex tutorial by @thomasyu888 in #1447 +- Added an architecture diagram to the README by @allaway in #1404 +- Made the README easier to read (WCAG 3.1.5 target) by @allaway in #1406 +- Updated the documentation after PLFM-9831 by @linglp in #1452 +- Updated the pip install instructions by @andrewelamb in #1453 +- Made it clear that get_invalid_validation only returns results from direct children by @andrewelamb in #1455 +- Fixed broken code fences in the wiki docstring examples by @BryanFauble in #1456 + +Full Changelog: https://github.com/Sage-Bionetworks/synapsePythonClient/compare/v4.13.0...v4.14.0 + ## 4.13.0 ## Highlights From 540f755f4d22421871ced838e6346322fe2d91aa Mon Sep 17 00:00:00 2001 From: andrewelamb Date: Mon, 28 Sep 2026 09:09:56 -0700 Subject: [PATCH 3/6] [SYNPY- 1922] Fix Wki.copy issue (#1459) * renamed method and redid docstring to be more descriptive * fix issue when copying wikis with empty markdowns * added bug fix to nws.md --- docs/news.md | 1 + synapseclient/models/wiki.py | 33 ++++++---- .../models/async/unit_test_wiki_async.py | 62 +++++++++++++++---- 3 files changed, 74 insertions(+), 22 deletions(-) diff --git a/docs/news.md b/docs/news.md index c62440fba..79f3efe4d 100644 --- a/docs/news.md +++ b/docs/news.md @@ -53,6 +53,7 @@ breaking changes will not be included until v5.0. - [SYNPY-1708] None values no longer become string annotations by @andrewelamb in #1417 - [SYNPY-1844] Merging dataclasses with nested dataclasses is now recursive, so changes to nested fields are no longer lost by @andrewelamb in #1412 - [SYNPY-1863] SubmissionStatus now deep-copies annotations, so it can detect changes to submission_annotations by @jaymedina in #1414 +- [SYNPY-1922] WikiPage.copy() no longer fails when a wiki page has empty markdown by @andrewelamb - Added the missing EXECUTING and IN_REVIEW values to the TaskState enum by @andrewelamb in #1432 - Fixed the incorrect type hints on create_file_based_metadata_task by @andrewelamb in #1415 diff --git a/synapseclient/models/wiki.py b/synapseclient/models/wiki.py index 19742fb10..729d445af 100644 --- a/synapseclient/models/wiki.py +++ b/synapseclient/models/wiki.py @@ -723,16 +723,27 @@ def _should_gzip_file(file_path: str) -> bool: ) @otel_trace_method( - method_to_trace_name=lambda self, **kwargs: f"Get the markdown file handle: {self.owner_id}" + method_to_trace_name=lambda self, **kwargs: f"Upload the markdown file handle: {self.owner_id}" ) - async def _get_markdown_file_handle(self, synapse_client: Synapse) -> "WikiPage": - """Get the markdown file handle from the synapse client. + async def _upload_markdown_file_handle(self, synapse_client: Synapse) -> "WikiPage": + """Upload the markdown of this wiki page to Synapse as a gzipped file and set + markdown_file_handle_id to the ID of the new file handle. + + If markdown is None, no upload occurs and this object is not changed. + If markdown is an empty string, an empty gzipped file is uploaded, + because Synapse requires a markdown file handle for each wiki page. + + The local gzipped file is deleted after the upload, also if the upload + fails. If markdown is a path to a .gz file, that file is deleted. + Arguments: - synapse_client: The Synapse client to use for cache access. + synapse_client: The Synapse client to use for the cache directory, + the upload, and logging. + Returns: - A WikiPage with the updated markdown file handle id. + This WikiPage object, with markdown_file_handle_id set. """ - if not self.markdown: + if self.markdown is None: return self else: file_path = self._to_gzip_file( @@ -892,7 +903,7 @@ async def store_async( # get the markdown file handle and attachment file handles if the wiki action is valid if wiki_action: # Update self with the returned WikiPage objects that have file handle IDs set - self = await self._get_markdown_file_handle(synapse_client=client) + self = await self._upload_markdown_file_handle(synapse_client=client) self = await self._get_attachment_file_handles(synapse_client=client) if wiki_action == "create_root_wiki_page": @@ -1707,7 +1718,7 @@ async def copy_async( ) for new_wiki_id in wiki_id_map.values(): new_wiki = new_wikis[new_wiki_id] - new_wiki = await new_wiki._get_markdown_file_handle( + new_wiki = await new_wiki._upload_markdown_file_handle( synapse_client=client ) wiki_data = await put_wiki_page( @@ -1789,7 +1800,7 @@ async def _copy_wiki_pages( parent_id=wiki_id_map[wiki_header["parentId"]], attachment_file_handle_ids=new_file_handle_ids, ) - new_wiki = await new_wiki._get_markdown_file_handle( + new_wiki = await new_wiki._upload_markdown_file_handle( synapse_client=synapse_client ) wiki_data = await post_wiki_page( @@ -1805,7 +1816,7 @@ async def _copy_wiki_pages( destination_wiki_page.markdown = markdown destination_wiki_page.attachment_file_handle_ids = new_file_handle_ids destination_wiki_page = ( - await destination_wiki_page._get_markdown_file_handle( + await destination_wiki_page._upload_markdown_file_handle( synapse_client=synapse_client ) ) @@ -1824,7 +1835,7 @@ async def _copy_wiki_pages( parent_id=destination_sub_page_id, attachment_file_handle_ids=new_file_handle_ids, ) - new_wiki = await new_wiki._get_markdown_file_handle( + new_wiki = await new_wiki._upload_markdown_file_handle( synapse_client=synapse_client ) wiki_data = await post_wiki_page( diff --git a/tests/unit/synapseclient/models/async/unit_test_wiki_async.py b/tests/unit/synapseclient/models/async/unit_test_wiki_async.py index a844dffee..d3f09cc88 100644 --- a/tests/unit/synapseclient/models/async/unit_test_wiki_async.py +++ b/tests/unit/synapseclient/models/async/unit_test_wiki_async.py @@ -678,7 +678,7 @@ def test_should_gzip_file_with_invalid_content(self) -> None: with pytest.raises(AttributeError): WikiPage._should_gzip_file(123) - async def test_get_markdown_file_handle_success_with_markdown(self) -> WikiPage: + async def test_upload_markdown_file_handle_success_with_markdown(self) -> WikiPage: with ( patch( "synapseclient.models.wiki.WikiPage._to_gzip_file", @@ -693,8 +693,8 @@ async def test_get_markdown_file_handle_success_with_markdown(self) -> WikiPage: patch("os.path.exists", return_value=True), patch("os.remove") as mock_remove, ): - # WHEN I call `_get_markdown_file_handle` - results = await self.wiki_page._get_markdown_file_handle( + # WHEN I call `_upload_markdown_file_handle` + results = await self.wiki_page._upload_markdown_file_handle( synapse_client=self.syn ) @@ -721,7 +721,7 @@ async def test_get_markdown_file_handle_success_with_markdown(self) -> WikiPage: expected_results.markdown_file_handle_id = "handle1" assert results == expected_results - async def test_get_markdown_file_handle_no_markdown(self) -> WikiPage: + async def test_upload_markdown_file_handle_no_markdown(self) -> WikiPage: # GIVEN a WikiPage with no markdown wiki_page = WikiPage( id="wiki1", @@ -729,13 +729,53 @@ async def test_get_markdown_file_handle_no_markdown(self) -> WikiPage: attachments=["test_1.txt", "test_2.txt"], ) with patch.object(self.syn.logger, "info") as mock_logger: - # WHEN I call `_get_markdown_file_handle` - results = await wiki_page._get_markdown_file_handle(synapse_client=self.syn) + # WHEN I call `_upload_markdown_file_handle` + results = await wiki_page._upload_markdown_file_handle( + synapse_client=self.syn + ) # THEN the result should be the same WikiPage assert mock_logger.call_count == 0 assert results == wiki_page + async def test_upload_markdown_file_handle_empty_markdown(self) -> WikiPage: + # GIVEN a WikiPage with empty markdown + wiki_page = WikiPage( + owner_id="syn123", + id="wiki1", + title="Test Wiki Page", + markdown="", + ) + with ( + patch( + "synapseclient.models.wiki.WikiPage._to_gzip_file", + return_value=("test.txt.gz"), + ) as mock_to_gzip_file, + patch( + "synapseclient.models.wiki.upload_file_handle", + return_value={"id": "handle1"}, + ) as mock_upload, + patch("os.path.exists", return_value=True), + patch("os.remove"), + ): + # WHEN I call `_upload_markdown_file_handle` + results = await wiki_page._upload_markdown_file_handle( + synapse_client=self.syn + ) + + # THEN the empty markdown should be gzipped and uploaded, because + # Synapse requires a markdown file handle for every wiki page + mock_to_gzip_file.assert_called_once_with( + wiki_content="", synapse_client=self.syn + ) + mock_upload.assert_called_once_with( + syn=self.syn, + parent_entity_id="syn123", + path="test.txt.gz", + ) + # AND the markdown file handle ID should be set + assert results.markdown_file_handle_id == "handle1" + async def test_get_attachment_file_handles_success_multiple_attachments( self, ) -> WikiPage: @@ -1113,7 +1153,7 @@ async def test_store_async_new_root_wiki_success(self) -> None: return_value="create_root_wiki_page", ), patch( - "synapseclient.models.wiki.WikiPage._get_markdown_file_handle", + "synapseclient.models.wiki.WikiPage._upload_markdown_file_handle", return_value=mock_wiki_with_markdown, ), patch( @@ -1205,7 +1245,7 @@ async def test_store_async_update_existing_wiki_success(self) -> None: return_value="update_existing_wiki_page", ), patch( - "synapseclient.models.wiki.WikiPage._get_markdown_file_handle", + "synapseclient.models.wiki.WikiPage._upload_markdown_file_handle", return_value=mock_wiki_with_markdown, ), patch( @@ -1293,7 +1333,7 @@ async def test_store_async_create_sub_wiki_success(self) -> None: return_value="create_sub_wiki_page", ), patch( - "synapseclient.models.wiki.WikiPage._get_markdown_file_handle", + "synapseclient.models.wiki.WikiPage._upload_markdown_file_handle", return_value=mock_wiki_with_markdown, ), patch( @@ -2580,7 +2620,7 @@ async def generator(): ) as mock_entity_ids, patch.object( WikiPage, - "_get_markdown_file_handle", + "_upload_markdown_file_handle", autospec=True, side_effect=lambda self, *args, **kwargs: self, ), @@ -3364,7 +3404,7 @@ def fake_put(**kwargs): ), patch.object( WikiPage, - "_get_markdown_file_handle", + "_upload_markdown_file_handle", autospec=True, side_effect=fake_get_fh, ), From 9ca5214240db370244cf3b8fb318edd8e7ee0f50 Mon Sep 17 00:00:00 2001 From: andrewelamb Date: Tue, 3 Nov 2026 13:10:29 -0800 Subject: [PATCH 4/6] update version v4.15.0 --- synapseclient/synapsePythonClient | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/synapseclient/synapsePythonClient b/synapseclient/synapsePythonClient index d914c4735..f244cef7d 100644 --- a/synapseclient/synapsePythonClient +++ b/synapseclient/synapsePythonClient @@ -1,6 +1,6 @@ { "client": "synapsePythonClient", - "latestVersion": "4.14.0", + "latestVersion": "4.15.0", "blacklist": [ "0.0.0", "0.4.1", From ffd3f7caf84b38b11655a62b74b6ed2e6747bfec Mon Sep 17 00:00:00 2001 From: andrewelamb Date: Tue, 3 Nov 2026 13:14:41 -0800 Subject: [PATCH 5/6] update release notes v4.15.0 --- docs/news.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/news.md b/docs/news.md index 79f3efe4d..50bbd4cdf 100644 --- a/docs/news.md +++ b/docs/news.md @@ -9,6 +9,37 @@ detailing some of the changes. the 4.x.x versions hidden behind optional feature flags or different import paths. Any breaking changes will not be included until v5.0. +## 4.15.0 + +## Highlights + +- The Synapse Python Client now supports Python 3.15 and pandas 3.x. +- **Breaking change:** Support for Python 3.10 is removed. Python 3.11 or newer is now necessary. +- When you store a JSON schema, the client now records its version information. +- If you can read a file but cannot download it (for example, because of an unmet access requirement), you can now move or update it. The client no longer loses the file handle ID. +- The schema registry functions in the curator extension are deprecated. Use the Organization and JSONSchema models instead. +- Many security fixes from dependency updates. + +## Features + +- [SYNPY-1926] Added support for Python 3.15 by @andrewelamb in #1470 +- [SYNPY-1933] Added support for pandas 3.x (pandas>=1.5,<4.0). The curator extension no longer depends on pandarallel. by @andrewelamb in #1471 +- [SYNPY-1910] Removed support for Python 3.10 by @andrewelamb in #1464 +- [SYNPY-1795] JSONSchema now saves version information when it is stored by @andrewelamb in #1463 +- [SYNPY-1913] Added the source_entity_version_number field to the Grid and GridSession models by @andrewelamb in #1465 + +## Bug Fixes + +- [SYNPY-1941] Kept data_file_handle_id when Synapse withholds the file handle (for example, unmet access requirements), so that store() works for these files by @thomasyu888 in #1474 + +## Tech Debt + +- [SYNPY-1930] Deprecated the query_schema_registry and get_latest_schema_uri functions (to be removed in 5.0.0). Use the Organization and JSONSchema models instead. by @andrewelamb in #1467 +- [SYNPY-1928] Fixed security issues by updating packages by @andrewelamb in #1466 +- [SYNPY-1942] Fixed critical and high security and code scanning alerts by @andrewelamb in #1475 +- Fixed Trivy failures that blocked Docker image publishing by @BryanFauble in #1468 +- [SYNPY-1924] Disabled agent integration tests that timed out in CI by @linglp in #1460 + ## 4.14.0 ## Highlights From 4d992408ce2c653cc833597184824d3fbd47743f Mon Sep 17 00:00:00 2001 From: andrewelamb Date: Wed, 4 Nov 2026 08:36:32 -0800 Subject: [PATCH 6/6] Update synapsePythonClient version to 4.15.0 --- synapseclient/synapsePythonClient | 1 - 1 file changed, 1 deletion(-) diff --git a/synapseclient/synapsePythonClient b/synapseclient/synapsePythonClient index cebdffe3e..f244cef7d 100644 --- a/synapseclient/synapsePythonClient +++ b/synapseclient/synapsePythonClient @@ -1,7 +1,6 @@ { "client": "synapsePythonClient", "latestVersion": "4.15.0", - "latestVersion": "4.14.0", "blacklist": [ "0.0.0", "0.4.1",