Repository navigation
Description
Describe the bug
Follow-up to #411. A PartiQL statement can carry a parameter of the wrong JSON type, such as {"BOOL": "x"}, {"SS": [1]}, {"BS": [1]} or a B that is not base64. The service and ExtendDB answer it with a 400 SerializationException. The whole request fails on both sides, in ExecuteStatement, BatchExecuteStatement and ExecuteTransaction, and in a batch no statement runs. The texts differ. The service has a text for each case. ExtendDB answers Start of structure or map found where not expected: and a detail that names the member and the JSON type it needs.
Measured against Amazon DynamoDB in us-east-1 on 2026-11-06. Table: pk S hash key, sk S range key. The measurement sent each value as the only parameter of UPDATE "<t>" SET v = ? WHERE pk = 'k' AND sk = 'b'. It sent it in ExecuteStatement, and in a BatchExecuteStatement with a second statement INSERT INTO "<t>" VALUE {'pk': 'two', 'sk': 'b'}. In ExecuteTransaction it sent it as the only member and as the second member after a valid UPDATE. On each side the three operations give one answer. The ExtendDB texts are the ones check_parameter_types builds at 0e4b4a8, measured on the build of the commit that added it, with the SQLite backend, debug build. In the table, Start of ...: stands for Start of structure or map found where not expected:.
| Parameter | DynamoDB | ExtendDB at 0e4b4a8 |
|---|---|---|
{"BOOL": "x"} |
Unexpected token received from parser |
Start of ...: BOOL value must be a boolean |
{"BOOL": 1} |
NUMBER_VALUE cannot be converted to Boolean |
Start of ...: BOOL value must be a boolean |
{"NULL": "x"} |
Unexpected token received from parser |
Start of ...: NULL value must be a boolean |
{"S": 1} and {"N": 1} |
NUMBER_VALUE cannot be converted to String |
Start of ...: S value must be a string and Start of ...: N value must be a string |
{"SS": [1]} and {"NS": [1]} |
NUMBER_VALUE cannot be converted to String |
Start of ...: SS member must be a string and Start of ...: NS member must be a string |
{"SS": [true]} |
TRUE_VALUE cannot be converted to String |
Start of ...: SS member must be a string |
{"SS": [{"S": "a"}]} |
Start of structure or map found where not expected |
Start of ...: SS member must be a string |
{"SS": "a"} and {"L": "x"} |
Unexpected field type |
Start of ...: SS value must be an array and Start of ...: L value must be an array |
{"M": []} and {"S": []} |
an empty message | Start of ...: M value must be an object and Start of ...: S value must be a string |
{"BS": [1]} |
only base-64-encoded strings are convertible to bytes |
Start of ...: BS member must be a base64 string |
{"B": "%%%"} |
Base64 encoded length is expected a multiple of 4 bytes but found: 3 |
Start of ...: invalid base64: Invalid symbol 37, offset 0. |
{"B": "AQ="} |
Base64 encoded length is expected a multiple of 4 bytes but found: 3 |
Start of ...: invalid base64: Invalid padding |
"a" |
Unexpected value type in payload |
Start of ...: invalid type: string "a", expected a DynamoDB AttributeValue map with exactly one type descriptor |
A value inside {"M": {"x": ...}} or {"L": [...]} gets the answer it gets alone, on both sides. {"BOOL": "x"} under 40 levels of L gets the BOOL answer.
Both sides answer the wrong JSON type ahead of every other check of the request. The measured checks were:
- a parameter refused for its content (
{"SS": []},{"N": "abc"}, a value nested past the limit), written before or after it Limit0, a statement over 8,192 characters, a blank statement and a syntax error- 26 statements in a batch and 101 members in a transaction
Not measured: a classic request with the same value (PutItem with {"v": {"BOOL": "x"}} in Item, or UpdateItem with it in ExpressionAttributeValues).
To Reproduce
- Start ExtendDB at 0e4b4a8 with the SQLite backend.
- Create the table above.
- Send the ExecuteStatement request with
Parameters[{"BOOL": "x"}]. - Read the error text.
Expected behavior
SerializationException with the service's text for the value: Unexpected token received from parser for {"BOOL": "x"}, NUMBER_VALUE cannot be converted to String for {"SS": [1]}, and the DynamoDB column for the others.
Actual behavior
SerializationException with the text Start of structure or map found where not expected: BOOL value must be a boolean for {"BOOL": "x"}, and the ExtendDB column for the others.
Environment
- ExtendDB version: the feat(partiql): implement ExecuteStatement, BatchExecuteStatement, ExecuteTransaction #411 branch at 0e4b4a8, SQLite backend, debug build
- Operating system: Linux
- Rust version (if building from source): 1.97.0
- Client SDK/driver and version: boto3
- Deployment method (binary, container, source): source
Proposed change
check_parameter_types (crates/core/src/partiql/params.rs) walks the JSON of each parameter and returns a detail for the first payload of the wrong JSON type. wrong_payload gives <tag> value must be a string, a base64 string, a boolean, an object or an array. A set member gives <tag> member must be a string or BS member must be a base64 string. check_base64 gives invalid base64: and the decoder's text. A parameter that is not an object gives the attribute value deserializer's text. check_request_parameter_types (crates/engine/src/partiql/mod.rs) wraps the detail as a SerializationException with the prefix Start of structure or map found where not expected: . ExecuteStatement, batch::run (crates/engine/src/partiql/batch.rs) and transaction::run (crates/engine/src/partiql/transaction.rs) call it over every statement's Parameters before their request checks.
The prefix is the one the classic handlers get from deserialize_error (crates/engine/src/lib.rs) around the text of the attribute value deserializer (from_descriptor, crates/core/src/types/attribute_value.rs). PutItem (crates/engine/src/put_item.rs) formats the same prefix itself.
The change is to give check_parameter_types the service's texts, one for each kind of fault in the table:
NUMBER_VALUE cannot be converted to String,TRUE_VALUE cannot be converted to StringandNUMBER_VALUE cannot be converted to Booleanfor the JSON type found and the type the member takesUnexpected token received from parserfor a string whereBOOLorNULLtakes a booleanUnexpected field typefor a string where the member takes an array or an object- an empty message for an array where the member takes an object or a string
only base-64-encoded strings are convertible to bytesfor aBorBSmember that is not a string- the base64 length text for a string that does not decode
Start of structure or map found where not expectedfor an object as a set memberUnexpected value type in payloadfor a parameter that is not an object
Before a change, measure a classic request with the same values against the service. If the service answers the classic request with the same texts, they belong in deserialize_error and the attribute value deserializer too. Both paths then get them. If the classic texts differ, the texts stay in check_parameter_types.
Additional context
#629 and #630 cover an unknown type descriptor and a repeated type descriptor in the attribute value deserializer, which the classic requests use. For a PartiQL parameter at 0e4b4a8, check_parameter_values (params.rs) answers those two as the service does. It ignores a member the model does not name. It gives a value with no typed member, or with two, the service's text.
Activity