Skip to content

PartiQL (#411): a parameter of the wrong JSON type gets other SerializationException texts than DynamoDB gives #884

Open

Description

@LeeroyHannigan

Describe the bug

Follow-up to #411. A PartiQL statement can carry a parameter of the wrong JSON type, such as {"BOOL": "x"}, {"SS": [1]}, {"BS": [1]} or a B that is not base64. The service and ExtendDB answer it with a 400 SerializationException. The whole request fails on both sides, in ExecuteStatement, BatchExecuteStatement and ExecuteTransaction, and in a batch no statement runs. The texts differ. The service has a text for each case. ExtendDB answers Start of structure or map found where not expected: and a detail that names the member and the JSON type it needs.

Measured against Amazon DynamoDB in us-east-1 on 2026-11-06. Table: pk S hash key, sk S range key. The measurement sent each value as the only parameter of UPDATE "<t>" SET v = ? WHERE pk = 'k' AND sk = 'b'. It sent it in ExecuteStatement, and in a BatchExecuteStatement with a second statement INSERT INTO "<t>" VALUE {'pk': 'two', 'sk': 'b'}. In ExecuteTransaction it sent it as the only member and as the second member after a valid UPDATE. On each side the three operations give one answer. The ExtendDB texts are the ones check_parameter_types builds at 0e4b4a8, measured on the build of the commit that added it, with the SQLite backend, debug build. In the table, Start of ...: stands for Start of structure or map found where not expected:.

Parameter DynamoDB ExtendDB at 0e4b4a8
{"BOOL": "x"} Unexpected token received from parser Start of ...: BOOL value must be a boolean
{"BOOL": 1} NUMBER_VALUE cannot be converted to Boolean Start of ...: BOOL value must be a boolean
{"NULL": "x"} Unexpected token received from parser Start of ...: NULL value must be a boolean
{"S": 1} and {"N": 1} NUMBER_VALUE cannot be converted to String Start of ...: S value must be a string and Start of ...: N value must be a string
{"SS": [1]} and {"NS": [1]} NUMBER_VALUE cannot be converted to String Start of ...: SS member must be a string and Start of ...: NS member must be a string
{"SS": [true]} TRUE_VALUE cannot be converted to String Start of ...: SS member must be a string
{"SS": [{"S": "a"}]} Start of structure or map found where not expected Start of ...: SS member must be a string
{"SS": "a"} and {"L": "x"} Unexpected field type Start of ...: SS value must be an array and Start of ...: L value must be an array
{"M": []} and {"S": []} an empty message Start of ...: M value must be an object and Start of ...: S value must be a string
{"BS": [1]} only base-64-encoded strings are convertible to bytes Start of ...: BS member must be a base64 string
{"B": "%%%"} Base64 encoded length is expected a multiple of 4 bytes but found: 3 Start of ...: invalid base64: Invalid symbol 37, offset 0.
{"B": "AQ="} Base64 encoded length is expected a multiple of 4 bytes but found: 3 Start of ...: invalid base64: Invalid padding
"a" Unexpected value type in payload Start of ...: invalid type: string "a", expected a DynamoDB AttributeValue map with exactly one type descriptor

A value inside {"M": {"x": ...}} or {"L": [...]} gets the answer it gets alone, on both sides. {"BOOL": "x"} under 40 levels of L gets the BOOL answer.

Both sides answer the wrong JSON type ahead of every other check of the request. The measured checks were:

  • a parameter refused for its content ({"SS": []}, {"N": "abc"}, a value nested past the limit), written before or after it
  • Limit 0, a statement over 8,192 characters, a blank statement and a syntax error
  • 26 statements in a batch and 101 members in a transaction

Not measured: a classic request with the same value (PutItem with {"v": {"BOOL": "x"}} in Item, or UpdateItem with it in ExpressionAttributeValues).

To Reproduce

  1. Start ExtendDB at 0e4b4a8 with the SQLite backend.
  2. Create the table above.
  3. Send the ExecuteStatement request with Parameters [{"BOOL": "x"}].
  4. Read the error text.

Expected behavior

SerializationException with the service's text for the value: Unexpected token received from parser for {"BOOL": "x"}, NUMBER_VALUE cannot be converted to String for {"SS": [1]}, and the DynamoDB column for the others.

Actual behavior

SerializationException with the text Start of structure or map found where not expected: BOOL value must be a boolean for {"BOOL": "x"}, and the ExtendDB column for the others.

Environment

Proposed change

check_parameter_types (crates/core/src/partiql/params.rs) walks the JSON of each parameter and returns a detail for the first payload of the wrong JSON type. wrong_payload gives <tag> value must be a string, a base64 string, a boolean, an object or an array. A set member gives <tag> member must be a string or BS member must be a base64 string. check_base64 gives invalid base64: and the decoder's text. A parameter that is not an object gives the attribute value deserializer's text. check_request_parameter_types (crates/engine/src/partiql/mod.rs) wraps the detail as a SerializationException with the prefix Start of structure or map found where not expected: . ExecuteStatement, batch::run (crates/engine/src/partiql/batch.rs) and transaction::run (crates/engine/src/partiql/transaction.rs) call it over every statement's Parameters before their request checks.

The prefix is the one the classic handlers get from deserialize_error (crates/engine/src/lib.rs) around the text of the attribute value deserializer (from_descriptor, crates/core/src/types/attribute_value.rs). PutItem (crates/engine/src/put_item.rs) formats the same prefix itself.

The change is to give check_parameter_types the service's texts, one for each kind of fault in the table:

  • NUMBER_VALUE cannot be converted to String, TRUE_VALUE cannot be converted to String and NUMBER_VALUE cannot be converted to Boolean for the JSON type found and the type the member takes
  • Unexpected token received from parser for a string where BOOL or NULL takes a boolean
  • Unexpected field type for a string where the member takes an array or an object
  • an empty message for an array where the member takes an object or a string
  • only base-64-encoded strings are convertible to bytes for a B or BS member that is not a string
  • the base64 length text for a string that does not decode
  • Start of structure or map found where not expected for an object as a set member
  • Unexpected value type in payload for a parameter that is not an object

Before a change, measure a classic request with the same values against the service. If the service answers the classic request with the same texts, they belong in deserialize_error and the attribute value deserializer too. Both paths then get them. If the classic texts differ, the texts stay in check_parameter_types.

Additional context

#629 and #630 cover an unknown type descriptor and a repeated type descriptor in the attribute value deserializer, which the classic requests use. For a PartiQL parameter at 0e4b4a8, check_parameter_values (params.rs) answers those two as the service does. It ignores a member the model does not name. It gives a value with no typed member, or with two, the service's text.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions