Skip to content

PartiQL (#411): for a statement with two faults, the first refusal differs from DynamoDB, and so does the number set text #886

Open

Description

@LeeroyHannigan

Describe the bug

Follow-up to #411. Two classes of answer differ from the service. One is the refusal a statement with two faults reports. The other is the text for a number set literal with two equal members.

The parser at 0e4b4a8 refuses two forms while it reads the statement. One is a bag with a member twice or with members of two types. The other is a call as an INSERT value. It keeps the other refusals until the end of the statement:

  • a sign before a path (-b)
  • a negative zero decimal (-0.0)
  • an IS test in a SET value
  • a map literal with a key twice
  • a ? inside a literal ([?])

The checks for a call in a SET value and for an empty bag run after the parse (check_statement, crates/engine/src/partiql/mod.rs). So ExtendDB reports the parser's refusal, wherever the statement puts the call or the empty bag. The service reports the call or the empty bag when they come first in written order. It reports them ahead of a sign in either order. In a WHERE condition, the service reports an empty bag written before a number that breaks a number rule. It does the same for an empty bag written before an arithmetic operator or before an unknown function. ExtendDB reports the later refusal.

The service reports the operand type of an arithmetic literal in a SET value (SET a = 'x' + 1) after the other refusals in the SET values. Every number rule, empty bag and call comes first, in any written order. The service refuses SET a = 'x' + 1, b = size(m) for size, SET a = 'x' + 1, b = <<>> for the bag, and SET a = 'x' + 1, b = 1e126 for the number. ExtendDB checks that operand type in the translation of the statement, after check_statement, and gave the service's answer in 18 cases on 2026-11-06 (test_the_operand_type_of_an_arithmetic_literal_comes_after_the_other_set_value_checks in tests/test_partiql_grammar.py). That check stays where it is.

A number set literal with two equal members gets One or more parameter values were invalid: Input collection [...] contains duplicates! under root on both sides. The AST holds a number set as a sorted set of texts, so ExtendDB lists the members in text order. The service lists them in written order, in normalized form. The parser refuses two members written alike (<<1, 1>>, <<+1, 1>>) with the classic text, which ends in contains duplicates. and has no path.

Measured against Amazon DynamoDB in us-east-1 on 2026-11-06. Table: pk S hash key, sk S range key. The item pk = 'h', sk = 's' holds a 1, x 0, m {'k': 1}, l [1, 2] and ss <<'a'>>. Operation: ExecuteStatement for every row. In the tables, U stands for UPDATE "<t>" SET, W for WHERE pk = 'h' AND sk = 's', and INSERT ... for INSERT INTO "<t>" VALUE {'pk': 'i', 'sk': 's', ...} where no item has pk = 'i'. The positions in the texts are for a table name of 15 characters. The ExtendDB answers are from 0e4b4a8 with the SQLite backend, debug build.

The first refusal in written order (21 cases):

Statement DynamoDB ExtendDB at 0e4b4a8
U a = size(m), w = -0.0 W The function is not allowed in an update expression; function: size Negative zero decimal is not supported under key root at 1:48:3
U a = size(m), w = -b W the function text Incorrect number of operands for operator or function; operator or function: -, number of operands: 1
U w = -b, a = size(m) W the function text the operand count text
U a = upper(m), w = -b W Unrecognized function: upper at 1:34:5 the operand count text
U w = -b, a = upper(m) W Unrecognized function: upper at 1:42:5 the operand count text
U a = size(m), w = b IS NULL W the function text IS operator is not supported in update expressions
U a = size(m), w = <<'a','a'>> W the function text One or more parameter values were invalid: Input collection [a, a] contains duplicates.
U v = {'x': <<>>, 'x': 1} W Empty bags are not supported Duplicate keys in tuple. Key: x under root at 1:34:1
U v = <<>>, w = [?] W, one parameter Empty bags are not supported Unsupported data type: Parameter under key root[0] at 1:45:1
U v = <<>>, w = -b W and U w = -b, v = <<>> W Empty bags are not supported the operand count text
U v = <<>>, w = -0.0 W Empty bags are not supported Negative zero decimal is not supported under key root at 1:45:3
INSERT ... 'v': <<>>, 'w': [?], one parameter Empty bags are not supported Unsupported data type: Parameter under key root[0] at 1:110:1
INSERT ... 'v': <<>>, 'w': -0.0 Empty bags are not supported Negative zero decimal is not supported under key root at 1:104:3
INSERT ... 'v': <<>>, 'w': size(m) Empty bags are not supported Unsupported data type: size under key root at 1:104:4
INSERT ... 'v': <<>>, 'v': 1 Empty bags are not supported Duplicate keys in tuple. Key: v under root at 1:91:3
INSERT ... 'v': <<>>, 'w': <<'a','a'>> Empty bags are not supported One or more parameter values were invalid: Input collection [a, a] contains duplicates.
INSERT ... 'v': <<>>, 'w': <<'a',1>> Empty bags are not supported Unsupported data type in Bag. DynamoDB only supports either numbers or strings in bags
U a = 1 W AND v = <<>> AND x = 1e126 Empty bags are not supported Number overflow. Attempting to store a number with magnitude larger than supported range under root
U a = 1 W AND v = <<>> AND a + 1 = 2 Empty bags are not supported Unsupported operator in Condition Expression. Operator: +
U a = 1 W AND v = <<>> AND lower(x) = 1 Empty bags are not supported Unrecognized function: lower at 1:107:5

The number set text (7 cases):

Statement DynamoDB ExtendDB at 0e4b4a8
U v = <<1.50, 1.5>> W One or more parameter values were invalid: Input collection [1.50, 1.5] contains duplicates! under root the same text with [1.5, 1.50]
U v = <<-1e2, -100>> W the text with [-1E+2, -100] the text with [-100, -1E+2]
U v = <<12e3, 12000>> W the text with [1.2E+4, 12000] the text with [12000, 1.2E+4]
U v = <<2, 1, 1.0>> W the text with [2, 1, 1.0] the text with [1, 1.0, 2]
U v = <<1, 1>> W One or more parameter values were invalid: Input collection [1, 1] contains duplicates! under root One or more parameter values were invalid: Input collection [1, 1] contains duplicates.
INSERT ... 'v': <<1, 1>> the text with [1, 1] and under root the classic text
U v = <<+1, 1>> W the text with [1, 1] and under root the classic text

Not measured for these cases: BatchExecuteStatement and ExecuteTransaction. For a statement with one fault, the batch gives the statement's own error and the transaction prefixes Validation failed in TransactStatements[i]: on both sides.

To Reproduce

  1. Start ExtendDB at 0e4b4a8 with the SQLite backend.
  2. Create the table above and put the item.
  3. Send the statements in the two tables with ExecuteStatement.
  4. Compare the answers with the DynamoDB column.

Expected behavior

The DynamoDB column. Among the refusals in the first table, the service reports the first in written order. It reports a call or an empty bag ahead of a sign in either order. It lists the members of a number set in written order, in normalized form, with ! under root at the end.

Actual behavior

The ExtendDB column. ExtendDB reports the parser's refusal, raised during the parse or kept until its end. It lists the members of a number set in text order, and gives two members written alike the classic text.

Environment

Proposed change

The checks are in crates/core/src/partiql/values.rs: check_condition_values, check_written_values, number_set and decimal_text. check_statement (crates/engine/src/partiql/mod.rs) runs them after the parse. The parser's walk (crates/core/src/partiql/parser/walk.rs) holds the deferred refusals, and bag_members (crates/core/src/partiql/parser/literals.rs) refuses two members written alike with duplicate_bag_message.

Three options:

  1. Move the empty bag, number set and call checks into the parser's walk. The walk meets values in written order and holds the deferred refusals. ExtendDB then reports the first refusal in written order. The walk can also keep the written order of the members of a bag. The operand type of an arithmetic literal stays in the translation. Cost: changes in parser/walk.rs, parser/forms.rs and parser/literals.rs, and values.rs shrinks to what the parser cannot see. This option needs the parser free of other work.
  2. Keep the checks after the parse and have the parser expose each deferred refusal with its token position, so that check_statement reports the earliest. Cost: a smaller parser change, but two walks that must agree on positions.
  3. Keep the behaviour and list the cases in docs/differences-from-dynamodb.md. Cost: none in code. Each case needs two faults in one statement.

Proposed approach: option 1, in a change of its own after 0e4b4a8 merges, so that no other parser change runs at the same time.

Additional context

#887 covers the parser forms that differ from the service. That work starts with SET a = a + 1 + 1, the one statement among them that the service runs and ExtendDB refuses.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions