Repository navigation
Description
Describe the bug
Follow-up to #411. Two classes of answer differ from the service. One is the refusal a statement with two faults reports. The other is the text for a number set literal with two equal members.
The parser at 0e4b4a8 refuses two forms while it reads the statement. One is a bag with a member twice or with members of two types. The other is a call as an INSERT value. It keeps the other refusals until the end of the statement:
- a sign before a path (
-b) - a negative zero decimal (
-0.0) - an
IStest in aSETvalue - a map literal with a key twice
- a
?inside a literal ([?])
The checks for a call in a SET value and for an empty bag run after the parse (check_statement, crates/engine/src/partiql/mod.rs). So ExtendDB reports the parser's refusal, wherever the statement puts the call or the empty bag. The service reports the call or the empty bag when they come first in written order. It reports them ahead of a sign in either order. In a WHERE condition, the service reports an empty bag written before a number that breaks a number rule. It does the same for an empty bag written before an arithmetic operator or before an unknown function. ExtendDB reports the later refusal.
The service reports the operand type of an arithmetic literal in a SET value (SET a = 'x' + 1) after the other refusals in the SET values. Every number rule, empty bag and call comes first, in any written order. The service refuses SET a = 'x' + 1, b = size(m) for size, SET a = 'x' + 1, b = <<>> for the bag, and SET a = 'x' + 1, b = 1e126 for the number. ExtendDB checks that operand type in the translation of the statement, after check_statement, and gave the service's answer in 18 cases on 2026-11-06 (test_the_operand_type_of_an_arithmetic_literal_comes_after_the_other_set_value_checks in tests/test_partiql_grammar.py). That check stays where it is.
A number set literal with two equal members gets One or more parameter values were invalid: Input collection [...] contains duplicates! under root on both sides. The AST holds a number set as a sorted set of texts, so ExtendDB lists the members in text order. The service lists them in written order, in normalized form. The parser refuses two members written alike (<<1, 1>>, <<+1, 1>>) with the classic text, which ends in contains duplicates. and has no path.
Measured against Amazon DynamoDB in us-east-1 on 2026-11-06. Table: pk S hash key, sk S range key. The item pk = 'h', sk = 's' holds a 1, x 0, m {'k': 1}, l [1, 2] and ss <<'a'>>. Operation: ExecuteStatement for every row. In the tables, U stands for UPDATE "<t>" SET, W for WHERE pk = 'h' AND sk = 's', and INSERT ... for INSERT INTO "<t>" VALUE {'pk': 'i', 'sk': 's', ...} where no item has pk = 'i'. The positions in the texts are for a table name of 15 characters. The ExtendDB answers are from 0e4b4a8 with the SQLite backend, debug build.
The first refusal in written order (21 cases):
| Statement | DynamoDB | ExtendDB at 0e4b4a8 |
|---|---|---|
U a = size(m), w = -0.0 W |
The function is not allowed in an update expression; function: size |
Negative zero decimal is not supported under key root at 1:48:3 |
U a = size(m), w = -b W |
the function text | Incorrect number of operands for operator or function; operator or function: -, number of operands: 1 |
U w = -b, a = size(m) W |
the function text | the operand count text |
U a = upper(m), w = -b W |
Unrecognized function: upper at 1:34:5 |
the operand count text |
U w = -b, a = upper(m) W |
Unrecognized function: upper at 1:42:5 |
the operand count text |
U a = size(m), w = b IS NULL W |
the function text | IS operator is not supported in update expressions |
U a = size(m), w = <<'a','a'>> W |
the function text | One or more parameter values were invalid: Input collection [a, a] contains duplicates. |
U v = {'x': <<>>, 'x': 1} W |
Empty bags are not supported |
Duplicate keys in tuple. Key: x under root at 1:34:1 |
U v = <<>>, w = [?] W, one parameter |
Empty bags are not supported |
Unsupported data type: Parameter under key root[0] at 1:45:1 |
U v = <<>>, w = -b W and U w = -b, v = <<>> W |
Empty bags are not supported |
the operand count text |
U v = <<>>, w = -0.0 W |
Empty bags are not supported |
Negative zero decimal is not supported under key root at 1:45:3 |
INSERT ... 'v': <<>>, 'w': [?], one parameter |
Empty bags are not supported |
Unsupported data type: Parameter under key root[0] at 1:110:1 |
INSERT ... 'v': <<>>, 'w': -0.0 |
Empty bags are not supported |
Negative zero decimal is not supported under key root at 1:104:3 |
INSERT ... 'v': <<>>, 'w': size(m) |
Empty bags are not supported |
Unsupported data type: size under key root at 1:104:4 |
INSERT ... 'v': <<>>, 'v': 1 |
Empty bags are not supported |
Duplicate keys in tuple. Key: v under root at 1:91:3 |
INSERT ... 'v': <<>>, 'w': <<'a','a'>> |
Empty bags are not supported |
One or more parameter values were invalid: Input collection [a, a] contains duplicates. |
INSERT ... 'v': <<>>, 'w': <<'a',1>> |
Empty bags are not supported |
Unsupported data type in Bag. DynamoDB only supports either numbers or strings in bags |
U a = 1 W AND v = <<>> AND x = 1e126 |
Empty bags are not supported |
Number overflow. Attempting to store a number with magnitude larger than supported range under root |
U a = 1 W AND v = <<>> AND a + 1 = 2 |
Empty bags are not supported |
Unsupported operator in Condition Expression. Operator: + |
U a = 1 W AND v = <<>> AND lower(x) = 1 |
Empty bags are not supported |
Unrecognized function: lower at 1:107:5 |
The number set text (7 cases):
| Statement | DynamoDB | ExtendDB at 0e4b4a8 |
|---|---|---|
U v = <<1.50, 1.5>> W |
One or more parameter values were invalid: Input collection [1.50, 1.5] contains duplicates! under root |
the same text with [1.5, 1.50] |
U v = <<-1e2, -100>> W |
the text with [-1E+2, -100] |
the text with [-100, -1E+2] |
U v = <<12e3, 12000>> W |
the text with [1.2E+4, 12000] |
the text with [12000, 1.2E+4] |
U v = <<2, 1, 1.0>> W |
the text with [2, 1, 1.0] |
the text with [1, 1.0, 2] |
U v = <<1, 1>> W |
One or more parameter values were invalid: Input collection [1, 1] contains duplicates! under root |
One or more parameter values were invalid: Input collection [1, 1] contains duplicates. |
INSERT ... 'v': <<1, 1>> |
the text with [1, 1] and under root |
the classic text |
U v = <<+1, 1>> W |
the text with [1, 1] and under root |
the classic text |
Not measured for these cases: BatchExecuteStatement and ExecuteTransaction. For a statement with one fault, the batch gives the statement's own error and the transaction prefixes Validation failed in TransactStatements[i]: on both sides.
To Reproduce
- Start ExtendDB at 0e4b4a8 with the SQLite backend.
- Create the table above and put the item.
- Send the statements in the two tables with ExecuteStatement.
- Compare the answers with the DynamoDB column.
Expected behavior
The DynamoDB column. Among the refusals in the first table, the service reports the first in written order. It reports a call or an empty bag ahead of a sign in either order. It lists the members of a number set in written order, in normalized form, with ! under root at the end.
Actual behavior
The ExtendDB column. ExtendDB reports the parser's refusal, raised during the parse or kept until its end. It lists the members of a number set in text order, and gives two members written alike the classic text.
Environment
- ExtendDB version: the feat(partiql): implement ExecuteStatement, BatchExecuteStatement, ExecuteTransaction #411 branch at 0e4b4a8, SQLite backend, debug build
- Operating system: Linux
- Rust version (if building from source): 1.97.0
- Client SDK/driver and version: boto3
- Deployment method (binary, container, source): source
Proposed change
The checks are in crates/core/src/partiql/values.rs: check_condition_values, check_written_values, number_set and decimal_text. check_statement (crates/engine/src/partiql/mod.rs) runs them after the parse. The parser's walk (crates/core/src/partiql/parser/walk.rs) holds the deferred refusals, and bag_members (crates/core/src/partiql/parser/literals.rs) refuses two members written alike with duplicate_bag_message.
Three options:
- Move the empty bag, number set and call checks into the parser's walk. The walk meets values in written order and holds the deferred refusals. ExtendDB then reports the first refusal in written order. The walk can also keep the written order of the members of a bag. The operand type of an arithmetic literal stays in the translation. Cost: changes in
parser/walk.rs,parser/forms.rsandparser/literals.rs, andvalues.rsshrinks to what the parser cannot see. This option needs the parser free of other work. - Keep the checks after the parse and have the parser expose each deferred refusal with its token position, so that
check_statementreports the earliest. Cost: a smaller parser change, but two walks that must agree on positions. - Keep the behaviour and list the cases in
docs/differences-from-dynamodb.md. Cost: none in code. Each case needs two faults in one statement.
Proposed approach: option 1, in a change of its own after 0e4b4a8 merges, so that no other parser change runs at the same time.
Additional context
#887 covers the parser forms that differ from the service. That work starts with SET a = a + 1 + 1, the one statement among them that the service runs and ExtendDB refuses.
Activity