Repository navigation
Run the window aggregates of the catalog on both typed SQL surfaces - #169
Merged
estebanzimanyi merged 1 commit intoOct 10, 2026
Merged
Conversation
An aggregate of the catalog's aggregates section taking more than one argument runs from its roles as one taking a value does, the transition taking the arguments of a row in the order the SQL arguments come: wCount, wMin, wMax, wSum and wAvg pass a temporal value and the interval of the window to temporal_wcount_transfn, tint_wmin_transfn and their siblings, which read them in that order. _catalog_aggregates of tools/codegen_jvm.py carries an aggregate whose first argument is a value of a surface type and each further one a value or an interval, and MeosAggregate's transition takes the arguments of a row. The typed Flink surface registers an accumulate per overload over all its arguments, a row with a null argument skipped; the typed Spark surface reads the arguments of a row through the tuple of their encoders and matches them through MeosSqlRuntime.fit, which the scalar builder shares, so INTERVAL '1' DAY reaches an overload taking an interval of days to seconds. Witness: over MobilityDB 754ea69fe6 main's generator registers no window aggregate on either typed surface, the 30 overloads of wCount, wMin, wMax, wSum and wAvg among the 48 it leaves out as taking two arguments. Measured over MobilityDB 754ea69fe6 with the catalog of MEOS-API ff5f838173: both typed surfaces register 20 aggregates of 171 overloads against 15 of 141, the 30 added being the window aggregates over the temporal types, and the 1164 SQL functions and 8956 overloads main's generator registers; beside the aggregate classes, the generated sources differ from main's by MeosSqlRuntime.fit on the Spark surface alone. The 77 overloads without a combine (appendInstant, appendSequence) and the 12 over a value of no surface type are counted as left out. MobilitySpark passes its 39 tests and MobilityFlink its GeneratedSqlSurfaceTest with assertions that wCount, wMax and wAvg over two temporal integers and wSum over a discrete temporal float, with a window of one day, answer what MobilityDB answers, on Spark over values in partitions of their own and on Flink in a streaming job and in a two-phase batch job. Why: a window aggregate answers on both engines as PostgreSQL answers it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An aggregate of the catalog's aggregates section taking more than one argument runs from its
roles as one taking a value does, the transition taking the arguments of a row in the order the
SQL arguments come: wCount, wMin, wMax, wSum and wAvg pass a temporal value and the interval of
the window to temporal_wcount_transfn, tint_wmin_transfn and their siblings, which read them in
that order. _catalog_aggregates of tools/codegen_jvm.py carries an aggregate whose first argument
is a value of a surface type and each further one a value or an interval, and MeosAggregate's
transition takes the arguments of a row. The typed Flink surface registers an accumulate per
overload over all its arguments, a row with a null argument skipped; the typed Spark surface reads
the arguments of a row through the tuple of their encoders and matches them through
MeosSqlRuntime.fit, which the scalar builder shares, so INTERVAL '1' DAY reaches an overload
taking an interval of days to seconds.
Witness: over MobilityDB 754ea69fe6 main's generator registers no window aggregate on either
typed surface, the 30 overloads of wCount, wMin, wMax, wSum and wAvg among the 48 it leaves out
as taking two arguments.
Measured over MobilityDB 754ea69fe6 with the catalog of MEOS-API ff5f838173: both typed surfaces
register 20 aggregates of 171 overloads against 15 of 141, the 30 added being the window
aggregates over the temporal types, and the 1164 SQL functions and 8956 overloads main's
generator registers; beside the aggregate classes, the generated sources differ from main's by
MeosSqlRuntime.fit on the Spark surface alone. The 77 overloads without a combine
(appendInstant, appendSequence) and the 12 over a value of no surface type are counted as left
out. MobilitySpark passes its 39 tests and MobilityFlink its GeneratedSqlSurfaceTest with
assertions that wCount, wMax and wAvg over two temporal integers and wSum over a discrete
temporal float, with a window of one day, answer what MobilityDB answers, on Spark over values in
partitions of their own and on Flink in a streaming job and in a two-phase batch job.
Why: a window aggregate answers on both engines as PostgreSQL answers it.