Skip to content

Run the window aggregates of the catalog on both typed SQL surfaces - #169

Merged
estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:codegen/window-aggregates
Oct 10, 2026
Merged

estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:codegen/window-aggregates

Conversation

@estebanzimanyi

Copy link
Copy Markdown
Member

An aggregate of the catalog's aggregates section taking more than one argument runs from its
roles as one taking a value does, the transition taking the arguments of a row in the order the
SQL arguments come: wCount, wMin, wMax, wSum and wAvg pass a temporal value and the interval of
the window to temporal_wcount_transfn, tint_wmin_transfn and their siblings, which read them in
that order. _catalog_aggregates of tools/codegen_jvm.py carries an aggregate whose first argument
is a value of a surface type and each further one a value or an interval, and MeosAggregate's
transition takes the arguments of a row. The typed Flink surface registers an accumulate per
overload over all its arguments, a row with a null argument skipped; the typed Spark surface reads
the arguments of a row through the tuple of their encoders and matches them through
MeosSqlRuntime.fit, which the scalar builder shares, so INTERVAL '1' DAY reaches an overload
taking an interval of days to seconds.

Witness: over MobilityDB 754ea69fe6 main's generator registers no window aggregate on either
typed surface, the 30 overloads of wCount, wMin, wMax, wSum and wAvg among the 48 it leaves out
as taking two arguments.

Measured over MobilityDB 754ea69fe6 with the catalog of MEOS-API ff5f838173: both typed surfaces
register 20 aggregates of 171 overloads against 15 of 141, the 30 added being the window
aggregates over the temporal types, and the 1164 SQL functions and 8956 overloads main's
generator registers; beside the aggregate classes, the generated sources differ from main's by
MeosSqlRuntime.fit on the Spark surface alone. The 77 overloads without a combine
(appendInstant, appendSequence) and the 12 over a value of no surface type are counted as left
out. MobilitySpark passes its 39 tests and MobilityFlink its GeneratedSqlSurfaceTest with
assertions that wCount, wMax and wAvg over two temporal integers and wSum over a discrete
temporal float, with a window of one day, answer what MobilityDB answers, on Spark over values in
partitions of their own and on Flink in a streaming job and in a two-phase batch job.

Why: a window aggregate answers on both engines as PostgreSQL answers it.

An aggregate of the catalog's aggregates section taking more than one argument runs from its
roles as one taking a value does, the transition taking the arguments of a row in the order the
SQL arguments come: wCount, wMin, wMax, wSum and wAvg pass a temporal value and the interval of
the window to temporal_wcount_transfn, tint_wmin_transfn and their siblings, which read them in
that order. _catalog_aggregates of tools/codegen_jvm.py carries an aggregate whose first argument
is a value of a surface type and each further one a value or an interval, and MeosAggregate's
transition takes the arguments of a row. The typed Flink surface registers an accumulate per
overload over all its arguments, a row with a null argument skipped; the typed Spark surface reads
the arguments of a row through the tuple of their encoders and matches them through
MeosSqlRuntime.fit, which the scalar builder shares, so INTERVAL '1' DAY reaches an overload
taking an interval of days to seconds.

Witness: over MobilityDB 754ea69fe6 main's generator registers no window aggregate on either
typed surface, the 30 overloads of wCount, wMin, wMax, wSum and wAvg among the 48 it leaves out
as taking two arguments.

Measured over MobilityDB 754ea69fe6 with the catalog of MEOS-API ff5f838173: both typed surfaces
register 20 aggregates of 171 overloads against 15 of 141, the 30 added being the window
aggregates over the temporal types, and the 1164 SQL functions and 8956 overloads main's
generator registers; beside the aggregate classes, the generated sources differ from main's by
MeosSqlRuntime.fit on the Spark surface alone. The 77 overloads without a combine
(appendInstant, appendSequence) and the 12 over a value of no surface type are counted as left
out. MobilitySpark passes its 39 tests and MobilityFlink its GeneratedSqlSurfaceTest with
assertions that wCount, wMax and wAvg over two temporal integers and wSum over a discrete
temporal float, with a window of one day, answer what MobilityDB answers, on Spark over values in
partitions of their own and on Flink in a streaming job and in a two-phase batch job.

Why: a window aggregate answers on both engines as PostgreSQL answers it.
@estebanzimanyi
estebanzimanyi merged commit 9628af6 into MobilityDB:main Oct 10, 2026
2 checks passed
@estebanzimanyi
estebanzimanyi deleted the codegen/window-aggregates branch October 10, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant