Skip to content

Improve event timezone handling, avoid misleading Z in JSON-LD and json_api - #1392

Closed
stain wants to merge 10 commits into
ElixirTeSS:masterfrom
stain:event-timezones
Closed

stain wants to merge 10 commits into
ElixirTeSS:masterfrom
stain:event-timezones

Conversation

@stain

@stain stain commented Oct 22, 2026 •

Copy link
Copy Markdown

Summary of changes

  1. ISO 8601 times with the event's offset (Bioschemas startDate/endDate, JSON-API start/end), using the
    offset of the event's time zone on that date, so daylight saving time is right:
    "2026-10-22T09:00:00-03:00". Without a time zone, the time is given without an offset (local time,
    "2026-10-22T09:00:00") rather than falsely as UTC. One method, Event#local_iso8601, for both.
  2. Missing time zones inferred from the location: when an event without a time zone is saved, it is looked up
    offline from its coordinates (timezone-boundary-builder polygons via the
    wheretz gem, so e.g. Evansville, Indiana is correctly Central, not
    Eastern), or else from a country with a single time zone. The result is one of the time zone names TeSS accepts.
    A time zone set by the user or the source is never overwritten. Online events often have no location and keep
    none.
  3. Marked as inferred: new timezone_inferred column, shown as "London (inferred)" on the event page and as
    timezone-inferred in JSON-API (OpenAPI schema updated), until the time zone is changed otherwise.
  4. Event form: suggests the location's time zone when none is selected, and warns when the selected one differs
    (ignoring zones with the same offsets, e.g. Edinburgh/London), with a "Use …" button. Advisory only. It updates as
    the time zone, country, city or postcode change, or the address finder sets a location; when the address fields
    are edited by hand after the address finder, only the country is used.
  5. rake tess:infer_event_timezones sets the time zone of existing events (keeps updated_at, reindexes Solr).
    Documented in docs/docker.md.
  6. iCalendar times stored as local time: the iCalendar, Indico and Gray Scott ingestors stored
    DTSTART;TZID=... times converted to UTC, unlike the event form and the other ingestors, which store local time.
    E.g. a 09:45–12:50 event in Perth (from the ical test fixture) was shown on its page as 01:45–04:50 Perth time,
    and exported to .ics and Google Calendar as 17:45 UTC the day before (8 hours early). All-day dates became
    midnight in the server's time zone. A shared Ingestor#ical_local_time keeps the calendar's local time. The
    ingestor tests now check the actual UTC time (start_utc) and local time with offset, rather than the column.
    Upcoming events are corrected when their source is ingested again (checked locally: the event is updated, and
    then shows 09:45–12:50 and exports DTSTART:20220211T014500Z). Past events stay as they are.
    rake tess:report_ical_event_times lists, per iCalendar/Indico/Gray Scott source, how many upcoming and past
    events with a non-UTC time zone came from it (and, for events scraped before last_scraped_by was recorded, from
    the same content provider).
  7. No dependency on the server's time zone: the new event form defaulted to 09:00–17:00 in the server's system
    time zone (DateTime.now; 00:00–08:00 on a server in Tokyo), and the event calendar used the server's date
    (Date.today). Now Time.zone.now and Date.current. Storage itself was not affected: Rails uses UTC.
  8. Exports of events without a time zone in local time: these were exported to iCal and Google Calendar as UTC,
    so a 09:00–17:00 event in Australia appeared from 20:00 until 04:00 the next day (the end falls on the next day
    for everyone more than 7 hours east of UTC, e.g. Australia, Japan, China; the start on the previous day in e.g.
    Hawaii). They are now exported as floating local times (DTSTART:20261105T090000, Google dates without Z),
    which calendars show at the same clock time and day in their own time zone, assuming people are near the event.
    The Google Calendar link gives the IANA time zone (ctz=Australia/Perth, was Perth).
  9. Default times marked, and no time zone inferred for them: events given only dates get 09:00–17:00
    (set_default_times). These are now marked with 1 microsecond (09:00:00.000001, not shown anywhere; start and
    end both, so durations stay whole), so Event#default_times? can tell them apart from real 09:00–17:00 events.
    No time zone is inferred for them, as a guessed time zone on top of guessed times would pin them to a time and
    day that may not apply, while their floating exports show on the right day. An inferred time zone is cleared if
    the times become default again, e.g. when ingested again from a date-only source. The ingestor tests that check
    default times now expect the marker.
  10. Developer documentation: docs/timezones.md
    describes how event times and time zones are stored, the default time marker, and what ingestors and their tests
    need to do; linked from the README.

Motivation and context

See #1391

TeSS stores an event's start/end as local (wall clock) time in a UTC datetime column, with the zone in the
separate timezone field. The Bioschemas JSON-LD and the JSON-API gave that local time as if it were UTC, e.g. an
event at 09:00–17:00 in Buenos Aires:

.jsonld     "startDate": "2026-10-22 09:00:00 UTC"   (also not ISO 8601)
.json_api   "start": "2026-10-22T09:00:00.000Z"

Both say 09:00 UTC, but the event starts at 12:00 UTC. (The iCal export already converts correctly for such events.)

Some ingestors stored times the other way, as actual UTC: e.g. the iCalendar test fixture
DTSTART;TZID=Australia/Perth:20220211T094500 (09:45–12:50 Perth) was stored as 01:45 with timezone: Perth, so its
page showed 01:45–04:50, and .ics/Google Calendar gave 17:45 UTC the day before (8 hours early).

Events without a time zone were exported to calendars as UTC, so a 09:00–17:00 event in Australia ended at 04:00 the
next day in a calendar there.

Most events have no time zone at all: on tess.elixir-europe.org, 191 of 231 upcoming events (83%), including 117 of
145 onsite events.

Top providers missing timezones: VIB Training & Conferences (95), EBI.

Screenshots

image image
{
  "@context": "http://schema.org",
  "@id": "http://localhost:3000/events/asdf-d5238c25-b9e9-4d39-9e02-c31c62b3ca2e",
  "@type": "Event",
  "dct:conformsTo": {
    "@type": "CreativeWork",
    "@id": "https://bioschemas.org/profiles/Event/0.2-DRAFT-2019_06_14"
  },
  "name": "asdf",
  "url": "http://example.com/?asd",
  "startDate": "2026-10-22T09:00:00+03:00",
  "endDate": "2026-10-22T17:00:00+03:00",
  "location": {
    "@type": "Place",
    "address": {
      "@type": "PostalAddress",
      "streetAddress": "Düşecek, Emet, Kütahya, Aegean Region, Turkey",
      "addressLocality": "Emet",
      "addressCountry": "Turkey",
      "postalCode": "03220"
    },
    "latitude": "39.260377",
    "longitude": "29.422771"
  }
}

..but .json legacy APi is still lying with Z time zone:
image

Questions for maintainers

  1. Legacy JSON API: events.json and events/1.json still give the stored local time as UTC (…T09:00:00.000Z).
    This PR leaves the legacy API unchanged. Should it get the same timestamp fix? It is ready, without adding any
    keys, as one commit on
    event-timezones-legacy-json,
    which can be added to this PR or a follow-up.

  2. Times without an offset in JSON-API: for events without a time zone, JSON-API now gives
    "2026-10-22T09:00:00". That is valid ISO 8601 but not an RFC 3339 date-time, which strict OpenAPI clients
    may expect (the schema currently says format: date for these fields, which neither the old nor the new values
    match). Would you prefer to keep Z there for compatibility, even though it claims UTC?

  3. Dependency size: wheretz bundles the boundary polygons, 151 MB, which adds to the Docker image. Lookups only
    happen when an event without a time zone is saved, and read only the polygon files whose bounding box contains
    the point (0–50 ms). The alternatives are the timezone gem with the GeoNames API (account and network calls), or
    a less exact nearest-city lookup with tzinfo alone (wrong near zone borders).

  4. Default time marker: is marking default times with 1 microsecond acceptable, or would you prefer an explicit
    column (e.g. default_times boolean)? The marker needs no migration and survives ingestion updates, but is
    implicit. Events saved before this change have no marker, so rake tess:infer_event_timezones treats their
    09:00–17:00 as real times; for upcoming events this is undone when their source is next ingested.

Notes

  • API clients that parse the times with their offset now get the correct instant. This includes TeSS's own
    TessEventIngestor, which reads JSON-API and sets the time zone to UTC, so events imported from another TeSS
    instance were previously off by their UTC offset.
  • The existing rake tess:check_timezones task fails on its first event (it calls the private
    WithTimezone#check_timezone); not changed here.

Tests

Lookup (test/unit/timezone_lookup_test.rb), model callbacks and inferred flag, geocoding worker, suggestion
endpoint, "(inferred)" on the event page, Bioschemas and JSON-API times with and without a time zone, including
daylight saving, iCalendar local times and new event defaults whatever the server's time zone, floating exports,
the default time marker, and the lookup's failure paths. Full suite: 1638 tests, 0 failures; all changed lines in
app and lib are covered by tests (the rake tasks, which have no tests in TeSS, were run on development data instead).

🤖 Generated with Claude Code

Checklist

  • I have read and followed the CONTRIBUTING guide.
  • I confirm that I have the authority necessary to make this contribution on behalf of its copyright owner and agree to license it to the TeSS codebase under the BSD license.

stain and others added 4 commits October 22, 2026 23:00
startDate and endDate were serialised by JSON.pretty_generate with
Time#to_s ("2015-11-23 10:16:33 UTC") and presented the event's local
(wall clock) time, which TeSS stores in a UTC column, as UTC. Give them
in ISO 8601 with the offset of the event's time zone on that date, e.g.
"2015-11-23T10:16:33-05:00", or without an offset (local time) when the
time zone is unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most events have no time zone, so their times cannot be placed. When an
event without one is saved, look it up offline from its coordinates
(timezone-boundary-builder polygons, via the wheretz gem) or, failing
that, from a country with a single time zone, as one of the time zone
names TeSS accepts. Online events often have neither and keep none.

The time zone is marked as inferred (new timezone_inferred column),
shown as "(inferred)" on the event page and in the JSON API, until it
is changed, e.g. by the user. The event form suggests the time zone of
the location, and warns if the selected one differs, without
preventing the user from keeping their own choice.

rake tess:infer_event_timezones sets the time zone of existing events.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The JSON and JSON-API representations gave the stored local (wall
clock) time as UTC, e.g. "2026-10-22T09:00:00.000Z" for an event at
09:00 in Buenos Aires. Give them like the Bioschemas dates, with the
offset of the event's time zone ("2026-10-22T09:00:00-03:00"), or
without an offset when the time zone is unknown, using one
Event#local_iso8601 for all three. The JSON event list now includes
timezone and timezone_inferred, like a single event.

Clients that parse the times with their offset now get the right
instant, including TeSS's own TeSS ingestor, which reads JSON-API and
sets the time zone to UTC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the legacy JSON representation (events.json, events/1.json) as it
was, without new keys and with the stored times as before; whether its
timestamps should also carry the event's offset is left as a separate
change (branch event-timezones-legacy-json).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stain
stain marked this pull request as draft October 22, 2026 23:32
stain and others added 6 commits October 22, 2026 23:48
TeSS stores event times as local (wall clock) time, with the zone in
the event's timezone. The iCalendar, Indico and Gray Scott ingestors
stored DTSTART;TZID=... times converted to UTC instead, so e.g. a
09:45 event in Perth was shown as 01:45 Perth time, and exported to
iCal and Google Calendar 8 hours early. All-day dates became midnight
in the server's time zone. Store the local time of the calendar
(Ingestor#ical_local_time). Upcoming events are corrected when their
source is ingested again.

The ingestor tests now check the actual UTC time (start_utc) and the
local time with its offset, rather than the stored column.

rake tess:report_ical_event_times lists, per iCalendar, Indico and
Gray Scott source, how many upcoming and past events are affected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The new event form defaulted to 09:00-17:00 in the server's system
time zone (DateTime.now), giving e.g. 00:00-08:00 on a server in
Tokyo, and the event calendar used the server's date (Date.today).
Use the application time zone (Time.zone.now, Date.current).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Events without a time zone were exported to iCal and Google Calendar as
UTC, so e.g. a 09:00-17:00 event in Australia appeared from 20:00 to
04:00 the next day. Export them as floating local times
(DTSTART:20261105T090000), which calendars show at the same clock time
and day in their own time zone, assuming people are near the event. The
Google Calendar link now gives the IANA time zone (ctz=Australia/Perth
rather than Perth).

Events given only dates get 09:00-17:00 by default. Mark these default
times with 1 microsecond (09:00:00.000001, not shown anywhere) so they
can be told apart from real 09:00-17:00 events (Event#default_times?),
and do not infer a time zone for them: combining the guessed times with
a guessed time zone would pin them to a time and day that may not apply,
while their floating local times show on the right day. An inferred
time zone is cleared if the times become default again, e.g. when the
event is ingested again from a source that only gives dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A short developer note on local (wall clock) storage with a separate
time zone, exports, the 1 microsecond marker on default times,
inferred time zones, and what ingestors and their tests need to do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use the Rails 8.1 migration version like the other migrations, address
rubocop remarks on the changed code (simpler TimezoneLookup.rails_name
and timezone_suggestion, no parallel assignment, zero?, frozen string
literals), and test the lookup failure paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stain

stain commented Oct 23, 2026

Copy link
Copy Markdown
Author

Split into #1393 and #1394.

@stain stain closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant