PHOENIX-7996 Enable server-side maintenance of immutable indexes by default

[lokiore]

What changes were proposed in this pull request?

This flips the client default DEFAULT_SERVER_SIDE_IMMUTABLE_INDEXES_ENABLED from false to true. With the flag enabled, immutable, global, non-transactional secondary indexes are maintained server-side by IndexRegionObserver (added in PHOENIX-7426) rather than by the client. The client no longer generates index mutations for these tables; it ships the serialized IndexMaintainer and lets the region server build the index updates exactly once.

To keep the flip safe, immutable data tables that declare a ROW_TIMESTAMP column continue to be maintained client-side regardless of the flag. The decision is centralized in a new helper, IndexUtil.isServerSideImmutableIndexMaintenanceEnabled(...), and applied at every data-table gate that reads the flag:

• IndexUtil.getClientMaintainedIndexes
• IndexMaintainer.maintainedLocalOrGlobalIndexesWithoutMatchingStorageScheme (the INDEX_UUID gate)
• MutationState.filterIndexCheckerMutations
• DeleteCompiler.isMaintainedOnClient (signature extended to take the data table so the guard resolves ROW_TIMESTAMP against the data table, not a projected or index table)
• IndexMetaDataCacheClient.setMetaDataOnMutations (the send-metadata gate)

Routing all gates through the same helper keeps the client and server in agreement on which side maintains a given table; a disagreement would cause a ROW_TIMESTAMP table to be maintained on both sides.

Enabling server-side maintenance by default also surfaced two paths that must be corrected so the server maintains every affected index correctly:

• Partial upserts on immutable global indexes could lose or misbuild an index column. IndexRegionObserver skips the current-row read-back for immutable batches. A partial upsert that omits an indexed, covered, or index-WHERE column then builds the index entry from the partial mutation alone. For a covered index this drops the column, so it is lost from the index while it survives in the data table (silent divergence, the same class of hazard as PHOENIX-7961). For an uncovered index the omitted indexed column is materialized as null, so a spurious null-keyed index entry is written while the data row keeps the earlier value: a point lookup self-heals through the index read-repair join-back, but a server-side aggregate over the index rows returns the wrong result. The read-back gate now forces a read-back for immutable batches carrying a covered or uncovered global index when any enabled mutation omits one of that index's on-disk columns. Columns are resolved to their on-disk qualifiers per the data table storage scheme, so full-row upserts and single-cell tables keep the fast path with no read-back. The gate trades a bounded in-memory column scan (only on the immutable global-index write path) for avoiding a disk read-back on the common full-row-upsert case.
• Uncovered global immutable indexes with a matching storage scheme were maintained by nobody. The immutable server-serialize filter in IndexMaintainer matched IndexType.GLOBAL only. An uncovered global immutable index whose storage scheme matched the data table was therefore serialized by neither the client (which returns no client-maintained indexes for these tables when the flag is on) nor the server (whose filter dropped it), so it went unmaintained. The filter now matches IndexUtil.isGlobalIndex, which covers both GLOBAL and UNCOVERED_GLOBAL.

Why are the changes needed?

PHOENIX-7426 added server-side maintenance of immutable-table indexes behind a flag that defaulted to off. Enabling it by default removes per-batch client index-mutation generation for immutable tables and lets IndexRegionObserver (the default index path for years, DEFAULT_INDEX_REGION_OBSERVER_ENABLED=true) build the index updates, reducing client-side work and mutation payload.

The ROW_TIMESTAMP carve-out is required for correctness. Server-side maintenance re-stamps every data cell — including the ROW_TIMESTAMP column — with the server batch timestamp, overwriting the user-supplied ROW_TIMESTAMP value. ROW_TIMESTAMP range predicates push an HBase scan TimeRange, so re-stamped cells fall outside it and rows are silently dropped on range reads (SCN-based visibility breaks for the same reason). This is the same hazard behind CANNOT_CREATE_INDEX_ON_MUTABLE_TABLE_WITH_ROWTIMESTAMP, which already forbids the mutable variant; the immutable variant was safe only because it was client-maintained.

The two additional fixes are required because the default flip is the point at which the server becomes responsible for these indexes: without the read-back, a partial immutable upsert would serve stale/missing covered values or a wrong-keyed uncovered entry from the index; without the broadened serialize filter, an uncovered global immutable index would receive no updates at all.

Does this PR introduce any user-facing change?

Yes. Immutable, global, non-transactional secondary indexes are now maintained server-side by default (previously client-side unless the flag was set explicitly).

Upgrade notes:

• Upgrade region servers before clients. Server-side maintenance rides the generic IndexRegionObserver path, which is enabled by default; on a default cluster a new client talking to an already-upgraded (or default-configured) server loses no index data.
• Silent index-data loss is only reachable on the deprecated legacy-Indexer configuration (phoenix.index.region.observer.enabled=false). Clusters still on that configuration must move off it before adopting this default.
• Immutable tables with a ROW_TIMESTAMP column are unaffected — they remain client-maintained.
• The previous behavior can be restored by setting phoenix.server.side.immutable.indexes.enabled=false.

How was this patch tested?

• RowTimestampIT — regression lock; asserts raw-scan cell timestamps equal the user ROW_TIMESTAMP on both data and index tables for the immutable case. Passes with the guard.
• GlobalIndexCheckerIT#testPartialRowUpdateForImmutable — regression lock for the read-back fix on a ONE_CELL_PER_COLUMN immutable covered index; a full upsert is followed by a partial upsert that omits a covered column, and the test asserts the index read still returns the earlier value. Fails without the read-back fix (expected:<abcd> but was:<null>) and passes with it.
• GlobalIndexCheckerIT#testPartialRowUpdateForImmutableUncovered — regression lock for the read-back fix on a ONE_CELL_PER_COLUMN immutable uncovered index; a full upsert is followed by a partial upsert that omits the indexed column, and the test asserts (via the index path) that the row still resolves under its original key, that a count over the index is exactly 1, and that IS NULL returns nothing. Fails without the read-back fix (the count returns 0) and passes with it.
• UncoveredGlobalImmutableNonTxIndexIT and UncoveredGlobalImmutableNonTxIndex2IT — pass with the broadened serialize filter (uncovered global immutable indexes are now maintained). GlobalImmutableNonTxIndexIT (covered global) stays green, confirming no regression to the covered path.
• ServerSideImmutableIndexIT and ClientSideImmutableIndexIT — pass.
• PhoenixMetricsIT, PhoenixLoggingMetricsIT, and PhoenixTableLevelMetricsIT#testMetricsWithIndexUsage — pass; the flag is pinned off at the driver level for the assertions that account for client-side index mutations.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)