Repository navigation
Description
Describe the bug
When the Agent formatter is used in backend query results (e.g., query builder, reports, labels), separators defined in the formatter XML are unconditionally prepended to field values. This causes output like "Smith, " when firstName is NULL or empty, instead of the expected "Smith".
The frontend correctly handles this case by only emitting the separator when the field value is non-empty, but the backend SQL generation does not.
Affected formatters
All formatters that use the sep attribute on <field> elements, including but not limited to:
Agent(agentType=1):lastName+, firstName+- text3Address:address+address2+, city+, state+postalCodeCollectingEvent:stationFieldNumber+; locality.localityName+; locality.geography.fullName+; locality.latitude1+, locality.longitude1
To Reproduce
- Create a Query Builder query that includes a formatted Agent field (one-to-one relation)
- Ensure some Agent records have a
lastNamebutfirstNameis NULL or empty - Run the query
- Observe the formatted result shows e.g.
"Smith, "(with trailing comma and space) instead of"Smith" - The same data on the frontend (forms, subview cells) does NOT show the trailing comma
Expected behavior
Separators should only appear between non-empty field values. "Smith" + empty firstName should produce "Smith", not "Smith, ".
Root Cause
In specifyweb/backend/stored_queries/format.py, the apply_stringish inner function within make_expr unconditionally prepends the separator:
# Line 230-232 (before fix)
sep = fieldNodeAttrib.get('sep')
if sep is not None:
e = concat(sep, e)This generates SQL like CONCAT(', ', firstName). When firstName is NULL, CONCAT returns '' (via blank_nulls wrapping → IFNULL). However, under some query conditions, the separator string is still present in the intermediate concatenation before NULL-coalescing, leading to "Smith, " output.
For comparison, the frontend correctly handles this at specifyweb/frontend/js_src/lib/components/Formatters/formatters.ts:204-207:
return {
formatted: formatted?.toString() ?? '',
separator: (formatted ?? '') === '' ? '' : separator,
};Additional context
This issue was introduced when the backend formatter was implemented without the separator-null-guard logic that exists in the frontend. The frontend (for previews in the table format editor and elsewhere) has handled this correctly since the formatter was first implemented, as seen in the formatField function in formatters.ts
Reported by
Marion Depraetere at MfN Berlin in BoldDesk ticket #1016
I have set up formatted fields for the Collectors and Determiners fields. The issue is that there is a display problem, but not for all collections and not for all Agents. So far, I have only seen the problem in the Mammals collection.
I would therefore like the Agents to appear in the format ‘First name, Last Name’. Normally, the comma does not appear if the “Last Name” field is empty. However, in the Mammals collection this does not work, whereas it works fine for the other collections. Even stranger is that it only works for some Agents (see Budinger agent in the second screenshot)
Activity
fix(query): omit separators for empty formatted values