[bot] Add google-cloud-discoveryengine (Vertex AI Search) instrumentation for answer generation, grounded generation, and rank (1,517,119 weekly downloads)

[starfolkai]

Summary

google-cloud-discoveryengine (Vertex AI Search / Discovery Engine) has zero Braintrust instrumentation, and unlike most remaining provider gaps it is an actively maintained, non-deprecated package with genuine generative execution surfaces.

• Current version 0.20.3 (2026-09-03), regular releases
• ~1.52M weekly downloads (1,517,119; 7,385,931/month)
• No integration directory, wrapper, patcher, nox session, matrix entry, or auto_instrument() support
• grep -rn "discoveryengine\|discovery_engine" py/src/braintrust/ py/pyproject.toml py/noxfile.py → zero matches on main @ 10bc8a17

This is a distinct package from both google-genai (instrumented) and google-cloud-aiplatform (see #549). It is the client library for Vertex AI Search, grounded generation, and the Discovery Engine assistant.

Measured gap

Run with auto_instrument() active — i.e. every integration this SDK ships turned on — against google-cloud-discoveryengine==0.20.3, with only the gRPC transport stubbed:

google-cloud-discoveryengine: 0.20.3
  ConversationalSearchServiceClient.answer_query:             patched=False
  GroundedGenerationServiceClient.generate_grounded_content:  patched=False
  RankServiceClient.rank:                                     patched=False
  SearchServiceClient.search:                                 patched=False

  answer_query returned: 'Braintrust is an eval platform.'
  SPANS PRODUCED: 0

What needs to be instrumented

All of these have both sync and async clients (client.py / async_client.py) in discoveryengine_v1.

Answer generation — highest priority

Client / Method	Description
ConversationalSearchServiceClient.answer_query	RAG answer generation over a datastore — the core generative surface
ConversationalSearchServiceClient.stream_answer_query	Streaming variant
ConversationalSearchServiceClient.converse_conversation	Legacy multi-turn conversational search

Grounded generation

Client / Method	Description
GroundedGenerationServiceClient.generate_grounded_content	Generation grounded against configured retrieval sources
GroundedGenerationServiceClient.stream_generate_grounded_content	Streaming variant
GroundedGenerationServiceClient.check_grounding	Scores how well an answer candidate is supported by supplied facts

Rank

Client / Method	Description
RankServiceClient.rank	Reranks a record set against a query

Precedent for the span shape already exists in this repo: cohere.rerank is instrumented at py/src/braintrust/integrations/cohere/patchers.py:58-61.

Deferred to follow-ups

Client / Method	Why deferred
SearchServiceClient.search / search_lite	Retrieval that conditionally generates — an LLM summary is produced only when ContentSearchSpec.SummarySpec is set. Span typing is ambiguous; see open questions.
AssistantServiceClient.stream_assist	Agentic surface; needs the agentic span contract (parent task + child llm/tool) worked out against what the API actually exposes.

Open questions — these should be settled before implementation

1. There are no token counts anywhere in this API.

Every discoveryengine_v1 type was checked. The only token-named fields are answer_query_token (conversational_search_service.py), assist_token (assistant_service.py), and attribution_token (search_service.py, recommendation_service.py, user_event.py) — all opaque tracking/attribution IDs, not usage counts. Confirmed at runtime:

answer_query_token: 'tok-abc123'   (opaque tracking id, not a token count)
response token-named fields: ['answer_query_token']

So prompt_tokens, completion_tokens, and tokens can never be populated, and cost cannot be attributed from the payload. features/token-and-cost-metrics.md treats these as MUST-when-reported; here they are structurally never reported. Does an llm span with no token metrics clear the bar, or does this need a spec amendment first?

2. Model is available request-side only.

• AnswerQueryRequest.AnswerGenerationSpec.ModelSpec.model_version
• SearchRequest.ContentSearchSpec.SummarySpec.ModelSpec.version
• GenerateGroundedContentRequest.GenerationSpec.model_id

None of these appear on the response, and all are optional — when unset the server picks a default that the client cannot observe. The spec says metadata.model SHOULD come from the resolved response model; the best available here is echoing the request, or omitting it entirely when unset. Which?

3. Span typing per surface. answer_query and generate_grounded_content are clearly llm. rank should follow the cohere.rerank precedent. search is retrieval that sometimes generates and maps cleanly to neither — it may warrant tool, or splitting on whether SummarySpec is present.

4. Partial overlap with existing coverage. Vertex AI Search used as a grounding tool inside a Gemini call — types.Retrieval(vertex_ai_search=types.VertexAISearch(datastore=...)) in google-genai — is already captured within the existing google_genai span. Only direct discoveryengine client calls are dark. Instrumentation must not double-count if both paths are ever active in one process.

Implementation notes

Patch targets are the generated GAPIC client methods (ConversationalSearchServiceClient.answer_query etc.), not an HTTP layer. Both _v1 and the _v1alpha / _v1beta surfaces exist; _v1 is the reasonable scope.

Testing should be viable with normal HTTP cassettes. The GAPIC clients ship a rest.py / rest_base.py transport alongside grpc.py / grpc_asyncio.py, so constructing clients with transport="rest" should let the existing VCR setup record cassettes rather than requiring gRPC-level recording. Worth confirming early — it materially affects the cost of this work.

Repo plumbing per CLAUDE.md: a [tool.braintrust.matrix.google-cloud-discoveryengine] entry with explicit pins, a [tool.braintrust.cassette-dirs] entry, per-version cassettes under integrations/<name>/cassettes/<version>/, a noxfile.py session, registration in integrations/__init__.py, and auto_instrument() wiring.

References

• google-cloud-discoveryengine on PyPI: https://pypi.org/project/google-cloud-discoveryengine/ (v0.20.3, 2026-09-03)
• Vertex AI Search docs: https://cloud.google.com/generative-ai-app-builder/docs/introduction
• answer_query reference: https://cloud.google.com/generative-ai-app-builder/docs/answer
• Grounded generation reference: https://cloud.google.com/generative-ai-app-builder/docs/grounded-gen
• Ranking API reference: https://cloud.google.com/generative-ai-app-builder/docs/ranking
• Related: [bot] Add google-cloud-aiplatform (vertexai.*) instrumentation #549 (google-cloud-aiplatform) — a different Google package, recommended there as won't-do

_{Measurements taken on google-cloud-discoveryengine==0.20.3 against main @ 10bc8a17 with auto_instrument() active; network transport stubbed, so they establish call path and span plumbing, not payload fidelity against a live endpoint. Download figures from pypistats, 2026-09-15.}