Skip to content

Google adds AI agent observability in BigQuery and Looker

Telemetry from first-party and custom agents now lands in customer-owned BigQuery tables, with dashboards, token caps and an open-source evaluation tool.

Draftship Editorial4 min read

Google extends agent observability into BigQuery and Looker

Google Cloud AI and ML announced expanded AI agent observability and evaluation capabilities across Google Data Cloud on November 14, 2026, in a post on the Google Cloud Blog. The AI agent observability in BigQuery and Looker now covers first-party data agents in those two products, plus custom third-party agents built with frameworks such as Agent Development Kit (ADK) or LangGraph.

The post argues that deploying an agent is only part of the work. Teams also need to see how it reasons, which data it touches, and what it costs to run.

The three problems it targets

Google names three operational challenges: tracing multi-step reasoning while protecting sensitive data, controlling token consumption and query spend, and evaluating answer accuracy before prompt or schema changes reach users.

The pattern is the same for every agent. Capture telemetry from the agent, land it in BigQuery tables the customer owns, then analyze it in the tools teams already use.

How telemetry gets in

First-party Conversational Analytics agents emit telemetry automatically with no setup, according to the post. Queries routed to BigQuery data agents in Gemmi Enterprise through agent-to-agent (A2A) workflows, now GA, send Cloud Monitoring metrics, Cloud Trace spans and BigQuery conversation logs into the agent's project. Custom agents built on ADK or LangGraph stream the same structured telemetry through single-line SDK instrumentation.

Conversation logging in BigQuery is GA and writes to tables you own, so Conversational Analytics traces, including prompts and responses, can be inspected inside the warehouse. The Conversational Analytics API is also GA, with always-on admin audit logs, optional data-read audit logs in Cloud Audit Logs, support for VPC Service Controls, customer-managed encryption keys (CMEK), and regional and multi-regional data residency.

What stays out of the logs

Admins can turn on Cloud Monitoring (aggregate metrics such as request volume, latency and token usage) and Cloud Trace (step-by-step spans) at project or organisation level, and Cloud Trace observability scopes consolidate telemetry across multiple agent projects. Those metrics and sanitised first-party spans exclude customer core content: user prompts, model responses, internal thoughts and generated SQL.

Deeper auditing is opt-in per agent. When enabled, a Google-managed service agent streams prompts, responses and turn-level telemetry into a customer-owned table called agent_events via the BigQuery Storage Write API, inheriting the dataset's IAM permissions, CMEK and VPC Service Controls. A fail-closed check runs before each turn and skips logging when a knowledge source uses row-level security, column-level policy tags or dynamic data masking, while fail-open execution keeps chat sessions running if logging errors. Custom third-party agents can add the open-source BigQueryAgentAnalyticsPlugin for full tool-invocation execution trees, time-series telemetry and multimodal payloads.

Dashboards, spend caps and evaluation

In BigQuery Studio, agent monitoring dashboards are GA and track usage, active sessions, top knowledge sources and latency for BigQuery Conversational Analytics agents. In Looker, System Activity dashboards for Conversational Analytics are GA in Looker (Google Cloud core) and Looker (original), covering engagement and token usage, while user ratings and query review in the Responses & feedback tab are in preview. Google also points to an open-source Data Studio report template and a Looker Block for Agent Analytics.

Each logged turn records input, output, thought, cached and total token counts alongside total turn latency, and Looker System Activity reports estimated input and output tokens across BI workloads. For spend control, per-query byte scan caps (big_query_max_billed_bytes) use dry-run estimates to block oversized queries before on-demand scan charges apply. Daily input and output token quotas are in preview per project, user or agent in the Google Cloud console and gcloud CLI. The post notes promotional trial pricing is currently available for Conversational Analytics.

On evaluation, the open-source Prism tool tests agents grounded in BigQuery tables or Looker Explores through the Conversational Analytics API, with reusable test suites, regex assertions and multi-region agent discovery deployed behind an access layer such as Cloud IAP. Completed runs can stream into day-partitioned BigQuery tables so benchmark scores sit next to production agent_events logs. Native BigQuery evaluations are in preview via API and a guided Colab notebook, built on the Gemmi Enterprise Agent Platform evaluation service with bring-your-own-service-account support.

Limits and open questions

Several pieces are in preview rather than GA: token quotas, native BigQuery evaluations, and the Looker ratings and query review tab. Google gives no figures for telemetry volume, retention periods or the cost of storing conversation logs, beyond the note about promotional trial pricing. Custom agent coverage is described for agents instrumented through ADK, LangGraph or the plugin, so agents on other frameworks fall outside what the post describes. It is also unclear whether any of this works with warehouses other than BigQuery.

What this means for building apps with AI

For teams running agents over BigQuery or Looker data, the practical change is that production traces, token counts and benchmark results can sit in one place instead of in separate export pipelines. That is a governance story, not a feature you drop into a small app.

If you build on Draftship, none of this is wired in for you. You can still add AI behaviour to an app and keep its data in the built-in database; the guide to adding AI to your app covers the app-side part, and the observability described here stays a Google Cloud concern.

Primary sources

This article was drafted with AI assistance from the primary sources listed below and checked automatically against them. An editor hasn't reviewed it yet.