Skip to main content
Bridge

November 15, 2026

How to Catch Silent Truncation in Grounded Gemini Calls

Detect and resolve silent response truncation in Google-Search-grounded Gemini queries where reasoning tokens exhaust max output limits. Learn to inspect candidate finish reasons, safely scale token ceilings on retry, and track cumulative API usage before returning partial outputs downstream.

Review and oversightGeminiSearchGroundingTokenManagementPlaybookHow-to
HyunKi Lee··5 min read

Verified on November 15, 2026

To prevent grounded Gemini calls from silently truncating output when internal thinking tokens exhaust token limits, inspect candidate.finish_reason on every call path and retry with doubled token limits up to a hard ceiling. In the Google Gen AI API, max_output_tokens bounds reasoning thoughts and visible text together. Catching MAX_TOKENS ensures partial research is rejected and audit logs record cumulative token consumption across all retry attempts.

Before you start

  • Python 3.10 or later installed with the google-genai SDK (pip install google-genai).
  • A Gemini API key set in your environment as GEMINI_API_KEY.
  • An agent workflow where web-grounded research is passed to downstream reasoning nodes.

How it works

flowchart LR
  A[Execute grounded Gemini call] --> B[Record attempt token costs]
  B --> C{Check candidate finish reason}
  C -- STOP --> D[Return complete text and usage]
  C -- MAX_TOKENS --> E{Attempts under limit and ceiling?}
  E -- Yes --> F[Double max output tokens and retry]
  F --> A
  E -- No --> G[Raise TruncatedOutputError]

Step 1: Define a truncation error class

Create a custom exception that preserves cumulative token telemetry and partial text when retries fail. This prevents broken output from contaminating downstream systems.

class TruncatedOutputError(RuntimeError):
    """Raised when an LLM call hits MAX_TOKENS and exhausts retries."""
    def __init__(self, message: str, cumulative_usage: dict, partial_text: str):
        super().__init__(message)
        self.cumulative_usage = cumulative_usage
        self.partial_text = partial_text

Step 2: Implement the grounded caller with token doubling

Wrap the client call to enforce Google Search grounding, accumulate usage across attempts, inspect candidate.finish_reason, and back off with doubled output limits when hit by MAX_TOKENS.

from google import genai
from google.genai import types

def call_gemini_grounded_safe(
    prompt: str,
    client: genai.Client | None = None,
    model: str = "gemini-2.5-flash",
    initial_max_tokens: int = 4096,
    max_token_ceiling: int = 32768,
    backoff_factor: int = 2,
    max_attempts: int = 3,
) -> dict:
    if client is None:
        client = genai.Client()

    current_max_tokens = initial_max_tokens
    attempt = 0
    cumulative_cost = {
        "prompt_tokens": 0,
        "candidates_tokens": 0,
        "thoughts_tokens": 0,
        "total_tokens": 0,
        "attempts": 0,
    }

    while attempt < max_attempts:
        attempt += 1
        cumulative_cost["attempts"] = attempt

        config = types.GenerateContentConfig(
            max_output_tokens=current_max_tokens,
            tools=[{"google_search": {}}],
            temperature=0.2,
        )

        response = client.models.generate_content(
            model=model,
            contents=prompt,
            config=config,
        )

        if response.usage_metadata:
            meta = response.usage_metadata
            cumulative_cost["prompt_tokens"] += meta.prompt_token_count or 0
            cumulative_cost["candidates_tokens"] += meta.candidates_token_count or 0
            if hasattr(meta, "candidates_tokens_details") and meta.candidates_tokens_details:
                for detail in meta.candidates_tokens_details:
                    cumulative_cost["thoughts_tokens"] += getattr(detail, "thought_tokens", 0) or 0
            cumulative_cost["total_tokens"] = (
                cumulative_cost["prompt_tokens"] + cumulative_cost["candidates_tokens"]
            )

        candidate = response.candidates[0] if response.candidates else None
        if not candidate:
            raise RuntimeError("Model returned no candidate responses.")

        finish_reason = candidate.finish_reason
        is_truncated = finish_reason in (
            types.FinishReason.MAX_TOKENS,
            "MAX_TOKENS",
            "LENGTH",
        )

        if not is_truncated:
            return {
                "text": response.text,
                "finish_reason": str(finish_reason),
                "truncated": False,
                "attempts_needed": attempt,
                "final_max_output_tokens": current_max_tokens,
                "usage": cumulative_cost,
            }

        next_max_tokens = current_max_tokens * backoff_factor
        if next_max_tokens > max_token_ceiling or attempt >= max_attempts:
            raise TruncatedOutputError(
                f"Call halted mid-output due to MAX_TOKENS after {attempt} attempts.",
                cumulative_usage=cumulative_cost,
                partial_text=response.text or "",
            )

        current_max_tokens = min(next_max_tokens, max_token_ceiling)

    raise TruncatedOutputError(
        "Max retry attempts exhausted.",
        cumulative_usage=cumulative_cost,
        partial_text="",
    )

Step 3: Run the caller and handle failures

Invoke the safe caller inside your pipeline. Catch TruncatedOutputError to stop truncated summaries from entering downstream tasks as verified research.

if __name__ == "__main__":
    prompt = "Search for recent Gemini architecture updates and summarize key details."
    try:
        result = call_gemini_grounded_safe(
            prompt=prompt,
            initial_max_tokens=4096,
            max_token_ceiling=16384,
        )
        print("Completed generation:", result["text"][:120], "...")
        print("Total tokens consumed:", result["usage"]["total_tokens"])
    except TruncatedOutputError as err:
        print(f"Call failed safely: {err}")
        print("Total audited token waste:", err.cumulative_usage)

Common mistakes

  • Checking HTTP status codes instead of finish_reason. The SDK returns HTTP 200 OK even when generation stops mid-sentence. Inspect response.candidates[0].finish_reason directly on every code path.
  • Dropping discarded attempt tokens from billing audits. Discarding a failed attempt without tracking its tokens hides real operational costs. Accumulate prompt, thinking, and candidate tokens across every attempt.
  • Assuming output caps apply only to visible text. Thinking models consume max_output_tokens for internal reasoning and visible output combined. A call spending ~3,900 tokens thinking leaves only 196 tokens under a 4,096 cap.
  • Prompting the model to continue from truncated text. Appending cut-off assistant text into conversational history corrupts internal grounding state. Retry the clean prompt with an increased token limit instead.

Frequently asked questions

Why does Gemini return HTTP 200 when an answer is cut off?

The API protocol treats reaching max_output_tokens as a normal completion event rather than a server error. The server returns HTTP 200 with finish_reason="MAX_TOKENS" on the candidate. The SDK does not throw an exception, so your application code must inspect candidate.finish_reason explicitly.

Why do search-grounded calls consume thinking tokens so quickly?

When Google Search is active, reasoning models generate thinking tokens to construct queries, analyze search results, and evaluate citations. For complex queries, this synthesis step can consume thousands of thinking tokens before visible generation starts, exhausting small output limits prematurely.

Should I retry a truncated call with the same token cap?

No. Repeating an identical grounded query with the same max_output_tokens will repeat the reasoning cycle and fail at the same boundary. You will pay double the token costs without receiving a complete answer. Always scale up the token limit or abort.

Related

Want to try Bridge?

Bridge ships November 2026 on Android. Become a founding member and lock in founders pricing.

Become a founding member