In-Process AI Bug Reporters

TLDR: When your application fails, the best moment to write the bug report is now, from inside the process, while the state that explains the failure still exists. An embedded LLM agent can do exactly that: inspect the live runtime state, ask for more, and draft a report that a human approves before it is sent.

Definition

An in-process LLM bug reporter is a component embedded in an application that, upon a failure (uncaught exception, error log, failed assertion, or a softer signal such as a frustrated user), starts an LLM agent that:

1. receives the failure and a first snapshot of the runtime state (stack, locals, configuration, recent logs),
2. queries additional state through read-only tools, like a coding agent explores a repository,
3. writes a structured bug report (title, what happened, expected behavior, evidence, likely cause),
4. stores it locally as a draft and sends it only after explicit approval.

The key word is in-process. Classical crash reporters ship a dump and analyze it later. Server-side AI debuggers read telemetry after the fact. Here, the agent runs at the failure site, before the process exits and the heap is gone forever.

Prior art

• Claude Code now does this for itself. Since version 2.1.238 (August 2026), when a tool keeps failing, when the task cannot be completed, or when the user or the model notices a mistake, the model drafts a feedback report with title, category, and description, with access to the whole session context. Nothing is transmitted before the user presses send (FrontierNews, 36kr).
• Automatic crash reporting. Windows Error Reporting has collected crash reports from millions of machines since the late 1990s; the key idea is bucketing, grouping reports by failure signature so that one bug gives one bucket (Glerum et al., Debugging in the (Very) Large, SOSP 2009). Mozilla’s Breakpad/Socorro does the same with minidumps, and restricts minidump access to a few engineers because dumps contain private data (Firefox crash reporter docs).
• Exception-to-issue libraries. PyBugReporter catches exceptions and opens GitHub issues; its README warns it is “not recommended for public repos, as a malicious user could spam your issues with fake exceptions.” llm-exceptions (2024) asks an LLM to explain a traceback in Jupyter. Neither combines state exploration and reporting.
• LLM-driven debuggers. ChatDBG (UMass Amherst & Williams College, FSE 2025) gives the LLM “autonomy to take the wheel”, issuing pdb/lldb/gdb commands to navigate stacks and inspect state; it finds root causes in 67–85% of cases (paper). Microsoft’s debug-gym (2025) is an environment to train agents to use pdb.
• LLM-enhanced crash reports. Fahim et al. enhance 492 real Java crash reports with an LLM: problem localization goes from 10.6% (developer-written) to 40.2–43.1%, and the agentic variant, which explores the call graph, gives more complete root causes (58% vs 46%) for $0.01 per report (arXiv 2509.13535). Sentry’s Seer does root-cause analysis on production telemetry, server-side.

The empirical signal is clear: LLM-written reports are better than human-written ones, and agentic ones are better than one-shot ones. What is missing is putting the agent where the state is.

Why in-process?

Because runtime state is perishable. Once the process dies, you keep what you serialized, and you serialized what you guessed would matter, before knowing the bug. An embedded agent reverses this: it decides what to look at after seeing the failure.

• The value of user.settings in frame 3? Ask.
• Which version of libfoo is actually loaded? Ask.
• Is the config file on disk the same as the one in memory? Ask.
• What were the last 20 log lines of this request? Ask.

This is exactly what a good developer does in a debugger, except it happens on the user’s machine, in production, at the moment of failure, without the developer.

A second benefit: the LLM widens the notion of failure. Claude Code triggers not only on errors but also on detected user frustration. An embedded reporter can be triggered by a wrong-looking output, a timeout, or a user saying “this is broken”.

Properties of a good LLM bug reporter

• Read-only tools. The agent inspects, it does not act. Beware: in Python, reading an attribute can run arbitrary code (properties, __repr__, __getattr__). Inspection must be side-effect-aware. And when the agent asks for a frame that does not exist or a path that does not resolve, the tool error must teach it the valid options, see my self-teaching error surface pattern: the bug reporter has one shot and a small budget, every wasted call counts.
• Redaction before the LLM, not after. Secrets in environment variables, API keys in locals, tokens in headers must be masked before being put in the prompt, especially when the LLM is remote.
• Bounded. A budget in time, tokens, tool calls and money. An error in a loop must not produce 10,000 LLM calls: rate limiting and deduplication (bucketing à la WER) are mandatory.
• Fail-safe. The reporter must never make things worse: if it crashes, times out, or has no network, the original failure is reported as it would have been without it.
• Injection-aware. Exception messages and variable values often contain user input. They are attacker-controlled text fed to an agent with access to process state and a send button. Runtime state is data, never instructions.

Conclusion

Automatic crash reporting gave developers volume. LLM-based bug reporting gives them explanation. The design pattern is simple: at failure time, before the state is gone, let an agent explore it and write the report, then let the human decide.