LangChain launches LangSmith LLM Gateway in beta

LangChain has introduced the LangSmith LLM Gateway in private beta, a runtime governance tool designed to sit directly between AI agents and LLM providers. Unlike traditional post-hoc logging tools, this gateway actively intercepts requests to enforce budget limits and redact personally identifiable information before data ever reaches an external model. Setting up the gateway requires a simple one-line change, swapping the application's base_url to the LangSmith endpoint and adding provider keys to workspace secrets.

The gateway addresses common production hazards, such as a coding agent getting trapped in an overnight retry loop that triggers 10,000 LLM calls and a massive invoice. To prevent this, developers can establish hard spend caps at the organization, workspace, user, or API key level. When an agent hits these limits, the gateway blocks the request and returns a 402 error response. It also features real-time cost rollups and automated PII and secrets detection, redacting sensitive data from both the model requests and downstream traces.

For AI practitioners, this release unifies observability and enforcement within a single interface. Instead of stitching together separate network gateways, guardrail platforms, and data catalogs, developers can manage policies where they already build and evaluate agents. When a policy violation occurs, it surfaces as a traceable event inside the LangSmith Engine. Users can click directly from a blocked request to the exact trace that triggered it, allowing them to debug prompts or tool configurations without switching tools.

Looking ahead, LangChain plans to expand the gateway's capabilities beyond basic LLM calls. Future updates will introduce deeper security controls, flexible enforcement options like model fallbacks and soft rollouts, and integration with tool calls and Model Context Protocol (MCP) servers. By embedding governance into the active agent lifecycle, the company aims to make policy enforcement a native component of agent development.