For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Backends
Verified Code examples on this page have been automatically tested and verified.Configure backends to route traffic to hostnames, LLM providers, and MCP servers.
Agentgateway backendsBackendA destination service that receives traffic from agentgateway. Backends can be static hosts, MCP servers, LLM providers, or other services. control where traffic is routed to. Agentgateway supports a variety of backends, such as simple hostnames and IP addresses, LLM providersProviderA service that provides LLM capabilities, such as OpenAI, Anthropic, or Azure. Agentgateway supports multiple LLM providers and can route to different providers based on configuration., and MCP servers.
llm or mcp modes, the examples on this page show each option in tabs. For more information, see Routing-based configuration.Static Hosts
The simplest form of backend is a static hostname or IP address. Static hosts are a routing-based backend, so they are configured under binds; the simplified llm and mcp modes model only LLM providers and MCP targets. For example:
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
binds:
- port: 3000
listeners:
- protocol: HTTP
routes:
- backends:
- host: example.com:8080
weight: 1
- host: 127.0.0.1:80
weight: 9MCP Servers
The MCP backend allows you to connect to an MCP server. Below shows a simple example, exposing a local and remote MCP server. See the MCP connectivity guide for more information.
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
mcp:
port: 3000
targets:
- name: stdio-server
stdio:
cmd: npx
args: ["@modelcontextprotocol/server-everything"]
- name: http-server
mcp:
host: https://example.com/mcpSession routing
By default, MCP backends use stateful session routing, where the gateway tracks session IDs and routes subsequent requests to the same upstream. For upstreams that do not maintain server-side session state, you can set statefulMode: stateless. In stateless mode, the gateway automatically wraps each request with an initialization sequence, so the upstream server processes every request independently.
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
mcp:
port: 3000
statefulMode: stateless
targets:
- name: openapi-server
openapi:
host: petstore3.swagger.io:443
schema:
url: https://petstore3.swagger.io/api/v3/openapi.jsonLLM Providers
Agentgateway natively supports connecting to LLM providers, such as OpenAI and Anthropic. Below shows a simple example, connecting to OpenAI. See the LLM consumption guide for more information.
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
models:
- name: openai
provider: openAI
params:
model: gpt-3.5-turbo
apiKey: "$OPENAI_API_KEY"