FastAPI LangGraph Agent Production-Ready Template
A production-ready backend template for AI agent services, bundling LangGraph, long-term memory, observability, and auth.
Evidence shows: README and SECURITY.md mention JWT auth, rate limiting, CORS, bcrypt password hashing, input sanitization, but no implementation details. Dependencies are listed but no vulnerability scanning or pinned versions (except langfuse). External effects: template calls external LLM APIs and Langfuse tracing, but data flow not clearly documented. Rollback: not mentioned. Source attribution: publisher not verified. Deductions: lack of concrete implementation evidence, dependency security not fully addressed, external effects and data flow transparency insufficient.
Evidence shows: README and pyproject.toml dependencies are consistent, but no test results. Dependency availability: many dependencies but no lock file or version ranges. Failure messages: docs mention retries and fallback but no specific error messages. Deductions: lack of test evidence, dependency availability not fully ensured, failure messages not specific.
Evidence shows: README clearly targets AI engineers and lists multiple use cases. Capability boundaries: docs mention OpenAI support and planned multi-provider, but no explicit limits. Trigger precision: no specific trigger conditions. Environment fit: Docker and local setup provided, but not detailed for all environments. Deductions: capability boundaries and trigger precision unclear.
Evidence shows: README provides clear project structure, install steps, FAQ, and doc links. Naming stability: not mentioned. Known limitations: FAQ mentions OpenAI-only but not comprehensive. License: MIT license present. Versioning: pyproject.toml has version but no changelog. Maintenance responsibility: SECURITY.md mentions maintainer but not explicit. Deductions: missing changelog and incomplete known limitations.
Evidence shows: README describes output as API endpoints but no specific output format. Marginal value: template provides many production features but no comparison with existing solutions. Cost-benefit: no performance or cost data. Deductions: output usability not specific, cost-benefit lacks data.
Evidence shows: README claims are partially supported by docs, but no external verification. Cross-source: none. Fact-inference separation: docs distinguish facts and plans but not clearly. Deductions: lack of external verification and clear separation.
- Publisher identity unverified; treat with caution.
- Dependencies not pinned, supply chain risk.
- External LLM and Langfuse calls may involve data exfiltration; clarify data flow.
- No test results provided; reliability uncertain.
What does this agent do, and when should you use it?
FastAPI LangGraph Agent Template is a production-grade starting point for building AI agent backends. It handles stateful conversations, long-term memory (via mem0 and pgvector), tool calling, observability (Langfuse, Prometheus, Grafana), rate limiting, and JWT authentication, so developers can focus on agent logic. The template includes an LLMRegistry service with circular fallback and exponential backoff retries, working with OpenAI-compatible endpoints like Atlas Cloud. The project provides Alembic migrations, an optional Valkey/Redis cache layer, structured logging, and extensive documentation covering architecture, configuration, and evaluation. It is deployable via Docker Compose or locally with Makefile commands.
On startup, it boots a FastAPI application exposing API routes under /docs, and connects to PostgreSQL (SQLModel ORM with Alembic migrations). The agent graph lives in app/core/langgraph/ and supports stateful conversations with checkpointing and tool calling; tools can be added in app/core/langgraph/tools/. LLM calls go through the LLMRegistry, which uses langchain_openai.ChatOpenAI and is configured with DEFAULT_LLM_MODEL, OPENAI_BASE_URL, and OPENAI_API_KEY environment variables; the service rotates across models on failure with a total timeout budget. Long-term memory is provided by in-process mem0 persisted to PostgreSQL with pgvector for semantic search. All LLM calls are traced with Langfuse, and Prometheus exposes metrics. JWT authentication handles sessions, and slowapi enforces rate limits. An evaluation framework (evals/) is also included.
- AI engineers who want a production-grade agent backend in hours rather than assembling all integrations from scratch
- Teams that need long-term semantic memory for multiple users without outsourcing to an external memory cloud
- Developers who require full LLM call tracing in Langfuse and metrics in Prometheus and Grafana
- Projects that need JWT authentication with session management and rate limiting for stateful conversations
- Engineers who want to experiment with different LLMs via a unified OpenAI-compatible endpoint like Atlas Cloud
- Teams that need a complete template with Docker, database migrations, and monitoring stack for quick project bootstrapping
What are this agent's strengths and limitations?
- Includes a full set of production components out of the box: Alembic migrations, mem0+pgvector long-term memory, Langfuse tracing, Prometheus/Grafana metrics, JWT sessions, and slowapi rate limiting
- LLM service features exponential backoff retries and circular fallback, ensuring high availability
- Supports 59+ OpenAI-compatible models via Atlas Cloud, enabling quick model switching
- Comprehensive documentation from architecture to configuration and evaluation lowers onboarding effort
- Currently only natively supports OpenAI-compatible APIs; multi-provider support (Anthropic, Google) is still planned (issue #51) and requires extra adaptation
- Long-term memory depends on PostgreSQL + pgvector, adding database management overhead
- The full production feature set (JWT, caching, monitoring) may be overkill for simple projects
- Limited to OpenAI provider for now; using other providers may require waiting for updates or manual code changes
How do you install or deploy this agent?
Clone the repository and enter the directory, then copy .env.example to .env.development and fill in keys (OPENAI_API_KEY or Atlas Cloud credentials, plus database config). Ensure Docker is installed, then run make install and make docker-up to start the API and PostgreSQL. For local development, refer to docs/getting-started.md.
How do you use this agent?
After starting Docker, visit [http://localhost:8000/docs](http://localhost:8000/docs) for the interactive API. Apply migrations with make migrate. Set DEFAULT_LLM_MODEL in .env.development to select a model. Add custom tools by placing a LangChain @tool-decorated function in app/core/langgraph/tools/ and registering it in the tools list exported from that package. Use Langfuse tracing, or disable it with LANGFUSE_TRACING_ENABLED=false. For full configuration, see docs/configuration.md.
FAQ
Does this template support non-OpenAI models like Anthropic or Google?
LLMRegistry using langchain_openai.ChatOpenAI). Multi-provider support is planned in issue #51.Does long-term memory require a separate mem0 cloud account?
OPENAI_API_KEY for fact extraction and embeddings.Can I disable Langfuse tracing?
LANGFUSE_TRACING_ENABLED=false or omit the Langfuse keys. Structured logs still capture request/session/user context.What if the API won't start?
make docker-up brings it up), .env.development exists with required keys, and run make migrate to apply migrations.How do I adjust the rate limiting policy?
app/core/limiter.py (slowapi). You can change per-route decorators or the default rate, and related environment variables are documented in docs/configuration.md.