NVIDIA NeMo Guardrails Library
Programmable guardrails for LLM-based conversational systems to ensure safety, trust, and control.
- Source repo
- NVIDIA-NeMo/Guardrails
- Stars
- ★ 7.3k
- Last updated
- 3d ago
- License
- NOASSERTION
- Primary language
- Python
- FA score
- 54/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Developers building a customer service bot with GPT-4 that must stay on-topic and fact-check responses.
- Not a fit if
- Developers unwilling to learn the Colang modeling language
- Teams with strict compliance rules against anonymous telemetry leaving the machine
- Source review
- 54/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
NeMo Guardrails is an open-source toolkit from NVIDIA for adding programmable guardrails (rails) to LLM-based conversational applications. It defines five types of rails—input, dialog, retrieval, execution, and output—using the Colang modeling language, and includes built-in guardrails for fact-checking, hallucination detection, jailbreak prevention, and sensitive data masking. The library provides a Python API (LLMRails) and a CLI server (nemoguardrails server), supports multiple LLM providers (e.g., OpenAI, NVIDIA NIM), and offers optional LangChain integration. Telemetry is collected anonymously but can be disabled via environment variables. Licensed under Apache 2.0.
The library inserts a guardrails layer between application code and the LLM. Users call LLMRails.generate() or generate_async() with a list of messages, and the layer applies configured input/dialog/retrieval/execution/output rails—such as jailbreak detection, sensitive data masking, self-check facts, and hallucination detection—before returning a standard Chat Completions format response. Configuration is defined in config.yml and .co files, with custom actions in actions.py. The CLI supports interactive chat (nemoguardrails chat), server startup, and evaluation (nemoguardrails evaluate).
- Developers building a customer service bot with GPT-4 that must stay on-topic and fact-check responses.
- Enterprise RAG systems needing to filter toxic retrieved chunks and validate generated content.
- Domain-specific assistants that must follow standard operating procedures (authentication, support flows).
- Teams wanting to add a safety layer to their custom LLM endpoints to prevent jailbreaks and prompt injections.
- LangChain users who want to wrap their chains with guardrails for extra safety.
- Product teams that need to hardcode conversational paths for compliance and consistency.
How do you install or deploy this agent?
Install via pip: pip install nemoguardrails. Requires Python 3.10 to 3.13 and API keys from your chosen LLM provider (e.g., OpenAI).
How do you use this agent?
- Create a config directory with config.yml (model and rail settings), .co files (Colang flows), and optionally actions.py. 2. Use the LLMRails class: from nemoguardrails import LLMRails, RailsConfig; config = RailsConfig.from_path('PATH/TO/CONFIG'); rails = LLMRails(config); completion = rails.generate(messages=[{'role': 'user', 'content': 'Hello world!'}]). 3. Optionally start a server: nemoguardrails server --config PATH/TO/CONFIGS --port 8000, and interact via the /v1/chat/completions endpoint.
What are this agent's strengths and limitations?
- Five guardrail types cover input, dialog, retrieval, execution, and output, providing comprehensive protection.
- Built-in guardrails library includes jailbreak detection, fact-checking, hallucination detection, and integrations with NVIDIA safety models and third-party APIs.
- Supports multiple LLMs (OpenAI, NVIDIA NIM, etc.) with sync and async APIs for flexible integration.
- Includes evaluation tools and vulnerability scanning reports to quantify safety.
- Apache 2.0 license, open for community contributions.
- Requires learning Colang modeling language, adding a learning curve.
- Dependent on LLM providers, requiring API keys and incurring costs.
- Default telemetry collection requires opt-out via environment variables.
- LangChain integration is optional and requires additional installation and configuration.
- Built-in guardrails may not be suitable for all production scenarios without adjustment.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| NVIDIA NeMo Guardrails Library This agent | 54 · Major gaps | Library / SDKFree + model costs | ★ 7.3k | 3d ago | Python | — |
| OpenAI Agents SDK | 64 · Some gaps | Library / SDKFree + model costs | ★ 30k | 2d ago | Python | OpenAI API |
| Generative AI with LangChain | 52 · Major gaps | Library / SDKFree + model costs | ★ 1.4k | 2mo ago | Jupyter Notebook | OpenAI API · Claude API |
| Promptise Foundry | 44 · Major gaps | Library / SDKFree + model costs | ★ 874 | today | Python | Claude.ai · OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: README clearly describes telemetry collection and opt-out methods, data flow transparency is good; sensitive data handling has built-in detection and masking; dependency security has version pins and dependency groups, but no vulnerability scanning evidence; external effects: telemetry is on by default but opt-out is provided; rollback mechanism not mentioned; source attribution is clear with authors and copyright. Deductions: telemetry on by default without explicit user consent; dependency security lacks specific vulnerability scanning evidence; rollback mechanism absent.
Evidence shows: README and pyproject.toml descriptions are consistent, self-consistency is good; dependencies have version ranges but no availability guarantees; failure messages: test files have assertions on error messages, but overall failure handling documentation is insufficient. Deductions: dependency availability not guaranteed; failure message documentation insufficient.
Evidence shows: README clearly identifies target audience (developers) and multiple use cases; capability boundaries are described (e.g., built-in guardrails may not be suitable for production); trigger precision: Colang language provides fine-grained control; environment fit: supports multiple Python versions and optional dependencies. Deductions: capability boundary description is brief; environment fit does not cover all platforms.
Evidence shows: information architecture is clear with README, docs links, examples; install notes are detailed; naming stability with version numbers; examples and FAQ provided; known limitations mentioned but not detailed; license is clearly Apache-2.0; versioning changelog link provided; maintenance responsibility clearly assigned to NVIDIA. Deductions: known limitations description not detailed.
Evidence shows: output format compatible with OpenAI Chat Completions API, good usability; marginal value high with unique Colang language and multiple guardrails; cost-benefit: performance benchmarks exist but no specific cost data. Deductions: cost-benefit lacks specific data.
Evidence shows: README claims are supported by docs and paper, traceability is good; cross-source corroboration: test files and CI config exist but no independent verification; fact-inference separation: README distinguishes facts and inferences but not clearly. Deductions: cross-source corroboration insufficient; fact-inference separation not clear.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Telemetry is on by default, requiring user opt-out, which may raise privacy concerns.
- Dependency security lacks vulnerability scanning evidence; users should assess themselves.
- Rollback mechanism not mentioned; may not be able to revert after upgrade failure.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
What's Changed This patch release addresses bug fixes and improvements. 🐛 Bug Fixes - *(benchmark)* Correct Locust concurrency measurements and sweeps ([#2323](https://github.com/NVIDIA-NeMo/Guardrails/issues/2323)) - *(content-safety)* Parse Nemotron 3.5 responses ([#2370](https://github.com/NVIDIA-NeMo/Guardrails/issues/2370)) - *(server)* Return overload response for IORails admission shedding ([#2334](https://github.com/NVIDIA-NeMo/Guardrails/issues/2334)) - *(rails)* Load config.py for combined configurations ([#2371](https://github.com/NVIDIA-NeMo/Guardrails/issues/2371)) …View on GitHub ↗ -
Release notes excerpt
What's Changed NeMo Guardrails 0.24.0 substantially expands the built-in rails available through IORails. IORails can now execute 59 of the library's 67 action-backed input and output surfaces, including community integrations and rails that transform content, while sharing the same underlying rail implementations with LLMRails. Three related foundations make this possible: - RailOutcome: decide once, render many. A rail action returns one engine-neutral allow, block, or transform result that either LLMRails or IORails can enforce. …View on GitHub ↗ -
Release notes excerpt
What's Changed This release expands tool calling and observability in IORails. Tool calling now works for streaming and non-streaming requests, including local rails that validate model-emitted tool calls and application-returned tool results. The OpenAI-compatible server also supports tool calling and adds a new /v1/checks endpoint for running input or output rails without generating a new model response. …View on GitHub ↗