NVIDIA NeMo Guardrails Library
Programmable guardrails for LLM-based conversational systems to ensure safety, trust, and control.
Evidence shows: README clearly describes telemetry collection and opt-out methods, data flow transparency is good; sensitive data handling has built-in detection and masking; dependency security has version pins and dependency groups, but no vulnerability scanning evidence; external effects: telemetry is on by default but opt-out is provided; rollback mechanism not mentioned; source attribution is clear with authors and copyright. Deductions: telemetry on by default without explicit user consent; dependency security lacks specific vulnerability scanning evidence; rollback mechanism absent.
Evidence shows: README and pyproject.toml descriptions are consistent, self-consistency is good; dependencies have version ranges but no availability guarantees; failure messages: test files have assertions on error messages, but overall failure handling documentation is insufficient. Deductions: dependency availability not guaranteed; failure message documentation insufficient.
Evidence shows: README clearly identifies target audience (developers) and multiple use cases; capability boundaries are described (e.g., built-in guardrails may not be suitable for production); trigger precision: Colang language provides fine-grained control; environment fit: supports multiple Python versions and optional dependencies. Deductions: capability boundary description is brief; environment fit does not cover all platforms.
Evidence shows: information architecture is clear with README, docs links, examples; install notes are detailed; naming stability with version numbers; examples and FAQ provided; known limitations mentioned but not detailed; license is clearly Apache-2.0; versioning changelog link provided; maintenance responsibility clearly assigned to NVIDIA. Deductions: known limitations description not detailed.
Evidence shows: output format compatible with OpenAI Chat Completions API, good usability; marginal value high with unique Colang language and multiple guardrails; cost-benefit: performance benchmarks exist but no specific cost data. Deductions: cost-benefit lacks specific data.
Evidence shows: README claims are supported by docs and paper, traceability is good; cross-source corroboration: test files and CI config exist but no independent verification; fact-inference separation: README distinguishes facts and inferences but not clearly. Deductions: cross-source corroboration insufficient; fact-inference separation not clear.
- Telemetry is on by default, requiring user opt-out, which may raise privacy concerns.
- Dependency security lacks vulnerability scanning evidence; users should assess themselves.
- Rollback mechanism not mentioned; may not be able to revert after upgrade failure.
What does this agent do, and when should you use it?
NeMo Guardrails is an open-source toolkit from NVIDIA for adding programmable guardrails (rails) to LLM-based conversational applications. It defines five types of rails—input, dialog, retrieval, execution, and output—using the Colang modeling language, and includes built-in guardrails for fact-checking, hallucination detection, jailbreak prevention, and sensitive data masking. The library provides a Python API (LLMRails) and a CLI server (nemoguardrails server), supports multiple LLM providers (e.g., OpenAI, NVIDIA NIM), and offers optional LangChain integration. Telemetry is collected anonymously but can be disabled via environment variables. Licensed under Apache 2.0.
The library inserts a guardrails layer between application code and the LLM. Users call LLMRails.generate() or generate_async() with a list of messages, and the layer applies configured input/dialog/retrieval/execution/output rails—such as jailbreak detection, sensitive data masking, self-check facts, and hallucination detection—before returning a standard Chat Completions format response. Configuration is defined in config.yml and .co files, with custom actions in actions.py. The CLI supports interactive chat (nemoguardrails chat), server startup, and evaluation (nemoguardrails evaluate).
- Developers building a customer service bot with GPT-4 that must stay on-topic and fact-check responses.
- Enterprise RAG systems needing to filter toxic retrieved chunks and validate generated content.
- Domain-specific assistants that must follow standard operating procedures (authentication, support flows).
- Teams wanting to add a safety layer to their custom LLM endpoints to prevent jailbreaks and prompt injections.
- LangChain users who want to wrap their chains with guardrails for extra safety.
- Product teams that need to hardcode conversational paths for compliance and consistency.
What are this agent's strengths and limitations?
- Five guardrail types cover input, dialog, retrieval, execution, and output, providing comprehensive protection.
- Built-in guardrails library includes jailbreak detection, fact-checking, hallucination detection, and integrations with NVIDIA safety models and third-party APIs.
- Supports multiple LLMs (OpenAI, NVIDIA NIM, etc.) with sync and async APIs for flexible integration.
- Includes evaluation tools and vulnerability scanning reports to quantify safety.
- Apache 2.0 license, open for community contributions.
- Requires learning Colang modeling language, adding a learning curve.
- Dependent on LLM providers, requiring API keys and incurring costs.
- Default telemetry collection requires opt-out via environment variables.
- LangChain integration is optional and requires additional installation and configuration.
- Built-in guardrails may not be suitable for all production scenarios without adjustment.
How do you install or deploy this agent?
Install via pip: pip install nemoguardrails. Requires Python 3.10 to 3.13 and API keys from your chosen LLM provider (e.g., OpenAI).
How do you use this agent?
- Create a config directory with config.yml (model and rail settings), .co files (Colang flows), and optionally actions.py. 2. Use the LLMRails class: from nemoguardrails import LLMRails, RailsConfig; config = RailsConfig.from_path('PATH/TO/CONFIG'); rails = LLMRails(config); completion = rails.generate(messages=[{'role': 'user', 'content': 'Hello world!'}]). 3. Optionally start a server: nemoguardrails server --config PATH/TO/CONFIGS --port 8000, and interact via the /v1/chat/completions endpoint.