Haystack
A Python framework for composing controlled, production-ready LLM pipelines and agent workflows.
What does this agent do, and when should you use it?
Haystack is an open-source AI orchestration framework for building production-ready LLM applications in Python. Its named execution primitives include Pipeline, which runs synchronously or asynchronously and streams token by token, and Agent, which can execute concurrent tool calls. Developers compose retrieval, ranking, filtering, memory, tools, routing, and generation into workflows with loops, branches, and conditional logic. The project supplies components for retrieval, indexing, tool calling, memory, and evaluation, while supporting custom components through a consistent interface. It targets RAG, semantic search, question answering, multimodal applications, and autonomous agents; for REST API or MCP-server delivery, the README points to the companion Hayhooks project.
An application connects built-in or custom components in a Pipeline to retrieve, rank, filter, combine, structure, and route context before model generation or tool calls. Pipeline runs synchronously or asynchronously and can stream token by token, while Agent can run tool calls concurrently. Agent behavior can be extended with before_llm, before_tool, and on_exit lifecycle hooks, while step_count, token_usage, and tool calls are tracked. SkillToolset can introduce skill descriptions into context only when needed, and Agent Pack includes ready-made agents such as a deep research agent and an advanced RAG agent.
- A Python team building a RAG question-answering system that needs document retrieval, ranking, and generation as separately inspectable steps.
- An enterprise-search team assembling indexing, retrieval, and filtering components for semantic search over an internal knowledge base.
- An application team that needs streaming responses and wants one Pipeline usable in both synchronous and asynchronous execution modes.
- An agent team that needs concurrent tool calls, lifecycle hooks, and token_usage tracking for monitoring and cost control.
- A product team building a multimodal application that needs to place non-text inputs in an explicit workflow.
- A service team that wants to expose an existing Haystack pipeline through HTTP REST endpoints or MCP using Hayhooks.
What are this agent's strengths and limitations?
- Pipeline supports synchronous execution, asynchronous execution, and token-by-token streaming in one named primitive.
- The workflow model supports loops, branches, and conditional logic while keeping retrieval, memory, tools, and generation explicit and traceable.
- Agent exposes lifecycle hooks including before_llm, before_tool, and on_exit, and tracks step_count, token_usage, and tool calls.
- The README documents a multi-provider model and infrastructure path that permits components to be swapped without rewriting the whole system.
- The README does not include a minimal runnable program, credential configuration, or a complete first invocation, so implementation details must be obtained elsewhere.
- Serving pipelines or agents as REST APIs or MCP servers is not presented as an in-repository command; it relies on the companion Hayhooks project.
- Cloud-model and cloud-infrastructure use still requires provider-specific integration and credentials; the README does not define a unified authentication mechanism.
- Anonymous telemetry sends an event when pipeline components are initialized; opting out requires following separate telemetry documentation.
How do you install or deploy this agent?
Run: pip install haystack-ai
For nightly pre-releases, the README also provides: pip install --pre haystack-ai. The documented installation path requires Python and pip.
How do you use this agent?
The README does not provide a copyable first Pipeline or Agent program, provider credential setup, environment-variable names, configuration format, or minimum working invocation. It does state that Python applications compose components in Pipeline and run them synchronously or asynchronously, or use Agent to orchestrate tool calls. Listed model and infrastructure integrations include OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, and local models. REST API or MCP-server delivery requires the separately referenced Hayhooks project.