Functionary: Conversational LLM for Controlled Tool Use
Functionary is a language model that can interpret and execute functions/plugins, deciding when to call tools and understanding their results.
- Source repo
- MeetKai/functionary
- Stars
- ★ 1.6k
- Last updated
- 3mo ago
- License
- MIT
- Primary language
- Python
- FA score
- 30/100 · Major gaps
At a glance
- How it runs
- Works with
- Platform-specificOpenAI API
- Cost
- Free, no paid service needed
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- Developers integrating a chat model with external APIs for tasks like weather queries.
- Not a fit if
- Teams needing maintained, supported software; the project is deprecated with no fixes
- Users with a single consumer GPU; medium models need 4×A6000 or 2×A100 80GB
- Anyone wanting a turnkey app rather than deploying their own inference server
- Source review
- 30/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
Functionary is a language model developed by MeetKai that specializes in function calling. It can decide whether to invoke functions based on user input, supports parallel calls, and generates responses grounded in tool execution results. The project is deprecated but provides multiple model versions, such as meetkai/functionary-v4r-small-preview and functionary-medium-v3.1. Deployment is possible via vLLM, SGLang, or TGI servers, with an OpenAI-compatible API. Despite being unmaintained, the code and documentation serve as a valuable reference.
Functionary takes user messages and tool definitions in JSON Schema format, uses the model to determine if function calls are needed, and generates tool_calls with function names and arguments. It supports parallel execution and can process tool results to produce final responses. Deployed via servers, it exposes an OpenAI-compatible chat completions endpoint at /v1/chat/completions, accepting standard requests and returning chat responses.
- Developers integrating a chat model with external APIs for tasks like weather queries.
- Scenarios requiring the model to call multiple tools in parallel, e.g., booking flights and hotels.
- Customer support systems that need structured extraction from user complaints.
- Researchers studying function-calling model implementations.
- Teams needing a self-hosted LLM service with an OpenAI-compatible interface.
How do you install or deploy this agent?
The project is deprecated, but the README provides installation steps: clone the repository, then install dependencies using pip: for vLLM run pip install -e .[vllm]; for SGLang run pip install -e .[sglang] --find-links ....
How do you use this agent?
Start the server using vLLM: python3 server_vllm.py --model "meetkai/functionary-v4r-small-preview" --host 0.0.0.0 --port 8000 --max-model-len 8192. Then use the OpenAI client: set base_url="http://localhost:8000/v1" and api_key="functionary", and send chat.completions requests with messages and tools.
What are this agent's strengths and limitations?
- Supports parallel function calls and code interpreter, offering rich functionality.
- Ranked 2nd on Berkeley Function-Calling Leaderboard, demonstrating strong performance.
- Provides OpenAI-compatible API, easing integration with existing applications.
- Multiple deployment backends (vLLM, SGLang, TGI) allow flexibility.
- Officially deprecated and no longer maintained, with no updates or support.
- Medium models require multiple GPUs (e.g., 4×A6000), incurring significant hardware costs.
- GGUF integration in llama.cpp may lag, necessitating custom scripts.
- Dependent on specific inference servers; not a standalone application.
How does this agent compare with similar options?
The README compares Functionary with NexusRaven, Gorilla, Glaive, and GPT-4-1106-preview, highlighting its support for multi-turn conversations, following up on missing arguments, and generating responses grounded in tool results.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Functionary: Conversational LLM for Controlled Tool Use This agent | 30 · Major gaps | Self-hosted serviceFree | ★ 1.6k | 3mo ago | Python | OpenAI API |
| LLM-in-Sandbox | 52 · Major gaps | CLIFree + model costs | ★ 245 | 1mo ago | Python | OpenAI API · Claude API |
| Forge: Reliability Layer for Self-Hosted LLM Tool Calling | 49 · Major gaps | CLIFree + model costs | ★ 2.3k | 4d ago | Python | OpenAI API · Claude API |
| Qwen-Agent | 42 · Major gaps | Library / SDKFree + model costs | ★ 17k | 7mo ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: the repository provides no documentation on permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust criteria lack evidence, hence score 0.
Evidence shows: README and test files provide some consistency evidence, such as test cases and prompt template tests, but no failure message handling or dependency availability guarantees. Self-consistency scores 1, dependency availability scores 1, failure messages scores 0.
Evidence shows: README provides multiple deployment scenarios and model choices, but does not clearly define capability boundaries or trigger precision. Audience and scenarios scores 2, capability boundaries scores 1, trigger precision scores 1, environment fit scores 1.
Evidence shows: README provides installation instructions, examples, and model lists, but lacks known limitations, version changelog, and maintenance responsibility. Information architecture scores 2, install notes scores 2, naming stability scores 1, examples and FAQ scores 2, known limitations scores 1, license scores 2, versioning changelog scores 1, maintenance responsibility scores 1.
Evidence shows: README provides usage examples and output formats, but no performance benchmarks or cost-benefit analysis. Output usability scores 2, marginal value scores 2, cost-benefit scores 1.
Evidence shows: claims in README are partially supported by tests, but lack independent verification and separation of facts and inferences. Claim traceability scores 1, cross-source corroboration scores 1, fact-inference separation scores 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- The repository is deprecated, code and documentation may be outdated, and should not be used in production.
- No security audit or permission management documentation is provided; assess risks before use.
- Dependencies are pinned but no vulnerability scanning or update policy is provided.