PageIndex: Vectorless, Reasoning-based RAG
Build hierarchical tree indexes and reason over them for context-aware retrieval - no vector DB, no chunking.
- Source repo
- VectifyAI/PageIndex
- Stars
- ★ 39k
- Last updated
- today
- License
- MIT
- Primary language
- Python
- FA score
- 33/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free tier plus a paid hosted plan
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- Financial analysts extracting specific data points from SEC filings or earnings reports for investment decisions.
- Not a fit if
- Teams that need fully offline, on-premises data handling
- Users with scanned or image-heavy PDFs (local mode is text-only)
- Teams needing stable production features (Flash mode is preview)
- Source review
- 33/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
PageIndex is an open-source vectorless, reasoning-based RAG engine that constructs a hierarchical tree index from long documents and uses LLMs to reason over that index for retrieval. Unlike traditional vector similarity search, PageIndex mirrors human reading patterns through tree search, prioritizing relevance and interpretability. The repository provides a self-hosted Python package that generates tree structures from PDF or Markdown documents, with standard PDF parsing and an optional 'Flash' mode for faster generation. The project claims state-of-the-art 98.7% accuracy on FinanceBench, outperforming vector RAG. Deployment options include self-hosted, cloud service via API/MCP, and enterprise private deployment.
PageIndex transforms PDF or Markdown documents into a hierarchical tree structure (like a table of contents), where each node includes a title, summary, and page ranges. Retrieval involves LLM-based reasoning over the tree to find the most relevant sections. Key operations: run run_pageindex.py to generate the tree with --pdf_path or --md_path; optionally use --flash for heuristic-based fast generation; the examples/agentic_vectorless_rag_demo.py demonstrates Agentic RAG. Output is a JSON tree with fields like node_id, start_index, end_index, and summary.
- Financial analysts extracting specific data points from SEC filings or earnings reports for investment decisions.
- Legal teams searching through thousands of pages of regulations to find relevant clauses for compliance.
- Medical researchers retrieving specific study findings from academic literature for systematic reviews.
- Technical writers looking up API details in product manuals to update user guides.
- Enterprises implementing traceable and explainable document Q&A for internal knowledge management.
How do you install or deploy this agent?
Clone the repository from GitHub and install dependencies: pip3 install --upgrade -r requirements.txt. Set your LLM API key by creating a .env file in the root directory with OPENAI_API_KEY=your_key (LiteLLM supports multiple providers).
How do you use this agent?
Generate a tree from a PDF using python3 run_pageindex.py --pdf_path /path/to/document.pdf or from Markdown with --md_path. Optional arguments include --model (default gpt-4o-2024-11-20), --max-pages-per-node, etc. Use --flash for faster heuristic-based generation. For Agentic RAG, install openai-agents and run python3 examples/agentic_vectorless_rag_demo.py.
What are this agent's strengths and limitations?
- No vector database or chunking, avoiding the similarity-relevance mismatch of traditional RAG.
- Tree-based reasoning provides better traceability and explainability.
- Achieved 98.7% accuracy on FinanceBench, outperforming vector RAG.
- Flexible deployment: self-hosted, cloud, and enterprise options.
- Reliance on LLM reasoning leads to higher latency and cost, plus requires API keys.
- Standard PDF parsing struggles with complex layouts; cloud service needed for better OCR.
- Self-hosted version only supports PDF and Markdown; advanced features like OCR are paid.
- Project is early-stage; Flash mode is preview and may be unstable.
How does this agent compare with similar options?
Compared to traditional vector-DB RAG systems (e.g., based on FAISS or Pinecone), PageIndex emphasizes reasoning over similarity, claiming higher accuracy on professional documents.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| PageIndex: Vectorless, Reasoning-based RAG This agent | 33 · Major gaps | Library / SDKFreemium | ★ 39k | today | Python | OpenAI API |
| AI Bank Statement Automation | 48 · Major gaps | CLIFree + model costs | ★ 627 | 2mo ago | Jupyter Notebook | OpenAI API |
| Controllable Complex RAG | 40 · Major gaps | Library / SDKFree + model costs | ★ 1.6k | 27d ago | Jupyter Notebook | OpenAI API |
| CAMEL Multi-Agent Framework | 61 · Some gaps | Library / SDKFree + model costs | ★ 18k | 7d ago | Python | OpenAI API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence: The repository provides no permission model, user confirmation mechanism, data flow transparency, sensitive data handling policy, dependency security audit, external impact assessment, rollback mechanism, or source attribution. All trust criteria lack evidence, hence score 0.
Evidence: Code and tests are internally consistent, tests cover key paths, but dependencies are pinned (e.g., litellm==1.84.0) without availability guarantees; error handling is tested but production failure messages are not explicit. Thus self-consistency 2, dependency availability 1, failure messages 1.
Evidence: README defines target audience (professionals handling long documents) and scenarios (finance, legal), capability boundaries (standard PDF parsing, complex PDFs need cloud) are stated, but trigger precision (e.g., CLI args) not detailed, environment fit (OS, Python version) not specified. Hence audience/scenarios 2, capability boundaries 2, trigger precision 1, environment fit 1.
Evidence: README is well-structured, install steps clear, examples and FAQ exist, known limitations (e.g., standard PDF parsing) mentioned, MIT license explicit, but versioning/changelog missing, maintenance responsibility unclear (publisher unverified). Thus information architecture 2, install notes 2, naming stability 1, examples/FAQ 2, known limitations 1, license 2, versioning/changelog 0, maintenance responsibility 1.
Evidence: Output is tree-structured JSON usable for RAG, marginal value in avoiding vector DB, but cost-benefit not quantified (e.g., API costs, performance comparisons). Hence output usability 2, marginal value 2, cost-benefit 1.
Evidence: README claims 98.7% accuracy but no reproducible verification steps; cross-source corroboration limited (only external benchmark mentioned); facts vs inferences not clearly separated. Hence claim traceability 1, cross-source corroboration 1, fact/inference separation 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- No security or privacy documentation provided; assess data exposure risks before use.
- Dependencies are pinned but not security-audited; check for known vulnerabilities.
- Performance claims (98.7% accuracy) lack reproducible verification steps; treat with caution.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v0.4.4 PageIndex 0.4.4 — cheaper than vector index; also runs on your Claude subscription
Release notes excerpt
- Unified SDK: vectorless RAG. Local or cloud. Index or chat. - Faster: trees are created from layout stats. - Cheaper than vector index: LLMs only write node summaries. - Runs on your Claude subscription. Index, to chat, to agent integration, one client. Local mode needs no server, no vector DB, no PageIndex API key. ```python client = PageIndexClient() client.submit_document("report.pdf") client.chat("What does the report conclude?") ``` Highlights …View on GitHub ↗ -
v0.4.3 PageIndex 0.4.3 — cheaper than vector index; also runs on your Claude subscription
Release notes excerpt
- Unified SDK: vectorless RAG. Local or cloud. Index or chat. - Faster: trees are derived from layout stats. - Cheaper than Vector Index: LLMs only write node summaries. - Runs on your Claude subscription. Index, to chat, to agent integration, one client. Local mode needs no server, no vector DB, no PageIndex API key. ```python client = PageIndexClient() client.submit_document("report.pdf") client.chat("What does the report conclude?") ``` Highlights …View on GitHub ↗ -
Release notes excerpt
PageIndex is now cheaper to build than a vector index. - The PageIndex SDK, local or cloud — vectorless, reasoning-based RAG, end to end. - Much faster indexing — the PageIndex Flash engine gets the tree from layout stats: no LLM involved for the structure generation itself, LLMs only write the node summaries, and tree expansion proposes a wave of nodes concurrently. ```python client = PageIndexClient() client.submit_document("report.pdf") client.chat("What does the report conclude?") ``` Index, to chat, to agent integration, one client. …View on GitHub ↗