Hiring Agent Resume Scorer
Turns resume PDFs, GitHub signals, and role-specific rubrics into evidence-backed scores.
The data flow, model providers, GitHub lookups, caching, and CSV writes are clearly disclosed; local Ollama operation and an optional GitHub token show some least-privilege consideration. Deductions apply because resumes are sensitive personal data, yet the supplied sources specify no encryption, access control, retention, deletion, redaction, or cloud-transfer privacy safeguards. Invoking the CLI provides broad consent, but there is no separate confirmation for Gemini/GitHub disclosure or persistent output. Dependencies are exactly pinned, but no vulnerability scanning, supply-chain verification, or update policy is shown. Generated files are identifiable, though no formal rollback or cleanup procedure exists. MIT copyright and external discussions are attributed, while publisher identity remains unknown.
The architecture, structured schemas, role weights, and processing stages are internally coherent. The README candidly acknowledges LLM nondeterminism and invisible-text manipulation, so those weaknesses are not treated as reliability guarantees. Ollama and Gemini offer backend alternatives, with reasonably clear prerequisites and model acquisition steps; however, operation still depends on model services, GitHub availability, and configured model names without a documented degradation strategy. No exception taxonomy, actionable error-message examples, or recovery guidance is evidenced, so failure_messages receives zero.
Local and hosted models, different model sizes, and extensible role directories support role-specific resume scoring across several environments. The explicit statements that this is not an ATS, customer product, or HackerRank open-role screening system thoroughly define capability boundaries. A required --role and explicit PDF path make invocation reasonably precise, but input validation, role compatibility checks, and safeguards against unsafe configurations are not fully documented. Python requirements, cross-platform virtual-environment instructions, and environment variables are covered, while containers, resource budgets, and enterprise deployment needs are not.
The README has strong organization across contents, architecture, configuration, workflow, and directory layout; prerequisites and installation commands are sufficient for ordinary setup, justifying full marks there. Module and role naming are generally consistent, but several model names are presented without stability or deprecation commitments. Examples are extensive, although there is no formal FAQ or systematic troubleshooting section. Nondeterminism, invisible-text attacks, GitHub-centric bias, and ethical concerns are disclosed thoroughly. The complete MIT license and copyright notice are present. No release-version policy or changelog is supplied. CONTRIBUTING is referenced, but its content is absent from the evidence, and no maintainer contact, support term, or explicit update owner is established; registry non-verification means only that publisher identity is unknown.
The proposed output includes category scores, evidence, bonuses, deductions, a readable report, and role-shaped CSV data, making the static output design highly usable. Structured extraction, GitHub enrichment, and ranking provide clear marginal value when triaging a large application pool; nevertheless, disclosed score variance, manipulation risk, and GitHub-centric bias reduce decision value. A local path avoids mandatory cloud API charges, but hardware needs, latency, token/API expense, and human-review costs are not quantified, so cost-benefit support is thin.
Scoring rules, role weights, prompt locations, processing modules, and evidence fields map claims to inspectable repository artifacts, supporting strong traceability. The README summarizes multiple external analyses that corroborate its own disclosures about variance, security, and bias, but the underlying articles are not included as source files, and claims about production scale and configuration lack a second supplied source. The document distinguishes intended use, excluded uses, demo configuration, and external criticism; however, evaluative claims such as “fair,” “objective,” and “explainable” are not fully substantiated by the static evidence, preventing full fact-inference separation.
- Resumes contain highly sensitive personal data. Before enabling Gemini, GitHub enrichment, development caching, or CSV export, establish data destinations, lawful basis, access controls, retention limits, and deletion procedures.
- The README reports substantial repeated-score variance and manipulation through invisible PDF text. A single score should not drive automatic rejection or final hiring decisions.
- A public, GitHub-centric rubric may disadvantage candidates whose work is private and may encourage optimization against disclosed rules; human review, bias monitoring, and an appeal path are needed.
- Exact dependency pins do not establish supply-chain safety. Scan for vulnerabilities and verify current model, GitHub API, and provider availability and terms before deployment.
What does this agent do, and when should you use it?
Hiring Agent is a self-hosted Python CLI for parsing, enriching, and evaluating resumes against role-specific rules; it is not a full applicant tracking system. It converts PDFs into Markdown-like text with PyMuPDF, then uses Jinja prompts and an LLM to extract structured Basics, Work, Education, Skills, Projects, and Awards data. When a resume includes a GitHub profile, `github.py` retrieves profile and repository signals and asks the model to select seven qualifying projects. `evaluator.py` applies the rubric in `roles/<role_name>/` and returns category scores, supporting evidence, bonuses, and deductions, while `score.py` orchestrates the pipeline and can cache JSON and append CSV output. It supports local Ollama models and Google Gemini, but the repository explicitly distinguishes its demo defaults from HackerRank's production configuration and places final hiring decisions with human reviewers.
On python score.py <PDF> --role <role_name>, pymupdf_rag.py and pdf.PDFHandler read the resume and produce Markdown-like content. Templates in prompts/templates/*.jinja instruct the model to extract individual sections and assemble the JSONResume schema defined in models.py; transform.py normalizes loose model JSON into a JSON Resume-style structure. If a GitHub username is present, github.py fetches the profile and repositories, classifies projects, and asks the model to select exactly seven distinct projects that meet a minimum author-commit threshold. evaluator.py then uses the selected role's role.json, criteria.jinja, and system_message.jinja to calculate category scores, evidence, bonuses, and deductions. score.py prints the report to standard output and, when DEVELOPMENT_MODE=True, writes intermediate cache files and appends resume_evaluations_<role>.csv.
- A recruiting team receiving tens of thousands of internship applications can rank which resumes humans should read first while retaining human review for most applicants.
- A hiring or governance team can inspect category evidence, bonuses, deductions, and role-specific prompt files when reviewing how a score was produced.
- A team hiring software engineering interns can start with the included
software_engineering_internrubric covering open source, personal projects, production work, and technical skills. - A maintainer evaluating a different position can scaffold a role with
--init-role, then define its categories, weights, score bounds, and evaluation prompts. - A researcher studying scoring behavior across model providers can switch between Ollama and Gemini and use development-mode caches and CSV exports for analysis.
What are this agent's strengths and limitations?
- Each role owns its categories, weights, score bounds, prompts, printed report structure, and CSV columns in a dedicated directory.
- It supports both local Ollama inference and Google Gemini, so the default path can run without a cloud API key.
- Results include category-level evidence, bonuses, and deductions instead of only an opaque total score.
- The CLI connects PDF extraction, structured parsing, GitHub enrichment, evaluation, caching, and CSV export in one pipeline.
- Analyses cited by the project report substantial score variation across repeated runs of the same resume, attributed to LLM nondeterminism.
- The documented coverage identifies invisible text embedded in PDFs as a way to inflate scores, so production adoption needs additional input defenses.
- The included rubric emphasizes GitHub activity and may underrepresent engineers whose contributions are primarily in private enterprise repositories.
- This is not an ATS and does not manage the candidate lifecycle; organizations must supply workflow integration, review procedures, and final decision-making.
- GitHub enrichment and Gemini require network services; Gemini also requires an API key, while local operation requires maintaining an Ollama service and model.
How do you install or deploy this agent?
Install Python 3.11+ and choose either Ollama or Google Gemini as the model backend.
git clone https://github.com/interviewstreet/hiring-agent
cd hiring-agent
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .envFor Ollama, install it, run ollama serve, and pull a model listed in providers.json, for example:
ollama pull gemma4:latestSet the matching DEFAULT_MODEL in .env. For Gemini, set DEFAULT_MODEL to a Gemini model listed in providers.json and provide GEMINI_API_KEY. GITHUB_TOKEN is optional and improves GitHub API rate limits.
How do you use this agent?
Score a resume with the included software engineering intern role:
python score.py ./resume/sample.pdf --role software_engineering_intern--role is required and must name a directory under roles/. To scaffold another role:
python score.py --init-role backend_engineerEdit roles/backend_engineer/role.json, criteria.jinja, and system_message.jinja, then run:
python score.py ./resume/sample.pdf --role backend_engineer--init-role only creates placeholder files; it does not score a resume. Setting DEVELOPMENT_MODE=True in config.py enables intermediate JSON caching and per-role CSV output. If the resume contains a GitHub profile, the run also retrieves GitHub data.
How does this agent compare with similar options?
Compared with a full ATS, Hiring Agent does not track candidates or manage an end-to-end recruiting workflow; it ranks and scores resumes. Compared with HackerRank AI Interviewer (Chakra), Hiring Agent evaluates resume and GitHub evidence, while Chakra automates the first interview round. For inference, Ollama offers a local path without a cloud API key, whereas Gemini requires credentials and network access; the repository also states that HackerRank's actual intern evaluations use a top-tier Gemini model rather than the public demo default.
FAQ
Does it automatically decide whom to hire or reject?
Can the pipeline run entirely on local infrastructure?
gemma4:latest path does not require a cloud API key. GitHub enrichment still uses the network when a GitHub profile is found.Is a GitHub token mandatory?
GITHUB_TOKEN is optional, but it improves GitHub API rate limits. The repository lookup stage runs only when the resume contains a GitHub profile.Are scores deterministic and resistant to resume manipulation?
Does the public configuration reproduce HackerRank's production setup?
gemma4:latest as a demo, while the documentation says actual intern resumes are evaluated with a top-tier Gemini model and that the production configuration is not included.