Judge0 Online Code Execution System
A robust, fast, scalable, and sandboxed open-source online code execution system for humans and AI.
- Source repo
- judge0/judge0
- Stars
- ★ 4.5k
- Last updated
- 1d ago
- License
- GPL-3.0
- Primary language
- HTML
- FA score
- 41/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free tier plus a paid hosted plan
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- AI developers can use Judge0 to safely execute AI-generated code in a sandboxed environment.
- Not a fit if
- Teams that need a built-in GUI or admin console
- Users who want a fully offline setup without Docker, Redis, or PostgreSQL
- Organizations that cannot promptly patch sandbox escape CVEs
- Source review
- 41/100 · Major gaps 6 safety controls not found
What does this agent do, and when should you use it?
Judge0 is an open-source online code execution system established in August 2016, designed to safely run untrusted code. It offers a simple HTTP JSON API, supports 90+ programming languages, and can execute multi-file projects with custom compiler options, command-line arguments, and configurable time and memory limits. Judge0 can be self-hosted via Docker or used as a managed SaaS, with official Python and JavaScript SDKs. Its scalable architecture is documented in a research paper, and it is widely used to build online judges, competitive programming platforms, and AI agent code execution tools.
Judge0 accepts source code and standard input, compiles and runs it in an isolated sandbox, and returns the execution result including stdout, exit code, and resource usage. Users submit tasks via the HTTP JSON API (e.g., POST to /submissions with language_id, source_code, stdin) or via the official Python SDK. It supports multi-file programs, additional files, custom compiler flags, and command-line arguments, and can notify via webhooks.
- AI developers can use Judge0 to safely execute AI-generated code in a sandboxed environment.
- E-learning platforms can integrate Judge0 to automatically grade students' programming assignments.
- Technical interview platforms can run candidate code in real-time for coding assessments.
- Competitive programming organizers can deploy Judge0 to handle high concurrency during contests.
- Developers can quickly build LeetCode-style platforms with multi-language code execution.
How do you install or deploy this agent?
Quick self-hosting: Ensure Docker is installed, then use the judge0/judge0 image and follow the deployment procedure in the official CHANGELOG. You'll also need Redis and PostgreSQL. See the official documentation for exact steps.
How do you use this agent?
Use the HTTP API: send a POST request to https://ce.judge0.com/submissions?wait=true with a JSON body including language_id, source_code, and stdin. For example: curl -H "Content-Type: application/json" -d '{"language_id": 109, "source_code": "print(f\"hello, {input()}\")", "stdin": "Alice"}' https://ce.judge0.com/submissions?wait=true. Alternatively, use the Python SDK: pip install judge0, then call judge0.run(source_code, stdin, language).
What are this agent's strengths and limitations?
- Supports 90+ programming languages covering most mainstream languages.
- Simple and consistent REST API reduces integration effort.
- Self-hosting option gives full control over data and infrastructure.
- Sandboxed execution safely handles untrusted code.
- Horizontally scalable to handle high loads.
- Self-hosting requires Docker, Redis, PostgreSQL, adding operational complexity.
- Sandboxing is not foolproof; historical CVEs (e.g., CVE-2024-28185) require prompt patching.
- No built-in GUI; management features must be developed separately.
How does this agent compare with similar options?
Compared to E2B and similar AI code sandboxes, Judge0 offers a self-hosting path and broader language support, while E2B provides more AI-native features like prebuilt agents.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Judge0 Online Code Execution System This agent | 41 · Major gaps | Self-hosted serviceFreemium | ★ 4.5k | 1d ago | HTML | — |
| Open Terminal | 50 · Major gaps | Self-hosted serviceFree | ★ 3.4k | today | Python | — |
| OtoDock | 77 · Good | Self-hosted serviceFreemium | ★ 191 | 5d ago | Python | Codex · Claude Code |
| Synapse AI — Multi-Agent Orchestration Platform | 70 · Some gaps | CLIFree + model costs | ★ 329 | 2mo ago | Python | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows the project claims sandboxed execution of untrusted code, but no implementation details or least-privilege measures are provided, so score 1. User confirmation mechanism not mentioned, data flow transparency not explained, sensitive data handling not addressed, dependency security not assessed, external effects not described, rollback not mentioned. Source attribution: README lists authors and contributors, but identity not verified, so score 1.
Self-consistency: README description aligns with feature list, but no test evidence, so score 2. Dependency availability: no dependency list or version pinning, so score 1. Failure messages: API docs may include error info, but not evident in provided files, so score 1.
Audience and scenarios clearly defined, including AI agents, education platforms, etc., so score 2. Capability boundaries: lists supported languages and features, but no explicit limitations, so score 2. Trigger precision: API call examples clear, but trigger conditions not detailed, so score 1. Environment fit: supports self-hosting and cloud, so score 2.
Information architecture clear with TOC and doc links, so score 2. Install notes provide multiple options, so score 2. Naming stability: API version stability not mentioned, so score 1. Examples and FAQ: API and SDK examples provided, so score 2. Known limitations not explicitly listed, so score 1. License is GPL-3.0, so score 2. Versioning and changelog linked, so score 2. Maintenance responsibility: security policy and contact provided, so score 2.
Output usability: API returns detailed results, so score 2. Marginal value: multiple integration methods, so score 2. Cost-benefit: open-source and self-hostable, so score 2.
Claim traceability: README cites research paper and docs, so score 2. Cross-source corroboration: many academic citations, so score 2. Fact-inference separation: some claims like 'robust' and 'fast' lack specific data, so score 1.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Sandbox implementation details not provided; review code to confirm least privilege.
- User confirmation mechanism not mentioned; code execution may be automatic.
- Dependency security not assessed; check dependency list and vulnerabilities.
- Rollback mechanism not provided; upgrade failures may impact service.
- Publisher identity not verified; proceed with caution.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v1.13.1 Judge0 CE v1.13.1
Release notes excerpt
v1.13.1 (2024-04-18) Security researcher [Daniel Cooper](https://www.linkedin.com/in/daniel-cooper4) (@stacksparrow4) found three critical vulnerabilities in Judge0 versions <= 1.13.0, which this release fixes. Thank you, Daniel, for your contribution. Security Fixes This release fixes the following security vulnerabilities: - [CVE-2024-28185](https://github.com/judge0/judge0/security/advisories/GHSA-h9g2-45c8-89cf) - [CVE-2024-28189](https://github.com/judge0/judge0/security/advisories/GHSA-3xpw-36v7-2cmg) …View on GitHub ↗ -
v1.13.1-extra Judge0 Extra CE v1.13.1
Release notes excerpt
v1.13.1-extra (2024-04-18) Security researcher [Daniel Cooper](https://www.linkedin.com/in/daniel-cooper4) (@stacksparrow4) found three critical vulnerabilities in Judge0 versions <= 1.13.0-extra, which this release fixes. Thank you, Daniel, for your contribution. Security Fixes This release fixes the following security vulnerabilities: - [CVE-2024-28185](https://github.com/judge0/judge0/security/advisories/GHSA-h9g2-45c8-89cf) - [CVE-2024-28189](https://github.com/judge0/judge0/security/advisories/GHSA-3xpw-36v7-2cmg) …View on GitHub ↗ -
v1.13.0 Judge0 CE v1.13.0
Release notes excerpt
v1.13.0 (2021-03-10) Huge thanks to [Filtered](https://www.filtered.ai) for sponsoring this release. New Features - Added support for enable_network configuration flag. With the new ALLOW_ENABLE_NETWORK configuration variable, usage of this flag can be permitted or denied. Furthermore, with the new ENABLE_NETWORK configuration variable the default value of this flag can be set for every submission. - Commits: [@62a00520](https://github.com/judge0/judge0/commit/62a00520692c2b7a8bf0f54528aee8fbbf864f36) …View on GitHub ↗