KrillinAI Video Localization
Transcribe, translate, dub, and reformat videos for multilingual publishing across major social platforms.
- Source repo
- krillinai/OpenCreator
- Stars
- ★ 14k
- Last updated
- today
- License
- Apache-2.0
- Primary language
- TypeScript
- FA score
- 62/100 · Some gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A multilingual YouTube or TikTok team localizing existing videos into translated, dubbed channel editions.
- Not a fit if
- Users unwilling to hand-edit config.toml: the non-desktop edition requires manual setup
- Intel Mac users: WhisperKit local transcription is M-series only
- Source review
- 62/100 · Some gaps
What does this agent do, and when should you use it?
KrillinAI is an audio and video localization system for human operators and AI agents, covering acquisition, transcription, subtitle translation, TTS dubbing, horizontal and vertical composition, and cover generation. Human users can run the workflow through a desktop client or web interface, while automation can invoke the staged subtitle, tts, render-horizontal, render-vertical, pipeline, and cover CLI commands. The CLI runs synchronously by default, prints one JSON line to stdout when finished, and writes krillinai_manifest.json so later stages can reuse generated artifacts. It accepts local videos or online sources downloaded through yt-dlp and prepares outputs for platforms including Bilibili, Xiaohongshu, Douyin, WeChat Video, Kuaishou, YouTube, and TikTok. The project supports Windows, Linux, and macOS, offers both local and cloud transcription choices, and can use cloud or locally deployed language models compatible with the OpenAI API format. Adoption requires the relevant model or service credentials plus media-processing and download dependencies.
Input can be a YouTube or Bilibili URL or a local video. The subtitle command first attempts to obtain platform captions and falls back to a Whisper-based transcription service, producing files such as origin_language_srt.srt, target_language_srt.srt, bilingual_srt.srt, and short_origin_mixed_srt.srt; translation uses contextual LLM processing through an OpenAI-compatible API and supports terminology replacement. The tts command reads target-language subtitles and creates tts_final_audio.wav, optionally combining it with video as video_with_tts.mp4; documented providers include Alibaba Cloud Voice Service, OpenAI TTS, and MiniMax TTS, while the feature list also mentions CosyVoice tones and custom voice cloning. render-horizontal combines the source with bilingual subtitles, or a dubbed video with target subtitles. render-vertical converts footage to portrait presentation and applies short subtitles and title layout. pipeline chains selected stages through --outputs, while cover creates generated_cover.png from the original thumbnail and prompt templates. Each stage records paths in outputs, subsequent commands can reuse the manifest by receiving only --workdir, and --dry-run validates parameters without downloading media or calling external AI services.
- A multilingual YouTube or TikTok team localizing existing videos into translated, dubbed channel editions.
- A creator repurposing landscape footage for Douyin, Xiaohongshu, Kuaishou, or WeChat Video with portrait formatting and shorter subtitles.
- A subtitle production team generating source, translated, bilingual, and short-form SRT files from local footage before human review.
- A developer or AI agent building a media automation pipeline around staged commands, structured JSON, and reusable manifest artifacts.
- A self-hosting team controlling transcription costs with FasterWhisper, WhisperKit, or WhisperCpp where the selected platform is supported.
- A content operations team generating platform covers from source thumbnails and prompt templates alongside localized videos.
How do you install or deploy this agent?
The shortest path is to download the operating-system-specific executable from https://github.com/KrillinAI/KrillinAI/releases and place it in an empty directory. The desktop edition starts directly and is configured in the application. For the non-desktop edition, create config/config.toml, copy the contents of config/config-example.toml into it, enter the required settings, run the executable, and open http://127.0.0.1:8888 unless a different port was configured. The unsigned macOS desktop build requires sudo xattr -cr ./KrillinAI_1.0.0_desktop_macOS_arm64, sudo chmod +x ./KrillinAI_1.0.0_desktop_macOS_arm64, and then ./KrillinAI_1.0.0_desktop_macOS_arm64; the non-desktop build uses sudo xattr -rd com.apple.quarantine instead. Docker deployment is also documented. To build the CLI from source, install Go and run: go build -o build/krillinai-cli ./cmd/cli. Actual processing requires ffmpeg, ffprobe, and yt-dlp; cloud transcription, translation, and TTS also require credentials for the selected providers.
How do you use this agent?
For subtitle-only localization, set provider.name to openai under [transcribe] and supply the OpenAI API key in [llm]. For local transcription, select fasterwhisper, set transcribe.fasterwhisper.model to large-v2, and configure the model used under [llm]. A first CLI invocation is: ./build/krillinai-cli subtitle "https://www.youtube.com/watch?v=dQw4w9WgXcQ" --origin-lang en --target-lang zh_cn --workdir tasks/demo --caption-source any. Generate dubbing with: ./build/krillinai-cli tts --workdir tasks/demo --input-srt tasks/demo/target_language_srt.srt --line-mode target-only --video tasks/demo/origin_video.mp4. Render a horizontal edition with: ./build/krillinai-cli render-horizontal --workdir tasks/demo --video tasks/demo/origin_video.mp4 --subtitle tasks/demo/bilingual_srt.srt. Render a portrait edition with: ./build/krillinai-cli render-vertical --workdir tasks/demo --video tasks/demo/origin_video.mp4 --subtitle tasks/demo/short_origin_mixed_srt.srt --major-title "今日话题" --minor-title "AI Video". Automation should parse the final JSON line on stdout and krillinai_manifest.json rather than plain-text logs. Use --dry-run before contacting real services to validate parameters.
What are this agent's strengths and limitations?
- One pipeline spans downloading, transcription, translation, dubbing, horizontal and portrait rendering, and cover generation, reducing artifact handoffs between separate applications.
- The CLI separates subtitle, TTS, and rendering stages while exposing JSON, outputs, and a manifest contract suitable for scripts and agent orchestration.
- Transcription choices include OpenAI Whisper, FasterWhisper, WhisperKit, WhisperCpp, and Alibaba Cloud ASR, allowing a choice between local cost control and cloud convenience.
- The language-model layer accepts OpenAI-compatible services, with OpenAI, Gemini, DeepSeek, Tongyi Qianwen, and locally deployed open-source models explicitly listed.
- Desktop, web-server, and CLI delivery modes are available across Windows, Linux, and macOS.
- The full workflow depends on ffmpeg, ffprobe, and yt-dlp; cloud transcription, translation, and TTS also require connectivity, credentials, and potentially paid usage.
- Local transcription options have platform restrictions: FasterWhisper is listed for Windows and Linux, while WhisperKit is limited to M-series Macs.
- The macOS binaries are unsigned and require manual quarantine removal and executable permissions; the desktop edition is also described as new and still receiving bug fixes.
- The non-desktop edition requires manual creation and editing of config.toml, making initial setup more involved than the desktop path.
- The pipeline and cover Agent Skills are described as planning or reserved guides until those execution paths are fully wired, limiting confidence in those skill wrappers.
- The documented configuration requires Alibaba Cloud as the TTS provider for voice cloning, so that capability is not provider-neutral.
How does this agent compare with similar options?
Compared with cloud-only transcription, FasterWhisper, WhisperKit, and WhisperCpp can run locally and avoid cloud transcription charges, but they differ in platform coverage and local model requirements. OpenAI Whisper is the all-platform cloud option described as fast and effective, while Alibaba Cloud ASR is positioned as a way to avoid network issues in mainland China.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| KrillinAI Video Localization This agent | 62 · Some gaps | Desktop appFree + model costs | ★ 14k | today | TypeScript | OpenAI API |
| Video Use | 54 · Major gaps | Agent plugin / skillFree + model costs | ★ 29k | 10d ago | Python | Codex · Claude Code |
| OH Story — All-in-One Skill Pack for Chinese Web Fiction | 75 · Good | Agent plugin / skillFree + model costs | ★ 7.7k | 1d ago | Python | Codex · Claude Code |
| ShortGPT Video Automation | 0 · Major gaps | Self-hosted serviceFree + model costs | ★ 8k | 1y ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
The README distinguishes local and cloud providers and identifies downloads, external AI calls, generated files, and a dry-run mode; the release workflow limits its declared permission to contents: write, providing moderate transparency about data flow and effects. Deductions apply because the exact payloads sent to providers, retention rules, log redaction, and credential-storage protections are not documented; ordinary execution has no per-effect confirmation, rollback is limited to an implied ability to remove work-directory artifacts, and dependency controls are weakened by master, latest, and major-only action references without an audit or update policy. Attribution names only the Krillin AI team and community channels, not a verified or individually accountable maintainer.
The single-line JSON convention, reusable manifest, and usage/retryable/dependency error classes support automation, while ffmpeg, ffprobe, yt-dlp, and alternative service backends are identified. Deductions apply because pipeline and cover are presented as commands but their Skills are simultaneously described as planning or reserved until execution is fully wired; dependency installation, compatibility, and fallback details are incomplete, and no concrete error-object example is supplied.
The source clearly addresses desktop users, server operators, scripts, and AI Agents, with precise stage commands, inputs, outputs, artifact reuse, and target platforms. Deductions apply because some advertised paths remain incompletely wired, source-language coverage is narrower than translation-language coverage, and disclosed macOS signing, chip, and provider-platform constraints are not accompanied by complete resource or compatibility guidance.
The README is organized around features, provider matrices, quick start, CLI examples, configuration, FAQ, and contact routes; command, artifact, and manifest names are reasonably stable, and desktop bugs, signing issues, and reserved capabilities are disclosed. The complete GPL-3.0 text matches the supplied metadata, justifying full license credit. Deductions apply because installation depends partly on unavailable linked documents and includes a questionable Google-search-style Alibaba link; there is no changelog or compatibility policy, tag-based releases provide only limited versioning evidence, and maintenance ownership is expressed only as a team label and community contacts.
Stage-specific commands, structured JSON, a reusable manifest, intermediate artifacts, and explicit media outputs make results directly usable in automated workflows. Combining acquisition, transcription, translation, dubbing, and horizontal or vertical rendering adds substantial value over a single-stage utility. The cost-benefit score is reduced because the source only broadly contrasts local inference with cloud-service cost and gives no estimates for APIs, compute, storage, downloads, or long-video processing; some end-to-end paths are also described as not fully wired.
Claims are partly traceable to command tables, examples, provider matrices, go.mod, the release workflow, and the license file. Deductions apply because central capability and quality claims are mostly self-reported in the README without tests, benchmarks, or corroboration from the other supplied files, while assertions such as no omissions or overlaps, very high quality, and perfect presentation are not clearly separated from demonstrated facts or inference.
- Before sending media, subtitles, or voice samples to OpenAI, Alibaba Cloud, MiniMax, or compatible providers, independently verify uploaded fields, retention, training use, and logging policies.
- Confirm the actual wiring at this revision before unattended production use of pipeline or cover; the README presents them both as commands and as reserved capabilities that are not fully wired.
- Pin and audit build dependencies and GitHub Actions, especially PairZhu/gpt-translate@master, the latest Docker image, and the latest GoReleaser distribution.
- Independently verify content licenses, likeness and voice authorization, and destination-platform rules when downloading, translating, dubbing, cloning voices, or generating covers.
- The macOS instructions remove quarantine attributes for unsigned binaries; independently verify release provenance and integrity before execution.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v3.2.8 OpenCreator v3.2.8
Release notes excerpt
OpenCreator v3.2.8 What's New - Expanded subtitle translation languages with searchable native and English names, source subtitle language detection, and transcription model compatibility guidance. - Improved multilingual subtitle layout with bundled fallback fonts, safer handling of complex scripts, and more reliable sentence boundaries and subtitle coverage. - Added more detailed transcription, translation, dubbing, and rendering progress, including completed and total item counts. …View on GitHub ↗ -
v3.2.7 OpenCreator v3.2.7
Release notes excerpt
OpenCreator v3.2.7 What's New - Expanded official text, image, voiceover, transcription, and video services with recommended models and pricing from the gateway model catalog. - Standardized credit pricing and whole-number balance displays, with clear billing units for tokens, images, video seconds, and transcription minutes. - Accelerated account state restoration at startup and improved gateway migration, authorization callbacks, and top-up confirmation. …View on GitHub ↗ -
v3.2.6 OpenCreator v3.2.6
Release notes excerpt
OpenCreator v3.2.6 What's New - Connect an OpenCreator account to use official model and media services, view credit balances, and manage account settings. - Simplify the personal center and open the website for subscriptions and credit purchases, with payment status refreshed when returning to the app. - Show sign-in status promptly after browser authorization and allow signing in or out while tasks are pending. - Connect YouTube through an existing Chrome profile and save its login session for video downloads and translation. …View on GitHub ↗