KrillinAI Video Localization
Transcribe, translate, dub, and reformat videos for multilingual publishing across major social platforms.
The README distinguishes local and cloud providers and identifies downloads, external AI calls, generated files, and a dry-run mode; the release workflow limits its declared permission to contents: write, providing moderate transparency about data flow and effects. Deductions apply because the exact payloads sent to providers, retention rules, log redaction, and credential-storage protections are not documented; ordinary execution has no per-effect confirmation, rollback is limited to an implied ability to remove work-directory artifacts, and dependency controls are weakened by master, latest, and major-only action references without an audit or update policy. Attribution names only the Krillin AI team and community channels, not a verified or individually accountable maintainer.
The single-line JSON convention, reusable manifest, and usage/retryable/dependency error classes support automation, while ffmpeg, ffprobe, yt-dlp, and alternative service backends are identified. Deductions apply because pipeline and cover are presented as commands but their Skills are simultaneously described as planning or reserved until execution is fully wired; dependency installation, compatibility, and fallback details are incomplete, and no concrete error-object example is supplied.
The source clearly addresses desktop users, server operators, scripts, and AI Agents, with precise stage commands, inputs, outputs, artifact reuse, and target platforms. Deductions apply because some advertised paths remain incompletely wired, source-language coverage is narrower than translation-language coverage, and disclosed macOS signing, chip, and provider-platform constraints are not accompanied by complete resource or compatibility guidance.
The README is organized around features, provider matrices, quick start, CLI examples, configuration, FAQ, and contact routes; command, artifact, and manifest names are reasonably stable, and desktop bugs, signing issues, and reserved capabilities are disclosed. The complete GPL-3.0 text matches the supplied metadata, justifying full license credit. Deductions apply because installation depends partly on unavailable linked documents and includes a questionable Google-search-style Alibaba link; there is no changelog or compatibility policy, tag-based releases provide only limited versioning evidence, and maintenance ownership is expressed only as a team label and community contacts.
Stage-specific commands, structured JSON, a reusable manifest, intermediate artifacts, and explicit media outputs make results directly usable in automated workflows. Combining acquisition, transcription, translation, dubbing, and horizontal or vertical rendering adds substantial value over a single-stage utility. The cost-benefit score is reduced because the source only broadly contrasts local inference with cloud-service cost and gives no estimates for APIs, compute, storage, downloads, or long-video processing; some end-to-end paths are also described as not fully wired.
Claims are partly traceable to command tables, examples, provider matrices, go.mod, the release workflow, and the license file. Deductions apply because central capability and quality claims are mostly self-reported in the README without tests, benchmarks, or corroboration from the other supplied files, while assertions such as no omissions or overlaps, very high quality, and perfect presentation are not clearly separated from demonstrated facts or inference.
- Before sending media, subtitles, or voice samples to OpenAI, Alibaba Cloud, MiniMax, or compatible providers, independently verify uploaded fields, retention, training use, and logging policies.
- Confirm the actual wiring at this revision before unattended production use of pipeline or cover; the README presents them both as commands and as reserved capabilities that are not fully wired.
- Pin and audit build dependencies and GitHub Actions, especially PairZhu/gpt-translate@master, the latest Docker image, and the latest GoReleaser distribution.
- Independently verify content licenses, likeness and voice authorization, and destination-platform rules when downloading, translating, dubbing, cloning voices, or generating covers.
- The macOS instructions remove quarantine attributes for unsigned binaries; independently verify release provenance and integrity before execution.
What does this agent do, and when should you use it?
KrillinAI is an audio and video localization system for human operators and AI agents, covering acquisition, transcription, subtitle translation, TTS dubbing, horizontal and vertical composition, and cover generation. Human users can run the workflow through a desktop client or web interface, while automation can invoke the staged subtitle, tts, render-horizontal, render-vertical, pipeline, and cover CLI commands. The CLI runs synchronously by default, prints one JSON line to stdout when finished, and writes krillinai_manifest.json so later stages can reuse generated artifacts. It accepts local videos or online sources downloaded through yt-dlp and prepares outputs for platforms including Bilibili, Xiaohongshu, Douyin, WeChat Video, Kuaishou, YouTube, and TikTok. The project supports Windows, Linux, and macOS, offers both local and cloud transcription choices, and can use cloud or locally deployed language models compatible with the OpenAI API format. Adoption requires the relevant model or service credentials plus media-processing and download dependencies.
Input can be a YouTube or Bilibili URL or a local video. The subtitle command first attempts to obtain platform captions and falls back to a Whisper-based transcription service, producing files such as origin_language_srt.srt, target_language_srt.srt, bilingual_srt.srt, and short_origin_mixed_srt.srt; translation uses contextual LLM processing through an OpenAI-compatible API and supports terminology replacement. The tts command reads target-language subtitles and creates tts_final_audio.wav, optionally combining it with video as video_with_tts.mp4; documented providers include Alibaba Cloud Voice Service, OpenAI TTS, and MiniMax TTS, while the feature list also mentions CosyVoice tones and custom voice cloning. render-horizontal combines the source with bilingual subtitles, or a dubbed video with target subtitles. render-vertical converts footage to portrait presentation and applies short subtitles and title layout. pipeline chains selected stages through --outputs, while cover creates generated_cover.png from the original thumbnail and prompt templates. Each stage records paths in outputs, subsequent commands can reuse the manifest by receiving only --workdir, and --dry-run validates parameters without downloading media or calling external AI services.
- A multilingual YouTube or TikTok team localizing existing videos into translated, dubbed channel editions.
- A creator repurposing landscape footage for Douyin, Xiaohongshu, Kuaishou, or WeChat Video with portrait formatting and shorter subtitles.
- A subtitle production team generating source, translated, bilingual, and short-form SRT files from local footage before human review.
- A developer or AI agent building a media automation pipeline around staged commands, structured JSON, and reusable manifest artifacts.
- A self-hosting team controlling transcription costs with FasterWhisper, WhisperKit, or WhisperCpp where the selected platform is supported.
- A content operations team generating platform covers from source thumbnails and prompt templates alongside localized videos.
What are this agent's strengths and limitations?
- One pipeline spans downloading, transcription, translation, dubbing, horizontal and portrait rendering, and cover generation, reducing artifact handoffs between separate applications.
- The CLI separates subtitle, TTS, and rendering stages while exposing JSON, outputs, and a manifest contract suitable for scripts and agent orchestration.
- Transcription choices include OpenAI Whisper, FasterWhisper, WhisperKit, WhisperCpp, and Alibaba Cloud ASR, allowing a choice between local cost control and cloud convenience.
- The language-model layer accepts OpenAI-compatible services, with OpenAI, Gemini, DeepSeek, Tongyi Qianwen, and locally deployed open-source models explicitly listed.
- Desktop, web-server, and CLI delivery modes are available across Windows, Linux, and macOS.
- The full workflow depends on ffmpeg, ffprobe, and yt-dlp; cloud transcription, translation, and TTS also require connectivity, credentials, and potentially paid usage.
- Local transcription options have platform restrictions: FasterWhisper is listed for Windows and Linux, while WhisperKit is limited to M-series Macs.
- The macOS binaries are unsigned and require manual quarantine removal and executable permissions; the desktop edition is also described as new and still receiving bug fixes.
- The non-desktop edition requires manual creation and editing of config.toml, making initial setup more involved than the desktop path.
- The pipeline and cover Agent Skills are described as planning or reserved guides until those execution paths are fully wired, limiting confidence in those skill wrappers.
- The documented configuration requires Alibaba Cloud as the TTS provider for voice cloning, so that capability is not provider-neutral.
How do you install or deploy this agent?
The shortest path is to download the operating-system-specific executable from https://github.com/KrillinAI/KrillinAI/releases and place it in an empty directory. The desktop edition starts directly and is configured in the application. For the non-desktop edition, create config/config.toml, copy the contents of config/config-example.toml into it, enter the required settings, run the executable, and open http://127.0.0.1:8888 unless a different port was configured. The unsigned macOS desktop build requires sudo xattr -cr ./KrillinAI_1.0.0_desktop_macOS_arm64, sudo chmod +x ./KrillinAI_1.0.0_desktop_macOS_arm64, and then ./KrillinAI_1.0.0_desktop_macOS_arm64; the non-desktop build uses sudo xattr -rd com.apple.quarantine instead. Docker deployment is also documented. To build the CLI from source, install Go and run: go build -o build/krillinai-cli ./cmd/cli. Actual processing requires ffmpeg, ffprobe, and yt-dlp; cloud transcription, translation, and TTS also require credentials for the selected providers.
How do you use this agent?
For subtitle-only localization, set provider.name to openai under [transcribe] and supply the OpenAI API key in [llm]. For local transcription, select fasterwhisper, set transcribe.fasterwhisper.model to large-v2, and configure the model used under [llm]. A first CLI invocation is: ./build/krillinai-cli subtitle "https://www.youtube.com/watch?v=dQw4w9WgXcQ" --origin-lang en --target-lang zh_cn --workdir tasks/demo --caption-source any. Generate dubbing with: ./build/krillinai-cli tts --workdir tasks/demo --input-srt tasks/demo/target_language_srt.srt --line-mode target-only --video tasks/demo/origin_video.mp4. Render a horizontal edition with: ./build/krillinai-cli render-horizontal --workdir tasks/demo --video tasks/demo/origin_video.mp4 --subtitle tasks/demo/bilingual_srt.srt. Render a portrait edition with: ./build/krillinai-cli render-vertical --workdir tasks/demo --video tasks/demo/origin_video.mp4 --subtitle tasks/demo/short_origin_mixed_srt.srt --major-title "今日话题" --minor-title "AI Video". Automation should parse the final JSON line on stdout and krillinai_manifest.json rather than plain-text logs. Use --dry-run before contacting real services to validate parameters.
How does this agent compare with similar options?
Compared with cloud-only transcription, FasterWhisper, WhisperKit, and WhisperCpp can run locally and avoid cloud transcription charges, but they differ in platform coverage and local model requirements. OpenAI Whisper is the all-platform cloud option described as fast and effective, while Alibaba Cloud ASR is positioned as a way to avoid network issues in mainland China.