Auto Video Pipeline with AI Avatar — Case Study

This is the case study for the fully automated video pipeline I built that turns structured game data into finished news-recap videos without any human in the loop. The pipeline ran on a schedule, consumed game events as input, and produced complete narrated videos with an AI talking avatar, automatic subtitles, and AI-generated visuals as output. Spain-born, Paraguay-based, working solo on this build.

What was the problem?

The team behind the game wanted regular news-recap videos covering in-game events — patches, tournaments, balance changes, community highlights. The content was valuable, the audience was hungry for it, and the schedule should have been weekly or better. The problem was that producing a single video manually took a video editor's full day: scripting from the raw data, recording narration, sourcing visuals, editing the timeline, generating subtitles. That cost made the cadence quarterly at best, and quarterly is too slow for game communities that operate on a daily news cycle.

The hypothesis was that every step of that workflow was either deterministic (the data was already structured) or automatable with the AI tooling available in 2023 (script-writing from data, TTS narration, AI-generated visuals, lip-synced avatar frames, automatic subtitling, FFmpeg-driven assembly). If the pipeline could run unattended, the cadence problem solved itself.

The constraints:

• Quality bar. The output had to be watchable. Not Hollywood-quality, but high enough that the audience would actually consume it rather than bounce on the first awkward cut.
• Brand consistency. The talking avatar had to match the game's visual identity, not look like a generic stock AI presenter. That ruled out off-the-shelf avatar tools.
• Unattended reliability. A pipeline that needed a human to babysit failed runs would not solve the cadence problem. Failure isolation and retries had to be first-class.
• Reasonable cost per run. The pipeline had to be cheap enough that weekly cadence was sustainable.

What was the approach?

I designed the pipeline as a sequence of discrete stages, each with a clear input contract and a persistent output artifact. The stages could be run independently, retried in isolation, and inspected at each boundary. That structure was the most important architectural decision in the entire build.

The stages in order:

1. Ingest game data. Pull the structured event data from the game's reporting layer. This was the input contract for everything downstream.
2. Script generation. LLM-driven step that turned the structured data into a narration script. Prompt-engineered to match the show's voice and length budget.
3. Text-to-speech. Render the script to audio. The TTS engine choice was important for the quality bar; a robotic voice would have killed watch time.
4. Avatar frame rendering. Lip-synced frames of the talking avatar matching the TTS audio. The avatar was built from the game's own character assets so it looked native to the game's universe.
5. Visual generation through ComfyUI. Background scenes and B-roll visuals generated from prompts derived from the script, matching the game's art style. The ComfyUI documentation covers the workflow-graph model that made this composable.
6. Subtitle generation. Forced alignment on the TTS audio to produce frame-accurate subtitles in the styling of the brand.
7. Final assembly with FFmpeg. Concatenate, overlay the avatar, mix the audio, burn the subtitles, output the finished MP4.

Every stage wrote its output to a persistent artifact store. If stage five failed because of a model hiccup, stage six could resume from stage five's outputs once they were regenerated. No work was redone unnecessarily.

What is in the stack?

• Python. Orchestration glue. The right call when the pipeline spans data processing, model invocation, and shell-level tooling. Each stage was a Python module with a clean function signature.
• FFmpeg. The video-assembly workhorse. Predictable behavior, exhaustive options, well-understood performance. The FFmpeg documentation is the canonical reference. Higher-level video libraries were considered and rejected — the loss of control was not worth the convenience.
• Text-to-speech engine. Modern neural TTS — naturalistic, prosody-aware, controllable through SSML or equivalent markup. The choice of engine was the difference between a watchable video and an unwatchable one.
• ComfyUI for image generation. The workflow-graph model meant complex image-generation pipelines could be versioned and shared as graph files, not as ad-hoc scripts. Game-style consistent visuals were the goal.
• Custom avatar module. Built around a lip-sync model with the game's character assets as the visual base. Modular so a different presenter could be swapped in without touching the rest of the pipeline.
• Subtitle generation. Forced alignment on the rendered audio to produce frame-accurate subtitles, then burned in with FFmpeg in the brand's styling.
• Job orchestration. A simple sequential runner with per-stage retries and persistent intermediate artifacts. No need for a heavier workflow engine at this scale.

The interesting work in this project was not any single component — it was the discipline of separating stages with persistent intermediate artifacts. That structure made every other operational decision easier.

What was the outcome?

The pipeline shipped and ran on a schedule. It produced complete videos without a human touching the timeline. The cadence problem that motivated the project — quarterly when it should have been weekly — went away because the cost of producing a video collapsed from a workday to a few minutes of compute.

Qualitative observations:

• The avatar's brand consistency was the single most positive piece of feedback. Audiences responded to a presenter that looked like part of the game's universe, not a generic AI presenter dropped in from elsewhere.
• The TTS choice carried more weight than expected. A small improvement in narration naturalness produced a disproportionately large improvement in perceived production quality.
• Per-stage retries paid for themselves the first time a single model invocation timed out — the rest of the pipeline did not have to restart.
• The FFmpeg decision held up well. Higher-level libraries would have hit a wall around the avatar overlay step; FFmpeg handled it directly.

No view-count or engagement metrics go on this page that I cannot independently verify. The honest claim is that the pipeline shipped, ran unattended, and produced watchable videos at a cadence manual production could not match.

What did I learn?

Persistent intermediate artifacts are the load-bearing decision in any automation pipeline. Every stage writes its outputs somewhere durable; every stage can resume from the previous stage's outputs without redoing earlier work. This is boring and unglamorous and it pays for itself the first time a single stage fails.

Stage isolation beats pipeline integration. A monolithic pipeline is easier to write and harder to debug. Per-stage isolation costs a little more upfront and saves a lot over the operational life of the system. Default to isolation.

Quality bar is a real constraint, not a vague aspiration. Specifically: TTS naturalness, visual coherence, subtitle accuracy, audio mixing. A small regression in any of those breaks the watch-time numbers in ways that are hard to recover from. Test against a quality bar with real viewers, not against an internal "this looks fine" feeling.

Brand consistency at the avatar level is worth the engineering investment. Off-the-shelf avatar tools were faster to integrate and produced presenters that did not belong to the game's universe. Building the avatar from the game's own assets cost more upfront and paid for itself in audience reception.

For broader stack reasoning on AI-driven products, see best stack for AI SaaS in 2026. The trade-offs between RAG and other AI patterns are covered in RAG vs fine-tuning for chatbots. The AI automation service page has the scope and pricing band for engagements in this shape; the RAG chatbot service covers a related but different pattern.

Email hello@ignax.dev if you have a structured input source and a repetitive output product — the pipeline pattern probably fits. Repository style at github.com/ignaxdev.