How my pipeline picks b-roll and draws diagrams from a script
# How my pipeline picks b-roll and draws diagrams from a script
The worst thing this system ever shipped was a green rectangle. A visual step failed silently mid render, and the finished video played a blank green slide for twenty full seconds while the narration carried on underneath, perfectly happy. Nothing errored. No log complained. I only caught it by watching the file before it went out.
That bug is a good place to start, because everything interesting about automating the visual side of a faceless video lives in moments like that one. The footage, the animated diagrams and the captions in my videos are all chosen and placed by a program, assembled from nothing except the words of the script. People ask about this part of the pipeline more than any other, and I think that is because it is the part that separates a video that looks made from a video that looks cheap. So here is how the visual track actually gets built, plus the failures that shaped it.
Why the visuals carry the whole video
A faceless video is narration with pictures on top. There is no presenter, no face holding your attention, so the picture track does that work alone. The trap that catches almost everyone early is pasting the words of the script on screen as static text while the voiceover reads them out. It looks lazy within seconds, and viewers leave. A wall of your own narration sitting motionless on screen is the fastest known way to lose someone.
The bar I hold the system to is simple. Every moment on screen should be worth looking at, and the picture should change before the eye gets bored. Everything below exists to clear that bar.
Three layers on screen
At any given moment the frame is built from three layers. B-roll sits underneath: real stock footage of the world, matched to whatever I am talking about. Diagrams sit above it: small animated tables, flows and comparisons drawn on the fly. Captions run along the bottom edge, following the narration word for word, for everyone watching on mute, which on social platforms is most viewers.
Each layer has a job. Footage carries feeling. Diagrams carry structure. Captions carry the exact words. The system decides, paragraph by paragraph, which layer should lead, based on what that paragraph is doing.
Matching footage to the meaning of a line
For every paragraph, the system reads what I am saying and writes a short scene description, the kind of phrase you would type into a stock library yourself. Talking about a data center gets a data center. Talking about someone working late gets exactly that: a person at a laptop, tired, bad lighting. The library returns candidate clips, the best one gets trimmed to the length of the sentence, and it lands at the right moment.
The detail that keeps this from looking random is where the search phrase comes from. A lazy version grabs the first noun in the sentence and searches for that, and the results are nonsense. The version that works describes the scene the sentence is painting, so the footage matches the feeling of the line rather than one keyword plucked out of it. It still misses sometimes. When it lands, it looks deliberate.
Take a single line as an example. Say the script reads, most engineers hit the same wall when their setup grows. The system decides that line is a feeling rather than a structure, so it reaches for footage. It writes a phrase like a frustrated developer at a cluttered desk late at night, pulls a few matches, trims the best one, and pins the clip to the words the same wall so it appears exactly as I say them. The very next paragraph might list four fixes, and now the system sees a shape instead of a feeling, so it draws an animated list that builds one item at a time. Two adjacent lines, two completely different visual decisions, both made from nothing except the words.
Diagrams drawn from structure
The diagrams are the part I am proudest of, because nothing about them comes off a shelf. A lot of spoken explanation has a hidden shape. A comparison between two things. A process with steps. A short list of options. A few numbers worth seeing side by side. The system checks each paragraph for one of those shapes, and when it finds one it builds the matching visual: a clean two column table for the comparison, a flow that reveals each step as I describe it, a list that builds in as I go.
Because the content of each diagram comes straight out of my own words, the picture carries the actual point being made at that moment. That is the difference between an explainer and a slideshow with clip art on it.
There is a real cost here. Each diagram is rendered frame by frame into a small video clip, and one video can need a dozen or more of them. I draw them locally, on my own graphics card, because paying a service per render across that many clips and that many videos would add up fast. Doing it on hardware I already own makes each diagram cost nothing beyond a few minutes of machine time. The trade is speed: the diagram pass is the slowest single step in the whole pipeline, sometimes the difference between a render finishing in five minutes and one dragging on for twenty. I consider that a fair price for visuals built to the content, and it is a genuine price, worth knowing before you build your own version.
Captions from the script, never from the audio
Most tools caption a video by listening to the audio and guessing at the words. That works until a product name or an unusual technical term shows up, at which point the tool writes the wrong thing with total confidence. Automated videos are full of exactly this kind of embarrassing caption error.
My captions skip the guessing entirely. The whole video is generated from a script I wrote, so the caption text is lifted straight from that script, spelled the way I spelled it. The audio contributes only timing. The system lines the script up against the voiceover, finds the moment each word is spoken, and places the caption there. Tricky names come out right every time, because the system never had to hear them. It already knew them.
One design choice, one entire category of error gone.
Timing is the trick underneath everything
A good visual at the wrong moment reads worse than no visual at all. Every cue in the visual track is anchored to a short phrase of narration. When the voiceover is generated, the system finds the exact instant those anchor words are spoken and fires the visual right there. The data center clip appears as I say data center, never five seconds early. The third step of a flow appears as I describe the third step. Because every cue pins to the actual audio, nothing drifts out of sync, however the narration timing shifts.
When the system gets this right, you do not notice it at all, which is the entire point. Good timing is invisible.
Pacing, which a number cannot fully capture
Pacing took me the longest to get right. Change shots too slowly and a single clip sits on screen until attention drifts and the whole video feels sluggish. Change too fast and the frame flickers, and nobody can actually read the diagrams. My working rhythm is a fresh visual every several seconds, with diagrams held a little longer because reading takes time.
The system spaces cues to that rhythm automatically. I still watch every finished video to feel it, because a number gets you close and your eye tells you the truth. You know pacing is right when you stop noticing the cuts and just follow the idea.
The self checks that caught my worst bugs
Which brings me back to the green rectangle.
Early versions failed in ways I only found funny later. A search phrase would return footage related to a keyword and wrong in spirit. A diagram would cram in so much text that nobody could read it in the seconds it stayed on screen. And a failed visual could leave that blank green slide playing while the narration kept going. Every one of those failures earned a permanent check. The finished video now gets inspected by the same system that made it: frames get sampled and examined for the tells. A frame that is mostly flat green. A long stretch where nothing on screen has changed. Footage that sits badly against its line. Anything suspicious gets flagged to me before publish.
That habit of self inspection, more than any clever generation trick, turned this from a fragile toy into a system I trust to run.
If you build your own
The most important lesson I can pass on has nothing to do with a specific tool. Make the script the single source of truth and grow every layer out of it. The footage, the diagram content, the caption text and the cue timing should all trace back to the same words. Any layer that drifts off to live on its own is where the desync starts, and desync is what makes automated video look automated in the bad way.
The second lesson: make the system check its own output. Across dozens of videos you will never catch everything by hand, and a tool that tells you loudly when it failed is worth ten of one that ships broken work in silence.
Third, build one layer at a time. Get footage matching solid before you touch diagrams. Get diagrams solid before you obsess over pacing. Each layer is honest work on its own, and stacking three shaky ones gives you a system that fails in three directions at once.
From the outside, the finished product looks like someone spent an afternoon dragging clips around a timeline. Inside, it is one script, three layers grown out of it, and a set of checks watching the result. The machine does the assembly. I do the judging. So far, that split holds.
Get new guides and videos first — join the Telegram channel.