Keeping the same AI face across every frame: everything we learned in production

Generating a single AI image is a solved problem now. The real difficulty is this: keeping the same person the SAME human across days, across hundreds of images and videos. One frame looks great, the next one the eyes drift, the jaw changes, the person turns into their own sibling. After months of producing and shipping a persona, here's what we learned — not the theory, the part that actually works.

One-line summary: consistency doesn't come from writing a better prompt. It comes from taking the face out of the prompt and locking it to a reference.

1 — Lock the face to a reference, not the prompt.

The first beginner mistake is describing the face in words: 'green-eyed, freckled, heart-shaped face, 25-year-old woman.' That never gives you a consistent person — each generation draws a DIFFERENT human matching that description. The fix is to pick one master reference face and lock every image to it (identity transfer via face-id / reference image). The prompt describes the scene, light, outfit; the reference carries the identity.

• ▸Generate one canonical face first, approve it, freeze it. That's your identity anchor.
• ▸Feed that face as a reference into every subsequent image — identity comes from the picture, not the prompt.
• ▸Reference-driven models like nano-banana preserve identity largely on their own; that solves most of the work.

2 — Separate consistency from aesthetics.

Trying to dump two separate problems onto one model was where we wasted the most time. 'Is the face the same' is one job; 'is the image real/beautiful' is another. For a realistic, amateur-selfie aesthetic, SDXL-based models (like RealVisXL) often look more 'human' than FLUX — FLUX is technically clean but too polished, with an 'AI-smelling' skin. Once we put identity preservation in one layer and the look in another, both improved.

“FLUX renders cleaner; but the feel of 'a selfie a real person took' lands more often with SDXL/RealVisXL. Perfect ≠ believable.”

3 — Don't pose to camera.

Even after you solve consistency, the image can still smell 'AI.' The reason is almost always the same: the subject is staring dead-center and smiling, giving a studio pose. Real people don't pose in most photos. Candid framing — a side glance, a half profile, a caught-in-motion moment, standing at the edge of the frame — adds instant realism. We rotate poses with a 'shot beat' system that varies them daily instead of a fixed list, so the feed never goes monotone.

• ▸Don't make every frame look at the camera; vary the angles (side, back, over-the-shoulder).
• ▸Build a stable setting (e.g. the same café) but produce different poses/moments inside it — like a story.
• ▸Instead of a studio smile, aim for a 'caught moment': reaching for a cup, laughing, turning their head.

4 — Negative-prompt and seed discipline.

Two small levers with outsized payoff. A good negative prompt cuts the classic AI-face distortions up front. And the same seed + same reference is the cheapest way to produce 'sibling frames' — you can change the angle of a scene without moving the identity.

negative: deformed face, asymmetric eyes, plastic skin,
          extra fingers, fused fingers, warped jaw,
          over-smoothed skin, uncanny, waxy

seed: keep fixed → sibling frames of the same identity
seed: change   → same person, new scene

5 — In video, the real enemy is face flicker.

Solving consistency in stills is half the battle. The moment you turn a still into video (image-to-video), a new problem appears: the face flickers frame to frame, the model slightly redraws it each frame, and the person 'boils.' The most-tried-and-discarded approach was pure i2v — it can't hold the face stable. Our winning chain: generate the motion separately with a driving video, carry the identity separately, then layer lip-sync as its own step. Separating motion from identity makes the flicker disappear.

• ▸Pure i2v: the face flickers and 'boils' — it gives itself away on a long shot. Discarded.
• ▸Winner: motion via a driving-video → identity stays fixed → a separate lip-sync step.
• ▸Don't 'recolor' the background afterward; shifting color breaks face consistency.

6 — Vary the motion, don't flatten the voice.

If even silent/ambient videos go monotone, the cause is usually repeating a single motion (like constant breathing). Varying motion from a 'beat' pool too — sometimes fixing hair, sometimes turning the head, sometimes laughing — keeps the video alive. In voiced content, a flat/robotic tone is the killer: breathy intonation, a laugh tucked in, and natural pauses are what carry the 'real human' feel.

7 — A persona isn't an image, it's an ongoing story.

This is the highest-leverage and most-skipped thing: individually beautiful images don't build an audience; continuity does. Each post should continue the last one — the same places, an evolving storyline, building on what happened yesterday. We keep this in a 'story state' file so the character never starts from scratch. People don't follow a perfect face, they follow a story worth following.

“An audience follows the ongoing story, not the perfect pixel. Continuity is a stronger glue than resolution.”

8 — Chase the cost out from the start.

Left unchecked, this chain eats money. A few images plus a few seconds of video per day multiplies directly by model choice and step count. We tag every step with 'what does this cost'; generate-and-discard on a cheap model, save the expensive model only for the final frame. Scaling before consistency is solved only multiplies the error expensively.

Summary: lock the face to a reference, separate the look, don't pose to camera, separate motion from identity in video, and weave a story over all of it. Order matters — chasing aesthetics or scale before identity was solved was our most expensive mistake.