LatentSync 1.6 needs 18 GB VRAM — that's the broadcast bar

Why LatentSync 1.6 Draws the Broadcast Line at 18 GB

LatentSync 1.6 draws the broadcast line at roughly 18 GB of VRAM because it raised training and inference resolution to 512×512, up from the ~256-pixel face region common to earlier latent-space lip-sync methods. That single resolution jump is the main reason memory nearly doubled from v1.5's ~8 GB to v1.6's ~18 GB . LatentSync 1.6 shipped on June 11, 2025 under Apache-2.0, and the resolution upgrade was aimed squarely at the softness that plagues diffusion-based lip sync at 256px .

Quick Answer: LatentSync 1.6, ByteDance's Apache-2.0 lip-sync model released June 11, 2025, needs about 18 GB of VRAM — up from ~8 GB in v1.5 — because it moved to 512×512 resolution to cut the blurriness of 256px latent-space methods. An RTX 4090 or A10-class GPU is the documented comfortable-throughput entry point.

The architecture explains why the memory footprint is steep. LatentSync is an audio-conditioned latent-diffusion model built on a Stable Diffusion U-Net, with no intermediate motion representation. Whisper converts mel-spectrograms into audio embeddings that are injected into the U-Net via cross-attention, while reference frames and masked frames are concatenated directly to the noised latents . Each of those parallel streams — the noised latent, the reference, the mask, and the audio conditioning — occupies VRAM simultaneously during a forward pass, so doubling the spatial resolution compounds across every tensor at once.

Version 1.5 laid the groundwork before the resolution bump: it added a temporal-consistency layer, improved Chinese-language performance, and trained with TREPA, LPIPS and SyncNet losses to hold identity and sync steady across frames . As the project frames the tradeoff:

"512×512 training resolution was introduced specifically to reduce the blurriness of latent-space generation" — LatentSync project documentation (source: bytedance/LatentSync).

For a developer scoping hardware, the practical line is throughput, not just fitting the weights in memory. Reviewers document the RTX 4090 / A10-class GPU as the comfortable entry point for LatentSync 1.6; configurations below 18 GB will complete inference, but at speeds community benchmarks describe as production-prohibitive . That is the real meaning of "broadcast bar" here — not a capability gate but a cost gate. If your pipeline can afford a 24 GB card, LatentSync 1.6 gives you the highest-fidelity open-source re-dub available. If it can't, the sections that follow explain where MuseTalk and MOVA trade resolution or generation model for a lighter, or heavier, footprint.

LatentSync vs. MuseTalk vs. MOVA: VRAM, fps, and Resolution Side by Side

No single open-source lip-sync model wins every column, which is the trade the rest of this guide resolves. LatentSync 1.6 leads on visual fidelity at 512×512 but asks for roughly 18 GB VRAM ; MuseTalk 1.5 leads on throughput, clearing 30 fps on a Tesla V100 inside 8 GB ; and MOVA leads on sync coherence for footage it synthesizes from scratch, at a far heavier hardware cost . Match the column that matters to your shot, not the model with the loudest headline.

Wav2Lip anchors the low end as the zero-shot baseline: its SyncNet discriminator locks mouth timing to phonemes for any speaker with no per-subject training, and it runs on a mid-range card — at the cost of soft, low-resolution mouths . The three columns that decide most pipelines are approach type and VRAM. Re-dub models (Wav2Lip, MuseTalk, LatentSync) edit an existing clip's mouth region, so length is effectively unbounded; MOVA generates picture and synchronized speech in one pass but caps out at 8-second, ≤720p clips . That distinction — editing footage you already have versus synthesizing new footage — matters more than any single fps number, and the next three sections break each option down.

MuseTalk 1.5: Why 30 fps and 8 GB VRAM Come With a 256px Tradeoff

MuseTalk 1.5 hits 30 fps because it is single-step latent-space inpainting, not iterative diffusion. Where LatentSync runs repeated denoising passes per frame, MuseTalk does one forward pass per frame — a single inpainting operation over the masked mouth region — which is what lets it clear 30 fps on a Tesla V100 . That architectural choice is the entire reason it fits comfortably in the ~8 GB VRAM tier and runs on hardware LatentSync 1.6 cannot touch.

Tencent Lyra Lab shipped MuseTalk 1.5 on March 28, 2025, and the update was targeted, not cosmetic . It added perceptual, GAN, and sync losses on top of a two-stage training process with a spatio-temporal sampling strategy . Those additions map directly to the two weaknesses reviewers flagged in earlier builds: temporal coherence between frames and lip-to-audio correspondence. The sync loss tightens phoneme timing; the GAN and perceptual losses sharpen the generated mouth; the spatio-temporal sampling reduces the frame-to-frame instability that single-frame generation tends to introduce.

The cost is resolution. MuseTalk generates on a 256×256 face region, and that ceiling is where broadcast delivery starts to strain . Mustache structure, precise lip shape, and lip-color preservation degrade visibly when the output is scaled to 1080p or 4K timelines . Jitter from single-frame generation is the other documented artifact. The standard workaround is a super-resolution or face-restore pass — GFPGAN is the commonly cited choice — run after lip-sync to rebuild facial detail . That recovers a lot, but it adds a pipeline stage, more compute, and another place for identity drift to creep in. If you are comparing raw model output, MuseTalk trades detail for speed; if you are comparing finished deliverables, you are really comparing MuseTalk-plus-restoration.

Licensing is where MuseTalk pulls ahead for production teams. It ships under permissive MIT terms — the most commercially friendly of the three options here — versus LatentSync's Apache-2.0 . For a studio wiring lip-sync into a dubbing pipeline, MIT means integration without the legal-review overhead that more restrictive terms can trigger. Combined with the 8 GB VRAM floor and 30 fps throughput, that makes MuseTalk the pragmatic default for high-volume, cost-sensitive dubbing — provided you budget for the restoration step that broadcast resolution demands.

MOVA: Generating Speech and Footage Together, No Cascade

MOVA (MOSS Video and Audio) is a joint audio-visual foundation model that generates video and its synchronized speech in a single inference pass, rather than dubbing existing footage. Released January 29, 2026 under Apache-2.0 by the OpenMOSS team with MOSI, it targets a different problem than LatentSync and MuseTalk: it synthesizes new content from scratch, so there is no source video to re-sync. That single-pass design eliminates the error accumulation that compounds in cascaded pipelines, where a separate audio model feeds a separate video model sequentially and each stage inherits the previous stage's drift .

Architecturally, MOVA uses an asymmetric dual-tower design fused by bidirectional cross-attention, with an "Aligned RoPE" mechanism handling modality alignment between the picture and speech streams. The backbone is a Mixture-of-Experts model reported at roughly 32B total parameters with about 18B active, trained in three stages . It ships in two variants — MOVA-360p and MOVA-720p — on Hugging Face, with Day-0 SGLang inference support at launch . A companion paper, MOVA: Towards Scalable and Synchronized Video-Audio Generation (arXiv:2602.08794), was listed as forthcoming at release .

The practical ceiling is duration. MOVA generates clips up to 8 seconds long , which frames it as a shot-generation tool rather than a long-form pipeline. For a talking-head sequence longer than a single line of dialogue, you are stitching multiple generations, and identity consistency across those cuts is not something a per-clip generator guarantees.

"MOVA generates video and audio jointly in one pass, avoiding the error accumulation of cascaded generate-then-dub pipelines," — the OpenMOSS team, on the MOVA release (source: OpenMOSS/MOVA).

The distinction matters for tool selection. MOVA is not a re-dubbing option: if you already have footage — an interview, a shot film, an existing avatar clip — and want the mouth to match a new audio track, MuseTalk or LatentSync remain the applicable choices. MOVA competes only when you are creating the visual and the voice together from a prompt. That makes it a complement to, not a replacement for, the face-region editors, and it explains why teams shipping the best dubbed content in 2026 route each shot to the model built for that shot's problem .

VRAM by Approach, and Where to Offload

VRAM is the single hardest constraint that decides which lip-sync approach you can actually run, and the four leading options span a wide band. Wav2Lip runs on a mid-range consumer card; MuseTalk 1.5 and LatentSync v1.5 both sit near 8 GB; LatentSync v1.6 raised that to roughly 18 GB when it moved to 512×512 training; and MOVA is heaviest by far. OpenMOSS documents MOVA at about 48 GB VRAM with component-wise offload, dropping to ~12 GB with layerwise offload, tested on RTX 4090 and H100. The offload tradeoff is throughput: less VRAM means more weight shuffling per step.

If local hardware is the blocker, hosted playgrounds cover the two face-region editors. LatentSync and MuseTalk both run on Fal, Sieve, Replicate and Hugging Face Spaces, so you can test either without provisioning a GPU [hosted paths]. MOVA has no announced hosted inference path as of July 2026, which leaves local execution or a private cloud instance as the only routes — a meaningful gate for teams without in-house H100 or 4090 capacity.

The practical read: match the card to the job. A 24 GB 4090 clears every re-dubbing workflow and reaches MOVA only with offload penalties, while true no-compromise MOVA generation still points at datacenter-class silicon.

The Limitations No 2026 Lip Sync Approach Has Closed

No open-source lip-sync approach in 2026 has closed the core quality gaps, and matching the card to the job does not remove the need for human finishing. Each of the three leading paths — LatentSync's latent re-dubbing, MuseTalk's single-step inpainting, and MOVA's joint generation — trades one hard limitation for another, and none delivers untouched broadcast masters end to end.

LatentSync's latent-space pipeline still produces subtle blurriness and fine-detail loss even after 1.6 raised training resolution to 512×512 . The common remedy is a super-resolution or face-restore pass with a model such as GFPGAN, which recovers sharpness but adds a stage to the pipeline and compounds latency on every frame . That extra step is manageable for offline work and awkward for anything near real time.

MuseTalk trades depth for speed the other way. Its single-frame generation introduces per-frame jitter, and the 256×256 face-region ceiling means soft mouths on broadcast-resolution footage no matter how much post-processing you apply — upscaling a 256px source cannot invent lip, mustache, or color detail the model never rendered . GFPGAN helps, but it restores faces generically rather than reconstructing the original talent's exact mouth.

MOVA sidesteps both by generating audio and picture together, yet its 8-second clip maximum makes it unusable for long-form dubbing without external segmentation and stitching logic . Once you cut a scene into 8-second pieces and rejoin them, identity consistency across the seams becomes an open problem — face, lighting, and voice timbre can drift shot to shot.

Three failure modes remain unresolved across all of them:

• Cross-lingual robustness — phoneme coverage and mouth shapes degrade outside the languages each model was tuned on.
• Dental interior clarity — teeth and tongue texture stay mushy under motion, the most visible tell on close-ups.
• Long-take identity stability — faces drift over extended takes, whether from temporal accumulation or stitched segments.

The practical consequence is that no single entry wins on every axis. Teams shipping the best dubbed content in 2026 route each shot to the best-fit model and human-finish the deliverable rather than trusting one pipeline end to end .

MOVA's LSE Scores and Arena ELO: Unconfirmed by Peer Review

MOVA's headline lip-sync numbers are promising but self-reported, so treat them as vendor claims until an outside lab reproduces them. MOVA-720p lists an LSE-D of 7.094 and an LSE-C of 7.452 with Dual CFG, an arena ELO near 1113.8, and win rates above 70% against models including LTX-2 and OVI . Every one of those figures traces back to OpenMOSS release materials, not an independent evaluation — which matters when you are deciding whether MOVA belongs in a broadcast pipeline.

The gap is procedural, not accusatory. The accompanying paper, "MOVA: Towards Scalable and Synchronized Video-Audio Generation" (arXiv:2602.08794), was listed as forthcoming at the January 29, 2026 launch . As of July 2026, no third-party replication of these scores on a shared, public test set has been published. Until that appears, the "film-grade" framing carried by the release write-ups is a claim, not a validated result .

"Paper coming soon," — OpenMOSS team, MOVA project note (source: OpenMOSS/MOVA).

Why the caution specifically here: LSE-D and LSE-C are standard lip-sync metrics, and ELO is a reasonable ranking device, but a self-reported ELO is especially sensitive to test-set construction. Choose the prompts, the opponents, and the audio, and you shape the score. An ELO of 1113.8 is only interpretable once an external lab regenerates it on a common benchmark against the same fixed set of competitors. By contrast, Wav2Lip's SyncNet-discriminator scores remain the most widely and independently replicated baseline in open-source lip sync, because the metric and the evaluation setup have been reused across many papers and repos .

The concrete takeaway: for a shipping decision today, size your rig around LatentSync 1.6 at ~18 GB or MuseTalk 1.5 at ~8 GB — both are reproducible and battle-tested — and pilot MOVA as a promising but unproven joint generator. Bookmark arXiv:2602.08794; the moment an independent group reproduces those win-rates on a public set, MOVA's claims graduate from provisional to bankable, and that is when it earns a slot in a broadcast route, not before.

Frequently asked questions

What VRAM do I need to run LatentSync 1.6 locally?

LatentSync 1.6 needs roughly 18 GB of VRAM to run inference at its 512×512 training resolution, while the earlier v1.5 fits in about 8 GB . Reviewers point to an RTX 4090 or A10-class GPU as the documented entry point for comfortable throughput . Inference can complete on less, but at speeds that are impractical for production batches.

What is the difference between MuseTalk and LatentSync?

MuseTalk uses single-step latent-space inpainting on a 256×256 face region: it is fast, runs in about 8 GB of VRAM, and clears 30 fps on a Tesla V100 . LatentSync instead runs iterative audio-conditioned latent diffusion built on Stable Diffusion, which is slower and needs roughly 18 GB for v1.6 but reaches 512×512 with higher visual fidelity . Pick MuseTalk for speed and low hardware, LatentSync for detail.

What makes MOVA different from LatentSync and MuseTalk?

MOVA synthesizes new video and synchronized audio in a single inference pass rather than re-dubbing existing footage, which removes the error accumulation of cascaded generate-then-add-sound pipelines . That capability is heavier: OpenMOSS documents about 48 GB of VRAM, dropping to roughly 12 GB with layerwise offload, and clips cap out at 8 seconds and ≤720p . It solves a different problem than the two re-dubbing models.

Can I use MuseTalk commercially?

Yes. MuseTalk ships under the MIT license, the most commercially permissive option among these tools and the friendliest fit for broadcast pipelines . LatentSync and MOVA are both Apache-2.0, which is also permissive for commercial use . All three permit commercial deployment, but confirm enterprise-specific conditions with your own legal team before shipping.

Are MOVA's benchmark scores independently verified?

Not as of July 2026. MOVA's headline figures — LSE-D 7.094, LSE-C 7.452, an arena ELO near 1113.8, and win rates above 70% against rivals — are self-reported, and the companion paper (arXiv:2602.08794) was listed as forthcoming at launch . Treat the numbers as provisional vendor claims until an independent group reproduces them on a public set.

Enjoyed this article? Subscribe to get new stories by email whenever they're published.

Subscribe