LTX-2.5 at 4K, in half the time

LTX-2.5 is LTX's state-of-the-art open world model, which generates, simulates, and shapes video, audio, and the physical world. doubleAI has made LTX-2.5 generate 4K video nearly twice as fast.

The speedup comes from Maxwell, doubleAI's performance engineering Code Execution Model. Maxwell has previously demonstrated super human capabilities on a host of performance engineering benchmarks, exceeding the performance of human experts. LTX’s pipeline represents the top human expert performance for their workload, and Maxwell’s gains come on top of a cutting edge highly-optimised inference pipeline. The improvements made by Maxwell range from algorithmic, all the way down to a bespoke CuTe DSL kernels. Read on for the details.

Inference on LTX-2.5

LTX-2.5 was built for efficiency from the start: it is light enough to run on edge devices, and the inference pipeline behind it is state-of-the-art. The pipeline is compiled end to end, runs across multiple GPUs with sequence parallelism, and decodes video with a hand-written attention kernel. LTX and doubleAI set Maxwell to push this already efficient pipeline further: to profile, inspect and optimise it.

Maxwell came back with more than a dozen significant changes. Some are algorithmic, and change how the work is divided between tiles and between GPUs. Others are low-level, down to edits inside individual GPU kernels.

With these changes, the pipeline is not only faster but also more robust to differences between machines. The original pipeline had bottlenecks that made its speed depend on the host CPU: some work ran on the CPU itself, and other GPU work was issued as thousands of tiny launches that a slow CPU couldn't keep up with. Maxwell moved the CPU work onto the GPU and batched the tiny launches into a handful, so the optimised pipeline is now almost entirely insensitive to the CPU. All of these changes are available to the community, open source.

The same video

None of Maxwell's speedups come at the cost of quality. The soundtrack is preserved bit for bit. The video is preserved just as well: on LTX's benchmark prompt, the PSNR between the original and optimised videos is 41 dB. In other words, the optimised pipeline changes the output less than simply turning torch.compile on or off! PSNR (peak signal-to-noise ratio) compares two videos pixel by pixel on a logarithmic scale, where higher means closer; above 40 dB, two videos are considered visually indistinguishable.

Below are three examples, each from a different level of the stack. How the two GPUs share the work of attention, how the 4K frame is cut into tiles, and what happens inside the decoder's hand-written kernel. There’s more than a dozen other changes, all quite interesting. Those can be found in LTX’s source code.

Attention across two GPUs

LTX-2.5 generates video and sound together, and in every transformer block the audio looks at the video through cross attention. The two sides are highly asymmetric: about a hundred audio tokens against tens of thousands of video tokens. Each GPU holds all of the audio but only its half of the video.

For the audio to see the whole video, the original code had the GPUs exchange their video keys and values: 262 MB per GPU, in every block, at every step.

Maxwell turned this around: send the answers, not the data. Each GPU runs the audio queries against the video keys and values it already holds, giving a partial attention answer computed over its half of the video. That answer is tiny, because there are only about a hundred queries. The GPUs swap these partial answers, about 1 MB instead of 262, and merge them exactly. The merge works because each partial answer carries its running maximum and sum of exponentials, the same trick FlashAttention uses inside a single GPU. Maxwell applied it across two.

Kernel selection mattered too: small details decide which kernel PyTorch runs, and the difference can be large. For this shape, PyTorch picked cuDNN's attention kernel: it takes the first kernel in a fixed list that supports the call, not the fastest one. With so few queries, cuDNN ran one thread block per head and left most of the GPU idle. PyTorch's FlashAttention kernel, which splits the work along the keys, is nine times faster (0.1 ms instead of 0.9), so Maxwell moved it to the front of the list for this shape. And because the number of video tokens was odd, the original code padded the sequence with one extra token so it would split evenly, then masked that token out. The mask alone pushed PyTorch onto a slower kernel. Dropping the padded key instead of masking it gives the same result in less than half the time.

Thinner seams

LTX-2.5 generates 4K video through an iterative process. It first generates the clip at a lower resolution and frame rate, then refines it in further passes, the last of which brings it up to 4K. That final pass, the spatial epilogue, accounted for more than half of the original runtime. A 4K frame is too large to denoise whole, so at every step the pipeline cuts it into overlapping tiles, denoises each one, and blends the results back together. Neighbouring tiles never quite agree about the pixels they share, and the overlap is there to hide the seam.

The original overlaps were overlarge: on average, every pixel was denoised almost twice. Maxwell measured how far neighbouring tiles actually drift apart from one step to the next, and found that much thinner overlaps hide the seams just as well. Together with one row of tiles fewer, which keeps each tile close to its original shape on a 16:9 frame, almost every pixel is now denoised once.

The same pass also became cheaper per pixel. LTX-2.5 ships its weights in FP8, but the original code converted them to BF16 before every matrix multiply. The optimised pipeline leaves them in FP8, converts the activations to FP8 with a separate scale for each row, and accumulates in FP32, so the tensor cores run at their FP8 rate.

Inside the decoder

LTX-2.5 does its work in a compressed latent space, and a final decoding step turns the result into actual 4K frames. This decoding was the next-largest component in the chart, and most of its time is spent in a single hand-written GPU kernel for neighbourhood attention, written in CuTe DSL. Maxwell made three key changes to it.

The first removed wasted work. Each tile has room for four frames, but the keyframe tiles held one real frame and three copies of it, so the projections and the MLP ran four times over on the same data. Filling every tile with real frames halved the number of launches.

The second fixed a stall. The kernel carried an identical copy of its code for each of its two warp groups, and together the copies no longer fit in the instruction cache, so the kernel spent a large share of its time waiting for instructions. Maxwell replaced them with a single copy that both warp groups share, each reaching its own data through an offset.

Along the way, Maxwell even found a rare hang in the kernel, and fixed it. At the end of each tile, the kernel's warps poll a shared barrier, and the next tile resets it, with nothing to ensure that every warp has finished polling first. When another CUDA context shared the GPU, a late warp could find the barrier already reset and wait forever. A single synchronisation before the reset closes the race.

Afterword

Thanks to the LTX and doubleAI teams for making this collaboration possible. There is plenty more code to optimize, and we’re excited to see what Maxwell finds next.

The faster LTX-2.5 pipeline is open source. Try it on your own videos, or bring your own workload to Maxwell and see how much faster it can run, even if you’ve already optimized it.