Karma XPU Multi-GPU Scaling: Why 4 GPUs Is the Sweet Spot
Why Karma XPU gets less from each extra GPU, and how to find the right number for your scene:
- Every GPU renders the same frame: Karma XPU adds the samples from every OptiX GPU and the CPU into one image, and each GPU has to hold the whole scene, because memory is not pooled.
- Some work does not speed up with more GPUs: scene loading, shader compiling and the CPU-side steps that blend samples and write AOVs take about the same time on 1 or 4 GPUs, so each extra card saves less.
- Measure it on your scene: SideFX staff suggest switching GPUs off with KARMA_XPU_DISABLE_DEVICE_n and timing 1, 2 and 4 GPUs; speed-up is the 1-GPU time divided by the multi-GPU time.
- On iRender: in our experience Karma XPU runs best on 4x RTX 5090 servers; to finish sooner, start more 4-GPU servers and give each one a frame range.
It is easy to assume that a Karma XPU scene will render four times faster on four GPUs. It does not. Each extra GPU helps a little less than the one before, and for Karma XPU the curve flattens sooner than for GPU-only renderers.
This guide explains how Karma XPU splits a frame between devices, why part of every frame does not get faster with more cards, how to measure scaling on your own scene, and what we see on our own multi-GPU servers. Technical points come from the SideFX documentation and from SideFX staff answers on the SideFX forum.
How Karma XPU uses several GPUs on one frame
Karma XPU uses every compatible device it finds: each NVIDIA GPU becomes an OptiX device and the CPU becomes an EmbreeCPU device. All of them work on the same image, and the viewport render stats show each device’s share of it. A SideFX developer explained on the forum that, for this to work, every device has to produce exactly the same result, so XPU uses fp32 calculations on all devices.
Two consequences matter for multi-GPU machines:
- Memory is not pooled. SideFX’s documentation says each GPU is addressed separately, so every card must hold the whole scene. Four 24 GB cards still give you a 24 GB limit per scene.
- The CPU still has work to do. Before and during rendering, Houdini has to load and update the USD stage, compile shaders for each device, combine the samples from all devices and write the AOVs. A SideFX developer noted that writing AOV data was made multi-threaded for the case where very fast or multiple GPUs are paired with a slow CPU.
Why the speed-up flattens after a few GPUs
Think of a frame as two parts. One part is the GPU rendering itself, which really is shared between the cards. The other part is work that does not get shorter when you add cards: loading and updating the scene, compiling shaders, and the CPU-side steps that combine samples and write the image. The more GPUs you add, the bigger the share of the frame that the fixed part takes.
The figure is an illustration, not a measurement. With a fixed CPU-side share of 20%, the best possible speed-up is 1 ÷ (0.2 + 0.8 ÷ n): 1.7x on 2 GPUs and 2.5x on 4 (our calculation). Your scene’s fixed share may be larger or smaller, which is why you should measure it, as shown below.
What we see on iRender servers
On our multi-GPU servers, Karma XPU gains clearly going from 1 to 2 GPUs and keeps gaining up to 4. Our 8x RTX 5090 servers still render Karma XPU, but a single job finishes only a little faster than on 4 GPUs, so we offer Karma XPU on 4x RTX 5090 servers. To finish a sequence sooner, start more 4-GPU servers and give each one a frame range.
We do not publish a scaling percentage for Karma XPU, because it depends heavily on the scene: a light scene with many AOVs behaves very differently from a long volume render. Measure your own.
How to measure Karma XPU scaling on your scene
A SideFX developer’s suggested method on the forum is to render the same frame with different devices switched off using the per-device variables, which take a device number:
| Test | Example variables (3 GPUs: 0, 1, 2) | What it tells you |
|---|---|---|
| One GPU | KARMA_XPU_DISABLE_DEVICE_0=0, KARMA_XPU_DISABLE_DEVICE_1=1, KARMA_XPU_DISABLE_DEVICE_2=1 | Your 1-GPU baseline |
| Two GPUs | KARMA_XPU_DISABLE_DEVICE_0=0, KARMA_XPU_DISABLE_DEVICE_1=0, KARMA_XPU_DISABLE_DEVICE_2=1 | The gain from a second card |
| All GPUs, no CPU | KARMA_XPU_DISABLE_EMBREE_DEVICE=1 | GPU-only speed, with no CPU help |
| CPU only | KARMA_XPU_DISABLE_OPTIX_DEVICE=1 | How much the CPU device adds on its own |
Per-device variables as given by a SideFX developer on the SideFX forum; the Embree and OptiX variables are in SideFX’s Karma XPU documentation. Device numbers depend on the machine; check which number belongs to which card.
- Pick a representative frame and render it from the command line with husk, so the viewport is not involved. Forum users found that the viewport can keep one GPU busy, which skews a viewport test.
- Render it four times: CPU only, 1 GPU, 2 GPUs and all GPUs, using the variables above. Restart the render process between runs so the new variables are read.
- Write down the full time of each run, from start to the written image, not only the sampling time.
- Work out the speed-up: the 1-GPU time divided by the multi-GPU time. A result of 1.7 on 2 GPUs means the frame finished 1.7 times as fast.
- Check the device shares in the render stats; if one GPU contributes far less than the others, look for memory or driver problems before blaming scaling.
Help the CPU side keep up
- Give blending more threads. For a slow multi-GPU setup, a SideFX developer suggested raising KARMA_XPU_NUM_PER_DEVICE_BLENDING_THREADS from its default of 1 to 2 or 3, which assigns more CPU threads to combining each device’s samples. Test it on your frame; it is not a universal setting.
- Keep Houdini up to date. SideFX has made CPU-side steps such as AOV writing multi-threaded for machines with fast or multiple GPUs.
- Write only the AOVs you need. Every AOV, deep image or Cryptomatte layer adds CPU-side work per frame.
- Load caches, not live simulations. Render from cached geometry and volumes on fast local storage, so each frame does not wait for Houdini to cook upstream nodes.
- Use a strong CPU. On iRender, the 4x RTX 5090 servers use a 32-core AMD Ryzen™ Threadripper™ PRO 5975WX.
SaaS or IaaS render farm for multi-GPU Karma XPU?
Both farm models render the same frames. The difference for multi-GPU Karma XPU is who chooses how many GPUs work on each frame, and whether you can measure the scaling yourself.
| Your situation | SaaS render farm | IaaS render farm (iRender) |
|---|---|---|
| The shot is approved and you only need frames | A good fit. Upload the project and the farm renders the frames, often across many machines, with no server to manage. | Also works, but you start, use and stop the server yourself. |
| You have a deadline and need to know when rendering starts and ends | Jobs join the farm’s queue, so the start time depends on how busy the farm is; many farms sell higher-priority plans to start sooner | The server runs only your job from the moment it boots, so you control when rendering starts and can time the whole job |
| You want to choose how many GPUs render each frame | The farm decides how jobs are placed on its machines. | Pick a 1, 2 or 4-GPU server, or switch GPUs off per process with the device variables. |
| You want to measure scaling on your scene | You compare finished jobs and their logs. | Run the 1, 2 and 4-GPU tests on the same server and compare the times directly. |
| The scene is close to the memory of one GPU | Check the farm’s GPU memory before you submit. | RTX 5090 cards have 32 GB each; memory is per card, not added up. |
A comparison of how the two farm models work for this task, not a speed test.
A simple rule: if the shot is approved and you only need frames, a SaaS farm is a good fit. If you want to choose the GPU count, measure scaling on your own scene or split GPUs between husk processes, use an IaaS server.
A scaling test on an iRender server
- Upload the project and caches to iRender Drive before you start the server; uploading is free and does not use rental time.
- Start a 4x RTX 5090 server, install Houdini once and sign in with your own SideFX license. Your server image is saved between sessions. Billing is per minute.
- Time one representative frame with husk on 1, 2 and 4 GPUs using the device variables above.
- Pick the setup with the best time per frame for the money, and render the sequence.
- Need it sooner? Start more 4-GPU servers and give each one a frame range; the total cost stays about the same.
Before you render the sequence on another machine
- Use the same Houdini release and an NVIDIA driver of version 570 or later.
- Make sure the heaviest frame fits in one GPU’s memory; memory is not pooled across GPUs.
- Remove device test variables you no longer need, or keep them on purpose if you split GPUs between processes.
- Render with husk and check the device shares on the first frame.
- Time the heaviest frame first and compare it with your test.
Frequently asked questions
1. Does Karma XPU scale linearly with more GPUs?
No. Every GPU helps, but part of each frame, such as loading the scene, compiling shaders and the CPU-side steps that combine samples and write AOVs, does not get faster with more cards. Each extra GPU therefore saves less time than the one before.
2. How many GPUs should I use for Karma XPU?
In our experience on iRender servers, Karma XPU gains clearly from 1 to 2 GPUs and keeps gaining up to 4. Past 4 GPUs the extra speed for one job is small, so we recommend 4-GPU servers for Karma XPU and more servers when you need the sequence sooner.
3. Does Karma XPU combine the VRAM of several GPUs?
No. SideFX’s documentation says each GPU is addressed separately and memory is not pooled, so every GPU must hold the whole scene. Four 24 GB cards still give a 24 GB limit per scene.
4. How do I test Karma XPU with only some of my GPUs?
Use the per-device variables suggested by a SideFX developer, such as KARMA_XPU_DISABLE_DEVICE_1=1 to switch off the second GPU. Render the same frame with 1, 2 and all GPUs and divide the 1-GPU time by each multi-GPU time.
5. Can the CPU slow down a multi-GPU Karma XPU render?
Yes. The CPU still loads the scene, compiles shaders, combines the samples from every device and writes the AOVs. SideFX has made some of these steps multi-threaded, and a SideFX developer suggested raising KARMA_XPU_NUM_PER_DEVICE_BLENDING_THREADS when fast GPUs wait for a slow CPU.
6. Can I check Karma XPU scaling on the machine that renders my final frames?
Yes, on an IaaS render farm. On iRender you rent the whole server, so you can time the same frame on 1, 2 and 4 GPUs with husk, see each device in the render stats and then render the sequence on the same hardware.
Render your Houdini Karma XPU scenes on iRender
iRender is an IaaS GPU render farm: you rent a bare-metal server with 4x NVIDIA RTX 5090 (32 GB) GPUs, an AMD Ryzen™ Threadripper™ PRO CPU, 256 GB ECC RAM and a 2 TB NVMe SSD, connect by remote desktop and work on it like your own workstation, with full administrator access. You install Houdini and use your own SideFX license; your server image is saved between sessions, so Houdini, your packages and the Karma XPU kernel cache stay installed. For Karma XPU we recommend the 4-GPU servers.
Multi-GPU servers are billed per minute (1-GPU servers have a one-hour minimum), and sessions of 3 hours or more get 10% off. Our 4-GPU servers are liquid-cooled; in our own measurements the GPUs hold 60–70°C under sustained render load. Elite support is available 7 days a week by hotline, live chat, Skype, WhatsApp and email.
This video walks through rendering a Houdini Karma XPU scene on an iRender server in five steps:
New users get a 100% bonus on the first top-up (minimum $50). See the Karma XPU render farm page for server options, or download the iRender GPU app.


