Simplifying Realistic Character Creation with NVIDIA Omniverse Reallusion Connector

In this article, iRender team will explore the contribution between Reallusion Connector with Nvidia Omniverse through a blog. That blog was written by Weien Chen who is a senior software engineer on the Graphics team at Reallusion. He participated in shipping iClone 7/Character Creator 3 and their Omniverse Connector. He also worked on Reallusion PopVideo enhancement, VXGI, and PopcornFX integration and Reallusion skin, SSS, and hair shader development.

Let’s start!

Character creation and animation are two distinct disciplines that demand the skill of well-trained artists with specialized background knowledge. These domains can be difficult and frustrating for artists who come from unrelated backgrounds with different skill sets. And it’s a pain point that Character Creator and iClone were specifically created to resolve.

Through, Character Creator is well-positioned as the go-to solution for creating realistic and stylized characters with robust pipelines for mainstream tools like ZBrush, Substance, and Blender. Combined with the ability to export FBX with LODs (levels of detail), digital human shaders, and a rich collection of motion assets, iClone stands out as an animation editor without a steep learning curve.

On the other hand, the launch of NVIDIA Omniverse in 2020 was a momentous occasion that attracted our attention. Omniverse represents the next-generation 3D virtual collaboration and real-time simulation platform that connects people and applications for broad-based collaboration. The Reallusion software suite combined with a massive library of high-quality character assets and motions can play a crucial role in this ecosystem, while Omniverse provides the path-traced rendering and AI technology that makes for a powerful synergy.

To build the connector, you start by referencing the Connect Sample code available for download on the NVIDIA launcher application.

On the Omniverse Youtube channel, there is a great beginner tutorial: Create an Omniverse USD App from the Connect Sample.

iClone and Character Creator’s 3D scene consists of nodes with basic transforms consisting of translation, rotation, and scale values. Next, Characters, meshes, lights, and cameras are all attached under these nodes.

Furthermore, the bones of a character skeleton are also represented by these nodes. Nodes that only represent transforms are exported as USD XForm and nodes that represent the body and facial bones are exported as USD skeleton joints. Additional bones and skinning are added to accessory nodes that are attached to bones before converting to USD format.

USD Xform scaling works in fundamentally different ways to iClone. In iClone, a node can be made to inherit or ignore the parent scale. In Omniverse, the node scale is always inherited from its parent node. Under such conditions, bone node scale inheritance must be removed with its values reset before exporting to Omniverse, in order for the scale values to match.

Besides, most of iClone’s facial expressions are composed of morph animations that are exported as USD blend-shapes. In contrast to FBX blend-shapes, which are stored in local-space positions, USD blend-shapes store positional offsets.

Because iClone also stores positional offsets (in combination with strength multipliers). It is completely compatible with Omniverse, enabling direct conversion to USD format. It should be noted that Omniverse requires a skeleton root for props with blend-shapes attached and additional processing may be required.

The following section contains MDL code excerpts for iClone’s USD exporter. For more information, see USD Shader Attributes.

Within the USD file, MDLs are specified using info:mdl:sourceAsset with info:mdl:sourceAsset:subIdentifier as the entry function. The new subIndentifier attribute was introduced by NVIDIA and PIXAR. input: is later called to feed the texture and material parameters. OmniPBR.mdl and OmniSurface(Base).mdl provided with Omniverse were used as starting points.

Then omniPBR.mdl was chosen from the start because it works well in both NVIDIA RTX Real-time and Path-traced mode. On the other hand, OmniSurface and OmniHairare photo-realistic MDLs for RTX Path-traced mode. The existing PBR, Skin, Head, and SSS shaders were then rewritten from HLSL to MDL.

Flowing water in the swimming pool is another good example:

float2 inputs:texture_translate.timeSamples = {
                    0: (0, 0),
                    4000: (4, 8),
                }

Besides the previously mentioned built-in MDLs, there is also a base.mdl on GitHub with some reusable functions that can be deployed in a jiffy.

Point lights and spotlights use UsdLuxSphereLight with adjusted cone angles. Tube and rectangle lights use UsdLuxCylinderLight and UsdLuxRectLight, respectively. The Light IES profile file is also one of the shaping attributes. Light intensity in USD is similar to luminous intensity per unit surface area. The USD intensity of a spherical light with radius (r in meters) can be approximated with the following formula:

USD intensity = candela * 1000 * / (4PI r*r)

The following formula is used when radius is in centimeters:

USD intensity = candela * 1000 / (4PI(0.01r)*(0.01r))

Radius is a significant attribute in Omniverse Renderer. We recommend imposing a minimum radius of 2 cm. For more examples, see iClone Omniverse Connector Tutorial – Light Settings in iClone & Omniverse.

iRender is an appropriate and effective service for special software such as iClone Connector and NVIDIA Omniverse. We provide high-performance servers for GPU-based engine rendering through the Remote Desktop application. 

In addtition, we are releasing new servers for iRender’s users, which have powerful processing capabilities of AMD Ryzen Threadripper Pro 3955WX @ 3.90GHz. Besides, they have multi-GPUs: 2/ 4/ 6/ 8 RTX 3090 24GB VRAM to boost performance in the rendering process.

iRender’s workflow is simple and easy. You need to create an image and then boot a system. Next, you connect to our remote servers and take full control of our machines. There you can install any software that you want.

We not only have powerful configurations, but also we have excellent service with a great support team. In addition, we want to bring a comfortable and helpful feeling to all customers. They will always be satisfied when they use our service. I would like to sponsor hours rendering on power servers for you if you CREATE AN ACCOUNT right now and contact via email: [email protected] or Whatsapp/Telegram: +84 0394000881. 

Source and images: developer.nvidia.com