RSS< Twitter< etc

XMesh for Maya - Introduction

What Is XMesh?

XMesh is a mesh caching solution. It stores the render-time representation of one or more geometry objects to disk files, and can restore them for viewport playback and rendering while preserving all relevant information including vertex deformations, vertex velocity data, topology changes, mapping coordinates, smoothing information and material assignments.

XMesh is currently available as a plugin for Autodesk Maya and Autodesk 3ds Max, and is under development for other 2D and 3D DCC applications.

The XMesh system consists of an XMesh Saver utility for caching geometry objects to XMesh file sequences, and an XMesh Loader object for loading the saved data back into the 3D application.

XMesh has been used in heavy visual effects production since 2006 and works reliably with multi-million polygon meshes and complex particle systems setups. Its saves the data in a compact and extensible channel-oriented format and supports caching of geometry on the network using Thinkbox's Deadline Network Manager, including the parallel processing of the same animation sequence on multiple network machines.

Why Use XMesh?

In a visual effects pipeline, working with dynamic scenes at all stages of production can be a problem. For example, the results of the Animation, Simulation and Effects departments have to be passed to the Lighting And Rendering department, but a lighting artist does not need access the actual animation rigs, simulation data or effects/particles setups to perform his work. In fact, having access to the underlying rigs from the previous stages could be dangerous since some vital elements like nulls or particle system components could be moved, modified or deleted accidentally, affecting the final render output.

XMesh provides a way to cache the geometry data and removes the dependency on animation, simulation and effects rigs, giving the Lighting and Rendering department the precise results it needs in an unchangeable form.

In addition, it allows the combining of multiple meshes into a single mesh, significantly reducing the performance hit on the 3D scene where one object with millions of polygons is typically faster to work with than thousands of objects with a few polygons each.

The data is externalized to file sequences outside of the main Maya scene, allowing for easy updates to the scene geometry data by simply resaving the caches.

Finally, thanks to the support for 3D applications like Autodesk Maya and Autodesk 3ds Max, XMesh can be used to exchange mesh data between the packages.

How Does It Work?

An XMesh file sequence uses multiple files per frame, resulting in a large number of files per sequence. This has several positive implications: 

  • Unchaging channel data can be stored just once for the whole sequence, reducing disk usage;
  • Multiple machines can save to the same sequence in parallel for faster turnaround;
  • A sequence can be saved in multiple runs, adding more frames to existing ones if needed;
  • Sub-ranges of an existing sequence can be resaved / updated without the need to resave the whole sequence;
  • Corrupted data can be fixed easily by resaving the individual frame without the need to resave the whole sequence;
  • Multiple CPUs can save and compress individual data channels of the same frame in parallel;
  • Multiple CPUs can decompress and load individual data channels of the same frame in parallel

and so on. 

To learn more about the structure and content of the XMesh file sequence, see the topic "XMesh MY File Sequence".

Saving Modes

The XMesh Saver provides several saving modes:

  • Saving of multiple objects as a single mesh in world space - this mode lets you combine many objects into a single mesh to optimize the performance of heavy scenes when access to the individual entities is not needed.
  • Saving of multiple objects as individual caches in world space - thos mode preserves the individual objects by saving one XMesh sequence per object, but bakes the world transforms into the mesh cache, making the recreation of the original scene as easy as placing all XMesh Loaders at the world origin.
  • Saving of multiple objects as individual caches in object space - this mode bakes only the object space transforms and allows each XMesh Loader to replace the respective source object in an existing hierarchy, while caching object-space deformations only.

What Data Does XMesh Support?

XMesh can save and load arbitrary named data channels, but the current implementation of XMesh for Maya supports a hard-coded set of channels, as well as a specific subset of the channels saved by 3ds Max. Only the Vertex and Face lists are madatory and always included, all other channels are optional and can be disabled when saving.

In the case of Autodesk Maya, XMesh saves

  • Vertex list (always included)
  • Face list (always included)
  • Edge Smoothing (automatically converted to 3ds Max Smoothing Groups when loading in 3ds Max)
  • Vertex Normals (overriding Smoothing Groups if both are available when loading in 3ds Max)
  • Material assignments (loaded as Material IDs in 3ds Max)
  • Current UV set (as map1 channel)
  • Custom UV sets (named from map2 to map99), if present. These are mainly for round-trip data transfer between 3ds Max and Maya.
  • Vertex Velocity (if available and explicitly requested by the user)

In the case of loading files saved from Autodesk 3ds Max, XMesh MY currently supports the following channels:

  • Vertex list 
  • Face list
  • Per-face Smoothing Groups converted to Maya Edge Smoothing
  • Vertex Normals (overriding Edge Smoothing, if Smoothing Groups channel is also available)
  • Per-face Material IDs (converted to Shading Groups when loading in Maya)
  • VertexColor channel (Mapping Channel 0), loaded as vertex color set.
  • TextureCoord channel (a.k.a. Mapping Channel 1), loaded as UV set named map1. Note that the 3rd component of 3ds Max UVW channels will be lost.
  • Mapping channels (from Mapping2 to Mapping99), loaded as map2 to map99. Note that the 3rd component of 3ds Max UVW channels will be lost.
  • Vertex Velocity (used by the Loader to extrapolate vertex positions on sub-frames, if requested)

How Does The XMesh Loader Work?

The XMesh Loader node is a geometry object and can be modified like any other geometry object in Maya. It can be freely translated, rotated or scaled to transform the loaded geometry, including animated transformations. The XMesh Loader can be parented to other objects and be part of a hierarchy. Topology-dependent modifications should not be used though since the mesh can change completely from one frame to the next.

File Sequences

The XMesh Loader object can load one or two XMesh sequences from disk - a Render Sequence and a Proxy Sequence.

The Render Sequence is used for the render-time representation of the object, and optionally for viewport display if no Proxy Sequence was saved. Even if a Proxy sequence is present, the viewpor display can be forced to show the Render Sequence when needed.

The optional Proxy sequence, usually saved with lower resolution, can be used as a viewport proxy for faster performance, but can be selected to be rendered too if needed (e.g. for fast previews).

The two sequences can be saved with different sampling step and can include completely different channels (for example the Proxy sequence could exclude Velocities, Mapping, Materials or Smoothing data to reduce the data amount and improve playback speed). In addition, the Proxy sequence can be optionally saved with polygon reduction of a user-defined percentage.

The XMesh Loader lets you pick any XMesh file from the sequence and will replace the frame number with the current or custom time as needed. It can also optionally load only one frame from the sequence at every time, in which case the frame selected as representative of the whole sequence will be used without replacing its frame number.

Retiming

The XMesh Loader exposes controls to offset the playback in time, or to freely retime the sequence using a custom graph.

The XMesh Loader can also limit the loading of the sequence to a user-defined frame range to ensure that the retiming controls don't request a non-existing frame.

Memory Caching

The XMesh Loader does not pre-load frames into memory, nor does it keep previously loaded frames in memory. This is done to avoid problems with very heavy meshes where a single frame can use up nearly all available RAM.

The XMesh Loader does support memory caching of individual channels to accelerate the loading of non-changing channel data though. For example, if the Face List is consistent between multiple frames, it will be loaded just once on the first frame and reused to build the mesh on the following frames without accessing the compressed data from disk.

When the XMesh Loader is set to use the same single frame on every frame, it will be cached and will not be reloaded on every frame. If the playback is limited to a custom range though where the first and last frame are the same, the specified frame will be reloaded from disk on each time change.

Multi-Threading

The XMesh Loader uses multi-threading to accelerate the loading process. Since every data channel is stored in a separated compressed file, and decompression is rather CPU-intensive and single-threaded, a separate thread will be spawned to decompress each data channel. This means that a multi-core machine with 8 threads could load up to 8 data channels at the same time.

 

How Does The Licensing Work?

XMesh Saver Application-Agnostic Floating Licenses

The XMesh Saver requires a floating license. A single XMesh Saver license can be used by multiple artists on different machines as long as they are not saving at the same time.

Multiple XMesh Saver licenses would be needed to save XMesh files in parallel, including simultaneous processing on Deadline.

The XMesh Saver license is the same for all implementations of XMesh - it can be used to cache from both Autodesk Maya and Autodesk 3ds Max, and will be usable with any future implementations of the XMesh Saver in other applications.

XMesh Loader License-Free Loading And Rendering

The XMesh Loader does not require a license in any of the supported applications. It can be downloaded, installed and used for free to load, view and render XMesh data saved with the commercial XMesh Saver.