XGen was easily the most difficult to integrate into our pipeline. It was quite a task getting it to work, between the unfinished state, the bugs and crashes, and all the missing features we needed. If it wouldn't have been a requirement we definitely would have rather waited for the updated version in Maya 2016.python, xgen, arnold, pipeline, mayaCraig Barnett

XGen with Arnold Pipeline

XGen was easily the most difficult to integrate into our pipeline. We had to figure out the best way to allow XGen collections to transfer between assets and shots, it had to be Perforce friendly, it had to take into account Arnold, it had to be compatible with our render farm software Qube, and most importantly couldn't be too complicated for our artists. All of this and we had to stay on Maya 2015, the first revision of XGen which still had a lot of bugs and crashes, and missing features. Thankfully it ended up being worth all the trouble, as you can see in the image below that one of our artists made using the tools.

Example of XGen hair by using tools

Our main goal seemed rather simple in the beginning, get these characters that have XGen hair, throw them in a shot, and render them. We quickly realized how very wrong we were. The main problems we stumbled across were:

  • XGen in Maya 2015 doesn't support namespaces.
  • XGen doesn't render in Arnold very easily.
  • XGen in the scene greatly slows it down, especially with a shot with multiple characters.
  • XGen is not open source, and therefore couldn't edit any source code.
  • XGen and Maya 2015, require all painting of files to happen from the 3D Paint Textures folder.
  • XGen is extremely difficult to move around files.
  • XGen crashes on export during the last frame with motion blur turned on.
  • XGen isn't included in bounding box calculations.
  • Lack of XGen and Arnold documentation
  • How can we have a shot update when an Asset changes if nothing in the shot has changed?
  • How the heck do we get XGen to the farm?

Our first thing we needed to figure out was just how to get Xgen files moving through the pipeline to at least some degree. After hitting several walls and limitations, like having no source code, we found our solution.

  1. The artist imports a model.
  2. The artist creates their XGen collection and saves.
  3. Animation caches are imported into temporary scenes, making sure to keep unique namespaces.
  4. The user selects the cache and using a slightly modified XGen Import Collection tool the user brings in their collection.
  5. Using a custom tool the user exports a Arnold standin of the XGen contents either locally or on the render farm.
  6. These standins can then be imported and used in their shot like any other standin.

Now that we had a workflow set up, we needed to see what we had to change to prevent all the different errors from popping up. So we start digging into the code and fixing what we find. Our solution to the colon in file name problem, was to convert colons to two underscores, you could of course use something different.

First we fix exporting patches when the collection includes a namespace:

1
2
3
4
5
6
7
8
9
    | +++ {maya_root}/plug-ins/xgen/scripts/xgenm/ui/xgDescriptionEditor.py
    | @@ -1917,7 +1917,7 @@ class DescriptionEditorUI(QtGui.QWidget):
    |      cmdAlembicBase = cmdAlembicBase + ' -uvWrite -attrPrefix xgen -worldSpace'
    |      palette = cmds.ls( exactType="xgmPalette" )
    |      for p in range( len(palette) ):
    | -        filename = strScenePath+ "/" + strSceneName + "__" + palette[p] + ".abc"
    | +        filename = strScenePath+ "/" + strSceneName + "__" + palette[p].replace(':', '__') + ".abc"
    |          descShapes = cmds.listRelatives( palette[p], type="xgmDescription", ad=True )
    |          cmdAlembic = cmdAlembicBase

Then we fix a Maya error because of .xgen file read permissions (Useful for perforce and other versioning systems that may lock the file. For this to work the user must have permissions to modify the file in the first place):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    | +++ {maya_root}/plug-ins/xgen/scripts/xgenm/xmaya/xgmExternalAPI.py
    | @@ -385,10 +385,11 @@ def importPalette( fileName, deltas, nameSpace="", wrapPatches=True, wrapGuides=
    |      if validatePath:
    |          path = base.getAttr( 'xgDataPath', palName )
    |          # create the directory if we can, otherwise log an error
    | -        expandedPath = base.expandFilepath( path, "", True, True )
    | -        if not os.path.exists(expandedPath):
    | -            msg = maya.stringTable[ 'y_xgmExternalAPI.kxgDataPathBroken' ] % (palName,path)
    | -            XGError( msg )
    | +        for p in path.split(';'):
    | +            expandedPath = base.expandFilepath( p, "", True, True )
    | +            if not os.path.exists(expandedPath):
    | +                msg = maya.stringTable[ 'y_xgmExternalAPI.kxgDataPathBroken' ] % (palName, p)
    | +                XGError( msg )
    |
    |      # setup all imported descriptions
    |      _setupDescriptionFolder( palName )

Next we fix XGen failing on windows to set modifier load paths because of backslashes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
    | +++ {maya_root}/plug-ins/xgen/scripts/xgenm/ui/tabs/xgFXStackTab.py
    | @@ -318,9 +321,9 @@ class FXModuleLayerUI(QtGui.QWidget):
    |          _localFXMenu.clear()
    |          _userFXMenu.clear()
    |          _allFXMenus = []
    | -        self.buildMenu(_globalFXMenu,xg.globalRepo()+"fxmodules/")
    | -        self.buildMenu(_localFXMenu,xg.localRepo()+"fxmodules/")
    | -        self.buildMenu(_userFXMenu,xg.userRepo()+"fxmodules/")
    | +        self.buildMenu(_globalFXMenu, path_normalize(os.path.join(xg.globalRepo(), "fxmodules/")))
    | +        self.buildMenu(_localFXMenu, path_normalize(os.path.join(xg.localRepo(), "fxmodules/")))
    | +        self.buildMenu(_userFXMenu, path_normalize(os.path.join(xg.userRepo(), "fxmodules/")))
    |
    |      def buildMenu(self,topmenu,startDir):
    |          # first verify that the directory exists
    | @@ -351,7 +354,7 @@ class FXModuleLayerUI(QtGui.QWidget):
    |                  else:
    |                      menus.append(menu)
    |                  for item in files:
    | -                    long = os.path.join(dir,item)
    | +                    long = path_normalize(os.path.join(dir,item))
    |                      if os.path.isfile(long):
    |                          parts = os.path.splitext(item)
    |                          if parts[1] == ".xgfx":
    | @@ -378,7 +381,7 @@ class FXModuleLayerUI(QtGui.QWidget):
    |              moduleName = self.getCurrentModuleName()
    |              if ( moduleName == "" ):
    |                  return
    | -            startDir  = xg.userRepo() + "fxmodules/"
    | +            startDir  = os.path.join(xg.userRepo(), "fxmodules/").replace('\\', '/')
    |              try:
    |                  buf = os.stat(startDir)
    |              except:
    | @@ -409,7 +412,7 @@ class FXModuleLayerUI(QtGui.QWidget):
    |          moduleName = self.getCurrentModuleName()
    |          if ( moduleName == "" ):
    |              return
    | -        tempFileName = str(tempfile.gettempdir()) + "/xgen_dup.xgfx"
    | +        tempFileName = os.path.join(tempfile.gettempdir(), "xgen_dup.xgfx")
    | +        tempFileName = tempFileName.replace( "\\", "/" )
    |          xg.exportFXModule(pal, desc, moduleName, tempFileName )
    |          dup = xg.importFXModule( pal, desc, tempFileName )

When resetting back to a slider in XGen from a map, it fails to reset correctly, and when the scene is saved and reopened it errors, this is because the newline character isn’t escaped:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    | +++ /plug-ins/xgen/scripts/xgenm/ui/widgets/xgExpressionUI.py
    | @@ -2293,14 +2293,19 @@ class ExpressionUI(QtGui.QWidget):
    |                  de.playblast()
    |
    |      def resetToSlider(self):
    | -        expr = "$a=0.000;#-1.0,1.0\n$a"
    | +        expr = "$a=0.000;#-1.0,1.0\\n$a"
    |          self.setValue(expr)
    | +        if xgg.Maya:
    | +            mel.eval( 'XgExpressionEditor();' )             
    | +        if self.object=="" or xgg.DescriptionEditor is None:
    | +            return
    |          if ptexBaker.g_Mode != "" and ptexBaker.g_xgenAttr == self.attr:
    |              cmds.setToolTo( "selectSuperContext" )

When creating a new description for a collection XGen clears out any custom XgDataPaths (Using Edit Collection Paths under the XGen File menu):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    | +++ {maya_root}/plug-ins/xgen/scripts/xgenm/xmaya/xgmExternalAPI.py
    | @@ -139,7 +139,8 @@ def _setupProject( palette, newDesc='' ):
    |
    |      # store the unresolved version
    |      palFolder = palettePathVar(palette)
    | -    base.setAttr( 'xgDataPath', palFolder, palette )
    | +    if not base.getAttr( 'xgDataPath', palette ):
    | +        base.setAttr( 'xgDataPath', palFolder, palette )
    |
    |      # setup the description folder
    |      _setupDescriptionFolder( palette, newDesc )

This might be a limited issue because of combining the XGen installs into one cross-platform solution. However the import error we receive was because of load order and seems like it would be a problem to everyone, so just in case we provided our fix here too:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
    | +++ {maya_root}/plug-ins/xgen/scripts/xgenm/__init__.py
    | @@ -36,18 +36,28 @@
    |  # @version Created 06/01/09
    |  #
    |
    | +import XgExternalAPI
    | +
    | +import xgGlobal
    |  import xgGlobal as xgg
    |
    |  # The base level API
    |  from XgExternalAPI import *
    |
    |  # If in maya bring in the Maya wrappers and extensions
    | -if xgg.Maya:
    | -    from xmaya.xgmExternalAPI import *
    | -else:
    | -    # stub this thing out because it's used everywhere
    | -    def getOptionVar( varName ):
    | -        return None
    | +try:
    | +    if xgg.Maya:
    | +        from xmaya.xgmExternalAPI import *
    | +    else:
    | +        # stub this thing out because it's used everywhere
    | +        def getOptionVar( varName ):
    | +            return None
    | +except:
    | +    pass
    |
    |  # General utilities
    |  from xgUtil import *

Arnold fix for colons in the patch abc path (This fix included modifying Arnold source code, I would be extremely surprised if this hasn't been fixed long since then):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    | +++ mtoa/contri{maya_root}/extensions/xgen/plugin/XGenTranslator.cpp
    | @@ -115 +115 @@
    | + bool replace(std::string& str, const std::string& from, const std::string& to) {
    | +     size_t start_pos = str.find(from);
    | +     if(start_pos == std::string::npos)
    | +         return false;
    | +     str.replace(start_pos, from.length(), to);
    | +     return true;
    | + }
    |
    | @@ -376 +376 @@
    | -   std::string strGeomFile = info.strScene + "__" + info.strPalette + ".abc";
    | +  std::string replacementPalette(info.strPalette);
    | +
    | +  replace(replacementPalette, ":", "__");
    | +
    | +  std::string strGeomFile = info.strScene + "__" + replacementPalette + ".abc";

Now that we have some of the core code working, we can look into the tools we need. We'll start with the modifications to importing XGen collections. First we go ahead and set a starting folder for browsing that makes more sense for us. We use a method called getEntityPath, that uses Shotgun methods from a shotgun wrapper we developed called Trak, to find the correct folder for the assets outputs. While this isn't required, the artists desperately wanted this feature. You could easily create your own Shotgun methods to do the same thing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
    |  +++ {maya_root}/plug-ins/xgen/scripts/xgenm/ui/dialogs/xgImportFile.py
    |  @@ -82 +82 @@
    |    def paletteUI(self):
    |        self.palPart = QtGui.QWidget()
    |        vbox = QtGui.QVBoxLayout()
    |        vbox.setAlignment(QtCore.Qt.AlignTop)
    |        self.palPart.setLayout(vbox)
    |        self.palFile = BrowseUI(maya.stringTable[ 'y_xgImportFile.kFileName'  ],
    |                              maya.stringTable[ 'y_xgImportFile.kFileNameAnn'  ],
    |                               "","*.xgen","in")
    |        self.palFile.optionButton.setToolTip(maya.stringTable[ 'y_xgImportFile.kBrowseForFiles'  ])
    | -      self.palFile.textValue.setText(xg.userRepo())
    | +      entityPath = self.getEntityPath()
    | +      if entityPath:
    | +          LOG.debug("Setting import dialog path to: {0}".format(entityPath))
    | +          self.palFile.textValue.setText(entityPath)
    | +      else:
    | +          if pm.workspace(fn=True):
    | +              LOG.debug("Setting import dialog path to: {0}".format(pm.workspace(fn=True)))
    | +              self.palFile.textValue.setText(pm.workspace(fn=True))
    | +          else:
    | +              LOG.debug("Setting import dialog path to: {0}".format(xg.userRepo()))
    | +              self.palFile.textValue.setText(xg.userRepo())

We then go ahead and add a warning for when there's no selection. To prevent the amount of editing in XGen source code we create and emit callback methods (importDialogCallback, preImportCallback and postImportCallback) that can be created in a number of different ways but we go ahead and use our custom framework and declare them at the top of file. This way the chunk of code that will run can be in a separate location. Last we go ahead and set up an error to force the user to save there scene before importing XGen. This helped us prevent a variety of issues in Perforce and with how we check for file paths in Shotgun.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
    |  +++ {maya_root}/plug-ins/xgen/scripts/xgenm/ui/dialogs/xgImportFile.py
    |  @@ -77 +77 @@
    |     if which == 'description':
    |         self.tabs.setCurrentWidget( self.descPart )
    |     else:
    |         self.tabs.setCurrentWidget( self.palPart )
    | +   importDialogCallback(self)
    |
    |  @@ -256 +256 @@
    |    def importCB(self):
    | +      if xgg.maya:
    | +          if not pm.ls(sl=True):
    | +              LOG.warning("No geometry selected to connect imported collection to.")
    |        if self.getType() == 0:
    |            # Import palette
    |            xgenFile = self.getPalFile()
    |            if not ImportFileUI.fileExists(xgenFile,maya.stringTable[ 'y_xgImportFile.kFileDoesNotExist'  ]):
    |                return
    | +          originalXgenFile = xgenFile
    | +          data = dict(path=xgenFile)
    | +          preImportCallback(xgenFile, data)
    | +          xgenFile = data['path']
    |
    |  @@ -277 +277 @@
    |            if (xgg.DescriptionEditor != 0 ):
    |                xgg.DescriptionEditor.refresh("Full")
    | +          postImportCallback(xgenFile, originalXgenFile, validator)
    |
    |  @@ -404 +404 @@
    |    def importFile( which='palette' ):
    |        """Function to import a file using a dialog.
    |
    |        This provides a simple dialog to import either a palette or a
    |        description from a file. The user can use a browser to find the file
    |        and specify other options specific to the type of input.
    |        """
    | +      if xgg.maya:
    | +          import pymel.core as pm
    | +          if not pm.sceneName():
    | +              QtGui.QMessageBox.warning(None, "Scene Not Saved!", "In order to set correct paths you must save your scene first!")
    | +              result = pm.mel.eval('projectViewer SaveAs;')
    | +              if not result:
    | +                  return
    |        return ImportFileUI( which ).exec_()

We use the callbacks we set before for several different purposes. First we use the importDialogCallback to populate the namespace in the dialog automatically based on the users selection, we use a couple different methods to get that data:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
    def asList(value):
        """
        Return the given object as a list
            >>> asList(2)
            [2]
            >>> asList([1, 2, 3])
            [1, 2, 3]
            >>> asList(None)
            []
        """
        if value is None:
            return []
        if not isinstance(value, (list, tuple)):
            return [value]
        return value

    def getSelectedAssembly():
        """ Returns the assembly for the selected object in Maya """
        rootNodes = []
        selection = pm.ls(sl=True)
        if not selection:
            return None
        for obj in selection:
            rootNodes.append(obj.root())
        assemblies = pm.ls(list(set(rootNodes)), assemblies=True)
        if len(assemblies) > 1:
            raise Exception("Select only 1 asset at a time to import XGen with")
        if not assemblies:
            LOG.debug("No assemblies could be found for selection")
            return None
        return asList(assemblies)[0]

    def getNamespaceForAssembly():
        """ Gets the namespace for the selected assembly in Maya """
        assembly = getSelectedAssembly()
        namespace = None
        if assembly:
            if pm.attributeQuery('Namespace', node=assembly, exists=True):
                namespace = pm.getAttr('{assembly}.Namespace'.format(assembly=assembly))
            if not namespace:
                assembly = getSelectedAssembly()
                if assembly:
                    namespace = assembly.split(':')[0]
                else:
                    assembly = pm.ls(sl=True)[0]
                    namespace = assembly.split(':')[0]
            if not namespace:
                LOG.warning("Couldn't find namespace")
                namespace = ""
        return namespace

    def importDialogCallback(QImportDialog):
        """Populates Import Dialog"""
        LOG.debug('Populating Import Dialog...')
        namespace = core.getNamespaceForAssembly()
        LOG.debug('Setting dialog namespace to: {0}'.format(namespace))
        QImportDialog.nameSpace.textValue.setText(namespace)

Next we use the preImportCallback to build a list of data paths to use in the new collection, create a copy of the XGen file being imported, then parse and modify all the required fields, essentially creating a new XGen file for the import to use.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
    def preImportCallback(xgenFile, data):
        """ Gets data paths from file and adds to list to use in new collection """
        xgDataPath = xgm.XgExternalAPI.getAttrFromFile('xgDataPath', 'Palette', xgenFile)
        if not xgDataPath:
            raise Exception("Could not read or find XGen file: {0}".format(xgenFile))

        # Try and get a project path based on the Shotgun connection to the XGen file, use the workspace default if we can't find one.
        try:
            project = trak.Project.from_path(xgenFile)
            root = project.primary_root()
            project_path = root.path(project)
        except:
            LOG.warning("Couldn't get project path for XGen file - using current workspace")
            project_path = str(pm.workspace(fn=True))

        xgDataPath = filter(None, xgDataPath[0].split(';'))
        xgDataPath = [xgm.expandFilepath(path, project_path) for path in xgDataPath]

        CollectionCallback.IMPORT_PATHS = asList(xgDataPath)  # We store these paths on a class object until we need them later
        editFile(xgenFile, data)

    def editFile(xgenFile, data):
        LOG.debug('Running File Editor...')
        with ROOTLOG:
            editor = XgenFileEditor(xgenFile)
            fileDescriptions = xgm.XgExternalAPI.getAttrFromFile('name', 'Description', xgenFile)

            path = editor.run(descriptionOverride=(getPrefix(), 'prepend'), collectionOverride=(getPrefix(),'prepend'))
            data['path'] = path
            LOG.debug('New Xgen File Path: {0}'.format(path))
            return path

    def getPrefix():
        """ Generate a prefix standard name for a the current scene """
        prefix = ''
        try:
            entity = trak.Entity.from_path(pm.sceneName())
        except Exception:
            entity = None
        if entity:
            entity.load_field('code')
            prefix += entity['code']
        else:
            name = os.path.basename(pm.sceneName())
            name = name.split('.')[0]
            if len(name.split('_')) > 1:
                prefix += name.split('_')[0]
            else:
                if len(name) > 6:
                    prefix += name[0:6]
                else:
                    prefix += name[0:3]
        prefix += '__'
        namespace = core.getNamespaceForAssembly()
        if not namespace:
            namespace = core.getSelectedAssembly()
            if namespace:
                namespace = namespace.split(':')[0]
        if not namespace:
            namespace = pm.ls(sl=True)[0]
        prefix += namespace
        LOG.debug('Namespace set as: {0}'.format(namespace))
        LOG.debug('Prefix set as: {0}'.format(prefix))
        return prefix

The class we use to edit the XGen files with is provided below. It's goal is to be able to take in new collection and new description names, and go through and make those changes throughout the file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
    class XgenFileEditor():
        """ Used to edit xgen files and provide custom overrides to certain settings """
        def __init__(self, xgenFile, ):
            self.filename = xgenFile
            self.collectionRegex = r'Palette'
            self.descriptionRegex = r'Description'
            self.patchRegex = r'Patches\s(?P<value>.*?)\t'
            self.geometryRegex = r'Patch\s.*'
            self.nameRegex = r'\s+name\s+(?P<value>.*)'

        def run(self, descriptionOverride=None, collectionOverride=None, geometryOverride=None):
            ''' Overrides require a tuple, (NewText, Mode), where mode is either 'replace', 'prepend' or 'append' '''
            if not collectionOverride and not descriptionOverride and not geometryOverride:
                raise Exception('No overrides provided for xgen file editor')
            with open(self.filename) as oldFile:
                tempPath = os.path.join(tempfile.gettempdir(), 'xgenTemporaryImportFile.xgen')
                if os.path.isfile(tempPath):
                    os.remove(tempPath)
                with open(tempPath, 'w') as tempFile:
                    nextLine = False
                    for currentLine in oldFile:
                        newLine = None
                        if nextLine:
                            nameMatch = re.match(self.nameRegex, currentLine)
                            if nameMatch:
                                newLine = self.editLine(nextLine, nameMatch, currentLine)
                                LOG.debug('New line to write: {0}'.format(newLine))
                                tempFile.write(newLine)
                                nextLine = False
                        if descriptionOverride and not newLine:
                            descriptionMatch = re.match(self.descriptionRegex, currentLine)
                            if descriptionMatch:
                                nextLine = descriptionOverride
                        if descriptionOverride and not newLine:
                            patchMatch = re.match(self.patchRegex, currentLine)
                            if patchMatch:
                                newLine = self.editLine(descriptionOverride, patchMatch, currentLine)
                                LOG.debug('New line to write: {0}'.format(newLine))
                                tempFile.write(newLine)
                        if collectionOverride and not newLine:
                            collectionMatch = re.match(self.collectionRegex, currentLine)
                            if collectionMatch:
                                nextLine = collectionOverride
                        if geometryOverride and not newLine:
                            geometryMatch = re.match(self.geometryRegex, currentLine)
                            if geometryMatch:
                                nextLine = geometryOverride
                        if not newLine:
                            newLine = currentLine
                            tempFile.write(newLine)
            return path_normalize(tempPath)

        def editLine(self, override, match, currentLine):
            matchData = match.groupdict()
            value = matchData['value']
            text = override[0]
            mode = override[1]
            if mode == 'prepend':
                newValue = '{0}__{1}'.format(text, value)
            elif mode == 'append':
                newValue = '{0}__{1}'.format(value, text)
            elif mode == 'replace':
                newValue = value
            else:
                raise Exception("Invalid mode provided: {0}".format(mode))
            LOG.debug("New override value: {0}".format(newValue))
            newLine = currentLine.replace(value, newValue)
            return newLine

Now for the postImportCallback, where we make sure grooms are all reconnected correctly and connect maps and attributes. The great thing about this setup is the newly created XGen collection still references the contents like grooms and paint maps from the original XGen file. Only once the user decides to paint or change something in the new XGen file, does it copy over the old XGen file and create a completely new connection. This is where the majority of the code we had to write is located.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
    def postImportCallback(xgenFile, originalXgenFile):
        LOG.debug("Performing Post Import Functions...")
        with ROOTLOG:
            fileDescriptions = asList(xgm.XgExternalAPI.getAttrFromFile('name', 'Description', xgenFile))
            LOG.debug("Descriptions Found in File: {0}".format(fileDescriptions))
            fileCollection = xgm.XgExternalAPI.getAttrFromFile('name', 'Palette', xgenFile)[0]
            matchingPalette = setMatchingPalette(fileCollection)

            setupCollectionMaps(fileDescriptions)
            reconnectGrooms(fileDescriptions, xgenFile, originalXgenFile)
            setupCollectionAttrs(fileDescriptions, fileCollection)
        LOG.debug("Finished Importing Xgen File")

    def setMatchingPalette(palette):
        for i in range(xggm.DescriptionEditor.palettes.count()):
            scenePalette = xggm.DescriptionEditor.palettes.itemText(i)
            if palette in scenePalette or scenePalette in palette:
                xggm.DescriptionEditor.palettes.setCurrentIndex(i)
                xggm.DescriptionEditor.refresh("Palette")
                return scenePalette

    def setMatchingDescription(description):
        for i in range(xggm.DescriptionEditor.descs.count()):
            sceneDesc = xggm.DescriptionEditor.descs.itemText(i)
            if description in sceneDesc or sceneDesc in description:
                xggm.DescriptionEditor.descs.setCurrentIndex(i)
                xggm.DescriptionEditor.refresh("Description")
                return sceneDesc

    def setupCollectionMaps(fileDescriptions):
        """ Modifies attributes to match imported descriptions"""
        for fileDescription in fileDescriptions:
            # xgm.ui.createDescriptionEditor(True).preview(True)
            splitDescription = fileDescription.split('__')
            description = None
            namespace = None
            if len(splitDescription) == 3:
                description = splitDescription[2]
                namespace = splitDescription[1]

            elif len(splitDescription) == 2:
                description = splitDescription[1]
                namespace = splitDescription[0]
            elif len(splitDescription) > 1:
                description = splitDescription[-1]
            else:
                description = splitDescription[0]
            matchedDescription = setMatchingDescription(description)
            if matchedDescription:
                LOG.debug('Old Map Description: {0}'.format(description))
                possibleMaps = core.getPaintableAttrs()
                LOG.debug("Possible Maps: {0}".format(possibleMaps))
                renameImportedMaps(description, possibleMaps)
                fxModules = xgm.fxModules(xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription())
                renameImportedModifiers(description, fxModules)

    def renameImportedMaps(description, possibleMaps):
        """ Rename imported maps """
        for attr, attrObject in [obj.split('.') for obj in possibleMaps]:
            if not xgm.attrExists(attr, xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), attrObject):
                continue
            currVal = xgm.getAttr(attr, xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), attrObject)
            if not currVal:
                continue
            newVal = currVal.replace('${DESC}', description)
            if not newVal == currVal:
                xgm.setAttr(attr, newVal, xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), attrObject)
                LOG.debug("Reconnected Map Attr from: {0} to: {1}".format(currVal, newVal))

    def renameImportedModifiers(description, fxModules):
        """ Renaming imported modifiers """
        for module in fxModules:
            attrObject = str(module)
            attrs = xgm.attrs(xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), module)
            for attr in attrs:
                if not xgm.attrExists(attr, xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), attrObject):
                    continue
                currVal = xgm.getAttr(attr, xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), attrObject)
                if not currVal:
                    continue
                newVal = currVal.replace('${DESC}', description)
                if not newVal == currVal:
                    xgm.setAttr(attr, newVal, xggm.DescriptionEditor.currentPalette(), xggm.DescriptionEditor.currentDescription(), attrObject)
                    LOG.debug("Reconnected Modifier Attr from: {0} to: {1}".format(currVal, newVal))

    def reconnectGrooms(fileDescriptions, xgenFile, originalXgenFile):
        """ Import and Reconnect Grooms """
        fileGrooms = asList(xgm.XgExternalAPI.getAttrFromFile('groom', 'Description', xgenFile))
        LOG.debug("Grooms found for imported file: {0}".format(fileGrooms))
        for index, groom in enumerate(fileGrooms):
            if not groom:
                continue
            fileDescription = fileDescriptions[index]
            splitDescription = fileDescription.split('__')
            description = None
            namespace = None
            if len(splitDescription) == 3:
                description = splitDescription[2]
                namespace = splitDescription[1]
            elif len(splitDescription) == 2:
                description = splitDescription[1]
                namespace = splitDescription[0]
            elif len(splitDescription) > 1:
                description = splitDescription[-1]
            else:
                description = splitDescription[0]
            matchedDescription = setMatchingDescription(description)
            if not matchedDescription:
                continue
            base = os.path.split(originalXgenFile)[0]
            xgenDir = os.path.join(base, 'xgen', description, 'groom')
            LOG.debug("Expected Groom Directory: {0}".format(xgenDir))
            if os.path.exists(xgenDir):
                LOG.debug("Found Groom, importing from: {0}".format(xgenDir))
                connectGroom(xgenDir, xggm.DescriptionEditor.currentDescription())
            else:
                LOG.warning("Could not find path for groom: {0}, groom was not imported!".format(groom))

    def connectGroom(groomFolder, description):
        igDescr = xgm.igDescription(description)
        expandedPath = xgm.ui.widgets.xgBrowseUI.FileBrowserUI.folderExists(groomFolder, description, None)
        if expandedPath:
            try:
                pm.waitCursor(state=True)
                pm.mel.eval('iGroom -im "{0}" -d "{1}";'.format(expandedPath, igDescr))
            finally:
                pm.waitCursor(state=False)
            xgm.XGWarning(3, 'Groom imported for description <{0}> imported from: {1}'.format(description, expandedPath))
            LOG.debug("Connected Groom Succesfully")

    def setupCollectionAttrs(fileDescriptions, fileCollection):
        """ Add imported attributes to collection for debug purposes and to keep track of where things came from"""
        descriptions = []
        for description in fileDescriptions:
            split = description.split('__')
            if len(split) == 3:
                descriptions += split[2],

            elif len(split) == 2:
                descriptions += split[1],
            elif len(split) > 1:
                descriptions += split[-1],
            else:
                descriptions += split[0],

        currentCollection = xggm.DescriptionEditor.currentPalette()
        LOG.debug('Adding attributes to current collection connecting imported file...')
        pm.addAttr(currentCollection, ln="ImportedCollection", dt="string")
        pm.setAttr('{0}.ImportedCollection'.format(currentCollection), fileCollection, type="string")
        pm.addAttr(currentCollection, ln="ImportedDescription", dt="string")
        pm.setAttr('{0}.ImportedDescription'.format(currentCollection), ', '.join(descriptions), type="string")

To use the xgDataPath we stored before we have a class to manage collection paths.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
    class CollectionCallback():
        IMPORT_PATHS = []

        def __init__(self, *args):
            self.setupPath(args=args)

        def setupPath(self, args=None):
            newPath = ''
            relPath = self.relativePath
            pathList = [relPath]
            pathList.extend(self.IMPORT_PATHS)
            self.IMPORT_PATHS = []
            pathList.append(self.projectPath)
            paths = filter(None, pathList)
            paths = [path[:-1] if path[-1] == '/' else path for path in paths]
            paths = self.removeDuplicates(paths)
            for path in paths:
                path = path_normalize(path)
                if path:
                    path = path[0].lower() + path[1:]
                root = xgm.getProjectPath()
                if root:
                    root = root[0].lower() + root[1:]
                if root in path:
                    relative = path.split(root)[-1]
                    newPath += ('${{PROJECT}}{0};'.format(relative))
                else:
                    newPath += ('{0};'.format(path))
            if not newPath:
                LOG.warning("No paths found, is the scene saved?")
                return
            numberOfPaths = filter(None, newPath.split(';'))
            if len(numberOfPaths) == 1 and newPath[-1] == ';':
                newPath = newPath[0:-1]
            if args:
                self.setPath(newPath, args[0])
            else:
                try:
                    collection = xggm.DescriptionEditor.currentPalette()
                except:
                    pass
                if collection:
                    self.setPath(newPath, collection)

        @property
        def entityPath(self):
            path = pm.sceneName()
            if not os.path.isfile(path):
                return None
            try:
                entity = trak.Entity.from_path(path)
            except:
                LOG.warning("File not an entity file: {0}".format(path))
                return None
            if entity['type'] not in ['Asset', 'Shot']:
                return None
            root = entity.primary_root()
            root_path = root.path(entity)
            return os.path.join(root_path, 'xgen')

        @property
        def relativePath(self):
            path = pm.sceneName()
            path = os.path.split(path)[0]
            if path == '':
                return None
            path = os.path.join(path, 'xgen')
            if not os.path.exists(path):
                os.makedirs(path)
            return path

        @property
        def projectPath(self):
            path = pm.workspace(fn=True)
            path = os.path.join(path, 'xgen')
            if os.path.exists(path):
                return path
            else:
                return None

        def removeDuplicates(self, paths):
            items = set()
            items_add = items.add
            return [x for x in paths if x not in items and not items_add(x)]  # Used because a normal set changes order which we dont want

        def setPath(self, newPath, collection):
            return xgm.setAttr("xgDataPath", str(newPath), collection)

Now all thats left is to create our exporter. There were a lot of problems we ran into when developing this. We noticed an extremely high change for Maya to crash when doing a simple export the more frames there were in the scene. The solution was simple and involves exporting one frame to a file at a time. Bounding box generation is also tricky, since XGen curves don't count in Maya when doing any of the bounding box calculations. Our solution to this was create a series of curves to represent the bounding box (Something that wouldn't render), that would cause Maya to calculate bounding boxes correctly.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
    XGEN_OUTPUT_OPERATION_RENDER = 'RendermanRenderer'

    def getCollectionPatches(collection=None):
        if not collection:
            return None
        descriptions = xgenm.descriptions(collection)
        geometry = []
        for d in descriptions:
            boundGeo = xgenm.boundGeometry(collection, d)
            if boundGeo:
                geometry.extend(boundGeo)
        return geometry

    def createBoundingBoxForCollection(collection, startFrame=None, endFrame=None, convertToCurves=True):
        """
        Create a bounding box for an xgen collection. This is needed when exporting since an xgen collection
        by itself doesn't calculate into the bounding box size of a standin once imported. Arnold currently
        doesn't support exporting bounding boxes so they will need to be converted to be used once imported.

        Args:
            collection (xgen Collection / Palette)
            startFrame (int or None): If None, uses start of play range
            endFrame (int or None): If None, uses end of play range
        Returns:
            List: Bounding Box Cubes
        """
        LOG.debug("Creating Bounding Boxes for Collection: {0}".format(collection))
        with ROOTLOG:
            # Get selection to save / restore later
            selection = pm.ls(sl=True)
            pm.cmds.select(cl=True)
            patches = getCollectionPatches(collection)
            LOG.debug("Patches: {0}".format(patches))

            if startFrame is None:
                pm.playbackOptions(q=True, min=True)
            if endFrame is None:
                pm.playbackOptions(q=True, max=True)

            # We do one bounding box at a time here to prevent crashing in MayaPy
            bboxes = []
            curves = []
            for patch in patches:
                pm.select(patch)
                _bbox = pm.cmds.geomToBBox(
                    patch,
                    nameSuffix="_BBox",
                    single=True,
                    # shaderColor=(0.5, 0.5, 0.5),
                    startTime=startFrame,
                    endTime=endFrame,
                    sampleBy=1,
                    bakeAnimation=True,
                    keepOriginal=True
                )

                if convertToCurves:
                    shapes = pm.listRelatives(_bbox, shapes=True, f=True)
                    for shape in shapes:
                        for edge in shape.e:
                            curve = pm.polyToCurve(edge)
                            if isinstance(curve, list):
                                curve = curve[0]
                            curves.append(curve)
                bboxes.append(_bbox)
            pm.cmds.select(selection)

            if convertToCurves:
                bboxParents = [pm.listRelatives(b, p=True) for b in bboxes]
                pm.delete(bboxParents)
                return curves
            return bboxes


    def getMotionBlurFrames():
        """
        Gets the number of frames before and after each frame that would be required for Arnold motion blur to work correctly
        The number is based on motion blur settings in the arnold render settings.
        Returns:
            Tuple: (Frames needed before for motion blur, Frames needed after for motion blur)
        """
        # Get motion blur frame settings
        preFrames = 0
        postFrames = 0
        if pm.objExists('defaultArnoldRenderOptions'):
            if pm.getAttr('defaultArnoldRenderOptions.motion_blur_enable'):
                blurType = pm.getAttr('defaultArnoldRenderOptions.range_type')
                blurLength = pm.getAttr('defaultArnoldRenderOptions.motion_frames')
                if blurType == 0:  # Start On Frame
                    postFrames = int(math.ceil(blurLength))
                elif blurType == 1:  # Center On Frame
                    preFrames = int(math.ceil(blurLength/2))
                    postFrames = int(math.ceil(blurLength/2))
                elif blurType == 2:  # End On Frame
                    preFrames = int(math.ceil(blurLength))
                elif blurType == 3:  # Custom
                    preFrames = int(math.ceil(abs(pm.getAttr('defaultArnoldRenderOptions.motion_start'))))
                    postFrames = int(math.ceil(abs(pm.getAttr('defaultArnoldRenderOptions.motion_end'))))
        return preFrames, postFrames


    class XgenCollectionsToStandinExporter(object):
        def __init__(self, outputFolder, collections=None, variation='main', startFrame=None, endFrame=None, expandProcedurals=True, translateCallback=None, compress=True, createBoundingBoxes=True, **kwargs):
            """
            Convert a list of collections to standins for a range of frames

            Args:
                outputFolder (str): Path to the outputFolder for where to export the standins
                collections (list or None): List of xgen collections.
                    If list of collections, must be python instances of Xgen collections.
                    If not supplied, use all collections in the current scene.
                startFrame (int or None): Start frame for the export.
                    If not supplied, use the start of the play range
                endFrame (int or None): End frame for the export.
                    If not supplied, use the end of the play range
            """
            self.outputFolder = outputFolder
            self.expandProcedurals = expandProcedurals
            self._collections = None
            self._startFrame = None
            self._end = None
            self._translateCallback = translateCallback
            self.kwargs = kwargs

            self.variation = variation
            self.collections = collections
            self.startFrame = startFrame
            self.endFrame = endFrame
            self.compress = compress
            self.createBoundingBoxes = createBoundingBoxes

        @property
        def startFrame(self):
            if self._startFrame is None:
                self._startFrame = pm.playbackOptions(q=True, min=True)
            return self._startFrame

        @startFrame.setter
        def startFrame(self, value):
            self._startFrame = value

        @property
        def endFrame(self):
            if self._endFrame is None:
                self._endFrame = pm.playbackOptions(q=True, max=True)
            return self._endFrame

        @endFrame.setter
        def endFrame(self, value):
            self._endFrame = value

        @property
        def collections(self):
            if self._collections is None:
                self._collections = [str(x) for x in pm.ls(type="xgmPalette")]
            return self._collections

        @collections.setter
        def collections(self, value):
            if not value:
                self._collections = [str(x) for x in pm.ls(type="xgmPalette")]
            else:
                if not isinstance(value, list):
                    value = envtools.asList(value)
                self._collections = [str(x) for x in value]
            return self._collections

        def start(self):
            self.run()

        def run(self):
            startTime = time.time()
            revertCallbacks = self._prepare()
            try:
                for col in self.collections:
                    self._prepareAndExportCollection(col)
            finally:
                self._cleanup(revertCallbacks)
                totalTime = time.time() - startTime
            return True

        def _prepare(self):
            """ Prepare the scene for export. Set required global arnold and xgen settings """
            revertCallbacks = []
            revertCallbacks.extend(self._setRequiredArnoldSettings())
            return revertCallbacks

        def _setRequiredArnoldSettings(self):
            LOG.debug("Setting Required Arnold Path Settings")

            pm.setAttr('defaultRenderGlobals.ren', 'arnold', type='string')
            mtoa.core.createOptions()

            originalAbsoluteTexturePath = pm.getAttr("defaultArnoldRenderOptions.absoluteTexturePaths")
            originalAbsoluteProceduralPath = pm.getAttr("defaultArnoldRenderOptions.absoluteProceduralPaths")
            originalTextureSearchPath = pm.getAttr("defaultArnoldRenderOptions.texture_searchpath")

            pm.setAttr("defaultArnoldRenderOptions.absoluteTexturePaths", 1)
            pm.setAttr("defaultArnoldRenderOptions.absoluteProceduralPaths", 0)
            pm.setAttr("defaultArnoldRenderOptions.texture_searchpath", "[MBOT_RENDER_ROOT]", type="string")

            def revertArnoldSettings():
                LOG.debug("Reverting Arnold Path Settings")
                pm.setAttr("defaultArnoldRenderOptions.absoluteTexturePaths", originalAbsoluteTexturePath)
                pm.setAttr("defaultArnoldRenderOptions.absoluteProceduralPaths", originalAbsoluteProceduralPath)
                pm.setAttr("defaultArnoldRenderOptions.texture_searchpath", originalTextureSearchPath, type="string")
            return [revertArnoldSettings]

        def _cleanup(self, revertCallbacks):
            """ Cleanup after export and revert any changed settings that occurred during prepare """
            for cb in revertCallbacks:
                cb()

        def _prepareAndExportCollection(self, collection):
            """ Export the specified collection """
            LOG.debug("Preparing and Exporting collection {0}".format(collection))
            with ROOTLOG:
                revertCallbacks = []
                revertCallbacks.extend(self._prepareCollection(collection))

                try:
                    results = self._exportCollection(collection)
                finally:
                    LOG.debug("Cleaning Up Export")
                    with ROOTLOG:
                        self._cleanup(revertCallbacks)
            return results

        def _exportCollection(self, collection, additionalItems=[]):
            """ Preforms the actual export code """
            results = []

            # Get a filename based on the collection
            cleanName = collection
            fullCleanName = cleanName.replace(':', '_').replace('|', '_')
            shortCleanName = cleanName = cleanName.split(':')[-1].replace('|', '_')
            cleanName = shortCleanName

            path = pm.sceneName()
            # Generate a unique filename to use for output
            outputPath = generateOutputPath(path, cleanName)

            LOG.debug("Ass Output Path: {0}".format(outputPath))
            outputDir = os.path.dirname(outputPath)
            if not os.path.exists(outputDir):
                os.makedirs(outputDir)

            # Store and clear our current selection and select what we need to export
            selection = pm.ls(sl=True)
            pm.cmds.select(cl=True)
            pm.select([collection] + additionalItems)
            pm.refresh()

            LOG.debug("Exporting: {0}".format(pm.ls(sl=True)))

            LOG.debug("Calculating extra frames for patches and motion blur")
            frames = range(self.startFrame, self.endFrame + 1)
            preFrames, postFrames = getMotionBlurFrames()

            # Make Sure we always have at least 2 frames to work with since export patches needs it
            preFrames = max(preFrames, 2)
            postFrames = max(postFrames, 2)

            # Export each frame to a file
            # We only do one at a time due random crashes, especially across large frameranges
            for frame in frames:

                self.exportPatches(frame-preFrames, frame+postFrames)
                pm.setCurrentTime(frame)
                result = pm.arnoldExportAss(
                    f=outputPath,
                    s=True,
                    startFrame=frame,
                    endFrame=frame,
                    frameStep=1.0,
                    mask=255,
                    lightLinks=1,
                    shadowLinks=1,
                    boundingBox=False,
                    expandProcedurals=self.expandProcedurals,
                    asciiAss=False,
                    compressed=self.compress,
                    )

                if self._translateCallback and callable(self._translateCallback):
                    self._translateCallback(result, **self.kwargs)

                if self.createBoundingBoxes:
                    bboxPath = result[0]
                    if bboxPath.endswith('.gz'):
                        bboxPath = bboxPath[:-3]
                    bboxPath = bboxPath + 'toc'
                    descShapes = []
                    descriptions = xgenm.descriptions(collection)
                    for description in descriptions:
                        transforms = pm.listRelatives(description, type='transform')
                        for transform in transforms:
                            descShape = pm.listRelatives(transform, type='shape')[0]
                            if isinstance(descShape, pm.nodetypes.XgmSubdPatch):
                                descShapes.append(descShape)

                    x1, y1, z1, x2, y2, z2 = pm.exactWorldBoundingBox(descShapes, calculateExactly=True)
                    data = "bounds {x1} {y1} {z1} {x2} {y2} {z2}".format(x1=x1, y1=y1, z1=z1, x2=x2, y2=y2, z2=y2)

                    with open(bboxPath, 'w') as f:
                        f.write(data)
                results.append(result[0])
            pm.cmds.select(selection)
            return results

        def _prepareCollection(self, collection):
            """ Prepares the collection by setting any needed settings and creating needed bounding boxes """
            revertCallbacks = []

            for description in xgenm.descriptions(collection):
                # Set the renderer - no need to revert
                self.setRenderer(collection, description, "Arnold Renderer")

                # Set the renderer operation
                prevRenderOp = self.getRendererOperation(collection, description)
                self.setRendererOperation(collection, description, XGEN_OUTPUT_OPERATION_RENDER)

                def revertRenderOperation():
                    self.setRendererOperation(collection, description, prevRenderOp)
                revertCallbacks.append(revertRenderOperation)

                prevRenderMode = self.getRenderMode(collection, description)
                self.setRenderMode(collection, description, 1)

                def revertRenderMode():
                    self.setRenderMode(collection, description, prevRenderMode)
                revertCallbacks.append(revertRenderMode)

            return revertCallbacks

        def exportPatches(self, startFrame=None, endFrame=None, path=None, dryRun=False):
            results = {}
            if not pm.sceneName():
                raise Exception("Scene Not Saved")
            scene = pm.sceneName()
            sceneName = scene.basename().splitext()[0]
            if not path:
                path = os.path.split(scene)[0]
            cmdAlembicBase = 'AbcExport -j "'
            if startFrame and endFrame:
                cmdAlembicBase = cmdAlembicBase + '-frameRange '+str(startFrame)+' '+str(endFrame)
            cmdAlembicBase = cmdAlembicBase + ' -uvWrite -attrPrefix xgen -worldSpace -stripNamespaces'
            palette = pm.cmds.ls(exactType="xgmPalette")
            for p in range(len(palette)):
                filename = envtools.path_normalize(os.path.join(path, sceneName + "__" + palette[p].replace(':', '__') + ".abc"))
                descShapes = pm.cmds.listRelatives(palette[p], type="xgmDescription", ad=True)
                cmdAlembic = cmdAlembicBase
                for d in range(len(descShapes)):
                    descriptions = pm.cmds.listRelatives(descShapes[d], parent=True)
                    if len(descriptions):
                        patches = xgenm.descriptionPatches(descriptions[0])
                        for patch in patches:
                            cmd = 'xgmPatchInfo -p "'+patch+'" -g'
                            geom = pm.mel.eval(cmd)
                            geomFullName = pm.cmds.ls(geom, l=True)
                            cmdAlembic += " -root " + geomFullName[0]
                cmdAlembic = cmdAlembic + ' -file ' + filename + '";'
                LOG.debug('Export Patches Command: {0}'.format(cmdAlembic))
                if not dryRun:
                    pm.mel.eval(cmdAlembic)
                results[palette[p].replace(':', '__')] = filename
            return results

        def getRendererOperation(self, collection, description):
            operation = xgenm.getActive(collection, description, 'Renderer')
            return operation

        def setRendererOperation(self, collection, description, operation):
            try:
                xgenm.setActive(collection, description, operation)
            except Exception, e:
                raise Exception("Error setting renderer operation: {0}".format(e))

        def setRenderer(self, collection, description, renderMethod=None):
            if not renderMethod:
                renderMethod = 'Arnold Renderer'
            if not self.getRendererOperation(collection, description) == 'RendermanRenderer':
                raise Exception("Renderer Operation Not Set To Render")
            try:
                xgenm.setAttr('renderer', renderMethod, collection, description, 'RendermanRenderer')
            except Exception, e:
                raise Exception("Error setting renderer: {0}".format(e))

        def getRenderMode(self, collection, description):
            if not self.getRendererOperation(collection, description) == 'RendermanRenderer':
                raise Exception("Renderer Operation Not Set To Render")
            try:
                mode = xgenm.getAttr('custom__arnold_rendermode', collection, description, 'RendermanRenderer')
            except Exception, e:
                raise Exception("Error getting render mode: {0}".format(e))
            return mode

        def setRenderMode(self, collection, description, mode):
            if not self.getRendererOperation(collection, description) == 'RendermanRenderer':
                raise Exception("Renderer Operation Not Set To Render")
            try:
                xgenm.setAttr('custom__arnold_rendermode', str(mode), collection, description, 'RendermanRenderer')
            except Exception, e:
                raise Exception("Error setting render mode: {0}".format(e))

From here it was just a matter of getting the file structure and naming down. Some other tools I won't go over here, but that were useful to implement include:

  • Paint map copier, edit XGen code to copy paint maps from the asset folder to and from the project 3dPaintTexture folder using pre and post paint callbacks.
  • Implementing live XGen with render farm to prevent the need of caching all XGen to standins.
  • XGen publishing / Copies, verifies, and connects versions of XGen to Shotgun.
  • Keep your XGen projects in Perforce or some other versioning software, these files get corrupted or just break all the time.

Example of XGen hair using tools

The above and below images are from the two first projects we used these XGen tools on. We had an extremely short deadline to get XGen up and going and had to rush through most of the code and planning. It was easily the most difficult tools to develop so far. I highly recommend skipping to Maya 2016 or higher to avoid some of these issues and not have to go through so much trouble. It was however still a success, the hair in our project ended up looking great, and since artists had to suffer shortly without the tools they were exceptionally glad to have something that made them work when we were done.

Example of XGen hair by using tools