[Maya] The case of Xgen and [Error 6] The handle is invalid

This post is about a Maya issue on Window. It’s not an MtoA or Arnold issue.

Whenever I tried to export the selection as an XGen archive, I’d get [Error 6] The handle is invalid.

[Error 6] The handle is invalid
xgmArchiveExportBatchUI.py except block
xgmArchiveExportBatchUI.py call subprocess.Popen
mayapy "C:/Program Files/Autodesk/Maya2014/plug-ins/xgen/scripts/xgenm/xmaya/xgmArchiveExportBatch.py" 0 0 0 1.0 1.0 -destName ["'DragonArchive'"] -destDir ["'C:/Users/SOLIDANGLE/Documents/maya/projects/Support/xgen/archives/'"] -sourceFiles ["'C:/Users/SOLIDANGLE/Documents/maya/projects/Support/scenes/Dragon1.mb'"] -scriptPaths ["'C:/Program Files/Autodesk/Maya2014\\plug-ins\\xgen\\scripts'"] -loadPlugins ["'AbcExport'", "'Mayatomr'", "'xgenMR.py'"]
xgmArchiveExportBatchUI logging to C:/Users/SOLIDANGLE/Documents/maya/projects/Support/xgen/archives/DragonArchive.log

Googling [Error 6] The handle is invalid subprocess.Popen led me to the problem: I was starting Maya from the command line. When I started Maya from the Start menu, then the XGen Export Selection worked.

The reason I start Maya from the command line is because that’s the only way I can get the MtoA diagnostics in the Output Window. So something seems a little weird about how Maya handles STDOUT and STDERR 😉

[Maya] Socket error creating command port

NOTE: This is a Maya issue that can happen with any renderer, including mental ray.

Like many other Maya users, I started getting this error when I tried to batch render:

// Error: line 1: Socket error creating command port SOLIDANGLE-PC:7835 (10048).

I checked the port 7835 with netstat, and that gave me a PID (process ID) of 109736

C:\Program Files\Autodesk\Maya2014\bin>netstat -ano | find "7835"
  TCP    XXX.XXX.XXX.XXX:7835              LISTENING       109736

But unfortunately, there didn’t seem to be any process with that PID, so I couldn’t kill it to free up the port. tasklist | find “109736” came up empty; there was no such PID in the Task Manager or Process Explorer; and TCP View said that PID was non-existent.
I was resigned to having to reboot my machine until I saw this. Then I used Process Explorer to find the process that was referencing the “non-existent” PID, and when I killed that process, I could batch render again from Maya.

Another case of a missing entry point

In this case, a user installed the latest MtoA, but when he went to load mtoa.mll in the Plug-in Manager, he got this error:

Settings/Preferences: maya.exe - Entry Point Not Found
The procedure entry point ?IsRenderablePath@CArnoldSession@@SA_NVMDagPath@@@Z could not be located in the dynamic link library mtoa_api.dll. 

From past experience, I knew that the problem was probably the Maya.env file. And sure enough, it looked something like this:

MAYA_RENDER_DESC_PATH = C:\solidangle\mtoadeploy\2013.0.22.0
PATH = %PATH%;C:\solidangle\mtoadeploy\2013.0.22.0\bin;

MAYA_RENDER_DESC_PATH = C:\solidangle\mtoadeploy\2013.0.22.1
PATH = %PATH%;C:\solidangle\mtoadeploy\2013.0.22.1\bin;

Notice that there are two MAYA_RENDER_DESC_PATH and PATH settings. Maya reads just the top two, which are for an older version of mtoa, so when Maya loads mtoa.mll we end up with a “missing entry point” error.

How did this happen? Well, in this case, in-between installs and uninstalls of different versions MtoA, the user had updated Maya.env with his own MAYA_RENDER_DESC_PATH and PATH settings for MtoA. And then the next install of MtoA appended its own settings to the Maya.env file.

The MtoA Installer works like this:

  • Installs make a backup of the existing Maya.env, and then append the environment settings to the existing Maya.env.
  • Uninstalls restore the backup version of Maya.env.

So if your Maya.env already has MtoA settings for MAYA_RENDER_DESC_PATH and PATH settings, then when you install another MtoA, you end up with a problem (because the new settings will be at the bottom, where they are ignored).

Ok, so that’s problem solved. Just remove those extra entries from Maya.env.

[MtoA] Editing Arnold attributes on multiple objects

If you need to change the same Arnold attributes on many nodes, use the Attribute Spreadsheet. For example, suppose you wanted to turn off the Opaque attribute on a number of objects in your scene. Here’s how to do it:

  1. Select the objects.
  2. Press the Down arrow (aka Pick Walk) to select the shape nodes.
  3. Open the Attribute Spreadsheet.
  4. Click the All tab, and find the Ai (Arnold) attributes you want to change. Drag across all rows, and then type “off” in the last row.

Exporting user data parameters to Arnold from Maya

You can add attributes to shape nodes in Maya, and MTOA will export those extra attributes as user data in Arnold shape nodes. For example, you use this mechanism to add user data to a procedural node.

Add some extra attributes to a shape node, using the naming convention mtoa_constant_parameter-name. That is, the attribute name must always start with “mtoa_constant_”.

Export to ASS. In the .ass file, your extra attributes will be declared as user parameters:

 name ArnoldStandInShape
 dso "C:/Users/SOLIDANGLE/Dev/MyProcedural.dll"
 min -9.63329697 0 0.861843109
 max -6.73797035 3.62841082 9.87797546
  1 0 0 0
  0 1 0 0
  0 0 1 0
  0 0 0 1 
 declare MyVector constant VECTOR
 MyVector 0 0 0
 declare MyFloat constant FLOAT
 MyFloat 0
 declare MyInteger constant INT
 MyInteger 0
 declare MyBoolean constant BOOL
 MyBoolean off
 declare MyString constant STRING
 MyString ""