Vicon Shogun Post 1.8 - Merging StretchSense Data

Pre-requisites: 

• Hand Engine 2.X or later
• Vicon Shogun Post 1.8 or later
• Latest StretchSense Script from Vicon 

• https://www.vicon.com/software/models-and-scripts/stretchsense-scripts/

• Latest VSS files from Vicon 

• https://www.vicon.com/software/models-and-scripts/vss/

Motion Capture Setup and Data Management: 

• Set the Performer name in the Hand Engine exactly the same as the performer name in Shogun Live. This will make sure that the automated merging of hand and body data will be successful. 
• Ensure that you have set up timecode sync between Shogun Live and Hand Engine, or that the two software are operating on the same PC with the same clock: 

• https://get.stretchsense.com/knowledge/hand-engine-user-guide-2.0#Timecode

• Before recording, ensure that the Master Timecode in Hand Engine is live. 
  
  Example: Shogun Live operating on same PC as Hand Engine:
  

 
Pre-Recording Check:
  

• Test the timecode stream to ensure that timecode is live in Hand Engine and the frame rate is set correctly. Each glove (Left and Right, for each performer) should also be jam-synced to this timecode)

• Ensure that you have set up remote triggering from Shogun Live to Hand Engine. This will ensure that recordings are started and stopped at the same time: 

• https://get.stretchsense.com/knowledge/hand-engine-user-guide-2.0#RemoteTrigger
  
  Setting in Shogun Live
  
  Setting in Hand Engine - Settings menu
  

 
Pre-Recording Check: 

1. Check that the take name is being updated in Hand Engine according to what is set in Shogun Live.
2. Test remote trigger, ensuring that Hand Engine starts and stops recording when it is triggered from Shogun Live.
3. Check the recording folder to ensure the take name is correct, and each take folder contains the left and right hand .FBX files. 

• Set the Hand Engine recording folder to the same as Shogun Live. This will ensure that all data recorded will be in the same place
• Ensure the frame rate is the same for the Hand data from Hand Engine and the body MoCap from Shogun Live. If you are not sure, you can check this by importing the recording FBX from Hand Engine into Shogun Post and checking the frame rate at the bottom right-hand side. 
• Make sure the time code for the Hand Recordings closely matches the body MoCap recordings to ensure alignment of the hand and body data. 

Setting the Take Folder:
For ease of data ingest, it is suggested to set the same capture path for both Hand Engine and Shogun Live. 

If you set the capture path in both Shogun and Hand Engine to:
G:\MOCAP\StretchSense\ViconCapture
The following are produced in that folder:
From Shogun: An MCP file named MyTake_001.mcp (Example)
From Hand Engine: A take folder named after the date, within that, subfolders named MyTake_001 etc. (Example), and within these take folders, the FBX files required for the data merge. 

NOTE: Hand Engine will create a take folder of the current date. You will need to copy the take folders from within this and move them into the root folder of the Vicon Takes. See the example below for clarification.
Example:
The root of the capture folder:

Within Hand Engine Take folder:


Shogun Post Script - Automatic Merge

Provided all the above prerequisites have been met, and the data is organized as per the above instruction, Follow these steps to merge the data. 

1. Open Shogun Post 1.8 and load the MCP file of the take containing the solved actor Vicon body skeleton(s)
2. In the script tab, run the following:
   ImportStretchSenseFBXFiles.hsl;
   
3. All hand data FBX files with the correct naming scheme will be imported and merged to the respective performers' skeleton in the scene. 

Shogun Post Script - Manual Merge
If the data from Hand Engine has a different performer name or does not meet the above naming schema, you can still manually merge the left and right-hand data onto the performers' skeleton in Shogun Post. 

1. Open Shogun Post 1.8 and load the MCP file of the take containing the solved actor Vicon body skeleton(s)
2. Set the current subject to the performer you wish to merge that hand data onto.
3. In the script tab, run the following:
   ImportStretchSenseFBXFiles.hsl;
4. A file browser window will open, please select the Left and Right hand .FBX files you wish to merge. 
5. Shogun Post will then merge the Hand data onto the solved actor vicon body skeleton. 

 
Troubleshooting
Hands are merging at odd angles (Pre-rotation set incorrectly)
If the hands, once merged, are offset in rotation from the wrist, you may need to edit the pre-rotation offsets in the importStretchSenseFBXFile.hsl script.   

1. Assess the pre-rotation by loading the Vicon .MCP and merging the hand data as per above instructions
2. Select the wrist of each hand and check the pre-rotation in the attributes  window. 
   
3. Adjust the pre-rotations in Shogun Post until the hands are aligned correctly with the wrist . Take note of these values (X, Y, Z)
   
   
   
4. Open importStretchSenseFBXFile.hsl  using a text editor such as Notepad++
5. Insert these changes into this part of the script. Note the first set of rotations is for the Left hand, and the second (After the "else") is for the right hand.
   The order is as follows:
   
   setProperty"Pre_Rotation" X Y Z -onMod $SS_HandName;
   
   Note in the below script section, the numbers have been updated to match the above images. 

// Remove translation offset and set pre rotation to  
// orient StretchSense hand relative to Vicon hand  
select $SS_HandName;  
selectProperty Translation;  
cutKeys -all;  
setProperty "Translation.X" 0.000000 -d  -onMod $SS_HandName;  
if( $side == "L" )  
{  
setProperty "Pre_Rotation" 180.000000 0.000000 90.00 -onMod $SS_HandName;  
}  
else  
{  
setProperty "Pre_Rotation" 0.000000 0.000000 -90.0 -onMod $SS_HandName;   

6. Save the script  (You may be asked to save as administrator)
7. Open a new scene, and import the Vicon MCP.
8. Run the ImportStretchSenseFBXFiles; script 
9. Select the hand data files
10. Check if the hands are aligned as expected with the wrist.

Timecode misalignment between Hand and Body MCP files:
If there is a mis-match in the timecode between Hand Engine and Shogun Live at the time of recording, you are still able to perform the merge. Please follow these steps to remedy: 

1. In Shogun Post, load the Left and Right-hand FBX files (Separately) with the following import settings:
   
2. Import FBX as Skeleton
   
3. Select the clip
   
4. In the attributes section, select the advanced attributes
   
5. Set SMPTE_Offset to 0:
   
6. Select the SolvingBone branch:
   
7. Export FBX with these settings:
   
8. Start a new scene in Shogun Post, and import the vicon body MCP, perform the same actions above to set SMPTE_Offset to 0 before running the ImportStretchSenseFBXFiles.hsl; script.
9. After merging, the clip timecode can be restored in the advanced attributes.

The frame rate between Hand and Body data does not match
If the frame rate at the time of recording was not set the same in Shogun Live and Hand Engine, your body and hand recordings may have mis-matching frame rates. To remedy this, please follow the below instructions. 

1. In Shogun Post, Load your Vicon body recording MCP and note the frame rate shown at the bottom right hand side
   
2. In Shogun Post, start a new scene and load the Left and Right-hand FBX files (Separately) with the following import settings:
   
3. In the Script Editor, for this example, run the following script. Note: Adjust the resample encoding and frame rate to suit your body MCP
   
   selectByType SolvingBone;
   resample "NTSC" 119.88; 
   
   
    
4. Export FBX with these settings:
   
5. You should now be ready to run the merge script as per the above instructions.