[SmartBody-dev] Smartbody with Unity

[elena]

Dear Sir/Madam,

My name is Elena Kokkinara and I am working for the SPECS lab in University of Pompeu Fabra in Barcelona.

Getting started with some new E.U projects assigned to the lab, we are required to build an ECA engine for virtual environments. After some research I have done, it seems that Smartbody is the most complete and appropriate tool for our needs.

From what I have been reading in Smartbody's documentation, there are already full integrations with renderers such as Unreal. Since our system supports Unity, our idea was to use Smartbody as a separate process and try to integrate it with this game engine.
My questions arise based on this idea. I was wondering whether Smartbody has already been integrated with Unity, and if not, I would truly appreciate your opinion on how easy and efficient would be such an effort.

Your experience and your advices would be really valuable and helpful in order to proceed with our projects.

Best regards,


Elena Kokkinara
SPECS:
Synthetic Perceptive, Emotive and Cognitive Systems
Departament de Tecnologies de la Informació i
les Comunicacions
Universitat Pompeu Fabra

[Ari Shapiro]

Hi Elena,



The integration with Unity should be fairly straightforward. There are two easy ways to do this:



1) Run Unity and SmartBody as a separate processes. SmartBody will send information over the network to Unity. In Unity, you create an instance of the BoneBusServer, and then implement the following functions in Unity like:



...

typedef void (*OnCreateCharacterFunc)( const int characterID, const std::string characterType, const std::string characterName, const int skeletonType, void * userData );

typedef void (*OnDeleteCharacterFunc)( const int characterID, void * userData );

typedef void (*OnSetCharacterPositionFunc)( const int characterID, const float x, const float y, const float z, void * userData );

typedef void (*OnSetCharacterRotationFunc)( const int characterID, const float w, const float x, const float y, const float z, void * userData );

typedef void (*OnBoneRotationsFunc)( const BulkBoneRotations * bulkBoneRotations, void * userData );

typedef void (*OnBonePositionsFunc)( const BulkBonePositions * bulkBonePositions, void * userData );

typedef void (*OnSetCharacterVisemeFunc)( const int characterID, const int visemeId, const float weight, const float

...

...



So, SmartBody will send a 'character created' message, and in turn, you create a character in Unity. Then it will send a 'character foo has bone arm in position x,y,z' message, and you would then alter your Unity character according to that message.



2) Alternatively, you can combine Unity and SmartBody into the same process uses the smartbody-dll project. To do this, you create an instance of Smartbody_dll, then extend the SmartBodylistener class. Then within your Unity execution loop, call the GetNumberOfCharacters() function and then GetCharacter() for each one, and update your unity characters with the information contained within the SmartbodyCharacter that is returned with that last call. Where the SmartbodyCharacter looks like this:



class SmartbodyCharacter

{

public:

std::string m_name;

float x; <-- world position x

float y; <-- y

float z; <-- z

float rw; <-- world quaternion w

float rx; <-- x

float ry; <-- y

float rz; <-- z



std::vector< SmartbodyJoint > m_joints;

};



and the SmartbodyJoint looks like this:



class SmartbodyJoint

{

public:

std::string m_name;

float x;

float y;

float z;

float rw;

float rx;

float ry;

float rz;

};



I think that (1) is a little bit easier, and gives you a few other niceties as well (there are a number of viewers and debugging tools that come along for free if you use alternative (1) ). I would try that first.



I do think that SmartBody is a good choice for an ECA agent. We have a team supporting it, we are in active development on it, we've recently added some important features (such as locomotion on varying terrain), and we use it in many of our internal projects, so it undergoes thorough testing.



If you need more help or guidance, please continue to post here, or you can contact me directly at ***@ict.usc.edu.



Regards,



Ari Shapiro







From: elena [mailto:***@gmail.com]
Sent: Friday, October 22, 2010 9:07 AM
To: smartbody-***@lists.sourceforge.net
Subject: [SmartBody-dev] Smartbody with Unity



Dear Sir/Madam,



My name is Elena Kokkinara and I am working for the SPECS <http://SPECS.upf.edu> lab in University of Pompeu Fabra in Barcelona.



Getting started with some new E.U projects assigned to the lab, we are required to build an ECA engine for virtual environments. After some research I have done, it seems that Smartbody is the most complete and appropriate tool for our needs.