The Control Device

After having explored different systems and constructed some prototypes (that you will see later) I found a very interesting paper by a group of researchers at Nanyang Technological University in Singapore. The paper describes a very similar project in terms of actuation that has proved invaluable in understanding, among other things, the relationship and computation between the number of strings required and the degrees of freedom (DOF) desired.

I've since then contacted Professor I-Ming Chen of the School of Mechanical and Production Engineering at NTU, who very graciously offered to release their design so that I can integrate it to mine and suggested a possible collaboration as well. Thank you Dr. Chen


On the robo-puppet, the wires that would normally go to the handheld device, (sometimes called airplane control due to its cross shape) are driven by a series of servo motors, which wind or unwind the strings to achieve the needed posture. We will call the strings the actuators.

There are three different functions the strings perform.
a) support or constrain/reference.
b) movement control
c) special effects.

Since the puppet is basically stationary within the world (it does not go anywhere except your mind!) the support strings have a simple function and will probably only have an up-down movement. The reference string, will allow the puppet to bow and stand straight. As it name implies all other strings reference this string. This is taken from traditional Chinese marionettes and is called the backbone.

The motion control strings control the head and the limbs. Because the character makes ample and sweeping movements with his arms I originally considered controls mounted on swinging devices, which would function as the plucking of the strings do in traditional marionette control.

However, since the rig needs to be portable as well as easy to assemble, not to mention cost considerations I have opted, at least for now, for a linear extension that can be easily detached and does not need additional motors for swinging. An added benefit of this solution is that it provides faster response. It contemplates stationary arms in the control device, which go as far as the limit of the puppet movement. The servos at the end of the arms would control a wire loop that simulates the string plucking of a puppeteer.

3D simulation of control rig showing linear extensible arms and servo driven pulleys

After consulting with Dr. Antonio Arroyo 1 and Dr. Eric M. Schwartz I decided to add additional loop controls at the waist level to increment the degree of expressiveness of the puppet’s body language as well as install spring and/or elastic counterbalance and damping devices to minimize the inertial deviation of the limbs as they move rapidly into position.

My first basic prototype

Once the system is in place, the puppet, the control hardware and the software, it is necessary to train the puppet by establishing neutral, initial and target postures that can express meanings and range of emotions. This is similar to the numerous cell or keyframe elements that a conventional cel animator or “claymator”2 would have to prepare to follow a script. These groups of keyframes (one for every joint) would be called upon according to the response required by the sensors and their interpretation by the neural control program 3.

Some of the considerations that must be taken into account to perform the initial set of calculations include:

- The number of links in the system
- The link length
- The mass
- The center of mass
- The inertia moment
- The joint viscosity

1- Dr. Arroyo is the Director of the Machine Intelligence Lab (MIL)at the University of Florida and it is my designated mentor for this project. Dr. Eric Schwarz is Associate Director of the MIL at the University of Florida and gives me advice as a mentor as well.

2- Clay animation is one of the many forms of stop-motion animation where the clay is hand-shaped for every frame to achieve the illusion of movement. The term Claymation was popularized by animator Will Vinton who used it as his trademark. However it is now commonly used in the English language to refer to clay animation.

3- Input /Output into a simple neural net.
The input to the system consist primarily on tracking (image and pattern recognition), on the information provided by the ambient sensors, such as movement, light, sound, proximity, etc. This input is correlated with the matrix representing the mindset of the puppet; his “vision of the world”
The output affects the actuators (motors, sounds, lights), ambient displays and control systems.