X-Rated Super Mario

A Tale of Two Masters

First napkin sketch of the user interface

The intermediate stage of the motion control system has closely followed
the original concept although of course it has been greatly improved.

I would like to quote from the introduction to my Master's Thesis "Roboethics and Performance":

"Being as we are at the verge or perhaps in the middle of that threshold or bifurcation of our own destiny, it is hard if not impossible to observe with parsimony the course of action. In that liminal state where turbulence becomes organized into myriad eddies and flows and the “singularity” approaches, every theory or hypothesis seems to have some validity since the information or data we analyze is particular to our point of view and seldom encompasses a bigger picture. Gregory Bateson , reminiscing about the Macy conferences, goes further by suggesting that we never know the world as such. He states that “We are our epistemology” since we only perceive and understand the world through what our sensory apparatus allows.

However the birthing environment of the open network where data itself acts as a “controlling agent” is beginning to show a pattern that is itself fed back into the system, and once the critical threshold is achieved, some theorists suggest it will give raise to the emergence of a machinic consciousness.

The ability to enhance our body and mind utilizing radical nanotechnology beyond therapeutic practices, and the recent advances in bioinformatics and prosthetics as Katherine Hayles suggests, opens the discourse about the posthuman condition. All the “ecological variety” at the dawn of this new era, with the consequent interbreeding, is creating an explosion of possible species raising profound ethical questions and forcing us to rethink our position in the evolutionary sphere."

As some recent studies of the brain suggest*, there is not one but two control centers of control and command which work independently and actually incommunicated from each other even though both work towards a common goal.

The software that Philip Forget and myself have designed works as one of those control centers. The other stems from the independent actions of the performer which by interacting with the environment under surveillance by the Creator produces a synergistic feedback loop.

I am very excited by the software part of the project. It is in my biased opinion the best tool that I have seen in robotic puppetry control (not that there are many, mostly in research projects like mine). Although at first some closed systems were briefly considered, like National Instrument's LabView, to control servos based on image processing and other sensor input, the price and steep learning and development curve ruled it out early on, not to mention my dislike of anything that is not open source.

When Philip learned that I was using the MAKE microcontroller to drive my project and suggested that we use Flash as the controlling software I was right off skeptical. Not having used Flash in a couple of years I was not aware of recent developments.

He was of course referring to FlashDevelop, which is an open source ActionScript 2/3 and web development environment which integrates seamlessly with the Adobe product. This has allowed us to implement exactly what we wanted in a record time and in a very efficient and elegant way.

Our system is essentially a motion-sound-image-video-light sequencer/controller, driven in this case by image processing, but could as well be driven by any other sort of input. The heart of the system is a MAKE microcontroller which is directly controlled by the software right out of the box.

The screen capture above shows the position editor on the left and the corresponding key frame editor on the right. Each position is a collection of expressive movements achieved by "training" the marionette and recording the position, travel and speed of a group of eight servos. The sliders on the top right control or feedback both position and speed for each individual servo. The block below house the smaller sliders for light control as well as the sound control. The placeholder image is an automatic capture of the marionette position.

When played, the timeline runs thorough all the movements, light states and sounds that correspond to such moment of expression. The modules missing in the screen above are the library module and the main GRID which is the area under surveillance by the camera and which triggers actions defined by both timing and position of the performer or other beaming object (the performer wears an infrared beacon).

It is our intention to release the software once completed as an open source project under Creative Commons.



*Washington University School of Medicine (2007, June 21). Brain's Voluntary Chain-of-command Ruled By Not One But Two Captains. ScienceDaily. Retrieved June 15, 2008, from http://www.sciencedaily.com­ /releases/2007/06/070619134802.htm

Sense and sensibility and the Oneironaut

Visualizing the user interface for servo control

The nervous system is the current step (by step). Working on the sensors and on the beacons has given me a clearer understanding that there is nothing I know. Still very very far from The Docta Ignorantia.

Here is our microprocessor stack composed of two Make boards which were our final choice. The boards are highly integrated, very easy to interface and simply well designed. The only drawback was the limitation of 4 servo ports per board. I decided early on that 8 motors were as many as I wanted to use, both for practical and symbolic reasons. Four is the symbol of man generated by the trilogy which comes from the duality contained in the unity (bear with me, it is a stream of thought). 8 represents the power of the king in chess, and the queen in all her potency and (coming down to earth), I could only afford two Make controllers anyway! So, there. I had to do it with only 8 motors, period.

Then came the IR beacon circuit. That was a lot of fun to build although it took a lot of trial and error to do it with kitchen stuff. DYI instructions stipulate in no uncertain terms that you should use glossy inkjet paper and ONLY print with a laser printer, then iron transfer into the copper plate. Well, after many frustrating trials, botched plates and burnt fingers we (Philip and I) decided to try swhat you are not supposed to do. Went to our local print shop and had the circuit printed on their glossiest color laser paper with their fancy color printers. That worked beautifully. It transfered absolutely perfect the first time around. The entire coating of the paper stuck to the plate, and I am sure more than one has though that was wrong, but after soaking it in warm water the paper peeled off like a decal and left an impecable mask.

So here is Philip, which was the instigator of the whole DYI circuit idea cooking up the circuit with a coctel of muriatic acid mixed with hydrogen peroxide. In a few minutes our circuit was done. As usual he is multitasking with his totally hacked IPhone.

Guess where Philip is from...

Philip in my studio dremelin the circuit holes while Castro look approvingly at our socialist methodology.

He got a kick out of my retro-futuristic glasses. You look awful in them, but they have a variable focus that allows you to magnify minute stuff or read a newspaper across the room.

This section needs proofing and revising...tomorrow, after I give my class and deal with insane bureaucracy.

After dealing with the obvious and easy choices, like the arms (4 motors there) I realized the problem. My monster servo would lower the body, yes. But what about the head?, the shoulder strings? (they allow the marionette to turn the upper part of the body to face in another direction). They had to be lowered at the same time! otherwise I would need many more motors and complex programming to synchronize all these elements.

So the idea of the elevator was born, with all its ancillary problems, weight, friction, balance, stability, randomness, etc. At first I built tracks to guide the (square!) rods up and down. These rods (quads?) support a platform that houses the 3 remaining servos. The problem is that the head has more than one degree of freedom of course. It not only has to turn left and right, it also has to tilt sideways and bow as well. Or pitch, yaw and roll if you will.

All these simple motions are trivial (if difficult to master) in a marionette. Then I had the shoulder controls to turn the body around; how to do all that with 3 servos and control the wanted nuances?

After about two weeks (which is the time I have not posted!) I stumbled upon the solution, where but in a dream!. It is pretty obvious that we work as hard in our oneiric universe as we do in our waking world.

Next morning I skeptically went to examine the contraption with the suspicion that, like many other "great ideas" conceived in la-la land it would turn to be impractical, impossible or just plain turdy. It actually took me a while to connect my ideas with the reality of the mechanics, something was missing which I had apparently forgotten. Then, all of a sudden I saw it! make one of the servos perform double duty.

I cannot even explain how I visualized it. It was so obvious, simple, and yes, elegant. The only bummer is that I feel it did not come from me but from the collective unconscious debris. I just picked up the pieces.

I will post some pictures to explain how it works.

Divide and conquer?

That seems to be the design philosophy of the VEX guys. I promise (well, a weak promise) that I won't waste more time bitching about their horrible "close source" design decisions. But someone has to say something. Of course I used their stuff to prototype my project since I had bought 5 or 6 kits with the hope of actually using them for the final project. Now, to be fair, their nuts and screws are first rate and I have no complaints, as well as their cute chain transports if you use their square shafts of course, but the rest...


It was obvious pretty soon that the electronic/hardware side would not work due to their very flimsy nature and weak linkage (or lack of it). Take their servo for example. Not only is it far from standard, so that you cannot mix with any other brand, but it lacks fundamental elements like a horn or attachment plate of some sort so that you can drive something more than their absurd square shafts. Guess what, you cannot even screw anything to it, you can only use their "clutch" (the small cylinder with a short square stub) to attach another square shaft to drive any square holed device! BTW the clutch and therefore anything attached to it, will simply fall off if you turn it upside down. Am I missing a screw here?

You cannot see it in this next picture, but those arms or extensions that I need to drive the limbs of the marionette with have as much strength and stability as an apple stuck on the tip of a drinking straw and balanced on the edge of a glass, can you picture that? perhaps David Copperfield could use such a trick. The main reason being that as I explained, the servos connect to anything else through the wimpy clutch mechanism that cannot stand any side pressure, only direct transmission to a wheel at the most.


To be fair, after a great deal of literally hacking away with a saw, drilling and cutting and bending, I was able to put together a somewhat functional scaffolding for the stage. If I had the money I would simply make everything from scratch out of aluminum and some steel pieces. And, yes, chuck those servos away. I opened one to convert it, I was so shocked by the poor craftsmanship inside, probably built on a Chinese sweatshop (badly soldered by hand), that I just closed it and proceeded to order a set of excellent Hi-Tech servos.

The Beauty of the Fall

The beauty of the Püterschein system is, as you can see in this picture, that the puppet assumes very natural positions with no human intervention. I just cut the string that held it up and it fell on its knees just like that! A true predecessor of inverse kinematics!.


It seems like it is praying for arms, looking intently at the cut rods that will give it the expressiveness it needs.


I cut an old broom stick to use for the arms, thinking that it was pine and it would save me some time if it already had a round shape. Ha! I don't know what kind of wood it is, but it was hard as oak and I have gone through a whole set of carbon steel blades just cutting it down to shape. It has taken me three full days to get them ready for testing! So much for saving time...





You can see the Creator now just standing naturally, again with only spine support. I did not pose it, other than bringing it down to contact a surface. Can't wait to have the entire body functional. In case you are wondering the "feet" will be (at least that is the plan) two slim inverted cones. We'll see how it behaves. As for hands it will have none in this iteration, since it will be all dressed up and have very long sleeves like the human performer in an earlier post:
you can see it here and in the original design so you get the idea.


This is a detail of the hip and waist rigging that holds and constrains those parts as the marionette flexes and moves.



After three days, a few cuts and sore muscles overall, the arms are close to ready. I've tested them and they move gracefully and have the needed range. Under the torso you can see the channel I gouged out to make room for a couple of screw eyes to pass the string that connects it to the waist and hip.

Next task: the HEAD! this will be interesting since it will have a wooden core or "skull" and it will wear a "self animated" mask, a fancy way of saying it will have some springs to react to any movement of the head. If I have time, which I doubt, I would like to put a micro-servo on the latex mask so that it can have a range of expressions in addition to the eyes. Lorena, who designed the original creature does not like the idea. Perhaps she is right. The simplicity of the spring solution, which we have demonstrated on performance gives such a varied range of expression to the Creator's face that adding more might be less. So there!