The Small Bang!

Here the Creator is kneeling down since the high-torque spine servo is not powered up.

It has been quite an ordeal to get to this point. In fact a couple of years to be exact. But finally the creature moves, and not only moves but seems to have a life of its own. I will not post video until after the performance but I am very excited by what I see. Philip and I crack up laughing every time it moves because it is so life-like.



But come to think of it, what is happening (philosophy purists will probably disagree) is that life in fact has been transferred and conserved as energy in a different form. This is not something new to me. I consider any work of our hands to partake of this mysterious force that we call life, otherwise how would it inspire us or elevate our spirit?

When I see a painting (in reality, not a reproduction) by say, Vermeer, the emotion I feel is a product not only of the formal beauty, but of being in the presence of time standing still, a few years of someone's life (which just happens to be a master) condensed and trapped like a genius in a bottle. I, the recipient of this gift, am the vehicle necessary for this alchemical process to occur. I release the energy contained in the matter of the painting.

The order of the particles, held by resinous mediums, crystallized by the passage of time which conform the external appearances of the scene resonate and I believe connect me in the most direct form with the creator of the work of art, in the same way that we are in intimate connection with the writer that is sharing her thoughts and life experiences with us.

This is, has to be, an energy transfer that could probably be calculated by an Einstein, but I prefer to simply shed a tear of joy when I find myself in front of the work of art. That is what art means to me.


In case you have not seen a previous post this is what the Creator's creature looks like. Here Lorena is getting some exercise and getting fit to withstand the wild and strenuous performance ahead, after all it is all her fault since she created the Creator. Playing god or goddess does not come cheap. You can see a little animation here

The OZ connection

The software is finally (well...) done. I mean it is totally functional, does what it is supposed to be and it is absolutely beautiful to look at. The design is old-school, meaning lean, efficient and therefore aesthetically perfect on my book. Thanks to Philip for never being satisfied. In fact every day there is some internal change that makes it run faster or adds even more functionality.

I will provide pictures of the UI in full operation as it grabs puppet positions, controls lights, sounds etc. It feels a bit like those old music sequencers which were so intuitive and logical and therefore easy to use, yet produced any result that your imagination demanded. This is the case.

These are some current pics of the UI before anything is loaded. The puppet figure is just a place holder until the cameras, both for tracking and for position display are connected.



This last shot shows the tracking grid. All the application is done modularly so that everything can be customized, such as the resolution of the grid which in this case represents the size of the stage, basically 6x8 meters. Every cell triggers different responses from the Creator based on the performer actions and timing.

Out of Sight

A few more days and we will be driving up the east coast to NY. I hate to think of all the gas we will burn, but I am sure it will be less than taking a plane, no to mention the cost. Nowadays it is impossible (and this is good) to do anything without minding the consequences in the bigger picture of life as we know it. I feel that finally a swell of inconformity is rising against the many tyrants that rule our world, including our inner one.

Back to business. The weight of the eyes proved too much for the tiny springs, so I decided to encase the spring inside a sturdier one and on top of that I put a couple of coats of latex to hold it together and still be flexible. That proved an overkill. The eyes became too stiff and did not respond to the movement as expected. So I carved them from an ultralight foam, and gave the a few coats of liquid gesso. After sanding them they had just the right look and weight to be supported by the tiny springs.

On the left the amputated wooden eye vs. the light foam eyeballs shown at the end of the spring. The toothpick became the optic nerve and it is passively string-controlled for the basic position and orientation of the eyes. They work beautifully and add a lot of expressiveness. The strings are connected to the end of the neck, so that when the Creator bows its head or kneels, the eyes "relax", and when he stands up they perk up very nicely. As everything puppetry and theatrical, their exaggerated irritation is understandable, given the great effort to see and make sense of an utterly absurd world.

The focal point of the entire universe of the Creator is ultimately its creature. Regardless of the external appearance, which only a parent would love, the important thing is that which radiates from it and communicates its presence. This electronic soul is extremely simple yet fascinating. We started creating the tracking beacon using standard IR (infrared) LED's. We were having problems achieving the necessary brightness to be detected at a long distance. On top of that, since the creature moves around the stage wildly shaking every part of its body and sometimes lowering its head it was important that the beacon be visible in all circumstances. Adding the half-pong ball (sanded to paper thin) helped the angle spread of the light but we thought that it would cut the light considerably.

And indeed our first tests confirmed that. So we bought the super-wide angle IR LED's in the hope of solving the angle problem and still retaining the strength of the light. (see the previous post on that here)


Now comes the ignoramus part, which if we had done a little more research would have been avoided. All the webcams we tested have an internal filter that blocks IR, not exactly sure why (have not had a minute to explore that). So we proceeded to dismantle the lenses to remove such filters. (unfortunately we were so engrossed in doing so that I forgot to take pictures of the process!) This tested every sense and the steadiness of our hands, not to mention memorizing the order and orientation of every microlens. In fact, in the case of our wide-angle camera shown previously, after removing the IR blocker, we had to rearrange all 9 lenses and spacers in any imaginable way until we hit on the right one. Philip which has much better eyesight ended doing the final assemblies and it was nerve wracking to watch, almost like a brain surgeon removing a micro tumor.

The original beacon, which we will use as a backup since it is even brighter because it has more LED's to begin with.

Well, then we were up for a big surprise, in fact we are still amazed at what we saw. First of all, the camera with the exposed film in place (to filter most of the visible light) not only saw the light, but it was as bright as pointing the camera to the sun! At this point we realized that both the standard LED's as well as the wide-angle ones work practically the same. You could turn the beacon even 180 degrees (in other words showing the back) and the camera would still see it very brightly.

But then we were in shock. By holding the IR device in our hand we observe something amazing. We could not only see the light going through our hand, we could see every vein as well! very much like x-rays, except we could not see the bones. Again, the surprise was such that I forgot to take pictures of this, but tomorrow I will. We even put the thing inside our mouth, but the cheeks were way to thin for the strength of the light. This immediately brought questions about the safety of this light which we had not considered.

So I have been reading papers and reports by organizations such as The International Commission on Non-Ionizing Radiation Protection, but have found little indication of danger at that level. However, although LED's are classified as IR-C are supposed to have a minimal penetration of the skin, while IR-A which is labeled as hazardous is said to penetrate the skin only a few millimeters. So something is odd here. More food for thought.

A Tale of Two Masters

First napkin sketch of the user interface

The intermediate stage of the motion control system has closely followed
the original concept although of course it has been greatly improved.

I would like to quote from the introduction to my Master's Thesis "Roboethics and Performance":

"Being as we are at the verge or perhaps in the middle of that threshold or bifurcation of our own destiny, it is hard if not impossible to observe with parsimony the course of action. In that liminal state where turbulence becomes organized into myriad eddies and flows and the “singularity” approaches, every theory or hypothesis seems to have some validity since the information or data we analyze is particular to our point of view and seldom encompasses a bigger picture. Gregory Bateson , reminiscing about the Macy conferences, goes further by suggesting that we never know the world as such. He states that “We are our epistemology” since we only perceive and understand the world through what our sensory apparatus allows.

However the birthing environment of the open network where data itself acts as a “controlling agent” is beginning to show a pattern that is itself fed back into the system, and once the critical threshold is achieved, some theorists suggest it will give raise to the emergence of a machinic consciousness.

The ability to enhance our body and mind utilizing radical nanotechnology beyond therapeutic practices, and the recent advances in bioinformatics and prosthetics as Katherine Hayles suggests, opens the discourse about the posthuman condition. All the “ecological variety” at the dawn of this new era, with the consequent interbreeding, is creating an explosion of possible species raising profound ethical questions and forcing us to rethink our position in the evolutionary sphere."

As some recent studies of the brain suggest*, there is not one but two control centers of control and command which work independently and actually incommunicated from each other even though both work towards a common goal.

The software that Philip Forget and myself have designed works as one of those control centers. The other stems from the independent actions of the performer which by interacting with the environment under surveillance by the Creator produces a synergistic feedback loop.

I am very excited by the software part of the project. It is in my biased opinion the best tool that I have seen in robotic puppetry control (not that there are many, mostly in research projects like mine). Although at first some closed systems were briefly considered, like National Instrument's LabView, to control servos based on image processing and other sensor input, the price and steep learning and development curve ruled it out early on, not to mention my dislike of anything that is not open source.

When Philip learned that I was using the MAKE microcontroller to drive my project and suggested that we use Flash as the controlling software I was right off skeptical. Not having used Flash in a couple of years I was not aware of recent developments.

He was of course referring to FlashDevelop, which is an open source ActionScript 2/3 and web development environment which integrates seamlessly with the Adobe product. This has allowed us to implement exactly what we wanted in a record time and in a very efficient and elegant way.

Our system is essentially a motion-sound-image-video-light sequencer/controller, driven in this case by image processing, but could as well be driven by any other sort of input. The heart of the system is a MAKE microcontroller which is directly controlled by the software right out of the box.

The screen capture above shows the position editor on the left and the corresponding key frame editor on the right. Each position is a collection of expressive movements achieved by "training" the marionette and recording the position, travel and speed of a group of eight servos. The sliders on the top right control or feedback both position and speed for each individual servo. The block below house the smaller sliders for light control as well as the sound control. The placeholder image is an automatic capture of the marionette position.

When played, the timeline runs thorough all the movements, light states and sounds that correspond to such moment of expression. The modules missing in the screen above are the library module and the main GRID which is the area under surveillance by the camera and which triggers actions defined by both timing and position of the performer or other beaming object (the performer wears an infrared beacon).

It is our intention to release the software once completed as an open source project under Creative Commons.



*Washington University School of Medicine (2007, June 21). Brain's Voluntary Chain-of-command Ruled By Not One But Two Captains. ScienceDaily. Retrieved June 15, 2008, from http://www.sciencedaily.com­ /releases/2007/06/070619134802.htm

The Cyclopean Eye

The tiniest springs I could find in my "this could be useful" parts bin

The eyes are an important means of expression of the "soul" of a person, animal or character. The only external organ that connects directly to the brain through the shortest possible path.
The eyes of the creator move nervously as he scans its surroundings. Its world. They are animated by a simple spring mechanism that responds to the smallest movement. I did not know it was going to be so challenging to do this, except that, as we have found out, scale matters. And the smaller the scale the more critical the elements become.

The springs proved to be too weak to support the weight of the eye,
so I added a coat of latex to strengthen and stiffen it a bit,
but it proved to be a little too much so I ended up burning the latex away.
The residue that it left iwas just the right amount.

Lorena irritating the eyes

Of course the real "eye" of the Creator is up in the sky, or rather the ceiling in this case. This cyclopean eye that sees and controls everything is more and more a pervasi and accepted fact of life, in our streets, our buildings and even our homes, not to mention the "real" and inaccessible eye in the sky who observes and analyzes our every move, both as a species as well as individually when the need arises.

A very wide angle camera (76 degrees) was the choice after a good deal of research, and it proved to be the right one. I like that it is flat and although the base actually sucks if you are going to use it on a desk or a monitor it is perfect to tape to the roof and adjust the angle precisely. I needed a wide angle because the ceiling at the venue is not very tall and the camera must cover at least a 10x10 mt performing area, the world of the Creator, its Tartarus.

HP 2 megapixel camera, 76 degrees FOV

Related to the eyes or rather vision system is the light. Not the infrared light that the camera detects and processes but the actual "stage lighting". For quite a while I thought I was going to have to finagle a small lighting system form the theatre department at my university but since that was not very promising I started looking around for solutions.

I must say I am very happy with the outcome. I bought some inexpensive 12 LED flashlights at the local supermarket and hacked them (literally) so as to keep only the front portion. The advantage, in this case, of a cheap Chinese product is that it was very easy to solder to it, unlike more sophisticated models. The beam is extremely bright and is easily controlled by the microcontroller. So I have 4 "spots" with different color lights (a color filter on the LED's) to dramatize the scene and automatically create different moods according to the moment.

I got the plastic flexible contraption to where I attached the light base from one of my many "possible useful things" boxes. I wish I knew what it is or where it came from, maybe someone does becasue it is extremely practical for this and I am sure, other, applications. I suspect that it might belong to some fluid dispensing system.

The following are the very first tests with my new lighting system. I love it! absolute and precise positional and intensity control.

The Eyes of the Poet

One of the films that most impressed me when I was a young aspiring filmmaker was undoubtedly Un Chien Andalou and its memorable eye-slicing scene was particularly impressive to me. Whenn I was seven years old I had had my left eye cut by a piece of glass thrown to me by a childhood friend which required seven stitches right next to the pupil. This event changed my life forever and I believe in a positive way as I look back (pun not intended).

First of all the impact on my psyche was considerable. Since all eye operations occur with your eyes wide open, it is inevitable to see what it is being done, even if you are under total anesthesia, like I was.

The surgeon did not believed me at first when I told him that I had seen all the operation until I started describing in detail what I saw. This images were burned in my mind in a very permanent way, and for many years, until my young adult age the sequence of events (the operation itself) would run unannounced like a silent film of yore. When this happened I had to immediately stop whatever I was doing and change to something opposite of that, for example if I was alone I would try to find people, the more the better, like a crowd. But when I was in a crowd or a party and the film started to run I had to immediately find a place where I could be alone. Only then would the operation fade and let me go on with my life. Then one day this vanished never to come back again.

My sight never recovered and so I would see very clearly with my right eye and completely out of focus with my left, since the stitches distorted the pupil. This became extremely useful when I was a cameraman, since I did not have to close my left eye when shooting like everyone does. The main reason you would close the eye that is not looking through the viewfinder is to prevent confusion as to which image you are supposed to be seeing. The view from the lens would never be the same as the naked eye and therefore the mix of both images would be very disturbing to some people. However, my two vies were so radically different that I was able to use the out of focus image to my advantage. The left eye could (barely) see that which was outside the frame and therefore I would be aware of something that was just going to happen around and could pan to frame it or to avoid it.

Back to the creatures eyes and the relationship with all of this. I, and apparently Lorena, my wife, as well have lived and still do in a close relationship with the dream world, which as you know is not always your "dream world" since nightmares and surreal events can be pretty scary. It si just that we have both decided to turn these experiences into the stuff of art, bringing this parallel universe into the "real" world for other to see.

The touch of the Godess

The credit for the Creator's flesh and blood must go to Lorena, who with her hands brings to life the creatures of our nightmares, the future that we have already created even if we ourselves are not quite there yet to suffer the consequences, but getting very close...

Plasticine, plaster gauze and Papier-mâché is the skin of its face

Make up time

Hmmm, definetely some resemblance...

The Creator gets its make-up treatment. It was eerie to see how it resembled the original Creator's head, that you can see in the background. It was like seeing a clone emerge from another dimension. And God said, "Go ahead and multiply!"