The ethics of computers

Only six years ago the New York Times announced that a Japanese laboratory had built the world's fastest computer, ..."a machine so powerful that it matches the raw processing power of the 20 fastest American computers combined and far outstrips the previous leader, an I.B.M.-built machine." (That is 35.6 trillion mathematical operations per second)

As is true of the robotics field as well the Japanese machine reveals very different ethics and commitment than their US counterparts.

The Japanese government financed this computer to be used at the Earth Simulator Research and Development Center in Yokohama ..." to analyze climate change, including global warming, as well as weather and earthquake patterns. "

"By contrast", the article says, "the United States has predominantly focused its efforts on building powerful computers for simulating weapons, while its efforts have lagged in scientific areas like climate modeling."

Forward 6 years to June of this year and true to the double exponential acceleration towards the singularity suggested by Ray Kurzweil, CNN announced that engineers from IBM and Los Alamos (sounds a bell?) have designed a computer capable of sustaining 1,000 trillion operations per second.

Can you guess what this computer will be dedicated to?

• It will be used to help maintain the nation's nuclear weapons stockpile

So while most of the supercomputers around the world work towards solving some of the most urgent problems that we face, the US government through its military establishment keeps its accelerated pace towards our destruction.

It's Show Time

Am I ready yet?

Finally the expected moment when breaking a leg is the least of your worries, it is in fact a welcome omen!

I decided to post a slideshow, given that there are many pictures of different moments, from the preparation before the show to pictures showing the actual marionette, since many people believed that it was a pre-made video agains which the performance took place.

As I had discussed with my mentor Deborah Aschheim, this was going to be an experiment to see if the concept worked. The concept being that The Creator represented, among other things, an omnipresence that we took for granted, whose reality was ignored. The pervasive surveillance and control that is now an accepted state in the life of so many people who fail to question some very basic assumptions.

This presence whose control was embodied by the environment itself could become simply entertainment, a surface to which we have become fully accustomed and to which we in fact give our compliance voluntarily.

So perhaps conceptually the experiment worked, since as a "show" it was considered fascinating by most of the audience I had a chance to speak with. But I think that I would like to eventually construct a full scale marionette as originally planned (5 mts. tall). I am more interested in telling a compelling story that can directly connect with the stored archetypes that we carry in our brains and , because of scale and the amazement associated with seeing something so out of the ordinary, our defenses are suspended, letting the images and sounds seep undetected into our psyche.

Later on, perhaps,we will encounter in our dreams or nightmares, or worse in our waking reality, images,facts and ideas that evoke that inexplicable and uncanny feeling of the loss of our mythical innocence. Optimistically I would hope that this would result in recognizing like Pink Floyd says "the writing on the wall".

The Creator was born in Germany as a concept, or rather as a nightmare that my wife Lorena had. As artists we become attached to the object of our creation, even if it represents something as terrible as to what a post-apocalyptic scenario suggests. We treat the creator and the creature with loving care and feel both awe and compassion for them, or perhaps for what they represent. A loss of our own humanity.

It is fitting, and part of the mystery of our lives that we simply accept things as they happen. The performance took place in Linz, of all places. Linz, the birthplace of Hitler bears everywhere you turn and in spite of the efforts to conceal it, an atmosphere of sadness which even the clowns and street performers and musicians who entertain tourists and passerby s fail to diffuse. At our graduation ceremony, Virgil Wong had invited a pair of these street musicians to perform for us. It was interesting to see how the director of the Institute felt so uncomfortable as they sang beautiful but obviously sorry laden songs filled with nostalgia for what is not, sang in a language which nobody spoke but which everyone understood.

Ir Rays

A few posts before I mentioned our amazement at the strength and penetration of the infrared LED's that we are using as the beacon for our performance. I just took a screen grab of what the webcam was recording as proof that I was not seeing visions. You can clearly see the veins on the hand. I really like the shot. It reminds me of the photograms of László Moholy-Nagy

"The reality of our century is technology: the invention, construction and maintenance of machines. To be a user of machines is to be of the spirit of this century. Machines have replaced the transcendental spiritualism of past eras."
--László Moholy-Nagy

The Small Bang!

Here the Creator is kneeling down since the high-torque spine servo is not powered up.

It has been quite an ordeal to get to this point. In fact a couple of years to be exact. But finally the creature moves, and not only moves but seems to have a life of its own. I will not post video until after the performance but I am very excited by what I see. Philip and I crack up laughing every time it moves because it is so life-like.



But come to think of it, what is happening (philosophy purists will probably disagree) is that life in fact has been transferred and conserved as energy in a different form. This is not something new to me. I consider any work of our hands to partake of this mysterious force that we call life, otherwise how would it inspire us or elevate our spirit?

When I see a painting (in reality, not a reproduction) by say, Vermeer, the emotion I feel is a product not only of the formal beauty, but of being in the presence of time standing still, a few years of someone's life (which just happens to be a master) condensed and trapped like a genius in a bottle. I, the recipient of this gift, am the vehicle necessary for this alchemical process to occur. I release the energy contained in the matter of the painting.

The order of the particles, held by resinous mediums, crystallized by the passage of time which conform the external appearances of the scene resonate and I believe connect me in the most direct form with the creator of the work of art, in the same way that we are in intimate connection with the writer that is sharing her thoughts and life experiences with us.

This is, has to be, an energy transfer that could probably be calculated by an Einstein, but I prefer to simply shed a tear of joy when I find myself in front of the work of art. That is what art means to me.


In case you have not seen a previous post this is what the Creator's creature looks like. Here Lorena is getting some exercise and getting fit to withstand the wild and strenuous performance ahead, after all it is all her fault since she created the Creator. Playing god or goddess does not come cheap. You can see a little animation here

The OZ connection

The software is finally (well...) done. I mean it is totally functional, does what it is supposed to be and it is absolutely beautiful to look at. The design is old-school, meaning lean, efficient and therefore aesthetically perfect on my book. Thanks to Philip for never being satisfied. In fact every day there is some internal change that makes it run faster or adds even more functionality.

I will provide pictures of the UI in full operation as it grabs puppet positions, controls lights, sounds etc. It feels a bit like those old music sequencers which were so intuitive and logical and therefore easy to use, yet produced any result that your imagination demanded. This is the case.

These are some current pics of the UI before anything is loaded. The puppet figure is just a place holder until the cameras, both for tracking and for position display are connected.



This last shot shows the tracking grid. All the application is done modularly so that everything can be customized, such as the resolution of the grid which in this case represents the size of the stage, basically 6x8 meters. Every cell triggers different responses from the Creator based on the performer actions and timing.

Out of Sight

A few more days and we will be driving up the east coast to NY. I hate to think of all the gas we will burn, but I am sure it will be less than taking a plane, no to mention the cost. Nowadays it is impossible (and this is good) to do anything without minding the consequences in the bigger picture of life as we know it. I feel that finally a swell of inconformity is rising against the many tyrants that rule our world, including our inner one.

Back to business. The weight of the eyes proved too much for the tiny springs, so I decided to encase the spring inside a sturdier one and on top of that I put a couple of coats of latex to hold it together and still be flexible. That proved an overkill. The eyes became too stiff and did not respond to the movement as expected. So I carved them from an ultralight foam, and gave the a few coats of liquid gesso. After sanding them they had just the right look and weight to be supported by the tiny springs.

On the left the amputated wooden eye vs. the light foam eyeballs shown at the end of the spring. The toothpick became the optic nerve and it is passively string-controlled for the basic position and orientation of the eyes. They work beautifully and add a lot of expressiveness. The strings are connected to the end of the neck, so that when the Creator bows its head or kneels, the eyes "relax", and when he stands up they perk up very nicely. As everything puppetry and theatrical, their exaggerated irritation is understandable, given the great effort to see and make sense of an utterly absurd world.

The focal point of the entire universe of the Creator is ultimately its creature. Regardless of the external appearance, which only a parent would love, the important thing is that which radiates from it and communicates its presence. This electronic soul is extremely simple yet fascinating. We started creating the tracking beacon using standard IR (infrared) LED's. We were having problems achieving the necessary brightness to be detected at a long distance. On top of that, since the creature moves around the stage wildly shaking every part of its body and sometimes lowering its head it was important that the beacon be visible in all circumstances. Adding the half-pong ball (sanded to paper thin) helped the angle spread of the light but we thought that it would cut the light considerably.

And indeed our first tests confirmed that. So we bought the super-wide angle IR LED's in the hope of solving the angle problem and still retaining the strength of the light. (see the previous post on that here)


Now comes the ignoramus part, which if we had done a little more research would have been avoided. All the webcams we tested have an internal filter that blocks IR, not exactly sure why (have not had a minute to explore that). So we proceeded to dismantle the lenses to remove such filters. (unfortunately we were so engrossed in doing so that I forgot to take pictures of the process!) This tested every sense and the steadiness of our hands, not to mention memorizing the order and orientation of every microlens. In fact, in the case of our wide-angle camera shown previously, after removing the IR blocker, we had to rearrange all 9 lenses and spacers in any imaginable way until we hit on the right one. Philip which has much better eyesight ended doing the final assemblies and it was nerve wracking to watch, almost like a brain surgeon removing a micro tumor.

The original beacon, which we will use as a backup since it is even brighter because it has more LED's to begin with.

Well, then we were up for a big surprise, in fact we are still amazed at what we saw. First of all, the camera with the exposed film in place (to filter most of the visible light) not only saw the light, but it was as bright as pointing the camera to the sun! At this point we realized that both the standard LED's as well as the wide-angle ones work practically the same. You could turn the beacon even 180 degrees (in other words showing the back) and the camera would still see it very brightly.

But then we were in shock. By holding the IR device in our hand we observe something amazing. We could not only see the light going through our hand, we could see every vein as well! very much like x-rays, except we could not see the bones. Again, the surprise was such that I forgot to take pictures of this, but tomorrow I will. We even put the thing inside our mouth, but the cheeks were way to thin for the strength of the light. This immediately brought questions about the safety of this light which we had not considered.

So I have been reading papers and reports by organizations such as The International Commission on Non-Ionizing Radiation Protection, but have found little indication of danger at that level. However, although LED's are classified as IR-C are supposed to have a minimal penetration of the skin, while IR-A which is labeled as hazardous is said to penetrate the skin only a few millimeters. So something is odd here. More food for thought.

A Tale of Two Masters

First napkin sketch of the user interface

The intermediate stage of the motion control system has closely followed
the original concept although of course it has been greatly improved.

I would like to quote from the introduction to my Master's Thesis "Roboethics and Performance":

"Being as we are at the verge or perhaps in the middle of that threshold or bifurcation of our own destiny, it is hard if not impossible to observe with parsimony the course of action. In that liminal state where turbulence becomes organized into myriad eddies and flows and the “singularity” approaches, every theory or hypothesis seems to have some validity since the information or data we analyze is particular to our point of view and seldom encompasses a bigger picture. Gregory Bateson , reminiscing about the Macy conferences, goes further by suggesting that we never know the world as such. He states that “We are our epistemology” since we only perceive and understand the world through what our sensory apparatus allows.

However the birthing environment of the open network where data itself acts as a “controlling agent” is beginning to show a pattern that is itself fed back into the system, and once the critical threshold is achieved, some theorists suggest it will give raise to the emergence of a machinic consciousness.

The ability to enhance our body and mind utilizing radical nanotechnology beyond therapeutic practices, and the recent advances in bioinformatics and prosthetics as Katherine Hayles suggests, opens the discourse about the posthuman condition. All the “ecological variety” at the dawn of this new era, with the consequent interbreeding, is creating an explosion of possible species raising profound ethical questions and forcing us to rethink our position in the evolutionary sphere."

As some recent studies of the brain suggest*, there is not one but two control centers of control and command which work independently and actually incommunicated from each other even though both work towards a common goal.

The software that Philip Forget and myself have designed works as one of those control centers. The other stems from the independent actions of the performer which by interacting with the environment under surveillance by the Creator produces a synergistic feedback loop.

I am very excited by the software part of the project. It is in my biased opinion the best tool that I have seen in robotic puppetry control (not that there are many, mostly in research projects like mine). Although at first some closed systems were briefly considered, like National Instrument's LabView, to control servos based on image processing and other sensor input, the price and steep learning and development curve ruled it out early on, not to mention my dislike of anything that is not open source.

When Philip learned that I was using the MAKE microcontroller to drive my project and suggested that we use Flash as the controlling software I was right off skeptical. Not having used Flash in a couple of years I was not aware of recent developments.

He was of course referring to FlashDevelop, which is an open source ActionScript 2/3 and web development environment which integrates seamlessly with the Adobe product. This has allowed us to implement exactly what we wanted in a record time and in a very efficient and elegant way.

Our system is essentially a motion-sound-image-video-light sequencer/controller, driven in this case by image processing, but could as well be driven by any other sort of input. The heart of the system is a MAKE microcontroller which is directly controlled by the software right out of the box.

The screen capture above shows the position editor on the left and the corresponding key frame editor on the right. Each position is a collection of expressive movements achieved by "training" the marionette and recording the position, travel and speed of a group of eight servos. The sliders on the top right control or feedback both position and speed for each individual servo. The block below house the smaller sliders for light control as well as the sound control. The placeholder image is an automatic capture of the marionette position.

When played, the timeline runs thorough all the movements, light states and sounds that correspond to such moment of expression. The modules missing in the screen above are the library module and the main GRID which is the area under surveillance by the camera and which triggers actions defined by both timing and position of the performer or other beaming object (the performer wears an infrared beacon).

It is our intention to release the software once completed as an open source project under Creative Commons.



*Washington University School of Medicine (2007, June 21). Brain's Voluntary Chain-of-command Ruled By Not One But Two Captains. ScienceDaily. Retrieved June 15, 2008, from http://www.sciencedaily.com­ /releases/2007/06/070619134802.htm

The Cyclopean Eye

The tiniest springs I could find in my "this could be useful" parts bin

The eyes are an important means of expression of the "soul" of a person, animal or character. The only external organ that connects directly to the brain through the shortest possible path.
The eyes of the creator move nervously as he scans its surroundings. Its world. They are animated by a simple spring mechanism that responds to the smallest movement. I did not know it was going to be so challenging to do this, except that, as we have found out, scale matters. And the smaller the scale the more critical the elements become.

The springs proved to be too weak to support the weight of the eye,
so I added a coat of latex to strengthen and stiffen it a bit,
but it proved to be a little too much so I ended up burning the latex away.
The residue that it left iwas just the right amount.

Lorena irritating the eyes

Of course the real "eye" of the Creator is up in the sky, or rather the ceiling in this case. This cyclopean eye that sees and controls everything is more and more a pervasi and accepted fact of life, in our streets, our buildings and even our homes, not to mention the "real" and inaccessible eye in the sky who observes and analyzes our every move, both as a species as well as individually when the need arises.

A very wide angle camera (76 degrees) was the choice after a good deal of research, and it proved to be the right one. I like that it is flat and although the base actually sucks if you are going to use it on a desk or a monitor it is perfect to tape to the roof and adjust the angle precisely. I needed a wide angle because the ceiling at the venue is not very tall and the camera must cover at least a 10x10 mt performing area, the world of the Creator, its Tartarus.

HP 2 megapixel camera, 76 degrees FOV

Related to the eyes or rather vision system is the light. Not the infrared light that the camera detects and processes but the actual "stage lighting". For quite a while I thought I was going to have to finagle a small lighting system form the theatre department at my university but since that was not very promising I started looking around for solutions.

I must say I am very happy with the outcome. I bought some inexpensive 12 LED flashlights at the local supermarket and hacked them (literally) so as to keep only the front portion. The advantage, in this case, of a cheap Chinese product is that it was very easy to solder to it, unlike more sophisticated models. The beam is extremely bright and is easily controlled by the microcontroller. So I have 4 "spots" with different color lights (a color filter on the LED's) to dramatize the scene and automatically create different moods according to the moment.

I got the plastic flexible contraption to where I attached the light base from one of my many "possible useful things" boxes. I wish I knew what it is or where it came from, maybe someone does becasue it is extremely practical for this and I am sure, other, applications. I suspect that it might belong to some fluid dispensing system.

The following are the very first tests with my new lighting system. I love it! absolute and precise positional and intensity control.

The Eyes of the Poet

One of the films that most impressed me when I was a young aspiring filmmaker was undoubtedly Un Chien Andalou and its memorable eye-slicing scene was particularly impressive to me. Whenn I was seven years old I had had my left eye cut by a piece of glass thrown to me by a childhood friend which required seven stitches right next to the pupil. This event changed my life forever and I believe in a positive way as I look back (pun not intended).

First of all the impact on my psyche was considerable. Since all eye operations occur with your eyes wide open, it is inevitable to see what it is being done, even if you are under total anesthesia, like I was.

The surgeon did not believed me at first when I told him that I had seen all the operation until I started describing in detail what I saw. This images were burned in my mind in a very permanent way, and for many years, until my young adult age the sequence of events (the operation itself) would run unannounced like a silent film of yore. When this happened I had to immediately stop whatever I was doing and change to something opposite of that, for example if I was alone I would try to find people, the more the better, like a crowd. But when I was in a crowd or a party and the film started to run I had to immediately find a place where I could be alone. Only then would the operation fade and let me go on with my life. Then one day this vanished never to come back again.

My sight never recovered and so I would see very clearly with my right eye and completely out of focus with my left, since the stitches distorted the pupil. This became extremely useful when I was a cameraman, since I did not have to close my left eye when shooting like everyone does. The main reason you would close the eye that is not looking through the viewfinder is to prevent confusion as to which image you are supposed to be seeing. The view from the lens would never be the same as the naked eye and therefore the mix of both images would be very disturbing to some people. However, my two vies were so radically different that I was able to use the out of focus image to my advantage. The left eye could (barely) see that which was outside the frame and therefore I would be aware of something that was just going to happen around and could pan to frame it or to avoid it.

Back to the creatures eyes and the relationship with all of this. I, and apparently Lorena, my wife, as well have lived and still do in a close relationship with the dream world, which as you know is not always your "dream world" since nightmares and surreal events can be pretty scary. It si just that we have both decided to turn these experiences into the stuff of art, bringing this parallel universe into the "real" world for other to see.

The touch of the Godess

The credit for the Creator's flesh and blood must go to Lorena, who with her hands brings to life the creatures of our nightmares, the future that we have already created even if we ourselves are not quite there yet to suffer the consequences, but getting very close...

Plasticine, plaster gauze and Papier-mâché is the skin of its face

Make up time

Hmmm, definetely some resemblance...

The Creator gets its make-up treatment. It was eerie to see how it resembled the original Creator's head, that you can see in the background. It was like seeing a clone emerge from another dimension. And God said, "Go ahead and multiply!"

Re Cycle beneath the wheel

The Creator stage in progress

People, myself included, talk about reducing our "carbon footprint", a term about to disappear from overexposure the same way any word repeated over and over loses its meaning as it is replaced by noise. At the same time we blog about it, consuming terawatts of carbon producing power exacerbated by the inefficiency of bloated operating systems, the apparently innocuous white screen of Google, by the energy consumed by the avatars in virtual worlds that need to be kept virtually alive by the server nodes and the computers around the world that hum incessantly without any hope of ever being silent.

Our self-made society has plugged a cannula into our collective vein, to transfuse itself into the mind of the machine, which like a life support system we cannot disconnect, or at least we think we can't, that is, until the plug is pulled by the system itself when no longer needs any more information, other than the one it itself generates.

WW II direct interhuman syringe.

That is the story of The Creator with which we are trying to portray, using puppetry as a transference medium, the stark reminder of our self-dependency. Who or what controls who or what?

On to the practicum.

Since I detest plastic and the many woes and sickness that has come as a result of its indiscriminate use, I try to target it as a primary candidate for recycling, which is just a palliative approach to the real problem. In any case, I took the vacuformed package of the "energy saving" bulbs that we use nowadays, and just as I was about to send it into its recyling path, I noticed a form which reminded me of ancient alchemical apparatuses which I needed to create for the robotic-marionette stage. So here is the process:






The background, built over the "hardware cloth" is semi-translucent, with the most translucent parts being the "genetic" containers which will transmute common information into a panacea.

A digital projector, fed with data generated imagery will fill up the vessels as well as create the fluid digital environment where the Creator lives.


Here is the very first test as seen from behind, where I am trying unsuccessfully to line up, scale and correct the angle distortion. As you can see I missed the mark by a few centimeters. I am now in the process of positioning the elements as close as I can to the target, given the limitations of inexact everything.

Chakra

Nowadays it has become fashionable to "go green", like trying to patch a dam when it is about to burst. We have forgotten how, not too long ago, recycling was for many simply a way of life. But perhaps that is just because some of us grew up in a very different circumstance. People today would say that we lived in misery or poverty, therefore we were forced to straighten a bent rusted nail instead of simply buying a new one.

For me that was what made my life extraordinary. My grandmother, who had taken a "poverty vow" (go explain that to someone living in a consumer society!) lived a life where everything around her had a purpose and had to be taken care of. Since I grew up with her it was natural for me to learn and understand how to create with what the world around me, particularly our shelter, provided. To what others was a miserable shack, to me was a magnificent castle, full of adventure and stories, constantly morphing through the tension between entropy and our creative survival.

I tell all this (and there is so much more to tell) because that was, among everything else I know and care about, the root and the source of my current work. In that privileged environment where the fruit of our labor and our play (is there any difference?) both fed us and entertained us, I learned to make my first puppets and my first theater. My audience (beside my grandmother) were the three or four kids that survived in the vicinity of our house.

I say survive, because in their case even I, in my young age could tell the difference between my life, where every stone, stick or piece of paper was an immense treasure, to be enjoyed and shared and their miserable existence from which they could not possibly escape on their own.

Hundreds of years of oppression and slavery had transformed them into nothing more than a basic self-consuming organism that labored, like a fungus, to produce what others needed but without being able to make use of it themselves. They were the kids of adobe and brick makers and yet they could not build even the smallest adobe house to defend themselves from the extreme elements. Why not? think about it. If you were starving, would you build your house out of bread?

They lived in cardboard shelters, made from discarded boxes (they smeared the cardboard with asphalt to fend of the tremendous storms that are typical of the high plateaus at the center of Mexico, above 2,240 meters) just big enough to cover their bodies and not much more.

I remember, as one of the first tragedies that I witnessed in my childhood, a fire, produced by their simple kiln, that consumed their "house" in just a few minutes. I watched from my side of the river how they just stood helpless, since there was nothing they could do. The water they used for their adobes had to be carried in buckets from the river (more like a big sewage that came from the big city) and this took the majority of the day of my small friends who had to carry the heavy load.

After that I never saw them again. Perhaps something else happened within the cardboard walls that made them move away from that patch of dirt to another.


I guess this memories are prompted by the tragedy of the millions who have lost their homes and their lives. In their case, the solidity and the weight of their shelter crushed them to death, bricks and water, wind and fire. In the kids of my childhood it was the asphalt, which protected them from the rain, which burned their cardboard shacks in a flash. But they did not lose their lives just then, they had lost it many hundreds of years ago and continue to lose it under the empires who dominate the world through terror, so that their fat children can become obese with the food that the others don't eat, and their governors, kings and presidents can rot in gold and riches and trillions of petroleum dollars tinted with the blood of thousands and hundreds of thousands of people before they themselves die their solitary death.

So then I am here, putting up a sort of morality play, told from the point of view of the machine which will survive us and carry with them the good and the bad programming of the species that preceded them.

Chicken Coop

After thinking for what seemed years, about what the material for the background for the stage should be I decided to use something called "hardware cloth". I must admit I had never heard such term until I described what I wanted to a person I know at my local Loew's store. He immediately said, "you want hardware cloth", I said "no it must be metallic!". So he simply lead me to the garden area and noted that it was just a sort of "chicken wire". So that was exactly what I wanted, but did not know the name, so now you know too.

Hardware cloth

It is pretty sharp and can cut you very nicely if you are not careful. I did not use gloves because I had to bend the ends and the gloves prevented that, so it took quite a while to do it without bleeding to chicken death.



Here it is already cut and tried for a good fit. I like how it seems a natural fit to the rest of the stage. The idea is that not only supports the materials and "stuff" that I will put there, but it also acts as a grid with all the connotations that the word has in terms of media. Once I project from behind (over a translucent substrate) it will look, or so I hope, as a kind of coarsely pixilated environment, very apropos with the theme.

IR Beacon

This is the first finished beacon. It has 24 IR (infrared) LED's. The idea was that, since the tracking camera must have a wide angle in order to see the entire stage (which is about 6x6 meters, much smaller than I expected) from above, we needed an array big enough that would be easily tracked.

What I did not realize was that the angle of the LED's that I got was extremely narrow, so that when the performer lowers or tilts her head (the beacon is on top of her head) the LED's practically disappear. So we came up with the idea of using a diffusion dome to spread the light. That of course is half a ping-pong ball which happened to have the exact required diameter.

The ball was too thick and did not let enough light through, so I sanded it until it felt like a turtle egg, if you ever touched one. The solution worked, however there was still a considerable loss of light.

So I started to look for new LED's. This time I made sure that the lens of the LED would be wide enough to allow the free movement of the performer's head. This is what I found:

Ultra Wide Angle IR 850nm

These LED's are awesome. We could not believe how wide they emit light and how bright they are. You can almost turn the away from the camera and you can still see them!. So they will definitely do the trick without the need for a diffuser. We will use a set of ten which has the added benefit of resulting in a much smaller device and, because it is IR, it will be invisible to the audience.

I bought them form the guys at NaturalPoint which have a wide (yes, intended) variety of optical devices, mostly for motion-tracking. A set of ten cost me US$ 15 which is about half of the price of "cheap" LED's at RadioCrap.

So now we will design and burn another circuit and probably use a coin battery instead, to make everything really compact and unobtrusive to wear.

As soon as the UI (user interface) is ready for the tracking and motion-assigning program I will post pics of the UI and the tracking screen which is a lot of fun to see in operation.

How to solve it

My apologies to George Pólya for quoting the title of his jewel of a book, but "How to solve it" made an indelible impression on me when I read it decades ago. I wish I had had him as my math teacher instead of the monster that expelled me from his class for having the nerve to record his "lectures" (which I did not understand) in what must have been one of the earliest portable recorders (the size of a worker's box lunch) that my dad, a very forward thinking man gave me as a present so I could use technology to help me in my studies. This wicked teacher foamed (he became redder than he was and I remember his veins bulging in his neck) that my intent was to make money with his awful expositions, when my only intention was to listen to it again at home and review the parts that were unintelligible to me. Remember it was the time (1960's) of slide rulers and solving logarithms, square roots etc. with pen and paper! and teachers which threw solid wooden rulers at students if they failed to pay attention.

In any case, this is how I went about it:

The two control strings for the head (1 and 4) allow it to not only turn left and right, but as I said before to tilt sideways and bow as well. In a traditional marionette these strings are played like an instrument, varying the tension, twisting and plucking.

When the marionette needs to kneel or jump etc. you are not really concerned with this since the entire body, head and all is balanced by your hand, which uses your own personal embedded experience as a subject of gravity to effect movements which prove extremely hard to program convincingly. Imagine for example that you want the character to tilt his head slightly in an inquisitive way while bowing and kneeling at the same time. If you try to break that into individual movements performed by individual muscles of your hand, fingers and arm you will picture the complexity of the apparent simplicity. Don't forget that I said tilt his head in an inquisitive way!

So, in my dream I must have followed some of the principles Pólya suggests to poor souls in distress, like for example:

• Look for a pattern
• Draw a picture
• Solve a simpler problem
• Use a model
• Work backward
• Use a formula
• Be ingenious

The 'head' strings reach the horn which would turn the head left and right and you would think that is where the strings should end. Then I visualized (and this is what comes from the dream) that the strings at that point corresponded to my left hand fingers twisting them while my right hand controlled the other movements in unison.

So I let the strings continue on to the other 'hand' which would pull or let go to let the gravity do its thing. The combination of these elements (including the most important, gravity) give you a very rich and expressive palette of head "language". This of course I cannot convey simply by pictures, but I will eventually post a video of the movements.

The 'other hand'

I continued raiding my wife's sewing supplies and used this perfect and beautiful spool, which I epoxied to the servo horn (you can still see behind the spool the gray irregular shape of the rock hard Apoxie that I used. Need to refine that.)

There was another problem, which because of its apparent simplicity tends to be forgotten until it is too late. Remember the analogy with a string instrument. So it is that you must tune the strings of your marionette to the perfect tension which allows precise control while at the same time giving gravity and inertia room to infuse the movement with the illusion of life.

Easier said than done. First of all the strings are very thin and usually waxed or Teflon coated to prevent fraying. That makes tying and untying knots practically impossible. Then consider the awkward and tight places where the strings are connected. Finally, size. Things that are very easy at a human scale become wickedly difficult in Lilliput.

Bowl of Sushi by Hiroshige (Edo Period)

My love of sushi came to the rescue. Chopsticks! the bamboo sticks have the right diameter, are pleasant to touch and sight (and taste) and its transversal structure provides the friction necessary to prevent the 80 lb. Teflon string from sliding. This was my prototype but although it stopped that string diameter from sliding, my preferred string is .006" Fireline, which is a braided bead thread very strong and resistant to abrasion (it is used to weave the smallest glass beads). This string just went through my tensioner like a water snake.


So I refined my prototype incorporating an opposing path that effectively holds it in place. And what is more important and the reason for this exercise, it is easy to adjust as needed, to tune the strings of the instrument.

Tension is good when needed

Sense and sensibility and the Oneironaut

Visualizing the user interface for servo control

The nervous system is the current step (by step). Working on the sensors and on the beacons has given me a clearer understanding that there is nothing I know. Still very very far from The Docta Ignorantia.

Here is our microprocessor stack composed of two Make boards which were our final choice. The boards are highly integrated, very easy to interface and simply well designed. The only drawback was the limitation of 4 servo ports per board. I decided early on that 8 motors were as many as I wanted to use, both for practical and symbolic reasons. Four is the symbol of man generated by the trilogy which comes from the duality contained in the unity (bear with me, it is a stream of thought). 8 represents the power of the king in chess, and the queen in all her potency and (coming down to earth), I could only afford two Make controllers anyway! So, there. I had to do it with only 8 motors, period.

Then came the IR beacon circuit. That was a lot of fun to build although it took a lot of trial and error to do it with kitchen stuff. DYI instructions stipulate in no uncertain terms that you should use glossy inkjet paper and ONLY print with a laser printer, then iron transfer into the copper plate. Well, after many frustrating trials, botched plates and burnt fingers we (Philip and I) decided to try swhat you are not supposed to do. Went to our local print shop and had the circuit printed on their glossiest color laser paper with their fancy color printers. That worked beautifully. It transfered absolutely perfect the first time around. The entire coating of the paper stuck to the plate, and I am sure more than one has though that was wrong, but after soaking it in warm water the paper peeled off like a decal and left an impecable mask.

So here is Philip, which was the instigator of the whole DYI circuit idea cooking up the circuit with a coctel of muriatic acid mixed with hydrogen peroxide. In a few minutes our circuit was done. As usual he is multitasking with his totally hacked IPhone.

Guess where Philip is from...

Philip in my studio dremelin the circuit holes while Castro look approvingly at our socialist methodology.

He got a kick out of my retro-futuristic glasses. You look awful in them, but they have a variable focus that allows you to magnify minute stuff or read a newspaper across the room.

This section needs proofing and revising...tomorrow, after I give my class and deal with insane bureaucracy.

After dealing with the obvious and easy choices, like the arms (4 motors there) I realized the problem. My monster servo would lower the body, yes. But what about the head?, the shoulder strings? (they allow the marionette to turn the upper part of the body to face in another direction). They had to be lowered at the same time! otherwise I would need many more motors and complex programming to synchronize all these elements.

So the idea of the elevator was born, with all its ancillary problems, weight, friction, balance, stability, randomness, etc. At first I built tracks to guide the (square!) rods up and down. These rods (quads?) support a platform that houses the 3 remaining servos. The problem is that the head has more than one degree of freedom of course. It not only has to turn left and right, it also has to tilt sideways and bow as well. Or pitch, yaw and roll if you will.

All these simple motions are trivial (if difficult to master) in a marionette. Then I had the shoulder controls to turn the body around; how to do all that with 3 servos and control the wanted nuances?

After about two weeks (which is the time I have not posted!) I stumbled upon the solution, where but in a dream!. It is pretty obvious that we work as hard in our oneiric universe as we do in our waking world.

Next morning I skeptically went to examine the contraption with the suspicion that, like many other "great ideas" conceived in la-la land it would turn to be impractical, impossible or just plain turdy. It actually took me a while to connect my ideas with the reality of the mechanics, something was missing which I had apparently forgotten. Then, all of a sudden I saw it! make one of the servos perform double duty.

I cannot even explain how I visualized it. It was so obvious, simple, and yes, elegant. The only bummer is that I feel it did not come from me but from the collective unconscious debris. I just picked up the pieces.

I will post some pictures to explain how it works.