REGISTER for Eyebeam workshops. Web Hacking, Virtual Sculpture, RGBDToolkit, and more!
APPLY NOW for Eyebeam Residencies and Fellowships. Open Call deadline is June 14.
I downloaded his file and ran it through my Stranger Visions software today to generate my take on his face. You can see a side by side comparison of my generated face and his actual face (well pulled from google image search results) below.
I am still very much in the middle of my wetlab work for Stranger Visions but the software side – the part that takes DNA information and produces a 3d model of a face – is mostly complete. I thought I would talk a little bit about the software and the concept of inductive bias in this next installment on Stranger Visions. This is a slightly technical post but there is a kind of ethical question at the end…
I have worked with facial recognition algorithms in the past and one technique I had read about was the use of a morphable 3d facial model to attempt to recognize faces at weird angles. As I have previously turned systems like this, intended for surveillance, into systems for creativity, it naturally occurred to me that the same system could be used to generate faces. And if you can determine what correlates certain types of faces you can then generate faces with those characteristics using principal components analysis.
It turns out the research group behind the Basel Morphable Model realized this as well and after much digging I figured out how to use their matlab model as a starting point for many different types of parameters. They had already found the primary axes for gender, age and weight so the main parameter missing was ethnicity.
The issue here of course is that all these models are heavily skewed towards people of european descent so I knew from the beginning that I would have to deal with significant inductive bias. For those who have never read AI textbooks, inductive bias describes the set of assumptions a programmer makes in the design of a machine learning system. Or as Luger puts it in Artificial Intelligence “Inductive bias refers to any criteria a learner uses to constrain the concept space or select concepts within that space”. It is actually a pretty fascinating subject in its own right that dates back to the origins of inductive logic with Aristotle. The basic idea is that by looking at a certain number of examples of something we can form generalization about all things of this type. Now we all know what happens when we make assumptions based on limited experiences, so the flaws of inductive reasoning are pretty obvious. Then again, all reasoning rests on making some initial assumptions (otherwise we have nothing to deduce from!) so at the end of the day some amount of induction is unavoidable. For more on induction, deduction and their interdependence see Hume’s An Enquiry Concerning Human Understanding.
In programming inductive bias can be anything as simple as how many clusters to look for in a data set, to much more difficult issues like how to get a representative sample of images of a population for a face recognition system. (And just what is “representative” anyway?)
The Basel model I started with was trained on an even split of men and women but the majority of people in the training set were of european descent. This means that even though the model can be fitted to parameters representing other ethnicities it always has a slight in-built bias toward caucasian faces. With this in mind I then trained a new version of this program by fitting images from the FERET face recognition dataset (which is more representative of ethnicity composition in the states but still not perfect). I trained the system based on my own subjective interpretations and thus, like much of an artist’s work, the faces it produces are as much a reflection of me and my view of the world as they are any kind of objective scientific model. Then again, most academic researchers follow the same technique, utilizing the judgements of 5 men in an office instead of 1 woman in a studio.
I did all this work in matlab which I am by no means an expert in, and so I switched to Python for the rest of the code – mainly reading in various data files and turning that info into parameters for the matlab morphable model. The Biopython library proved valuable here!
And now I have a bit of an internal debate as to whether I should open source all this work or not. My impulse is to post the code online and let people do what they wish with it. But then another part of me wonders who will want to use it and what for? The point of the project is criticality and dialogue, not proving that this is technically possible. I never had the intention of making a “genetic portrait toolkit”. Do I really want to make this easier? Thoughts?
Thanks to everyone who made it out to Eyebeam Open Studios! For those who didn’t, here are some pictures of my installation at the event. ALSO Stranger Visions is now up in a snazzy presentation in the Eyebeam bookstore/lobby til August, check it out if you are in Chelsea. (and consider donating to Eyebeam’s kickstarter campaign if you like their work!)
Working on the Zajal installation for Open Studios. It’s going to be a hackable asteroids/shmup game with physical Arduino-driven controls. Exciting!
These screenshots demonstrate the asteroid generation algorithm I’m working on.
Second prototype of قلب’s code calligraphy. This one says مرحبا يا عالم (
hello world), which isn’t executable code, but code-related and fun anyway.
This is seven tiles high, while the last one was only five. Seven tiles allow a lot more flexibility, and that’s what I’m going to go forward with. I was being overly minimal with the five-tile high constraint.