Category Archives: rambling

Audience Interactions

In recent years there seems to be a wave of Audience Interaction (participation?) technologies. Here I just list a few that I am aware of, if you know others, let me know!

One of the earliest systems I know of is an experiment by Loren Carpenter presented at the Siggraph conference in 1991, (historic video here). He provided an audience with small paddles, reflective green one side and reflective red on the other. Using cameras and some algorithms he projected just these paddles on a screen. Then he went through a series of applications for these image processing algorithms. Ranging from simply showing the paddles as pixels to having (parts of) the audience control the movements of game-elements depending on the red/green ratio. For example the paddles of a game of Pong.

Cinematrix game example


This technology was patented and further exploited in a company called Cinematrix.


At ICMI2002 , Dan Aminzade, Randy Pausch and Steve Seitz presented work on Interactive Audience Participation. On his (old) work pages, Aminzade presents some of the ways the techniques presented were implemented. Watch him talk about it:


Less interactive, but still causing awesome effects are systems that more or less turn audience members into pixels.

in 2012 FanFlash saw its premiere on German TV:


Recently you may have seen the halftime show of the 2014 Superbowl. It’s the clever guys at PixMob that were responsible for the light effects there.

They have done and are still doing various versions of their technology for turning the audience into pixels that are part of the light show. My personal favourite is the beachballs at Coachella in 2011.

Someone who has always been very effective at engaging audiences is DJ Tiësto. He uses PixMob’s audience-as-pixels technologies well in his sets.


Another company, Embraceled, has a similar system that was used at at a party called Sensation.

They also promote another application for business events, that bridges online and offline social networking. (Warning: corporate company presentation video with cheesy background music, in dutch)


People as pixels centrally controlled, reminds me of something:





I am still musing on what axis or in what space these different projects may be placed.



Scalable group touch: Spots

screen capture of 3 spots

Up until now, I have avoided making computer (screen) based group interaction media. The reason being that it seems more fitting to work in the tangible, physical domain; We understand things when we engage our bodies in interacting with the world on a level different from when we interpret things rationally.
I have made contraptions and installations through which groups of a few more than two people can interact. The problem with working with such media is the cost (in terms of time, effort and money) involved in scaling up to much larger groups.
Screen based interaction media are more easily scaled. A middle-road between completely screen-based and completely tangible, seems to be the mobile phone. Also for the role that they play in our lives it seems to be quite fitting to develop an interaction medium on that platform.

So I did.


In its current  -pilot- stage, Spots allows people to experience a sense of mutual touch through a mobile phone.
When you start Spots, you see an empty grey screen. When you drag your finger across the screen, a spot of approximately the size of your finger follows your finger. These actions are broadcast to other people running Spots.
When other people with the app come ‘online’, a vaguely visible spot appears with a soft sound and their touch actions leave fading traces. When your spot and one of theirs overlap, a vibration can be felt and a quickly fading ripple is visible, showing a trace of your touch.

You can try it out! Currently there are these versions:
Spots for Android
Spots for Mac (nb: this is not a signed app, see below)
If the app starts but nothing happens, either you have no internet connection or the server is down. Tap/Click the top 10 pixels of the app screen to bring up some detailed info. Let me know your experiences.
On Mac, if it says the file is damaged when you try to open it, you need to allow your mac to run unsigned applications: go to System Preferences>Security&Privacy, on the General tab select Allow applications downloaded from: Anywhere.

The video above shows what it looks like from the Spots server point of view (which runs on a mac).

Spots is a re-imagining for my own research context of the app touchThrough developed by Gabrielle Le Bihan cs. (Gabrielle has presented touchThrough at TEI’13 and published some of her research with it at CHI’13).

Yes, I am aware of the very nice Feel Me project by Marco Triverio. There certainly are similarities, but our intentions are quite different in my view.
I am developing a research design that mediates between an individual and the group dynamics he is part of. Marco developed an app that aims to establish an intimate link between individuals.

One end of the spectrum: Inspired by Cololo

I am building the slider systems (v01 and v02) to do some experiments from which we hope to study how varying qualities of an interaction medium influences the experience.
A project that shows a minimalist version of mediated interaction and telepresence, is Cololo, from the Uchiyama lab in Tsukuba. (be sure to have a look at some of their other projects)

I have now gotten my slider system to mimic the Cololo behavior as follows: When one slider is moved, the other slider moves randomly for about 4 seconds. During that period, the system does not respond to input on either slider.

I set out to maintain the possibility to have feedback on slider A about the random movement of slider B in response to the initial moving of slider A. However this proved rather tricky in a closed loop feedback system. It did teach me a thing or two about how to implement such behavior. Moreover it proved once more on the one hand that my current platform has its limits for more complex behaviors, and on the other hand my own limitations when it comes to ‘control systems’ theory and implementation.

Luckily the Cololo system doesn’t have any direct feedback, so for now I don’t need it. In my current code I applied a bit of a blunt method to get the Cololo behavior. In future iterations I will definitely need the feedback, so I will have to come up with a more elegant solution. I am now looking into possible collaborations with experts in the field of mechanical engineering and control systems theory.
My current arduino code for the Cololo behavior can be downloaded here (as a zip archive).

IASDR 2011 paper submitted

I have been working with Pierre Lévy of the DQI group here in Eindhoven, on a paper for the IASDR 2011 conference in Delft.
The abstract was accepted some weeks ago and now, after a spurt effort, the paper is done and submitted. Thank you Pierre for taking the lead and pulling this through. Now all we can do is keep our fingers crossed. As soon as we know if it is accepted, I’ll give a synopsis here.

good intentions

I solemnly vow to keep notes on my research on this page;

  • thoughts and summaries on books and papers I read
  • updates, video and pictures on builds and hacks
  • others’ inspiring work
  • ramblings and so on