Category Archives: inspiration

Audience Interactions

In recent years there seems to be a wave of Audience Interaction (participation?) technologies. Here I just list a few that I am aware of, if you know others, let me know!

One of the earliest systems I know of is an experiment by Loren Carpenter presented at the Siggraph conference in 1991, (historic video here). He provided an audience with small paddles, reflective green one side and reflective red on the other. Using cameras and some algorithms he projected just these paddles on a screen. Then he went through a series of applications for these image processing algorithms. Ranging from simply showing the paddles as pixels to having (parts of) the audience control the movements of game-elements depending on the red/green ratio. For example the paddles of a game of Pong.

Cinematrix game example


This technology was patented and further exploited in a company called Cinematrix.


At ICMI2002 , Dan Aminzade, Randy Pausch and Steve Seitz presented work on Interactive Audience Participation. On his (old) work pages, Aminzade presents some of the ways the techniques presented were implemented. Watch him talk about it:


Less interactive, but still causing awesome effects are systems that more or less turn audience members into pixels.

in 2012 FanFlash saw its premiere on German TV:


Recently you may have seen the halftime show of the 2014 Superbowl. It’s the clever guys at PixMob that were responsible for the light effects there.

They have done and are still doing various versions of their technology for turning the audience into pixels that are part of the light show. My personal favourite is the beachballs at Coachella in 2011.

Someone who has always been very effective at engaging audiences is DJ Tiësto. He uses PixMob’s audience-as-pixels technologies well in his sets.


Another company, Embraceled, has a similar system that was used at at a party called Sensation.

They also promote another application for business events, that bridges online and offline social networking. (Warning: corporate company presentation video with cheesy background music, in dutch)


People as pixels centrally controlled, reminds me of something:





I am still musing on what axis or in what space these different projects may be placed.



‘Affective Computing’ – Affective Interaction

Over at, they have been publishing chapters of an online interaction design encyclopedia, where each chapter is written by an expert in a particular area in the field. I had come across it before for the chapter on Social Computing, which discusses crowd-sourcing type implementations of social computing such as wikipedia and scoring systems in e.g. Amazon and discussing mediated communication systems like twitter and facebook. A nice read and particularly a possibly helpful breakdown of crucial elements of such social systems.

Now I am posting as there is a thought provoking discussion of Affective Computing in response to a chapter of the same name written by Kristina Höök. The chapter  is followed by a well written response from prof. Roz Picard, who coined the term Affective Computing and drafted the first maps of that then poorly-charted terrain in her book Affective Computing. Next a rather artless response from one of the bigger names in the field of design and emotion, prof. Paul Hekkert. I do have to agree with the jest of his response. He amongst others, quotes the work on the compelling concept of ‘inherent feedback’ by my colleague and friend Miguel Bruns Alonso. The discussion centers on wether some HCI/AI approaches to Affective Computing is reductionist and cognitivist versus a more holistic and phenomenological approach to Affective Interaction that Höök describes. the Interactional Approach. That concept reminds me of the work of Stephan Wensveen, e.g. his paper at DIS in  2000 Touch Me, Hit Me and I Know How You Feel. A Design Approach to Emotionally Rich Interaction and then in his 2005 dissertation: A Tangibility Approach to Affective Interaction.

Against this backdrop I am currently reading the PhD thesis work of Joris Janssen, Joris’ work at first sight seems to take a reductionistic approach to empathic mediation (what he calls physiosocial technology), but particularly in the later chapters walks he shows examples of both a reductionist -lab based- approach and more holistic -real life- setting.
What fascinates me most in Joris’ work is the idea of technology that supports (even promotes?) empathy in dyads (2 people). Of particular interest for me is his (forthcoming) research on providing some form of feedback based on the correlation between one persons physiological signals and those of another. About a year ago Joris and I discussed social bio-feedback in more detail, after we got into contact over my Masters’ thesis on that topic and his paper on Intimate Heartbeats. Particular issue we discussed then was the modality of such feedback.

Control theory demo

A nice example of control theory relevant to interaction design, made by Aldo Hoeben (thanks for the link!) at the TUD IO StudioLab, back in 2002/2003

full screen

DDD: Light

Lamp_turret — Once in a while the DQI group at ID TU/e does a few days’ workshop: DQI Doing Days. This time we explored the design of complex systems through creating dynamic light objects that communicate. Three groups each created two objects that communicate either through their light behavior, wirelessly (through xBee’s) or both. We were asked to create functions in the arduino code that could be activated  by other light-objects. This workshop was organized by Remco Magielse and Serge Offermans of the Intelligent Lighting Institute. My group set out to create an object that would disrupt the behavior of the others’ objects, both through light and through xBee communication. We created two turrets, one with a light sensor and one with 3W LED’s. The turret with the sensor continuously searches for the lightest spot around it, telling the other turret where it is. The turret with the super bright LED’s will then turn in the same direction to ‘blind’ the brightest spot. (we assumed that other light-objects would also have light sensors and that delivering a bright light to them would then disable any of their behavior that would be based on varying light levels). In the end the behavior is a bit more surveillance than virus-like. Continue reading

One end of the spectrum: Inspired by Cololo

I am building the slider systems (v01 and v02) to do some experiments from which we hope to study how varying qualities of an interaction medium influences the experience.
A project that shows a minimalist version of mediated interaction and telepresence, is Cololo, from the Uchiyama lab in Tsukuba. (be sure to have a look at some of their other projects)

I have now gotten my slider system to mimic the Cololo behavior as follows: When one slider is moved, the other slider moves randomly for about 4 seconds. During that period, the system does not respond to input on either slider.

I set out to maintain the possibility to have feedback on slider A about the random movement of slider B in response to the initial moving of slider A. However this proved rather tricky in a closed loop feedback system. It did teach me a thing or two about how to implement such behavior. Moreover it proved once more on the one hand that my current platform has its limits for more complex behaviors, and on the other hand my own limitations when it comes to ‘control systems’ theory and implementation.

Luckily the Cololo system doesn’t have any direct feedback, so for now I don’t need it. In my current code I applied a bit of a blunt method to get the Cololo behavior. In future iterations I will definitely need the feedback, so I will have to come up with a more elegant solution. I am now looking into possible collaborations with experts in the field of mechanical engineering and control systems theory.
My current arduino code for the Cololo behavior can be downloaded here (as a zip archive).

google scholar skimming

don’t forget to skim:

google scholar with force feedback in mediated interaction

google scholar with inTouch article