Costumes as Game Controllers: Lightning Bug Game Projection Testing

Costumes as Game Controllers, Face-to-face Games, tweeted!

I am a huge fan of cinema, and I like projection mapping and know of the magic projections can do, but I have never been on the development side of the experiences. To make the Lightning Bug game, I originally envisioned a large dome with the hope to make the experience more immersive for the players. Note in this sketch that the projectors are on the outside of the dome.


At the point in the project, I think half a dome, a quarter of a sphere, should be enough. Concentrating on the half dome will make it exponentially easier to make it a full dome later. I don’t know enough about projections and to take small steps is the saner route to take.

I will be fabricating the dome myself, keeping in mind portability and stability. I need to travel with the dome, and so it needs to be lightweight and collapsible. It needs to also be stable enough so that we can create some kind of consistency with the projection so that it can be set up anywhere. The dome will probably be some kind of stretch fabric with conduits for flexible plastic rods, kind of like a tent. I am not sure what the flexible plastic rod material would be, so right now let’s call them “bones”.

Fortunately, my good friend Kyle Li is working with me on the projection part of the game. He’s got experience working on projected installations and dome projections too. He’s amazing! Last week, at our first meeting Kyle suggested a few alternative ways to project on to the dome. One way is to project from the inside against a curved mirror. This is a good alternative, because the seams on the screen won’t block out the projection, because the “bones” would be on the outside. There is also the option to projection from the inside without mirrors, but the players could interfere with the projection, depending on where the projector is and how high the projected image would be on the dome surface.


Yesterday Kyle and I had our second meeting, and we tried a paper miniature version of the dome. Kyle took some amazing photos of the various options. The two tree models represent the 2 players.



I am so excited about where this is going. The next steps in the actual projection testing will take place inside the large space that Eyebeam is going to let me use for the last several days of this month. In the meantime, I need to do more research about the following to prepare to work in the large space:

  • Calculating and making 2D patterns for the gores, sections on the dome. I was using a parachute calculator on the internet, but I soon found out parachute domes are not exactly what I am looking for. *fail*
  • Researching fabric. Will probably go to the fabric stores in the garment district and look for a semi-opaque material that is tough, possibly stretchy and white.
  • Researching materials for “bones”

Hit Me! 2011 Project list

Eyebeam, Face-to-face Games, Github, Hit Me!

On Friday, I figured out the doorbell receiver signals and how arduino can tell which doorbell is being rung. Here is the project list which helps me code but also know exactly what goes where.

1. Rest Start Button which goes to Intro
Show screenshot of last game
Start button Nothing Background loops
2. Intro Intro background
Show 2 streaming videos
Wait for snapbutton
Timer to 3 mins then returns to Rest
Exit button
Exit button Snapbutton Introduction!
3. Game Game background
Timer runs for 30 sec
Has 2 pics of players
If there has been a previous hit
the snap shot from the prev hit
if game has ended
then go to ScoreMail
2 streamed video off screen
Exit button
Exit button Doorbell signals Hit Me!
Game Loop
4. Hit Takes snapshot from whatever stream
puts it on the screenshot
Who did hit
If judge adds points
then show total points
score from hit
return to game
Exit button
Exit button Judge point input Ding ding ding
5. ScoreMail Final score
Final winner
Archived snapshots
Email data
Exit button
Exit button Ding ding ding

Hit Me! 2.0

Eyebeam, Face-to-face Games, Hit Me!, Physical Computing

I will be showing Hit Me! at the next Eyebeam Mixer. I am really excited about it. I need to do some updating to the game. Here is a list of intended updates:

1. Better wireless pin-hole cameras. For the game, I need 2 cameras that are same but run on different channels. I found some rechargeable ones at Geeks that have a choice of 4 different channels around the 2.4 Ghz frequency. Unfortunately I won’t know how the system would run in a space until I actually try it out. There are always going to be things that run on the 2.4 Ghz range, as well as the 900 Mhz range that my older cameras ran on. In the Chelsea Museum show, the old cameras conflicted with the project that was running right before mine, but it worked fine as soon as the previous project was turned off. So I am looking forward to getting them quick to try out in the space.

2. Better doorbell system. For the game, I need 2 that run on different channels like the cameras, but they can share the same receiver. I found a set on amazon which can run on a range of frequencies it seems, but I am waiting for an email reply from the manufacturer about what frequency it runs on. I don’t want anything that would interfere with the cameras. Also I would like to get ones with lower latency from my previous set. However, latency is not really an important function when it comes to doorbells, so it’s not a function that is highlighted or written about. Basically it’s hard to research that information before purchasing.

3. Rewriting software on Processing.

4. Composite Video to USB adapter. I already have a one that goes to firewire.

MM5000 Vest

Eyebeam, Face-to-face Games, Mary Mack 5000

Originally the MM5000 shoulder and thigh pads were made from bandanas and D-rings. However, it’s been quite a hassle. They slide off while playing, they take a long time to put on and are equally annoying to take off.

So the solution to this problem is to create a vest interface that is much easier to put on and off. And not only does the vest add to the “rocker” feel of the game, but also it provides a nice surface to mount the sensor pads on to. I found some RJ50, also known as 10P10C cables, which have 10 conductors. They look like ethernet cables and fit into a similar looking jack. These jacks I will place on the vest and the console box. Colin Leipelt constructed the muslin version (center) from paper patterns (right) that I made off of the terribly hideous looking vest (left) I got from a thrift store. I have revised the paper pattern since then to give a more “rocker”, jean-jacket vest type feel silhouette-wise. The vest will be made from black denim and will be appropriately distressed.

To Do List for Kagemusha: Shadow Warrior

Eyebeam, Face-to-face Games, Ninja Shadow Warrior

Coding: Finish coding game in Processing

Research Social Software: check out open source software like Drupal. Need to be able to do following:

  • Can upload photos to
  • Can draw on photos and saves that image
  • Can have people register individually and then create teams
  • Individuals can keep accounts which are then connected to teams
  • Individuals can score or rank images
  • Game results by groups can be posted on a leaderboard
  • Detailed results of a game (score, snapshot, ranking) can be shown
  • Results of a game can be sent via email to anyone
  • Can connect somewhat seamlessly to the actual game coded in Processing

Start Thinking About Harware and Enclosure:

  • computer/laptop
  • screen
  • camera
  • arcade game like interface: push buttons
  • wooden standalone game enclosure reminiscent of arcade games

Kagemusha: Shadow Warrior

Eyebeam, Face-to-face Games, Ninja Shadow Warrior

Kagemusha: Shadow Warrior is a new game I have been working on. It is a stand alone arcade game built with a screen and webcam. The goal is to fill out the shadow of an object as accurately as possible. Scores based on accuracy are kept on a leaderboard. The game structure of Shadow Warrior naturally supports face-to-face interaction, as more details can be filled out when more people are playing at once. This game will experiment with crowd sourcing content for the shadow object database.

Mary Mack 5000: Brainstorming

Eyebeam, Face-to-face Games, Mary Mack 5000, Physical Computing

The most important role of gloves in Mary Mack 5000 is to measure the accuracy of the claps. The technology must be able to assess whether or not the correct claps are being made at the correct time.

I first started out mapping out all possible clapping combinations.

Ideas like pressure sensor combined with color sensors & colorful gloves or pressure sensors combined with RFID were considered, but ultimately a simple method of ID through contact would be most suitable for the game. I bounced around several scenarios with Marko, and while a voltage ID would work, it was a mysetery how the gloves would be able to read each other’s voltage ID. Remember, each gloves would have to have both voltage and input to work with all clapping combos (clapping hands together alone, clapping hands with the other person, clapping just the right hand with the other’s right hand etc…). Combining two voltages would lead to some new voltage that stood between the 2 original voltages and creating 2 pads per gloves wouldn’t work with the various combos with the different directions the pads would face for different claps.

On Jie Qi’s last day at Eyebeam, I explained her the problem, and she suggested flipping really fast between voltage and input. So each glove would have its own unique voltage ID and yet be able to read the unique voltage ID of another glove. Thanks to Jie, this was the elegant solution to the problem!

Mary Mack 5000

Eyebeam, Face-to-face Games, Mary Mack 5000

Mary Mack 5000 is a project I have been working on at my Eyebeam fellowship. It’s a game that reintroduces the popular Patty Cake little girl’s games in a hyper, rocked out, digitized version. It uses special finger-less gloves with conductive pads, as well as pads on the thighs and upper arms. The sensors help measure timing and accuracy of two people playing the clapping game, and the score is based on these two factors. A projected interface keeps the players and spectators informed of the score and types of claps.

I am collaborating with Lina Fenequito. There are also some incredible music tracks made by Ray Mancini and Sergey Popovich of Goodswan.

The game debuted last Friday at the Come Out and Play Festival at the Brooklyn Lyceum.

I will be updated more about the development of the game here as it progresses.