int 0×30:

Below is a short movie demonstrating my recent E15 iPhone integration, enabling 3D navigation functionality with an image collection obtained from YouTube video thumbnails. It also shows E15 playing live video from YouTube, something I’ve never quite gotten around to showing in anything other than static images on my Flickr account and E15:Web.

When all of the excitement about the beta iPhone developer program was happening a few months ago, I must’ve been busy writing my thesis or something, because I really didn’t take much notice. About a week and a half ago, I decided I really wanted to build some iPhone support into E15, so I applied to the developer program, expecting to get rejected as so many people have. Then, two days later, I got accepted. Excellent.

Although I really haven’t been looking, I don’t think I’ve seen any iPhone applications that use multi-touch for 3D navigation, so I thought I’d see if I could come up with a simple mapping suitable for navigation in E15, effectively turning the iPhone into a device for remote navigation of a 3D space. The iPhone multi-touch interface provides additional degrees of freedom that can be exploited for navigating spaces that are not inherently 2D. The diagram below shows the mapping I came up with, using the pinching gesture to navigate in the z direction, and parallel two-fingered gestures for rotation about the x and y axes (rotation about z doesn’t seem to be very interesting at the moment). In the diagram, x runs horizontally, and y runs vertically. Solid lines denote the directions that are manipulated by each gesture.

After sketching these up, I wrote an iPhone application that detects the relevant gestures, and communicates with a running E15 instance on another machine, sending this gesture information over the network. I added E15 support to handle this incoming data, as well as built some infrastructure for invoking user-defined Python callbacks in response to actions received from the iPhone (e.g. tapping and shaking).