Nobody really knows what the future of human interfaces and gaming will look like, but Andrew Fentem—who went from working on classified missile systems to developing multi-touch human interfaces, kinetic surfaces and motion sensing technologies before almost anyone else in the planet—gave us a fascinating vision on where we are headed in this exclusive interview. Work like his Fentix Cube, a motion- and touch-sensing cube which can play Pac-Man among other games, have all the big companies taking notes. The videos speak for themselves.
The Fentix Cube is just the tip of the iceberg of his stunning work. Many of his inventions are still ahead of current technology, things which we are only starting to get familiar with now. I talked with Andrew about his inventions and the future of human interfaces.
Jesús Díaz: We are big fans of multi-touch technology and think it's the future of adaptive user interfaces. Do you see them replacing the keyboard and mouse in many applications? I'm just looking at your sequencer now...
Andrew Fentem: Touchscreen and 'multi-touch' technologies have a bright future, and will certainly replace the keyboard and mouse in *some* applications. However, the keyboard and mouse have some BIG advantages that have proved hard to overcome: A physical keyboard provides great tactile feedback - meaning that you don't have to look at what your fingers are doing while you type. And the great thing about a mouse pointer is that it doesn't obscure what it's pointing at on the screen (unlike your fat dirty fingers - this is why the buttons are so big on touchscreen ATMs - wasting valuable screen real estate).
Another issue with touchscreen technologies (and multi-touch systems in particular) is their inability to track rapid finger movements. This not only puts many applications (especially certain types of games) off-limits, but can also interfere with gesture recognition.
The key future developments of touchscreen/multi-touch systems will be:
1. faster touch sensing hardware and firmware
2. improved (i.e. more intelligent) gesture-sensing software
3. improved tactile feedback
4. larger touchscreens
The ultra-high-speed touchscreen that I built back in 2001/2002 — which I am still confident is the fastest large-scale touchscreen ever built - demonstrated how improved touchscreen technologies could be used to create exciting high-speed touch-oriented game systems like virtual air-hockey. There are now plenty of other virtual air hockey systems on Youtube - but they're all a bit sluggish because the off-the-shelf touch sensing and data processing sub-systems that have been employed by the designers are too slow for the job.
The "Tactile Multi-touch Sequencer" that I developed in 2004 showed how combining multi-touch finger sensing with multi-object sensing could improve a multi-touch system - enabling you to program the machine with your fingers, but also by moving a multitude of small objects around the surface (thereby freeing up some of your fingers, and making the interface more tactile).
JD: How long have you been investigating touch surfaces and alternative user interfaces (like accelerometers or kinetic surfaces)?
AF: Since 2001. Before that I was running a consultancy company advising market-leading companies about product innovation. I was always shocked how unreceptive big-name hi-tech companies were to new ideas.
So, having a fairly unusual background in both user-interface research, and in military and music electronics research, I thought I could maybe do better myself. Everyone at the end of the 90s was obsessed with software and the Internet. It looked like no one in the West was really innovating hardware, so I guessed it might be easy to develop novel eye-catching stuff.
I was also keen to challenge the received wisdom that the complexities of modern electronics, operating systems, firmware, and software mean that you need a large team to develop a sophisticated world-class gadgets. I suspected that all you really need is some creativity and one or two ultra-motivated alpha geeks.
Expressing my motivations in more artistic/cultural terms, I suppose I also wanted to make an 'intervention' in the gadget market - i.e. just put some stuff out there and see what came back. One of my friends calls this calculated recklessness "Gonzo style" product development. I've always been interested in art - I attended art college for a bit, and ran a kitsch/ironic hairdressing salon/DIY clothes boutique for year or so when I was at college.
So anyway, in 2001 I started developing ideas around a concept that at the time I called "Couture Electronics" - i.e. hardware that is big, expensive, fast, hi-spec, and beautiful.
I went to trade shows and asked the touchscreen market leaders if they could sell me a multi-touch touchscreen system - the sales reps just looked at me like I was weird, and asked me why anyone would want one. I just shrugged and thought, "Hmm, this could be an interesting opportunity.."
JD: Are you working in developing better tactile feedback to touch surfaces beyond haptic vibration or is the technology not there yet? I'm imagining flexible OLED surfaces that can have bits raising, for example...
AF: No. It would be nice, but other companies have moved heavily into this area now - Sony, Apple etc.., so I've moved on - you have to stay agile and on the edge if you're a small operator. (Obviously if Apple, Sony, Mattel, Microsoft, Motorola, or whoever want to pay me to design edgy stuff for them I'd be happy to oblige...)
One of the reasons for publicizing the cube was to attract investment for the development of other gadget technologies that I'm currently developing - gadgets promising even better fun/dollar ratios.
Judging by the traffic on my website and the positive global reaction to the Fentix Cube, this exercise seems to have gone reasonably according to plan.
JD: The Fentix Cube seems to have definitive commercial potential, have you commercialized any of your developments?
AF: Yes, mainly in the form of one-off projects though. As well as developing gadgets, I'm currently being asked to consider some pretty interesting architectural electronics for skyscrapers in London. These are going to be BIG gadgets!
However, my main aim for 2008 is to launch at least one major mass-market commercial product.
The nearest I came to commercializing the multi-touch technology was back in 2002-2004. I negotiated a deal with a manufacturer (Novation EMS Ltd) to start manufacturing multi-touch hardware interfaces, but in mid-2004 they went into administration after losing money on other projects. A UK Government R&D fund who were also backing the project then bailed because of the "increased risk". It was a great shame and a gut-wrenching experience after having been so far ahead of Apple's teams of 'innovation gurus' for such a long time.
People in the UK tend to be extremely risk-averse - consequently there's not much VC culture over here. I'm currently trying to forge links outside of the UK, and would love to get the opportunity to work somewhere more like the US. (I spent a very short period at Harvard - it was a great experience.)
JD: In the BBC article I read about your work a while ago, you mention you are being bombarded by Korean-based toy firms. Have there been any interest by mainstream giant toy groups like Mattel and the like?
AF: I get a lot of traffic on my website - mostly from universities and a broad range of hi-tech companies in the US and Europe - everyone from Lucasfilm and Disney, to Sony and Microsoft. The big companies don't get in touch directly though - it seems their employees just spend all day gazing at my website. (I *adore* Google Analytics!)
Hasbro (the giant toy group) appears to be my largest single source of traffic. I would *love* to know what they're up to. Maybe they just want to see what the future looks like ;-)
Most of the serious offers that I've received have been from firms in Asia - they tend to be super keen and want everything done yesterday. It's a refreshing attitude.
JD: What price do you think the Fentix Cube could have in the market?
AF: It all really comes down to the cost of the screen hardware. I can't see it being much less than $100 as things stand at the moment. There are a lot of decisions to be made that could dramatically effect this pricing though.
JD: In a commercial Fentix Cube, would users be able to load new software?
AF: Yes. I'm a great believer in DIY, open-source development, and end-user customization. The Rubik's Cube and Pac-Man style games were written to demonstrate the potential of the 3D touchscreen and motion-sensing capabilities. I'm hoping that people will come up with game ideas that could merge the two. I've already been emailed an enormous range of ideas - 3D Snake games, modeling fluid dynamics, math puzzles, aids for the handicapped, game controllers, you name it...
JD: Does it have wireless networking? In other words, can the Fentix Cube connect to other Fentix Cubes either directly or through the Internet?
AF: Yes...but at the moment it's fairly rudimentary.
JD: I'm also imagining using the cube as an alternative interface for computers. Like a custom interface for editing video or music, either off-line or in real time (for performances). Can the Fentix Cube connect to other devices and act as a UI front-end?
AF: Yes. Theoretically. But I'm sure that there are plenty of mouse manufacturers like Logitech working on this kind of thing already. I'll leave it to them.
Hopefully, we will see Andrew's stunning work in commercial products soon. Steve (Jobs) or Steve (Ballmer), grab this guy's stuff, pronto! [Andrew Fentem]