Microsoft’s new tabletop Surface technology, six years in the making, moves optical technology forward by combining physical and virtual worlds.
When Steve Bathiche and Andy Wilson conceived their idea of “surface computing” at Microsoft in 2001, they were way ahead of their time. Fortunately, they could afford to wait more than six years to get the idea to market.
The engineers wanted to create a table that could interact with objects placed on top of it. They led a small team starting in October of 2001 that sought to build a new way of interacting with a computer through touch and vision. About six years later, Microsoft has finally unveiled its Surface Computer. It enables people to directly interact with the interface on the table top, dragging and dropping virtual objects such as digital photos with their fingers. It also recognizes physical objects placed on top of it, including Microsoft’s Zune portable media players. For engineers, this is the ultimate play toy, allowing for experimentation with a wide array of applications that combine the physical and virtual worlds.
The Microsoft Surface engineers weren’t under pressure to get something out fast, but they knew their work would never see the light of day unless they came up with something far different from today’s personal computer. They had to embrace ideas that required new technologies to be invented or that would come down in cost over time. One of the fundamental things the team had to wait for was Windows Vista, since the table’s interface sits on top of the operating system.
The nearly six years was enough time for the team to make a series of tough engineering decisions on its code-named “Milan” project. One of the reasons it took so long was that the team created prototype after prototype in an effort to create not just a product but something that could become a universal platform for computers embedded in walls, desks, tables, refrigerators or other surfaces.
Early on, Bathiche and Wilson thought they would build a PlayTable, something that would be used for games such as pinball. By early 2003, the team made a presentation to Bill Gates. He liked the idea and encouraged more exploration. They build a modest prototype dubbed T1, based on an IKEA table with a hole cut in the top and a sheet of architect vellum used as a diffuser. The team started developing applications such as a pinball game, a photo browser and a video puzzle.
They favored applications that exploited the unique ability of a surface computer to recognize an object placed on it. The first prototype had an internal projector that bounced images off of a mirror. But the team found it hard to get stable results from the mirror. One of the problems was dust. The cameras below the table detected too much detail, so even the smallest specks of dust threw off the recognition, says Nigel Keam, who joined the team to lead software development in 2004. The team found they could engineer around that problem by sealing the table, but that messed up the electronics, which needed air flow to keep the overall computer system cool.
And with the low height of a typical table, they couldn’t find a camera with a wide enough lens to clearly recognize everything that was placed on top of it. Because of these tough problems, the team built more than 85 prototypes.
They settled upon 8-bit, .75-inch tags that the cameras underneath the table could visually recognize. The tags would have patterns of dots that the cameras could see and translate into the computer. Hence, the system would be able to tell one object apart from another. Keam, who eventually became the team’s architect, says they could have used radio frequency identification tags, but the technology wasn’t widely accepted at the time they were exploring options.
“We don’t rule anything out,” Keam says. “If near-field communications becomes popular, then we could incorporate it. We are trying not to invent anything that we don’t need to in that regard.”
Keam says it made sense to use the same digital light processing technology that is behind rear-projection TVs to project images from the bottom of the table to the top, which is a translucent acrylic screen. And the team ultimately had to use five cameras to be able to capture objects that could be placed on any part of the table. The extra cameras helped get around the field of view problem and allow for shorter tables, and they also fed a lot more data to the computer in a parallel fashion, allowing for instantaneous recognition of many objects placed on the table at the same time.
Because the dots were new, Keam had to coordinate with other product divisions. The team developing the Zune portable media player, for instance, had to be aware of the tags and the way that the table would recognize a Zune that was placed upon it.
Keam says that most of the technology in the table is off-the-shelf electronics that can be cost reduced over time. The table includes a computer with an Intel Pentium 4 microprocessor and a fairly powerful graphics processing unit. The system relies more heavily upon the GPU than the CPU for processing tasks – a contrast to the typical PC.
The team can also change the building blocks, swapping out a rear-projection screen for a front-projection screen. That way, they can change from tables to desks or walls more easily.
“It’s not a vertical application with custom-built hardware,” Keam says. “We are aware of other work out there that can do that. But we wanted to add value as a general platform that is practical and ready to go.”
The final technology is still expensive. The tables that ship in November will likely cost $5,000 to $10,000. That’s why Microsoft is relying upon a series of partners to bring them out in public places. T-Mobile will use the tables as retail kiosks in its cell phone stores. Starwood Hotels and Resorts, as well as Harrah’s Entertainment, will use the tables in its hotel chains as a virtual concierge desk or slot machines.
The business model assumes that the company will come out with high-end products for public places at first and then cost reduce it over three to five years until it becomes a viable consumer product for the home, Keam says.
The team has now grown to more than 100 employees headed by General Manager Pete Thompson. Looking back, Keam says one engineering lesson was that optical technology is still difficult to master and not as predictable.
“Every time you make an assumption or change an optical technology, it has to be thoroughly researched,” he says. “We didn’t have a lot of optical specialists on hand at first, so we had to do a lot by trial and error.”
Tweet This
7 
