ENLIGHTEN3D

By Joshua Koopferstock

Back in the hotel after day 1 at CES, and I am in awe.  This show is gigantic.  I’m pretty sure that if it was 3 weeks long, I still wouldn’t have seen all of the booths.  From a 3D industry perspective, I noticed one thing of interest today while walking the floor.  People are going crazy for 3D displays.  Absolutely crazy.

Vegas, baby!

Around nVidia’s booth where they were showing off a 3D gaming computer rig (and selling 3D shutter glasses for $199; I was tempted…), there was such a crowd of people I couldn’t even shove my way through the booth, and almost got bowled over in the process of trying.  At the DALITE booth, where they were promoting their projection screen that preserves polarization, I had to wait in line for a demo.  Of a projection screen company!  People (well, CES technology nerds, anyhow) are eating up the 3D home entertainment technology, and this is despite the fact that in many cases, it is not quite ready for prime-time.  The DALITE demo was neat, but I felt like I was looking at it cross-eyed for half the time (not the fault of the screen they were selling, but still).

A view of the show floor

I’m not sure if it’s a failing of the technology, or a lack of experience in using the tools in these nascent stages, but there is definite room for improvement.  The polarized glasses seem to especially require controlled viewing conditions (i.e. viewer’s position relative to the display) to work effectively.

That’s it for the quick day 1 report.  I’ll try to take more relevant pictures tomorrow; I spent most of the day with my jaw hanging open, and forgot to take out the camera for the blog.

By Joshua Koopferstock

2009 promises to be a formative year for 3D and Computer Vision. 3D films are getting all the press at the moment (see this excellent blog on 3D Movies), but this year should bring us much more than just a new dimension from Hollywood.

Just today, Apple announced at MacWorld that its new iPhoto program would have facial recognition built in, following Picasa’s lead of a few months back, and bringing yet another computer vision application to the masses.  Expect visual search to keep blossoming in 2009, with uses you may not have considered before.

In the next weeks and months, I will have many more posts on the innovative technologies that are going to shape our fields for 2009 and the years to come.  From computer vision research analysis to 3D film reviews to chipset technology, there is so much to write about, I’m having trouble containing myself and not going on for a few thousand words right here.

To begin an exciting 2009, I’ll be heading to CES later this week, so if you want to meet me there for a chat or a beer, just send me an e-mail.  Alternatively, you can look for me as I wander around; I’ll be the guy standing in front of the 3D displays, drooling.

Happy New Year to all of you!

by Joshua Koopferstock

Constructive 3D Furniture Demo from joshk on Vimeo.

Usability and time-to-learn are two big concerns with any web application.  For 3D applications, both of these factors have led to the downfall of some pretty innovative sites.

In the video above, I demonstrate how we use the Constructive technology to refurnish our own conference room in 3D without having to learn any new or foreign process. With a few points tagged in two pictures by an operator, Constructive understands the camera position, orientation, and focal length, allowing me to move around the 3D furniture along the floor with the correct perspective.  And since Constructive understands the relationship between the photos, moving a piece of furniture (or a French maid, as the case may be) in one photo will also move it with the correct perspective in any and all others.

This application also addresses the issue of scalability; by not requiring any highly trained or skilled work to be done, the simple manual tagging process can be cheaply outsourced or put “in the cloud” using Amazon’s Mechanical Turk.

I look forward to seeing innovative retailers using Computer Vision technology like Constructive to add a level personalization that doesn’t exist today.  Soon, we won’t have to try to imagine what our new room will look like; you will be able to perfect it before ever putting any money down, and even share your ideas with friends and family to get their thoughts too.  A mother knows best, right?

All 3D models used come from www.Presto3D.com

By Christian Laforte

In the last month, Josh and I have had the opportunity to meet with several companies on the cutting edge of the computer vision and graphics field. First, we sat down at SIGGRAPH with the team leaders on Intel’s Larrabee project (the topic of a future post). More recently, we talked to Videosurf CTO Eitan Sharon, our pick for top company presenting at the TechCrunch50 conference two weeks ago. Last week, we got to chat with the founders of a local startup, Float4 Interactive, a Montreal start-up, who is turning image processing and computer vision techniques into an interactive art form.

Although Float4’s custom software technology works with cheap cameras (e.g. eyeToy) and regular screens, the effects are most impressive when displayed life-size using a back-projected display and two high-performance cameras. They provide turn-key solutions and even rent the hardware for special events, such as extravagant wedding ceremonies, industry expositions, and advertising installations.

Applications and clients.

Other companies experiment similar technologies — I remember seeing some at SIGGRAPH – but Float4 has raised the bar in interactivity, robustness and aesthetic quality.

Here are a couple examples of the applications of their current technology:

• Move your body to create unique animations (such as juggling a soccer ball)
• Experience the joy of flying like an airplane or a bird, by shifting or waving your arms.

Already, the company has attracted attention from notable clients such as the Cirque du Soleil. Despite our best efforts to pry it out of them, the Float4 folks are staying quiet on exactly what kind of display they’re building for the Cirque. Personally, I’m excited to see how this state-of-the-art graphical technology gets integrated into what is already a visually astounding performance.

Visit Float4 Interactive

By Joshua Koopferstock

If creating 3D models was as easy as taking photos, it is safe to say that the use of 3D would be far more widespread than it is today.  From e-commerce to virtual tourism to casual games, reducing the cost and complexity of creating 3D models would have a widespread effect on multiple industries.

Feeling Software is making that possible.  Over the last 2 years, we have worked to develop a technology that allows anyone to create 3D models with little effort and no training.  Our goal: simplicity.  You take a bunch of photos with a regular camera from any angle you please, and we automatically create a 3D model.  The demo video below discusses our project in detail.

Feeling Software Demo from joshk on Vimeo.

We have thought of a variety ways that this technology can be applied to solve problems for consumers.  For our readers, imagine that you could take photos of an object or scene, press a button, and instantly have a high-quality 3D model of that object or scene.  If this technology were available today, how would you use it?

By Joshua Koopferstock

You tag this photo:

Source: www.bt.dk

Polar Rose finds this photo:

Source: www.newprophecy.net

With facial recognition in Picasa Web Albums launched yesterday, an exciting computer vision application once again bumps heads against privacy concerns.  On the one hand, automatic tagging of photos through facial recognition can be a useful time saver; if I have an album of 100 pictures of my family vacation with the same 5 people, I would much rather have my computer tag them for me than having to sift through them one-by-one adding tags.  On the other hand, I might not necessarily want all photos of me to be so easily found by anyone.

Picasa Web Albums seems to be OK in how it deals with this issue, at least for now.  Users only tag their own albums, and, as far as I can tell, the information gathered is not used to search Google Imagesand automatically tag images of the people you tag in your web albums.

This is not the case with every player stepping into this industry.  Polar Rose, announced late-2006, uses facial recognition on user-tagged photos to search for more photos of an individual in any publicly available images.  The service is designed as a browser plug-in and an embeddable widget for photo-sharing sites, and rumour has it that a partnership with one of the major sites is imminent.  Users tag photos of people that they find anywhere online, and Polar Rose uses that information to find that same face across other images.

The example at the beginning of this post should illustrate why this may be a concern.  For Paris Hilton, perhaps her image benefits when scandalous pictures surface, but this is not the case for most of us.  Should photos of people really be that searchable?

In reality, though, the point is moot.  Using Google Image search, I probably could have found the same 2 pictures shown above; with facial recognition, this just becomes more efficient.  If you have been using Facebook for the past few years, you have probably already come to terms with the fact that people can quickly find many pictures of you, including ones that others took without your consent (though in fairness to Facebook, you can untag and render unsearchable pictures you don’t like).

Conclusion: from a computer vision standpoint, it is neat to see these technologies reach the mass market.  From a privacy outlook, more (visual) information about us is going to become accessible without our direct consent, but only information that was publicly available in the first place, and this development is probably inevitable.

On a final note, there is one feature proposed by Polar Rose which I can’t help but find far more creepy than useful: personal photo RSS feeds.  Basically, get instantly updated by RSS each time a new photo of a targeted person is found.  Stalkers rejoice!

By Joshua Koopferstock

As if getting the required precision for surgery wasn’t hard enough in person, surgeons can now perform surgery remotely from thousands of miles away.  While it is far from mainstream, robotic surgery, including remote robotic surgery, has made leaps and bounds in the past decade.  At SIGGRAPH, I came across a technology that might give robotic surgery another shove forward.

Butterfly Haptics, launched earlier this year as a spinoff company from Carnegie Mellon University, is hard at work trying to commercialize a magnetic levitation haptic device (pictured below).  The grey handle in the center of the device floats in a magnetic field.

What Sets it Apart

The device allows 6 degrees of freedom (translation in any direction, rotation in any direction) like many haptic devices.  However, the Butterfly Haptic device separates itself from the rest in two ways.

First, despite the fact the the device is floating in a magnetic field, it can still very effectively stop your motion completely.  In one demo, you could control objects in 3D space; when you ran the object into a wall, the feeling of hitting a solid object was extremely convincing.

Second, no static friction is present as the device is floating in a magnetic field and is not mechanical.  The surface texture demo illustrating this property sold me completely to the benefits of maglev haptics.  In this demo, you were presented on screen with different surface textures that you could run over with the device, such as a solid wavy surface, a tiny ridged surface which felt something like running your fingernail over the the paper edge of a closed book and, most impressively, the “ice” (frictionless surface).  Pushing down on the surface, it was completely solid, but as you move along it in the other two dimensions, it feels absolutely frictionless like perfect ice.

Butterfly Haptics Device

Back to Surgery

It does not require explaining that in surgery, having maximum freedom of movement and realistic force feedback is optimal, if not necessary.  And it is on these two fronts that Butterfly Haptics excels.  Not only would this technology be beneficial in remote robotic surgery, but also for surgical training simulations.  With immersive 3D displays like those used in currently available robotic surgery devices and realistic force feedback, surgeons-in-training can perform highly realistic surgeries on “humans” (anatomically correct 3D models) without ever making a true incision!

The Future for Butterfly Haptics

While the maglev haptic device is currently more academic than commercial, the fact that Butterfly Haptics has been spun out of academia into the business world suggests to me that these devices may find exciting real-world applications in the near future.  What exactly those applications may be are uncertain, but the company suggests on their site that beyond medicine, the devices may be used for CAD applications, data visualization, and character animation.  The medical applications appear most promising to me, but in any case, this is a company and technology well worth keeping an eye on in the next few years.

This is only one of the many interesting technologies and research papers that we came across during SIGGRAPH last week. Expect to find more blog posts about what we saw at SIGGRAPH in the upcoming days now that I am back in beautiful Montreal.