Who hasn’t always dreamed of having a $2000 sculpture of their own head? I know I have. And finally, thatsmyface.com will make this dream come true. With the upload of 2 photos of your face, one in front view and one in profile, thatsmyface.com will reconstruct a 3D model of your head in a semi-automatic fashion. This is a pretty neat attempt to bring a computer vision & 3D technology to the consumer market.
The technology behind the site is nothing new. I can’t say for sure, but I would be willing to bet that thatsmyface.com is using FaceGen technology, which has been around for 10 years.
Similar?
Still, you don’t need cutting edge technology AND an exciting product to make a successful business; one of the two will usually do just fine. Thatsmyface.com now just needs to convince people that what they’ve always wanted was random gear with their head as the main feature. For the sake of entrepreneurship, good luck to them!
In the last month, Josh and I have had the opportunity to meet with several companies on the cutting edge of the computer vision and graphics field. First, we sat down at SIGGRAPH with the team leaders on Intel’s Larrabee project (the topic of a future post). More recently, we talked to Videosurf CTO Eitan Sharon, our pick for top company presenting at the TechCrunch50 conference two weeks ago. Last week, we got to chat with the founders of a local startup, Float4 Interactive, a Montreal start-up, who is turning image processing and computer vision techniques into an interactive art form.
Although Float4’s custom software technology works with cheap cameras (e.g. eyeToy) and regular screens, the effects are most impressive when displayed life-size using a back-projected display and two high-performance cameras. They provide turn-key solutions and even rent the hardware for special events, such as extravagant wedding ceremonies, industry expositions, and advertising installations.
Applications and clients.
Other companies experiment similar technologies — I remember seeing some at SIGGRAPH – but Float4 has raised the bar in interactivity, robustness and aesthetic quality.
Here are a couple examples of the applications of their current technology:
Move your body to create unique animations (such as juggling a soccer ball)
Experience the joy of flying like an airplane or a bird, by shifting or waving your arms.
Already, the company has attracted attention from notable clients such as the Cirque du Soleil. Despite our best efforts to pry it out of them, the Float4 folks are staying quiet on exactly what kind of display they’re building for the Cirque. Personally, I’m excited to see how this state-of-the-art graphical technology gets integrated into what is already a visually astounding performance.
If creating 3D models was as easy as taking photos, it is safe to say that the use of 3D would be far more widespread than it is today. From e-commerce to virtual tourism to casual games, reducing the cost and complexity of creating 3D models would have a widespread effect on multiple industries.
Feeling Software is making that possible. Over the last 2 years, we have worked to develop a technology that allows anyone to create 3D models with little effort and no training. Our goal: simplicity. You take a bunch of photos with a regular camera from any angle you please, and we automatically create a 3D model. The demo video below discusses our project in detail.
We have thought of a variety ways that this technology can be applied to solve problems for consumers. For our readers, imagine that you could take photos of an object or scene, press a button, and instantly have a high-quality 3D model of that object or scene. If this technology were available today, how would you use it?
After months of development, Presto3D left closed beta last week and has opened to the public. If you missed the post a few weeks ago, Presto3D is a 3D model marketplace that automatically creates 3D previews of the content that is submitted.
The reception has been positive, and we have gotten press coverage on several major animation and game development sites. If you haven’t taken a look at the site yet, come check out what we’ve been up to for the last few months. If you already visited the site during the closed beta, it’s worth going back just to see the fullscreen 3D previews that we’ve added in the latest release.
With facial recognition in Picasa Web Albumslaunched yesterday, an exciting computer vision application once again bumps heads against privacy concerns. On the one hand, automatic tagging of photos through facial recognition can be a useful time saver; if I have an album of 100 pictures of my family vacation with the same 5 people, I would much rather have my computer tag them for me than having to sift through them one-by-one adding tags. On the other hand, I might not necessarily want all photos of me to be so easily found by anyone.
Picasa Web Albums seems to be OK in how it deals with this issue, at least for now. Users only tag their own albums, and, as far as I can tell, the information gathered is not used to search Google Imagesand automatically tag images of the people you tag in your web albums.
This is not the case with every player stepping into this industry. Polar Rose, announced late-2006, uses facial recognition on user-tagged photos to search for more photos of an individual in any publicly available images. The service is designed as a browser plug-in and an embeddable widget for photo-sharing sites, and rumour has it that a partnership with one of the major sites is imminent. Users tag photos of people that they find anywhere online, and Polar Rose uses that information to find that same face across other images.
The example at the beginning of this post should illustrate why this may be a concern. For Paris Hilton, perhaps her image benefits when scandalous pictures surface, but this is not the case for most of us. Should photos of people really be that searchable?
In reality, though, the point is moot. Using Google Image search, I probably could have found the same 2 pictures shown above; with facial recognition, this just becomes more efficient. If you have been using Facebook for the past few years, you have probably already come to terms with the fact that people can quickly find many pictures of you, including ones that others took without your consent (though in fairness to Facebook, you can untag and render unsearchable pictures you don’t like).
Conclusion: from a computer vision standpoint, it is neat to see these technologies reach the mass market. From a privacy outlook, more (visual) information about us is going to become accessible without our direct consent, but only information that was publicly available in the first place, and this development is probably inevitable.
On a final note, there is one feature proposed by Polar Rose which I can’t help but find far more creepy than useful: personal photo RSS feeds. Basically, get instantly updated by RSS each time a new photo of a targeted person is found. Stalkers rejoice!
These product concept designs by Mac Funamizu, a Japanese graphic designer, are among the most amazing applications of computer vision technology that I have come across. What I found especially inspiring is that some of the technology that we are working on at Feeling Software will be key to making a concept such as this a reality. And without too much effort, I believe that just about everyone reading this blog can see how their own work in 3D or computer vision will be a necessary building block to make this possible. I hope Mr. Funamizu’s work concept fires the imagination of many people regarding the possibilities and usefulness of augmented reality.
“Future of Internet Search: Mobile Version” Product Concept
This Photosynth-esque approach (above) shows you other photos of the same scene you are looking at, from the same angle. Here, the designer demonstrates looking at the scene in front of you over time through historical photographs.
Image recognition + mobile internet + Wikipedia?
Text recognition + mobile internet + babel fish? I wish I had that when I was trying to decipher menus in Slovakia!
See more possibilities for this concept at Mac Funamizu’s blog, petitinvention.
As if getting the required precision for surgery wasn’t hard enough in person, surgeons can now perform surgery remotely from thousands of miles away. While it is far from mainstream, robotic surgery, including remote robotic surgery, has made leaps and bounds in the past decade. At SIGGRAPH, I came across a technology that might give robotic surgery another shove forward.
Butterfly Haptics, launched earlier this year as a spinoff company from Carnegie Mellon University, is hard at work trying to commercialize a magnetic levitation haptic device (pictured below). The grey handle in the center of the device floats in a magnetic field.
What Sets it Apart
The device allows 6 degrees of freedom (translation in any direction, rotation in any direction) like many haptic devices. However, the Butterfly Haptic device separates itself from the rest in two ways.
First, despite the fact the the device is floating in a magnetic field, it can still very effectively stop your motion completely. In one demo, you could control objects in 3D space; when you ran the object into a wall, the feeling of hitting a solid object was extremely convincing.
Second, no static friction is present as the device is floating in a magnetic field and is not mechanical. The surface texture demo illustrating this property sold me completely to the benefits of maglev haptics. In this demo, you were presented on screen with different surface textures that you could run over with the device, such as a solid wavy surface, a tiny ridged surface which felt something like running your fingernail over the the paper edge of a closed book and, most impressively, the “ice” (frictionless surface). Pushing down on the surface, it was completely solid, but as you move along it in the other two dimensions, it feels absolutely frictionless like perfect ice.
Butterfly Haptics Device
Back to Surgery
It does not require explaining that in surgery, having maximum freedom of movement and realistic force feedback is optimal, if not necessary. And it is on these two fronts that Butterfly Haptics excels. Not only would this technology be beneficial in remote robotic surgery, but also for surgical training simulations. With immersive 3D displays like those used in currently available robotic surgery devices and realistic force feedback, surgeons-in-training can perform highly realistic surgeries on “humans” (anatomically correct 3D models) without ever making a true incision!
The Future for Butterfly Haptics
While the maglev haptic device is currently more academic than commercial, the fact that Butterfly Haptics has been spun out of academia into the business world suggests to me that these devices may find exciting real-world applications in the near future. What exactly those applications may be are uncertain, but the company suggests on their site that beyond medicine, the devices may be used for CAD applications, data visualization, and character animation. The medical applications appear most promising to me, but in any case, this is a company and technology well worth keeping an eye on in the next few years.
This is only one of the many interesting technologies and research papers that we came across during SIGGRAPH last week. Expect to find more blog posts about what we saw at SIGGRAPH in the upcoming days now that I am back in beautiful Montreal.
For the last few months, we have been quietly working on Presto3D, a 3D model marketplace which integrates our 3D web viewer to display user-generated 3D models in 3D within the browser. Finally, it is out the door and in closed beta! If you want to check it out, go to www.presto3d.com and enter this beta referral key: “presto0845″ (without quotes).
Presto3D is a marketplace where 3D models can be bought or sold. All 3D assets are user-generated and user-priced. When a model is uploaded, we convert the 3D model into COLLADA to create a 3D preview for our web viewer. With a small, one-time plugin download, potential buyers can see the models in 3D, rotate and zoom the models within the browser.
To ensure an optimal performance and to keep the models safe from petty thefts, we automatically reduce the resolution of textures, compress and encrypt all the data.
Why is Presto3D so exciting?
To the best of our knowledge, there exists nothing on the web that allows such openness for the display of user-generated 3D content. Due to our automatic conversion, on Presto3D, users can upload files in any Maya (.ma, .mb) or 3dsMax (.max, .3ds) format and see them in 3D in the browser (.dae, .obj, & .fbx are also supported). Even outside of 3D marketplace websites, other sites will require that you use file formats specially created for the specific web viewer, or create the files within their proprietary platform (such as within some web games and virtual worlds).
The goals of Presto3D are two-fold. First, we aim to drastically improve the experience of buying and selling 3D content. Second, we will create the most direct path to display 3D content online, irregardless of the software used to create it.
Go give Presto3D a try, and tell us what you think!
For you technology lovers who are still kids at heart, Disneyland has recently opened up the Innoventions Dream Home, showcasing cool high tech integration in a futuristic home. What caught my attention was one invention called the Magic Mirror. Getting its name from the mirror in Snow White, this Magic Mirror does not tell you “who is the fairest of them all” (for that you’ll still need HotOrNot.com). What it does do is allow you to virtually try on clothes in your wardrobe. In fact, the Magic Mirror is not a mirror at all; it is a large display monitor with a video camera next to it.
Trying on a dress in the Magic Mirror. Photo: cepro.com
While the concept is neat and would probably be even more useful in the department store dressing room than the bedroom, by the looks of it in the video below, the concept is still far from the realism necessary for a technology like this to take off. A few years back, it was thought that by today, virtual clothes shopping would be mainstream, and companies like My Virtual Model had signed contracts with major apparel retailers to integrate their technology into online stores.
It turns out that the technology wasn’t ready, and by the looks of this Magic Mirror, it still has a long way to go.
Here’s how I think they do it:
The dress moves roughly according to the orientation of the head, so they are most likely using a simple real-time head tracker and applying the pose of the head to the top of the dress. The bottom seems to be animated randomly, or maybe through secondary animation.
Later this week I’ll post on a face tracking algorithm that could make this easily possible.
How could we do it better?
One imperfection is very noticeable: the dress doesn’t follow the shoulders and hips properly. Part of it may be anatomical (this is a guy after all), but I think this problem should be easily solved by tracking the silhouette (using background subtraction) and identifying the shoulders and hips using simple heuristics, e.g. areas of low curvature and roughly horizontal or vertical slopes. This would immediately improve the realism of this solution.
Another improvement would be to track features on the user’s T-shirt, so we can have a better estimate of the body pose, its size and maybe even the person’s sex. I’d start my search with Automatic Non-Rigid 3D Modeling from Video (Torresani and Hertzmann, 2004), since I remember being impressed with the results way back then: it handles occlusions and variations in illumination very nicely. In the picture below, you can see one of the researchers moving his hands in front of his T-shirt… the algorithm can still capture a 3D representation of the deforming T-shirt. Doing this in real-time may be challenging, but fast GPUs and multi-core systems should make it possible.
Still a Long Way to Go
With the method that we have suggested, there is one major sticking point that we have not addressed: content creation. Assuming you have the ability to accurately track a person and render the image in real time, you still need a way to create the clothes in 3D. This is not a simple task, as clothes can be highly variable in elasticity, reflectiveness, etc., which would make automation of the modeling process complex.
Before we see these Magic Mirrors in department stores like Sears or Macy’s that have hundreds of thousands of different apparel items each year, a method for automatic creation of clothing content will have to be developed. And while automatic 3D content creation is going to take great strides in the next couple of years, the quality of 3D reconstruction needed for clothing is still a long way off.
I posted a little while back about car-maker Renault teaming up with Holografika to create 3D displays in their vehicles, but based on the information provided, we were only left to guess what they might be doing with this technology.
Today I came across a very similar story from “way back” in March about Daihatsu (majority-owned by Toyota) working with ProVision, another 3D holographic display developer and vendor. Happily, Daihatsu is providing us with a bit more information as to what exactly they will use a 3D display for. According to the article from Reuters:
“The traditional 2D flat screen acts as a traditional dashboard, displaying gauges such as the speedometer and tachometer. The 3D holographic screen presents warnings and vehicle information as easy-to-understand 3D images to facilitate driver awareness and recognition. These features will be integrated as part of Daihatsu’s OPCS (Omni-directional Pre-Crash Safety Support System) to create the next generation of digital dashboards.”
Unfortunately it looks like we’re going to have to wait until 2012 before we actually see these things in anything besides concept cars, but it’s still interesting to have a preview of what the future of 3D displays might be.
