Who hasn’t always dreamed of having a $2000 sculpture of their own head?  I know I have.  And finally, thatsmyface.com will make this dream come true.  With the upload of 2 photos of your face, one in front view and one in profile, thatsmyface.com will reconstruct a 3D model of your head in a semi-automatic fashion. This is a pretty neat attempt to bring a computer vision & 3D technology to the consumer market.

The technology behind the site is nothing new.  I can’t say for sure, but I would be willing to bet that thatsmyface.com is using FaceGen technology, which has been around for 10 years.

Similar?

Still, you don’t need cutting edge technology AND an exciting product to make a successful business; one of the two will usually do just fine.  Thatsmyface.com now just needs to convince people that what they’ve always wanted was random gear with their head as the main feature.  For the sake of entrepreneurship, good luck to them!

Today I came across a neat computer-vision based installation/game that was presented at the FILE 2008 expo in Brazil.  The company that put this together, Colmeia, is an interactive production company that creates digital installations for advertising.  The game involves using one’s silhouette, reprojected as dark blobs, to block out certain parts on a screen.

Tantalus Quest at FILE 2008 from eduardo omine on Vimeo.

We wrote a couple of weeks ago about Float4, a Montreal-based company also working on interactive display installations.  I am hopeful that what we are observing is the beginning of a trend, and that in the next couple of years we will see a rising number of computer vision applications in advertising.  This is certainly a trend that we will be following closely on ENLIGHTEN3D.

Spies rejoice!  Computer vision scientists at UC San Diego have devised a way to create a duplicate of a key using a only one digital photo, apparently taken in normal conditions.

By identifying a few “key” points (lame pun intended) in the photograph, the researchers are able to extract all the information required by a locksmith to make a copy of that key.  So, to all our loyal readers in the secret agent and espionage professions, send this report to your R&D department today and get this mildly amusing incredibly practical technology out of the labs!

Read the full article here.

The folks over at the Orkut blog announced a few days ago a new filter for their social network that can screen out “sexually suggestive phrases or images”.  However, no detail has been published (to my knowledge) on exactly how they filter out these images, whether they use text-based clues or somehow incorporate computer vision to detect the content of photos.  Until they respond to my e-mail I cannot say for sure, but my guess is that they are using only a text-based filter like the one used for Google Images SafeSearch.  If this is the case, I doubt it will be as effective on a social network where there are far fewer textual clues to put an image in context than there are on, say, a pornography web site.

Regardless of whether Orkut is using it or not, it is only a matter of time before computer vision techniques drive the next wave of “safe content” filtering.  As we discussed a few weeks ago, VideoSurf, the video search engine, has already introduced an image/video-based filter.  Once one major player in search adopts computer vision for content filtering, their competitors will have a strong incentive  to follow or risk being seen as the “unsafe” search engine.  Whether or not Google started the trend with Orkut this week, it’s only a matter of time until this application of computer vision hits the mainstream.

The New York Times, that is.  In what can only be perceived as the deepest form of flattery, last week the NY Times wrote a full length article on the “beautification algorithm”; the exact same paper that we wrote about 2 months ago.  Personally, I was excited to see a major mainstream publication taking an interest in computer vision, and I hope that our field continues to push into the thoughts of those who are not working directly in it.

Most folks that I encounter who aren’t working directly in computer vision or computer science are unfamiliar with the field entirely, and, more importantly, are unaware of the amazing things that can and are being done with computer vision today.  In my own experience, some of the best ideas come from those who are not the scientists directly doing the research, but who are working on something completely different yet see the potential of the research in their own, seemingly unrelated, application.  So, New York Times, show the potential of computer vision to the world!  And any time you need inspiration for an article, we will be flattered if you come check by ENLIGHTEN3D.

In the last month, Josh and I have had the opportunity to meet with several companies on the cutting edge of the computer vision and graphics field. First, we sat down at SIGGRAPH with the team leaders on Intel’s Larrabee project (the topic of a future post). More recently, we talked to Videosurf CTO Eitan Sharon, our pick for top company presenting at the TechCrunch50 conference two weeks ago. Last week, we got to chat with the founders of a local startup, Float4 Interactive, a Montreal start-up, who is turning image processing and computer vision techniques into an interactive art form.

Although Float4’s custom software technology works with cheap cameras (e.g. eyeToy) and regular screens, the effects are most impressive when displayed life-size using a back-projected display and two high-performance cameras. They provide turn-key solutions and even rent the hardware for special events, such as extravagant wedding ceremonies, industry expositions, and advertising installations.

Applications and clients.

Other companies experiment similar technologies — I remember seeing some at SIGGRAPH – but Float4 has raised the bar in interactivity, robustness and aesthetic quality.

Here are a couple examples of the applications of their current technology:

• Move your body to create unique animations (such as juggling a soccer ball)
• Experience the joy of flying like an airplane or a bird, by shifting or waving your arms.

Already, the company has attracted attention from notable clients such as the Cirque du Soleil. Despite our best efforts to pry it out of them, the Float4 folks are staying quiet on exactly what kind of display they’re building for the Cirque. Personally, I’m excited to see how this state-of-the-art graphical technology gets integrated into what is already a visually astounding performance.

Visit Float4 Interactive

With 50 startups launching at the TechCrunch50 conference last week, the blogosphere has been abuzz with high tech news. In all that media frenzy, one company stood out and impressed us more than the rest. Videosurf, a video search engine, combines traditional text metadata search with computer vision to provide higher quality search results. After seeing a video of their presentation at TC50, I wanted to learn more, so I arranged an interview with Videosurf CTO and computer vision expert Dr. Eitan Sharon. The following article is based on our discussion.

How Videosurf Works

Videosurf starts by looking through textual tags and dividing video into shots. Unlike other video search engines though, Videosurf goes much further in its analysis. A facial recognition system goes through the films frame-by-frame, collects faces that look alike in different shots, and tries to match them against the appearance of known actors using the cast information. In the beginning, an operator had to assist the system, but now, most of the common actors are easily recognized automatically.

Here’s an example. I put in a search for “Star Trek”. At the top of the page, I am presented with a thumbnail of people that are associated with my search. In this case the list includes William Shatner, Jeri Ryan, Leonard Nimoy, and Patrick Stewart, all actors who have played major characters in a Star Trek TV series. By clicking on one of the thumbnails (Patrick Stewart), I can find all of the Star Trek-related clips in which Patrick Stewart appears.

From the screenshots, you may have noticed that next to each clip there appears to be a storyboard. Appearing simple at first glance, these “video summaries” actually employ a complex computer vision approach in trying to automatically determine the most “important” frames in a clip. According to Dr. Sharon, creating these visual summaries involves a combination of techniques such as determining the uniqueness or similarity of objects across frames, the depth of field, motion in the scenes, etc. If these automatic storyboards work effectively, and from my own testing they seem to do a good job, this should save users time so that they don’t have to wait for a video to finish downloading just to realize that it is not what they were looking for.

Other Features

Perhaps the feature with the most immediate profit potential is their adult content filter. Though we did not get too far into the details, they have employed a type of “safe search” filter that adds computer vision techniques to traditional text search to determine whether content should be filtered. Like Google’s Safe Search, this filter can be toggled on or off. If I was Yahoo’s chief marketing officer, I would immediately license this technology exclusively for a few years. Then I would run an ad campaign about how our search engine is the only safe place for children on the internet. But that’s just me.

One other neat feature lets you share a specific segment from a video, increasing the viral potential of online video beyond where it already is today.

Will it take off?

Saying “Will I use this?” is not really the most unbiased kind of market analysis, but I can say that I will use this (free!) service now. I already have, actually; having “Videosurfed” 3 times in the past 24 hours since receiving my beta invite, and I am confident that I will keep coming back if it keeps giving me quality results.

From a profitability perspective, Videosurf may have an ace in the hole that they haven’t talked about yet: copyright detection. After Google (owner of YouTube) was sued for one billion dollars by Viacom last, it is safe to bet that they and others are interested in being able to better detect whether the material on their sites is copyrighted. Computer vision has finally come to online video search, and we can safely say that it is here to stay.

You can find a list of Dr. Sharon’s computer vision publications on his personal web site: http://www.dam.brown.edu/people/eitans/

If creating 3D models was as easy as taking photos, it is safe to say that the use of 3D would be far more widespread than it is today.  From e-commerce to virtual tourism to casual games, reducing the cost and complexity of creating 3D models would have a widespread effect on multiple industries.

Feeling Software is making that possible.  Over the last 2 years, we have worked to develop a technology that allows anyone to create 3D models with little effort and no training.  Our goal: simplicity.  You take a bunch of photos with a regular camera from any angle you please, and we automatically create a 3D model.  The demo video below discusses our project in detail.

Feeling Software Demo from joshk on Vimeo.

We have thought of a variety ways that this technology can be applied to solve problems for consumers.  For our readers, imagine that you could take photos of an object or scene, press a button, and instantly have a high-quality 3D model of that object or scene.  If this technology were available today, how would you use it?

After months of development, Presto3D left closed beta last week and has opened to the public.  If you missed the post a few weeks ago, Presto3D is a 3D model marketplace that automatically creates 3D previews of the content that is submitted.

The reception has been positive, and we have gotten press coverage on several major animation and game development sites.  If you haven’t taken a look at the site yet, come check out what we’ve been up to for the last few months.  If you already visited the site during the closed beta, it’s worth going back just to see the fullscreen 3D previews that we’ve added in the latest release.

All feedback is appreciated!

You tag this photo:

Source: www.bt.dk

Polar Rose finds this photo:

Source: www.newprophecy.net

With facial recognition in Picasa Web Albums launched yesterday, an exciting computer vision application once again bumps heads against privacy concerns.  On the one hand, automatic tagging of photos through facial recognition can be a useful time saver; if I have an album of 100 pictures of my family vacation with the same 5 people, I would much rather have my computer tag them for me than having to sift through them one-by-one adding tags.  On the other hand, I might not necessarily want all photos of me to be so easily found by anyone.

Picasa Web Albums seems to be OK in how it deals with this issue, at least for now.  Users only tag their own albums, and, as far as I can tell, the information gathered is not used to search Google Imagesand automatically tag images of the people you tag in your web albums.

This is not the case with every player stepping into this industry.  Polar Rose, announced late-2006, uses facial recognition on user-tagged photos to search for more photos of an individual in any publicly available images.  The service is designed as a browser plug-in and an embeddable widget for photo-sharing sites, and rumour has it that a partnership with one of the major sites is imminent.  Users tag photos of people that they find anywhere online, and Polar Rose uses that information to find that same face across other images.

The example at the beginning of this post should illustrate why this may be a concern.  For Paris Hilton, perhaps her image benefits when scandalous pictures surface, but this is not the case for most of us.  Should photos of people really be that searchable?

In reality, though, the point is moot.  Using Google Image search, I probably could have found the same 2 pictures shown above; with facial recognition, this just becomes more efficient.  If you have been using Facebook for the past few years, you have probably already come to terms with the fact that people can quickly find many pictures of you, including ones that others took without your consent (though in fairness to Facebook, you can untag and render unsearchable pictures you don’t like).

Conclusion: from a computer vision standpoint, it is neat to see these technologies reach the mass market.  From a privacy outlook, more (visual) information about us is going to become accessible without our direct consent, but only information that was publicly available in the first place, and this development is probably inevitable.

On a final note, there is one feature proposed by Polar Rose which I can’t help but find far more creepy than useful: personal photo RSS feeds.  Basically, get instantly updated by RSS each time a new photo of a targeted person is found.  Stalkers rejoice!