According to Israeli newspaper Haaretz, Microsoft is set to buy 3DV, the developer of a markerless motion capture technology called ZCam. Their system, which is something of a next-generation Playstation Eyetoy, should open up new doors for future game development on the Microsoft Xbox console line. If it works as well as advertised, the possibilities are endless for new styles of game controls, but without having tried it out myself, I can’t say whether this is going to revolutionize gaming or be a neat add-on like the Eyetoy was for Sony.
According to the CrunchGear blog, the company had actually raised $38.6 million over the past 12 years, so despite the large dollar number, it’s hard to judge whether this will be viewed as a big win for 3DV or not.
In a demo presented at the TED conference last week, an MIT research group gave us a flavor of what the future of augmented reality might be. Basically, their system involved a wearable projector, camera, and computer. Using computer vision, the camera would identify what was in front of you, then project information on to this object or person. Have a look at the video below for some neat augmented reality applications that we might see in the future.
Don’t expect this technology to hit the mass market next week; yet, what’s exciting to see is that researchers today are not only thinking of useful applications of augmented reality, but that they are then able to actually build and demonstrate them, even if the conditions must be controlled. Visually augmented reality is already in the mass market in the form of in-vehicle GPS (amongst others), and as technology improves to be able to serve us more and more relevant information with fewer and fewer manually given clues, this field will only grow.
Personally, I doubt that carebear-style “light streaming from tummy” design will take off; with improvements in displays and optics, information-empowered glasses or contact lenses would be far less obtrusive. Regardless of how this technology eventually gains market acceptance, kudos to professor Pattie Maes and student researcher Pranav Mistry for an inspiring technology demonstration.
Here are a few screenshots of a project that we have been working on at Feeling Software. From aerial photographs, this proof-of-concept shows how we can automatically build a 3D city model using our Feeling Constructive computer vision technology.
This was produced only using data that we were able to obtain internally. Better imagery, along with GPS position information, can be used to create 3D city models of much greater quality.
In a second proof-of-concept, we use 5 ground-level photos taken with a standard digital camera to create a near photo-realistic 3D model of one of the buildings nearby to our office in Montreal. This is done in a semi-automatic way, such that no 3D modeling experience is necessary to create the high quality 3D model.
Below, you can see how the City of Montreal uses the Feeling Engine GIS Edition as a visualization and planning tool for the city’s GIS department. Our office is in the red building!
Today, local governements today are creating impressive 3D representations of their cities, both for internal planning purposes and as external marketing materials. As the need for tools to create 3D cities rises across the globe, Feeling Software is looking for partners to bring these cutting-edge computer vision technologies to market.
One of my favorite blogs on the web is PetitInvention by a Tokyo-based concept designer named Mac Funamizu. A few months ago, I posted how one of his concepts, the “Future of Mobile Search“, should be an inspiration to all computer vision scientists.
It seems like the folks at Google Earth find Mac’s work inspiring too. Now, I have no way of knowing for sure that this was the source of their inspiration or whether it is just a happy coincidence, but have a look at the screenshots below and be the judge.
(above) Google Earth 5, released February 2nd, 2009
Future of Mobile Search - Search Beyond Time, posted July 9, 2008
Both Mac’s concept and Google Earth’s new release demonstrate a slider to look at imagery of the same scene over time. It’s still a ways away from the location and orientation-matched mobile design, but nonetheless, it is impressive to see one of Mac’s more imaginative concepts already appearing in a real-world product. We need more designers like Mac inspiring us with their ideas. Does anyone else have a favorite design site that they use for inspiration?
2009 promises to be a formative year for 3D and Computer Vision. 3D films are getting all the press at the moment (see this excellent blog on 3D Movies), but this year should bring us much more than just a new dimension from Hollywood.
Just today, Apple announced at MacWorld that its new iPhoto program would have facial recognition built in, following Picasa’s lead of a few months back, and bringing yet another computer vision application to the masses. Expect visual search to keep blossoming in 2009, with uses you may not have considered before.
In the next weeks and months, I will have many more posts on the innovative technologies that are going to shape our fields for 2009 and the years to come. From computer vision research analysis to 3D film reviews to chipset technology, there is so much to write about, I’m having trouble containing myself and not going on for a few thousand words right here.
To begin an exciting 2009, I’ll be heading to CES later this week, so if you want to meet me there for a chat or a beer, just send me an e-mail. Alternatively, you can look for me as I wander around; I’ll be the guy standing in front of the 3D displays, drooling.
Usability and time-to-learn are two big concerns with any web application. For 3D applications, both of these factors have led to the downfall of some pretty innovative sites.
In the video above, I demonstrate how we use the Constructive technology to refurnish our own conference room in 3D without having to learn any new or foreign process. With a few points tagged in two pictures by an operator, Constructive understands the camera position, orientation, and focal length, allowing me to move around the 3D furniture along the floor with the correct perspective. And since Constructive understands the relationship between the photos, moving a piece of furniture (or a French maid, as the case may be) in one photo will also move it with the correct perspective in any and all others.
This application also addresses the issue of scalability; by not requiring any highly trained or skilled work to be done, the simple manual tagging process can be cheaply outsourced or put “in the cloud” using Amazon’s Mechanical Turk.
I look forward to seeing innovative retailers using Computer Vision technology like Constructive to add a level personalization that doesn’t exist today. Soon, we won’t have to try to imagine what our new room will look like; you will be able to perfect it before ever putting any money down, and even share your ideas with friends and family to get their thoughts too. A mother knows best, right?
Unless you are Haley Joel Osment, odds are you possess 5 senses. On this blog, we are focused on how computers can interpret one of those senses, vision. It’s the field where we have our technical expertise, and where we believe we can make our biggest contribution to technological advancement (see our current R&D project to turn photos to 3D models). However, in the same way that computer vision scientists are trying to help machines understand what they see, the researchers over at mufin are trying to teach computers how to hear.
mufin is an automated music recommendation system that takes a different, one could say more technical, approach to helping you find music that you like. Services like Pandora and iTunes Genius use human expertise or song meta-data where mufin actually analyzes the audio content. This is from the mufin website:
How does mufin work?
mufin knows the musical essence of millions of songs and connects those songs that have a similar essence. This essence consists of sound properties like tempo, instruments, sound density or harmony. Whether the music is well-known or not, which genre it belongs to, when it was released or where in the world it was made, plays no role when you discover music using mufin. What matters is the sound!
Much like those of us in Computer Vision, Computer Audition scientists sit at the meeting point between art and mathematics. As effectively as Computer Audition algorithms can objectively break down music into a series of variables, it requires a subjective human to determine which combination of those variables implies a “similar” song. It is this subjectivity that keeps both of our fields fascinating.
Though the early reviews of this site seem to be not entirely positive, the folks over at mufin already have my respect for taking a stab at a truly complex problem. Whether mufin succeeds or not, I expect to see Computer Audition techniques applied in other music recommendation services in the future, though undoubtedly in combination with other, more manual, techniques rather than in isolation.
