ENLIGHTEN3D

By Joshua Koopferstock

Being able to see through haze would be neat enough in its own right, but DARPA-funded scientists are going one step beyond that and using the haze to actually see further than they would be able to see if it wasn’t there.  More specifically, these researchers believe that the shimmering of heat “waves” can in fact be used as a lens, with the right image recognition technology.

Photo by Keirn

The goal? 90% accurate facial recognition at 1km with a 6cm lens.  As the system combines the data from multiple images, you will not be able to see 1km away in real time; the aim is 1 frame per second.

What I like about this approach is that it takes something that is impeding the goal — haze blocks your ability to see far — and turns it into an improvement — haze helps you see farther!  More problems should be solved this way.

View the technical presentation

Source: New Scientist

By Joshua Koopferstock

These product concept designs by Mac Funamizu, a Japanese graphic designer, are among the most amazing applications of computer vision technology that I have come across. What I found especially inspiring is that some of the technology that we are working on at Feeling Software will be key to making a concept such as this a reality. And without too much effort, I believe that just about everyone reading this blog can see how their own work in 3D or computer vision will be a necessary building block to make this possible.  I hope Mr. Funamizu’s work concept fires the imagination of many people regarding the possibilities and usefulness of augmented reality.

“Future of Internet Search: Mobile Version” Product Concept

This Photosynth-esque approach (above) shows you other photos of the same scene you are looking at, from the same angle.  Here, the designer demonstrates looking at the scene in front of you over time through historical photographs.

Image recognition + mobile internet + Wikipedia?

Text recognition + mobile internet + babel fish?  I wish I had that when I was trying to decipher menus in Slovakia!

See more possibilities for this concept at Mac Funamizu’s blog, petitinvention.

By Joshua Koopferstock

As if getting the required precision for surgery wasn’t hard enough in person, surgeons can now perform surgery remotely from thousands of miles away.  While it is far from mainstream, robotic surgery, including remote robotic surgery, has made leaps and bounds in the past decade.  At SIGGRAPH, I came across a technology that might give robotic surgery another shove forward.

Butterfly Haptics, launched earlier this year as a spinoff company from Carnegie Mellon University, is hard at work trying to commercialize a magnetic levitation haptic device (pictured below).  The grey handle in the center of the device floats in a magnetic field.

What Sets it Apart

The device allows 6 degrees of freedom (translation in any direction, rotation in any direction) like many haptic devices.  However, the Butterfly Haptic device separates itself from the rest in two ways.

First, despite the fact the the device is floating in a magnetic field, it can still very effectively stop your motion completely.  In one demo, you could control objects in 3D space; when you ran the object into a wall, the feeling of hitting a solid object was extremely convincing.

Second, no static friction is present as the device is floating in a magnetic field and is not mechanical.  The surface texture demo illustrating this property sold me completely to the benefits of maglev haptics.  In this demo, you were presented on screen with different surface textures that you could run over with the device, such as a solid wavy surface, a tiny ridged surface which felt something like running your fingernail over the the paper edge of a closed book and, most impressively, the “ice” (frictionless surface).  Pushing down on the surface, it was completely solid, but as you move along it in the other two dimensions, it feels absolutely frictionless like perfect ice.

Butterfly Haptics Device

Back to Surgery

It does not require explaining that in surgery, having maximum freedom of movement and realistic force feedback is optimal, if not necessary.  And it is on these two fronts that Butterfly Haptics excels.  Not only would this technology be beneficial in remote robotic surgery, but also for surgical training simulations.  With immersive 3D displays like those used in currently available robotic surgery devices and realistic force feedback, surgeons-in-training can perform highly realistic surgeries on “humans” (anatomically correct 3D models) without ever making a true incision!

The Future for Butterfly Haptics

While the maglev haptic device is currently more academic than commercial, the fact that Butterfly Haptics has been spun out of academia into the business world suggests to me that these devices may find exciting real-world applications in the near future.  What exactly those applications may be are uncertain, but the company suggests on their site that beyond medicine, the devices may be used for CAD applications, data visualization, and character animation.  The medical applications appear most promising to me, but in any case, this is a company and technology well worth keeping an eye on in the next few years.

This is only one of the many interesting technologies and research papers that we came across during SIGGRAPH last week. Expect to find more blog posts about what we saw at SIGGRAPH in the upcoming days now that I am back in beautiful Montreal.

By Joshua Koopferstock & Christian Laforte

For the last few months, we have been quietly working on Presto3D, a 3D model marketplace which integrates our 3D web viewer to display user-generated 3D models in 3D within the browser.  Finally, it is out the door and in closed beta!  If you want to check it out, go to www.presto3d.com and enter this beta referral key: “presto0845″ (without quotes).

Presto3D Tutorial from joshk on Vimeo.

What is Presto3D?

Presto3D is a marketplace where 3D models can be bought or sold.  All 3D assets are user-generated and user-priced.  When a model is uploaded, we convert the 3D model into COLLADA to create a 3D preview for our web viewer.  With a small, one-time plugin download, potential buyers can see the models in 3D, rotate and zoom the models within the browser.

To ensure an optimal performance and to keep the models safe from petty thefts, we automatically reduce the resolution of textures, compress and encrypt all the data.

Why is Presto3D so exciting?

To the best of our knowledge, there exists nothing on the web that allows such openness for the display of user-generated 3D content.  Due to our automatic conversion, on Presto3D, users can upload files in any Maya (.ma, .mb) or 3dsMax (.max, .3ds) format and see them in 3D in the browser (.dae, .obj, & .fbx are also supported).  Even outside of 3D marketplace websites, other sites will require that you use file formats specially created for the specific web viewer, or create the files within their proprietary platform (such as within some web games and virtual worlds).

The goals of Presto3D are two-fold.  First, we aim to drastically improve the experience of buying and selling 3D content.  Second, we will create the most direct path to display 3D content online, irregardless of the software used to create it.

Go give Presto3D a try, and tell us what you think!

By Christian Laforte

Panic and surprise

Three days ago I received a panicked call from my wife. She had been arrested while driving on the highway near the office in Montreal, Canada with our 10-months old daughter. I ran to the scene and was told by the policeman that my wife drove safely, but we had neglected to renew our license plate on time. We had to accompany him to the police station and pay $600 in fines and towing charges.

Not the actual scene, but you get the idea.

How could this happen? We always pay our bills right away. We notified the government of our new address before moving apartment last year. But more interesting to the readers of this blog, how did the police identify my wife’s car out of the dozens that pass every minute on the highway?

The policeman — let’s call him Joe — gave me a lift to the traffic authorities, and explained how this all works. A real-time license plate scanner is installed on a patrol car on the side of the highway. Using an active light source and high-speed cameras, it tracks every license plate that passes and compares it against an on-board database, updated once a week through a USB key. The device costs $25,000.

“Isn’t that expensive?”, I asked Joe.

“Listen to the radio… They just arrested a guy who already lost his permit. He was driving a car with an expired license plate and he was wanted for petty crimes and unpaid parking tickets. He’s looking at a fine of at least $900, plus the old parking tickets. We would have never caught the guy otherwise. No wonder the big boss wants to equip at least 100 cars with the device by the end of year.”

Frustration instantly switched into interest (and a bit of envy)

That’s a market of $2.5M for a small city like Montreal. A great market for a computer vision technology, with a lot of potential growth in years to come.

Still until now, I’ve always been an optimistic proponent of computer vision technologies. I wasn’t too worried about privacy. Being arrested certainly gave me a fresh perspective. Especially, as it turns out, because the government admitted having a bug in their address change software, which explained why we never got the license plate renewal notice.

Anyway, I still love computer vision and this is a cool technology, so let’s explore how it works and how it could be improved.

Description of the system

I haven’t seen the system but on the spot I asked Joe a lot of questions to have a better idea. The device is bolted on the roof of another patrol car, stopped on the highway. It has two cameras and one red, intense light source, like those used in barcode scanners. The cameras and the light source are tuned to focus on highly reflective surfaces, like a clean license plate. It can be fooled if the license plate is dirty or if there are other highly reflective surfaces in the field of view, e.g. a policeman badge, or I assume, when the sun reflects toward the camera. Otherwise the system appears quite robust: it works night and day, it can deal with partial occlusions of the license plate, and it can read multiple license plates in the same image.

Limitations of the system

- The database is only updated once a week. People can get arrested more than once even though they paid the fine.

- The device only scans plates. It cannot recognize a stolen car with a valid plate. As Joe explained, organized criminals are smart: they wouldn’t risk getting arrested with a false or expired plate.

- The device, I presume, can be fooled easily by adding a filter (e.g. transparent film or grease) on the plate to absorb the red wavelength, or by adding a mirror next to it to distract the cameras. To the human eye, the plate would look fine, but it would no longer be detected by the device.

- Joe explained that, if the driver were to speed away, he probably couldn’t do anything. The police no longer engage in speed chases since it’s too dangerous for the police and the general public. They have a hard time tracking dangerous drivers that speed away.

Clearly, recognizing a license plate is too simplistic. Pretty soon, criminals will know how to fool the system and the only honest people like my wife will be apprehended.

A better solution

For a device like this to be truly useful, it would first need to be connected to the central station database. Just plug it into a cellular network, e.g. using an iPhone or Android (link). With a fast enough connection, the video stream could be uploaded, recorded and processed in a central server farm. This could vastly reduce the size and cost of the device and increase the recognition capability of the overall system. The cheaper system could be installed on every patrol car or traffic light. A dangerous driver speeding away could be tracked across the city and apprehended when finally stops.

Using high resolution cameras, It would be pretty easy to recognize a car color, brand and year from the video stream: all you need is a database of logos and a good feature detector. Getting this to run at real-time would be challenging, but I’m confident this can be achieved given a year or two of development. Looking at the car as a whole would help identify stolen vehicles.

Pushing this farther, cars could be tracked across an entire city, e.g.: London with its networks of surveillance cameras. Criminals could be followed to their lair hours after a crime is reported. Hopefully the people in charge will re-think the overall process so honest people aren’t harassed or tracked without a good reason.

(Note: this is a draft of the post. I haven’t had the time to research the solution, but I’m posting it early anyway since the Washington post and Slashdot just featured a similar story.)