A research team from Carnegie Mellon University has claimed to have developed an optical microphone or camera system that can monitor sound vibrations very precisely. The precision is so high that the camera can capture separate audio of individual guitars playing simultaneously. That allows the camera to reconstruct the music faithfully and accurately from a single instrument even when it is playing in an orchestra or band.
Even using the most directed and high-powered microphones it is not possible to totally eliminate neighboring sounds, effects of acoustics, and ambient noise when capturing audio. The research team has used a novel approach. They used two cameras along with a laser beam. This allows them to sense the high-speed but low-amplitude vibrations from the surface of the instrument. The team uses these vibrations for reconstructing sound. The unique arrangement allows them to isolate the audio and capture it without a microphone and with no interference.
The team claims to have invented a new way of seeing sound. The camera system is innovative, represents a new device for imaging, and makes it possible to see things that are not visible ordinarily to the naked eye. The team has successfully completed several demonstrations for showcasing the effectiveness of sensing vibrations and reconstructing sound faithfully and with quality.
During their demonstrations, the team was able to successfully capture isolated audio from separate guitars that were playing together, and the audio of individual speakers that were playing assorted music at the same time. For instance, they analyzed vibrations from a tuning fork. They also captured the vibrations on a bag of burritos placed near a speaker thereby capturing the sound from the speaker.
The team significantly improves the work done earlier for capturing sound by computer vision. Where earlier researchers used high-speed cameras for producing a high-quality recording, the present researchers used ordinary cameras costing only a fraction. The dual-camera is necessary for capturing vibrations from moving objects, such as the movements of the instrument when the musician is playing it, while at the same time, sensing individual sounds from many other points.
The team claims they have improved the optical microphone to make it more usable and practical. They claim to have improved the quality while reducing the expenses.
According to the team, the system operates with two types of shutters—a global shutter and a rolling shutter. An algorithm analyzes the difference in speckle patterns between the two streams of video. It converts the differences into vibrations for the reconstruction of the sound.
A speckle pattern is a result that coherent light or laser generates after its reflection off a rough surface. By aiming a laser beam at the vibrating surface, the team created the speckle pattern. This speckle pattern changes with the changes on the surface as it vibrates. The rolling shutter rapidly scans the speckle pattern from top to bottom and produces the image by stacking rows of pixels one on top of the other. At the same time, a global shutter captures the entire speckle pattern in a single instance.
