WOW! M.I.T. Researchers can recreate sound from objects in the room

a reply to: neoholographic


Ok -- so in a controlled environment, the minute vibrations of a bag of chips of a plant can be read to recreate a sound in that room.

Great. I think everyone here agrees with that (incluing ZetaRediculian). I'm not sure what your argument beyond that is. It seems to me that everyone else is right that in a non-controlled real-world environment, it would be difficult to recreate a conversation in that room (i.e., if the air conditioning was blowing on the plant, or if music in the room was making the bag vibrate along with the vibrations from the conversation).

Is there any indication that this technology can effectively cancel out any other real-world vibrations and movements other than the intended "target" vibrations? If not, then the conversation may have so much background noise (i.e., from the other vibrations). It seems that in a real world application, air blowing through the room will make the plant move, too. How are they cancelling out this movement?

Again, I'm not sure what it is that you (neoholographic) are arguing.


I will say, however, that it is certainly possible to do this with certain items -- such as a Window. back in the 1980s, the CIA and KGB were spying on each other by using a laser to read the vibrations of the windows in each others offices, and recreating the sound from those window vibrations. However, that's slightly different due to the window perimeter being fixed on four side, which reduces unwanted "free" oscillations. This way, the windows acts like a speaker diaphragm,and is vibrating mainly due to sound. of course, those windows would be vibrating due to sound outside, also, so those vibrations from outside noise may have been required to be cancelled out.

a reply to: Soylent Green Is People

What??

First off, if you would have read the thread you would clearly see that these guys have been debating against things that were never said from magic waves to the human voice being unique.

You asked:

It seems that in a real world application, air blowing through the room will make the plant move, too. How are they cancelling out this movement?

Canceling out this movement? Why would they need to cancel out this movement when the vibrations are captured in frame and are less than 100th of a pixel?

This is why the question doesn't make any sense. The wind isn't going to blow fast enough unless you're in a tornado or a hurricane to disrupt vibrations that are less than 100th of a pixel. This is why waving the bag of chips nonsense is just that, nonsense. Because you're not going to move your arm fast enough to not be captured in frame and that will disturb vibrations that are less than 100th of a pixel.

originally posted by: neoholographic
Canceling out this movement? Why would they need to cancel out this movement when the vibrations are captured in frame and are less than 100th of a pixel?

Because the vibration they are recording will change as soon as there is other forces applied to whatever they are recording.

If you don't understand how harmonics work and interact, should you be involved in this conversation?

a reply to: neoholographic

WOW!! COMPLETELY STILL LOL!!!

you really missed the point. Let me try again because it was pretty obvious. Yes, the objects were isolated and completely still except for the vibration. There is nothing else indicating any other interference. Everything was done in a completely controlled environment under ideal conditions. The points you are making are entertaining. Not much else. The more you carry on, the less seriously you will be taken. Take a breath and explain it again. I want to make sure I get this.

originally posted by: neoholographic

Canceling out this movement? Why would they need to cancel out this movement when the vibrations are captured in frame and are less than 100th of a pixel?

For the same reason that if I want to hear a whisper over the sound of a rock-n-roll band, I would need to cancel out that rock-n-roll band.

That's why.


That movement of the plant caused by the air conditioner would be part of the 100th of a pixel vibration. It would be additive to the "target" vibration of the voice, but that total sum vibration (the air conditioning and target sound together) would just be one movement. It's not like the plant is sometimes moving due to the target sound and sometimes due to the air conditioning.

originally posted by: neoholographic

I said it will be harder to mask the audio because you're dealing with vibrations that are less than 100th of a pixel that gets better as you increase the frames per second. So it will not be like picking up sound with a bug that's in the room that depends strictly on audio data. This technology depends on visual data as well and that's why they did the experiment through soundproof glass. This is because they're recreating sound through visual data.

A recording device in a room such as a "bug" works because the audio causes "vibrations" in the detecting equipment, which then translates the vibrations into an electronic signal.

This experiment simply replaces the microphone with a bag, and uses video combined with the algorithm to convert the visually observed vibrations into an electronic signal. It's the same fundamental concept. Sound makes something vibrate, and we use equipment to detect the vibration. This experiment still relies entirely on audio data as the initial input, just like a microphone. It simply replaces a fairly efficient method of capturing that data, with a fairly inefficient method.

This technique suffers from all the same weaknesses as a "traditional" recording method such as a "bug", because it relies on exactly the same principle. In fact it has several more weaknesses that, to my mind, outweigh the advantages it can offer.

You have been offered several opportunities to discuss this in detail and expand on your points - and those offers have been far more polite than your responses merit, to be honest.

If you are still interested in trying to make this into a reasonable thread, I would be greatful if you could address the following point:

You have drawn attention to the fact that the process works through a soundproof window - do you believe it would still work if the bag was placed outside the glass, near the camera? If you believe it will only work when on the same side of the glass as the audio source, how is this any different from a traditional microphone?

a reply to: neoholographic

You're still not getting it. There is nothing magical about the frequency band the human voice typically resides in. ANY background noise occupying this band (LOTS of everyday sounds) WILL mask the signal. I'm sorry that you're failing to grasp the principles at play here, how many times does it have to be said? How many people must say it?

originally posted by: EvillerBob
A recording device in a room such as a "bug" works because the audio causes "vibrations" in the detecting equipment, which then translates the vibrations into an electronic signal.

This experiment simply replaces the microphone with a bag, and uses video combined with the algorithm to convert the visually observed vibrations into an electronic signal. It's the same fundamental concept. Sound makes something vibrate, and we use equipment to detect the vibration. This experiment still relies entirely on audio data as the initial input, just like a microphone. It simply replaces a fairly efficient method of capturing that data, with a fairly inefficient method.

Exactly.

Just like the spy devices I was talking about a couple of posts up that used a laser to read the vibrations of a Window, this method uses a video camera to read those vibrations. However, as you said, they are still all reading vibrations, just like a speaker diaphragm or a microphone diaphragm uses vibrations. It's all the some concept.

All of these methods simply measure the amplitude and frequency of a vibration. If one knows the amplitude and frequency of a vibration, then they could artificially recreate the sound that vibration was making.

That's also the basic concept behind any recording method (vinyl records, cassette tapes, CDs, mp3, etc.) . The difference with those is that the "source" sound was captured by a microphone, and then broken down into electrical signals that corresponded to the amplitude and frequency of the source sound. In this video camera method posted by the OP, the amplitude and frequency of the vibrations are instead read by a video camera rather than a microphone. However, the concept is identical; both methods are reading the vibrations.

a reply to: Soylent Green Is People

Again, this comes from a lack of understanding of the technology being used. You said:

That movement of the plant caused by the air conditioner would be part of the 100th of a pixel vibration. It would be additive to the "target" vibration of the voice, but that sum vibration would just be one movement. It's not like the plant is sometimes moving due to the target sound and sometimes due to the air conditioning.

It will not disturb the recreation of sound because the recreation of sound depends on frames per second. The movement from the air conditioner would have to be so fast that it will distort the visual and it couldn't be captured in frame. This is why they talked about how the fast moving rotor of a helicopter can distort the visual. In order to disrupt the recreation of sound that's being recreated through VISUAL DATA, you would have to move fast enough to distort the image. This is why they talked about caputuring these small vibrations through frames by second and then running them through the technology.

Reconstructing audio from video requires that the frequency of the video samples — the number of frames of video captured per second — be higher than the frequency of the audio signal. In some of their experiments, the researchers used a high-speed camera that captured 2,000 to 6,000 frames per second. That’s much faster than the 60 frames per second possible with some smartphones, but well below the frame rates of the best commercial high-speed cameras, which can top 100,000 frames per second.

Unless you can move fast enough to distort the image then wind blowing from an open window or an air condition will not move fast enough to distort the image being captured in frame.

This is just basic common sense.

a reply to: neoholographic

So then mask your voice with a stereo playing music.
Also, what about other harmonic vibrations in a building that would be affecting the plant movement?

The plant and the chip bag are simply microphone diaphragms, and microphone diaphragms can pick up lots of unwanted sounds -- sounds that may create unwanted 100th of a pixel vibrations.

originally posted by: neoholographic
a reply to: Soylent Green Is People

Again, this comes from a lack of understanding of the technology being used.

No my friend, the lack of understanding is on your end.

Your voice creates pressure waves.

Other background sources create pressure waves.

These will all contribute to the waveform you are recordingregardless of the recording method.

If the background noise is of sufficient amplitude, the masking will be so great that the signal you wish to record (someone's voice) will be masked to the point of intelligibility because the sum of these different pressure waves makes it impossible to pick out the original signal.

It doesn't matter whether you use a microphone, a high speed camera, a piece of string and some plastic cups, you cannot get around this caveat.

a reply to: EvillerBob

Again, you're not making any sense as it pertains to this technology.

It's not the same as capturing audio on a microphone and this is why they specifically recreated the sound through soundproof glass.

Again, this is more efficient because you're dealing with VISUAL DATA.

How hard is this to understand?

It's much easier to mask the audio on a bug because it's picking up the sound from the room not recreating the sound at less than a 100th of a pixel through VISUAL DATA.

In order to stop this technology from recreating the sounds in the room you would have to move fast enough to distort the images so they can't be captured in frame.

HOW HARD IS THIS TO UNDERSTAND?

It's simple and laid out in black and white. They even talk about how a fast moving object like the rotor of a helicopter can distort the image and make it harder to pick up visual data.

a reply to: neoholographic

lets pretend you have some understanding of this beyond everyone else.

None of it really matters until its demonstrated. Its very simple. What don't you understand?

a reply to: neoholographic

I have got a three year old grandson who can do the same thing with only a wooden spoon.
Then his mother takes the spoon away and stops all the fun .

a reply to: neoholographic

a reply to: neoholographic

Is the bag of chips attached to their vocal cords?

No.

Then what is the source of the vibrations?

Air pressure waves.

Then any other sources will contribute to the pressure waves interacting with the bag of chips, contributing to the masking of the signal.

There is NOTHING in the video that speaks about this method is exempt from masking (and why would it be? There is absolutely no reason why it would be). This is a complete fabrication on your part.

originally posted by: GetHyped
If the background noise is of sufficient amplitude, the masking will be so great that the signal you wish to record (someone's voice) will be masked to the point of intelligibility because the sum of these different pressure waves makes it impossible to pick out the original signal.

Yes -- but there is noise-cancelling technology such as noise-cancelling headphones.

However, that noise-cancelling technology is listening is a specific location for that noise. For example, noise-cancelling headphones listen to the environment outside the headphones and adds sound waves opposite to those outside sounds to the music in order to cancel that outside noise.

But the headphones don't inherently know the difference between unwanted sounds and wanted sounds. It simply is told which direction (outside the headphones) those unwanted sounds would be coming from.

a reply to: Soylent Green Is People

The problem is that noise cancellation won't help if you cannot distinguish the noise from the signal. Turning up a CD player to mask the voice will make it impossible to distinguish the voice with a camera/microphone/whatever alone.

a reply to: neoholographic

neo --

The sound of the voice is not making the plant vibrate more minutely than the sound of, say, music. Nor is the voice causing the plant to vibrate more minutely than, say, the vibrations of someone walking near the plant (and vibrating the floor, which then transfers to the plant.

All of these vibrations cause the plant to move 100th of a pixel -- NOT just the sound of the voice.

The visual data cannot distinguish the difference between a 100th of a pixel vibration caused by a voice or a 100th of a pixel vibration caused by some unwanted sound/vibration. It cannot inherently separate out the voice vibration from the unwanted vibration just by visually looking at the vibration.

Nowhere in the article in the OP do the MIT people say "voice vibrations are different than other vibrations, therefore we can separate voice from unwanted noise". Forgive me if I'm wrong, but that's what you seem to be suggesting.

originally posted by: neoholographic
a reply to: EvillerBob

HOW HARD IS THIS TO UNDERSTAND?

I take this to mean you are not prepared to answer the question? Colour me surprised

At this stage, you've had the principle explained to you in not one, not two, but three or more different ways by different people. In return, you have basically repeated the same incorrect statement over and over again, and refused to answer any of the genuine and useful questions that have been put forward.

Will there come a point when you start to ponder - hey, everyone else who has posted is consistently saying that I'm wrong, maybe I need to think about this a bit more? I rather hope not, this is quite entertaining. Unfortunate because of how it reflects on ATS as a site that seeks to deny ingnorance, but entertaining.

originally posted by: Soylent Green Is People

Nowhere in the article in the OP do the MIT people say "voice vibrations are different than other vibrations, therefore we can separate voice from unwanted noise". Forgive me if I'm wrong, but that's what you seem to be suggesting.

Exactly. This chap seems convinced that voice audio is some kind of magic wave that creates a special vibration that can't be overridden by any other type of vibration.