Computer program reveals invisible motion in videos - Amazing!!!!!

RubberDuckGB
But since the source code is now freeware, if they don't do it maybe we as a ATS team can do an smart phone app.

It's freeware but it has a licence that allows only "non-commercial research purposes" and they retain the copyright.

You can read the licence here.

reply to post by ArMaP


Yup, so the app will be no commericial.. Aka free.

Danny

reply to post by ArMaP



Sorry I am falling asleep. What I meant was if MIT don't do a smart app some one else can as the source code is in the public arena. As long it's not used for comercial purposes. That's all. Also submit any ideas thoughts to MIT because I feel they seem really gen, as you are ATS ers are! Lol

Danny

reply to post by ipsedixit


Is there anyway of downloading videos that personally have been uploaded onto this website?

reply to post by zilebeliveunknown


That is really amazing, and demonstrates quite nicely what this code is capable of. Thanks for uploading!

I can't seem to do this right even though I followed the README to the best of my abilities. I hope anyone that reads this comment can type out a step by step procedure. I downloaded MCR from the right link. I used a windows-64 link. I then downloaded the EVM file and placed them into the same folder. I opened up the EVM and created a folder called "data". I don't know what else they meant because there isn't any specifics on it. I tried opening up the .exe and got the 3 errors that it later talked about in README. I couldn't understand what to do from what they said afterwards from their answer to the error.

Can someone please tell me what I need to do?

Zer0megAlpha
I downloaded MCR from the right link. I used a windows-64 link.

I suppose you are using some 64 bits version of Windows, that you installed the MCR and the install didn't throw any errors.

I opened up the EVM and created a folder called "data".

You need to copy the videos you want to process into that folder. If you want to recreate the examples you need to download the videos they used.

Hi Now I know many members are getting really excited about this and it is a great bit of software BUT you have to give it the correct data to work with.

So just in case some people haven't looked at whats required from the site.

Tips for recording and processing videos:

At capture time:
- Minimize extraneous motion. Put the camera on a tripod. If appropriate, provide support for your subject (e.g. hand on a table, stable chair).
- Minimize image noise. Use a camera with a good sensor, make sure there is enough light.
- Record in the highest spatial resolution possible and have the subject occupy most of the frame. The more pixels covering the object of interest - the better the signal you would be able to extract.
- If possible, record/store your video uncompressed. Codecs that compress frames independently (e.g. Motion JPEG) are usually preferable over codecs exploiting inter-frame redundancy (e.g. H.264) that, under some settings, can introduce compression-related temporal signals to the video.

Now most people video cameras phones etc will record using a codec which will compress the video also if you look at the baby video the have a photographers gray & colour card to get white balance and colour accurate.

So as you can see source material has to be of good quality and the part underlined above would rule out most cheap small sensor cameras & mobile phones.

reply to post by thaelin


This is awesome !!!!

i was thinking maybe it would be interesting to apply this to nixon in the nixon/frost interviews - see what his pulse rate does when answering certain questions. there's no movement to speak of either so it should be do-able (?)

reply to post by RoScoLaz


or the apollo astronauts' press conference...

reply to post by RubberDuckGB


Thank You for such an intereting item, But..... I just wondered if the sking / blood circulation caused such, or could it be the screen pulsating ?????? seeeing how they started such via pixelation etc.

reply to post by RoScoLaz


As a part of experiment with that Nixon/Frost interview I did this:



I haven't watched the whole video to see where he would just sit still.

Hello , this could be interesting in the paranormal field, my thought is what if this is applied to someone on his deathbed? catching the actual moment of death ..............what would we see then, some traces of a soul or other entity/aura etc....

Just a thought.

Zer0megAlpha
I can't seem to do this right even though I followed the README to the best of my abilities. I hope anyone that reads this comment can type out a step by step procedure. I downloaded MCR from the right link. I used a windows-64 link. I then downloaded the EVM file and placed them into the same folder. I opened up the EVM and created a folder called "data". I don't know what else they meant because there isn't any specifics on it. I tried opening up the .exe and got the 3 errors that it later talked about in README. I couldn't understand what to do from what they said afterwards from their answer to the error.

Can someone please tell me what I need to do?

Did you figure it out? i am having the same issue...i dble clik EVM and it opens the CMD window then closes after a few seconds...i have a data folder....and a video in it as well..nothing...

Hey guys. I see quite a few of you are having problems using this software, so here is a quick guide. Like I said earlier, this isn't a plug-n-play experience. You need to put effort both into the video that you use [ensure that the subject, camera, and light sources are all still] as well as the settings you use. Anyways, this might help some of you.

----------------------------------------------------------

1. Extract the ZIP to a folder. To keep things tidy, extract it to a new folder.
2. You should now have a folder with the following contents:
-changelog [unimportant]
-evm [this is the program, but you will never have to use this, so just leave it be]
-LICENSE [unimportant]
-README [explains how to use the software]
-reproduce_results [batch file]
3. create a folder called "data" and a folder called "results" in the same folder you extracted the ZIP to

The easiest way to use this software is by executing "batch files". A batch file is basically a text file that contains commands. When you extract the ZIP, it comes with a batch file called "reproduce_results". If you right-click the file, and select "edit" it will open the batch file in notepad, or whatever your default text editor is.

The batch file that extracts from the ZIP contains the commands needed to reproduce the results that MIT shows on its site. The baby, face, shadow, wrist etc. It contains a lot of stuff that isn't really important to an amateur users. I'll try and break down the contents the best that I can though, without dragging this on. I won't cover every single line/command.

----------------------------------------------------------

-":: Windows Batch Script for generating example results" - This is simply a "comment". Anything in the batch file that follows two colons "::" does not actually execute anything. It is just a note. If you want, you can just get rid of these to make the file a bit easier on the eyes.

Everything between "set f=evm" and "mkdir %RDIR%" is necessary. It is just for finding, calling, defining directories and other boring stuff. Don't change any of this unless you know what you're doing.

This is the part that we will actually play with.
-"set inFile=%SDIR%/file.mov" - This is just a bunch of coding jargon that essentially finds/defines the file that the program will analyse. In your batch file, it will probably say "baby.mp4" or something like that. This line has to have the name and correct filetype in order for the software to actually find and choose the video that it will analyse. With all my batch files, I have the filetypes set to "file.mov" and "file.mp4". When I want to analyse a video, I drop it into the "data" folder that I made, rename it "file" and then double click the batch file. Simple.

-"%f% %inFile% %RDIR%" - This here is again just a bunch of nifty coding. %f% is just the EVM program, and %inFile% is just the file name and type, %RDIR% is just the directory to save the file to etc. All this can be left alone, as it is all defined in earlier bits of the script.

And finally, the fun bits. "30 color 140/60 160/60 150 ideal 1 6". These are the actually parameters for the analysis. The "README" actually lays out and defines all these terms plainly for us.

EVM_BIN vidFile outDir samplingRate magType lo cutoff hi cutoff alpha filterType [mag par]

vidFile: the input video file
outDir: directory under which the output video will be stored (the actual
name of the output video file is set automatically according to the
choice of filter, amplification, etc.)
samplingRate: the sampling frequency of video [Hz] (notice this may be different
from the frame rate of the actual video file)
magType: type of magnification (without quotes):
"color": for amplifying color
"motion": for amplifying motion
lo: low frequency cutoff [Hz]
hi: high frequency cutoff [Hz]
alpha: amplification factor
filterType: the type of temporal filter (without quotes)
"ideal" [default]
"butter"
----------------------------------------------------------
So there you go. The readme will explain some stuff in better detail, like the magnification parameters. Also note that the lo and hi cutoffs are written as fractions in the default batch, but the software accepts plain decimals too. Just play around with it. I'll post an example batch file below, so you guys have an easy place to start from. Everything will be good to go, and you can just tweak the settings. Just make a copy of the "reproduce_results" batch, erase all the text inside, and copy what I've pasted below. Everything from "@echo" and down. Then save the batch, put a video in the data folder, call the video "file", and then run the batch file. That's it! If it doesn't work, check the filetype. The batch below is set to use .mp4 files, so if you are using .mov or some other file, just change that one part in the batch file. Note that the space after "C:" is added by me, because it is interpretted as a emoticon and then lost in copy/paste. So just make sure to remove spaces in "C: P".

@echo off
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
set f=evm

set SDIR=./data
set MCR=C: Program FilesMATLABMATLAB Compiler Runtimev80runtimewin64

set PATH=%MCR%;%PATH%

set verNum=v80
set RDIR=Results

mkdir %RDIR%
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

::----------------------------------------------------------
set inFile=%SDIR%/file.mp4
call:printFun

%f% %inFile% %RDIR% 30 color 2 2.4 50 ideal 1 4

:: printing helper Function, should NOT come before all others
:printFun
echo Processing %inFile%
echo.
goto:eof


:: MCR verNum:v80
:: MCR defaultPath: C: Program FilesMATLABMATLAB Compiler Runtimev80

----------------------------------------------------------

Hope this helps folks. I'm excited to what what this community does with this software!

reply to post by LeviWardrobe


Error Message:
Could not find version 8.0 of the MCR.
Attempting to load mclmcrrt8_0.dll
Please install the correct version of the MCR.

Fix:
(I) If you are calling EVM_BIN from command line:
Windows:
$>> set PATH=C:\Program Files\MATLAB\MATLAB Compiler Runtime\v80\runtime\win64;%PATH%
$>> export PATH
$>> evm.exe vidFile outDir samplingRate magType lo hi alpha filterType [magnification_parameters]

Yeah, so how do I do this?

This is really very very cool, nice find!!!!

I look forward to see what it can be applied to in the future.

thaelin
Wonder how the politicians will accept being filmed now? Within minutes
we could know what the lies were. KOOL.......

I laughed... ALOT
Very interesting technology indeed, I wasn't expecting the change there was in the clip where the baby's breathing, at first, was invisble, but after, you could see the motion in the baby's clothes!

reply to post by Deaf Alien


and what does it show?

Its not a magical technique that can be applied to videos like that.