Massive US military social media spying archive left wide open in AWS S3 buckets.

Massive US military social media spying archive left wide open in AWS S3 buckets

Dozens of terabytes exposed, your tax dollars at work

By Iain Thomson in San Francisco 17 Nov 2017 at 20:08

Three misconfigured AWS S3 buckets have been discovered wide open on the public internet containing "dozens of terabytes" of social media posts and similar pages – all scraped from around the world by the US military to identify and profile persons of interest.

The archives were found by veteran security breach hunter UpGuard's Chris Vickery during a routine scan of open Amazon-hosted data silos, and these ones weren't exactly hidden. The buckets were named centcom-backup, centcom-archive, and pacom-archive.

CENTCOM is the common abbreviation for the US Central Command, which controls army operations in the Middle East, North Africa and Central Asia. PACOM is the name for US Pacific Command, covering the rest of southern Asia, China and Australasia.

Vickery told The Register today he stumbled upon them by accident while running a scan for the word "COM" in publicly accessible S3 buckets. After refining his search, the CENTCOM archive popped up, and at first he thought it was related to Chinese multinational Tencent, but quickly realized it was a US military archive of astounding size.

"For the research I downloaded 400GB of samples but there were many terabytes of data up there," he said. "It's mainly compressed text files that can expand out by a factor of ten so there's dozens and dozens of terabytes out there and that's a conservative estimate."

Just one of the buckets contained 1.8 billion social media posts automatically fetched over the past eight years up to today. It mainly contains postings made in central Asia, however Vickery noted that some of the material is taken from comments made by American citizens.

www.theregister.co.uk...

Persons of interest might not only include suspected terrorists, but anyone who might be a person of interest for whatever reason.

Somewhere there's a video with former NSA Director Michael Hayden talking about this very thing, and he gets a wry grin and smile on his face when he speaks of "persons of interest" to mean anyone at all who they find to be of interest for any reason, even preumably for sh*t's and giggles or whatever, even corporate espionage. I've seen reports about CIA and NSA officers high fiving one another over how much money they were going to make as a result of their special access.

What are your thoughts?

Ankh

a reply to: AnkhMorpork

A file on S3 storage is spread over thousands of physical drives. I'd rather format a 40 megabyte hard drive with bit poker than try to assemble an S3 index by hand.

Try not to post you face on Youtube, that is a start.

This proves it. Everyone is famous!!

originally posted by: IgnoranceIsntBlisss
This proves it. Everyone is famous!!

Why is everyone a fascist...oh famous. I misread it.

originally posted by: musicismagic
Try not to post you face on Youtube, that is a start.

Or, in my experience, to speak out loudly and forcefully in opposition to a certain action or policy position by then President Barack Obama.

Black hats can fly out at you from all directions like bats from a cave, although most like to work remotely and in more ways than one.

They betray themselves and their position however, and can find to their dismay that they might have placed themselves in a double bind and a hoodwink because isn't the devil always blind?

Btw, I didn't know where to post this. Looked through the forums and i thought to myself, well, it's a military project.

The militarization of Facebook and Google.

Strange times..

Maybe just turning off the computer and going out for a walk might not be such a bad idea. If I don't report back just assume that i got hit by a truck while on my way to Starbucks (God forbid!).

Best regards,

Ankh

originally posted by: dfnj2015

Why is everyone a fascist...

Because Donald Trump.

JK!

ARMY training SIR!
I wonder how many Cryptos suicide without EXTERIOR help?

What really freaks me out most about this story, is that no one seems freaked out by it at all really.


..and in today's news, billions and billions of social media posts are sucked up and stored by the US military.

It's like that Obama authorized program which was unmasking (internally) 10's of millions of Americans.

That was against the law! No one was charged or held accountable.

The difference with social media posts i suppose is that in making the post in the public forum makes the data public domain.

I'm thinking seriously about suicide, Facebook suicide.

I've also changed my mind about being a Youtube star.. lol

First thing I thought of was that scene in the "Snowden" movie where he watches the No Such Agency dude do the search through damn near the same type of database. He asks the guy what's in the database, and the guy just looks at him with a smile, and answers "Uhh, everything"

Gives me a chill every time I watch it. The BAD kind of chill.

I guess it's my Libertarian/Anarchist side freaking out.

originally posted by: AnkhMorpork
What really freaks me out most about this story, is that no one seems freaked out by it at all really.

I assure you, my friend, some of us are!

It makes me wonder why the military would even use AWS. I'll have to research that. You'd think that the US military could afford people to write proprietary code because "security".

a reply to: AnkhMorpork

Because Donald Trump.

What a novel way to derail your own thread!

Ahhh, quick question!

What is AWS S3 buckets?

Thank you.

I'm wondering if when I know it will be less freaky, or moreso?

originally posted by: AnkhMorpork
What really freaks me out most about this story, is that no one seems freaked out by it at all really.


..and in today's news, billions and billions of social media posts are sucked up and stored by the US military.

It's like that Obama authorized program which was unmasking (internally) 10's of millions of Americans.

That was against the law! No one was charged or held accountable.

The difference with social media posts i suppose is that in making the post in the public forum makes the data public domain.

I'm thinking seriously about suicide, Facebook suicide.

I've also changed my mind about being a Youtube star.. lol

Land of free, home of the watched.

You're free to do whatever like, within the boundaries allocated by your owners.

a reply to: AnkhMorpork

Nothing changed after Snowden.

It's bad enough that all our comms are stored by our own Intel/Military groups. Now we learn they've been wide open to foreign governments and everyone else since 2009.

From the report:

The poor CSTAR cyber risk scores of CENTCOM and PACOM - 542 and 409 , respectively, out of a maximum of 950 - is a further indication that even the most sensitive intelligence organizations are not immune to sizable cyber risk. Finally, the collection of billions of internet posts in several unsecured data repositories raises further questions about online privacy, as well as regarding the right to freely express your beliefs online.

UpGuard

It looks like privacy has been sacrificed for non-existent safety. Tax cuts for the already wealthy, legal tax avoidance havens and we pay more taxes to allow Intel/Military to sit on our shoulders and spy. What a wonderful system we're paying for.

a reply to: Kandinsky

It's both absurd and an outrage and a major abuse of power and a violation of the fourth amendment.

Maybe all is not lost. Don't ask me why but I remain an eternal optimist.

I really wish they'd cut it out though, as well as what I call "satanics".

Enough's enough already! Sheesh!

What I discovered, via my own encounter with "them" is that they're not even that smart and all their games and shinanigans add up to nothing, and can even make things much worse, for the observers themselves.

You're right, it's totally messed up, but at some level it helps us know who all the assholes are where for every action there's a reaction and where it may be said that to be is to be perceived.

They're screwed themselves, particularly when these tools start to get used on they themselves ie: like the Podesta brothers and the like.

Matt Drudge is headlining it right now..

a reply to: AnkhMorpork

I had a look for the archives in the Amazon 'buckets' and couldn't see them.

They're screwed themselves, particularly when these tools start to get used on they themselves ie: like the Podesta brothers and the like.

Your comment here reminds me of stages of moral reasoning in young children. Toddlers will cheat at games, steal stuff and sometimes give other kids a slap. Nearly all of them have a moment when they realise that playing by the rules benefits everyone - they go up a stage in moral reasoning. Likewise they realise that slaps are cool until someone slaps back and it hurts - they stop slapping.

The people who do all this spying and gaming the system don't appear to have the morals of an enlightened toddler. As you point out, they're #ing oblivious to the inevitability of others treating them the same way. They also seem to have that me, me, me vibe too because they're also oblivious to the repercussions on wider society.

2009- a year when many became targeted individuals. Noticing they were being gestured to, then started hearing voices. All satallite and emf based technology.