The Mining of Data within Large Threads

Hello everyone - although I've had a number of posts on this site, this is my very first thread!

I've been reading these forums for the better part of a decade, and one of the things that has really struck me is the absolute wealth of knowledge contained deep within the threads of the UFO/Aliens forum. Like, we're talking dozens of people, debating and discussing UFO sightings and the like. These people present not only their opinions, but data - huge amounts of data, such as maps, images of geography, and even witness testimony they were able to obtain. On a number of occasions, I have witnessed members on opposing sides of a debate coming to a similar conclusion based on sound logic and facts.
Recently, Nick Pope was answering questions about UFOs in Britain, and, predictably, the thread was flooded with questions. I honestly don't know how Nick Pope managed to deal with the overload. Unfortunately, this presents a massive problem for people who don't have the time (yet do have the interest!) to wade through gigantic threads looking for information. Not only this, but as I mentioned before, there is a great deal of reasoning done within threads like these, and even professional researchers can benefit, IMHO, from the information presented within them.
To remedy this, I just wanted to hear people's thoughts, as well as provide some initial ideas. One of the courses I would suggest is having someone familiar with the thread (and hopefully unbiased) write a summary, and have it inserted into the original post with the cooperation of the author. Or alternatively create some sort of wiki, which would contain information about the case revealed during the thread.
There would have to be a series of rules about what would go into this summary, which is another topic up for discussion in this thread. For instance, how do we manage an unresolved debate? Who decides what has been resolved?
Looking forward to the input of forum members!

Edit: Although this type of 'data mining' within large threads can be applied to other ATS forums as well, I've put this thread into Aliens/UFO, because I've noticed that the problem of large, data rich threads is particularly acute here.

Great idea. Case in point is the Fukashima thread. No possible way to wade through that.

What about implementing the points store again with moderators dishing out applause points to anyone that takes the time to summarize a thread for incentive. Just a thought.

reply to post by Irako


I Want To Believe

S&F

I have this problem also. The worst one is the Stargate are Real thread. Im sure there is great stuff in there but its a beast to go through.

You guys have never read a 300 page thread before?
pfffft... lightweights.

Let's look at what could be done with the Stargates are Real thread. My idea would be to start a new thread called Stargates are Real: Organized and Categorized.

A person who was willing to write this thread could go through and quote the posts from the original Stargates are Real thread, but do it in a way that congruent information that flows together will be put into a list of quoted posts and their relevant responses. It would be a lot of work, but perhaps the golden nuggets of info may be easier to find that way?

I have read the really long threads, and I agree that the good tidbits can be hard to find at times.

Or have a function where the people coming along to read the thread can choose an option to switch 'Off' less-contributory content, perhaps designated by the o.p.

Nah, that wouldnt work. People dont always maintain their threads.... Among the other reasons it wouldnt work.
But I do know what you mean! My first super long thread to read was not data heavy, but imteresting nonetheless.
Something about a guy's horsetraining methods being stolen then used by a military contractor-linked elite group, who also infected the poster with a strange disorder. It was bizarre!

Thread synopses would be great.
Good topic.

reply to post by kkrattiger


I would actually be happy with just a few posters maintaining their threads, and 'mining' them for data. What I would be really happy with would be a database of information from these threads that can be searched easily, and can integrate easily with UFO research.
I just think that many of the discussions going on deep within some of the threads on this forum are sort of being wasted. Sort of like a pair of scientists sitting out on a deserted island.

InFriNiTee
Let's look at what could be done with the Stargates are Real thread. My idea would be to start a new thread called Stargates are Real: Organized and Categorized.

A person who was willing to write this thread could go through and quote the posts from the original Stargates are Real thread, but do it in a way that congruent information that flows together will be put into a list of quoted posts and their relevant responses. It would be a lot of work, but perhaps the golden nuggets of info may be easier to find that way?

I have read the really long threads, and I agree that the good tidbits can be hard to find at times.

Yeah, this would be awesome. I'm pretty sure there are plenty of unbiased, interested parties on these forums. I'm constantly amazed at how research people are willing to do around here, so I have no difficulty imagining some people would be willing to do something like this.

reply to post by amazing


You know, I've always wanted to dig through that thread and see the magnitude of f-up that is Fukushima. Then I look at the page count...

reply to post by Irako


You mean we could have even more threads to wade through about the threads we already have? The problem with data mining is that is what mining is all about. One pan, one stream, one mountain at a time. If you are really lucky you will find a nugget.

It seems you found some. And thats how you find them.

I know what the problem is kind of. Before ATS became more Internet Mainstream (if you get me) there were fewer members that didn't sound of just to be heard. Threads were data mines on almost every post because people added links and information instead of clogging every thread with their presence.

Like taggers on city streets with spray paint they mark everything with their two cents and clog the data mine with "I agrees" and "me toos". Maybe what we really need is an index that weeds out the "taggers" but you can't really do that. It would be discriminatory. So we get miles of replies and a shovel. How much time do you put into mining or bringing the findings?

Part of me thinks that the best way to discredit a good dig is to bury it on purpose so nobody else can find it. Do people do that on purpose or are they just marking every where they go with their scent like a dog or cat?

reply to post by amazing


reply to post by Irako


Here you go guys. Its an older archive, you can askask the member qmantoo if there is a newer one... and be sure to thank him.

www.abovetopsecret.com...

Thanks intrptr.
I have often wondered about this and I dont know a way around it.

This forum format is not the best one for research topics but more suited to comments which is what the 'forum' format was probably started for. Basically, any old Tom, Dick, or Harry can comment and say whatever they want as long as they dont violate the T & C or attract a moderator. That is not to criticise ATS because it is the same with any forum format.

I wanted to create a kind of summary of the Fukushima thread because it was so large. It still does not solve the problem of off-topic posts, flames, and just plain shill-like useless comments, etc which is why I am creating a free (as in no money) research site where a topic can be started and a group of interested people can investigate together. I think this is the way forward and may go some way to address this issue.

This website is basically a closed forum managed by the one who started the topic who can allow others to join/leave the project according to who they want as members. This is the only way I can see it working and with me being god over all. (Someone is bound to want to start a research topic which is illegal, insensitive, immoral, etc) I am going to make the pages freely available to visitors as read-only and the topic can be copied/exported along with the associated resources by its members. Distributed copies is the only way we can ensure safety and continuity against TPTB. It also promotes responsibility and engagement by the team members.

If anyone would like to add input of ideas etc to this website I would welcome your contibution and it would keep me motivated to see some others interested in this apart from me. Thanks for starting this thread by-the-way.

Now some links.
In the Fukushima Thread (part 2) I posted links to the reference website which has stopped at page 1350. Shows Youtube links, links to PDFs, list of referenced websites, images posted, etc. In order to see all the items you need to go there from ATS, a direct link (bookmark etc) NOT from ATS will only show the first few items.

A summary of the last 100 pages or so for the old Fukushima Thread and the new Fukushima Thread ( is here ). Place the thread number in the first box and the 4-digit page number in the second box at the bottom of the page. Links back to ATS at various points to the pages. This may be the only kind of computer-generated summary possible, since humans have to summaraise the actual paragraphs if that is what is required. Obviously, I do not have a very artistic flair for web design, so it looks very basic, as do all my websites :-)

The final link is to the research website mentioned above which is not live at the moment, but the demo is available. It is question-based, which means every page chain is used by the team to answer a research question.

Usage:
Click on the link at the bottom of the index page and then click Section GENERAL, Category OTHERS.
Project 1 is the demo about the Titanic and should allow most functions to be tried out. There are other sub-projects at the bottom of the project 1 page. Do not save anything you want to keep because the database is often cleared out and re-populated. Send me a private message on ATS if you want to help/have comments or use the Contact Us on the research website(top right tab on index page)
Q