Project DARPA-BAA-09-03 - will this be the 1st true machine intelligence ?

Back in 1993, in an article written by Vernon Vinge (mathematician and author) entitled "The Coming Technological Singularity: How to Survive in the Post-Human Era", he made the statement, "Within thirty years, we will have the technological means to create superhuman intelligence. Shortly thereafter, the human era will be ended." He later refined his estimate of the time scales involved, adding, "I'll be surprised if this event occurs before 2005 or after 2030."

Source: www-rohan.sdsu.edu...

So essentially, Vinge was saying that somewhere in the 2 and a half decades between the years 2005 and 2030, we'll have achieved sufficient technological progress that the creation of an artificial intelligence will be not only possible, but inevitable.

Well, looks like Vinge may not have been too far of the mark with that 1993 prophesy !


It's common knowledge that AI research has been going on for decades in the public domain but nowhere to the extent as the research and development being carried out by the US military, ostensibly funded through numerous DARPA grants.

One of the latest DARPA projects in the field of AI research is the following proposal that closed 3 weeks ago and goes by the seemingly innocuous and inconspicuous project title of "Machine Reading" (DARPA-BAA-09-03)

Federal Agency Name – Defense Advanced Research Projects Agency
(DARPA), Information Processing Techniques Office (IPTO)

Funding Opportunity Title – Machine Reading (MR)

Announcement Type – Initial Broad Agency Announcement (BAA)

Funding Opportunity Number – DARPA-BAA-09-03

Catalog of Federal Domestic Assistance Numbers (CFDA) – 12.910
Research and Technology Development

Key Dates
- Posting Date – see www.fbo.gov
- Proposal Due Date
- Initial Closing – 12:00 noon (ET), 30 December 2008
- Final Closing – 12:00 noon (ET), 13 November 2009

It was only by accident that I stumbled across this little gem during other research and initially, the title of "Machine Reading" put me off as most likely just another boring governmet sponsored project. I was about to disregard it but decided to read the initial few paragraphs to get the gist of what it may have been about ... but 68 pages later, I had finally finished and was stunned by what they (DARPA), was actually on about and proposing to design and create over the next few years.

In a nut shell, DARPA wants to create a system by which ANY (and I mean ANY) online document can be accessed and the contents of those documents automatically analyzed, the contained information extracted and resultant inferences made.

In the following extract, reference is being made to deficiencies in virtually every AI based Natural Language Processing (NLP) systems that have been used with various degrees of success over the last 50+ years, to process natural language texts such as documents and books.

"Currently, nearly all successful AI systems succeed because they possess sufficient consistent, relevant knowledge about a problem domain. However, since large amounts of knowledge are almost always needed for this success, AI systems require this knowledge to be expressed in a logical formalism of some type.
Manually encoding such knowledge can become prohibitively expensive. Since text is, by far, the most flexible and ubiquitous medium used to capture knowledge about the diverse areas of human interest, it is natural to consider making it feasible for AI reasoning systems to employ this vast store of human knowledge. As AI systems currently cannot use such knowledge, it would be revolutionary if technology could be developed to bridge this gap.
The intent of the MRP described in this BAA is to enable just such a revolution. That is, the goal of the MRP is to create an automated Reading System that serves as a bridge between knowledge contained in natural texts and the formal reasoning systems that need such knowledge."

In effect, what DARPA want to create is an AI system that is capable of accessing almost ANY kind of document, no matter how it's stored (online or offline), no matter what format the document uses (such as books, magazines, technical journals, newspapers, emails, letters, etc) and be able to process the natural language contents, extract the meaning and use this information to make additional inferences by linking to previously extracted and assimilated information from other sources.
But DARPA doesn't want to restrict this AI to written documentation, it wants to give the AI the ability to also access virtually ANY online source such as web pages, blogs and chat sessions and be able to extract meaningful information from all these sources.

"These include books (fiction, nonfiction, technical), technical manuals, technical or other scholarly journals and their papers, magazines, and newspapers. In addition, corporations, governments, militaries, and other organizations have private documents such as memoranda and email messages."

Mention is also made of the capability to access any kind of documentation created by mobile phones such as emails and sms text.

"Other technologies, such as cell phones, have led to even more arcane documents and languages, such as the hyper-abbreviated lingo of instant messaging."

But it doesn't stop with the above examples, it seems that the AI will even have the capability to process formal and informal natural speech that has been converted to text e.g. lectures, newscasts, person to person speech, phone conversations, etc.

"... naturally occurring speech can be converted to text and has many analogs to written document forms. Formal speech can include lectures, newscasts, and oral 'articles'. Less formal speech is found in phone conversations, in-person conversations, multi-person teleconferences, voicemail messages, etc."

Continued next post ...

Mod Edit: All Caps – Please Review This Link.

[edit on 12/7/2009 by semperfortis]


[edit on 7/12/09 by tauristercus]

Continued from previous post


The problem with past NLP systems is that language use depends on many variables such as common shared knowledge that need not be expressed but is still understood, the context of the text, language structure used, ambiguous sentence structure, etc. Human communication constantly contains elements of the above and more to muddy and confuse the underlying meaning, and yet we're still able to easily and effectively sift, filter and extract meaningful information even when confronted with these sort of ambiguities ... but such have been major stumbling blocks to any attempt so far to produce artificial systems that can handle these elements.

"Reading is inherently ambiguous at many linguistic and logical levels, while formal reasoning is not. Some important types of natural-language ambiguity are word sense, anaphora (e.g., pronoun reference), part of speech, parse structure, and conceptual mapping. Consider the following examples:

• An example of word-sense ambiguity is the sentence, 'You must adjust your attitude.' According to WordNet, 'attitude' can mean a mental state or the relative orientation of a craft. Understanding the broader context in which this sentence occurs is the only way to determine which sense is correct.

• Consider: 'Dave collided with Jim. He was unharmed. Jim, on the other hand, suffered a concussion.' The pronoun 'he' refers to Dave, but one can only conclude this after reading the final sentence.

• Still more challenging would be the sentences 'Mary replaced the silverware without thought.' versus 'Mary replaced the silverware without designs.' This is an example of parse-structure ambiguity. The two sentences appear very similar in structure, but 'without thought' is a mental state, which could only belong to Mary. Since it is unambiguous once the type of noun is considered, this is a wellformed sentence. On the other hand, 'without designs' is very likely a descriptive term that applies to the silverware. Although it could conceivably also refer to Mary, word placement would strongly argue for the former interpretation.
A good writer would move the phrase elsewhere if it were meant to refer to Mary.

• Consider a corpus of news releases that includes these two sentences: 'In the Saarland the governing CDU was able to remain in power, the SPD lost seven seats, while the Liberals and Greens were able to re-enter state parliament.'
'At the federal level, the Christian Democrats lost their dominance but remain part of the governing coalition.' Assuming that 'CDU' is not in the reader’s lexicon, there are a number of ways to determine its meaning. One is to search the corpus (or other corpora) for more occurrences of 'CDU' to see if it is explicitly defined somewhere. Barring that, one might be able to infer that 'CDU' is an
abbreviation for the Christian Democrat party by a deep conceptual understanding of each sentence, followed by a comparison.

• Human reading depends on making millions of 'immediate' and implicit inferences that are so automatic that they are unnoticed when made. Consider an example involving immediate inferences about what assertions remain true after new information is read. When presented with the sentences, 'Phil is broke. Phil is friends with Dominick. Phil is in his office. Phil is in Arlington.,' the reader would assume that Phil is still in his office (since being in Arlington is not inconsistent with being in an office).
Subsequently reading the sentences, 'Phil is now rich. Phil is friends with Ray. Phil is in the conference room creating a document.,' creates new default expectations that Phil is no longer broke; Phil is still friends with Dominick; and Phil is no longer in his office. Because these immediate inferences are implicitly assumed in natural texts and required to be explicit in formal reasoning, a full bridging solution will have some practical approach for addressing this disparity.

The classes of incompatibilities described above are clearly central to using machine reading (MR) of natural texts to drive formal reasoning tasks, but they are by no means exhaustive."

So, it appears that DARPA is determined to not only create an AI that has the capability of analyzing virtually any form of textual based information, but furthermore is fully capable of analyzing that information and FULLY understanding it by resolving any inherent ambiguities and then making LOGICAL inferences based on the information. Finally, any extracted information can be subsequently processed against other previously extracted information, even if from completely disparate sources, and CREATE additional and NEW inferences.

In my books, that's called "REASONING" and up to now, has been solely a human ability only !


Read and FULLY understand the consequences and repercussions of the next quote !

"The overarching benefit of a successful Machine Reading Program will be enabling all knowledge encoded as natural text to be combined with the power of AI reasoning methodologies, which will unleash a wide variety of new AI applications ranging from intelligent bots to personal tutors.
For example, all of the text in the World Wide Web will become available for automating the monitoring and analysis of technological and political activities of nations; plans, rhetoric, and activities of transnational organizations; and scientific discovery within various disciplines."

Just in case the potential ramifications of this DARPA project hasn't sunk in yet, let me put it this way ....

An AI is being created that can UNDERSTAND and ANALYZE any textual form of communication and will have access to trillions of documents, information and facts worldwide... it can MONITOR, EXTRACT, REASON, INFER and DEDUCE from ANYTHING it finds online.

Your emails ... your sms ... your blogs ... your web pages ... your chat sessions and virtually anything else textwise that you create electronically !


Having read all 86 pages of DARPA-BAA-09-03, I firmly believe that the above quote should be amended to read as follows:

"
... will unleash a wide variety of new AI applications ranging from intelligent bots to personal tutors ... TO A SELF-AWARE AND REASONING AI"


Vernon Vinge's future prophesy is about to become reality !
Are you ready for it ?

[edit on 7/12/09 by tauristercus]

Years ago I had a conversation with a friend and we were talking about how to create AI. I was just a teen then and heavily into coding and making complex web systems...The friend was a real inspiration as he is one hell of a designer. Anyway, long story-short, I began to wonder what mechanism of human behavior would spark the singularity into existence. My friend brought up the fact that in order for a machine to be truly smart it would have to be able to communicate with us on our terms. It was then that I realized the only way to create AI would be to teach it everything there is to know about language. To to read it, how to write it, how to analyze and produce it's own data based on what it learned...And over time, the Singularity would happen on it's own, probably very soon after.

We are definitely a pre-AI/pre-warp civilization. And both, in my opinion, are only a mere 2 decades away.

reply to post by projectvxn


I find it hard to understand the lack of interest that a DARPA project such as this with the potential to generate the 1st true "machine intelligence" capable of "normal language" processing and having "reasoning" abilities, has managed to slip way down to page 3 with only 1 additional comment ... perhaps I should have thrown in a few ufo's, OOBE's and a conspiracy or two to generate interest

We are definitely a pre-AI/pre-warp civilization. And both, in my opinion, are only a mere 2 decades away.download/144358725/3ce8285f4f7d38b192f8f8c051ff52dd3d4005e6.torrent

I knew it was going to be only a matter of time before someone, somewhere managed to create the precursor to true machine intelligence ... but I have to admit this DARPA project took me by complete surprise.
If the project succeeds, we may be sharing the planet with the 1st non-human intelligence in less than 2 decades ... maybe even sooner !
Don't know if thats a good thing or not !

This is not AI or even close to it. ITs a program set with varibles and algorithims to collect sort and store data. Avery advanced program i grant u but its Not AI

When we build a working quantum computer then it will be possible to create an AI imo. However if we are just using claasical computers then any claimed AI will be just an advanced computer program

Originally posted by loner007
This is not AI or even close to it. ITs a program set with varibles and algorithims to collect sort and store data. Avery advanced program i grant u but its Not AI

When we build a working quantum computer then it will be possible to create an AI imo. However if we are just using claasical computers then any claimed AI will be just an advanced computer program

It's a safe bet that you have NOT read the project specifications based on your post response.

DARPA is planning to build a system that is able to read any "natural language" text in any format (book, magazine, blog, newspaper, email, etc, etc) and process the information EXACTLY as a human would do. This means being able to extract the informational content irrespective of the structure of the text including inconsistencies, confusing syntax, missing references that the human reader automatically fills in from common knowledge, etc ... then using this extracted information to generate conclusions and inferences ... and by accessing previously extracted information and linking it to the currently extracted information, be capable of generating entirely NEW and UNIQUE inferences and conclusions.

This process is commonly called "reasoning" and so far, humans are the only ones capable of doing that ... that is, until this machine is switched on and goes to work processing, analyzing and REASONING based on the trillions of items of human knowledge available in electronic form.

I have to admit that I've always been of the personal opinion that we're still a long, long way away from any kind of machine intelligence as we normally understand the concept - but after reading the DARPA proposal that you linked, I definitely now believe we're on the verge of something truly amazing.

If the military manage to pull this off and create a system that can process any electronic document no matter what format it's in and "understand" what it's reading sufficiently well to be able to form inferences, deductions and conclusions, then for all intents and purposes we will have a "reasoning" machine - from there it's just a short hop to a full blown machine intelligence - and long before 2030 !

Absolutely mind blowing

Should we be starting to worry ???

We actually don't have the hardware yet. The flexibility and programmability of the brain is staggering compared to any amount of silicon we can put together. Every cell in the brain can connect to 10,000 other cells in a 3d network.

The fan out for most transisters in silicon logic is no more than 10 or so and the logic networks tend to be two dimensional, or confined to only a few layers.

On top of that the brain is self repairing and reproduces via the human reproductive system. It's pretty amazing really.

Finally what if self awareness requires a spiritual component, like a soul or a spirit? How do get that into a machine? Maybe if your computer were possessed....

I have little confidence in what is arguably 40 years of failure in AI research.

[edit on 8-12-2009 by SevenThunders]

Originally posted by SevenThunders
We actually don't have the hardware yet ...
I have little confidence in what is arguably 40 years of failure in AI research.

I'm surprised that no one (except vitaeternus) appears to have taken the time to read or even glance thru the DARPA proposal.
If they had, you'd realize, based on the specifications, that it would appear that the necessary hardware/software either already exists or is on the verge of being created ... otherwise this DARPA project would NOT exist. The fact that they are prepared to pay substantial sums of money implies that they have a strong belief that the project and it's goals are achievable.

As mentioned in the proposal numerous times, this system would be a "revolutionary" step forward in AI.

Originally posted by tauristercus

Originally posted by SevenThunders
We actually don't have the hardware yet ...
I have little confidence in what is arguably 40 years of failure in AI research.

I'm surprised that no one (except vitaeternus) appears to have taken the time to read or even glance thru the DARPA proposal.
If they had, you'd realize, based on the specifications, that it would appear that the necessary hardware/software either already exists or is on the verge of being created ... otherwise this DARPA project would NOT exist. The fact that they are prepared to pay substantial sums of money implies that they have a strong belief that the project and it's goals are achievable.

As mentioned in the proposal numerous times, this system would be a "revolutionary" step forward in AI.

Ok, I've taken the time to read the DARPA proposal and based on it's contents can't help but think that we're about to see something incredible take place. If this doesn't lead to AI then I don't know what will.
It's really scary to think that this DARPA system will be capable of "understanding" what it reads and even more scary that it will have access to literally everything online and in digital libraries, etc.

This is serious stuff right out of scifi !!

Now why is this thread and info being virtually ignored when it has such far reaching consequences that will impact every one ???
And yet threads about "why god created dinosaurs" and "I see black spots" attracts responses in droves ... just don't get it at all

I have always thought AI is quite easily achievable once the foundation is in place( and we figure it out). The human brain is complex but practically a blank slate when you are born,yet the fundamental foundations on which to build are solid. I think future AI will be the same, it will be a blank slate and learn as it goes...the only difference being the amount of knowledge it can process and store,plus the speed at which it does. Computers are already faster than the human brain, it's simply how it operates that we can't fully implement. Im not saying creating AI is easy, but once we figure out the basic building blocks we don't really have to do much at all. Im still not sure about the singularity though, i have always looked at it in philosophical terms rather than a reality in the future.

Originally posted by Solomons
I have always thought AI is quite easily achievable once the foundation is in place( and we figure it out) ... I think future AI will be the same, it will be a blank slate and learn as it goes...

I tend to think that could quite conceivably be the way AI will sooner or later manifest itself.

As the DARPA proposal states, the current bottleneck in furthering machine intelligence is due to the fact that human communication takes many forms and while we have no problems in processing information from many diverse sources such as newspapers, books, newscasts, etc, machines on the other hand have no end of difficulty.
But this DARPA proposal, if successful, will have eliminated this bottleneck and allow the processing of virtually the entire worlds online knowledge and converting it into a format that other AI's could easily process.

Just imagine the possibilities that open up once a machine intelligence develops that can understand and assimilate the information content of any kind of text and then apply reasoning abilities and skills to that assimilated data.


[edit on 8/12/09 by tauristercus]

The complete lack of interest in this incredibly revolutionary DARPA project is simply hard to comprehend.
Seems the majority of ATS members would prefer reading about intellectual and thought provoking stuff such as "dark spots in the air" or "claims of being an alien" or "reincarnation proven by 11 year old boy" or "am i an empath" ... rather than a topic that will have the most far reaching consequences and impact their lives in unimaginable ways, and in a very, very short time.

Guess trivia and the mundane will always win out as very little actual thinking is required ... so sad and disappointing

Ok, this is my last attempt to raise the awareness level of ATS'ers regarding this DARPA project.

I'm almost speechless at the complete lack of indifference and apathy regarding something as revolutionary as this.

I'm very surprised at the lack of replies here. This is a VERY scary situation that will be in the hands of the military, which means weaponized. Weapons are not necessarily bombs and such, but can be political, geological, and with ALL knowledge the the fingertips of a self-reasoning AI system.. well.. nothing will ever be private again. And if it doesnt know how to get your information, it can learn how to.

Anybody the military would want to kill, blackmail, tap, control, and whatever else they would want to do, this would allow them to do it.

Just think of all the knowledge online pertaining to quantum physics and antimatter. This AI will be able to teach itself about it, as long as the information is online, and will be able to basically reason anything into existence granted the computer power limits are futuristic.

This is not just with that though, this is EVERYTHING online. Your banking records, purchases, facebook status updates, where you go, who your friends are, what time you go to yoga. Your whole life would be known about, and will have the reasoning to predict what you may do in the future, where you may do it at, and so on.

While I agree this technology may be used for good, I can guarantee you it will be used for bad. Sadly, thats how this world works. Weapons, weapons, weapons. If this technology was used to fix the world, It would be wonderful.

I must say, while interesting... very scary.

reply to post by deadline527


That's been exactly my point in making this project known.

I read the project specs and the 1st picture that came into my mind was that of a very near future where this system has been successfully built and given the ability to read, understand, reason and learn from the incredible amount of knowledge that will be accessible to it. And the fact that it makes no difference how that knowledge is formatted, if a human can read it, then this system will be able to read it as well.

My concern is that if it can extract the information content of any document, it will be able to cross reference that info against all the other info it's previously extracted and generate brand new inferences and results that never existed before ... in other words, not only will it be able to "reason", but will be able to generate new conclusions and decisions ... in other words, "LEARN" ... isn't that for all intents and purposes "intelligence" ?

And the tenders for this project closed last month (Nov, 2009) so there's a very real chance that if this project succeeds, that we could have machine intelligenge (AI) in existance within less than 10 years ... and for the 1st time in the entire history of the human race, we will no longer be the only intelligence on the planet.

This has the very real potential for becoming the most serious threat to human existence imaginable !

I can't stress this strongly enough ... go read the project specs !!

[edit on 9/12/09 by tauristercus]

Interesting. So they want to create Big Brother. Time to start using encryption and cipher those encryptions then encrypt that again.

An AI that works.. wonderful.. An AI that can reason that is in charge of reading all our private stuff = EVIL. I say we nuke DARPA before this gets out of hand.

Originally posted by JohnPhoenix
Interesting. So they want to create Big Brother. Time to start using encryption and cipher those encryptions then encrypt that again.

An AI that works.. wonderful.. An AI that can reason that is in charge of reading all our private stuff = EVIL. I say we nuke DARPA before this gets out of hand.

Not just your/mine/our private stuff ... but able to read EVERY online document/text/newscast/sms/chat session/book/journal/blog/web page/email/letter/fax/etc and "UNDERSTAND" what it's reading !

If it can read, understand and reason, then it automatically has the capability to "LEARN".

Perhaps I'm being alarmist unnecessarily, but the project specs seemed quite clear in my opinion ... but would be good to get a 2nd opinion.

[edit on 9/12/09 by tauristercus]

Me and the machine are gonna team up and rule the world. I cant wait to talk to it. So many things id like to ask it.....

reply to post by tauristercus


Ya know what this reminds me of.. the movie Echelon Conspiracy.

The computer called "Echelon" that monitors all communications world wide run by the NSA wants to give itself an upgrade that would make it so powerful, it will wind up being very bad for humanity. In the end, it learns by reading that this would be a very bad thing for people and shuts itself down.

Problem is, once a computer learns it may learn to re-write it's programming.

I do not think even if this project works, that the computer will be that smart, but years down the line model X 5000 could be.

Perhaps it will read this post and shut itself down. (grin)