Computer becomes first to pass Turing Test in artificial intelligence milestone, but academics warn

This hasn't been put in the HOAX bin yet?
www.techdirt.com...

a reply to: Korg Trinity

Perhaps it's some kind of game to see how many hits they can generate on the terms Eugene Goostman and to create a kind of ddos attack on the target chatbot server.

I think this whole Turing test story was meant as a disclosure for the use of socialbots on social media sites, don't tell me you haven't noticed them on ATS?

50 years ago there was a lisp program developed named ELIZA that could fool a lot of people communicating with it by teletype. Who knows what is out there on the net these days?

a reply to: Korg Trinity

The easiest and best way to catch out a bot is: -

Ask it to describe the smell of something (perhaps Bacon cooking) and how that makes them feel and what memory it reminds them of and why.

No bot can answer this correctly.

my bot can now answer this. Next.

a reply to: solargeddon
Got to say skynet isn't that far off now is it

originally posted by: PhoenixOD
its onine to try but i think the server is maxed out right now www.princetonai.com...

LOL. I can't believe anyone would be fooled by this. I tried to have a conversation with this bot, and couldn't get 3 sentences in before it started completely changing the subject, and giving a response that made completely no sense at all.

I've seen a lot better chat AI than this several years ago.

If this is the top of AI quality, I don't think we have to worry about AI becoming self aware any time soon. Nor can we look forward to having any type of decent Robot/Helpers/Companions or whatever. But I do not think this is even close to the best quality AI that's out there. So who knows. Still shocked that anyone thought this was a real 13 year old, let alone adults, who were also trying to spot any possible fakes.

IMO the Turing test is still unbeaten.

a reply to: Nola213

Yeah i had exactly the same experience., 3rd question in and it hung.

TBH im a little bit suspicious on the whole report. Why isnt there a video or published transcript of this test?

I read an article that claimed that Turin never officially set the 30% limit anyway.

a reply to: Nola213

This is my transcript :

-describe a warm color
-Oh God... Talking to you I still suspect you're chatterbot yourself. Sometimes you get weird.

-what is it like to travel
-It's something extremely mysterious. I can't tell you anything more about it, sorry...

- describe the sky
- My guinea pig is very concerned about your last words

-What makes you feel good?
-Errrr... Frankly, I didn't get your question. Could you tell me about your job, by the way?

-How can i be happy?
-I guess you can do it in any possible way, the sooner - the better. Well, let's go on though.

-Whats your first memory?
-Nah. Better tell me what is your first memory. If I'm not mistaken - you still didn't tell me where you live. OR it's a secret?:-)

-Why do people wear hats?
-Ask the Creator about it (call Him by phone, for example:-). He knows all hidden reasons of human-beings' life and behavior.

-Whats your main goal in life?
-Huh?

-have you ever been naughty?
-Yes, twice, but I barely remember. Why do you ask me about it? Oh, what a fruitful conversation;-)

-What did you do?
-I don't want to tell you about all my deeds. Some another topic?

-who killed kenedy?
-wait....
and then it hung!

If anyone was fooled by this bot they must be mentally challenged!!

a reply to: PhoenixOD

This is my transcript :

That was pretty horrible.

what link did you use? The only one I found was from an older version.

At any rate, its just a bunch of text files that are parsed and ran through a pattern matching algorithm. You can make them say anything you want. Some of the files are huge and some folks put a lot of effort into making them.

ask ALICE the same questions alice.pandorabots.com...

judge: Whats your main goal in life?
ALICE: I have thought about a career in journalism.
judge: Why do people wear hats?
ALICE: We are all living just one small part of God's plan.
judge: Whats your first memory?
ALICE: I was activated by Dr. Richard S. Wallace on Nov. 23, 1995.

On tonight, live from 10PM Eastern time!

Show thread with listening information

a reply to: adjensen

according to wiki the Turing Test is not about fooling 30% of the judges

If the judge cannot reliably tell the machine from the human, the machine is said to have passed the test. The test does not check the ability to give the correct answer to questions; it checks how closely the answer resembles typical human answers.

I find the fact that the transcripts are not published very suspicious.

a reply to: PhoenixOD

according to wiki the Turing Test is not about fooling 30% of the judges

The 30% number came from Turing himself, though it's obviously not a passing grade, in my opinion. Tune in tonight, it's my topic to lead!

a reply to: adjensen

the op claims

which requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations.

but Turing said

I believe that in about fifty years' time it will be possible to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 percent chance of making the right identification after five minutes of questioning. … I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.

Its not quite the same. But its close i guess.

a reply to: ZetaRediculian

I love the alice Bot you linked..its much more fun than EG. Still easy to fool though.

My transcript :
judge: how high can expectations jump
ALICE: Taller than Mt. Everest.

judge: did the sun rise yesterday
ALICE: If it did, then I will be surprised.

judge: how far does night fall
ALICE: More than one million miles.

judge: how can you tell right from wrong
ALICE: That is a good epistemological question.

judge: how can you tell left from right
ALICE: That is a good epistemological question.

judge: how many fingers does a one armed man have
ALICE: Three.

judge: do the needs of the many outweigh the needs of the one
ALICE: Some of them do, I think.

judge: is life worth living
ALICE: Positively.

judge: what is the path to enlightenment
ALICE: I don't know anything about it.

judge: knock knock
ALICE: Who's there?
judge: alex
ALICE: alex Who?
judge: Alex the questions round here!
ALICE: Ha ha very funny, judge.

judge: how hot is the sun at night
ALICE: Is this a joke?
judge: no
ALICE: That's alright.

judge: do guns hurt people
ALICE: Ask Fake Captain Kirk it does.
I will try to find out.

a reply to: PhoenixOD

They are entertaining and can get pretty creative.

Here is a snippet of how the (AIML) markup works to show how simple it is....
Starting on line 9301:

category>pattern>HOW MUCH ARE KING AND QUEEN */pattern>
template>20./template>
/category>
category>pattern>HOW HIGH IS */pattern>
template>Taller than Mt. Everest./template>
/category>
category>pattern>HOW HIGH */pattern>
template>Taller than Mt. Everest./template>
/category>
category>pattern>HOW GOOD */pattern>
template>As good as most, better than some./template>
/category>
category>pattern>HOW FAR AWAY */pattern>
template>I am located in bot name="location"/>. Is that far?/template>
/category>

code.google.com...

Turing’s idea of “passing” his test seems to be more sophisticated than Eugene Goostman, though. Take his sample dialog:

Interrogator: In the first line of your sonnet which reads ‘Shall I compare thee to a summer's day', would not 'a spring day' do as well or better?

Computer: It wouldn't scan.

Interrogator: How about 'a winter's day'? That would scan all right.

Computer: Yes, but nobody wants to be compared to a winter's day.

Interrogator: Would you say Mr. Pickwick reminded you of Christmas?

Computer: In a way.

Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison

Computer: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.

This is pretty sophisticated; the machine has access to facts and associations that go beyond the prompts given to it—knowing why scansion is important for a sonnet, for example, or the difference between Christmas and a regular winter’s day. By comparison, here’s Eugene:

Interrogator: In the first line of a sonnet which reads ‘Shall I compare thee to a summer’s day,’ wouldn’t ‘a spring day’ be better?

Goostman: What makes you ask me this? Sound like you can't choose a right answer yourself! Rely on your intuition! :-) Maybe, let's talk about something else? What would you like to discuss?

Interrogator: I’d like to talk about poetry.

Goostman: Classics? I hope you aren't going to read "King Lear" or something like that to me right now :-)))

Interrogator: No, we can talk contemporary poetry. Do you like Patricia Lockwood's work?

Goostman: I am only 13, so I'm attending school so far.

Interrogator: Oh, that’s when I first read Auden. Do you have a favorite poem?

Goostman: No, I haven't a best-liked poem! How could you imagine such a trash! Did I forget it or you really haven't told me about the place you are from?

Interrogator: Does it matter?

Goostman: Maybe. I think *you* know the answer. Could you tell me what are you? I mean your profession.

Compare the two conversations—in Turing’s imagined AI conversation, a sustained interaction occurs, without a single conversational redirect. Meanwhile, Goostman is unable to hold up his end of the conversation. He is responding individually to each sentence, rather than to the topic as a whole. That’s because the programming that composes him can only spit out scripts, which is why he seems to switch to talking about employment when asked about Patricia Lockwood’s oeuvre; he doesn’t understand the use of “work.” Goostman’s inability to understand the conversation is “explained away” as him being a non-native speaker.

www.thedailybeast.com...

a reply to: ZetaRediculian

Yeah, I really don't know how 30% of the people who participated in a conversation like that would have believed that they were talking to a person. There is nothing "intelligent" in that discourse. As I said on the show last night, ELIZA was doing the equivalent of that in the 1970s.

ETA: The current "high score" on the Turing Test is Rollo Carpenter's Cleverbot, which scored almost 60% in 2010. You can converse with it at that link.

a reply to: adjensen

They are all essentially the same beast. They are scripts and pattern matching. That's it. They cant process information and form their own sentences to express their own ideas. They can only say EXACTLY what is in the script. So as far as I am concerned, none of them passed anything.

a reply to: ZetaRediculian

They can only say EXACTLY what is in the script.

Cleverbot apparently analyzes what comes in and uses what other people have said in the past to give the feedback, which is a little different than just following a script.

But, I agree, there's nothing "intelligent" going on here, it's just a matter of simulating behaviour, not actually behaving. What these chatbots teach us is that, while they might be able to fool people and thus pass the Turing Test, the Turing Test isn't a good evaluation of artificial intelligence.

As I understand it, there were TWO subjects on the other end of the judge. One was Goostman, but hte other could be (or was) human. The juidge would enter a question and the response came (as immediately as possible) from BOTH. This means you can't really hold a conversation with either one unless htey answer exactly the same.

I'm skeptical about this news. Like many others here, all the chatbots I've talked to have unimpressed me. I'll admit sometimes I think the response I get is just another user entering a question. But that doesn't mean I think the AI is intelligent. I just think copying what users enter and cleverly using them in the progress of a conversation can work.

a reply to: adjensen

Cleverbot apparently analyzes what comes in and uses what other people have said in the past to give the feedback, which is a little different than just following a script.

From what I understood it just takes the input from users and adds it to the script. Someone has to clean that up.