Testing Wolfram Alpha, post your results...

I know there’s already a thread on WA but I thought it deserved a new one now it’s up and running.

I don’t know about anyone else but I was pretty excited by the buzz and I was looking forward to seeing how it performed. So far I’m very far from impressed; it’s certainly not living up to the hype.

I decided to give a whirl with 20 questions.

Based on the tips given on the site I asked the following general knowledge questions plus a few calculations. I stayed away from the more esoteric “what is the meaning of life” kind of questions as the site recommends.

Anyway, here are the results. Please feel free to add to them.

- Questions

1. What is the average height of a British male?
2. What will the population of the Earth be in 2010?
3. How many countries are there?
4. What is the largest aerospace company by turn over?
5. What is the GDP of the UK?
6. What is the GDP of France?
7. What is the GDP of the UK divided by the GDP of France? (This was used in a demonstration before the site went live)
8. Where is the nearest Tesco to me? (it knows my IP address so there is no reason it shouldn’t be able to do this if the hype is to be believed)
9. What is the average cost of a house in the UK?
10. What is the average house price in the UK?
11. Who is the current speaker of the House of Commons?
12. Who is Barack Obama?
13. What is Barack Obama’s occupation?
14. What is the 234th digit of pi?
15. What are the odds of flopping a royal flush in Texas Hold'em?
16. What is the distance between London and Leeds?
17. What is the seating capacity of the Albert Hall?
18. How many MPs sit in the House of Commons?
19. Where is Wolfram Alpha based?
20. Who is the current leader of Japan?

- Answers

1. Came up with 5.3 feet which is not correct.
2. Could not decide what to do with the input.
3. Came up with an unclear 238 which contrasts the 195 number found elsewhere.
4. Could not decide what to do with the input.
5. $2.768 trillion. Correct.
6. $2.546 trillion. Correct.
7. $1.087 trillion. Correct.
8. Could not decide what to do with the input.
9. Could not decide what to do with the input.
10. Could not decide what to do with the input.
11. Could not decide what to do with the input.
12. Gave basic information including full name, place of birth and date of birth.
13. Could not decide what to do with the input.
14. Gave Pi to 234 places.
15. Could not decide what to do with the input.
16. 273km Correct.
17. Could not decide what to do with the input.
18. Could not decide what to do with the input.
19. Could not decide what to do with the input.
20. Could not decide what to do with the input.

I also tried a few variations on these questions. For example on the last one I also tried “leader Japan” and just “Japan”. Neither of these gave me the answer I was looking for.

Out of 20 questions I got six sort of satisfactory answers. Overall though I think I’d be better off with wikipedia and a calculator. Very disappointing.


Please post your results.

I haven't found it very useful either. The thing I find most annoying is that the answer text is rendered as an image and you can't copy and paste it. Maybe that's why it's so slow.

well I can confirm that it does not give winning lotto numbers

I've done a search on "the meaning of life, the universe and everything".

Yes, it is what you would expect it to be.

reply to post by Mike_A


The button that you click on is = (equals), so you should modify your question accordingly to avoid Alpha having a hard time to understand your question. Don't include pronouns like when, what, etc in the input. Just type in all nouns that are particular to your question. If you want to know something particular about Barack Obama, just type in the name. The engine returns all info about Barack Obama that is available in Alpha database.

In many respects Alpha is vastly inferior to the input that you hit other search engines, and that is the subject followed by "wiki." For example, if you input Alpha with "tangent, the engine returns an answer regarding tangent as a function; it doesn't expand to tangent line or tangent point. If you google up "tangent wiki," you get exhaustive info regarding the query "tangent."

Alpha has its own database that is only a tiny fraction of what is available on the Internet. Be careful not to misspell your input. Alpha doesn't recognize misspelled words the way google does and lets you know with the "did you mean . . . ?" option. Look at the difference between "stratosphere" and "strattosphere" when asking Alpha.

I bet you that the traffic to Alpha will slow down considerably within a month or so despite all that hype and hoopla.

reply to post by Avarus


lol just tried it. Shows you where their priorities are.

All I can say is I'm glad I didn't take all the "next Google!" hype to heart and invest my life savings.

reply to post by stander


reply to post by stander


Like I said I tried reducing most of the questions and still got unsatisfactory results.

The way it was sold though made it seems like an intelligent data miner that could interpret a question rather than just search for keywords and extrapolate a cogent meaning. But it just can’t do that.

It’s useful if someone can type in an intuitive question like “where is the nearest dentist” or “when will Star Trek be released” and have the answer. If it can’t do this then it’s just a glorified calculator and a pretty limited encyclopaedia. I’m struggling to see a use for it, it’s too limited, too pernickety, and not really very accurate.

Of course this is it’s first day so maybe it’ll get better but I don’t hold my breath.

I agree that traffic will slow, personally I think it'll dry up completely.

What an incredibly bad way to start out.
First of all ease of use is not very good.
In order to get anything of use out of the search result. I had to do a basic 2+2 and it gave me four. Then I tried the airspeed of a swallow.
Humorously enough I got a Monty Python reference.

All in all. I'd rate this SE a 1 on a scale of 1 to 10. IMO it's nothing more than a glorified calculator. It needs alot of work still to get it to be easy to use and actually useful.

It's definately not a search engine.

Well it's kinda cool how you can put in complex math problems and it does it for you
I can see this helpful for school.

Originally posted by grey580
All in all. I'd rate this SE a 1 on a scale of 1 to 10. IMO it's nothing more than a glorified calculator.
It's definately not a search engine.

Woah you must have had such high hopes, i most certainly would give it atlest more then 1. Its a good program just dont expect it to know the future. It has great simplified answers for many questions you put in.

reply to post by Mike_A

Regarding your question #2 that couldn't be understood by Alpha. Just follow what I said in my previous post concerning the way you input Alpha.
Just type in "human population 2010." The input interpretation says "global human population 2010" and the answer is 13.62 trillion people!

A trillion?
That's how Alpha "computes" the answer.
My verdict: Alpha = unreliable junk.

Well here's the issue. Touting this as a Google killer when it's not doesn't bode well.
Google really set the bar pretty high with their search engine.

And to get users to stop using google and use a different search engine is pretty hard. Google does what it does well and simply. Asking users to change their way of doing things is not an easy thing to do. The average user is going to say why should I change when google does everything i want it to do.

If this SE can survive and improve the way the search is handled. Then maybe it will do well. maybe.

reply to post by jman1a


Can you give some examples of the questions it can provide answers for?

I’ve been asking it more and the only things it can reliably answer are fairly simple general knowledge type questions or very well defined mathematical equations. Both of these are better answered by Google and a calculator.

It’s even struggling when it comes to its own examples. In its examples page it offers “weather forecast Paris”; it did this fine but then I expect it to. However when I tried “weather forecast Teeside” (and Teesside) it didn’t know what I was talking about.

The developers claimed that WA would be able to understand a properly formulated question but it can’t, as the population 2010 illustrates.


I’m struggling to see the market for the site. Who would use Wolfram Alpha and for what?

reply to post by Mike_A


One question i asked was "What is the world population?" and it gave me a good amount of information about the number of the population and density and life expectancy. So it answered what i wanted to know. Another one i asked (simple but just for a test) was "Who is the president of The united states?" It gave me the answer and dates of his predictors and their terms of office and basic info about the man himself. Im not saying its super perfect but its a good tool for students and people who just want a quick answer. So thats why i think its justified a better ranking then 1.

[edit on 17-5-2009 by jman1a]

reply to post by jman1a


But that’s the point, those questions are very, very simple and in general pretty well known anyway. If you don’t know them then you’ll find the answer very quickly on Google or other existing websites. What has WA got going for it that necessitates its existence?

I tried your question about the population of the world and all I got was Result: 6.53 billion which itself is pretty outdated.

A Google search gave me this:

en.wikipedia.org...

Which has a hell of a lot more information and is well sourced allowing me to decide for myself the validity of the content.

I did get a detailed answer when I just typed in “population” but if someone wants to know the population of the world then they’ll probably type something along the lines of “world population” not just population; again very pernickety. However I’m dubious as to how accurate the information provided is given that WA claimed the average man in the UK is 5’3”

On very limited subjects it can provide a quick answer but it doesn’t do it any better than existing services and in more obscure cases it completely drops the ball.

Using Google to search for answers to my original questions I found them on the first page in the cases of questions 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19 and 20.

That’s 17 out of 20, WA scored 6.


If you want a quick answer then Google is still your best friend.

Well as long as we are not hearing " Yes, Dave", I think we will be ok . I do not see this taking over google. Google is to BIG to fail

Originally posted by jman1a
reply to post by Mike_A

 

Another one i asked (simple but just for a test) was "Who is the president of The united states?" It gave me the answer and dates of his predictors and their terms of office and basic info about the man himself. Im not saying its super perfect but its a good tool for students and people who just want a quick answer. .

Well, the pot-smoking students of Fasttime High would surely find WA useful to brush up on the civics.

I believe that we are heading toward the conclusion that for a general purpose, WA is inferior to Google. Here is an example that understrikes the conclusion:

WA
Q: the largest volcano in the solar system
A: Wolfram Alpha isn't sure what to do with your input.

So let's take the same question to Google . . .
Clicking on the first item returned gets you the answer pretty quickly -- with a pic:
www.universetoday.com...

But if you ask WA to "differentiate x^2 + 3^x," the query goes straight to Matematica and you get the answer. Google can't retrieve an answer for such an esoteric question, especially when this function doesn't have some wide-spread usage.

WA is a specialized and very restrictive engine; it can be helpful to some but completely useless to others. I think it is a very poor general-purpose SE.

[edit on 5/18/2009 by stander]

You stole my thunder... Sadface.

Thread Here

I asked it "The meaning of life" it gave me the answer 42.

Originally posted by stander
reply to post by Mike_A

 

Regarding your question #2 that couldn't be understood by Alpha. Just follow what I said in my previous post concerning the way you input Alpha.
Just type in "human population 2010." The input interpretation says "global human population 2010" and the answer is 13.62 trillion people!

A trillion?
That's how Alpha "computes" the answer.
My verdict: Alpha = unreliable junk.

No it's just predicting the coming alien invasion.
Or maybe its counting the mole people?
What about ghosts?!
Is that the added up amount of humans ever to live on the face of the Earth?
Maybe it mistook humans for Ants...