For a system like this, I've been trained through repeated disappointments to ignore hype and only look at results. I might also be influenced by knowing a bit about AI.
it doesn't use natural language processing, it *computes* the answer.
mathematica is a reimplementation of similar systems that were already proven to work. alpha is a reimplementation of similar systems that were proven not to work.
Ah, the "witty put down" at its finest! While nothing of this sort has worked to date, the same could have been said of powered flight on December 16, 1903. In any event... Thanks for making me smile :-)
But the more interesting question is "why is that?" And the answer is that you've used Google for so long that you've thoroughly internalized its capabilities that you think in Google and can't easily see outside of those capabilities.
In the programming language context, we call this the Blub paradox, but Blub happens everywhere, and is hardest to see when there is nothing to step up to.
If this works (a big if, IMHO, but I'm willing to give it a try), what might happen is that it might entirely recalibrate what queries you can conceive of. Until we see the system, though, it's hard to even begin to imagine what those queries might be.
How many queries in 1980 did people wish they could query Google for? Nearly none, not because people didn't have questions Google could answer, but because only a bare handful of visionaries could even conceive of Google.
Awesome point. It's also why any serious contender to Google won't really look like Google at all.
Because if it looks like Google, then users will interface with it like they're used to interfacing with Google, and then you really have to win by beating Google at its own game. Much better to pick your own place & time.
You can already ask such questions on Google and get back answers in many cases in the web page snippets. Try this query for example: how many bones are there in the human body?
So if you don't normally issue queries of this sort on Google even though you would probably get back an answer in the web page snippets, then perhaps you would not find this new system all that useful.
Google is great for finding random data, but the results generally require human perusal. You couldn't feed them into a running program without at minimum a hand-written screen scraper.
"Gibberish" - why? Many of the commercially used products that deal with representation of natural language use naive bayesian inference or statistical classification to return results.
Further, one of the promising approaches in this area involves using generative grammars (or other generative, non-parametric approaches) to approximate natural language representations.
Both these approaches 'compute' answers without a notion of natural language grammars that are usually associated with natural language processing.
I've definitely read papers about using those methods you describe on processing text. They are all just algorithms attacking a problems, so the distinction between NLP and computing is gibberish.
Yes, I agree that all NLP must include some computational model. However, it is an interesting distinction to the reader of this article that the natural language engine is not based off of a linguistically derived grammar - like Powerset, and many of the larger, more notable NLP efforts.
This is the point the author was making. Had you not paraphrased the article to skew the intended meaning of this observation, it would have taken on a different meaning.
The actual text: "It doesn't simply parse natural language and then use that to retrieve documents, like Powerset... Instead, Wolfram Alpha actually computes the answers to a wide range of questions"
Computing the answer for 'what does a string of natural language mean' and 'what is the intended answer of the question being asked' are 2 different things.
I see the distinction, but I think the review is light on content, so the criticism is fair. Why, pray tell, do we need to be hyped about a great new piece of software?
If it even close to real, the results will certainly speak for themselves. And the results might even be really awesome! I just would advise people to not pay attention until they see them.
I'm sort of assuming that its going to be an extension of the online data sources that Mathematica introduced a little while back, with perhaps a natural language interface.
Numenta makes software that is used in (prototype) missile systems. The vision systems in missiles and aeronautics generally are awesome. They are much closer to practical, deployable software than this sounds.
Agreed. Plus, I doubt the companies using Numenta's sdk want it advertised that they are using it.
If you have not tried their sdk, I highly recommend it. It is open source(not free though). You can play around with it for free but if you want to use it commercially you need to pay.
I have taken a look at the website. Apart from hype there is nothing that I have seen that makes me want to use it for any AI (which I do a lot of commercially).
How does it compare to other architectures? What sort of problems does it excel on? Where are it's weaknesses? What is the training like compared to other methods?
I wouldn't really take them seriously until a paper is linked to from their front page explaining the above. At the moment it just looks like hype.
I had a chance to see this in action a while back. While I, and none of the people I saw this with, were not at all impressed by NKS, this project blew our minds. We watched as it pulled up and manipulated everything from Egyption fraction expansions to historic weather data to the human genome. If the author of this article is exaggerating, it's not by a whole lot. While Wolfram may not be bringing about the revolution in science he hoped to, don't forget that he and his crew made Mathematica, and are very capable of creating impressive software.
Yeah, I know it looks suspicious. I've been lurking around here for a while, but hadn't really been compelled to comment before. I'm happy to try and answer questions about what I saw, but it was a while ago, so I don't remember the details that well.
The demo was very impressive, but in practice they just couldn't match the scope of Wikipedia or the wider Web. It may be the same way with Wolfram|Alpha.
> When Google came out they didn't make any claims. Only factual performance counts.
A cite to some Google claims when they came out would be nice. (Academic papers don't count.)
I was in the SF Bay Area at the time and the "publicity" that I remember was friends saying "check this out". I don't recall Google saying anything beyond "here's how many pages we indexed".
If all Wolfram will be spitting back is an answer, I'm wondering how the user will determine the answer's correctness. Will there be a "proof" of some sort, or a list of references for the underlying facts and assumptions. With information found via Google, at the very least you'll be able to assess the reliability of the author/source (random message board v. NYTimes article) - not saying this is perfect, but a good measure.
Let's not get too excited here. Generally, stuff that's hyped up before it launches tends to suck.
Was Twitter hyped up like this before it launched? Facebook? Google? Microsoft? Apple? TechCrunch? Hacker News? Wikipedia? Heck, pretty much ANYTHING that's successful now? (Even small stuff like Balsamiq that's currently very successful in a small way wasn't hyped before it launched).
Now think of stuff that was hyped massively before launch. Cuil. Powerset. Yeah.
Stuff that ultimately becomes super successful becomes successful over quite a long period of time and due to the excitement of users after launch - not the bleatings of gurus before launch.
Or it could be as unimportant as Powerset. It's best not to hype it up too much since the odds are that it won't be a panacea to everyone and a lot of people will be disappointed.
Products that tend to be modest initially and improve and prove themselves rapidly tend to do better than products that are hyped up beyond all proportions.
Indeed, why hype a product that is so insanely great the news will spread practically by itself? I expect a product to be worthless if its hype goes beyond a certain treshold.
I said something similar about A New Kind of Science, and that was ridiculously underwhelming. I respect Wolfram like crazy, but I want my money back on that thing :)
Do you feel that Wolfram can rightly claim the thesis offered and explored in the book? Regardless, how well do you think he supported the thesis?
The thesis of A New Kind of Science is something like "systems comprised of a small number of simple rules can perform arbitrarily complex computations."
The book proceeds to support the thesis. The content is comprised of descriptions of such systems, corresponding Mathematica execution trace diagrams, and analysis. These analyses are related to a ambitiously large scope of natural phenomenon and scientific knowledge.
Knowing no more about this than the PR thus far, I am pessimistic, but for not quite the same reason as some other commenters.
I think a couple of things are clear.
(1) We are at the point where something impressive is likely to be able to be produced, and Wolfram may very well have the resources to do it.
(2) We are not at the point where the be-all-end-all version of this can be produced.
Compare this with the symbolic computation packages (Mathematica, Maple, etc.). Around 1990, we were at the point where we could produce a very good one. Several were written. They have been improved since, but only marginally. We're still pretty much using 1990 technology.
And that's fine. We knew how to make a really good symbolic computation package. We did. End of story.
But consider the proposed packages (Alpha, etc.). We might produce something impressive. But we are not ready to produce something reallygood and useful. Our initial efforts will require lots of improving.
And Wolfram is definitely not the one to do that improving. He runs an aggressively closed shop. Always has. I predict, therefore, that the cathedral-bazaar effect is going to mean his product will be difficult to improve, and so will never become truly useful.
This too is my concern. The guy is brilliant, but I really feel technology like this can only reach its full potential by being open and extendible by domain experts. Hopefully Wolfram realizes this as well -- it sounds like he has put forth significant effort and money bootstrapping the engine with knowledge and so on, hopefully he passes the torch to the rest of the world and doesn't propose his company be the only source of information into this engine.
If he provides not just technology and data but also the means to extend that technology and data by following his example he might be contributing something truly revolutionary.
Excellent article describing what could be the biggest advance in the web since the launch of Google. However, I wonder if it will be inundated with 99% of the questions being about who Miley Cyrus's current boyfriend is - or some other frivolous usage.
Seems to me that this technology should have been released for some other scientific usage first (if it is indeed that powerful). It could be valuable as an engine for other applications as well in this manner.
I would also argue that one of Google's advantages is that it enables discovery of new information instead of just giving you the one page you for which you are looking.
Successful search engines are innundated with questions about what its users care about.
> Seems to me that this technology should have been released for some other scientific usage first
Why?
Technologists need to get over the idea that technology is for science/technology.
> I would also argue that one of Google's advantages is that it enables discovery of new information instead of just giving you the one page you for which you are looking.
Huh? If there isn't a page that states which city is the fourth largest in eastern Montana, how will Google help you answer that question? (No fair going to the "populations of cities in eastern montana" page.)
Sounds a lot like what cyc (http://www.cyc.com/cyc/company/about) has been trying to do for the last 25 years and actually the holy grail of AI since the 50s.
I think its still way out of reach for non-trivial data-sets. Something like this doesn't just show up out of the blue, its not a problem amenable to some single new algorithm or breakthrough.
I don't think a formal system with symbolic inference is useful for describing the knowledge of any reasonably complex domain that doesn't have a mathematical model. And most of the human knowledge tends to be like this.
I'll add another tautology to counteract your argument: isn't it true that any domain that can be described by a formal system is essentially equivalent to mathematics?
Which brings us back to the original point: human stuff is not very math-friendly. I want to deal with emotions, politics, etc.
Think Bush and torture: can this system give me any definitive answers?
And when Joe started doing something new, you'd just extend your formal system to include all of Joe's new behaviors. Given enough symbolic content, no matter what Joe comes up with, you can model it.
Just like we extended our formal number system into negatives, imaginaries, quaternions, etc.
Math is a terrifically abstract, self-consistent model of reality. But that's all it is: a model. Sometimes the model tells us things we didn't know before, and sometimes we have to change the model to make it work with what we're observing.
I don't think so. My assumption is that Joe is predictable by a mathematical equation. I find that this is only true of certain people and only true in certain circumstances.
People, in general, operate the opposite of computers. We don't think about what we should do, we think about we shouldn't do. So, it is very hard, if not impossible, to represent human behavior by a mathematical system.
This argument is explored in great detail in the book: Godel, Escher, Bach.
I think that's where the "restricted to domains where everyone agrees that there is a factual answer" comes in. (I'd like to have a system answer me if I really violated any traffic laws when I got a ticket last week, but I don't think that's going to be very easy...)
"Wolfram Alpha is not HAL 9000, and it wasn't intended to be. It doesn't have a sense of self or opinions or feelings."
Too bad for that because right away I was thinking "Wow! It's a sentient version of Google only a bazillion times better!" but then I realized it's just a parser that turns natural language questions into queries against a large dataset then I became all sad and disappointed.
How will they prevent malicious questions such as prime number factorization, np problems, from eating all processing time??
Im asking it seriously!
At least they have to enumerate all these questions to prevent system abuse.
Funnyly "The Last Question", Asimov's short story also comes to my mind.
I wonder how useful this will be if you can only ask a single question or a set of questions that can be easily expressed in single line text field?
If I'm asking something like, "What is the capitol of Nebraska?" why not just get directed to the Wikipedia entry where I can learn a lot more than the one fact that answers what I just asked?
If Alpha is actually going to do computation, I'd rather be able to use it for something more complex than a single natural language query.
I would love to gather good questions and discuss the results when they are available. I think it is important to find questions to which google, yahoo, powerset or wikipedia don't provide a straight answer.
How about:
1) What is the smallest unknown prime number? ;)
2) Where on earth is the rainy days to sunny days ratio the lowest?
3) How many languages does the average person from the Benelux countries speak?
The other big question: will they hardwire it to come up with "42" for "the answer to life, the universe, and everything" or will it actually source that "fact" from the web?
If you ask Google "What is the capital city of the country that has the 15th highest average rainfall in the world?" then you don't get a straight answer.
This software should be able to look up rainfall data from Stephan Wolfram's Bumper Book of Trivia, work out average rainfall for each country, work out which country has the 15th-highest rainfall from that result, then look up the capital city for that country.
All determinable facts with a straight answer; you should simply get the name of the city as a result.
I tried the query and the top of the list is the parent comment. (Google seems to be paying close attention to HN.) The second was an article about Uruguay which said it was colonized from Spain in the 15th century. Google replied with any old 15th. (The article is actually wrong. It was settled in 1536.)
The parent posting is correct. Google is particularly poor at complex questions, especially if you aren't sure about the exact phrasing. I have had several queries in the past couple of weeks where I have had to spend 30 to 90 minutes trying to get the right set of terms to return the right set of documents to look at. This is particularly true if the name of the product is a common English word.
This is as you say a fact, something you can look up in a geographical info database. As I understand it Wolfram's goal is to also answer questions which require computation and application of laws or formulas, or inference.
If there is only one answer to one factual question, why not have it already written and available like wikipedia?
Practically speaking, static content that doesn't change often is better served by models like wikipedia.
If Wolfram knows all the answers, write them all in static html for the world to use, search, browse, replicate and extend instead of stored on semantic databases or ethereal brains.
I am not pissing on their parade, I know the scientific work is commendable, but practically speaking it can't compete with more efficient models.
Gödel might have said something about the possibility of a universal inference engine.
Like TrueKnowledge and the Freebase answers in Powerset, this system will likely be good at answering a small subset of very direct questions. Having access to Mathematica's symbolic solver algorithms would definitely help in building this system.
If it's successful it will either be faster than current inference engines, or capable of solving more complex queries. Or perhaps both. We'll see.
