May 19, 2009

Wolfram Alpha: Bad Idea!!

I will write about this later, but I will just note this for the moment: Wolfram Alpha is not just not ready for prime time. It is an intellectual travesty. The idea of jumping from concepts to numbers without any attention to how the numbers are produced and how they can or can’t be compared is simply anti-intellectual. It is incredibly irresponsible and a symptom of a disease, a disease of non-thought and non-reflection. If Wolfram Alpha became “better” to the point that it began to be relied upon for actual practical purposes, the result would be dangerous to society, culture, the environment, and the human soul. It is ironic that it’s coming out now, at a time when the financial system is in tatters because a similar breed of quantitative ideologues (bankers) played with numbers while ignoring the reality behind them. It led to disaster and it will lead to disaster again unless we can learn from it.

Wolfram Alpha is a big mistake from the start.

More later.

Share on Facebook

20 Comments »

  1. This is all part of that kind of new-agey dude’s work, right? That’s a suspect starting point, in my ill-educated opinion.

    Comment by syntaxfactory — May 20, 2009 @ 7:35 am

  2. It doesn’t strike me as being a new agey problem or a hippy dippy problem, except in the sense that there is a huge naivete involved.

    Comment by Rory Litwin — May 20, 2009 @ 7:49 am

  3. Kramer auto Pingback[…] a comment   libraryjuice2 19 May 2009 @ 11:12 pm Wolfram Alpha: Bad Idea!!   http://libraryjuicepress.com/blog/?p=1391I will write about this later, but I will just note this for the moment: Wolfram Alpha is not just […]

    Pingback by Friends — May 20, 2009 @ 8:36 am

  4. He’s not new-agey; he’s a science/math nerd with a big ego. He *has* accomplished a lot so maybe the ego is justified. His huge book about cellular automata was called A New Kind of Science, which is a bit of an exaggeration (CA is a useful niche within computer science but nothing as revolutionary as he claims, in my humble semi-informed opinion).

    WolframAlpha disturbs me too but I hadn’t thought about exactly why yet. Rory, I think your comments are on target. WolframAlpha is “Making the World’s Knowledge Computable,” which presupposes that this is possible and that it’s a good thing. It’s a naive view of knowledge, though I’m sure Google’s view is similar.

    Comment by Kevin Arthur — May 20, 2009 @ 3:23 pm

  5. I see your argument. I am all for people being thoughtful and reflective and I agree that there may be less of that these days, unfortunately.

    But saying this tool is an intellectual travesty is like saying calculators are intellectual travesties. We do not question a calculator when it tells us 31*14=434. A calculator simply does these computations better and quicker than us.

    The same is true for a tool like Wolfram Alpha. For example, it is able to find the GDP of China and the US from public data and then compare them side by side giving us numbers, graphs, and charts. It even allows to look at the source information for its data, which as a librarian I see as very important. A human, though, finding this information and then computing it would take considerably longer.

    This is why we have designed computers, so they can do the grunt work for us and allow us to be reflective about the bigger ideas that computers can’t tackle like “what are the implications of these numbers and comparisons on our citizens, trade, economy, etc.”

    Thanks to computers we are modeling all sorts of new information and accomplishing feats that humans could never do on their own, like mapping the genome or space travel.

    I don’t think computational tools make us less reflective. They allow us to reflect on more important things.

    Comment by Andy Burkhardt — May 21, 2009 @ 7:41 am

  6. Andy, you’re missing the point. The labor that Wolfram Alpha would be saving us is the labor of examining how statistics are produced so that we can tell how they can or cannot validly be used together for computations. GDP is computed differently in different countries at different times, and calculations need to take this into account (to follow your example; there are a million others). Accepting a number based on a keyword, with no examination of how the number is derived, and then using that number in a computation involving other numbers brought out of the database in the same way is just stupid. It really is anti-intellectual and it really is an intellectual travesty. And I’m not merely talking about ivory-tower standards. I’m talking about a problem that could have dire consequences if this tool were ever used for a serious purpose, to make decisions. It seems like this is a lesson that we have just learned, with the explosion of mortgage-backed securities….

    Comment by Rory Litwin — May 21, 2009 @ 8:13 am

  7. I understand that you are talking about questioning where the information comes from and how they arrive at these statistics. I agree that this is necessary when making big decisions. That is a big part of information literacy.

    But I also think that doesn’t make Wolfram Alpha an intellectual travesty. Wolfram Alpha allows people to access information that they otherwise would not be able to find (or at least not very easily).

    Wikipedia has information that a person might not be sure where it came from, but that does not make it an intellectual travesty or any less useful. It is a good starting point, and finding out more about the sources and questioning it further is just a deeper point in the research.

    Having more tools at our disposal does not seem to me to be a bad idea. I think the bad idea is people not questioning and digging deeper into their information.

    Comment by Andy Burkhardt — May 21, 2009 @ 2:36 pm

  8. My thoughts exactly, Rory. What would Edward Tufte say!? I think he needs to weigh in on this.

    Comment by Ramona — May 21, 2009 @ 3:10 pm

  9. Andy, it’s not just about locating information, it is about using it to do calculations. As the site says, “Today’s Wolfram|Alpha is the first step in an ambitious, long-term project to make all systematic knowledge immediately computable by anyone.” The basic problem is there, in the idea that knowledge can be immediately computable. The notion of doing that effaces all of the choices, often incompatible choices, that go into the gathering of quantitative data in the first place. It is an important principle of using quantitative information that anybody who has studied statistics knows – you have to consider what is behind the data and how it can be compared or not compared to other data. I am not saying this as an anti-quantitative person – I think quantitative data has its place. But that’s precisely it – it has its place. The idea of a search engine that returns just data and not methodology, and then uses the data in computations without any analysis involved, flies in the face of any thoughtful approach to information.

    Comment by Rory Litwin — May 21, 2009 @ 3:25 pm

  10. Repeat after me, “It is my responsibility to be the thinker in this human computer relationship. It is the computer’s responsibility to compute.”

    That means, as the human, you have the responsiblity to take the computed knowledge and consider:
    —-What is the source of the data? Don’t know? How much does it matter? If it’s important enough, look it up from another source. Or go gather the data yourself.
    —-What is the methodolgy used to produce the result? Don’t know? How much does it matter? If it’s important enough, look it up from another source. Or derive it yourself. etc…

    A travesty is when people blindly believe what they are told. It doesn’t matter if a machine does the telling or a person that has thoughtfully conveyed a lie.

    A sophisticated calculator is not a travesty. It is a sophisticated calculator.

    Comment by Michael Mersic — May 21, 2009 @ 7:53 pm

  11. Michael – The average web user will accept that the numerical “facts” are as advertised and won’t ask questions about how concepts are defined for quantitative purposes. The assumption seems to be that if they don’t know enough to look at their information critically, then their uses don’t need to be taken seriously. I have two objections to this idea as a justification for Wolfram Alpha. First, we don’t know when people who are less than information literate might use this search engine as the basis for decisions that are important in their own lives. My second objection to that kind of assumption is that it misses the fact that Wolfram Alpha intends to be just the beginning of a technology that would eventually be refined and put to serious use on a larger scale. The problem at its core is not a consequence of its not being ready for “prime time” but a problem with the concept itself, and would still be there after decades of development. Anybody who has worked with statistics knows that you can’t blindly make computations without looking at what the data actually is.

    Comment by Rory Litwin — May 21, 2009 @ 8:15 pm

  12. I completely agree, Rory. The calculator analogy, of course, is meant to cast any alarm about a technology like Wolfram Alpha as alarm-ist. But it’s disingenuous, because the potential for mischief inherent in something like WA is several orders of magnitude higher than that in a calculator. And calculators really do make your brain that much more lazy.

    The perception of WA as harmless, or at least faultless, hinges on the idea that tools are inert—that the relationship between user and tool is one-way, and that tools don’t affect users, except on an individual basis, in which case that individual’s irresponsibility is to blame. But in this day and age, that also seems very disingenuous. Successful technologies use convenience to entrench themselves in society, and the most successful actually change society so as to make themselves indispensible. Look at cell phones, look at Google, look at the automobile. WA is obviously looking to be a similarly game-changing technology—if it succeeds it will change the way research is done—period.

    Wolfram Alpha is a way of—benevolently, helpfully, and cheerfully—enclosing what was an information commons. Not, obviously, by restricting access to information, but by centralizing and offering “value-added” information of such utility and with such ease as to make the alternative—first-person research—no longer viable.

    Comment by John — May 27, 2009 @ 11:51 pm

  13. […] Nei, svarar bibliotekaren og bloggaren Rory Litwin. Wolfram er ifølgje han ein ”bad idea” og ein ”intellectual travesty”. Tenesta er […]

    Pingback by Wolfram – eit framsteg? « Se også — June 5, 2009 @ 4:28 am

  14. Kramer auto Pingback[…] Litwin makes a quite different and critical approach: http://libraryjuicepress.com/blog/?p=1391 […]

    Pingback by Wolfram Alpha and Libraries | ALA TechSource — June 5, 2009 @ 11:31 am

  15. Kramer auto Pingback[…] positive take on Wolfram|Alpha over on the ALA Tech Source blog, while Rory Litwin calls it “an intellectual travesty” for its presentation of numbers and stats without proper attention paid to their source. Comparing […]

    Pingback by Academic Newswire — June 15, 2009 @ 4:32 pm

  16. Check out what this Chemistry blogger found out about Wolfram Alpha, at http://usefulchem.blogspot.com/2010/01/dangerous-data-lessons-from-my-cheminfo.html.

    Comment by Ramona — January 19, 2010 @ 2:18 pm

  17. Thanks, Ramona – that is a really interesting and useful post.

    Comment by Rory Litwin — January 19, 2010 @ 3:06 pm

  18. Kramer auto Pingback[…] Rory Litwin, Library Juice […]

    Pingback by » Wolfram|Alpha Information/Science — April 14, 2010 @ 8:56 am

  19. Kramer auto Pingback[…] What's that, you say? We already have such a service? Well, for all the fears that Google is making us stupid by making it too easy to look up information, at least Google and its rivals enable the critical thinking that comes from scoping out multiple sources. Unlike search engines that deliver links that match keywords in your query, WolframAlpha is more of a black box. If you have it perform a calculation, it gives you an answer, along with a small link for "source information." Open that and you'll generally be told the data was "curated"—found and verified—by the company behind WolframAlpha. In other words, "trust us."The site does suggest ways to track down similar information from other sources, including government statistics, proprietary databases, almanacs and the collaborative encyclopedia Wikipedia. To confirm WolframAlpha's data I went a suddenly old-fashioned route—through Web searches on Google and Yahoo. I didn't find any errors, but taking that step made me wonder why I didn't just use Google or Yahoo to begin with.Link: Review: Flaws in Web's much-touted WolframAlpha.Rory Litwin has posted similar comments at Library Juice: Wolfram Alpha: Bad Idea! […]

    Pingback by Question Technology: Education — November 24, 2011 @ 7:41 pm

  20. Kramer auto Pingback[…] (Source: Slashdot) See also: Wolfram Alpha: Bad Idea!! http://libraryjuicepress.com/blog/?p=1391 See also: Wolfram|Alpha – computational is the key word! […]

    Pingback by Welcome to The Rowland Institute at Harvard — January 6, 2012 @ 6:25 am

RSS feed for comments on this post. | TrackBack URI
You can also bookmark this on del.icio.us or check the cosmos

Leave a comment

XHTML ( You can use these tags): <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> .