As a Google shareholder, I really want to know whether Bing search results are better than Google. And I really, really want to know what the trend is over time.
However I don't think Microsoft's study is any good. They say, "In the test, participants were shown the main web search results pane of both Bing and Google for 10 search queries of their choice."
So it seems that the study participant had to think of 10 search queries, one after the other. But this is not how search works. In the real world users need to perform one query every so often, and their choice of search terms is based on an immediate need rather than what comes to mind.
A good Bing vs Google study would ask 1000 people to use a split-test search engine for their browser search box, then monitor the results over time. I've asked HN about this before and got nowhere:
http://news.ycombinator.com/item?id=3966121
So is there business opportunity here? I imagine that there must be thousands of Google shareholders willing to pay £100 per year for the results of an unbiased, ongoing study into who gives the best search results. In fact, why doesn't Google set up the study and provide such information to shareholders?
However I don't think Microsoft's study is any good. They say, "In the test, participants were shown the main web search results pane of both Bing and Google for 10 search queries of their choice."
So it seems that the study participant had to think of 10 search queries, one after the other. But this is not how search works. In the real world users need to perform one query every so often, and their choice of search terms is based on an immediate need rather than what comes to mind.
A good Bing vs Google study would ask 1000 people to use a split-test search engine for their browser search box, then monitor the results over time. I've asked HN about this before and got nowhere: http://news.ycombinator.com/item?id=3966121
So is there business opportunity here? I imagine that there must be thousands of Google shareholders willing to pay £100 per year for the results of an unbiased, ongoing study into who gives the best search results. In fact, why doesn't Google set up the study and provide such information to shareholders?