Machine Learning and Friends Lunch





home
past talks
resources
conferences

Challenges in Running a Commercial Search Engine


Amit Singhal
Google

Abstract

The real world poses unique challenges for search algorithms. First, they operate at unprecedented scales and over a wide diversity of information. Serving millions of queries against billions of documents in fractions of a second poses unique system design challenges. Second, they must serve the masses, many of whom find it hard to formulate effective queries. This raises the challenge of interpreting what the user really wants given what they actually said. Finally, we have entered an unprecedented world of "Adversarial Information Retrieval". The lure of billions of dollars of commerce, guided by search engines, motivates all kinds of people to try all kinds of tricks to get their sites to the top of the search results. How can a search engine function effectively in this adversarial environment? In this talk we will discuss the above challenges, what Google is doing about them, and what are some of the open problems.

The world of a commercial search engine can be described by two of Murphy's Laws: a) If anything can go wrong, it will, and b) If anything cannot go wrong, it will anyway.

Dr. Singhal is a Distinguished Engineer at Google, with a research focus on information retrieval (IR). Recent IR research has included web search, speech retrieval, document ranking, question answering, document routing/filtering, and automatic text summarization.

Back to ML Lunch home