Ad verba per numeros

Swarm Information Foraging (i.e. another way of exploiting crowd intelligence to improve search engines)

Wednesday, November 25, 2009, 01:05 AM
Last friday I took notice of Wowd; it is, in their words, "a real-time search engine for discovering what's popular on the web right now". Wowd exploits crowd intelligence in a really smart way: first, there is no crawler, those pages visited by the users are submitted to the index and, secondly, ranking is determined from the attention the users pay to each page.

All of this is somewhat related to a paper we have under review at this moment and, thus, we decided to release a draft report as a preprint:

Making the road by searching - A search engine based on Swarm Information Foraging by Daniel Gayo-Avello and David J. Brenes. Abstract: Search engines are nowadays one of the most important entry points for Internet users and a central tool to solve most of their information needs. Still, there exist a substantial amount of users' searches which obtain unsatisfactory results. Needless to say, several lines of research aim to increase the relevancy of the results users retrieve. In this paper the authors frame this problem within the much broader (and older) one of information overload. They argue that users' dissatisfaction with search engines is a currently common manifestation of such a problem, and propose a different angle from which to tackle with it. As it will be discussed, their approach shares goals with a current hot research topic (namely, learning to rank for information retrieval) but, unlike the techniques commonly applied in that field, their technique cannot be exactly considered machine learning and, additionally, it can be used to change the search engine's response in real-time, driven by the users behavior. Their proposal adapts concepts from Swarm Intelligence (in particular, Ant Algorithms) from an Information Foraging point of view. It will be shown that the technique is not only feasible, but also an elegant solution to the stated problem; what's more, it achieves promising results, both increasing the performance of a major search engine for informational queries, and substantially reducing the time users require to answer complex information needs.

Curiously enough, a few hours after the preprint was available I received an interview request from a French journalist which ended in this description of our research (in french).


Back Next