David Green BA (Hons), PgDipLIS, MCLIP    
Home CV Publications

Research Papers

Events Useful Links

 

Section five - Search utilities and Intelligent agents

Online Information Review - Apr 00 - The evolution of web searching

My tag cloud - semantic web and ai links

As already discussed, meta-search sites such as Dogpile and Mamma have grown in popularity as they allow users to search across different search indexes simultaneously with results also de-duplicated and re-ranked (depending on the meta-search service used).

Search utilities represent the logical evolution of this functionality. Unlike meta-search engines, where the processing power to refine results still remains on the server the user is interrogating, search utilities are programs that are installed onto the user's hard drive. By shifting processing power away from the server, and onto the user's own desktop, search utilities offer a much greater range of search and results analysis functionality.

Like several of the second-generation search technologies that have emerged (Electric Monk, Google) many of these search utilities incorporate intelligent agents (or bots). Indeed, many of the powerful features offered by search utilities, such as language independent searching, filtering, automatic refinement of results and document summaries, active hyperlinking of query words and live highlighting of search terms are possible because of the nature of intelligent agents. Unlike a standard software program that will execute specific functions within clearly defined parameters, agents/bots:

  • are adaptive - they can interpret monitored events to make appropriate decisions

  • are self-organising - they assimilate both information and experience

  • can communicate with both the user and other bots (12)

Agents can search across a wide range of document types and formats. They can provide a uniform interface for search queries across different sources and are true 'infomediaries' in that they can identify and search appropriate resources that may or may not be known to the searcher. The adaptive element of intelligent agents is central to the functionality of many search products that incorporate agents. The following popular search utilities, which all contain agent technology, are available as free downloads and as more comprehensive 'pay-for' versions:

Mata Hari can learn one set of power search commands and then automatically translate these for each search service/database that it queries for the user.

BullsEye Pro (Intelliseek) incorporates eleven different intelligent agents, including technology from Verity to conduct what it calls 'Web mining'. The different agents are used to target specific types of information such as business news in over 450 sources on both the visible and invisible web. It will automatically run searches, allows import/export of searches to other users, whilst users can chose to receive change alerts by HTML email, pager or other hand-held data devices.

Copernic can translate a search statement for different services and then simultaneously submit the query to these search engines, web directories and databases. There are also about 20 categories such as Business and Finance, Science etc. with predetermined web sources to search in.

Recognising the advantages offered by search utilities, some search providers have released a variety of free basic search utility programs as 'plug ins'. As the name suggests, once installed, they are incorporated within the user's web browser and enables the search engine provider to offer more features. Search providers that have released search utilities include Infoseek (Infoseek Express), AltaVista (AltaVista Discovery) and more recently Lycos (See More).

A common function of agents is that they allow the user to specify a high-level goal instead of issuing explicit instructions - leaving the 'how' and 'when' decisions to the agent. This, combined with their ability to search across data in unstructured format, to automatically learn and adapt to user preferences and to identify patterns is giving agent technology an ever increasing role in web searching.

Next: Section six - XML

Related article: Intelligent agents and peer-to-peer searching, IWR Dec 00

Back to the top