Aliweb: The World’s First Internet Search Engine

August 2014

I assume, like most Internet users you will use a search engine, probably Google. Did you know Nexor invented the first search Engine?

Aliweb

Back in 1992, Martijn Koster, a software developer at Nexor, built some software to manage and index the emerging Web. His work, called Aliweb, is acknowledged as the world’s first search engine. (The BBC incorrectly cite WebCrawler, but Aliweb was announced in 1993 prior to WebCrawler and presented at the first World Wide Web conference in 1994).

Here is the documentation from the time:

The World Wide Web is growing too big to find things easily. It is impossible to keep track of all the services other people provide; they change often, and there are simply too many of them.

Therefore ALIWEB proposes that people just keep track of the services they provide, in such a way that automatic programs can simply pick up their descriptions, and combine them into a searchable database.

Because automatic programs cannot understand natural language, these descriptions will have to be written in a concise and standard format.
So the way ALIWEB works is as follows:

• People write descriptions of their services in a standard format into a file on the Web, by hand or using automatic tools.
• They then tell ALIWEB about this file.
• ALIWEB regularly retrieves all these files, and combines them into a searchable database.
• Anybody can come and search this database from the Web.

Because the database can be updated regularly (currently once a day) the data is very up-to-date. Since ALIWEB does all the work of retrieving and combining these files, people only need to worry about descriptions of their own services; so the information is likely to be correct and informative. And as only these small description files need to be gathered there is little overhead.

As other search services started to emerge, using a spidering technique (invented by Koster at Nexor), Aliweb became one of many such search engines. Their individual coverage was limited, and to find what you wanted you really had to choose the right search engine.

CUSI

To solve this problem, Koster went on to develop CUSI, (Configurable Unified Search Interface) which enabled you to send a single query to multiple search engines. The interface is still visible on the WayBack Machine.

…then came Google…

NB: www.aliweb.com is a commercial company, and nothing to do with Nexor or the original Aliweb search engine research.

Author Bio – Colin Robbins

Colin Robbins is Nexor’s Managing Security Consultant. He is a Fellow of the IISP, and a NCSC certified Security and Information Risk Adviser (Lead CCP) and Security Auditor (Senior CCP). He has specific technical experience in Secure Information Exchange & Identity Systems and is credited as the co-inventor of LDAP. He also has a strong interest in security governance, being a qualified ISO 27001 auditor.

• LinkedIn profile
• View all of Colin's posts