What is Searching
The first tool for searching the Internet, created in 1990, was called “Archie”. It downloaded directory listings of all files located on public anonymous FTP servers; creating a searchable database of filenames. A year later “Gopher” was created. It indexed plain text documents. “Veronica” and “Jughead” came along to search Gopher’s index systems. The first actual Web search engine was developed by Matthew Gray in 1993 and was called “Wandex”.
The good news about the Internet and its most visible component, the World Wide Web, is that there are hundreds of millions of pages available, waiting to present information on an amazing variety of topics. The bad news about the Internet is that there are hundreds of millions of pages available, most of them titled according to the whim of their author, almost all of them sitting on servers with cryptic names. When you need to know about a particular subject, how do you know which pages to read? If you’re like most people, you visit an Internet search engine.
Internet search engines are special sites on the Web that are designed to help people find information stored on other sites. There are differences in the ways various search engines work, but they all perform three basic tasks:
- They search the Internet — or select pieces of the Internet — based on important words.
- They keep an index of the words they find, and where they find them.
- They allow users to look for words or combinations of words found in that index.
Early search engines held an index of a few hundred thousand pages and documents, and received maybe one or two thousand inquiries each day. Today, a top search engine will index hundreds of millions of pages, and respond to tens of millions of queries per day.
What is Search Engine
A search engine is a searchable database of Internet files collected by a computer program, called a crawler, robot, worm, or spider. Indexing is created from the collected files, e.g., title, full text, date last modified, URL, language, etc. Results are ranked by relevance; this will vary among search engines.
In essence, a search engine consists of three components:
- Spider: Program that traverses the Web from link to link, identifying and reading pages (Web crawler)
- Index: Database containing a copy of each Web page or other file gathered by the spider
- Search and retrieval mechanism: Technology that enables you to search the index and that returns results in a relevancy-ranked order
Four types of search engines
- General
- Meta
- Concept Categorizing
- Vertical
General Search Engines
A general search engine is a search engine that covers the overall Web, using its own spider to collect Web pages for its own index.
When to use a general search engine
- When you have a well-defined topic or idea to research
- When you are looking for a specific site
- When you want to search the full text of millions of Web pages
- When you want to retrieve a large number of Web sites on your topic
- When you want to search for particular types of documents, sites, file types, languages, date last modified, geographical location, etc.
Examples of general search engines
This is easy. General search engines have been popular and newsworthy for many years.
- Bing
- Yahoo!
Meta Search Engines
A meta search engine searches multiple search engines from a single search page.
Meta search engines work in various ways. With some, a single, simultaneous search retrieves results from multiple sources, usually with the duplicates removed. Others offer a separate search of multiple content sources, allowing you to select the source(s) you want for each search.
When to use a meta search engine
- When you want to retrieve a relatively small number of relevant results
- When your topic is obscure
- When you are not having luck finding what you want
- When you want the convenience of searching a variety of different content sources from one search page
Examples of meta search engines
- Browses
- Dogpile
Concept Categorizing Search Engines
A concept categorizing search engine organizes results into topical categories or concepts. These concepts are derived from the contents of your search results. This arrangement may be thought of as a horizontal layout of results, in contrast to a single, vertical list of results that you find on such search engines as Google. Most concept processing engines also offer a vertical list of results. The concept categories are an added bonus.
When to use a concept categorizing search engine
- When your search is a general one and you want to learn about the component concepts of your topic
- When you know very little about your topic and want help in identifying its scope and content
- When your topic is obscure
- When you are having trouble finding what you want when scanning a single list of search results
Vertical Search Engines
A vertical search engine searches a specific industry, topic, type of content (e.g., travel, movies, images, blogs, live events), piece of data, geographical location, and so on. It may help to think of vertical search as a search for a particular niche. Some of this content cannot be found, or is difficult to find, on general search engines.
To find a vertical search engine, you can use a general search engine and try to find a search site dedicated to a particular type of content, for example medical search, job search, and so on.
When to use a vertical search engine
- When your topic is focused on a specific topic, industry, content type, geographical location, language, etc.
- When you’re having difficulty locating what you want on general, meta, or concept categorizing search engines
A few examples of vertical search engines
Topical search engines
~ MedNar and PubMed
~ ScienceResearch.com, Scitopia and World Wide Science
Industry search engines
~ BizNar, the world of business
~ Industry Search, a search for industrial supplies limited to Australia and New Zealand
~ PaperPundit, searching the pulp and paper industry created with Google Custom Search
~ Stock Screener from Yahoo!
Image search engines
~ Picsearch
~ Library of Congress Prints & Photographs Online Catalog
tip! A good source for more of these tools can be found in a 2008 posting on the ReadWriteWeb blog, Digital Image Resources on the Deep Web.
News search engines
~ Alltop
~ NewsNow
Blog search engines
~ Google Blog Search
~ Technorati
Full text books and articles
~ Google Book Search
~ Google Scholar
~ The Online Books Page
~ Project Gutenberg
Social Web real time search
~ TweetScan for Twitter content
~ FriendFeed Search for social networking activities
