How do you use a search engine?
What gets indexed?
Which search engine?
Other engines

In the hundreds of millions of pages that make up the World Wide Web, how do you find that one piece of information you're desperately seeking? You use a search engine. The problem is, there are now well over a thousand search engines out there, so how do you go about choosing which to use and learning how to use it?

The good news is, all the major search engines do a good job of cataloguing a fair slice of the Web as well as other sections of the Internet. Most of them also work in basically similar ways, at least at the simple level, so you can transport many of your basic searching skills from one engine to another.

Search engines distinguish themselves in a variety of ways: some catalogue more of the Internet than others; some stay more up-to-date; some are easier to use; some add very advanced searching options. There's an incredible amount of jostling among the engines to be number one, so you'll find frequent upgrades to their interfaces and underlying technology. The engine that was hot three months ago may already be slipping out of favour today.

This guide to the major search engines gives you the current state of play, and lets you choose the search engine that will suit your needs and level of knowledge.

To use the category searches, you use a drill down approach, starting with general topics (such as The Arts, Business, Computers) and digging down to specific categories (such as Chinese Poets, Debit Cards or Electronic Mail Software) refining your search as you go.

You'll find there's a lot of cross-over, with almost all of the category-style engines also permitting keyword searches.

Category-based search engines usually require people to submit their sites for inclusion in the search engine's database, and then use a keyword or concept approach to indexing the sites. While this makes them less than all-inclusive, it certainly removes a lot of the chaff from your searches. Category search engines often include guides or selected sites which are not only catalogued but also reviewed. Such guides are a real help in winnowing out excess or irrelevant information.

If you're looking for obscure or highly specific details, use one of the keyword engines, such as AltaVista or -- a simpler alternative -- Open Text Index. To use these search engines effectively and not be totally overwhelmed by the amount of information they can toss at you, you'll need to take a little time getting to know the specific syntax they use. Each search engine uses its own search language, and the syntax varies slightly from one to another. Taking some time to learn the specific syntax for a search engine will increase the value of the results you retrieve many times over.

If you're wanting information about more generalised topics, such as "What sort of handheld computers are available?", then you'll find the category-based search engines are ideal. For example, to answer this question using Yahoo, you'd start by looking in the computers category, then hardware, then portable computers, and finally handheld computers.

The most effective way to use search engines is to choose two or three (include at least one keyword engine and one category engine) and begin to use them regularly. That way, you can become accustomed to the particular features and language used by the search engines without drowning in the sea of possibilities.

Meta-search engines offer another option. These search engines take your search query and submit it to a selection of other search engines, including both keyword and category engines. Meta-searches usually restrict the number of results either by limiting the amount of time they spend on the search, or by limiting the number of documents they'll return in the results list.

If you want to focus on sites in our neighbourhood, try one of the Australian search engines. Some of the Australian search engines are pretty lightweight in terms of features and content, largely because they don't attract the advertising revenue garnered by the big US-based engines, and thus don't have much in the development kitty. But there are some good engines among them and the recent addition of localised versions of Yahoo and Excite adds a real boost to local searching.

Many of the Australian engines index local and regional sites only; others, such as Anzwers, Yahoo Australia and Excite Australia, provide global searches but with the option of doing localised searches.

You can also use one of the big global search engines and restrict your searches to Australian sites by doing field searches. Field searches look for the terms you specify in particular parts of a Web document. For instance, to restrict searches in AltaVista to Australian sites, add AND host:au to your query.

Unfortunately, valid field search terms differ from engine to engine -- some don't recognise them at all -- so you'll need to learn the correct syntax for yourself. For instance, Infoseek uses the fields url:, site: (a URL and all its related pages), title: and link: (lets you search for any page that links to a specific URL). HotBot and Anzwers have their very own syntax, but let you use drop-down menus to perform field searches as well, which is a much easier, albeit slower, alternative.

If you're after a more specific search engine, you'll find lists of search engines categorised by other search engines. For instance, Yahoo has an excellent list in the sub-category Computer and Internet/Internet/World Wide Web/Searching the Web/Search Engines. Instead of drilling all the way down, you can find this page simply by typing search engines in the Yahoo search box.

by Rose Vines

Speaking Boolean

Search Engine comparison

Researching the Internet

The Lycos Pro Power Panel (available in Java or non-Java versions) gives you extraordinary control over searches merely by choosing from drop-down boxes and adjusting slider controls.

2. Use quotation marks to search for specific phrases, such as "sheep dip". (The same technique applies to many, but not all, search engines.)

3. You must use AND NOT instead of NOT to exclude items in Boolean searches. This is contrary to the information in AltaVista's online help. The same applies to Excite and HotBot.

4. Beware: AltaVista's Simple Search and Advanced Search use different syntax. The Simple Search does not recognise any Boolean operators. For example, if you type Oliver AND NOT stone in the Simple Search box, you'll get pages that include Oliver, stone and those that include Oliver Stone. Type the same query in the Advanced Search box, and you'll get what you expect: pages including Oliver or stone but not both.

by Rose Vines

2. The HotBot Guides (called the Wired Cybrarian) offer excellent collections of information, especially on computer and Internet topics.

3. Click the Save This Search button on the results page to save a complex search. You can then use the Saved Searches button to display all your saved searches and select the one you want to reuse.

by Rose Vines