Search engine VS Directories

The line between search engines and directories is a thin one. but there are some important differences below the surface.

The most important difference is in the way they collect sites to add to their databases.

Search engines rely on “crawlers” or “robots”.

crawlers are programs that continuously roam the web visiting web pages and depositing information on those pages Into a searchable database (also called an “index”). Spiders automatically follow links from one page to the next and from one site to the next.

SEARCH ENGINES

A popular myth is that search engines really “search the web“ every time someone enters a search query. if that was true, you'd have to wait days - perhaps even weeks - for the search engine to return the search results in response to a search.

Instead, search engines search through databases of indexed web sites, and it takes only a fraction of a second.

Search engine spiders can only find pages that are

1. linked to from other pages they know about or

2. that are submitted to them.

That's it.

So that means they only index part of the web?

Yes.

No search engine can claim to have indexed the entire web. They can only index a subset of the web that part of the web they “know about” and with millions of pages being added to the web daily, the part that they know about is only a small percentage of the total.

What matters to you and me is for our pages to be among those that are indexed. That’s simple enough.

E.g. Google,Bing,Yahoo!,Altavista and so on……

DIRECTORIES

Directories do not use spiders. Instead, they use real people (editors) who visit and evaluate sites and add them only if they meet the directory's minimum quality requirement.

This is an important difference:

search engine spiders can index thousands of pages a day.

Directory editors can't.

so why do we have directories If they can't compete?

The answer is quality.

Human directory editors are considerably harder to impress 1han spiders. The page has to offer unique Information or a unique product. When you submit a site to a directory, the editor of the category where you submitted will take a look at your site and decide if its good enough to add to the directory.

Expect directory editors to reject pages with typos, broken

links, unclear navigation etc.

That said, getting your site into a directory is not that hard.

If your site offers high-quality content, is not broken or under construction and you follow the directory's submission rules, you shouldn't have trouble getting in.

We'll look at the details in the Link Popularity section later in the book.