What Is a Search Engine? How Discovery, Indexing and Serving Work

In plain English

Every search engine does the same three jobs in sequence: find pages, store an understanding of them, then decide which stored pages best answer a given query. Google describes its own process as crawling, indexing and serving.

The important consequence is that these are separate stages. A page can be found but not indexed, or indexed but never served. Diagnosing an SEO problem usually means working out which stage it failed at.

What to know

Crawling

Bots follow links and fetch pages. Anything unreachable never gets further.

Indexing

Content is parsed, understood and stored, with duplicates resolved to one version.

Serving

For each query, candidates are ranked and presented as links, snippets, maps or summaries.

Learning

Behaviour and quality signals feed back into future ranking decisions.

Why it matters

Knowing the stages tells you where to look when something goes wrong. It also widens the definition usefully: YouTube, Amazon and app stores are search engines with their own indexes, and for some businesses those matter more than Google.

Common mistakes

×Assuming a published page is automatically an indexed page.

×Optimising for ranking while the crawling stage is broken.

×Ignoring the search engines inside the platforms your customers already use.

FAQs

Is Google the only one worth optimising for?

It is the largest, but AI assistants, YouTube and marketplace search all send meaningful traffic depending on the business.

Do all search engines work the same way?

The three stages are common to all of them. The ranking judgements differ considerably.

WRITTEN BY TARIQ SALLAM

Marketing Consultant. Entrepreneur. Content Creator.

I'm a marketing consultant, entrepreneur and content creator. I help businesses grow through practical marketing, websites, SEO, content and AI.

More About Tariq →