Skip to main content

Learn SEO

Menu

Class 2: How the Google Search Engine Works?
| FREE Advanced SEO Course

Watch the Full Video: How the Google Search Engine Works?

Website Side Process

Crawler: The crawler gathers data from websites and forwards it to the indexer for processing and indexing.

Indexer: The indexer analyzes and processes the data received from the crawler, then sends it to the index for future retrieval.

Index: The index takes the processed data, organizes it efficiently, and stores it in a database.

User Side Process

Interface: The user goes to a search engine and enters a query related to the information they need.

Query Engine: Whatever query the user enters is received by the query engine. The query engine sends the user’s query to the index, retrieves the relevant information, and then displays the results to the user.

Index: Based on the user’s query, the index provides relevant data to the search engine.

URL Server: It removes duplicate URLs, keeps only the unique URLs, and groups them into batches and provides them to the crawler.

Crawler: The Crawler visits the URLs received from the URL server through the internet, collects content, images, videos, and other resources, and saves them in HTML format.

Store Server: Store Server compresses the data received from the Crawler and assigns a unique ID to it.

Repository: The Store Server forwards the compressed data to the Repository, and the Repository stores that data.

Indexer: The data, which is obtained from the Repository! The Indexer breaks it down into individual words and assigns an ID to each and extracts links and anchor text.

Anchors: The Anchors collect the anchor texts generated by the Indexer and store them.

URL Resolver: After the Anchors System is processed, the information is sent to the URL Resolver. The URL Resolver has two main functions.

    1. The first function of the URL Resolver is to convert relative URLs into absolute URLs. For example, /blog/page1 becomes https://example.com/blog/page1.
    2. Route the URL to the correct destination(URL Resolver removes and replaces the ID assigned by the Repository(exa.com = 123, and is page k andar koy link hogi like abc.com isko wo id dega like abx.com = xxx))

URL Resolver to Links, Doc Index, Barrels Links: The Links System creates a map of all links, and it tracks which pages are connected, which pages are linking to a particular page, and which pages a page is linking to. This link information is then passed to the PageRank algorithm, which calculates the importance or weight of each page and provides that information to the ranking system.

Doc Index: The URL Resolver sends the data to the Doc Index, doc index means document index. It creates a version of the page that can be searched and retrieved quickly, and the data is forwarded to the Final Search System and the URL Server.

Barrels: Whatever data Barrels receives from the Indexer, the Barrels system matches the document ID with the IDs of all the texts contained in that document.

Sorter: The Inverter converts the forward index into an inverted index. Instead of organizing data by document IDs, it organizes data by word IDs. It keeps track of which documents contain a particular word. This inverted index is then used by the search engine to quickly find and rank relevant pages.

Forward Index:

    • Document ID 101 → Word IDs 5, 8, 12

    • Document ID 102 → Word IDs 5, 15, 20

Inverted Index:

    • Word ID 5 → Document IDs 101, 102

    • Word ID 8 → Document ID 101

  • Word ID 15 → Document ID 102

Lexicon: The Lexicon creates HITs using the text, word IDs, and their positions in the text. These HITs are then sent to the Indexer, and from the Indexer they are forwarded to the Barrels system.