Search Engines Index 85% of Visible Web Content


According to a recent study co-written by Ask Jeeves Director of Advanced Products Antonio Gulli and University of Iowa’s Alessio Signorini, the major search engines have managed to index nearly 85% of the estimated 11.5 billion documents that make up the “visible web”. While there are likely over 500 billion documents on the web, most are invisible files hidden in databases and password protected regions search spiders don’t travel. Collectively, the four major search engines are estimated to have indexed about 9.1 billion unique documents.

Google carries the most indexed content boasting 8.1 billion documents or 69.6% of the visible web in its search database. Yahoo comes in second with an estimated 6.6 billion or 57.4%. Ask Jeeves slips past MSN with an estimated 5.3 billion indexed documents (46.1%) to the 5.1 billion pages (44.3%) in MSN’s database.

While the study relies heavily on estimates and on reporting from search engines, it is one of the only recent examinations of the actual size of the web and of search engine coverage.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s