Posted by carinoverturf
This index finished up in just 13 days, thanks again to all the improvements our Big Data Processing team has been implementing to make our Mozscape processing pipeline more efficient. The team continues to dial out our virtual private cloud in Virginia as well as tweak, tune, and improve the time it takes to process 82 billion URLs.
We’ve been saying we’re close to releasing our first index created on our own hardware – and now we really are! Stay tuned for a deep dive blog post into why and how we built our own private cloud.
This index was kicked off the first week of March, so data in this index will span from late January through February, with a large percentage of crawl data from the last half of February.
Here are the metrics for this latest index:
- 83,122,215,182 (83 billion) URLs
- 12,140,091,376 (12.1 billion) Subdomains
- 141,967,157 (142 million) Root Domains
- 801,586,268,337 (802 billion) Links
Followed vs. Nofollowed
- 2.21% of all links found were nofollowed
- 55.23% of nofollowed links are internal
- 44.77% are external
- Rel Canonical – 15.70% of all pages now employ a rel=canonical tag
The average page has 74 links on it
- 63.56 internal links on average
- 10.65 external links on average
And the following correlations with Google’s US search results:
- Page Authority – 0.35
- Domain Authority – 0.19
- MozRank – 0.24
- Linking Root Domains – 0.30
- Total Links – 0.25
- External Links – 0.29
We always love to hear your thoughts! And remember, if you’re ever curious about when Mozscape next updates, you can check the calendar here. We also maintain a list of previous index updates with metrics here.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!