Hyperlink Graph of the World Wide Web of 2012 (aggregated by first level subdomains)

Item Type:	Dataset
Title:	Hyperlink Graph of the World Wide Web of 2012 (aggregated by first level subdomains)
Alternative Title:	Hyperlink Graph of the World Wide Web (aggregated by fist level subdomains) extracted from the Common Crawl Dataset 2012
Date:	12 November 2013
Creator:	Meusel, Robert ; Lehmberg, Oliver ; Bizer, Christian
Divisions:	School of Business Informatics and Mathematics > Wirtschaftsinformatik V (Bizer)

DDC Classification:	004 Computer science, internet
Keywords:	webgraph ; web graph ; hyperlink graph
Abstract:	Knowledge about the general graph structure of the hyperlink graph is important for designing ranking methods for search engines. To amend the ranking calculated by search engines for different websites, search engine optimization agencies focus on linkage structure for their clients. An extreme appearance of ranking manipulation manifests in spam networks, where pages and websites publishing dubious content try to increase their ratings by setting a massive number of links to other pages and retrieve backlinks. The WDC Hyperlink Graph on first level subdomain level has been extracted from the Common Crawl 2012 web corpus and covers 95 million first level subdomains, linked by almost 2 billion connections, which are derived from the hyperlinks of the pages contained by the first level subdomains.

URL:	https://madata.bib.uni-mannheim.de/50/
DOI:	https://doi.org/10.7801/50
Availability (Controlled):	Download
Availability:	Data is available as download of Index File and Arc files. For further information about the license of the extracted data see http://webdatacommons.org/hyperlinkgraph/2012-08/download.html#license.
Publication(s) (MADOC):	Meusel Robert und Vigna Sebastiano und Lehmberg Oliver und Bizer Christian (2015), The Graph Structure in the Web - Analyzed on Different Aggregation Levels Meusel Robert und Vigna Sebastiano und Lehmberg Oliver und Bizer Christian (2014), Graph Structure in the Web - Revisited
Reference URL (External):	http://webdatacommons.org/hyperlinkgraph/index.htm... http://webdatacommons.org/hyperlinkgraph/2012-08/t... http://webdatacommons.org/hyperlinkgraph/2012-08/d...
Project:	Project Title: Web Data Commons Hyperlink Graph Project Description: Knowledge about the general graph structure of this graph is important for designing ranking methods for search engines. To amend the ranking calculated by search engines for different websites, search engine optimization agencies focus on linkage structure for their clients. An extreme appearance of ranking manipulation manifests in spam networks, where pages and websites publishing dubious content try to increase their ratings by setting a massive number of links to other pages and retrieve backlinks. The WDC Hyperlink Graph has been extracted from the Common Crawl 2012 web corpus and covers 3.5 billion web pages and 128 billion hyperlinks between these pages. To the best of our knowledge, this graph is the largest hyperlink graph that is available to the public outside companies such as Google, Yahoo, and Microsoft.

Metadata export

HTML CitationEP3 XMLReference ManagerMultiline CSVBibTeX

File	Filename / Infos	Link
	Archive Filename: sd1-arc.gz No License defined	Download (8GB)
	Archive Filename: sd1-index.gz No License defined	Download (793MB)

Depositing User:	Robert Meusel
Date Deposited:	09 May 2017 09:14
Last Modified:	29 Feb 2024 16:08

Actions (login required)

View Item