wikilinkgraphs-revisionlist --------------------------- This dataset contains lists of all revisions for each Wikipedia article (`namespace 0`) from Wikimedia's [history dumps](https://dumps.wikimedia.org/) for the languages ***de, en, es, fr, it, nl, pl, ru, sv***. ### Descriptions of the data * `page_id`: an integer, the page identifier used by MediaWiki. This identifier is not necessarily progressive, there may be gaps in the enumeration; * `page_title`: a string, the title of the Wikipedia article; * `revision_id`: an integer, the identifier of a revision of the article, also called a *permanent id*, because it can be used to link to that specific revision of a Wikipedia article; * `revision_parent_id`: an integer, the identifier of the parent revision. In general, each revision as a unique parent; going back in time before 2002, however, we can see that the oldest articles present non-linear edit histories. This is a consequence of the import process from the software previously used to power Wikipedia, MoinMonWiki, to MediaWiki; * `revision_timestamp`: date and time of the edit that generated the revision under consideration; * `user_type`: a string (`"registered"` or `"anonymous"`), specifying whether the user making the revision was logged-in or not; * `user_username`: a string, the username of the user that made the edit that generated the revision under consideration; * `user_id`: an integer, the identifier of the user that made the edit that generated the revision under consideration; * `revision_minor`: a boolean flag, with value 1 if the edit that generated the current revision was marked as *minor* by the user, 0 otherwise; * `bytes`: an integer, the length in bytes of the text of that revision; * `change_bytes`: an integer, the difference in length between a revision and the previous; ### Sample Extract of the file `enwiki-20180301-pages-meta-history1.xml-p10p2115.7z.revisionlist.csv.gz` in `enwiki/20180301/`: ``` page_id,page_title,revision_id,revision_parent_id,revision_timestamp,user_type,user_username,user_id,revision_minor,bytes,change_bytes 10,AccessibleComputing,233192,,2001-01-21T02:12:21Z,registered,RoseParks,99,0,119,119 10,AccessibleComputing,862220,233192,2002-02-25T15:43:11Z,registered,Conversion script,0,1,35,-84 10,AccessibleComputing,15898945,862220,2003-04-25T22:18:38Z,registered,Ams80,7543,1,34,-1 10,AccessibleComputing,56681914,15898945,2006-06-03T16:55:41Z,registered,Nzd,516514,1,36,2 10,AccessibleComputing,74466685,56681914,2006-09-08T04:16:04Z,registered,Rory096,750223,0,57,21 10,AccessibleComputing,133180268,74466685,2007-05-24T14:41:58Z,registered,Ngaiklin,4477979,1,53,-4 10,AccessibleComputing,133452289,133180268,2007-05-25T17:12:12Z,registered,Gurch,241822,1,57,4 10,AccessibleComputing,381200179,133452289,2010-08-26T22:23:51Z,ip,76.28.186.133,-1,0,1875,1818 10,AccessibleComputing,381202555,381200179,2010-08-26T22:38:36Z,registered,OlEnglish,7181920,1,57,-1818 10,AccessibleComputing,631144794,381202555,2014-10-26T04:50:23Z,registered,Paine Ellsworth,9092818,0,69,12 ``` ## FAQs ### What is the total size of the dataset, the number of files and the largest file in the dataset? The total dataset size is 21GB, and it contains 987 files. The average file size is 21.8MB and the largest file is 588MB. ### How are files organized? Files are divided in directories, one for each language, like this: ``` . ├── dewiki │   └── 20180301 │   ├── dewiki-20180301-pages-meta-history1.xml-p1p3405.7z.revisionlist.csv.gz │   ├── dewiki-20180301-pages-meta-history1.xml-p3406p6349.7z.revisionlist.csv.gz │ ├── ... │   └── dewiki.revisionlist.sha512sums.txt ├── enwiki │   └── 20180301 │   ├── enwiki-20180301-pages-meta-history1.xml-p10p2115.7z.revisionlist.csv.gz │   ├── enwiki-20180301-pages-meta-history1.xml-p2116p4236.7z.revisionlist.csv.gz │   ├── ... │   └── enwiki.revisionlist.sha512sums.txt ├── eswiki │   └── 20180301 │   ├── eswiki-20180301-pages-meta-history1.xml-p5p3721.7z.revisionlist.csv.gz │   ├── eswiki-20180301-pages-meta-history1.xml-p3722p7536.7z.revisionlist.csv.gz │   ├── ... │   └── eswiki.revisionlist.sha512sums.txt ├── frwiki │   └── 20180301 │   ├── frwiki-20180301-pages-meta-history1.xml-p3p3554.7z.revisionlist.csv.gz │   ├── frwiki-20180301-pages-meta-history1.xml-p3555p9507.7z.revisionlist.csv.gz │   ├── ... │   └── frwiki.revisionlist.sha512sums.txt ├── itwiki │   └── 20180301 │   ├── itwiki-20180301-pages-meta-history1.xml-p2p12411.bz2.revisionlist.csv.gz │   ├── itwiki-20180301-pages-meta-history1.xml-p12412p29362.bz2.revisionlist.csv.gz │   ├── ... │   └── itwiki.revisionlist.sha512sums.txt ├── nlwiki │   └── 20180301 │   ├── nlwiki-20180301-pages-meta-history1.xml-p1p6842.7z.revisionlist.csv.gz │   ├── nlwiki-20180301-pages-meta-history1.xml-p6843p18427.7z.revisionlist.csv.gz │   ├── ... │   └── nlwiki.revisionlist.sha512sums.txt ├── plwiki │   └── 20180301 │   ├── plwiki-20180301-pages-meta-history1.xml-p2p7835.7z.revisionlist.csv.gz │   ├── plwiki-20180301-pages-meta-history1.xml-p7836p27363.7z.revisionlist.csv.gz │   ├── ... │   └── plwiki.revisionlist.sha512sums.txt ├── ruwiki │   └── 20180301 │   ├── ruwiki-20180301-pages-meta-history1.xml-p4p5387.7z.revisionlist.csv.gz │   ├── ruwiki-20180301-pages-meta-history1.xml-p5388p15885.7z.revisionlist.csv.gz │   ├── ... │   └── ruwiki.revisionlist.sha512sums.txt └── svwiki └── 20180301 ├── svwiki-20180301-pages-meta-history.xml.7z.revisionlist.csv.gz └── svwiki.revisionlist.sha512sums.txt ``` ### Who produced this dataset and why? * This dataset has been produced by Cristian Consonni, David Laniado and Alberto Montresor. * Cristian Consonni and Alberto Montresor are affiliated with the Department of Information Engineering and Computer Science (DISI), University of Trento, Trento, Italy; David is affiliated with Eurecat - Centre Tecnològic de Catalunya, Barcelona, Spain. * This dataset has also been produced as part of the research related to the ENGINEROOM project. EU ENGINEROOM has received funding from the European Union's Horizon 2020 research and innovation programme under the Grant Agreement no 780643. ### Is this dataset used in currently in-review or future papers that cite it? This dataset is published as part of a paper *"WikiLinkGraphs: A complete, longitudinal and multi-language dataset of the Wikipedia link networks"*, currently under review at [ICWSM 2019](https://www.icwsm.org/2019/index.php). This dataset is published as part of the ENGINEROOM project and it will be cited in the related scientific publications. ## License This dataset is released under [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/legalcode). The original dump is released under the [GNU Free Documentation License (GFDL)](https://www.wikipedia.org/wiki/Wikipedia:Copyrights) and the [Creative Commons Attribution-Share-Alike 3.0 License](https://creativecommons.org/licenses/by-sa/3.0/), see [the legal info](https://dumps.wikimedia.org/legal.html).