The computations required to generate these corpora were performed at Google using the MapReduce

The computations required to generate these corpora were performed at Google using the MapReduce framework for distributed computing (Ref $5). Many computers were used as these computations would take many years on a single ordinary computer. Note that the ability to study the frequency of words or phrases in English over time was our primary focus in this study. As such, we went to significant lengths to ensure the quality of the general English corpora and their date metadata (i.e., Eng-all, Eng-1M, and Eng-Modern-1M). As a result, the accuracy of place- of-publication data in English is not as reliable as the accuracy of date metadata. In addition, the foreign language corpora are affected by issues that were improved and largely eliminated in the English data. For instance, their date metadata is not as accurate. In the case of Hebrew, the metadata for language is an oversimplification: a significant fraction of the earliest texts annotated as Hebrew are in fact hybrids formed from Hebrew and Aramaic, the latter written in Hebrew script. The size of these base corpora is described in Tables S3-S6. III. Culturomic Analyses In this section we describe the computational techniques we use to analyze the historical n-grams corpora. IlI.0. General Remarks III.0.1 On Corpora. There is significant variation in the quality of the various corpora during various time periods and their suitability for culturomic research. All the corpora are adequate for the uses to which they are put in the paper. In particular, the primary object of study in this paper is the English language from 1800-2000; this corpus during this period is therefore the most carefully curated of the datasets. However, to encourage further research, we are releasing all available datasets - far more data than was used in the paper. We therefore take a moment to describe the factors a culturomic researcher ought to consider before relying on results of new queries not highlighted in the paper. 1) Volume of data sampled. Where the number of books used to count n-gram frequencies is too small, the signal to noise ratio declines to the point where reliable trends cannot be discerned. For instance, if an n-gram's actual frequency is 1 part in n, the number of words required to create a single reliable timepoint must be some multiple of n. In the English language, for instance, we restrict our study to years past 1800, where at least 40 million words are found each year. Thus an n-gram whose frequency is 1 part per million can be reliably quantified with single-year resolution. In Chinese, there are fewer than 10 million words per year prior to the year 1956. Thus the Chinese corpus in 1956 is not in general as suitable for reliable quantification as the English corpus in 1800. (In some cases, reducing the resolution by binning in larger windows can be used to sample lower frequency n-grams in a corpus that is too smal for single-year resolution.) In sum: for any corpus and any n-gram in any year, one must consider whether the size of the corpus is sufficient to enable reliable quantitation of that n-gram in that year. 2) Composition of the corpus. The full dataset contains about 4% of all books ever published, which limits the extent to which it may be biased relative to the ensemble of all surviving books. Still, marked shifts in composition from one year to another are a potential source of error. For instance, book sampling patterns differ for the period before the creation of Google Books (2004) as compared to the period afterward. Thus, it is difficult to compare results from after 2000 with results from before 2000. As a result, significant changes in culturomic trends past the year 2000 may reflect corous composition issues. This was an important reason for our choice of the period between 1800 and 2000 as the target period. 12 HOUSE_OVERSIGHT_017020