reflected the diversity of book editions actually published in that year. We estimated the latter
Every indexed document
Browse the released records by type and release. Open any document to read its text and jump to the official source.
69,179 documents
based on eleven different subsets of books. Five of these are English
it may be preferable to examine the fraction of
for instance in a biography of 'Gottlieb Daimler' which mentions his name many times. This
reflects both the number of authors using an n-gram
so our ability to discern these trends was not compromised
rare spelling and OCR errors were also omitted. Since most n-grams are infrequent
three counts are generated: (1) the total number of times the n-gram appears; (2) the
a subset of book editions is chosen to serve as the
n-grams that span two consecutive pages were not counted.
a sequence of n 1-grams) for all n less than or equal to 5. Because
we counted the number of times each 1-gram appeared. We further counted the number of
we denote the corresponding n-gram by concatenating the 1-grams with
except it appears at the end of a sequence of alphanumeric
possibly containing a decimal point. Examples include $71 and $9.95
except where it is the first character of a word consisting
except when it is part of a number or price
digi-<newline>tized became “digitized”. This
may appear on two lines in a book as ""digi-<newline>ized"". Prior to tokenization
the tokenizer separates the character stream into words at the white space characters (\n