Bigrams

9 March 2019. Puzzling out a word jumble, I’m writing a python script to search a grid for words. Step one is to compile a list of legal bigrams in English. Bigrams are two letters that go side-by-side. So the letter <Q> in English has a limited list of bigrams. We see <QU> as in quit, <QA> as in Qatar (and a few others if you allow very rare words).

I found a huge list online of English words compiled from web pages. 2.5 megs of text file! Here is the resulting python dict of bigrams:

{'A':['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'],
'B':['A', 'B', 'C', 'D', 'E', 'F', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'R', 'S', 'T', 'U', 'V', 'W', 'Y', 'G', 'P', 'Z', 'Q'],
'C':['A', 'I', 'K', 'T', 'U', 'E', 'O', 'Y', 'H', 'C', 'L', 'M', 'N', 'Q', 'R', 'S', 'D', 'B', 'W', 'Z', 'G', 'P', 'F'],
'D':['V', 'W', 'E', 'I', 'O', 'L', 'N', 'A', 'U', 'G', 'Y', 'R', 'P', 'C', 'D', 'F', 'H', 'J', 'M', 'S', 'T', 'Z', 'B', 'K', 'Q'],
'E':['H', 'R', 'D', 'N', 'E', 'S', 'M', 'Y', 'V', 'L', 'A', 'C', 'I', 'P', 'T', 'K', 'Z', 'U', 'G', 'W', 'B', 'F', 'O', 'X', 'Q', 'J'],
'F':['F', 'T', 'A', 'U', 'O', 'E', 'I', 'Y', 'L', 'G', 'R', 'S', 'W', 'Z', 'N', 'V', 'H', 'B', 'K', 'D', 'M', 'J', 'P', 'C'],
'G':['I', 'E', 'H', 'L', 'N', 'A', 'Y', 'O', 'R', 'M', 'U', 'S', 'D', 'G', 'K', 'P', 'B', 'W', 'T', 'F', 'C', 'V', 'J', 'Z'],
'H':['R', 'E', 'L', 'M', 'I', 'Y', 'O', 'U', 'A', 'T', 'N', 'S', 'W', 'B', 'P', 'Z', 'G', 'C', 'F', 'D', 'H', 'J', 'K', 'V', 'Q'],
'I':['C', 'T', 'N', 'S', 'O', 'E', 'A', 'Z', 'R', 'L', 'D', 'U', 'P', 'G', 'B', 'V', 'F', 'M', 'I', 'X', 'K', 'Y', 'W', 'H', 'Q', 'J'],
'J':['E', 'O', 'U', 'A', 'I', 'H', 'J', 'R', 'Y', 'P', 'D', 'M', 'W', 'L', 'T', 'N', 'B', 'K'],
'K':['A', 'H', 'E', 'I', 'Z', 'M', 'N', 'B', 'S', 'L', 'O', 'C', 'K', 'P', 'R', 'T', 'U', 'W', 'Y', 'D', 'F', 'G', 'J', 'V'],
'L':['F', 'L', 'U', 'I', 'O', 'E', 'Y', 'A', 'M', 'T', 'S', 'N', 'V', 'C', 'D', 'B', 'G', 'H', 'P', 'R', 'K', 'W', 'J', 'Q', 'Z', 'X'],
'M':['A', 'P', 'E', 'B', 'I', 'O', 'H', 'U', 'Y', 'M', 'S', 'T', 'F', 'L', 'W', 'N', 'R', 'C', 'G', 'V', 'K', 'D', 'J', 'Z', 'Q'],
'N':['I', 'A', 'C', 'E', 'D', 'T', 'U', 'O', 'S', 'R', 'G', 'Y', 'M', 'N', 'Z', 'L', 'P', 'K', 'F', 'H', 'Q', 'B', 'J', 'V', 'X', 'W', '-'],
'O':['L', 'N', 'R', 'S', 'I', 'M', 'T', 'U', 'G', 'O', 'W', 'A', 'B', 'D', 'H', 'V', 'X', 'C', 'K', 'Z', 'P', 'Y', 'E', 'F', 'Q', 'J'],
'P':['E', 'T', 'O', 'Y', 'I', 'H', 'S', 'R', 'A', 'N', 'U', 'L', 'P', 'M', 'J', 'B', 'D', 'F', 'W', 'K', 'C', 'G', 'V', 'Q'],
'Q':['U', 'I', 'A', 'R', 'E', 'O', 'Q'],
'R':['D', 'O', 'U', 'E', 'A', 'I', 'T', 'Y', 'R', 'S', 'V', 'M', 'B', 'P', 'G', 'N', 'H', 'L', 'F', 'W', 'C', 'K', 'J', 'Q', 'X', 'Z'],
'S':['C', 'T', 'A', 'E', 'S', 'I', 'G', 'H', 'K', 'O', 'M', 'U', 'F', 'Q', 'V', 'Y', 'P', 'L', 'N', 'B', 'W', 'D', 'R', 'J', 'Z'],
'T':['E', 'I', 'O', 'H', 'A', 'T', 'U', 'C', 'N', 'S', 'R', 'M', 'L', 'Y', 'B', 'P', 'F', 'W', 'K', 'Z', 'D', 'G', 'J', 'V', 'Q', 'X'],
'U':['A', 'S', 'L', 'R', 'C', 'M', 'N', 'D', 'T', 'E', 'V', 'P', 'Z', 'B', 'I', 'O', 'X', 'G', 'K', 'F', 'Y', 'W', 'J', 'H', 'Q', 'U'],
'V':['A', 'E', 'I', 'O', 'U', 'Y', 'S', 'R', 'C', 'L', 'V', 'N', 'Z', 'D', 'K', 'G'],
'W':['O', 'H', 'A', 'E', 'I', 'L', 'N', 'S', 'T', 'R', 'M', 'U', 'Y', 'B', 'P', 'W', 'D', 'F', 'K', 'C', 'G', 'Z', 'Q', 'V', 'J'],
'X':['I', 'A', 'Y', 'T', 'E', 'O', 'U', 'M', 'P', 'C', 'B', 'F', 'H', 'L', 'S', 'W', 'R', 'D', 'K', 'N', 'G', 'Q', 'Z', 'V'],
'Y':['S', 'M', 'A', 'R', 'C', 'P', 'G', 'I', 'L', 'N', 'D', 'T', 'X', 'O', 'E', 'Z', 'U', 'F', 'W', 'H', 'B', 'Y', 'K', 'V', 'J', 'Q'],
'Z':['E', 'A', 'U', 'Z', 'I', 'O', 'L', 'G', 'Y', 'R', 'H', 'T', 'N', 'B', 'D', 'P', 'K', 'C', 'M', 'V', 'S', 'F', 'W']
}

And here is the code to get the bigrams (my file of words is called web2.txt, and each word is on a separate line). In order to limit the bigrams to a list of unique letters, I use set().

import os

path = os.getcwd()
path += '/web2.txt'

bigrams = {'A':[], 'B':[], 'C':[], 'D':[], 'E':[], 'F':[], 'G':[], 'H':[], 'I':[], 'J':[],
           'K':[], 'L':[], 'M':[], 'N':[], 'O':[], 'P':[], 'Q':[], 'R':[], 'S':[], 'T':[],
           'U':[], 'V':[], 'W':[], 'X':[], 'Y':[], 'Z':[]}

with open(path, 'r') as allwords:
    words = allwords.read().split('\n')
    allwords.close()

for letter in bigrams.keys():
    letter = letter.upper()

    for word in words:
        word = word.upper()
        if letter in word:
            if word.index(letter) < len(word):
                try:
                    nextletter = word[word.index(letter) + 1]
                    if nextletter not in set(bigrams[letter]):
                        bigrams[letter].append(nextletter)
                except IndexError:
                    continue

    print('\'{0}\':{1}, '.format(letter, bigrams[letter]))

Germanization of English

It seems that one of the background processes in American English is an increase in adnominal adjectives. You don’t make a choice about a college, but a college choice. The prepositional phrase is turned into a pre-position adjective, turning a noun into an adjective, rendering a compound noun worthy of German. I’ve noticed hundreds. Academics don’t have meetings of the faculty, they have faculty meetings; they no longer discuss the curriculum, they have curriculum discussions. In a recent memo, someone was described as a community heritage preserver, which is an astounding way of saying that she preserves the heritage of her community.

So what? Well, a tea cup is a thing to drink from, and a cup of tea is an amount of tea–I’m making a cup of tea is not the same as I’m making a tea cup. Likewise, a meeting of the faculty is a meeting, plain and simple, comprised of faculty. A faculty meeting is a kind of meeting. The first form describes a genus (meeting) populated by a species (faculty); the second introduces a new genus. English speakers do not have kinds of meetings (people meetings, carnival meetings, everyone-in-red meetings, cornhusker-fan meetings, and so forth). English speakers have meetings, plain and simple. Although English speakers can figure out what is meant by both forms, the compounded form adds unnecessary and sometimes misleading ambiguity.

Update [7/19/2015]: The metathesis of adnomial genitives (e.g., leaf of laurel) into denomial adjectives (laurel leaves, Lat. folia laurea) appears to have been popular during the late Republic and early Empire in Rome. Cicero shied from the pattern, preferring the genitives.

So, let’s say that someone studies metaphors of feces in the works of Chaucer, as in “fecopoetics“–read up on it for $105.00. Such a study would actually be a study of images of feces. (Chaucer left us no actual feces.) The order of nouns runs general-and-inclusive to specific-and-exclusive: a study, which is of images, and more specifically of images of feces. The last two nouns and their accompanying words get compounded to feces images, or rather, fecal images, which wrongly introduces a new genus of images. English speakers do not have categories of images (images of birds, images of Joe, images of animals recently returned from drinking, and so forth). English speakers have images, plain and simple. The compounding form fecal is then transferred backwards and upwards to make a new genus of study: fecal study. Or, more likely, pluralized to fecal studies.

Thus arises the proliferation of areas of study. We have fecal studies, gender studies, postcolonial studies, Marxist studies, and so forth. The implication is that each of these compounds represents a new field, which is incorrect. All belong to the same academic field: the study of culture, called “cultural studies.” And all employ the same general methods of the field of cultural studies. What differs is the object of study. One decent and clever fellow, Asa Mittman, wants to start the discipline of monster studies since he studies images of monsters. I honestly don’t see a university opening up a hiring line in monster studies. But I do see universities regularly hiring in cultural studies. So why hide the obvious strength of a mutually-supportive, communal enterprise in divisive compounds—or is that “compounds of divisiveness”?

UPDATE (1/29/2019): A terrific example just arrived by email, a description of a presentation. It reads:

Most methods for relation extraction from text rely on pre-trained entity resolution models in order to find the entities mentioned in text. Text-enhanced knowledge graph (KG) completion methods also rely on such an entity resolution model as a pre-processing step. We present a method to simultaneously learn entity resolution as well as relation extraction or KG completion without relying on a pre-trained entity resolution model or mention-level entity resolution data for training.

Wow! We can see a series of genitive + Noun-ion. So, to extract relations gets changed by turning the object (relations) into a plain adjective, and putting it before a noun ending in -ion (extraction). Similarly, a model that has been pre-trained to resolve entities becomes “pre-trained entity resolution model.” Try it at home. Consider a computer that has been programmed to calculate averages: would that be a programmed average-calculating computer. What a mess! But it’s the new norm. If you want to sound scientific, you take all of your descriptive or appositive phrases and cram them into adjectival positions. That would be positionally crammed adjectival phrase scientism.