We examine the theoretical foundations of causal language modeling, where natural language is modeled in terms of the probability distribution of a token in a sequence. This probability distribution is dependent on the values of all of the preceding tokens in the sequence.
Due to time constraints and real-life factors, the final deliverable was changed from a traditional thesis to a project. The project was to be an implementation of an algorithm to generate haiku. This algorithm need not be original.
We tried a number of generative language models, from using Markov chains and naive LSTM networks, to using textgenrnn by Max Woolf, and finally by fine-tuning a pretrained GPT-2 model on haiku scraped from online haiku submission and award websites.
As expected, the results from using a fine-tuned GPT-2 model were by far the best. Some generated haiku were eerily cogent, and many (though not all) generated haiku display many traits of haiku. A sample of some of the generated haiku is given below.
||summer / all these extra prayers / at the dead puppies|
|GPT-2 commandline||3191532796||spoonfuls / of medication / and loneliness|
|GPT-2 web||2607968983||vodka in the bottle / i go through / the exam|
|GPT-2 web||2913164516||thunder / the stock market goes up / and down|
|GPT-2 commandline||718585002||http haiku / some thoughts im afraid of / all alone at the coffee shop|
|GPT-2 web||1691016217||bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang|
|GPT-2 web||3138988667||system failure / a distant train / making a sudden silence|
|GPT-2 web||921792121||may the force of life / reminding me / not to call it quits|
|GPT-2 web||1338062952||team meeting / everyone is still / talking about joining us|
We examine a new complex contour integral-based eigensolver algorithm using ideas from functional analysis. This eigensolver algorithm extends previous eigensolvers that typically required some special structure, to an eigensolver that works on a more general class of complex-valued matrices.
The algorithm consists of two stages. First, it searches the complex plane for regions dense with eigenvalues. Second, it combs through those regions to find the eigenvalues using a modified form of the FEAST algorithm.
Due to time constraints we were only able to examine the first stage in detail, and for completeness provide a description for the second stage. Our work was exploratory in nature, which in practice consisted of building and testing a Matlab implementation of the first stage.