Research Projects

Haiku Generation with Causal Language Modeling
M.S. Project — 2018-2020

We examine the theoretical foundations of causal language modeling, where natural language is modeled in terms of the probability distribution of a token in a sequence. This probability distribution is dependent on the values of all of the preceding tokens in the sequence.

Due to time constraints and real-life factors, the final deliverable was changed from a traditional thesis to a project. The project was to be an implementation of an algorithm to generate haiku. This algorithm need not be original.

We tried a number of generative language models, from using Markov chains and naive LSTM networks, to using textgenrnn by Max Woolf, and finally by fine-tuning a pretrained GPT-2 model on haiku scraped from online haiku submission and award websites.

As expected, the results from using a fine-tuned GPT-2 model were by far the best. Some generated haiku were eerily cogent, and many (though not all) generated haiku display many traits of haiku. A sample of some of the generated haiku is given below.

model seed haiku
Markov -
summer / all these extra prayers / at the dead puppies
GPT-2 commandline 3191532796 spoonfuls / of medication / and loneliness
GPT-2 web 2607968983 vodka in the bottle / i go through / the exam
GPT-2 web 2913164516 thunder / the stock market goes up / and down
GPT-2 commandline 718585002 http haiku / some thoughts im afraid of / all alone at the coffee shop
GPT-2 web 1691016217 bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang bang
GPT-2 web 3138988667 system failure / a distant train / making a sudden silence
GPT-2 web 921792121 may the force of life / reminding me / not to call it quits
GPT-2 web 1338062952 team meeting / everyone is still / talking about joining us
Understanding A Fast Contour-Integral Eigensolver for Non-Hermitian Matrices
B.S. Project — 2017-2018

We examine a new complex contour integral-based eigensolver algorithm using ideas from functional analysis. This eigensolver algorithm extends previous eigensolvers that typically required some special structure, to an eigensolver that works on a more general class of complex-valued matrices.

The algorithm consists of two stages. First, it searches the complex plane for regions dense with eigenvalues. Second, it combs through those regions to find the eigenvalues using a modified form of the FEAST algorithm.

Due to time constraints we were only able to examine the first stage in detail, and for completeness provide a description for the second stage. Our work was exploratory in nature, which in practice consisted of building and testing a Matlab implementation of the first stage.