What I’m interested in at the moment
Recently I’ve been thinking about reactive probabilistic programming languages and using them to build dynamical systems models of cooperation between agents. This is appealing because it pulls together a lot of the things I like: functional programming, non-equilibrium statistical mechanics, and a longstanding curiosity about cognition and convention.
Physics
At Berkeley, I worked at the intersection of computational statistics and physics. These fields turn out to be very deeply connected, although there is surprisingly little cross-talk (possibly because both statisticians and physicists each have a different, totally incomprehensible jargon).
In particular, I worked on the Microcanonical Hamiltonian Monte Carlo sampler, which is inspired, as the name suggests, by the microcanonical ensemble in statistical mechanics. I worked on applying this to many-body problems in condensed matter physics. I also spent time thinking about flow-based models, and non-equilibrium physics, which led to counterdiabatic Hamiltonian Monte Carlo.
I’m also interested in how numerical methods in the applied sciences can be bolstered by theorem provers like Lean. As a step in that direction, I’ve been working on using the type system of a statically typed language to enforce correctness of the fusion rules for Lie group representations (and other fusion categories). Physicists never get exposed to this kind of computer science (try asking a physicist what the lambda calculus is), but I think this sort of approach will prove increasingly powerful in the future.
Functional programming
After my PhD, I worked in industry for a few years, on a variety of projects. One was a natural language interpreter. I think symbolic AI is conceptually misguided, but the opportunity to code full-time in Haskell was very appealing. One thing I worked on was incorporating multi-world expressions in a compositional semantics and grammar via very general recursion schemes from category theory (histomorphisms). The result was fairly fancy natural language parser and interpreter in the spirit of construction grammar. I continue to think that the connection between free functors and fragment grammars is very beautiful, and would love to finish this project some day.
I also finally worked out how to use dependent types for categorial grammars, and how to parse by lazily generating all possible sentences and folding them into a parser.
A more recent thing I’ve worked on is to extend reactive programming to handle stochasticity, a project that Manuel Bärenz is mainly responsible for, to get online particle filters expressed as Feynman-Kac processes. This is really cool!
Bayesian models of communication
During my PhD, I worked on unifying logical and statistical perspectives on meaning in natural language using probabilistic models of pragmatic reasoning.
The general idea is to model the interpretation of a linguistic expression (e.g. a sentence) as a process of Bayesian inference, to ask: given that this sentence is true (or, more to the point, given that someone said it) what must the world be like. This turns out to be a nice viewpoint for integrating a traditional logical perspective on meaning with an information-theoretic one, as well as handling semantic and pragmatic meaning in a single framework. I say a little bit more about that in the introduction of my dissertation.
Below are some of the projects I never quite finished; for one reason or another, a lot of the most interesting projects in grad school never ended up published. (For the others, see Google Scholar.)
Direction 1 of PhD research: scaling the models
Metaphor and Linguistic Creativity
This paper explores the technical and conceptual consequences of a model of meaning where the listener’s prior is over a vector space. This allows integration with word embeddings.
(Cohn-Gordon and Bergen).Lost in Machine Translation: A Method to Reduce Meaning Loss
This and some related papers look at models of meaning where the utterance space is recursively generated. This allows for integration with a neural semantics, in particular a conditional language model.
(NAACL 2019 - Cohn-Gordon and Goodman).
Direction 2 of PhD research: enriching the models
- Verbal Irony, Pretense, and the Common Ground
This paper looks at models where the listener is uncertain not only of the state of the world, but also the state of the common ground. In a nutshell, if I tell you something, you learn not only that thing, but also that I believed you didn’t already know it (an inference about my belief about your prior). A speaker can leverage this to communicate and that yields a very satisfying account of a very distinctive feature of natural languages, namely sarcasm.
(Cohn-Gordon and Bergen).
- The Pragmatics of Multiparty Communication
This project looked at what novel dynamics emerge when there are multiple listeners, so any one can explain away a speaker’s utterance on the assumption that it was directed towards a different listener. The interesting idea lurking in the background is that the joint common ground is not the union of the pairwise common grounds; at some point I should sit down and write out clearly what this means. It also gives a nice model of the semantics of proper names as presupposed variable assignments, which shows how parts of a 1st order logical semantics can be lifted into a Bayesian model.
(Cohn-Gordon, Levy, and Bergen).