Nicholas Jennings
Here's all my research projects.
Procedural content generation (PCG), the process of algorithmically creating game components instead of manually, has been a common tool of game development for decades. Recent advances in large language models (LLMs) enable the generation of game behaviors based on player input at runtime. Such code generation brings with it the possibility of entirely new gameplay interactions that may be difficult to integrate with typical game development workflows. We explore these implications through GROMIT, a novel LLM-based runtime behavior generation system for Unity. When triggered by a player action, GROMIT generates a relevant behavior which is compiled without developer intervention and incorporated into the game. We create three demonstration scenarios with GROMIT to investigate how such a technology might be used in game development. In a system evaluation we find that our implementation is able to produce behaviors that result in significant downstream impacts to gameplay. We then conduct an interview study with n=13 game developers using GROMIT as a probe to elicit their current opinion on runtime behavior generation tools, and enumerate the specific themes curtailing the wider use of such tools.
Paper
Genetic studies indicate that more than 50% of women are genetically tetrachromatic, expressing four distinct types of color photoreceptors (cone cells) in the retina. At least one functional tetrachromat has been identified in laboratory tests. We hypothesize that there is a large latent group in the population capable of fundamentally richer color experience, but we are not yet aware of this group because of a lack of tetrachromatic colors in the visual environment. This paper develops theory and engineering practice for fabricating tetrachromatic colors and potentially identifying tetrachromatic color vision in the wild.
Paper
I'm working on several projects related to beyond-real interactions in virtual reality. I tend to focus on interaction signifiers, and on multi-body interactions (where the user controls both their 1st person virtual body and 1+ 3rd person bodies at the same time). The gif (and the linked video) shows an example where 2 additional bodies (one small, one big) are used to enable interacting with an enviroment at multiple scales.
Video Link
Computational design tools can automatically generate large quantities of viable designs for a given design problem. This raises the challenge of how to enable designers to efficiently and effectively evaluate and select preferred designs from a large set of alternatives. In GeneratiVR, we present two novel interaction techniques to address this challenge, by leveraging Virtual Reality for rich, spatial user input. With these interaction methods, users can directly manipulate designs or demonstrate desired design functionality. The interactions allow users to rapidly filter through an expansive design space to specify or find their preferred designs.
GeneratiVR Paper VR or Not? Paper Video
Android virtualization software can be a powerful tool for app development, but it can also be exploited for malicious purposes. During my time on the Anti-Virtualization research project led by Eleonora Losiouk, I conducted a review of prior work to create a taxonomy of known virtualization-based malwares and attacks for Android, built Gradle scripts for virtualizing malwares and assessing their detectability by commercial antivirus apps, and used Bash, Gradle, and Python scripts to create an semi-automated system for identifying virtualization-based malwares from virus databases, allowing for more in-depth analysis.