Justus-Jonas Erker

📍 S2|02 D017

UKP Lab, TU Darmstadt

Hochschulstraße 10

64289 Darmstadt

Germany

Hey there 👋, I am a PhD Candidate in Natural Language Processing and fortunate to be supervised by Dr. Nils Reimers and Prof. Dr. Iryna Gurevych.

I am deeply fascinated by the ability to encode sequential information into Sentence-Transformers representations, enabling emergent properties and behaviors using just cosine similarity. The elegance of representing intricate sequential structures through such a simple yet powerful similarity measure is still stealing my sleep. Apart from my passion on representation learning and retrieval my research interests evolve around:

Retrieval-Augmented Generation
Synthetic Data Generation
Dialog Modeling
Efficient NLP

Hobbies:

In my free time I like to spend time outside in nature, either hiking (with my dogs) or mountain biking. You most probably will see me walking around with headphones, not because I am not willing to talk to you but because I am very passionate music listener. (I also play some music, preferred instruments: piano, guitar and drums)

news

Aug 10, 2023	I will start a Ph.D. in representation learning at the UKP Lab, TU Darmstadt supervised by Dr. Nils Reimers and Prof. Dr. Iryna Gurevych. in October 2023 🥳
May 2, 2013	My paper Imagination Is All You Need! got accepted in the 2023 Findings of the Association of Computational Linguistics (ACL). If you are also joining ACL 2023 in Toronto, Canada from the 9th to the 14th of July, hit me up. I would be keen to have a chat there!

selected publications

Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning

Justus-Jonas Erker, Stefan Schaffer, and Gerasimos Spanakis

In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023

Abs HTML

Inspired by the curvature of space-time, we introduce Curved Contrastive Learning (CCL), a novel representation learning technique for learning the relative turn distance between utterance pairs in multi-turn dialogues. The resulting bi-encoder models can guide transformers as a response ranking model towards a goal in a zero-shot fashion by projecting the goal utterance and the corresponding reply candidates into a latent space. Here the cosine similarity indicates the distance/reachability of a candidate utterance toward the corresponding goal. Furthermore, we explore how these forward-entailing language representations can be utilized for assessing the likelihood of sequences by the entailment strength i.e. through the cosine similarity of its individual members (encoded separately) as an emergent property in the curved space. These non-local properties allow us to imagine the likelihood of future patterns in dialogues, specifically by ordering/identifying future goal utterances that are multiple turns away, given a dialogue context. As part of our analysis, we investigate characteristics that make conversations (un)plannable and find strong evidence of planning capability over multiple turns (in 61.56% over 3 turns) in conversations from the DailyDialog dataset. Finally, we show how we achieve higher efficiency in sequence modeling tasks compared to previous work thanks to our relativistic approach, where only the last utterance needs to be encoded and computed during inference.
A Cancel Culture Corpus through the Lens of Natural Language Processing

Justus-Jonas Erker, Catalina Goanta, and Gerasimos Spanakis

In Proceedings of the First Workshop on Language Technology and Resources for a Fair, Inclusive, and Safe Society within the 13th Language Resources and Evaluation Conference, Jun 2022

Abs HTML

Cancel Culture as an Internet phenomenon has been previously explored from a social and legal science perspective. This paper demonstrates how Natural Language Processing tasks can be derived from this previous work, underlying techniques on how cancel culture can be measured, identified and evaluated. As part of this paper, we introduce a first cancel culture data set with of over 2.3 million tweets and a framework to enlarge it further. We provide a detailed analysis of this data set and propose a set of features, based on various models including sentiment analysis and emotion detection that can help characterizing cancel culture.