CLunch
CLunch is the weekly Computational Linguistics lunch run by the NLP group. We invite external and internal speakers to come and present their research on natural language processing, computational linguistics, and machine learning.
Interested in attending CLunch? Sign up for our mailing list here.
View older talks at the CLunch archive.
Past Talks
Past talks from the current and previous semesters are shown below. View older talks at the CLunch archive.
TTIC
October 26th, 2026
Olmo Hybrid: From Theory to Practice and Back
Recent work has demonstrated the potential of non-transformer language models, especially linear recurrent neural networks (RNNs) and hybrid models that mix recurrence and attention. Yet there is no consensus on whether the potential benefits of these new architectures justify the risk and effort of scaling them up. To address this, we provide evidence for the advantages of hybrid models over pure transformers on several fronts. First, theoretically, we show that hybrid models do not merely inherit the expressivity of transformers and linear RNNs, but can express tasks beyond both, such as code execution. Putting this theory to practice, we train Olmo Hybrid, a 7B-parameter model largely comparable to Olmo 3 7B but with the sliding window layers replaced by Gated DeltaNet layers. We show that Olmo Hybrid outperforms Olmo 3 across standard pretraining and mid-training evaluations, demonstrating the benefit of hybrid models in a controlled, large-scale setting. We find that the hybrid model scales significantly more efficiently than the transformer, explaining its higher performance. However, its unclear why greater expressivity on specific formal problems should result in better scaling or superior performance on downstream tasks unrelated to those problems. To explain this apparent gap, we return to theory and argue why increased expressivity should translate to better scaling efficiency, completing the loop. Overall, our results suggest that hybrid models mixing attention and recurrent layers are a powerful extension to the language modeling paradigm: not merely to reduce memory during inference, but as a fundamental way to obtain more expressive models that scale better during pretraining.
Johns Hopkins University
October 19th, 2026
Toward Human-Level Machine Social Intelligence for Adaptive AI Partners
Despite our tremendous progress in AI, current AI agents still cannot adequately understand, interact with, and learn from humans in real-world settings over extended periods. One of the key missing ingredients is social intelligence. In this talk, I will discuss how we can engineer human-level machine Theory of Mind for adaptive AI partners. I will begin by showing how cognitive modeling and foundation models can be combined to build scalable, robust, model-based approaches for multimodal Theory of Mind. I will then discuss how Theory of Mind reasoning can enhance multimodal human-AI collaboration. Finally, I will present some of our recent efforts on scaling Theory of Mind in the real world, including (1) internalizing explicit mental reasoning into LLMs through self-supervised reinforcement learning, (2) ground mental reasoning to the 3D world via generative world models, and (3) latent user thoughts as a new training data modality for enhancing human-model alignment.
NYU
October 12th, 2026
Discovering and Engineering the Computation Underlying Large Intelligent Agents
The richness of language and intelligent behavior has often been attributed to latent compositional structure. Can we build tools for discovering how deep networks learn and represent this latent structure implicitly? And more importantly, can we use this knowledge to improve generalization in largely structure-less general purpose models or refine our understanding of the world they describe? In this talk, I present three perspectives to answer these questions. I will discuss experimental methods to functionally characterize the space of learnt solutions in LLMs and demonstrate how this understanding can be used to improve their empirical generalization in a gradient free manner, sometimes by as much as 30% points on language understanding benchmarks. Following that, I show how to decipher the structure of another (black box) language-like system, the naturally arising communication system of sperm whales in the wild, discovering for the first time a unique combinatorial communication system. Finally, I apply insights from these results to equip embodied agents with a latent language of thought– hierarchical and compositional—and show how it can enable long-horizon reasoning and planning in these systems.
Northeastern University
October 5th, 2026
The Language of Language Models
As language models have grown nearly as fluent as human writers, the details of how they use language have become far less studied. However, humans absorb linguistic patterns from the language they're exposed to, meaning LLM-generated text will likely have wide-ranging effects on human linguistics. In this talk, I ask what studying LMs as language users reveals about how they actually process and produce natural language, and how that will shape our language use in turn.\n First, I show that LMs exhibit systematic typological biases in language learning for both morphological and syntactic phenomena that often diverge from human linguistic universals, though data effects can override these biases at scale. I will then discuss how LMs' accommodation behavior differs from humans in both synthetic and real human-LLM conversations, with LLMs strongly overfitting to users' linguistic style far beyond human norms. In contrast, humans in dialogue with LLMs accommodate them as they would a human interlocutor. Together, these findings show that LLMs bring artificial learning and pragmatic biases that diverge from human behavior, with significant implications for both long-term linguistic diversity and the trajectory of natural language as LLMs continue to be integrated into everyday life.
University of South Florida
September 28th, 2026
Lost in Translation, and Found: Detecting and Interpreting Translation Effects in Large Language Models
Large language models are able to generate highly fluent text, in large part because they are trained on massive amounts of data. This data may contain “translationese”: hallmarks which distinguish translated texts from texts originating in the language. Though individual translated texts are often fluent and preserve meaning, at a large scale, the presence of translated texts in training data negatively impacts performance and in test data inflates evaluation scores. In this work, I show that linguistic tools can be used to mitigate the surface-level features (and thus harmful impacts) of translationese. Further, I investigate how accurately texts can be automatically classified as original or translated, and what features encoded within neural classifiers enable them to distinguish original and translated English texts much more accurately than humans.
Mila - Québec AI Institute
September 21st, 2026
“A natural language interface for everyday life”: Unpacking the social and political functions of AI capabilities discourses
Marketing materials position generative AI models as smarter, faster, and more useful than ever, with “with built-in thinking that puts expert-level intelligence in everyone’s hands” (OpenAI). A substantial body of scholarship and journalistic reporting has characterized such discourses as hype. I’ll describe recent work in which, rather than interrogating particular claims for their truth value, we are interested in the form these discourses take: What is required to make statements like “built-in thinking” sayable, “poetry” and “understanding cancer risk” comparable as items that belong in the same list, and all of these components interpretable as features of an advanced technology that is also a desirable consumer product? I'll describe how we analyze a collection of marketing materials from four companies and interpret what we refer to as AI capabilities discourses, which we argue render a wide variety of social activities as interchangeable, where situated, relational, embodied practices are cast as things subject to AI capabilities within a broader landscape of increasingly flexibilized, precarious cognitive work—thus narrowing the terrain for contestation.
Harvard University
September 14th, 2026
Customizing Foundation Models on a Budget
The rising cost of frontier models is pushing organizations to adapt existing models rather than train new ones, often under tight compute and data budgets. This raises two practical challenges: tailoring model size to deployment constraints and selecting the most useful data for fine-tuning. In this talk, I will present work addressing each. First, I will introduce boomerang distillation, a surprising phenomenon we identified in which patching teacher layers into a distilled student produces intermediate-sized models without any additional training. I will show that performance interpolates smoothly between the student and teacher as model size increases. Next, I will introduce an efficient patching algorithm that improves these intermediate models under a fixed inference-time compute budget and extend the method to post-trained models. Second, I will examine targeted instruction selection and show that only gradient-based representations reliably predict downstream performance. These representations yield the strongest results at small data budgets, while random sampling becomes competitive as budgets grow. Together, these results offer practical guidance for adapting foundation models under real-world compute and data constraints.
Massachusetts Institute of Technology
April 20, 2026
New Advances in Multimodal and Agentic Reasoning
Today's language models are increasingly capable of reasoning over multiple steps with verification and backtracking to solve challenging problems. However, multimodal reasoning agents that can reason and act over an integrated set of modalities such as text, images, audio, video, and physical sensors are sorely lacking, and can pave the way for a next frontier of AI. I will describe our group's work on advancing the frontiers of multimodal and agentic reasoning, from new multimodal reasoning benchmarks and foundation models, to long-horizon and self-evolving agents, with applications to science and engineering.
Carnegie Mellon University
April 13, 2026
Inducing Functions to Improve LLM Agents
Programs provide a structured, reusable, and verifiable means for people to carry out digital tasks. We show that LLM-based agents also benefit from generating code, executing that code in an environment, and abstracting functions from correct code. We present tool induction methods that build libraries of reusable functions online as the agent interacts with the environment. Our methods allow agents to carry out tasks more accurately and efficiently in grounded environments including performing tasks on the web and answering questions about structured data and images. We also find that induced tools make agent trajectories easier to verify for people and generalize well to complex tasks with shared sub-structure.
University of Texas at Austin
April 6, 2026
Multi-Model Training for Multi-Agent Communication Skills
As we scale from individual agents to teams of agents, inter-agent communication will become increasingly important. In this talk, I will describe a general paradigm for teaching multi-agent communication skills through multi-model reinforcement learning, which I will illustrate via three key collaborative skills: expressing confidence in a calibrated way, responding robustly to positive and negative persuasion, and expressing reasoning faithfully. I will show how these problems can be framed in terms of speaker-listener games, and how this framing allows us to teach models collaborative skills, often using games simulated on smaller models to train larger models. Time permitting, I will also touch on how we can infer skills to combine multiple agents more productively.
Carnegie Mellon University
March 30, 2026
Making AI Systems Work for Imperfect Humans
General-purpose AI systems are increasingly envisioned to support users on any tasks. However, a prerequisite for this vision is that the user need is clearly “communicated” to the AI, which is in itself a nontrivial step: users often begin with vague or unformed goals, and even when they have a clear idea of what they want, their instructions may be ambiguous or misaligned with how the AI interprets them. Simply put, humans are not perfect oracles of their own intentions. How can we design AI systems that better support imperfect users? In this talk, I will share some of our recent work aimed at making AI more practically useful. This involves reflections on the right representations and metrics to capture user needs and task utility, and methods for improving the goal capturing, either by training the model to better guess the user need, or training the human to better express themselves.
Ohio State University
March 23, 2026
Computer Use: Modern Moravec’s Paradox
Computer-use agents reveal a modern form of Moravec’s Paradox: today’s AI excels at symbolic tasks like math and coding yet struggles with the everyday cognitive work humans perform effortlessly on computers. We will discuss the inherent challenges of computer use such as idiosyncratic environments and contextual understanding. We also argue that computer use is not only one of the hardest frontiers for AI but may also be the most important, because it can potentially offer the next Internet-scale learning opportunity and the most immediate path toward general, practical, goal-directed machine intelligence.
Northeastern University
March 16, 2026
Beyond the Surface: How Post-Training Artifacts Shape LLM Diversity and Safety
Post-training alignment makes LLMs helpful, but also introduces unintended artifacts. This talk explores two such artifacts, their impact on LLM diversity and safety, and presents corresponding solutions. (1) We begin with a data-driven artifact from RLHF, showing how a "typicality bias" in human preferences leads to mode collapse. I will introduce Verbalized Sampling, a principled prompting method that restores diversity across creative writing, social simulation, and synthetic data generation tasks. (2) Next, we shift to a mechanistic artifact from SFT, uncovering how LLMs encode "harmfulness" and "refusal" separately. This insight demystifies how jailbreaks work and enables the Latent Guard, an intrinsic safeguard built on the model's internal beliefs. Together, these findings call for an artifact-aware approach that looks beyond surface-level behaviors when building and evaluating LLMs.
University of Massachusetts Lowell
March 02, 2026
AI at Two Levels: From Efficient Pre-Training to Co-Creative Science
This talk addresses AI research at two levels: efficient methods for training large language models, and the emerging practice of using those models as collaborative partners in scientific discovery. Over the past few years, scaling large language models (LLMs) has become the go-to AI solution for solving progressively more complicated tasks. However, as the models scaled towards hundreds of billions of parameters, not just training them from scratch, but even fine-tuning to adapt to specific tasks has become computationally expensive and unmanageable for most practitioners. This has led to a crisis, in which only a few well-funded industry labs have the resources to develop high-quality LLMs. In response to these developments, numerous parameter-efficient techniques revolutionized the accessibility of fine-tuning LLMs. However, until recently, such methods were not available for pre-training. In the first part of the talk, I will present our work on ReLoRA (ICLR 2024), the first parameter-efficient method for training models from scratch, which utilizes low-rank updates to train high-rank networks, with demonstrated results for models with up to 1.3 billion parameters and throughput improvements of up to 40%. Since its publication, this line of work has grown into an active subfield, with dozens of new low-rank pre-training methods building on these ideas. In the second part, I will turn to the question of how the practice of science itself is changing as human researchers begin to collaborate with AI models. Today's models are trained on virtually all of codified human knowledge. But how can we teach them to do science with us, rather than merely for us? I will describe the Human–AI Co-Creativity Consortium (HAI3C), a cross-disciplinary effort across multiple universities with the goal of building the infrastructure, datasets, and evaluation frameworks needed to support genuine human-AI scientific collaboration. I will discuss our first pilot in mathematics, in which domain experts and multiple AI agents work together on open problems, and how this process generates annotated reasoning traces that can inform the next generation of models.
Ben-Gurion University of the Negev
February 23, 2026
Building Character: Tokenization in Theory and in Practice
The end-to-end nature of recent NLP models, most prominently large language models (LLMs), makes it hard for us to figure out how their individual processing steps are affected by properties of text and language, for example how they represent words in relation to their low-level composition or in alignment with how humans perceive them. We can find instances where models struggle with lexical changes in register, domain, or innovation, but the underlying mechanisms still mostly elude us. In this talk, I will focus on the lexical and sub-lexical levels of LLM representations, challenging models with recognition of words and the processes they are formed through, looking at subword tokenization algorithms from the point of view of their downstream performance requirements and the challenges posed by different languages. I will talk about SaGe/MINT, a subword tokenizer that incorporates context into the vocabulary creation objective; BoundlessBPE, which allows breaking past the limits of regex pretokenization; Splinter, a pre-tokenization algorithm that relinearizes Hebrew text to conform the concatenative tokenization pipeline with Semitic templatic morphology; and CharBench, an evaluation dataset for surface-form tasks such as character counting.
Carnegie Mellon University (Incoming Assistant Professor), Allen Institute for AI
February 16, 2026
Deep Research Agents: From Evaluation to Training and Open Challenges
Deep research agents that plan, search, and synthesize evidence into long-form, well-attributed reports are gaining rapid traction, yet most research still focuses on short-form, easily verifiable tasks. In this talk, I will present our recent work on evaluation, model development, and open challenges for deep research agents. First, I will briefly discuss the evaluation framework developed for OpenScholar (Nature, 2026), designed to assess long-form, citation-grounded scientific synthesis. I will describe our new multi-faced approach which combines rubric-based automated metrics, fine-grained citation accuracy verification, and expert preference judgments, and share key findings. Next, I will present DR Tulu, the first open model trained end-to-end for long-form deep research via Reinforcement Learning with Evolving Rubrics (RLER), where evaluation rubrics co-evolve with the policy to provide increasingly discriminative, on-policy feedback. DR Tulu-8B matches or outperforms proprietary systems like OpenAI Deep Research or Gemini 3 Pro across science, healthcare, and general-domain benchmarks while remaining significantly smaller and cheaper. Finally, I will draw on insights from our NeurIPS 2025 MMU-RAGent competition, which evaluates deep research systems through both static benchmarks and live user-facing arena comparisons. Key takeaways include limitations of current inflexible deep research agent designs as well as evaluation protocols. I will conclude with a roadmap toward more reliable deep research evaluation and training.
New York University
February 09, 2026
Linear Truth Encoding in LLMs: Emergence and Generalization
Large language models often exhibit surprisingly simple linear structure: directions and subspaces in their hidden states align with human-interpretable concepts. In this talk, I'll introduce the linear representations hypothesis and zoom in on a particularly striking case: linear "truth" encoding. Using a transparent one-layer transformer toy model with a deliberately simple, well-defined notion of truth, we trace the end-to-end mechanism by which a subspace emerges that linearly separates true from false statements. This analysis is based on a simple generative story in which true statements tend to co-occur with other true statements, and likewise for false ones. It turns out that this structure alone suffices to produce a fully characterizable truth subspace. In modern LLMs trained on natural data, however, the picture is messier. Recent results suggest that any apparent "truth direction" can be brittle: it may shift under distribution change, conflate truth with surface cues, or fragment across domains. I'll use these challenges to motivate ongoing work that moves from controlled setups to realistic regimes, probing real models under distribution shift and disentangling different notions of truth. The emerging picture is more complex than previously assumed: what looks like a single "truth subspace" is often a superposition of directions, some spurious and domain-specific, others more abstract and transferable.
Massachusetts Institute of Technology (MIT)
December 08, 2025
Just Asking Questions
In the age of deep networks, "learning" almost invariably means "learning from examples". We train language models with human-generated text and labeled preference pairs, image classifiers with large datasets of images, and robot policies with rollouts or demonstrations. When human learners acquire new concepts and skills, we often do so with richer supervision, especially in the form of language---we learn new concepts from examples accompanied by descriptions or definitions, and new skills from demonstrations accompanied by instructions. Crucially, language-based supervision involves not only instructions but *questions*---students ask questions to elicit the most useful pieces of supervision, and teachers ask questions to probe student knowledge and encourage them to acquire new skills or aspects of understanding on their own. This talk will focus on a few recent projects focused on building computational models that can ask good questions for both learning and teaching, with applications spanning LM alignment, policy learning, and education. This is joint work with Belinda Li, Alex Tamkin, Noah Goodman, Andi Peng, Ilia Sucholutsky, Nishanth Kumar, Julie A Shah, Andreea Bobu, Alexis Ross, Gabe Grand, Valerio Pepe and Josh Tenenbaum.
Meta Fundamental AI Research (FAIR)
November 21, 2025
Self-Improvement of LLMs
Classically, learning algorithms were designed to improve their performance by updating their parameters (weights), while keeping other components, such as the training data, loss function, and algorithm, fixed. We argue that fully intelligent systems will be able to self-improve across all aspects of their makeup. We describe recent methods that enable large language models (LLMs) to self-improve in various ways, increasing their performance on tasks relevant to human users. In particular, we describe methods whereby models are able to create their own training data (self-challenging), train on this data using themselves as their own reward model (self-rewarding), and train themselves to better provide their own rewards (meta-rewarding). We then discuss the future of self-improvement for AI and key challenges that remain unresolved.
University of Southern California
November 10, 2025
Understanding Model Nature of LLMs
The widespread adoption of large language models (LLMs) places a responsibility on the AI research community to understand them and develop abstractions that encapsulate their behavioral tendencies. In this talk, I will describe my group’s attempts to make sense of LLMs at three levels: their memorization of pre-training data, their internal mechanisms, and their advice-giving behavior. First, I will introduce the Hubble project, in which we have pre-trained LLMs (up to 8B parameters) on controlled pre-training corpora to understand when and how they memorize sensitive data related to copyright risks, privacy leakage, and test set contamination; we envision these models as a valuable open-source resource for scientific inquiry into LLM memorization. Next, I will describe my group’s work on understanding how language models work internally and how we can make these insights actionable. Finally, I will highlight a recent collaboration with USC oncologists in which we uncover LLM sycophancy issues that arise when patients ask these models for medical advice.
Yale University
November 03, 2025
Evaluating and Understanding LLMs: From Scientific Reasoning to Alignment as Judges
We present our recent work on evaluating and understanding large language models in scientific contexts and understanding them in context of evaluation-generation capabilities. First, we'll introduce SciArena, an open evaluation platform for literature-grounded scientific tasks that uses expert preferences to rank models on long-form, literature-grounded responses. The platform currently supports a broad set of open and proprietary models and has already accumulated a large pool of high-quality preferences. Using these data, we release SciArena-Eval, a meta-evaluation benchmark for training and stress-testing automated judges on science tasks. We will then turn to scientific problem solving. We discuss a holistic suite of scientific reasoning tasks, and a new framework for studying the role of knowledge in scientific problem solving and its interaction with reasoning. Our analysis shows that retrieving task-relevant knowledge from model parameters is the primary bottleneck for science reasoning; in-context external knowledge systematically helps even strong reasoning models; and improved verbalized reasoning increases a model’s ability to surface the right knowledge. Finally, if there is time, we will present a work on generation–evaluation consistency and show that models that judge well also tend to generate outputs that align with human preferences. This enables alignment benchmarking that evaluates models in their role as judges without scoring their generations directly.
Toyota Technological Institute at Chicago (TTIC)
October 27, 2025
Language Models as User Models
If we want to build collaborative language models, we'll need to find the right training objective. One promising direction involves simulating human users at scale and using these simulations as a training signal to develop models that better understand and interact with people. In this talk, I’ll discuss key challenges in simulating human behavior, ranging from hallucinations and coherence to knowledge consistency and memory. Then, I’ll discuss some ongoing work and outline future directions for building more human-like user simulators and training more useful, collaborative models.
University of Southern California
October 20, 2025
Building Personalized AI Assistants: From Task Execution to Human Alignment
Large language models have transformed how we interact with technology, yet most current systems remain reactive, executing tasks without understanding user context or values. This talk explores the path toward personalized, trustworthy AI assistants that can reason, adapt, and align with human preferences. I will introduce Computer-Using Agents (CUAs) that combine GUI operations and code generation to efficiently complete real-world tasks, and present CoAct-1, a multi-agent system that coordinates planning and execution. I will then discuss SEA, an algorithm for uncovering LLM knowledge deficiencies, and WildFeedback, a framework that learns user preferences from natural interactions. Finally, I will highlight ongoing work on human-centered and culturally aware AI, from understanding intention and reasoning consistency to enabling proactive, value-aligned collaboration. Together, these advances move us closer to AI systems that truly understand and assist people.
Northwestern University
October 13, 2025
From Large Language Models to Large Agent Models: The Reasoning Interface for World Interaction
The leap from Large Language Models to Large Agent Models is to unfold reasoning as multi-turn interactions with the world. We take the first step by formalizing agent training as a Markov Decision Process (MDP), introducing an agent reasoning interface (RAGEN) to avoid “reasoning collapse”, where LLM agents loop into repetitive reasoning and fail to explore. Extending to Partially Observable MDPs (POMDPs), we propose VAGEN, training VLM agents to internalize world models for state estimation (“what is seen”) and transition modeling (“what comes next”). We further reveal shallow exploitation over true exploration, and use self-play to inject world model knowledge to diversify exploration. Finally, we introduce cognitive maps as reasoning interfaces for VLMs to integrate partial observations into coherent world beliefs in the 3D space. In the end, we will lay out how these advances chart a path to agents that simulate, explore, and actively construct internal world models, as a decisive step from LLMs to LAMs.
Northeastern University
September 22, 2025
Some potential uses for and (current) limitations of mechanistic interpretability methods
LLMs are increasingly being deployed in high-stakes domains like healthcare and law. But the black box nature of such models brings real risks. Emerging "mechanistic" interpretability methods seek to characterize the inner-workings of generative models, providing insights into how they come to make particular predictions. This may in turn offer transparency and provide levers to improve model outputs in targeted ways. But while the science of mechanistic interpretability has progressed, the degree to which such methods might offer actionable insights in realistic and domain-specific tasks remains unclear. In this talk I will discuss some recent and ongoing work on such techniques, highlighting both potential applications (e.g., for transparency in healthcare settings, and for efficient model distillation) and some current limitations.
University of Maryland
September 15, 2025
Robust, Fair, and Culturally-Adaptive Social Cognition in LLMs
Successful communication in natural language is dependent upon interlocutors’ ability to build and maintain common ground, or a mutual understanding of the world. Thus, for LLMs to understand and serve the needs of their human users, they require not only basic knowledge of the world (“common sense”), but also the ability to engage in social cognition: reasoning under uncertainty about peoples’ beliefs, intents, and other mental states. In the first part of the talk, I will discuss my lab’s work on measuring robustness of LLM inference under uncertainty and inherent challenges of establishing ground truth in tasks of plausibility assessment. In the second part of the talk, I will present our findings concerning the fairness of social inference in LLMs, in particular the finding that LLMs demonstrate human-like inter-group empathy gaps. Finally, I will discuss my lab’s work developing evaluations of cross-cultural knowledge based on social theories of cultural consensus.
University of Maryland
September 08, 2025
A Human-Centered Approach to Trustworthy Machine Translation
Machine Translation (MT) tools are more accessible than ever, but a key challenge remains: how can we help users rely on them appropriately, especially when they don't understand one of the languages involved? This talk will describe how my group has addressed this problem in recent years. I'll begin by sharing the results of user studies designed to understand how people build trust in and reliance on MT. These findings led us to reframe MT evaluation to provide actionable feedback that helps people assess the impact of translation errors. To achieve this, we introduce techniques based on large language models to ask and answer questions about a translated text, and to detect errors in speech translations with minimal supervision. Throughout the talk, I will highlight open questions and research directions for designing language technology that genuinely supports people in communicating across language barriers.