CLunch
CLunch is the weekly Computational Linguistics lunch run by the NLP group. We invite external and internal speakers to come and present their research on natural language processing, computational linguistics, and machine learning.
Interested in attending CLunch? Sign up for our mailing list here.
View older talks at the CLunch archive.
Past Talks
Past talks from the current and previous semesters are shown below. View older talks at the CLunch archive.
Massachusetts Institute of Technology
April 20, 2026
New Advances in Multimodal and Agentic Reasoning
Today's language models are increasingly capable of reasoning over multiple steps with verification and backtracking to solve challenging problems. However, multimodal reasoning agents that can reason and act over an integrated set of modalities such as text, images, audio, video, and physical sensors are sorely lacking, and can pave the way for a next frontier of AI. I will describe our group's work on advancing the frontiers of multimodal and agentic reasoning, from new multimodal reasoning benchmarks and foundation models, to long-horizon and self-evolving agents, with applications to science and engineering.
Carnegie Mellon University
April 13, 2026
Inducing Functions to Improve LLM Agents
Programs provide a structured, reusable, and verifiable means for people to carry out digital tasks. We show that LLM-based agents also benefit from generating code, executing that code in an environment, and abstracting functions from correct code. We present tool induction methods that build libraries of reusable functions online as the agent interacts with the environment. Our methods allow agents to carry out tasks more accurately and efficiently in grounded environments including performing tasks on the web and answering questions about structured data and images. We also find that induced tools make agent trajectories easier to verify for people and generalize well to complex tasks with shared sub-structure.
University of Texas at Austin
April 6, 2026
Multi-Model Training for Multi-Agent Communication Skills
As we scale from individual agents to teams of agents, inter-agent communication will become increasingly important. In this talk, I will describe a general paradigm for teaching multi-agent communication skills through multi-model reinforcement learning, which I will illustrate via three key collaborative skills: expressing confidence in a calibrated way, responding robustly to positive and negative persuasion, and expressing reasoning faithfully. I will show how these problems can be framed in terms of speaker-listener games, and how this framing allows us to teach models collaborative skills, often using games simulated on smaller models to train larger models. Time permitting, I will also touch on how we can infer skills to combine multiple agents more productively.
Carnegie Mellon University
March 30, 2026
Making AI Systems Work for Imperfect Humans
General-purpose AI systems are increasingly envisioned to support users on any tasks. However, a prerequisite for this vision is that the user need is clearly “communicated” to the AI, which is in itself a nontrivial step: users often begin with vague or unformed goals, and even when they have a clear idea of what they want, their instructions may be ambiguous or misaligned with how the AI interprets them. Simply put, humans are not perfect oracles of their own intentions. How can we design AI systems that better support imperfect users? In this talk, I will share some of our recent work aimed at making AI more practically useful. This involves reflections on the right representations and metrics to capture user needs and task utility, and methods for improving the goal capturing, either by training the model to better guess the user need, or training the human to better express themselves.
Ohio State University
March 23, 2026
Computer Use: Modern Moravec’s Paradox
Computer-use agents reveal a modern form of Moravec’s Paradox: today’s AI excels at symbolic tasks like math and coding yet struggles with the everyday cognitive work humans perform effortlessly on computers. We will discuss the inherent challenges of computer use such as idiosyncratic environments and contextual understanding. We also argue that computer use is not only one of the hardest frontiers for AI but may also be the most important, because it can potentially offer the next Internet-scale learning opportunity and the most immediate path toward general, practical, goal-directed machine intelligence.
Northeastern University
March 16, 2026
Beyond the Surface: How Post-Training Artifacts Shape LLM Diversity and Safety
Post-training alignment makes LLMs helpful, but also introduces unintended artifacts. This talk explores two such artifacts, their impact on LLM diversity and safety, and presents corresponding solutions. (1) We begin with a data-driven artifact from RLHF, showing how a "typicality bias" in human preferences leads to mode collapse. I will introduce Verbalized Sampling, a principled prompting method that restores diversity across creative writing, social simulation, and synthetic data generation tasks. (2) Next, we shift to a mechanistic artifact from SFT, uncovering how LLMs encode "harmfulness" and "refusal" separately. This insight demystifies how jailbreaks work and enables the Latent Guard, an intrinsic safeguard built on the model's internal beliefs. Together, these findings call for an artifact-aware approach that looks beyond surface-level behaviors when building and evaluating LLMs.
University of Massachusetts Lowell
March 02, 2026
AI at Two Levels: From Efficient Pre-Training to Co-Creative Science
This talk addresses AI research at two levels: efficient methods for training large language models, and the emerging practice of using those models as collaborative partners in scientific discovery. Over the past few years, scaling large language models (LLMs) has become the go-to AI solution for solving progressively more complicated tasks. However, as the models scaled towards hundreds of billions of parameters, not just training them from scratch, but even fine-tuning to adapt to specific tasks has become computationally expensive and unmanageable for most practitioners. This has led to a crisis, in which only a few well-funded industry labs have the resources to develop high-quality LLMs. In response to these developments, numerous parameter-efficient techniques revolutionized the accessibility of fine-tuning LLMs. However, until recently, such methods were not available for pre-training. In the first part of the talk, I will present our work on ReLoRA (ICLR 2024), the first parameter-efficient method for training models from scratch, which utilizes low-rank updates to train high-rank networks, with demonstrated results for models with up to 1.3 billion parameters and throughput improvements of up to 40%. Since its publication, this line of work has grown into an active subfield, with dozens of new low-rank pre-training methods building on these ideas. In the second part, I will turn to the question of how the practice of science itself is changing as human researchers begin to collaborate with AI models. Today's models are trained on virtually all of codified human knowledge. But how can we teach them to do science with us, rather than merely for us? I will describe the Human–AI Co-Creativity Consortium (HAI3C), a cross-disciplinary effort across multiple universities with the goal of building the infrastructure, datasets, and evaluation frameworks needed to support genuine human-AI scientific collaboration. I will discuss our first pilot in mathematics, in which domain experts and multiple AI agents work together on open problems, and how this process generates annotated reasoning traces that can inform the next generation of models.
Ben-Gurion University of the Negev
February 23, 2026
Building Character: Tokenization in Theory and in Practice
The end-to-end nature of recent NLP models, most prominently large language models (LLMs), makes it hard for us to figure out how their individual processing steps are affected by properties of text and language, for example how they represent words in relation to their low-level composition or in alignment with how humans perceive them. We can find instances where models struggle with lexical changes in register, domain, or innovation, but the underlying mechanisms still mostly elude us. In this talk, I will focus on the lexical and sub-lexical levels of LLM representations, challenging models with recognition of words and the processes they are formed through, looking at subword tokenization algorithms from the point of view of their downstream performance requirements and the challenges posed by different languages. I will talk about SaGe/MINT, a subword tokenizer that incorporates context into the vocabulary creation objective; BoundlessBPE, which allows breaking past the limits of regex pretokenization; Splinter, a pre-tokenization algorithm that relinearizes Hebrew text to conform the concatenative tokenization pipeline with Semitic templatic morphology; and CharBench, an evaluation dataset for surface-form tasks such as character counting.
Carnegie Mellon University (Incoming Assistant Professor), Allen Institute for AI
February 16, 2026
Deep Research Agents: From Evaluation to Training and Open Challenges
Deep research agents that plan, search, and synthesize evidence into long-form, well-attributed reports are gaining rapid traction, yet most research still focuses on short-form, easily verifiable tasks. In this talk, I will present our recent work on evaluation, model development, and open challenges for deep research agents. First, I will briefly discuss the evaluation framework developed for OpenScholar (Nature, 2026), designed to assess long-form, citation-grounded scientific synthesis. I will describe our new multi-faced approach which combines rubric-based automated metrics, fine-grained citation accuracy verification, and expert preference judgments, and share key findings. Next, I will present DR Tulu, the first open model trained end-to-end for long-form deep research via Reinforcement Learning with Evolving Rubrics (RLER), where evaluation rubrics co-evolve with the policy to provide increasingly discriminative, on-policy feedback. DR Tulu-8B matches or outperforms proprietary systems like OpenAI Deep Research or Gemini 3 Pro across science, healthcare, and general-domain benchmarks while remaining significantly smaller and cheaper. Finally, I will draw on insights from our NeurIPS 2025 MMU-RAGent competition, which evaluates deep research systems through both static benchmarks and live user-facing arena comparisons. Key takeaways include limitations of current inflexible deep research agent designs as well as evaluation protocols. I will conclude with a roadmap toward more reliable deep research evaluation and training.
New York University
February 09, 2026
Linear Truth Encoding in LLMs: Emergence and Generalization
Large language models often exhibit surprisingly simple linear structure: directions and subspaces in their hidden states align with human-interpretable concepts. In this talk, I'll introduce the linear representations hypothesis and zoom in on a particularly striking case: linear "truth" encoding. Using a transparent one-layer transformer toy model with a deliberately simple, well-defined notion of truth, we trace the end-to-end mechanism by which a subspace emerges that linearly separates true from false statements. This analysis is based on a simple generative story in which true statements tend to co-occur with other true statements, and likewise for false ones. It turns out that this structure alone suffices to produce a fully characterizable truth subspace. In modern LLMs trained on natural data, however, the picture is messier. Recent results suggest that any apparent "truth direction" can be brittle: it may shift under distribution change, conflate truth with surface cues, or fragment across domains. I'll use these challenges to motivate ongoing work that moves from controlled setups to realistic regimes, probing real models under distribution shift and disentangling different notions of truth. The emerging picture is more complex than previously assumed: what looks like a single "truth subspace" is often a superposition of directions, some spurious and domain-specific, others more abstract and transferable.
Massachusetts Institute of Technology (MIT)
December 08, 2025
Just Asking Questions
In the age of deep networks, "learning" almost invariably means "learning from examples". We train language models with human-generated text and labeled preference pairs, image classifiers with large datasets of images, and robot policies with rollouts or demonstrations. When human learners acquire new concepts and skills, we often do so with richer supervision, especially in the form of language---we learn new concepts from examples accompanied by descriptions or definitions, and new skills from demonstrations accompanied by instructions. Crucially, language-based supervision involves not only instructions but *questions*---students ask questions to elicit the most useful pieces of supervision, and teachers ask questions to probe student knowledge and encourage them to acquire new skills or aspects of understanding on their own. This talk will focus on a few recent projects focused on building computational models that can ask good questions for both learning and teaching, with applications spanning LM alignment, policy learning, and education. This is joint work with Belinda Li, Alex Tamkin, Noah Goodman, Andi Peng, Ilia Sucholutsky, Nishanth Kumar, Julie A Shah, Andreea Bobu, Alexis Ross, Gabe Grand, Valerio Pepe and Josh Tenenbaum.
Meta Fundamental AI Research (FAIR)
November 21, 2025
Self-Improvement of LLMs
Classically, learning algorithms were designed to improve their performance by updating their parameters (weights), while keeping other components, such as the training data, loss function, and algorithm, fixed. We argue that fully intelligent systems will be able to self-improve across all aspects of their makeup. We describe recent methods that enable large language models (LLMs) to self-improve in various ways, increasing their performance on tasks relevant to human users. In particular, we describe methods whereby models are able to create their own training data (self-challenging), train on this data using themselves as their own reward model (self-rewarding), and train themselves to better provide their own rewards (meta-rewarding). We then discuss the future of self-improvement for AI and key challenges that remain unresolved.
University of Southern California
November 10, 2025
Understanding Model Nature of LLMs
The widespread adoption of large language models (LLMs) places a responsibility on the AI research community to understand them and develop abstractions that encapsulate their behavioral tendencies. In this talk, I will describe my group’s attempts to make sense of LLMs at three levels: their memorization of pre-training data, their internal mechanisms, and their advice-giving behavior. First, I will introduce the Hubble project, in which we have pre-trained LLMs (up to 8B parameters) on controlled pre-training corpora to understand when and how they memorize sensitive data related to copyright risks, privacy leakage, and test set contamination; we envision these models as a valuable open-source resource for scientific inquiry into LLM memorization. Next, I will describe my group’s work on understanding how language models work internally and how we can make these insights actionable. Finally, I will highlight a recent collaboration with USC oncologists in which we uncover LLM sycophancy issues that arise when patients ask these models for medical advice.
Yale University
November 03, 2025
Evaluating and Understanding LLMs: From Scientific Reasoning to Alignment as Judges
We present our recent work on evaluating and understanding large language models in scientific contexts and understanding them in context of evaluation-generation capabilities. First, we'll introduce SciArena, an open evaluation platform for literature-grounded scientific tasks that uses expert preferences to rank models on long-form, literature-grounded responses. The platform currently supports a broad set of open and proprietary models and has already accumulated a large pool of high-quality preferences. Using these data, we release SciArena-Eval, a meta-evaluation benchmark for training and stress-testing automated judges on science tasks. We will then turn to scientific problem solving. We discuss a holistic suite of scientific reasoning tasks, and a new framework for studying the role of knowledge in scientific problem solving and its interaction with reasoning. Our analysis shows that retrieving task-relevant knowledge from model parameters is the primary bottleneck for science reasoning; in-context external knowledge systematically helps even strong reasoning models; and improved verbalized reasoning increases a model’s ability to surface the right knowledge. Finally, if there is time, we will present a work on generation–evaluation consistency and show that models that judge well also tend to generate outputs that align with human preferences. This enables alignment benchmarking that evaluates models in their role as judges without scoring their generations directly.
Toyota Technological Institute at Chicago (TTIC)
October 27, 2025
Language Models as User Models
If we want to build collaborative language models, we'll need to find the right training objective. One promising direction involves simulating human users at scale and using these simulations as a training signal to develop models that better understand and interact with people. In this talk, I’ll discuss key challenges in simulating human behavior, ranging from hallucinations and coherence to knowledge consistency and memory. Then, I’ll discuss some ongoing work and outline future directions for building more human-like user simulators and training more useful, collaborative models.
University of Southern California
October 20, 2025
Building Personalized AI Assistants: From Task Execution to Human Alignment
Large language models have transformed how we interact with technology, yet most current systems remain reactive, executing tasks without understanding user context or values. This talk explores the path toward personalized, trustworthy AI assistants that can reason, adapt, and align with human preferences. I will introduce Computer-Using Agents (CUAs) that combine GUI operations and code generation to efficiently complete real-world tasks, and present CoAct-1, a multi-agent system that coordinates planning and execution. I will then discuss SEA, an algorithm for uncovering LLM knowledge deficiencies, and WildFeedback, a framework that learns user preferences from natural interactions. Finally, I will highlight ongoing work on human-centered and culturally aware AI, from understanding intention and reasoning consistency to enabling proactive, value-aligned collaboration. Together, these advances move us closer to AI systems that truly understand and assist people.
Northwestern University
October 13, 2025
From Large Language Models to Large Agent Models: The Reasoning Interface for World Interaction
The leap from Large Language Models to Large Agent Models is to unfold reasoning as multi-turn interactions with the world. We take the first step by formalizing agent training as a Markov Decision Process (MDP), introducing an agent reasoning interface (RAGEN) to avoid “reasoning collapse”, where LLM agents loop into repetitive reasoning and fail to explore. Extending to Partially Observable MDPs (POMDPs), we propose VAGEN, training VLM agents to internalize world models for state estimation (“what is seen”) and transition modeling (“what comes next”). We further reveal shallow exploitation over true exploration, and use self-play to inject world model knowledge to diversify exploration. Finally, we introduce cognitive maps as reasoning interfaces for VLMs to integrate partial observations into coherent world beliefs in the 3D space. In the end, we will lay out how these advances chart a path to agents that simulate, explore, and actively construct internal world models, as a decisive step from LLMs to LAMs.
Northeastern University
September 22, 2025
Some potential uses for and (current) limitations of mechanistic interpretability methods
LLMs are increasingly being deployed in high-stakes domains like healthcare and law. But the black box nature of such models brings real risks. Emerging "mechanistic" interpretability methods seek to characterize the inner-workings of generative models, providing insights into how they come to make particular predictions. This may in turn offer transparency and provide levers to improve model outputs in targeted ways. But while the science of mechanistic interpretability has progressed, the degree to which such methods might offer actionable insights in realistic and domain-specific tasks remains unclear. In this talk I will discuss some recent and ongoing work on such techniques, highlighting both potential applications (e.g., for transparency in healthcare settings, and for efficient model distillation) and some current limitations.
University of Maryland
September 15, 2025
Robust, Fair, and Culturally-Adaptive Social Cognition in LLMs
Successful communication in natural language is dependent upon interlocutors’ ability to build and maintain common ground, or a mutual understanding of the world. Thus, for LLMs to understand and serve the needs of their human users, they require not only basic knowledge of the world (“common sense”), but also the ability to engage in social cognition: reasoning under uncertainty about peoples’ beliefs, intents, and other mental states. In the first part of the talk, I will discuss my lab’s work on measuring robustness of LLM inference under uncertainty and inherent challenges of establishing ground truth in tasks of plausibility assessment. In the second part of the talk, I will present our findings concerning the fairness of social inference in LLMs, in particular the finding that LLMs demonstrate human-like inter-group empathy gaps. Finally, I will discuss my lab’s work developing evaluations of cross-cultural knowledge based on social theories of cultural consensus.
University of Maryland
September 08, 2025
A Human-Centered Approach to Trustworthy Machine Translation
Machine Translation (MT) tools are more accessible than ever, but a key challenge remains: how can we help users rely on them appropriately, especially when they don't understand one of the languages involved? This talk will describe how my group has addressed this problem in recent years. I'll begin by sharing the results of user studies designed to understand how people build trust in and reliance on MT. These findings led us to reframe MT evaluation to provide actionable feedback that helps people assess the impact of translation errors. To achieve this, we introduce techniques based on large language models to ask and answer questions about a translated text, and to detect errors in speech translations with minimal supervision. Throughout the talk, I will highlight open questions and research directions for designing language technology that genuinely supports people in communicating across language barriers.