#01Nov 4, 2026
math.ST
Regularization vs. Convolution in Kernel-Based Hypothesis Testing
Omar Hagrass, Bharath K. Sriperumbudur
The maximum mean discrepancy (MMD) test is a popular nonparametric test on general domains. Two lines of work have studied its optimality through structurally different approaches. The first, a convolution approach on $\mathbb{R}^d$, uses translation-invariant kernels with a shrinking bandwidth and is minimax optimal over Sobolev balls under $L^2$-separation; the second, a regularization approach on general measurable spaces, uses spectral regularization of the kernel covariance operator and is minimax optimal over Hellinger- and $χ^2$-separated alternatives. In this paper, we study the regularization approach over Sobolev balls under $L^2$-separation, which reveals the connection between the two approaches. We show that, with a translation-invariant kernel on $\mathbb{R}^d$, both are ways of meeting a single condition on the kernel's spectral density: the spectral regularized discrepancy captures the $L^2$-separation $Δ$ of a Sobolev alternative of smoothness $s$ as soon as the regularization parameter $λ$ is below the spectral density on frequencies up to $Δ^{-1/(2s)}$. A kernel with a shrinking bandwidth $h$ meets this condition with $λ\asymp h^d$, and under this coupling the constant regularizer, which gives the MMD test itself, attains the Sobolev minimax rate. This explains, from a spectral perspective, the need for a shrinking bandwidth in the convolution approach. With a non-constant spectral regularizer, our analysis requires the bandwidth to shrink only when the spectral density decays faster than any polynomial, as for the Gaussian kernel. Thus, for Matérn and Laplace kernels, a fixed bandwidth with $λ\to 0$ attains the Sobolev minimax rate. Finally, an adaptive test over a grid of bandwidths and regularization parameters is minimax optimal, up to a $\log\log$ factor, over both classes without prior knowledge of the smoothness.
#02Nov 4, 2026
cs.LG
Context--Answer Metamodels: Forecasting LLM Activations Before the First Answer Token
Tommy He, Thomas Jiralerspong
Under stochastic decoding, a single context can elicit many different answers from a language model. Characterizing this variation typically requires many expensive autoregressive rollouts, each revealing only one realization. We ask whether the activations produced while reading the context can directly forecast the model's rollout activations. We train context--answer metamodels that map a context activation sequence to point or distribution forecasts of mean-pooled answer activations. We find that (i) the four metamodel families we study forecast mean-pooled answer activations with high fidelity, and the distribution metamodels also capture rollout variability; (ii) performance improves as the training set grows to 500,000 prompts, and the metamodels generalize zero-shot to out-of-domain datasets; (iii) their forecasts retain behavior-predictive signal; and (iv) context activations from Qwen2.5 predict mean-pooled answer activations from Qwen3.5 and Gemma 4, suggesting that activation forecasts can transfer across models. Together, these results suggest that context activation traces can support scalable forecasts of behaviors in LLM answers before the first answer token is generated.
#03Nov 4, 2026
cs.MA
A Framework for Context Enrichment and Selective Robustness in LLM Social Agents
Mattia Greiche, Reihaneh Rabbany, Maximilian Puelma Touzel
Researchers use large language models as agents in social simulations, where agents are prompted with observations and background information intended to shape their actions. Yet LLM agents have been shown to substantially change their behavior under meaning-preserving prompt variations, creating a validation problem for social simulation. One plausible hypothesis is that this brittleness reflects a lack of grounding and can be reduced by enriching the context provided to the agents. Whether richer context improves robustness remains unclear, in part because context richness and robustness are rarely defined in ways that support controlled comparisons. We address the first need with controlled context enrichment, a set of prompt substitution rules that systematically enriches distinct types of context: neutral padding, generic social background, descriptive decision grounding, and normative decision grounding. We address the second by introducing selective robustness as a validation criterion: insensitivity to meaning-preserving prompt variation combined with a directional response to decision-relevant changes in the social world. We demonstrate the framework in a Prisoner's Dilemma-like game, showing how it can be used to identify which forms of context enrichment produce more desirable and robust agent behavior. We find that these forms of enrichment produce distinct behavioral patterns that vary between the two models tested.
#04Nov 4, 2026
cs.AR
Enabling AWS Cloud FPGA Development Using High-Level Vitis HLS
David Rostcheck
We present a toolchain for development of ML models on AWS EC2 F2 cloud FPGA infrastructure from high-level language, implementing a complete Vitis HLS to HDK bridge to AFI workflow that has been absent from the academic literature despite eight years of AWS FPGA availability. We introduce a four-level methodology (FP32 PyTorch, bit-accurate quantized Python, HLS C-simulation, physical FPGA) to aid in localizing errors, including FPGA-specific circuit errors. Together these tools allow coding agents to effectively implement FPGA designs from user directions, testing and debugging as needed. We document the deployment workflow, timing closure strategies, and practical HLS engineering decisions that proved crucial. We validate using TinyBERT fine-tuned for address Named Entity Recognition, implementing the embedding layer and the first of four encoder layers on the FPGA hardware. Our implementation streams weights from host DRAM, yielding poor performance compared to CPU/GPU implementations; we discuss forward paths.
#05Nov 4, 2026
cs.AI
ConceptARC at Scale: Studying Conceptual Understanding in Reasoning Models
Philipp Mondorf, Adrien Deliège, Solim LeGris and 5 more
Large reasoning models (LRMs) perform well on challenging benchmarks, yet it remains unclear whether they form generalizable abstractions across conceptually related reasoning tasks. The Abstraction and Reasoning Corpus (ARC) provides a controlled testbed for studying abstraction, requiring solvers to infer transformation rules over two-dimensional grids from a small set of examples. While ARC evaluates rule abstraction within individual tasks, different tasks may share higher-level structure despite requiring different transformations$\unicode{x2014}$for example, several tasks may involve reasoning about objects contained within other objects. Benchmarks such as ConceptARC capture this structure by grouping ARC-style tasks into recurring spatial and semantic concepts, but they often contain only a few manually constructed tasks per concept. We introduce ConceptARC at Scale, a generator for creating thousands of ARC-style tasks targeting a specific concept. Using this generator, we construct ConceptARC-100, a dataset of 1,600 new, manually reviewed tasks, with 100 tasks for each of the 16 concepts defined by ConceptARC$\unicode{x2014}$ten times the size of the original benchmark. We evaluate a range of recent LRMs and models trained from scratch on ConceptARC-100 to characterize which concepts they solve reliably and where they struggle. We further analyze the models' internal representations and find clear semantic structure: task representations cluster according to the operations applied to objects, such as recoloring, removal, or translation, even when realized through substantially different rules and scenarios. Together, our results suggest that models form stable abstractions of transformation operations that generalize across diverse tasks.