#01Nov 8, 2026
cs.CV
How to "train" a surgeon? Benchmarking generalist agents in surgical scene understanding
Gengyuan Zhang, Xiao Han, Shijie Zhou and 7 more
Evaluating general-purpose multimodal large language models (MLLMs) for surgical scene understanding remains limited due to scarce surgical data and fragmented evaluation protocols. We address these challenges through a two-stage evaluation framework for surgical scene understanding. In Stage I, we introduce SurgKnowBench, a diagnostic benchmark that harmonizes public surgical datasets across procedures and tasks and evaluates proprietary and open-source MLLMs to map the boundary of pretrained surgical competence. SurgKnowBench reveals that current models retain partial visual priors but remain weak on relational and procedural understanding and workflow assessment. Building on this failure frontier and motivated by the scaffolded nature of surgical training, we introduce SurgAgentGym, an agentic environment where general-purpose MLLM agents can use bounded knowledge search over SurgWiki and general-purpose visual grounding for surgical tasks. SurgAgentGym evaluates whether generalist agents can improve surgical understanding at inference time through tool-mediated evidence use, without additional surgical training. Our results show that agentic inference narrows the gap mainly for perception-heavy tasks, while fine-grained localization and expert-style assessment remain persistent bottlenecks. This opens a path toward future studies of agentic surgical understanding.
#02Nov 8, 2026
cs.CV
DynOLAT: OLAT Video Prediction for Controllable Dynamic Full-Body Relighting
Taoming Liu, Yinji ShenTu, Guangzhao He and 7 more
Accurate lighting response is essential for realistic dynamic human relighting, because shadows, highlights, and color changes should follow the target illumination in both direction and relative strength. Current learning-based relighting methods can generate visually plausible videos, but they often exhibit imprecise lighting responses: shadows and highlights may drift away from the target direction, and the output brightness may not faithfully follow the relative intensities in the target environment, making the result closer to appearance harmonization than controllable relighting. To address this issue, we propose DynOLAT, a generalizable OLAT video prediction framework for dynamic full-body humans. Instead of directly synthesizing a single relit appearance, our method predicts temporally consistent One-Light-At-a-Time (OLAT) basis videos from flat-lit inputs, and then linearly combines the predicted bases according to the target environment map. This OLAT-based formulation makes the directional response explicit in the predicted bases, while the relative lighting strength is determined by the linear weights derived from the environment map. We further introduce a dynamic capture pipeline and the first large-scale dynamic full-body OLAT dataset, comprising over 24 million frames from 224 identity-outfit combinations, where different outfits of the same person define separate combinations. Experiments show that DynOLAT responds more faithfully to variations in target lighting direction and relative intensity than the tested baselines, enabling controllable video relighting while preserving fine appearance details and temporal coherence. The dataset and processing tools will be publicly released to foster future research.
#03Nov 8, 2026
cs.CV
Auto-GlaKG: Automated Biomarker Reasoning for Explainable Glaucoma Detection
Cheng Huang, Zeyu Han, Yian Hu and 3 more
Interpretable glaucoma detection requires a transparent connection between fundus images, clinically meaningful biomarkers, and diagnostic predictions. Knowledge graph (KG) reasoning provides this structure, but performance measured using annotated biomarkers does not establish performance when only images are available. We present Auto-GlaKG, an automated framework that combines image-based biomarker prediction with fixed clinical-prior rules. A ResNet50 network predicts the cup-to-disc ratio (CDR) and six morphological signs using biomarker supervision, and a KG reasoner aggregates their signed rule contributions into a glaucoma score. On 689 fundus images evaluated using stratified, group-aware five-fold cross-validation, Auto-GlaKG achieves a pooled area under the receiver operating characteristic curve (AUC) of 0.951 (95\% confidence interval: 0.931--0.969), with sensitivity of 0.970 and specificity of 0.779. The matched image classifier also achieves an AUC of 0.951, while the rule stage adds 0.2~ms per image. Replacing predicted CDR with its annotation increases fold-mean AUC from 0.953 to 0.995. The framework provides an explicit rule-level account of image-based predictions and identifies CDR estimation as a priority for validation using independently measured biomarkers.
#04Nov 8, 2026
cs.LG
A Unified Framework for License Plate Identification and Character-Recognition
Saurabh Kumar, Dewant Katare, Prakriti Jain
This paper presents a unified framework for license plate detection and character recognition on low-power edge hardware. The proposed network uses a shared CSP-Darknet and PANet feature encoder with an anchor-free localization head and a RoIAlign-based sequence-recognition branch. Character decoding is performed directly on aligned feature maps, avoiding image-space plate cropping and a second recognition backbone. The model is trained on an AI-assisted dataset of approximately 37,000 manually verified images covering license plate layouts from all 50 United States. The dataset includes variations in illumination, viewpoint, weather, blur, occlusion, and plate design. The framework is converted to an INT8 TensorRT engine and evaluated on an NVIDIA Jetson Orin Nano with 8 GB memory. The INT8 model shows an mAP50 of 97.8% and a full-sequence accuracy of 90.7%. It records 147 FPS at 6.8 ms end-to-end latency, with 315 MB inference memory and 9.4 W average power consumption. Relative to the YOLOv8s+CRNN baseline, the proposed deployment provides a 7.1x throughput increase and a 77.8% reduction in allocated inference memory
#05Nov 8, 2026
cs.CV
Conditional Diffusion-Based Proxy Representation Recovery for BCVA Prediction with Missing SLO Modality
Chuyi He, Hua Zou, Fei Luo and 1 more
Preoperative prediction of postoperative best-corrected visual acuity (BCVA) can support prognosis assessment and individualized decision-making in cataract surgery. Multimodal learning can integrate preoperative optical coherence tomography (OCT) images, scanning laser ophthalmoscopy (SLO) images, and clinical text to characterize patient status. However, SLO images are often missing in real-world clinical data due to acquisition or data-management issues, resulting in incomplete multimodal inputs. To address this problem, this paper proposes a conditional diffusion-based method for SLO proxy representation recovery and fusion prediction. Instead of reconstructing raw SLO images, the proposed method formulates SLO missingness as proxy representation recovery in the latent feature space. Specifically, OCT image features and clinical text features are first fused to construct an OCT-text conditional representation, which guides the reverse denoising process of the diffusion model to estimate sample-specific SLO proxy representations. The recovered SLO proxy representation is then used as a substitute for the unavailable SLO modality and incorporated into the downstream multimodal fusion prediction network for postoperative BCVA prediction. Experiments were conducted on 2,141 preoperative multimodal cataract samples, with an SLO missing rate of 58.80%. The proposed method achieved an accuracy of 0.9714 under the criterion of an absolute prediction error within 0.1 LogMAR and a mean absolute error of 0.0423, outperforming representative postoperative visual acuity prediction and missing-modality learning methods. These results indicate that representation-level proxy recovery can alleviate the impact of SLO modality missingness on postoperative BCVA prediction.