AI
Terminal-Bench 2.1: when the benchmark is part of the problem
The audit found issues in roughly 30% of Terminal-Bench 2.0 tasks. The authors reviewed task instructions, grading code, participant reports and execution traces. They repaired 28 of 89 tasks in 2.1. In the release comparison, the largest gain was 12.1 percentage points for Claude Code with Opus 4.6, while some configurations declined slightly. Rank agreement remained high in the audit, with a Spearman correlation of 0.958.
What to check. If you are choosing an agent, record the dataset and harness versions, inspect the execution logs and use the same time budgets. The paper also shows that changing the time limit can narrow gaps between agent setups. A different score after the benchmark is repaired is a reason to rerun the comparison.
Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks
Compute
KOSMOS: overlap communication with computation
Training GPUs can sit idle while another group transfers data. KOSMOS uses persistent kernels and queues that let operations proceed when their input is ready. The authors report 1.10–1.28× end-to-end BF16 training speedups in Megatron-LM on AMD MI350X and MI355X, and 1.29–1.83× versus DeepEP on DeepSeek-V3 and Qwen3-235B proxy workloads.
What to check. Use the profiler to find idle periods at the communication boundary before testing a replacement. Re-run on your own topology and workload. The benchmark measures the reported setups and does not establish the same gain for a different cluster or a long production run.
KOSMOS: Self-scheduling Persistent Kernels for Communication-Compute Overlap
Robotics
VLA-Tug: detect a likely failure before the action completes
A compact monitor reads internal states of a vision-language-action model and classifies likely failure types before the action ends. The authors report up to 30% better early detection than their strongest baseline at comparable false-alarm levels. On unseen tasks the reported statistical bounds held in up to 86% of runs, versus at most 21% for the baselines. The researchers provide 3,180 labelled episodes.
What to check. Test on unfamiliar objects and measure the cost of unnecessary stops alongside prevented failures. Integrate the monitor with a predictable recovery procedure and a hardware stop where appropriate. The research monitor itself provides no safety certification.
Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
Energy
Data-centre batteries: schedule dispatch around prices and demand
Van Ham, Liu and Wang combine electricity-price and load forecasts with dynamic programming to choose when a battery charges or discharges. In the evaluated configurations, electricity procurement savings were 2.63 times the savings obtained by a rule-based dispatcher. This is a model comparison of savings, with no claim that the full electricity bill falls by the same factor.
What to check. Backtest against your own hourly tariffs and load. Include conversion losses, cycling limits, battery wear and grid constraints. If the advantage survives those costs, a pilot can test whether it carries over to operations.
Biotech
BILBY-MD: a good static structure can still move out of shape
BILBY-PRO expands scaffold testing to 220 cases linked to 174 functional PROSITE motifs. BILBY-MD tests what happens when previously successful designs are put through molecular dynamics. Of 55 design-task pairs that passed the static check, 31 met the dynamics-supported criterion and only eight passed all the checks.
What to check. Use static recovery as one stage in a screening pipeline. Check distinct properties such as design diversity and stability in motion before committing to expensive experiments. The results concern the chosen computational criteria and motion scales.
Static Recovery Is Not Dynamic Stability: Dynamics-Aware Benchmarking of Protein Motif Scaffolding
Security
SensorGuard-RV: protect the reading before it reaches software
A driver, bus or memory buffer can be a point of tampering or replay. SensorGuard-RV authenticates and encrypts samples after owner enrolment and uses a RISC-V instruction to deliver verified readings to a CPU register. The prototype used a NEORV32 core, Artix-7 FPGA and ADXL362 accelerometer. At 100 MHz, a secure transaction took 51.9 μs and throughput reached 19,015 transactions per second, 5.3% above the chosen software baseline.
What to check. Check the power budget before choosing a sensor-to-register design. The authors estimated system power rising from 118 to 368 mW. The FPGA prototype demonstrates the mechanism and gives a concrete trade-off to test on your hardware.
SensorGuard-RV: Sensor-to-Register Data Protection for Resource-Constrained RISC-V Devices