The day in conversation for 2026-10-21
discussion
Transcript
The discussion: Tom: That 43% video-level accuracy for EVID-Bench is pretty concerning. It suggests that even with a robust search mechanism, the systems are fundamentally failing to connect the dots when the ground truth isn't perfectly present in the immediate vicinity of the manipulated footage. Jane: I agree. And it brings back that same issue we saw last week with editorial splicing detection—where models mislabel genuine editing as manipulation because they can't fully contextualize where the *real* evidence should be. It seems like search-grounded verification introduces a whole new layer of ambiguity because the necessary verification anchors are outside the video frame entirely. Lu: Exactly. We’re moving from detecting internal inconsistencies—like audio glitches or frame skips—to needing external, real-time knowledge retrieval to validate claims, and the system breaks down precisely where that retrieval is imperfect or irrelevant. It's not a failure of detection so much as a failure of scope. Meng: I think the problem is compounded by how these frontier multimodal models are trained. They seem hyper-focused on the immediate visual evidence they *can* process, ignoring those peripheral search results because they aren't weighted correctly against the primary video content. They’re getting distracted by irrelevant anchors, as the summary mentioned. Lalam: So if we look at this through the lens of our previous work on single-source editing versus multi-source editing, it seems like search grounding is a multi-source problem taken to an extreme. We can handle two sources within a known context, but here we require an infinite pool of external evidence that the model
More episodes
- 2506.04019-LLMs are Brittle to Simple Code Transformations: Introducing CETBench - A Benchmark for Code-Equivalence Checking
- 2506.23053-Double-Diffusion: Balancing Speed, Accuracy, and Uncertainty in Probabilistic Forecasting for Urban Sensor Networks
- 2506.23921-The Trilemma of Truth in Large Language Models
- 2507.11228-Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
- 2507.21299-Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamics
- 2509.00362-Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
- 2509.05865-The Measure of Deception: An Analysis of Data Forging in Machine Unlearning
- 2509.14203-Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gain
- 2509.24608-Evaluating utility of clinical prediction models: A comparison of decision curve analysis and cost curves
- 2510.03811-Curriculum-Augmented GFlowNets For mRNA Sequence Generation