Video AI That Knows When to Do Less Work.
Published research on a video AI that recognises human actions, and skips its most expensive analysis whenever its first answer is already confident.
Key numbers
83.12%
mean accuracy on the JHMDB benchmark
71.41%
average estimated reduction in computation
78.73%
of videos skipped the expensive motion step
84.17%
estimated reduction at best, on one test split
What was getting in the way
Recognising actions like walking or sitting from video works best when visual, body-joint and motion data are combined. But motion data is expensive to compute, and most systems calculate it for every video, even when it adds nothing.
What we built
A three-stream model that reads a video three ways: what’s visible (RGB), how the body’s joints move (skeleton) and pixel-level motion (optical flow). The key idea is a gate that decides when the costly third stream is worth running.
How it works
- Fast pass first. The visual and skeleton streams make a quick prediction.
- Check the uncertainty. The system measures how confident that prediction is.
- Confident? Done. The answer is returned and motion analysis is skipped.
- Unsure? Look closer. Only then does it compute optical flow and combine all three streams.
The savings are estimates of computation, not measured speed or energy. Possible uses include health monitoring, assisted living and workplace safety; these are future applications, not deployments from the study.
Research facts
- Type
- Research Project
- Authors
- Bevin Benet et al.
- Field
- Computer vision
- Published in
- Smart Health, Vol. 41 (2026), Article 100699
- Tools
- PyTorch, ConvNeXt, HT-ConvNet, RAFT
Where this expertise applies
Custom AI Systems & Fine-Tuned Models
Build AI-powered applications around your specific processes, information and requirements.
Your workflow could be next
Have a Problem Worth Solving?
Tell us what your team keeps doing manually. We’ll help identify what can be improved and how to validate the results before a wider rollout.