Real-time vision AI is trying to eat everything, overshoot is a pick and shovel play
For the past two years, all AI demos have been the same. Upload your image, wait 3 seconds and get your results. This is enough to generate a meme. It’s useless for anything that actually moves.
Physical security cameras don’t pause for inference. Robots do not wait for API calls to return. The goalkeeper cannot sit still while the VLM knows where the ball is going. Real-time vision AI requires a fundamentally different infrastructure stack, and few have built it.
There is overshoot. Less than 200ms from stream input to structured JSON output. Any video source. Any VLM. 3 lines of code. That’s the pitch, and the team has the receipts to back it up.
What overshoot actually does
Overshoot is an API infrastructure layer for real-time vision applications. Connect your video streams, phone cameras, webcams, YouTube Live, RTSP security feeds, and screen sharing, and Overshoot handles everything from frame extraction to VLM inference to returning structured results.
Customers are developers building vision apps. They don’t want to manage FFmpeg pipelines, figure out WebRTC handshakes, or benchmark which VLM gives the fastest response for a particular use case. they want to write overshoot.connect(stream_url, on_result=callback) And move on with their lives.
The business model is usage-based API pricing. You pay per inference run, per frame analyzed, or per minute of stream time. It’s a classic pick-and-shovel approach that Overshoot can scale with its customers without having to build the end application itself.
300 developers are already using it. Use cases range from physical security (real-time anomaly detection), gaming (AI that monitors on-screen content), robotics (recognition pipelines), sports analytics (tracking player movement at 60fps), and consumer apps (live translation, accessibility tools). Importantly, they all require fast visual inference at streaming frame rates.
How pipelines actually work
Video ingest: All protocols, one endpoint
The first problem is getting the video cleanly into the system. Overshoot accepts WebRTC (browser and mobile), RTSP (IP cameras, security systems), HLS (live streaming platforms, CDN distributed content), and direct file uploads for near-live processing.
Each protocol has different delay characteristics and buffering behavior. Although WebRTC is designed for sub-100ms delivery, it is notoriously difficult to implement correctly, with ICE negotiation, STUN/TURN server management, and codec negotiation all adding complexity. RTSP is simpler, but has longer delays by default. HLS operates in segment windows of 2 to 10 seconds. Handling all four properly without inadvertently inheriting each protocol’s worst delay characteristics is a nontrivial task.
Codec Layer: Where Real Latency Matters Most
This is the reason for Overshoot’s performance claims. Most Vision API platforms treat video as a container of JPEG frames, decode the video, extract the frames, JPEG compress them, and send them to the VLM. Be frank. It’s also catastrophically inefficient.
H.264 and H.265 video is already spatially compressed. Intelligently, the codec mostly only saves what has changed from the previous frame (P and B frames). Decoding to RAW pixels and re-encoding to JPEG discards this information and consumes CPU cycles that didn’t need to be spent.
Overshoot’s custom codec-level frame extraction works at the compressed bitstream level. The team, which includes Meta AI’s GPU kernel engineers and Intel Computer Vision AI framework engineers, can extract frame data without full decompression whenever possible, and can make smarter decisions about which frames are worth decoding based solely on the compressed domain signal.
This is not something you can look up on Google. This is the kind of knowledge that exists in compiler engineering teams at chip companies and distributed systems teams at video platforms. That’s why the team is just as important as the product.
Adaptive frame sampling: Not every frame requires a Ph.D.
A 30fps video stream is 30 frames per second. Running VLM inference on every frame is astronomically expensive and is completely unnecessary for most applications. A security camera monitoring an empty parking lot doesn’t need GPT-4o Vision 30 times per second.
Overshoot’s adaptive sampling layer determines which frames actually require inference. The core signals are scene change detection (Did something visually important happen?), magnitude of movement (Where is it moving and how much?), and application-specific heuristics (a sports analytics app might consider every frame in play, but rarely the frames in between).
Scene change detection at the codec level is actually low-cost, with the codec already incorporating motion vectors and residual energy signals. You can calculate a meaningful “How much has changed?” Get the score without fully decoding the frame. This is exactly the compression domain trick that turns expensive inference pipelines into efficient ones.
As a result, the sampling engine automatically handles decisions and provides a configurable target inference rate. 2 frames per second for static scenes and 10 frames per second for high activity scenes.
VLM Routing: Choosing the Right Model for the Job
There is no single VLM that is optimal for all visual tasks. GPT-4o Vision provides high accuracy but has slower latency and higher cost. Gemini Flash is fast and cheap, but there are various feature trade-offs. Claude has particular strengths in reasoning over structured output and visual content. Open source models running on dedicated GPU clusters can be even faster for certain fine-tuned use cases.
Overshoot routes inference requests between models based on configurable criteria such as latency budget (is this a real-time alert or background analysis?), cost tolerance, accuracy requirements, and historical performance with similar frame content. This routing layer also handles retries, fallbacks, and load balancing between API providers.
The complexity of operations here is considerable. Manage API keys, rate limits, error handling, and latency SLAs across multiple external providers simultaneously while ensuring end-to-end latency promises to your customers.
Edge Deployment: Geographic Proximity Matters
Physics is the ultimate latency constraint. A round trip from the camera in Frankfurt to the inference server on the East Coast of the United States adds 80 to 120 milliseconds to complete the calculation. If the total time is less than 200 milliseconds, more than half of the budget will be spent on the speed of light.
Overshoot runs geographically distributed edge inference servers that route incoming streams to the closest available worker. It goes beyond just running several VMs in different regions and builds an intelligent routing layer that takes into account the current server load, the availability of VLM at each edge, and the latency and cost tradeoffs of serving from a more distant but less loaded edge node.
Combined with codec optimization, adaptive sampling, and optimized VLM routing, we achieve consistent sub-200ms inference, which runs on centralized infrastructure and is 10x faster than alternatives that treat video like a series of slow images.
difficulty score
| discipline | Score | Precautions |
|---|---|---|
| ML/AI | 8/10 | While VLM integration itself is an API call, building adaptive sampling, scene change heuristics, and high-quality routing logic across the model requires true ML intuition. It’s really hard to fine-tune the trade-off between latency and accuracy. |
| data | 5/10 | The data challenges are real but standard, such as high-throughput streaming ingest, time-series inference logging, and usage measurement. Nothing special, but I/O needs to be handled with appropriate volumes. |
| backend | 9/10 | This is mainly a backend issue. Simultaneously perform codec-level video processing, streaming protocol processing, low-latency distribution systems, multi-region routing, and stateful stream management across WebRTC/RTSP/HLS. This is the difficult part. |
| front end | 4/10 | The product is an API. There’s a dashboard, documentation, and maybe a stream preview UI, but none of it is particularly special. The problem is entirely with the plumbing. |
| DevOps | 8/10 | Deploying a multi-region edge with a sub-200ms SLA is a significant infrastructure effort. GPU cluster management, streaming server orchestration, autoscaling for bursty video workloads, and zero-downtime deployment of always-on streams. Kubernetes is a starting point, not an end point. |
Hori: What’s really difficult here?
API surfaces are not moats. Anyone can wrap the VLM API into a streaming endpoint this weekend. it doesn’t matter.
This moat is a combination of codec-level optimizations, streaming protocol expertise, GPU kernel tuning, and an inference service architecture that produces stable sub-200ms latencies at scale. Each of those areas is independently unusual. It’s really rare to have all four of Uber’s pricing systems (distributed, latency-sensitive), the Meta GPU kernel, the Intel CV framework, and the founding engineers from the Intel acquisition on a founding team.
Data flywheels are also starting to emerge. As more developers use the platform, Overshoot accumulates signals about which VLMs perform best for which frame types, which sampling rates work for which application categories, and where geographic latency bottlenecks occur. Routing intelligence improves over time.
Difficult technical work also strengthens you. Building a custom codec-level extractor that is significantly faster than a naive approach creates a performance gap that competitors cannot close without the same thoroughness of work. It’s not a one-way door, but it’s a heavy door.
Easier is to build a basic demo by connecting a webcam to GPT-4o Vision and calling it a real-time vision platform. It will take a weekend. There are many of these. It doesn’t run at 30fps, has no latency below 200ms, and drops under real load. The demo is easy. Infrastructure is not.
Reproducibility score
52out of 100 points
Here’s an honest breakdown of why we reach 52.
Ready to replicate: API design, SDK interfaces, basic form of VLM routing logic, billing infrastructure, dashboards, and documentation. A talented team of two engineers can create a working prototype that passes demo in a week. Accept streams. JSON is returned. It will feel like overshoot.
Things that aren’t immediately reproducible include codec-level frame extraction to achieve latency numbers, adaptive sampling for cost efficiency at scale, inference batch processing to keep GPU utilization high without exceeding latency budgets, and multi-region edge deployments to handle geographic latency. These require engineers with previous experience doing this particular combination of tasks.
The expertise required is not just a “good engineer,” but a GPU kernel engineer who specifically understands video codecs, and a streaming infrastructure engineer who understands inference services. It is a narrow Ven diagram. You can also hire for 6-12 months. You can’t spend a weekend working on it yourself.
52 means you can compete if you really want to. It requires two to three qualified engineers and a year of intensive work. We’ll probably still keep track of Overshoot’s performance numbers as we iterate through the next optimization layer. It’s not impossible. It’s not a weekend project. A true engineering commitment to a team with a great start and precise expertise.
why now?
VLM will cross a threshold of capability in 2024-2025 and become truly useful not only for image description but also for real-time vision tasks such as spatial reasoning, object tracking, action recognition, and anomaly detection. The model exists. The demand for developers is there. The infrastructure was not.
Overshoot is a Cloudflare Workers moment for Vision AI, a layer that gives developers access to powerful underlying technology without having to all become systems engineers. The timing is good because the model is good enough to be worth building on and the systems engineering gap remains large enough to matter.
300 developers won’t adopt an API infrastructure tool because the demo is beautiful. They adopt it because it solves a real problem. Overshoot solves a real problem.
