Types of Video Analytics Four Generations, and Which One You Are Being Sold

The oldest and still the most widely deployed. The software compares consecutive frames and fires when enough pixels change. It has no concept of objects, so a cat, a plastic bag, and a burglar are the same event. Every camera and recorder made in the last twenty years includes it, which is precisely why so many security teams have long since muted their alert channel.

VMD is not useless. It is a cheap way to decide which footage is worth recording, and on an indoor camera in a locked room after hours it is close to sufficient. It fails the moment the scene contains weather, traffic, foliage, or anything else that legitimately moves.

The second generation adds geometry. The software tracks the moving region as a blob and applies rules: crossed this line, entered this zone, stayed longer than thirty seconds, moved in the wrong direction, was left behind. Tripwire, loitering, and object-left detection all live here.

This is a real improvement, because the rule filters out movement in places you do not care about. But the system still does not know what the blob is. A shopping cart rolling across a tripwire crosses it exactly as convincingly as a person, and a large enough shadow at sunset will do the same. Tuning becomes an exercise in shrinking the zone until the noise is tolerable, which is also how real events get missed.

The third generation replaces the blob with a classification. A neural network trained on large labeled datasets labels each object in the frame as a person, vehicle, bicycle, package, or weapon, and a tracker links those labels across frames so one car crossing a lot is a single tracked object rather than eighty separate detections.

This is the shift that makes alerting usable. Because the system knows what an object is, a rule can require a person in the loading dock zone between 10pm and 5am, and simply never fire for the raccoon or the rain. Weapon detection, license plate recognition, PPE compliance, people counting, and behavior analysis such as loitering, tailgating, and fighting all sit on this generation.

Accuracy here depends far more on the camera than the model. Pixels on target, compression, and lighting decide whether the network has anything to classify, which is why the accuracy of AI video analytics is a property of a deployment rather than of a product.

The newest generation embeds video frames and text into the same mathematical space, so a typed description can be matched against footage directly. An operator writes "white SUV in the north lot after 9pm" or "person in a green jacket near the side entrance" and gets matching clips across every camera and every day of retention.

This is a retrieval capability rather than an alerting one, and it changes investigation more than it changes monitoring. Work that used to mean an afternoon of scrubbing becomes a query and an evidence export. In practice, modern platforms run generations three and four together: classification drives the alerts, and language search drives the investigation.