This document provides an introduction to IndigoVision 8000 Analytics. The document covers the all-important process of qualification, configuration and a basic understanding of each of the different filter types. The document also examines common sources of false alarms, and how to significantly reduce their occurrence.
3 ANALYTICS AND ANALYTICS FILTERS
This document provides an introduction to IndigoVision 8000 Analytics. The document covers the all-important process of qualification, configuration and provides a basic understanding of the detection process. The document will further examine common sources of false alarms, and how to significantly reduce their occurrence.
This document refers to the IndigoVision 2.12.0 8000 release firmware.
This document supersedes the “Understanding Motion Detection” white paper.
It is fundamental to understand that the success of any VMD system is determined during system qualification - well before the process of any demonstrations, on-site configuration and system field trials.
Analytical processing of real-time video is exceptionally complex, but is something that people can perform exceptionally well and with ease. Unfortunately, this leads to an over-expectation in what VMD systems can achieve.
Qualified, fit-for-purpose, correctly installed and purposefully configured VMD systems work exceptionally well, and provide a significant aid to security systems. Poorly qualified, incorrectly installed, and inappropriately configured VMD systems lead to high false alarm rates, low detection rates, frustration and disappointment.
It is vitally important to have an analytics qualification process, covering conception to installation. The process can be based on simple common sense and experience. Use the following six-step guide as a template for a simple self-qualification process:
This section of the document introduces the basics of the Analytics and Analytics Filters. In 2.12.0 firmware two filters are provided: “Filter 1” and “Filter 2”, with six types of filter: Activity, Motion, Museum, Congestion, Counter-Flow and Tripwire.
Analytics is the process within the 8000 transmitter of analysing the incoming video stream from the camera using various statistical techniques. In general, there are many classes of digital video analytics. IndigoVision have identified five distinct classes of algorithms, which are described in Table 2.
Class |
Name |
Description |
I |
Global |
Analyzes video without respect to individual objects. |
II |
Object |
Detection of localized motion at individual instances in time. |
III |
Tracking |
Detection of localized moving objects over time. |
IV |
Flow |
Tracking and estimation of true speed and direction. |
V |
Behavioral |
Analysis for abnormalities in object behavior. |
Global (class I) algorithms have no concept of objects e.g. cars or individual people. They provide statistical measures averaged across a region-of-interest (ROI). Being global monitors the same level of motion in each frame of video can be achieved by a lot of small, distributed motion but also by a large amount of concentrated motion.
Object (class II) algorithms improve detection by analyzing video on a localized basis within each frame of video. Co-located areas of motion or change in each frame are combined to form individual objects. These objects can be filtered based on features such as object size.
Tracking (class III) algorithms attempt to track objects over a number of frames of video. This can provide for significant improvement in both the determination of direction and object shape, by averaging over time.
Although Tracking algorithms have a rudimentary notion of object motion over time they are unable to determine the true speed and direction of an object i.e. they have no concept of depth e.g. how far away a person is from the camera. Information intelligently or manually gathered by Flow (class IV) algorithms can be used to gauge an estimate the true size, direction and speed of an object.
Behavioral (class V) algorithms intelligently monitor all objects over a large period of time, in order to determine what behaviour is typical, and which is atypical. Atypical behaviour will trigger an alarm.
Each new class introduces a major step up in terms of detection performance but also computational complexity i.e. cost. It is also important to understand that not all applications require a class V algorithm.
Filters in 8000 transmitters vary in their class type. Activity and Congestion are class I algorithms. Motion and museum filters are class II algorithms. Filters that perform such tasks as virtual tripwire and counter-flow operations are class III algorithms.
Filters provide the 8000 with its detection functionality and are each designed for specific applications. Six types of filter are currently available: Activity, Motion, Museum, Congestion, Counter-Flow and Virtual Tripwire.
The “Normal” filter in 2.10.0 firmware was renamed “Motion” in 2.11.0.
Table 3 describes the main differences between these six types of filters.
Filter Type |
Application |
Activity |
Very sensitive filter to detect the slightest movement in the video. Restricted for use with ACF only. |
Motion |
Filter to detect significant motion in the cameras field of view e.g. a door entry system to record personnel entering a building |
Museum |
Filter to detect background changes, e.g. vehicle abandonment, whilst ignoring foreground motion. |
Congestion |
Filter to detect the build up of large amounts of moving, possibly slow moving objects, within a specified region of interest. |
Counter-Flow |
Filter to detect motion in an illegal direction, such as a vehicle traveling the wrong way down a road. |
Virtual Tripwire |
Filter for detecting motion over a user-defined tripwire, for applications such as perimeter intrusion. |
An Activity Filter can be applied to detect any form of activity in the video. The filter is very sensitive and its use is restricted to ACF and cannot be used to generate analytics events. See “Understanding ACF” [1].
All filters subdivide the camera’s field of view into a regular grid of non-overlapping detection cells. For PAL this grid measures 22 by 18 cells in size and for NTSC sources this grid measures 22 by 15 cells. A PAL example grid is shown in Figure 1.
During configuration the user can select the cells in which to detect activity. This allows a user to select a Region Of Interest (ROI) within the camera’s field of view. This is demonstrated in Figure 2 where a simple ROI has been configured. In this example, the filter is looking for activity in cells around one of the doors.
Each cell in the ROI is individually processed for activity. If significant motion is detected the cell is marked as activated, as shown in Figure 3. The activity level output is the sum of the activity in the activated cells. The activity level output is used by ACF to determine whether to switch between active and passive modes.
A Motion Filter can be applied to detect simple moving objects, and can be used to generate analytics events.
The Motion Filter uses the activated cells and attempts to combine neighbouring activated cells into a single object. In Figure 4 ten neighbouring cells have been combined into a single object of size ten.
In other situations a number of objects, of varying sizes, may have been detected. The Motion Filter then processes all the objects in order to reject objects that are too small or too big, which as we will see later, is useful for false alarm rejection.
The final stage in the Motion Filter is to reject false spurious motion. Only when an object persists by remaining present and moving will an analytics event be generated. Figure 5 shows an object persisting over a period of 1 second.
The Museum Filter can be applied to detect static background changes in the video.
In an identical manner to the Motion Filter an ROI can be configured for use with the Museum Filter. An example ROI is shown in Figure 6 where it has been configured to detect vehicles blocking the entrance to a building and also priority parking spaces.
Once configured the Museum Filter analyzes each cell in the ROI for changes in the static background content. With the Museum Filter cars moving through the ROI will not trigger a cell to be activated. However, if a car drives into and stops in the ROI the cell is activated (marked as changed) as shown in Figure 7.
The activated cells are then processed in the same way as with the Motion Filter. Neighbouring activated cells are joined together to form objects, which can then be filtered by their size. If an object of a suitable size is detected and has existed over the persistence time an analytics event is generated. Once generated, as shown in Figure 8, the object is now treated as part of a new background.
Since the object is now treated as part of the static background scene, exactly the same process will occur when the car is removed, as shown in Figure 9. In this case a second analytics event will be generated.
This illustrates the dual role of a Museum Filter as a method of detecting the addition of new static objects, and conversely the removal of static objects.
A Congestion Filter can be used to detect the build up of congestion. This would be typically used in applications for traffic or train station platforms.
Congestion detection is a class I algorithm. The algorithm works by detecting masses of cells that have either fast or slow moving motion. Congestion is not designed for detecting completely static congestion – this could potentially be achieved using a museum filter i.e. the detection of a large change in the static background. This filter requires motion in order to come to a decision regarding the build up of congestion.
Setting up a region of interest for congestion is important for congestion. The ROI defines the area in which you are interested in detecting congestion, where the level of congestion is measured as a percentage of the ROI. For example at a train station you should set up the ROI for the platform and ignore the train tracks. This can avoid false alarms due to trains coming in and out of the station.
Figure 10 shows a region of interest configured for detecting traffic congestion. In this case congestion is being detected only on one side of the road. If the region of interest had been set to the entire scene, as by default, other motion and small changes, such as on the other side of the road, may have erroneously contributed to the level of congestion.
A Counter-Flow Filter is an advanced Motion Filter. A Counter-Flow filter applies a similar technique to determine objects on a frame-by-frame basis as a Motion Filter, but then uses a tracking technique to follow objects over time, measuring the distance and direction from the first point where the object is detected.
This tracking technology gives a far more robust way of detecting motion, and in particular motion in a specific direction, such as in applications to monitor vehicles moving in an illegal direction. The Counter-Flow Filter can also be used as an upgrade to the simple Motion Filter potentially providing a much improved FAR due to the ability to reject false alarms that do not appear to move over time, such as a foliage moving in the wind.
Note: In 2.12.0 firmware a maximum of 4 objects are tracked concurrently.
The Virtual Tripwire Filter employs the same Analytics technology as the Counter- Flow Filter but introduces the concept of a tripwire. This filter tracks objects over time but only produces a significant output when an object moves over a defined tripwire.
In this example a vertical tripwire is shown in yellow. Note how it is only until the center of the object passes over the tripwire that the filter is alarmed, and will remain alarmed even if the object tries to return back over the tripwire. As such care should be taken with configuring a tripwire with large object sizes or aspect ratios.
The Virtual Tripwire Filter is another potential way of reduce the number of false alarms over the much simpler Motion Filter.
Note: In 2.12.0 firmware a maximum of 4 objects are tracked concurrently.
Installing an analytics enabled 8000 transmitter can be performed as per the IndigoVision 8000 installation documentation. This section provides some analytics specific installation tips and some example configurations.
This section provides a reminder of a few of the points from the previous section and a few further helpful common-sense hints regarding installation that should provide for maximum analytics performance.
There are a number of different ways to configure an analytics system. Three very simple configurations are shown in the following diagrams.
In Figure 13 detection is set up in a passive configuration. Analytics events and video are recorded to an IndigoVision NVR. At some later point the recorded alarms and video are reviewed with Control Center. This is suitable for applications whereby no action needs to be taken as a result of an analytics event but video is required before, during and after an event.
In other scenarios a Control Center operator wishes to see what caused the event immediately. This configuration is illustrated in Figure 14. In this configuration it is possible to view the live information at the time of the event.
A third potential configuration is shown in Figure 15. In this case only what happens after the event is important, and in this example the system is configured to record video when an analytics event has occurred. This reduces the required storage. Events and recorded video can then be viewed at a later time.
These are three simple examples of potential configurations for an 8000 analytics enabled transmitter. For more information regarding configuring these types of applications please refer to Control Center documentation. For information regarding configuring the 8000 transmitters see the 8000 documentation and for specific configuration tips see the section in this document entitled “Configuring Analytics”.
Analytics is configured via the standard 8000-transmitter interface, as described in the 8000 documentation. This section provides some more in-depth information.
Two filters are provided for configuration: “Filter 1” and “Filter 2”, with 6 types of filter also provided. Use of some of the more advanced filter types requires a valid Analytics License Key. Table 4 shows the types of filter available.
License |
Filter 1 |
Filter 2 |
Unlicensed |
Activity Filter |
Motion Filter |
1 Filter
|
Activity Filter |
Motion Filter or
|
With 2.12.0 firmware a 2-FIL license provides no extra functionality.
To use the more advanced types of filters enter a valid 1-FIL license key via the Firmware Upgrade page, as shown in Figure 16, else these more advanced types of filter will remain greyed-out (disabled) on the Analytics Configuration page.
To configure Analytics navigate to the Analytics Configuration page, as shown in Figure 17. On the Analytics Configuration page you will see two filters “Filter 1” and “Filter 2” that are available for configuration. You will also see a link to a Region-Of- Interest editor page at the top, which is common to all filters.
In this version of firmware “Filter 1” is used exclusively with ACF. For details on how to configure “Filter 1” refer to “Understanding ACF” [1].
To configure “Filter 2” for detection:
1. Select the Type of filter you wish to use, and Submit.
2. Once submitted click on the Edit button to configure the Region-Of-Interest.
3. The ROI editor will appear as shown in Figure 18. Functionality on the ROI editor page will depend on the current submitted filter type.
4. Use the ROI editor to remove as many potential false alarms and as much background clutter as possible. An example is shown in Figure 19. Note the total size, in cells, of the ROI circled.
5. Determine the size of the ROI, as a percentage of the total area is not too small i.e. it passes the qualification process.
6. For the motion-based filters (Motion, Counter-Flow, Tripwire) use the arrows around the ROI to select all, 1 or 2 directions in which you are interested in detecting motion. An example of 2 directions is shown in Figure 19. Note: This does not include the Activity Filter, which does not use direction.
7. For the tripwire filter use a CTRL, left mouse-click and drag operation to define a tripwire, over which you are attempting to detect motion. An example is shown in Figure 20 – the tripwire is shown in yellow. Note: Only the parts of the tripwire within the ROI will used for detection.
|
8. Submit your changes.
9. Adjust the minimum and maximum object sizes to reject false alarms due to their size e.g. it is unlikely in most situations that the objects you are attempting to detect will be the same size as the ROI. The upper range is determined by the ROI size you noted in step 5. Note: Object size does not apply to the Congestion Filter.
|
10. Enter a persistence time over which the motion, change, congestion must persist in order to trigger an event. Use this feature to reject noise-related or transient motion and changes e.g. caused by a light being switch on or off.
11. Choose an appropriate dwell time. The dwell time controls the period of time after an analytics event before another analytics event can possibly occur, regardless of the filter output.
12. Tick the Events check box to enable the transmission of analytics events.
13. Submit changes.
14. In Control Center under Site Setup click on the site in which your transmitter resides. Click on the Alarm Sources tab, and enable the Video Analysis source for your transmitter, as shown in Figure 21.
15. Monitor for events from the camera whilst viewing video under Live Video
16. Return to Analytics Configuration page and adjust the sensitivity level for “Filter 2” if too many or too few events appear to be produced. The higher the value entered the greater the sensitivity, and vice-versa. Adjust this value to achieve the expected performance.
17. Record the stream for 24 hours, and review the results to get an estimate of FAR and DR. Alternatively use the performance evaluation process determined by your qualification process.
For 2.11.0 firmware or later (requires Control Center 2.8 or later):
In order to aid the process of configuration you can embed important configuration information in a video stream via the Video Configuration page. Figure 23 shows that “Stream 1” has this feature enabled.
This embedded analytics information includes a list of the objects detected in every frame of video, including each objects location, as well as information designed to help with filter configuration.
1. Select Embed analytics data from the Video Configuration page and Submit.
2. Right-click on video pane in Control Center and select Show Video Analysis
3. Two bars will appear in the left of the video pane representing the outputs from the two Analytics filters, “Filter 1” and “Filter 2”. The levels represent the current instantaneous output from either filter. See Figure 24.
4. The white horizontal bars represent a filter threshold. An analytics event will be generated if the level remains above this threshold for a time greater than the persistence time. Adjust the sensitivity value on the appropriate filter on the Analytics Configuration page to adjust this threshold. The greater the sensitivity the lower the threshold. Note: The Museum Filter is slightly different in this release with the level representing the time a change has been detected, as a percentage of the persistence time.
|
In order to view individual objects overlaid in a Control Center video pane
1. Select Embed analytics data from the Video Configuration page and Submit.
2. Right-click on video pane in Control Center and select Highlight Motion
3. Rectangular object bounding boxes will be displayed where appropriate, as shown in Figure 25.
Note: Green boxes on the screen do not necessarily mean that an analytics event will be generated. Also note that again the Museum Filter is slightly different in that it only displays green boxes in-between 90% and 100% of the persistence time, and will disappear as soon as they have generated an event.
|
With the exception of the I-frame interval, none of the parameters that are configurable on the “Video” section should cause serious degradation in the performance of analytics. However, it is recommended that the highest resolution, frame rate and bitrate on “Stream 1” be used to ensure best performance.
The Detection Rate (DR) of an Analytics system can be defined as the number of successfully detected objects of interest determined over a period time, expressed as a percentage of the number of objects of interest that actually occurred. For example, an analytics system set up to detect people entering a building detects 49 people over a 24-hour period. However, records show that actually 50 people arrived – this gives a DR estimate of (49/50)*100%=98% per day.
The False Alarm Rate (FAR) of Analytics system can be defined as the number of incorrectly detected objects over a period of time, expressed as a percentage of the number of objects of interest. Continuing from the previous example, if 51 objects were detected, that means 2 objects were not actually people – this gives a FAR of (2/50)*100%=4% per day.
The ultimate goal of analytics is to find a level of sensitivity that balances between maximising the DR whilst minimising the FAR.
The probability of a false alarm is often related to the environment. Table 5 shows some example environments and the likelihood of a false alarm.
Environment |
Probability |
Closed environment with constant lighting, such as an internal stairwell or corridor. |
Very low |
Restricted indoor area, such as a warehouse. |
Low |
Open office area with exterior windows and entrances, and mixed or varying lighting. |
Medium |
Exterior areas, such as car parks with day-night operation, with multiple objects to detect, such as cars and people. |
High |
False alarms can come from a number of different sources
Type |
Cause |
Camera shake |
Extreme weather, heavy machinery and traffic |
Camera noise |
High-gain in dark areas, low-light cameras, AGC |
Obscuration |
Dirt, water on lens, reflections. |
False targets |
Leaves, animals, insects or litter. |
Weather |
Heavy rain, snow, fog |
Lighting |
Headlights, PC monitors, streetlights, office lights |
Windows |
Irrelevant motion occurring through an exterior window. |
Office |
Blinds and curtains by open windows |
The following images show examples of false alarms, shown in white rectangular boxes, and potential targets shown in red rectangular boxes (illustration only.)
There are a number of tools available to reduce FAR
1. Qualification: The ultimate and best solution. Remove the need to reject false alarms by correct qualification i.e. by not introducing false alarms in the first place means you don’t need to get rid of them.
2. Filter Choice: More advanced filters should have a lower FAR e.g. using a Tripwire Filter with tracking may provide better results than a simple Motion Filter with very basic directional filtering.
3. Region Of Interest: The second best way to remove false alarms. Remove everything you are not interested in and all potential false alarms. If this involves removing more than 75% of the cells return to qualification.
4. Object Size: Simply ignore objects that are too big or too small.
5. Persistence: Simply ignore objects that do not persist for long periods of time. A lot of false alarms are very transient – only occurring for a brief period of time. Real events often persist for much longer.
6. Dwell Time: Use the dwell time to ignore secondary events from the same incident, such as a person entering a room and walking about in the room.
7. Sensitivity: Use the sensitivity to adjust the balance between DR and FAR. Lower the sensitivity as much as possible without adversely affecting DR.
1. What is the difference between Control Center and 8000 analytics? Analytics in 8000 transmitters are designed to control ACF and for creating events (Control Center alarms) based on video content and behaviour. Analytics in Control Center, such as Motion Search, is a repeatable off-line process in Control Center for analyzing recordings to create intelligent indices into the video using a configurable region of interest.
2. Why do I have to tune analytics? Every situation is different. You need to instruct Analytics what to do by tuning the settings. See Qualifying Analytics.
3. Why do I get multiple events when people walk in front of the camera? Analytics has no understanding of “people” or “walking” and will produce events when, for example, it detects motion. The number of events generated for a single incident can be controlled using the dwell time.
4. What is the difference between dwell time and persistence time? Dwell time controls how often an 8000 transmitter can potentially transmit analytics events. The persistence time controls how long certain behaviour, such as motion, must persist in order to generate an event.
5. Two separate incidents have occurred in a short period of time but I only have one event? Reduce dwell time to improve probability of detecting secondary incidents.
6. What is the smallest region of interest I can set? Theoretically, it is possible to set a region of interest with only a single detection cell. However, this is not recommended. See Installation Tips.
7. Should I get an event for every spike in a Control Center motion search? No, not necessarily. The Analytics Filters may reject the source of these spikes as false alarms.
8. I have video streaming and I can see significant motion but I don’t get any Analytics events from a Motion Filter? Check that events are enabled for the filter on the web page and that the motion is occurring inside the ROI. Use the 8000s Events page to test for analytics events. Check Control Center has the alarm source enabled.
If still no events configure the filter:
9. I have a Motion Filter with a large persistence time but my detection rate has reduced significantly. What can I do? Motion must be sustained throughout the persistence time. Try reducing the persistence time as low as allowable by your application and/or try increasing the sensitivity and the minimum object size.
10. Will motion detection work with a PTZ camera? Yes. However, it is not recommended. PTZ operations will cause motion detection events to occur. Further, any ROI that has been configured will be potentially rendered useless after the first change in position.
11. What is the relationship between ACF and Analytics? This is described in detail in [1].
12. What is the recommended video I-frame interval? It is strongly recommended that the I-frame interval be left at the default of 4000ms, though it can be lowered to 1000ms. Analytics Filters will not work on an I-frame only bitstream.
13. The bounding rectangles shown in Control Center seem to be bigger than the maximum object size? An object may comprise of an irregular, non-rectangular, combination of cells. As such not every cell within the bounding box will be part of the object.
14. I cannot select either any filter type apart from Motion? In order to use the other filters a valid licence key must be entered on the Firmware Upgrade page.
15. Why don’t I get any green boxes with congestion? Congestion is a class I algorithm and does not identify specific objects.
16. Does switching on Embed analytics data impact video quality? To carry the extra information the actual video is compressed into approximately 8kbps less than the stipulated target when selected.
[1] “Understanding ACF”, IC-COD-REP011-1.5, 27 July 2006.
Term | Definition |
ACF | Activity Controlled Frame rate |
AGC | Automatic Gain Control |
Cell | Smallest unit of area used in the detection process |
Congestion | Process of detecting large localized slow or fast motion |
Counter-flow | Process of detecting motion moving in an illegal direction |
DR | Detection rate: Number of correctly identified objects over time |
False alarm | A spurious or unwanted event generated by a filter |
FAR | alse alarm rate: Number of false alarms over time |
Filter | Processes analytics data for a specific task e.g. tripwire monitor |
FOV | Field-of-view |
Museum | Process of detecting changes in the static background |
ROI | Region Of Interest |
VMD | Video Motion Detection |
Virtual tripwire | Process of detecting motion over a user-defined line |