The premiss of this project is that we will be able to visually track multiple objects using Kalman filters to constantly update the position estimates for each object even if no measurements are currently available for the objects based on their current path and trajectory.
To become familiar with tools and equipment that I'd be working with on this project I started out doing some simple image filtering based on the brightness of a given pixel and the length of edges in the picture. This program uses Intel's OpenCV library to manipulate the given image based on the before mentioned parameters. To see an example of this click here.
After becoming familiar with the OpenCV toolset I started familiarizing myself with Kalman filters. What a Kalman filter tries to do is calculate the best possible estimate for the state of a system at a particular instance in time based on the previous prediction and the current measurements. Kalman filters use a vector to model the system and a vector of measurements used to update the system model. At each time interval measurements of the systems velocities and accelerations are taken and from those measurements the system model is updated. Provided that the object being tracked is moving in a semi-linear fashion and follows the basic laws of physics, the kalman filter will be able to predict the trajectory of the system.
The program I initially looked at when trying to better understand how a Kalman filter works involved an object traveling in a circle, using only the angle to determine the objects location, at varying speeds while a Kalman filter tracked it, the program allowed me to view the measurement coordinates, the predicted coordinates, and the actual coordinates of the object so I would be able to see how well the Kalman filter did at tracking the object. Click here for an example of what this looks like. Once I had a basic understanding of how Kalman filters work I transformed that program into one where the object is moving along in a less predictable pattern, one that constantly evolves from a circle to a figure eight and back again, and added track bars to control the speed, measurement noise, and process noise. This allowed me to see how Kalman filters are able to predict the location of faster vs. slower moving objects and the changes in the accuracy of the filter when the gains are changed all in a dynamic setting. This step was necessary to gain some intuition on the types of things that could possibly throw off the accuracy of the filter once I get further along in the project. Click here to see a snapshot of the program.
The next step in the project was to redesign the structure of the code, making it as object oriented as possible, so it would not only be possible, but very simple, to add multiple objects and have them all tracked by their own Kalman filter. This helped to better simulate real world situations with tracking multiple objects. Once the code was redesigned and working efficiently, a Pan-Tilt-Zoom (PTZ) camera was added to the project. The SPU-Toolbox was utilized to help with controlling the movements of the camera and an algorithm was written so the camera could track the different objects as they moved about the screen depending on which object is being focused on at the time. An explanation of the structure and layout of the code can be found here, and this is an example of what the running program looks like. In this example the red dots represent the objects being tracked and the green cross represents the Kalman filter for the specific object being tracked at any given moment.
The next phase of the project involves figuring out the different threat levels of each object depending on their location relative to other objects on the screen at a given time and creating a scheduling algorithm that will allow the PTZ to pay extra attention to the objects that posses higher threat levels while still maintaining surveillance on the others.
