Video Analysis and Content Management for Surveillance

The Video Analysis and Content Management for Surveillance (VACMS) project aims to develop intelligent video surveillance systems that can recognise specific events by understanding their context and make a note of them or alert the operator.

In the traditional Closed Circuit Television (CCTV) systems, human operators have to monitor large numbers of terminals to process the captured visual information. Such a process is inefficient and difficult, which means human operators are likely to miss unusual or suspicious situations. The challenge is to give these automated systems semantic or human-like understanding when detecting and interpreting events of interest.

The project has applied research in three areas: novel algorithms, novel applications and a prototype system framework. Mathematical models have been developed to teach a computer to recognise a sequence of events from different angles and views.

What will this research achieve?

An intelligent video monitoring and management system aims to free security operators of the mundane and repetitive task of monitoring video information, allowing them to focus on higher-value tasks such as responding to the event. The new technology can recognise such things as a traffic accident or a bag left unattended and alert the operator. It can also scan large databases of stored video footage at high speed and find examples of specific types of occurrences. The socio-economic value of this research includes the potential contribution to counter-terrorism efforts, national security and public safety.

Who will benefit?

Security officers who use these new surveillance techniques and everyone who is protected by them.

What are the key features?

The team’s challenge was to develop a computer’s semantic understanding capability when detecting and interpreting events of interest:

  • By applying novel algorithms to identify the association of objects in complex scenes, for example a robbery. The team will develop mathematical models to teach the computer to recognise the sequence of events from all possible angles and views. The computer deconstructs the main event in order to recognise each pattern in the sequence and finally matching it to what it has been programmed to remember.
  • By applying novel applications – combining research with target applications such as scene analysis. For example, developing an active reaction to unwanted events by sounding an alarm, activating door locks or providing a geographical position or an image or video of the subject.
  • By applying both novel algorithms and novel applications in a software-based prototype system framework. The prototype system framework is the software that links it all together. It can bring about fundamental changes to the whole surveillance system with advanced media technologies for national security and public safety.

Progress update

The project has successfully developed a platform (hardware and software integration) showcasing key features of the system including a graphics database and an event detection module.

Selected publications:

[1]      S. Paisitkriangkrai, C. Shen and J. Zhang “Pedestrian classification using a cascade of boosted covariance features,” Accepted by Special Issue on Video Surveillance of IEEE Transactions on Circuits and Systems for Video Technology, August, 2008

[2]      Y. Wang, “Real-Time moving vehicle detection with cast shadow removal in video based on conditional random field,” accepted as a short paper by Special Issue on Video Surveillance of IEEE Transactions on Circuits and Systems for Video Technology, August, 2008

[3]   S. Paisitkriangkrai, C. Shen and J. Zhang, “Fast Pedestrian Detection using multiple layer boosting,” Accepted by The Eighth International Workshop on Visual Surveillance (VS2008) in conjunction with 2008 European Conference on Computer Vision

[4]   Z, Li and J. Chen, " An Improved Mean-Shift Tracker with Kernel Prediction and Scale Optimization Targeting for Low-Frame-Rate Video Tracking, " Acepted by IEEE International Conference on Pattern Recognition 2008

[5]   Yang Wang, “Joint Random Fields for Moving Vehicle Detection,” Accepted by British Machine Vision Conference 2008

[6]   J. Xu, G. Ye, G. Herman, B Zhang,”An Efficient Approach to Detecting Pedestrians in Video”, Accepted as a short paper by ACM International Conference on Multimedia 2008

[7]   G. Herman, G. Ye, J. Xu & B. Zhang, “Improving Object Detection by Pruning Noisy Samples from Training Sets,” Accepted by ACM International Conference on Multimedia Information Retrieval 2008

[8]   J. Xu, G. Ye, G. Herman, B Zhang, “Detecting and Recognizing Moving Pedestrians in Video,” Accepted by IEEE Workshop on Multimedia Signal Processing, 2008

[9]   C. Luo, X. Cai, J. Zhang, “Robust Object Tracking Using the Particle Filtering and Level Set Method: A Comparative Experiment”, Accepted by IEEE Workshop on Multimedia Signal Processing, 2008

[10]  Z. Li, J. Chen, A. Chong, Z. Yu, N. Schraudolph., “Using Stochastic Gradient-Descent Scheme in Appearance Model Based Face Tracking”, Accepted by IEEE Workshop on Multimedia Signal Processing, 2008

[11]  S. Lu, J. Zhang, David Feng, “Detecting Unattended Packages through Human Activity Recognition and Object Association”, the Journal of Pattern Recognition, Volume 40, Number 8, August 2007

[12]  P. Janney and J. Yu, “Invariant Feature of Local Texture – a rotation invariant local texture descriptor”, Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Minneapolis(USA), June 17-22,  2007

[13]  S. Lu, J. Zhang, David Feng, “An Efficient Method for Detecting Ghost and Left Objects in Surveillance Video”, IEEE International Conference on Advanced Video and Signal based Surveillance, September 2007

[14]  Y. Wang, “Foreground and shadow detection based on conditional random field,” Proc. Int’l Conf. Computer Analysis of Images and Patterns (CAIP), 2007

[15]  S. Lu, J. Zhang, D. Feng, “A Knowledge-based Approach for Detecting Unattended Packages in Surveillance Video”, IEEE International Conference on Advanced Video and Signal based Surveillance, November 2006

[16]  R. Mathew and D. S. Taubman, “Motion modeling with geometry and quad-tree leaf merging”, Proc. IEEE International Conference on Image Processing, pp. 297-300, September 2007

[17]  R. Mathew and D. S. Taubman, “Hierarchical and polynomial motion modeling with quad-tree leaf merging”, Proc. IEEE International Conference on Image Processing, pp. 1881-1884, October 2, 2006

 

Participants:

Professor David Feng, Dr. Weidong Cai  and Dr Zhiyong Wang, University of Sydney

Prof. Xuemin Lin, University of New South Wales

Roads and Traffic Authority, New South Wales

Research team

Dr Jian Zhang (project leader)

Dr. Yang Wang

Mr. Sijun Lu

Dr. Jing Chen

Dr. Getian Ye

Mr. Kostia Robter

Mr. Gunawan Herman (PhD student)

Mr. Pranam Janney (PhD student)

Mr. Sakrapee (Paul) Paisitkriangkrai (PhD student)

Mr. Jun Yang (PhD student)

Mr. Jie Xu (PhD student)

Ms. Jue (Shana) Wang (postgraduate student)

Contacts

For more information about the VACMS project, please contact Dr Jian Zhang, project leader, on +612 8306 0780 or Jian.Zhang@nicta.com.au.