SAIL Home                       SAIL Project                        SAIL Team 

Summary

In the Pilot Phase, the SAIL Project has successfully:

• Used formal techniques for inputting, representing, reasoning, fusing and querying information obtained from diverse information sources.
  
• Implemented a novel layered architecture that integrates First Order Logic and Description Logic reasoners as well as Control Natural Language Processing.
  
• Designed and built a prototype SAIL platform to demonstrate the use of these techniques.
  

Materials

• SAIL Research Paper
• SAIL Platform Architecture
• SAIL Animated Powerpoint Presentation
• SAIL TechFest 2008 Poster
  
  

Functionality

The prototype SAIL platform takes low level sensor data such as radar tracks and integrates it with eyewitness reports and background information such as maps (GIS data) and databases (e.g. information about vehicles, political situations, etc) to construct a representation that is both real-time and at the same semantic level as the human operator, enabling effective situation management. It also identifies emerging factors of interest in the situation and alerts the operator. 

Functionality is provided in successive tiers (see diagram above):

• Data Collection:  Collects input data from sensors and other sources, including track data that captures track locations and velocities at successive time points, and eyewitness reports in text form.
  
• Data Aggregation: Collected data is analysed to identify entities of a situation and their low level properties, for instance, that a track corresponds to an object that is an aircraft, and has a trajectory that forms a circle.  This layer includes integration with databases and GIS information.
  
• Semantic Analysis: Interpretation and evaluation of identified entities to identify their significance e.g that the behaviour shown by a fighter plane indicates it is being aggressive to some object.
• Alert Generation:  Monitors and projects events unfolding over time to identify possible high impact eventualities and warns the operator.  For example, a surveillance operation taking place now may indicate a future aggressive action, so the operator is alerted.
  
  

Key Capabilities

• Data inputs:  accepts input from both electronic sensors and from eyewitness reports structured in Controlled Natural Language (a predefined grammar and vocabulary).  
• Data fusion: fuses data from multiple sensors or sources that capture the same objects or events.
• Data integration: integrates input data with background information about the environment, objects and events, including GIS data and knowledge bases.
• Domain representation: the system needs to be able to recognise a range of things from simple objects, through to complex events and behaviours.  For instance, in the military domain, these may range from objects like aircraft and ships through to a complex behaviour like the enemy preparing for an aggressive manouvre. The design phase involves working with domain experts to identify and create a suitable representation of the kinds of objects, events and behaviours of interest to operators and of relevance for effective situation management.  Part of this work involves specifying the criteria for classification. 
  Note: this is a modular part of the SAIL architecture; the SAIL technology can be applied in any domain.
• Object and event recognition: the system is able to use the domain representation to recognize specific objects, events and behaviours from input data as they occur.
• Projecting situations into the future: given adequate classification criteria, the system is able to identify initial signs of potential events of interest, and to alert operators
• Alert generation: the system may be set to generate alerts to operators whenever it recognises actual or emerging events of interest.   This is particularly useful in case the operator's attention has been diverted.  Early alerts assist the operator to  effectively manage the current situation effectively and prevent undesirable outcomes. 
• Question answering: the system is able to answer a large variety of questions posed by the operator about the situation, objects and events.  These questions must be phrased in Controlled Natural Language  (a predefined grammar and vocabulary).
• Visual interface: the system is able to be integrated with visual interfaces.  The pilot uses Google Earth to illustrate this.
• Dealing with time and space: the system is able deal effectively with temporal and spatial factors, primarily via its novel architecture.
• Real-time: the system is designed to operate in real-time.
  
  

Approach

The SAIL platform has utilized a declarative approach that uses formal logics to state rules and definitions and to calculate their consequences.  It also utilizes a novel architecture that allows these to be used together effectively to deal with space and time.

Key Features

• Two tiered architecture uses both first order logic (FOL) and description logic (DL) formalisms.
• FOL is used for data aggregation.  If-then rules are used to specify data aggregation in a declarative way.  The core component is E-KRHyper, a FOL theorem prover. The result is passed on to the DL layer.
  
• DL is used for semantic analysis and alert generation. For semantic analysis, conceptually high-level concepts and roles are introduced as DL axioms using the primitive concepts defined by the data aggregation layer.  RacerPro is used as the reasoner for this layer.  Alert generation is also implemented using DL / RacerPro, realizing an approach based on Linear Time Logics.
  
• Control Natural Language queries are parsed, converted to logical form and submitted to RacerPro to interrogate the DL ontology and return results, which are then parsed back into CNL.
  
  

Benefits of the SAIL Approach

The SAIL approach is novel and offers significant benefits compared to current state-of-the-art Situation Awareness Systems, as explained below.

State-of-the-art Situation Awareness systems

Current systems provide only limited functionality due to insufficient capabilities regarding aggregation of sensor data, representing domain knowledge, reasoning with time and space, and interacting with human operators.

State-of-the-art systems typically:

• support some data fusion to aggregate data about an object from multiple sensors, but do not integrate sensor data effectively with other background knowledge
  
• present the operator with a "dots-on-maps" interface, but do little to provide semantic information about those dots e.g. what is this dot? who does it belong to? what is it doing? where is it going? is it a threat?
  
• use some procedural processing (conventional programming techniques) but do not use a declarative approach, where definitions are stated formally and processing is done by computing inferences from their logical consequences.
  
• are unable to process inputs presented in natural language
  
  

Novelty and Benefits of the SAIL approach

SAIL's novel system architecture provides the capabilities lacking in state-of-the-art systems described above.  In particular:

• The SAIL system integrates sensor data, GIS and background knowledge into a single semantic representation of the system that the operator can interact with directly.
  
• SAIL has a modular architecture which supports a "plug and play" approach with ontologies. By simply replacing the ontology, the input data can be aggregated and interpreted in a completely different way.
  
• SAIL has the ability to interface with the human operator in English. Using a limited subset of English known as "Controlled Natural Language" that has a limited grammar and vocabulary, the operator can ask questions and receive answers in English, and will also be alerted to emerging events that merit operator attention.
  
• Inputs presented in Controlled Natural Language, such as eyewitness reports, are accepted and processed by the system.
  
• SAIL uses a declarative approach rather than a procedural approach.  It defines concepts and their relationships in an ontology and uses automated reasoning techniques to build the overall picture of the situation.  This obviates the need for conventional programming.
  
• Being modular and reusable, the SAIL platform reduces the development time for decision-support systems while substantially enhancing their ability to manage complex information from multiple data sources.
  
  

Future Extensions

In Phase Two, SAIL researchers plan to:

• use state-of-the art data mining techniques and algorithms to recognise patterns emerging from low level data.
• explore formal methods for dealing with uncertain and probabilistic data.