Learning and Representation

The Learning and Representation (LEAR) program aims to develop software which will help the military make superior battle decisions and also be applicable in areas such as movies and video games.

The threat of terrorism has resulted in a dramatic increase in the attention placed on improving the efficiencies of warfare, reducing loss of life and costs.

Military personnel are faced with managing and interpreting copious amounts of information, and the expertise to make the right decisions is currently trapped in the minds of the highest-ranking commanders who have years of experience. The LEAR project is creating innovative learning algorithms which enable even lower-ranking officers to make superior decisions. The program will allow users to ‘play forward’ a real battle to better evaluate strategies, as well as help them make decisions on purchasing weaponry and on logistics.

There are many important non-military applications of this technology. Already computers have been used to create realistic battle scenes in films such as the Lord of the Rings trilogy, and there is enormous potential for creating computer-controlled characters in video games which behave in a human-like manner.

What will this research achieve?

The greatest loss during war is human life, followed by the financial costs. LEAR will be able to predict the outcome of particular tactics to ensure the most effective strategy can be chosen, from a range of options, that results in the most effective and satisfactory results for that circumstance. The most important end result being the reduction of the number of human casualties incurred.

Who will benefit?

While the direct end-user of this research would be military strategists, the basic concept behind LEAR can be used in many important non-military applications involving simulation of large numbers of people or other self-directed entities.

What are the key features?

LEAR is building a mathematical description of how self-directed entities, known as agents, behave by themselves and in reaction to others. An analogy would be simulating a person driving a car. While most drivers might drive as fast as possible until reaching a certain following distance, an aggressive driver would switch back and forth between lanes and honk the car horn. In the case of LEAR, the simulation is done with troops on the ground, soldiers driving tanks and pilots flying jets.

The most important feature of LEAR is that it allows the agents to learn to work together. The agents try various behaviours and remember what works best so it can be used in future similar situations.

Progress update

Scenarios involving jets flying to fixed target locations while avoiding enemy jets have been simulated using a program developed by the Defence Science and Technology Organisation.

Experimentation with controlling agents in a demo video game in collaboration with a small Australian video game development start-up has also been investigated.

Due to a change in end-user priorities the participants agreed to terminate the project early.

Research team

Bernhard Hengst (Project Leader)

CheeWee Phua

Robert Fitch

Participants

Defence Science and Technology Organisation

Swedish National Defence College

Publications

Structural Abstraction Experiments in Reinforcement Learning (202 KB) Robert Fitch, Bernhard Hengst, Dorian Suc, Greg Calbert, Jason Scholz, The 18th Australian Joint Conference on Artificial Intelligence, Sydney Australia December 2005.

Machine Learning for Adversarial Agent Microworlds (498 KB) J. Scholz, B. Hengst, G. Calbert, A. Antoniades, P. Smet, L. Marsh, H-W. Kwok, D. Gossink, MODSIM'05 Melbourne Australia December 2005. 

Tracking Value Function Dynamics to Improve Reinforcement Learning with Piecewise Linear Function Approximation. Chee Wee Phua, Robert Fitch, ICML 2007