Can Automatic Speech Recognition reduce fuel consumption?

DLR, together with 10 air traffic controllers (ATCO) from Austro Control supported by two safety experts from Integra validated the influence of automatic speech recognition on flight efficiency and ATM safety in DLR’s ATMOS simulation environment.

The trials were conducted within solution 96 of the SESAR-IR funded PROSA project. The speech recognition and understanding architecture of HAAWAII is used. The same applies for the improved transcription and annotation rules.

Five DLR experts for the ATM simulator and for speech recognition and understanding (ASRU), one subject matter expert from Austro Control, one observer from SESAR Joint Undertaking also participated In these successful validations to quantify the benefits of ASRU for Vienna approach.

Each ATCO was involved in four different simulation runs either with a medium or a high (42 arrivals per hour) traffic scenario. Both scenarios were conducted once with ASRU support and once without it. To avoid sequence effects five of the ATCOs started with ASRU support and the other five started without it (so called baseline runs).

DLR is currently evaluating the effect of ASRU on the flown trajectory length, which correlates of course with kerosene consumption. In baseline runs the ATCOs entered all spoken commands manually into the aircraft radar labels, i.e. they entered heading, speed, flight level and altitude values as well as cleared waypoints, transitions and descent rates. In simulation runs where ASRU was active, these inputs were done automatically, only required correcting actions when ASRU did not recognize a clearance correctly. It can be already stated now, that the number of necessary HMI inputs was reduced by a factor between 8 and 12 with ASRU support.

Left side: Radar Screen with output of ASRU in purple, right side: output of ASRU on word level and on semantic level

This can increase ATM safety, because the time saved on HMI inputs, is available as an additional buffer for safety. When it becomes necessary, the ATCO can use this additional time for taking care of emergency flights, for coordinating things with other sectors or simply for improving the performance by giving the right clearance at the “perfect” time.

These safety buffers were not only claimed, but measured by a secondary task, which the ATCOs performed on top of controlling the traffic.

ATCO performing secondary task to select the colour of a displayed word

From observations (without detailed evaluation) it can be assumed that the command recognition rate was above 90%. Detailed evaluation of ASRU performance is in progress and will be reported later. Safety experts from Integra, on behalf of Air Navigation Services of the Czech Republic, are currently evaluating, if the remarkable performance of  ASRU could also have negative effects on safety due to over trust in the system. The solution in this case is the integration of ASRU output with the downlinked settings from the aircraft, so that an additional safety net is available.

The benefit of the presented ASRU solution: The ATCO can concentrate on his/her main task, monitoring and sequencing aircraft and can avoid the tedious work of maintaining the radar label contents!