Hide menu

The Complex Acoustic Surveillance and Tracking (COAST) Project

  • Funding agency: CENIIT
  • Project leader: Isaac Skog
  • Duration: 2019 - onward

Background

Historically, Sweden holds a strong position in the area of underwater technology, with world-class submarines and autonomous underwater systems. An important component needed to ensure that Sweden remain in the technological forefront within this area, and maintains its industrial leadership and defence capabilities, is the development of new technologies for underwater monitoring and surveillance. During the last decades — driven by the development in sensing, computational, and communication capabilities — a range of new signal processing theories and methods for target detection and tracking, such as random finite set filters, adaptive waveform design, and track-before-detect filters, have been developed. Along the same line of research, methods for sensor management and distributed sensor fusion in sensor networks subjected to energy, computational, and communication constraints, have been developed. These methods have successfully been applied in various terrestrial radar systems and sensor networks. However, due to the complex acoustic environment the application of these theories and methods in sub-surface monitoring and surveillance systems remains a challenge. This is especially the case for systems that are to operate in littoral waters, such as the Swedish coast. The shallow depths, surface heating and cooling, salinity changes, dense civil traffic, etc., result in an ever-changing acoustic environment; an environment to which the theories and methods must be adapted and tuned. Generally, the complex acoustic environment also limits the communication capabilities within underwater sensor networks to a point beyond that imposed on terrestrial sensor networks. This implies that the employed distributed target detection and tracking methods have to function with a minimum of information exchange, and new communication strategies that make dual use of the communication signals both for sensing and information exchange, are necessary.

In addition to the mentioned challenges, generally comes the problem of not knowing the exact location of the sensors within the surveillance system, which can severally degrade the performance of the system. The uncertainties in the location of the sensors is caused by the need to, in many situations, quickly and quietly deploy the system. This implies that it is not feasible to apply active calibration methods to infer the location of the sensors. Neither can, due to the poor propagation of radio waves in water, standard radio based positioning tools commonly employed in terrestrial systems, such as the Global Navigation Satellite Systems (GNSS), be used to infer the location of the sensors. Further, other sensing platforms, such as hydrophone arrays, may be towed behind a vessel and constantly change location and geometry. Therefore, new methods for passive calibration, a.k.a self-calibration, of the sensor locations, must be developed.

Goal

The goal of the Complex Acoustic Surveillance and Tracking (COAST) project is to, in cooperation with the Dept. of Naval Systems & Underwater Technology, Swedish Defence Research Agency (FOI), and Saab Dynamics, research and develop methods to adapt current state-of-the-art target detection and tracking methods to the complex acoustic environment encountered by sub-surface systems. To obtain results of long-lasting impact and relevance to the Swedish industry and defence, experimental driven research where theories and methods are developed, verified, and adapted based upon sea trial data, will constitute a core part of the project activities. Focus will be on combining the theoretical excellence of the Dept. of Automatic Control, Linköping University (LiU), with the excellent knowledge about the underwater domain and unique experimental resources, of the Dept. of Naval Systems & Underwater Technology, FOI, and the knowledge about system integration and operational requirements at Saab Dynamics.

Project description and work packages

As a first step in the development and adaption of current state-of-the-art target detection and tracking methods to the complex acoustic environment encountered by sub-surface systems, the following three work packages (WPs) have been framed.

WP1: Sensor Location Calibration

For a sensor network or sensor array to reliably detect and track a signal sources, the location of the sensors within the system must be known with high accuracy. Although the location of the sensors are generally known with high accuracy when released at the surface, due to their drift in the water, the exact locations of the sensors when they have sunk to the bottom are unknown. Needless to say, for sensor systems towed behind vessels the sensor locations are constantly changing and are therefore unknown. Thus, various active calibrations methods for estimating the location of the sensors have been developed. However, in many situations it is due to time and cost constraints, and the risk of revealing the existence of the surveillance system or disturbing the marine wildlife, desirable to avoid active calibration methods. Therefore, methods that may be used to either track the motions of the sensors while they sink or to passively estimate their locations once they are on the bottom, needs to be developed.

Aim: To study the feasibility of using: (i) inertial sensors to track the sensors while sinking or towed behind a vessel; and (ii) signals of opportunity, or in the case of towed system the self-noise of the vessel, for estimating the locations of the sensors.

WP2: Track-Before-Detect in Non-Stationary Disturbances

Track-before-detect have been shown to be a powerful technique for simultaneous target detection and tracking within the same stochastic filtering framework. By coherently integrating the measured signals along possible target trajectories, and thereby increasing the signal-to-disturbances ratio, it enables targets to be detected and tracked at low a signal-to-disturbance ratio. Fundamental for a successful use of the track-before-detect technique is access to realistic target motion models and disturbance models. Realistic motion models are generally readily available for the considered targets. However, reliable disturbances models are generally not available. The disturbances include ambient noise caused by water turbulence, shipping, wave action, rainfall, etc., and reverberations due to bottom, surface, and in volume reflections. The properties of these disturbances varies with the location, observation direction, time, etc. Therefore, even though well-established models for the individual disturbance sources under different environmental conditions exist, it is difficult to create generic applicable models. Further, many of the models captures only the temporal behavior of the disturbances and not the, for track-before-detect applications, important spatial behavior. Hence, new methods for online learning of disturbances models suitable for track-before-detect applications are needed.

Aim: To study and develop methods for online learning of disturbance models suitable for track-before-detect applications in underwater surveillance systems.

WP3: Sensor Management for Joint Sensing and Communication

To optimize the detection and tracking performance in underwater surveillance systems, various sensor management strategies for selecting the optimal sonar waveforms, sensor locations, and paths for reconnaissance vessels, have been proposed. In distributed underwater surveillance networks, one of the main bottlenecks when implementing a detection and tracking framework is the, in general, limited bandwidth available for sensing and communication; the limited bandwidth generally implies that suboptimal distributed information fusion strategies have to be used, with a loss in detection and tracking performance as a result. One possible way to better use the available bandwidth and improve the performance is to use the, within the radar community, recently proposed idea of joint sensing and communication. That is, the communication signals are design so that they can simultaneously be used both for information exchange and active sensing. To maximize the benefits of the joint sensing and communication concept new distributed sensor management and information strategies must be developed; strategies that, based upon current available information decides on which sources that are to communicate, when to communicate, and what waveforms to be used.

Aim: To research and develop sensor management methods for joint sensing and communication in underwater sensor networks.

Project Status and Results

Summary of Activities Conducted Within The Project 2020-2021

  • Sea-trial where inertial and pressure data was recorded from sinking underwater sensor nodes.
  • Sea-trial where data to investigate the possibility of using communication signals as sonar pulses was recorded.
  • Guest lectures on Array signal processing in TSRT78 Digital Signal Processing, Magnus Lundberg Nordenvad, FOI.
  • MSc thesis: Tracking of sinking underwater node using inertial navigation, Kristoffer Lindve, FOI/MH.
  • Two conference papers (see [1] and [5]) and three journal papers (see [6,7,8]) published.
  • Development of software toolbox for array calibration during sea-trials; based upon [3] and successfully used during several sea-trials.
Further the project Tensor-field based localization has been started. The project is funded by the Swedish Research Council, and will study how magnetic field variations and sensor arrays can be used for localization in GNSS denied environments, such as underwater environments.

Summary of Activities Conducted Within The Project 2019-2020

  • PhD student Daniel Bosser recruited to work in the project.
  • Seminaries on Underwater acoustics in marine environments for the benefit of defence and security , by Lena Lund, FOI.
  • Seminaries on Signal processing in marine bioacoustics , by Mathias Andersson, FOI.
  • Guest lectures on Array signal processing in TSRT78 Digital Signal Processing, Magnus Lundberg Nordenvad, FOI.
  • MSc thesis: Trajectory and pulse optimization for active towed array sonar using MPC and information measures, , Fabian Ekdahl Filipsson, FOI/UU.
  • MSc thesis: Bistatic sonar using communication pulses , (ongoing), Erica Svensson, FOI/UU.
  • One conference paper (see [2]) published and one journal paper (see [3]) submitted.

Results WP1: Sensor Location Calibration

Signals of opportunit based calibration

A simultaneous localization and mapping (SLAM) method to calibrate the geometries of hydrophone arrays using the sound emitted from nearby ships have been developed. The performance of the proposed calibration method have been evaluated using data from two PASS-2447 Omnitech Electronics Inc. 56- element hydrophone arrays. Tests with three data sets have shown that array geometries in the north-east plane can be consistently estimated. Further, the calibration of the array geometries have shown to increase the source localization accuracy significantly. The findings are presented in the conference paper:
Illustration of the signals of opportunity based array shapes estimation setup. From the sound emitted by a ship passing by the arrays time difference of arrival estimates are calculated and used to estimate the location and shape of the arrays.
A video illustrating the calibration method in action is shown below. Both the location of the source (MS Gotland) and the shape of the hydrophone arrays are estimated simultaneously. Two different models are used to describe the array geometries. The first model (red color in the video) does not impose any constraints on the location of the hydrophones. The second model (blue color in the video) perseveres the a priori known maximum distance between the hydrophones.


Inertial tracking of sinking sensor node

The possibility to use inertial and pressure sensors to track the location and orientation of a sinking sensor node has been investigated. Sensor data was collected during a sea-trial, and then processed using an extended Kalman filter based depth aided inertial navigation system. The results are presented in the MSc thesis Tracking of sinking underwater node using inertial navigation by K. Lindve.
Underwater sensor node used in the sea-trial .

Results WP2: Track-Before-Detect

Together with Robin Forsling and Gustaf Hendeby, a study on passive multi-sonar track-before-detect has been initiated. Experimental data from a test with an unmanned underwater vehicles (UUV) moving in littoral water has been collected by the Underwater Department, FOI. The initial results shows that use of track-before-detect enables earlier detection at a lower false alarm rate compared to conventional methods. Current focus is on enhancing the performance further by learning the background disturbances. A short video of the algorithm in action is shown below.

Project Collaborations

FOI Swedish Defense Research Agency

The PI has a close collaboration with the Underwater Department at FOI, Swedish Defense Research Agency, and the PI currently hold a 10% adjunct position a FOI. The collaboration has this far results in four joint publications ([2,3,4,5]) and three MSc thesis projects. Further, researchers from FOI have at two occasion given seminars on underwater acoustics at ISY, LiU. Moreover, PhD. M. L. Nordenvaad, FOI, has become a co-supervisor for PhD student D. Bossér. He has also given guest lectures on array signal processing in the TSRT 78 Digital Signal Processing course during 2020 and 2021.

Beyond the collaboration within the COAST project, I and FOI has received joint funding from Security Link for a project on localization in GNSS denied environments, such as underwater environments. See project description about Tensor-field based localization project.

WASP WARA PS

In collaboration with the CENIIT project Applied Research Platform for Sensor Fusion and WASP WARA PS (contact: Lars Rundqwist) a demonstrator platform for real-time detection of anomalies in ship routes is currently under development. The underlying algorithm is based upon the results presented in [1], and sea-trial to collect experimental data is scheduled for end of September (postpone due to covid).

Publications

Project publications

The research within the COAST project has resulted in the following publications:
  1. A. Kullberg, I. Skog, and G. Hendeby, Learning Motion Patterns in AIS Data and Detecting Anomalous Vessel Behavior, Proc. FUSION 2021, South Africa, Nov. 2021.
  2. I. Skog and E. Gudmundson, Signals of Opportunity based Geometry Calibration of Hydrophone Arrays , Proc. OCEANS 2019, Seattle, WA, Oct. 2019.
  3. I. Skog and M.L. Nordenvaad, Signals of Opportunity Based Hydrophone Array Location, Orientation, and Shape Calibration , In review: IEEE Journal of Oceanic Engineering, 2021.
  4. R.L. Nordström, E. Lalander, I. Skog, and M. Andersson, Maximum likelihood separation of anthropogenic and wind-generated underwater noise, Submitted to: The Journal of the Acoustical Society of America, 2021.

Related publications

Research conducted in projects closely connected to the COAST project has resulted in the following publications:
  1. I. Skog, G. Hendeby, and F. Trulsson, Magnetic-field Based Odometry – An Optical Flow Inspired Approach, IEEE Int. Conf. Indoor Positioning and Indoor Navigation (IPIN), Nov. 2021.
  2. H. Carlsson, I. Skog, T.B. Schön, and J. Jaldén, Quantifying the Uncertainty of the Relative Geometry in Inertial Sensors Arrays, IEEE Sensors Journal, vol. 21, no. 17, 2021
  3. H. Carlsson, I. Skog, and J. Jaldén, Self-Calibration of Inertial Sensor Arrays, IEEE Sensors Journal, vol. 21, no. 6, 2021
  4. A. Kullberg, I. Skog, and G. Hendeby, Online Joint State Inference and Learning of Partially Unknown State-Space Models, IEEE Trans. Signal Processing, DOI: 10.1109/TSP.2021.3095709, 2021.
  5. A. Kullberg, I. Skog, and G. Hendeby, Learning driver behaviors using a Gaussian process augmented state-space models, Proc. Int. Conf on Information Fusion, Online, Jul. 2020.
  6. J. Wahlström and I. Skog, Fifteen Years of Progress at Zero Velocity: A Review, IEEE Sensors Journal, DOI: 10.1109/JSEN.2020.3018880, Aug. 2020.
Isaac Skog

Associate Professor in Automatic Control,
Docent in Signal Processing

(Swedish: Universitetslektor i reglerteknik)

Phone:
+46 708186805
E-mail:
isaac.skog_at_liu.se
Address:
Dept. of Electrical Engineering
Linköping University
SE-581 83 Linköping
Sweden
Visiting Address:
Campus Valla
Building B
Room 2A:526


Page responsible: Isaac Skog
Last updated: 2021-09-29