README.md 6.59 KB
Newer Older
Oliver Kirsebom's avatar
Oliver Kirsebom committed
1

Oliver Kirsebom's avatar
Oliver Kirsebom committed
2
3
4
5
6
7
8
9
10
11
12
13
# MERIDIAN Acoustic Boat Detector

Using the [Ketos](https://docs.meridian.cs.dal.ca/ketos/) package we have created a little command-line tool for processing hydrophone data and searching for the acoustic signature of motorized vessels. In other words, a boat detector!


# Installation

```terminal
pip install -r requirements.txt
python setup sdist
pip install dist/boat_detector-1.0.0.tar.gz
```
Oliver Kirsebom's avatar
Oliver Kirsebom committed
14
15
16
17


# Windows users

Oliver Kirsebom's avatar
Oliver Kirsebom committed
18
19
20
21
22
See the [Step-by-step guide for Windows users](https://gitlab.meridian.cs.dal.ca/data_analytics_dal/projects/boat_detector/wikis/Step-by-step-guide-for-Windows-users).


# Usage example

Oliver Kirsebom's avatar
Oliver Kirsebom committed
23
24
To illustrate the usage of the boat detector, we have included an example data set in the 
`assets/` directory, which 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
25
consists of 60 short audio recordings, each lasting no more than approximately 10 seconds. 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
26
The recordings were made over the course of 1 hour using a broadband hydrophone with a sampling 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
27
28
29
30
31
32
33
rate of 20 kHz. The hydrophone was deployed in a shallow bay that is busy with recreational 
boating. The data were kindly shared with us by Mitchell Rider from the Rosenstiel School 
of Marine & Atmospheric Science in the U.S. To save space the audio files have been 
downsampled to 4 kHz. The start time of each audio file is encoded into the file name using 
the format `HMS_%H_%M_%S__DMY_%d_%m_%y` where `%H` is the hour, `%M` is the minute, `%S` 
is the second, `%d` is the day, `%m` is the month, and `%y` is the year.

Oliver Kirsebom's avatar
Oliver Kirsebom committed
34
The boat detection program consists of two steps, data pre-processing and signal detection:
Oliver Kirsebom's avatar
Oliver Kirsebom committed
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

## Step I: Pre-processing

Pre-process the audio files in the `assets/` directory with the command
```terminal 
$ boat-preproc -c settings.json -i assets/ -v
```
This should produce the following output,
```terminal 
 Found 60 files
 Processing batch #1 ...
 Processing batch #2 ...
 Processing batch #3 ...
 Successfully processed 60 files
 Processed data saved to: output.csv
 Time table saved to: output_time_table.csv
 Elapsed time: 00:00:07
```

## Step II: Detection

Search for acoustic signatures of boats in the preprocessed data
```terminal
$ boat-detect -c settings.json -i output.csv -S
```
This should produce the following output,
```terminal 
 7 boats detected
 Detection report saved to: detections.csv
```
and this figure:

![](assets/boat_detections.png)


# Options

## Pre-processing script
```terminal
$ boat-preprocess -h
usage: boat-preprocess [-h] [-c CONFIG_FILE] [-i INPUT] [-o OUTPUT_FILE] [-r]
                       [-v]

Split audio signal into frequency bands and produce a time series of the noise
magnitude in each band.

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config_file CONFIG_FILE
                        path to json configuration file.
  -i INPUT, --input INPUT
                        path to the wav file to be analyzed or directory
                        containing multiple wav files.
  -o OUTPUT_FILE, --output_file OUTPUT_FILE
                        path to output csv file
  -r, --recursive_search
                        Include subdirectories in search for wav files
  -v, --verbose         Print progress updates during processing
```

## Detection script
```terminal
$ boat-detect -h
usage: boat-detect [-h] [-c CONFIG_FILE] [-i INPUT_FILE] [-o OUTPUT_FILE] [-S]

Perform outlier- and peak analysis of time-series data.

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config_file CONFIG_FILE
                        path to json configuration file.
  -i INPUT_FILE, --input_file INPUT_FILE
                        .csv file containing time-series data to be analyzed.
  -o OUTPUT_FILE, --output_file OUTPUT_FILE
                        .csv file where analysis report will be outputted.
  -S, --show_graph      Show time-series data
```

# Configuration files

Oliver Kirsebom's avatar
Oliver Kirsebom committed
115
## General settings (settings.json)
Oliver Kirsebom's avatar
Oliver Kirsebom committed
116
117
118

The main configuration file informs the program about 1) the date-time filename 
labeling convetion, 2) the maximum amount of audio data to be processed in a single 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
119
120
batch, and 3) the path to the detector configuration file.
```terminal
Oliver Kirsebom's avatar
Oliver Kirsebom committed
121
122
123
124
125
126
127
128
$ more settings.json 
{
    "date_time_format": "HMS_%H_%M_%S__DMY_%d_%m_%y",
    "batch_size": "5E6",
    "detector": "detectors/fav.json"
}
```

Oliver Kirsebom's avatar
Oliver Kirsebom committed
129
## Detectors
Oliver Kirsebom's avatar
Oliver Kirsebom committed
130

Oliver Kirsebom's avatar
Oliver Kirsebom committed
131
132
The `detectors/` folder contains two detector configuration files, [fav.json](detectors/fav.json) and [obi.json](detectors/obi.json), 
which implement two different detection strategies.
Oliver Kirsebom's avatar
Oliver Kirsebom committed
133
134
135
136
137
138
139
140
141
142

### fav.json

To detect the presence of a boat, the FAV (Frequency Amplitude Variation) detector takes advantage of the fact 
that the frequency spectrum of a boat often contains several narrow peaks, which occur at frequencies ranging 
from tens of Hz to several hundred Hz. More specifically, the FAV detector computes the Short-Time Fourier 
Transform (STFT) of the audio data, searches for narrow peaks, and reports a boat detection if a few such 
peaks are found. The peak search algorithm is similar, though not identical, to the one described in 
[this paper](https://doi.org/10.1111/2041-210X.13245). 

Oliver Kirsebom's avatar
Oliver Kirsebom committed
143
The performance of the FAV detector can be optimized by adjusting the parameters found in the [fav.json](detectors/fav.json) 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
144
configuration file. While this file contains many parameters, the most relevant are the `threshold` parameter (found in 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
145
146
147
148
149
150
151
152
the `fav_config` section) and the `height` parameter (found in the `detect_config` section). The former determines how sharp and tall 
a frequency peak must be to be counted as a peak, while the latter determines how many such peaks must be present at given 
instant to produce a boat detection.

### obi.json

The OBI (Octave Band Intensity) detector, on the other hand, detects boats by looking for variations in the 
sound intensity within the 2nd, 3rd, and 4th octave band, centered at 125 Hz, 250 Hz, and 500Hz.
Oliver Kirsebom's avatar
Oliver Kirsebom committed
153
The performance of the OBI detector can be optimized by adjusting the parameters found in the [obi.json](detectors/obi.json) 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
154
configuration file. While this file contains many parameters, the most relevant are the `prominence` and `multiplicity` 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
155
156
parameters (found in the `detect_config` section). The former determines how prominent the increase in the 
sound intensity must be to be counted as signal, while the latter determines in how many octave bands such 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
157
a signal must be detected in to produce a boat detection.
Oliver Kirsebom's avatar
Oliver Kirsebom committed
158

Oliver Kirsebom's avatar
Oliver Kirsebom committed
159
160
Below, you see the spectrogram of a 10-second long audio recordering containing noise from a boat. 
Narrow peaks in the frequency spectrum are seen as sharp horizontal lines in the spectrogram, visible 
Oliver Kirsebom's avatar
Oliver Kirsebom committed
161
162
at about 20 Hz, 50 Hz, 95 Hz, and 145 Hz, as well as an overall increase of sound intensity, especially 
in the third octave band centered at 250 Hz.
Oliver Kirsebom's avatar
Oliver Kirsebom committed
163
164
165

![](assets/boat.png)

Oliver Kirsebom's avatar
Oliver Kirsebom committed
166
167
168