GitLab

requested review from @kirsebom

@kirsebom the test_transmission_loss_real_world_env is failing, do you know whats going wrong here?

I may have broken something by changing the ocean module functions 'bathy()' and 'temp()' to 'bathymetry()' and 'temperature()'

========================================================== FAILURES ===========================================================
____________________________________________ test_transmission_loss_real_world_env ____________________________________________
kadlu/tests/sound/test_geophony.py:156: in test_transmission_loss_real_world_env
    transm_loss, ocean = transmission_loss(seafloor=seafloor,
kadlu/sound/geophony.py:143: in transmission_loss
    ss = SoundSpeed(ssp=k['ssp']) if 'ssp' in k.keys() else SoundSpeed(
kadlu/sound/sound_speed.py:95: in __init__
    depths = self._depth_coordinates(ocean,
kadlu/sound/sound_speed.py:185: in _depth_coordinates
    indices, _ = interp_grid_1d(y=c,
kadlu/utils.py:335: in interp_grid_1d
    norm = np.max(y) - np.min(y)
<__array_function__ internals>:180: in amax
    ???
../env_kadlu/lib/python3.10/site-packages/numpy/core/fromnumeric.py:2791: in amax
    return _wrapreduction(a, np.maximum, 'max', axis, None, out,
../env_kadlu/lib/python3.10/site-packages/numpy/core/fromnumeric.py:86: in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
E   ValueError: zero-size array to reduction operation maximum which has no identity
=================================================== short test summary info ===================================================
FAILED kadlu/tests/sound/test_geophony.py::test_transmission_loss_real_world_env - ValueError: zero-size array to reduction ...
========================================= 1 failed, 5 passed, 73 deselected in 1.72s ==========================================

changed the description

added 1 commit

ba7b3cc7 - convert elevation to depths in load_from_file.py plotting

Compare with previous version

added 1 commit

c9e211d1 - fix wwiii fetching

Compare with previous version

I will take a look at this later today (this evening)

In step 1 of the installation instructions, we should include guidance on how to check if GCC is installed and how to install it, if it is not already installed.

Result of running pytest kadlu/ --doctest-modules after fresh clone of kadlu on this branch:

============================================================================== short test summary info ===============================================================================
FAILED kadlu/index.py::kadlu.index.index
FAILED kadlu/tests/geospatial/test_ocean.py::test_small_full_ocean - requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='tds.hycom.org', port=443): Read timed out. (read ti...
FAILED kadlu/tests/geospatial/test_ocean.py::test_wind_water_uv - requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='tds.hycom.org', port=443): Read timed out. (read timeo...
FAILED kadlu/tests/geospatial/data_sources/test_hycom.py::test_load_water_uv - requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='tds.hycom.org', port=443): Read timed out...
FAILED kadlu/tests/geospatial/data_sources/test_hycom.py::test_load_water_v - requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='tds.hycom.org', port=443): Read timed out....
FAILED kadlu/tests/sound/test_geophony.py::test_transmission_loss_real_world_env - AttributeError: 'Ocean' object has no attribute 'bathy'

I will investigate the failing test in the geophony module. Can you take a look at the others, @matt_s ?

Yeah, im having the same issue as well. Hycom is very slow, so I added a time limit of 5 minutes without response before failing. I had to try multiple times before it worked, but once it actually downloaded it seemed to work as intended

Can you share the full error log for FAILED kadlu/index.py::kadlu.index.index? this one is unexpected

I think it has to do with the --doctest-modules flag, I've added some code examples in the docstrings which might be failing

added 1 commit

741eacf5 - Update index.py

Compare with previous version

It might be fixed by the change in 741eacf5 but I will do more thorough testing on the issue later

added 1 commit

a990b60a - remove index.py docstring example

Compare with previous version

should be fixed now

added 2 commits

bcf82e61 - partially fixed test
18a59d1f - Merge branch 'new_docker_img' of...

Compare with previous version

I have updated the Ocean class method names in the test_transmission_loss_real_world_env test that was causing problems. However, the test still fails because the interpolated bathymetry values differ from the expected values. The expected values were based on the GEBCO 2020 NetCDF bathymetry data. I would like to run this test using the GEBCO 2021 NetCDF data instead of the Geotiff data in order to understand if the difference in bathy values is due 1) a difference between using NetCDF or Geotiff data, or 2) a difference between the GEBCO 2021 and 2020 datasets. The differences are quite substantial so I doubt that 2) is the reason but I would like to check. @matt_s , How do I go about loading from NetCDF instead of Geotiff? Is the NetCDF loading function still available and working?

>       np.testing.assert_array_almost_equal(seafloor_depth,
                                             answ_seafloor_depth,
                                             decimal=2)
E       AssertionError: 
E       Arrays are not almost equal to 2 decimals
E       
E       Mismatched elements: 9 / 9 (100%)
E       Max absolute difference: 98.22117791
E       Max relative difference: 0.26499711
E        x: array([[ 267.21,  594.1 , 1470.33],
E              [  99.98,  279.2 , 1661.36],
E              [  83.73,  468.87, 1304.41]])
E        y: array([[ 267.75,  606.44, 1398.15],
E              [  99.94,  273.12, 1651.99],
E              [  80.17,  370.65, 1307.41]])

I will revert the gebco module to the previous behaviour... in the meantime, kadlu.load_file() should work (for this specific file) after removing the assertion statements

Thanks Matt. And yes, agreed. The reason for the discrepancy could be that the newer grids simply have different data. But I would like to explore the alternative explanation too (i.e. that there is a difference between the netcdf and geotiff formats - or the way we handle them).

added 1 commit

dec765ba - reverting gebco

Compare with previous version

added 1 commit

d3ac1efd - remove old backup

Compare with previous version

added 1 commit

42ad84ee - remove old CHS code

Compare with previous version

added 1 commit

1818f5b7 - update install instructions explaining how to install GCC

Compare with previous version

Re-running the test with Gebco NetCDF instead of Geotiff, I still get an error, but the numerical deviations are different:

>       np.testing.assert_array_almost_equal(seafloor_depth,
                                             answ_seafloor_depth,
                                             decimal=2)
E       AssertionError: 
E       Arrays are not almost equal to 2 decimals
E       
E       Mismatched elements: 9 / 9 (100%)
E       Max absolute difference: 127.73378904
E       Max relative difference: 0.13175636
E        x: array([[ 269.1 ,  616.11, 1466.31],
E              [ 101.66,  268.88, 1595.38],
E              [  85.25,  419.49, 1435.14]])
E        y: array([[ 267.75,  606.44, 1398.15],
E              [  99.94,  273.12, 1651.99],
E              [  80.17,  370.65, 1307.41]])

So, to summarize:

there are differences between the two formats
and there are differences between the two years

The situation is a little confusing, to say the least. I may do a bit more digging ...

added 1 commit

abd4526a - fixed test

Compare with previous version

I have fixed the test. I think this can be merged now? I will investigate the issues with differences between the GEBCO NetCDF/GeoTiff data at a later time.