A Grid for Multidimensional and Multivariate Spatial Representation and Data Processing

Tobias Stål; Anya M. Reading

doi:10.5334/jors.287

A Grid for Multidimensional and Multivariate Spatial Representation and Data Processing

Journal of Open Research Software

Volume 8 (2020): Issue 1

By: Tobias Stål and Anya M. Reading

Open Access

|Jan 2020

Abstract

Researchers use 2D and 3D spatial models of multivariate data of differing resolutions and formats. It can be challenging to work with multiple datasets, and it is time consuming to set up a robust, performant grid to handle such spatial models. We share ‘agrid’, a Python module which provides a framework for containing multidimensional data and functionality to work with those data. The module provides methods for defining the grid, data import, visualisation, processing capability and export. To facilitate reproducibility, the grid can point to original data sources and provides support for structured metadata. The module is written in an intelligible high level programming language, and uses well documented libraries as numpy, xarray, dask and rasterio.

Funding statement: This research was supported under Australian Research Council’s Special Research Initiative for Antarctic Gateway Partnership (Project ID SR140300001).

References

An, M, Wiens, D A, Zhao, Y, Feng, M, Nyblade, A A, Kanao, M, Li, Y, Maggi, A and Lévêque, J-J 2015 S-velocity model and inferred Moho topography beneath the Antarctic Plate from Rayleigh waves. Journal of Geophysical Research, Solid Earth, 120: 2007–2010. DOI: 10.1002/2014JB011332
Open DOI Search in Google Scholar Back to article
Bergstra, J, Breuleux, O, Bastien, F, Lamblin, P, Pascanu, R, Desjardins, G, Turian, J, Warde-Farley, D and Bengio, Y 2010 Theano: A cpu and gpu math compiler in python. In Proc. 9th Python in Science Conf, 1.
Search in Google Scholar Back to article
Beyreuther, M, Barsch, R, Krischer, L, Megies, T, Behr, Y and Wassermann, J 2010 ObsPy: A Python toolbox for seismology. Seismological Research Letters, 81(3): 530–533. ISSN 0895-0695. DOI: 10.1785/gssrl.81.3.530
Open DOI Search in Google Scholar Back to article
de la Varga, M, Schaaf, A and Wellmann, F 2019 Gempy 1.0: Open-source stochastic geological modeling and inversion. Geoscientific Model Development. DOI: 10.5194/gmd-2018-61
Open DOI Search in Google Scholar Back to article
Fomel, S 2013 Tour with Madagascar, Revisiting s. e. p. and SCons. Journal of Open research software.
Search in Google Scholar Back to article
Fomel, S and Hennenfent, G 2007 Computational experiments using SCons, reproducible. ICASSP, 2007, 1257–1260. DOI: 10.1109/ICASSP.2007.367305
Open DOI Search in Google Scholar Back to article
Fretwell, P, Pritchard, H D, Vaughan, D G, Bamber, J L N, Barrand, E, Bell, R, Bianchi, C, Bingham, R G, Blankenship, D D, Casassa, G, Catania, G, Callens, D, Conway, H, Cook, A J, Corr, H F J, Damaske, D, Damm, V, Ferraccioli, F, Forsberg, R, Fujita, S, Gim, Y, Gogineni, P, Griggs, J A, Hindmarsh, R C A, Holmlund, P, Holt, J W, Jacobel, R W, Jenkins, A, Jokat, W, Jordan, T, King, E C, Kohler, J, Krabill, W, Riger-Kusk, M, Langley, K A, Leitchenkov, G, Leuschen, C, Luyendyk, B P, Matsuoka, K, Mouginot, J, Nitsche, F O, Nogi, Y, Nost, O A, Popov, S V, Rignot, E, Rippin, D M, Rivera, A, Roberts, J, Ross, N M, Siegert, J, Smith, A M, Steinhage, D, Studinger, M, Sun, B, Tinto, B K, Welch, B C, Wilson, D, Young, D A, Xiangbin, C and Zirizzotti, A 2013 Bedmap2: Improved ice bed, surface and thickness datasets for Antarctica. The Cryosphere, 7(1): 375–393. DOI: 10.5194/tcd-6-4305-2012
Open DOI Search in Google Scholar Back to article
Gillies, S 2013 The Shapely user manual. URL https://pypi.org/project/Shapely/.
Search in Google Scholar Back to article
Gillies, S 2014 The Fiona user manual. URL https://fiona.readthedocs.io/en/latest/manual.html.
Search in Google Scholar Back to article
Gillies, S 2018 Rasterio: Geospatial raster I/O for Python programmers. URL https://github.com/mapbox/rasterio.
Search in Google Scholar Back to article
Hinsen, K 2011 A data and code model for reproducible research and executable papers. Procedia Computer Science, 4: 579–588. ISSN 1877-0509. DOI: 10.1016/j.procs.2011.04.061
Open DOI Search in Google Scholar Back to article
Hoyer, S and Hamman, J 2017 xarray: N-D labeled arrays and datasets in Python. Journal of Open Research Software, 5(1). DOI: 10.5334/jors.148
Open DOI Search in Google Scholar Back to article
Hoyer, S, Fitzgerald, C, Hamman, J, Kleeman, A, Kluyver, T, Roos, M, Helmus, J J, Markel, M, Cable, P, Maussion, F, Miles, A, Kanmae, T, Wolfram, P, Sinclair, S, Bovy, B, Ebrevdo, R, Guedes, R, Abernathey, R, Filipe, S, Hill, N, Richards, A, Lee, N, Koldunov, M, Maciekswat, G, Gerard, J, Babuschkin, I, Deil, C, Welch, E and Hilboll, A 2019 Xarray:. http://xarray.pydata.org/en/stable/.
Search in Google Scholar Back to article
Hunter, J D May 2007 Matplotlib: A 2D graphics environment. Computing In Science & Engineering, 9(3): 90–95. ISSN 1521-9615. DOI: 10.1109/MCSE.2007.55
Open DOI Search in Google Scholar Back to article
Jones, E, Oliphant, T, Peterson, P, et al. 2001 SciPy: Open Source scientific tools for Python. URL http://www.scipy.org/. [Online; accessed 13 Dec 2018].
Search in Google Scholar Back to article
Jordahl, K 2014 GeoPandas: Python tools for geographic data. URL https://github.com/geopandas/geopandas.
Search in Google Scholar Back to article
Knight, S 2010 Scons user guide. Python Software Foundation.
Search in Google Scholar Back to article
Lewis, A, Lymburner, L, Purss, M B J, Brooke, B, Evans, B, Ip, A, Dekker, A G, Irons, J R, Minchin, S, Mueller, N, et al. 2016 Rapid, high-resolution detection of environmental change over continental scales from satellite data-the Earth Observation Data Cube. International Journal of Digital Earth, 9(1): 106–111. DOI: 10.1080/17538947.2015.1111952
Open DOI Search in Google Scholar Back to article
Lewis, A, Oliver, S, Lymburner, L, Evans, B, Wyborn, L, Mueller, N, Raevksi, G, Hooke, J, Woodcock, R, Sixsmith, J, Wu, W, Tan, P, Li, F, Killough, B, Minchin, S, Roberts, D, Ayers, D, Bala, B, Dwyer, J, Dekker, A, Dhu, T, Hicks, A, Ip, A, Purss, M, Richards, C, Sagar, S, Trenham, C, Wang, P and Wang, L-W December 2017 The Australian geoscience data cube-Foundations and lessons learned. Remote Sensing of Environment, 202: 276–292. DOI: 10.1016/j.rse.2017.03.015
Open DOI Search in Google Scholar Back to article
McKinney, W 2015 Pandas: A Python data analysis library. URL http://pandas.pydata.org.
Search in Google Scholar Back to article
Megies, T, Beyreuther, M, Barsch, R, Krischer, L and Wassermann, J 2011 ObsPy – what can it do for data centers and observatories? Annals of Geophysics, 54(1): 47–58. ISSN 1593-5213. DOI: 10.4401/ag-4838
Open DOI Search in Google Scholar Back to article
Morse, P, Reading, A M and Stål, T 2019 Well-posed geoscientific visualization through interactive and color mapping. Frontiers in Earth Science. DOI: 10.3389/feart.2019.00274
Open DOI Search in Google Scholar Back to article
Oliphant, T E 2006 A guide to NumPy, 1. Trelgol Publishing USA.
Search in Google Scholar Back to article
Pedregosa, F, Varoquaux, G, Gramfort, A, Michel, V, Thirion, B, Grisel, O, Blondel, M, Prettenhofer, P, Weiss, R, Dubourg, V, et al. 2011 Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct): 2825–2830.
Search in Google Scholar Back to article
QGIS. 2015 QGIS geographic information system. Open Source Geospatial Foundation Project. By development team and community.
Search in Google Scholar Back to article
Ramachandran, P and Varoquaux, G 2011 Mayavi: 3D visualization of scientific data. Computing in Science and Engineering, 13(2): 40–51. ISSN 1521-9615. DOI: 10.1109/MCSE.2011.35
Open DOI Search in Google Scholar Back to article
Rew, R and Davis, G 1990 NetCDF: An Interface for Scientific Data Access. IEEE Computer Graphics and Applications, 10(4): 76–82. ISSN 0272-1716. DOI: 10.1109/38.56302
Open DOI Search in Google Scholar Back to article
Robitaille, T P, Tollerud, E J, Greenfield, P, Droettboom, M, Bray, E, Aldcroft, T, Davis, M, Ginsburg, A, Price-Whelan, A M, Kerzendorf, W E, et al. 2013 Astropy: A community python package for astronomy. Astronomy & Astrophysics, 558: A33. DOI: 10.1051/0004-6361/201322068
Open DOI Search in Google Scholar Back to article
Rocklin, M 2015 Dask: Parallel computation with blocked algorithms and task scheduling. In Proceedings of the 14th Python in Science Conference, number 130–136. Citeseer. DOI: 10.25080/Majora-7b98e3ed-013
Open DOI Search in Google Scholar Back to article
Roth, G, Matsuoka, K, Skoglund, A, Melvær, Y and Tronstad, S February 2018 Quantarctica: A unique, open, standalone GIS package for Antarctic research and education. URL http://quantarctica.npolar.no.
Search in Google Scholar Back to article
Schöberl, J, Martin, K and Lorensen, B 2006 Kitware, The Visualization Toolkit. Technical report. (VTK).
Search in Google Scholar Back to article
Stål, T January 2020 Agrid code and tests v. 0.4.0. Zenodo. DOI: 10.5281/zenodo.2553966
Open DOI Search in Google Scholar Back to article
Stål, T, Reading, A M, Halpin, J A and Whittaker, J M August 2019 A multivariate approach for mapping lithospheric domain boundaries in east Antarctica. Geophysical Research Letters. DOI: 10.1029/2019GL083453
Open DOI Search in Google Scholar Back to article
Stål, T, Reading, A M, Halpin, J and Whittaker, J 2020 The Antarctic crust and upper mantle: A 3D model and framework for interdisciplinary research. Manuscript in preparation.
Search in Google Scholar Back to article
Thompson, J F, Soni, B K and Weatherill, N P 1998 Handbook of grid generation. CRC press.
Search in Google Scholar Back to article
Uieda, L October 2018 Verde: Processing and gridding spatial data using green’s functions. Journal of Open Source Software, 3(30): 957. DOI: 10.21105/joss.00957
Open DOI Search in Google Scholar Back to article
van der Walt, S, Colbert, S C and Varoquaux, G 2011 The NumPy array: A structure for efficient numerical computation. Computing in Science & Engineering, 13(2): 22–30. DOI: 10.1109/MCSE.2011.37
Open DOI Search in Google Scholar Back to article
Warmerdam, F and GDAL/O. G. R. contributors. software Library, GDAL/OGR Geospatial Data Abstraction. Foundation, Open Source Geospatial, 2018. URL http://gdal.org.
Search in Google Scholar Back to article
Wilson, G, Aruliah, D A, Brown, C T, Chue Hong, N P, Davis, M, Guy, R T, Haddock, S H D, Huff, K D, Mitchell, I M, Plumbley, M D and Others 2014 Best practices for scientific computing. PLoS biology, 12(1): e1001745. DOI: 10.1371/journal.pbio.1001745
Open DOI Search in Google Scholar Back to article
Zwally, H J, Giovinetto, M B, Beckley, M A and Saba, J L 2012 Antarctic and Greenland drainage systems, gsfc cryospheric sciences laboratory. Available at icesat4.gsfc.nasa.gov/cryo_data/ant_grn_drainage_systems.php. Accessed March 1, 2015.
Search in Google Scholar Back to article
Met Office 2019 Cartopy: A cartographic python library with a matplotlib interface. Exeter, Devon. URL http://scitools.org.uk/cartopy.
Search in Google Scholar Back to article