I am really looking forward to a time when scientific data analysis is less of a constant fuckaround and fight with technical bullshit. I'd *really* like
- #netCDF natively supporting complex numbers
- #Python #xarray and #pandas to natively support physical units (#pint is great on its own but the integrations leave a LOT to be desired)
- #Jupyter notebooks to suck less (crashes, glitches, widget plots not saved statically, an effing BUILTIN formatter, etc.)
- proper data pipeline systems
...
Satisfaction comes from wherever you can get it. An issue I filed on Github in 2021 and which users gradually piled onto to say, oh yeah, we need that? It's resolved!
New release of my software coming in the next couple days.
𝗖𝗙 𝗠𝗲𝘁𝗮𝗱𝗮𝘁𝗮 𝗖𝗼𝗻𝘃𝗲𝗻𝘁𝗶𝗼𝗻𝘀 support #OpenScience by automating processes via metadata in #NetCDF and #Zarr files. They establish a unified language for the #weather, #climate, #ocean, and #EO community. https://buff.ly/rGPuoku
#remotesensing #EarthObs
how to tell if #netcdf was compiled with s3 support ? (on linux, just installed from RPM)
even an example netcdf url that should work from a bucket would be helpful
GSPy - A New Toolbox And Data Standard For Geophysical Datasets
--
https://doi.org/10.3389/feart.2022.907614 <-- shared paper
--
https://doi.org/10.5066/P9XNQVGQ | https://code.usgs.gov/g3sc/gspy <-- shared code repository
--
[an older paper, but code is in active and ongoing development/evolution]
#GIS #spatial #mapping #geophysics #geophysical #NetCDF #datatypes #code #opensource #library #dataformats #standardisation #standardization #openstandard #portable #metadata #Python #package #GSPy #methods #workflows #xarray #CRS #opendata #architecture #toolbox
Excited to share that I'm now a member of the CF Conventions Committee! 
CF (Climate and Forecast) conventions standardise how scientific data and metadata are stored in NetCDF files, ensuring consistency and machine readability. Looking forward to contributing and collaborating with the community to shape its future! 
Excited to share my new FREE video tutorial on creating CF-NetCDF files using R! 
In this video, you'll learn to:
• Ensure compliance with CF and ACDD conventions
• Work with 1D, 2D, and 3D data
• Manage irregular grids and moving instruments
Watch it here: https://youtu.be/IZDygRjfMIg
Plus, check out the accompanying Jupyter book for code and explanations: https://nordatanet.github.io/NetCDF_in_R_from_beginner_to_pro/04_creating_a_cfnetcdf_file.html
I’m pleased to share that my NetCDF in Python and R tutorials are now available on NorDataNet.
Access them here:
Python: https://www.nordatanet.no/NetCDF-in-Python-from-beginner-to-pro
R: https://www.nordatanet.no/NetCDF-in-R-from-beginner-to-pro
These tutorials cover everything from basic data extraction to creating your own NetCDF files, ensuring they meet CF and ACDD standards for FAIR data.
NorDataNet enhances access to environmental datasets, supporting long-term preservation and interoperability. 
Using Mesh Data In QGIS [including accessing NetCDF data]
--
https://courses.gisopencourseware.org/course/view.php?id=72 <-- shared tutorial
--
https://youtu.be/OBi4wuxLiAE?si=_5DCAcsy8qjhsA-k <-- tutorial as a video
--
“Mesh data represents an unstructured grid that can include temporal and other components. It consists of vertices, edges, and faces that form a spatial structure in 2D or 3D space. Each vertex can store multiple datasets, which can also have a temporal dimension, making mesh data highly versatile for various applications such as meteorology, hydrodynamics, and environmental modeling…
This course is designed to equip you with the knowledge and skills needed to effectively work with mesh data within the QGIS environment…"
#QGIS #GIS #mesh #datavisualisation #NetCDF #climate #animation #openaccess #onlinelearning #learning #spatial #mapping #tutorial
Heading to the 2024 CF Workshop in Norrköping, Sweden! 
CF conventions are vital for ensuring data adheres to the FAIR principles. While NetCDF files are great, CF standards make data truly interoperable across disciplines.
I'll be sharing ideas on introducing CF to new disciplines like biology and geology, helping them unlock the full potential of their data. Excited to learn how CF conventions evolve and meet the community!
Want to work with NetCDF files in Python? My tutorial series covers everything from opening and plotting NetCDF data to creating CF-compliant files for FAIR data publication.
Whether you're new to NetCDF or looking to enhance your skills, I've got you covered! Check it out: https://lhmarsden.github.io/NetCDF_in_Python_from_beginner_to_pro
Topics include:
Extracting data 
Plotting 
Creating CF-compliant files 
Granularity 
CF & ACDD
Suggestions? Let me know! #Python #DataScience #NetCDF #xarray #FAIRData #ClimateData
New Video Alert!
I’ve just released a tutorial on how to convert NetCDF data to CSV or XLSX using R. Whether you’re dealing with 1D, 2D, or 3D variables, this video will guide you through the process step-by-step.
Watch here: https://youtu.be/9-EDaRQ8Aps
Day six of the sixth eruption near #Svartsengi and the town of #Grindavík This eruption is the largest one yet. Here is a 4-day series of #Sentinel5p satellite data from the #TROPOMI instrument showing the mean amount of SO₂ in a vertical column of atmosphere (Dobson units). With time this will turn into sulfate particles forming volcanic smog.
This level 3 data is made available by The German Aerospace Center as #NetCDF files and can easily be opend in #QGIS and visualized with MDAL library.
@roadskater
1) #netCDF is so much better than the individual formats thousands of scientists have made up over the decades. But I really wish people would use the metadata fields consistently (or at all).
2) Have you discovered IrfanView? It’s mostly just a viewer, but it handles every image type and many grid types that I’ve thrown at it. It is great for checking whether the netCDF shows what it’s supposed to.
The person who pinged me about this won't look too good dealing with the data product developers on his end if he keeps arguing for standards that, e.g., mean my software can read and plot the data. But forget moi, CF rules!
Otherwise. Ooof. My bad.
Meanwhile, other software packages apparently can plot this stuff? I guess they have multiple coders, and funding to pay them all?
Otherwise, crud. The data grid mapping issue is a pain, but I figured that out. But… the data are in a structure, which… the library doesn't handle well (cough, gag) and disaster ensues.
There could be a career in sci software dev here. If one had the funding.
Spent some time the last few days examining whether I need to blow up and rewrite code in order to plot a satellite data product. Seemed from perusing a very necessary library API that it wouldn't need much work. On re-read, maybe more work. Sigh, now realizing that the API itself would need refactoring. Which, 1) their developers might say no, or 2) worse, they might say yes but they have SO many other demands.
The user who pinged me about this will probably be less than surprised. F!
A Gentle Introduction to GDAL Part 8 - Reading Scientific Data Formats
--
https://medium.com/@robsimmon/a-gentle-introduction-to-gdal-part-8-reading-scientific-data-formats-1a1f70d5388c <-- shared technical article / tutorial
--
“Among its many well-known capabilities, GDAL has a hidden superpower — the ability to read scientific data formats like Hierarchical Data Format (HDF), Network Common Data Form (NetCDF), and Gridded Binary (GRIB). Many essential climate and satellite datasets created by the likes of NASA, NOAA, the World Meteorological Organization (WMO), and the European Space Agency (ESA) are stored and distributed in one of these formats. They contain records of everything from global temperatures to land cover to ocean salinity. Unfortunately, many people who’d be interested in using these data don’t even know they exist…”
#GIS #spatial #mapping #remotesensing #earth #global #gdal #opensource #opendata #tutorial #learning #onlinelearning #introduction #scientificdata #HDF #NetCDF #GRIB #NASA #NOAA #WMO #ESA
A new release of the Panoply data visualization app (v. 5.5.0) has been posted. A bunch of refactoring which hopefully the user will never notice, so the biggest apparent change this release is the ability to save plots in SVG format.
Greg Cocks 
