GRIB Processor

ETL pipeline for grib metadata.

Description

The grib processor extracts metadata from grib files and can optionally publish it to RabbitMQ servers.

This package is made for use by the Satellite Data Service at the SSEC.

Limitations

Currently, this package can only process GRIB2 files, GRIB1 files will be ignored.

Installation

Requirements

• Python >= 3.8
• NCEPLIBS-g2c (libg2c)
  • This is the C library used to read GRIB files.

If the lib folder with libg2c is in a "common" installation path (/usr/local, /sw, /opt, /opt/local, /opt/homebrew, or /usr) then you can simply install the package from the GitLab package registry.

python3 -m pip install grib-processor --upgrade --index-url https://gitlab.ssec.wisc.edu/api/v4/projects/2732/packages/pypi/simple

If lib folder with libg2c is on the system, but not in one of the previously listed paths, you can manually tell grib_processer where it is through the G2C_DIR environment variable. E.g. libg2c is in /home/user/my_libs/lib/, install grib_processer as so:

G2C_DIR=/home/user/my_libs python3 -m pip install grib-processor --upgrade --index-url https://gitlab.ssec.wisc.edu/api/v4/projects/2732/packages/pypi/simple

If the system doesn't have the libg2c binary at all, you can install it from source, or, more easily, with conda.

conda create -n my_env -y python=3.12 nceplibs-g2c -c conda-forge
conda run -n my_env python3 -m pip install grib-processor --upgrade --index-url https://gitlab.ssec.wisc.edu/api/v4/projects/2732/packages/pypi/simple
conda activate my_env

Usage

After installation, the command grib_processer will be added to your path. use grib_processer --help for more information about how to use the command.

The package also comes with a python interface to parse GRIB metadata.

import grib_processer.grib as grib

# If you would like to include XCD data.
grib.load_xcd_models()

grib_path = '/path/to/my/grib_file.grib'

for data in grib.itergrib(grib_path):
  print(data)

Author

Created by Max Drexler.

License

This software can be freely distributed and/or modified without consent of the original author. For more information, see the LICENSE file.