The grib processor extracts [metadata](/docs/payload.md) from grib files and can optionally publish it to RabbitMQ servers.
The main use-case of this package is for the SSEC with the main purpose being to source grib files by watching a directory and publish all new metadata.
This package is made for use by the Satellite Data Service at the SSEC.
### Limitations
...
...
@@ -17,96 +17,46 @@ Currently, this package can only process GRIB2 files, GRIB1 files will be ignore
If `libg2c` is in a "common" installation path (`/usr/local`, `/sw`, `/opt`, `/opt/local`, `/opt/homebrew`, or `/usr`) then you can simply install the package from the GitLab package registry.
If `libg2c` is on the system, but not in one of the previously listed paths, you can manually tell grib_processer where it is through the `G2C_DIR` environment variable. E.g. `libg2c` is in `/home/user/my_libs/lib/`, install grib_processer as so:
This will show you how to set up a virtual environment to run the processor with. It assumes that you've installed the grib processor and are in the `grib_rmq_stream` directory.
There are a couple of ways to get your environment working based on your environment manager. See [anaconda](#anaconda) if you want to use anaconda or [virtualenv](#virtualenv) if you just want to use virutalenv.
If the system doesn't have the `libg2c` binary at all, you can install it from [source](https://github.com/NOAA-EMC/NCEPLIBS-g2c), or, more easily, with conda.
```bash
python3 -m pip install-r requirements.txt
```
If you get an error during installation about not finding a library, set the environment variable `G2C_DIR` to be the path to the folder containing the 'lib' folder for [NCEPLIBS-g2c](https://github.com/NOAA-EMC/NCEPLIBS-g2c). e.g.
If `libg2c.so` is under `/home/user/.conda/envs/lib-env/lib/`, you could install using:
Once you've [setup](#setup) an environment, you can start the event processor simply by calling `./grib_processor {dir}`. Where {dir} is the directory to watch for grib files. See [configuration](#configuration) for more information about how to change the behavior of the processor.
After installation, the command `grib_processer` will be added to your path. use `grib_processer --help` for more information about how to use the command.
### Configuration
The package also comes with a python interface to parse GRIB metadata.
There are two ways to configure the grib_processer script: through a .env file and through the CLI. The .env file contains secret information that you might not want in the command name, and the CLI has all other configuration.
```python3
import grib_processer.grib as grib
#### .env File
# If you would like to include XCD data.
grib.load_xcd_models()
The .env file can have two keys: `RMQ_USER` and `RMQ_PASS` which specify the RabbitMQ username and password respectively.
grib_path = '/path/to/my/grib_file.grib'
#### CLI Options
Use `./grib_processor --help` for a full list of command line arguments and what they do.