Skip to content
Snippets Groups Projects
Commit 05297dc2 authored by Alan De Smet's avatar Alan De Smet
Browse files

Import of ../wgrib2.tgz.v3.0.0

parent f095c91b
No related branches found
No related tags found
No related merge requests found
Showing
with 329 additions and 46 deletions
......@@ -43,8 +43,8 @@ Default makefile options
USE_AEC=1
To compile without netcdf, set USE_NETCDF3=0 and USE_NETCDF4=0
To compile without netcdf v3, set USE_NETCDF3=1 and USE_NETCDF4=0
To compile without netcdf v4, set USE_NETCDF3=0 and USE_NETCDF4=1
To compile with netcdf v3, set USE_NETCDF3=1 and USE_NETCDF4=0
To compile with netcdf v4, set USE_NETCDF3=0 and USE_NETCDF4=1
For netcdf4, the netcdf4 and hdf5 libraries are usually not included
in the wgrib2.tgz file to save space. The makefile suggests
......
See _README.clang
Using the DWD's ICON forecast grib files with wgrib2 updated 12/2019, 4/2020
The DWD is making global forecasts using the ICON model. This model uses a triangular
mesh, and the forecast quantities are valid for the center of the triangles. The DWD
opendata server is distributing data in grib format for the forecast values from
the center of the triangles. This note shows how to process the grib data using
wgrib2.
Basics that DWD may change:
https://opendata.dwd.de/weather/nwp/icon/grib/HH
HH = 00, 06, 12 or 18
Step 1. Download the CLAT and CLON file
CLAT=latitude of the center of the triangles
CLON=longitude of the center of the triangles
https://opendata.dwd.de/weather/nwp/icon/grib/00/clat/icon_global_icosahedral_time-invariant_YYYYMMDDHH_CLAT.grib2.bz2
https://opendata.dwd.de/weather/nwp/icon/grib/00/clon/icon_global_icosahedral_time-invariant_YYYYMMDDHH_CLON.grib2.bz2
Step 2: Download some forecast files
Example
https://opendata.dwd.de/weather/nwp/icon/grib/00/t_2m/icon_global_icosahedral_single-level_YYYYMMDDHH_000_T_2M.grib2.bz2
https://opendata.dwd.de/weather/nwp/icon/grib/00/t_2m/icon_global_icosahedral_single-level_YYYYMMDDHH_001_T_2M.grib2.bz2
..
https://opendata.dwd.de/weather/nwp/icon/grib/00/t_2m/icon_global_icosahedral_single-level_YYYYMMDDHH_180_T_2M.grib2.bz2
Step 3: Uncompress the data
bunzip2:
Step 4: Combining the files
Bash:
cat icon_global_icosahedral_time-invariant_YYYYMMDDHH_CLAT.grib2 \
icon_global_icosahedral_time-invariant_YYYYMMDDHH_CLON.grib2 \
icon_global_icosahedral_single-level_YYYYMMDDHH_006_TMAX_2M.grib2 >icon.grb
Windows:
copy /b icon_global_icosahedral_time-invariant_YYYYMMDDHH_CLAT.grib2 +
icon_global_icosahedral_time-invariant_YYYYMMDDHH_CLON.grib2 +
icon_global_icosahedral_single-level_YYYYMMDDHH_006_TMAX_2M.grib2 icon.grb
(all of above on one line)
Contents of icon.grb
$ wgrib2 icon.grb
1:0:d=2019040900:GEOLON:surface:anl:
2:5898409:d=2019040900:GEOLAT:surface:anl:
3:11796818:d=2019040900:TMP:2 m above ground:0-360 min max fcst:
Comment:
Regridding takes a long time for the first field because wgrib2 searches
each grid point to find the nearest neighbor. The rest of the fields
is much faster because wgrib2 retains a list of the nearest neighbors.
So processing is faster if all the fields that need regridding are put
into one file. (The unix cat command works for grib files.) This
slow first field behavior also works for the -lon option. The
nearest neighbor search is faster when using multiple cores and the
OpenMP version of wgrib2.
Example 1: Obtaining values for (10E, 20N) and (10W, 30S)
wgrib2 v2.0.9 (in development)
v2.0.9 adds -else, -elseif and -endif
v2.0.9 updates -grid_def to use GEOLAT and GEOLON
$ wgrib2 icon.grb -if "^(1|2):" -grid_def -else -s -lon 10 20 -lon 15 -30 -endif
1:0
2:5898409
3:11796818:d=2019040900:TMP:2 m above ground:0-6 hour max fcst::lon=9.968750,lat=20.012680,
val=296.588:lon=15.078125,lat=-30.069351,val=290.953
wgrib2 v2.0.6 - v2.0.8 (earlier versions of wgrib2 had a bug in -grid_def)
$ wgrib2 icon.grb \
-if ":GEOLAT:" -set center 7 -set_var NLAT -fi \
-if ":GEOLON:" -set center 7 -set_var ELON -fi \
-grid_def -s \
-not_if "^(1|2):" -lon 10 20 -lon 15 -30 -fi
1:0:d=2019040900:ELON:surface:anl:
2:5898409:d=2019040900:NLAT:surface:anl:
3:11796818:d=2019040900:TMP:2 m above ground:0-360 min max fcst::lon=9.968750,lat=20.012680,val=296.588:
lon=15.078125,lat=-30.069351,val=290.953
Example 2: a 1x1 degree global grid by nearest neighbor interpolation
wgrib2 v2.0.9 (in development)
$ wgrib2 icon.grb -if "^(1|2):" -grid_def -else -s -lola 0:360:1 -90:181:1 1x1.grb grib -endif
1:0
2:5898409
3:11796818:d=2019040900:TMP:2 m above ground:0-6 hour max fcst:
wgrib2 v2.0.6 - v2.0.8 (earlier versions of wgrib2 had a bug in -grid_def)
$ wgrib2 icon.grb \
-if ":GEOLAT:" -set center 7 -set_var NLAT -fi \
-if ":GEOLON:" -set center 7 -set_var ELON -fi \
-grid_def -s \
-not_if "^(1|2):" -lola 0:360:1 -90:181:1 1x1.grb grib
1:0:d=2019040900:ELON:local level type 1 0:anl:
2:5898409:d=2019040900:NLAT:local level type 1 0:anl:
3:11796818:d=2019040900:TMP:local level type 103 2:0-6 hour max fcst:
Example 3: Making a netcdf file
The raw ICON grib files do not have latitude and longitude information. By prepending
the CLON and CLAT files, the file has the longitude and latitude information. However,
the wgrib2 cannot make a netcdf file because the data are not on a lat-lon grid. One
could update the netcdf converter to output the ICON data on a trianglular mesh, but
how many visualization codes could read that netcdf file and make a plot?
The suggested method to make a netcdf file using wgrib2 is by making a lat-lon grib
file. See example 2. Once you have made the lat-lon file, you can make a netcdf
file using the grib2->netcdf utility of your choice.
The conversion from the trianglar mesh to a lat-lon grid is slow because a linear search
is used to find the nearest neighbor. The conversion can be made faster by using more cores
and setting the appropriate number of cores to use (export OMP_NUM_THREADS=n). This is why
you want more cores!
Compiling with clang (C language frontend for LLVM)
Clang: 9/2019
tested with Ubuntu 12.04 and 14.xx
Clang, gfortran/dragonegg:
export CC=clang
export FC=gfortran -fplugin=dragonegg
The original wgrib2 support for Clang was for Ubuntu 12.04, using
gfortran with dragonegg. Unfortunately dragonegg requires gcc 4.x
and gcc is already up to gcc 10. As a result, support for
clang+gfortran-dragonegg has ended.
note: at time of testing, clang did not have OpenMP support
Clang, Flang and AOCC:
When I last checked (2018), flang was available from github with compile
directions. Not simple to install.
AMD released AOCC (AMD Optimizing C Compiler) suite v2.0 which is based
on clang/flang with optional optimizations for Zen and Zen 2. This
wgrib2 makefile has been tested with AOCC and a Ryzen 3600 on a Ubuntu
18.04LTS.
According to AMD release documentation, AOCC has been tested by AMD on
REL 7.4, SLES 12 sp3, and Ubuntu 18.04LTS. AOCC requires linux with
glibc 2.17+. AOCC is freely available, works on Intel and AMD x86 cpus,
and the AMD modifications get sent to LLVM for inclusion in to clang/flang.
IMHO, it is an easy to install clang/flang. (Sooner or later, flang will
be easy to install.)
https://developer.amd.com/amd-aocc/
To compile with AOCC
export CC=clang
export FC=flang
make
To compile for Zen2 with AOCC wgrib2 v2.0.9
export CC=clang
export FC=flang
(add more info here)
make
9/2020 Notes on installing cygwin on Windows 10
go to cygwin.com
run setup-x86-64.exe (need system admin)
(64-bits is needed for wgrib2 to process files > 2GB)
(I installed from the internet)
(I installed all recommended packages)
(I installed icons in both locations)
The rest is run from cygwin's bash (run Cygwin Terminal icon)
See if you have gcc, gfortran, g++, and make installed.
gcc
(if you see gcc: command not found, you need to download gcc and gfortran)
(go to cygwin.com, run setup-x86-64.exe)
(select packages: view:full search: gcc-fortran, I selected the latest version)
gfortran
(if you see gfortran: command not found, you need to download gfortran, see above)
(go to cygwin.com, run setup-x86-64.exe)
(select packages: view:full search: gcc-fortran, I selected the latest version)
make
(if you see make: command not found, you need to download make, if nothing happens,
type control-c to quit the make program)
(go to cygwin.com, run setup-x86-64.exe)
(select packages: view:full search: make, I selected latest version)
g++
(if you see g++: command not found, you need to download gcc-g++)
(go to cygwin.com, run setup-x86-64.exe)
(select packages: view:full search: gcc-g++, I selected latest version)
Get wgrib2 source code and install.
get wgrib2.tgz from ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/wgrib2.tgz
If you use a browser to download wgrib2.tgz, you can find the download in
/cygdrive/c/Users/YourName/Downloads/wgrib2.tgz
(copy to home directory using bash)
mv /cygdrive/c/Users/YourName/Downloads/wgrib2.tgz ~
Of course "/cygdrive/c" could be replaced by "/cygdrive/d" if your
windows home directory is on "d:".
The rest is run from cygwin's bash (run Cygwin Terminal icon)
cd
(cd to your home directory)
tar -xzvf wgrib2.tgz
(expand wgrib2.tgz, now you should see the grib2 directory)
cd grib2
Create a cygwin makefile (makefile.cygwin)
sed 's/\\#/#/' <makefile >makefile.cygwin
(this creates makefile.cygwin which is compatible with the cygwin make)
Compile
export CC=gcc
export FC=gfortran
(you may want to alter the "makefile.cygwin" to make a custom configuration)
make
(this should make wgrib2/wgrib2.exe)
tested 9/24/2020: window 10, latest cygwin
......@@ -2,7 +2,7 @@ Intel ICC: updated 2/2017
prior to wgrib2 v2.0.6
The makefile doesn't compile Jasper 1.900.1 "correctly". Testing
The makefile did not compile Jasper 1.900.1 "correctly". Testing
fails with writing of jpeg2000 encoded files. The portland
C compiler fails with the exact same symptoms. May be
a Jasper source code problem rather than a compiler problem.
......@@ -28,18 +28,18 @@ Bash:
make lib makes lib/*.a lib/*.mod for wgrib2api
WARNING WARNING - Compiling wgrib2 on WINDOWS
WARNING WARNING - Compiling wgrib2 on WINDOWS (including using icc)
Wgrib2 follows the standards (no Microsoft Visual C is not a standard). Wgrib2
uses fseek(FILE *stream, long offset, int whence) for random access. Most 64-bit
systems define long to be a 64 bit integer. As a result, wgrib2 on will support
systems define long to be a 64 bit integer. As a result, wgrib2 will support
large files.
Unfortunately long is usually defined as 32-bits on 64-bit Windows compilers including the
windows version of icc. This is entirely appropriate as "640K ought to be enough for anybody".
Icc/windows defines long to be 32 bits long, and wgrib2 compiled with icc is limited to 2GB
files. You could use fseeko(FILE *stream, off_t offset, int whence) to access large files.
files. One could use fseeko(FILE *stream, off_t offset, int whence) to access large files.
Unfortunately this comforms to standards SUSv2 and POSIX.1-2001 which are not usually supported
by Windows C compilers. (I could find no suggestion that it was supported by icc/windows.)
For Windows, I suggest that you use the 64-bit Cgywin gcc/gfortran compilers. They are the
......
Compiling wgrib2 with netcdf-4
9/2013:
8/2020:
NetCDF4 support has been in shambles for many years. The good news,
Redhat with its vintage compilers has been working fine. If you
are using the Intel compilers, it compiles and probably works.
(I have heard no complaints but number of users is extremely small.)
I had no problems with Ubuntu 12.04LTS until it lost support. I
had to move to something newer like Ubuntu 18.04LTS. Now for the
bad news. HDF5 would not compile with the default gnu compilers and
needed something from the GNU 4.xx series. Finally hdf5 10.6.0 was
released with support for not-so-old gnu compilers. I could now
compile hdf5 on my unbuntu 18.04 system. Being able to provide
NetCDF4 support was looking better.
Unfortunately hdf5 10.6.0 breaks the -import_netcdf option on my
Redhat system with the vintage compilers. Maybe I made a mistake
with my netcdf calls, the only error message is that hdf5 had
a problem. It's not helpful. I had convert my personal copy back
to hdf 10.4.0 to get some scripts working.
8/2020: There is not support for netcdf4. You may be able to
get netcdf4 to work with gnu 4.xx compilers. You may be able to
use the system installed netcdf4 libraries. However, you are on
your own in modifying the makefiles to already installed netcdf4
libraries. You are on your own in verifying that wgrib2 is working
correctly with your system libraries.
It has been a long time since I compiled wgrib2 with netcdf4.
To get the makefile working with the new netcdf4 and hdf5 src
code required a couple of days of working.
Configuration:
......
OpenMP revised 7.2016
OpenMP revised 7.2016, 1.2020
OpenMP is a shared-memory parallel-programming API. As the grid dimensions get larger,
the need for parallel computing becomes more important. With OpenMP, wgrib2 will use
multiple threads, typically one thread per core. For the typical PC, wgrib2 will run
on the number of cores that are on the CPU chip which is typically 2, 3, 4, 6 or 8.
(Yes, I've owned machines with 2, 3, 4, 6 and 8 cores.) The multi-core speedup is only
significant when the grids have several million grid points.
significant when the grids have several million grid points. (Update: 2019 brings
12 and 16 core PCs.)
You can control the number of cores used by the environment variable OMP_NUM_THREADS
and the -ncpu option. The latter overrides the former.
Status: working
Status: working, requires OpenMP v3.1
Tested: AIX: xlc
Ubuntu 12.04+
Redhat 4: gcc did not officially support OpenMP, had problems
Redhat 5+: gcc, open64
Clang 9: tested AOCC version
Hints:
complex-packing reading is parallelized when bitmaps are not used (-g2clib 0/1)
complex-packing reading is partly parallelized if bitmaps are used (-g2clib 0/1)
complex-packing reading is parallelized when bitmaps are not used
complex-packing reading is partly parallelized if bitmaps are used
complex-packing writing is partly parallelized.
simple-packing is parallelized
jpeg2000, png, AEC are not parallelized because they depends on an external libraries
jpeg2000, png, AEC are not parallelized because they depends on external libraries
geolocation is parallelized except when using Proj4.
Running multiple copies of wgrib2 can be done along with OpenMP.
Nodes with 24+ cores should have environment variable OMP_NUM_THREADS set to
......
PGI: pgcc
Preliminary testing using pgcc 11-10.0
Preliminary testing using community version 2019.
The makefile doesn't compile Jasper "correctly". Testing
fails with writing of jpeg2000 encoded files. The Intel
C compiler fails with the exact same symtoms. May be
a Jasper source code problem rather than a compiler problem.
Compiling by pgcc was turned off in the makefile
No go.
Shared Library
This is experimental. Do not enable unless you know what
you are doing.
......@@ -40,7 +40,7 @@ int wgrib2_list_cmd() {
fprintf(stderr,"no wgrib2 cmds\n");
return 0;
}
fprintf(stderr,"wgrib2 ", cmd[i]);
fprintf(stderr,"wgrib2 ");
for (i = 1; i < n_cmds; i++) {
fprintf(stderr,"%s ", cmds[i]);
}
......
......@@ -10,3 +10,4 @@
grb2_wrt, grb2_inq, added optional parameter debug
grb_inq: added copy=file
1/2018: added real, parameter :: grb2_UNDEFINED
11/2019: added optional parameter use_ncep_table=logical_value
#
# v1.1 7.2020 WNE: support make -j N makefile, simplify clean
#
ARFLAGS=-ruv
all: wgrib2lowapi.o wgrib2api.o fort_wgrib2.o libwgrib2_api.a
......@@ -5,22 +9,17 @@ all: wgrib2lowapi.o wgrib2api.o fort_wgrib2.o libwgrib2_api.a
.c.o:
$(CC) ${CPPFLAGS} ${CFLAGS} -c $<
wgrib2api.o: wgrib2lowapi.o wgrib2lowapi.mod wgrib2api.f90
wgrib2api.o: wgrib2lowapi.mod wgrib2api.f90
$(FC) ${FFLAGS} -c wgrib2api.f90
wgrib2lowapi.o: wgrib2lowapi.f90
$(FC) ${FFLAGS} -c wgrib2lowapi.f90
wgrib2lowapi.mod: wgrib2lowapi.f90
wgrib2lowapi.mod: wgrib2lowapi.f90 wgrib2lowapi.o
$(FC) ${FFLAGS} -c wgrib2lowapi.f90
libwgrib2_api.a: wgrib2lowapi.o wgrib2api.o fort_wgrib2.o
ar ${ARFLAGS} libwgrib2_api.a *.o
clean:
touch fort_wgrib2.o ; rm fort_wgrib2.o
touch wgrib2api.o ; rm wgrib2api.o
touch wgrib2api.mod ; rm wgrib2api.mod
touch wgrib2lowapi.o ; rm wgrib2lowapi.o
touch wgrib2lowapi.mod ; rm wgrib2lowapi.mod
touch junk.o junk.mod junk.a ; rm *.o *.mod *.a
......@@ -214,15 +214,23 @@ contains
! grb2_mk_inv
! writes the match inventory to a file
!
integer function grb2_mk_inv(grbfile, invfile)
integer function grb2_mk_inv(grbfile, invfile, use_ncep_table)
use wgrib2lowapi
implicit none
character (len=*), intent(in):: grbfile, invfile
logical, optional, intent(in):: use_ncep_table
character (len=300) :: cmd(10)
integer :: n, i
n = 0
if (present(use_ncep_table)) then
if (use_ncep_table) then
i = add_line(cmd,n,'-set')
i = add_line(cmd,n,'center')
i = add_line(cmd,n,'7')
endif
endif
i = add_line(cmd,n,grbfile)
i = add_line(cmd,n,'-rewind_init')
i = add_line(cmd,n,grbfile)
......@@ -258,7 +266,7 @@ contains
! i = add_line(cmd,n,'-grib')
! i = add_line(cmd,n,outgrbfile)
! i = add_line(cmd,n,'-inv')
!! ** this is linux/unix
! ** this is linux/unix
! i = add_line(cmd,n,'/dev/null')
! ** this is for windows
! i = add_line(cmd,n,'NUL')
......
......@@ -70,14 +70,14 @@ module wgrib2lowapi
USE ISO_C_BINDING
integer (C_INT), value :: n
integer (C_SIZE_T), value :: size_buffer
character (kind=c_char) :: buffer(*)
character (kind=c_signed_char) :: buffer(*)
end function wgrib2_get_mem_buffer
integer (C_INT) function wgrib2_set_mem_buffer(buffer, size_buffer, n) bind(C)
USE ISO_C_BINDING
integer (C_INT), value :: n
integer (C_SIZE_T), value :: size_buffer
character (kind=c_char) :: buffer(*)
character (kind=c_signed_char) :: buffer(*)
end function wgrib2_set_mem_buffer
integer (C_INT) function wgrib2_free_file(string) bind(C)
......
......@@ -15,6 +15,7 @@ include make.filelist
all: ${LIBRARY}
${LIBRARY}: $(OBJ_FILES)
touch ${LIBRARY} ; rm ${LIBRARY}
$(AR) $(ARFLAGS) $@ $(OBJ_FILES)
clean:
......
#define USE_SPECTRAL 0
#define USE_SPECTRAL 1
......@@ -205,7 +205,15 @@
RLON2=FLOAT(IGDTMPL(16))/FLOAT(ISCALE)
ISCAN=MOD(IGDTMPL(19)/128,2)
HI=(-1.)**ISCAN
DLON=HI*(MOD(HI*(RLON2-RLON1)-1+3600,360.)+1)/(IM-1)
! WNE DLON=HI*(MOD(HI*(RLON2-RLON1)-1+3600,360.)+1)/(IM-1)
if (HI.eq.1) then
if (RLON2 <= RLON1) RLON2 = RLON2 + 360.0
DLON = (RLON2-RLON1)/(IM-1)
else
if (RLON1 <= RLON2) RLON1 = RLON1 + 360.0
DLON = (RLON1-RLON2)/(IM-1)
endif
DLAT=(RLAT2-RLAT1)/(JM-1)
XMIN=0
XMAX=IM+1
......
......@@ -35,6 +35,7 @@ FSRC_FILES = \
polatev4.f90 \
polatev6.f90 \
polfixs.f90 \
polfixs_nn.f90 \
polfixv.f90
# The complete list of source files
......
......@@ -52,6 +52,7 @@
! VERSION OF GDSWZD.
! 2015-07-13 GAYNO CONVERT TO GRIB 2. REPLACE GRIB 1 KGDS ARRAYS
! WITH GRIB 2 GRID DEFINITION TEMPLATE ARRAYS.
! 2019-05-24 GAYNO POLE FIX
!
! USAGE: CALL POLATES2(IPOPT,IGDTNUMI,IGDTMPLI,IGDTLENI, &
! IGDTNUMO,IGDTMPLO,IGDTLENO, &
......@@ -121,7 +122,7 @@
! GDSWZD GRID DESCRIPTION SECTION WIZARD
! IJKGDS0 SET UP PARAMETERS FOR IJKGDS1
! IJKGDS1 RETURN FIELD POSITION FOR A GIVEN GRID POINT
! POLFIXS MAKE MULTIPLE POLE SCALAR VALUES CONSISTENT
! POLFIXS_NN MAKE MULTIPLE POLE SCALAR VALUES CONSISTENT
! CHECK_GRIDS2 DETERMINE IF INPUT OR OUTPUT GRIDS HAVE CHANGED
! BETWEEN CALLS TO THIS ROUTINE.
!
......@@ -275,7 +276,7 @@
IBO(K)=IBI(K)
IF(.NOT.ALL(LO(1:NO,K))) IBO(K)=1
ENDDO
IF(IGDTNUMO.EQ.0) CALL POLFIXS(NO,MO,KM,RLAT,IBO,LO,GO)
IF(IGDTNUMO.EQ.0) CALL POLFIXS_NN(NO,MO,KM,RLAT,IBO,LO,GO)
! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ELSE
IF(IRET.EQ.0) IRET=IRETX
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment