Getting RUC/RAP data into WDSS-II

Written by Bryan Burlingame [Part 1 of How to Create Reflectivity -10C]

To start, the first steps require the requesting, and downloading of large datasets. This process can be time consuming in both waiting for the NCDC order to come through, and also in downloading the data.

RAP/RUC Data:

Download RAP/RUC from:

http://nomads.ncdc.noaa.gov/data.php?name=access#hires_weather_datasets

After downloading the needed RAP data, organize the data into directories so that only the times you want processed are in a directory. The WDSS-II program reads in whole directories worth of data at a time. So to minimize the processing time, and also the size of your data, only keep the data need for the time of interest. Utilities such as wgrib or wgrib2, available from NCEP, are particularly useful for this task.

Converting RAP/RUC Data to WDSS-II NetCDF

There are two different processes for grib1 data and for grib2 data. Typically, RAP data will be in grib2 format and RUC data will be in grib1. In order for WDSS-II to read in RAP/RUC data, it needs to be in the WDSS-II NetCDF format. Below are the steps needed to convert both GRIB 1 & 2 to the required format.

GRIB 1

  1. The command used to convert GRIB 1 to WDSS-II NetCDF is gribToNetcdf. Here is an example of the command:

gribToNetcdf -i /path/to/grb1 -o /creates/new/directory/NetCDF -k -a -1 -p 2013 -t “50 -111” -b “27 -88” -s “0.03 0.03” –verbose

    1. i = input directory of the GRIB 1 data.
    2. o = where you want the generated NetCDF files to go (creates directory if it DNE).
    3. k = tells command NOT to delete the original GRIB 1 files after processing.
    4. a = if on, will read the existing files in the directory on startup.
    5. -1 = if on, will execute just once (not real-time).
    6. p = pattern recognition (if all files start with rap_130… you would use –p rap_130)
    7. t = the northwest corner of the domain you wish to use (Should be same as below with w2merger).
    8. b = the southeast corner of the domain you wish to use (Should be same as below with w2merger).
    9. s = the horizontal spacing of the domain
  1. Run “replaceIndex” to make a .xml file from .fam directory. The code_index.fam will be the same as the outputDir specified in the griddataingest.xml file you copied over and edited.

replaceIndex -i /path/to/code_index.fam -o /path/to/code_index.xml

  1. i = input directory (path to code_index.fam/) of now WDSS-II NetCDF files (same as output directory used above in gribToNetcdf ).
  2. o = same directory as above, however the output is code_index.xml rather than .fam
  1. Run “nse” to create various products that RAP/RUC does not have, but which WDSS-II needs to interpolate to the various Isotherms in the RAP/RUC data.

nse -i xml:/path/to/code_index.xml -o NSE/ -3 –verbose

GRIB 2

  1. There are two options for converting GRIB 2 to the WDSS-II NetCDF. Option 1 (the recommended option) is to continue onto step 2 below. The other option is to convert the GRIB 2 data to GRIB 1 using the command: cnvgrib –g21 <inputfile> <outputfile>
    1. cnvgrib readme: http://www.nco.ncep.noaa.gov/pmb/docs/grib2/download/cnvgrib.readme
    2. After converting to GRIB 1, revert back to the previous section for converting GRIB 1 to WDSS-II NetCDF.
  1. For GRIB 2, use the WDSS-II Java package to convert GRIB 2 to WDSS-II NetCDF. In steps (a.) and (b.), make sure the proper paths are declared.
    1. export JAVA_HOME=”/usr/bin”
    2. export WDSSII_INSTALL_DIR=”/path/to/wdssii/wdssiijava”
  1. Next you need to copy the file wdssiijava/example/griddataingest.xml into a directory of your choice, and rename it rap.xml or ruc.xml (any name .xml works). Next, edit the newly copied .xml file. At the top of the file change the inputDir to where your data are, and outputDir to where you want it to go. Also change the filenamePatterns to match some part of your input files (if all files start with rap_130… you would use rap_130 as your filenamePattern). You also will want to uncomment the listVariables entry and set it to true.

 

  1. Now run the following command:

w2java.sh org.wdssii.ncingest.GridDatasetIngest /path/to/copied/griddatasetingest.xml

  1. This will create your configuration file, varList.xml. This file contains all of the meteorological variables in your RAP/RUC data. By editing this file you can change which variables are processed in the new NetCDF files.
  1. Next, edit griddatasetingest.xml that you copied and renamed so to set listVariables to false.
  1. Now run the following command: (Yes, same as above)

w2java.sh org.wdssii.ncingest.GridDatasetIngest /path/to/copied/griddatasetingest.xml

  1. Run “replaceIndex” to make a .xml file from .fam directory. The code_index.fam will be the same as the outputDir specified in the griddataingest.xml file you copied over and edited.

replaceIndex -i /path/to/code_index.fam -o /path/to/code_index.xml

    1. i = input directory (path to code_index.fam/) of now WDSS-II NetCDF files (same directory as outputDir mentioned above).
    2. o = same directory as above, however the output is code_index.xml rather than .fam
  1. Run “nse” to create various products that RAP/RUC does not have, but which WDSS-II needs to interpolate to the various Isotherms in the RAP/RUC data.

nse -i /path/to/code_index.xml -o /path/to/NSE/ -3 –verbose