Cleaner, crisper rotation tracks

We are really happy when public safety agencies use the imagery from http://ondemand.nssl.noaa.gov/ to show the impact of the recent tornadoes in Illinois.  Hey, that’s our stuff, we want to shout. It reminds us of why we do what we do.

But there’s a lot of noise on those accumulation products, noise that can be removed by the use of Multiple Hypothesis Tracking (MHT).  We couldn’t do MHT on-demand or in real-time because it is so slow, but just a few weeks ago, we figured out a way to do it faster with not much of a tradeoff in noise removal.

We repeated the analysis of the Illinois outbreak using the faster method and boy, is it faster! We can process an hour of data in 5 minutes!  (Here are the cleaned-up rotation tracks for the Illinois tornadoes. It is a KML file, so view it in Google Earth.) MHT will be implemented on the ondemand website in a few days.  So, the next time you see folks sharing Rotation Tracks images, they will be cleaner and crisper.

Rotation Tracks today

Rotation Tracks today

Rotation Tracks with MHT (very, very slow)

Rotation Tracks with MHT (very, very slow)

Optimized MHT

Optimized MHT

 

w2accumulator applies MHT faster

One of the best ways to improve the quality of the Rotation Tracks products is to apply spatial QC using hysteresis and temporal QC using Multiple Hypothesis Tracking.

Unfortunately, this used to be quite slow. An hour of azimuthal shear data covering the CONUS could take as much as two hours to process. Therefore, it was used only in research studies and off-line, but not to produce the post-event rotation tracks that you can download from http://ondemand.nssl.noaa.gov/

w2accumulator’s -Q option now supports two vastly more efficient optimizations. You can specify that the number of hypothesis is 1 (meaning to only keep the best track, and not bother about second-best, third-best, etc.) or that the algorithm should retain all potential tracks (specifying -1 for number of hypotheses).  These are the most likely values that you will want to specify and with these, the algorithm runs 20x faster.  Yup, you can now process an hour of data in about 6 minutes.

Method -Q option CPU Time (microseconds)
No QC ” “ 46
5 best blob:0.002:0.005:2:azshear,mht:1:2:1800:5:5 2046
Only best blob:0.002:0.005:2:azshear,mht:1:2:1800:5:1 132
All reasonable blob:0.002:0.005:2:azshear,mht:1:2:1800:5:-1 138

You used to have only the first two options for -Q available. Now, you have two more, and these two “special” values are highly optimized.

What’s the impact of these options? (Open the images in different tabs in your browser and switch between them so that you can see the differences between the last two images more readily)

w2image-input_MergedAzShear_0-2kmAGL-20130520-215833-1

Azimuthal shear field without QC

Keeping only best hypothesis

Keeping only best hypothesis

Keep all reasonable hypotheses

Keep all reasonable hypotheses


For more details about MHT-QC and its application to rotation tracks products, please see these scientific articles:

M. Miller, V. Lakshmanan, and T. Smith, “An automated method for depicting mesocyclone paths and intensities,” Wea. Forecasting, vol. 28, pp. 570-585, 2013.

V. Lakshmanan, M. Miller, and T. Smith, “Quality control of accumulated fields by applying spatial and temporal constraints,” J. Atmos. Ocean. Tech., vol. 30, pp. 745-757, 2013.

w2qcnndp now handles Sun Spikes and Electronic Interference

w2qcnndp is a WDSS-II algorithm that employs polarimetric moments to do quality-control of weather radar reflectivity data.  Single-pol QC (via w2qcnn) has lots of problems distinguishing bioscatter from light precipitation, but polarimetric moments (variance of Zdr, especially) help w2qcnndp outperform w2qcnn in terms of removing bioscatter. If you are using w2qcnn on US weather radar data from after the polarimetric upgrade, you should definitely start using w2qcnndp instead.

w2qcnndp used to have lots of problems with electronic interference and with sun spikes.  Recently, we added two modules to w2qcnndp (they are turned on by default; use -m “-sunstrobe -electronicinterference” to turn these off).  Here are a couple of examples to show the impact of these modules.

First, electronic interference:

Raw data from KEWX

Raw data from KEWX

QCed KEWX data

QCed KEWX data

w2qcnndp used to have problems with sun spikes (sun strobes) that were connected to valid echoes, but this has been improved:

Raw data from KTHX

Raw data from KTHX

QCed KHTX data

QCed KHTX data

For more details about w2qcnndp, please refer to the following two scientific articles:

V. Lakshmanan, C. Karstens, J. Krause, and L. Tang, “Quality control of weather radar data using polarimetric variables,” J. Atm. Ocea. Tech.,

V. Lakshmanan, C. Karstens, K. Elmore, S. Berkseth, and J. Krause, “Which polarimetric variables are important for weather/no-weather discrimination?,” J. Atmos. Ocean. Tech.

What’s this blog about?

Users of WDSS-II range from students and researchers who download WDSS-II from the wdssii.org website, engineers at private companies that license WDSS-II from the University to forecasters at government agencies and meteorological departments.

A question that I often get from users of WDSS-II is: “What’s new? What have you done in the past X months?”.  This blog is an attempt to answer that. We will add examples of changes we have made to small parts of the system, the sort of changes that can go unnoticed in a system where the names of the tools and algorithms tend to remain the same but the tools get more and more powerful and the algorithms more and more skillful.

Another purpose of this blog is to point WDSS-II users to tips and tricks. If you have a cool way to use or customize a WDSS-II application, please consider contributing a guest post.

Questions about use of the software or products, however, should be posted on the WDSS-II forum.