MWRpy processing

In this tutorial MWRpy products are generated from raw data, including quality control and visualization. This example utilizes files taken from the ACTRIS site Hyytiala:

• RPG-HATPRO microwave radiometer:

  • Brightness temperatures: 230406.BRT

  • Housekeeping data: 230406.HKD

  • Elevation scans: 230406.BLB

  • Weather station: 230406.MET

  • Infrared radiometer: 230406.IRT

Note

.BRT and .HKD files are mandatory in MWRpy for processing

Level 1c

First we convert RPG-HATPRO microwave radiometer (MWR) binary files, including brightness temperature (TB) and housekeeping data (*.BRT, *.HKD), into a Level 1c netCDF file. Data from optional elevation scans (*.BLB, *.BLS), weather station (*.MET) and infrared radiometer (*.IRT) are combined in this process and the following quality flags are derived:

• Bit 1: missing_tb

• Bit 2: tb_below_threshold

• Bit 3: tb_above_threshold

• Bit 4: spectral_consistency_above_threshold

• Bit 5: receiver_sanity_failed

• Bit 6: rain_detected

• Bit 7: sun_moon_in_beam

Quality flags are stored as bits and Bit 1-3 include checks for missing brightness temperature values and their valid range (2.7 - 330 K). The spectral consistency flag (Bit 4) compares measured and retrieved TB. For this flag, it is expected to have the corresponding RPG retrieval coefficient file (SPC*.RET) in /mwrpy/site_config/{site}/coefficients/. Data from HKD files are used to determine the stability of the receiver components (Bit 5). The sensor from the attached weather station detects rain for quality Bit 6 and the sun and moon orbits are calculated and compared to the measurement geometry to detect potential interferences (Bit 7). A quality flag status variable contains information whether the flag is active.

import os
from mwrpy.level1.write_lev1_nc import lev1_to_nc

SITE = "hyytiala"
PACKAGE_DIR = os.getcwd()
DATA_DIR = f"{PACKAGE_DIR}/tests/data/{SITE}"

hatpro_raw = lev1_to_nc(
    "1C01",
    DATA_DIR,
    site=SITE,
    output_file="mwr_1c.nc",
)

The data format of the generated mwr_1c.nc file, including metadata information and variable names, is compliant with the data structure and naming convention developed in the EUMETNET Profiling Programme E-PROFILE.

Variables such as brightness temperature can be plotted from the newly generated file.

import os
from mwrpy.plots.generate_plots import generate_figure
PACKAGE_DIR = os.getcwd()
generate_figure('mwr_1c.nc', ['tb'], save_path=f"{PACKAGE_DIR}/")

Level 2 Single Pointing

Based on the Level 1c netCDF file mwr_1c.nc, MWR single pointing data are extracted and product specific retrieval coefficients are applied to generate the Level 2 single pointing product:

from mwrpy.level2.lev2_collocated import generate_lev2_single
hatpro_prod = generate_lev2_single("hyytiala", "mwr_1c.nc", "mwr-single.nc")

Variables such as integrated water vapor (IWV) can be plotted from the newly generated file.

import os
from mwrpy.plots.generate_plots import generate_figure
PACKAGE_DIR = os.getcwd()
generate_figure('mwr-single.nc', ['iwv'], save_path=f"{PACKAGE_DIR}/")

Level 2 Multiple Pointing

Based on the Level 1c file, MWR multiple pointing data (elevation scans) are extracted and product specific retrieval coefficients are applied to generate the Level 2 multiple pointing product:

from mwrpy.level2.lev2_collocated import generate_lev2_multi
hatpro_prod = generate_lev2_multi("hyytiala", "mwr_1c.nc", "mwr-multi.nc")

Variables such as temperature profiles can be plotted from the newly generated file.

import os
from mwrpy.plots.generate_plots import generate_figure
PACKAGE_DIR = os.getcwd()
generate_figure('mwr-multi.nc', ['temperature'], save_path=f"{PACKAGE_DIR}/")

Command line usage

After defining the instrument type and site specific configuration files (including input/output data paths) in mwrpy/site_config/, MWRpy can also be run using the command line tool mwrpy/cli.py:

mwrpy/cli.py [-h] -s SITE [-d YYYY-MM-DD] [--start YYYY-MM-DD]
                       [--stop YYYY-MM-DD] [-p ...] [{process,plot}]

Arguments

Short	Long	Default	Description
-h	–help		Show help and exit.
-s	–site		Site to process data from, e.g, hyytiala. Required.
-d	–date		Single date to be processed. Alternatively, –start and –stop can be defined.
	–start	current day - 1	Starting date.
	–stop	current day	Stopping date.
-p	–products	1C01, single, multi	Processed products, e.g, 1C01, 2I02, 2P03, single, see Data Types below.

Commands

Command	Description
process	Process data and generate plots (default).
plot	Only generate plots.

Data Types

Level 1

• 1B01: MWR brightness temperatures from .BRT and .BLB/.BLS files + retrieved spectrum

• 1B11: IR brightness temperatures from .IRT files

• 1B21: Weather station data from .MET files

• 1C01: Combined data type with time corresponding to 1B01

Level 2

• 2I01: Liquid water path (LWP)

• 2I02: Integrated water vapor (IWV)

• 2P01: Temperature profiles from single-pointing observations

• 2P02: Temperature profiles from multiple-pointing observations

• 2P03: Absolute humidity profiles

• 2P04: Relative humidity profiles (derived from 2P01/2P02 + 2P03)

• 2P07: Potential temperature (derived from 2P01/2P02 + 2P03)

• 2P08: Equivalent potential temperature (derived from 2P01/2P02 + 2P03)

• single: Single pointing data product (including 2I01, 2I02, 2P01, 2P03)

• multi: Multiple pointing data product (including 2P02, 2P04, 2P07, 2P08)