Reproducing the single-system graphical products for additional variables

Reproducing the single-system graphical products for additional variables#

Introduction#

This Jupyter Notebook shows how the products shown in the C3S seasonal graphical products can be calculated from data in the Climate Data Store (CDS), and plotted. The C3S seasonal graphical and data products are described in this documentation page.

This example code can be used as the basis for creating graphical products which are not part of the C3S suite. In this example we look at monthly mean daily minimum temperature real-time forecasts (for one system, ECMWF System 51), which is a variable available in the CDS dataset but not the graphical products. A ‘tercile summary’ is computed from tercile probabilities. An ensemble mean anomaly is also calculated and plotted with significance testing applied. This will be used in a further example to create multi-system combinations.

Configuration
CDS API requests
Load real-time forecast and hindcast data
Category based probability forecast product
Deterministic forecast product

Note

Some of the python code uses a set of utility functions available in the GitHub respository which hosts these Notebooks.

Configuration#

Here we set which variable(s) will be downloaded, for which C3S seasonal system. We also set which real-time forecast date we will create a real-time forecast for, and which hindcast period to use (which will be used to calculate the terciles and anomalies).

Note that the URL and KEY need to be filled in with the details from your CDS account, and the cdsapi package needs to be installed. Ideally, a .cdsapirc file should be created, to avoid the possibility of exposing credentials when sharing Notebooks. CDS API requests can now also be made using earthkit, as shown in this example.

Import required modules and configure CDS API key and client.

# import required modules
import cdsapi
import os
import xarray as xr
import numpy as np
import pandas as pd
import sys
from pathlib import Path
# Date and time related libraries
from dateutil.relativedelta import relativedelta
from calendar import monthrange
# Plotting libraries
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import cartopy.feature as cfeature


# Add parent directory to path so we can import the utils package
sys.path.insert(0, str(Path.cwd().parent))
from utils.tools import getForecastsystemDetails

# config to avoid issues saving to netcdf if needed
os.environ['HDF5_USE_FILE_LOCKING'] = 'FALSE'

URL = 'https://cds.climate.copernicus.eu/api'
KEY = '' # INSERT CDS KEY HERE IF NEEDED
c = cdsapi.Client(url=URL, key=KEY) # if a .cdsapirc file is used, url and key can be omitted
c = cdsapi.Client() 

CDS API requests#

Here we request the desired hindcast and real-time forecast data in GRIB format using the CDS API, and save it within a ‘data’ folder in the current working directory, organized by originating centre and real-time forecast system. For this example, the CDS API keywords used are:

Format: Grib
Variable: minimum_2m_temperature_in_the_last_24_hours set via ‘cds_var_name’
Originating centre: ECMWF set via ‘origin’
System: 51 this refers to SEAS5 system 51, set via ‘system’
Product type: Monthly mean all ensemble members will be retrieved
Year: 1993 to 2016 for the hindcast 2024 for the real-time forecast, set via ‘hc_years’ and ‘fc_year’
Month: 02 February, set via ‘st_mon’
Leadtime month: 1 to 6 all lead months available, February to July in this case

Set some parameters for the request, and to be used later during plotting.

origin = 'ecmwf'  # originating centre label for the CDS request
system = '51'  # forecast system identifier for the CDS request
prov = f'{origin}.s{system}'

# Get system details from configuration files in utils directory using format ukmo.s604
system_details = getForecastsystemDetails(prov)
lagged = system_details.get('system_details',{}).get('isLagged',False)
print(system_details)

# define some other parameters for the data request
fc_yr = '2024' # the real-time forecast year
st_mon = '02' # the real-time forecast start month

# select a variable, tmin in this case
var = 'tmin'
var_str = 'daily mean T2m' # for plotting
var_cds_name = '2m_temperature'


lt_mons = [1, 2, 3, 4, 5, 6]  # cover all lead months when plotting month by month
hc_start = '1993'
hc_end = '2016'
hc_years = '_'.join([hc_start, hc_end])  # string to print the hindcast period, and label the file

# data path in cwd 
data_path = os.sep.join(['data', origin, system])
try:
   os.makedirs(data_path)
except FileExistsError:
   # directory already exists
   pass

{'origin_details': {'MARS_origin': 'ecmf', 'label': 'ECMWF'}, 'system_details': {'name': 'SEAS5 (C3Sv5.1)', 'in_c3s': '20221101', 'ens_size': {'products': {'hc': 25}}}}

Make the hindcast request and save the data.

#HINDCAST REQUEST - 1993-2016
c.retrieve(
    'seasonal-monthly-single-levels',
    {
        'format': 'grib',
        'variable': var_cds_name,
        'originating_centre': origin,
        'system': system,
        'product_type': 'monthly_mean',
        'year': [
            '1993', '1994', '1995',
            '1996', '1997', '1998',
            '1999', '2000', '2001',
            '2002', '2003', '2004',
            '2005', '2006', '2007',
            '2008', '2009', '2010',
            '2011', '2012', '2013',
            '2014', '2015', '2016',
        ],
        'month': [st_mon],
        'leadtime_month': lt_mons
    },
    data_path + f'/{var}_mm_{origin}_{system}_{hc_years}_{st_mon}.grib'

Make the real-time forecast request and save the data.

#FORECAST REQUEST
c.retrieve(
    'seasonal-monthly-single-levels',
    {
        'format': 'grib',
        'variable': [var_cds_name],
        'originating_centre': origin,
        'system': system,
        'product_type': 'monthly_mean',
        'year': [fc_yr],
        'month': [st_mon],
        'leadtime_month': lt_mons
    },
    data_path + f'/{var}_mm_{origin}_{system}_{fc_yr}_{st_mon}.grib' 
)