Worksheet 2a: Introduction to using CDO for analysis and NCL for visualisation



In this worksheet sample PRECIS output over the CORDEX Africa domain driven by HadCM3Q ensemble members is compared with observations for validation purposes. The validation of model results by comparison with observed data is essential. It is the measure by which we can assess the quality of the model and it informs appropriate uses of the data. For more information on evaluating and validating model results, see the material from the PRECIS lecture on validation.

Here, we use PRECIS output driven by two different GCM ensemble members. Using data from both experiments will give us two representations of present day climate and two possible climate scenarios. For more details on multimodel approaches see the PRECIS workshop lecture on climate model ensembles.

The following are examples of types of analyses undertaken as part of a model validation. The methods shown are not necessarily the only way to proceed and are intended to demonstrate the use of CDO tools in model validation, and provide a starting point for your own analyses.

The worksheet uses xconv to visualise the data. While this is a useful tool to look quickly at results, it is not suitable for use in reports and papers. NCL visualisation software can produce high quality plots. NCL will be introduced in this worksheet.

Contents


2.1

Introduction to CDO

2.2

Climatological mean calculation

2.3

Mean annual cycle calculation

2.4

Comparing models and observations

2.5

Climatological mean and annual cycle for an ensemble

2.6

Introduction to NCL for visualisation

2.7

Optional question


Note: The data used here has been processed in the same way as Worksheet 1. The 8 point-rim has been removed and it has been converted from PP to netCDF format.







2.1 Introduction to CDO


CDO stands for Climate Data Operators. CDO is a free open source tool set and like PRECIS tools, it is run from the Linux command line.

For further details and documentation seehttps://code.zmaw.de/projects/cdo/wiki.

The datasets used here are daily and monthly data from two PRECIS runs carried out over Africa: one driven by HadCM3Q0 and the other driven by HadCM3Q2.

The observations used for comparison are WATCH WFDEI.

1. ) Find out more information about the monthly precipitation file 1981-2010 of experiment akyjy.

infov writes information about the structure and content of the netCDF file to screen.

% DATA=$LOCALDATA/WS # change this for your computer if necessary
% cd $DATA/monthly
% ls

The runid of the HadCM3Q0 driven run is akyjy and the runid of the HadCM3Q2 driven run is akyuy.

% cd akyjy/05216
% cdo infov akyjya.pm.8110.05216.rr8.nc



2.2 Climatological mean calculation

1 a. ) Calculate the 1980-2010 seasonal mean precipitation field for October-December (OND) from both the HadCM3Q0 (akyjy) and HadCM3Q2 (akyuy) driven PRECIS runs.

timmean calculates the mean over all timesteps in a file. We have combined timmean with selmon and selyear to be selective about the time period (e.g. selecting only OND - months 10,11 and 12.)

Firstly for akyjy

Note: sometimes commands are too long to fit on one line, if a line does not start with %, the command is continued on the next line and you should not press enter until it is complete.

% cd $DATA/monthly
% mkdir climatology # creates a new directory called climatology
% cdo timmean -selmon,10/12 -selyear,1981/2010 akyjy/05216/akyjya.pm.8110.05216.rr8.nc climatology/akyjya.OND.mean.baseline.05216.nc
% xconv -i climatology/akyjya.OND.mean.baseline.05216.nc

Secondly for akyuy

% cdo timmean -selmon,10/12 -selyear,1981/2010 akyuy/05216/akyuya.pm.8110.05216.rr8.nc climatology/akyuya.OND.mean.baseline.05216.nc
% xconv -i climatology/akyuya.OND.mean.baseline.05216.nc

1 b. ) Convert the OND mean fields for both runs and the observations from kg/m2/s to mm/day (multiply by 86400.)

mulc multiplies the field by a constant. Similar operators exist for division, subtraction and addition.

% cdo mulc,86400 climatology/akyjya.OND.mean.baseline.05216.nc climatology/akyjya.OND.mean.baseline.05216.mmday.nc
% cdo mulc,86400 climatology/akyuya.OND.mean.baseline.05216.nc climatology/akyuya.OND.mean.baseline.05216.mmday.nc


% cdo mulc,86400 WATCH_WFDEI/watch.mon.7912.05216.nc WATCH_WFDEI/watch.mon.7912.05216.mmday.nc

2. ) Calculate the 1981-2010 seasonal mean for OND from the WATCH WFDEI observation data.

% cd $DATA/monthly/WATCH_WFDEI
% cdo timmean -selmon,10/12 -selyear,1981/2010 watch.mon.7912.05216.mmday.nc ../climatology/watch.OND.mean.baseline.05216.nc
% xconv -i ../climatology/watch.OND.mean.baseline.05216.nc

Note: WATCH-Forcing-Data-ERA-Interim: WFDEI is a high resolution (0.5 degree) meteorolical forcing dataset applied to ERA-Interim reanalysis data. It provides global 3-hourly and daily precipitation and temperature fluxes from 1979-2012. See http://www.eu-watch.org/data_availability for more information.



2.3 Mean annual cycle calculation

1 a. ) Extract the area around Cairo from the monthly PRECIS precipitation data for both the HadCM3Q0 and HadCM3Q2 driven runs by specifying lon and lat co-ordinates.

sellonlatbox allows you to extract an area from fields by choosing lon1,lon2,lat1,lat2 (W,E,S,N).

% cd $DATA/monthly/akyjy/05216
% cdo sellonlatbox,30.7,31.7,29.5,30.5 akyjya.pm.8110.05216.rr8.nc akyjya.pm.8110.05216.rr8.ext.nc

% cd $DATA/monthly/akyuy/05216
% cdo sellonlatbox,30.7,31.7,29.5,30.5 akyuya.pm.8110.05216.rr8.nc akyuya.pm.8110.05216.rr8.ext.nc

1 b. ) Calculate monthly mean fields for 1981-2010 for each of the twelve months for the Cairo area.

ymonmean computes the mean of all the time steps of multiple years in each month.

% cd - # returns you to the directory you were previously in
% cdo ymonmean -selyear,1981/2010 akyjya.pm.8110.05216.rr8.ext.nc $DATA/monthly/climatology/akyjya.monmean.baseline.05216.rr8.ext.nc
% cd $DATA/monthly/akyuy/05216
% cdo ymonmean -selyear,1981/2010 akyuya.pm.6190.05216.rr8.ext.nc $DATA/monthly/climatology/akyuya.monmean.baseline.05216.rr8.ext.nc

Convert the monthly mean fields to mm/day

% cd $DATA/monthly/climatology
% cdo mulc,86400 akyjya.monmean.baseline.05216.rr8.ext.nc akyjya.monmean.baseline.05216.rr8.ext.mmday.nc
% cdo mulc,86400 akyuya.monmean.baseline.05216.rr8.ext.nc akyuya.monmean.baseline.05216.rr8.ext.mmday.nc

% cdo infov akyjya.monmean.baseline.05216.rr8.ext.mmday.nc
% cdo infov akyuya.monmean.baseline.05216.rr8.ext.mmday.nc







What is the mean precipitation amount in mm/day for the HadCM3Q0 and HadCM3Q2 driven PRECIS runs?

Month

akyjy (mm/day)

akyuy (mm/day)

1 - January

 

 

2 - February

 

 

3 - March

 

 

4 - April

 

 

5 - May

 

 

6 - June

 

 

7 - July

 

 

8 - August

 

 

9 - September

 

 

10 - October

 

 

11 - November

 

 

12 - December

 

 

2. ) Now find monthly means 1981-2010 for observations.

% cd $DATA/monthly/WATCH_WFDEI
% cdo sellonlatbox,30.7,31.7,29.5,30.5 -selyear,1981/2010 watch.mon.7912.05216.nc watch.mon.8110.05216.ext.nc
% cdo ymonmean watch.mon.8110.05216.ext.nc ../climatology/watch.monmean.baseline.05216.ext.nc



























How do the observations compare to the models? In what months are there the greatest differences?

Month

WATCH WFDEI observations (mm/day)

1 - January

 

2 - February

 

3 - March

 

4 - April

 

5 - May

 

6 - June

 

7 - July

 

8 - August

 

9 - September

 

10 - October

 

11 - November

 

12 - December

 



______________________________________________________

______________________________________________________

______________________________________________________


2.4 Comparing models and observations


To fairly compare spatial model and observation fields they must firstly be on the same grid. We will regrid to the coarsest grid and then subtract the model field from the observations. In this case, the observations have the coarsest resolution so we will regrid the model data onto the observation grid.

1 . ) Regrid the multiannual OND mean model fields onto the observations grid.

griddes a command which prints a description of the input field(s) grid.
remapbil this operator remaps all input fields to a new horizontal grid using bilinear interpolation.

% cd $DATA/monthly/climatology
% cdo griddes watch.OND.mean.baseline.05216.nc > mygrid
% cdo remapbil,mygrid akyjya.OND.mean.baseline.05216.mmday.nc akyjya.OND.mean.baseline.05216.mmday.rg.nc
% cdo remapbil,mygrid akyuya.OND.mean.baseline.05216.mmday.nc akyuya.OND.mean.baseline.05216.mmday.rg.nc

% xconv -i akyjya.OND.mean.baseline.05216.mmday.rg.nc

Note: mygrid is used as the target grid for remapping.

2 a. ) Find the difference between the model and observation OND multiannual mean fields.

sub this operator subtracts one field from another, similar operators exist for addition, multiplication and division.

% cdo sub akyjya.OND.mean.baseline.05216.mmday.rg.nc watch.OND.mean.baseline.05216.nc diff.akyjy_watch.OND.baseline.nc
% cdo sub akyuya.OND.mean.baseline.05216.mmday.rg.nc watch.OND.mean.baseline.05216.nc diff.akyjy_watch.OND.baseline.nc
% xconv -i diff.akyuy_watch.OND.baseline.nc

2 b. ) Extract the RCM domain from the difference fields (this improves plotting.)

% cdo sellonlatbox,-25,60,-45,42 diff.akyjy_watch.OND.baseline.nc diff.akyjya_watch.OND.baseline.ext.nc
% cdo sellonlatbox,-25,60,-45,42 diff.akyuy_watch.OND.baseline.nc diff.akyuy_watch.OND.baseline.ext.nc
% xconv -i diff.akyuy_watch.OND.baseline.ext.nc

2.5 Climatological mean and annual cycle for an ensemble

So far data from two models downscaled with PRECIS have been used. In this section we use an ensemble of five models from the HadCM3Q QUMP ensemble (discussed in the model ensemble lecture). The five members of the QUMP ensemble are HadCM3Q0 (akyjy), HadCM3Q2 (akyuy), HadCM3Q9 (akzcy), HadCM3Q13 (akzjb) and HadCM3Q14 (akzja).

Taking an ensemble approach allows us to account for a range of uncertainty in the model projections. Typing the CDO commands into the command line for all six ensemble members would be very time consuming, so we use a script intead.

1 . ) Calculate the OND seasonal mean and annual cycle (for the Cairo area) for 1.5m temperature for all five ensemble members and WATCH WFDEI observations. Also find the difference between model runs and observations.

The script worksheet2.sh contains all the CDO commands needed to calculate the OND mean, annual cycle and difference between models and observations for akyjy, akyuy, akzcy, akxjb, and akzja.

The top of the script needs to be edited to correct the location of DATA before running.

% cd $DATA
% emacs worksheet2.sh &
% ./worksheet2.sh

2 . ) Calculate the OND seasonal mean and annual cycle (for the Cairo area) for precipitation for all five ensemble members and WATCH WFDEI observations. Also find the difference between model runs and observations.

Use the same script (worksheet2.sh) but change stash from 03236 to 05216 to change the script from calculating temperature to precipitation.

% emacs worksheet2.sh &
% ./worksheet2.sh

2.6 Introduction to NCL for visualisation


NCL is a powerful visualisation tool. It is easy to learn and can be scripted. Many examples of scripts and plots are available from http://www.ncl.ucar.edu/


To check if NCL is working simply type ncl on the command line.You should see something like:

Copyright (C) 1995-2011 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.0.0
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ for more details.
ncl 0>


Now type exit

1. ) Run an NCL script to plot the annual cycle of precipitation and temperature for the model and observations.


The script annualcycle.ncl uses the fields generated in section 2.5 to plot the annual cycle of precipitation and temperature. It reads in the NetCDF fields and finds the area mean for each month before plotting. It outputs a .png plot called annualcycle.png. Open the script for editing.

% cd $DATA/ncl_scripts
% emacs annualcycle.ncl & # you can use which ever text editor you prefer

Before running this script two changes need to be made. At the top of the script, change data_dir to be the location of the climatology directory which contains the data to be plotted. Also change plot_location to the location of the ncl_scripts directory.

% ncl annualcycle.ncl
% display annualcycle.png


How does the annual cycle of the six ensemble members compare to observations and each other, what are the differences?

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

2. ) Run an NCL script to plot mean OND precipitation fields for both PRECIS experiments and APHRODITE observations.


The script precip_OND_mean.ncl uses the fields generated in section 2.2 to plot OND mean precip fields. It outputs a .png plot called precip_OND_mean.png, open the script for editing.

% emacs precip_OND_mean.ncl & # you can use which ever text editor you prefer

Before running this script two changes need to be made. At the top of the script, change data_dir to be the location of the climatology directory which contains the data to be plotted. Also change plot_location to the location of the ncl_scripts directory.

% ncl precip_OND_mean.ncl
% display precip_OND_mean.png


How do the two PRECIS runs compare to each other and observations?

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

3. ) Run an NCL script to plot OND model minus observation fields for all five ensemble members. Do this for both precipitation and temperature


The script OND_diff.ncl uses the fields generated in section 2.5 to plot two difference fields. It outputs a .png plot called OND_diff.png, open the script for editing.

% emacs OND_diff.ncl & # you can use which ever text editor you prefer

Before running this script two changes need to be made. At the top of the script, change data_dir to be the location of the climatology directory which contains the data to be plotted. Also change plot_location to the location of the ncl_scripts directory.

% ncl OND_diff.ncl
% display OND_diff.png


How do the five ensemble members compare to observations, where are the biggest temperature/precipitation differences?


________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

2.7 Optional question

If you have time, repeat the analysis of the above worksheet for the season June, July and August e.g. JJA.

(1) How do the seasons compare to each other?

(2) Are the model biases with respect to the observation the same in both seasons?

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________

________________________________________________________