Revisión de 11:32 16 jul 2019

ORCHIDEE's work-flow management for 1 processor.

Climate runs require simulations for long periods of time. Geophysical models require different sources of forcing. Usually one should not and ofently can not prepare boundary conditions for all the period of the run (usually more than 10 years). Instead, models perform long simulations by `chunks' or fractions of the period making use of restart files. For each of these chunks, forcing data has to be prepared.

One could do this manually waiting until the end of the run. It is much more efficient if a work-flow can be generated in order to concatenated the chunks in an automatic way.

Here is presented a very simple work-flow manager for off-line ORCHIDEE simulations on single processor machines.

Files & structure

It is assumed that ORCHIDEE has already been installed into the machine, and user already has downloaded all the required data for the run. On a main folder called [foldOR1proc] (as example) it is expected to contain:

• components: with two additional folders
  • ORtemplate: Folder with an example of ORCHIDEE's run.def; bin/, folder with the compiled binaries: orchideedriver, orchidee_ol, rebuild
  • python: Folder with required work-flow python scripts: change_namelist.py
  • shell: Folder with the work-flow scripts: run_multi-or_1proc.bash, prepare_forcings.bash; configuration file: or_multi-config.txt; rebuild.bash shell to rebuild ORCHIDEE's output files
• forcings: Folder with all the anciallary data and meteorological forcings

or_multi-config.txt

This is the main file for the work-flow and the unique one to be modified by the user. It contains the definition of a series of variables. As an example:

# scratch
scratch=true

# Period of simulation
period=1990-2010

# config
foldOR1proc=/home/administrador/installOR
components=${foldOR1proc}/components
batsHOME=${components}/shell
pyH=${components}/python
binfolder=${components}/ORtemplate/bin
forcingfolder=${foldOR1proc}/forcings
OUTfolder=/home/administrador/installOR/test_multi-run

# Anciliary data 1 file without year (original name of the files with the ancilliary data [origFilen]@[ORfilen])
filestolink='alb_bg_modisopt_2D_ESA_v2.nc@alb_bg.nc:'
filestolink=${filestolink}'cartepente2d_15min.nc@cartepente2d_15min.nc:'
filestolink=${filestolink}'reftemp.nc@reftemp.nc:routing.nc@routing.nc:'
filestolink=${filestolink}'soil_bulk_and_ph.nc@soil_bulk_and_ph.nc:'
filestolink=${filestolink}'soils_param.nc@soils_param.nc'

# Anciliary data 1 file with year (original name of the files with the ancilliary data [origFilen]@[ORfilen], 
  [YYYY] to be changed by the correspondant year)
filestolinkyr='PFTmap_YYYY.nc@PFTmap.nc:woodharvest_YYYY.nc@woodharvest.nc'

# Anciliary data 3 files with year-1,year,year+1 (original name of the files with the ancilliary data 
  [origFilen]@[ORfilen], [YYYY] to be changed by the correspondant year-1, year, yerar+1)
filestolink3yr='cruncep_halfdeg_YYYY.nc@cruncep_halfdeg_YYYY.nc'

# Atmospheric forcing as [ForcingName]@[HeaderForcingFiles] [assuming ${forcingfolder}/ATMOS/{ForcingName}/
  {HeaderForcingFiles}_YYYY.nc]
atmos='CRUNCEP@cruncep_halfdeg'

# Additional parameters for run.def as [paramName1]:[values1];...;[paramNameN]:[valuesN] (! for spaces)
addparams=

• period: period of simulation providing the initial and the ending year in format [iyear]-[eyear]
• pyBIN: path to the binary of python (2.x version is required)
• OUTfolder: path to the place where the simulation will be performed
• foldOR1proc: path to the main folder ot the work-flow manager
• components: path to the components of the work-flow (no need to be changed)
• shellHOME: path to the shell scripts of the work-flow (no need to be changed)
• pyH: path to the python scripts of the work-flow (no need to be changed)
• binfolder: path to the binaries of the work-flow (no need to be changed)
• forcingfolder: path to the forcings of the work-flow (no need to be changed)
• filestolink: ':' separated list of values for the anciliary data that comes from files without year dependency (in format [origFilen]@[ORfilen]; [origFilen], original name of the file; [ORfilen]: expected name by ORCCHIDEE)
• filestolinkyr: ':' separated list of values for the ancillary data that comes from files with year dependency (in format [origFilen]@[ORfilen]; [origFilen], original name of the file; [ORfilen]: expected name by ORCCHIDEE. Name of the files must contain the string chain 'YYYY' that will be automatically changed)
• filestolink3yr: ':' separated list of values for the ancillary data that comes from files with year dependency (in format [origFilen]@[ORfilen]; [origFilen], original name of the file; [ORfilen]: expected name by ORCCHIDEE. Name of the files must contain the string chain 'YYYY' that will be automatically changed as year-1, year, yerar+1)
• atmos Atmospheric forcing as [ForcingName]@[HeaderForcingFiles] [assuming ${forcingfolder}/ATMOS/{ForcingName}/{HeaderForcingFiles}_YYYY.nc]
• addparams: ';' separated list of additional parameters for run.def as [paramName1]:[values1] (using ! for spaces)

One located at the desired output folder <code>[OUTfolder]:

1. copy the file or_multi-config.txt<code>
2. link the main work-flow manager script: <code>run_multi-or_1proc.bash
3. adapt the content of or_multi-config.txt<code> to the experiment
4. run the work-flow (recommending to run it in background as the example)

$ cd [OUTfolder]
$ cp [foldOR1proc]/components/shell/or_multi-config.txt ./
$ ln -s [foldOR1proc]/components/shell/run_multi-or_1proc.bash ./
$ vim or_multi-config.txt 
$ nohup ./run_multi-or_1proc.bash >& run_multi_or_1proc.log &

Along the experiment the work-flow will create in <code>[OUTfolder] a new directory called orout within which will be antother folder with the output of each year of simulation. Work-flow manager looks for the presence of sechiba_rest_out.nc in orout/[year] in order to determine if such chunk should be run.

Inside the folder user will also find:

• run.def: ORCHIDEE's configuration file used to run the chunk
• used_run.def: ORCHIDEE's full configuration (also with the parameters not set in run.def) file used to run the chunk
• out_orchidee_0000: ORCHIDEE's standard output for the simulation