CLI Commands

initialize_adcirc

initialize_adcirc creates JSON Configurations from initial parameters of a model run (model start and end times, forcing files, storm track ID, etc.). These JSON files provide a portable encapsulation of the entire model run, including model configurations in fort.15 and NEMS couplings. The files are read by generate_adcirc to generate the actual model configuration; generate_adcirc must be run again every time the JSON files change to keep the configuration up to date.

/home/docs/checkouts/readthedocs.org/user_builds/coupledmodeldriver/envs/latest/lib/python3.10/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.2-CAPI-1.17.2) is incompatible with the GEOS version PyGEOS was compiled with (3.10.4-CAPI-1.16.2). Conversions between both will be slow.
  warnings.warn(
usage: initialize_adcirc [-h] --platform PLATFORM --mesh-directory
                         MESH_DIRECTORY --modeled-start-time
                         MODELED_START_TIME --modeled-duration
                         MODELED_DURATION --modeled-timestep MODELED_TIMESTEP
                         [--nems-interval NEMS_INTERVAL]
                         [--modulefile MODULEFILE] [--forcings FORCINGS]
                         [--adcirc-executable ADCIRC_EXECUTABLE]
                         [--adcprep-executable ADCPREP_EXECUTABLE]
                         [--aswip-executable ASWIP_EXECUTABLE]
                         [--adcirc-processors ADCIRC_PROCESSORS]
                         [--job-duration JOB_DURATION]
                         [--output-directory OUTPUT_DIRECTORY]
                         [--skip-existing] [--absolute-paths] [--verbose]

options:
  -h, --help            show this help message and exit
  --platform PLATFORM   HPC platform for which to configure
  --mesh-directory MESH_DIRECTORY
                        path to input mesh (`fort.13`, `fort.14`)
  --modeled-start-time MODELED_START_TIME
                        start time within the modeled system
  --modeled-duration MODELED_DURATION
                        duration within the modeled system
  --modeled-timestep MODELED_TIMESTEP
                        time interval within the modeled system
  --nems-interval NEMS_INTERVAL
                        main loop interval of NEMS run
  --modulefile MODULEFILE
                        path to modulefile to source before model execution`
  --forcings FORCINGS   comma-separated list of forcings to configure, from
                        ['tidal', 'atmesh', 'besttrack', 'owi', 'ww3data']
  --adcirc-executable ADCIRC_EXECUTABLE
                        filename of compiled `adcirc` or `NEMS.x`
  --adcprep-executable ADCPREP_EXECUTABLE
                        filename of compiled `adcprep` (mesh decomposition)
  --aswip-executable ASWIP_EXECUTABLE
                        filename of compiled `aswip` (preprocessing of best
                        track / ATCF file)
  --adcirc-processors ADCIRC_PROCESSORS
                        numbers of processors to assign for ADCIRC
  --job-duration JOB_DURATION
                        wall clock time for job
  --output-directory OUTPUT_DIRECTORY
                        directory to which to write configuration files
                        (defaults to `.`)
  --skip-existing       skip existing files
  --absolute-paths      write paths as absolute in configuration
  --verbose             show more verbose log messages

ADCIRC run options that are not exposed by this command, such as runs or gwce_solution_scheme, can be specified by directly modifying the JSON files.

The following command creates JSON files for coupling (ATMESH + WW3DATA) -> ADCIRC over a small Shinnecock Inlet mesh:

initialize_adcirc \
    --platform HERA \
    --mesh-directory /scratch2/COASTAL/coastal/save/shared/models/meshes/shinnecock/v1.0 \
    --output-directory hera_shinnecock_ike_spinup_tidal_atmesh_ww3data \
    --modeled-start-time 20080823 \
    --modeled-duration 14:06:00:00 \
    --modeled-timestep 00:00:02 \
    --nems-interval 01:00:00 \
    --adcirc-executable /scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/ALLBIN_INSTALL/NEMS-adcirc-atmesh-ww3data.x \
    --adcirc-processors 40
    --adcprep-executable /scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/ALLBIN_INSTALL/adcprep \
    --modulefile /scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/modulefiles/envmodules_intel.hera \
    --forcings tidal,atmesh,ww3data \
    --tidal-source TPXO \
    --tidal-path /scratch2/COASTAL/coastal/save/shared/models/forcings/tides/h_tpxo9.v1.nc \
    --tidal-spinup-duration 12:06:00:00 \
    --atmesh-path /scratch2/COASTAL/coastal/save/shared/models/forcings/shinnecock/ike/wind_atm_fin_ch_time_vec.nc \
    --ww3data-path /scratch2/COASTAL/coastal/save/shared/models/forcings/shinnecock/ike/ww3.Constant.20151214_sxy_ike_date.nc

This will create the directory hera_shinnecock_ike_spinup_tidal_atmesh_ww3data/ with the following JSON configuration files:

πŸ“‚ hera_shinnecock_ike_spinup_tidal_atmesh_ww3data/
┣ πŸ“œ configure_adcirc.json
┣ πŸ“œ configure_atmesh.json
┣ πŸ“œ configure_modeldriver.json
┣ πŸ“œ configure_nems.json
┣ πŸ“œ configure_slurm.json
┣ πŸ“œ configure_tidal_forcing.json
β”— πŸ“œ configure_ww3data.json

These files contain relevant configuration values for an ADCIRC run. You will likely wish to change these values to alter the resulting run, before generating the actual model configuration. For instance, NEMS connections and the run sequence need to be manually specified in configure_nems.json.

generate_adcirc

generate_adcirc reads JSON files (created by initialize_adcirc) and creates a set of ADCIRC configurations (fort.14, fort.15, etc.), as well as a script with which to submit the model run to a job manager such as Slurm.

/home/docs/checkouts/readthedocs.org/user_builds/coupledmodeldriver/envs/latest/lib/python3.10/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.2-CAPI-1.17.2) is incompatible with the GEOS version PyGEOS was compiled with (3.10.4-CAPI-1.16.2). Conversions between both will be slow.
  warnings.warn(
usage: generate_adcirc [-h]
                       [--configuration-directory CONFIGURATION_DIRECTORY]
                       [--output-directory OUTPUT_DIRECTORY]
                       [--relative-paths] [--overwrite] [--parallel]

options:
  -h, --help            show this help message and exit
  --configuration-directory CONFIGURATION_DIRECTORY
                        path containing JSON configuration files
  --output-directory OUTPUT_DIRECTORY
                        path to store generated configuration files
  --relative-paths      use relative paths in output configuration
  --overwrite           overwrite existing files
  --parallel            generate configurations concurrently

The following command will read the JSON files created in the example above and generate the following files:

cd hera_shinnecock_ike_spinup_tidal_atmesh_ww3data
generate_adcirc

πŸ“‚ hera_shinnecock_ike_spinup_tidal_atmesh_ww3data/
┣ πŸ“œ configure_adcirc.json
┣ πŸ“œ configure_atmesh.json
┣ πŸ“œ configure_modeldriver.json
┣ πŸ“œ configure_nems.json
┣ πŸ“œ configure_slurm.json
┣ πŸ“œ configure_tidal_forcing.json
┣ πŸ“œ configure_ww3data.json
┣ πŸ“‚ spinup/
┃  ┣ πŸ“œ fort.13
┃  ┣ πŸ”— fort.14 -> ../fort.14
┃  ┣ πŸ“œ fort.15
┃  ┣ πŸ“œ nems.configure
┃  ┣ πŸ“œ model_configure
┃  ┣ πŸ”— atm_namelist.rc -> ./model_configure
┃  ┣ πŸ“œ config.rc
┃  ┣ πŸ“œ setup.job
┃  β”— πŸ“œ adcirc.job
┣ πŸ“‚ runs/
┃  β”— πŸ“‚ unperturbed/
┃    ┣ πŸ“œ fort.13
┃    ┣ πŸ”— fort.14 -> ../../fort.14
┃    ┣ πŸ“œ fort.15
┃    ┣ πŸ”— fort.67.nc -> ../../spinup/fort.67.nc
┃    ┣ πŸ”— fort.68.nc -> ../../spinup/fort.68.nc
┃    ┣ πŸ“œ nems.configure
┃    ┣ πŸ“œ model_configure
┃    ┣ πŸ”— atm_namelist.rc -> ./model_configure
┃    ┣ πŸ“œ config.rc
┃    ┣ πŸ“œ setup.job
┃    β”— πŸ“œ adcirc.job
┣ πŸ“œ fort.14
┣ πŸ“œ cleanup.sh
β”— πŸ“œ run_hera.sh

check_completion

check_completion checks the completion status of a running model directory.

/home/docs/checkouts/readthedocs.org/user_builds/coupledmodeldriver/envs/latest/lib/python3.10/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.2-CAPI-1.17.2) is incompatible with the GEOS version PyGEOS was compiled with (3.10.4-CAPI-1.16.2). Conversions between both will be slow.
  warnings.warn(
usage: check_completion [-h] [--model MODEL] [--verbose] [directory ...]

positional arguments:
  directory      directory containing model run configuration

options:
  -h, --help     show this help message and exit
  --model MODEL  model that is running, one of: `ADCIRC`
  --verbose      list all errors and problems with runs

check_completion

{
    "hera_shinnecock_ike_spinup_tidal_atmesh_ww3data": {
        "spinup": "running - 15%",
        "runs": "not_started - 0%"
    }
}

you can also pass a specific directory (or several directories):

check_completion spinup

{
    "spinup": "running - 27%"
}

cd run_20211027_florence_besttrack_250msubset_quadrature
check_completion runs/*_13

{
    "vortex_4_variable_perturbation_13": "completed - 100.0%",
    "vortex_4_variable_quadrature_13": "not_started - 0%"
}

if a run has an error, you can pass –verbose to see detailed logs:

check_completion spinup

{
    "spinup": "error - 0%"
}

check_completion spinup --verbose

{
    "spinup": {
        "status": "error",
        "progress": "0%",
        "error": {
            "ADCIRC_SETUP_SPINUP.err.log": [
                "forrtl: severe (24): end-of-file during read, unit -4, file /proc/92195/fd/0\n",
                "Image              PC                Routine            Line        Source             \n",
                "adcprep            000000000069A72E  Unknown               Unknown  Unknown\n",
                "adcprep            00000000006CBAAF  Unknown               Unknown  Unknown\n",
                "adcprep            000000000050A5CB  openprepfiles_           6996  prep.F\n",
                "adcprep            0000000000507F22  prep13_                   753  prep.F\n",
                "adcprep            000000000042E2E9  prepinput_                717  adcprep.F\n",
                "adcprep            000000000042BCDB  MAIN__                    239  adcprep.F\n",
                "adcprep            000000000040B65E  Unknown               Unknown  Unknown\n",
                "libc-2.17.so       00002AAEC02EB555  __libc_start_main     Unknown  Unknown\n",
                "adcprep            000000000040B569  Unknown               Unknown  Unknown\n",
                "srun: error: h24c51: task 0: Exited with exit code 24\n",
                "srun: launch/slurm: _step_signal: Terminating StepId=25366266.1\n"
            ]
        }
    }
}

check_completion runs

{
    "spinup": "failed - 0%"
}

check_completion runs --verbose

{
    "runs": {
        "status": "failed",
        "progress": "0%",
        "failed": {
            "fort.16": "ADCIRC output file `fort.16` not found"
        },
        "error": {
            "ADCIRC_SETUP_unperturbed.err.log": [
                "slurmstepd: error: execve(): /scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/ADCIRC/ALLBIN_INSTALL/adcprep: No such file or directory\n",
                "srun: error: h18c49: task 0: Exited with exit code 2\n",
                "srun: launch/slurm: _step_signal: Terminating StepId=25366268.0\n"
            ]
        }
    }
}

unqueued_runs

unqueued_runs finds and optionally submits runs that haven’t been queued to a job manager.

/home/docs/checkouts/readthedocs.org/user_builds/coupledmodeldriver/envs/latest/lib/python3.10/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.2-CAPI-1.17.2) is incompatible with the GEOS version PyGEOS was compiled with (3.10.4-CAPI-1.16.2). Conversions between both will be slow.
  warnings.warn(
usage: unqueued_runs [-h] [--model MODEL] [--dependency DEPENDENCY] [--submit]
                     [directory ...]

positional arguments:
  directory             directory containing model run configuration

options:
  -h, --help            show this help message and exit
  --model MODEL         model that is running, one of: `ADCIRC`
  --dependency DEPENDENCY
                        job ID of dependency
  --submit              whether to submit unqueued runs

corresponding Python functions

coupledmodeldriver.client.initialize_adcirc.initialize_adcirc(platform: Platform, mesh_directory: PathLike, modeled_start_time: datetime, modeled_duration: timedelta, modeled_timestep: timedelta, tidal_spinup_duration: timedelta | None = None, perturbations: Dict[str, Dict[str, Any]] | None = None, nems_interval: timedelta | None = None, nems_connections: List[str] | None = None, nems_mediations: List[str] | None = None, nems_sequence: List[str] | None = None, modulefile: PathLike | None = None, forcings: List[ForcingJSON] | None = None, adcirc_executable: PathLike | None = None, adcprep_executable: PathLike | None = None, aswip_executable: PathLike | None = None, adcirc_processors: int | None = None, job_duration: timedelta | None = None, output_directory: PathLike | None = None, absolute_paths: bool = True, overwrite: bool | None = None, verbose: bool = False)

creates a set of JSON configuration files according to the given parameters

Parameters:

  • platform – HPC platform for which to configure

  • mesh_directory – path to input mesh (fort.13, fort.14)

  • modeled_start_time – start time within the modeled system

  • modeled_duration – duration within the modeled system

  • modeled_timestep – time interval within the modeled system

  • tidal_spinup_duration – duration of tidal spinup in model time

  • perturbations – mapping of perturbation names to changed values

  • nems_interval – modeled time interval of NEMS main loop

  • nems_connections – list of NEMS connections as strings (i.e. ATM -> OCN)

  • nems_mediations – list of NEMS mediations, including functions

  • nems_sequence – list of NEMS entries in sequence order

  • modulefile – path to modulefile to source before model execution

  • forcings – list of forcings to configure, from [β€˜tidal’, β€˜atmesh’, β€˜besttrack’, β€˜owi’, β€˜ww3data’]

  • adcirc_executable – filename of compiled adcirc` `or ``NEMS.x

  • adcprep_executable – filename of compiled adcprep (mesh decomposition)

  • aswip_executable – filename of compiled aswip (preprocessing of best track / ATCF file)

  • adcirc_processors – numbers of processors to assign for ADCIRC

  • job_duration – wall clock time for job

  • output_directory – directory to which to write configuration files (defaults to .)

  • absolute_paths – whether to use absolute paths in configuration

  • overwrite – whether to overwrite existing files

  • verbose – whether to log extra debugging messages

coupledmodeldriver.generate.adcirc.generate.generate_adcirc_configuration(configuration_directory: PathLike, output_directory: PathLike | None = None, relative_paths: bool = False, overwrite: bool = False, parallel: bool = True)

generate an ADCIRC run configuration from JSON configuration files generated by initialize_adcirc

Parameters:

  • configuration_directory – path containing JSON configuration files

  • output_directory – path to store generated configuration files

  • relative_paths – whether to write relative paths in generated configuration files

  • overwrite – whether to overwrite existing files

  • parallel – generate configurations concurrently

coupledmodeldriver.client.check_completion.check_completion(directory: PathLike | None = None, verbose: bool = False, model: ModelJSON | None = None) → Dict[str, Any]

check the completion status of a running model

Parameters:

  • directory – directory containing model run configuration

  • verbose – list all errors and problems with runs

  • model – model that is running, one of: ADCIRC

Returns:

JSON output of completion status

coupledmodeldriver.client.unqueued_runs.get_unqueued_runs(directories: List[PathLike], model: ModelJSON | None = None, **kwargs) → Dict[str, Path]

get runs in the local configuration that have not been submitted / queued to the job manager

Parameters:

  • directories – directory containing model run configuration

  • model – model that is running, one of: ADCIRC

Returns:

mapping of unqueued run names to their directory paths