Usage: The pycmor CLI

Usage: The pycmor CLI#

pycmor is the command line interface to the pycmor package. It provides a simple way to interface with the underlying Python, without needing to know too many details about what is going on behind the scenes. The CLI is hopefully simple and is the recommended way to get going.

You can get help with:

pycmor -h

The CLI is divided into a few subcommands. The main one you will want to use is:

pycmor process <configuration_yaml>

This will process the configuration file and run the CMORization process. Read on for a full summary of the commands.

  • pycmor develop: Tools for developers

    • Subcommand ls: Lists a directory and stores the output as a yaml. Possibly useful for development work and creating in-memory representations of certain folders.

  • pycmor externals: List external program status

    You might want to use NCO or CDO in your workflows. The pycmor externals command lists information about the currently found versions for these two programs.

  • pycmor plugins: Extending the command line interface

    The user can extend the pycmor.CLI by adding their own plugins to the main command. This lists the docstrings of those plugins.

    Note

    Paul will probably throw this out when we clean up the project for release.

  • pycmor process: The main command. Takes a yaml file and runs through the CMORization process.

  • pycmor ssh-tunnel: Creates port forwarding for Dask and Prefect dashboards. You should provide your username and the remote compute node, not the login node. The tunnels will default to 8787 for Dask and 4200 for Prefect.

    Important

    You need to run this from your laptop!

  • pycmor table-explorer: Opens up the web-based table explorer. This is a simple way to explore the tables that are available in the CMIP6 data request.

  • pycmor validate: Runs checks on a configuration file.

Command Line Reference#

pycmor#

PyCMOR - Makes CMOR Simple

pycmor [OPTIONS] COMMAND [ARGS]...

Options

--version#

Show the version and exit.

-v, --verbose#

Log debugging info to stderr.

Default:

False

-q, --quiet#

Suppress info to stderr.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

--profile_mem#

Profile peak memory use.

Default:

False

cache#

pycmor cache [OPTIONS] COMMAND [ARGS]...

Options

--version#

Show the version and exit.

-v, --verbose#

Log debugging info to stderr.

Default:

False

-q, --quiet#

Suppress info to stderr.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

--profile_mem#

Profile peak memory use.

Default:

False

inspect-prefect-global#

Print information about items in Prefect’s storage cache

pycmor cache inspect-prefect-global [OPTIONS] [CACHE_DIR]

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

CACHE_DIR#

Optional argument

inspect-prefect-result#
pycmor cache inspect-prefect-result [OPTIONS] RESULT

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

RESULT#

Required argument

populate-cache#
pycmor cache populate-cache [OPTIONS] [FILES]...

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

FILES#

Optional argument(s)

develop#

pycmor develop [OPTIONS] COMMAND [ARGS]...

Options

--version#

Show the version and exit.

-v, --verbose#

Log debugging info to stderr.

Default:

False

-q, --quiet#

Suppress info to stderr.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

--profile_mem#

Profile peak memory use.

Default:

False

ls#
pycmor develop ls [OPTIONS] DIRECTORY [OUTPUT_FILE]

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

DIRECTORY#

Required argument

OUTPUT_FILE#

Optional argument

prefect-check#

pycmor prefect-check [OPTIONS] CONFIG_FILE

Arguments

CONFIG_FILE#

Required argument

process#

pycmor process [OPTIONS] CONFIG_FILE

Arguments

CONFIG_FILE#

Required argument

scripts#

Various utility scripts for Pycmor.

pycmor scripts [OPTIONS] COMMAND [ARGS]...
fesom1#
pycmor scripts fesom1 [OPTIONS] COMMAND [ARGS]...
convert#

Main CLI for FESOM unstructured-to-structured conversion.

pycmor scripts fesom1 convert [OPTIONS] MESHPATH IPATH... [OPATH] [VARIABLE]

Options

-n, --ncore <ncore>#

Number of cores (parallel processes)

Default:

2

-s, --skip#

Skip the calculation if the output file already exists.

Arguments

MESHPATH#

Required argument

IPATH#

Required argument(s)

OPATH#

Optional argument

VARIABLE#

Optional argument

update-dimensionless-mappings#

Update dimensionless_mappings.yaml file from CMIP6 Tables

pycmor scripts update-dimensionless-mappings [OPTIONS]

Options

--yaml-path <yaml_path>#

Required Path to dimensionless_mappings.yaml file

--tables-path <tables_path>#

Required Path to CMIP6 Tables directory

ssh-tunnel-cli#

Create an SSH tunnel to access Prefect and Dask dashboards on a remote compute node.

pycmor ssh-tunnel-cli [OPTIONS]

Options

--local-dask-port <local_dask_port>#

Local port to forward from (Dask) [default: 8787]

--remote-dask-port <remote_dask_port>#

Remote port to forward to (Dask) [default: 8787]

--local-prefect-port <local_prefect_port>#

Local port to forward from (Prefect) [default: 4200]

--remote-prefect-port <remote_prefect_port>#

Remote port to forward to (Prefect) [default: 4200]

--gateway <gateway>#

Gateway server hostname [default: levante1.dkrz.de]

--compute-node <compute_node>#

Required Compute node hostname

--username <username>#

Required Username for SSH connections

table-explorer#

pycmor table-explorer [OPTIONS]

validate#

pycmor validate [OPTIONS] COMMAND [ARGS]...

Options

--version#

Show the version and exit.

-v, --verbose#

Log debugging info to stderr.

Default:

False

-q, --quiet#

Suppress info to stderr.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

--profile_mem#

Profile peak memory use.

Default:

False

config#
pycmor validate config [OPTIONS] CONFIG_FILE

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

CONFIG_FILE#

Required argument

directory#
pycmor validate directory [OPTIONS] CONFIG_FILE OUTPUT_DIR

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

CONFIG_FILE#

Required argument

OUTPUT_DIR#

Required argument

table#
pycmor validate table [OPTIONS] CONFIG_FILE TABLE_NAME

Options

--profile_mem#

Profile peak memory use.

Default:

False

--logfile, --no-logfile#

Log to file.

Default:

True

-q, --quiet#

Suppress info to stderr.

Default:

False

-v, --verbose#

Log debugging info to stderr.

Default:

False

Arguments

CONFIG_FILE#

Required argument

TABLE_NAME#

Required argument

Contents