2.3.14. vacumm.misc.config – Configuration management

2.3.14.1. Overview

Functions:

print_short_help() opt2rst() cfg2rst() cfgargparse() cfgoptparse() get_spec() get_secnames() list_options() option2rst()

filter_section() register_config_validator() get_validator() validator_bbox() validator_cdtime() validator_cmap() validator_color() validator_datetime() validator_dict()

validator_eval() validator_figsize() validator_interval() validator_minmax() validator_numerics() validator_path() validator_timeunits() validator_workdir()

Classes:

ConfigException ValidationWarning ConfigManager cfgspecs configspecs specs validator arg_long_help() arg_parse() arg_patch() defaults() get_rst() get_spec() getspec() load() opt_long_help() opt_parse() opt_patch() patch() reset()

2.3.14.2. Content

Configuration management with validation capabilities and commandline interaction. It is based on configobj and validate modules.

List of builtin validators for validate, along with their plural form: bbox bbox_list() bboxes() boolean boolean_list() booleans() cdtime cdtime_list() cdtimes() cmap cmap_list() cmaps() color color_list() colors() date date_list() dates() datetime datetime_list() datetimes() dict dict_list() dicts() directories() directory directory_list() eval eval_list() evals() figsize figsize_list() figsizes() file file_list() files() float float_list() floats() integer integer_list() integers() interval interval_list() intervals() minmax minmax_list() minmaxes() numerics numerics_list() numericss() path path_list() paths() string string_list() strings() timeunits timeunits_list() timeunitss() workdir workdir_list() workdirs()

exception ConfigException

Bases: exceptions.Exception

args

message

exception ValidationWarning

Bases: exceptions.Warning

args

message

class ConfigManager(cfgspecfile=None, validator=None, interpolation='template', encoding=None, boolean_false=True, splitsecdesc=False, cfgfilter=None, cfgfilter_default=False, warn_empty_specs=False)

Bases: object

A configuration management class based on a configuration specification file and a validate.Validator

Example:

>>> Cfg = Config('config.ini', interpolation='template')
>>> Cfg.opt_parse()
>>> cfg = Cfg.load('config.cfg')

See also:

configobj.ConfigObj and validate.Validator

arg_long_help(rst=True, usage=None, description='Long help based on config specs')

Get the generic long help from config specs

Params:

• rst, optional: Reformat output in rst.

arg_parse(parser=None, exc=[], parse=True, args=None, getparser=False, getargs=False, cfgfile='config.cfg', patch=None, cfgfileopt='--cfgfile', cfgfilepatch='before', nested=None, extraopts=None)

Options (argparse) and config mixer.

1. Creates command-line options from config defaults
2. Parse command-line argument and create a configuration patch

For instance, the following config define the commandline option --section1-my-section2-my-key with value as a default value, s tored in a special group of options with a short name and a long description:

[section1] # Short name : long description of the group
    [[my_section2]]
        my_key=value

Warning

Section and option names must not contain any space-like character !

Note

If you want to prevent conflict of options, don’t use "_" in section and option names.

Params:

• parser: optional, a default one is created if not given. This can be:

  • a OptionParser instance
  • a dict with keyword arguments for the one to be created

• exc, optional: List of keys to be excluded from parsing.

• parse, optional: If True, parse commande line options and arguments

• args, optional: List of arguments to parse instead of default sys.argv[1:]

• getparser: allow getting the parser in addition to the config if parse=True

• getargs: allow getting the parsed arguments in addition to the config if parse=True

• patch, optional: used if parse is True.

  Can take the following values: - a bool value indicating wheter to apply defaults on the returned config

  before applying the command line config

  • a ConfigObj instance to apply on the returned config before applying the command line config

• cfgfileopt: optional, if present a config file option will be added.

  Can be a string or couple of strings to use as the option short and/or long name

• cfgfilepatch: specify if the returned config must be patched if a config file command

  line option is provided and when to patch it. Can take the following values: - True or ‘before’: the config file would be used before command line options - ‘after’: the config file would be used after command line options - Any False like value: the config file would not be used

• nested: Name of a section whose defines the configuration. It must be used when the configuration in nested in more general configuration.

• extraopts: Extra options to declare in the form [(args1, kwargs1), ((args2, kwargs2), ...]

Return:

• the OptionParser if parse is False
• the resulting ConfigObj if parse is True and getparser is not True
• the resulting ConfigObj and the OptionParser if both parse and getparser are True

arg_patch(parser, exc=[], cfgfileopt='cfgfile')

Call to arg_parse() with an automatic patching of current configuration

Return:cfg, args

cfgspecs

configspecs

defaults(nocomments=False, interpolation=None)

Get the default config

Params:

• nocomments, optional: Do not include option comments in config file. If equal to 2, remove section comments too.
• interpolation, optional: if True, interpolate values.

Return:

A ConfigObj instance

get_rst(mode='specs', **kwargs)

Convert the default configuration to rst declarations with cfg2rst()

get_spec(sec, key, **kwargs)

See get_spec()

If sec is a basestring, use configspec[sec][key] Otherwise use sec as a configspec, sec[key]

getspec(sec, key, **kwargs)

See get_spec()

If sec is a basestring, use configspec[sec][key] Otherwise use sec as a configspec, sec[key]

load(cfgfile='config.cfg', validate='fix', geterr=False, patch=None, force=True, cfgfilter=False, **kwpatch)

Get a ConfigObj instance loaded from a file

Params:

• cfgfile, optional: config file

  • a config file name
  • a ConfigObj instance
  • None: defaults to "config.cfg"

• validate, optional: Type of validation

  • False: no validation
  • "fix": validation fixes and reports errors
  • "report": validation reports errors
  • "raise": validation raises errors

• geterr, optional: Return validation results as well

• force, optional: Force re-instantiation of cfgfile when it is already a ConfigObj instance.

Return:

Depends on geterr

• if True: (cfg, err) where is the result of validate()
• else: cfg (ConfigObj instance)

opt_long_help(rst=True, usage='Usage: [prog] [options] ...', description='Long help based on config specs')

Get the generic long help from config specs

Params:

• rst, optional: Reformat output in rst.

opt_parse(parser=None, exc=[], parse=True, args=None, getparser=None, cfgfile='config.cfg', patch=None, cfgfileopt='--cfgfile', cfgfilepatch='before', nested=None)

Options (optparse) and config mixer.

1. Creates command-line options from config defaults
2. Parse command-line argument and create a configuration patch

For instance, the following config define the commandline option --section1-my-section2-my-key with value as a default value, s tored in a special group of options with a short name and a long description:

[section1] # Short name : long description of the group
    [[my_section2]]
        my_key=value

Warning

Section and option names must not contain any space-like character !

Note

If you want to prevent conflict of options, don’t use "_" in section and option names.

Params:

• parser: optional, a default one is created if not given. This can be:

  • a OptionParser instance
  • a dict with keyword arguments for the one to be created

• exc, optional: List of keys to be excluded from parsing.

• parse, optional: If True, parse commande line options and arguments

• args, optional: List of arguments to parse instead of default sys.argv[1:]

• getparser: allow getting the parser in addition to the config if parse=True

• patch, optional: used if parse is True.

  Can take the following values: - a bool value indicating wheter to apply defaults on the returned config

  before applying the command line config

  • a ConfigObj instance to apply on the returned config before applying the command line config

• cfgfileopt: optional, if present a config file option will be added.

  Can be a string or couple of strings to use as the option short and/or long name

• cfgfilepatch: specify if the returned config must be patched if a config file command

  line option is provided and when to patch it. Can take the following values: - True or ‘before’: the config file would be used before command line options - ‘after’: the config file would be used after command line options - Any False like value: the config file would not be used

Return:

• the OptionParser if parse is False
• the resulting ConfigObj if parse is True and getparser is not True
• the resulting ConfigObj and the OptionParser if both parse and getparser are True

opt_patch(parser, exc=[], cfgfileopt='cfgfile')

Call to arg_parse() with an automatic patching of current configuration

Return:cfg, args

patch(cfg, cfgpatch, validate=False)

Replace config values of cfg by those of cfgpatch

Params:

• cfg: A ConfigObj instance, a config file or a dictionary, that must be patched.
• cfgpatch: A ConfigObj instance, a config file or a dictionary, used for patching.
• validate, optional: If True, validate configs if they have a valid config spec.

reset(cfgfile='config.cfg', backup=True, nocomments=True, verbose=True)

Reset a config file to default values

Params:

• cfgfile, optional: The configuration file to reset.
• backup, optional: Backup the old config file.
• nocomments, optional: Do not include comment in config file.

Return:

A ConfigObj instance

See also:

defaults()

specs

validator

print_short_help(parser, formatter=None, compressed=False)

Print all help of an OptionParser instance but those of groups.

opt2rst(shelp, prog=None, secfmt=':%(secname)s:', descname='Description')

Convert options displayed in an help string to rst declarations of options

This is useful for autodocumenting executable python scripts that show a formatted help.

Params:

• shelp: Help string showing options (results from :option:--help).
• prog, optional: Program name, otherwise guess it from usage.

Output:

String converted to rst format (with cmdoption directives).

cfg2rst(cfg, mode='basic', optrole='confopt', secrole='confsec', **kwargs)

Convert a configuration to rst format

Configuration sections are declared with the rst directive “confsec” and options are declared with the rst directive “confopt”.

For instance:

a=1 # desc a
[s1] # desc s1
    b=2  # desc b
    [[s2]] # desc s2
        c=$a-$b # desd c
        [sec1] # section 1

is converted to:

.. confopt:: a

    desc a

.. confsec:: [s1]

    desc s1

    .. confopt:: [s1] b

        desc b

    .. confsec:: [s1][s2]

        desc s2

        .. confopt:: [s1][s2] c

            desd c

Then one can reference an option with for example ``[s1][s2]c.

Params:

• cfg: ConfigObj or ConfigManager instance. In the case of a ConfigManager, the defaults() are used.

Return:

rst string

cfgargparse(cfgspecfile, parser, cfgfileopt='cfgfile', cfgfile='config.cfg', exc=[], extraopts=None, args=None, **kwargs)

Merge configuration and commandline arguments

Params:

• cfgspecfile: Config specification file (.ini).
• parser: ArgumentParser instance.
• cfgfileopt, optional: Name of the option used to specify the user config file. Example: 'cfgfile' creates the option --cfgfile=<config file>.
• cfgfile, optional: Default name for the loaded config file.
• exc, optional: Config option name that must not be used to generated a commandline option.
• Extra params are passed to ConfigManager initialization.

Return:

A ConfigObj object

Tasks:

1. Initialize a default configuration (ConfigManager) from the specification file given by cfgspecfile.
2. Generate associated commandline options.
3. Load a user configuration file (specified with the option whose name is given by cfgfileopt).
4. Patch this configuration with user supplied options retrieved using the ArgumentParser parser parser.

Technically it combines ConfigManager and ConfigManager.arg_parse()

cfgoptparse(cfgspecfile, parser, cfgfileopt='cfgfile', cfgfile='config.cfg', exc=[], **kwargs)

Merge configuration and commandline arguments

Params:

• cfgspecfile: Config specification file (.ini).
• parser: ArgumentParser instance.
• cfgfileopt, optional: Name of the option used to specify the user config file. Example: 'cfgfile' creates the option --cfgfile=<config file>.
• cfgfile, optional: Default name for the loaded config file.
• exc, optional: Config option name that must not be used to generated a commandline option.
• Extra params are passed to ConfigManager initialization.

Tasks:

1. Initialize a default configuration (ConfigManager) from the specification file given by cfgspecfile.
2. Generate associated commandline options.
3. Load a user configuration file (specified with the option whose name is given by cfgfileopt).
4. Patch this configuration with user supplied options retrieved using the OptionParser parser parser.

Technically it combines ConfigManager and ConfigManager.opt_patch()

get_spec(spec, validator=None)

Get an option specification.

Params:

• spec: the specification string
• validator: (optional) the validator to use

Return:

A dict with keys: - funcname: the validation function name - args: the positionnal arguments - kwargs: the named arguments - default: the default value - iterable: if the value is list-like - func: the validation function - opttype: the function used with optparse - argtype: the function used with argparse

Read access to these keys can also be done as attribute of the returned dict (d.funcname == d[‘funcname’], …)

For example, a specification file containing:

[section]

option = integer(default=0, min=-10, max=10)

Would return: ``{‘funcname’: integer, ‘args’: [],

‘kwargs’: {‘min’: ‘-10’, ‘max’: ‘10’}, ‘default:’ 0, ‘opttype’: ‘int’, ‘argtype’: int, ‘func’:is_integer, ‘iterable’: None}

This can be usefull when you added extraneous named arguments into your specification file for your own use.

get_secnames(cfg)

Get section names as list from top to bottom [‘sec0’,’sec1’,…]

list_options(sec, optlist=None, parents=None, values=False, sections=False)

Get the list of options of section tree

Each list items has the format (sections, optname) or (sections, optname, optvalue) depending on the values keyword.

option2rst(option, optrole='confopt', secrole='confsec')

Format a tuple or (parents, optname) to a rst inline declaration

filter_section(sec, cfgfilter, default=False)

Recursively filter a section according to a dict of specifications

When encountering an option of sec, it removed if its value in cfgfilter is set to False. When not found, it default to the __default__ key of cfgfilter. And if the __default__ is not found, it defaults to False (filtered out). When an option is a section and its value in cfgfilter is a dictionary, this subsection is filtered in the same way with the value as restrictions (cfgfilter[subsection]).

Params:

• sec: A configobj.Section instance.
• cfgfilter: A dictionary tree with the same structure as sec.

register_config_validator(**kwargs)

Add a new configobj validator function

Example:

>>> register_config_validator(level=is_level)

get_validator(functions=None)

Get a default validator

validator_bbox(value, default=None)

Parse bbox coordinates with value format: x1,y1,x2,y2

validator_cdtime(value, min=None, max=None, default=None)

Validator of a date (compatible with cdtime.s2c())

validator_cmap(value, default=None)

Validator for colormaps

validator_color(value, default='k', alpha=False, as256=None)

validator_datetime(value, default=None, fmt='%Y-%m-%dT%H:%M:%S')

Parse value as a traditionnal python datetime object

validator_dict(value, default={}, vtype=None)

validator for dictionaries

value:

• dict(a=2, b=”x”)
• {“a”:2, “b”:”x”}
• a=2
• a=2, b=”x”
• OrderedDict(b=2)
• dict([(“a”,2),(“b”,”x”)])

validator_eval(value, default=None, unchanged_if_failed=True)

Validate a string that can be evaluated

validator_figsize(value, default=(6, 6), min=0, max=20)

Validator of a figure size (xsize,ysize)

validator_interval(value, default=None)

Validator of an interval of coordinates (min, max [,bounds])

validator_minmax(value, min=None, max=None, default=(0, 100), type='float')

Validator of a min,max pair

validator_numerics(value, default=None, min=None, max=None, type='float', n=None)

Validator of a tuple of numeric values

validator_path(value, default='', expand=None)

Parse a value as a path :Params:

– expand: expandvars and expandhome on loaded path

Warning: expand currently can’t work with interpolation

validator_timeunits(value, default='days since 1950-01-01')

Validator of standard time units

validator_workdir(value, default='')