2.3.1. vacumm.misc.misc
– Very generic tools¶
2.3.1.1. Overview¶
2.3.1.2. Content¶
Misc tools
Note
You can import it directly for example like this:
>>> from vacumm.misc import auto_scale
-
class
ArgList
(argsi)[source]¶ Bases:
object
Utility to always manage arguments as list and return results as input
Examples: >>> a = 'a' >>> al = ArgList(a) >>> al.get() # input for function as list ['a'] >>> al.put(['aa']) # output as input 'aa'
>>> a = ['a','b'] >>> al = ArgList(a) >>> al.get() ['a', 'b'] >>> al.put(['aa']) ['aa']
-
class
ArgTuple
(argsi)[source]¶ Bases:
object
Utility to always manage arguments as tuple and return results as input
Examples: >>> a = 'a' >>> al = ArgTuple(a) >>> al.get() # input for function as tuple ['a'] >>> al.put(['aa']) # output as input 'aa'
>>> a = ['a','b'] >>> al = ArgTuple(a) >>> al.get() ['a', 'b'] >>> al.put(['aa']) ['aa']
-
class
Att
[source]¶ Bases:
dict
Class to create a dictionnary and access and set keys as attributes.
You initialize and manage it as using
dict
.Example: >>> dd = Att(toto=3) >>> print dd['toto'] 3 >>> print dd.toto 3 >>> dd.toto = 5 >>> print dd['toto'] 5
-
class
CaseChecker
(valid_cases, errmode='raise', ic=True, casename='parameter')[source]¶ Bases:
object
Check that a string that represents a “case” is valid
-
class
FileTree
(input_dir, relative=False, scan=True, **kwargs)[source]¶ Bases:
object
Build a file tree
Params: - input_dir: Input directory.
- patterns, optional: A string (or a list of strings) indicating which REGULAR EXPRESSION patterns files must match (using glob.glob) [default: ‘.*’]
- exclude, optional:: A string (or a list of strings) indicating which REGULAR EXPRESSION patterns files must not match (using re.search) [default: [‘CVS/’,’.svn/’]]
- include, optional:: A string (or a list of strings) indicating which REGULAR EXPRESSION patterns files must match if they are excluded by ‘exclude’ patterns (using re.search) [default: None]
- relative, optional:: Return file names relative to input_dir [default: False]
-
default_patterns
= {'exclude': ['~$', '^CVS$/', '^\\.svn$/', '^\\.DS_Store$', '\\.db$', '\\.ini$'], 'include': [], 'patterns': ['.*']}¶
-
MV2_axisConcatenate
(axes, id=None, attributes=None, copy=True)[source]¶ Concatenate multiple axes including boundaries.
- axes:
- Axes to concatenate
- id:
- New axis identification (default None)
- attributes:
- Attributes to attached to the new Axis
Transient axis.
-
MV2_concatenate
(arrays, axis=0, axisid=None, axisattributes=None, copy=True)[source]¶ Concatenate the arrays along the given axis. Give the extended axis the id and attributes provided - by default, those of the first array.
-
N_choose
(a, choices, out=None, mode='raise')[source]¶ Robust version to
numpy.choose()
valid also for masked arrays.The func:numpy.choose function fails when length of choices is greater than 32! You can verify it with:
>>> numpy.choose([0], numpy.zeros((50, 1))) ! FAIL >>> N_choose([0], numpy.zeros((50, 1))) ! SUCCESS
Note
Performances of this function can be improved, even if it should not take alot of time to run in the vast majority of cases.
-
auto_scale
(data=None, nmax=None, vmin=None, vmax=None, separators=None, fractions=False, symetric=False, keepminmax=False, **kwargs)[source]¶ Computes levels according to a dataset and its range of values. Locators are on a 10-base. Different scaling can be used with this version.
Params: - data: The dataset
- vmax, optional: Replaces max(data)
- vmin, optional: // min(data)
- symetric, optional: min_value and max_value are made symetric with respect to zero [default: False]
- nmax, optional: Maximal number of levels
- separators, optional: Subdivision for irregular levels
- fractions, optional: Consider separators as fractions (]0.,1.[)
- steps, optional: Base 10 steps for finding levels [default: [1,2,2.5,5,10]]
- geo, optional: Treat levels as geographical degrees [default: False]
- Other parameters are given to MaxNLocator
Return: Array of levels See also: basic_auto_scale()
geo_scale()
-
basic_auto_scale
(vmin, vmax, nmax=7, steps=[1, 2, 2.5, 5, 10], geo=False, minutes=False, **kwargs)[source]¶ Computes levels according to a dataset and its range of values. Locators are on a 10-base.
Params: - vmin/vmax: Find levels around this range.
- nmax, optional: Maximal number of steps.
- steps, optional: Base 10 steps for finding levels [default: [1,2,2.5,5,10]]
- geo, optional: Assume longitude or latitude degrees [default: False].
- minutes, optional: If geo, find suitable levels to match nice minutes locations when locations have floating values (like 1.2) [default: False].
See also:
-
bivariate_normal
(X, Y, sigmax=1.0, sigmay=1.0, mux=0.0, muy=0.0, sigmaxy=0.0)[source]¶ Bivariate Gaussian distribution for equal shape X, Y. See bivariate normal at mathworld.
-
bound_ops
(bounds)[source]¶ Get operators that must be used for checking inclusion within boundaries. Returned operators (ops) must be used in the following way to return True if value is inside bounds:
Params: - bounds: Boundary closing indicator (like ‘co’)
Example: >>> ops[0](value,lower_bound) >>> ops[1](value,upper_bound)
Return: A 2-element tuple of operators taken within (operator.ge,operator.gt,operator.le,operator.lt).
-
broadcast
(set, n, mode='last', **kwargs)[source]¶ Broadcast
set
to the specified lengthn
Params: set: A single element or a sequence.
n: Final requested length.
mode, optional: Filling mode.
"last"
: Use the last element."first"
: Use the first element."cycle"
: Cycle through set.- if
fillvalue
is passed as a keyword, it is used to fill.
Example: >>> broadcast([2,4], 5) >>> broadcast(5, 4) >>> broadcast((2,3), 3) >>> broadcast((2,3), 3, mode='first') >>> broadcast([2,3], 3, mode='value', fillvalue=999)
-
check_case
(cases, case, mode='isvalid', **kwargs)[source]¶ Check that case is valid using :class:``CaseChecker` and allowed cases
-
check_def_atts
(obj, **defaults)[source]¶ Check defaults attributes and set them if empty
See also: get_atts()
cp_atts()
set_atts()
-
checkdir
(path, asfile=None, chmod=None)[source]¶ Make sure that a directory exists and has right access
Params: - path: Directory of file path.
- asfile, optional: Interpret
path
as file name instead of a directory name. WhenNone
, it is set toTrue
if it has a dot ("."
) inside.
Return: The directory name.
-
closeto
(a, b, rtol=1e-05, atol=1e-08)[source]¶ Check which values of a numeric array are close to those of another array
Params: - a: A
numpy.ndarray
variable. - b: Another array.
See also: allclose()
- a: A
-
cp_atts
(var1, var2, overwrite=True, select=None, exclude=None, extra=None, **kwargs)[source]¶ Copy all atributes from one variable to another
Params: - var1/var2: Copy var1 attributes to var2
- id, optional: Also copy id [default: False]
- overwrite, optional: Overwrite attributes of
var2
? - select, optional: Copy only these attributes.
- exclude, optional: Attributes that must be excuded from copy.
- Other keywords are set as attributes
See also:
-
cp_props
(var1, var2, axes=None, grid=True, atts=None, exaxes=None, exatts=None, owatts=True)[source]¶ Copy properties of a variable to another variabes
Proporties are attributes, axes and grid.
Params: - var1/var2: cdms variables.
- axes, optional: Position of axes to copy. Positions are
converted to relative position to that last dimension to handle
the case of variables that don’t have the same number of dimensions.
Set it to
False
to not copy axes. - exaxes, optional: Position of axes to not copy.
- grid, optional: Also copy the grid?
- atts, optional: Attributes to copy. Set it to
False
to not copy attributes. - exatts, optional: Attributes to exclude from copy.
- owatts, optional: Overwrite attributes?
Examples: >>> cp_props(var1, var2, atts=False) >>> cp_props(var1, var2, axes=[1,-1], atts=['units','long_name']) >>> cp_props(var1, var2, exaxes=0, grid=False) >>> cp_props(var1, var2, axes=False, exatts='units', owatts=False)
-
create_selector
(*args, **kwargs)[source]¶ Create a
cdms2.selectors.Selector
Params: args: Each item of args is treated in a special way:
- A dictionary is treated using:
selector.refine(**args[i])
- A list is treated using:
selector.refine(*args[i])
- Else (single slice, tuple, etc):
selector.refine(args[i])
- A dictionary is treated using:
kwargs: Items are integrated using:
selector.refine(**kwargs)
Examples: >>> create_selector((time1,time2),slice(0,1),lat=(lat1,lat2)) >>> create_selector([(time1,time2),(lat1,lat2)],(lon1,lon2)) >>> selector2 = create_selector(selector1, lon=(lon1,lon2))
-
deplab
(depths, fmt='%gm', auto=False, nosign=False)[source]¶ Return well formatted depth labels
Params: - depths: Numerical depths
- fmt, optional: Numeric format of the string (including units) [default: ‘%gm’]
- auto, optional: If True, find the ticks according to the range of depths [default: False]
- nosign, optional: Absolute values are used.
See also:
-
dict_aliases
(kwargs, aliases)[source]¶ Remove duplicate entries in a dictionnary according to a list of aliases. The first alias has priority over the following.
Example: >>> kwargs = dict(min=4, title='Title', label='Label') >>> dict_aliases(kwargs, ['label', 'title']) {'min': 4, 'label': 'Label'}
-
dict_copy_items
(ddi, ddo, keys)[source]¶ Copy existing items of an array to another one
Params: - ddi: Dictionary from which items are taken.
- ddo: Dictionary in which items are put.
- keys: single or list of keys.
-
dict_filter
(kwargs, filters, defaults=None, copy=False, short=False, keep=False, **kwadd)[source]¶ Filter out kwargs (typically extra calling keywords)
Params: - kwargs: Dictionnary to filter.
- filters: Single or list of prefixes.
- defaults: dictionnary of default values for output fictionnary.
- copy: Simply copy items, do not remove them from kwargs.
- short: Allow prefixes to not end with
"_"
. - keep: Keep prefix filter in output keys.
Example: >>> kwargs = {'basemap':'f', 'basemap_fillcontinents':True, 'quiet':False,'basemap_plot':False} >>> print kwfilter(kwargs,'basemap', defaults=dict(drawcoastlines=True,plot=True),good=True) {'plot': False, 'fillcontinents': True, 'good': True, 'basemap': 'f', 'drawcoastlines': True} >>> print kwargs {'quiet': False}
-
dict_filter_out
(kwargs, filters, copy=False, mode='start')[source]¶ Remove entries from a dictionary
Params: kwargs: Valid dictionary.
filters: Single or list of prefixes. Entries with a key starting with one of these filters are removed.
copy, optional: Create a copy or work on original dict?
mode, optional: Matching mode.
"start"
: remove if key starts with filter."end"
: remove if key ends with filter."regexp"
: remove if key match regexp match filter.- else remove if key is exactly filter.
-
dict_merge
(*dd, **kwargs)[source]¶ Merge dictionaries
First dictionaries have priority over next
Params: - dd: Argument are interpreted as dictionary to merge. Those who are not dictionaries are skipped.
- mergesubdicts, optional: Also merge dictionary items (like in a tree) [default: True].
- mergetuples, optional: Also merge tuple items [default: False].
- mergelists, optional: Also merge list items [default: False].
- unique, optional: Uniquify lists and tuples [default: True].
- skipnones, optional: Skip Nones [default: True].
- skipempty, optional: Skip everything that is not converted to False using bool [default: False].
- cls, optional: Class to use. Default to the first class found in arguments
that is not a
dict
, else defaults todict
.
Example: >>> d1 = dict(a=3, b=5) >>> d2 = dict(a=5, c=7) >>> print dict_merge(d1,d2) {'a': 3, 'c': 7, 'b': 5}
-
dicttree_set
(dd, *keys, **items)[source]¶ Set values in a tree of dicts
- dd: dict
- Dict of dicts to fill
keys: strings items: dict
To set>>> dd = {} >>> dicttree_set(dd, 'sec', 'subsec', 'subsubsec', option=15) >>> dicttree_set(dd, value1=30, value2=20)
-
dirsize
(folder, units='b')[source]¶ Get the size of a directory
Params: folder: path of the directory
units, optional:
"b"
: bytes"k"
: Kb"m"
: Mb"g"
: Gb"t"
: Tb
-
filter_level_selector
(selector, ids=None, **kwargs)[source]¶ Filter a
cdms2.selectors.Selector
instance to keep or remove z dimension
-
filter_selector
(selector, ids=None, copy=True, out=False, keeppos=False, noslice=False)[source]¶ Filter a
cdms2.selectors.Selector
instance to keep or remove only a list of idsParams: - selector: A
cdms2.selectors.Selector
instance. - ids, optional: Allowed ids.
- copy, optional: Filter original or copy?
- out, optional: Filter in (keep target selections) or out(remove target selections)?
- keeppos, optional: Keep positional components?
- noslice, optional: Remove slices?
- selector: A
-
geo_scale
(*args, **kwargs)[source]¶ auto_scale()
with geo=TrueSee also: basic_auto_scale()
auto_scale()
-
geodir
(direction, from_north=True, inverse=False)[source]¶ Return a direction in degrees in the form ‘WNW’
-
get_atts
(var, id=True, extra=None, **kwargs)[source]¶ Get all attributes from a variable
Params: - var: Target variable
- id, optional: Get also id [default: False]
- Other keywords are set as attributes
See also:
-
get_svn_revision
(path, max=False)[source]¶ Get the revision number of a path
Adapted from
numpy.disutils.misc_util.Configuration
-
grow_depth
(var, depth=None, mode=None, default=0.0, getvar=True)[source]¶ Make depth and variable have the same shape
Params: var: A MV2 array with a depth dimension.
depth, optional: Depth axis or array. If None, it is guessed from var. If not found, it defaults to
default
.default, optional: Default value for depth if not found.
mode, optional: How to handle case where shape of var is changed:
None
: Nothing happens"warn"
: A warning withwarn()
."raise"
: Raise aVACUMMError
exception.
Return: var, depth
wheredepth
may be a scalar equal todefault
-
grow_lat
(var, lat=None, mode=None, default=None, getvar=True)[source]¶ Make latitude and variable have the same shape
Params: var: A MV2 array with a lat dimension.
depth, optional: Latitude axis or array. If None, it is guessed from var.
default, optional: Default value for lat if not found.
mode, optional: How to handle case where shape of var is changed:
None
: Nothing happens"warn"
: A warning withwarn()
."raise"
: Raise aVACUMMError
exception.
Return: var, lat where lat may be ``None
-
grow_variables
(var1, var2)[source]¶ Grow dimensions of var1 and var2 until their match
It is an improved version
genutil.grower.grower()
.
-
intersect
(seg1, seg2, length=False)[source]¶ Intersection of two segments
Example: >>> intersect([1,10],(5,12)) [5, 10]
-
is_iterable
(obj, nostr=True, nogen=True)[source]¶ Check if an object is iterable or not.
Params: - obj: Object to check
Return: True/False
-
iterable
(obj, nostr=True, nogen=True)¶ Check if an object is iterable or not.
Params: - obj: Object to check
Return: True/False
-
kwfilter
(*args, **kwargs)[source]¶ Alias for
dict_filter()
-
kwfilterout
(kwargs, filters, copy=False, mode='start')¶ Remove entries from a dictionary
Params: kwargs: Valid dictionary.
filters: Single or list of prefixes. Entries with a key starting with one of these filters are removed.
copy, optional: Create a copy or work on original dict?
mode, optional: Matching mode.
"start"
: remove if key starts with filter."end"
: remove if key ends with filter."regexp"
: remove if key match regexp match filter.- else remove if key is exactly filter.
-
latlab
(latitudes, **kwargs)[source]¶ Return nice latitude labels
Params: - latitudes: Value of latitudes
- decimal, optional: Use decimal representation
- fmt, optional: Format for decimal representation
- no_seconds, optional: Do not add seconds to degrees representation
- tex, optional: Use Tex degree symbol
(depends on
rcParams['text.usetex']
by default) - no_symbol, optional: Don’t use degree symbol
- no_zeros, optional: Don’t insert zeros when integer < 10
- auto, optional: If True, find the ticks according to the range of values
- auto_minutes, optional: Automatically suppress degrees if value is not exactly a degree (just display minutes and seconds) else display degree
- bfdeg, optional: Use bold face for degrees when alone if auto_minutes and
rcParams['text.usetex']
are True.
See also:
-
lonlab
(longitudes, **kwargs)[source]¶ Return nice longitude labels
Params: - longitudes: Value of longitudes
- decimal, optional: Use decimal representation
- fmt, optional: Format for decimal representation
- no_seconds, optional: Do not add seconds to degrees representation
- tex, optional: Use Tex degree symbol
(depends on
rcParams['text.usetex']
by default) - no_symbol, optional: Don’t use degree symbol
- no_zeros, optional: Don’t insert zeros when integer < 10
- auto, optional: If True, find the ticks according to the range of values
- auto_minutes, optional: Automatically suppress degrees if value is not exactly a degree (just display minutes and seconds) else display degree
- bfdeg, optional: Use bold face for degrees when alone if auto_minutes and
rcParams['text.usetex']
are True.
See also:
-
main_geodir
(directions, amp=None, num=False, res=22.5, getamp=False, **kwargs)[source]¶ Return the dominant direction from a set of directions in degrees
-
mask_nan
(input)[source]¶ Mask NaN from a variable
Params: - input: Array (numpy, numpy.ma or MV2)
Note
If input is pure numpy, it is converted to numpy.ma
Example: >>> var = N.array([1,N.nan]) >>> print mask_nan(var) [1.0 --]
-
match_atts
(obj, checks, id=True, ignorecase=True, transform=None)[source]¶ Check that at least one of the attributes of an object matches check list
Params: - obj: An object
- checks: A dictionary of (attributes name, checklist)
-
match_string
(ss, checks, ignorecase=True, transform=None)[source]¶ Check that a string verify a check list that consists of a list of either strings or callables
-
numod
(*vv)[source]¶ Get the needed numeric module to be able to handle all specified variables
Params: Scalars or numeric arrays. Return: numpy
,numpy.ma
orMV2
.
-
phaselab
(vals, fmt='%.5g', label=None, decimal=True, tex=None, auto_minutes=False, no_seconds=False, no_symbol=False, no_zeros=False, auto=False, bfdeg=False, **kwargs)[source]¶ Return a nice label for degrees
Inspired from Basemap toolkit of Matplotlib
Remove html tags from a string
Params: - str: A string
Return: Cleaned string
Example:
>>> rm_html_tags('<title>My title</title>') My title
-
scalebox
(box, factor, square=False, xmargin=None, ymargin=None)[source]¶ Alter box limits with a zoom factor
Params: - box: A list of
[xmin,ymin,xmax,ymax]
. - factor: Zoom factor where 1 means no change, and a factor < 1 means a smaller box.
Example: >>> scalebox([0,0,1,2], 1.1) [-0.55, -1.10, 1.55, 2.55]
- box: A list of
-
scan_format_string
(format_string)[source]¶ Scan a format string using
string.Formatter
Source: SONAT
format_string: string
- dict:
- Positional and keyword fields with their properties in a dict
- dict:
- Dictionary of properties with the following keys: - positional: list of positional field keys - keyword: list of keyword field keys - with_time: list of keys that have a date pattern format
-
selector2str
(selector)[source]¶ Convert a selector to a generic string where <obj …> labels are removed
Example: >>> sel = cdms2.selectors.Selector(slice(2,5),time=(4,6)) >>> selector2str(sel) '((slice(2, 5, None)),("time", (4, 6)))'
-
set_atts
(var, atts=None, overwrite=True, **kwargs)[source]¶ Set attributes
Params: - var: Change attributes of var
- atts, optional: A dictionary of attributes
- overwrite, optional: Overwrite attributes of
var2
? - Other keywords are set as attributes
See also:
-
set_lang
(default='en_US.UTF-8', num=None)[source]¶ Set the default and numeric languages
The numeric language defaults to the default language.
-
set_lang_fr
(ennum=True)[source]¶ Set lang to french, except the numeric lang which is set to english by default
-
split_selector
(selector)[source]¶ Split a selector into (positionalComponents, axisComponents).
Params: - selector: cdms2.selectors.Selector
Return: a tuple of: - the list of positionalComponents - the
OrderedDict
of axisComponentsExemple: >>> selector = cdms2.selectors.Selector(slice(2,5),time=(4,6)) >>> split_selector(selector) ((slice(2, 5, None),), {'time': (4, 6)})
-
splitidx
(arr, crit)[source]¶ Return a list of split indices for a 1d array according to a criteria
arr: 1D array crit: int, list of int, int array, float
If a positive int, it refers to the number of almost equal intervals. If a negative int, it refers to the max number of item per interval. If a list or array of int, it is directly intepreted as split indices, with some checks. If a float, the input array is monotically increasing and the intervals are at least of size crit.- idx: list of ints
- Indices that are compatible ith
numpy.array_split()
.
-
squarebox
(box, scale=1)[source]¶ Get an approximate
[xmin, ymin, xmax, ymax]
that is square in meters
-
squeeze_variable
(var, spec=True, asmv=False)[source]¶ Remove singleton axes from a MV2 variable
Params: var: MV2 array.
spec, optional: Squeeze specification.
True
orNone
: Simply squeeze out all singletons.False
: Does nothing (return the same variable).- A string containing geo letters like ‘xyzt’ to remove axes according to their type.
asmv, optional: If True and input is a MV2 array, always return a MV2 array, even if all result is a scalar value or masked.
Examples: >>> squeeze_variable(var) >>> squeeze_variable(var, spec='tx')
-
write_ascii_time1d
(var, file, fmt='%g')[source]¶ Write an ascii file in the following format: YYY/MM/DD HH:MN:SS DATA where DATA is one column.
Params: - var: a cdms variable WITH A TIME AXIS
- file: output file name
- fmt, optional: format of DATA [default: ‘%g’]
-
xls_style
(style=None, b=None, i=None, u=None, c=None, o=None, bd=None, fmt=None, va=None, ha=None, f=None, s=None, n=None, copy=True, **kwargs)[source]¶ Excel style sheet for pyExcelerator cell objects
Params: - style, optional: Style object to update.
- scopy, optional: Make a copy of
style
if passed. - b, optional: Bold [True/False].
- i, optional: italic[True/False].
- u, optional: Underline [True/False].
- c, optional: Colour index [from 0 to 82!, with 0=black, 1=white, 2=red, 3=green, 4=blue, 5=yellow, 6=magenta, 7=cyan].
- o, optional: Outline [True/False].
- bd, optional: Borders [True/int/False/dict(top=int…)].
- fmt, optional: Format [‘general’,‘0.0’,…].
- va, optional: Vertical alignment [‘top’,’bottom’,’center’,’justified’].
- ha, optional: Horizontal alignment [‘left’,’right’,’center’,’justified’].
Example: >>> style_num = xls_style(fmt='0.00') >>> style_num_left = xls_style(style_num, left=2, copy=True) >>> style_full = xls_style(c=2, b=True, va='center', ha='justified')
Tutorial:
-
zoombox
(box, factor, square=False, xmargin=None, ymargin=None)[source]¶ Alias for
scalebox()
with1/factor
as zoom factor.