A Collection of Utilities#
Math#
- pytools.levi_civita(tup: Tuple[int, ...]) int #
Compute an entry of the Levi-Civita symbol for the indices tuple.
- pytools.perm(*args, **kwargs)#
- pytools.comb(*args, **kwargs)#
Assertive accessors#
- pytools.one(iterable: Iterable[T]) T #
Return the first entry of iterable. Assert that iterable has only that one entry.
- pytools.is_single_valued(iterable: ~typing.Iterable[~pytools.T], equality_pred: ~typing.Callable[[~pytools.T, ~pytools.T], bool] = <built-in function eq>) bool #
- pytools.all_roughly_equal(iterable, threshold)#
Memoization#
- pytools.memoize(*args: F, **kwargs: Any) F #
Stores previously computed function values in a cache.
Two keyword-only arguments are supported:
- Parameters
use_kwargs β Allows the caller to use keyword arguments. Defaults to
False
. Setting this toTrue
has a non-negligible performance impact.key β A function receiving the same arguments as the decorated function which computes and returns the cache key.
- pytools.memoize_on_first_arg(function: Callable[[Concatenate[T, P]], R], *, cache_dict_name: Optional[str] = None) Callable[[Concatenate[T, P]], R] #
Like
memoize_method()
, but for functions that take the object in which do memoization information is stored as first argument.Supports cache deletion via
function_name.clear_cache(self)
.
- pytools.memoize_method(method: Callable[[Concatenate[T, P]], R]) Callable[[Concatenate[T, P]], R] #
Supports cache deletion via
method_name.clear_cache(self)
.Changed in version 2021.2: Can memoize methods on classes that do not allow setting attributes (e.g. by overwritting
__setattr__
), e.g. frozendataclasses
.
- pytools.memoize_in(container: Any, identifier: Hashable) None #
Adds a cache to the function it decorates. The cache is attached to container and must be uniquely specified by identifier (i.e. all functions using the same container and identifier will be using the same cache). The decorated function may only receive positional arguments.
Note
This function works well on nested functions, which do not have stable global identifiers.
Changed in version 2020.3: identifier no longer needs to be a
str
, but it needs to be hashable.Changed in version 2021.2.1: Can now use instances of classes as container that do not allow setting attributes (e.g. by overwritting
__setattr__
), e.g. frozendataclasses
.
- pytools.keyed_memoize_on_first_arg(key: Callable[[P], Hashable], *, cache_dict_name: Optional[str] = None) None #
Like
memoize_method()
, but for functions that take the object in which memoization information is stored as first argument.Supports cache deletion via
function_name.clear_cache(self)
.- Parameters
key β A function receiving the same arguments as the decorated function which computes and returns the cache key.
cache_dict_name β The name of the dict attribute in the instance used to hold the cache.
New in version 2020.3.
- pytools.keyed_memoize_method(key: Callable[[P], Hashable], *, cache_dict_name: Optional[str] = None) None #
Like
memoize_method
, but additionally uses a function key to compute the key under which the function result is stored.Supports cache deletion via
method_name.clear_cache(self)
.- Parameters
key β A function receiving the same arguments as the decorated function which computes and returns the cache key.
New in version 2020.3.
Changed in version 2021.2: Can memoize methods on classes that do not allow setting attributes (e.g. by overwritting
__setattr__
), e.g. frozendataclasses
.
- pytools.keyed_memoize_in(container: Any, identifier: Hashable, key: Callable[[P], Hashable]) None #
Like
memoize_in
, but additionally uses a function key to compute the key under which the function result is memoized.- Parameters
key β A function receiving the same arguments as the decorated function which computes and returns the cache key.
New in version 2021.2.1.
Argmin/max#
- pytools.argmin2(iterable, return_value=False)#
- pytools.argmax2(iterable, return_value=False)#
- pytools.argmin(iterable)#
- pytools.argmax(iterable)#
Cartesian products#
- pytools.cartesian_product(*args)#
- pytools.distinct_pairs(list1, list2)#
Permutations, Tuples, Integer sequences#
- pytools.wandering_element(length, wanderer=1, landscape=0)#
- pytools.generate_nonnegative_integer_tuples_below(n, length=None, least=0)#
n may be a sequence, in which case length must be None.
- pytools.generate_nonnegative_integer_tuples_summing_to_at_most(n, length)#
Enumerate all non-negative integer tuples summing to at most n, exhausting the search space by varying the first entry fastest, and the last entry the slowest.
- pytools.generate_all_integer_tuples_below(n, length, least_abs=0)#
- pytools.generate_permutations(original)#
Generate all permutations of the list original.
Nicked from http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/252178
- pytools.generate_unique_permutations(original)#
Generate all unique permutations of the list original.
Formatting#
- class pytools.Table(alignments: Optional[Tuple[str, ...]] = None)#
An ASCII table generator.
- nrows#
- ncolumns#
- alignments#
A
tuple
of alignments of each column:"l"
,"c"
, or"r"
, for left, center, and right alignment, respectively). Columns which have no alignment specifier will use the last specified alignment. For example, withalignments=("l", "r")
, the third and all following columns will use right alignment.
- __str__() str #
Returns a string representation of the table.
>>> tbl = Table(alignments=['l', 'r', 'l']) >>> tbl.add_row([1, '|']) >>> tbl.add_row([10, '20||']) >>> print(tbl) 1 | | ---+------ 10 | 20||
- github_markdown() str #
Returns a string representation of the table formatted as GitHub-Flavored Markdown.
>>> tbl = Table(alignments=['l', 'r', 'l']) >>> tbl.add_row([1, '|']) >>> tbl.add_row([10, '20||']) >>> print(tbl.github_markdown()) 1 | \| :--|-------: 10 | 20\|\|
- csv(dialect: str = 'excel', csv_kwargs: Optional[Dict[str, Any]] = None) str #
Returns a string containing a CSV representation of the table.
- Parameters
dialect β String passed to
csv.writer()
.csv_kwargs β Dict of arguments passed to
csv.writer()
.
>>> tbl = Table() >>> tbl.add_row([1, ","]) >>> tbl.add_row([10, 20]) >>> print(tbl.csv()) 1,"," 10,20
- latex(skip_lines: int = 0, hline_after: Optional[Tuple[int, ...]] = None) str #
Returns a string containing the rows of a LaTeX representation of the table.
- Parameters
skip_lines β number of lines to skip at the start of the table.
hline_after β list of row indices after which to add an
hline
(the indices must subtract skip_lines, if non-zero).
>>> tbl = Table() >>> tbl.add_row([0, "skipped"]) >>> tbl.add_row([1, "apple"]) >>> tbl.add_row([2, "pear"]) >>> print(tbl.latex(skip_lines=1)) 1 & apple \\ 2 & pear \\
- pytools.merge_tables(*tables: Table, skip_columns: Optional[Tuple[int, ...]] = None) Table #
- Parameters
skip_columns β a
tuple
of column indices to skip in all the tables except the first one.
- pytools.string_histogram(iterable, min_value=None, max_value=None, bin_count=20, width=70, bin_starts=None, use_unicode=True)#
- pytools.word_wrap(text, width, wrap_using='\n')#
A word-wrap function that preserves existing line breaks and most spaces in the text. Expects that existing line breaks are posix newlines (
\n
).
Debugging#
- pytools.typedump(val, max_seq=5, special_handlers=None)#
- pytools.invoke_editor(s, filename='edit.txt', descr='the file')#
Progress bars#
Name generation#
- pytools.generate_unique_names(prefix)#
- pytools.generate_numbered_unique_names(prefix: str, num: Optional[int] = None) Iterable[Tuple[int, str]] #
- class pytools.UniqueNameGenerator(existing_names: Optional[Set[str]] = None, forced_prefix: str = '')#
Class that creates a new
str
on each__call__()
that is unique to the generator.- __init__(existing_names: Optional[Set[str]] = None, forced_prefix: str = '')#
Create a new
UniqueNameGenerator
.
- add_name(name: str, *, conflicting_ok: bool = False) None #
- Parameters
conflicting_ok β A flag to dictate the behavior when name is conflicting with the set of existing names. If True, a conflict is silently passed. If False, a
ValueError
is raised on encountering a conflict.
Deprecation Warnings#
- pytools.deprecate_keyword(oldkey: str, newkey: Optional[str] = None, *, deadline: Optional[str] = None)#
Decorator used to deprecate function keyword arguments.
- Parameters
oldkey β deprecated argument name.
newkey β new argument name that serves the same purpose, if any.
deadline β expected time frame for the removal of the deprecated argument.
Functions for dealing with (large) auxiliary files#
- pytools.download_from_web_if_not_present(url, local_name=None)#
New in version 2017.5.
Helpers for numpy
#
- pytools.reshaped_view(a, newshape)#
Create a new view object with shape
newshape
without copying the data ofa
. This function is different fromnumpy.reshape
by raising an exception when data copy is necessary.- Parameters
a β a
numpy.ndarray
object.newshape β an
int
object or a tuple ofint
objects.
New in version 2018.4.
Timing data#
Log utilities#
- class pytools.ProcessLogger(logger, description, silent_level=None, noisy_level=None, long_threshold_seconds=None)#
Logs the completion time of a (presumably) lengthy process to
logging
. Only uses a high log level if the process took perceptible time.- __init__(logger, description, silent_level=None, noisy_level=None, long_threshold_seconds=None)#
- done(extra_msg=None, *extra_fmt_args)#
- __enter__()#
- __exit__(exc_type, exc_val, exc_tb)#
- class pytools.DebugProcessLogger(logger, description, silent_level=None, noisy_level=None, long_threshold_seconds=None)#
- class pytools.log_process(logger, description=None, long_threshold_seconds=None)#
A decorator that uses
ProcessLogger
to log data about calls to the wrapped function.- __init__(logger, description=None, long_threshold_seconds=None)#
- __call__(wrapped)#
Call self as a function.
Sorting in natural order#
- pytools.natorder(item)#
Return a key for natural order string comparison.
See
natsorted()
.New in version 2020.1.
- pytools.natsorted(iterable, key=None, reverse=False)#
Sort using natural order 1, as opposed to lexicographic order.
Example:
>>> sorted(["_10", "_1", "_9"]) == ["_1", "_10", "_9"] True >>> natsorted(["_10", "_1", "_9"]) == ["_1", "_9", "_10"] True
- Parameters
iterable β an iterable to be sorted. It must only have strings, unless key is specified.
key β if provided, a key function that returns strings for ordering using natural order.
reverse β if True, sorts in descending order.
- Returns
a sorted list
New in version 2020.1.
Backports of newer Python functionality#
- pytools.resolve_name(name)#
A backport of
pkgutil.resolve_name()
(added in Python 3.9).New in version 2021.1.2.
Hashing#
- pytools.unordered_hash(hash_instance, iterable, hash_constructor=None)#
Using a hash algorithm given by the parameter-less constructor hash_constructor, return a hash object whose internal state depends on the entries of iterable, but not their order. If hash is the instance returned by evaluating
hash_constructor()
, then the each entry i of the iterable must permithash.upate(i)
to succeed. An example of hash_constructor ishashlib.sha256
fromhashlib
.hash.digest_size
must also be defined. If hash_constructor is not provided,hash_instance.name
is used to deduce it.- Returns
the updated hash_instance.
Warning
The construction used in this function is likely not cryptographically secure. Do not use this function in a security-relevant context.
New in version 2021.2.
Sampling#
- pytools.sphere_sample_equidistant(npoints_approx: int, r: float = 1.0)#
Generate points regularly distributed on a sphere based on https://www.cmu.edu/biolphys/deserno/pdf/sphere_equi.pdf.
- Returns
an
ndarray
of shape(3, npoints)
, wherenpoints
does not generally equal npoints_approx.
- pytools.sphere_sample_fibonacci(npoints: int, r: float = 1.0, *, optimize: Optional[str] = None)#
Generate points on a sphere based on an offset Fibonacci lattice from 2.
- Parameters
optimize β takes the values: None to use the standard Fibonacci lattice,
"minimum"
to minimize the nearest neighbor distances in the lattice and"average"
to minimize the average distances in the lattice.- Returns
an
ndarray
of shape(3, npoints)
.
String utilities#
- pytools.strtobool(val: Optional[str], default: Optional[bool] = None) bool #
Convert a string representation of truth to True or False. True values are βyβ, βyesβ, βtβ, βtrueβ, βonβ, and β1β; false values are βnβ, βnoβ, βfβ, βfalseβ, βoffβ, and β0β. Uppercase versions are also accepted. If default is None, raises ValueError if val is anything else. If val is None and default is not None, returns default. Based on
distutils.util.strtobool()
.- Parameters
val β Value to convert.
default β Value to return if val is None.
- Returns
Truth value of val.
Type Variables Used#
- class pytools.T#
- class pytools.R#
Generic unbound invariant
typing.TypeVar
.
- class pytools.F#
Generic invariant
typing.TypeVar
bound to atyping.Callable
.
- class pytools.P#
Generic unbound invariant
typing.ParamSpec
.
An in-memory relational database table#
- class pytools.datatable.DataTable(column_names, column_data=None)#
An in-memory relational database table.
- __init__(column_names, column_data=None)#
Construct a new table, with the given C{column_names}.
- Parameters
column_names β An indexable of column name strings.
column_data β None or a list of tuples of the same length as column_names indicating an initial set of data.
- copy()#
Make a copy of the instance, but leave individual rows untouched.
If the rows are modified later, they will also be modified in the copy.
- deep_copy()#
Make a copy of the instance down to the row level.
The copyβs rows may be modified independently from the original.
- join(column, other_column, other_table, outer=False)#
Return a tabled joining this and the C{other_table} on C{column}.
The new table has the following columns: - C{column}, titled the same as in this table. - the columns of this table, minus C{column}. - the columns of C{other_table}, minus C{other_column}.
Assumes both tables are sorted ascendingly by the column by which they are joined.
Dot helper functions#
- pytools.graphviz.dot_escape(s: str) str #
Escape the string s for compatibility with the dot language, particularly backslashes and HTML tags.
- Parameters
s β The input string to escape.
- Returns
s with special characters escaped.
- pytools.graphviz.show_dot(dot_code: str, output_to: Optional[str] = None) Optional[str] #
Visualize the graph represented by dot_code.
- Parameters
dot_code β An instance of
str
in the dot language to visualize.output_to β
An instance of
str
that can be one of:"xwindow"
to visualize the graph as an X window."browser"
to visualize the graph as an SVG file in the systemβs default web-browser."svg"
to store the dot code as an SVG file on the file system. Returns the path to the generated SVG file.
Defaults to
"xwindow"
if X11 support is present, otherwise defaults to"browser"
.
- Returns
Depends on output_to. If
"svg"
, returns the path to the generated SVG file, otherwise returnsNone
.