Collections (Data Structures + helpers)¶
Table of Contents
Functions, classes and/or types which either are, or are related to Python
variable storage types (dict
, tuple
, list
, set
etc.)
Object-like Dictionaries (dict’s)¶
Have you ever wanted a dictionary that works like an object, where you can get/set dictionary keys using
attributes (x.something
) as easily as you can with items (x['something']
)?
We did. So we invented DictObject
, a sub-class of the built-in dict
, making it compatible
with most functions/methods which expect a dict
(e.g. json.dumps()
).
You can create a new DictObject
and use it just like a dict
, or you can convert an existing
dict
into a DictObject
much like you’d cast any other builtin type.
It can also easily be cast back into a standard dict
when needed, without losing any data.
Creating a new DictObject and using it¶
Since DictObject
is a subclass of the builtin dict
, you can instantiate a new
DictObject
in the same way you would use the standard dict
class:
>>> d = DictObject(hello='world')
>>> d
{'hello': 'world'}
>>> d['hello']
'world'
>>> d.hello
'world'
>>> d.lorem = 'ipsum'
>>> d['orange'] = 'banana'
>>> d
{'hello': 'world', 'lorem': 'ipsum', 'orange': 'banana'}
Converting an existing dictionary (dict) into a DictObject¶
You can convert an existing dict
into a DictObject
in the same way you’d convert
any other object into a dict
:
>>> y = {"hello": "world", "example": 123}
>>> x = DictObject(y)
>>> x.example
123
>>> x['hello']
'world'
>>> x.hello = 'replaced'
>>> x
{'hello': 'replaced', 'example': 123}
It also works vice versa, you can convert a DictObject
instance back into a dict
just as
easily as you converted the dict into a DictObject.
>>> z = dict(x)
>>> z
{'hello': 'replaced', 'example': 123}
Dict-able NamedTuple’s¶
While collections.namedtuple()
’s can be useful, they have some quirks, such as not being able to access
fields by item/key (x['something']
). They also expose a method ._asdict()
, but cannot be directly casted
into a dict
using dict(x)
.
Our dictable_namedtuple()
collection is designed to fix these quirks.
What is a dictable_namedtuple and why use it?¶
Unlike the normal namedtuple()
types, ``dictable_namedtuple``s add extra convenience functionality:
Can access fields via item/key:
john['first_name']
Can convert instance into a dict simply by casting:
dict(john)
Can set new items/attributes on an instance, even if they weren’t previously defined.
NOTE: You cannot edit an original namedtuple field defined on the type, those remain read only
There are three functions available for working with dictable_namedtuple
classes/instances,
each for different purposes.
dictable_namedtuple()
- Create a newdictable_namedtuple
type for instantiation.
convert_dictable_namedtuple()
- Convert an existing namedtuple instance (not a type/class) into adictable_namedtuple
instance.
subclass_dictable_namedtuple()
- Convert an existing namedtuple type/class (not an instance) into adictable_namedtuple
type for instantiation.
Importing dictable_namedtuple functions¶
from collections import namedtuple
from privex.helpers import dictable_namedtuple, convert_dictable_namedtuple, subclass_dictable_namedtuple
Creating a NEW dictable_namedtuple type and instance¶
If you’re creating a new Named Tuple, and you want it to support dictionary-like access, and
have it able to be converted into a dict simply through dict(my_namedtuple)
, then you want
dictable_namedtuple()
Person = dictable_namedtuple('Person', 'first_name last_name')
john = Person('John', 'Doe')
dave = Person(first_name='Dave', last_name='Smith')
print(dave['first_name']) # Prints: Dave
print(dave.first_name) # Prints: Dave
print(john[1]) # Prints: Doe
print(dict(john)) # Prints: {'first_name': 'John', 'last_name': 'Doe'}
Converting an existing namedtuple instance into a dictable_namedtuple instance¶
If you have existing Named Tuple instances, e.g. returned from a python library, then you can use
convert_dictable_namedtuple()
to convert them into dictable_namedtuple
’s and gain all the
functionality mentioned at the start of this section.
Person = namedtuple('Person', 'first_name last_name') # This is an existing namedtuple "type" or "class"
john = Person('John', 'Doe') # This is an existing namedtuple instance
john.first_name # This works on a standard namedtuple. Returns: John
john[1] # This works on a standard namedtuple. Returns: Doe
john['first_name'] # However, this would throw a TypeError.
dict(john) # And this would throw a ValueError.
# We can now convert 'john' into a dictable_namedtuple, which will retain the functionality of a
# namedtuple, but add to the functionality by allowing dict-like key access, updating/creating new
# fields, as well as painlessly casting to a dictionary.
d_john = convert_dictable_namedtuple(john)
d_john.first_name # Returns: John
d_john[1] # Returns: Doe
d_john['first_name'] # Returns: 'John'
dict(d_john) # Returns: {'first_name': 'John', 'last_name': 'Doe'}
Converting an existing namedtuple type/class into a dictable_namedtuple type/class¶
If you have existing Named Tuple type/class then you can use subclass_dictable_namedtuple()
to convert the type/class into a dictable_namedtuple
type/class and gain all the functionality mentioned
at the start of this section. (NOTE: it’s usually easier to just replace your namedtuple
calls
with dictable_namedtuple
)
Person = namedtuple('Person', 'first_name last_name') # This is an existing namedtuple "type" or "class"
# We can now convert the 'Person' type into a dictable_namedtuple type.
d_Person = subclass_dictable_namedtuple(Person)
# Then we can use this converted type to create instances of Person with dictable_namedtuple functionality.
john = d_Person('John', 'Doe')
john.first_name # Returns: John
john[1] # Returns: Doe
john['first_name'] # Returns: 'John'
dict(john) # Returns: {'first_name': 'John', 'last_name': 'Doe'}
Copyright:
+===================================================+
| © 2019 Privex Inc. |
| https://www.privex.io |
+===================================================+
| |
| Originally Developed by Privex Inc. |
| License: X11 / MIT |
| |
| Core Developer(s): |
| |
| (+) Chris (@someguy123) [Privex] |
| (+) Kale (@kryogenic) [Privex] |
| |
+===================================================+
Copyright 2019 Privex Inc. ( https://www.privex.io )
-
class
privex.helpers.collections.
DictDataClass
[source]¶ This is a base class for use with Python 3.7+
dataclass
‘s, designed to make dataclasses more interoperable with existing dictionaries, and allows them to be used like dictionaries, similar toDictable
, but more powerful / flexible.The most notable difference between this and
Dictable
- is that DictDataClass uses the attributeraw_data
on your dataclass to store any excess attributes when your dataclass is initialised from a dictionary withfrom_dict()
orfrom_list()
, allowing you to retrieve any dictionary keys which couldn’t be stored on your dataclass instance.Basic Example:
>>> from dataclasses import dataclass, field >>> from privex.helpers import DictDataClass, DictObject >>> from typing import Union >>> >>> @dataclass >>> class ExampleDataclass(DictDataClass): ... example: str = 'hello world' ... lorem: int = 999 ... raw_data: Union[dict, DictObject] = field(default_factory=DictObject, repr=False) ... ### ^ The raw, unmodified data that was passed as kwargs, as a dictionary ... ### For DictDataClass to work properly, you must include the raw_data dataclass field in your dataclass. ... >>> edc = ExampleDataclass.from_dict(dict(example='test', hello='this is an example')) >>> edc.example 'test' >>> dict(edc) {'example': 'test', 'lorem': 999}
Thanks to
raw_data
- you can access any extraneous items contained within the dictionary used infrom_dict()
as if they were part of your dataclass. You can also access and set any attribute using standard dictionary item syntax with square brackets like so:>>> edc.hello # This is not in the original dataclass attributes, but is proxied from raw_data 'this is an example' >>> edc['hello'] # Also works with item / dict syntax 'this is an example' >>> edc['hello'] = 'world' # You can set attributes using "key" / dict-like syntax >>> edc.hello 'world' >>> edc.hello = 'test' # You can also set raw_data keys using standard attribute dot-notation syntax. >>> edc['hello'] 'test'
Dictionary casting modes / ``__iter__`` configuration modes
There are a total of four (4)
dict`
conversion modes that you may use for a given class.These modes control whether
raw_data
is used when an instance is being casted viadict(obj)
, the order in which it’s merged with the instance attributes, along with the option to include just the dataclass attributes, or just the raw_data.Available
DictConfig.dict_convert_mode
options:* **Fallback / Dataclass / Instance Attributes Only** - When ``dict_convert_mode`` is empty (``None`` or ``""``), or it can't be matched against a pre-defined conversion mode, when an instance is converted into a :class:`.dict` - only the attributes of the instance are used - :attr:`.raw_data` keys are ignored. ``dict_convert_mode`` settings: ``None``, ``""``, ``"none"`` or any other invalid option. * Raw Data Only ( ``raw`` / ``raw_data`` ) - When this mode is used, converting the instance to a ``dict`` will effectively just return ``raw_data``, while still enforcing the ``dict_exclude`` and ``dict_listify`` settings. ``dict_convert_mode`` settings: ``"raw"``, ``"raw_data"``, ``"rawdata"``, or any other value beginning with ``raw`` * Merge with Dataclass Priority - In this mode, both :attr:`.raw_data` and the instance's attributes are used when converting into a :class:`.dict` - first the :attr:`.raw_data` dictionary is taken, and we merge the instance attributes on top of it. This means the instance/dataclass attributes take priority over the ``raw_data`` attributes, which will generally result in only ``raw_data`` keys which don't exist on the instance having their values used in the final ``dict``. ``dict_convert_mode`` settings: ``"merge_dc"`` / ``"merge_ins"`` / ``"merge_dataclass"`` * Merge with :attr:`.raw_data` Priority - In this mode, both :attr:`.raw_data` and the instance's attributes are used when converting into a :class:`.dict` - first the instance attributes are converted into a dict, then we merge the :attr:`.raw_data` dictionary on top of it. This means the :attr:`.raw_data` keys take priority over the instance/dataclass attributes, which will generally result in only instance attributes which don't exist in ``raw_data`` having their values used in the final ``dict``. ``dict_convert_mode`` settings: ``"merge_rd"`` / ``"merge_raw"`` / ``"merge_raw_data"``
By default, the conversion mode is set to
merge_dc
, which means when an instance is converted into adict
- the dataclass instance attributes are merged on top of the raw_data dictionary, meaning raw_data is included, but dataclass attributes take priority overraw_data
keys.Changing the conversion mode
>>> from dataclasses import dataclass, field >>> >>> @dataclass >>> class MyDataclass(DictDataClass): ... class DictConfig: ... dict_convert_mode = 'merge_dc' ... ... hello: str ... lorem: str = 'ipsum' ... raw_data: Union[dict, DictObject] = field(default_factory=DictObject, repr=False) >>> dc = MyDataclass.from_dict(dict(hello='test', example=555, test='testing')) >>> dc.hello 'test' >>> dc.hello = 'replaced' >>> dict(dc) {'hello': 'replaced', 'example': 555, 'test': 'testing', 'lorem': 'ipsum'} >>> dc.DictConfig.dict_convert_mode = 'merge_raw' {'hello': 'test', 'lorem': 'ipsum', 'example': 555, 'test': 'testing'} >>> dc.DictConfig.dict_convert_mode = None {'hello': 'replaced', 'lorem': 'ipsum'}
-
privex.helpers.collections.
DictDataclass
¶
-
class
privex.helpers.collections.
DictObject
[source]¶ A very simple
dict
wrapper, which allows you to read and write dictionary keys using attributes (dot notation) PLUS standard item (key / square bracket notation) access.Example Usage (creating and using a new DictObject):
>>> d = DictObject(hello='world') >>> d {'hello': 'world'} >>> d['hello'] 'world' >>> d.hello 'world' >>> d.lorem = 'ipsum' >>> d['orange'] = 'banana' >>> d {'hello': 'world', 'lorem': 'ipsum', 'orange': 'banana'}
Example Usage (converting an existing dict):
>>> y = {"hello": "world", "example": 123} >>> x = DictObject(y) >>> x.example 123 >>> x['hello'] 'world' >>> x.hello = 'replaced' >>> x {'hello': 'replaced', 'example': 123}
-
class
privex.helpers.collections.
Dictable
[source]¶ A small abstract class for use with Python 3.7 dataclasses.
Allows dataclasses to be converted into a
dict
using the standarddict()
function:>>> @dataclass >>> class SomeData(Dictable): ... a: str ... b: int ... >>> mydata = SomeData(a='test', b=2) >>> dict(mydata) {'a': 'test', 'b': 2}
Also allows creating dataclasses from arbitrary dictionaries, while ignoring any extraneous dict keys.
If you create a dataclass using a
dict
and you have keys in yourdict
that don’t exist in the dataclass, it’ll generally throw an error due to non-existent kwargs:>>> mydict = dict(a='test', b=2, c='hello') >>> sd = SomeData(**mydict) TypeError: __init__() got an unexpected keyword argument 'c'
Using
from_dict
you can simply trim off any extraneous dict keys:>>> sd = SomeData.from_dict(**mydict) >>> sd.a, sd.b ('test', 2) >>> sd.c AttributeError: 'SomeData' object has no attribute 'c'
-
class
privex.helpers.collections.
Mocker
(modules: dict = None, attributes: dict = None, *args, **kwargs)[source]¶ This mock class is designed to be used either to act as a stand-in “noop” (no operation) object, which could be used either as a drop-in replacement for a failed module / class import, or for certain unit tests.
If you need additional functionality such as methods having actual behaviour, you can set attributes on a Mocker instance to either a lambda, or point them at a real function/method:
>>> m = Mocker() >>> m.some_func = lambda a: a+1 >>> m.some_func(5) 6
Example use case - fallback for unimportant module imports
Below is a real world example of using
Mocker
andprivex.helpers.decorators.mock_decorator()
to simulatepytest
- allowing your tests to run under the standardunittest
framework if a user doesn’t have pytest (as long as your tests aren’t critically dependent on PyTest).Try importing
pytest
then fallback to a mock pytest:>>> try: ... import pytest ... except ImportError: ... from privex.helpers import Mocker, mock_decorator ... print('Failed to import pytest. Using privex.helpers.Mocker to fake pytest.') ... # Make pytest pretend to be the class 'module' (the class actually used for modules) ... pytest = Mocker.make_mock_class('module') ... # To make pytest.mark.skip work, we add the fake module 'mark', then set skip to `mock_decorator` ... pytest.add_mock_module('mark') ... pytest.mark.skip = mock_decorator ...
Since we added the mock module
mark
, and set the attributeskip
to point atmock_decorator
, the test functiontest_something
won’t cause a syntax error.mock_decorator
will just call test_something() which doesn’t do anything anyway:>>> @pytest.mark.skip(reason="this test doesn't actually do anything...") ... def test_something(): ... pass >>> >>> def test_other_thing(): ... if True: ... return pytest.skip('cannot test test_other_thing because of an error') ... >>>
Generating “disguised” mock classes
If you need the mock class to appear to have a certain class name and/or module path, you can generate “disguised” mock classes using
make_mock_class()
like so:>>> redis = Mocker.make_mock_class('Redis', module='redis') >>> redis <redis.Redis object at 0x7fd7402ea4a8>
A :class:`.Mocker` instance has the following behaviour
Attributes that don’t exist result in a function being returned, which accepts any arguments / keyword args, and simply returns
None
Example:
>>> m = Mocker() >>> repr(m.randomattr('hello', world=123)) 'None'
Arbitrary attributes
x.something
and itemsx['something']
can be set on an instance, and they will be similarly returned when they’re accessed. Attributes and items share the same key/value’s, so the following examples are all accessing the same data:
Example:
>>> m = Mocker() >>> m.example = 'hello' >>> m['example'] = 'world' >>> print(m.example) world >>> print(m['example']) world
You can add arbitrary “modules” to a Mocker instance. With only the
name
argument,add_mock_module()
will add a “module” under the instance, which is really just anotherMocker
instance.
Example:
>>> m = Mocker() >>> m.add_mock_module('my_module') >>> m.my_module.example = 'hello' >>> print(m.my_module['example'], m.my_module.example) hello hello
-
add_mock_module
(name: str, value=None, mock_attrs: dict = None, mock_modules: dict = None)[source]¶ Add a fake sub-module to this Mocker instance.
Example:
>>> m = Mocker() >>> m.add_mock_module('my_module') >>> m.my_module.example = 'hello' >>> print(m.my_module['example'], m.my_module.example) hello hello
- Parameters
name (str) – The name of the module to add.
value – Set the “module” to this object, instead of an instance of
Mocker
mock_attrs (dict) – If
value
isNone
, then this can optionally contain a dictionary of attributes/items to pre-set on the Mocker instance.mock_modules (dict) – If
value
isNone
, then this can optionally contain a dictionary of “modules” to pre-set on the Mocker instance.
-
add_mock_modules
(*module_list, _dict_to_attrs=True, _parse_dict=True, **module_map)[source]¶ >>> hello = Mocker.make_mock_class('Hello') >>> hello.add_mock_modules( ... world={ ... 'lorem': 'ipsum', ... 'dolor': 123, ... } ... )
- Parameters
module_list –
_parse_dict –
_dict_to_attrs –
module_map –
- Returns
-
classmethod
make_mock_class
(name='Mocker', instance=True, simple=False, attributes: dict = None, modules: dict = None, **kwargs) → Union[Any, privex.helpers.collections.Mocker, Type[privex.helpers.collections.Mocker]][source]¶ Return a customized mock class or create an instance which appears to be named
name
Allows code which might check
x.__class__.__name__
to believe it’s the correct object.Using the kwarg
module
you can change the module that the class / instance appears to have been imported from, allowing for quite deceiving fake classes and instances.Example usage:
>>> redis = Mocker.make_mock_class('Redis', module='redis') >>> # As seen below, the class appears to be called Redis, and even claims to be from the module `redis` >>> redis <redis.Redis object at 0x7fd7402ea4a8> >>> print(f'Module: {redis.__module__} - Class Name: {redis.__class__.__name__}') Module: redis - Class Name: Redis
Creating methods/attributes dynamically
You can set arbitrary attributes to point at a function, or just set them to a lambda:
>>> redis.exists = lambda key: 1 >>> redis.exists('hello') 1 >>> redis.hello() # Non-existent attributes just act as a function that eats any args and returns None None
- Parameters
name – The name to write onto the mock class’s
__name__
(and__qualname__
if not specified)instance (bool) – If
True
then the disguised mock class will be returned as an instance. Otherwise the raw class itself will be returned for you to instantiate yourself.simple (bool) – When
True
, generates a very basic, new class - not based onMocker
, which contains the attributes/methods defined in the paramattributes
.kwargs – All kwargs (other than
qualname
) are forwarded to__init__
of the disguised class ifinstance
is True.attributes (dict) – If
simple
is True, then this dictionary of attributes is used to generate the class’s attributes, methods, and/or constructor. Ifsimple
is False, andinstance
is True, these attributes are passed to the constructor of theMocker
clone that was generated.modules (dict) – If
simple
is False, andinstance
is True, this dict of modules are passed to the constructor of theMocker
clone that was generated.
- Key str qualname
Optionally specify the “qualified name” to insert into
__qualname__
. If this isn’t specified, thenname
is used for qualname, which is fine for most cases anyway.- Key str module
Optionally override the module namespace that the class is supposedly from. If not specified, then the class will just inherit this module (
privex.helpers.common
)- Returns
-
class
privex.helpers.collections.
OrderedDictObject
[source]¶ Ordered version of
DictObject
- dictionary with attribute access. SeeDictObject
-
privex.helpers.collections.
convert_dictable_namedtuple
(nt_instance, typename=None, module=None, **kwargs) → Union[NamedTuple, Dict][source]¶ Convert an existing
collections.namedtuple()
instance into a dictable_namedtuple instance.Example
First we create a namedtuple type
Person
>>> from collections import namedtuple >>> Person = namedtuple('Person', 'first_name last_name')
Next we create an instance of
Person
called John Doe, and we can confirm it’s a normal namedtuple, as we can’t access first_name by item/key.>>> john = Person('John', 'Doe') >>> john['first_name'] TypeError: tuple indices must be integers or slices, not str
Using
convert_dictable_namedtuple()
, we can convertjohn
from a normalnamedtuple
, into adictable_namedtuple
.This enables many convenience features (see
dictable_namedtuple()
for more info) such as easy casting to adict
, and accessing fields by item/key (square brackets):>>> from privex.helpers import convert_dictable_namedtuple >>> d_john = convert_dictable_namedtuple(john) >>> d_john Person(first_name='John', last_name='Doe') >>> d_john['first_name'] 'John' >>> dict(d_john) {'first_name': 'John', 'last_name': 'Doe'}
- Parameters
nt_instance – An instantiated namedtuple object (using a type returned from
collections.namedtuple()
)typename (str) – Optionally, you can change the name of your instance’s class, e.g. if you provide a
Person
instance, but you set this toMan
, then this will return aMan
instance, like so:Man(first_name='John', last_name='Doe')
module (str) – Optionally, you can change the module that the type class belongs to. Otherwise it will inherit the module path from the class of your instance.
- Key bool read_only
(Default:
False
) If set toTrue
, the outputted dictable_namedtuple instance will not allow new fields to be created via attribute / item setting.- Return dictable_namedtuple
The instance you passed
nt_instance
, converted into a dictable_namedtuple
-
privex.helpers.collections.
copy_class
(obj: Type[T], name=None, deep_copy=True, deep_private=False, **kwargs) → Union[Type[T], type][source]¶ Attempts to create a full copy of a
type
or class, severing most object pointers such as attributes containing adict
/list
, along with classes or instances of classes.Example:
>>> class SomeClass: >>> example = 'lorem ipsum' >>> data = ['hello', 'world'] >>> testing = 123 >>> >>> from privex.helpers import copy_class >>> OtherClass = copy_class(SomeClass, name='OtherClass')
If you then append to the
list
attributedata
on both SomeClass and OtherClass - with a different item appended to each class, you’ll see that the added item was only added todata
for that class, and not to the other class, proving the original and the copy are independent from each other:>>> SomeClass.data.append('lorem') >>> OtherClass.data.append('ipsum') >>> SomeClass.data ['hello', 'world', 'lorem'] >>> OtherClass.data ['hello', 'world', 'ipsum']
- Parameters
obj (Type[T]) – A
type
/ class to attempt to duplicate, deep copying each individual object in the class, to avoid any object pointers shared between the original and the copy.name (str|None) – The class name to use for the copy of
obj
. If not specified, defaults to the original class name fromobj
deep_copy (bool) – (Default:
True
) If True, usescopy.deepcopy()
to deep copy each attribute inobj
to the copy. If False, then standard references will be used, which may result in object pointers being copied.deep_private (bool) – (Default:
False
) If True,copy.deepcopy()
will be used on “private” class attributes, i.e. ones that start with__
. If False, attributes starting with__
will not be deep copied, only a standard assignment/reference will be used.kwargs – Additional advanced settings (see
keyword
pydoc entries for this function)use_bases (bool) – (Default:
True
) If True, copy the inheritance (bases) fromobj
into the class copy.quiet (bool) – (Default
False
) If True, log deep copy errors asdebug
level (usually silent in production apps) instead of the louderwarning
.bases (tuple) – A
tuple
of classes to use as “bases” (inheritance) for the class copy. If not specified, copies__bases__
from the original class.module (str) – If specified, overrides the module
__module__
in the class copy with this string, instead of copying from the original class.
- Return Type[T] obj_copy
A deep copy of the original
obj
-
privex.helpers.collections.
copy_class_simple
(obj: Type[T], name=None, qualname=None, module=<class 'privex.helpers.types.AutoDetected'>, allow_attrs: list = None, ban_attrs: list = None, **kwargs)[source]¶ This is an alternative to
copy_class()
which simply creates a blank class, then iterates overobj.__dict__
, usingsetattr()
to copy each attribute over to the cloned class.It uses
_q_copy()
to safely deep copy any attributes which are object references, and thus need their reference pointers severed, to avoid edits to the copy affecting the original (and vice versa).- Parameters
obj – The class to duplicate
name (str) – The class name to set on the duplicate. If left as
None
, the duplicate will retain the originalobj
name.qualname (str) – The qualified class name to set on the duplicate.
module (Optional[str]) – The module path to set on the duplicate, e.g.
privex.helpers.common
allow_attrs (list) – Optionally, you may specify additional private attributes (ones which start with
__
) that are allowed to be copied from the original class to the duplicated class.ban_attrs (list) –
Optionally, you may blacklist certain attributes from being copied from the original class to the duplicate.
Blacklisted attributes take priority over whitelisted attributes, so you may use this to cancel out any attributes in the default attribute whitelist
DEFAULT_ALLOWED_DUPE
which you don’t want to be copied to the duplicated class.kwargs –
bases (tuple|list) – If specified, overrides the default inherited classes (
obj.__bases__
) which would be set on the duplicated class’s__bases__
.
- Returns
-
privex.helpers.collections.
copy_func
(f: Callable, rewrap_classmethod=True, name=None, qualname=None, module=<class 'privex.helpers.types.AutoDetected'>, **kwargs) → Union[Callable, classmethod][source]¶ Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)
-
privex.helpers.collections.
dictable_namedtuple
(typename, field_names, *args, **kwargs) → Union[Type[collections.namedtuple], dict][source]¶ Creates a dictable_namedtuple type for instantiation (same usage as
collections.namedtuple()
) - unlike namedtuple, dictable_namedtuple instances allow item (dict-like) field access, support writing and can be painlessly converted into dictionaries viadict(my_namedtuple)
.Named tuple instances created from
dictable_namedtuple
types are generally backwards compatible with any code that expects a standardcollections.namedtuple()
type instance.Quickstart
>>> from privex.helpers import dictable_namedtuple >>> # Define a dictable_namedtuple type of 'Person', which has two fields - first_name and last_name >>> p = dictable_namedtuple('Person', 'first_name last_name') >>> john = p('John', 'Doe') # Alternatively you can do p(first_name='John', last_name='Doe') >>> john.first_name # You can retrieve keys either via attributes (dot notation) 'John' >>> john['last_name'] # Via named keys (square brackets) 'Doe' >>> john[1] # Or, via indexed keys (square brackets, with integer keys) 'Doe' >>> john.middle_name = 'Davis' # You can also update / set new keys via attribute/key/index >>> dict(john) # Newly created keys will show up as normal in dict(your_object) {'first_name': 'John', 'last_name': 'Doe', 'middle_name': 'Davis'} >>> john # As well as in the representation in the REPL or when str() is called. Person(first_name='John', last_name='Doe', middle_name='Davis')
This function adds / overrides the following methods on the generated namedtuple type:
_asdict
__iter__
__getitem__
__getattribute__
__setitem__
__setattr__
__repr__
Extra functionality compared to the standard
namedtuple()
generated classes:Can access fields via item/key:
john['first_name']
Can convert instance into a dict simply by casting:
dict(john)
Can set new items/attributes on an instance, even if they weren’t previously defined.
john['middle_name'] = 'Davis'
orjohn.middle_name = 'Davis'
Example Usage
First we’ll create a named tuple typle called
Person
, which takes two arguments, first_name and last_name.>>> from privex.helpers import dictable_namedtuple >>> Person = dictable_namedtuple('Person', 'first_name last_name')
Now we’ll create an instance of
Person
calledjohn
. These instances look like normalnamedtuple
’s, and should be generally compatible with any functions/methods which deal with named tuple’s.>>> john = Person('John', 'Doe') # Alternatively you can do Person(first_name='John', last_name='Doe') >>> john Person(first_name='John', last_name='Doe')
Unlike a normal
namedtuple
type instance, we can access fields by attribute (.first_name
), index ([0]
), AND by item/key name (['last_name']
).>>> john.first_name 'John' >>> john[0] 'John' >>> john['last_name'] 'Doe'
Another potentially useful feature, is that you can also update / create new fields, via your preferred method of field notation (other than numbered indexes, since those don’t include a field name):
>>> john['middle_name'] = 'Davis' >>> john.middle_name = 'Davis'
We can also convert
john
into a standard dictionary, with a simpledict(john)
cast. You can see that the new field we added (middle_name
) is present in the dictionary serialized format.>>> dict(john) {'first_name': 'John', 'last_name': 'Doe', 'middle_name': 'Davis'}
- Parameters
- Key bool read_only
(Default:
False
) If set toTrue
, the outputted dictable_namedtuple instance will not allow new fields to be created via attribute / item setting.- Return Type[namedtuple] dict_namedtuple
A dict_namedtuple type/class which can be instantiated with the given
field_names
via positional or keyword args.
-
privex.helpers.collections.
generate_class
(name: str, qualname: str = None, module: str = None, bases: Union[tuple, list] = None, attributes: Dict[str, Any] = None, **kwargs) → Any[source]¶ A small helper function for dynamically generating classes / types.
Basic usage
Generating a simple class, with an instance constructor, a basic instance method, and an instance factory classmethod:
>>> import random >>> from privex.helpers.collections import generate_class >>> def hello_init(self, example: int): ... self.example = example ... >>> Hello = generate_class( ... 'Hello', module='hello', ... attributes=dict( ... __init__=hello_init, lorem=lambda self: self.example * 10, ... make_hello=classmethod(lambda cls: cls(random.randint(1, 100))) ... ) ... ) ... >>> h = Hello(123) >>> h.lorem() 1230 >>> j = Hello.make_hello() >>> j.example 77 >>> j.lorem() 770
Generating a child class which inherits from an existing class (the parent(s) can also be a generated classes):
>>> World = generate_class( ... 'World', module='hello', bases=(Hello,), attributes=dict(ipsum=lambda self: float(self.example) / 3) ... ) >>> w = World(130) >>> w.lorem() 1300 >>> w.ipsum() 43.333333333333336
- Parameters
name (str) – The name of the class, e.g.
Hello
qualname (str) – (Optional) The qualified name of the class, e.g. for nested classes
A -> B -> C
, classC
would have the__name__
:C
and__qualname__
:A.B.C
module (str) – (Optional) The module the class should appear to belong to (sets
__module__
)bases (tuple|list) – (Optional) A tuple or list of “base” / “parent” classes for inheritance.
attributes (dict) – (Optional) A dictionary of attributes to add to the class. (can include constructor + methods)
kwargs –
- Returns
-
privex.helpers.collections.
generate_class_kw
(name: str, qualname: str = None, module: str = None, bases: Union[tuple, list] = None, **kwargs) → Type[source]¶ Same as
generate_class()
, but instead of adict
attributes
parameter - all additional keyword arguments will be used forattributes
Example:
>>> def lorem_init(self, ipsum=None): ... self._ipsum = ipsum ... >>> Lorem = generate_class_kw('Lorem', ... __init__=lorem_init, hello=staticmethod(lambda: 'world'), ... ipsum=property(lambda self: 0 if self._ipsum is None else self._ipsum) ... ) >>> l = Lorem() >>> l.ipsum() 0 >>> l.hello() 'world'
-
privex.helpers.collections.
is_namedtuple
(*objs) → bool[source]¶ Takes one or more objects as positional arguments, and returns
True
if ALL passed objects are namedtuple instancesExample usage
First, create or obtain one or more NamedTuple objects:
>>> from collections import namedtuple >>> Point, Person = namedtuple('Point', 'x y'), namedtuple('Person', 'first_name last_name') >>> pt1, pt2 = Point(1.0, 5.0), Point(2.5, 1.5) >>> john = Person('John', 'Doe')
We’ll also create a
tuple
,dict
, andstr
to show they’re detected as invalid:>>> normal_tuple, tst_dict, tst_str = (1, 2, 3,), dict(hello='world'), "hello world"
First we’ll call
is_namedtuple()
with our Person NamedTuple objectjohn
:>>> is_namedtuple(john) True
As expected, the function shows
john
is in-fact a named tuple.Now let’s try it with our two Point named tuple’s
pt1
andpt2
, plus our Person named tuplejohn
.>>> is_namedtuple(pt1, john, pt2) True
Since all three arguments were named tuples (even though pt1/pt2 and john are different types), the function returns
True
.Now we’ll test with a few objects that clearly aren’t named tuple’s:
>>> is_namedtuple(tst_str) # Strings aren't named tuples. False >>> is_namedtuple(normal_tuple) # A plain bracket tuple is not a named tuple. False >>> is_namedtuple(john, tst_dict) # ``john`` is a named tuple, but a dict isn't, thus False is returned. False
Original source: https://stackoverflow.com/a/2166841
- Parameters
objs (Any) – The objects (as positional args) to check whether they are a NamedTuple
- Return bool is_namedtuple
True
if all passedobjs
are named tuples.
-
privex.helpers.collections.
make_dict_tuple
(typename, field_names, *args, **kwargs)[source]¶ Generates a
collections.namedtuple()
type, with added / modified methods injected to make it into adictable_namedtuple
.Note: You probably want to be using
dictable_namedtuple()
instead of calling this directly.
-
privex.helpers.collections.
subclass_dictable_namedtuple
(named_type: type, typename=None, module=None, **kwargs) → type[source]¶ Convert an existing
collections.namedtuple()
type into a dictable_namedtuple.If you have an INSTANCE of a type (e.g. it has data attached), use
convert_dictable_namedtuple()
Example:
>>> from collections import namedtuple >>> from privex.helpers import subclass_dictable_namedtuple >>> # Create a namedtuple type called 'Person' >>> orig_Person = namedtuple('Person', 'first_name last_name') >>> # Convert the 'Person' type into a dictable_namedtuple >>> Person = subclass_dictable_namedtuple(orig_Person) >>> john = Person('John', 'Doe') # Create an instance of this dictable_namedtuple Person >>> john['middle_name'] = 'Davis'
- Parameters
named_type (type) – A NamedTuple type returned from
collections.namedtuple()
typename (str) – Optionally, you can change the name of your type, e.g. if you provide a
Person
class type, but you set this toMan
, then this will return aMan
class type.module (str) – Optionally, you can change the module that the type class belongs to. Otherwise it will inherit the module path from
named_type
.
- Key bool read_only
(Default:
False
) If set toTrue
, the outputted dictable_namedtuple type will not allow new fields to be created via attribute / item setting.- Return type dictable_namedtuple
Your
named_type
converted into a dictable_namedtuple type class.
Module Contents¶
Functions
copy_class
(obj[, name, deep_copy, deep_private])Attempts to create a full copy of a
type
or class, severing most object pointers such as attributes containing adict
/list
, along with classes or instances of classes.
convert_dictable_namedtuple
(nt_instance[, …])Convert an existing
collections.namedtuple()
instance into a dictable_namedtuple instance.
dictable_namedtuple
(typename, field_names, …)Creates a dictable_namedtuple type for instantiation (same usage as
collections.namedtuple()
) - unlike namedtuple, dictable_namedtuple instances allow item (dict-like) field access, support writing and can be painlessly converted into dictionaries viadict(my_namedtuple)
.
is_namedtuple
(*objs)Takes one or more objects as positional arguments, and returns
True
if ALL passed objects are namedtuple instances
make_dict_tuple
(typename, field_names, …)Generates a
collections.namedtuple()
type, with added / modified methods injected to make it into adictable_namedtuple
.
subclass_dictable_namedtuple
(named_type[, …])Convert an existing
collections.namedtuple()
type into a dictable_namedtuple.Classes
Dictable
()A small abstract class for use with Python 3.7 dataclasses.
This is a base class for use with Python 3.7+
dataclass
‘s, designed to make dataclasses more interoperable with existing dictionaries, and allows them to be used like dictionaries, similar toDictable
, but more powerful / flexible.A very simple
dict
wrapper, which allows you to read and write dictionary keys using attributes (dot notation) PLUS standard item (key / square bracket notation) access.Ordered version of
DictObject
- dictionary with attribute access.alias of
builtins.dict
Dictable
()A small abstract class for use with Python 3.7 dataclasses.
Mocker
(modules, attributes, *args, **kwargs)This mock class is designed to be used either to act as a stand-in “noop” (no operation) object, which could be used either as a drop-in replacement for a failed module / class import, or for certain unit tests.
Ordered version of
DictObject
- dictionary with attribute access.