.. _handlers: ======== Handlers ======== Handlers are what provide functionality to serialize to and deserialize from various file formats. All of the supported file formats should be attribute-value pair data exchange formats that can be serialized to and from dictionaries. Whenever a new config instance is created some generic serialization and deserialization methods are *magically* created based on what handlers are currently available. The exposed methods are the following: - ``config_instance.dump_x(file_object)`` - *dumps an instance to a file object* - ``config_instance.dumps_x()`` - *dumps the content to a string* - ``ConfigClass.load_x(file_object)`` - *loads an instance from a file object* - ``ConfigClass.loads_x(string)`` - *loads an instance from a string* where ``x`` is the generic name of the handler (e.g. ``json``, ``ini``, ``pickle``, ``toml``, etc...). I will commonly refer to these methods as the **config serialization methods**. .. important:: Since handlers such as :class:`~.handlers.JSONHandler` have support for multiple serializer libraries, a keyword ``prefer`` is supplied to all the config serialization methods. You can use this keyword to indicate what library you wish to use for that specific dump or load. .. code-block:: python # if you have python-rapidjson installed but would prefer to use the builtin json config_instance.dumps_json(prefer="json") The value of the ``prefer`` keyword is the **module name** not the package name. For example `pyyaml `_'s package name is ``pyyaml`` but it's importable module name is ``yaml``. So you would put ``prefer="yaml"`` rather than ``prefer="pyyaml"``. For the following documentation I will be using the following config instance to showcase the handler's serialization. .. code-block:: python from typing import List, Dict from enum import Enum import file_config class Status(Enum): STOPPED = 0 STARTED = 1 @file_config.config class ProjectConfig(object): @file_config.config class Dependency(object): name = file_config.var(str, min=1) version = file_config.var(file_config.Regex(r"^v\d+$")) name = file_config.var(str, min=1) type_ = file_config.var(str, name="type", required=False) keywords = file_config.var(List[str], min=0, max=10) status = file_config.var(Status) dependencies = file_config.var(Dict[str, Dependency]) config_instance = ProjectConfig( name="My Project", type_="personal-project", keywords=["example", "test"], status=Status.STOPPED, dependencies={ "a-dependency": ProjectConfig.Dependency(name="A Dependency", version="v12") }, ) .. _handlers.json: JSON ==== `JSON `_ is probably the most popular, supported, and straightforward data exchange formats available right now. Available formatting options for JSON serializers are... - ``indent=2`` - *the number of spaces to use for indentation* - ``sort_keys=True`` - *sorts keys alphabetically in the resulting json* .. _handlers.json.json: :mod:`json` ----------- Uses the builtin :mod:`json` module for serialization. .. code-block:: python config_instance.dumps_json(prefer="json", indent=2) .. code-block:: json { "name": "My Project", "type": "personal-project", "keywords": [ "example", "test" ], "status": 0, "dependencies": { "a-dependency": { "name": "A Dependency", "version": "v12" } } } .. _handlers.json.python-rapidjson: `python-rapidjson `_ ---------------------------------------------------------------- Compatability with ``python-rapidjson`` requires the installation of it as an extra... .. code-block:: bash pipenv install file-config[python-rapidjson] Serialization is the same as the default :ref:`handlers.json.json` handler... .. code-block:: python config_instance.dumps_json(prefer="rapidjson", indent=2) .. code-block:: json { "name": "My Project", "type": "personal-project", "keywords": [ "example", "test" ], "status": 0, "dependencies": { "a-dependency": { "name": "A Dependency", "version": "v12" } } } .. _handlers.json.ujson: `ujson `_ ------------------------------------------ In my opinion people shouldn't be using ``ujson`` since they don't follow the JSON spec and are thus incompatible with the default :mod:`json` module for many edge cases. But I know that lots of packages still depend on it (*for unknown reasons*). Support for ``ujson`` requires it to be installed as an extra... .. code-block:: bash pipenv install file-config[ujson] Usage is similar, however notice the resulting json lacks whitespace between key value pairs (a quirk of ``ujson``). .. code-block:: python config_instance.dumps_json(prefer="ujson", indent=2) .. code-block:: json { "name":"My Project", "type":"personal-project", "keywords":[ "example", "test" ], "status":0, "dependencies":{ "a-dependency":{ "name":"A Dependency", "version":"v12" } } } .. _handlers.ini: INI === `INI `_ is another popular configuration file format. Although it lacks features available in other file formats people still tend to use it over better solutions (:ref:`handlers.toml` following a very similar specification). .. important:: **INI does not support arrays of dictionaries.** Doing so would break the specification and be difficult to read. If your resulting dictionary from ``file_config.to_dict(config_instance)`` has an array containing dictionaries the config serialization methods for ini serialization will raise a :class:`ValueError`. If you need to structure your data this way, please switch to using :ref:`handlers.toml` as that is one of their key features. Available formatting options for INI serializers are... - ``root="root"`` - *the name of the root section of the resulting ini* - ``delimiter=":"`` - *the delimiter to use to indicate nested dictionaries* - ``empty_sections=True`` - *allows empty ini sections to exist* .. _handlers.ini.configparser: :mod:`configparser` ------------------- Uses the builtin :mod:`configparser` module for serialization. A custom :class:`~.contrib.ini_parser.INIParser` is used since the default ``configparser`` module is pretty lackluster. .. code-block:: python config_instance.dumps_ini(prefer="configparser", root="root") .. code-block:: ini [root] name = "My Project" type = personal-project keywords = example test status = 0 [root:dependencies:a-dependency] name = "A Dependency" version = v12 .. _handlers.pickle: Pickle ====== You really shouldn't ever be serializing and storing things to :mod:`pickle` syntax since it has pretty serious security flaws with how it is loaded back in. Anyway, here is how you can do it in ``file_config``. **There are no format options available for the pickle handler.** .. _handlers.pickle.pickle: :mod:`pickle` ------------- Uses the builtin :mod:`pickle` module for serialization. .. code-block:: python config_instance.dumps_pickle(prefer="pickle") .. code-block:: bytes \x80\x04\x95\xc6\x00\x00\x00\x00\x00\x00\x00\x8c\x0bcollections\x94\x8c\x0bOrderedDict\x94\x93\x94)R\x94(\x8c\x04name\x94\x8c\nMy Project\x94\x8c\x04type\x94\x8c\x10personal-project\x94\x8c\x08keywords\x94]\x94(\x8c\x07example\x94\x8c\x04test\x94e\x8c\x06status\x94K\x00\x8c\x0cdependencies\x94}\x94\x8c\x0ca-dependency\x94h\x02)R\x94(h\x04\x8c\x0cA Dependency\x94\x8c\x07version\x94\x8c\x03v12\x94usu. .. _handlers.toml: TOML ==== There are a thousand libraries for parsing `toml `_ and they are all terrible. Maybe toml is just poorly designed but every single toml parsing library I've used (in Python) either doesn't fully parse toml correctly or has some odd quirks that make it hard to work with. Nevertheless, here are the three best libraries that *currently* exist for parsing toml. The format options available to toml are the following: - ``inline_tables=["dependences.*"]`` - *a glob pattern for inlining tables* .. _handlers.toml.tomlkit: `tomlkit `_ ---------------------------------------------- Using tomlkit requires it to be installed as an extra... .. code-block:: bash pipenv install file-config[tomlkit] Usage is just as you might expect... .. code-block:: python config_instance.dumps_toml(prefer="tomlkit", inline_tables=["dependencies.*"]) .. code-block:: toml name = "My Project" type = "personal-project" keywords = ["example", "test"] status = 0 [dependencies] a-dependency = {name = "A Dependency",version = "v12"} .. _handlers.toml.toml: `toml `_ ---------------------------------------- Using toml requires it to be installed as an extra... .. code-block:: bash pipenv install file-config[toml] Usage is just the same as :ref:`handlers.toml.tomlkit`... .. code-block:: python config_instance.dumps_toml(prefer="toml", inline_tables=["dependencies.*"]) .. code-block:: toml name = "My Project" type = "personal-project" keywords = [ "example", "test",] status = 0 [dependencies] a-dependency = { name = "A Dependency", version = "v12" } .. _handlers.toml.pytoml: `pytoml `_ -------------------------------------------- Using pytoml requires it to be installed as an extra... .. code-block:: bash pipenv install file-config[pytoml] Pytoml does not support serializing inline tables in any easy manner. So the ``inline_tables`` keyword won't be applied when dumping with `pytoml`_. .. code-block:: python config_instance.dumps_toml(prefer="pytoml") .. code-block:: toml name = "My Project" type = "personal-project" keywords = ["example", "test"] status = 0 [dependencies] [dependencies.a-dependency] name = "A Dependency" version = "v12" .. _handlers.yaml: YAML ==== `Yaml `_ is a data exchange langauge used by many projects (Docker, TravisCI, etc...) for simple configuration files. **There are no format options for yaml serialization.** .. _handlers.yaml.pyyaml: `pyyaml (yaml) `_ --------------------------------------------------- Using pyyaml requires it to be installed as an extra... .. code-block:: bash pipenv install file-config[pyyaml] Usage is straightforward... .. code-block:: python config_instance.dumps_yaml(prefer="yaml") .. code-block:: yaml name: My Project type: personal-project keywords: [example, test] status: 0 dependencies: a-dependency: name: A Dependency version: v12 .. _handlers.message-pack: Message Pack ============ `MessagePack `_ is a byte sized json format which retains the same structure as json. It's valuable for quick and fast json streams over network protocols. **There are no format options for msgpack.** .. _handlers.message-pack.msgpack: `msgpack `_ ---------------------------------------------- Using msgpack requires it to be installed as an extra... .. code-block:: bash pipenv install file-config[msgpack] Usage is just as you would expect... .. code-block:: python config_instance.dumps_msgpack(prefer="msgpack") .. code-block:: bytes \x85\xa4name\xaaMy Project\xa4type\xb0personal-project\xa8keywords\x92\xa7example\xa4test\xa6status\x00\xacdependencies\x81\xaca-dependency\x82\xa4name\xacA Dependency\xa7version\xa3v12 .. _handlers.xml: XML === `XML `_ is an older data exchange format that follows a nested tag-attribute type structure. A custom :class:`~.contrib.xml_parser.XMLParser` is used since the many dictionary to xml helper packages out there are not reflective. The format options available to xml are the following: - ``root="root"`` - *the name of the root element in the resulting xml* - ``pretty=True`` - *indicates that the resulting xml should be pretty formatted* - ``xml_declaration=True`` - *indicates that the xml header should be added* - ``encoding="utf-8"`` - *the encoding to use for the resulting xml document* .. _handlers.xml.lxml: `lxml `_ ---------------------------------------- Using XML requires ``lxml`` to be installed as an extra... .. code-block:: bash pipenv install file-config[lxml] Usage is straightforward... .. code-block:: python config_instance.dumps_xml(prefer="lxml", pretty=True, xml_declaration=True) .. code-block:: xml My Project personal-project example test 0 A Dependency v12