Python Dictionaries on Steroids with Python-benedict

2023-03-15

Python Development

Python-benedict is a powerful Python library that extends the capabilities of Python's built-in dictionary (or dict) class. The library enables you to easily access, search, and modify nested values, manipulate and transform data, and convert various formats to and from dictionaries. As the dictionary is one of the most commonly used data structures in Python, this library could be a potential boost to productivity.

This library is a dict subclass with keylist/keypath support, I/O shortcuts (base64, csv, ini, json, pickle, plist, query-string, toml, xls, xml, yaml) and many utilities... for humans, obviously.

Features

100% backward-compatible, you can safely wrap existing dictionaries.
NEW Keyattr support for get/set items using keys as attributes.
Keylist support using list of keys as key.
Keypath support using keypath-separator (dot syntax by default).
Keypath list-index support (also negative) using the standard [n] suffix.
Normalized I/O operations with most common formats: base64, csv, ini, json, pickle, plist, query-string, toml, xls, xml, yaml.
Multiple I/O operations backends: file-system (read/write), url (read-only), s3 (read/write).
Many utility and parse methods to retrieve data as needed (check the API section).
Well tested. ;)

Installation

If you want to install everything:

pip install "python-benedict[all]"

alternatively you can install the main package:

run pip install python-benedict, then install only the optional requirements you need.

Optional Requirements

Here the hierarchy of possible installation targets available when running pip install "python-benedict[...]" (each target installs all its sub-targets):

[all]
[io]
[toml]
[xls]
[xml]
[yaml]
[s3]

Usage

Basics

benedict is a dict subclass, so it is possible to use it as a normal dictionary (you can just cast an existing dict).

from benedict import benedict

# create a new empty instance
d = benedict()

# or cast an existing dict
d = benedict(existing_dict)

# or create from data source (filepath, url or data-string) in a supported format:
# Base64, CSV, JSON, TOML, XML, YAML, query-string
d = benedict("https://localhost:8000/data.json", format="json")

# or in a Django view
params = benedict(request.GET.items())
page = params.get_int("page", 1)

Keyattr

It is possible to get/set items using keys as attributes (dotted notation).

from benedict import benedict

d = benedict()
d.profile.firstname = "Fabio"
d.profile.lastname = "Caccamo"
print(d) 
# output:
{'profile': {'firstname': 'Fabio', 'lastname': 'Caccamo'}}

Disable keyattr functionality

You can disable the keyattr functionality passing keyattr_enabled=False in the constructor.

d = benedict(existing_dict, keyattr_enabled=False)

You can disable the keyattr functionality using the getter/setter property.

d.keyattr_enabled = False

Even if this feature is very useful, it has some obvious limitations: it works only for string keys that are unprotected (not starting with an _) and that don't clash with the currently supported methods names.

Keylist

Wherever a key is used, it is possible to use also a list (or a tuple) of keys.

from benedict import benedict

d = benedict()

# set values by keys list
d["profile", "firstname"] = "Fabio"
d["profile", "lastname"] = "Caccamo"
print(d) 
# output:
{'profile': {'firstname': 'Fabio', 'lastname': 'Caccamo'}}

print(d["profile"]) 
# output:
{'firstname': 'Fabio', 'lastname': 'Caccamo'}

# check if keypath exists in dict
print(["profile", "lastname"] in d) 
# output:
True

# delete value by keys list
del d["profile", "lastname"]
print(d["profile"])
# output:
{'firstname': 'Fabio'}

Keypath

. is the default keypath separator.

If you cast an existing dict and its keys contain the keypath separator a ValueError will be raised. In this case you should use a custom keypath separator or disable keypath functionality.

Custom keypath separator

You can customize the keypath separator passing the keypath_separator argument in the constructor. If you pass an existing dict to the constructor and its keys contain the keypath separator an Exception will be raised.

d = benedict(existing_dict, keypath_separator="/")

Change keypath separator

You can change the keypath_separator at any time using the getter/setter property. If any existing key contains the new keypath_separator an Exception will be raised.

d.keypath_separator = "/"

Disable keypath functionality

You can disable the keypath functionality passing keypath_separator=None in the constructor.

d = benedict(existing_dict, keypath_separator=None)

You can disable the keypath functionality using the getter/setter property.

d.keypath_separator = None

Examples

from benedict import benedict

data = {
    "id_1": {
        "name": "John",
        "surname": "Doe",
        "age": 25,
        "skills": {
            "programming": {
                "Python": "5",
                "Java": "4",
            },
        },
        "languages": ["English", "French", "Spanish"],
    },
    "id_2": {
        "name": "Bob",
        "surname": "Marley",
        "age": 29,
        "skills": {
            "programming": {
                "Python": "4",
                "JavaScript": "4",
            },
        },
        "languages": ["English", "Portugal", "Italy"],
    }
}

data = benedict(data)


[data[key, "name"] for key in data ]
# output:
['John', 'Bob']

[(data[key, "name"], data[key, "skills"]) for key in data ]
# output:
[('John', {'programming': {'Python': '5', 'Java': '4'}}), ('Bob', {'programming': {'Python': '4', 'JavaScript': '4'}})]



# keypaths
# Return a list of all keypaths in the dict.
# If indexes is True, the output will include list values indexes.
# k = d.keypaths(indexes=False)
data.keypaths()
# output:
['id_1', 'id_1.age', 'id_1.languages', 'id_1.name', 'id_1.skills', 'id_1.skills.programming', 'id_1.skills.programming.Java', 'id_1.skills.programming.Python', 'id_1.surname', 'id_2', 'id_2.age', 'id_2.languages', 'id_2.name', 'id_2.skills', 'id_2.skills.programming', 'id_2.skills.programming.JavaScript', 'id_2.skills.programming.Python', 'id_2.surname']


# filter
# Return a filtered dict using the given predicate function.
# Predicate function receives key, value arguments and should return a bool value.
# predicate = lambda k, v: v is not None
# f = d.filter(predicate)
data.filter(lambda key, value: value.age == 29)
# output:
{'id_2': {'name': 'Bob', 'surname': 'Marley', 'age': 29, 'skills': {'programming': {'Python': '4', 'JavaScript': '4'}}, 'languages': ['English', 'Portugal', 'Italy']}}
data.filter(lambda key, value: "Java" in value.skills.programming)
# output:
{'id_1': {'name': 'John', 'surname': 'Doe', 'age': 25, 'skills': {'programming': {'Python': '5', 'Java': '4'}}, 'languages': ['English', 'French', 'Spanish']}}


# flatten
# Return a new flattened dict using the given separator to join nested dict keys to flatten keypaths.
# f = d.flatten(separator="_")
print(data.flatten(separator="/").dump())
# output:
{
    "id_1/age": 25,
    "id_1/languages": [
        "English",
        "French",
        "Spanish"
    ],
    "id_1/name": "John",
    "id_1/skills/programming/Java": "4",
    "id_1/skills/programming/Python": "5",
    "id_1/surname": "Doe",
    "id_2/age": 29,
    "id_2/languages": [
        "English",
        "Portugal",
        "Italy"
    ],
    "id_2/name": "Bob",
    "id_2/skills/programming/JavaScript": "4",
    "id_2/skills/programming/Python": "4",
    "id_2/surname": "Marley"
}


# search
# Search and return a list of items (dict, key, value, ) matching the given query.
# r = d.search("hello", in_keys=True, in_values=True, exact=False, case_sensitive=False)
data.search("Java", in_keys=True, in_values=True, exact=False, case_sensitive=False)
# output:
[({'Python': '5', 'Java': '4'}, 'Java', '4'), ({'Python': '4', 'JavaScript': '4'}, 'JavaScript', '4')]
data.search("Java", in_keys=True, in_values=True, exact=True, case_sensitive=False)
# output:
[({'Python': '5', 'Java': '4'}, 'Java', '4')]


# subset
# Return a dict subset for the given keys.
# It is possible to pass a single key or more keys (as list or *args).
# s = d.subset(["firstname", "lastname", "email"])
data["id_2"].subset(["skills"])
# output:
{'skills': {'programming': {'Python': '4', 'JavaScript': '4'}}}
data.id_2.subset(["skills"])
# output:
{'skills': {'programming': {'Python': '4', 'JavaScript': '4'}}}