Introduction to bmemcached

A pure python module (thread safe) to access memcached via it's binary with SASL auth support.

The main purpose of this module it to be able to communicate with memcached using binary protocol and support authentication, so it can work with Heroku for example.

Latest compiled docs on Read The Docs here.

Installing

Use pip or easy_install.

pip install python-binary-memcached

Using

import bmemcached
client = bmemcached.Client(('127.0.0.1:11211', ), 'user',
                            'password')
client.set('key', 'value')
print(client.get('key'))

Using it with distributed keys

import bmemcached
client = bmemcached.DistributedClient(
    ('127.0.0.1:11211', ), 'user', 'password'
)
client.set('key', 'value')
print(client.get('key'))

Using it with raw data

By default, the client keeps the type of each value in the memcached flags field. Some applications already use the flags field for their own data, and a default client then fails to read their keys. Set raw_data=True to read and write those keys.

import bmemcached
client = bmemcached.Client(('127.0.0.1:11211', ), raw_data=True)
client.set('key', b'value')
print(client.get('key'))  # b'value'

In raw data mode:

• A read returns the stored bytes value. The client ignores the flags field.
• A write accepts only bytes values. Other types raise TypeError.
• The client does not compress values. The default compress_level=-1 means no compression, and a compress_level greater than 0 raises MemcachedException.

DistributedClient accepts the same raw_data argument.

Warning

A write in raw data mode sets the flags field of the key to 0. This overwrites the flags that another application set on the key.

Security

The client uses pickle to store Python objects. This is a deliberate choice. The client is for private caches that hold trusted data. Other Python memcached clients, for example pylibmc and python-memcached, also unpickle values by default.

Each item keeps its type in the memcached flags field. On a read, the client uses this field to select the decoder. If the object flag is set, the client unpickles the value.

Warning

Unpickling runs code that the data defines. A person who can write to your memcached servers can run code in every process that reads from them. Use the default settings only when your own applications are the only writers to the cache.

To protect the cache:

• Do not expose memcached to the internet or to a network that you do not trust.
• Turn on SASL authentication on the server. Then give the username and the password to the client.
• Use TLS on a network that you do not control. Without TLS, SASL PLAIN sends the password as clear text. Give an ssl.SSLContext to the client in tls_context.
• Do not share a memcached server with applications that you do not trust.

If the cache can contain data that you do not trust, use one of these options:

• Set raw_data=True. The client then reads and writes only bytes, and it never unpickles a value. See "Using it with raw data" above.
• Give the client an unpickler that accepts only the classes that you expect. The client calls unpickler(file).load() for each object value.

import pickle

import bmemcached


class RestrictedUnpickler(pickle.Unpickler):
    ALLOWED = {
        ("builtins", "set"),
        ("datetime", "datetime"),
        # Pickle protocols 0 to 2 store bytes, for example the state of a
        # datetime, through this function.
        ("_codecs", "encode"),
    }

    def find_class(self, module, name):
        # Pickle protocols 0 to 2 write the Python 2 module name.
        if module == "__builtin__":
            module = "builtins"
        if (module, name) not in self.ALLOWED:
            raise pickle.UnpicklingError(f"{module}.{name} is not allowed")
        return super().find_class(module, name)


client = bmemcached.Client(('127.0.0.1:11211', ), unpickler=RestrictedUnpickler)

The values dict, list, tuple, str, int, float, bool and None need no entry in the list. With the default pickle protocol 0, bytes inside an object needs the _codecs.encode entry. Add only data classes to the list. A class with side effects in its constructor gives an attacker a way to run code.

Testing

python-binary-memcached unit tests are found in the test/ directory and are designed to be run using pytest. pytest will discover the tests automatically, so all you have to do is:

$ pytest
...
170 passed in 4.43 seconds

This runs the tests with the default Python interpreter.

You can also verify that the tests pass on other supported Python interpreters. For this we use tox, which will automatically create a virtualenv for each supported Python version and run the tests. For example:

$ tox
...
py310: commands succeeded
py311: commands succeeded
py312: commands succeeded
ERROR:  py313: InterpreterNotFound: python3.13
py314: commands succeeded

You may not have all the required Python versions installed, in which case you will see one or more InterpreterNotFound errors.

Using with Django

If you want to use it with Django, go to django-bmemcached to get a Django backend.

Tests Status