Skip navigation links
MySQL HA/Scalability Guide :: 4 Using MySQL with memcached :: 4.3 memcached Interfaces :: 4.3.3 Using MySQL and memcached with Python
4.3.4 Using MySQL and memcached with PHP »
Section Navigation      [Toggle]

4.3.3. Using MySQL and memcached with Python

The Python memcache module interfaces to memcached servers, and is written in pure python (that is, without using one of the C APIs). You can download and install a copy from Python Memcached.

To install, download the package and then run the Python installer: 

python setup.py install
running install
running bdist_egg
running egg_info
creating python_memcached.egg-info
...
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing python_memcached-1.43-py2.4.egg
creating /usr/lib64/python2.4/site-packages/python_memcached-1.43-py2.4.egg
Extracting python_memcached-1.43-py2.4.egg to /usr/lib64/python2.4/site-packages
Adding python-memcached 1.43 to easy-install.pth file
Installed /usr/lib64/python2.4/site-packages/python_memcached-1.43-py2.4.egg
Processing dependencies for python-memcached==1.43
Finished processing dependencies for python-memcached==1.43

Once installed, the memcache module provides a class-based interface to your memcached servers. Serialization of Python structures is handled by using the Python cPickle or pickle modules.

To create a new memcache interface, import the memcache module and create a new instance of the memcache.Client class: 

import memcache
memc = memcache.Client(['127.0.0.1:11211'])

The first argument should be an array of strings containing the server and port number for each memcached instance you want to use. You can enable debugging by setting the optional debug parameter to 1.

By default, the hashing mechanism used is crc32. This provides a basic module hashing algorithm for selecting among multiple servers. You can change the function used by setting the value of memcache.serverHashFunction to the alternate function you want to use. For example: 

from zlib import adler32
memcache.serverHashFunction = adler32

Once you have defined the servers to use within the memcache instance, the core functions provide the same functionality as in the generic interface specification. A summary of the supported functions is provided in the following table.

Python memcache Function Equivalent to
get() Generic get()
get_multi(keys) Gets multiple values from the supplied array of keys. Returns a hash reference of key/value pairs.
set() Generic set()
set_multi(dict [, expiry [, key_prefix]]) Sets multiple key/value pairs from the supplied dict.
add() Generic add()
replace() Generic replace()
prepend(key, value [, expiry]) Prepends the supplied value to the value of the existing key.
append(key, value [, expiry[) Appends the supplied value to the value of the existing key.
delete() Generic delete()
delete_multi(keys [, expiry [, key_prefix]] ) Deletes all the keys from the hash matching each string in the array keys.
incr() Generic incr()
decr() Generic decr()

Note

Within the Python memcache module, all the *_multi()functions support an optional key_prefix parameter. If supplied, then the string is used as a prefix to all key lookups. For example, if you call: 

memc.get_multi(['a','b'], key_prefix='users:')

The function will retrieve the keys users:a and users:b from the servers.

An example showing the storage and retrieval of information to a memcache instance, loading the raw data from MySQL, is shown below: 

import sys
import MySQLdb
import memcache
memc = memcache.Client(['127.0.0.1:11211'], debug=1);
try:
    conn = MySQLdb.connect (host = "localhost",
                            user = "sakila",
                            passwd = "password",
                            db = "sakila")
except MySQLdb.Error, e:
     print "Error %d: %s" % (e.args[0], e.args[1])
     sys.exit (1)
popularfilms = memc.get('top5films')
if not popularfilms:
    cursor = conn.cursor()
    cursor.execute('select film_id,title from film order by rental_rate desc limit 5')
    rows = cursor.fetchall()
    memc.set('top5films',rows,60)
    print "Updated memcached with MySQL data"
else:
    print "Loaded data from memcached"
    for row in popularfilms:
        print "%s, %s" % (row[0], row[1])

When executed for the first time, the data is loaded from the MySQL database and stored to the memcached server. 

shell> python memc_python.py
Updated memcached with MySQL data

The data is automatically serialized using cPickle/pickle. This means when you load the data back from memcached, you can use the object directly. In the example above, the information stored to memcached is in the form of rows from a Python DB cursor. When accessing the information (within the 60 second expiry time), the data is loaded from memcached and dumped: 

shell> python memc_python.py
Loaded data from memcached
2, ACE GOLDFINGER
7, AIRPLANE SIERRA
8, AIRPORT POLLOCK
10, ALADDIN CALENDAR
13, ALI FOREVER

The serialization and deserialization happens automatically, but be aware that serialization of Python data may be incompatible with other interfaces and languages. You can change the serialization module used during initialization, for example to use JSON, which will be more easily exchanged.