API Documentation¶

Configuration¶

sherlock can be globally configured to set a lot of defaults using the sherlock.configure() function. The configuration set using sherlock.configure() will be used as the default for all the lock objects.

sherlock.configure(**kwargs)[source]¶

Set basic global configuration for sherlock.

Parameters:

backend – global choice of backend. This backend will be used for managing locks by sherlock.Lock class objects.
client – global client object to use to connect with backend store. This client object will be used to connect to the backend store by sherlock.Lock class instances. The client object must be a valid object of the client library. If the backend has been configured using the backend parameter, the custom client object must belong to the same library that is supported for that backend. If the backend has not been set, then the custom client object must be an instance of a valid supported client. In that case, sherlock will set the backend by introspecting the type of provided client object.
namespace (str) – provide global namespace
expire (float) – provide global expiration time. If expicitly set to None, lock will not expire.
timeout (float) – provide global timeout period
retry_interval (float) – provide global retry interval

Basic Usage:

>>> import sherlock
>>> from sherlock import Lock
>>>
>>> # Configure sherlock to use Redis as the backend and the timeout for
>>> # acquiring locks equal to 20 seconds.
>>> sherlock.configure(timeout=20, backend=sherlock.backends.REDIS)
>>>
>>> import redis
>>> redis_client = redis.StrictRedis(host='X.X.X.X', port=6379, db=1)
>>> sherlock.configure(client=redis_client)

Understanding the configuration parameters¶

sherlock.configure() accepts certain configuration patterns which are individually explained in this section. Some of these configurations are global configurations and cannot be overriden while others can be overriden at individual lock levels.

`backend`¶

The backend parameter allows you to set which backend you would like to use with the sherlock.Lock class. When set to a particular backend, instances of sherlock.Lock will use this backend for lock synchronization.

Basic Usage:

>>> import sherlock
>>> sherlock.configure(backend=sherlock.backends.REDIS)

Note

this configuration cannot be overriden at the time of creating a class object.

Available Backends¶

To set the backend global configuration, you would have to choose one from the defined backends. The defined backends are:

Etcd: sherlock.backends.ETCD
Memcache: sherlock.backends.MEMCACHE
Redis: sherlock.backends.REDIS

`client`¶

The client parameter allows you to set a custom clien object which sherlock can use for connecting to the backend store. This gives you the flexibility to connect to the backend store from the client the way you want. The provided custom client object must be a valid client object of the supported client libraries. If the global backend has been set, then the provided custom client object must be an instance of the client library supported by that backend. If the backend has not been set, then the custom client object must be an instance of a valid supported client. In this case, sherlock will set the backend by instrospecting the type of the provided client object.

The global default client object set using the client parameter will be used only by sherlock.Lock instances. Other Backend Specific Locks will either use the provided client object at the time of instantiating the lock object of their types or will default to creating a simple client object by themselves for their backend store, which will assume that their backend store is running on localhost.

Note

this configuration cannot be overriden at the time of creating a class object.

Example:

>>> import redis
>>> import sherlock
>>>
>>> # Configure just the backend
>>> sherlock.configure(backend=sherlock.backends.REDIS)
>>>
>>> # Configure the global client object. This sets the client for all the
>>> # locks.
>>> sherlock.configure(client=redis.StrictRedis())

And when the provided client object does not match the supported library for the set backend:

>>> import etcd
>>> import sherlock
>>>
>>> sherlock.configure(backend=sherlock.backends.REDIS)
>>> sherlock.configure(client=etcd.Client())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 148, in configure
    _configuration.update(**kwargs)
  File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 250, in update
    setattr(self, key, val)
  File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 214, in client
    self.backend['name']))
ValueError: Only a client of the redis library can be used when using REDIS as the backend store option.

And when the backend is not configured:

>>> import redis
>>> import sherlock
>>>
>>> # Congiure just the client, this will configure the backend to
>>> # sherlock.backends.REDIS automatically.
>>> sherlock.configure(client=redis.StrictRedis())

And when the client object passed as argument for client parameter is not a valid client object at all:

>>> import sherlock
>>> sherlock.configure(client=object())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 148, in configure
    _configuration.update(**kwargs)
  File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 250, in update
    setattr(self, key, val)
  File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 221, in client
    raise ValueError('The provided object is not a valid client'
ValueError: The provided object is not a valid client object. Client objects can only be instances of redis library's client class, python-etcd library's client class or pylibmc library's client class.

`namespace`¶

The namespace parameter allows you to configure sherlock to set keys for synchronizing locks with a namespace so that if you are using the same datastore for something else, the keys set by locks don’t conflict with your other keys set by your application.

In case of Redis and Memcached, the name of your locks are prepended with NAMESPACE_ (where NAMESPACE is the namespace set by you). In case of Etcd, a directory with the name same as the NAMESPACE you provided is created and the locks are created in that directory.

By default, sherlock does not namespace the keys set for locks.

Note

this configuration can be overriden at the time of creating a class object.

`expire`¶

This parameter can be used to set the expiry of locks. When set to None, the locks will never expire.

This parameter’s value defaults to 60 seconds.

Note

this configuration can be overriden at the time of creating a class object.

Example:

>>> import sherlock
>>> import time
>>>
>>> # Configure locks to expire after 2 seconds
>>> sherlock.configure(expire=2)
>>>
>>> lock = sherlock.Lock('my_lock')
>>>
>>> # Acquire the lock
>>> lock.acquire()
True
>>>
>>> # Sleep for 2 seconds to let the lock expire
>>> time.sleep(2)
>>>
>>> # Acquire the lock
>>> lock.acquire()
True

`timeout`¶

This parameter can be used to set after how much time should sherlock stop trying to acquire an already acquired lock.

This parameter’s value defaults to 10 seconds.

Note

this configuration can be overriden at the time of creating a class object.

Example:

>>> import sherlock
>>>
>>> # Configure locks to timeout after 2 seconds while trying to acquire an
>>> # already acquired lock
>>> sherlock.configure(timeout=2, expire=10)
>>>
>>> lock = sherlock.Lock('my_lock')
>>>
>>> # Acquire the lock
>>> lock.acquire()
True
>>>
>>> # Acquire the lock again and let the timeout elapse
>>> lock.acquire()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "sherlock/lock.py", line 170, in acquire
    'lock.' % self.timeout)
sherlock.lock.LockTimeoutException: Timeout elapsed after 2 seconds while trying to acquiring lock.

`retry_interval`¶

This parameter can be used to set after how much time should sherlock try to acquire lock after failing to acquire it. For example, a log has already been acquired, then when another lock object tries to acquire the same lock, it fails. This parameter sets the time interval for which we should sleep before retrying to acquire the lock.

This parameter can be set to 0 to continuously try acquiring the lock. But that will also mean that you are bombarding your datastore with requests one after another.

This parameter’s value defaults to 0.1 seconds (100 milliseconds).

Note

this configuration can be overriden at the time of creating a class object.

Generic Locks¶

sherlock provides generic locks that can be globally configured to use a specific backend, so that most of your application code does not have to care about which backend you are using for lock synchronization and makes it easy to change backend without changing a ton of code.

class sherlock.Lock(lock_name, **kwargs)[source]¶

A general lock that inherits global coniguration and provides locks with the configured backend.

Note

to use Lock class, you must configure the global backend to use a particular backend. If the global backend is not set, calling any method on instances of Lock will throw exceptions.

Basic Usage:

>>> import sherlock
>>> from sherlock import Lock
>>>
>>> sherlock.configure(sherlock.backends.REDIS)
>>>
>>> # Create a lock instance
>>> lock = Lock('my_lock')
>>>
>>> # Acquire a lock in Redis running on localhost
>>> lock.acquire()
True
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
True
>>>
>>> # Release the acquired lock
>>> lock.release()
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
False
>>>
>>> import redis
>>> redis_client = redis.StrictRedis(host='X.X.X.X', port=6379, db=2)
>>> sherlock.configure(client=redis_client)
>>>
>>> # Acquire a lock in Redis running on X.X.X.X:6379
>>> lock.acquire()
>>>
>>> lock.locked()
True
>>>
>>> # Acquire a lock using the with_statement
>>> with Lock('my_lock') as lock:
...     # do some stuff with your acquired resource
...     pass

Parameters:

lock_name (str) – name of the lock to uniquely identify the lock between processes.
namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
timeout (float) – set timeout to acquire lock
retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.

Note

this Lock object does not accept a custom lock backend store client object. It instead uses the global custom client object.

acquire(blocking=True)¶

Acquire a lock, blocking or non-blocking.

Parameters:	blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True.
Returns:	if the lock was successfully acquired or not
Return type:	bool

locked()¶

Return if the lock has been acquired or not.

Returns:	True indicating that a lock has been acquired ot a shared resource is locked.
Return type:	bool

release()¶: Release a lock.

Backend Specific Locks¶

sherlock provides backend specific Lock classes as well which can be optionally used to use different backend than a globally configured backend. These locks have the same interface and semantics as sherlock.Lock.

Redis based Locks¶

class sherlock.RedisLock(lock_name, **kwargs)[source]¶

Implementation of lock with Redis as the backend for synchronization.

Basic Usage:

>>> import redis
>>> import sherlock
>>> from sherlock import RedisLock
>>>
>>> # Global configuration of defaults
>>> sherlock.configure(expire=120, timeout=20)
>>>
>>> # Create a lock instance
>>> lock = RedisLock('my_lock')
>>>
>>> # Acquire a lock in Redis, global backend and client configuration need
>>> # not be configured since we are using a backend specific lock.
>>> lock.acquire()
True
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
True
>>>
>>> # Release the acquired lock
>>> lock.release()
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
False
>>>
>>> # Use this client object
>>> client = redis.StrictRedis()
>>>
>>> # Create a lock instance with custom client object
>>> lock = RedisLock('my_lock', client=client)
>>>
>>> # To override the defaults, just past the configurations as parameters
>>> lock = RedisLock('my_lock', client=client, expire=1, timeout=5)
>>>
>>> # Acquire a lock using the with_statement
>>> with RedisLock('my_lock') as lock:
...     # do some stuff with your acquired resource
...     pass

Parameters:

lock_name (str) – name of the lock to uniquely identify the lock between processes.
namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
timeout (float) – set timeout to acquire lock
retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
client – supported client object for the backend of your choice.

acquire(blocking=True)¶

Acquire a lock, blocking or non-blocking.

Parameters:	blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True.
Returns:	if the lock was successfully acquired or not
Return type:	bool

locked()¶

Return if the lock has been acquired or not.

Returns:	True indicating that a lock has been acquired ot a shared resource is locked.
Return type:	bool

release()¶: Release a lock.

Etcd based Locks¶

class sherlock.EtcdLock(lock_name, **kwargs)[source]¶

Implementation of lock with Etcd as the backend for synchronization.

Basic Usage:

>>> import etcd
>>> import sherlock
>>> from sherlock import EtcdLock
>>>
>>> # Global configuration of defaults
>>> sherlock.configure(expire=120, timeout=20)
>>>
>>> # Create a lock instance
>>> lock = EtcdLock('my_lock')
>>>
>>> # Acquire a lock in Etcd, global backend and client configuration need
>>> # not be configured since we are using a backend specific lock.
>>> lock.acquire()
True
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
True
>>>
>>> # Release the acquired lock
>>> lock.release()
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
False
>>>
>>> # Use this client object
>>> client = etcd.Client()
>>>
>>> # Create a lock instance with custom client object
>>> lock = EtcdLock('my_lock', client=client)
>>>
>>> # To override the defaults, just past the configurations as parameters
>>> lock = EtcdLock('my_lock', client=client, expire=1, timeout=5)
>>>
>>> # Acquire a lock using the with_statement
>>> with EtcdLock('my_lock') as lock:
...     # do some stuff with your acquired resource
...     pass

Parameters:

lock_name (str) – name of the lock to uniquely identify the lock between processes.
namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
timeout (float) – set timeout to acquire lock
retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
client – supported client object for the backend of your choice.

acquire(blocking=True)¶

Acquire a lock, blocking or non-blocking.

Parameters:	blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True.
Returns:	if the lock was successfully acquired or not
Return type:	bool

locked()¶

Return if the lock has been acquired or not.

Returns:	True indicating that a lock has been acquired ot a shared resource is locked.
Return type:	bool

release()¶: Release a lock.

Memcached based Locks¶

class sherlock.MCLock(lock_name, **kwargs)[source]¶

Implementation of lock with Memcached as the backend for synchronization.

Basic Usage:

>>> import pylibmc
>>> import sherlock
>>> from sherlock import MCLock
>>>
>>> # Global configuration of defaults
>>> sherlock.configure(expire=120, timeout=20)
>>>
>>> # Create a lock instance
>>> lock = MCLock('my_lock')
>>>
>>> # Acquire a lock in Memcached, global backend and client configuration
>>> # need not be configured since we are using a backend specific lock.
>>> lock.acquire()
True
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
True
>>>
>>> # Release the acquired lock
>>> lock.release()
>>>
>>> # Check if the lock has been acquired
>>> lock.locked()
False
>>>
>>> # Use this client object
>>> client = pylibmc.Client(['X.X.X.X'], binary=True)
>>>
>>> # Create a lock instance with custom client object
>>> lock = MCLock('my_lock', client=client)
>>>
>>> # To override the defaults, just past the configurations as parameters
>>> lock = MCLock('my_lock', client=client, expire=1, timeout=5)
>>>
>>> # Acquire a lock using the with_statement
>>> with MCLock('my_lock') as lock:
...     # do some stuff with your acquired resource
...     pass

Parameters:

lock_name (str) – name of the lock to uniquely identify the lock between processes.
namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
timeout (float) – set timeout to acquire lock
retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
client – supported client object for the backend of your choice.

acquire(blocking=True)¶

Acquire a lock, blocking or non-blocking.

Parameters:	blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True.
Returns:	if the lock was successfully acquired or not
Return type:	bool

locked()¶

Return if the lock has been acquired or not.

Returns:	True indicating that a lock has been acquired ot a shared resource is locked.
Return type:	bool

release()¶: Release a lock.

API Documentation¶

Configuration¶

Understanding the configuration parameters¶

`backend`¶

Available Backends¶

`client`¶

`namespace`¶

`expire`¶

`timeout`¶

`retry_interval`¶

Generic Locks¶

Backend Specific Locks¶

Redis based Locks¶

Etcd based Locks¶

Memcached based Locks¶

Indices and tables¶

Table Of Contents

Previous topic

Next topic

This Page

API Documentation¶

Configuration¶

Understanding the configuration parameters¶

backend¶

Available Backends¶

client¶

namespace¶

expire¶

timeout¶

retry_interval¶

Generic Locks¶

Backend Specific Locks¶

Redis based Locks¶

Etcd based Locks¶

Memcached based Locks¶

Indices and tables¶

`backend`¶

`client`¶

`namespace`¶

`expire`¶

`timeout`¶

`retry_interval`¶