Key-Based Locking - Upstash Documentation

Upstash Redis databases use locks to keep commands isolated while allowing independent keys to be processed in parallel. The engine automatically locks the hash tag of each key a command uses. Commands whose keys have different hash tags can run concurrently, subject to the parallelism available to your database. Key-based locking is transparent to clients. You do not need to change regular Redis commands to use it.

How It Works

Single-key commands (for example GET, SET, INCR, HSET) acquire a lock on that key’s hash tag.
Multi-key commands acquire locks on the hash tag of every key they reference, in a deterministic order to avoid deadlocks.
Read-only commands (for example GET, HGET, LRANGE) take a shared read lock, so multiple readers on the same hash tag run concurrently. Read locks block writers on that hash tag until they complete.
Commands that need a database-wide operation, such as FLUSHDB and FLUSHALL, take the global lock and can reduce concurrency while they run.

Hash Tags

A valid Redis hash tag is the non-empty value between the first { in a key and the first } that follows it. If a key has no valid hash tag, its full name acts as its hash tag. Locking works on hash tags: {queue}:wait and {queue}:active share the queue hash tag and lock together; queue:wait and queue:active have no hash tag, so each locks under its own full name. Commands that write indexed data also lock the matching Search indexes. Two commands can therefore contend even when their data keys differ if both commands update the same index.

Transactions

Transactions (MULTI/EXEC) use key-based locking at EXEC time. While commands are queued, Upstash collects the keys referenced by the transaction. When EXEC runs, the engine takes an exclusive write lock for the union of those keys and executes the queued commands atomically. Transactions whose keys have disjoint hash tags can run concurrently. Transactions that share a hash tag block each other until one transaction finishes.

MULTI
SET user:42:name "Ada"
INCR user:42:version
EXEC

In this example, EXEC locks user:42:name and user:42:version for the duration of the transaction. If a queued command requires a database-wide lock, the whole transaction uses the global lock. This includes commands such as FLUSHDB and FLUSHALL. Lua scripts queued inside a transaction always execute under the global lock, even if the script declares allow-key-locking. If you want script-level key locking, run the script directly with EVAL / EVALSHA outside of a transaction.

Lua Scripts

Lua scripts (EVAL, EVALSHA, EVAL_RO, EVALSHA_RO) default to the global lock because the engine cannot know in advance which keys the script will use. To opt into key-based locking, add the allow-key-locking flag to the script’s shebang line:

#!lua flags=allow-key-locking

redis.call('INCR', KEYS[1])
redis.call('INCRBY', KEYS[2], ARGV[1])
return "OK"

When the flag is set, Upstash locks the hash tag of each key passed through the KEYS array when the script is invoked. For writes, it also locks the matching Search indexes. Other commands and scripts whose keys have disjoint hash tags can run in parallel.

Rules for `allow-key-locking`

Every key passed to redis.call must be covered by an existing lock. A key is covered when its hash tag is already locked — usually because the key itself appears in KEYS, or because it shares a hash tag with one that does. A Search index matched by a declared key can also lock a hash tag. For example, declaring {queue}:wait locks the queue hash tag, which also covers a dynamic {queue}:active key. A key with a different hash tag is rejected, and a key with no valid hash tag — whose hash tag is its own full name — is covered only when it appears in KEYS itself. When a command writes indexed data, all matching Search indexes must also be covered by locks acquired before the script starts. Upstash automatically includes the indexes that match declared keys. An indexed dynamic key is rejected if it requires another index that is not already covered. An uncovered key or index produces an error such as:
ERR Dynamic keys are not allowed in Lua scripts when 'allow-key-locking' flag is set. Key was: <key>
Even when a dynamic key shares a declared hash tag, pass the fully resolved key through KEYS when possible. Declared keys and their matching indexes can be loaded before the script runs. A dynamic key can instead force a disk read while the lock is held. This is also worth avoiding in scripts that use the global lock. See Dynamic Keys and Latency.
Database-wide writes are not allowed. Commands that require database-wide exclusive access, such as FLUSHDB and FLUSHALL, cannot be called from a script with allow-key-locking. Run those scripts without the flag so the engine can use the global lock.

Read-only script variants and scripts with the no-writes flag also need allow-key-locking if you want them to use per-key read locks. Without it, they run under the global lock. To use both flags in a Lua script, separate them with a comma:

#!lua flags=no-writes,allow-key-locking

When to use it

Enable allow-key-locking for short scripts that operate on a small, known set of keys and are called frequently enough that the global lock becomes a bottleneck (for example counters, rate limiters, or per-user state transitions). For scripts that must scan or mutate many keys at once, leave the flag off so the engine uses the global lock.

Example: Key-Locked Counter

#!lua flags=allow-key-locking

local current = tonumber(redis.call('GET', KEYS[1]) or "0")
if current >= tonumber(ARGV[1]) then
  return 0
end
redis.call('INCR', KEYS[1])
return 1

Invoked with:

EVAL "<script>" 1 user:42:quota 100

Multiple clients calling this script for different users will execute concurrently, each holding a lock only on its own user:<id>:quota key.

Example: Dynamic Keys with a Shared Hash Tag

#!lua flags=allow-key-locking

local destination = '{queue}:' .. ARGV[1]
local job = redis.call('LPOP', KEYS[1])
if job then
  redis.call('RPUSH', destination, job)
end
return job

Invoke the script with {queue}:wait as its declared key and active as an argument:

EVAL "<script>" 1 {queue}:wait active

The declared key locks the queue hash tag, so the dynamically constructed {queue}:active key is covered by the same lock. Constructing {other}:active would be rejected because the other hash tag was not locked.

Redis Functions

Redis functions (FCALL, FCALL_RO) also default to the global lock. For functions, allow-key-locking is set on each registered function, not on the library shebang, and takes effect when the library is loaded with FUNCTION LOAD:

#!lua name=locks

local function incr_if_below(keys, args)
  local current = tonumber(redis.call('GET', keys[1]) or "0")
  if current >= tonumber(args[1]) then
    return 0
  end

  redis.call('INCR', keys[1])
  return 1
end

redis.register_function{
  function_name='incr_if_below',
  callback=incr_if_below,
  flags={'allow-key-locking'}
}

Invoked with:

FCALL incr_if_below 1 user:42:quota 100

The same rules apply to functions. A key is covered when it appears in the FCALL key list or shares a hash tag with an already locked key, including a Search index matched by a declared key. A key passed as a regular argument is rejected if its hash tag is not already locked. If the function is also read-only, include both flags in the function registration:

flags={'no-writes', 'allow-key-locking'}

Dynamic Keys and Latency

Pass every key a script or function touches through the key list of the call, where it arrives as KEYS, when possible. This remains the best choice even when you are not using allow-key-locking. With the flag set, a dynamic key is accepted only when its hash tag is already locked. Without the flag, the call uses the global lock and dynamic keys are accepted, but they can be slow, and the slowdown is not limited to the caller. Upstash keeps data in memory and on disk, and an entry that has been idle long enough to be evicted from memory is read back from disk on the next access. Declared keys are loaded before the script body starts running, and the engine releases the lock while it waits for that read, so other commands keep making progress. A key that only becomes known in the middle of the script cannot be loaded that way: script execution has to stay atomic, so the engine holds the lock across the disk read. With allow-key-locking, this blocks other work on the same hash tag. Under the global lock, the whole database waits for the disk read.

-- A same-tag dynamic key is locked, but may be read from disk under that lock
local key = '{user:42}:' .. ARGV[1]
redis.call('INCR', key)

-- Declared key: loaded before the script runs, outside the locked section
redis.call('INCR', KEYS[1])

The cost only appears when the key or a matching Search index is not already in memory, so it is easy to miss against a small, warm test dataset and easy to hit in production against a large one. Resolve key names in your application and pass them in the key list whenever possible.

​How It Works

​Hash Tags

​Transactions

​Lua Scripts

​Rules for allow-key-locking

​When to use it

​Example: Key-Locked Counter

​Example: Dynamic Keys with a Shared Hash Tag

​Redis Functions

​Dynamic Keys and Latency