Lua Scripting

Command table

A Lua script runs atomically on the server. Nothing interleaves, so read-then-decide logic that would need a WATCH retry loop becomes one round trip with no retry at all. Redis embeds Lua 5.1.

EVAL "return redis.call('SET', KEYS[1], ARGV[1])" 1 hu:k v1     # OK
sha=$(redis-cli SCRIPT LOAD "return redis.call('GET', KEYS[1])")
echo "$sha"                                    # e0e1f9fa...
redis-cli EVALSHA "$sha" 1 hu:k                # "v1"
redis-cli SCRIPT EXISTS "$sha" 0000000000000000000000000000000000000000
# 1) (integer) 1  2) (integer) 0

The production idiom is EVALSHA first, and on NOSCRIPT fall back to EVAL once and retry. Every real client does this for you, so you send a 40-byte hash instead of the whole source on every call.

KEYS and ARGV

Two global tables, both 1-based because this is Lua.

-- EVAL <script> 2 key:a key:b arg1 arg2
KEYS[1]  -- "key:a"
KEYS[2]  -- "key:b"
ARGV[1]  -- "arg1"
#KEYS    -- 2

Every value in both tables is a string. Numbers arrive as "42", so wrap anything arithmetic in tonumber() and remember that tostring(1) is "1" while tostring(1.0) is "1" too, since Lua 5.1 has one number type.

Every key the script touches must be declared in KEYS. This is not style, it is the contract that lets Redis Cluster route the call, and a script that builds a key name from ARGV or from a SCAN result will work on a standalone server and fail in Cluster with CROSSSLOT Keys in request don't hash to the same slot. In Cluster all declared keys must also live in the same slot, which is what hash tags are for: hu:{1042}:profile and hu:{1042}:sessions share a slot.

# WRONG, the key is invisible to the cluster router
EVAL "return redis.call('GET', 'hu:hardcoded')" 0

# RIGHT
EVAL "return redis.call('GET', KEYS[1])" 1 hu:hardcoded

redis.call versus redis.pcall

-- call: the script dies and the client sees the error
return redis.call('INCR', KEYS[1])     -- WRONGTYPE aborts everything

-- pcall: you decide
local res = redis.pcall('INCR', KEYS[1])
if type(res) == 'table' and res.err then
  return 'skipped: ' .. res.err
end
return res

Use redis.call by default. An unhandled error aborting the script is usually what you want, because the alternative is silently continuing with a half-applied change, and remember that a script is not rolled back either. Commands already executed inside it stay executed.

Return values and type conversion

Lua types convert to RESP on the way out, and the mapping is lossy in one place people trip over.

EVAL "return 3.9" 0                          # (integer) 3, the decimal is GONE
EVAL "return tostring(3.9)" 0                # "3.9", return a string for floats
EVAL "return {1, 2, 3}" 0                    # 1) 1  2) 2  3) 3
EVAL "return {1, nil, 3}" 0                  # 1) (integer) 1   <- truncated at the nil
EVAL "return redis.status_reply('DONE')" 0    # DONE, a simple status
EVAL "return redis.error_reply('MYERR nope')" 0  # (error) MYERR nope
EVAL "return cjson.encode({1, 2, 'x'})" 0     # "[1,2,\"x\"]"

Return floats as strings, and never rely on a nil in the middle of a returned array.

Libraries available

local payload = cjson.decode(ARGV[1])
redis.log(redis.LOG_WARNING, 'processing ' .. payload.id)
return redis.sha1hex(ARGV[1])

math.random is seeded deterministically per script call, and os.time, os.clock, and print are removed. Use redis.call('TIME') for the clock, or better, pass the timestamp in as an ARGV so the same script replays identically.

A correct rate limiter

Fixed window, atomic, returning both the verdict and how long until the window resets.

-- EVAL <script> 1 hu:rl:ip:1.2.3.4 <limit> <window_ms>
-- returns {allowed, count_or_retry_after_ms}
local n = redis.call('INCR', KEYS[1])
if n == 1 then
  redis.call('PEXPIRE', KEYS[1], tonumber(ARGV[2]))
end
if n > tonumber(ARGV[1]) then
  return {0, redis.call('PTTL', KEYS[1])}
end
return {1, n}

redis-cli EVAL "$script" 1 hu:rl:ip:1.2.3.4 3 60000
# 1) (integer) 1  2) (integer) 1     allowed
# ... after four calls:
# 1) (integer) 0  2) (integer) 59997 denied, retry in 60s

The PEXPIRE is inside the same atomic unit as the INCR, which closes the window where a crash between the two leaves an immortal counter and a permanently banned caller.

Token bucket, which allows bursts and refills smoothly instead of resetting on a cliff. Pass the clock in from the caller so the script stays deterministic.

-- EVAL <script> 1 hu:tb:user:1042 <rate/sec> <capacity> <now_ms> <cost>
local rate     = tonumber(ARGV[1])
local capacity = tonumber(ARGV[2])
local now      = tonumber(ARGV[3])
local cost     = tonumber(ARGV[4])

local b      = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(b[1])
local ts     = tonumber(b[2])
if tokens == nil then
  tokens = capacity
  ts     = now
end

local delta = math.max(0, now - ts) / 1000.0
tokens = math.min(capacity, tokens + delta * rate)

local allowed = 0
if tokens >= cost then
  tokens  = tokens - cost
  allowed = 1
end

redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / rate * 1000) + 1000)
return {allowed, math.floor(tokens)}

# rate 1/s, capacity 3: three go through, the fourth is denied
redis-cli EVAL "$script" 1 hu:tb:user:1042 1 3 "$now" 1   # 1) 1  2) 2
redis-cli EVAL "$script" 1 hu:tb:user:1042 1 3 "$now" 1   # 1) 1  2) 1
redis-cli EVAL "$script" 1 hu:tb:user:1042 1 3 "$now" 1   # 1) 1  2) 0
redis-cli EVAL "$script" 1 hu:tb:user:1042 1 3 "$now" 1   # 1) 0  2) 0
# two seconds later, two tokens have refilled
redis-cli EVAL "$script" 1 hu:tb:user:1042 1 3 "$now2" 1  # 1) 1  2) 1

The other script every codebase needs is the safe lock release, a compare-and-delete that cannot free someone else's lock:

-- EVAL <script> 1 hu:lock:job:9 <my-random-token>
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
end
return 0

Atomicity, and keeping scripts short

A script blocks the entire server for its whole duration. That is what buys you atomicity, and it is also the cost.

- Redis is single threaded, so a 200ms script adds 200ms to the p99 of every other client. Target sub-millisecond. - No blocking commands. BLPOP, BRPOP, XREAD BLOCK, and WAIT do not block inside a script, they return their empty reply. - busy-reply-threshold (formerly lua-time-limit, default 5000ms) is when the server starts answering other clients with BUSY. It does not kill the script. - SCRIPT KILL only works on a script that has not yet written. Once it has, the only exit is SHUTDOWN NOSAVE, which loses unpersisted data. Keep loops bounded. - Avoid unpack() on a large table, it overflows the Lua stack at a few thousand elements. Batch in the hundreds. - Scripts are propagated to replicas and the AOF as effects, the concrete write commands they performed, so nondeterminism is not a correctness problem the way it was pre-5.0. Passing timestamps and random values in as ARGV is still better style.

EVAL_RO (7.0+) declares a script read-only, so an accidental write is caught rather than silently executed, and the call can be served by a replica.

EVAL_RO "return redis.call('SET', KEYS[1], 'x')" 1 hu:k
# (error) ERR Write commands are not allowed from read-only scripts.

Functions (7.0+)

Functions are the successor to EVAL. A library is registered once, persists in the RDB and the AOF, replicates to replicas, and survives a restart, so there is no NOSCRIPT fallback and no shipping source at runtime.

#!lua name=hulib

local function rate_limit(keys, args)
  local n = redis.call('INCR', keys[1])
  if n == 1 then
    redis.call('PEXPIRE', keys[1], tonumber(args[2]))
  end
  if n > tonumber(args[1]) then
    return {0, redis.call('PTTL', keys[1])}
  end
  return {1, n}
end

redis.register_function('hu_rate_limit', rate_limit)
redis.register_function{
  function_name = 'hu_get',
  callback      = function(keys, args) return redis.call('GET', keys[1]) end,
  flags         = {'no-writes'}          -- enables FCALL_RO
}

redis-cli FUNCTION LOAD "$(cat hulib.lua)"      # "hulib"
redis-cli FCALL hu_rate_limit 1 hu:rl:ip 100 60000
redis-cli FCALL_RO hu_get 1 hu:k
redis-cli FUNCTION LIST LIBRARYNAME hulib
redis-cli FUNCTION DELETE hulib

The shebang #!lua name=<libname> is mandatory, the library body runs once at load time and may only register functions, and the callback signature is (keys, args) as lowercase locals rather than the KEYS/ARGV globals. Reloading the same library name needs REPLACE.

Use functions for logic you deploy alongside the app and call by name. Keep EVAL for short, ad-hoc, client-embedded scripts, which is still the majority of real usage.