Locking-Center

A small distributed lock server. Take a named lock, do the work, release it — blocking and FIFO-fair, from a single binary, with client libraries for nine languages.

Locking-Center is a mutex point to synchronize access between different services. Services on different machines take a named lock before they touch a shared resource, and only one of them holds that key at a time. The rest queue up and are served in the order they arrived.

It is a single binary with one dependency, no configuration file, and no coordination layer to operate.

At a glance

• Blocking, FIFO-fair acquisition. A caller parks until the key is free and is woken in arrival order — not a SETNX spin-and-retry with no ordering.
• Blocking and non-blocking. Lock waits its turn; TryLock takes the key only if it is free right now.
• Client libraries for nine languages — Go, Rust, Python, Java, C#, JavaScript, C, C++ and Zig — all speaking the same tiny TCP protocol. See the list.
• Optional durability. Set DATA_PATH and held keys survive a restart, on a write-ahead log that only ever errs on the safe side.
• Operable out of the box. Docker image, a single-replica Kubernetes manifest, Prometheus metrics with a stuck-lock alert, and an lc-cli that locks straight from a shell script.
• Deliberately simple. One instance, no auth, no fencing tokens — when to use it and when not to are spelled out below.

---

Contents

• Quick start
• What it is for
• When to use it
• When not to use it
• How it works
• Running with Docker
• Running on Kubernetes
• Running as a binary
• Ports
• Keeping locks across restarts
• Client libraries
• Wire protocol
• Management CLI
• Monitoring with Prometheus
• Shutting down
• Building from source
• License

---

Quick start

Run the server (see Running as a binary for the no-Docker route):

docker run -d --name locking-center \
  -p 22119:22119 -p 22120:22120 -p 22121:22121 \
  freakmaxi/locking-center:latest

Take a lock from your code — here in Go, see client libraries for the other eight:

m, _ := mutex.NewLockingCenter("localhost:22119")
m.Lock("orders/batch-7")
defer m.Unlock("orders/batch-7")
// ... exclusive work; every other caller for this key waits here until you unlock ...

Or straight from a shell, no code at all — the distributed flock(1):

lc-cli lock deploy/prod -- ./deploy.sh production   # acquire, run, release even if it fails

What it is for

Say you keep a text file in a shared location. One service reads it and appends at the end, another finds a section and removes it. If both reach the file at the same time you get a race, and one of them loses.

The usual ways around this are heavier than the problem;

• Put a service in front of the file. Every other service goes through it and it serializes the access. That works until you need to scale it, and then you need a message queue, and then you need a second queue to tell the callers what happened, because you no longer know the result at the moment you asked for it.
• Cache the resource in Redis and rely on its locking. That works until the resource is too big to keep in memory.

Locking-Center takes the lock out as its own primitive. You lock a key, do the work, unlock the key. The shared resource is untouched, you keep the result immediately and synchronously, and there is no queue architecture to build.

When to use it

Locking-Center fits when all of these are true;

• The critical sections are short, and the work inside them is retryable.
• Losing every lock during a restart is survivable, or you set DATA_PATH so they are kept.
• You want a caller to block until the key is free rather than spin and retry.
• The network between your services and the lock server is trusted.

Things it does well;

Use case	Why it fits
Stopping a scheduled job from running twice across replicas	Short, retryable, a missed lock just reruns
Serializing writes to a shared file or NFS mount	The original motivating case
Holding an external API to one caller at a time	Blocking acquire, no spin loop
Sequencing migration or deployment steps	Human paced, easy to reset by hand

The real differentiator is blocking, FIFO-fair acquisition. A Redis SETNX lock makes the caller spin and retry, and gives you no ordering at all. Locking-Center parks the caller and wakes it in arrival order.

When not to use it

Locking-Center is deliberately simple, and simple has a price. Do not reach for it when;

• A double acquire would corrupt data or move money. There are no fencing tokens, so a holder that stalls past its turn cannot be shut out of the resource.
• The lock server must not be a single point of failure. There is one instance and no replication. While it is down nobody locks anything. Use etcd, Consul or ZooKeeper.
• A crashed client must release its lock automatically. There are no leases or TTLs. See Monitoring with Prometheus for detecting stuck locks, and lc-cli reset for clearing them.
• The network is not trusted. There is no authentication. Anyone who can reach the port can lock any key, unlock somebody else's key, or reset everything.

If you already run PostgreSQL, pg_advisory_lock gives you much of this with the lock tied to the database session, so a crashed client releases automatically. On Kubernetes, a coordination.k8s.io Lease covers leader election without new infrastructure. Locking-Center earns its place when you want blocking, ordered, sub-millisecond locks without adding a database or a consensus system. With DATA_PATH set the locks are bound by the disk instead, see the cost.

How it works

Every key gets a Go channel with a buffer of one. Sending into it acquires the lock, receiving from it releases. When the buffer is full the next sender blocks, and that block is the queue, with ordering inherited from the Go runtime.

Each waiting request is also registered in a map by its own id. That map is not the queue, it is the cancellation registry, and it is the only way to reach a request that is parked inside a channel send in order to revoke it. That is what makes reset work on waiters and not just on the current holder, and what withdraws a waiter whose connection dropped so a dead client is never handed the key.

A lock is not bound to the connection that took it. The client connects, gets its answer, and the connection closes while the lock stays held. This is what makes a crashed client leave its key locked forever, and why reset exists. It is also what makes keeping the locks across a restart work, the holders never notice that the server went away.

Running with Docker

Build the image with the script under -build-/docker;

./-build-/docker/create_image.sh

It tags the calculated release version and latest. To build both architectures and push;

IMAGE_NAME="your-registry/locking-center" ./-build-/docker/create_image.sh --push

Variable	Default	Description
IMAGE_NAME	freakmaxi/locking-center	Image repository
IMAGE_TAG	latest	Extra tag next to the version
PLATFORMS	linux/amd64,linux/arm64	Targets used by --push

Run it;

docker run -d --name locking-center \
  -p 22119:22119 -p 22120:22120 -p 22121:22121 \
  freakmaxi/locking-center:latest

The image runs as a non root user, carries both lcd and lc-cli, and has a HEALTHCHECK on the metrics endpoint. docker stop sends SIGTERM, which goes through the graceful shutdown path.

To keep the locks across restarts, mount a volume on /data and point DATA_PATH at it;

docker run -d --name locking-center \
  -p 22119:22119 -p 22120:22120 -p 22121:22121 \
  -v locking-center-data:/data -e DATA_PATH=/data/locks.json \
  freakmaxi/locking-center:latest

The service runs as uid and gid 22119. A named volume picks up the ownership of /data from the image, a bind mount does not, so hand a host directory over first with chown 22119:22119 [directory]. Without it the process cannot create the file and exits with code 4.

docker exec locking-center lc-cli --manager-address 127.0.0.1:22120 keys -d

Running on Kubernetes

Run exactly one replica. Locking-Center keeps its lock table in memory and does not coordinate between instances. Two replicas behind one Service means two independent lock tables handing out the same key.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: locking-center
spec:
  replicas: 1
  strategy:
    type: Recreate            # never run two lock tables at once
  selector:
    matchLabels:
      app: locking-center
  template:
    metadata:
      labels:
        app: locking-center
    spec:
      terminationGracePeriodSeconds: 45   # above the 30s shutdown timeout
      securityContext:
        runAsNonRoot: true
        fsGroup: 22119                    # most block storage mounts as root, this hands /data to the service
      containers:
        - name: locking-center
          image: freakmaxi/locking-center:latest
          env:
            - { name: DATA_PATH, value: /data/locks.json }
          ports:
            - { name: mutex,   containerPort: 22119 }
            - { name: manager, containerPort: 22120 }
            - { name: metrics, containerPort: 22121 }
          readinessProbe:
            httpGet: { path: /metrics, port: metrics }
          livenessProbe:
            httpGet: { path: /metrics, port: metrics }
          volumeMounts:
            - { name: data, mountPath: /data }
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: locking-center-data
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: locking-center-data
spec:
  accessModes: [ReadWriteOnce]      # one replica, one writer
  resources:
    requests:
      storage: 100Mi                # the file holds the held keys only, it stays tiny

Leave out DATA_PATH, the volume and the claim to run without persistence.

Clients should pass their pod IP as the source address, using the downward API, so reset by source can release everything a single pod held;

env:
  - name: POD_IP
    valueFrom:
      fieldRef:
        fieldPath: status.podIP

Without it the server infers the source from the peer address, which a service mesh or SNAT can rewrite.

Running as a binary

• Download the latest release, or compile it with -build-/executable/create_release.sh.
• Copy lcd to /usr/local/bin and make it executable with sudo chmod +x /usr/local/bin/lcd
• Create a start up file, save it, make it executable, and run it;

#!/bin/sh

export BIND_ADDRESS="localhost:22119" # optional, defaults to `:22119`
/usr/local/bin/lcd

Ports

BIND_ADDRESS sets the mutex port. The two following ports are taken as well.

Service	Port	Protocol	Description
Mutex	22119	Binary TCP	Locking and unlocking, used by your services
Manager	22120	Binary TCP	Management commands, used by lc-cli
Metrics	22121	HTTP	Prometheus /metrics endpoint

Only the mutex port needs to be reachable by your applications. Keep the manager port restricted, it can release every lock in the system.

Keeping locks across restarts

By default the lock table lives in memory. A restart, including every deploy, drops every key, and the clients that were holding them are not told. Two of them can then end up inside the same critical section, one from before the restart and one from after it.

Set DATA_PATH to a file and the held keys are kept there;

export DATA_PATH="/var/lib/locking-center/locks.json"

• Every lock is on disk before it is acknowledged. The key is appended to the log and synced first, then the client gets +. A crash in between leaves a client that was never told it holds the key, which is the safe side.
• Every release is on disk before it takes effect. The other order could bring back a key that nobody holds.
• A lock that cannot be written is not handed out. The client gets -, the key is released again, and locking_center_store_write_errors_total goes up.
• A release goes through in memory regardless. The client gets + and the failure is logged and counted. The file still lists the key until the next write succeeds, so a restart in that window brings it back as held, and a reset clears it.
• Nothing else changes for the clients. The wire protocol is the same, no client needs to be upgraded.
• One process, one DATA_PATH. The log is written by a single process with no cross process locking. Two instances pointed at the same file would corrupt each other's log, so a shared DATA_PATH must never be handed to more than one instance. Running a single instance already, as the service requires, keeps this true; on Kubernetes a ReadWriteOnce volume enforces it.

The one thing that can be left behind is a key that nobody holds. A crash after the record is synced but before the client is told, or a sync that reports failure after the bytes already reached the disk, can restore a holder that was answered - and never entered its critical section. That is the safe side: it can never hand a key to two clients, it only ever leaves a key held by nobody, and that looks exactly like a crashed client to the alert and to lc-cli reset.

Only the held keys are stored. Requests that are waiting for a key are open connections, they die with the process and retry on their own, so on a restart they queue up again behind the restored holders.

The file is a write ahead log. Each lock and each release appends one small record, so the cost of an operation does not grow with the size of the table. When the log has roughly doubled over the set of held keys it is compacted, rewritten to hold only the keys that are currently held, through a temporary file and a rename so a crash in the middle leaves the previous log in place. locking_center_store_compactions_total counts the rewrites. The log stays proportional to the busiest the table has been since the last compaction rather than to the number of locks ever taken; a table that peaks large, drains small and then goes completely idle keeps the larger log until the next burst of activity compacts it again, which only costs a longer replay on the next restart.

Each record is a length, a CRC32C checksum and the JSON of one change. A crash during an append can only damage the last record: a record shorter than its length claims is a torn tail, it was never acknowledged and is dropped on the next start. A record that is fully present but fails its checksum is real corruption and refuses the start.

What it costs

Every lock and every release is one append and one sync of the log, regardless of how many keys are held. They go through the one writer that guards the table, so the locking rate is bound by the sync latency of the volume, roughly one operation per sync. A compaction is an extra full rewrite, but it happens about once per as many operations as there are held keys, so its cost is spread thin. The manager port and the metrics endpoint wait behind the same writer, so a volume that stalls shows up as a failed liveness probe. Without DATA_PATH none of this applies.

Two things follow from keeping the locks;

• A stuck lock is now stuck across restarts too. Before, a restart cleared the key that a crashed client left behind. Now it comes back, with its original timestamp, so the alert keeps firing until somebody runs lc-cli reset. That is the intended behaviour, but it makes the alert something to actually watch.
• A corrupt log refuses to start. A torn last record from the crash that stopped the previous run is dropped, it was never acknowledged. But a record that is whole and fails its checksum is real corruption, and starting with a wrong table would silently drop or misreport locks, so the process exits with code 4 and says why. Delete the file to start fresh, on purpose.

Client libraries

Every client speaks the same wire protocol and offers the same API: blocking Lock, non-blocking TryLock, Unlock, Wait, and the two resets. All are licensed under Apache-2.0 so they can be embedded in any service.

Language	Repository	Notes
C	locking-center-client-c	C11, POSIX sockets, static library
C#	locking-center-client-csharp	.NET Standard 2.0, on NuGet as LockingCenterClient
C++	locking-center-client-cpp	C++17, header-only
Go	locking-center-client-go	no dependencies
Java	locking-center-client-java	Java 21, no dependencies, on Maven Central as io.github.freakmaxi:locking-center-client
JavaScript	locking-center-client-js	Node.js, Promise-based, on npm as @freakmaxi/locking-center-client
Python	locking-center-client-python	3.10+, stdlib only, on PyPI as lockingcenter
Rust	locking-center-client-rust	std only, on crates.io as locking-center-client
Zig	locking-center-client-zig	Zig 0.16

If you write a client for another language, share it and it will be listed here.

Wire protocol

Every request is a new TCP connection. Write the request bytes, read a single byte answer, close the connection.

Answer	Meaning
+	The operation succeeded
-	The operation failed, for example a malformed key. Check the format, otherwise retry
#	A try-lock did not win the key, it is held by somebody else. Not a failure, and only ever the answer to a try-lock

Strings are length prefixed. One byte holds the length, the bytes that follow are the content. A key is at most 127 bytes. The server does not check the encoding, whatever bytes a client sends are the key.

The first byte of every request is the action;

Action	Byte	Request layout	Answer
Lock	1	[1][keySize][key][sourceSize][source]	+ when acquired, blocks until then
Unlock	2	[2][keySize][key]	+
Reset by key	3	[3][keySize][key]	+
Reset by source	4	[4][sourceSize][source]	+
Try lock	5	[5][keySize][key][sourceSize][source]	+ acquired, # held by another

source identifies the owner rather than the connection, and it is what reset by source matches on. Send it as an empty string, a length byte of 0, to let the server use the peer IP address instead.

Locking

To lock the key locking-me and let the server infer the source;

action    key size    "locking-me"                                 source size
   1          10      108 111 99 107 105 110 103 45 109 101             0

[1, 10, 108, 111, 99, 107, 105, 110, 103, 45, 109, 101, 0]

The answer may not come back immediately. That means the key is already held and you are queued behind its owner. Hang there until the byte arrives, then do your work and unlock.

Keep the connection open while you wait: the server watches it, and a request whose connection drops before the key is won is withdrawn from the queue. A client that dies while waiting therefore never ends up owning a key it cannot release; the next live requester gets it instead.

Your TCP client must not set a read timeout on this call. A queued request stays open for as long as the holder keeps the key, which is unbounded.

Try locking

Try locking is the non blocking form of locking. It takes the key only if it is free at that moment and answers right away, so the caller can decide what to do instead of waiting in the queue.

[5, 10, 108, 111, 99, 107, 105, 110, 103, 45, 109, 101, 0]

• + the key was free and is now yours, hold it and unlock it as usual.
• # the key is held by somebody else, you did not get it. Decide whether to try again later, wait with a blocking lock, or do something else.
• - the request was malformed or could not be made durable.

Unlike locking, this answer always comes back immediately, so a read timeout on the connection is fine here.

Unlocking

[2, 10, 108, 111, 99, 107, 105, 110, 103, 45, 109, 101]

Unlock is not checked against ownership. Any client that knows the key can release it.

Resetting

A crashed service does not release its lock, the key stays held until somebody clears it. Reset by key drops the key and lets the queued requests contend again;

[3, 10, 108, 111, 99, 107, 105, 110, 103, 45, 109, 101]

Reset by source drops everything a single owner holds, which is the one to use when a whole instance goes away;

[4, 8, 49, 48, 46, 48, 46, 48, 46, 52]     # source "10.0.0.4"

Management CLI

lc-cli is two tools in one binary. Its management commands talk to the manager port and inspect or clear the table; its lock commands talk to the mutex port like any other client, which makes locking-center usable from a shell script, a cron job or a CI pipeline without writing a program.

Inspecting and clearing

lc-cli --manager-address localhost:22120 keys
lc-cli --manager-address localhost:22120 keys -d      # with owner and duration
lc-cli --manager-address localhost:22120 reset [key]
lc-cli --manager-address localhost:22120 reset -s [source]   # everything one owner held

   10.0.0.4:51314 -> 2026 Aug 27 11:59:11 (   73.204s) locking-me (10.0.0.4)

Locking from a shell

lc-cli lock deploy/prod -- ./deploy.sh production  # acquire, run, release, even if deploy.sh fails
lc-cli trylock nightly-job && ./run-once.sh         # only one runner wins
lc-cli lock orders/batch-7                          # hold it across commands, until unlock or reset
lc-cli unlock orders/batch-7
lc-cli wait migration-done                          # block until free, hold nothing

Command	Blocks	Description
lock <key>	yes	Acquires the key. It stays held after lc-cli exits, until unlock or reset; useful to fence a resource while you work on it, and the easiest way to forget a lock
lock <key> -- <command…>	yes	Acquires the key, runs the command with the terminal attached, releases the key when it ends whatever the outcome, and exits with the command's exit code. The distributed flock(1)
trylock <key>	no	Acquires the key only if it is free right now. Exit 0 acquired (and held), exit 1 somebody else holds it
unlock <key>	no	Releases the key, whoever holds it
wait <key>	yes	Blocks until the key is free, then returns without holding it

Exit codes: 0 success, 1 trylock did not get the key, 2 any failure; lock with -- exits with the code of the command it ran. A SIGINT or SIGTERM during lock -- is forwarded to the command, so it gets to stop, and the key is still released before lc-cli exits.

The lock commands use --address (default localhost:22119) and identify themselves to the server with a source of lc-cli@<hostname>, so keys -d shows which machine holds a key and reset -s lc-cli@<hostname> sweeps everything a script there left behind. Override it with --source when a different identity is more useful, a deploy pipeline's job id for example. Keys are at most 127 bytes; an invalid key fails at once with exit 2 rather than retrying.

Monitoring with Prometheus

Locking-Center exposes the live state of the lock table at http://[host]:22121/metrics. The metrics come from a fresh snapshot on every scrape, so a key appears as soon as it is locked and disappears as soon as it is released. No stale series are left behind.

Metric	Type	Labels	Description
locking_center_locked_keys	gauge		Number of keys currently held under a lock
locking_center_lock_held_seconds	gauge	key, source_addr, remote_addr	Seconds the current owner has been holding the key
locking_center_lock_acquired_timestamp_seconds	gauge	key, source_addr, remote_addr	Unix timestamp of the moment the key was acquired
locking_center_store_write_errors_total	counter		Times a change could not be written to DATA_PATH since the start up. Only exported when DATA_PATH is set
locking_center_store_compactions_total	counter		Times the DATA_PATH log has been rewritten to drop released keys since the start up. Only exported when DATA_PATH is set

Standard Go runtime and process metrics are exported next to these. go_goroutines is worth watching, every client queued behind a lock is one parked goroutine, so it reads as queue depth.

# HELP locking_center_lock_held_seconds Duration in seconds that the current owner has been holding the lock of the key.
# TYPE locking_center_lock_held_seconds gauge
locking_center_lock_held_seconds{key="locking-me",remote_addr="10.0.0.4:51314",source_addr="10.0.0.4"} 73.204
locking_center_lock_held_seconds{key="orders/batch-7",remote_addr="10.0.0.9:44120",source_addr="10.0.0.9"} 2.981
# HELP locking_center_locked_keys Number of keys that are currently held under a lock.
# TYPE locking_center_locked_keys gauge
locking_center_locked_keys 2

scrape_configs:
  - job_name: locking-center
    static_configs:
      - targets: ['locking-center-host:22121']

Alerting on stuck locks

Since a crashed client never releases its key, this alert is the main way to find out that it happened.

groups:
  - name: locking-center
    rules:
      - alert: LockingCenterLockHeldTooLong
        expr: locking_center_lock_held_seconds > 60
        for: 0s
        labels:
          severity: warning
        annotations:
          summary: "Lock on key {{ $labels.key }} is held longer than 60 seconds"
          description: >-
            {{ $labels.source_addr }} ({{ $labels.remote_addr }}) has been holding the lock of
            {{ $labels.key }} for {{ $value | humanizeDuration }}. A crashed owner does not release
            its lock, it can be released with `lc-cli reset {{ $labels.key }}`.

The metric already carries the elapsed duration, so for: does not need to delay the alert. Leave it at 0s and move the 60 to change the threshold. Without DATA_PATH, a restart drops the key and the alert resolves on its own. With it, the key comes back with its original timestamp and the alert keeps firing until the key is reset.

Shutting down

Locking-Center handles SIGTERM and SIGINT. It stops accepting new mutex and manager connections first, then drains the metrics endpoint for up to 30 seconds, so the shutdown stays visible to Prometheus until the last moment, and exits. A mutex or manager request that is still in flight at that point is cut with the process. A client that was waiting for a key sees its connection drop and asks again, which is what the clients do anyway, and a key is only ever acknowledged after it is on disk, so nothing that a client was told is lost.

• Without DATA_PATH the locks do not survive a restart. Every key is released when the process exits, and the clients holding them are not notified. See keeping locks across restarts.
• On Kubernetes, keep terminationGracePeriodSeconds above 30. That budget covers the preStop hook and the SIGTERM to SIGKILL window together. If it runs out first the shutdown is killed part way through.

Building from source

Requires Go 1.25 or newer.

go build ./...                 # build everything
go test -race ./...            # unit tests with the race detector
./-build-/executable/create_release.sh    # cross compiled release binaries
./-build-/docker/create_image.sh          # container image

create_release.sh needs GNU date on macOS, install it with brew install coreutils.

License

GNU General Public License v3.0, see LICENSE.