SDB is an extensible post-mortem and live debugger for Linux kernels and
userland programs, built on top of drgn.
It provides a pipeline-based command architecture (similar to Unix pipes)
where typed drgn.Object instances flow between commands.
# Live kernel debugging
sudo sdb
# Debug a crash dump
sdb <vmlinux> <crash_dump>
# Debug a core dump
sdb <binary> <core_dump>
# Evaluate a single command and exit
sdb <vmlinux> <dump> -e "spa | member spa_name"
# Run multiple commands in sequence (stops on first error)
sdb <vmlinux> <dump> -e "spa | count" -e "threads | count"
# Read commands from stdin (one per line, # comments allowed)
echo -e "spa | count\nthreads | count" | sdb <vmlinux> <dump> -
# List all commands as JSON (for tooling/AI agents)
sdb --list-commands
# Machine-readable JSON output
sdb <vmlinux> <dump> -e "spa" --jsonCommands are composed using | pipes. Objects flow left-to-right through
the pipeline. Each command receives an iterable of drgn.Object and yields
an iterable of drgn.Object:
sdb> find_task 1 | member comm
(char [16])"systemd"
Shell pipes are supported with !:
sdb> addr modules | lxlist "struct module" list | member name ! sort | head -n 3
All commands inherit from sdb.Command (defined in sdb/command.py):
- Command - Base class. Implements
_call(objs) -> Iterable[drgn.Object]. - SingleInputCommand - Processes each input object independently; errors on one object don't stop processing of subsequent objects.
- Walker - Iterates over container data structures (linked lists, trees,
etc.). Implements
walk(obj) -> Iterable[drgn.Object]. Registered byinput_typeinWalker.allWalkers. - PrettyPrinter - Formats objects for human-readable display. Implements
pretty_print(objs) -> None. Registered byinput_typeinPrettyPrinter.all_printers. - Locator - Finds objects of a given type. Can start a pipeline (via
no_input()) or accept input (via@InputHandlerdecorated methods). Often also a PrettyPrinter (hybrid pattern).
class MyCommand(sdb.Command):
names = ["mycmd", "mc"] # CLI names/aliases
load_on = [sdb.Kernel()] # When to register: All(), Kernel(),
# Userland(), Module("zfs"), Library("libc")
input_type = "struct foo *" # Expected input C type (optional)Create a .py file in sdb/commands/ (or a subdirectory). The command is
auto-discovered via __init_subclass__:
import sdb
from typing import Iterable
import drgn
class MyLocator(sdb.Locator, sdb.PrettyPrinter):
"""
One-line summary of what this command does
DESCRIPTION
Detailed description here.
EXAMPLES
sdb> mycmd
...
"""
names = ["mycmd"]
input_type = "struct foo *"
output_type = "struct foo *"
load_on = [sdb.Kernel()]
def no_input(self) -> Iterable[drgn.Object]:
# Find all foo objects when used at pipeline start
...
def pretty_print(self, objs: Iterable[drgn.Object]) -> None:
for obj in objs:
print(f"foo at {hex(obj.value_())}")External commands can be loaded from arbitrary paths via:
--load-commands PATHCLI flagSDB_COMMANDS_PATHenvironment variable (colon-separated)%load-commands PATHin the REPL
sdb/
├── __init__.py # Public API exports
├── command.py # Command base classes (Command, Walker, PrettyPrinter, Locator)
├── pipeline.py # Pipeline execution engine
├── parser.py # Pipeline tokenizer/parser
├── target.py # drgn.Program wrapper and type utilities
├── loader.py # External command loading
├── error.py # Exception hierarchy
├── session.py # Session recording/replay
├── mdb_compat.py # mdb syntax preprocessing
├── internal/
│ ├── cli.py # CLI entry point (main())
│ └── repl.py # Interactive REPL
└── commands/ # Built-in commands
├── *.py # Core commands (echo, cast, filter, member, etc.)
├── linux/ # Linux kernel commands
├── zfs/ # ZFS-specific commands
└── spl/ # SPL commands
# Install in dev mode
pip install -e ".[dev]"
# Run unit tests (no crash dumps needed)
pytest -v tests/unit
# Lint
pylint -d duplicate-code -d invalid-name sdb
ruff check sdb tests
# Type check
mypy --strict --show-error-codes -p sdb
# Format check
yapf --diff --style google --recursive sdbIntegration tests require crash dumps and additional native libraries not
needed by unit tests. The GitHub Actions workflow (.github/workflows/main.yml,
job pytest-integration) is the authoritative reference for how to set this
up. The helper scripts live in .github/scripts/:
# 1. Install native dependencies for drgn and libkdumpfile
./.github/scripts/install-drgn.sh
./.github/scripts/install-libkdumpfile.sh
# 2. Download reference crash dumps from Google Drive (public folder)
# Uses the `gdown` Python package to pull from a shared gDrive folder.
./.github/scripts/download-dumps-from-gdrive.sh
# 3. Extract each dump archive into tests/integration/data/dumps/
./.github/scripts/extract-dump.sh dump.201912060006.tar.lzma
./.github/scripts/extract-dump.sh dump.202303131823.tar.gz
# 4. Run the integration tests
pytest -v tests/integrationSee each script for details (apt dependencies, build steps, dump archive profiles, etc.). All scripts assume they are run from the repo root.
Integration tests in tests/integration/test_*_generic.py compare command
output against checked-in reference baselines stored under
tests/integration/data/regression_output/.
When adding a new regression test module (i.e. a new
test_<module>_generic.py file with a CMD_TABLE), you must generate
and commit the reference output so the tests have baselines to compare
against:
# Re-generate ALL reference output (requires crash dumps in place)
python -m tests.integration.gen_regression_outputThis creates/overwrites files under data/regression_output/<dump>/<module>/.
Review the generated output carefully in the PR — it becomes the golden
baseline that future test runs are validated against.
When adding new commands to an existing CMD_TABLE, re-run the same
generation script and commit the new/updated output files.
JSON integration tests (test_json_generic.py) use programmatic validation
(not baseline files), so they do not require reference output generation.
Before opening a PR, every commit must pass the full check suite. Run these checks after each commit and fix any failures before proceeding:
# Unit tests (required — must all pass)
pytest -v tests/unit
# Lint (required — must be clean)
pylint -d duplicate-code -d invalid-name sdb
ruff check sdb tests
# Format (required — must produce no diff)
yapf --diff --style google --recursive sdbDo not open a pull request until all of the above pass for every commit in the branch.
SDB supports machine-readable output for AI agent integration:
sdb --list-commands- Dumps all registered commands as JSON with metadata (names, type, docstring, input/output types).sdb -e "command" --json- Emits pipeline output as JSON instead of human-readable format.- Exit codes: 0 = success, 1 = command error, 2 = argument error.
-e CMDflag for non-interactive single-command evaluation.