A Linux command-line and interactive disk manager for plain filesystems and LUKS containers. Friendly names map to persistent hardware paths instead of unstable /dev/sdX names. Includes inventory, mounting, provisioning, health checks, rsync backups, block cloning and entropy analysis.
.
|-- diskmap.tsv # Local disk mappings (runtime-created, gitignored)
|-- diskmgr # Thin executable compatibility wrapper
|-- diskmgrlib/ # Modular application package
| |-- app.py # CLI entry point
| |-- shell.py # Composed interactive shell
| |-- shell_core.py # History, prompts, and shell lifecycle
| |-- shell_helpers.py # Compatibility composition of helper interfaces
| |-- common.py # Backward-compatible re-exports for older callers
| |-- runtime.py # Privileged execution, logging, output, and timing
| |-- devices.py # lsblk/sysfs device resolution helpers
| |-- inventory.py # Device inventory and formatted list data
| |-- mappings.py # Atomic persistent friendly-name mappings
| |-- mount_policy.py # Mount, fstab, ownership, and compression policy
| |-- mounts.py # Mountpoint discovery and cleanup helpers
| |-- rawio.py # Erase, sanitize, and overwrite primitives
| |-- entropy_sampling.py # Read-only worker, histograms, overlapping windows
| |-- safety.py # Destructive-operation preflight and identity checks
| |-- smart.py # SMART parsing and vendor decoding
| `-- commands/ # Workflow-focused command mixins
| |-- listing.py # list
| |-- boot.py # layout and boot
| |-- mapping.py # map and unmap
| |-- mounting.py # open, close, label, remount
| |-- luks.py # LUKS management
| |-- destructive.py # Compatibility alias for block commands
| |-- entropy.py # entropy sampling
| |-- health.py # SMART and filesystem health commands
| |-- transfer.py # sync and diff
| |-- filesystem.py # Compatibility composition for maintenance/provisioning
| |-- filesystem_maintenance.py # defrag, fshealth, and scrub
| |-- provisioning.py # format, convert, create, and remove
| |-- block.py # erase, nuke, entropise, and clone
| `-- partition.py # Compatibility alias for provisioning commands
|-- tests/ # Safety, fault, transfer, terminal and compatibility tests
|-- docs/
| |-- readme-overview.md # Maintained overview used by gen_readme.py
| |-- implementation-plan.json # 27-step implementation and test inventory
| `-- audit-fixes.md # Changes, verification, measurements and limits
|-- DEPENDENCIES.md # External command and capability requirements
|-- pyproject.toml # Python requirement and optional entropy extra
|-- uv.lock # Reproducible Python dependency resolution
|-- gen_readme.py # Script to regenerate this documentation
`-- README.md # This file
- Description: The executable wrapper delegates to a composed interactive shell. Shared behavior is separated into runtime, device, inventory, mapping, mount-policy, safety, raw-I/O, and SMART modules; command groups live under
diskmgrlib/commands/. - Inputs: User commands via interactive shell or one-shot CLI invocation; system hardware information via
lsblk,udevadm,cryptsetup, and related tools. - Outputs: Formatted tables, system state changes (mounts, encryption status), command logs, and updates to
diskmap.tsv.
- Description: Tab-separated values file that stores the mapping between user-defined friendly names and persistent device paths (e.g.,
/dev/disk/by-id/...). - Format:
<friendly_name>\t<persistent_device_path>
- Description: Generates this README using an isolated temporary home with privilege escalation disabled.
- Inputs:
docs/readme-overview.md, structure text ingen_readme.py, and current one-shot command help from this checkout. - Outputs:
README.md; temporary mappings/history are discarded and temporary home paths become~. Failed help capture aborts before replacing the README.
- Description: Lists Python/runtime requirements and optional command-line capabilities by workflow.
- Inputs: None.
- Outputs: Documentation only; the application does not install packages.
Requires Linux, Python 3.12+, and external tools for the commands you use. Core
Python functionality has no third-party Python dependencies. See
DEPENDENCIES.md for filesystem, encryption, transfer and
diagnostic tools. The application uses sudo for privileged helpers; launch it
as your normal user so user state and filesystem ownership belong to that user.
From the repository root:
UV_CACHE_DIR=/data/.cache/uv uv sync
UV_CACHE_DIR=/data/.cache/uv uv run python diskmgr list
UV_CACHE_DIR=/data/.cache/uv uv run python diskmgrTo expose the executable elsewhere, keep the repository in place and create a
symlink in a directory on your PATH:
mkdir -p ~/.local/bin
ln -s "$(pwd)/diskmgr" ~/.local/bin/diskmgr
diskmgr help
diskmgr help formatThe symlink uses the python3 on your PATH. To use the optional NumPy entropy
accelerator from the uv environment, launch through uv:
UV_CACHE_DIR=/data/.cache/uv uv sync --extra entropy
UV_CACHE_DIR=/data/.cache/uv uv run --extra entropy python diskmgrCommands work as diskmgr COMMAND ... or at the (diskmgr) prompt. Quote
discovery IDs in an external shell, for example diskmgr map '#3' backup,
because an unquoted # can start a shell comment. Use help COMMAND, not
COMMAND --help, for documentation. exit, quit and Ctrl+D leave the shell;
pressing Enter on an empty line does not repeat a command.
Destructive commands can erase entire disks. Check the displayed device, persistent path and PCI path before confirming. Guards reduce mistakes; they cannot make failing hardware reliable or replace a backup.
- Inspect
listorlist concise; uselist verbose(list listis an alias) for identifiers, ownership, usage, mount options and fstab details. - Assign a friendly name with
map #N backup. IDs are temporary; check the current list first. Mapping or renaming a mapping does not format or relabel the filesystem. Prefer the friendly name for later commands. - Use
open backupfor existing data. Only useformat backup --fs ext4on an intended blank, unmounted target. For partitioned layouts, usecreatefirst and map the particular partition before formatting it. - Preview with
diff /home/user /media/user/backup --depth 1. - Run
sync /home/user /media/user/backupto make that destination match the source. Sync performs its own dry-run before destructive confirmation. - Leave directories on the disk and stop applications using it, then run
close backup. Use--forceonly when killing userspace holders is acceptable.
This is an interactive sequence, not an unattended provisioning script. Replace
example paths with the mount path printed by open. There is no additive copy
command in diskmgr: sync deletes destination-only entries.
list: per-disk layout followed by a standard table; no FSUUID column. SIZE and FSAVAIL use binary units with two decimal places.list concise: one compact present-device table and a separate missing-mapping table, without partition-layout diagrams or TYPE/FSLABEL columns.list verbose/list list: key/value entries with UUID/label paths, ownership, state, mount options and available reserved/metadata accounting. Empty fields are omitted. Partition-tablemsdos/dosis displayed asmbr.- Standard/concise modes skip invisible ownership, fstab, mount-option, serial/PCI and ext4 metadata queries. Persistent paths are indexed once per refresh; detailed ext4 static metadata is cached briefly while usage counters stay live.
- Listing authenticates once before its timed probes. Probe process groups retain the authenticated terminal session. Failed inventory reads abort explicitly; failed layout reads do not masquerade as zero-sized disks or absent tables.
format accepts ext4, XFS, Btrfs, FAT32 and exFAT. A whole-disk target must be
unpartitioned and receives a whole-device filesystem, optionally inside LUKS.
A partition target is formatted within its existing boundary. format does not
create or rearrange partitions. It checks required mkfs/encryption tools first.
Normal format refuses detected content. --reformat-existing requires additional
UUID/label confirmation and does not bypass mounted-device, partition-table or
existing-LUKS protections. Destructive preflight checks signatures, mountpoints,
swap, holders and device-mapper/RAID use; failed safety probes abort. Confirmation
shows the target entry and requires the device, persistent path and PCI path.
Identity and active use are checked again after confirmation, with a nonblocking
advisory lock. Format also revalidates after passphrase generation; conversion
holds a lock and rechecks before fsck and btrfs-convert.
open waits up to 60 seconds for discovery and reports continuing waits. A
system-wide USB bridge count is a hint, not proof that the intended disk is
connected. Mount attempts have a separate bounded wait. Existing fstab entries
choose the mountpoint/options; otherwise the mapped name, filesystem label or
device name selects /media/$USER/<name> in that order. Duplicate mounts are
cleaned with normal unmounts where possible; a live different device at the
preferred path blocks mounting.
For Btrfs, HDD mounts default to compress-force=zstd:12; other media get no
automatic compression option. Override with --compress=zstd:12 or
--compress-force=zstd:12 on open/remount. New filesystem ownership is assigned
to the invoking user; FAT/exFAT use mount uid/gid options instead of unsupported
chown. label NAME LABEL --fstab creates a UUID-based /mnt/LABEL entry with
nofail and GVfs display options.
close flushes individual filesystems, unmounts normally and handles mounted
children when given a whole disk. --force can kill userspace holders, not kernel
D-state I/O. Recovery is bounded; a mounted disk must not be kernel-detached and
timed-out helpers are tracked to prevent overlapping operations. That tracking
lasts only for the current diskmgr process. A timeout does not prove kernel I/O
has stopped or that unplugging is safe. No lazy unmount is used.
On an intended blank target, format backup --luks --fs btrfs --detached-header
creates encryption with the header off-device from the start. An optional path
after --detached-header chooses another location. LUKS2 PBKDF defaults are
Argon2id, 4 GiB memory, four threads and time cost eight.
luks backup backup saves a header to ~/.local/share/diskmgr/backup by default.
Detached-header associations are keyed by persistent device path in .headers.json;
custom paths remain discoverable after friendly-name changes without moving header
bytes. Opening and LUKS management use the registered header or legacy name-based
backup when the on-disk header is absent. luks passwd gets the current passphrase
from passgen and confirms the new passphrase with two generations. Use
luks params backup time=8 memory=4GiB parallelism=4 to change PBKDF parameters
without changing the passphrase.
Keep independent, protected header backups. Losing all usable headers can make
data unrecoverable; old backups can retain old keyslot credentials. luks wipe
is destructive and intended for controlled recovery tests, not routine use.
Source/primary means copied FROM; destination/secondary means changed to
match the source. Both commands preserve hardlinks and use rsync sparse-file
handling. Sync uses archive mode, --delete, protected arguments and progress2
data. It revalidates both endpoints after confirmation and retains directory
descriptors so a detached source does not silently become an empty mount folder.
When the source is /, both commands use --one-file-system and exclude
/proc, /sys, /run, /dev, /tmp, /mnt and /media. A separate destination
filesystem under an excluded root is supported. Excluded roots may still appear
as zero-change entries in the diff tree; that does not mean their contents copy.
Sync's pre-scan estimates file payload bytes, not final physical disk usage. Its progress percentage has two decimals; speed uses byte-change intervals rather than printing zero-speed samples between unchanged counters. Live source changes can make the scan and transfer disagree; this is not an atomic snapshot backup.
Diff normally shows logical created bytes, update size deltas and deleted bytes,
with hardlink-aware accounting. Use --physical-estimates for optional allocation
estimates from file block counts; these cost additional metadata reads and cannot
predict destination compression, reflinks or metadata overhead exactly. --depth
limits displayed hierarchy, not scan depth. --fast omits the hierarchy and shows
rsync stats; --checksum reads file contents for checksum comparison.
clone SOURCE DESTuses ddrescue and starts with a fresh map each time. Resume only an unfinished run with--resume MAPFILE, keeping the adjacent identity metadata. Both identities must match, and neither device's contents may have changed outside that run. Completed/legacy maps are rejected; identity checks cannot detect arbitrary intervening content changes.eraseremoves signatures/partition metadata; it is not a full data wipe.entropisewrites an exact device-length random pass with progress and duration.nukeuses target-scoped hardware erase or supported fallbacks, refuses subsystem-wide NVMe erasure, and never reports ordinary discard as secure erase.entropy NAME --begin 0 --end 1GiB --step 1MiBsamples a range; omitted window follows an explicit step.entropy NAME 1GiB --samples 1000stitches random windows into a graph, not a physical position map. Text data and PNGs are saved under/tmp, then a viewer is attempted. Overlapping reads are reused; NumPy accelerates counting when installed. Unexpected short reads fail rather than silently publishing a complete plot.health/smartdisplay SMART diagnostics;selftest NAME --watchmonitors a test.fshealth,defragandscrubinspect or maintain supported filesystems. Btrfs defrag is non-recompression by default;--compressopts into recompression. Balance status is polled while balance runs. An extents/files ratio alone is not a reliable performance score for compressed Btrfs data.bootreads GRUB/fstab information, scans supported layouts and attempts BIOS core-image recovery. It supports whole-device, encrypted-payload, LVM and separate/bootlayouts. It does not install or repair a bootloader.
See audit fixes and verification and the 27-step implementation inventory for regression coverage, measured entropy-counting performance and hardware-test limits.
| Location | Purpose |
|---|---|
diskmap.tsv beside the resolved entrypoint |
Local <friendly_name><TAB><persistent_path> mappings, written atomically; not tracked in Git. Override with DISKMGR_MAP_FILE. |
~/.local/state/diskmgr/history |
Command history; override with DISKMGR_HISTORY. Confirmation answers are excluded. |
~/.local/share/diskmgr/<name> |
Default LUKS header backup or detached-header file. |
~/.local/share/diskmgr/.headers.json |
Device-to-detached-header associations. |
~/.local/state/diskmgr/clone-maps/ |
ddrescue maps and adjacent .map.json identity/completion metadata. |
/tmp/diskmgr_* |
Operation logs, entropy data/plots and recovered boot images; commands print their paths. |
DISKMGR_PRIVILEGE_BACKEND defaults to sudo. none/direct disable the
privilege wrapper for controlled tests or an already-root process, not to bypass
permissions. Password entry precedes short probe timeouts; subsequent probes in
an authenticated batch are noninteractive.
The wrapper dispatches to app.py, then the composed shell and command mixin.
Commands resolve mappings, use shared safety and mount helpers, then invoke
external tools through runtime.py. Transfer progress and entropy samples flow
back from child processes to the display; the entropy worker consumes device
bytes and emits sample records for plotting. Compatibility modules preserve
older import surfaces rather than duplicating implementations.
Run from the repository root:
UV_CACHE_DIR=/data/.cache/uv uv run --no-extra entropy python -m unittest discover -s tests -v
UV_CACHE_DIR=/data/.cache/uv uv run --extra entropy python -m unittest discover -s tests -v
UV_CACHE_DIR=/data/.cache/uv uv run python -m compileall -q diskmgrlib tests gen_readme.py
UV_CACHE_DIR=/data/.cache/uv uvx ruff check --select E9,F63,F7,F82 .Tests cover compatibility, destructive-operation guards, simulated device changes, transfer accounting, subprocess deadlines, private-terminal session preservation, listing failures and entropy workers. Privileged operations are mocked; tests do not format or fault-test real disks. Firmware, USB disconnects and kernel D-state recovery still require disposable-device testing. Passing unit tests is not a hardware safety guarantee.
Update docs/readme-overview.md, structure text in gen_readme.py and relevant
command help first, then regenerate and review the README:
UV_CACHE_DIR=/data/.cache/uv uv run python gen_readme.pyDisplay the physical partition layout and free space for all plugged-in disks.
Usage:
list -> standard table (default)
list concise -> concise table
list verbose -> verbose key/value entries (alias: list list)
UNDER THE HOOD:
1. Hardware Scan: Identifies all physical 'disk' devices (excluding partitions).
2. Geometry Query: Runs 'sudo parted -m <dev> unit s print free' and 'blockdev --getsz'.
3. Parsing:
- Extracts Partition Table type (gpt/mbr) and sector sizes.
- Calculates total logical sectors from blockdev output.
4. Formatting:
- Adds GPT metadata blocks (Primary/Backup) if applicable.
- Identifies 'free' space segments.
- Calculates MiB and GiB values from sector counts.
Display boot entries from GRUB and fstab detection for each partition.
UNDER THE HOOD:
Scans partition devices. If mounted, it parses /boot/grub/grub.cfg
and checks for /etc/fstab inside that mounted partition.
If unmounted or encrypted, it explains why it cannot yet read the config.
Create or modify a persistent mapping: map <name/id> <name>
Usage:
map [#1] backup Assigns friendly name to discovery ID (e.g., map #1 backup)
map 1a backup Renames an existing mapping (e.g., map 1a backup)
Note: Raw device paths (e.g., /dev/sdb) are NOT allowed.
UNDER THE HOOD:
1. Input Resolution:
- discovery ID (e.g., [#1]): Resolves the temporary device to its Persistent Device Path (PDP).
- mapping name (e.g., 1a): Selects an existing mapping for RENAME operations.
2. PDP Linking: Extracts the /dev/disk/by-id/ path for the target hardware.
3. Conflict Check: Ensures the new friendly name is not already in use.
4. Persistence: Writes the [Name <TAB> PDP] pair to diskmap.tsv.
This ensures the disk is recognized correctly regardless of USB port or device node changes.
Remove a persistent mapping: unmap <name/id>
UNDER THE HOOD:
1. Resolution:
- Name mode: removes the exact mapping name.
- ID mode (#N): resolves to a device and removes mapping(s) pointing to that device.
2. Removal: Deletes the [Name <TAB> PDP] pair(s) from the internal dictionary.
3. Persistence: Re-writes diskmap.tsv with the mapping(s) removed.
Create partition table or partition on a whole disk: create <name/id> [--gpt|--mbr] [--partition] [--start X] [--end Y]
Scope:
- Whole disks only (not partitions).
- Table creation requires prior erase: target must look erased (no partitions, no PT metadata, no signatures).
- Partition-only mode can add a partition to an existing partitioned disk.
Actions:
- --gpt / --mbr: create a new partition table (erased disk only)
- --partition:
* with --gpt/--mbr: create first partition after table creation
* without --gpt/--mbr: create an additional partition on existing table
* when --start/--end are omitted, the largest free extent is selected automatically
* overlapping ranges are rejected; existing partitions are not overwritten
Examples:
erase 1b
create 1b --gpt
create 1b --gpt --partition
create 1b --partition
create 1b --partition --start 500GiB --end 100%
create #4 --mbr --partition
Format a superfloppy disk/partition volume: format <name/id> [options]
Note: You must 'map' a disk first to give it a name before initializing it.
NUANCES & SCOPE:
1. Running format on a Partition (e.g., sda2)
- Formats inside the existing partition boundary (plain or LUKS + payload FS).
- Other partitions on the disk are untouched.
2. Running format on a Whole Disk (e.g., sda)
- Creates a superfloppy-style volume directly on the disk (plain or LUKS + payload FS).
- Refuses if the disk already has a partition table (non-destructive policy).
- To wipe partition metadata first, use: erase <name>
Options:
--fs <ext4|xfs|btrfs|fat32|exfat> Filesystem type (default: ext4)
--label <label> Set a different internal filesystem label (other than <name>)
--luks Encrypt target first with LUKS2, then format payload filesystem.
PBKDF defaults: argon2id, memory=4GiB, threads=4, time=8.
--reformat-existing
Explicitly allow replacement of an existing filesystem/signature.
Requires an additional exact UUID or label confirmation.
--detached-header [FILE]
Store LUKS header detached from the target device.
If FILE is omitted: ~/.local/share/diskmgr/<name>
UNDER THE HOOD:
1. Safety: Probes the target and every child with wipefs/blkid, checks all
mounts, swaps, kernel holders, and mapper/RAID memberships before asking
for confirmation. Probe errors fail closed.
2. Disk Type Policy:
- If target is a whole disk, it must be unpartitioned (no GPT/MBR table present).
- If target is a partition, format is applied directly within that partition.
- Existing content requires --reformat-existing and an exact UUID/label.
3. LUKS Format (only when --luks is used):
- Uses 'passgen' to generate the passphrase (not the volume key).
- Runs 'cryptsetup luksFormat' with LUKS2 encryption
(and --header FILE when --detached-header is used).
- Opens the container as /dev/mapper/<name>.
4. Filesystem:
- Plain mode (default): formats target directly with the selected filesystem.
- --luks mode: formats the opened mapper payload with the selected filesystem.
- (ext4 only): Reclaims the 5% reserved space for root using 'tune2fs -m 0'.
5. Persistence: Adds the new disk's PDP to diskmap.tsv automatically (best-effort).
Note: This is a DESTRUCTIVE operation. You must type the resolved device,
persistent path, and PCI path to proceed.
Fast metadata wipe (soft erase): erase <name/id> [--soft]
This is a fast "re-provisioning" wipe. It removes recognizable signatures and zaps GPT/MBR metadata
(when the target is a whole disk). It is NOT a secure wipe.
It performs:
- wipefs -a (and --force for whole-disk partition-table signatures)
- sgdisk --zap-all (GPT) when available
- sfdisk (MBR)
Note: This is a DESTRUCTIVE operation. You must type the resolved device,
persistent path and PCI path to proceed.
Securely erase a disk: nuke <name/id>
NUANCES & SAFETY:
- Whole Disk (sda):
Attempts supported hardware-level wipes scoped to the confirmed target.
Destroys the Partition Table and ALL partitions on the drive.
- Partition (sda2):
Whole-device hardware wipes are skipped. Uses an exact partition-length
zero overwrite with beginning/middle/end verification. On flash media,
overwriting logical sectors cannot guarantee erasure of remapped cells.
- Mapped names and discovery IDs (#N) resolve to their disk or partition;
they do not imply a partition-only target.
UNDER THE HOOD:
1. Target Resolution: Maps friendly name to a raw block device.
2. Destructive Wipe:
- NVMe: Only namespace-local Format Crypto Erase / Format Block Erase
when supported. Subsystem-wide sanitize/format is refused, even if
only one namespace is currently attached. May try secure discard.
- SATA SSD: Tries ATA Sanitize / ATA Secure Erase, then secure discard.
Ordinary discard is never reported as secure erase; an unsupported
secure method causes failure rather than a silent downgrade.
- HDD: Prioritizes (1) ATA Sanitize, (2) ATA Secure Erase (Enhanced),
(3) ATA Secure Erase (Standard), and (4) Zero Overwrite + Verify.
3. Verification: Executes 'udevadm settle' and 'sync' to ensure all operations are committed.
Note: This is a DESTRUCTIVE operation. You must type the resolved device,
persistent path and PCI path to proceed.
WARNING: This operation is IRREVERSIBLE.
Plot entropy profile over a raw device range or stitched random samples.
Range mode (explicit flags required):
entropy <name/id> --begin 0 --end 1GiB [--step 1MiB] [--window 1MiB]
Random stitched mode:
entropy <name/id> <span> --samples N
Example: entropy 1a 1GiB --samples 1000
- window size is derived as: span / N (integer bytes)
- N random windows are sampled across the whole device
- graph X axis is stitched as one contiguous span (not physical offsets)
High-entropy random overwrite on a disk or partition: entropise <name/id>
Performs a full-device single pass using /dev/urandom via dd with an
explicit byte count (count_bytes) so completion is clean at end-of-device:
dd if=/dev/urandom of=<device> bs=16M count=<device_size_bytes> count_bytes status=progress iflag=fullblock oflag=direct conv=fsync
This destroys all existing data on the target.
Remove a partition from its parent disk: remove <name/id>
Scope:
- Partition targets only.
- Whole-disk targets are refused.
Start a SMART long self-test: selftest <name/id>
Runs smartctl long test against the underlying DISK device for the mapping.
- If the mapping points to a partition, diskmgr targets the parent disk.
- If the disk transport is USB and the device is /dev/sdX, diskmgr uses:
smartctl -d sat -t long /dev/sdX
(common for USB-SATA bridges).
Display SMART health for a mapped disk: health <name/id>
Runs smartctl against the underlying DISK device for the mapping.
- If the mapping points to a partition, diskmgr automatically targets the parent disk.
- If the disk transport is USB and the device is /dev/sdX, diskmgr uses:
smartctl -d sat -x /dev/sdX
(common for USB-SATA bridges).
Clone one disk or partition to another: clone <src_name/id> <dst_name/id> [--resume MAPFILE]
WARNING (DATA DESTRUCTION):
- This command writes directly to the destination block device (like running ddrescue/dd).
- The destination is overwritten starting at byte 0. Any existing partition table,
filesystems, and files on the destination WILL BE DESTROYED.
- If the destination is larger than the source, bytes beyond the source size are
not overwritten. Old data may still physically exist there, but it will not be
referenced by the cloned partition table.
- diskmgr does NOT unmount the destination for you. Unmount/close it first to
avoid live corruption.
- If you need to sanitize the destination (secure wipe), run: nuke <dst_name>
- If you only need a fast metadata wipe for re-provisioning, run: erase <dst_name>
Note: The target disk MUST be the same size or larger than the source.
Each invocation starts a NEW clone with a fresh mapfile. To resume an
unfinished run, use clone <src> <dst> --resume <mapfile>. Neither device's
contents may have changed since that run. Completed maps cannot be reused.
STEP-BY-STEP PROCESS:
1. Resolution: Maps both friendly names to their physical device nodes (PDP).
2. Size Validation: Queries 'blockdev --getsize64' for both. Aborts if dst < src.
3. Safety Audit: Verifies that the target is NOT the system root drive.
4. Confirmation: Requires the resolved device, persistent path and PCI path for the destination.
5. Cloning: Executes ddrescue in two phases:
- Pass 1: 'ddrescue --force <src> <dst> <mapfile>'
- Retry: 'ddrescue --force -r3 <src> <dst> <mapfile>'
6. Sync: Flushes kernel buffers to ensure all data is physically committed to disk.
Note: This is a DESTRUCTIVE operation. You must type the resolved device,
persistent path and PCI path to proceed.
SCENARIOS:
- Drive to Drive:
Creates a 1:1 bit-perfect clone. The target disk becomes an identical twin,
including the Partition Table, UUIDs, and all partitions.
Note: If the target is larger, the extra space appears as 'free' at the end.
- Partition to Partition:
Copies the internal data of the source partition into the target partition.
Useful for moving a LUKS container or a specific filesystem.
Warning: Filesystem UUIDs will be duplicated; avoid mounting both simultaneously.
- Partition to Drive:
The source partition's content is written to the start of the physical disk.
This destroys the target's partition table and turns the disk into a
"partitionless" volume (e.g., a raw LUKS device).
- Drive to Partition (DANGEROUS):
Writes the source's boot sectors and partition table into the target partition.
This usually results in an unreadable "nested" structure.
CLONING & ENCRYPTION (CRITICAL):
- Source is LOCKED (e.g., clone sda sdb):
Creates a bit-perfect "Encrypted Twin." The destination remains encrypted
and requires the same password. (Recommended for backups).
- Source is OPEN (e.g., clone sda sdb):
Copies encrypted data but may capture a "dirty" filesystem state if
files are currently being written. (Close before cloning if possible).
- Source is MAPPER (e.g., clone dm-0 sdb):
Performs a "Strip-and-Clone." The destination receives RAW DECRYPTED
DATA. The resulting clone will be completely UNENCRYPTED.
Unlock (if encrypted) and mount a disk: open <name/id>
UNDER THE HOOD:
1. Identity Resolution: Looks up the friendly name in diskmap.tsv.
2. Hardware Wait: Polls for up to 60 seconds to allow for hardware spin-up/udev events.
3. Validation:
- Runs 'cryptsetup isLuks' to check for encryption.
- If NOT encrypted (Plain Disk):
* Skips decryption step.
* Verifies the existence of a valid filesystem.
* Proceeds to label detection and mounting.
4. Decryption (LUKS only):
- Executes 'passgen' to retrieve the passphrase.
- Tries 'cryptsetup open' with on-disk LUKS header first.
- If that fails, retries with detached header at ~/.local/share/diskmgr/<mapping_name> when present.
5. Mounting:
- Uses /etc/fstab mountpoint/options when an entry exists for the device.
- Otherwise identifies preferred mountpoint: /media/$USER/<label>.
- If no hardware label is present, falls back to /media/$USER/<mapping_name>.
- Ensures the directory exists and attaches the device.
- Clears a stale mount whose /dev source no longer exists before reusing the preferred path.
A preferred path occupied by an existing different device remains blocked.
- For btrfs, default mount policy is HDD => compress-force=zstd:12; non-HDD => no compression option.
- Override per command with --compress=<mode> or --compress-force=<mode>.
6. Policy Enforcement: If the disk is already mounted at a non-standard path,
it unmounts and remounts it to the preferred path.
SAFETY NOTE:
- If your mapping points to a whole disk (e.g. /dev/sda) but the actual LUKS/filesystem
lives on a partition (e.g. /dev/sda2), diskmgr will only auto-select a partition when
it is unambiguous (exactly one candidate). Otherwise it will refuse and ask you to map
the correct partition explicitly.
Unmount and lock (if encrypted) a disk: close <name/id> [--force]
UNDER THE HOOD:
1. Unmounting (Encrypted & Plain):
- With --force, terminates userspace processes holding the filesystem.
- Flushes the target filesystem, then attempts a normal unmount.
- Waits for a bounded recovery period if only kernel writeback remains.
- --force cannot kill kernel tasks stuck in uninterruptible D-state I/O.
- Treats a vanished source as an unplug and cleans stale mount directories after teardown.
- Attempts unmount by mapper path (LUKS), source path (Plain), or guessed mountpoint.
- If target is a whole disk, also unmounts mounted child partitions on that disk.
2. Locking (LUKS only):
- Commands the kernel to wipe encryption keys from RAM.
- Removes the virtual cleartext device from /dev/mapper/.
3. Audit: Checks and displays remaining active mappings for security awareness.
4. Rescue: A --force kernel detach is single-attempt and only runs after all
filesystems and child LUKS mappings have been closed.
LUKS encryption management: luks <passwd|params|backup|restore|header|wipe> [options]
Subcommands:
passwd <name>
Change the LUKS passphrase (old from passgen, new passphrase confirmed via passgen twice).
params <name> time=<N> memory=<VALUE> parallelism=<N>
Change PBKDF parameters without changing passphrase.
Example: luks params 7a time=8 memory=4GiB parallelism=4
backup <name> [file] Save the LUKS header to a file.
Default file when omitted: ~/.local/share/diskmgr/<name>
restore <name> <file> Restore the LUKS header from a file (Destructive).
header <name> Print the current LUKS header (cryptsetup luksDump).
wipe <name> Overwrite LUKS header/keyslots with random data (Destructive; test helper).
Get or set the filesystem label of an OPEN disk: label <name> [new_label] [--fstab]
UNDER THE HOOD:
1. Validation: Verifies that the disk is currently open/unlocked.
2. Identification: Queries the filesystem type (ext4, xfs, etc.) via 'lsblk'.
3. Labeling:
- ext4: Uses 'e2label' on the active device.
- xfs: Requires a temporary unmount, then uses 'xfs_admin -L', then remounts.
4. Refresh: Executes 'udevadm trigger' to force tools like 'lsblk' to see the change.
5. Optional fstab update (--fstab):
- Removes old label-based /etc/fstab entry.
- Adds UUID-based entry with mountpoint /mnt/<new_label>.
The label is written directly to the disk hardware and persists across different computers.
Remount an OPEN disk to its label mountpoint: remount <name>
This fixes "mounted twice" and "data1/data2 suffix" issues by moving the mount
to the canonical path: /mnt/<label> when the device has an /etc/fstab entry;
otherwise /media/$USER/<label>.
SAFETY RULES:
- Refuses if the target mountpoint is already mounted by a different device.
- Refuses if the target directory exists, is not a mountpoint, and is non-empty.
- Refuses only when neither /etc/fstab entry nor filesystem LABEL is available.
UNDER THE HOOD:
1. Resolve Device: Uses /dev/mapper/<name> if present, otherwise the mapped source path.
If the mapping is LUKS and not OPEN, it refuses.
2. Target Mountpoint:
- If /etc/fstab entry exists, enforce /mnt/<label> (and update the fstab mountpoint if needed).
- Otherwise, fall back to /media/$USER/<label>.
3. Preflight: Validates selected mountpoint is safe to use.
4. Unmount: Unmounts all current mount targets for the device (if any).
5. Cleanup: Removes empty old mountpoint directories under /media/$USER (best-effort rmdir).
6. Mount: Uses fstab mount when present; otherwise direct mount to LABEL path.
For btrfs, default remount policy is HDD => compress-force=zstd:12; non-HDD => no compression option.
Override per command with --compress=<mode> or --compress-force=<mode>.
Synchronize two filesystems: sync <primary> <secondary>
UNDER THE HOOD:
1. Validation: Verifies both endpoints resolve to directories.
- Mapped names must already be mounted.
- Absolute paths must exist and be directories.
2. Pre-scan: Runs rsync dry-run stats to compute planned transfer bytes.
3. Confirmation: Requires the resolved device, persistent path and PCI path for the destructive secondary target.
4. Execution: Runs rsync and reports real progress as bytes_done / planned_bytes.
Note: The SECONDARY disk will be modified to match the PRIMARY disk.
All files on the secondary that do not exist on the primary will be DELETED.
Preview differences: diff <primary_name> <secondary_name> [--depth N] [-d] [--fast] [--checksum] [--physical-estimates]
Endpoints may be mapping names/IDs (must be mounted) or absolute directory paths.
Uses rsync dry-run itemized output (primary -> secondary) and prints:
1) Change counts and byte estimates (created/modified/deleted, net change).
2) Hierarchy summary by subtree up to --depth levels.
Add --physical-estimates to read allocated blocks for changed files.
These are estimates, not a prediction of destination compression or reflinks.
Defragment a mounted filesystem: defrag <name> [--compress]
UNDER THE HOOD:
1. Validation: Verifies the disk is mapped and currently mounted.
2. Confirmation: Requires typing YES.
3. Execution:
- ext4: runs 'sudo e4defrag <mountpoint>'
- btrfs: runs 'sudo btrfs filesystem defragment -r -v <mountpoint>'
optional: add '--compress' to use '-czstd' recompression mode.
with live progress counters (total files + current directory),
then 'sudo btrfs balance start -dusage=50 <mountpoint>'
4. Recording: Stores a timestamp on the mountpoint root via:
sudo setfattr -n user.last_defrag -v "<date>" <mountpoint>
Filesystem health/diagnostics: fshealth <name>
Shows filesystem-specific diagnostic output and local "maintenance" timestamps.
- ext4: sudo tune2fs -l <device>
sudo e4defrag -c <mountpoint> (fragmentation score + extents/files ratio)
- btrfs: sudo btrfs filesystem usage <mountpoint>
sudo btrfs filesystem show <mountpoint>
sudo btrfs filesystem df <mountpoint>
sudo btrfs device stats <mountpoint>
sudo btrfs scrub status <mountpoint>
sudo compsize <mountpoint> (extents/files ratio)
- xfs: xfs_info <mountpoint>
Also reads xattrs from the mountpoint root:
user.last_defrag, user.last_scrub
Scrub a mounted btrfs filesystem: scrub <name> [--no-watch]
UNDER THE HOOD:
1. Validation: Verifies the disk is mapped and currently mounted.
2. Confirmation: Requires typing YES.
3. Execution: Runs 'sudo btrfs scrub start -B -R <mountpoint>'.
4. Recording: Stores a timestamp on the mountpoint root via:
sudo setfattr -n user.last_scrub -v "<date>" <mountpoint>
OPTIONAL:
- default (watch mode): tails kernel logs during the scrub and prints checksum errors as they happen.
- --no-watch: disable log tailing (quiet; you only get the scrub summary output).
Btrfs typically logs logical addresses (and sometimes inode numbers); diskmgr will attempt
to resolve those to paths via:
btrfs inspect-internal logical-resolve <logical> <mountpoint>
btrfs inspect-internal inode-resolve <ino> <mountpoint>
Print diskmgr version
Convert ext4 -> btrfs in place (no data copy): convert <name/id>
Uses btrfs-convert on an UNMOUNTED ext4 filesystem.
- Plain ext4 targets are supported directly.
- If target is crypto_LUKS, diskmgr tries to resolve the open payload device
(e.g. /dev/mapper/<name> or a crypt child) and convert that.
Terrydaktal 9lewis9@gmail.com