diff --git a/.gitignore b/.gitignore index 9f85289e..cb871989 100644 --- a/.gitignore +++ b/.gitignore @@ -18,3 +18,5 @@ target # Build outputs vdf-test* + +.cursor \ No newline at end of file diff --git a/client/README.md b/client/README.md new file mode 100644 index 00000000..cde6eb26 --- /dev/null +++ b/client/README.md @@ -0,0 +1,398 @@ +# qclient + +`qclient` is the Quilibrium command-line client. It manages Quilibrium nodes, +keys, tokens, messages, hypergraph data, deployments, and compute operations. + +This README is written to be usable by both humans and AI coding agents. It +documents: + +1. How to build and install `qclient`. +2. The full command tree (every subcommand with its purpose and usage string). +3. Configuration and on-disk layout. +4. How to build new clients/tooling on top of `qclient` (Go packages, + wrapping the CLI, RPC reuse). + +All command usage strings below are pulled directly from the Cobra command +definitions in [`client/cmd/`](./cmd). When in doubt, run +`qclient --help`. + +--- + +## 1. Build and install + +### Install the latest release (no build required) + +The simplest way to get `qclient` is the published installer script, +[`install-qclient.sh`](../install-qclient.sh). It detects your OS/arch, +downloads the matching signed release from +`https://releases.quilibrium.com`, installs under the platform default +install root (`/opt/quilibrium` on Linux, `/usr/local/quilibrium` on +macOS), and creates the `/usr/local/bin/qclient` symlink. + +One-liner (requires `sudo`): + +```bash +curl -sSL https://raw.githubusercontent.com/QuilibriumNetwork/ceremonyclient/refs/heads/develop/install-qclient.sh | sudo bash +``` + +Or run a local copy: + +```bash +sudo ./install-qclient.sh +``` + +Use this path when you just want to run `qclient` and do not need to +modify or build from source. + +### Build with Task (preferred for development) + +Builds are driven by [Task](https://taskfile.dev) from the **repository root**, +not from `go build` inside this module — native builds require generated +artifacts and CGO libraries that only the root `Taskfile.yaml` wires up. + +```bash +task --list + +task build_qclient_amd64_linux +task build_qclient_arm64_linux +task build_qclient_amd64_darwin +task build_qclient_arm64_darwin + +task build:release +task build:source +``` + +Outputs land in `client/build/_/qclient` (e.g. `client/build/amd64_linux/qclient`). + +### Install / link + +```bash +qclient link # symlink current binary into /usr/local/bin (sudo) +qclient update # fetch and swap in a newer qclient release +qclient uninstall # remove binaries, symlink, and client config +qclient version # print version +qclient download-signatures +``` + +### Signature verification + +By default `qclient` verifies its own binary against signed digests from the +configured signatories before every command. Controls: + +- `--signature-check=false` — skip for this invocation. +- `-y, --yes` — auto-approve and bypass. +- `QUILIBRIUM_SIGNATURE_CHECK=false` — environment default. +- `qclient config signature-check false` — persistent. +- `qclient download-signatures` — fetch the `.dgst` + `.dgst.sig.N` files. + +Running from `go run` / source will fail signature check — use +`--signature-check=false`. + +### Dev mode (custom / locally-built qclient) + +A custom-built `qclient` has no signatures and will otherwise trip the +signature check on every invocation. `qclient dev` applies a sane set of +defaults for that workflow in one shot: + +```bash +qclient dev enable # signatureCheck=false, quiet=true +qclient dev disable # signatureCheck=true, quiet=false +qclient dev # toggle based on current state +``` + +After enabling, `qclient dev` also offers to symlink the current binary +into `/usr/local/bin/qclient` (equivalent to running `qclient link`), so +subsequent shells pick up your build without further setup. Decline the +prompt to skip linking; the config changes still apply. + +### `qclient link` relocation menu + +When the current binary is outside the standard install tree +(`/opt/quilibrium/bin//` on Linux, +`/usr/local/quilibrium/bin//` on macOS), `qclient link` offers a +menu: + +1. Link the current file path as-is (recommended for dev builds). +2. Copy into the standard location, then link (non-destructive install). +3. Move into the standard location, then link. +4. Copy into a custom directory, then link. +5. Abort. + +Options 2–4 also relocate any adjacent `.dgst` / `.dgst.sig.*` sidecar +files alongside the binary, so a signed build stays verifiable after the +move. + +### Global flags + +| Flag | Default | Purpose | +| --- | --- | --- | +| `--network ` | `$QUILIBRIUM_NETWORK` | Load `~/.quilibrium/configs//` as the active network config. | +| `--signature-check` | `true` or `$QUILIBRIUM_SIGNATURE_CHECK` | Verify the `qclient` binary signature. | +| `-y, --yes` | `false` | Auto-approve; implies `--signature-check=false`. | + +--- + +## 2. Command reference + +Root: `qclient` — *Quilibrium client. CLI for managing Quilibrium nodes.* + +Top-level groups: `node`, `config`, `token`, `hypergraph`, `compute`, +`deploy`, `key`, `message`, `alias`, plus the standalone commands +`cross-mint`, `download-signatures`, `link`, `uninstall`, `update`, +`version`, `quiet`, `dev`. + +### `qclient node` — Quilibrium node commands + +| Command | Purpose | +| --- | --- | +| `install [version]` | Install Quilibrium node | +| `update [version] [--restart\|-r]` | Update the Quilibrium node version | +| `uninstall` | Uninstall Quilibrium node | +| `auto-update [enable\|disable\|status]` | Setup automatic update checks | +| `info [config-name]` | Get information about the Quilibrium node | +| `link` | Create a symlink for a specific node version | +| `clean` | Clean old node files | +| `service [command]` | Manage the Quilibrium node service (systemd/launchd) | +| `grpc enable` / `grpc disable` | Set/clear `listenGrpcMultiaddr` | +| `rest enable` / `rest disable` | Set/clear `listenRESTMultiaddr` | +| `backup enable` / `backup disable` | Toggle node backups to S3-compatible storage | +| `backup config` | Interactively configure S3 credentials, endpoint, bucket, optional bucket prefix (namespace inside the bucket, e.g. `quilibrium/backups`), region, path-style | +| `backup config print` | Print the backup configuration (credentials masked) | +| `backup schedule [cron\|disable]` | Install/print/remove a crontab entry for `backup run` (default `0 * * * *`) | +| `backup run` | Upload config dir + store/ + worker-store/* to S3 as-is (no compression) | +| `backup restore [--name] [--force] [--dry-run] [--path-map OLD=NEW]` | Download objects back to their recorded paths. Automatically remaps absolute paths across hosts/OSes (Linux ↔ macOS, different `$HOME`, different install/state dirs) using host context recorded in `manifest.json`, and rewrites embedded paths in the restored `config.yml` (logger, alias, key, db). Use `--path-map` for any leftover absolute prefixes that auto-detection can't cover. | + +#### `qclient node config` — Manage node configuration + +| Command | Purpose | +| --- | --- | +| `create [name]` | Create a default configuration file set for a node | +| `import [name] [source_directory]` | Import `config.yml` and `keys.yml` from a source directory | +| `switch [name]` | Switch the config run by the node | +| `set [key] [value]` | Set a configuration value | +| `assign-rewards [target-config-name]` | Assign rewards to a config | + +#### `qclient node log` + +| Command | Purpose | +| --- | --- | +| `view` | View node logs | +| `enable` | Enable file-based logging for the active node config | +| `disable` | Disable file-based logging for the active node config | +| `clean` | Clean node logs | + +#### `qclient node prover` — prover/shard operations + +| Command | Purpose | +| --- | --- | +| `status` | List prover status and shard allocations | +| `shards` | List shards with estimated per-frame reward | +| `shardinfo` | List all known shards with prover counts and estimated rewards | +| `join [filter...]` | Join the prover to the network | +| `leave [filter...]` | Initiate a prover leave | +| `pause [filter]` | Pause a prover | +| `resume [filter]` | Resume a prover | +| `confirm [filter...]` | Confirm prover shard allocations | +| `reject [filter...]` | Reject prover shard allocations | +| `merge` | Merge config data for prover seniority | +| `delegate ` | Delegate prover rewards | +| `manage` | Interactive prover shard management TUI | +| `alt-shard-update ` | Submit an alternative shard state update | + +### `qclient config` — QClient configuration + +| Command | Purpose | +| --- | --- | +| `print` | Print the current configuration | +| `create-default` | Create a default configuration file | +| `service-name [name]` | Set the Linux systemd service name used by the node | +| `signature-check [true\|false]` | Set signature check setting | +| `public-rpc [true\|false]` | Set public RPC setting | +| `set-custom-rpc [url\|clear]` | Set custom RPC URL | + +### `qclient token` — Token operations + +| Command | Purpose | +| --- | --- | +| `account` | Show the account address of the managing account | +| `balance` | List the total balance of tokens in the managing account | +| `coins` | List all coins under control of the managing account | +| `mint []` | Mint tokens from proof of work | +| `transfer [RefundAccount] ` | Create a transfer of coin | +| `split` | Split a coin into multiple coins | +| `merge [all\|...]` | Merge multiple coins | +| `accept ` | Accept a pending transfer | +| `reject ` | Reject a pending transaction | + +### `qclient key` — Key management + +| Command | Purpose | +| --- | --- | +| `list` | List all available keys | +| `create [Purpose]` | Create a new key (purpose is informational) | +| `import ` | Import a private key (hex) | +| `delete ` | Delete a key | +| `sign [DomainHex]` | (DANGEROUS) Sign a raw payload | + +### `qclient message` — Messaging + +| Command | Purpose | +| --- | --- | +| `send ` | Send a message (`-` reads stdin) | +| `retrieve [InboxKeyName]` | Retrieve messages | +| `show ` | Display stored messages | +| `delete ` | Delete a message | + +### `qclient hypergraph` — Hypergraph operations + +| Command | Purpose | +| --- | --- | +| `get vertex ` | Retrieve and display vertex data | +| `get hyperedge ` | Retrieve and display hyperedge data | +| `put vertex [key=value...] [EncryptionKeyBytes]` | Insert or update vertex data | +| `put hyperedge [AtomAddresses\|Aliases...]` | Insert or update hyperedge data | +| `remove vertex ` | Remove a vertex | +| `remove hyperedge ` | Remove a hyperedge | + +### `qclient deploy` — Deploy to the network + +| Command | Purpose | +| --- | --- | +| `file [EncryptionKeyBytes]` | Deploy a file to the hypergraph | +| `token [Key=Value...]` | Deploy a token | +| `hypergraph [Key=Value...] [RDFFileName]` | Deploy a hypergraph schema | +| `compute [RDFFileName]` | Deploy a QCL compute program | +| `update [Key=Value...]` | Update a deployed token configuration | +| `update [RDFFileName] [key=value...]` | Update a deployed hypergraph/compute configuration | +| `get [DecryptionKey]` | Retrieve a deployed file | + +### `qclient compute` + +| Command | Purpose | +| --- | --- | +| `execute [Rendezvous] [PartyId] [ArgKey=ArgValue...]` | Execute a compute operation | + +### `qclient alias` — Manage address aliases + +| Command | Purpose | +| --- | --- | +| `list` | List all aliases | +| `add
[type]` | Add or update an alias | +| `remove ` | Remove an alias | +| `get ` | Get address for an alias | +| `find
` | Find alias for an address | +| `resolve ` | Resolve an alias or address | + +### Standalone + +| Command | Purpose | +| --- | --- | +| `cross-mint` | Sign a payload from the Quilibrium bridge to mint tokens on Ethereum L1; prints result to stdout | +| `download-signatures` | Download signature files for the current qclient binary | +| `link` | Symlink the qclient binary into `/usr/local/bin` (sudo) | +| `uninstall` | Uninstall qclient (binaries, symlink, client config) | +| `update [version]` | Update qclient version | +| `version` | Display the qclient version | +| `quiet [enable\|disable]` | Hide informational output when signature verification succeeds | +| `dev [enable\|disable]` | Apply sane defaults for custom/locally-built binaries (`signatureCheck=false`, `quiet=true`); offers to symlink the current binary | + +--- + +## 3. Configuration and on-disk layout + +### QClient config + +Created automatically on first run (or via `qclient config create-default`). +Manage interactively with the `qclient config` subcommands. Inspect with +`qclient config print`. + +Key fields include: `SignatureCheck`, `Quiet`, custom RPC URL, public RPC +preference, and active network. See +[`client/utils/clientConfig.go`](./utils/clientConfig.go) for the full +struct. + +### Network configs + +Selected with `--network ` or `QUILIBRIUM_NETWORK`. Resolved to +`~/.quilibrium/configs//`. + +### Node install paths + +Defined in [`client/utils/paths.go`](./utils/paths.go): + +| Platform | Install dir | State dir | Symlink dir | +| --- | --- | --- | --- | +| Linux (FHS) | `/opt/quilibrium` | `/var/lib/quilibrium` | `/usr/local/bin` | +| macOS | `/usr/local/quilibrium` | `/usr/local/var/quilibrium` | `/usr/local/bin` | + +Node configs live under `~/.quilibrium/configs/` by default; file logs go +in `.logs/` inside each config dir when enabled. + +Legacy installs under `/var/quilibrium` are detected for migration warnings +only and should not be used for new installs. + +--- + +## 4. Developing on top of qclient + +There are three supported integration shapes. + +### (a) Wrap the CLI + +Most automation should shell out to `qclient`. Tips for agents: + +- Use `-y` to bypass interactive prompts, or `QUILIBRIUM_SIGNATURE_CHECK=false` + for non-interactive environments where signatures are not available. +- Use `qclient config print` to discover the active configuration. +- Many `token`, `hypergraph`, `deploy`, and `compute` commands print + machine-parseable output to stdout; prefer parsing that over scraping + help text. +- `qclient cross-mint` prints only the signed payload to stdout, making it + safe to pipe. + +### (b) Import the Go packages + +The client is a Go module at `source.quilibrium.com/quilibrium/monorepo/client`. +Useful internal packages: + +| Package | Purpose | +| --- | --- | +| [`client/utils`](./utils) | Client config loading, paths, RPC client, downloads, node helpers, file utils, types, user input. | +| [`client/utils/rpc.go`](./utils/rpc.go) | `GetGRPCClient(...)` — construct a gRPC client against the configured/custom/public RPC. | +| [`client/utils/clientConfig.go`](./utils/clientConfig.go) | `ClientConfig`, `LoadClientConfig`, `CreateDefaultConfig`, `GetConfigPath`. | +| [`client/utils/paths.go`](./utils/paths.go) | Platform-aware install, state, and symlink directories. | +| [`client/hypergraph`](./hypergraph) | Remote hypergraph helpers (see `remote.go`). | +| [`client/pkg/yamlutil`](./pkg/yamlutil) | YAML helpers used by node/nodeconfig commands. | +| [`client/cmd/...`](./cmd) | Cobra commands — useful as references for how to call RPC + keys + tokens. | + +Pattern for adding a new subcommand (mirrors everything already in `cmd/`): + +1. Create `client/cmd//.go` exposing a `var FooCmd = &cobra.Command{...}`. +2. Register it from the parent group's `init()` via `parent.AddCommand(FooCmd)`. +3. New top-level groups are wired in from + [`client/cmd/root.go`](./cmd/root.go) in `init()`. +4. Use `utils.LoadClientConfig()` and `utils.GetGRPCClient(...)` rather than + re-implementing RPC setup. +5. Never bypass the root `PersistentPreRun` signature check logic — it runs + automatically for all subcommands except `help` and `download-signatures`. + +### (c) Reuse only the RPC layer + +If you only need to talk to a node, construct a gRPC client via +`utils.GetGRPCClient` with a loaded `ClientConfig`. The generated gRPC stubs +live in the sibling `node/` module of this monorepo — see the root +`go.work` and top-level README for how the node's protobuf definitions are +exposed. + +--- + +## Quick reference for agents + +- To install a release without building: `curl -sSL https://raw.githubusercontent.com/QuilibriumNetwork/ceremonyclient/refs/heads/develop/install-qclient.sh | sudo bash` (or `sudo ./install-qclient.sh` from a checkout). +- To build from source, prefer `task build_qclient__` from the repo root. +- Always pass `-y` in non-interactive automation. +- Config lives at the path returned by `utils.GetConfigPath()`; node configs + live under `~/.quilibrium/configs//`. +- Every command supports `--help`; treat that as the source of truth. +- For new functionality, add a Cobra command under `client/cmd//` + and register it — do not fork `main.go`. diff --git a/client/cmd/config/config.go b/client/cmd/config/config.go index 88da1c79..20c39ffa 100644 --- a/client/cmd/config/config.go +++ b/client/cmd/config/config.go @@ -15,4 +15,5 @@ func init() { ConfigCmd.AddCommand(ClientConfigPublicRpcCmd) ConfigCmd.AddCommand(ClientConfigSetCustomRpcCmd) ConfigCmd.AddCommand(ClientConfigSignatureCheckCmd) + ConfigCmd.AddCommand(ClientConfigServiceNameCmd) } diff --git a/client/cmd/config/print.go b/client/cmd/config/print.go index 90c0a45b..de3f0c86 100644 --- a/client/cmd/config/print.go +++ b/client/cmd/config/print.go @@ -19,9 +19,35 @@ var ClientConfigPrintCmd = &cobra.Command{ } // Print the config in a readable format - fmt.Printf("Data Directory: %s\n", config.DataDir) + fmt.Printf("QClient Install Dir: %s\n", utils.GetQClientInstallDir()) + fmt.Printf(" QClient Binary Dir: %s\n", utils.GetQClientBinaryDir()) fmt.Printf("Symlink Path: %s\n", config.SymlinkPath) fmt.Printf("Signature Check: %v\n", config.SignatureCheck) + fmt.Printf("Quiet: %v\n", config.Quiet) fmt.Printf("Public RPC: %v\n", config.PublicRpc) + serviceName := config.NodeServiceName + if serviceName == "" { + serviceName = utils.DefaultNodeServiceName + } + fmt.Printf("Node Service Name: %s\n", serviceName) + + fmt.Printf("Node Install Dir: %s\n", utils.GetNodeInstallDir()) + fmt.Printf(" Node Binary Dir: %s\n", utils.GetNodeBinaryDir()) + fmt.Printf("Node State Dir: %s\n", utils.GetNodeStateDir()) + fmt.Printf(" Node Env File: %s\n", utils.GetNodeEnvFilePath()) + // Node log location lives in the node config's logger.path, not + // the client config. Show the active one for convenience. + if resolved, err := utils.ResolveActiveNodeLog(); err == nil { + if resolved.FileBased { + fmt.Printf("Node Log Dir: %s (from %s/config.yml)\n", + resolved.LogDir, resolved.ConfigDir) + } else { + fmt.Printf("Node Log Dir: (none; active config %q has no logger block, node logs to system log)\n", + resolved.ConfigName) + } + } + fmt.Printf("Node Symlink Dir: %s\n", utils.GetNodeSymlinkDir()) + fmt.Printf(" Node Symlink: %s\n", utils.GetNodeSymlinkPath()) + fmt.Printf("Node Configs Dir: %s\n", utils.GetNodeConfigsDir()) }, } diff --git a/client/cmd/config/service-name.go b/client/cmd/config/service-name.go new file mode 100644 index 00000000..d0fa74c8 --- /dev/null +++ b/client/cmd/config/service-name.go @@ -0,0 +1,172 @@ +package config + +import ( + "fmt" + "os" + "os/exec" + "strings" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/cmd/node" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var ClientConfigServiceNameCmd = &cobra.Command{ + Use: "service-name [name]", + Short: "Set the Linux systemd service name used by the node", + Long: `Set the name of the systemd service unit for the Quilibrium node. + +On Linux, this controls the name used for commands like: + sudo systemctl start + sudo systemctl status +and the unit file written at /etc/systemd/system/.service. + +The default is "quilibrium-node". The binary symlink at /usr/local/bin is +always created as quilibrium-node regardless of this value. + +If a systemd unit is already installed under the previous name, this command +will migrate it: the old service is stopped/disabled/removed and a new unit +file is created under the new name (preserving enabled/active state). + +Examples: + qclient config service-name my-node + qclient config service-name # prints current value`, + Args: cobra.MaximumNArgs(1), + Run: func(cmd *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Fprintf(os.Stderr, "Error loading config: %v\n", err) + os.Exit(1) + } + + current := cfg.NodeServiceName + if current == "" { + current = utils.DefaultNodeServiceName + } + + if len(args) == 0 { + fmt.Printf("Node service name: %s\n", current) + return + } + + newName := strings.TrimSpace(args[0]) + if err := utils.ValidateNodeServiceName(newName); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + + if newName == current { + fmt.Printf("Node service name is already %q, nothing to do.\n", current) + return + } + + // On Linux, if the old unit file exists we need sudo up front to be + // able to migrate cleanly. + oldUnitPath := "/etc/systemd/system/" + current + ".service" + needsMigration := utils.OsType == "linux" && utils.FileExists(oldUnitPath) + + if needsMigration { + if err := utils.CheckAndRequestSudo( + "Renaming the installed systemd service requires root privileges", + ); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + } + + // Capture prior state before we touch anything. + wasActive := needsMigration && systemctlCheck("is-active", current) + wasEnabled := needsMigration && systemctlCheck("is-enabled", current) + + if needsMigration { + fmt.Printf("Migrating installed service %q -> %q...\n", current, newName) + + if wasActive { + if err := runSystemctl("stop", current); err != nil { + fmt.Fprintf(os.Stderr, + "Warning: failed to stop %q: %v\n", current, err, + ) + } + } + if wasEnabled { + if err := runSystemctl("disable", current); err != nil { + fmt.Fprintf(os.Stderr, + "Warning: failed to disable %q: %v\n", current, err, + ) + } + } + + // Remove old unit file directly (RemoveSystemdServiceFile reads + // the configured name, which we haven't rotated yet — but we + // know the exact path here). + if err := os.Remove(oldUnitPath); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, + "Warning: failed to remove old unit file %s: %v\n", + oldUnitPath, err, + ) + } + + if err := runSystemctl("daemon-reload"); err != nil { + fmt.Fprintf(os.Stderr, + "Warning: systemctl daemon-reload failed: %v\n", err, + ) + } + } + + // Persist the new name before writing the new unit file so that + // CreateSystemdServiceFile picks it up via GetNodeServiceName(). + cfg.NodeServiceName = newName + if err := utils.SaveClientConfig(cfg); err != nil { + fmt.Fprintf(os.Stderr, "Error saving config: %v\n", err) + os.Exit(1) + } + + if needsMigration { + if err := node.CreateSystemdServiceFile(false); err != nil { + fmt.Fprintf(os.Stderr, + "Error creating new systemd service file: %v\n", err, + ) + os.Exit(1) + } + + if wasEnabled { + if err := runSystemctl("enable", newName); err != nil { + fmt.Fprintf(os.Stderr, + "Warning: failed to enable %q: %v\n", newName, err, + ) + } + } + if wasActive { + if err := runSystemctl("start", newName); err != nil { + fmt.Fprintf(os.Stderr, + "Warning: failed to start %q: %v\n", newName, err, + ) + } + } + + fmt.Printf("Service migrated. Active=%v Enabled=%v\n", wasActive, wasEnabled) + } + + fmt.Printf("Node service name set to %q.\n", newName) + if !needsMigration && utils.OsType == "linux" { + fmt.Println( + "No existing systemd unit was found under the previous name; " + + "the new name will take effect the next time you install or " + + "update the service (e.g. `sudo qclient node service install`).", + ) + } + }, +} + +// systemctlCheck returns true when `systemctl ` exits 0. +func systemctlCheck(subcmd, unit string) bool { + cmd := exec.Command("systemctl", subcmd, unit) + return cmd.Run() == nil +} + +// runSystemctl runs `sudo systemctl ` and returns its error. +func runSystemctl(args ...string) error { + full := append([]string{"systemctl"}, args...) + cmd := exec.Command("sudo", full...) + return cmd.Run() +} diff --git a/client/cmd/crossMint.go b/client/cmd/crossMint.go index 6f849faa..1652f968 100644 --- a/client/cmd/crossMint.go +++ b/client/cmd/crossMint.go @@ -22,8 +22,9 @@ import ( ) var ( - NodeConfig *config.Config - ConfigDirectory string + NodeConfig *config.Config + ConfigDirectory string + resolvedConfigDirectory string ) var CrossMintCmd = &cobra.Command{ @@ -44,8 +45,18 @@ var CrossMintCmd = &cobra.Command{ var nodeConfig *config.Config var err error if ConfigDirectory != "" && ConfigDirectory != "default" { - nodeConfig, err = utils.LoadNodeConfig(ConfigDirectory) + resolvedConfigDirectory, err = utils.ResolveNodeConfigDir(ConfigDirectory) + if err != nil { + fmt.Printf("error loading node config: %s\n", err) + os.Exit(1) + } + nodeConfig, err = utils.LoadNodeConfig(resolvedConfigDirectory) } else { + resolvedConfigDirectory, err = utils.GetDefaultNodeConfigDir() + if err != nil { + fmt.Printf("error loading node config: %s\n", err) + os.Exit(1) + } nodeConfig, err = utils.LoadDefaultNodeConfig() } if err != nil { @@ -88,7 +99,7 @@ var CrossMintCmd = &cobra.Command{ // account if it was changed. if !filepath.IsAbs(NodeConfig.Key.KeyStoreFile.Path) { NodeConfig.Key.KeyStoreFile.Path = filepath.Join( - ConfigDirectory, + resolvedConfigDirectory, filepath.Base(NodeConfig.Key.KeyStoreFile.Path), ) } diff --git a/client/cmd/dev.go b/client/cmd/dev.go new file mode 100644 index 00000000..d68f6db4 --- /dev/null +++ b/client/cmd/dev.go @@ -0,0 +1,106 @@ +package cmd + +import ( + "fmt" + "os" + "strings" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var DevCmd = &cobra.Command{ + Use: "dev [enable|disable]", + Short: "Toggle developer-friendly defaults for custom qclient builds", + Long: `Dev mode applies sane defaults for locally-built / unsigned qclient binaries: + + enable: + - signatureCheck = false (skip signature verification) + - quiet = true (suppress informational output) + + disable: + - signatureCheck = true (restore signature verification) + - quiet = false (restore informational output) + +With no argument, the current state is toggled based on the signatureCheck flag +(dev mode is considered "enabled" when signatureCheck is false).`, + Run: func(_ *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Printf("Error loading config: %v\n", err) + os.Exit(1) + } + + var enable bool + if len(args) > 0 { + switch strings.ToLower(args[0]) { + case "enable": + enable = true + case "disable": + enable = false + default: + fmt.Printf("Error: Invalid value '%s'. Please use 'enable' or 'disable'.\n", args[0]) + os.Exit(1) + } + } else { + enable = cfg.SignatureCheck + } + + if enable { + cfg.SignatureCheck = false + cfg.Quiet = true + } else { + cfg.SignatureCheck = true + cfg.Quiet = false + } + + if err := utils.SaveClientConfig(cfg); err != nil { + fmt.Printf("Error saving config: %v\n", err) + os.Exit(1) + } + + status := "disabled" + if enable { + status = "enabled" + } + fmt.Printf("Dev mode has been %s (signatureCheck=%v, quiet=%v).\n", + status, cfg.SignatureCheck, cfg.Quiet) + + if enable { + maybeLinkDevBinary() + } + }, +} + +func maybeLinkDevBinary() { + execPath, err := os.Executable() + if err != nil { + fmt.Printf("Skipping link prompt: cannot resolve current executable: %v\n", err) + return + } + + if existing, err := os.Readlink(symlinkPath); err == nil && existing == execPath { + fmt.Printf("%s already points at this binary.\n", symlinkPath) + return + } + + fmt.Printf("Link this dev binary at %s -> %s? (y/n): ", symlinkPath, execPath) + var response string + fmt.Scanln(&response) + response = strings.ToLower(strings.TrimSpace(response)) + if response != "y" && response != "yes" { + fmt.Println("Skipping symlink.") + return + } + + if !utils.IsSudo() { + fmt.Printf("Cannot create symlink at %s without sudo. Re-run: sudo qclient link\n", symlinkPath) + return + } + + if err := utils.CreateSymlink(execPath, symlinkPath); err != nil { + fmt.Printf("Failed to create symlink: %v\n", err) + return + } + fmt.Printf("Symlink created at %s\n", symlinkPath) +} diff --git a/client/cmd/link.go b/client/cmd/link.go index 5bcf81be..8bdfcb73 100644 --- a/client/cmd/link.go +++ b/client/cmd/link.go @@ -1,7 +1,9 @@ package cmd import ( + "bufio" "fmt" + "io" "os" "path/filepath" "strings" @@ -20,78 +22,202 @@ var LinkCmd = &cobra.Command{ Example: qclient link`, RunE: func(cmd *cobra.Command, args []string) error { - // Get the path to the current executable execPath, err := os.Executable() if err != nil { return fmt.Errorf("failed to get executable path: %w", err) } - IsSudo := utils.IsSudo() - if IsSudo { - fmt.Println("Running as sudo, creating symlink at /usr/local/bin/qclient") + if utils.IsSudo() { + fmt.Printf("Running as sudo, creating symlink at %s\n", symlinkPath) } else { - fmt.Println("Cannot create symlink at /usr/local/bin/qclient, please run this command with sudo") + fmt.Printf("Cannot create symlink at %s, please run this command with sudo\n", symlinkPath) os.Exit(1) } - // Check if the current executable is in the expected location - expectedPrefix := utils.ClientDataPath + expectedPrefix := utils.GetQClientBinaryDir() - // Check if the current executable is in the expected location if !strings.HasPrefix(execPath, expectedPrefix) { - fmt.Printf("Current executable is not in the expected location: %s\n", execPath) - fmt.Printf("Expected location should start with: %s\n", expectedPrefix) - - // Ask user if they want to move it - fmt.Print("Would you like to move the executable to the standard location? (y/n): ") - var response string - fmt.Scanln(&response) - - if strings.ToLower(response) == "y" || strings.ToLower(response) == "yes" { - if err := moveExecutableToStandardLocation(execPath); err != nil { - return fmt.Errorf("failed to move executable: %w", err) - } - // Update execPath to the new location - execPath, err = os.Executable() - if err != nil { - return fmt.Errorf("failed to get new executable path: %w", err) - } - } else { - fmt.Println("Continuing with current location...") + newPath, err := promptRelocateExecutable(execPath, expectedPrefix) + if err != nil { + return err } + if newPath == "" { + fmt.Println("Aborted. No symlink created.") + return nil + } + execPath = newPath } - // Create the symlink (handles existing symlinks) if err := utils.CreateSymlink(execPath, symlinkPath); err != nil { return err } - fmt.Printf("Symlink created at %s\n", symlinkPath) + fmt.Printf("Symlink created at %s -> %s\n", symlinkPath, execPath) return nil }, } -func moveExecutableToStandardLocation(execPath string) error { - // Get the directory of the current executable +// promptRelocateExecutable asks the user how to handle a qclient binary that +// lives outside the standard install tree. Returns the path the symlink +// should target, or "" if the user aborted. +func promptRelocateExecutable(execPath, expectedPrefix string) (string, error) { + standardDir := filepath.Join(expectedPrefix, "bin", "") + + fmt.Println() + fmt.Println("Current executable is not in the standard location.") + fmt.Printf(" Current path: %s\n", execPath) + fmt.Printf(" Standard path: %s/\n", standardDir) + fmt.Println() + fmt.Println("Choose how to link this binary:") + fmt.Printf(" [1] Link the current file path as-is (recommended for dev builds)\n") + fmt.Printf(" symlink -> %s\n", execPath) + fmt.Printf(" [2] Copy into the standard location, then link (install this build, non-destructive)\n") + fmt.Printf(" [3] Move into the standard location, then link (removes binary from current path)\n") + fmt.Printf(" [4] Copy into a custom directory, then link\n") + fmt.Printf(" [a] Abort\n") + fmt.Print("Choice [1/2/3/4/a] (default 1): ") + + reader := bufio.NewReader(os.Stdin) + line, _ := reader.ReadString('\n') + choice := strings.ToLower(strings.TrimSpace(line)) + if choice == "" { + choice = "1" + } + + switch choice { + case "1", "y", "yes": + fmt.Printf("Linking current file path as-is: %s\n", execPath) + return execPath, nil + + case "2", "copy": + destDir, err := standardVersionedDir() + if err != nil { + return "", err + } + return relocateExecutable(execPath, destDir, false) + + case "3", "move", "n", "no": + destDir, err := standardVersionedDir() + if err != nil { + return "", err + } + return relocateExecutable(execPath, destDir, true) + + case "4", "custom": + fmt.Print("Enter destination directory: ") + dirLine, _ := reader.ReadString('\n') + destDir := strings.TrimSpace(dirLine) + if destDir == "" { + return "", fmt.Errorf("no destination directory provided") + } + return relocateExecutable(execPath, destDir, false) + + case "a", "abort", "q", "quit": + return "", nil + + default: + return "", fmt.Errorf("invalid choice %q", choice) + } +} + +// standardVersionedDir returns /bin//, creating +// it (with the invoking sudo user as owner) if necessary. +func standardVersionedDir() (string, error) { version, err := GetVersionInfo(false) if err != nil { - return fmt.Errorf("failed to get version info: %w", err) + return "", fmt.Errorf("failed to get version info: %w", err) } - destDir := filepath.Join(utils.ClientDataPath, "bin", version.Version) + // NB: historical layout — keep the extra "bin" segment to match + // what older qclient link behavior produced. + return filepath.Join(utils.GetQClientBinaryDir(), "bin", version.Version), nil +} - // Create the standard location directory if it doesn't exist +// relocateExecutable copies or moves the qclient binary (and any +// .dgst / .dgst.sig.N sidecar files sitting next to it) into destDir, +// renaming the binary to StandardizedQClientFileName. Returns the new +// path to the binary. +func relocateExecutable(execPath, destDir string, move bool) (string, error) { currentUser, err := utils.GetCurrentSudoUser() if err != nil { - return fmt.Errorf("failed to get current user: %w", err) + return "", fmt.Errorf("failed to get current user: %w", err) } if err := utils.ValidateAndCreateDir(destDir, currentUser); err != nil { - return fmt.Errorf("failed to create directory: %w", err) + return "", fmt.Errorf("failed to create directory %s: %w", destDir, err) + } + + verb := "Copying" + if move { + verb = "Moving" + } + + destBinary := filepath.Join(destDir, StandardizedQClientFileName) + fmt.Printf("%s binary: %s -> %s\n", verb, execPath, destBinary) + if err := relocateFile(execPath, destBinary, move); err != nil { + return "", fmt.Errorf("failed to relocate executable: %w", err) + } + + // Relocate sidecar digest / signature files if they exist next to + // the source binary. These share the binary's basename. + sidecarSuffixes := []string{".dgst"} + // Pick up any .dgst.sig.* siblings too. + if matches, err := filepath.Glob(execPath + ".dgst.sig.*"); err == nil { + for _, m := range matches { + sidecarSuffixes = append(sidecarSuffixes, strings.TrimPrefix(m, execPath)) + } } - // Move the executable to the standard location - if err := os.Rename(execPath, filepath.Join(destDir, StandardizedQClientFileName)); err != nil { - return fmt.Errorf("failed to move executable: %w", err) + for _, suffix := range sidecarSuffixes { + srcSidecar := execPath + suffix + if _, err := os.Stat(srcSidecar); err != nil { + continue + } + dstSidecar := filepath.Join(destDir, StandardizedQClientFileName+suffix) + fmt.Printf("%s sidecar: %s -> %s\n", verb, srcSidecar, dstSidecar) + if err := relocateFile(srcSidecar, dstSidecar, move); err != nil { + return "", fmt.Errorf("failed to relocate sidecar %s: %w", srcSidecar, err) + } } - return nil + return destBinary, nil +} + +// relocateFile moves or copies a single file, preserving the source's +// permission bits (important for the executable). +func relocateFile(src, dst string, move bool) error { + if move { + if err := os.Rename(src, dst); err == nil { + return nil + } + // Fall through to copy+remove in case of cross-device rename. + if err := copyFilePreservePerms(src, dst); err != nil { + return err + } + return os.Remove(src) + } + return copyFilePreservePerms(src, dst) +} + +func copyFilePreservePerms(src, dst string) error { + srcInfo, err := os.Stat(src) + if err != nil { + return err + } + in, err := os.Open(src) + if err != nil { + return err + } + defer in.Close() + + out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, srcInfo.Mode().Perm()) + if err != nil { + return err + } + if _, err := io.Copy(out, in); err != nil { + out.Close() + return err + } + if err := out.Close(); err != nil { + return err + } + return os.Chmod(dst, srcInfo.Mode().Perm()) } diff --git a/client/cmd/node/backup/backup.go b/client/cmd/node/backup/backup.go new file mode 100644 index 00000000..55a9963b --- /dev/null +++ b/client/cmd/node/backup/backup.go @@ -0,0 +1,35 @@ +package backup + +import ( + "github.com/spf13/cobra" +) + +// BackupCmd is the root of `qclient node backup`. +var BackupCmd = &cobra.Command{ + Use: "backup", + Short: "Manage node backups to S3-compatible object storage", + Long: `Configure and control node backups to any S3-compatible endpoint +(AWS S3, Quilibrium qstorage, MinIO, Backblaze B2, etc.). + +Typical flow: + + qclient node backup config # interactive setup (credentials, bucket, etc.) + qclient node backup config print # show the persisted configuration + qclient node backup enable # turn backups on + qclient node backup disable # turn backups off + qclient node backup schedule "0 * * * *" # install hourly cron + qclient node backup run # back up now (config + store + worker-store) + qclient node backup restore # download a backup to its original paths`, + Run: func(cmd *cobra.Command, args []string) { + cmd.Help() + }, +} + +func init() { + BackupCmd.AddCommand(enableCmd) + BackupCmd.AddCommand(disableCmd) + BackupCmd.AddCommand(configCmd) + BackupCmd.AddCommand(scheduleCmd) + BackupCmd.AddCommand(runCmd) + BackupCmd.AddCommand(restoreCmd) +} diff --git a/client/cmd/node/backup/config.go b/client/cmd/node/backup/config.go new file mode 100644 index 00000000..29e3b4c8 --- /dev/null +++ b/client/cmd/node/backup/config.go @@ -0,0 +1,287 @@ +package backup + +import ( + "bufio" + "context" + "fmt" + "os" + "strings" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + awsconfig "github.com/aws/aws-sdk-go-v2/config" + "github.com/aws/aws-sdk-go-v2/credentials" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/spf13/cobra" + "golang.org/x/term" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var configCmd = &cobra.Command{ + Use: "config", + Short: "Configure node backup S3-compatible storage", + Long: `Interactively configure the S3-compatible storage used by +` + "`qclient node backup`" + `. Values are persisted in the qclient config. + +Prompts for: + - access key ID + - secret access key (hidden) + - endpoint (default: ` + utils.DefaultBackupEndpoint + `) + - bucket (required, no default) + - bucket prefix (optional, e.g. "quilibrium/backups"; empty = bucket root) + - region (default: ` + utils.DefaultBackupRegion + `) + - path-style (default: true)`, + Run: func(cmd *cobra.Command, args []string) { + if err := runConfig(); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + }, +} + +var configPrintCmd = &cobra.Command{ + Use: "print", + Short: "Print the current node backup configuration", + Run: func(cmd *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Fprintf(os.Stderr, "Error loading config: %v\n", err) + os.Exit(1) + } + b := cfg.Backup + fmt.Printf("Enabled: %v\n", b.Enabled) + fmt.Printf("Access Key ID: %s\n", maskCred(b.AccessKeyID)) + fmt.Printf("Secret Access Key: %s\n", maskCred(b.SecretAccessKey)) + endpoint := b.Endpoint + if endpoint == "" { + endpoint = utils.DefaultBackupEndpoint + " (default)" + } + fmt.Printf("Endpoint: %s\n", endpoint) + bucket := b.Bucket + if bucket == "" { + bucket = "(unset)" + } + fmt.Printf("Bucket: %s\n", bucket) + bucketPrefix := b.BucketPrefix + if bucketPrefix == "" { + bucketPrefix = "(none — store at bucket root)" + } + fmt.Printf("Bucket Prefix: %s\n", bucketPrefix) + region := b.Region + if region == "" { + region = utils.DefaultBackupRegion + " (default)" + } + fmt.Printf("Region: %s\n", region) + fmt.Printf("Use Path Style: %v\n", b.UsePathStyle) + }, +} + +func init() { + configCmd.AddCommand(configPrintCmd) +} + +func runConfig() error { + cfg, err := utils.LoadClientConfig() + if err != nil { + return fmt.Errorf("load config: %w", err) + } + b := cfg.Backup + + reader := bufio.NewReader(os.Stdin) + + accessKey, err := promptString(reader, "Access Key ID", b.AccessKeyID, true) + if err != nil { + return err + } + secret, err := promptSecret("Secret Access Key", b.SecretAccessKey != "") + if err != nil { + return err + } + if secret == "" { + secret = b.SecretAccessKey + } + endpoint, err := promptString(reader, "Endpoint", firstNonEmpty(b.Endpoint, utils.DefaultBackupEndpoint), false) + if err != nil { + return err + } + bucket, err := promptString(reader, "Bucket", b.Bucket, true) + if err != nil { + return err + } + bucketPrefix, err := promptString(reader, "Bucket Prefix (optional, blank for none)", b.BucketPrefix, false) + if err != nil { + return err + } + bucketPrefix = normalizeBucketPrefix(bucketPrefix) + region, err := promptString(reader, "Region", firstNonEmpty(b.Region, utils.DefaultBackupRegion), false) + if err != nil { + return err + } + defaultPathStyle := utils.DefaultBackupUsePathStyle + if b.Bucket != "" || b.AccessKeyID != "" { + defaultPathStyle = b.UsePathStyle + } + usePathStyle, err := promptBool(reader, "Use path-style addressing", defaultPathStyle) + if err != nil { + return err + } + + cfg.Backup = utils.NodeBackupConfig{ + Enabled: b.Enabled, + AccessKeyID: accessKey, + SecretAccessKey: secret, + Endpoint: endpoint, + Bucket: bucket, + BucketPrefix: bucketPrefix, + Region: region, + UsePathStyle: usePathStyle, + } + + if err := utils.SaveClientConfig(cfg); err != nil { + return fmt.Errorf("save config: %w", err) + } + fmt.Println("Backup configuration saved.") + + if ok, err := verifyBucket(&cfg.Backup); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not verify bucket access: %v\n", err) + } else if ok { + fmt.Printf("Verified access to bucket %q.\n", cfg.Backup.Bucket) + } + + return nil +} + +func promptString(r *bufio.Reader, label, def string, required bool) (string, error) { + for { + if def != "" { + fmt.Printf("%s [%s]: ", label, def) + } else if required { + fmt.Printf("%s (required): ", label) + } else { + fmt.Printf("%s: ", label) + } + line, err := r.ReadString('\n') + if err != nil { + return "", err + } + line = strings.TrimSpace(line) + if line == "" { + line = def + } + if required && line == "" { + fmt.Println(" value is required") + continue + } + return line, nil + } +} + +func promptSecret(label string, hasExisting bool) (string, error) { + if hasExisting { + fmt.Printf("%s [keep existing, press enter to keep]: ", label) + } else { + fmt.Printf("%s: ", label) + } + if !term.IsTerminal(int(os.Stdin.Fd())) { + r := bufio.NewReader(os.Stdin) + line, err := r.ReadString('\n') + if err != nil { + return "", err + } + return strings.TrimSpace(line), nil + } + buf, err := term.ReadPassword(int(os.Stdin.Fd())) + fmt.Println() + if err != nil { + return "", err + } + return strings.TrimSpace(string(buf)), nil +} + +func promptBool(r *bufio.Reader, label string, def bool) (bool, error) { + defStr := "y" + if !def { + defStr = "n" + } + for { + fmt.Printf("%s [y/n] (default %s): ", label, defStr) + line, err := r.ReadString('\n') + if err != nil { + return false, err + } + line = strings.ToLower(strings.TrimSpace(line)) + switch line { + case "": + return def, nil + case "y", "yes", "true", "1": + return true, nil + case "n", "no", "false", "0": + return false, nil + default: + fmt.Println(" please answer y or n") + } + } +} + +func firstNonEmpty(a, b string) string { + if a != "" { + return a + } + return b +} + +// maskCred shows the first 4 characters of a credential and masks the +// rest. Short credentials (<=4 chars) are fully masked. +func maskCred(s string) string { + if s == "" { + return "(unset)" + } + if len(s) <= 4 { + return strings.Repeat("*", len(s)) + } + return s[:4] + strings.Repeat("*", len(s)-4) +} + +// verifyBucket attempts a HeadBucket call against the configured +// endpoint to confirm the credentials and bucket are usable. +func verifyBucket(b *utils.NodeBackupConfig) (bool, error) { + client, err := newS3Client(b) + if err != nil { + return false, err + } + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + _, err = client.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: aws.String(b.Bucket)}) + if err != nil { + return false, err + } + return true, nil +} + +func newS3Client(b *utils.NodeBackupConfig) (*s3.Client, error) { + region := b.Region + if region == "" { + region = utils.DefaultBackupRegion + } + cfg, err := awsconfig.LoadDefaultConfig( + context.Background(), + awsconfig.WithRegion(region), + awsconfig.WithCredentialsProvider( + credentials.NewStaticCredentialsProvider(b.AccessKeyID, b.SecretAccessKey, ""), + ), + ) + if err != nil { + return nil, err + } + opts := []func(*s3.Options){ + func(o *s3.Options) { + o.UsePathStyle = b.UsePathStyle + }, + } + if b.Endpoint != "" { + opts = append(opts, func(o *s3.Options) { + o.BaseEndpoint = aws.String(b.Endpoint) + }) + } + return s3.NewFromConfig(cfg, opts...), nil +} diff --git a/client/cmd/node/backup/disable.go b/client/cmd/node/backup/disable.go new file mode 100644 index 00000000..15530977 --- /dev/null +++ b/client/cmd/node/backup/disable.go @@ -0,0 +1,27 @@ +package backup + +import ( + "fmt" + "os" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var disableCmd = &cobra.Command{ + Use: "disable", + Short: "Disable node backups", + Run: func(cmd *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Fprintf(os.Stderr, "Error loading config: %v\n", err) + os.Exit(1) + } + cfg.Backup.Enabled = false + if err := utils.SaveClientConfig(cfg); err != nil { + fmt.Fprintf(os.Stderr, "Error saving config: %v\n", err) + os.Exit(1) + } + fmt.Println("Node backups disabled.") + }, +} diff --git a/client/cmd/node/backup/enable.go b/client/cmd/node/backup/enable.go new file mode 100644 index 00000000..c56b76a8 --- /dev/null +++ b/client/cmd/node/backup/enable.go @@ -0,0 +1,46 @@ +package backup + +import ( + "fmt" + "os" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var enableCmd = &cobra.Command{ + Use: "enable", + Short: "Enable node backups", + Long: `Enable node backups to the configured S3-compatible endpoint. + +Requires that ` + "`qclient node backup config`" + ` has been run first so that +bucket, credentials, and endpoint are set.`, + Run: func(cmd *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Fprintf(os.Stderr, "Error loading config: %v\n", err) + os.Exit(1) + } + if err := validateBackupConfigured(&cfg.Backup); err != nil { + fmt.Fprintf(os.Stderr, "Cannot enable backups: %v\n", err) + fmt.Fprintln(os.Stderr, "Run `qclient node backup config` first.") + os.Exit(1) + } + cfg.Backup.Enabled = true + if err := utils.SaveClientConfig(cfg); err != nil { + fmt.Fprintf(os.Stderr, "Error saving config: %v\n", err) + os.Exit(1) + } + fmt.Println("Node backups enabled.") + }, +} + +func validateBackupConfigured(b *utils.NodeBackupConfig) error { + if b.Bucket == "" { + return fmt.Errorf("bucket is not set") + } + if b.AccessKeyID == "" || b.SecretAccessKey == "" { + return fmt.Errorf("credentials are not set") + } + return nil +} diff --git a/client/cmd/node/backup/portable.go b/client/cmd/node/backup/portable.go new file mode 100644 index 00000000..7ebe1411 --- /dev/null +++ b/client/cmd/node/backup/portable.go @@ -0,0 +1,243 @@ +package backup + +import ( + "fmt" + "path/filepath" + "runtime" + "sort" + "strings" + + "source.quilibrium.com/quilibrium/monorepo/client/utils" + nodeconfig "source.quilibrium.com/quilibrium/monorepo/config" +) + +// pathMap is an ordered list of (old, new) path prefix substitutions +// used to translate absolute paths recorded on the backup source host +// to equivalent paths on the restore destination host. Order matters: +// more specific prefixes (longer) are applied before less specific +// ones so that a configs dir nested inside $HOME is rewritten via the +// configs-dir entry instead of the broader $HOME entry. +type pathMap struct { + entries []pathMapEntry +} + +type pathMapEntry struct { + old string + new string +} + +func (m *pathMap) add(oldP, newP string) { + oldP = strings.TrimRight(oldP, string(filepath.Separator)) + newP = strings.TrimRight(newP, string(filepath.Separator)) + if oldP == "" || newP == "" || oldP == newP { + return + } + for _, e := range m.entries { + if e.old == oldP { + return + } + } + m.entries = append(m.entries, pathMapEntry{old: oldP, new: newP}) +} + +// finalize sorts entries by descending length of the old prefix so +// longest-match wins during apply. +func (m *pathMap) finalize() { + sort.SliceStable(m.entries, func(i, j int) bool { + return len(m.entries[i].old) > len(m.entries[j].old) + }) +} + +// apply returns (rewritten, true) if any prefix matched p, otherwise +// (p, false). Matching is by path component boundary: "/a/b" matches +// "/a/b" and "/a/b/c" but not "/a/boom". +func (m *pathMap) apply(p string) (string, bool) { + if p == "" { + return p, false + } + for _, e := range m.entries { + if p == e.old { + return e.new, true + } + if strings.HasPrefix(p, e.old+string(filepath.Separator)) || + strings.HasPrefix(p, e.old+"/") { + rest := p[len(e.old):] + return e.new + rest, true + } + } + return p, false +} + +// buildPathMap constructs the default automatic path map by diffing the +// manifest's recorded source-host paths against the destination host's +// equivalents, and then overlays the user-supplied --path-map flags. +// Returns the map plus the destination configDir for the restored +// config (used when rewriting the "files/" half of object keys). +func buildPathMap(mf *manifest, userMaps []string) (*pathMap, string, error) { + pm := &pathMap{} + + dstConfigsDir := utils.GetNodeConfigsDir() + dstConfigDir := filepath.Join(dstConfigsDir, mf.ConfigName) + dstStateDir := utils.GetNodeStateDir() + dstInstallDir := utils.GetNodeInstallDir() + dstHome := currentHomeDir() + + // Per-config dir is the most specific entry and must win when the + // old configs dir happened to be nested under the old home. + if mf.ConfigDir != "" { + pm.add(mf.ConfigDir, dstConfigDir) + } + if mf.ConfigsDir != "" { + pm.add(mf.ConfigsDir, dstConfigsDir) + } + if mf.NodeStateDir != "" { + pm.add(mf.NodeStateDir, dstStateDir) + } + if mf.NodeInstallDir != "" { + pm.add(mf.NodeInstallDir, dstInstallDir) + } + if mf.Home != "" && dstHome != "" { + pm.add(mf.Home, dstHome) + } + + // User-supplied maps take precedence over auto-detected ones. We + // achieve "precedence" by inserting them first and having apply() + // be longest-match, but also by replacing any auto entry with the + // same key. + for _, raw := range userMaps { + oldP, newP, err := parsePathMap(raw) + if err != nil { + return nil, "", err + } + // If the user overrides an auto entry, drop the auto one. + filtered := pm.entries[:0] + for _, e := range pm.entries { + if e.old != oldP { + filtered = append(filtered, e) + } + } + pm.entries = filtered + pm.add(oldP, newP) + } + + pm.finalize() + return pm, dstConfigDir, nil +} + +// parsePathMap parses a single "OLD=NEW" argument. +func parsePathMap(raw string) (string, string, error) { + idx := strings.Index(raw, "=") + if idx <= 0 || idx == len(raw)-1 { + return "", "", fmt.Errorf("expected OLD=NEW, got %q", raw) + } + oldP := strings.TrimSpace(raw[:idx]) + newP := strings.TrimSpace(raw[idx+1:]) + if !filepath.IsAbs(oldP) || !filepath.IsAbs(newP) { + return "", "", fmt.Errorf("path-map entries must be absolute paths: %q", raw) + } + return oldP, newP, nil +} + +// destPathFor returns the local filesystem destination for a given +// backup entry, honoring the configName-prefix-relative object key to +// remap files/ entries into the destination configDir, and applying +// prefix remapping to absolute/ entries. +// +// - Entries under "/files/" land at +// filepath.Join(dstConfigDir, rel). +// - Entries under "/absolute/" are +// run through the path map; unmapped paths fall back to LocalPath. +// - Version 1 manifests (no host context) always fall back to +// LocalPath unless an entry is remapped by a user --path-map. +func destPathFor(mf *manifest, bf *backupFile, pm *pathMap, dstConfigDir string) (string, bool) { + configName := strings.TrimSuffix(mf.ConfigName, "/") + key := bf.ObjectKey + + // An optional bucket prefix may precede "/". Locate + // the "/files/" or "/absolute/" segment + // anywhere in the key so both prefixed and un-prefixed backups + // classify correctly. + filesMarker := "/" + configName + "/files/" + if strings.HasPrefix(key, configName+"/files/") && mf.Version >= 2 { + rel := strings.TrimPrefix(key, configName+"/files/") + return filepath.Join(dstConfigDir, filepath.FromSlash(rel)), true + } + if idx := strings.Index(key, filesMarker); idx >= 0 && mf.Version >= 2 { + rel := key[idx+len(filesMarker):] + return filepath.Join(dstConfigDir, filepath.FromSlash(rel)), true + } + + // For absolute/ entries, or for any entry in a v1 manifest, try + // the prefix map against LocalPath. + if mapped, ok := pm.apply(bf.LocalPath); ok { + return mapped, true + } + return bf.LocalPath, false +} + +// rewriteRestoredConfig opens the restored config.yml at configDir, +// rewrites any embedded absolute paths through pm, and saves it back. +// The rewrite only affects path-bearing fields (logger.path, +// alias.aliasFile.path, key.keyManagerFile.path, db.path, +// db.workerPaths, db.workerPathPrefix). Paths that do not match any +// mapping entry are left untouched so operator-pinned absolute paths +// outside the managed roots survive restore. +// +// Returns the list of fields that were actually changed (for human +// output), or a non-nil error if load/save fails. A missing config.yml +// is not an error: cross-platform restore may legitimately skip the +// config half. +func rewriteRestoredConfig(configDir string, pm *pathMap) ([]string, error) { + cfgPath := filepath.Join(configDir, "config.yml") + cfg, err := nodeconfig.NewConfig(cfgPath) + if err != nil { + return nil, err + } + + var changed []string + rewrite := func(label string, p *string) { + if p == nil || *p == "" { + return + } + if mapped, ok := pm.apply(*p); ok { + *p = mapped + changed = append(changed, label) + } + } + + if cfg.Logger != nil { + rewrite("logger.path", &cfg.Logger.Path) + } + if cfg.Alias != nil && cfg.Alias.AliasFile != nil { + rewrite("alias.aliasFile.path", &cfg.Alias.AliasFile.Path) + } + if cfg.Key != nil && cfg.Key.KeyStoreFile != nil { + rewrite("key.keyManagerFile.path", &cfg.Key.KeyStoreFile.Path) + } + if cfg.DB != nil { + rewrite("db.path", &cfg.DB.Path) + rewrite("db.workerPathPrefix", &cfg.DB.WorkerPathPrefix) + for i := range cfg.DB.WorkerPaths { + rewrite(fmt.Sprintf("db.workerPaths[%d]", i), &cfg.DB.WorkerPaths[i]) + } + } + + if len(changed) == 0 { + return nil, nil + } + + if err := nodeconfig.SaveConfig(configDir, cfg); err != nil { + return nil, fmt.Errorf("save rewritten config: %w", err) + } + return changed, nil +} + +// crossOSNote returns a short human-readable note to print when the +// source and destination OS differ, or an empty string otherwise. +func crossOSNote(mf *manifest) string { + if mf.HostOS == "" || mf.HostOS == runtime.GOOS { + return "" + } + return fmt.Sprintf("Cross-OS restore detected: backup taken on %s, restoring on %s", + mf.HostOS, runtime.GOOS) +} diff --git a/client/cmd/node/backup/portable_test.go b/client/cmd/node/backup/portable_test.go new file mode 100644 index 00000000..cc1ecebf --- /dev/null +++ b/client/cmd/node/backup/portable_test.go @@ -0,0 +1,202 @@ +package backup + +import ( + "path/filepath" + "testing" +) + +func TestPathMap_LongestMatchWins(t *testing.T) { + pm := &pathMap{} + pm.add("/home/alice", "/Users/alice") + pm.add("/home/alice/.quilibrium/configs", "/Users/alice/.quilibrium/configs") + pm.finalize() + + cases := []struct { + in, want string + mapped bool + }{ + { + in: "/home/alice/.quilibrium/configs/node-quickstart/config.yml", + want: "/Users/alice/.quilibrium/configs/node-quickstart/config.yml", + mapped: true, + }, + { + in: "/home/alice/other/file", + want: "/Users/alice/other/file", + mapped: true, + }, + { + in: "/home/alice", + want: "/Users/alice", + mapped: true, + }, + { + in: "/var/lib/quilibrium/quilibrium.env", + want: "/var/lib/quilibrium/quilibrium.env", + mapped: false, + }, + { + in: "/home/alicex/not-matched", + want: "/home/alicex/not-matched", + mapped: false, + }, + } + for _, tc := range cases { + got, ok := pm.apply(tc.in) + if got != tc.want || ok != tc.mapped { + t.Errorf("apply(%q) = (%q,%v); want (%q,%v)", + tc.in, got, ok, tc.want, tc.mapped) + } + } +} + +func TestParsePathMap(t *testing.T) { + good := []struct{ in, oldP, newP string }{ + {"/a=/b", "/a", "/b"}, + {" /foo/bar = /baz/qux ", "/foo/bar", "/baz/qux"}, + } + for _, tc := range good { + oldP, newP, err := parsePathMap(tc.in) + if err != nil || oldP != tc.oldP || newP != tc.newP { + t.Errorf("parsePathMap(%q) = (%q,%q,%v); want (%q,%q,nil)", + tc.in, oldP, newP, err, tc.oldP, tc.newP) + } + } + bad := []string{"", "no-equals", "=/b", "/a=", "relative=path", "/a=rel"} + for _, tc := range bad { + if _, _, err := parsePathMap(tc); err == nil { + t.Errorf("parsePathMap(%q) expected error", tc) + } + } +} + +func TestDestPathFor_FilesEntryRemapsToLocalConfigsDir(t *testing.T) { + mf := &manifest{ + Version: 2, + ConfigName: "node-quickstart", + ConfigDir: "/home/alice/.quilibrium/configs/node-quickstart", + ConfigsDir: "/home/alice/.quilibrium/configs", + Home: "/home/alice", + } + bf := &backupFile{ + LocalPath: "/home/alice/.quilibrium/configs/node-quickstart/config.yml", + ObjectKey: "node-quickstart/files/config.yml", + } + pm := &pathMap{} + pm.add(mf.ConfigDir, "/Users/bob/.quilibrium/configs/node-quickstart") + pm.finalize() + dstConfigDir := "/Users/bob/.quilibrium/configs/node-quickstart" + + dest, mapped := destPathFor(mf, bf, pm, dstConfigDir) + want := filepath.Join(dstConfigDir, "config.yml") + if dest != want || !mapped { + t.Errorf("destPathFor files entry = (%q,%v); want (%q,true)", dest, mapped, want) + } +} + +func TestDestPathFor_AbsoluteEntryUsesPathMap(t *testing.T) { + mf := &manifest{ + Version: 2, + ConfigName: "node-quickstart", + } + bf := &backupFile{ + LocalPath: "/mnt/big/store/worker-store/3/data.sst", + ObjectKey: "node-quickstart/absolute/mnt/big/store/worker-store/3/data.sst", + } + pm := &pathMap{} + pm.add("/mnt/big/store", "/data/quil/store") + pm.finalize() + + dest, mapped := destPathFor(mf, bf, pm, "/irrelevant") + want := "/data/quil/store/worker-store/3/data.sst" + if dest != want || !mapped { + t.Errorf("destPathFor abs entry = (%q,%v); want (%q,true)", dest, mapped, want) + } +} + +func TestDestPathFor_V1ManifestFallsBackToLocalPath(t *testing.T) { + mf := &manifest{ + Version: 1, + ConfigName: "node-quickstart", + } + bf := &backupFile{ + LocalPath: "/home/alice/.quilibrium/configs/node-quickstart/config.yml", + ObjectKey: "node-quickstart/files/config.yml", + } + pm := &pathMap{} + pm.finalize() + dest, mapped := destPathFor(mf, bf, pm, "/ignored") + if dest != bf.LocalPath || mapped { + t.Errorf("v1 fallthrough = (%q,%v); want (%q,false)", dest, mapped, bf.LocalPath) + } +} + +func TestDestPathFor_FilesEntryWithBucketPrefix(t *testing.T) { + mf := &manifest{ + Version: 2, + ConfigName: "node-quickstart", + } + bf := &backupFile{ + LocalPath: "/home/alice/.quilibrium/configs/node-quickstart/store/LOG", + ObjectKey: "quilibrium/backups/node-quickstart/files/store/LOG", + } + pm := &pathMap{} + pm.finalize() + dstConfigDir := "/Users/bob/.quilibrium/configs/node-quickstart" + dest, mapped := destPathFor(mf, bf, pm, dstConfigDir) + want := filepath.Join(dstConfigDir, "store", "LOG") + if dest != want || !mapped { + t.Errorf("bucket-prefixed files entry = (%q,%v); want (%q,true)", dest, mapped, want) + } +} + +func TestJoinKey(t *testing.T) { + cases := []struct{ prefix, want string }{ + {"", "node-quickstart/files/config.yml"}, + {"/", "node-quickstart/files/config.yml"}, + {"quilibrium/backups", "quilibrium/backups/node-quickstart/files/config.yml"}, + {"/quilibrium/backups/", "quilibrium/backups/node-quickstart/files/config.yml"}, + {" quilibrium/backups ", "quilibrium/backups/node-quickstart/files/config.yml"}, + } + for _, tc := range cases { + got := joinKey(tc.prefix, "node-quickstart", "files", "config.yml") + if got != tc.want { + t.Errorf("joinKey(%q,...) = %q; want %q", tc.prefix, got, tc.want) + } + } +} + +func TestNormalizeBucketPrefix(t *testing.T) { + cases := map[string]string{ + "": "", + "/": "", + "//": "", + "foo": "foo", + "/foo/": "foo", + " /foo/bar/ ": "foo/bar", + "quilibrium/backups": "quilibrium/backups", + "/quilibrium/backups/": "quilibrium/backups", + } + for in, want := range cases { + if got := normalizeBucketPrefix(in); got != want { + t.Errorf("normalizeBucketPrefix(%q) = %q; want %q", in, got, want) + } + } +} + +func TestBuildPathMap_UserMapOverridesAuto(t *testing.T) { + mf := &manifest{ + Version: 2, + ConfigName: "node-quickstart", + ConfigDir: "/home/alice/.quilibrium/configs/node-quickstart", + Home: "/home/alice", + } + pm, _, err := buildPathMap(mf, []string{"/home/alice=/opt/custom"}) + if err != nil { + t.Fatalf("buildPathMap: %v", err) + } + got, _ := pm.apply("/home/alice/somefile") + if got != "/opt/custom/somefile" { + t.Errorf("user map should override auto: got %q", got) + } +} diff --git a/client/cmd/node/backup/restore.go b/client/cmd/node/backup/restore.go new file mode 100644 index 00000000..ae5b81b4 --- /dev/null +++ b/client/cmd/node/backup/restore.go @@ -0,0 +1,519 @@ +package backup + +import ( + "context" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "regexp" + "runtime" + "sort" + "strconv" + "strings" + "sync" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var ( + restoreConfigName string + restoreForce bool + restoreDryRun bool + restoreConfigOnly bool + restoreMaster bool + restoreWorkerAll bool + restoreWorkers []string + restorePathSubs []string + restorePathMaps []string +) + +var restoreCmd = &cobra.Command{ + Use: "restore", + Short: "Download a previous backup back into its original locations", + Long: `Restore files from the configured S3-compatible bucket to the local +filesystem, placing each file at the absolute path recorded in manifest.json. + +By default the active node config name is used as the backup prefix. Override +with --name to restore a different config (e.g. when recovering onto a fresh +host). Existing files are skipped unless --force is passed. + +Selective restore (combinable, union semantics — a file matches if ANY filter +matches; with no filters, everything in the manifest is restored): + + --config only config files: config.yml, keys.yml, alias file, + logger dir, and anything else outside store/ and + worker-store/ + --master only the master store/ directory + --worker EXPR only the listed worker-store/ directories. EXPR + accepts integers, ranges, and comma-separated lists, + e.g. --worker 3 | --worker 1-3,5,7-16 | --worker 0,2,4. + Flag is repeatable; selections are unioned. + --worker-all all worker-store/* directories + --path SUBSTR only files whose local path contains SUBSTR (repeatable) + +Cross-OS / cross-host restore: + + Backups taken by a recent client record the source host's $HOME, + configs dir, node state dir, and install dir in manifest.json. On + restore, absolute paths are automatically remapped to the equivalent + locations on this host (e.g. Linux /home/alice → macOS /Users/alice, + Linux /var/lib/quilibrium → macOS /usr/local/var/quilibrium), and the + restored config.yml has its embedded logger/alias/key/db paths + rewritten to match. For any leftover absolute paths that can't be + mapped automatically, pass one or more --path-map OLD=NEW flags. + + --path-map OLD=NEW rewrite destination paths whose prefix is OLD + to NEW (both must be absolute; repeatable) + +Examples: + + qclient node backup restore --config + qclient node backup restore --master + qclient node backup restore --worker 0 --worker 3 + qclient node backup restore --worker 1-3,5,7-16 + qclient node backup restore --worker-all --master + qclient node backup restore --path config.yml`, + Run: func(cmd *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Fprintf(os.Stderr, "Error loading qclient config: %v\n", err) + os.Exit(1) + } + if err := validateBackupConfigured(&cfg.Backup); err != nil { + fmt.Fprintf(os.Stderr, "Error: backup not configured: %v\n", err) + fmt.Fprintln(os.Stderr, "Run `qclient node backup config` first.") + os.Exit(1) + } + + name := restoreConfigName + if name == "" { + resolvedName, _, _, rerr := resolveActiveNodeConfig() + if rerr != nil { + fmt.Fprintf(os.Stderr, "Error resolving active node config: %v\n", rerr) + fmt.Fprintln(os.Stderr, "Pass --name to specify a backup to restore.") + os.Exit(1) + } + name = resolvedName + } + + workers, wErr := parseWorkerSelectors(restoreWorkers) + if wErr != nil { + fmt.Fprintf(os.Stderr, "Error: invalid --worker value: %v\n", wErr) + os.Exit(1) + } + sel := restoreSelector{ + configOnly: restoreConfigOnly, + master: restoreMaster, + workerAll: restoreWorkerAll, + workers: workers, + pathSubs: restorePathSubs, + } + if err := runRestore(&cfg.Backup, name, sel, restorePathMaps, restoreForce, restoreDryRun); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + }, +} + +func init() { + restoreCmd.Flags().StringVar(&restoreConfigName, "name", "", "backup name to restore (defaults to active node config)") + restoreCmd.Flags().BoolVar(&restoreForce, "force", false, "overwrite existing files") + restoreCmd.Flags().BoolVar(&restoreDryRun, "dry-run", false, "show what would be downloaded without writing") + restoreCmd.Flags().BoolVar(&restoreConfigOnly, "config", false, "restore only config files (config.yml, keys.yml, alias, logger) and skip store/worker-store") + restoreCmd.Flags().BoolVar(&restoreMaster, "master", false, "restore only the master store/ directory") + restoreCmd.Flags().BoolVar(&restoreWorkerAll, "worker-all", false, "restore all worker-store/* directories") + restoreCmd.Flags().StringSliceVar(&restoreWorkers, "worker", nil, "worker indices to restore: single (3), range (1-3), or list (1-3,5,7-16); repeatable") + restoreCmd.Flags().StringSliceVar(&restorePathSubs, "path", nil, "restore files whose local path contains this substring (repeatable)") + restoreCmd.Flags().StringSliceVar(&restorePathMaps, "path-map", nil, "rewrite destination paths: --path-map /old/prefix=/new/prefix (repeatable)") +} + +// restoreSelector is the set of include filters parsed from flags. +// When all fields are zero-valued, every file in the manifest is +// selected (full restore). +type restoreSelector struct { + configOnly bool + master bool + workerAll bool + workers []int + pathSubs []string +} + +func (s restoreSelector) isFull() bool { + return !s.configOnly && !s.master && !s.workerAll && + len(s.workers) == 0 && len(s.pathSubs) == 0 +} + +// parseWorkerSelectors parses one or more --worker values into a +// deduplicated, sorted list of worker indices. Each value may be a +// comma-separated list of integers or inclusive ranges, e.g. +// "3", "1-3", "0,2,4", "1-3,5,7-16". Negative indices are rejected. +func parseWorkerSelectors(values []string) ([]int, error) { + set := make(map[int]struct{}) + for _, v := range values { + for _, part := range strings.Split(v, ",") { + part = strings.TrimSpace(part) + if part == "" { + continue + } + if strings.Contains(part, "-") { + bounds := strings.SplitN(part, "-", 2) + lo, err1 := strconv.Atoi(strings.TrimSpace(bounds[0])) + hi, err2 := strconv.Atoi(strings.TrimSpace(bounds[1])) + if err1 != nil || err2 != nil { + return nil, fmt.Errorf("bad range %q", part) + } + if lo < 0 || hi < 0 { + return nil, fmt.Errorf("negative worker index in %q", part) + } + if lo > hi { + return nil, fmt.Errorf("range %q: lower bound greater than upper bound", part) + } + for i := lo; i <= hi; i++ { + set[i] = struct{}{} + } + continue + } + n, err := strconv.Atoi(part) + if err != nil { + return nil, fmt.Errorf("bad worker index %q", part) + } + if n < 0 { + return nil, fmt.Errorf("negative worker index %d", n) + } + set[n] = struct{}{} + } + } + out := make([]int, 0, len(set)) + for k := range set { + out = append(out, k) + } + sort.Ints(out) + return out, nil +} + +// workerIndexRe matches .../worker-store//... or .../worker-store/ +// at end-of-path, with N as a non-negative integer. Capture group 1 is +// the index. Used against both LocalPath and ObjectKey so it works +// whether the file was backed up under files/ or absolute/. +var workerIndexRe = regexp.MustCompile(`(?:^|/)worker-store/(\d+)(?:/|$)`) + +// isStorePath reports whether p is inside a master store/ directory +// (not worker-store/). We check the "/store/" segment but exclude any +// path that also matches worker-store/. +func isStorePath(p string) bool { + norm := filepath.ToSlash(p) + if workerIndexRe.MatchString(norm) { + return false + } + return strings.Contains(norm, "/store/") || strings.HasSuffix(norm, "/store") +} + +// workerIndexFor returns the worker index for p, or -1 if p is not a +// worker-store path. +func workerIndexFor(p string) int { + norm := filepath.ToSlash(p) + m := workerIndexRe.FindStringSubmatch(norm) + if m == nil { + return -1 + } + n, err := strconv.Atoi(m[1]) + if err != nil { + return -1 + } + return n +} + +// selectFiles returns the subset of entries that matches sel. +func selectFiles(entries []backupFile, sel restoreSelector) []backupFile { + if sel.isFull() { + return entries + } + workerSet := make(map[int]struct{}, len(sel.workers)) + for _, w := range sel.workers { + workerSet[w] = struct{}{} + } + out := make([]backupFile, 0, len(entries)) + for _, f := range entries { + // Check both the local path and the object key so filters + // work regardless of whether a file was backed up under + // files/ (inside configDir) or absolute/ (outside it). + candidates := []string{f.LocalPath, f.ObjectKey} + + match := false + + if sel.configOnly { + isConfig := true + for _, c := range candidates { + if isStorePath(c) || workerIndexFor(c) >= 0 { + isConfig = false + break + } + } + if isConfig { + match = true + } + } + if !match && sel.master { + for _, c := range candidates { + if isStorePath(c) { + match = true + break + } + } + } + if !match && (sel.workerAll || len(workerSet) > 0) { + for _, c := range candidates { + idx := workerIndexFor(c) + if idx < 0 { + continue + } + if sel.workerAll { + match = true + break + } + if _, ok := workerSet[idx]; ok { + match = true + break + } + } + } + if !match && len(sel.pathSubs) > 0 { + for _, sub := range sel.pathSubs { + if sub == "" { + continue + } + if strings.Contains(f.LocalPath, sub) || strings.Contains(f.ObjectKey, sub) { + match = true + break + } + } + } + + if match { + out = append(out, f) + } + } + return out +} + +// resolvedFile pairs a manifest entry with its computed destination on +// the local filesystem (which may differ from bf.LocalPath after +// cross-host path remapping). +type resolvedFile struct { + bf backupFile + dest string + mapped bool +} + +func runRestore(b *utils.NodeBackupConfig, name string, sel restoreSelector, userPathMaps []string, force, dryRun bool) error { + client, err := newS3Client(b) + if err != nil { + return fmt.Errorf("s3 client: %w", err) + } + ctx := context.Background() + + bucketPrefix := normalizeBucketPrefix(b.BucketPrefix) + prefix := strings.TrimSuffix(name, "/") + mfKey := joinKey(bucketPrefix, prefix, manifestObjectKey) + mf, err := downloadManifest(ctx, client, b.Bucket, mfKey) + if err != nil { + return fmt.Errorf("download manifest %s: %w", mfKey, err) + } + + pm, dstConfigDir, err := buildPathMap(mf, userPathMaps) + if err != nil { + return fmt.Errorf("build path map: %w", err) + } + + files := selectFiles(mf.Files, sel) + fmt.Printf("Restoring backup %q (%d of %d files selected) created at %s\n", + mf.ConfigName, len(files), len(mf.Files), + mf.CreatedAt.Format("2006-01-02 15:04:05 MST")) + if note := crossOSNote(mf); note != "" { + fmt.Println(note) + } + if len(files) == 0 { + return fmt.Errorf("no files matched the selected filters (use --dry-run without filters to see what the backup contains)") + } + + resolved := make([]resolvedFile, 0, len(files)) + var unmappedAbs []string + for _, f := range files { + dest, mapped := destPathFor(mf, &f, pm, dstConfigDir) + resolved = append(resolved, resolvedFile{bf: f, dest: dest, mapped: mapped}) + if !mapped && strings.Contains(f.ObjectKey, "/absolute/") && + mf.HostOS != "" && mf.HostOS != runtime.GOOS { + unmappedAbs = append(unmappedAbs, f.LocalPath) + } + } + + if len(unmappedAbs) > 0 { + fmt.Fprintf(os.Stderr, + "Warning: %d absolute-path file(s) from a %s backup could not be remapped to %s equivalents and will be written verbatim. Pass --path-map OLD=NEW to redirect them. Examples:\n", + len(unmappedAbs), mf.HostOS, runtime.GOOS) + shown := unmappedAbs + if len(shown) > 5 { + shown = shown[:5] + } + for _, p := range shown { + fmt.Fprintf(os.Stderr, " %s\n", p) + } + if len(unmappedAbs) > len(shown) { + fmt.Fprintf(os.Stderr, " ... and %d more\n", len(unmappedAbs)-len(shown)) + } + } + + if dryRun { + for _, r := range resolved { + marker := "" + if r.bf.LocalPath != r.dest { + marker = " (remapped)" + } + fmt.Printf(" would download %s -> %s (%d bytes)%s\n", + r.bf.ObjectKey, r.dest, r.bf.Size, marker) + } + // In dry-run, preview config rewrites too (without touching + // disk) by showing what would change after a real restore. + return nil + } + + if err := downloadFiles(ctx, client, b.Bucket, resolved, force); err != nil { + return err + } + + // Post-restore: if we actually wrote a config.yml for this + // backup, rewrite its embedded absolute paths to match the + // destination host. Skipped silently when config.yml wasn't + // restored in this invocation (e.g. --master only). + if configRestored(resolved, dstConfigDir) { + changed, rerr := rewriteRestoredConfig(dstConfigDir, pm) + if rerr != nil { + fmt.Fprintf(os.Stderr, "Warning: could not rewrite embedded paths in %s/config.yml: %v\n", dstConfigDir, rerr) + } else if len(changed) > 0 { + fmt.Printf("Rewrote %d embedded path(s) in %s/config.yml: %s\n", + len(changed), dstConfigDir, strings.Join(changed, ", ")) + } + } + + return nil +} + +// configRestored reports whether the set of resolved files included +// config.yml for the destination config dir. +func configRestored(files []resolvedFile, dstConfigDir string) bool { + target := filepath.Join(dstConfigDir, "config.yml") + for _, r := range files { + if r.dest == target { + return true + } + } + return false +} + +func downloadManifest(ctx context.Context, client *s3.Client, bucket, key string) (*manifest, error) { + out, err := client.GetObject(ctx, &s3.GetObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(key), + }) + if err != nil { + return nil, err + } + defer out.Body.Close() + data, err := io.ReadAll(out.Body) + if err != nil { + return nil, err + } + mf := &manifest{} + if err := json.Unmarshal(data, mf); err != nil { + return nil, fmt.Errorf("parse manifest: %w", err) + } + return mf, nil +} + +func downloadFiles(ctx context.Context, client *s3.Client, bucket string, entries []resolvedFile, force bool) error { + type result struct { + idx int + err error + msg string + } + results := make(chan result, len(entries)) + sem := make(chan struct{}, backupConcurrency) + + var wg sync.WaitGroup + for i := range entries { + i := i + wg.Add(1) + sem <- struct{}{} + go func() { + defer wg.Done() + defer func() { <-sem }() + msg, err := downloadOne(ctx, client, bucket, &entries[i], force) + results <- result{idx: i, err: err, msg: msg} + }() + } + go func() { + wg.Wait() + close(results) + }() + + var firstErr error + done := 0 + for r := range results { + done++ + if r.err != nil { + fmt.Fprintf(os.Stderr, " [%d/%d] %s: %v\n", done, len(entries), entries[r.idx].dest, r.err) + if firstErr == nil { + firstErr = r.err + } + continue + } + fmt.Printf(" [%d/%d] %s %s\n", done, len(entries), r.msg, entries[r.idx].dest) + } + return firstErr +} + +func downloadOne(ctx context.Context, client *s3.Client, bucket string, rf *resolvedFile, force bool) (string, error) { + bf := &rf.bf + dest := rf.dest + if fi, err := os.Stat(dest); err == nil && fi.Size() == bf.Size && !force { + return "skipped", nil + } + + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return "", fmt.Errorf("mkdir: %w", err) + } + + out, err := client.GetObject(ctx, &s3.GetObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(bf.ObjectKey), + }) + if err != nil { + return "", err + } + defer out.Body.Close() + + tmp := dest + ".qclient-restore.tmp" + f, err := os.OpenFile(tmp, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o644) + if err != nil { + return "", err + } + if _, err := io.Copy(f, out.Body); err != nil { + f.Close() + os.Remove(tmp) + return "", err + } + if err := f.Close(); err != nil { + os.Remove(tmp) + return "", err + } + if err := os.Rename(tmp, dest); err != nil { + os.Remove(tmp) + return "", err + } + return "restored", nil +} diff --git a/client/cmd/node/backup/run.go b/client/cmd/node/backup/run.go new file mode 100644 index 00000000..14c6cfa5 --- /dev/null +++ b/client/cmd/node/backup/run.go @@ -0,0 +1,464 @@ +package backup + +import ( + "context" + "crypto/md5" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "os" + "os/user" + "path/filepath" + "runtime" + "sort" + "strings" + "sync" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" + nodeconfig "source.quilibrium.com/quilibrium/monorepo/config" +) + +// manifestVersion is the schema version written by this client. Version +// 2 adds host-context fields (HostOS, Home, ConfigsDir, NodeStateDir, +// NodeInstallDir) so cross-OS restore can remap absolute paths to the +// new host's equivalents. Restore accepts v1 manifests for back-compat. +const manifestVersion = 2 + +// normalizeBucketPrefix trims surrounding whitespace and leading/ +// trailing slashes from a user-supplied bucket prefix. Empty input +// returns "" (meaning: store at the bucket root). +func normalizeBucketPrefix(p string) string { + p = strings.TrimSpace(p) + p = strings.Trim(p, "/") + return p +} + +// joinKey joins a bucket prefix with one or more path segments using +// S3-style forward slashes. An empty prefix yields the segments joined +// directly (no leading slash). Empty segments are skipped. +func joinKey(prefix string, segs ...string) string { + prefix = normalizeBucketPrefix(prefix) + parts := make([]string, 0, len(segs)+1) + if prefix != "" { + parts = append(parts, prefix) + } + for _, s := range segs { + s = strings.Trim(s, "/") + if s != "" { + parts = append(parts, s) + } + } + return strings.Join(parts, "/") +} + +const ( + // manifestObjectKey is the per-config manifest name at the root of + // the config's S3 prefix. Clients read this on restore. + manifestObjectKey = "manifest.json" + + // backupConcurrency is the number of parallel uploads/downloads. + backupConcurrency = 4 +) + +var runCmd = &cobra.Command{ + Use: "run", + Short: "Run the node backup now (uploads config and worker data)", + Long: `Upload the active node config plus its store/ and worker-store/* +directories to the configured S3-compatible bucket. Files are uploaded as-is +(no compression, no encryption beyond TLS to the endpoint). + +The object layout is: + + /manifest.json + /files/ + +Files that live outside the config directory (if a custom DB.Path or +WorkerPaths is configured) are uploaded under: + + /absolute/ + +Restore uses manifest.json to place each object back at its recorded +absolute path.`, + Run: func(cmd *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Fprintf(os.Stderr, "Error loading qclient config: %v\n", err) + os.Exit(1) + } + if !cfg.Backup.Enabled { + fmt.Fprintln(os.Stderr, "Warning: backups are disabled in qclient config. Proceeding anyway because `run` was invoked explicitly.") + } + if err := validateBackupConfigured(&cfg.Backup); err != nil { + fmt.Fprintf(os.Stderr, "Error: backup not configured: %v\n", err) + fmt.Fprintln(os.Stderr, "Run `qclient node backup config` first.") + os.Exit(1) + } + if err := runBackup(&cfg.Backup); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + }, +} + +// backupFile describes a single file selected for backup. +type backupFile struct { + LocalPath string `json:"localPath"` + ObjectKey string `json:"objectKey"` + Size int64 `json:"size"` + MD5Hex string `json:"md5"` +} + +// manifest is the JSON document uploaded alongside the files. +// +// Host-context fields (HostOS, Home, ConfigsDir, NodeStateDir, +// NodeInstallDir) are populated starting at Version 2 and let restore +// remap absolute paths recorded on the source host to the equivalent +// locations on the destination host (e.g. Linux /home/alice → +// macOS /Users/alice, Linux /var/lib/quilibrium → +// macOS /usr/local/var/quilibrium). They are optional — a Version 1 +// manifest restores to the recorded paths verbatim, as before. +type manifest struct { + Version int `json:"version"` + ConfigName string `json:"configName"` + ConfigDir string `json:"configDir"` + CreatedAt time.Time `json:"createdAt"` + Files []backupFile `json:"files"` + + // v2 host context + HostOS string `json:"hostOS,omitempty"` + Home string `json:"home,omitempty"` + ConfigsDir string `json:"configsDir,omitempty"` + NodeStateDir string `json:"nodeStateDir,omitempty"` + NodeInstallDir string `json:"nodeInstallDir,omitempty"` +} + +func runBackup(b *utils.NodeBackupConfig) error { + configName, configDir, cfg, err := resolveActiveNodeConfig() + if err != nil { + return err + } + fmt.Printf("Backing up config %q from %s\n", configName, configDir) + + localFiles, err := gatherFilesForBackup(configDir, cfg) + if err != nil { + return fmt.Errorf("gather files: %w", err) + } + if len(localFiles) == 0 { + return fmt.Errorf("no files found to back up in %s", configDir) + } + + client, err := newS3Client(b) + if err != nil { + return fmt.Errorf("s3 client: %w", err) + } + + bucketPrefix := normalizeBucketPrefix(b.BucketPrefix) + prefix := strings.TrimSuffix(configName, "/") + entries := make([]backupFile, 0, len(localFiles)) + for _, local := range localFiles { + key := objectKeyFor(bucketPrefix, prefix, configDir, local) + entries = append(entries, backupFile{LocalPath: local, ObjectKey: key}) + } + + ctx := context.Background() + if err := uploadFiles(ctx, client, b.Bucket, entries); err != nil { + return err + } + + mf := manifest{ + Version: manifestVersion, + ConfigName: configName, + ConfigDir: configDir, + CreatedAt: time.Now().UTC(), + Files: entries, + HostOS: runtime.GOOS, + Home: currentHomeDir(), + ConfigsDir: utils.GetNodeConfigsDir(), + NodeStateDir: utils.GetNodeStateDir(), + NodeInstallDir: utils.GetNodeInstallDir(), + } + if err := uploadManifest(ctx, client, b.Bucket, bucketPrefix, prefix, &mf); err != nil { + return fmt.Errorf("upload manifest: %w", err) + } + + fmt.Printf("Backup complete: %d files uploaded to s3://%s/%s/\n", + len(entries), b.Bucket, joinKey(bucketPrefix, prefix)) + return nil +} + +// resolveActiveNodeConfig loads the active node config and returns a +// short name, its absolute config directory, and the loaded config. +// Mirrors the logic in client/cmd/node/node.go's PersistentPreRun so +// the backup package can run without importing its parent and causing +// an import cycle. +func resolveActiveNodeConfig() (name, dir string, cfg *nodeconfig.Config, err error) { + cfg, err = utils.LoadDefaultNodeConfig() + if err != nil { + return "", "", nil, fmt.Errorf("load node config: %w", err) + } + resolved, dErr := utils.GetDefaultNodeConfigDir() + if dErr == nil { + dir = resolved + } else { + dir = utils.GetDefaultNodeConfigSymlink() + } + abs, err := filepath.Abs(dir) + if err == nil { + dir = abs + } + name = filepath.Base(dir) + if name == "" || name == "/" { + name = utils.DefaultNodeConfigName + } + return name, dir, cfg, nil +} + +// gatherFilesForBackup returns all local files under configDir plus +// any external worker-store / db paths resolved from cfg. +func gatherFilesForBackup(configDir string, cfg *nodeconfig.Config) ([]string, error) { + seen := make(map[string]struct{}) + var out []string + + add := func(p string) { + if p == "" { + return + } + if _, ok := seen[p]; ok { + return + } + seen[p] = struct{}{} + out = append(out, p) + } + + // Walk the entire config directory (config.yml, keys.yml, alias + // file, and by default store/ + worker-store/* since they live + // under configDir). + if err := walkRegularFiles(configDir, add); err != nil { + return nil, err + } + + // If DB.Path or WorkerPaths point outside configDir, pick them up + // too. Defaults keep them inside configDir so this is a no-op in + // the common case. + if cfg != nil && cfg.DB != nil { + if cfg.DB.Path != "" { + if err := walkRegularFiles(cfg.DB.Path, add); err != nil { + return nil, err + } + } + for _, wp := range cfg.DB.WorkerPaths { + if err := walkRegularFiles(wp, add); err != nil { + return nil, err + } + } + // WorkerPathPrefix with %d is expanded by the node per core + // at runtime; we can't know the core count here, so we + // best-effort-glob siblings under the prefix's parent dir. + if cfg.DB.WorkerPathPrefix != "" { + if paths := expandWorkerPrefix(cfg.DB.WorkerPathPrefix); paths != nil { + for _, wp := range paths { + if err := walkRegularFiles(wp, add); err != nil { + return nil, err + } + } + } + } + } + + sort.Strings(out) + return out, nil +} + +// expandWorkerPrefix returns directories matching a WorkerPathPrefix +// like "/worker-store/%d" by listing sibling directories under +// "/worker-store" whose names are pure integers. +func expandWorkerPrefix(prefix string) []string { + idx := strings.LastIndex(prefix, "%d") + if idx < 0 { + if _, err := os.Stat(prefix); err == nil { + return []string{prefix} + } + return nil + } + parent := strings.TrimRight(prefix[:idx], "/") + entries, err := os.ReadDir(parent) + if err != nil { + return nil + } + var out []string + for _, e := range entries { + if !e.IsDir() { + continue + } + // Only accept numeric names to avoid swallowing unrelated + // siblings the operator may have put under worker-store/. + if _, err := fmt.Sscanf(e.Name(), "%d", new(int)); err == nil { + out = append(out, filepath.Join(parent, e.Name())) + } + } + return out +} + +func walkRegularFiles(root string, add func(string)) error { + info, err := os.Stat(root) + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + if !info.IsDir() { + abs, _ := filepath.Abs(root) + add(abs) + return nil + } + return filepath.Walk(root, func(p string, fi os.FileInfo, err error) error { + if err != nil { + return err + } + if fi.IsDir() { + return nil + } + if !fi.Mode().IsRegular() { + return nil + } + abs, aerr := filepath.Abs(p) + if aerr != nil { + abs = p + } + add(abs) + return nil + }) +} + +// objectKeyFor maps a local absolute path to an S3 object key under +// the (optional) bucket prefix and the configName prefix. Paths inside +// configDir become "//files/"; +// paths outside become +// "//absolute/". +// When bucketPrefix is empty the layout is unchanged from v1. +func objectKeyFor(bucketPrefix, configName, configDir, localPath string) string { + absConfig, _ := filepath.Abs(configDir) + absLocal, _ := filepath.Abs(localPath) + if rel, err := filepath.Rel(absConfig, absLocal); err == nil && + !strings.HasPrefix(rel, "..") { + return joinKey(bucketPrefix, configName, "files", filepath.ToSlash(rel)) + } + stripped := strings.TrimPrefix(absLocal, string(filepath.Separator)) + return joinKey(bucketPrefix, configName, "absolute", filepath.ToSlash(stripped)) +} + +func uploadFiles(ctx context.Context, client *s3.Client, bucket string, entries []backupFile) error { + type result struct { + idx int + err error + } + results := make(chan result, len(entries)) + sem := make(chan struct{}, backupConcurrency) + + var wg sync.WaitGroup + for i := range entries { + i := i + wg.Add(1) + sem <- struct{}{} + go func() { + defer wg.Done() + defer func() { <-sem }() + size, md5Hex, err := uploadOne(ctx, client, bucket, &entries[i]) + if err == nil { + entries[i].Size = size + entries[i].MD5Hex = md5Hex + } + results <- result{idx: i, err: err} + }() + } + go func() { + wg.Wait() + close(results) + }() + + var firstErr error + done := 0 + for r := range results { + done++ + if r.err != nil { + fmt.Fprintf(os.Stderr, " [%d/%d] %s: %v\n", done, len(entries), entries[r.idx].LocalPath, r.err) + if firstErr == nil { + firstErr = r.err + } + continue + } + fmt.Printf(" [%d/%d] uploaded %s (%d bytes)\n", done, len(entries), entries[r.idx].ObjectKey, entries[r.idx].Size) + } + return firstErr +} + +func uploadOne(ctx context.Context, client *s3.Client, bucket string, bf *backupFile) (int64, string, error) { + f, err := os.Open(bf.LocalPath) + if err != nil { + return 0, "", err + } + defer f.Close() + + fi, err := f.Stat() + if err != nil { + return 0, "", err + } + + h := md5.New() + if _, err := io.Copy(h, f); err != nil { + return 0, "", err + } + md5Hex := hex.EncodeToString(h.Sum(nil)) + + if _, err := f.Seek(0, io.SeekStart); err != nil { + return 0, "", err + } + + _, err = client.PutObject(ctx, &s3.PutObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(bf.ObjectKey), + Body: f, + }) + if err != nil { + return 0, "", err + } + return fi.Size(), md5Hex, nil +} + +// currentHomeDir returns the invoking user's home directory, preferring +// the sudo-invoking user so a root-run backup still records the human +// user's $HOME (matching GetNodeConfigsDir's resolution). Falls back to +// os.UserHomeDir, then empty string. +func currentHomeDir() string { + if u, err := utils.GetCurrentSudoUser(); err == nil && u != nil && u.HomeDir != "" { + return u.HomeDir + } + if u, err := user.Current(); err == nil && u != nil && u.HomeDir != "" { + return u.HomeDir + } + if h, err := os.UserHomeDir(); err == nil { + return h + } + return "" +} + +func uploadManifest(ctx context.Context, client *s3.Client, bucket, bucketPrefix, prefix string, mf *manifest) error { + data, err := json.MarshalIndent(mf, "", " ") + if err != nil { + return err + } + key := joinKey(bucketPrefix, prefix, manifestObjectKey) + _, err = client.PutObject(ctx, &s3.PutObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(key), + Body: strings.NewReader(string(data)), + ContentType: aws.String("application/json"), + }) + return err +} diff --git a/client/cmd/node/backup/schedule.go b/client/cmd/node/backup/schedule.go new file mode 100644 index 00000000..2e488199 --- /dev/null +++ b/client/cmd/node/backup/schedule.go @@ -0,0 +1,297 @@ +package backup + +import ( + "bytes" + "fmt" + "os" + "os/exec" + "regexp" + "strconv" + "strings" + + "github.com/spf13/cobra" +) + +// DefaultBackupCronSchedule runs hourly at minute 0. +const DefaultBackupCronSchedule = "0 * * * *" + +// cronMarker tags the crontab line managed by qclient so we can update +// or remove it without disturbing other user cron entries. +const cronMarker = "# managed-by: qclient node backup" + +var scheduleCmd = &cobra.Command{ + Use: "schedule [cron-expression]", + Short: "Configure a cron schedule that runs `qclient node backup run`", + Long: `Install or show the cron schedule for periodic node backups. + +Examples: + qclient node backup schedule # print current schedule (or default) + qclient node backup schedule "0 * * * *" # hourly (default) + qclient node backup schedule "*/15 * * * *" # every 15 minutes + qclient node backup schedule "0 3 * * *" # daily at 03:00 + qclient node backup schedule disable # remove the managed cron entry + +The cron entry invokes the current qclient binary as the invoking user and +runs ` + "`qclient node backup run`" + ` with signature checks disabled (the +binary path resolved at install time is written into the cron line).`, + Args: cobra.MaximumNArgs(1), + Run: func(cmd *cobra.Command, args []string) { + if len(args) == 0 { + if err := printSchedule(); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + return + } + arg := strings.TrimSpace(args[0]) + if strings.EqualFold(arg, "disable") || strings.EqualFold(arg, "remove") { + if err := removeSchedule(); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + return + } + if err := installSchedule(arg); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + }, +} + +func printSchedule() error { + existing, err := readCrontab() + if err != nil { + return err + } + line := findManagedLine(existing) + if line == "" { + fmt.Printf("No qclient-managed backup schedule installed.\n") + fmt.Printf("Default suggestion: %q\n", DefaultBackupCronSchedule) + return nil + } + expr, cmdStr := parseManagedLine(line) + fmt.Printf("Schedule: %s\n", expr) + fmt.Printf("Command: %s\n", cmdStr) + return nil +} + +func installSchedule(expr string) error { + if err := ValidateCronExpression(expr); err != nil { + return fmt.Errorf("invalid cron expression %q: %w", expr, err) + } + + execPath, err := os.Executable() + if err != nil { + return fmt.Errorf("resolve qclient binary path: %w", err) + } + // Use signature-check=false because a cron job runs + // non-interactively and the daily signature rotation would + // otherwise break scheduled runs. + cmdStr := fmt.Sprintf("%s --signature-check=false node backup run", execPath) + newLine := fmt.Sprintf("%s %s %s", expr, cmdStr, cronMarker) + + existing, err := readCrontab() + if err != nil { + return err + } + updated := replaceOrAppendManagedLine(existing, newLine) + if err := writeCrontab(updated); err != nil { + return err + } + fmt.Printf("Installed backup schedule: %s\n", expr) + fmt.Printf(" %s\n", cmdStr) + return nil +} + +func removeSchedule() error { + existing, err := readCrontab() + if err != nil { + return err + } + if findManagedLine(existing) == "" { + fmt.Println("No qclient-managed backup schedule to remove.") + return nil + } + updated := removeManagedLine(existing) + if err := writeCrontab(updated); err != nil { + return err + } + fmt.Println("Removed qclient-managed backup schedule.") + return nil +} + +// readCrontab returns the current user crontab content (empty string +// when no crontab exists). +func readCrontab() (string, error) { + if _, err := exec.LookPath("crontab"); err != nil { + return "", fmt.Errorf("crontab command not found in PATH") + } + cmd := exec.Command("crontab", "-l") + var stdout, stderr bytes.Buffer + cmd.Stdout = &stdout + cmd.Stderr = &stderr + if err := cmd.Run(); err != nil { + // `crontab -l` exits non-zero when there is no crontab; treat + // that specific message as empty rather than an error. + msg := stderr.String() + if strings.Contains(msg, "no crontab") || strings.Contains(strings.ToLower(msg), "no crontab for") { + return "", nil + } + if stdout.Len() == 0 && stderr.Len() == 0 { + return "", nil + } + return "", fmt.Errorf("crontab -l: %w: %s", err, strings.TrimSpace(msg)) + } + return stdout.String(), nil +} + +func writeCrontab(content string) error { + cmd := exec.Command("crontab", "-") + cmd.Stdin = strings.NewReader(content) + var stderr bytes.Buffer + cmd.Stderr = &stderr + if err := cmd.Run(); err != nil { + return fmt.Errorf("crontab write: %w: %s", err, strings.TrimSpace(stderr.String())) + } + return nil +} + +func findManagedLine(content string) string { + for _, line := range strings.Split(content, "\n") { + if strings.Contains(line, cronMarker) { + return line + } + } + return "" +} + +func parseManagedLine(line string) (expr, cmdStr string) { + // 5 cron fields + command + marker + fields := strings.Fields(line) + if len(fields) < 6 { + return line, "" + } + expr = strings.Join(fields[:5], " ") + rest := strings.TrimSpace(strings.TrimSuffix(strings.Join(fields[5:], " "), cronMarker)) + return expr, rest +} + +func replaceOrAppendManagedLine(content, newLine string) string { + lines := strings.Split(content, "\n") + replaced := false + for i, line := range lines { + if strings.Contains(line, cronMarker) { + lines[i] = newLine + replaced = true + } + } + if !replaced { + // Drop trailing empty line to avoid double blanks. + if len(lines) > 0 && strings.TrimSpace(lines[len(lines)-1]) == "" { + lines = lines[:len(lines)-1] + } + lines = append(lines, newLine) + } + // Ensure trailing newline. + out := strings.Join(lines, "\n") + if !strings.HasSuffix(out, "\n") { + out += "\n" + } + return out +} + +func removeManagedLine(content string) string { + lines := strings.Split(content, "\n") + filtered := make([]string, 0, len(lines)) + for _, line := range lines { + if strings.Contains(line, cronMarker) { + continue + } + filtered = append(filtered, line) + } + out := strings.Join(filtered, "\n") + if out != "" && !strings.HasSuffix(out, "\n") { + out += "\n" + } + return out +} + +// ValidateCronExpression checks that expr is a well-formed 5-field +// classic-vixie cron expression: minute hour day-of-month month day-of-week. +// It supports *, */n, a-b, a,b,c, and plain integers. It does not support +// macros (@hourly, @daily, etc.) or seconds fields. +func ValidateCronExpression(expr string) error { + fields := strings.Fields(expr) + if len(fields) != 5 { + return fmt.Errorf("expected 5 fields, got %d", len(fields)) + } + ranges := [5][2]int{ + {0, 59}, // minute + {0, 23}, // hour + {1, 31}, // day of month + {1, 12}, // month + {0, 6}, // day of week (0 or 7 = Sunday; we accept 0-6) + } + names := [5]string{"minute", "hour", "day-of-month", "month", "day-of-week"} + for i, f := range fields { + if err := validateCronField(f, ranges[i][0], ranges[i][1]); err != nil { + return fmt.Errorf("%s field: %w", names[i], err) + } + } + return nil +} + +var cronTokenRe = regexp.MustCompile(`^[\d\*/,\-]+$`) + +func validateCronField(f string, min, max int) error { + if f == "" { + return fmt.Errorf("empty") + } + if !cronTokenRe.MatchString(f) { + return fmt.Errorf("unsupported characters in %q", f) + } + for _, part := range strings.Split(f, ",") { + if err := validateCronPart(part, min, max); err != nil { + return err + } + } + return nil +} + +func validateCronPart(part string, min, max int) error { + step := 1 + rangeStr := part + if idx := strings.Index(part, "/"); idx >= 0 { + rangeStr = part[:idx] + stepStr := part[idx+1:] + s, err := strconv.Atoi(stepStr) + if err != nil || s <= 0 { + return fmt.Errorf("bad step %q", stepStr) + } + step = s + } + if rangeStr == "*" { + _ = step + return nil + } + if strings.Contains(rangeStr, "-") { + parts := strings.SplitN(rangeStr, "-", 2) + lo, err1 := strconv.Atoi(parts[0]) + hi, err2 := strconv.Atoi(parts[1]) + if err1 != nil || err2 != nil { + return fmt.Errorf("bad range %q", rangeStr) + } + if lo < min || hi > max || lo > hi { + return fmt.Errorf("range %d-%d out of bounds [%d,%d]", lo, hi, min, max) + } + return nil + } + n, err := strconv.Atoi(rangeStr) + if err != nil { + return fmt.Errorf("bad number %q", rangeStr) + } + if n < min || n > max { + return fmt.Errorf("%d out of bounds [%d,%d]", n, min, max) + } + return nil +} diff --git a/client/cmd/node/clean.go b/client/cmd/node/clean.go index bacfbbff..1749298f 100644 --- a/client/cmd/node/clean.go +++ b/client/cmd/node/clean.go @@ -49,38 +49,59 @@ To remove the current version of the node, use 'qclient node uninstall'`, }, } -// cleanNodeLogs removes all log files from the node's log directory +// cleanNodeLogs removes all log files from every node config's logger +// directory. Configs without a logger block (stdout/journal logging) are +// skipped — use the system log tooling to rotate/clean those. func cleanNodeLogs() { if err := utils.CheckAndRequestSudo("Cleaning logs requires root privileges"); err != nil { fmt.Fprintf(os.Stderr, "Error: %v\n", err) return } - logDir := utils.LogPath - entries, err := os.ReadDir(logDir) - if err != nil { - if os.IsNotExist(err) { - fmt.Println("No logs directory found.") - } else { - fmt.Fprintf(os.Stderr, "Error reading log directory: %v\n", err) + logDirs := utils.ResolveAllNodeLogDirs() + // Include the active config's log dir too in case it isn't listed + // by name (e.g. when the user only has a "default" symlink). + if resolved, err := utils.ResolveActiveNodeLog(); err == nil && resolved.FileBased { + present := false + for _, d := range logDirs { + if d == resolved.LogDir { + present = true + break + } } + if !present { + logDirs = append(logDirs, resolved.LogDir) + } + } + + if len(logDirs) == 0 { + fmt.Println("No node configs have a logger block set; nothing to clean.") return } - removed := 0 - for _, entry := range entries { - name := entry.Name() - if strings.HasSuffix(name, ".log") || strings.HasSuffix(name, ".log.gz") { - path := filepath.Join(logDir, name) - if err := os.Remove(path); err != nil { - fmt.Fprintf(os.Stderr, "Warning: could not remove %s: %v\n", name, err) + for _, logDir := range logDirs { + entries, err := os.ReadDir(logDir) + if err != nil { + if os.IsNotExist(err) { continue } - removed++ + fmt.Fprintf(os.Stderr, "Error reading log directory %s: %v\n", logDir, err) + continue + } + removed := 0 + for _, entry := range entries { + name := entry.Name() + if strings.HasSuffix(name, ".log") || strings.HasSuffix(name, ".log.gz") { + path := filepath.Join(logDir, name) + if err := os.Remove(path); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not remove %s: %v\n", name, err) + continue + } + removed++ + } } + fmt.Printf("Removed %d log file(s) from %s\n", removed, logDir) } - - fmt.Printf("Removed %d log file(s) from %s\n", removed, logDir) } // cleanNodeBinaries removes old node binary versions and signatures, @@ -91,16 +112,17 @@ func cleanNodeBinaries() { return } + binDir := utils.GetNodeBinaryDir() // Determine which version is currently active via the symlink currentVersion := "" - target, err := os.Readlink(utils.DefaultNodeSymlinkPath) + target, err := os.Readlink(utils.GetNodeSymlinkPath()) if err == nil { - // target looks like /var/quilibrium/bin/node//node--- + // target looks like /bin/node//node--- dir := filepath.Dir(target) currentVersion = filepath.Base(dir) } - entries, err := os.ReadDir(utils.NodeDataPath) + entries, err := os.ReadDir(binDir) if err != nil { if os.IsNotExist(err) { fmt.Println("No node binaries directory found.") @@ -118,7 +140,7 @@ func cleanNodeBinaries() { if entry.Name() == currentVersion { continue } - path := filepath.Join(utils.NodeDataPath, entry.Name()) + path := filepath.Join(binDir, entry.Name()) if err := os.RemoveAll(path); err != nil { fmt.Fprintf(os.Stderr, "Warning: could not remove %s: %v\n", entry.Name(), err) continue @@ -126,7 +148,7 @@ func cleanNodeBinaries() { removed++ } - fmt.Printf("Removed %d old version(s) from %s\n", removed, utils.NodeDataPath) + fmt.Printf("Removed %d old version(s) from %s\n", removed, binDir) if currentVersion != "" { fmt.Printf("Kept current version: %s\n", currentVersion) } @@ -134,8 +156,9 @@ func cleanNodeBinaries() { // RemoveNodeBinary removes a specific version's binary directory. func RemoveNodeBinary(version string) error { + binDir := utils.GetNodeBinaryDir() // Determine which version is currently active via the symlink - target, err := os.Readlink(utils.DefaultNodeSymlinkPath) + target, err := os.Readlink(utils.GetNodeSymlinkPath()) if err == nil { dir := filepath.Dir(target) currentVersion := filepath.Base(dir) @@ -144,9 +167,9 @@ func RemoveNodeBinary(version string) error { } } - versionDir := filepath.Join(utils.NodeDataPath, version) + versionDir := filepath.Join(binDir, version) if _, err := os.Stat(versionDir); os.IsNotExist(err) { - return fmt.Errorf("version %s not found in %s", version, utils.NodeDataPath) + return fmt.Errorf("version %s not found in %s", version, binDir) } return os.RemoveAll(versionDir) diff --git a/client/cmd/node/grpc_rest.go b/client/cmd/node/grpc_rest.go new file mode 100644 index 00000000..481022e1 --- /dev/null +++ b/client/cmd/node/grpc_rest.go @@ -0,0 +1,191 @@ +package node + +import ( + "fmt" + "os" + + "github.com/multiformats/go-multiaddr" + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/config" +) + +const ( + defaultListenGRPCMultiaddr = "/ip4/127.0.0.1/tcp/8337" + // Matches the REST listener default in node service templates (gRPC 8337, REST 8338). + defaultListenRestMultiaddr = "/ip4/127.0.0.1/tcp/8338" +) + +var ( + grpcEnableAddr string + restEnableAddr string +) + +// NodeGrpcCmd groups gRPC listen settings for the node. +var NodeGrpcCmd = &cobra.Command{ + Use: "grpc", + Short: "Configure node gRPC listen multiaddr", + Long: `Configure the node's gRPC listen address (listenGrpcMultiaddr in config.yml). + +Subcommands set or clear the value; restart the node service for changes to take effect.`, + Run: func(cmd *cobra.Command, args []string) { + cmd.Help() + }, +} + +var nodeGrpcEnableCmd = &cobra.Command{ + Use: "enable", + Short: "Enable gRPC by setting listenGrpcMultiaddr", + Long: fmt.Sprintf(`Set listenGrpcMultiaddr to enable the gRPC server. + +The default multiaddr is %s. Override with --addr. + +Restart the node service after changing this setting.`, defaultListenGRPCMultiaddr), + Run: func(cmd *cobra.Command, args []string) { + if err := enableListenGRPC(); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } + }, +} + +var nodeGrpcDisableCmd = &cobra.Command{ + Use: "disable", + Short: "Disable gRPC by clearing listenGrpcMultiaddr", + Long: `Set listenGrpcMultiaddr to empty (disabled). + +Restart the node service after changing this setting.`, + Run: func(cmd *cobra.Command, args []string) { + if err := disableListenGRPC(); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } + }, +} + +// NodeRestCmd groups REST (HTTP gateway) listen settings for the node. +var NodeRestCmd = &cobra.Command{ + Use: "rest", + Short: "Configure node REST listen multiaddr", + Long: `Configure the node's REST gateway listen address (listenRESTMultiaddr in config.yml). + +Subcommands set or clear the value; restart the node service for changes to take effect.`, + Run: func(cmd *cobra.Command, args []string) { + cmd.Help() + }, +} + +var nodeRestEnableCmd = &cobra.Command{ + Use: "enable", + Short: "Enable REST by setting listenRESTMultiaddr", + Long: fmt.Sprintf(`Set listenRESTMultiaddr to enable the HTTP/JSON gateway. + +The default multiaddr is %s. Override with --addr. + +Restart the node service after changing this setting.`, defaultListenRestMultiaddr), + Run: func(cmd *cobra.Command, args []string) { + if err := enableListenRest(); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } + }, +} + +var nodeRestDisableCmd = &cobra.Command{ + Use: "disable", + Short: "Disable REST by clearing listenRESTMultiaddr", + Long: `Set listenRESTMultiaddr to empty (disabled). + +Restart the node service after changing this setting.`, + Run: func(cmd *cobra.Command, args []string) { + if err := disableListenRest(); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } + }, +} + +func init() { + nodeGrpcEnableCmd.Flags().StringVar(&grpcEnableAddr, "addr", "", "gRPC listen multiaddr (default "+defaultListenGRPCMultiaddr+")") + NodeGrpcCmd.AddCommand(nodeGrpcEnableCmd) + NodeGrpcCmd.AddCommand(nodeGrpcDisableCmd) + + nodeRestEnableCmd.Flags().StringVar(&restEnableAddr, "addr", "", "REST listen multiaddr (default "+defaultListenRestMultiaddr+")") + NodeRestCmd.AddCommand(nodeRestEnableCmd) + NodeRestCmd.AddCommand(nodeRestDisableCmd) +} + +func ensureNodeConfigLoaded() error { + if NodeConfig == nil || NodeConfigDir == "" { + return fmt.Errorf("no active node config loaded. Run `qclient node config create` first, or pass --config ") + } + return nil +} + +func validateMultiaddr(label, s string) error { + if _, err := multiaddr.NewMultiaddr(s); err != nil { + return fmt.Errorf("invalid %s multiaddr %q: %w", label, s, err) + } + return nil +} + +func enableListenGRPC() error { + if err := ensureNodeConfigLoaded(); err != nil { + return err + } + addr := grpcEnableAddr + if addr == "" { + addr = defaultListenGRPCMultiaddr + } + if err := validateMultiaddr("gRPC", addr); err != nil { + return err + } + NodeConfig.ListenGRPCMultiaddr = addr + if err := config.SaveConfig(NodeConfigDir, NodeConfig); err != nil { + return fmt.Errorf("save config: %w", err) + } + fmt.Printf("Set listenGrpcMultiaddr to %s in %s/config.yml\n", addr, NodeConfigDir) + return nil +} + +func disableListenGRPC() error { + if err := ensureNodeConfigLoaded(); err != nil { + return err + } + NodeConfig.ListenGRPCMultiaddr = "" + if err := config.SaveConfig(NodeConfigDir, NodeConfig); err != nil { + return fmt.Errorf("save config: %w", err) + } + fmt.Printf("Cleared listenGrpcMultiaddr in %s/config.yml\n", NodeConfigDir) + return nil +} + +func enableListenRest() error { + if err := ensureNodeConfigLoaded(); err != nil { + return err + } + addr := restEnableAddr + if addr == "" { + addr = defaultListenRestMultiaddr + } + if err := validateMultiaddr("REST", addr); err != nil { + return err + } + NodeConfig.ListenRestMultiaddr = addr + if err := config.SaveConfig(NodeConfigDir, NodeConfig); err != nil { + return fmt.Errorf("save config: %w", err) + } + fmt.Printf("Set listenRESTMultiaddr to %s in %s/config.yml\n", addr, NodeConfigDir) + return nil +} + +func disableListenRest() error { + if err := ensureNodeConfigLoaded(); err != nil { + return err + } + NodeConfig.ListenRestMultiaddr = "" + if err := config.SaveConfig(NodeConfigDir, NodeConfig); err != nil { + return fmt.Errorf("save config: %w", err) + } + fmt.Printf("Cleared listenRESTMultiaddr in %s/config.yml\n", NodeConfigDir) + return nil +} diff --git a/client/cmd/node/install.go b/client/cmd/node/install.go index 16a0f52b..6fa2c087 100644 --- a/client/cmd/node/install.go +++ b/client/cmd/node/install.go @@ -1,14 +1,55 @@ package node import ( + "bufio" "fmt" "os" "path/filepath" + "runtime" + "strings" "github.com/spf13/cobra" "source.quilibrium.com/quilibrium/monorepo/client/utils" ) +// Install-time flags that let the user override the persisted install +// directories. Empty string means "unchanged" (leave the existing config +// value, or its default, alone). +var ( + installDirFlag string + stateDirFlag string + symlinkDirFlag string + configsDirFlag string + serviceNameFlag string + interactiveFlag bool +) + +// ExitUnlessSudoForInstall exits immediately if the process is not running +// with elevated privileges. NodeCmd.PersistentPreRun calls this for install +// before any other node setup (config load, default config creation, etc.). +func ExitUnlessSudoForInstall() { + if utils.IsSudo() { + return + } + osLabel, details := sudoInstallMessageForGOOS(runtime.GOOS) + fmt.Fprintf(os.Stderr, "This command must be run with sudo on %s before any install steps.\n\n", osLabel) + fmt.Fprintln(os.Stderr, " sudo qclient node install") + fmt.Fprintln(os.Stderr) + fmt.Fprintln(os.Stderr, details) + os.Exit(1) +} + +func sudoInstallMessageForGOOS(goos string) (osLabel string, details string) { + switch goos { + case "linux": + return "Linux", "Sudo is required to write binaries under /opt/quilibrium (default install root), write the environment file under /var/lib/quilibrium (default state root), install a systemd unit, place the quilibrium-node symlink (often under /usr/local/bin), and set ownership for binaries and logs." + case "darwin": + return "macOS", "Sudo is required to write binaries under /usr/local/quilibrium (default install root), write the environment file under /usr/local/var/quilibrium (default state root), install a launchd plist, place the quilibrium-node symlink (often under /usr/local/bin), and set ownership for binaries and logs." + default: + return goos, fmt.Sprintf("Sudo is required on %s to install system paths, the node service, binaries, and related config.", goos) + } +} + // installCmd represents the command to install the Quilibrium node var NodeInstallCmd = &cobra.Command{ Use: "install [version]", @@ -50,8 +91,44 @@ var NodeInstallCmd = &cobra.Command{ qclient node set-default [name-for-config] + ## Install Directories + + The following paths can be overridden at install time and are persisted + to the qclient config, so later commands (service, log, clean, etc.) + read the same values automatically: + + --install-dir Root install directory for node binaries (defaults + to /opt/quilibrium on Linux and + /usr/local/quilibrium on macOS). Binaries go to + /bin/node//. + --state-dir Root directory for mutable node state (defaults + to /var/lib/quilibrium on Linux and + /usr/local/var/quilibrium on macOS). The systemd + EnvironmentFile lives at + /quilibrium.env. + --symlink-dir Directory holding the quilibrium-node symlink + (defaults to /usr/local/bin). Make sure this is on + your $PATH if you change it. + --configs-dir Directory holding named node configs (defaults to + ~/.quilibrium/configs). + --service-name Name of the systemd service unit for the node + (defaults to "quilibrium-node"). Can also be + changed later via 'qclient config service-name'. + + The node log directory is not a qclient setting; it lives in the + node config's logger.path. On install, qclient ensures the active + node config has a logger block pointing to a .logs directory next to + that config's config.yml and creates that directory with the correct + ownership. Change the log location later with: + + qclient node config set logger.path /custom/log/dir + + Passing a flag updates the saved config. If the node is already + installed and the new value differs from the current one, the new + value is saved but takes effect only on the next install/update. + ## Binary Management - Binaries and signatures are installed to /var/quilibrium/bin/node/[version]/ + Binaries and signatures are installed to /bin/node/[version]/ You can update the node binary with: @@ -72,15 +149,16 @@ var NodeInstallCmd = &cobra.Command{ qclient node auto-update status ## Log Management - Logging uses system logging with logrotate installed by default. + Logs are controlled by the active node config's logger block and + written (and rotated) by the node itself via its lumberjack-based + logger. qclient does not install a separate logrotate rule. - Logs are installed to /var/log/quilibrium + The default log directory is /.logs/. - The logrotate config is installed to /etc/logrotate.d/quilibrium + You can view and clean logs with: - You can view the logs with: - - qclient node logs [version] + qclient node log view + qclient node log clean When installing with this command, if no version is specified, the latest version will be installed. @@ -93,6 +171,13 @@ Examples: # Install a specific version qclient node install 2.1.0 + + # Install into a custom directory tree + qclient node install --install-dir /opt/quilibrium + + # Interactively prompt for every install setting + qclient node install --interactive + qclient node install -i `, Args: cobra.RangeArgs(0, 1), Run: func(cmd *cobra.Command, args []string) { @@ -103,14 +188,39 @@ Examples: return } - if !utils.IsSudo() { - fmt.Println("This command must be run with sudo: sudo qclient node install") - fmt.Println("Sudo is required to install the node binary, logging, systemd (on Linux) service, and create the config directory.") + // If --interactive was passed, prompt the user for every + // install-time setting before we persist anything. The prompts + // write back into the same flag variables so the rest of this + // command sees them exactly as if they'd been passed on the CLI. + var interactiveVersion string + if interactiveFlag { + v, err := runInteractiveInstallPrompts(args) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + interactiveVersion = v + } + + // Apply any --install-dir / --state-dir / --symlink-dir / + // --configs-dir overrides to the persisted client config before + // we start laying files down, so every subsequent path lookup + // reads the new value. + if err := applyInstallDirFlags(); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) os.Exit(1) } - // Determine version to install - version := determineVersion(args) + warnLegacyInstallLayout() + + // Determine version to install. Interactive mode wins over + // positional args when the user selected something there. + var version string + if interactiveVersion != "" { + version = interactiveVersion + } else { + version = determineVersion(args) + } // Download and install the node if version == "latest" { @@ -143,7 +253,7 @@ Examples: // installNode installs the Quilibrium node func InstallNode(version string) { // Create installation directory - if err := utils.ValidateAndCreateDir(utils.NodeDataPath, NodeUser); err != nil { + if err := utils.ValidateAndCreateDir(utils.GetNodeBinaryDir(), NodeUser); err != nil { fmt.Fprintf(os.Stderr, "Error creating installation directory: %v\n", err) return } @@ -166,7 +276,7 @@ func InstallNode(version string) { // installByVersion installs a specific version of the Quilibrium node func InstallByVersion(version string) error { - versionDir := filepath.Join(utils.NodeDataPath, version) + versionDir := filepath.Join(utils.GetNodeBinaryDir(), version) if err := utils.ValidateAndCreateDir(versionDir, NodeUser); err != nil { return fmt.Errorf("failed to create version directory: %w", err) } @@ -183,3 +293,346 @@ func InstallByVersion(version string) error { return nil } + +// applyInstallDirFlags persists any --install-dir/--symlink-dir/ +// --configs-dir overrides to the client config. It validates that each +// supplied path is absolute and warns (but does not block) when an +// existing installation would need to be rebuilt for the change to take +// full effect. +func applyInstallDirFlags() error { + cfg, err := utils.LoadClientConfig() + if err != nil { + return fmt.Errorf("loading client config: %w", err) + } + + nodeInstalled := utils.FileExists(utils.GetNodeSymlinkPath()) + + updates := []struct { + name string + flagValue string + current *string + // prevResolvedPath describes the currently effective path we print + // in the "already installed" warning, to help the user understand + // what would be rebuilt. + prevResolved string + }{ + {"install-dir", installDirFlag, &cfg.NodeInstallDir, utils.GetNodeInstallDir()}, + {"state-dir", stateDirFlag, &cfg.NodeStateDir, utils.GetNodeStateDir()}, + {"symlink-dir", symlinkDirFlag, &cfg.NodeSymlinkDir, utils.GetNodeSymlinkDir()}, + {"configs-dir", configsDirFlag, &cfg.NodeConfigsDir, utils.GetNodeConfigsDir()}, + } + + changed := false + for _, u := range updates { + if u.flagValue == "" { + continue + } + if !filepath.IsAbs(u.flagValue) { + return fmt.Errorf( + "--%s must be an absolute path, got %q", u.name, u.flagValue, + ) + } + if u.flagValue == *u.current { + continue + } + + if nodeInstalled && u.flagValue != u.prevResolved { + fmt.Fprintf(os.Stderr, + "Warning: --%s changes %s -> %s, but an existing node "+ + "installation was detected. The new value has been "+ + "saved to the qclient config and will take effect on "+ + "the next install/update; existing files at the old "+ + "path have not been moved.\n", + u.name, u.prevResolved, u.flagValue, + ) + } + + *u.current = u.flagValue + changed = true + } + + if serviceNameFlag != "" { + if err := utils.ValidateNodeServiceName(serviceNameFlag); err != nil { + return fmt.Errorf("--service-name: %w", err) + } + if cfg.NodeServiceName != serviceNameFlag { + if nodeInstalled { + fmt.Fprintf(os.Stderr, + "Warning: --service-name changes %s -> %s, but an "+ + "existing node installation was detected. The new "+ + "value has been saved to the qclient config and "+ + "will take effect on the next install/update; the "+ + "previously installed service unit has not been "+ + "renamed. Use 'qclient config service-name' to "+ + "migrate an installed unit in place.\n", + cfg.NodeServiceName, serviceNameFlag, + ) + } + cfg.NodeServiceName = serviceNameFlag + changed = true + } + } + + if !changed { + return nil + } + + if err := utils.SaveClientConfig(cfg); err != nil { + return fmt.Errorf("saving client config: %w", err) + } + return nil +} + +// warnLegacyInstallLayout emits a one-shot warning when the install +// would land in (or leave behind) the pre-FHS-split /var/quilibrium +// layout. No files are moved; the user is told how to opt in to the +// new defaults or explicitly pin to the old path. +func warnLegacyInstallLayout() { + cfg, err := utils.LoadClientConfig() + if err != nil { + return + } + + resolvedInstall := utils.GetNodeInstallDir() + resolvedState := utils.GetNodeStateDir() + + pinnedLegacyInstall := cfg.NodeInstallDir == utils.LegacyNodeInstallDir + pinnedLegacyState := cfg.NodeStateDir == utils.LegacyNodeInstallDir + legacyTreeExists := utils.FileExists( + filepath.Join(utils.LegacyNodeInstallDir, "bin", "node"), + ) || utils.FileExists( + filepath.Join(utils.LegacyNodeInstallDir, "quilibrium.env"), + ) + + if !pinnedLegacyInstall && !pinnedLegacyState && !legacyTreeExists { + return + } + + defaultInstall := utils.DefaultNodeInstallDir() + defaultState := utils.DefaultNodeStateDir() + + fmt.Fprintln(os.Stderr) + fmt.Fprintln(os.Stderr, + "Notice: the default install layout has moved off "+ + utils.LegacyNodeInstallDir+".") + fmt.Fprintf(os.Stderr, + " New defaults: binaries at %s, env/state at %s.\n", + defaultInstall, defaultState, + ) + if pinnedLegacyInstall || pinnedLegacyState { + var pins []string + var flagsToDrop []string + if pinnedLegacyInstall { + pins = append(pins, fmt.Sprintf("install-dir=%s", resolvedInstall)) + flagsToDrop = append(flagsToDrop, "--install-dir") + } + if pinnedLegacyState { + pins = append(pins, fmt.Sprintf("state-dir=%s", resolvedState)) + flagsToDrop = append(flagsToDrop, "--state-dir") + } + fmt.Fprintf(os.Stderr, + " Your qclient config currently pins %s to the legacy layout;\n"+ + " this install will keep writing there.\n", + strings.Join(pins, ", "), + ) + fmt.Fprintf(os.Stderr, + " To adopt the new default(s), run 'sudo qclient node uninstall' "+ + "then reinstall without %s.\n", + strings.Join(flagsToDrop, "/"), + ) + } else if legacyTreeExists { + fmt.Fprintf(os.Stderr, + " A legacy install tree was detected under %s but this install "+ + "will use the new defaults (%s and %s).\n", + utils.LegacyNodeInstallDir, resolvedInstall, resolvedState, + ) + fmt.Fprintf(os.Stderr, + " To stay on the legacy layout, rerun with "+ + "--install-dir %s --state-dir %s.\n", + utils.LegacyNodeInstallDir, utils.LegacyNodeInstallDir, + ) + fmt.Fprintf(os.Stderr, + " Files under %s are NOT moved automatically; remove them "+ + "manually once you've verified the new install.\n", + utils.LegacyNodeInstallDir, + ) + } + fmt.Fprintln(os.Stderr) +} + +func init() { + NodeInstallCmd.Flags().StringVar( + &installDirFlag, "install-dir", "", + "Root install directory for node binaries (defaults to "+ + "/opt/quilibrium on Linux, /usr/local/quilibrium on macOS). "+ + "Persisted to qclient config.", + ) + NodeInstallCmd.Flags().StringVar( + &stateDirFlag, "state-dir", "", + "Root directory for mutable node state / env file (defaults "+ + "to /var/lib/quilibrium on Linux, /usr/local/var/quilibrium "+ + "on macOS). Persisted to qclient config.", + ) + NodeInstallCmd.Flags().StringVar( + &symlinkDirFlag, "symlink-dir", "", + "Directory for the quilibrium-node symlink (defaults to "+ + "/usr/local/bin). Persisted to qclient config.", + ) + NodeInstallCmd.Flags().StringVar( + &configsDirFlag, "configs-dir", "", + "Directory holding named node configs (defaults to "+ + "~/.quilibrium/configs). Persisted to qclient config.", + ) + NodeInstallCmd.Flags().StringVar( + &serviceNameFlag, "service-name", "", + "Name of the systemd service unit for the node (defaults to "+ + "\"quilibrium-node\"). Persisted to qclient config.", + ) + NodeInstallCmd.Flags().BoolVarP( + &interactiveFlag, "interactive", "i", false, + "Prompt for each install setting (version and directories) "+ + "instead of requiring flags. Pressing Enter at a prompt "+ + "keeps the current/default value.", + ) +} + +// runInteractiveInstallPrompts walks the user through each install-time +// setting and writes the answers back into the same package-level flag +// variables that --install-dir / --state-dir / --symlink-dir / +// --configs-dir populate. It returns the version string the user +// selected (or "" to fall back to the positional arg / "latest"). +// +// Each prompt shows the currently effective value in brackets; an empty +// response keeps that value. +func runInteractiveInstallPrompts(args []string) (string, error) { + reader := bufio.NewReader(os.Stdin) + + fmt.Fprintln(os.Stdout, "Interactive install. Press Enter to accept the shown default.") + fmt.Fprintln(os.Stdout) + + versionDefault := "latest" + if len(args) == 1 && strings.TrimSpace(args[0]) != "" { + versionDefault = strings.TrimSpace(args[0]) + } + version, err := promptString(reader, "Node version to install", versionDefault) + if err != nil { + return "", err + } + + dirPrompts := []struct { + label string + target *string + cur string + }{ + {"Install directory (binaries)", &installDirFlag, utils.GetNodeInstallDir()}, + {"State directory (env file / mutable state)", &stateDirFlag, utils.GetNodeStateDir()}, + {"Symlink directory (must be on $PATH)", &symlinkDirFlag, utils.GetNodeSymlinkDir()}, + {"Configs directory (named node configs)", &configsDirFlag, utils.GetNodeConfigsDir()}, + } + + for _, p := range dirPrompts { + val, err := promptAbsPath(reader, p.label, p.cur) + if err != nil { + return "", err + } + if val != p.cur { + *p.target = val + } + } + + curServiceName := utils.GetNodeServiceName() + svcName, err := promptServiceName( + reader, "Service name (systemd unit name)", curServiceName, + ) + if err != nil { + return "", err + } + if svcName != curServiceName { + serviceNameFlag = svcName + } + + fmt.Fprintln(os.Stdout) + fmt.Fprintf(os.Stdout, "Summary:\n") + fmt.Fprintf(os.Stdout, " version : %s\n", version) + fmt.Fprintf(os.Stdout, " install-dir : %s\n", effective(installDirFlag, utils.GetNodeInstallDir())) + fmt.Fprintf(os.Stdout, " state-dir : %s\n", effective(stateDirFlag, utils.GetNodeStateDir())) + fmt.Fprintf(os.Stdout, " symlink-dir : %s\n", effective(symlinkDirFlag, utils.GetNodeSymlinkDir())) + fmt.Fprintf(os.Stdout, " configs-dir : %s\n", effective(configsDirFlag, utils.GetNodeConfigsDir())) + fmt.Fprintf(os.Stdout, " service-name : %s\n", effective(serviceNameFlag, utils.GetNodeServiceName())) + fmt.Fprintln(os.Stdout) + + ok, err := promptYesNo(reader, "Proceed with these settings?", true) + if err != nil { + return "", err + } + if !ok { + return "", fmt.Errorf("install cancelled by user") + } + + return version, nil +} + +func effective(flagVal, current string) string { + if flagVal != "" { + return flagVal + } + return current +} + +func promptString(r *bufio.Reader, label, def string) (string, error) { + fmt.Fprintf(os.Stdout, "%s [%s]: ", label, def) + line, err := r.ReadString('\n') + if err != nil { + return "", fmt.Errorf("reading input: %w", err) + } + line = strings.TrimSpace(line) + if line == "" { + return def, nil + } + return line, nil +} + +func promptAbsPath(r *bufio.Reader, label, def string) (string, error) { + for { + val, err := promptString(r, label, def) + if err != nil { + return "", err + } + if !filepath.IsAbs(val) { + fmt.Fprintf(os.Stderr, " path must be absolute, got %q\n", val) + continue + } + return val, nil + } +} + +func promptServiceName(r *bufio.Reader, label, def string) (string, error) { + for { + val, err := promptString(r, label, def) + if err != nil { + return "", err + } + if err := utils.ValidateNodeServiceName(val); err != nil { + fmt.Fprintf(os.Stderr, " %v\n", err) + continue + } + return val, nil + } +} + +func promptYesNo(r *bufio.Reader, label string, def bool) (bool, error) { + hint := "[y/N]" + if def { + hint = "[Y/n]" + } + fmt.Fprintf(os.Stdout, "%s %s: ", label, hint) + line, err := r.ReadString('\n') + if err != nil { + return false, fmt.Errorf("reading input: %w", err) + } + line = strings.TrimSpace(strings.ToLower(line)) + if line == "" { + return def, nil + } + return line == "y" || line == "yes", nil +} diff --git a/client/cmd/node/link.go b/client/cmd/node/link.go index a91c035b..5a8aeee5 100644 --- a/client/cmd/node/link.go +++ b/client/cmd/node/link.go @@ -57,14 +57,14 @@ func NodeCreateSymlink() error { } if latestVersion == "" { - return fmt.Errorf("no node versions found in %s", utils.NodeDataPath) + return fmt.Errorf("no node versions found in %s", utils.GetNodeBinaryDir()) } Version = latestVersion } // Construct the path to the binary with the highest version normalizedBinaryName := fmt.Sprintf("node-%s-%s-%s", Version, OsType, Arch) - nodeBinaryPath := filepath.Join(utils.NodeDataPath, Version, normalizedBinaryName) + nodeBinaryPath := filepath.Join(utils.GetNodeBinaryDir(), Version, normalizedBinaryName) // Check if the binary exists if _, err := os.Stat(nodeBinaryPath); os.IsNotExist(err) { @@ -72,7 +72,7 @@ func NodeCreateSymlink() error { } // Check if we need sudo privileges for creating symlink in system directory - symlinkPath := filepath.Join("/usr/local/bin", utils.NodeServiceName) + symlinkPath := utils.GetNodeSymlinkPath() if err := utils.CheckAndRequestSudo(fmt.Sprintf("Creating symlink at %s requires root privileges", symlinkPath)); err != nil { return fmt.Errorf("failed to get sudo privileges: %w", err) } @@ -88,10 +88,11 @@ func NodeCreateSymlink() error { // findHighestNodeVersion finds the highest version number in the node binary directory func findLatestNodeVersion() (string, error) { + binDir := utils.GetNodeBinaryDir() // Read the directory contents - entries, err := os.ReadDir(utils.NodeDataPath) + entries, err := os.ReadDir(binDir) if err != nil { - return "", fmt.Errorf("failed to read node data directory %s: %w", utils.NodeDataPath, err) + return "", fmt.Errorf("failed to read node data directory %s: %w", binDir, err) } var versions []string diff --git a/client/cmd/node/log/clean.go b/client/cmd/node/log/clean.go index 19cb2c3d..8a48926d 100644 --- a/client/cmd/node/log/clean.go +++ b/client/cmd/node/log/clean.go @@ -13,7 +13,12 @@ import ( var LogCleanCmd = &cobra.Command{ Use: "clean", Short: "Clean node logs", - Long: `Remove all log files from the Quilibrium node log directory. + Long: `Remove all log files from the active node config's log directory. + +The log directory is resolved from the active node config's +logger.path. If the active config has no logger block (i.e. the node +logs to the system log), there is nothing for this command to clean — +use the system log tooling (e.g. journalctl --vacuum-time=...) instead. Examples: qclient node log clean`, @@ -23,34 +28,52 @@ Examples: return } - logDir := utils.LogPath - entries, err := os.ReadDir(logDir) + resolved, err := utils.ResolveActiveNodeLog() if err != nil { - if os.IsNotExist(err) { - fmt.Println("No logs directory found.") - } else { - fmt.Fprintf(os.Stderr, "Error reading log directory: %v\n", err) - } + fmt.Fprintf(os.Stderr, "Error resolving node log: %v\n", err) return } - - removed := 0 - for _, entry := range entries { - name := entry.Name() - if strings.HasSuffix(name, ".log") || strings.HasSuffix(name, ".log.gz") { - path := filepath.Join(logDir, name) - if err := os.Remove(path); err != nil { - fmt.Fprintf(os.Stderr, "Warning: could not remove %s: %v\n", name, err) - continue - } - removed++ - } + if !resolved.FileBased { + fmt.Fprintf(os.Stderr, + "Node config %q at %s has no logger block; the node "+ + "logs to the system log, which qclient does not "+ + "clean. Use journalctl --vacuum-time=... or set "+ + "logger.path first.\n", + resolved.ConfigName, resolved.ConfigDir, + ) + return } - fmt.Printf("Removed %d log file(s) from %s\n", removed, logDir) + removed := cleanLogsIn(resolved.LogDir) + fmt.Printf("Removed %d log file(s) from %s\n", removed, resolved.LogDir) }, } +func cleanLogsIn(logDir string) int { + entries, err := os.ReadDir(logDir) + if err != nil { + if os.IsNotExist(err) { + fmt.Println("No logs directory found.") + } else { + fmt.Fprintf(os.Stderr, "Error reading log directory: %v\n", err) + } + return 0 + } + removed := 0 + for _, entry := range entries { + name := entry.Name() + if strings.HasSuffix(name, ".log") || strings.HasSuffix(name, ".log.gz") { + path := filepath.Join(logDir, name) + if err := os.Remove(path); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not remove %s: %v\n", name, err) + continue + } + removed++ + } + } + return removed +} + func init() { LogCmd.AddCommand(LogCleanCmd) } diff --git a/client/cmd/node/log/disable.go b/client/cmd/node/log/disable.go new file mode 100644 index 00000000..2ebf4c40 --- /dev/null +++ b/client/cmd/node/log/disable.go @@ -0,0 +1,72 @@ +package log + +import ( + "fmt" + "os" + "path/filepath" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" + "source.quilibrium.com/quilibrium/monorepo/config" +) + +var LogDisableCmd = &cobra.Command{ + Use: "disable", + Short: "Disable file-based logging for the active node config", + Long: `Disable file-based logging by removing the logger block from the +active node config's config.yml. The node will fall back to stdout, +which the service manager captures (journalctl on Linux, launchd +StandardOutPath on macOS). + +Existing log files on disk are not deleted; use ` + "`qclient node log clean`" + ` +first if you want to wipe them. + +Examples: + qclient node log disable + qclient node log disable --config mynode`, + Run: func(cmd *cobra.Command, args []string) { + resolved, err := utils.ResolveActiveNodeLog() + if err != nil { + fmt.Fprintf(os.Stderr, "Error resolving active node config: %v\n", err) + os.Exit(1) + } + if resolved.ConfigDir == "" { + fmt.Fprintln(os.Stderr, + "No active node config found. Run `qclient node config create` first.", + ) + os.Exit(1) + } + + cfgPath := filepath.Join(resolved.ConfigDir, "config.yml") + cfg, err := config.NewConfig(cfgPath) + if err != nil { + fmt.Fprintf(os.Stderr, "Error reading %s: %v\n", cfgPath, err) + os.Exit(1) + } + + if cfg.Logger == nil { + fmt.Printf("File-based logging is already disabled for %q (%s).\n", + resolved.ConfigName, cfgPath) + return + } + + prevPath := cfg.Logger.Path + cfg.Logger = nil + + if err := config.SaveConfig(resolved.ConfigDir, cfg); err != nil { + fmt.Fprintf(os.Stderr, "Error saving %s: %v\n", cfgPath, err) + os.Exit(1) + } + + fmt.Printf("Disabled file-based logging for %q (%s).\n", + resolved.ConfigName, cfgPath) + if prevPath != "" { + fmt.Printf("Existing log files under %s were left in place; "+ + "run `qclient node log clean` to remove them.\n", prevPath) + } + }, +} + +func init() { + LogCmd.AddCommand(LogDisableCmd) +} diff --git a/client/cmd/node/log/enable.go b/client/cmd/node/log/enable.go new file mode 100644 index 00000000..15c3cca0 --- /dev/null +++ b/client/cmd/node/log/enable.go @@ -0,0 +1,181 @@ +package log + +import ( + "fmt" + "os" + "path/filepath" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" + "source.quilibrium.com/quilibrium/monorepo/config" +) + +var ( + enablePath string + enableMaxSize int + enableMaxBackups int + enableMaxAge int + enableCompress bool + enableForce bool + + // Track which flags the user actually set so we only override when + // they asked us to (vs. falling back to the existing value or the + // built-in default). + enableCompressSet bool +) + +var LogEnableCmd = &cobra.Command{ + Use: "enable", + Short: "Enable file-based logging for the active node config", + Long: fmt.Sprintf(`Enable file-based logging by writing a logger block into the +active node config's config.yml. + +If the active node config already has a logger block with a non-empty +path, this command leaves existing values in place unless --force is +passed. Missing fields are filled in with the defaults below. Any flag +you pass overrides both the existing value and the default. + +Defaults: + path /%s + max-size %d # Rotate after %dMB + max-backups %d # Keep %d old log files + max-age %d # Delete logs older than %d days + compress %t # Gzip rotated files + +Examples: + qclient node log enable + qclient node log enable --path /mnt/logs/quil --max-size 200 + qclient node log enable --max-backups 10 --max-age 60 --compress=false + qclient node log enable --force --path /var/log/quilibrium/mynode`, + utils.DefaultNodeLogRelDir, + utils.DefaultLoggerMaxSize, utils.DefaultLoggerMaxSize, + utils.DefaultLoggerMaxBackups, utils.DefaultLoggerMaxBackups, + utils.DefaultLoggerMaxAge, utils.DefaultLoggerMaxAge, + utils.DefaultLoggerCompress, + ), + Run: func(cmd *cobra.Command, args []string) { + enableCompressSet = cmd.Flags().Changed("compress") + + resolved, err := utils.ResolveActiveNodeLog() + if err != nil { + fmt.Fprintf(os.Stderr, "Error resolving active node config: %v\n", err) + os.Exit(1) + } + if resolved.ConfigDir == "" { + fmt.Fprintln(os.Stderr, + "No active node config found. Run `qclient node config create` first.", + ) + os.Exit(1) + } + + cfgPath := filepath.Join(resolved.ConfigDir, "config.yml") + cfg, err := config.NewConfig(cfgPath) + if err != nil { + fmt.Fprintf(os.Stderr, "Error reading %s: %v\n", cfgPath, err) + os.Exit(1) + } + + preexisting := cfg.Logger != nil && cfg.Logger.Path != "" + if cfg.Logger == nil { + cfg.Logger = &config.LogConfig{} + } + + applyEnableFlags(cfg.Logger, resolved.ConfigDir, preexisting) + + if err := config.SaveConfig(resolved.ConfigDir, cfg); err != nil { + fmt.Fprintf(os.Stderr, "Error saving %s: %v\n", cfgPath, err) + os.Exit(1) + } + + if preexisting && !enableForce && !anyEnableFlagSet(cmd) { + fmt.Printf("Logger already enabled for %q (%s); leaving existing settings in place.\n", + resolved.ConfigName, cfgPath) + } else { + fmt.Printf("Enabled file-based logging for %q (%s).\n", + resolved.ConfigName, cfgPath) + } + printLoggerSummary(cfg.Logger) + }, +} + +// applyEnableFlags fills in the logger block using, in priority order: +// 1. values explicitly passed on the command line, +// 2. existing values in the config (unless --force is set), +// 3. built-in defaults from the utils package. +func applyEnableFlags(lg *config.LogConfig, configDir string, preexisting bool) { + if enablePath != "" { + lg.Path = enablePath + } else if lg.Path == "" || enableForce { + lg.Path = utils.DefaultNodeLogDirForConfig(configDir) + } + + if enableMaxSize > 0 { + lg.MaxSize = enableMaxSize + } else if lg.MaxSize == 0 || enableForce { + lg.MaxSize = utils.DefaultLoggerMaxSize + } + + if enableMaxBackups > 0 { + lg.MaxBackups = enableMaxBackups + } else if lg.MaxBackups == 0 || enableForce { + lg.MaxBackups = utils.DefaultLoggerMaxBackups + } + + if enableMaxAge > 0 { + lg.MaxAge = enableMaxAge + } else if lg.MaxAge == 0 || enableForce { + lg.MaxAge = utils.DefaultLoggerMaxAge + } + + if enableCompressSet { + lg.Compress = enableCompress + } else if !preexisting || enableForce { + lg.Compress = utils.DefaultLoggerCompress + } +} + +func anyEnableFlagSet(cmd *cobra.Command) bool { + names := []string{"path", "max-size", "max-backups", "max-age", "compress"} + for _, n := range names { + if cmd.Flags().Changed(n) { + return true + } + } + return false +} + +func printLoggerSummary(lg *config.LogConfig) { + fmt.Println(" logger:") + if lg == nil { + fmt.Println(" (none)") + return + } + fmt.Printf(" path: %s\n", lg.Path) + fmt.Printf(" maxSize: %d\n", lg.MaxSize) + fmt.Printf(" maxBackups: %d\n", lg.MaxBackups) + fmt.Printf(" maxAge: %d\n", lg.MaxAge) + fmt.Printf(" compress: %t\n", lg.Compress) + if len(lg.LogFilters) > 0 { + fmt.Println(" logFilters:") + for k, v := range lg.LogFilters { + fmt.Printf(" %s: %s\n", k, v) + } + } +} + +func init() { + LogEnableCmd.Flags().StringVar(&enablePath, "path", "", + "Directory where the node writes logs (default: /"+utils.DefaultNodeLogRelDir+")") + LogEnableCmd.Flags().IntVar(&enableMaxSize, "max-size", 0, + "Megabytes per log file before rotation (default: 100)") + LogEnableCmd.Flags().IntVar(&enableMaxBackups, "max-backups", 0, + "Number of rotated log files to keep (default: 5)") + LogEnableCmd.Flags().IntVar(&enableMaxAge, "max-age", 0, + "Days to keep rotated log files (default: 30)") + LogEnableCmd.Flags().BoolVar(&enableCompress, "compress", true, + "Gzip rotated log files") + LogEnableCmd.Flags().BoolVar(&enableForce, "force", false, + "Overwrite existing logger fields with defaults/flags") + + LogCmd.AddCommand(LogEnableCmd) +} diff --git a/client/cmd/node/log/log.go b/client/cmd/node/log/log.go index 69a9bc37..dd8dc544 100644 --- a/client/cmd/node/log/log.go +++ b/client/cmd/node/log/log.go @@ -5,7 +5,7 @@ import ( "os" "os/exec" "os/signal" - "path/filepath" + "runtime" "strconv" "syscall" @@ -23,6 +23,10 @@ var LogCmd = &cobra.Command{ Short: "View and manage node logs", Long: `View and manage Quilibrium node logs. +Logs are read from the active node config's logger.path. If the active +node config has no logger block, qclient falls back to the system log +(journalctl on Linux, launchd StandardOutPath on macOS). + Examples: qclient node log view qclient node log view --lines 200 @@ -36,26 +40,91 @@ Examples: var LogViewCmd = &cobra.Command{ Use: "view", Short: "View node logs", - Long: `View the Quilibrium node log file. + Long: `View the Quilibrium node log. + +The log source is resolved from the active node config's logger.path. +If the config has no logger block, the system log is used instead +(journalctl -u on Linux, the launchd StandardOutPath on macOS). Examples: qclient node log view # show last 100 lines qclient node log view --lines 200 # show last 200 lines qclient node log view --follow # follow log output`, Run: func(cmd *cobra.Command, args []string) { - logFile := filepath.Join(utils.LogPath, "quilibrium-node.log") + resolved, err := utils.ResolveActiveNodeLog() + if err != nil { + fmt.Fprintf(os.Stderr, "Error resolving node log: %v\n", err) + return + } - if _, err := os.Stat(logFile); os.IsNotExist(err) { - fmt.Fprintf(os.Stderr, "Log file not found: %s\n", logFile) + if resolved.FileBased { + logFile := resolved.MasterLogFile() + if _, err := os.Stat(logFile); os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, + "Log file not found: %s\n"+ + "The node config at %s declares logger.path=%s "+ + "but no master.log has been written yet. "+ + "Has the node started?\n", + logFile, resolved.ConfigDir, resolved.LogDir, + ) + return + } + if follow { + tailFollow(logFile) + } else { + tailLines(logFile) + } return } + viewSystemLog(resolved) + }, +} + +// viewSystemLog falls back to journalctl (Linux) or the launchd stdout +// path (macOS) when the active node config has no logger block. +func viewSystemLog(resolved utils.ResolvedNodeLog) { + fmt.Fprintf(os.Stderr, + "Node config %q at %s has no logger block; reading from the "+ + "system log instead. Run `qclient node config set "+ + "logger.path ` to enable file-based logging.\n", + resolved.ConfigName, resolved.ConfigDir, + ) + + switch runtime.GOOS { + case "linux": + service := utils.GetNodeServiceName() + args := []string{"-u", service, "-n", strconv.Itoa(lines), "--no-pager"} + if follow { + args = append(args, "-f") + } + runStreaming("journalctl", args...) + case "darwin": + // launchd writes stdout/stderr into /node.log per the + // plist installed by qclient. Fall back to the pre-refactor + // root log directory so existing installs keep working. + stdoutPath := "/var/log/quilibrium/node.log" + if _, err := os.Stat(stdoutPath); err != nil { + fmt.Fprintf(os.Stderr, + "No launchd stdout log found at %s. Check the service "+ + "plist under /Library/LaunchDaemons for the "+ + "StandardOutPath.\n", + stdoutPath, + ) + return + } if follow { - tailFollow(logFile) + runStreaming("tail", "-n", strconv.Itoa(lines), "-f", stdoutPath) } else { - tailLines(logFile) + runStreaming("tail", "-n", strconv.Itoa(lines), stdoutPath) } - }, + default: + fmt.Fprintf(os.Stderr, + "System log fallback is not supported on %s; set "+ + "logger.path in the node config to use file logging.\n", + runtime.GOOS, + ) + } } func tailLines(logFile string) { @@ -69,25 +138,30 @@ func tailLines(logFile string) { } func tailFollow(logFile string) { - cmd := exec.Command("tail", "-n", strconv.Itoa(lines), "-f", logFile) + runStreaming("tail", "-n", strconv.Itoa(lines), "-f", logFile) +} + +// runStreaming runs an external command, wiring stdout/stderr through +// to this process and forwarding SIGINT/SIGTERM so Ctrl+C behaves +// correctly when following logs. +func runStreaming(name string, args ...string) { + cmd := exec.Command(name, args...) cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr if err := cmd.Start(); err != nil { - fmt.Fprintf(os.Stderr, "Error starting log follow: %v\n", err) + fmt.Fprintf(os.Stderr, "Error starting %s: %v\n", name, err) return } - // Handle signals to clean up the tail process sigCh := make(chan os.Signal, 1) signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM) - go func() { <-sigCh - cmd.Process.Kill() + _ = cmd.Process.Kill() }() - cmd.Wait() + _ = cmd.Wait() } func init() { diff --git a/client/cmd/node/node.go b/client/cmd/node/node.go index 839865da..0b1c5ff2 100644 --- a/client/cmd/node/node.go +++ b/client/cmd/node/node.go @@ -4,10 +4,10 @@ import ( "fmt" "os" "os/user" - "path/filepath" "github.com/spf13/cobra" "github.com/spf13/viper" + backupCmd "source.quilibrium.com/quilibrium/monorepo/client/cmd/node/backup" logCmd "source.quilibrium.com/quilibrium/monorepo/client/cmd/node/log" configCmd "source.quilibrium.com/quilibrium/monorepo/client/cmd/node/nodeconfig" proverCmd "source.quilibrium.com/quilibrium/monorepo/client/cmd/node/prover" @@ -21,10 +21,14 @@ var ( ConfigDirectory string NodeConfig *config.Config + // NodeConfigDir is the absolute directory holding the active node + // config.yml (either the resolved --config value or the default + // config's symlink target). Subcommands that write to config.yml + // should use this, not the bare "default" symlink. + NodeConfigDir string - NodeUser *user.User - ConfigDirs string - NodeConfigToRun string + NodeUser *user.User + ConfigDirs string ) // NodeCmd represents the node command @@ -33,6 +37,12 @@ var NodeCmd = &cobra.Command{ Short: "Quilibrium node commands", Long: `Run Quilibrium node commands.`, PersistentPreRun: func(cmd *cobra.Command, args []string) { + // Install must be sudo from the first moment: skip root/node init + // (config load, default config creation, etc.) until privileges are OK. + if cmd == NodeInstallCmd { + ExitUnlessSudoForInstall() + } + // Store reference to parent's PersistentPreRun to call it first parent := cmd.Parent() if parent != nil && parent.PersistentPreRun != nil { @@ -48,27 +58,48 @@ var NodeCmd = &cobra.Command{ os.Exit(1) } NodeUser = userLookup - ConfigDirs = filepath.Join(userLookup.HomeDir, ".quilibrium", "configs") + ConfigDirs = utils.GetNodeConfigsDir() if ConfigDirectory != "" { + resolved, rErr := utils.ResolveNodeConfigDir(ConfigDirectory) + if rErr != nil { + fmt.Printf("error resolving node config: %s\n", rErr) + os.Exit(1) + } + NodeConfigDir = resolved NodeConfig, err = utils.LoadNodeConfig(ConfigDirectory) + if err != nil { + fmt.Printf("error loading node config: %s\n", err) + os.Exit(1) + } } else { NodeConfig, err = utils.LoadDefaultNodeConfig() - } - if err != nil { - if err.Error() == utils.ErrConfigNotFoundErrorMessage { - fmt.Println("Config not found, creating default configuration...") - nodeConfig, err := utils.CreateDefaultNodeConfig(utils.DefaultNodeConfigName) - if err != nil { - fmt.Printf("error creating default node config: %s\n", err) + if err != nil { + if err.Error() == utils.ErrConfigNotFoundErrorMessage { + fmt.Println("Config not found, creating default configuration...") + nodeConfig, err := utils.CreateDefaultNodeConfig(utils.DefaultNodeConfigName) + if err != nil { + fmt.Printf("error creating default node config: %s\n", err) + os.Exit(1) + } + NodeConfig = nodeConfig + } else { + fmt.Printf("error loading node config: %s\n", err) os.Exit(1) } - NodeConfig = nodeConfig + } + // Resolve the default symlink to an absolute path so writes + // target the actual config directory rather than the + // symlink path (which breaks if the symlink is later + // re-pointed mid-operation). + if dir, dErr := utils.GetDefaultNodeConfigDir(); dErr == nil { + NodeConfigDir = dir } else { - fmt.Printf("error loading node config: %s\n", err) - os.Exit(1) + NodeConfigDir = utils.GetDefaultNodeConfigSymlink() } } proverCmd.NodeConfig = NodeConfig + configCmd.NodeConfig = NodeConfig + configCmd.ActiveNodeConfigDir = NodeConfigDir }, Run: func(cmd *cobra.Command, args []string) { cmd.Help() @@ -91,6 +122,13 @@ func init() { NodeCmd.AddCommand(NodeUninstallCmd) NodeCmd.AddCommand(NodeLinkCmd) NodeCmd.AddCommand(logCmd.LogCmd) + NodeCmd.AddCommand(NodeGrpcCmd) + NodeCmd.AddCommand(NodeRestCmd) + NodeCmd.AddCommand(backupCmd.BackupCmd) + + for _, c := range ServiceAliasCommands() { + NodeCmd.AddCommand(c) + } OsType = utils.OsType Arch = utils.Arch diff --git a/client/cmd/node/nodeconfig/config.go b/client/cmd/node/nodeconfig/config.go index 39ee058f..9752dc0e 100644 --- a/client/cmd/node/nodeconfig/config.go +++ b/client/cmd/node/nodeconfig/config.go @@ -2,9 +2,7 @@ package nodeconfig import ( "fmt" - "os" "os/user" - "path/filepath" "github.com/spf13/cobra" "source.quilibrium.com/quilibrium/monorepo/client/utils" @@ -12,11 +10,22 @@ import ( ) var ( - NodeUser *user.User - ConfigDirs string + NodeUser *user.User + ConfigDirs string + // NodeConfigToRun is the default-config symlink path + // (~/.quilibrium/configs/default). `config create`, `config + // import`, and `config switch` use it as the *link destination* so + // the node always loads whichever real config is currently aliased + // as "default". NodeConfigToRun string - SetDefault bool - NodeConfig *config.Config + // ActiveNodeConfigDir is the absolute path to the directory of the + // currently-active config.yml — either the resolved --config value + // or the default symlink's target. Commands that write to + // config.yml (e.g. `config set`) use this so writes land in the + // real config dir rather than in the CWD. + ActiveNodeConfigDir string + SetDefault bool + NodeConfig *config.Config ) // ConfigCmd represents the node config command @@ -38,13 +47,19 @@ This command provides utilities for configuring your Quilibrium node, such as: parent.PersistentPreRun(parent, args) } - // Check if the config directory exists - user, err := utils.GetCurrentSudoUser() - if err != nil { - fmt.Println("Error getting current user:", err) - os.Exit(1) - } - ConfigDirs = filepath.Join(user.HomeDir, ".quilibrium", "configs") + ConfigDirs = utils.GetNodeConfigsDir() + NodeConfigToRun = utils.GetDefaultNodeConfigSymlink() + // NodeConfig and ActiveNodeConfigDir are populated by the + // parent node command's PersistentPreRun (which has already + // run above via parent.PersistentPreRun) so that --config is + // honored. + + NodeConfigSwitchCmd.Long = fmt.Sprintf(`Switch the configuration to be run by the node by creating a symlink. + +Example: + qclient node config switch mynode + +This will symlink %s/mynode to %s`, ConfigDirs, NodeConfigToRun) }, Run: func(cmd *cobra.Command, args []string) { cmd.Help() diff --git a/client/cmd/node/nodeconfig/set.go b/client/cmd/node/nodeconfig/set.go index 19fc0ebe..c5bcdab9 100644 --- a/client/cmd/node/nodeconfig/set.go +++ b/client/cmd/node/nodeconfig/set.go @@ -3,6 +3,8 @@ package nodeconfig import ( "fmt" "os" + "strconv" + "strings" "github.com/spf13/cobra" "source.quilibrium.com/quilibrium/monorepo/config" @@ -12,40 +14,151 @@ var NodeConfigSetCmd = &cobra.Command{ Use: "set [key] [value]", Short: "Set a configuration value", Long: `Set a configuration value in the node config.yml file. - - To specify a config other than the default, use the --config flag. -Example: - qclient node config set mynode engine.statsMultiaddr /dns/stats.quilibrium.com/tcp/443 - + +To specify a config other than the default, use the --config flag. + +Supported keys: + engine.statsMultiaddr + p2p.listenMultiaddr + listenGrpcMultiaddr + listenRestMultiaddr + logger.path Directory where the node writes logs + logger.maxSize Megabytes per log file before rotation + logger.maxBackups Number of rotated files to keep + logger.maxAge Days to keep rotated files + logger.compress true/false — gzip rotated files + logger.logFilters Comma-separated list of component=level pairs, + e.g. "p2p=debug,engine=warn" + +Examples: + qclient node config set engine.statsMultiaddr /dns/stats.quilibrium.com/tcp/443 qclient node config set --config mynode engine.statsMultiaddr /dns/stats.quilibrium.com/tcp/443 + qclient node config set logger.path /path/to/my/logs + qclient node config set logger.compress true + qclient node config set logger.logFilters p2p=debug,engine=warn `, Args: cobra.ExactArgs(2), Run: func(cmd *cobra.Command, args []string) { key := args[0] value := args[1] - // Update the config based on the key - switch key { - case "engine.statsMultiaddr": - NodeConfig.Engine.StatsMultiaddr = value - case "p2p.listenMultiaddr": - NodeConfig.P2P.ListenMultiaddr = value - case "listenGrpcMultiaddr": - NodeConfig.ListenGRPCMultiaddr = value - case "listenRestMultiaddr": - NodeConfig.ListenRestMultiaddr = value - default: - fmt.Printf("Unsupported configuration key: %s\n", key) - fmt.Println("Supported keys: engine.statsMultiaddr, p2p.listenMultiaddr, listenGrpcMultiaddr, listenRestMultiaddr") + if NodeConfig == nil || ActiveNodeConfigDir == "" { + fmt.Println("No active node config loaded. Run `qclient node config create` first, or pass --config .") + os.Exit(1) + } + + if err := setConfigKey(key, value); err != nil { + fmt.Println(err) os.Exit(1) } - // Save the updated config - if err := config.SaveConfig(NodeConfigToRun, NodeConfig); err != nil { + if err := config.SaveConfig(ActiveNodeConfigDir, NodeConfig); err != nil { fmt.Printf("Failed to save config: %s\n", err) os.Exit(1) } - fmt.Printf("Successfully updated %s to %s in %s\n", key, value, NodeConfigToRun) + fmt.Printf("Successfully updated %s to %s in %s/config.yml\n", key, value, ActiveNodeConfigDir) }, } + +// setConfigKey mutates NodeConfig in place based on the key/value. It +// returns an error rather than exiting so the caller can decide how to +// report it. +func setConfigKey(key, value string) error { + switch key { + case "engine.statsMultiaddr": + NodeConfig.Engine.StatsMultiaddr = value + case "p2p.listenMultiaddr": + NodeConfig.P2P.ListenMultiaddr = value + case "listenGrpcMultiaddr": + NodeConfig.ListenGRPCMultiaddr = value + case "listenRestMultiaddr": + NodeConfig.ListenRestMultiaddr = value + case "logger.path": + ensureLogger() + NodeConfig.Logger.Path = value + case "logger.maxSize": + n, err := strconv.Atoi(value) + if err != nil { + return fmt.Errorf("logger.maxSize must be an integer, got %q", value) + } + ensureLogger() + NodeConfig.Logger.MaxSize = n + case "logger.maxBackups": + n, err := strconv.Atoi(value) + if err != nil { + return fmt.Errorf("logger.maxBackups must be an integer, got %q", value) + } + ensureLogger() + NodeConfig.Logger.MaxBackups = n + case "logger.maxAge": + n, err := strconv.Atoi(value) + if err != nil { + return fmt.Errorf("logger.maxAge must be an integer, got %q", value) + } + ensureLogger() + NodeConfig.Logger.MaxAge = n + case "logger.compress": + b, err := strconv.ParseBool(value) + if err != nil { + return fmt.Errorf("logger.compress must be true or false, got %q", value) + } + ensureLogger() + NodeConfig.Logger.Compress = b + case "logger.logFilters": + filters, err := parseLogFilters(value) + if err != nil { + return err + } + ensureLogger() + NodeConfig.Logger.LogFilters = filters + default: + return fmt.Errorf( + "Unsupported configuration key: %s\n"+ + "Supported keys: engine.statsMultiaddr, p2p.listenMultiaddr, "+ + "listenGrpcMultiaddr, listenRestMultiaddr, logger.path, "+ + "logger.maxSize, logger.maxBackups, logger.maxAge, "+ + "logger.compress, logger.logFilters", + key, + ) + } + return nil +} + +func ensureLogger() { + if NodeConfig.Logger == nil { + NodeConfig.Logger = &config.LogConfig{} + } +} + +// parseLogFilters accepts "component=level,component=level" and returns +// the equivalent map. Whitespace around entries is ignored; an empty +// string clears the filter map. +func parseLogFilters(value string) (map[string]string, error) { + out := map[string]string{} + value = strings.TrimSpace(value) + if value == "" { + return out, nil + } + for _, part := range strings.Split(value, ",") { + part = strings.TrimSpace(part) + if part == "" { + continue + } + kv := strings.SplitN(part, "=", 2) + if len(kv) != 2 { + return nil, fmt.Errorf( + "logger.logFilters entry %q must be component=level", part, + ) + } + k := strings.TrimSpace(kv[0]) + v := strings.TrimSpace(kv[1]) + if k == "" || v == "" { + return nil, fmt.Errorf( + "logger.logFilters entry %q has an empty component or level", part, + ) + } + out[k] = v + } + return out, nil +} diff --git a/client/cmd/node/nodeconfig/switch.go b/client/cmd/node/nodeconfig/switch.go index cc5e252e..23ab418a 100644 --- a/client/cmd/node/nodeconfig/switch.go +++ b/client/cmd/node/nodeconfig/switch.go @@ -11,12 +11,12 @@ import ( var NodeConfigSwitchCmd = &cobra.Command{ Use: "switch [name]", Short: "Switch the config to be run by the node", - Long: fmt.Sprintf(`Switch the configuration to be run by the node by creating a symlink. + Long: `Switch the configuration to be run by the node by creating a symlink. Example: qclient node config switch mynode -This will symlink %s/mynode to %s`, ConfigDirs, NodeConfigToRun), +This will symlink the chosen config directory to the default symlink path.`, Args: cobra.ExactArgs(1), Run: func(cmd *cobra.Command, args []string) { var name string diff --git a/client/cmd/node/service.go b/client/cmd/node/service.go index a897621b..2cfa48b6 100644 --- a/client/cmd/node/service.go +++ b/client/cmd/node/service.go @@ -66,6 +66,39 @@ Examples: }, } +// serviceAliasCommands returns top-level aliases for service subcommands +// (everything except install/update/uninstall) so users can run e.g. +// `qclient node start` instead of `qclient node service start`. +func ServiceAliasCommands() []*cobra.Command { + aliases := []struct { + use string + short string + run func() + }{ + {"start", "Start the node service (alias for 'service start')", startService}, + {"stop", "Stop the node service (alias for 'service stop')", stopService}, + {"restart", "Restart the node service (alias for 'service restart')", restartService}, + {"status", "Check the status of the node service (alias for 'service status')", checkServiceStatus}, + {"enable", "Enable the node service to start on boot (alias for 'service enable')", enableService}, + {"disable", "Disable the node service from starting on boot (alias for 'service disable')", disableService}, + {"reload", "Reload the node service (alias for 'service reload')", reloadService}, + } + + cmds := make([]*cobra.Command, 0, len(aliases)) + for _, a := range aliases { + a := a + cmds = append(cmds, &cobra.Command{ + Use: a.use, + Short: a.short, + Args: cobra.NoArgs, + Run: func(cmd *cobra.Command, args []string) { + a.run() + }, + }) + } + return cmds +} + // installService installs the appropriate service configuration for the current OS func installService() { if err := utils.CheckAndRequestSudo("Installing service requires root privileges"); err != nil { @@ -84,7 +117,7 @@ func installService() { // install systemd if not found installSystemd() } - if err := createSystemdServiceFile(true); err != nil { + if err := CreateSystemdServiceFile(true); err != nil { fmt.Fprintf(os.Stderr, "Error creating systemd service file: %v\n", err) return } @@ -131,7 +164,7 @@ func startService() { } } else { // Linux systemd command - cmd := exec.Command("sudo", "systemctl", "start", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "start", utils.GetNodeServiceName()) if err := cmd.Run(); err != nil { fmt.Fprintf(os.Stderr, "Error starting service: %v\n", err) return @@ -157,7 +190,7 @@ func stopService() { } } else { // Linux systemd command - cmd := exec.Command("sudo", "systemctl", "stop", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "stop", utils.GetNodeServiceName()) if err := cmd.Run(); err != nil { fmt.Fprintf(os.Stderr, "Error stopping service: %v\n", err) return @@ -189,7 +222,7 @@ func restartService() { } } else { // Linux systemd command - cmd := exec.Command("sudo", "systemctl", "restart", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "restart", utils.GetNodeServiceName()) if err := cmd.Run(); err != nil { fmt.Fprintf(os.Stderr, "Error restarting service: %v\n", err) return @@ -251,7 +284,7 @@ func checkServiceStatus() { } } else { // Linux systemd command - cmd := exec.Command("sudo", "systemctl", "status", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "--no-pager", "status", utils.GetNodeServiceName()) cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr if err := cmd.Run(); err != nil { @@ -277,7 +310,7 @@ func enableService() { } } else { // Linux systemd command - cmd := exec.Command("sudo", "systemctl", "enable", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "enable", utils.GetNodeServiceName()) if err := cmd.Run(); err != nil { fmt.Fprintf(os.Stderr, "Error enabling service: %v\n", err) return @@ -304,7 +337,7 @@ func disableService() { } } else { // Linux systemd command - cmd := exec.Command("sudo", "systemctl", "disable", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "disable", utils.GetNodeServiceName()) if err := cmd.Run(); err != nil { fmt.Fprintf(os.Stderr, "Error disabling service: %v\n", err) return @@ -317,7 +350,7 @@ func disableService() { func createService() { // Create systemd service file if OsType == "linux" { - if err := createSystemdServiceFile(true); err != nil { + if err := CreateSystemdServiceFile(true); err != nil { fmt.Fprintf(os.Stderr, "Warning: Failed to create systemd service file: %v\n", err) } } else if OsType == "darwin" { @@ -335,7 +368,7 @@ func removeService() { } if OsType == "linux" { - if err := removeSystemdServiceFile(); err != nil { + if err := RemoveSystemdServiceFile(); err != nil { fmt.Fprintf(os.Stderr, "Warning: Failed to remove systemd service file: %v\n", err) } } else if OsType == "darwin" { @@ -347,8 +380,8 @@ func removeService() { } } -func removeSystemdServiceFile() error { - servicePath := "/etc/systemd/system/" + utils.NodeServiceName + ".service" +func RemoveSystemdServiceFile() error { + servicePath := "/etc/systemd/system/" + utils.GetNodeServiceName() + ".service" if err := os.Remove(servicePath); err != nil { return fmt.Errorf("failed to remove systemd service file: %w", err) } @@ -367,7 +400,7 @@ func removeMacOSService() error { func updateServiceFile() { // Create systemd service file if OsType == "linux" { - if err := createSystemdServiceFile(false); err != nil { + if err := CreateSystemdServiceFile(false); err != nil { fmt.Fprintf(os.Stderr, "Warning: Failed to create systemd service file: %v\n", err) } } else if OsType == "darwin" { @@ -384,13 +417,19 @@ func CreateEnvFile() error { // Create environment file content envContent := `# Quilibrium Node Environment` + envPath := utils.GetNodeEnvFilePath() + // Ensure the install directory exists before writing the env file. + if err := utils.ValidateAndCreateDir(filepath.Dir(envPath), NodeUser); err != nil { + return fmt.Errorf("failed to create install directory: %w", err) + } + // Write environment file - if err := os.WriteFile(utils.NodeEnvPath, []byte(envContent), 0640); err != nil { + if err := os.WriteFile(envPath, []byte(envContent), 0640); err != nil { return fmt.Errorf("failed to create environment file: %w", err) } // Set ownership of environment file - chownCmd := utils.ChownPath(utils.NodeEnvPath, NodeUser, false) + chownCmd := utils.ChownPath(envPath, NodeUser, false) if chownCmd != nil { return fmt.Errorf("failed to set environment file ownership: %w", chownCmd) } @@ -398,8 +437,11 @@ func CreateEnvFile() error { return nil } -// createSystemdServiceFile creates the systemd service file with environment file support -func createSystemdServiceFile(createEnvFile bool) error { +// CreateSystemdServiceFile creates the systemd service file with environment file support. +// The unit file is written under the user-configured service name, while the +// ExecStart/ExecReload commands continue to invoke the fixed binary name so +// that the /usr/local/bin/quilibrium-node symlink does not need to change. +func CreateSystemdServiceFile(createEnvFile bool) error { if !utils.CheckForSystemd() { installSystemd() } @@ -409,13 +451,17 @@ func createSystemdServiceFile(createEnvFile bool) error { return fmt.Errorf("failed to get sudo privileges: %w", err) } - envPath := filepath.Join(utils.RootQuilibriumPath, "quilibrium.env") + envPath := utils.GetNodeEnvFilePath() if createEnvFile { if err := CreateEnvFile(); err != nil { return fmt.Errorf("failed to create environment file: %w", err) } } + serviceName := utils.GetNodeServiceName() + binaryPath := utils.GetNodeSymlinkPath() + configPath := filepath.Join(utils.GetNodeConfigsDir(), "default") + // Create systemd service file content serviceContent := fmt.Sprintf(`[Unit] Description=Quilibrium Node Service @@ -424,26 +470,26 @@ Wants=network-online.target [Service] Type=simple -User=quilibrium -EnvironmentFile=/var/quilibrium/quilibrium.env -ExecStart=/usr/local/bin/` + utils.NodeServiceName + ` --config ` + ConfigDirs + `/default +User=%s +EnvironmentFile=%s +ExecStart=%s --config %s Restart=always RestartSec=10 ExecStop=/bin/kill -s SIGINT $MAINPID -ExecReload=/bin/kill -s SIGINT $MAINPID && /usr/local/bin/` + utils.NodeServiceName + ` --config ` + ConfigDirs + `/default +ExecReload=/bin/kill -s SIGINT $MAINPID && %s --config %s KillSignal=SIGINT -RestartSignal=SIGINT +RestartKillSignal=SIGINT FinalKillSignal=SIGKILL -KillSignal=SIGKILL +SendSIGKILL=yes TimeoutStopSec=240 LimitNOFILE=65535 [Install] WantedBy=multi-user.target -`) +`, NodeUser.Username, envPath, binaryPath, configPath, binaryPath, configPath) // Write service file - servicePath := "/etc/systemd/system/quilibrium-node.service" + servicePath := "/etc/systemd/system/" + serviceName + ".service" if err := utils.WriteFileAuto(servicePath, serviceContent); err != nil { return fmt.Errorf("failed to create service file: %w", err) } @@ -474,9 +520,9 @@ func installMacOSService() { {{.Label}} ProgramArguments - /usr/local/bin/quilibrium-node + {{.BinaryPath}} --config - /opt/quilibrium/config/ + {{.ConfigPath}} EnvironmentVariables @@ -509,16 +555,41 @@ func installMacOSService() { ` // Prepare template data + // Resolve the active node config's logger.path for launchd's + // StandardOutPath/StandardErrorPath. If the active config has no + // logger block, fall back to the per-config default directory so + // launchd still has a stable place to send stdout/stderr; the node + // itself will log to stdout in that case, which launchd will capture. + logPath := utils.DefaultNodeLogDirForConfig(filepath.Join( + utils.GetNodeConfigsDir(), utils.DefaultNodeConfigName, + )) + if resolved, err := utils.ResolveActiveNodeLog(); err == nil { + if resolved.FileBased { + logPath = resolved.LogDir + } else if resolved.ConfigDir != "" { + logPath = utils.DefaultNodeLogDirForConfig(resolved.ConfigDir) + } + } else if dir, err := utils.GetDefaultNodeConfigDir(); err == nil { + logPath = utils.DefaultNodeLogDirForConfig(dir) + } + if err := os.MkdirAll(logPath, 0o755); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not create log dir %s: %v\n", logPath, err) + } + data := struct { Label string DataPath string ServiceName string LogPath string + BinaryPath string + ConfigPath string }{ Label: fmt.Sprintf("com.quilibrium.node"), - DataPath: utils.NodeDataPath, + DataPath: utils.GetNodeBinaryDir(), ServiceName: "node", - LogPath: utils.LogPath, + LogPath: logPath, + BinaryPath: utils.GetNodeSymlinkPath(), + ConfigPath: filepath.Join(utils.GetNodeConfigsDir(), "default"), } // Parse and execute template diff --git a/client/cmd/node/shared.go b/client/cmd/node/shared.go index 035846ce..7e987bf1 100644 --- a/client/cmd/node/shared.go +++ b/client/cmd/node/shared.go @@ -29,53 +29,46 @@ func determineVersion(args []string) string { // setOwnership sets the ownership of directories to the node user func setOwnership() { - + binDir := utils.GetNodeBinaryDir() // Change ownership of installation directory - err := utils.ChownPath(utils.NodeDataPath, NodeUser, true) + err := utils.ChownPath(binDir, NodeUser, true) if err != nil { - fmt.Fprintf(os.Stderr, "Warning: Failed to change ownership of %s: %v\n", utils.NodeDataPath, err) + fmt.Fprintf(os.Stderr, "Warning: Failed to change ownership of %s: %v\n", binDir, err) } } -// setupLogRotation creates a logrotate configuration file for the Quilibrium node -func setupLogRotation() error { - // Check if we need sudo privileges for creating logrotate config - if err := utils.CheckAndRequestSudo("Creating logrotate configuration requires root privileges"); err != nil { +// ensureNodeLogDirs makes sure the active node config has a logger +// block and that its log directory exists with the right ownership so +// the node can start writing to it immediately. Rotation itself is +// handled in-process by the node's lumberjack-based logger — we don't +// install a logrotate rule. +func ensureNodeLogDirs() error { + if err := utils.CheckAndRequestSudo("Preparing node log directory requires root privileges"); err != nil { return fmt.Errorf("failed to get sudo privileges: %w", err) } - // Create logrotate configuration - configContent := fmt.Sprintf(`%s/*.log { - daily - rotate 7 - compress - delaycompress - missingok - notifempty - create 0640 %s %s - postrotate - systemctl reload quilibrium-node >/dev/null 2>&1 || true - endscript -}`, utils.LogPath, NodeUser.Username, NodeUser.Username) - - // Write the configuration file - configPath := "/etc/logrotate.d/" + utils.NodeServiceName - if err := utils.WriteFile(configPath, configContent); err != nil { - return fmt.Errorf("failed to create logrotate configuration: %w", err) + activeDir, err := utils.GetDefaultNodeConfigDir() + if err != nil { + return fmt.Errorf("resolving active node config dir: %w", err) } - - // Create log directory with proper permissions - if err := utils.ValidateAndCreateDir(utils.LogPath, NodeUser); err != nil { - return fmt.Errorf("failed to create log directory: %w", err) + activeLogDir, modified, err := utils.EnsureNodeConfigLogger(activeDir) + if err != nil { + return fmt.Errorf("ensuring logger block on active config: %w", err) + } + if modified { + fmt.Fprintf(os.Stdout, + "Populated logger block in %s/config.yml (logger.path=%s)\n", + activeDir, activeLogDir, + ) } - // Set ownership of log directory - err := utils.ChownPath(utils.LogPath, NodeUser, true) - if err != nil { - return fmt.Errorf("failed to set log directory ownership: %w", err) + if err := utils.ValidateAndCreateDir(activeLogDir, NodeUser); err != nil { + return fmt.Errorf("failed to create log directory %s: %w", activeLogDir, err) + } + if err := utils.ChownPath(activeLogDir, NodeUser, true); err != nil { + return fmt.Errorf("failed to set ownership on %s: %w", activeLogDir, err) } - fmt.Fprintf(os.Stdout, "Created log rotation configuration at %s\n", configPath) return nil } @@ -86,29 +79,36 @@ func finishInstallation(version string) { normalizedBinaryName := "node-" + version + "-" + OsType + "-" + Arch // Finish installation - nodeBinaryPath := filepath.Join(utils.NodeDataPath, version, normalizedBinaryName) + nodeBinaryPath := filepath.Join(utils.GetNodeBinaryDir(), version, normalizedBinaryName) fmt.Printf("Making binary executable: %s\n", nodeBinaryPath) // Make the binary executable if err := utils.ChmodPath(nodeBinaryPath, 0755, "executable"); err != nil { fmt.Fprintf(os.Stderr, "Warning: Failed to make binary executable: %v\n", err) } + symlinkPath := utils.GetNodeSymlinkPath() // Check if we need sudo privileges for creating symlink in system directory - if strings.HasPrefix(utils.DefaultNodeSymlinkPath, "/usr/") || strings.HasPrefix(utils.DefaultNodeSymlinkPath, "/bin/") || strings.HasPrefix(utils.DefaultNodeSymlinkPath, "/sbin/") { - if err := utils.CheckAndRequestSudo(fmt.Sprintf("Creating symlink at %s requires root privileges", utils.DefaultNodeSymlinkPath)); err != nil { + if strings.HasPrefix(symlinkPath, "/usr/") || strings.HasPrefix(symlinkPath, "/bin/") || strings.HasPrefix(symlinkPath, "/sbin/") { + if err := utils.CheckAndRequestSudo(fmt.Sprintf("Creating symlink at %s requires root privileges", symlinkPath)); err != nil { fmt.Fprintf(os.Stderr, "Warning: Failed to get sudo privileges: %v\n", err) return } } + // Ensure the symlink directory exists for non-standard locations. + if err := utils.ValidateAndCreateDir(utils.GetNodeSymlinkDir(), NodeUser); err != nil { + fmt.Fprintf(os.Stderr, "Warning: Failed to create symlink directory %s: %v\n", utils.GetNodeSymlinkDir(), err) + } + // Create symlink using the utils package - if err := utils.CreateSymlink(nodeBinaryPath, utils.DefaultNodeSymlinkPath); err != nil { + if err := utils.CreateSymlink(nodeBinaryPath, symlinkPath); err != nil { fmt.Fprintf(os.Stderr, "Error creating symlink: %v\n", err) } - // Set up log rotation - if err := setupLogRotation(); err != nil { - fmt.Fprintf(os.Stderr, "Warning: Failed to set up log rotation: %v\n", err) + // Ensure the node's log directory exists (rotation is handled by + // the node itself, so no logrotate rule is installed). + if err := ensureNodeLogDirs(); err != nil { + fmt.Fprintf(os.Stderr, "Warning: Failed to prepare log directory: %v\n", err) } // Print success message @@ -118,11 +118,15 @@ func finishInstallation(version string) { // printSuccessMessage prints a success message after installation func printSuccessMessage(version string) { fmt.Fprintf(os.Stdout, "\nSuccessfully installed Quilibrium node %s\n", version) - fmt.Fprintf(os.Stdout, "Binary download directory: %s\n", filepath.Join(utils.NodeDataPath, version)) - fmt.Fprintf(os.Stdout, "Binary symlinked to %s\n", utils.DefaultNodeSymlinkPath) - fmt.Fprintf(os.Stdout, "Log directory: %s\n", utils.LogPath) - fmt.Fprintf(os.Stdout, "Environment file: /etc/default/quilibrium-node\n") - fmt.Fprintf(os.Stdout, "Service file: /etc/systemd/system/quilibrium-node.service\n") + fmt.Fprintf(os.Stdout, "Binary download directory: %s\n", filepath.Join(utils.GetNodeBinaryDir(), version)) + fmt.Fprintf(os.Stdout, "Binary symlinked to %s\n", utils.GetNodeSymlinkPath()) + if resolved, err := utils.ResolveActiveNodeLog(); err == nil && resolved.FileBased { + fmt.Fprintf(os.Stdout, "Log directory: %s\n", resolved.LogDir) + } else { + fmt.Fprintf(os.Stdout, "Log directory: (no logger block in active node config; using system log)\n") + } + fmt.Fprintf(os.Stdout, "Environment file: %s\n", utils.GetNodeEnvFilePath()) + fmt.Fprintln(os.Stdout, "Service file: /etc/systemd/system/"+utils.GetNodeServiceName()+".service") fmt.Fprintf(os.Stdout, "\nConfiguration:\n") fmt.Fprintf(os.Stdout, " To create a new configuration:\n") @@ -131,7 +135,7 @@ func printSuccessMessage(version string) { fmt.Fprintf(os.Stdout, "\n To use an existing configuration:\n") fmt.Fprintf(os.Stdout, " qclient node config import [name] /path/to/your/existing/config --default\n") fmt.Fprintf(os.Stdout, " # Or modify the service file to point to your existing config:\n") - fmt.Fprintf(os.Stdout, " sudo nano /etc/systemd/system/"+utils.NodeServiceName+".service\n") + fmt.Fprintf(os.Stdout, " sudo nano /etc/systemd/system/"+utils.GetNodeServiceName()+".service\n") fmt.Fprintf(os.Stdout, " # Then reload systemd:\n") fmt.Fprintf(os.Stdout, " sudo systemctl daemon-reload\n") @@ -143,7 +147,7 @@ func printSuccessMessage(version string) { fmt.Fprintf(os.Stdout, "\nTo manually start the node (must create a config first), you can run:\n") fmt.Fprintf(os.Stdout, " "+utils.NodeServiceName+" --config "+ConfigDirs+"/myconfig/\n") fmt.Fprintf(os.Stdout, " # Or use systemd service using the default config:\n") - fmt.Fprintf(os.Stdout, " sudo systemctl start "+utils.NodeServiceName+"\n") + fmt.Fprintf(os.Stdout, " sudo systemctl start "+utils.GetNodeServiceName()+"\n") fmt.Fprintf(os.Stdout, "\nFor more options, run:\n") fmt.Fprintf(os.Stdout, " "+utils.NodeServiceName+" --help\n") diff --git a/client/cmd/node/uninstall.go b/client/cmd/node/uninstall.go index c07956b8..dcdae648 100644 --- a/client/cmd/node/uninstall.go +++ b/client/cmd/node/uninstall.go @@ -2,15 +2,24 @@ package node import ( "bufio" + "errors" "fmt" "os" "os/exec" "strings" + "syscall" "github.com/spf13/cobra" "source.quilibrium.com/quilibrium/monorepo/client/utils" ) +// isDirNotEmpty reports whether err represents a non-empty-directory +// error from os.Remove. Used so uninstall can silently skip removing +// a state dir that still has user files in it. +func isDirNotEmpty(err error) bool { + return errors.Is(err, syscall.ENOTEMPTY) || errors.Is(err, syscall.EEXIST) +} + var ( Force bool ) @@ -27,7 +36,7 @@ The following will be removed: - All node binaries and signatures - Node symlink - Log files - - Logrotate configuration + - Any leftover legacy logrotate configuration from older installs The following will NOT be removed: - Configuration files (~/.quilibrium/configs/) @@ -72,28 +81,64 @@ func uninstallNode() { fmt.Println("Removing node service...") removeNodeService() + binDir := utils.GetNodeBinaryDir() + symlinkPath := utils.GetNodeSymlinkPath() + logDirs := utils.ResolveAllNodeLogDirs() + if resolved, err := utils.ResolveActiveNodeLog(); err == nil && resolved.FileBased { + present := false + for _, d := range logDirs { + if d == resolved.LogDir { + present = true + break + } + } + if !present { + logDirs = append(logDirs, resolved.LogDir) + } + } + envPath := utils.GetNodeEnvFilePath() + // 3. Remove all binaries fmt.Println("Removing node binaries...") - if err := os.RemoveAll(utils.NodeDataPath); err != nil { - fmt.Fprintf(os.Stderr, "Warning: could not remove binaries at %s: %v\n", utils.NodeDataPath, err) + if err := os.RemoveAll(binDir); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not remove binaries at %s: %v\n", binDir, err) } // 4. Remove symlink fmt.Println("Removing node symlink...") - if err := os.Remove(utils.DefaultNodeSymlinkPath); err != nil && !os.IsNotExist(err) { - fmt.Fprintf(os.Stderr, "Warning: could not remove symlink at %s: %v\n", utils.DefaultNodeSymlinkPath, err) + if err := os.Remove(symlinkPath); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove symlink at %s: %v\n", symlinkPath, err) } // 5. Remove logs fmt.Println("Removing log files...") - if err := os.RemoveAll(utils.LogPath); err != nil && !os.IsNotExist(err) { - fmt.Fprintf(os.Stderr, "Warning: could not remove logs at %s: %v\n", utils.LogPath, err) + for _, logDir := range logDirs { + if err := os.RemoveAll(logDir); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove logs at %s: %v\n", logDir, err) + } + } + + // 6. Best-effort removal of any legacy logrotate config left over + // from previous qclient versions. Current installs don't create + // one; the node rotates its own logs. + legacyLogrotate := "/etc/logrotate.d/" + utils.NodeServiceName + if err := os.Remove(legacyLogrotate); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove legacy logrotate config at %s: %v\n", legacyLogrotate, err) } - // 6. Remove logrotate config - logrotateConfig := "/etc/logrotate.d/" + utils.NodeServiceName - if err := os.Remove(logrotateConfig); err != nil && !os.IsNotExist(err) { - fmt.Fprintf(os.Stderr, "Warning: could not remove logrotate config at %s: %v\n", logrotateConfig, err) + // 7. Remove environment file, and the state dir itself if empty. + fmt.Println("Removing environment file...") + if err := os.Remove(envPath); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove environment file at %s: %v\n", envPath, err) + } + stateDir := utils.GetNodeStateDir() + if err := os.Remove(stateDir); err != nil && !os.IsNotExist(err) { + // Non-empty or permission error: leave it alone, but only + // report unexpected errors (ENOTEMPTY is expected when the + // user has other state files in there). + if !isDirNotEmpty(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove state directory at %s: %v\n", stateDir, err) + } } fmt.Println() @@ -112,7 +157,7 @@ func stopNodeService() { fmt.Fprintf(os.Stderr, " Note: could not stop service (may not be running): %v\n", err) } } else { - cmd := exec.Command("sudo", "systemctl", "stop", utils.NodeServiceName) + cmd := exec.Command("sudo", "systemctl", "stop", utils.GetNodeServiceName()) if err := cmd.Run(); err != nil { fmt.Fprintf(os.Stderr, " Note: could not stop service (may not be running): %v\n", err) } @@ -121,12 +166,13 @@ func stopNodeService() { func removeNodeService() { if OsType == "linux" { + serviceName := utils.GetNodeServiceName() // Disable service first - disableCmd := exec.Command("sudo", "systemctl", "disable", utils.NodeServiceName) + disableCmd := exec.Command("sudo", "systemctl", "disable", serviceName) disableCmd.Run() // ignore error // Remove service file - servicePath := "/etc/systemd/system/" + utils.NodeServiceName + ".service" + servicePath := "/etc/systemd/system/" + serviceName + ".service" if err := os.Remove(servicePath); err != nil && !os.IsNotExist(err) { fmt.Fprintf(os.Stderr, " Warning: could not remove service file: %v\n", err) } diff --git a/client/cmd/node/update.go b/client/cmd/node/update.go index d8941de2..8fe08338 100644 --- a/client/cmd/node/update.go +++ b/client/cmd/node/update.go @@ -76,7 +76,7 @@ func restartNode() { // updateNode handles the node update process func updateNode(version string) { // Check if we need sudo privileges - if err := utils.CheckAndRequestSudo(fmt.Sprintf("Updating node at %s requires root privileges", utils.NodeDataPath)); err != nil { + if err := utils.CheckAndRequestSudo(fmt.Sprintf("Updating node at %s requires root privileges", utils.GetNodeBinaryDir())); err != nil { fmt.Fprintf(os.Stderr, "Error: %v\n", err) return } diff --git a/client/cmd/quiet.go b/client/cmd/quiet.go new file mode 100644 index 00000000..6190e73a --- /dev/null +++ b/client/cmd/quiet.go @@ -0,0 +1,52 @@ +package cmd + +import ( + "fmt" + "os" + "strings" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var QuietCmd = &cobra.Command{ + Use: "quiet [enable|disable]", + Short: "Hide informational output when signature verification succeeds", + Long: `When quiet mode is enabled, qclient does not print progress lines for a successful +signature check, and does not print the banner when signature verification is bypassed. +Verification errors and prompts are always shown. + +With no argument, the current setting is toggled.`, + Run: func(_ *cobra.Command, args []string) { + cfg, err := utils.LoadClientConfig() + if err != nil { + fmt.Printf("Error loading config: %v\n", err) + os.Exit(1) + } + + if len(args) > 0 { + switch strings.ToLower(args[0]) { + case "enable": + cfg.Quiet = true + case "disable": + cfg.Quiet = false + default: + fmt.Printf("Error: Invalid value '%s'. Please use 'enable' or 'disable'.\n", args[0]) + os.Exit(1) + } + } else { + cfg.Quiet = !cfg.Quiet + } + + if err := utils.SaveClientConfig(cfg); err != nil { + fmt.Printf("Error saving config: %v\n", err) + os.Exit(1) + } + + status := "disabled" + if cfg.Quiet { + status = "enabled" + } + fmt.Printf("Quiet mode has been %s and will apply to future commands.\n", status) + }, +} diff --git a/client/cmd/root.go b/client/cmd/root.go index 8bc16b04..7e587b2f 100644 --- a/client/cmd/root.go +++ b/client/cmd/root.go @@ -81,11 +81,13 @@ It provides commands for installing, updating, and managing Quilibrium nodes.`, checksum := sha3.Sum256(b) - // First check var data path for signatures - varDataPath := filepath.Join(utils.ClientDataPath, config.GetVersionString()) - digestPath := filepath.Join(varDataPath, StandardizedQClientFileName+".dgst") + // First check the qclient binary directory for signatures. + versionDataPath := filepath.Join(utils.GetQClientBinaryDir(), config.GetVersionString()) + digestPath := filepath.Join(versionDataPath, StandardizedQClientFileName+".dgst") - fmt.Printf("Checking signature for %s\n", digestPath) + if !clientConfig.Quiet { + fmt.Printf("Checking signature for %s\n", digestPath) + } // Try to read digest from var data path first digest, err := os.ReadFile(digestPath) @@ -140,8 +142,11 @@ It provides commands for installing, updating, and managing Quilibrium nodes.`, count := 0 for i := 1; i <= len(config.Signatories); i++ { - // Try var data path first for signature files - signatureFile := filepath.Join(varDataPath, fmt.Sprintf("%s.dgst.sig.%d", filepath.Base(ex), i)) + // Try var data path first for signature files. Use the + // standardized release filename (qclient---) + // rather than the running executable's basename, since + // signatures on disk are named after the release artifact. + signatureFile := filepath.Join(versionDataPath, fmt.Sprintf("%s.dgst.sig.%d", StandardizedQClientFileName, i)) sig, err := os.ReadFile(signatureFile) if err != nil { // Fall back to checking next to executable @@ -165,12 +170,16 @@ It provides commands for installing, updating, and managing Quilibrium nodes.`, os.Exit(1) } - fmt.Println("Signature check passed") + if !clientConfig.Quiet { + fmt.Println("Signature check passed") + } } } else { - fmt.Println("Signature check bypassed, be sure you know what you're doing") - fmt.Println("----------------------------------------------------------") - fmt.Println("") + if !clientConfig.Quiet { + fmt.Println("Signature check bypassed, be sure you know what you're doing") + fmt.Println("----------------------------------------------------------") + fmt.Println("") + } } }, PersistentPostRun: func(cmd *cobra.Command, args []string) { @@ -250,5 +259,8 @@ func init() { rootCmd.AddCommand(CrossMintCmd) rootCmd.AddCommand(DownloadSignaturesCmd) rootCmd.AddCommand(LinkCmd) + rootCmd.AddCommand(UninstallCmd) rootCmd.AddCommand(VersionCmd) + rootCmd.AddCommand(QuietCmd) + rootCmd.AddCommand(DevCmd) } diff --git a/client/cmd/token/token.go b/client/cmd/token/token.go index 94ddf0c7..eff6e481 100644 --- a/client/cmd/token/token.go +++ b/client/cmd/token/token.go @@ -35,24 +35,27 @@ var TokenCmd = &cobra.Command{ fmt.Println("Loading node config...") if ConfigDirectory != "" { NodeConfig, err = utils.LoadNodeConfig(ConfigDirectory) + if err != nil { + fmt.Printf("error loading node config: %s\n", err) + os.Exit(1) + } } else { NodeConfig, err = utils.LoadDefaultNodeConfig() - } - - if err != nil { - if err.Error() == utils.ErrConfigNotFoundErrorMessage { - fmt.Println("Config not found, creating default configuration...") - nodeConfig, err := utils.CreateDefaultNodeConfig( - utils.DefaultNodeConfigName, - ) - if err != nil { - fmt.Printf("error creating default node config: %s\n", err) + if err != nil { + if err.Error() == utils.ErrConfigNotFoundErrorMessage { + fmt.Println("Config not found, creating default configuration...") + nodeConfig, err := utils.CreateDefaultNodeConfig( + utils.DefaultNodeConfigName, + ) + if err != nil { + fmt.Printf("error creating default node config: %s\n", err) + os.Exit(1) + } + NodeConfig = nodeConfig + } else { + fmt.Printf("error loading node config: %s\n", err) os.Exit(1) } - NodeConfig = nodeConfig - } else { - fmt.Printf("error loading node config: %s\n", err) - os.Exit(1) } } diff --git a/client/cmd/uninstall.go b/client/cmd/uninstall.go new file mode 100644 index 00000000..c2615b82 --- /dev/null +++ b/client/cmd/uninstall.go @@ -0,0 +1,145 @@ +package cmd + +import ( + "bufio" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + + "github.com/spf13/cobra" + "source.quilibrium.com/quilibrium/monorepo/client/utils" +) + +var uninstallForce bool + +// qclientSymlinkPath is the symlink created by `qclient link`. +const qclientSymlinkPath = "/usr/local/bin/qclient" + +var UninstallCmd = &cobra.Command{ + Use: "uninstall", + Short: "Uninstall qclient (binaries, symlink, client config)", + Long: `Uninstalls the qclient binary tree, the /usr/local/bin/qclient symlink, +and the qclient client config file. Node configs under ~/.quilibrium/configs/ +are preserved. + +This command will prompt for confirmation unless the --force flag is used. + +The following will be removed: + - qclient install dir (versioned binaries + signatures) + - Legacy qclient binary dir (/var/quilibrium/bin/qclient), if present + - /usr/local/bin/qclient symlink + - qclient client config file (~/.quilibrium/qclient.yml) + - The currently running qclient executable (scheduled after exit) + +The following will NOT be removed: + - Node configs (~/.quilibrium/configs/) + - Anything installed by 'qclient node install' (use 'qclient node uninstall') + +Examples: + sudo qclient uninstall + sudo qclient uninstall --force`, + // Skip the signature check PersistentPreRun for this command so + // users can uninstall even when signatures are missing/stale. + PersistentPreRun: func(cmd *cobra.Command, args []string) {}, + Run: func(cmd *cobra.Command, args []string) { + if !utils.IsSudo() { + fmt.Println("This command must be run with sudo: sudo qclient uninstall") + os.Exit(1) + } + + if !uninstallForce { + fmt.Println("This will remove qclient binaries, the /usr/local/bin/qclient symlink,") + fmt.Println("and the qclient client config file.") + fmt.Println("Node configs in ~/.quilibrium/configs/ will NOT be removed.") + fmt.Print("\nAre you sure you want to continue? [y/N]: ") + + reader := bufio.NewReader(os.Stdin) + response, _ := reader.ReadString('\n') + response = strings.TrimSpace(strings.ToLower(response)) + if response != "y" && response != "yes" { + fmt.Println("Uninstall cancelled.") + return + } + } + + uninstallQClient() + }, +} + +func uninstallQClient() { + binDir := utils.GetQClientBinaryDir() + + fmt.Println("Removing qclient binaries...") + if err := os.RemoveAll(binDir); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not remove qclient binaries at %s: %v\n", binDir, err) + } + + // Best-effort: remove legacy pre-FHS-split location too. + if _, err := os.Stat(utils.LegacyQClientBinaryDir); err == nil { + fmt.Printf("Removing legacy qclient binaries at %s...\n", utils.LegacyQClientBinaryDir) + if err := os.RemoveAll(utils.LegacyQClientBinaryDir); err != nil { + fmt.Fprintf(os.Stderr, "Warning: could not remove legacy qclient binaries: %v\n", err) + } + } + + fmt.Println("Removing qclient symlink...") + if err := os.Remove(qclientSymlinkPath); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove symlink at %s: %v\n", qclientSymlinkPath, err) + } + + configPath := utils.GetConfigPath() + fmt.Println("Removing qclient client config...") + if err := os.Remove(configPath); err != nil && !os.IsNotExist(err) { + fmt.Fprintf(os.Stderr, "Warning: could not remove client config at %s: %v\n", configPath, err) + } + + // Schedule self-deletion of the running executable after we exit. + // The install dir RemoveAll above may have already removed it on + // Linux (unlink-while-running is allowed), but on macOS and when + // the binary was copied/linked elsewhere we still need to clean it + // up. Best-effort — ignore errors. + scheduleSelfDelete() + + fmt.Println() + fmt.Println("qclient uninstalled successfully.") + fmt.Println() + fmt.Println("Your node configs have been preserved at:") + if cu, err := utils.GetCurrentSudoUser(); err == nil { + fmt.Printf(" %s\n", filepath.Join(cu.HomeDir, utils.DefaultNodeConfigsSubdir)) + } else { + fmt.Println(" ~/.quilibrium/configs/") + } +} + +// scheduleSelfDelete forks a detached shell that waits briefly for this +// process to exit, then removes the currently running executable. This +// is the cross-platform way to let a binary "delete itself": on Linux +// unlink-while-running works, but on macOS the file is still on disk +// after RemoveAll until the last reference is dropped. +func scheduleSelfDelete() { + ex, err := os.Executable() + if err != nil { + return + } + resolved, err := filepath.EvalSymlinks(ex) + if err == nil { + ex = resolved + } + // Detached shell: sleep briefly so our parent process can exit, + // then rm. We intentionally don't wait on this command. + sh := fmt.Sprintf("sleep 1; rm -f %q", ex) + cmd := exec.Command("/bin/sh", "-c", sh) + cmd.Stdin = nil + cmd.Stdout = nil + cmd.Stderr = nil + _ = cmd.Start() + if cmd.Process != nil { + _ = cmd.Process.Release() + } +} + +func init() { + UninstallCmd.Flags().BoolVar(&uninstallForce, "force", false, "Skip confirmation prompt") +} diff --git a/client/cmd/update.go b/client/cmd/update.go index 02fa5f8d..9fa8f249 100644 --- a/client/cmd/update.go +++ b/client/cmd/update.go @@ -87,21 +87,25 @@ func updateClient(version string) { return } + qclientBinDir := utils.GetQClientBinaryDir() + // Check if we need sudo privileges - if err := utils.CheckAndRequestSudo(fmt.Sprintf("Updating client at %s requires root privileges", utils.ClientDataPath)); err != nil { + if err := utils.CheckAndRequestSudo(fmt.Sprintf("Updating client at %s requires root privileges", qclientBinDir)); err != nil { fmt.Fprintf(os.Stderr, "Error: %v\n", err) return } + warnLegacyQClientLayout() + // Create version-specific installation directory - versionDir := filepath.Join(utils.ClientDataPath, version) + versionDir := filepath.Join(qclientBinDir, version) if err := os.MkdirAll(versionDir, 0755); err != nil { fmt.Fprintf(os.Stderr, "Error creating installation directory: %v\n", err) return } - // Create data directory - versionDataDir := filepath.Join(utils.ClientDataPath, version) + // Create data directory (same as versionDir today). + versionDataDir := versionDir if err := os.MkdirAll(versionDataDir, 0755); err != nil { fmt.Fprintf(os.Stderr, "Error creating data directory: %v\n", err) return @@ -136,7 +140,7 @@ func updateClient(version string) { func finishInstallation(version string) { // Construct executable path - execPath := filepath.Join(utils.ClientDataPath, version, "qclient-"+version+"-"+osType+"-"+arch) + execPath := filepath.Join(utils.GetQClientBinaryDir(), version, "qclient-"+version+"-"+osType+"-"+arch) // Make the binary executable if err := os.Chmod(execPath, 0755); err != nil { @@ -165,3 +169,57 @@ func finishInstallation(version string) { fmt.Fprintf(os.Stdout, "Executable: %s\n", execPath) fmt.Fprintf(os.Stdout, "Symlink: %s\n", symlinkPath) } + +// warnLegacyQClientLayout emits a one-shot warning when the qclient +// update would land in (or leave behind) the pre-FHS-split +// /var/quilibrium/bin/qclient layout. No files are moved; the user is +// told how to opt in to the new defaults. +func warnLegacyQClientLayout() { + cfg, err := utils.LoadClientConfig() + if err != nil { + return + } + + resolved := utils.GetQClientBinaryDir() + pinned := filepath.Clean(cfg.DataDir) == utils.LegacyQClientBinaryDir || + filepath.Clean(resolved) == utils.LegacyQClientBinaryDir + legacyTreeExists := utils.FileExists(utils.LegacyQClientBinaryDir) + + if !pinned && !legacyTreeExists { + return + } + + defaultInstall := utils.DefaultQClientInstallDir() + defaultBin := filepath.Join(defaultInstall, "bin", string(utils.ReleaseTypeQClient)) + + fmt.Fprintln(os.Stderr) + fmt.Fprintln(os.Stderr, + "Notice: the default qclient install layout has moved off "+ + utils.LegacyQClientBinaryDir+".") + fmt.Fprintf(os.Stderr, + " New default: %s.\n", defaultBin, + ) + if pinned { + fmt.Fprintf(os.Stderr, + " Your qclient config currently pins the qclient binary dir to %s;\n"+ + " this update will keep writing there.\n", + resolved, + ) + fmt.Fprintln(os.Stderr, + " To adopt the new default, clear 'dataDir' and "+ + "'qclientInstallDir' from your qclient-config.yaml and "+ + "re-run this update.") + } else if legacyTreeExists { + fmt.Fprintf(os.Stderr, + " A legacy qclient tree was detected under %s but this update "+ + "will use the new default (%s).\n", + utils.LegacyQClientBinaryDir, resolved, + ) + fmt.Fprintf(os.Stderr, + " Files under %s are NOT moved automatically; remove them "+ + "manually once you've verified the new install.\n", + utils.LegacyQClientBinaryDir, + ) + } + fmt.Fprintln(os.Stderr) +} diff --git a/client/cmd/version.go b/client/cmd/version.go index ccc10220..a4500294 100644 --- a/client/cmd/version.go +++ b/client/cmd/version.go @@ -5,6 +5,7 @@ import ( "os" "path/filepath" "regexp" + "strings" "github.com/spf13/cobra" "source.quilibrium.com/quilibrium/monorepo/client/utils" @@ -18,9 +19,14 @@ var ( ) func versionWithPatch(base string) string { + // If the base already contains a 4th (patch) component like "2.1.0.22", + // return it as-is. Otherwise, append the compiled-in patch number. + if strings.Count(base, ".") >= 3 { + return base + } patch := config.GetPatchNumber() if patch != 0x00 { - return fmt.Sprintf("%s-p%d", base, patch) + return fmt.Sprintf("%s.%d", base, patch) } return base } @@ -45,7 +51,7 @@ func GetVersionInfo(calcChecksum bool) (VersionInfo, error) { // Extract version from executable name (e.g. qclient-2.0.3-linux-amd) baseName := filepath.Base(executable) - versionPattern := regexp.MustCompile(`qclient-([0-9]+\.[0-9]+\.[0-9]+)`) + versionPattern := regexp.MustCompile(`qclient-([0-9]+\.[0-9]+\.[0-9]+(?:\.[0-9]+)?)`) matches := versionPattern.FindStringSubmatch(baseName) version := DefaultVersion diff --git a/client/go.mod b/client/go.mod index c0b0f3aa..76ab7a79 100644 --- a/client/go.mod +++ b/client/go.mod @@ -72,6 +72,24 @@ require ( require ( filippo.io/edwards25519 v1.0.0-rc.1 // indirect + github.com/aws/aws-sdk-go-v2 v1.41.6 // indirect + github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect + github.com/aws/aws-sdk-go-v2/config v1.32.16 // indirect + github.com/aws/aws-sdk-go-v2/credentials v1.19.15 // indirect + github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect + github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect + github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 // indirect + github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22 // indirect + github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1 // indirect + github.com/aws/aws-sdk-go-v2/service/signin v1.0.10 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.30.16 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect + github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect + github.com/aws/smithy-go v1.25.0 // indirect github.com/benbjohnson/clock v1.3.5 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/btcsuite/btcd v0.21.0-beta.0.20201114000516-e9c7a5ac6401 // indirect @@ -209,7 +227,8 @@ require ( golang.org/x/mod v0.25.0 // indirect golang.org/x/net v0.41.0 // indirect golang.org/x/sync v0.19.0 // indirect - golang.org/x/sys v0.42.0 // indirect + golang.org/x/sys v0.43.0 // indirect + golang.org/x/term v0.42.0 // indirect golang.org/x/text v0.26.0 // indirect golang.org/x/time v0.12.0 // indirect golang.org/x/tools v0.34.0 // indirect diff --git a/client/go.sum b/client/go.sum index afe80bb0..d401a2c8 100644 --- a/client/go.sum +++ b/client/go.sum @@ -24,6 +24,42 @@ github.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 h1:bsU8Tzxr/P github.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54/go.mod h1:0tr7FllbE9gJkHq7CVeeDDFAFKQVy5RnCSSNBOvdqbc= github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII= github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c= +github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg= +github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ= +github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 h1:adBsCIIpLbLmYnkQU+nAChU5yhVTvu5PerROm+/Kq2A= +github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9/go.mod h1:uOYhgfgThm/ZyAuJGNQ5YgNyOlYfqnGpTHXvk3cpykg= +github.com/aws/aws-sdk-go-v2/config v1.32.16 h1:Q0iQ7quUgJP0F/SCRTieScnaMdXr9h/2+wze1u3cNeM= +github.com/aws/aws-sdk-go-v2/config v1.32.16/go.mod h1:duCCnJEFqpt2RC6no1iK6q+8HpwOAkiUua0pY507dQc= +github.com/aws/aws-sdk-go-v2/credentials v1.19.15 h1:fyvgWTszojq8hEnMi8PPBTvZdTtEVmAVyo+NFLHBhH4= +github.com/aws/aws-sdk-go-v2/credentials v1.19.15/go.mod h1:gJiYyMOjNg8OEdRWOf3CrFQxM2a98qmrtjx1zuiQfB8= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 h1:IOGsJ1xVWhsi+ZO7/NW8OuZZBtMJLZbk4P5HDjJO0jQ= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22/go.mod h1:b+hYdbU+jGKfXE8kKM6g1+h+L/Go3vMvzlxBsiuGsxg= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 h1:GmLa5Kw1ESqtFpXsx5MmC84QWa/ZrLZvlJGa2y+4kcQ= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22/go.mod h1:6sW9iWm9DK9YRpRGga/qzrzNLgKpT2cIxb7Vo2eNOp0= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 h1:dY4kWZiSaXIzxnKlj17nHnBcXXBfac6UlsAx2qL6XrU= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22/go.mod h1:KIpEUx0JuRZLO7U6cbV204cWAEco2iC3l061IxlwLtI= +github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23 h1:FPXsW9+gMuIeKmz7j6ENWcWtBGTe1kH8r9thNt5Uxx4= +github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23/go.mod h1:7J8iGMdRKk6lw2C+cMIphgAnT8uTwBwNOsGkyOCm80U= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8 h1:HtOTYcbVcGABLOVuPYaIihj6IlkqubBwFj10K5fxRek= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8/go.mod h1:VsK9abqQeGlzPgUr+isNWzPlK2vKe9INMLWnY65f5Xs= +github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14 h1:xnvDEnw+pnj5mctWiYuFbigrEzSm35x7k4KS/ZkCANg= +github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14/go.mod h1:yS5rNogD8e0Wu9+l3MUwr6eENBzEeGejvINpN5PAYfY= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22 h1:PUmZeJU6Y1Lbvt9WFuJ0ugUK2xn6hIWUBBbKuOWF30s= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22/go.mod h1:nO6egFBoAaoXze24a2C0NjQCvdpk8OueRoYimvEB9jo= +github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22 h1:SE+aQ4DEqG53RRCAIHlCf//B2ycxGH7jFkpnAh/kKPM= +github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22/go.mod h1:ES3ynECd7fYeJIL6+oax+uIEljmfps0S70BaQzbMd/o= +github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1 h1:kU/eBN5+MWNo/LcbNa4hWDdN76hdcd7hocU5kvu7IsU= +github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1/go.mod h1:Fw9aqhJicIVee1VytBBjH+l+5ov6/PhbtIK/u3rt/ls= +github.com/aws/aws-sdk-go-v2/service/signin v1.0.10 h1:a1Fq/KXn75wSzoJaPQTgZO0wHGqE9mjFnylnqEPTchA= +github.com/aws/aws-sdk-go-v2/service/signin v1.0.10/go.mod h1:p6+MXNxW7IA6dMgHfTAzljuwSKD0NCm/4lbS4t6+7vI= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.16 h1:x6bKbmDhsgSZwv6q19wY/u3rLk/3FGjJWyqKcIRufpE= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.16/go.mod h1:CudnEVKRtLn0+3uMV0yEXZ+YZOKnAtUJ5DmDhilVnIw= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 h1:oK/njaL8GtyEihkWMD4k3VgHCT64RQKkZwh0DG5j8ak= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20/go.mod h1:JHs8/y1f3zY7U5WcuzoJ/yAYGYtNIVPKLIbp61euvmg= +github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 h1:ks8KBcZPh3PYISr5dAiXCM5/Thcuxk8l+PG4+A0exds= +github.com/aws/aws-sdk-go-v2/service/sts v1.42.0/go.mod h1:pFw33T0WLvXU3rw1WBkpMlkgIn54eCB5FYLhjDc9Foo= +github.com/aws/smithy-go v1.25.0 h1:Sz/XJ64rwuiKtB6j98nDIPyYrV1nVNJ4YU74gttcl5U= +github.com/aws/smithy-go v1.25.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc= github.com/aymanbagabas/go-udiff v0.4.1 h1:OEIrQ8maEeDBXQDoGCbbTTXYJMYRCRO1fnodZ12Gv5o= github.com/aymanbagabas/go-udiff v0.4.1/go.mod h1:0L9PGwj20lrtmEMeyw4WKJ/TMyDtvAoK9bf2u/mNo3w= github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= @@ -710,6 +746,8 @@ golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI= +golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= @@ -723,6 +761,8 @@ golang.org/x/term v0.16.0/go.mod h1:yn7UURbUtPyrVJPGPq404EukNFxcm/foM+bV/bfcDsY= golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk= golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY= golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM= +golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY= +golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= diff --git a/client/utils/clientConfig.go b/client/utils/clientConfig.go index 8cd846c5..78b8d557 100644 --- a/client/utils/clientConfig.go +++ b/client/utils/clientConfig.go @@ -12,12 +12,17 @@ func CreateDefaultConfig() { configPath := GetConfigPath() fmt.Printf("Creating default config: %s\n", configPath) + // Leave NodeInstallDir / NodeStateDir / NodeSymlinkDir / + // QClientInstallDir / DataDir empty here so the OS-aware helpers + // in paths.go supply the current defaults lazily. Persisting them + // would pin the user to whatever default was in effect at + // config-creation time. SaveClientConfig(&ClientConfig{ - DataDir: ClientDataPath, - SymlinkPath: DefaultQClientSymlinkPath, - SignatureCheck: true, - PublicRpc: false, - CustomRpc: "", + SymlinkPath: DefaultQClientSymlinkPath, + SignatureCheck: true, + PublicRpc: false, + CustomRpc: "", + NodeServiceName: DefaultNodeServiceName, }) sudoUser, err := GetCurrentSudoUser() @@ -32,14 +37,16 @@ func CreateDefaultConfig() { func LoadClientConfig() (*ClientConfig, error) { configPath := GetConfigPath() - // Create default config if it doesn't exist + // Create default config if it doesn't exist. Leave node and + // qclient path fields empty so OS-aware defaults from paths.go + // apply lazily. if _, err := os.Stat(configPath); os.IsNotExist(err) { config := &ClientConfig{ - DataDir: ClientDataPath, - SymlinkPath: filepath.Join(ClientDataPath, "current"), - SignatureCheck: true, - PublicRpc: false, - CustomRpc: "", + SymlinkPath: DefaultQClientSymlinkPath, + SignatureCheck: true, + PublicRpc: false, + CustomRpc: "", + NodeServiceName: DefaultNodeServiceName, } if err := SaveClientConfig(config); err != nil { return nil, err @@ -58,6 +65,16 @@ func LoadClientConfig() (*ClientConfig, error) { return nil, err } + // Backfill fields that may be missing from older configs. Only + // backfill the service name here; leave NodeInstallDir / + // NodeStateDir / NodeSymlinkDir empty so the OS-aware path + // accessors apply current defaults. Older configs that already + // have an explicit NodeInstallDir (e.g. the legacy + // /var/quilibrium) keep their persisted value untouched. + if config.NodeServiceName == "" { + config.NodeServiceName = DefaultNodeServiceName + } + return config, nil } diff --git a/client/utils/download.go b/client/utils/download.go index f7281d18..c4f57348 100644 --- a/client/utils/download.go +++ b/client/utils/download.go @@ -12,11 +12,22 @@ import ( var BaseReleaseURL = "https://releases.quilibrium.com" +// releaseBaseDir returns the base directory that holds versioned +// subdirectories for the given release type. Both node and qclient +// respect the user's configured install directory; defaults are +// OS-aware (see GetNodeBinaryDir / GetQClientBinaryDir). +func releaseBaseDir(releaseType ReleaseType) string { + if releaseType == ReleaseTypeNode { + return GetNodeBinaryDir() + } + return GetQClientBinaryDir() +} + // DownloadRelease downloads a specific release file func DownloadRelease(releaseType ReleaseType, version string) error { fileName := fmt.Sprintf("%s-%s-%s-%s", releaseType, version, OsType, Arch) fmt.Printf("Getting binary %s...\n", fileName) - fmt.Println("Will save to", filepath.Join(BinaryPath, string(releaseType), version)) + fmt.Println("Will save to", filepath.Join(releaseBaseDir(releaseType), version)) url := fmt.Sprintf("%s/%s", BaseReleaseURL, fileName) if !DoesRemoteFileExist(url) { @@ -63,7 +74,7 @@ func GetLatestVersion(releaseType ReleaseType) (string, error) { // DownloadReleaseFile downloads a release file from the Quilibrium releases server func DownloadReleaseFile(releaseType ReleaseType, fileName string, version string, showError bool) error { url := fmt.Sprintf("%s/%s", BaseReleaseURL, fileName) - destDir := filepath.Join(BinaryPath, string(releaseType), version) + destDir := filepath.Join(releaseBaseDir(releaseType), version) os.MkdirAll(destDir, 0755) destPath := filepath.Join(destDir, fileName) @@ -102,7 +113,7 @@ func DownloadReleaseSignatures(releaseType ReleaseType, version string) error { var files []string baseName := fmt.Sprintf("%s-%s-%s-%s", releaseType, version, OsType, Arch) fmt.Printf("Searching for signatures for %s from %s\n", baseName, BaseReleaseURL) - fmt.Println("Will save to", filepath.Join(BinaryPath, string(releaseType), version)) + fmt.Println("Will save to", filepath.Join(releaseBaseDir(releaseType), version)) // Add digest file URL files = append(files, baseName+".dgst") diff --git a/client/utils/fileUtils.go b/client/utils/fileUtils.go index 0291f9f2..73993294 100644 --- a/client/utils/fileUtils.go +++ b/client/utils/fileUtils.go @@ -88,10 +88,11 @@ func ValidateAndCreateDir(path string, user *user.User) error { // Directory doesn't exist, try to create it if os.IsNotExist(err) { - fmt.Printf("Creating directory %s\n", path) + fmt.Printf("Creating directory %s...", path) if err := os.MkdirAll(path, 0755); err != nil { return fmt.Errorf("failed to create directory %s: %v", path, err) } + fmt.Printf(" done\n") if user != nil { ChownPath(path, user, false) } @@ -150,15 +151,17 @@ func IsSudo() bool { func ChownPath(path string, user *user.User, isRecursive bool) error { // Change ownership of the path if isRecursive { - fmt.Printf("Changing ownership of %s (recursive) to %s\n", path, user.Username) + fmt.Printf("Changing ownership of %s (recursive) to %s...", path, user.Username) if err := exec.Command("chown", "-R", user.Uid+":"+user.Gid, path).Run(); err != nil { return fmt.Errorf("failed to change ownership of %s to %s (requires sudo): %v", path, user.Uid, err) } + fmt.Printf(" done\n") } else { - fmt.Printf("Changing ownership of %s to %s\n", path, user.Username) + fmt.Printf("Changing ownership of %s to %s...", path, user.Username) if err := exec.Command("chown", user.Uid+":"+user.Gid, path).Run(); err != nil { return fmt.Errorf("failed to change ownership of %s to %s (requires sudo): %v", path, user.Uid, err) } + fmt.Printf(" done\n") } return nil diff --git a/client/utils/node.go b/client/utils/node.go index 444eec70..1efa2f21 100644 --- a/client/utils/node.go +++ b/client/utils/node.go @@ -6,6 +6,7 @@ import ( "os" "os/exec" "path/filepath" + "regexp" "github.com/pkg/errors" @@ -18,11 +19,8 @@ import ( var ( NetworkConfigOverride string DefaultNodeConfigName = "node-quickstart" - NodeDataPath = filepath.Join(BinaryPath, string(ReleaseTypeNode)) - NodeEnvPath = filepath.Join(RootQuilibriumPath, "quilibrium.env") NodeServiceName = "quilibrium-node" - DefaultNodeSymlinkPath = filepath.Join(DefaultSymlinkDir, NodeServiceName) - LogPath = "/var/log/quilibrium" + DefaultNodeServiceName = "quilibrium-node" ) func GetPeerIDFromConfig(cfg *config.Config) peer.ID { @@ -56,7 +54,37 @@ func GetPrivKeyFromConfig(cfg *config.Config) (crypto.PrivKey, error) { } func IsExistingNodeVersion(version string) bool { - return FileExists(filepath.Join(NodeDataPath, version)) + return FileExists(filepath.Join(GetNodeBinaryDir(), version)) +} + +// GetNodeServiceName returns the user-configured systemd/launchd service name, +// falling back to DefaultNodeServiceName when unset or when the config cannot +// be read. It is used for Linux systemd unit operations; callers that must +// reference the fixed binary/package name (e.g. the /usr/local/bin symlink, +// the macOS launchd label, or cleanup of legacy logrotate configs) +// should continue to use DefaultNodeServiceName directly. +// nodeServiceNameRegex restricts service names to characters that are safe +// for systemd unit filenames and shell invocation. +var nodeServiceNameRegex = regexp.MustCompile(`^[A-Za-z0-9._-]+$`) + +// ValidateNodeServiceName returns an error when name contains characters +// that are unsafe for systemd unit filenames / shell invocation. +func ValidateNodeServiceName(name string) error { + if !nodeServiceNameRegex.MatchString(name) { + return fmt.Errorf( + "invalid service name %q. Allowed characters: letters, digits, '.', '_', '-'", + name, + ) + } + return nil +} + +func GetNodeServiceName() string { + cfg, err := LoadClientConfig() + if err != nil || cfg == nil || cfg.NodeServiceName == "" { + return DefaultNodeServiceName + } + return cfg.NodeServiceName } func CheckForSystemd() bool { @@ -65,20 +93,17 @@ func CheckForSystemd() bool { return err == nil } +// GetNodeConfigHomeDir is retained as a thin wrapper over +// GetNodeConfigsDir so older callers continue to compile. New code should +// call GetNodeConfigsDir directly. func GetNodeConfigHomeDir() string { - userLookup, err := GetCurrentSudoUser() - if err != nil { - fmt.Fprintf(os.Stderr, "Error getting current user: %v\n", err) - os.Exit(1) - } - - path := filepath.Join(userLookup.HomeDir, ".quilibrium", "configs") - - if _, err := os.Stat(path); os.IsNotExist(err) { - ValidateAndCreateDir(path, userLookup) - } + return GetNodeConfigsDir() +} - return path +// GetDefaultNodeConfigSymlink returns the path of the "default" symlink that +// the node follows to locate its active configuration directory. +func GetDefaultNodeConfigSymlink() string { + return filepath.Join(GetNodeConfigHomeDir(), "default") } func GetDefaultNodeConfigDir() (string, error) { @@ -101,9 +126,10 @@ func GetDefaultNodeConfigDir() (string, error) { ) } - fmt.Printf("Default node config directory does not exist, creating it\n") + fmt.Printf("Default node config directory does not exist, creating it...") // if neither exists, create it CreateDefaultNodeConfig(DefaultNodeConfigName) + fmt.Printf(" done\n") return configPath, nil } // Check if the config path is a symlink @@ -148,7 +174,58 @@ func LoadNodeConfig(configDirectory string) (*config.Config, error) { return LoadDefaultNodeConfig() } - return config.LoadConfig(configDirectory, "", false) + resolved, err := ResolveNodeConfigDir(configDirectory) + if err != nil { + return nil, err + } + + return config.LoadConfig(resolved, "", false) +} + +// ResolveNodeConfigDir resolves the value passed to --config into an absolute +// filesystem path, without creating anything on disk. It accepts either a +// named config (looked up under ~/.quilibrium/configs/) or a direct +// path (absolute or relative to the current working directory). The resolved +// directory must exist and contain both config.yml and keys.yml, otherwise an +// error is returned explaining what was checked. +func ResolveNodeConfigDir(value string) (string, error) { + if value == "" { + return "", fmt.Errorf("config directory not specified") + } + + namedPath := filepath.Join(GetNodeConfigHomeDir(), value) + if info, err := os.Stat(namedPath); err == nil && info.IsDir() { + if !HasNodeConfigFiles(namedPath) { + return "", fmt.Errorf( + "%s: %s", ErrNotValidConfigDirMessage, namedPath, + ) + } + return namedPath, nil + } + + if info, err := os.Stat(value); err == nil { + if !info.IsDir() { + return "", fmt.Errorf( + "config path is not a directory: %s", value, + ) + } + abs, err := filepath.Abs(value) + if err != nil { + abs = value + } + if !HasNodeConfigFiles(abs) { + return "", fmt.Errorf( + "%s: %s", ErrNotValidConfigDirMessage, abs, + ) + } + return abs, nil + } + + return "", fmt.Errorf( + "config directory not found: %q (looked for a named config at %s "+ + "and as a filesystem path)", + value, namedPath, + ) } // HasNodeConfigFiles checks if a directory contains both config.yml and diff --git a/client/utils/nodelog.go b/client/utils/nodelog.go new file mode 100644 index 00000000..de168cbf --- /dev/null +++ b/client/utils/nodelog.go @@ -0,0 +1,172 @@ +package utils + +import ( + "fmt" + "os" + "path/filepath" + + "source.quilibrium.com/quilibrium/monorepo/config" +) + +// Default rotation knobs used when qclient populates a node config's +// logger block at create/install time. These are safe middle-of-the-road +// values — users can override them with `qclient node config set +// logger.maxSize 50` etc. +const ( + DefaultLoggerMaxSize = 100 // megabytes per file before rotation + DefaultLoggerMaxBackups = 5 // rotated files to keep + DefaultLoggerMaxAge = 30 // days to keep rotated files + DefaultLoggerCompress = true +) + +// ResolvedNodeLog describes where a given node config writes its logs. +// When FileBased is false, the node is logging to stdout and callers +// should fall back to the system log (journalctl on Linux, launchd +// StandardOutPath on macOS). +type ResolvedNodeLog struct { + // ConfigName is the name of the node config (e.g. "node-quickstart") + // whose logger block was resolved. Empty for an ad-hoc path. + ConfigName string + // ConfigDir is the absolute directory of the node config this + // resolution is for (contains config.yml). + ConfigDir string + // FileBased is true when the node config has a logger block with a + // non-empty path; callers can then read LogDir / MasterLogFile. + FileBased bool + // LogDir is the logger.path from the node config. Valid only when + // FileBased is true. + LogDir string +} + +// MasterLogFile returns the conventional path to the node's primary +// (coreId=0) log file inside LogDir, matching what the node's logger +// actually writes (utils/logging.filenameForCore). +func (r ResolvedNodeLog) MasterLogFile() string { + if !r.FileBased { + return "" + } + return filepath.Join(r.LogDir, "master.log") +} + +// ResolveActiveNodeLog resolves the log destination for the node's +// currently-active (default) config. It never creates or mutates files +// on disk; if the default config isn't present it returns an error. +func ResolveActiveNodeLog() (ResolvedNodeLog, error) { + dir, err := GetDefaultNodeConfigDir() + if err != nil { + return ResolvedNodeLog{}, err + } + return resolveNodeLogForDir(dir) +} + +// ResolveNodeLogByName resolves the log destination for the named node +// config under the configs directory. +func ResolveNodeLogByName(name string) (ResolvedNodeLog, error) { + dir := filepath.Join(GetNodeConfigHomeDir(), name) + if _, err := os.Stat(dir); err != nil { + return ResolvedNodeLog{}, fmt.Errorf( + "node config %q not found at %s", name, dir, + ) + } + return resolveNodeLogForDir(dir) +} + +// ResolveAllNodeLogDirs returns every log directory referenced by an +// installed node config that currently has a logger block. This is used +// by uninstall/clean helpers that need to sweep log files across every +// known config. +func ResolveAllNodeLogDirs() []string { + configsDir := GetNodeConfigHomeDir() + entries, err := os.ReadDir(configsDir) + if err != nil { + return nil + } + seen := map[string]struct{}{} + dirs := make([]string, 0, len(entries)) + for _, e := range entries { + if !e.IsDir() || e.Name() == "default" { + continue + } + resolved, err := resolveNodeLogForDir(filepath.Join(configsDir, e.Name())) + if err != nil || !resolved.FileBased { + continue + } + if _, ok := seen[resolved.LogDir]; ok { + continue + } + seen[resolved.LogDir] = struct{}{} + dirs = append(dirs, resolved.LogDir) + } + return dirs +} + +func resolveNodeLogForDir(configDir string) (ResolvedNodeLog, error) { + abs, err := filepath.EvalSymlinks(configDir) + if err != nil { + abs = configDir + } + name := filepath.Base(abs) + + cfg, err := config.NewConfig(filepath.Join(abs, "config.yml")) + if err != nil { + return ResolvedNodeLog{ + ConfigName: name, + ConfigDir: abs, + FileBased: false, + }, nil + } + if cfg == nil || cfg.Logger == nil || cfg.Logger.Path == "" { + return ResolvedNodeLog{ + ConfigName: name, + ConfigDir: abs, + FileBased: false, + }, nil + } + return ResolvedNodeLog{ + ConfigName: name, + ConfigDir: abs, + FileBased: true, + LogDir: cfg.Logger.Path, + }, nil +} + +// EnsureNodeConfigLogger makes sure the config.yml at configDir has a +// logger block pointing at DefaultNodeLogDirForConfig(configDir). If a +// logger block already exists, the function leaves it untouched and +// returns the existing path. The returned boolean reports whether the +// config was modified on disk. +func EnsureNodeConfigLogger(configDir string) (string, bool, error) { + abs, err := filepath.EvalSymlinks(configDir) + if err != nil { + abs = configDir + } + + cfg, err := config.NewConfig(filepath.Join(abs, "config.yml")) + if err != nil { + return "", false, fmt.Errorf("loading node config at %s: %w", abs, err) + } + if cfg.Logger != nil && cfg.Logger.Path != "" { + return cfg.Logger.Path, false, nil + } + if cfg.Logger == nil { + cfg.Logger = &config.LogConfig{} + } + cfg.Logger.Path = DefaultNodeLogDirForConfig(abs) + if cfg.Logger.MaxSize == 0 { + cfg.Logger.MaxSize = DefaultLoggerMaxSize + } + if cfg.Logger.MaxBackups == 0 { + cfg.Logger.MaxBackups = DefaultLoggerMaxBackups + } + if cfg.Logger.MaxAge == 0 { + cfg.Logger.MaxAge = DefaultLoggerMaxAge + } + if !cfg.Logger.Compress { + cfg.Logger.Compress = DefaultLoggerCompress + } + + if err := config.SaveConfig(abs, cfg); err != nil { + return "", false, fmt.Errorf("saving node config at %s: %w", abs, err) + } + return cfg.Logger.Path, true, nil +} diff --git a/client/utils/paths.go b/client/utils/paths.go new file mode 100644 index 00000000..c71a6139 --- /dev/null +++ b/client/utils/paths.go @@ -0,0 +1,212 @@ +package utils + +import ( + "fmt" + "os" + "path/filepath" + "runtime" +) + +// Default install-time paths. These intentionally follow the FHS on +// Linux and Homebrew-style conventions on macOS so binaries, state, +// and symlinks land in the locations users expect for a system-wide +// install managed by sudo + systemd/launchd. +const ( + // LegacyNodeInstallDir is the pre-FHS-split install root. It is + // kept only for detecting legacy installs so we can warn the user; + // new installs should not land here. + LegacyNodeInstallDir = "/var/quilibrium" + + // DefaultNodeLogRelDir is the directory name for file logs under + // each node config directory (the folder containing config.yml) + // when logger.path is populated by qclient defaults. + DefaultNodeLogRelDir = ".logs" + + // DefaultNodeConfigsSubdir is the subdirectory of the user's home + // directory where node configs live when no override is set. + DefaultNodeConfigsSubdir = ".quilibrium/configs" +) + +// DefaultNodeInstallDir returns the OS-appropriate default root for +// node binaries: /opt/quilibrium on Linux (FHS), /usr/local/quilibrium +// on macOS (Homebrew-style). Unknown GOOS falls back to the Linux +// default. +func DefaultNodeInstallDir() string { + switch runtime.GOOS { + case "darwin": + return "/usr/local/quilibrium" + default: + return "/opt/quilibrium" + } +} + +// DefaultNodeStateDir returns the OS-appropriate default root for +// mutable node state (env file, future state/spool): /var/lib/quilibrium +// on Linux (FHS), /usr/local/var/quilibrium on macOS. +func DefaultNodeStateDir() string { + switch runtime.GOOS { + case "darwin": + return "/usr/local/var/quilibrium" + default: + return "/var/lib/quilibrium" + } +} + +// DefaultNodeSymlinkDir returns the directory where the node symlink +// is created. /usr/local/bin on both Linux and macOS. +func DefaultNodeSymlinkDir() string { + return "/usr/local/bin" +} + +// DefaultQClientInstallDir returns the OS-appropriate default root for +// the qclient binary tree: /opt/quilibrium on Linux, /usr/local/quilibrium +// on macOS. Matches the node install root so both trees live together. +func DefaultQClientInstallDir() string { + switch runtime.GOOS { + case "darwin": + return "/usr/local/quilibrium" + default: + return "/opt/quilibrium" + } +} + +// LegacyQClientBinaryDir is the pre-FHS-split qclient binary root. Kept +// only for detecting legacy installs so we can warn the user. +const LegacyQClientBinaryDir = "/var/quilibrium/bin/qclient" + +// loadConfigOrDefault returns the persisted client config, or a zero-value +// config if loading fails. Path accessors are best-effort: callers should +// always get a usable default even when the config file is missing or +// temporarily unreadable. +func loadConfigOrDefault() *ClientConfig { + cfg, err := LoadClientConfig() + if err != nil || cfg == nil { + return &ClientConfig{} + } + return cfg +} + +// GetNodeInstallDir returns the configured node install root, or the +// OS-appropriate default when unset. +func GetNodeInstallDir() string { + cfg := loadConfigOrDefault() + if cfg.NodeInstallDir != "" { + return cfg.NodeInstallDir + } + return DefaultNodeInstallDir() +} + +// GetNodeStateDir returns the configured node state root, or the +// OS-appropriate default when unset. The env file and any future +// mutable node state live here. +func GetNodeStateDir() string { + cfg := loadConfigOrDefault() + if cfg.NodeStateDir != "" { + return cfg.NodeStateDir + } + return DefaultNodeStateDir() +} + +// GetNodeBinaryDir returns the directory that holds versioned node binary +// subdirectories, e.g. /bin/node/. +func GetNodeBinaryDir() string { + return filepath.Join(GetNodeInstallDir(), "bin", string(ReleaseTypeNode)) +} + +// GetNodeEnvFilePath returns the path to the systemd EnvironmentFile +// used by the node service, e.g. /quilibrium.env. +func GetNodeEnvFilePath() string { + return filepath.Join(GetNodeStateDir(), "quilibrium.env") +} + +// DefaultNodeLogDirForConfig returns the default logger directory for a +// node config located at configDir (the directory containing config.yml), +// i.e. /.logs. This is the value qclient writes into the node +// config's logger.path when creating/installing a config. +func DefaultNodeLogDirForConfig(configDir string) string { + return filepath.Join(configDir, DefaultNodeLogRelDir) +} + +// GetNodeSymlinkDir returns the directory where the node binary symlink is +// created, defaulting to /usr/local/bin. +func GetNodeSymlinkDir() string { + cfg := loadConfigOrDefault() + if cfg.NodeSymlinkDir != "" { + return cfg.NodeSymlinkDir + } + return DefaultNodeSymlinkDir() +} + +// GetNodeSymlinkPath returns the full path of the node binary symlink, +// e.g. /usr/local/bin/quilibrium-node. The symlink file name itself is +// always the fixed DefaultNodeServiceName so that existing shell usage +// of `quilibrium-node` keeps working regardless of the service name. +func GetNodeSymlinkPath() string { + return filepath.Join(GetNodeSymlinkDir(), DefaultNodeServiceName) +} + +// GetQClientInstallDir returns the configured qclient install root. +// Resolution order: cfg.QClientInstallDir → legacy cfg.DataDir's parent +// (for back-compat with configs that pre-date QClientInstallDir and +// still pin DataDir to /var/quilibrium/bin/qclient) → OS-appropriate +// default. +func GetQClientInstallDir() string { + cfg := loadConfigOrDefault() + if cfg.QClientInstallDir != "" { + return cfg.QClientInstallDir + } + // cfg.DataDir historically points at /bin/qclient. If it + // is set, reverse-derive the install root so existing configs + // keep working without rewrites. + if cfg.DataDir != "" { + // Expect layout /bin/qclient; strip the trailing + // "bin/qclient" when present. + dir := filepath.Clean(cfg.DataDir) + parent := filepath.Dir(dir) // /bin + grandparent := filepath.Dir(parent) // + if filepath.Base(dir) == string(ReleaseTypeQClient) && filepath.Base(parent) == "bin" { + return grandparent + } + } + return DefaultQClientInstallDir() +} + +// GetQClientBinaryDir returns the directory that holds versioned +// qclient binary subdirectories, e.g. /bin/qclient. +func GetQClientBinaryDir() string { + return filepath.Join(GetQClientInstallDir(), "bin", string(ReleaseTypeQClient)) +} + +// GetNodeConfigsDir returns the configured node configs directory, or the +// default $HOME/.quilibrium/configs resolved against the invoking (sudo) +// user's home. The directory is created on demand. +func GetNodeConfigsDir() string { + cfg := loadConfigOrDefault() + if cfg.NodeConfigsDir != "" { + ensureDirExistsForSudoUser(cfg.NodeConfigsDir) + return cfg.NodeConfigsDir + } + + userLookup, err := GetCurrentSudoUser() + if err != nil { + fmt.Fprintf(os.Stderr, "Error getting current user: %v\n", err) + os.Exit(1) + } + path := filepath.Join(userLookup.HomeDir, DefaultNodeConfigsSubdir) + ensureDirExistsForSudoUser(path) + return path +} + +// ensureDirExistsForSudoUser creates the given path if missing, owned by +// the invoking sudo user when available. +func ensureDirExistsForSudoUser(path string) { + if _, err := os.Stat(path); err == nil { + return + } + userLookup, err := GetCurrentSudoUser() + if err != nil { + _ = os.MkdirAll(path, 0755) + return + } + _ = ValidateAndCreateDir(path, userLookup) +} diff --git a/client/utils/types.go b/client/utils/types.go index f9f8230d..00c7d676 100644 --- a/client/utils/types.go +++ b/client/utils/types.go @@ -4,11 +4,70 @@ type ClientConfig struct { DataDir string `yaml:"dataDir"` SymlinkPath string `yaml:"symlinkPath"` SignatureCheck bool `yaml:"signatureCheck"` + Quiet bool `yaml:"quiet"` PublicRpc bool `yaml:"publicRpc"` CustomRpc string `yaml:"customRpc"` NodeSymlinkName string `yaml:"nodeSymlinkName"` + NodeServiceName string `yaml:"nodeServiceName"` + // QClientInstallDir is the root directory for the qclient binary + // tree. Defaults to /opt/quilibrium on Linux and + // /usr/local/quilibrium on macOS. Binaries live under + // /bin/qclient//. When empty, the + // legacy cfg.DataDir is consulted for back-compat. + QClientInstallDir string `yaml:"qclientInstallDir"` + // NodeInstallDir is the root directory for the node binary tree. + // Defaults to /opt/quilibrium on Linux and /usr/local/quilibrium on + // macOS. The actual binaries live under + // /bin/node//. + NodeInstallDir string `yaml:"nodeInstallDir"` + // NodeStateDir is the root directory for mutable node state + // (currently the systemd EnvironmentFile). Defaults to + // /var/lib/quilibrium on Linux and /usr/local/var/quilibrium on + // macOS. + NodeStateDir string `yaml:"nodeStateDir"` + // NodeSymlinkDir is the directory where the node binary symlink + // (quilibrium-node) is created. Defaults to /usr/local/bin. + NodeSymlinkDir string `yaml:"nodeSymlinkDir"` + // NodeConfigsDir is the directory that holds named node configs. + // Defaults to $HOME/.quilibrium/configs (resolved from the invoking + // sudo user's home directory). + NodeConfigsDir string `yaml:"nodeConfigsDir"` + + // Backup holds S3-compatible node backup settings. Populate via + // `qclient node backup config`. + Backup NodeBackupConfig `yaml:"backup"` +} + +// NodeBackupConfig holds S3-compatible object storage settings used by +// `qclient node backup`. Credentials are stored alongside the rest of +// the qclient configuration. +type NodeBackupConfig struct { + Enabled bool `yaml:"enabled"` + AccessKeyID string `yaml:"accessKeyId"` + SecretAccessKey string `yaml:"secretAccessKey"` + Endpoint string `yaml:"endpoint"` + Bucket string `yaml:"bucket"` + // BucketPrefix is an optional key prefix inside the bucket, used + // when the bucket is shared with other data and backups should be + // namespaced (e.g. "quilibrium/backups"). All object keys and the + // per-config manifest are written under this prefix. Leading and + // trailing slashes are tolerated on input and normalized away. + // Empty means "store at the bucket root" (current behavior). + BucketPrefix string `yaml:"bucketPrefix"` + Region string `yaml:"region"` + // UsePathStyle controls S3 path-style addressing + // (bucket.host vs host/bucket). Most S3-compatible providers + // require path-style; defaults to true. + UsePathStyle bool `yaml:"usePathStyle"` } +// Default values for NodeBackupConfig. +const ( + DefaultBackupEndpoint = "https://qstorage.quilibrium.com" + DefaultBackupRegion = "q-world-1" + DefaultBackupUsePathStyle = true +) + type NodeConfig struct { ClientConfig RewardsAddress string `yaml:"rewardsAddress"` diff --git a/config/config.go b/config/config.go index b4d2b131..6e4de667 100644 --- a/config/config.go +++ b/config/config.go @@ -426,9 +426,12 @@ func LoadConfig(configPath string, proverKey string, skipGenesisCheck bool) ( } func SaveConfig(configPath string, config *Config) error { + // O_TRUNC is important: without it, writing a shorter YAML on top + // of an existing longer file would leave trailing garbage after the + // encoded document. file, err := os.OpenFile( filepath.Join(configPath, "config.yml"), - os.O_CREATE|os.O_RDWR, + os.O_CREATE|os.O_WRONLY|os.O_TRUNC, os.FileMode(0600), ) if err != nil { @@ -560,11 +563,7 @@ func PrintVersion(network uint8, char string, ver string) { schar = " " } - patch := GetPatchNumber() patchString := "" - if patch != 0x00 { - patchString = fmt.Sprintf("-p%d", patch) - } if network != 0 { patchString = fmt.Sprintf("-b%d", GetRCNumber()) } diff --git a/config/version.go b/config/version.go index 1d083945..e98397f1 100644 --- a/config/version.go +++ b/config/version.go @@ -25,7 +25,11 @@ func GetVersion() []byte { } func GetVersionString() string { - return FormatVersion(GetVersion()) + base := FormatVersion(GetVersion()) + if patch := GetPatchNumber(); patch != 0x00 { + return fmt.Sprintf("%s.%d", base, patch) + } + return base } func FormatVersion(version []byte) string { diff --git a/docker/.dockerignore b/docker/.dockerignore index 36d102c1..14b80ef5 100644 --- a/docker/.dockerignore +++ b/docker/.dockerignore @@ -5,6 +5,22 @@ github.env Taskfile.yaml .git +# CMake in-tree / out-of-tree junk (host paths break Docker builds if copied) +emp-tool/CMakeCache.txt +emp-tool/CMakeFiles +emp-tool/Makefile +emp-tool/cmake_install.cmake +emp-tool/CTestTestfile.cmake +emp-tool/install_manifest.txt +emp-tool/build +emp-ot/CMakeCache.txt +emp-ot/CMakeFiles +emp-ot/Makefile +emp-ot/cmake_install.cmake +emp-ot/CTestTestfile.cmake +emp-ot/install_manifest.txt +emp-ot/build + # Rust target vdf/generated diff --git a/docker/Dockerfile.source b/docker/Dockerfile.source index 4d5ea6d9..a3dec962 100644 --- a/docker/Dockerfile.source +++ b/docker/Dockerfile.source @@ -63,11 +63,11 @@ COPY emp-ot emp-ot RUN bash install-emp.sh -# Fix emp-tool to be static and install -RUN cd emp-tool && sed -i 's/add_library(${NAME} SHARED ${sources})/add_library(${NAME} STATIC ${sources})/g' CMakeLists.txt && mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && cd .. && make -j$(nproc) && make install +# Fix emp-tool to be static and install (build only in build/ so we never pick up a host CMakeCache.txt from COPY) +RUN cd emp-tool && sed -i 's/add_library(${NAME} SHARED ${sources})/add_library(${NAME} STATIC ${sources})/g' CMakeLists.txt && rm -rf build CMakeCache.txt CMakeFiles Makefile cmake_install.cmake CTestTestfile.cmake install_manifest.txt && mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && make -j$(nproc) && make install # Install emp-ot -RUN cd emp-ot && mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && cd .. && make -j$(nproc) && make install +RUN cd emp-ot && rm -rf build CMakeCache.txt CMakeFiles Makefile cmake_install.cmake CTestTestfile.cmake install_manifest.txt && mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && make -j$(nproc) && make install # ----------------------------------------------------------------------------- # Stage: go-base diff --git a/docker/Dockerfile.sourceavx512 b/docker/Dockerfile.sourceavx512 index 5f447aa5..568a11e0 100644 --- a/docker/Dockerfile.sourceavx512 +++ b/docker/Dockerfile.sourceavx512 @@ -124,9 +124,9 @@ COPY go-libp2p-blossomsub/go.mod go-libp2p-blossomsub/go.sum go-libp2p-blossomsu RUN bash install-emp.sh ENV CFLAGS="-march=skylake-avx512 -mtune=skylake-avx512" -RUN cd emp-tool && sed -i 's/add_library(${NAME} SHARED ${sources})/add_library(${NAME} STATIC ${sources})/g' CMakeLists.txt && mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && cd .. && make && make install && cd .. +RUN cd emp-tool && sed -i 's/add_library(${NAME} SHARED ${sources})/add_library(${NAME} STATIC ${sources})/g' CMakeLists.txt && rm -rf build CMakeCache.txt CMakeFiles Makefile cmake_install.cmake CTestTestfile.cmake install_manifest.txt && mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && make -j$(nproc) && make install && cd .. -RUN cd emp-ot && mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && cd .. && make && make install && cd .. +RUN cd emp-ot && rm -rf build CMakeCache.txt CMakeFiles Makefile cmake_install.cmake CTestTestfile.cmake install_manifest.txt && mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local && make -j$(nproc) && make install && cd .. ## Generate Rust bindings for channel WORKDIR /opt/ceremonyclient/channel diff --git a/install-emp.sh b/install-emp.sh old mode 100644 new mode 100755 index 467dbaba..4c0d4fc4 --- a/install-emp.sh +++ b/install-emp.sh @@ -17,6 +17,8 @@ fi for tool in ${TOOLS[@]};do cd $tool + # Drop stale configure output (e.g. host paths after repo move or COPY into Docker). + rm -rf CMakeCache.txt CMakeFiles Makefile cmake_install.cmake CTestTestfile.cmake install_manifest.txt cmake . make -j4 make install diff --git a/install-qclient.sh b/install-qclient.sh index 642477d7..f4c1bcb0 100755 --- a/install-qclient.sh +++ b/install-qclient.sh @@ -7,12 +7,27 @@ # Check if the script is run with sudo privileges if [ "$EUID" -ne 0 ]; then - echo "This script must be run as root (use sudo) to install the Quilibrium client to /var/quilibrium/ directory" + echo "This script must be run as root (use sudo) to install the Quilibrium client under the system install root and create /usr/local/bin/qclient" exit 1 fi BASE_URL="https://releases.quilibrium.com" +# Legacy pre-FHS-split install root. Used only to detect existing +# installs so we can warn the user; new installs do not land here. +LEGACY_QCLIENT_BIN_DIR="/var/quilibrium/bin/qclient" + +# default_install_root prints the OS-appropriate default install root +# for qclient. Matches DefaultQClientInstallDir() in client/utils/paths.go: +# Linux: /opt/quilibrium (FHS) +# macOS: /usr/local/quilibrium (Homebrew-style) +default_install_root() { + case "$(uname -s)" in + Darwin) echo "/usr/local/quilibrium" ;; + *) echo "/opt/quilibrium" ;; + esac +} + # Function to detect OS and architecture detect_os_arch() { OS=$(uname -s | tr '[:upper:]' '[:lower:]') @@ -93,6 +108,7 @@ download_release_file() { # Parse command line arguments DRY_RUN=false +INSTALL_ROOT_OVERRIDE="" while [[ "$#" -gt 0 ]]; do case $1 in --dry-run) @@ -100,6 +116,27 @@ while [[ "$#" -gt 0 ]]; do echo "[DRY RUN] enabled" shift ;; + --install-dir) + INSTALL_ROOT_OVERRIDE="$2" + shift 2 + ;; + --install-dir=*) + INSTALL_ROOT_OVERRIDE="${1#*=}" + shift + ;; + -h|--help) + cat </bin/qclient//. + --dry-run Print actions without downloading or modifying files. + -h, --help Show this help. +EOF + exit 0 + ;; *) echo "Unknown option: $1" exit 1 @@ -115,12 +152,37 @@ echo "Detected OS and architecture: $OS_ARCH" LATEST_VERSION=$(get_latest_release "$OS_ARCH") echo "Latest release version: $LATEST_VERSION" -# Download binary, digest, and signatures +# Resolve the install root. Precedence: +# 1. --install-dir flag +# 2. OS-appropriate default (Linux: /opt/quilibrium, macOS: /usr/local/quilibrium) +if [ -n "$INSTALL_ROOT_OVERRIDE" ]; then + INSTALL_ROOT="$INSTALL_ROOT_OVERRIDE" +else + INSTALL_ROOT="$(default_install_root)" +fi -INSTALL_DIR="/var/quilibrium/bin/qclient/$LATEST_VERSION" +QCLIENT_BIN_DIR="$INSTALL_ROOT/bin/qclient" +INSTALL_DIR="$QCLIENT_BIN_DIR/$LATEST_VERSION" + +echo "Install root: $INSTALL_ROOT" +echo "QClient binary dir: $QCLIENT_BIN_DIR" + +# Warn if a pre-FHS-split install is still on disk. Files are NOT moved. +if [ "$QCLIENT_BIN_DIR" != "$LEGACY_QCLIENT_BIN_DIR" ] && [ -d "$LEGACY_QCLIENT_BIN_DIR" ]; then + echo + echo "Notice: a legacy qclient install was detected under $LEGACY_QCLIENT_BIN_DIR." + echo " This install will use the new default ($QCLIENT_BIN_DIR)." + echo " Files under $LEGACY_QCLIENT_BIN_DIR are NOT moved automatically;" + echo " remove them manually once you've verified the new install." + echo +fi # Ensure the install directory exists -sudo mkdir -p "$INSTALL_DIR" +if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] mkdir -p $INSTALL_DIR" +else + sudo mkdir -p "$INSTALL_DIR" +fi # Get the list of release files for the detected OS and architecture echo "Fetching release files for $OS_ARCH..."