Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ jobs:
args: release --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ dist/

# profiles hold passwords
profiles.json

.playwright-mcp
35 changes: 35 additions & 0 deletions .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,38 @@ changelog:
sort: asc
filters:
exclude: ["^docs:", "^test:", "^chore:"]

# Package managers. Both publish to repos owned by the same account, which the default
# GITHUB_TOKEN cannot write to, so the release workflow passes a PAT as
# HOMEBREW_TAP_GITHUB_TOKEN (contents: write on homebrew-tap and scoop-bucket).
homebrew_casks:
- name: dbclone
repository:
owner: phuthuycoding
name: homebrew-tap
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Casks
homepage: https://phuthuycoding.github.io/dbclone/
description: Clone MongoDB and MySQL databases between any two connections through local Docker containers
license: MIT
binaries:
- dbclone
commit_author:
name: goreleaserbot
email: bot@goreleaser.com

scoops:
- name: dbclone
repository:
owner: phuthuycoding
name: scoop-bucket
branch: main
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: bucket
homepage: https://phuthuycoding.github.io/dbclone/
description: Clone MongoDB and MySQL databases between any two connections through local Docker containers
license: MIT
commit_author:
name: goreleaserbot
email: bot@goreleaser.com
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,13 @@ Checking local setup
## Install

```bash
go install github.com/phuthuycoding/dbclone@latest
brew install phuthuycoding/tap/dbclone # macOS (Homebrew)
scoop bucket add phuthuycoding https://github.com/phuthuycoding/scoop-bucket && scoop install dbclone # Windows
go install github.com/phuthuycoding/dbclone@latest # any OS with Go
```

or download a binary from [Releases](https://github.com/phuthuycoding/dbclone/releases).
or download a binary from [Releases](https://github.com/phuthuycoding/dbclone/releases)
(linux / macOS / windows, amd64 / arm64).

## Usage

Expand Down
35 changes: 35 additions & 0 deletions docs/launch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Lịch launch dbclone (bắt đầu 2026-10-07)

Nguồn và rule từng nền tảng: `docs/reference/2026-10-07-launch-platforms.md`.
Text từng nơi: các file cùng thư mục. **Show HN và Reddit anh phải tự viết lại bằng lời
mình** (HN cấm text AI, Reddit phạt post giống marketing); file chỉ là fact sheet.

## Tuần 1 — chuẩn bị

- [x] Repo `homebrew-tap` + `scoop-bucket` tạo xong; goreleaser có `homebrew_casks` + `scoops`.
- [ ] **Anh:** tạo PAT (fine-grained, Contents: Read & write trên `homebrew-tap` và
`scoop-bucket`) rồi `gh secret set HOMEBREW_TAP_GITHUB_TOKEN -R phuthuycoding/dbclone`.
- [ ] Merge PR https://github.com/phuthuycoding/dbclone/pull/1 (nhánh `phuthuycoding/feat/launch-prep`), tag `v0.3.0` → release tự đẩy cask + manifest.
- [ ] Kiểm tra `brew install phuthuycoding/tap/dbclone` trên máy Mac thật.
- [ ] GIF demo (vhs qua Docker) chèn README + site.
- [x] Reddit đã login (Google) trong Chrome profile phuthuycoding.
- [ ] Comment thật ở r/golang, r/devops, HN mỗi ngày (không link dbclone).
- [x] PR awesome-mongodb mở rồi: https://github.com/ramnes/awesome-mongodb/pull/177 (mục Tools › Data).
- [ ] Email Console.dev (hello@console.dev) + Golang Weekly (editor@cooperpress.com): text sẵn trong `emails.md`; Gmail compose bị đóng trước khi autosave nên chưa có draft, anh copy gửi tay.
- [x] Bài dev.to **đã publish 2026-10-07**: https://dev.to/phuthuycoding/cloning-a-staging-mongodb-and-mysql-to-your-laptop-without-installing-a-single-database-client-12dm (tag go/docker/mongodb/mysql, AI disclosure = AI-assisted).

## Tuần 2 — launch (giờ VN)

- [ ] Thứ Ba 2026-10-13, 20:00–22:00: Show HN (`show-hn.md`), trực thread ~6 tiếng.
- [ ] Cùng tối: publish dev.to, post X + LinkedIn kèm GIF (`social.md`).
- [ ] Thứ Năm 2026-10-15: r/golang.
- [ ] Thứ Sáu 2026-10-16: comment trong r/selfhosted "New Project Friday".
- [ ] Thứ Bảy 2026-10-17: r/devops. Thứ Hai 19/10: r/mongodb. Thứ Tư 21/10: r/mysql.
- [ ] 2026-10-20..21: gom feedback thành issues, retro.

## Sau đó

- ~2026-11-01: đủ 30 ngày tuổi cho homebrew-core; cần 225 sao (tự submit) hoặc 75 (người khác).
- Có invite Lobsters: post tag `show` + `go` + `databases`, giữ < 25% self-promo.
- 2027-01-02: r/selfhosted post riêng. 2027-03-02: awesome-go (coverage ≥ 90%).
- Driver PostgreSQL → resubmit Show HN.
18 changes: 18 additions & 0 deletions docs/launch/awesome-lists.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Curated lists

## ramnes/awesome-mongodb — PR now (no star/age gate; one item per PR; alphabetical)

Section: Tools (or "Backup"/"Migration" if a closer fit on the day). Entry:

- [dbclone](https://github.com/phuthuycoding/dbclone) - CLI to clone databases or selected collections between any two servers through the official Docker image, with parallel streams and retries

## shlomi-noach/awesome-mysql — after traction ("reasonably recognized and adopted")

Section: Backup (nearest). Entry (lowercase description, no superlatives):

- [dbclone](https://github.com/phuthuycoding/dbclone) - clone databases or selected tables between servers through the official docker image, parallel mysqldump streams, retries.

## avelino/awesome-go — not before 2027-03-02 (5 months since first commit) and coverage ≥ 90%

- [dbclone](https://github.com/phuthuycoding/dbclone) - Clone MongoDB and MySQL databases between any two servers through local Docker containers.
PR body must include the pkg.go.dev link: https://pkg.go.dev/github.com/phuthuycoding/dbclone
95 changes: 95 additions & 0 deletions docs/launch/dev-to.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
title: Cloning a staging MongoDB and MySQL to your laptop without installing a single database client
published: false
tags: go, docker, mongodb, mysql
canonical_url: https://phuthuycoding.github.io/dbclone/
---

Every week someone on my team needs a copy of our staging data on their laptop. Two
databases, MongoDB and MySQL, a few gigabytes. For years the answer was a wiki page:
install `mongodump` of the right major version, install the MySQL client, run four
commands, wait, and hope the SSH session doesn't drop halfway through a 3 GB restore.

I got tired of the wiki page and wrote [dbclone](https://github.com/phuthuycoding/dbclone),
a small Go CLI. This post is about the three decisions that made it pleasant, and the one
limit you should know before you trust it.

## The trick: your Docker images already contain the tools

If you run MongoDB or MySQL locally, you almost certainly run the official `mongo` and
`mysql` images. Those images ship `mongodump`, `mongorestore`, `mysqldump` and `mysql`.
dbclone never installs anything on your host; every dump and restore is a `docker exec`
into the container you already have:

```
┌──────────────── local container ────────────────┐
source ──────────▶│ mongodump / mysqldump ──pipe──▶ mongorestore / mysql │──────────▶ target
(any profile) └──────────────────────────────────────────────────┘ (any profile)
```

Two consequences fell out of this that I did not plan for:

- **Version skew disappears.** The client that dumps is the same major version as the
server that restores, because they are the same image.
- **Any direction works.** The container is just the pipe. `staging → local` is the
common case, but `local → staging` and `staging → another server` go through the same
code path. Your local containers are simply a built-in profile called `local`.

## Streams, never stages

Each dump is piped straight into its restore. There is no dump file on disk, so there is no
"out of disk at 90%" and no cleanup step.

Big work gets its own stream. A MongoDB collection of 64 MiB or more becomes one
`mongodump --collection | mongorestore` pipe; everything smaller travels together in one
extra stream. MySQL tables are split into up to `-w` size-balanced groups, each its own
`mysqldump --single-transaction | mysql` pipe, and triggers, views, routines and events
follow once the data has landed.

All streams from all databases share one weighted pool of `-j` slots (default 8) and are
dispatched biggest first, so a slot freed by a tiny database goes straight to the large one
instead of waiting for its siblings.

```
Cloning 3 databases prod → local · pool 8 streams · max 4 streams/db

⠹ mongo:shop orders ⇉2 [==========>-------------] 42% 1.2 GiB / ~2.9 GiB 18.4 MiB/s 01:07
✓ mysql:app done [========================] 100% 235.6 MiB 57.2 MiB/s 00:04
· mysql:crm waiting for slot [------------------------] 0% 0.0 b / ~80.0 MiB
```

## Flaky links and the rails that stop you from hurting yourself

Every stream is retried three times with backoff. Because restores drop and recreate the
object, a retry starts clean instead of appending to a half-written table. Errors that no
retry can fix, such as access denied, fail at once.

Then the part I care about most, because the tool can write to a server:

- Writing to anything but `local` means typing the target's name, interactively or with
`-confirm staging`.
- `-fresh` (drop whole databases first) only works when the target is `local`.
- A server is never cloned onto itself.
- Passwords reach the tools as environment variables, never on a command line. Profiles are
stored with mode `0600`. Every log line is scrubbed of `user:password@`.

## The limit you should know

MySQL table groups each take their own `--single-transaction` snapshot. The copy is
therefore not one consistent point in time across tables. That is fine for development
data, which is what this is for. It is not a backup tool, and I would not use it as one.

## Using it

```bash
brew install phuthuycoding/tap/dbclone # or scoop, go install, or a binary from Releases
dbclone -check # Docker + containers OK? prints the fix if not
dbclone # first run: add a profile, pick FROM, TO, databases
dbclone -from staging -only mongo:shop,mysql:app -yes
```

The engine interface is one package per database (`internal/driver/<engine>/`), and
PostgreSQL is the obvious next one. If you try it on a setup that breaks, an issue with the
`logs/` folder from that run is the most useful thing you can send me.

Repo: https://github.com/phuthuycoding/dbclone — MIT.
41 changes: 41 additions & 0 deletions docs/launch/emails.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Newsletter / curated submissions

## Console.dev — hello@console.dev (selection criteria: developer is the user, maintained, good docs)

Subject: Submission: dbclone — clone MongoDB/MySQL between servers using the official Docker images

Hi,

I'd like to submit dbclone for consideration.

- What: an open-source (MIT) Go CLI that clones MongoDB and MySQL databases between any
two connections (staging → local, prod → local, local → staging), whole databases or
selected tables/collections.
- Why it's different: nothing to install on the host. Dump and restore run with the tools
inside the official `mongo`/`mysql` images already on your machine, piped directly, no
dump files. One weighted pool of parallel streams, retries with backoff, live progress.
- Docs: README with every flag and an "what gets overwritten" table; `dbclone -check`
prints the exact command to fix a missing requirement.
- Repo: https://github.com/phuthuycoding/dbclone
- Site: https://phuthuycoding.github.io/dbclone/
- Install: brew / scoop / go install / binaries for linux, macOS, windows.

Thanks,
Quyen (phuthuycoding)

## Golang Weekly — editor@cooperpress.com (no public submit form; contact page says email)

Subject: Link suggestion: dbclone — Go CLI to clone MongoDB/MySQL via Docker, weighted worker pool

Hi,

A link suggestion for Golang Weekly: dbclone, a Go CLI that clones MongoDB/MySQL
databases between servers by piping mongodump/mysqldump straight into the restore inside
the official Docker images. Interesting bits for Go readers: one weighted worker pool
shared across databases dispatched biggest-first, per-stream retries, and a driver
interface that is one package per engine.

https://github.com/phuthuycoding/dbclone (MIT)

Thanks,
Quyen
63 changes: 63 additions & 0 deletions docs/launch/reddit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Reddit posts

r/golang rules verified 2026-10-07 in the browser (rule 9 "Must be Go Related" explicitly
allows "Announcements & articles about open source Go libraries or applications"; rule 4 bans
engagement hooks; rule 6 bans "Ask an AI" posts). Other subs: see the verified block in
`docs/reference/2026-10-07-launch-platforms.md` §1.2. Keep self-posts under
10% of your activity; comment for real in these subs for 1–2 weeks first.

## r/golang — text post (technical, Go-first), Thu 2026-10-15

**Title:** dbclone: a Go CLI that pipes mongodump/mysqldump into the restore inside Docker, with a weighted worker pool

**Body (rewrite freely):**

I wrote a small CLI to clone MongoDB/MySQL databases between servers (mostly staging → my
laptop). The Go-specific parts people here might find interesting:

- The dump and restore are two `exec.CommandContext` processes run through `docker exec`,
with the dump's stdout wired to the restore's stdin. No temp files, and the progress bar
reads byte counts off the pipe.
- All streams from all databases share one weighted pool (`-j` slots, default 8). Big
Mongo collections (≥ 64 MiB) and size-balanced groups of MySQL tables each get a stream,
dispatched biggest first.
- Retries: 3 attempts with backoff per stream; restores drop-and-recreate so a retry starts
clean. Errors that can't be fixed by retrying (auth) short-circuit.
- Engines are adapters behind one `driver.Driver` interface, one package per engine. Adding
Postgres should touch nothing in scheduling/progress/prompts.
- Safety rails because this thing can write to a server: non-local targets require typing
the target's name; `-fresh` only on `local`; passwords only via env vars, never argv.

Repo: https://github.com/phuthuycoding/dbclone (MIT).

(r/golang rule 4, verified 2026-10-07: no "engagement hooks" such as "redundantly asking what
others think" — so end on the facts, not on a question.)

## r/devops — ONLY as a comment in the weekly self-promotion thread (rule 4, verified). Sat 2026-10-17

Start the comment with "I'm the author." (rule 4: affiliation at the top). 5–6 lines, your own
words (rule 5 removes LLM-sounding text). Problem (client version skew, dump files, no
progress on a 3 GB restore) → the pipe trick → the pool → the honest limit → link. No blog
link (rule 4: self-contained).

## r/mongodb — rules still unverified (login wall). Mon 2026-10-19, after checking the rules

**Title:** Open-source CLI: clone a MongoDB database (or just some collections) from staging to local, using the mongo image's own mongodump/mongorestore

Body: 4 lines. Mention per-collection streams ≥ 64 MiB, retries, picker with `/` filter.

## r/mysql — DROPPED. Rule 2 "No Self Promotion" (verified). Only mention dbclone when it
genuinely answers someone's question.

## r/selfhosted — ONLY in the "New Project Megathread" until 2027-01-02 (rule 6, verified)

Comment, Fri 2026-10-16: 3 lines. Rule 2 wants "production ready and have docs" — point at
the README flag table and `-check`. Wednesday exception (rule 5) allows tool posts with the
right flair; still a 3-month-old project is megathread-only, so wait.

## r/commandline — NOT before 2026-11-02 (rule 5: no projects newer than 30 days, verified)

Rule 4: title/text must be human-written; if a meaningful share of the code was
AI-generated the post must say "This software's code is partially AI-generated" — decide
honestly. Rule 8: list similar tools and the difference (manual mongodump/mysqldump,
pgsync / replibyte / neosync for Postgres-first setups). Post the GIF of the progress view.
61 changes: 61 additions & 0 deletions docs/launch/show-hn.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Show HN — material (REWRITE IN YOUR OWN WORDS before posting)

HN guideline: "Please don't post generated text or AI-edited text." This file is a fact sheet
and a structure, not the post. Type the title and the text yourself from it.

**URL to submit:** https://github.com/phuthuycoding/dbclone (the repo, not the website —
Show HN wants something people can run).

## Title candidates (≤ 80 chars, name the databases, name the job)

1. Show HN: Dbclone – Clone MongoDB/MySQL from prod or staging to local, Docker only
2. Show HN: Dbclone – Copy a staging MongoDB/MySQL database to your laptop in one command
3. Show HN: Dbclone – Stream MongoDB/MySQL databases between servers with nothing installed

Why this shape: Neosync's vague first post got 4 points; the relaunch titled
"…for Postgres and MySQL" got 246. Replibyte's "A tool to seed your dev database with real
data" got 129.

## Facts to build the text from (all true per README / code)

- Why I built it: our team clones a shared staging MongoDB + MySQL to laptops every week.
mongodump/mysqldump by hand meant installing matching client versions on every machine,
dump files on disk, and no idea how far along a 3 GB restore was.
- What it does: picks source and target from named profiles (`staging`, `prod`, `local`),
lists databases, lets you tick collections/tables, then streams each dump straight into the
restore. No dump files on disk.
- Nothing to install on the host: the dump/restore binaries are the ones already inside the
official `mongo` and `mysql` Docker images on your machine. Works between two remote
servers too; the local containers are just the pipe.
- Parallelism: one global pool of streams (`-j`, default 8) shared across all databases,
biggest work dispatched first. Mongo collections ≥ 64 MiB get their own stream; MySQL
tables are split into size-balanced groups (`-w`).
- Flaky links: every stream retried 3× with backoff; restores drop-and-recreate so a retry
starts clean. Auth errors fail immediately instead of retrying.
- Safety: writing to anything but `local` requires typing the target's name; `-fresh` (drop
whole DB) only works on `local`; a server is never cloned onto itself. Passwords go to the
tools via env vars, never argv; profiles are `0600`; logs are scrubbed of `user:password@`.
- Honest limits (say them; HN rewards it): MySQL table groups each take their own snapshot,
so the copy is not one consistent point in time — dev data, not a backup tool. Needs Docker
with containers from the official images. Postgres is not there yet; the driver interface is
one package per engine and it is the obvious next one.
- Install: `brew install phuthuycoding/tap/dbclone`, Scoop, `go install`, or a binary from
Releases (linux/macOS/windows, amd64/arm64). MIT.

## Suggested structure (6–8 lines, first person)

1. One sentence: what it is and the exact pain.
2. One sentence: the trick (dump piped into restore inside the official images).
3. Two sentences: scheduling + retries, with the numbers.
4. One sentence: the safety rails.
5. One sentence: what it is NOT (consistency, backup).
6. Ask: what engine next, what breaks on your setup.

## Day-of checklist

- Post Tue–Thu, 20:00–22:00 Vietnam time (morning US Pacific). Not an official HN rule;
it is where the audience is awake.
- Stay in the thread for ~6 hours. Answer every comment, including the harsh ones.
- Do NOT ask anyone to upvote. Do not post the link in team chats with "please upvote".
- If it does not take off, it is allowed to resubmit later on a big release (Postgres
driver). lazydocker hit the front page 4 times over 5 years.
Loading
Loading