Skip to content

About

A Node CLI that trains a tiny deterministic character-level n-gram model live on a text file you provide, then serves a local page where dragging temperature, top-p, and top-k sliders reshapes the…

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

5 Commits

Folders and files

Repository files navigation

simplexview

A command-line tool that trains a tiny, deterministic character-level n-gram model directly from a text file you provide. It is the trainer at the core of a larger project that will let you manipulate a next-character probability simplex (temperature, top-p, top-k) and watch seeded sampling build text token by token.

So far this project implements:

  • The trainer: given a text file and a context length (order), it counts, for every context of that many preceding characters, how often each character followed it. The counts are written out as a canonical, sorted JSON model.
  • The sampling transforms (lib/sampling.js) that reshape a next-character probability distribution: temperature scaling, top-p (nucleus) filtering, and top-k filtering. These are pure functions over plain distribution arrays, independent of the trainer.
  • A local HTTP server (lib/server.js) and canvas page (public/) that serve the trained model: a JSON endpoint for a single reshaped distribution, a streaming endpoint that plays out a seeded sample token by token, and a page that projects the top candidates onto a simplex.

Install

Requires Node.js 18 or later. No external dependencies are needed — the project uses only the Node standard library.

npm install

Optionally link the CLI so simplexview is on your PATH:

npm link

Usage

Train a model from a text file and print it as JSON:

node bin/simplexview.js train path/to/text.txt

Or, if linked:

simplexview train path/to/text.txt

Options:

  • --n <order> — number of preceding characters used as context (default: 3).
  • --out <path> — write the JSON model to a file instead of stdout.

Example:

simplexview train corpus.txt --n 2 --out model.json

The output model has the shape:

{
  "order": 2,
  "startSymbol": "\u0001",
  "contexts": {
    "th": { "e": 3 },
    "he": { " ": 2, "l": 1 }
  }
}

contexts maps each observed context string to the counts of characters that followed it in the training text. Contexts near the start of the text are left-padded with the startSymbol control character so every context has a fixed length. Training is a single deterministic left-to-right pass over the input, so the same file and --n always produce byte-identical output.

Sampling transforms

lib/sampling.js exports pure functions that reshape a probability distribution. A distribution is a plain array of { token, prob } entries whose prob values sum to 1:

  • toDistribution(counts) — normalizes a map of raw counts (such as one context's entry from a trained model) into a sorted, normalized distribution.
  • applyTemperature(distribution, temperature) — raises each probability to the power 1 / temperature and renormalizes. Values below 1 sharpen the distribution toward its most likely token; values above 1 flatten it toward uniform; 1 leaves it unchanged.
  • applyTopP(distribution, p) — nucleus sampling. Keeps the smallest set of highest-probability tokens whose cumulative probability reaches p, zeroes out the rest, and renormalizes.
  • applyTopK(distribution, k) — keeps only the k highest-probability tokens, zeroes out the rest, and renormalizes.

All three reshaping functions preserve the original entry order in their output and break probability ties deterministically by ascending token, so results are reproducible for a given input.

Snapshot tests

test/snapshot.test.js pins the entire pipeline — training, the reshaping transforms, the seeded RNG, and the simplex projection — against one fixed text, context, seed, and set of slider params. It asserts the exact sampled character sequence, the exact /api/generate NDJSON output, and the exact simplex centroid and landing-vertex coordinates for every step, rounded to six decimal places. A separate test confirms a different seed changes the sampled sequence, and another confirms the pipeline itself is deterministic across repeated runs. Any change to the trainer, the sampling transforms, the RNG, or the geometry that shifts this fixed input's output — even by a rounding error — fails these tests.

Serving the simplex view

Train a model on a text file and serve it over HTTP:

simplexview serve path/to/text.txt --n 2 --port 8080

Options:

  • --n <order> — context length used to train the model before serving (default: 3).
  • --port <port> — port to listen on (default: 8080); use 0 to let the operating system pick a free port, which is printed on startup.

This starts a server (lib/server.js) with no runtime dependencies beyond Node's built-in http module:

  • GET / — a canvas page (public/index.html, public/app.js) with sliders for temperature, top-p, and top-k. Moving a slider re-fetches the reshaped distribution and redraws it; a "generate" button streams a seeded sample and animates it token by token.
  • GET /api/model — the model's order, context count, and a few sample contexts to try.
  • GET /api/distribution?context=<ctx>&temperature=<t>&topP=<p>&topK=<k> — the next-character distribution for context (must be exactly order characters long) after applying the given reshaping parameters, as JSON.
  • GET /api/generate?context=<ctx>&steps=<n>&seed=<s>&temperature=<t>&topP=<p>&topK=<k> — streams up to steps sampled tokens as newline-delimited JSON, one reshaped distribution and sampled token per line, drawn with a deterministic PRNG (lib/rng.js) seeded from seed; the same seed always reproduces the same sample.

The page projects the current candidate tokens onto a simplex (lib/geometry.js, also served at /geometry.js for the browser): each candidate gets a vertex spaced evenly around a circle, and the current distribution is drawn as an amber dot at the probability-weighted centroid of those vertices — a point that is always inside the polygon they form, since the probabilities sum to 1.

Moving a slider recomputes that reshaped distribution immediately and redraws the polygon and centroid — there is no extra step to "apply" a change. During generation, each streamed step also draws a second, green dot that animates from the centroid out to the vertex of the token the seeded RNG actually drew for that step, easing to a stop exactly on it (findVertex, tweenFrames, and easeOutCubic in lib/geometry.js), so a sample is seen landing on the simplex rather than simply appearing there.

Status

This project is built and maintained autonomously. Changes are only kept when the automated test suite passes.

About

A Node CLI that trains a tiny deterministic character-level n-gram model live on a text file you provide, then serves a local page where dragging temperature, top-p, and top-k sliders reshapes the…

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages