From c0b9d92d23f6e59b798a20c4074429ea931c733c Mon Sep 17 00:00:00 2001 From: JohnnyT Date: Mon, 5 Oct 2026 05:40:53 -0600 Subject: [PATCH] Adds an explanation of why the language is its own A new explanation page, docs/explanation/why-a-predicate-language-of- its-own.md, sets out who writes a rule and who executes it, the alternatives turned down (rules in code, evaluating the text as Elixir, a tree walker calling host functions by name, a rule format of the application's own), what the language buys, why it compiles to instructions, and what it costs. Examples are in the library loan. The page is an extra in the Explanation group and ships in the package files; the README's Documentation map links it under Understand. No library code changes. Refs: px-wwd7 --- README.md | 1 + .../why-a-predicate-language-of-its-own.md | 158 ++++++++++++++++++ mix.exs | 9 +- 3 files changed, 166 insertions(+), 2 deletions(-) create mode 100644 docs/explanation/why-a-predicate-language-of-its-own.md diff --git a/README.md b/README.md index 6207f65..5abe616 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ process's state or in a database column until the next loan comes back. - [API reference](https://hexdocs.pm/predicator/api-reference.html): every public module and function. - [The changelog](CHANGELOG.md): what changed in each version. - Understand + - [Why a predicate language of its own](docs/explanation/why-a-predicate-language-of-its-own.md): the alternatives turned down - rules in code, `eval`, host calls by name, a JSON rule format - and what the choice costs. - [Architecture](docs/architecture.md): the compilation pipeline, the component map, and the design decisions behind them. - [Cross-language siblings](docs/architecture.md#cross-language-siblings): how implementations in other languages adopt each instruction-set version. - [Why `=` is assignment, never equality](https://github.com/riddler/predicator-ex/blob/main/docs/adr/0002-the-equals-grammar-break.md): the grammar break and what it left untouched. diff --git a/docs/explanation/why-a-predicate-language-of-its-own.md b/docs/explanation/why-a-predicate-language-of-its-own.md new file mode 100644 index 0000000..5537f96 --- /dev/null +++ b/docs/explanation/why-a-predicate-language-of-its-own.md @@ -0,0 +1,158 @@ +# Why a predicate language of its own + +A library lends copies of books to patrons, and the rules around a loan change +more often than the application that enforces them: how many times a loan may +be renewed, when a patron with unpaid fines stops borrowing, how long an +overdue copy waits before it is declared lost. The people who know those rules +are circulation staff, not the developers. This page is about why Predicator +answers that situation with a small language of its own, which alternatives it +turned down, and what the choice costs. + +## Who writes a rule, and who executes it + +A rule such as `status == 'on_loan' AND renewals < 3 AND NOT on_hold` passes +through three hands. A member of staff writes it in a settings form. The +application stores it. Later, every time a patron asks to renew, a process the +author never sees evaluates it against that loan. Nobody reviews the text +between the first step and the last, and nothing assumes it was written in +good faith: a typo, a misunderstanding or a hostile string arrives at the +third step exactly as it left the first. + +That is the threat model the whole package is shaped by. The process that +evaluates a rule has to stay safe against whoever wrote it, whatever they +wrote. Each alternative below is measured against that one requirement. + +## The alternatives + +### Every rule written in code + +The safest answer is to keep rules out of the data altogether: each renewal +policy is a function a developer writes, reviews and releases. Nothing a +member of staff types can reach the process, because they type nothing. + +The cost is that every policy change waits for a release, and the person who +understands the policy cannot make the change. Rules that differ by branch, by +patron type or by season multiply into code paths. For an application whose +users or configuration genuinely decide when something may happen, this +answer moves the problem rather than solving it. + +### Evaluating the text as Elixir + +The cheapest way to let staff write rules is to evaluate what they type with +`Code.eval_string/3`. It works on the first afternoon, and it hands the author +the host's whole runtime: shell commands, the file system, every module loaded +in the node. There is no subset of Elixir it can be restricted to, so a rule +field becomes a way to execute anything at all. + +### A tree walker that calls host functions by name + +A middle road parses the text into a tree and walks it, calling a host +function whenever the tree names one. If the name chooses the function - +through `apply/3` on whatever the parse produced, or a lookup across every +loaded module - the author still chooses which host code executes. The +vocabulary is smaller than with `eval`, but the vocabulary was never the +boundary. A quieter version of the same mistake is turning names from the text +into atoms: atoms are never garbage collected and the atom table is a fixed +size, so a stream of invented variable names can bring down the node without +executing any code. + +### A rule format of the application's own + +Some applications avoid text entirely: a rule is a JSON structure of field, +operator and value rows, drawn by a form and compiled to an expression only +when it is evaluated. For a form that is the right shape, but as the rule's +stored form it means two representations of one rule, two serializations, two +migration stories when a field is added, and a translator that is a second +implementation of the language's precedence rules. Predicator keeps one +language and offers the form-shaped part of it as a value computed from the +parsed tree instead; [the simple subset](../guides/simple-subset.md) describes +it, and +[the decision record](https://github.com/riddler/predicator-ex/blob/main/docs/adr/0017-structured-authoring-is-a-subset-value.md) +weighs the routes it turned down, including a restricted parser mode. + +## What a language of its own buys + +**The rule stays data at every stage.** Source text becomes tokens, tokens +become a tree, the tree becomes a flat list of instructions, and a small stack +machine interprets that list over a closed set of operations it defines +itself. No stage hands any part of the text to the host language. The pipeline +is not an implementation detail wrapped around the safety property; it is the +mechanism that provides it. + +**Functions come from the host, by its choice.** A rule can call a function, +but only one the application made available: the builtins, plus +whatever functions the host passes in. A name the rule invents comes back as an "Unknown function" +error rather than reaching for a module, and a host function that raises is +caught and turned into an error value. The author of a rule selects from what +the host offered and nothing else; [custom functions](../guides/custom-functions.md) +covers how a host offers more. + +**Failures are values.** A rule that names a value the loan does not carry, +or calls a function the host never offered, returns an error value that names +the problem; it does not raise. The reason is the same threat model applied to +failure: if bad input raised, the author of a rule would choose, by choosing +input, which exception the host process sees and how far its stack unwinds. +The bang variants (`evaluate!/3`, `compile!/1`) do raise, because there the +host asked for an exception at a call site it wrote. + +**A compiled rule is something you can keep.** The instruction list is a plain +term. It can sit in a process's state, in ETS or in a database column, and be +evaluated against each returned copy without compiling the text again. It +carries the version of the instruction set it was compiled for, so a rule +stored under an earlier release is either executed as written or refused with +a reason, never silently misread; [the embedding guide](../guides/embedding.md) +covers that check. + +**The same rule means the same thing in another language.** Because the +instruction list, not the source text, is the contract, an implementation in +another language can execute what this one compiles and prove it against a +shared conformance corpus. [The architecture page](../architecture.md#cross-language-siblings) +describes how those implementations adopt each version of the instruction set. + +## Why compile at all + +Once the language is its own, a tree walker over the parsed rule would be the +simpler interpreter, and it would get one thing free that the stack machine +had to earn: `AND` and `OR` that stop at the first decisive side, so +`renewals < 3 AND NOT on_hold` never looks at `on_hold` once the renewals are +used up. Predicator compiles anyway, and added jump instructions to make the +operators short-circuit, because the compiled list is what makes a rule +storable and what other languages share. A tree is tied to this +implementation's data structures; a list of instructions with a version +number is an artifact. +[The decision record](https://github.com/riddler/predicator-ex/blob/main/docs/adr/0001-keep-the-stack-vm-revise-the-instruction-set.md) +sets out that trade in full. + +## What it costs + +Every feature of the language is built rather than borrowed. Elixir already +has a tokenizer, a parser, operator precedence, arithmetic, comparison rules +and a standard library, and none of them can be used: Predicator carries its +own lexer, parser, compiler and evaluator to re-earn a fraction of them, and +its own function set to re-earn a standard library one function at a time. +Every new operator or literal is work that `eval` would have supplied for +nothing, and that bill is paid on every feature. + +The same rule refuses a whole class of convenient requests: a lambda inside a +rule, a call into an arbitrary host module, a pattern compiled from rule text, +a template that interpolates into code. Each would be easy to add and each +would reopen the question the package exists to close. A condition that needs +more than the language offers belongs in a host function the application +writes and reviews, which the rule then calls by name. + +It is also one more language for the people who write rules to learn. The +grammar leans on what they already know - comparisons, `AND`, `OR`, `NOT`, +`in`, dates and durations - and departs from it deliberately in places, most +visibly in reading `=` as assignment rather than equality; +[that decision record](https://github.com/riddler/predicator-ex/blob/main/docs/adr/0002-the-equals-grammar-break.md) +explains the break. + +## Related reading + +- [The language reference](../reference/language.md) lists every operator, + function, data type and error shape the language has. +- [The architecture page](../architecture.md) maps the compilation pipeline + component by component. +- [The decision record on no `eval`](https://github.com/riddler/predicator-ex/blob/main/docs/adr/0004-no-eval-errors-are-values.md) + states the threat model this page starts from, and the boundaries of the + errors-as-values rule. diff --git a/mix.exs b/mix.exs index 402c225..f7a42bc 100644 --- a/mix.exs +++ b/mix.exs @@ -92,6 +92,7 @@ defmodule Predicator.MixProject do docs/guides/simple-subset.md docs/guides/porting.md docs/architecture.md + docs/explanation/why-a-predicate-language-of-its-own.md docs/contributing.md ), exclude_patterns: [~r{\Alib/predicator/conformance/}], @@ -132,7 +133,8 @@ defmodule Predicator.MixProject do "docs/guides/nested-data-access.md", "docs/guides/location-expressions.md", "docs/guides/simple-subset.md", - "docs/architecture.md" + "docs/architecture.md", + "docs/explanation/why-a-predicate-language-of-its-own.md" ], groups_for_extras: [ "How-to guides": [ @@ -148,7 +150,10 @@ defmodule Predicator.MixProject do "docs/guides/location-expressions.md", "docs/guides/simple-subset.md" ], - Explanation: ["docs/architecture.md"] + Explanation: [ + "docs/architecture.md", + "docs/explanation/why-a-predicate-language-of-its-own.md" + ] ], skip_undefined_reference_warnings_on: ["CHANGELOG.md"] ]