Summary
The cases/ directory contains numerical oracle data (finite-difference and PyTorch references) for scalar operations, but lacks documentation of the analytic derivative formulas (frule/rrule) that implementations should follow.
Motivation
chainrules-scalarops in tenferro-rs is being extended to provide frule/rrule for all standard scalar operations. Having the canonical formulas documented here ensures:
- A single source of truth for derivative correctness across implementations
- Clear specification for complex-valued Wirtinger derivatives (∂f/∂z̄ convention)
- Easy review of edge cases (e.g., branch cuts, singularities at zero)
Scope
Add a document (e.g., docs/scalar-ad-formulas.md) covering at minimum:
Unary real/complex
| op |
primal |
frule: df = ? |
rrule: ∂L/∂x = ? |
| sqrt |
√x |
dx / (2√x) |
ḡ / (2√x) |
| exp |
eˣ |
eˣ · dx |
ḡ · eˣ |
| expm1 |
eˣ−1 |
eˣ · dx |
ḡ · eˣ |
| log |
ln x |
dx / x |
ḡ / x |
| log1p |
ln(1+x) |
dx / (1+x) |
ḡ / (1+x) |
| sin |
sin x |
cos(x) · dx |
ḡ · cos(x) |
| cos |
cos x |
−sin(x) · dx |
−ḡ · sin(x) |
| tan |
tan x |
dx / cos²(x) |
ḡ / cos²(x) |
| tanh |
tanh x |
(1−tanh²x) · dx |
ḡ · (1−tanh²x) |
| asin |
arcsin x |
dx / √(1−x²) |
ḡ / √(1−x²) |
| acos |
arccos x |
−dx / √(1−x²) |
−ḡ / √(1−x²) |
| atan |
arctan x |
dx / (1+x²) |
ḡ / (1+x²) |
| sinh |
sinh x |
cosh(x) · dx |
ḡ · cosh(x) |
| cosh |
cosh x |
sinh(x) · dx |
ḡ · sinh(x) |
| asinh |
arcsinh x |
dx / √(x²+1) |
ḡ / √(x²+1) |
| acosh |
arccosh x |
dx / √(x²−1) |
ḡ / √(x²−1) |
| atanh |
arctanh x |
dx / (1−x²) |
ḡ / (1−x²) |
| conj |
x̄ |
d̄x |
ḡ̄ |
Binary
| op |
frule |
rrule (∂L/∂x, ∂L/∂y) |
| add |
dx + dy |
(ḡ, ḡ) |
| sub |
dx − dy |
(ḡ, −ḡ) |
| mul |
dx·ȳ + dy·x̄ |
(ḡ·ȳ, ḡ·x̄) |
| div |
dx/ȳ − dy·x̄/y² |
(ḡ/ȳ, −ḡ·x̄/y²) |
| powf |
n·x^(n−1)·dx |
ḡ·n·x^(n−1) |
| powi |
n·x^(n−1)·dx |
ḡ·n·x^(n−1) |
Notes to include
- Complex convention: Wirtinger calculus with conj() on partial derivatives where needed
- Edge cases / singularities (sqrt at 0, log at 0, etc.)
- Relationship to existing oracle test data in
cases/
Blocked by
- tensor4all/tenferro-rs chainrules-scalarops Phase 2a (adding the missing frule/rrule implementations)
Once the implementations are finalized, the formulas here should match exactly.
Summary
The
cases/directory contains numerical oracle data (finite-difference and PyTorch references) for scalar operations, but lacks documentation of the analytic derivative formulas (frule/rrule) that implementations should follow.Motivation
chainrules-scalaropsin tenferro-rs is being extended to provide frule/rrule for all standard scalar operations. Having the canonical formulas documented here ensures:Scope
Add a document (e.g.,
docs/scalar-ad-formulas.md) covering at minimum:Unary real/complex
df = ?∂L/∂x = ?Binary
Notes to include
cases/Blocked by
Once the implementations are finalized, the formulas here should match exactly.