RFC: Inferlet Package Manager

[ingim]

RFC: Inferlet Package Manager

• Feature name: inferlet-package-manager
• Start date: 2025-10-07
• RFC PR: (TBD)
• Tracking issue: (TBD)

Summary

Introduce a first-class way to discover, version, and reuse inferlets in Pie via an identifier and registry model. Users reference inferlets by stable IDs and versions instead of uploading binaries per run. This aims to improve reuse, reproducibility, and discoverability while reducing operational overhead.

Motivation

• Reuse & productivity: Inferlets are often general-purpose; packaging encourages sharing and reduces boilerplate.
• Operational efficiency: Avoid repeated uploads and reduce storage bloat.
• Discoverability: A searchable index helps users find the right inferlet faster.

Example

As a user, you can:

• Run an inferlet by ID and version:

  pie run ingim:tree-of-thought@0.1.0 -- --num-branch 10 --max-depth 3

• Discover inferlets by name/keywords and inspect metadata (description, inputs/outputs, compatibility).
• Publish a compiled inferlet with associated metadata so others can find and use it.

Example CLI interactions (non-normative):

pie inferlet search "react"
pie inferlet info ingim:react
pie inferlet get ingim:react@^1
pie inferlet publish --wasm ./dist/react.wasm --meta ./inferlet.yaml

Open Items

Concepts

• Inferlet Identifier: A namespace-qualified name plus a version, e.g. namespace:name@version.
• Namespaces delineate ownership; versioning follows SemVer-like conventions.
• Artifact: A compiled WASM binary plus a minimal manifest describing metadata, I/O, and engine compatibility.
• Registry: A logical service that stores/serves inferlet artifacts and metadata.
• A default public registry exist (e.g., pie-project/inferlet-index); private registries may be supported.

Execution

• The engine fetches/verifies the artifact, checks compatibility (engine version, WASI features), and executes.

Manifests (minimum expectations)

A manifest SHOULD include:

• Human metadata (name, description, authors)
• Identifier and version
• Declared inputs/outputs (for CLI and UI help)
• Engine compatibility hints (minimum engine version, required features)
• Optional provenance (repository, license)
• Optional integrity data (checksums/signatures)

The precise schema and formats are left open; one canonical schema will be proposed during implementation.

Drawbacks

• Additional surface area (registry, resolver, metadata) increases complexity.
• Governance and moderation for public namespaces can be non-trivial.
• Versioning policy introduces expectations around compatibility.

[ingim]

@zyma98 what do you think about ditching the namespace in Bakery?
I.e., inferlet names would be globally unique.
Having namespaces, in my opinion, has two problems:

• Rust crates and Python packages do not use namespaces. It will become tricky to expose bindings with namespaces.
• It adds complexity to the package manager, package names, etc.

Many package managers (almost everything I know except npm and maven) use global namespaces without any issues.
Also, it would make Pie.toml 's dependency section cleaner, as we can just adopt Rust's syntax.

[ingim]

Proposed manifest format (inherits Cargo.toml semver conventions):

[package]
name = "package-name"
version = "0.1.0"
description = "Package description"
authors = ["Name <e@mail.com>"]
repository = "https://github.com/pie-project/pie"
readme = "README.md"

[runtime]
core = "0.2.0"
mcp = "0.2.0"

[parameters]
prompt = { type = "string", description = "The user message to complete" }
system = { type = "string", optional = true, description = "System prompt to set assistant behavior" }
max_tokens = { type = "int", optional = true, description = "Maximum tokens to generate (default: 256)" }
temperature = { type = "float", optional = true, description = "Sampling temperature (default: 0.6)" }
top_p = { type = "float", optional = true, description = "Top-p nucleus sampling (default: 0.95)" }

[dependencies]
foo = "0.1.0"
bar = "0.1.0"

The runtime section defines the WIT interface between the inferlet and the engine.

[zyma98]

Looks very good to me!

[ingim]

I've also built the preliminaries for bindings serving in the registry. During the inferlet publish procedure, the registry server now:

1. Runs wasm-tools component wit ... to extract the WIT from the uploaded Wasm binary
2. Finds the package named pie:{inferlet_name} and extracts all its interfaces
3. Stores the extracted interfaces (if any) to the database

Before we move on to wit-bindgen for generating language-specific bindings, I'd like to discuss a couple of things:

1. Package naming convention for exported interfaces

Since we're using global namespaces, I think pie:{inferlet_name} is unique and clean. it keeps all inferlets under the pie namespace. Thoughts?

2. The run interface problem

Currently, run is exported as part of the core pie package. The issue: since all inferlets export this interface via pie:core, other inferlets can't import another inferlet's run.

I see a few options:

• Keep the current approach. run is only meant for the engine. We can consider runtime inferlet launching (multiprocessing) later.
• Move run under pie:{inferlet_name}. Cleaner, but I'm not sure how to make this work with inferlet-macros.

[zyma98]

1. pie:{inferlet_name} sounds good to me.
2. I prefer moving run under pie:{inferlet_name}. I believe we can somehow let the macro pull required information from Pie.toml. I'm experimenting on this.

[zyma98]

Here's the proof of concept code that we can pull the package name from Pie.toml in the main macro.

macro-test.zip

[ingim]

I guess this is why we love Rust!

[zyma98]

Indeed, the procedural macro is so versatile.