Skip to content

README.md

Latest commit

 

History

History
374 lines (269 loc) · 15.1 KB

README.md

File metadata and controls

374 lines (269 loc) · 15.1 KB
Raw

Midnight.js

CI License TypeScript Node.js GitHub Release PRs Welcome

TypeScript application development framework for the Midnight blockchain. Similar to Web3.js for Ethereum or polkadot.js for Polkadot.

Installation

yarn add @midnight-ntwrk/midnight-js

Quick Start

import { deployContract } from '@midnight-ntwrk/midnight-js-contracts';
import { levelPrivateStateProvider } from '@midnight-ntwrk/midnight-js-level-private-state-provider';
import { indexerPublicDataProvider } from '@midnight-ntwrk/midnight-js-indexer-public-data-provider';
import { httpClientProofProvider } from '@midnight-ntwrk/midnight-js-http-client-proof-provider';
import { FetchZkConfigProvider } from '@midnight-ntwrk/midnight-js-fetch-zk-config-provider';
import { setNetworkId } from '@midnight-ntwrk/midnight-js-network-id';
import type { MidnightProviders } from '@midnight-ntwrk/midnight-js-types';

// Set network
setNetworkId('testnet');

// Configure providers
const zkConfigProvider = new FetchZkConfigProvider('https://artifacts.example.com');

const providers: MidnightProviders = {
  privateStateProvider: levelPrivateStateProvider({
    privateStoragePasswordProvider: () => 'your-secure-password',
    accountId: 'user-wallet-address'
  }),
  publicDataProvider: indexerPublicDataProvider(
    'https://indexer.example.com/graphql',
    'wss://indexer.example.com/graphql'
  ),
  zkConfigProvider,
  proofProvider: httpClientProofProvider('https://proof-server.example.com', zkConfigProvider),
  walletProvider,    // from @midnight-ntwrk/wallet-sdk
  midnightProvider,  // from @midnight-ntwrk/wallet-sdk
};

// Deploy a contract
const deployed = await deployContract(providers, {
  compiledContract: myContract,
  privateStateId: 'my-state',
  initialPrivateState: { counter: 0n }
});

// Call a circuit
const result = await deployed.callTx.increment();

Features

Standard Blockchain Operations

  • Creating and submitting transactions
  • Interacting with wallets
  • Querying block and contract state
  • Subscribing to chain events

Privacy-Preserving Utilities

  • Executing smart contracts locally
  • Incorporating private state into contract execution
  • Persisting, querying, and updating private state
  • Creating and verifying zero-knowledge proofs

Packages

Package Description
@midnight-ntwrk/midnight-js Barrel package for Midnight.js core framework
@midnight-ntwrk/midnight-js-types Shared types, interfaces, and provider contracts
@midnight-ntwrk/midnight-js-contracts Contract deployment and interaction utilities
@midnight-ntwrk/midnight-js-dapp-connector-proof-provider Proof provider delegating to DApp Connector wallet
@midnight-ntwrk/midnight-js-indexer-public-data-provider GraphQL-based blockchain data provider
@midnight-ntwrk/midnight-js-level-private-state-provider Encrypted LevelDB private state storage
@midnight-ntwrk/midnight-js-http-client-proof-provider HTTP client for proof server
@midnight-ntwrk/midnight-js-fetch-zk-config-provider Browser-based ZK artifact retrieval
@midnight-ntwrk/midnight-js-node-zk-config-provider Node.js filesystem-based ZK artifact retrieval
@midnight-ntwrk/midnight-js-network-id Network identifier management
@midnight-ntwrk/midnight-js-logger-provider Pino logger wrapper for diagnostics
@midnight-ntwrk/midnight-js-compact Compact compiler manager
@midnight-ntwrk/midnight-js-protocol Version-agnostic re-exports of Midnight protocol packages
@midnight-ntwrk/midnight-js-utils General utility functions

Architecture

Deployment Diagram

Element Package
Contracts @midnight-ntwrk/midnight-js-contracts
PublicDataProvider @midnight-ntwrk/midnight-js-indexer-public-data-provider
PrivateStateProvider @midnight-ntwrk/midnight-js-level-private-state-provider
ProofProvider @midnight-ntwrk/midnight-js-http-client-proof-provider or @midnight-ntwrk/midnight-js-dapp-connector-proof-provider
ZKConfigProvider @midnight-ntwrk/midnight-js-fetch-zk-config-provider
DappConnector @midnight-ntwrk/dapp-connector-api
Wallet @midnight-ntwrk/wallet
Ledger @midnight-ntwrk/ledger
Compact Runtime @midnight-ntwrk/compact-runtime
Impact VM @midnight-ntwrk/onchain-runtime

Transaction Flow

Providers

Midnight.js uses a provider pattern for modularity. Each provider implements an interface from @midnight-ntwrk/midnight-js-types:

interface MidnightProviders {
  privateStateProvider: PrivateStateProvider;  // Private state storage
  publicDataProvider: PublicDataProvider;      // Blockchain queries
  zkConfigProvider: ZKConfigProvider;          // ZK artifact retrieval
  proofProvider: ProofProvider;                // ZK proof generation
  walletProvider: WalletProvider;              // Transaction balancing
  midnightProvider: MidnightProvider;          // Transaction submission
  loggerProvider?: LoggerProvider;             // Optional logging
}

Contracts

Compiling a Compact smart contract with compactc produces a JavaScript file (circuit execution logic) and a TypeScript declaration file (type definitions). Midnight.js uses these to execute circuits locally and create type-safe transactions. See Contract Compilation Details for more.

Security

The LevelPrivateStateProvider encrypts all private state data at rest using:

Parameter Value
Algorithm AES-256-GCM
Key Derivation PBKDF2-SHA256
Iterations 600,000
Password Minimum 16 characters

Development

Prerequisites

  • nvm - Node.js version management
  • direnv (optional) - Automatically sets COMPACTC_VERSION, activates Node via nvm, and configures GPG signing

Setup

# If using direnv (sets environment automatically)
direnv allow

# Install dependencies
yarn install

# Build
yarn build

# Test
yarn test

# Lint
yarn lint:fix

Available Scripts

Build

Script Description
yarn build Build all packages
yarn build:core Build everything except testkit-*
yarn build:testkit Build only testkit-*
yarn clean Clean all build outputs
yarn clean-build Clean then rebuild

Test

Script Description
yarn test Run all tests
yarn test:unit Run unit tests (excludes testkit)
yarn it Run integration tests
yarn e2e Run e2e tests
yarn e2e-single Run a single e2e test file

Lint & Check

Script Description
yarn lint Run ESLint
yarn lint:fix Fix linting issues
yarn check Type-check + lint + workspace constraints
yarn check:core Same as check but excludes testkit
yarn typecheck:tests Type-check test files

Release & Publish

Script Description
yarn changelog Generate/update CHANGELOG.md
yarn deploy Publish all packages
yarn deploy:core Publish core packages (excludes testkit)
yarn build:markdown-docs Generate API docs via TypeDoc

Further Reading

Contributing

Branch off main for all new features. Use Conventional Commits:

<type>[optional scope]: <description>

Types: feat, fix, docs, style, refactor, perf, test, chore, ci, build, revert

Scopes: midnight-js, testkit-js, deps, deps-dev, config, release

New functionality must include unit and integration tests (TDD — tests first). See CONTRIBUTING.md for full guidelines.

Git Hooks

  • pre-commit: Runs lint-staged
  • commit-msg: Validates commit message format
  • pre-push: Runs yarn lint and yarn typecheck:tests

Glossary

Term Description
Witness Private computation performed on the end user's device
Private State State updated by witnesses, stored on user's device
Circuit Smart contract function that can be executed and proven
ZKIR Zero-Knowledge Intermediate Representation

Detailed

Design Objectives

Type-Safety

  • Preserves contract circuit argument/return types and user-defined types (PS, Witnesses) throughout the data model
  • Infers types using infer instead of introducing additional generic parameters
  • Uses least restrictive generic constraints necessary for type safety
  • Uses any only for truly unconstrained user-supplied types, not to fix compilation issues
  • Does not require manual specification of concrete types for generic parameters in most scenarios
  • Employs branded types to distinguish domain concepts sharing the same representation (e.g., HexString vs string)

Modularity

  • Allows custom implementations of API clients via the "provider" pattern
  • Collects commonly used types in @midnight-ntwrk/midnight-js-types to standardize across applications

Interoperability

  • Isomorphic design supporting both browser and Node.js environments
  • Supports different Midnight networks (TestNet, MainNet)

Reusability

  • Builds high-level features from low-level utility functions, exporting both
  • Provides default implementation for each API client
  • Places each API client in a separate package for selective installation

Usability

  • Minimizes boilerplate required to set up a dApp
  • Supplies common sense defaults to avoid application errors

Contract Compilation Details

This section assumes familiarity with the Compact language.

When compiling a Compact smart contract with compactc, the JavaScript file contains:

  • Execution logic for each circuit in the source contract
  • Logic for constructing the contract's initial state
  • Utilities for converting on-chain contract state into JavaScript representation

The TypeScript declaration file contains:

  • Type definition for the contract (Contract)
  • Type definitions for circuits defined within the contract
  • Type definition for required witnesses (Witnesses)
  • Type definition for on-chain state (Ledger)

If the Compact source includes witness declarations, the Witnesses type is a non-empty object with a generic type parameter PS representing the private state that witnesses modify during circuit execution.

The term runtime describes the JS executable for a contract, distinct from @midnight-ntwrk/compact-runtime which provides utilities each executable uses.

Architecture Details

The Contracts element contains utilities for creating and submitting transactions. All clients are concrete instances of provider interfaces defined in @midnight-ntwrk/midnight-js-types.

Key relationships:

  • Ledger and Compact Runtime both depend on Impact VM (on-chain runtime)
  • Midnight.js and Substrate Node both depend on Ledger

This allows "rehearsing" circuit calls (running Impact VM) such that execution can be replayed on the node, and creating transactions from rehearsal results that the node accepts.

Release Process

Releases are prepared by the Release (prepare PR) GitHub Actions workflow (.github/workflows/release-prepare.yml). Trigger it manually from the Actions tab with the target version; it bumps versions in all workspaces, regenerates CHANGELOG.md, and opens a release/v<version> PR. Full CI runs on that PR and gates the merge. Merging it to main triggers the publish job in the CI workflow: once the build, unit, integration, and e2e jobs pass, it publishes packages to GitHub Packages and creates the GitHub Release (which creates the v<version> tag on the main merge commit). Publishing is gated on the same test run, so no release can ship ahead of its e2e.

Preparing a release

  1. Actions tab → Release (prepare PR) → Run workflow, from main.
  2. Enter the version (e.g. 5.1.0 or 5.1.0-beta.1).
  3. Review the opened release/v<version> PR (version bumps + changelog), wait for CI, merge.

Local fallback

If CI is unavailable, scripts/release.sh performs the same preparation locally:

# Dry-run (default): updates files only, no git operations — review with `git diff`
./scripts/release.sh 5.1.0

# Only file changes (bump + changelog), no git — what CI runs
./scripts/release.sh 5.1.0 --files-only

# Full local release: bump + changelog + branch + commit + push
./scripts/release.sh 5.1.0 --execute

# Full local release with build and tests beforehand
./scripts/release.sh 5.1.0 --execute --with-tests

Prerequisites for the local path: on main with a clean working tree, all previous release PRs merged, and GPG signing configured. After pushing, open a PR to main; merging it publishes the release.

What happens on merge to main

When the release PR is merged, the version bump on main triggers the publish job in the CI workflow. Once the build, unit, integration, and e2e jobs pass in the same run, it:

  1. Extracts release notes from CHANGELOG.md (section ## [<VERSION>])
  2. Validates and publishes packages/* to GitHub Packages
  3. Creates a GitHub Release with the extracted notes (creating the v<VERSION> tag on the merge commit)
  4. Publishes testkit-js/* if those paths changed

Resources

Terms & License

By using this package, you agree to Midnight's Terms and Conditions and Privacy Policy.

Licensed under Apache License 2.0.