Shapes

On-device single-stroke shape recognition for Swift, Android, and JavaScript. Draw one stroke and Shapes turns it into a clean vector shape: a line, rectangle, triangle, ellipse, or star. Everything runs locally, so the stroke never leaves the device or browser.

A small classifier proposes a shape, a geometric fitter produces the clean parameters, and the stroke is accepted only if it clears that class's calibrated gate. The result snaps to nice axes, circles, squares, and 15° rotations.

✎  a wobbly hand-drawn box   ->   Shape.rectangle(corners: [...])   clean and axis-aligned

• Features
• Swift
  • Install
  • Usage
  • Example
• Android
  • Install
  • Usage
  • Example
• JavaScript and TypeScript
  • Install
  • Usage
  • Example
• Shapes
• Model and caching
• License

Features

• Runs fully on device or in the local runtime. The stroke never leaves the machine.
• Recognizes line, rectangle, triangle, ellipse, and star, and rejects scribbles.
• Fits clean vector geometry and snaps it to axes, circles, squares, and 15° rotations.
• One and the same recognition pipeline on every platform, so results match: Core ML on Apple, LiteRT on Android and Linux, LiteRT.js in the browser.
• Small model, downloaded on demand and cached by default, or bundled for offline apps (about 0.2 MB on Apple, ~1.3 MB LiteRT); recognition is typically a few milliseconds.
• Apple bonus: one-line live snapping on a PencilKit canvas with an undo-safe preview.

Swift

Install

Requirements: iOS 16+, macOS 13+, tvOS 16+, watchOS 9+, visionOS 1+, and Swift 5.9+.

Add Shapes with Swift Package Manager:

.package(url: "https://github.com/Desert-Ant-Labs/shapes.git", from: "0.3.0")

Then add the Shapes product to your app target. Live PencilKit snapping is part of the Shapes product.

To bundle the Core ML model for fully offline Apple apps, also add ShapesCoreMLResources to your target.

Usage

Create one Shapes and reuse it. Construction is cheap and non-blocking. The model loads on first use, or earlier if you call download.

import Shapes

let shapes = Shapes()
if let shape = try await shapes.recognize(points: strokePoints) {
    switch shape {
    case let .rectangle(corners): ...       // [Point]
    case let .ellipse(center, semiMajor, semiMinor, rotation): ...
    default: break
    }
}

recognize accepts [Point] or, on Apple platforms, [CGPoint] and PencilKit PKStroke. On Apple, Shape.path gives a renderable CGPath.

Choose where the model comes from:

let shapes = Shapes()                       // managed cache, download on demand
let shapes = Shapes(directory: myModelDir)  // explicit model directory
let shapes = Shapes(bundle: myBundle)       // bundled model resources

Download ahead of time, for example from an onboarding screen:

let shapes = Shapes()
if !shapes.isDownloaded() {
    try await shapes.download { fraction in
        print("\(Int(fraction * 100))%")
    }
}

Bundle the model in an Apple app:

import Shapes
import ShapesCoreMLResources

let shapes = Shapes(bundle: ShapesCoreMLResourcesBundle.bundle)

Live PencilKit snapping (iOS/visionOS):

import Shapes

canvasView.enableShapeSnapping()   // pause while drawing to preview; lift to snap
// Offline/instant: enableShapeSnapping(using: Shapes(bundle: ShapesCoreMLResourcesBundle.bundle))

Example

SwiftUI example app

Android

Install

Requirements: Android API 31+. The AAR contains prebuilt arm64-v8a and x86_64 native libraries.

Shapes is published to Maven Central.

// settings.gradle.kts
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
    }
}

// build.gradle.kts
dependencies {
    implementation("ai.desertant:shapes:0.3.0")
}

ai.desertant:shapes downloads the model on demand and caches it under the app cache directory. To ship the LiteRT model inside your APK or app bundle instead, add the resources artifact and use Shapes.bundled():

dependencies {
    implementation("ai.desertant:shapes:0.3.0")
    implementation("ai.desertant:shapes-tflite-resources:0.3.0")
}

Usage

import ai.desertant.shapes.Point
import ai.desertant.shapes.Shapes

val shapes = Shapes(context)                 // download on demand, cached
val shape = shapes.recognize(strokePoints)   // Shape? (null if rejected)
when (shape) {
    is Shapes.Rectangle -> shape.corners
    is Shapes.Ellipse -> shape.center
    else -> {}
}
shapes.close()

recognize and download are suspend functions. Use use to close the native handle automatically:

Shapes(context).use { shapes ->
    val shape = shapes.recognize(strokePoints)
}

Download before first use:

val shapes = Shapes(context)
if (!shapes.isDownloaded()) {
    shapes.download()
}

Use an explicit model directory or bundled resources:

val cached = Shapes(context)                         // managed cache
val explicit = Shapes(context, directory = modelDir) // explicit model directory
val offline = Shapes.bundled()                       // needs shapes-tflite-resources

Example

Android example app

JavaScript and TypeScript

Install

Requirements: a browser (or browser-like) runtime with @litertjs/core (LiteRT.js). Inference runs in the browser (XNNPACK-accelerated CPU by default, optional WebGPU); in plain Node the pipeline loads but the model session is unavailable.

npm install @desert-ant-labs/shapes @litertjs/core

@litertjs/core is an optional peer dependency.

Usage

import { Shapes } from "@desert-ant-labs/shapes";

const shapes = await Shapes.load();               // download on demand, cached
const shape = await shapes.recognize(points);     // [{x, y}, ...] or [x0, y0, ...]
if (shape?.kind === "ellipse") shape.center;

Control loading:

const shapes = await Shapes.load({
  directory: "/var/cache/shapes",       // Node only, optional
  onProgress: (fraction) => console.log(fraction),
});

Bring your own LiteRT.js module, useful for browser bundlers and React Native:

import * as litert from "@litertjs/core";
import { Shapes } from "@desert-ant-labs/shapes";

const shapes = await Shapes.load({ litert, litertWasmDir: "/path/to/@litertjs/core/wasm/" });

Example

JavaScript examples

Shapes

All platforms return the same shape, discriminated by kind, or null when the stroke is rejected or degenerate:

• line(from, to)
• rectangle(corners) - four points around the perimeter
• triangle(vertices) - three vertices
• ellipse(center, semiMajor, semiMinor, rotation) - rotation in radians
• star(center, outerRadius, innerRadius, rotation, pointCount)

The field names and shape kinds are identical across Swift, Kotlin, and TypeScript. minimumConfidence (default 0) raises the classifier threshold on top of each class's calibrated gate.

Model and caching

The model artifacts are published at desert-ant-labs/shapes on Hugging Face. Each SDK pins the model revision to its own package version, and downloads are SHA-256 verified.

Default behavior:

• Swift: downloads the Core ML model on demand to a managed cache, or uses bundled ShapesCoreMLResources.
• Android: downloads the LiteRT model on demand to app cache, or uses bundled ai.desertant:shapes-tflite-resources.
• JavaScript: downloads the LiteRT model on Shapes.load() to the managed cache in Node (~/.cache/desert-ant-models/...) or browser cache storage when available.

Passing an explicit directory makes that directory the model home. Existing valid files are adopted for offline use; otherwise Shapes downloads into that directory and reuses it later.

License

Desert Ant Labs Source-Available License. Free for most apps; a commercial license is required at scale. Full terms are at the link. Licensing: licensing@desertant.ai.

Third-party data and model attributions are in THIRD_PARTY_NOTICES.md.