Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/sandbox-capabilities.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
- mosaic
- namespace
- northflank
- opencomputer
# - opencomputer
- runloop
- run-cloud
- sandbox0
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sandbox-dax-benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ jobs:
- mosaic
- namespace
- northflank
- opencomputer
# - opencomputer
- run-cloud
- runloop
- sail
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sandbox-tti-benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ jobs:
- mosaic
# - namespace
- northflank
- opencomputer
# - opencomputer
# - quilt
# - railway
- run-cloud
Expand Down
48 changes: 24 additions & 24 deletions benchmarks/sandbox/providers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,14 @@ import { givemeanode } from '@computesdk/givemeanode';
import { hopx } from '@computesdk/hopx';
import { isorun } from '@computesdk/isorun';
// import { lelantos } from '@computesdk/lelantos';
import { lightning } from '@computesdk/lightning';
// import { lightning } from '@computesdk/lightning';
import { modal } from '@computesdk/modal';
import { microsandbox } from '@computesdk/microsandbox';
import { miosa } from '@computesdk/miosa';
import { mosaic } from '@computesdk/mosaic';
import { namespace } from '@computesdk/namespace';
import { northflank } from '@computesdk/northflank';
import { opencomputer } from '@computesdk/opencomputer';
// import { opencomputer } from '@computesdk/opencomputer';
// import { quilt } from '@computesdk/quilt';
// import { railway } from '@computesdk/railway';
import { runCloud } from '@computesdk/run-cloud';
Expand Down Expand Up @@ -144,19 +144,19 @@ export const providers: ProviderConfig[] = [
// requiredEnvVars: ['LELANTOS_API_KEY'],
// createCompute: () => lelantos({ apiKey: process.env.LELANTOS_API_KEY! }),
// },
{
name: 'lightning',
requiredEnvVars: ['LIGHTNING_API_KEY'],
// Lightning sizes sandboxes via `instanceType` on the provider factory (the SDK
// ignores an instanceType passed to sandbox.create()), so it can't be sized through
// DAX_RESOURCE_OPTIONS like most providers. The dax workflow sets
// LIGHTNING_INSTANCE_TYPE=cpu-8 for the standardized 8 vCPU / 16 GiB profile;
// it defaults to cpu-1 (the SDK default) everywhere else.
createCompute: () => lightning({
apiKey: process.env.LIGHTNING_API_KEY!,
instanceType: process.env.LIGHTNING_INSTANCE_TYPE || 'cpu-1',
}),
},
// {
// name: 'lightning',
// requiredEnvVars: ['LIGHTNING_API_KEY'],
// // Lightning sizes sandboxes via `instanceType` on the provider factory (the SDK
// // ignores an instanceType passed to sandbox.create()), so it can't be sized through
// // DAX_RESOURCE_OPTIONS like most providers. The dax workflow sets
// // LIGHTNING_INSTANCE_TYPE=cpu-8 for the standardized 8 vCPU / 16 GiB profile;
// // it defaults to cpu-1 (the SDK default) everywhere else.
// createCompute: () => lightning({
// apiKey: process.env.LIGHTNING_API_KEY!,
// instanceType: process.env.LIGHTNING_INSTANCE_TYPE || 'cpu-1',
// }),
// },
{
name: 'modal',
requiredEnvVars: ['MODAL_TOKEN_ID', 'MODAL_TOKEN_SECRET'],
Expand Down Expand Up @@ -202,15 +202,15 @@ export const providers: ProviderConfig[] = [
runtime: 'node',
}),
},
{
name: 'opencomputer',
requiredEnvVars: ['OPENCOMPUTER_API_KEY', 'OPENCOMPUTER_API_URL'],
createCompute: () => opencomputer({
apiKey: process.env.OPENCOMPUTER_API_KEY!,
apiUrl: process.env.OPENCOMPUTER_API_URL!,
}),
sandboxOptions: { timeout: 600_000 },
},
// {
// name: 'opencomputer',
// requiredEnvVars: ['OPENCOMPUTER_API_KEY', 'OPENCOMPUTER_API_URL'],
// createCompute: () => opencomputer({
// apiKey: process.env.OPENCOMPUTER_API_KEY!,
// apiUrl: process.env.OPENCOMPUTER_API_URL!,
// }),
// sandboxOptions: { timeout: 600_000 },
// },
Comment on lines +205 to +213

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 OpenComputer benchmark command always fails

Running pnpm bench:opencomputer selects the removed opencomputer participant. resolveParticipants finds no participant and aborts the command.

Learn more

The package command remains exposed at bench:opencomputer, but tti.bench.ts now receives a participant list without opencomputer. Explicit participant selection produces an empty list, and resolveParticipants throws NoAvailableParticipantsError before executing tasks.

Example: A contributor runs pnpm bench:opencomputer using the repository's named command. The runner receives --provider opencomputer, selects zero participants, and exits instead of benchmarking OpenComputer.

Recommended fix: Remove or disable the bench:opencomputer package script alongside this provider entry. Restore both together when OpenComputer is re-enabled.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, though this matches an existing pattern in the repo: bench:railway (and bench:render, bench:just-bash) still exist even though those providers are commented out of providers.ts, so a bench:* script pointing at a disabled provider already fails the same way today.

Keeping bench:opencomputer makes re-enabling a single-file uncomment; happy to remove it if we want the disable to be more permanent.

// {
// name: 'quilt',
// requiredEnvVars: ['QUILT_API_KEY', 'QUILT_BASE_URL'],
Expand Down
Loading