Too many invocations cause WasmtimeException "sgen-stw.c:77, condition ... not met"

[warappa]

Setup

I wanted to try out the limits of DotNetIsolator and changed the ConsoleSample.Program.cs file to have call runtime.Invoke(...) many more times (because you don't know how often a 3rd party code would call Invoke(...)). I also emptied the lambda body to speed up the invocation and to not have parameters, captured variables or return values have any influence.

Any non-empty lambda body causes the error to happen at an even lower invocation count.

Issue

Up to 35317 invocations there is no problem: The final output of the host is printed in the console (with impressive 0,04ms/call!).

But when incrementing numCalls by just 1 to 35318, it breaks with a WasmtimeException (see below).

Maybe this is an issue of Wasmtime itself, but I'm not sure as I'm not experienced in this kind of magic 😉.

Edited ConsoleSample

using var host = new IsolatedRuntimeHost().WithBinDirectoryAssemblyLoader();
using var runtime = new IsolatedRuntime(host);

//int numCalls = 35317; // GOOD
int numCalls = 35318; // BAD
var sw = new Stopwatch();
sw.Start();
for (var i = 0; i < numCalls; i++)
{
    runtime.Invoke(() =>
    {
    });
}
sw.Stop();
Console.WriteLine($"Done in {sw.ElapsedMilliseconds:F0}ms ({(double)sw.ElapsedMilliseconds / numCalls:F4} ms/call)");

Raised Exception

[wasm_trace_logger] * Assertion at /home/runner/work/dotnet-wasi-sdk/dotnet-wasi-sdk/modules/runtime/src/mono/mono/metadata/sgen-stw.c:77, condition `info->client_info.stack_start >= info->client_info.info.stack_start_limit && info->client_info.stack_start < info->client_info.info.stack_end' not met

Unhandled exception. Wasmtime.WasmtimeException: error while executing at wasm backtrace:
    0: 0x40754b - <unknown>!<wasm function 7705>
    1: 0x405089 - <unknown>!<wasm function 7618>
    2: 0x407a16 - <unknown>!<wasm function 7714>
    3: 0xe2e68 - <unknown>!<wasm function 709>
    4: 0x2d2c58 - <unknown>!<wasm function 4879>
    5: 0x3f7937 - <unknown>!<wasm function 7321>
    6: 0x3f77e3 - <unknown>!<wasm function 7319>
    7: 0x3f7a2b - <unknown>!<wasm function 7323>
    8: 0x3f7ab6 - <unknown>!<wasm function 7324>
    9: 0x2b7fce - <unknown>!<wasm function 4637>
   10: 0x2b7d14 - <unknown>!<wasm function 4635>
   11: 0x30b42c - <unknown>!<wasm function 5606>
   12: 0x30ade8 - <unknown>!<wasm function 5605>
   13: 0x30ac21 - <unknown>!<wasm function 5604>
   14: 0x30ab95 - <unknown>!<wasm function 5603>
   15: 0x3003b6 - <unknown>!<wasm function 5444>
   16: 0x2bd5e5 - <unknown>!<wasm function 4725>
   17: 0x2292bf - <unknown>!<wasm function 3284>
   18: 0x229110 - <unknown>!<wasm function 3283>
   19: 0x92199 - <unknown>!<wasm function 333>
   20: 0x62d24 - <unknown>!<wasm function 332>
   21: 0x367621 - <unknown>!<wasm function 6309>
   22: 0x2110a7 - <unknown>!<wasm function 3085>
   23: 0x212d90 - <unknown>!<wasm function 3101>
   24: 0x21c3f8 - <unknown>!<wasm function 3187>
   25: 0xe3202 - <unknown>!<wasm function 714>
   26: 0xe3427 - <unknown>!<wasm function 715>
   27: 0x3cd2 - <unknown>!<wasm function 43>
   28: 0x4477 - <unknown>!<wasm function 46>
   29: 0x419242 - <unknown>!<wasm function 7924>

Caused by:
    Exited with i32 exit status 1
   at Wasmtime.Function.Invoke(Span`1 argumentsAndResults, StoreContext storeContext)
   at Wasmtime.Function.<>c__DisplayClass187_0`3.<WrapFunc>b__0(T1 p0, T2 p1)
   at DotNetIsolator.IsolatedRuntime.CopyObject[T](T value) in C:\source\DotNetIsolator\src\DotNetIsolator\IsolatedRuntime.cs:line 115
   at DotNetIsolator.IsolatedRuntime.Invoke(Action value) in C:\source\DotNetIsolator\src\DotNetIsolator\IsolatedRuntime.cs:line 308
   at Program.Main(String[] args) in C:\source\DotNetIsolator\sample\ConsoleSample\Program.cs:line 17

Expected Result

There should be no upper limit in invocation count.

Additional Information

Tested on 6dbe1a3
This error also happened on previous versions of DotNetIsolator.

dotnet --info

Details

.NET SDK: Version: 8.0.100-preview.3.23178.7 Commit: e300b0e1e6

Runtime Environment:
OS Name: Windows
OS Version: 10.0.22621
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.100-preview.3.23178.7\

.NET workloads installed:
There are no installed workloads to display.

Host:
Version: 8.0.0-preview.3.23174.8
Architecture: x64
Commit: 47bad717bd

.NET SDKs installed:
3.1.426 [C:\Program Files\dotnet\sdk]
5.0.408 [C:\Program Files\dotnet\sdk]
5.0.416 [C:\Program Files\dotnet\sdk]
6.0.311 [C:\Program Files\dotnet\sdk]
7.0.100-preview.7.22377.5 [C:\Program Files\dotnet\sdk]
7.0.100-rc.1.22431.12 [C:\Program Files\dotnet\sdk]
7.0.105 [C:\Program Files\dotnet\sdk]
7.0.302 [C:\Program Files\dotnet\sdk]
7.0.400-preview.23225.8 [C:\Program Files\dotnet\sdk]
8.0.100-preview.3.23178.7 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-preview.7.22376.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-rc.1.22427.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0-preview.3.23177.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-preview.7.22375.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-rc.1.22426.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0-preview.3.23174.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.0-preview.7.22377.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.0-rc.1.22427.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.0-preview.3.23178.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
arm64 [C:\Program Files\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\arm64\InstallLocation]
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

dotnet workload list

Details

Installed Workload Id Manifest Version Installation Source --------------------------------------------------------------------------------------------- macos 13.1.462-net8-p3/8.0.100-preview.3 VS 17.7.33711.374 maccatalyst 16.2.462-net8-p3/8.0.100-preview.3 VS 17.7.33711.374 maui-windows 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374 maui-maccatalyst 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374 maui-ios 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374 wasm-tools 8.0.0-preview.3.23174.8/8.0.100-preview.3 VS 17.7.33711.374 wasm-tools-net6 8.0.0-preview.3.23174.8/8.0.100-preview.3 VS 17.7.33711.374 runtimes-windows 8.0.0-preview.3.23174.8/8.0.100-preview.3 VS 17.7.33711.374 ios 16.2.462-net8-p3/8.0.100-preview.3 VS 17.7.33711.374 android 34.0.0-preview.3.224/8.0.100-preview.3 VS 17.7.33711.374 maui-android 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374

Personal Note

That's some awesome stuff your doing here! Please keep it on!

[CalebSerafin]

I've observed that this is not only caused by too many invocations, but also by too much data transfer.
By limiting to only 1 invocation, using the following code I manually binary searched the maximum amount of bytes I could return.

I cause the same error by either: Incrementing that amount by 1; Or the sum of multiple invocations exceeds that number.

WasiConfiguration wasiConfiguration = new WasiConfiguration().WithInheritedStandardOutput().WithInheritedStandardError();
using var host = new IsolatedRuntimeHost().WithAssemblyLoader(LoadAssembly);
using var isolatedRuntime = new IsolatedRuntime(host);

// Maximum bytes without crash: 0b0111_0110_0100_1010_1110_0011 = 7752419 = 7.39 MiB
const int bytesToGenerate = 0b0111_0110_0100_1010_1110_0011;

IsolatedObject byteGenerator = isolatedRuntime.CreateObject<ReturnManyPseudoRandomByteGenerator>();
byteGenerator.Invoke<int, int, byte[]>("GenerateBytes", 1, bytesToGenerate);

class ReturnManyPseudoRandomByteGenerator {
    public byte[] GenerateBytes(int seed, int count) {
        Random random = new(seed);
        byte[] bytes = new byte[count];
        random.NextBytes(bytes);
        return bytes;
    }
}