main domain blocking with Unix.sleep only called inside tasks

[hyphenrf]

While attempting to port this Rust snippet, which executes in ~3 seconds + some overhead irrespective of thread count

let mut tasklist = vec![];
for _ in 1..tasks {
    let t = thread::spawn(|| thread::sleep(Duration::from_secs(3)));
    tasklist.push(t)
}
for t in tasklist { t.join().unwrap() }

to domainslib as so

let tasklist =
  Array.init tasks (fun _ -> async pool (fun _ -> Unix.sleep 10))
in
run pool (fun _ -> Array.iter (await pool) tasklist)

where tasks is user-supplied, I found that for n > num_domains + 1, the wait times start to change significantly.
e.g. at num_domains = 2, and tasks = 100, the execution time is ~33x what's expected.

I've searched the docs here for "the right way to go idle" in different wordings but found nothing on this. I'm not sure what's different with Unix.sleep or domainslib's task implementation.

[polytypic]

This is unfortunately the case with pretty much all blocking functions in the OCaml Stdlib (sleep, lock, select, ...). Those functions do not (at least not yet) work in a scheduler friendly manner, so to speak. They will instead block the current thread or the entire domain if you only have a single thread running in a domain.

Using a currently experimental domain-local-timeout mechanism for setting timeout callbacks in a scheduler friendly manner and a domain-local-await mechanism for blocking in a scheduler friendly manner, it is possible to implement a sleepf function (among many other things) that works in a scheduler friendly manner:

let sleepf seconds =
  let t = Domain_local_await.prepare_for_await () in
  let _ = Domain_local_timeout.set_timeoutf seconds t.release in
  t.await ()

As explained in the documentation of domain-local-timeout, you need to explicitly tell it that it can use the Thread and Unix libraries (to avoid depending on them directly):

let () = Domain_local_timeout.set_system (module Thread) (module Unix)

But with the above setup done, one can then implement a program that behaves closer to what is desired:

let () =
  let tasks = try int_of_string Sys.argv.(1) with _ -> 1000 in
  let seconds = try float_of_string Sys.argv.(2) with _ -> 1.0 in
  let tasklist =
    Array.init tasks (fun _ -> async pool (fun _ -> sleepf seconds))
  in
  run pool (fun _ -> Array.iter (await pool) tasklist)

Running that program (all of the above snippets as a program) takes roughly the specified number of seconds.

Note that, at the time of writing this, the default implementation of the experimental set_timeoutf is not optimized as it uses a simple list resulting in quadratic behaviour and for large number of timeouts (or tasks in this case) that will cause problems.

Note that memory usage grows rapidly with very large numbers of tasks and that can cause issues.