Skip to content

Add MTCL backend and multinode integration tests - #223

Merged
GlassOfWhiskey merged 37 commits into
masterfrom
mtcl-backend
Sep 24, 2026
Merged

GlassOfWhiskey merged 37 commits into
masterfrom
mtcl-backend

Conversation

@marcoSanti

@marcoSanti marcoSanti commented Apr 13, 2026 •

Copy link
Copy Markdown
Member
  • Add MTCL remote backend with dynamic discovery, connection lifecycle management, asynchronous sends, and framed request/file transfers.
  • Extend the backend API to send request metadata and file data as one ordered operation while preserving existing backend behavior.
  • Harden remote request parsing and read handling, including malformed requests, zero-length reads, bounded byte counts, and correct file offsets.
  • Replace the legacy integration test executable with split, map, and merge programs executed across three containers.
  • Add Docker Compose multinode coverage to CI, improve failure log reporting, and make the test image configurable by base image.
  • Add unit coverage for owned remote requests and compound backend sends.

@marcoSanti
marcoSanti marked this pull request as draft April 13, 2026 14:58
@codecov

codecov Bot commented Apr 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 26.51685% with 327 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.12%. Comparing base (ea6aab0) to head (848e695).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
capio/server/src/mtcl_backend.cpp 32.91% 44 Missing and 115 partials ⚠️
capio/tests/unit/server/src/remote_backend.cpp 0.00% 58 Missing ⚠️
capio/tests/unit/server/src/cli_parser.cpp 0.00% 29 Missing ⚠️
capio/server/include/remote/listener.hpp 25.71% 13 Missing and 13 partials ⚠️
capio/tests/unit/server/src/none_backend.cpp 0.00% 14 Missing ⚠️
capio/server/src/mpi_backend.cpp 0.00% 5 Missing and 8 partials ⚠️
capio/tests/unit/server/src/storage_manager.cpp 0.00% 11 Missing ⚠️
capio/tests/unit/server/src/main.cpp 0.00% 7 Missing ⚠️
capio/tests/unit/server/src/capio_file.cpp 0.00% 6 Missing ⚠️
capio/server/src/remote_request.cpp 87.50% 0 Missing and 2 partials ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #223      +/-   ##
==========================================
- Coverage   59.81%   55.12%   -4.69%     
==========================================
  Files          81       89       +8     
  Lines        3175     3951     +776     
  Branches     1973     2507     +534     
==========================================
+ Hits         1899     2178     +279     
- Misses        984     1134     +150     
- Partials      292      639     +347     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

server_thread.join();
remote_listener_thread.join();

delete backend;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed as this was duplicated (in signals.hpp it is already removed) and some times it was cause for a segfault during termination (double free)

@marcoSanti
marcoSanti force-pushed the mtcl-backend branch 4 times, most recently from ef344cf to 7479343 Compare August 13, 2026 09:29
= and others added 18 commits September 18, 2026 09:35
Added NoBackend as default CAPIO backend

Split backend declaration and implementation

minor fixes

Fixes to tests

Fixes to no_backend

test

fix

const refs

fix to segfault

Messages

Messages

bugfix

format

format

Final classes
…hmetics

WIP: MTCL backend

WIP backend

WIP: mtcl backend

Fixed handshake

First alpha implementation
WIP: rewriting in progress of MTCL backend control logic for cleaner
code
@marcoSanti
marcoSanti force-pushed the mtcl-backend branch 8 times, most recently from 6cd27b9 to 65808d8 Compare September 24, 2026 12:48
@marcoSanti
marcoSanti force-pushed the mtcl-backend branch 2 times, most recently from f4acfbe to 64422f5 Compare September 24, 2026 13:26

@GlassOfWhiskey GlassOfWhiskey left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@GlassOfWhiskey
GlassOfWhiskey merged commit 0a7cd9c into master Sep 24, 2026
41 checks passed
@GlassOfWhiskey
GlassOfWhiskey deleted the mtcl-backend branch September 24, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants