problem
currently, several client libraries in the Typelevel ecosystem (like Ember, Skunk, and Rediculous) implement their own individual pooling logic for TCP connections. This leads to duplicated effort and inconsistent behavior across the ecosystem.
I am proposing TCP connection pool library designed specifically for these client libraries to share.i am seeking design advice from the community . my goal is to build a shared API that handles pooling, error conditions, and metrics consistently.
I have started the project using sbt new typelevel/typelevel.g8 and have built a basic connection pool that successfully holds connections.
Steps I have taken so far
1.PoolTracer: implemented a tracking mechanism using a trait and console logger to monitor real-time pool activity, such as additions, removals, and total available connections.
2.ConnectionStore: Built the core storage layer using cats.effect.Ref and Map .
3.TcpConnectionFactory: Created a factory to establish real fs2.io.net.Socket[IO] connections, demonstrating how the library integrates with the FS2 Network module.
4.ConnectionStoreSuite (Tests): Developed an MUnit test suite that verifies the logic using both dummy strings and real local TCP server sockets to prove the store can safely hold and retrieve active socket references.
Speaking more specifically about the connection pool, I need help from the community with these questions
1.can this pool support both HTTP/1.1 and HTTP/2?
2.What should the interface look like for client libraries to integrate with this pool?
3.What's the best internal state manager? internally, to manage the queue of waiting clients and idle connections, what is the most robust approach? Should we rely on cats.effect.std.Queue combined with Semaphore, or is a custom state machine backed by cats.effect.Ref and Deferred the better way to go for performance and fairness?
These are the immediate questions that came to my mind as I started building the core logic, and I would really appreciate the community's guidance on them. As the project progresses, I will definitely be looking for more feedback and help to ensure the pool's design perfectly aligns with the ecosystem.
problem
currently, several client libraries in the Typelevel ecosystem (like Ember, Skunk, and Rediculous) implement their own individual pooling logic for TCP connections. This leads to duplicated effort and inconsistent behavior across the ecosystem.
I am proposing TCP connection pool library designed specifically for these client libraries to share.i am seeking design advice from the community . my goal is to build a shared API that handles pooling, error conditions, and metrics consistently.
I have started the project using sbt new typelevel/typelevel.g8 and have built a basic connection pool that successfully holds connections.
Steps I have taken so far
1.PoolTracer: implemented a tracking mechanism using a trait and console logger to monitor real-time pool activity, such as additions, removals, and total available connections.
2.ConnectionStore: Built the core storage layer using cats.effect.Ref and Map .
3.TcpConnectionFactory: Created a factory to establish real fs2.io.net.Socket[IO] connections, demonstrating how the library integrates with the FS2 Network module.
4.ConnectionStoreSuite (Tests): Developed an MUnit test suite that verifies the logic using both dummy strings and real local TCP server sockets to prove the store can safely hold and retrieve active socket references.
Speaking more specifically about the connection pool, I need help from the community with these questions
1.can this pool support both HTTP/1.1 and HTTP/2?
2.What should the interface look like for client libraries to integrate with this pool?
3.What's the best internal state manager? internally, to manage the queue of waiting clients and idle connections, what is the most robust approach? Should we rely on cats.effect.std.Queue combined with Semaphore, or is a custom state machine backed by cats.effect.Ref and Deferred the better way to go for performance and fairness?
These are the immediate questions that came to my mind as I started building the core logic, and I would really appreciate the community's guidance on them. As the project progresses, I will definitely be looking for more feedback and help to ensure the pool's design perfectly aligns with the ecosystem.