Add a guide for usage with multiple server instances

[MatthewWid]

Add a guide to the documentation on using the library with multiple server instances.

This seems to be a fairly common query from users trying to architect their application with server redundancy in mind (1, 2, 3, 4).

---

In general, the most scalable approach is to use some external pub-sub or message broker solution (Eg, Redis, RabbitMQ, NATS, etc.) and for each subscription to a topic you create a dedicated channel.

Then, when you receive an incoming request, simply register its session to the corresponding channel that the user is requesting to be subscribed to.

Now, when you want to broadcast messages, instead of calling Channel#broadcast you instead publish a message to the messaging service directly, triggering all of your subscribed channels to rebroadcast the event across all server instances.

---

An example flow would be the following:

1. Server boots and connects to Redis.
2. Server subscribes to the stock-updates topic and creates and stores a new channel associated with the name stock-updates.
3. Client makes a request to GET /subscribe?name=stock-updates.
4. Server checks that a channel exists with the name stock-updates, then creates a session and registers it to the channel.
5. Some service (maybe the Server itself) publishes to the stock-updates Redis topic.
6. Server receives the event from Redis and calls the broadcast method on the channel associated with stock-updates.
7. Client receives the broadcasted event.

The documentation should include sample code for a few of the most popular pub-sub and/or message broker technologies.

---

See this comment for more details.

[Georgeab9]

Hey @MatthewWid , thanks for the hard work, the package is very well structured with a clear mental model.

I'm trying to have a deeper understanding of the implementation you mention for more advance use cases.

2.Server subscribes to the stock-updates topic and creates and stores a new channel associated with the name stock-updates

In the use case of "rooms" where a channel is a roomId and connected clients are users who are in the /:roomId route, the subscription should trigger only once the first GET request to /:roomId route is made ? And would we need to initialize 1 Redis subscriber client or 1 for each room ? So the flow for, let's say an event new-user where all users in the same roomId should be notified i.e. UI update indicates new user added would be :

1. Server boots and connects to Redis client. (Singleton publisher instance in each machine).
2. First GET request by GUEST1 registered to room with id 123 ( first user to navigates and submit create username form to /rooms/123)
3. New channel created rooms:room123 (or any other notations) and New session created for GUEST1 which is registered in the channel rooms:room123
4. New Redis subscriber client initialized and subscribe to the channel
5. Redis publisher instance publishes event "new-user" to channel rooms:room123 and calls broadcast method on the channel
6. All clients on route /rooms/123 receive the new event (connected via EventSource)
7. GUEST2 navigates to /rooms/123 and submits create username form
8. Server checks that channel already exists for /rooms/123 then creates a session for GUEST2 and registers the session to the channel rooms:room123
9. Redis publisher instance publishes event "new-user" to channel rooms:room123 and calls broadcast method on the channel
10. All clients on route /rooms/123 receive the new event (GUEST1 only in this example)

Would be interested to see your persepctive whenever you have time to check this.. also added some visual to illustrate the example

[MatthewWid]

@Georgeab9 Thanks for your interest!

Just some clarification on terminology so we don't get confused:

• Redis: Subscribers subscribe to topics, receiving messages that are published by publishers.
• Better SSE: Sessions register with channels, receiving events that are broadcasted.

I understand that doesn't necessarily agree with how the Redis docs use those words, but it's how I like to mentally separate them because there are a lot of overlapping terms going on here.

In the use case of "rooms" where a channel is a roomId and connected clients are users who are in the /:roomId route, the subscription should trigger only once the first GET request to /:roomId route is made ?

You can either create the subscription pre-emptively when the server boots or lazily the first time a user makes a request for a previously-unused room ID.

The example workflow in the original issue shows the former, with a topic name that is static and known ahead of time (stock-updates) so we can set up the subscription on server boot to save some time not having to set it up during the first time a user makes a request for it.

When the topic name is dynamic and given by the user (rooms:roomXXX) like in your use case then yes, it would be fine to only create the channel and set up the subscription the first time we receive a request for a new room ID (and perhaps consider unsubscribing once the last session is deregistered from the channel, but whether that is worth it depends on what you expect your traffic patterns to be like.)

And would we need to initialize 1 Redis subscriber client or 1 for each room ?

A Redis client can subscribe or unsubscribe to one or many topics at any time, so per server instance you only need one client that manages all your subscriptions and one client that manages all your publishing (RESP3 even allows publishing from a subscribing client, so you'd only need one client to handle both, but this is currently unsupported by Node Redis.)

On server boot, you create one Redis client for subscription and another for publishing. When a request is received for a new room ID you create one subscription for it using the subscription-client and create one channel associated with the room ID. When another request is received for the same room ID, you reuse the subscription by instead simply registering the session with the existing channel.

So the flow for, let's say an event new-user where all users in the same roomId should be notified i.e. UI update indicates new user added would be :

What you've written there is mostly correct, though for #4 you would just create a subscription using the existing client that was created on boot, not create a whole new client for it.

Here's how I would write it (a bit more verbose):

1. Server boots and creates singleton Redis subscriber-client and singleton Redis publisher-client.
2. GUEST1 makes request to GET /rooms/123.
3. Server creates session for GUEST1 and checks if rooms:room123 exists (it does not.)
4. Server creates new channel associated with rooms:room123 and registers GUEST1's session with it.
5. Server uses Redis subscriber-client to create new subscription to rooms:room123 and sets up its listener to broadcast events on the new channel.
6. Server uses Redis publisher-client to publish new-user message to Redis channel rooms:room123.
7. Server Redis subscriber-client receives message and broadcasts it on the associated channel.
8. All clients (including GUEST1*) connected to GET /rooms/123 via EventSource receive the event.
9. GUEST2 makes request to GET /rooms/123.
10. Server creates session for GUEST2 and checks if rooms:room123 exists (it does!)
11. Server registers GUEST2's session with the channel associated with rooms:room123.
12. Server uses Redis publisher-client to publish new-user message to Redis channel rooms:room123.
13. Server Redis subscriber-client receives message and broadcasts it on the associated channel.
14. All clients connected to GET /rooms/123 via EventSource (GUEST1 and GUEST2*) receive the event.

* The new session is already registered by this point so they, too, would receive the event. You could add some meta-data to the published message about who published it and then filter them out in the subscription listener, or otherwise just publish new-user messages before the new users' session is registered with the channel.

Your diagram is correct, but just remember that Subscriber1, Subscriber2, etc. would be subscriptions created with the same Redis client, not separate clients created for each subscription.

Let me know if I've misunderstood anything.

[Georgeab9]

Thank you so much for the detailed and clear explanation. I will try to implement this in my use case, hopefully i am able to do it properly.

Appreciate your support ! 🙏