ADR 003: Workflow Orchestration via Handler Services

[monkeypants]

Summary

Cherry-picks ADR 003 from the super-branch (PR #56).

Documents the Handler pattern for decoupled use case orchestration:

• Handlers have domain interfaces (accept domain objects, not requests)
• Use cases hand off to handlers without knowing what happens next
• "Green-dotted-egg" principle for separation of concerns
• Acknowledgement semantics (wilco/roger)
• Fine-grained vs coarse-grained handlers
• Cross-BC coordination via composition

Related

Closes #62
Cherry-picked from PR #56

Test plan

• ADR content matches super-branch
• Index updated

[monkeypants]

Question: Which BC Owns the Handler Protocol?

The ADR shows handler protocols in {bc}/services/, but doesn't address a subtlety: what if the entity being handed off is defined in a different BC than the use case doing the handoff?

For example, if CreateStoryUseCase lives in BC-X but Story is defined in BC-Y, where does OrphanStoryHandler live?

Two options:

(A) With the entity being handed off — OrphanStoryHandler lives in the BC that defines Story

(B) With the use case that hands off — OrphanStoryHandler lives wherever CreateStoryUseCase lives

---

Proposal: Option A (with the entity)

Rationale:

• Per ADR 009, protocols are organized by the entity types they're semantically bound to
• A handler protocol signature like handle(story: Story) is bound to Story
• Multiple use cases might hand off to the same handler protocol (e.g., CreateStory and ImportStory both might produce orphan stories)
• The handler protocol represents "what can I do with this entity after a condition is recognized" — that's entity-centric, not use-case-centric
• Dependency graph consideration: Option A never introduces new dependencies — the use case BC already depends on the entity BC. Option B may add new vertices to the dependency graph (the entity BC would need to depend on the use case BC to access the protocol).

The ADR should clarify this placement rule explicitly.

[monkeypants]

Dispatcher Pattern for Handler Orchestration

Problems with Current Design

1. Dependency Rule Tension

The ADR shows coarse-grained handlers that call use cases:

class StoryOrchestrationHandler:
    def __init__(self, orchestration_use_case: StoryOrchestrationUseCase):
        self._use_case = orchestration_use_case

The handler implementation must know:

• Which use case(s) to call
• How to construct use case request types
• How to interpret responses

This creates structural coupling. Where does this handler live? It has dependencies on both the handler protocol (domain) and the use case API (application). Cross-BC handlers are worse — they depend on multiple BCs.

2. Bootstrapping / Instantiation Ordering

Handlers need use cases. Use cases need handlers. The DI container must instantiate them in the right order:

1. Instantiate UseCase B
2. Instantiate Handler (inject UseCase B)
3. Instantiate UseCase A (inject Handler)

This works for linear chains but creates complexity for cross-BC wiring and makes the dependency graph fragile.

---

Proposed Design: Handler Dispatcher

Introduce a HandlerDispatcher at the composition root that routes handler protocol invocations to implementations.

Components:

Component	Location	Responsibility
Handler Protocol	BC domain (services/)	Declares the intent shape
BC-local Handler	BC infrastructure	Handles without calling use cases (logging, notifications)
OrchestrationHandler	Core infrastructure (generic)	Routes to use cases via factories
HandlerDispatcher	Composition root	Routes protocol invocations to implementations
Proxy	Generated by dispatcher	Satisfies protocol, delegates to dispatcher

OrchestrationHandler receives use case factories, not instances:

# Core infrastructure - generic
class OrchestrationHandler:
    def __init__(
        self,
        routes: list[tuple[
            Callable[[], UseCase],   # use case factory (callable)
            Callable[..., Request],  # request builder
        ]],
    ):
        self._routes = routes
    
    async def handle(self, *args, **kwargs) -> Acknowledgement:
        for get_use_case, build_request in self._routes:
            use_case = get_use_case()  # lazy instantiation
            request = build_request(*args, **kwargs)
            await use_case.execute(request)
        return Acknowledgement.wilco()

DI composes existing factories:

# Composition root (dependencies.py)
def get_handler_dispatcher() -> HandlerDispatcher:
    dispatcher = HandlerDispatcher()
    
    # BC-local handler (no use case knowledge)
    dispatcher.register(OrphanStoryHandler, get_slack_orphan_handler())
    
    # Orchestration handler (composes use case factories)
    dispatcher.register(
        OrphanStoryHandler,
        OrchestrationHandler(
            routes=[
                (get_assign_to_epic_use_case, lambda story: AssignToEpicRequest(story_slug=story.slug)),
                (get_notify_team_use_case, lambda story: NotifyRequest(message=f"Orphan: {story.slug}")),
            ]
        ),
    )
    return dispatcher

# Use case receives proxy from dispatcher
def get_create_story_use_case() -> CreateStoryUseCase:
    return CreateStoryUseCase(
        get_story_repository(),
        post_create_handler=get_handler_dispatcher().proxy_for(OrphanStoryHandler),
    )

---

Why This Works

1. Dependency rule satisfied — The DI container already knows about use case implementations (it wires repositories/services to use cases). Extending it to know about inter-use case orchestration is cohesive.

2. Bootstrapping solved — Use case factories (callables) are passed, not instances. Lazy instantiation at handle() time eliminates ordering constraints.

3. Multiple use cases per handler — One handler invocation can trigger multiple use cases (add tuples to routes).

4. BC-local handlers unchanged — Handlers that don't call use cases (logging, Slack, etc.) remain simple infrastructure implementations in the BC.

5. Cross-BC wiring explicit — The composition root is where BC boundaries are bridged, which is already its job.

[absoludity]

So this one confused me up until the last part where it explicitly talks about the polling app. I've left my (confused) comments in place, as it might help see why... I think it's because something is being proposed as a general pattern (using a generic "Story" use case) which is really (?) just for a specific case like the contrib polling app where a re-usable use-case needs to be wired up to orchestrate another use-case or workflow?

[absoludity]

Hmm, this sounds like yet another layer of complexity? Ah, or is it actually just a service in the normal sense?

[monkeypants]

it is a service, putting the word "handler" in front of it is documentation: it's a service that is responsible for knowing what to do with these things (i.e. has responsibility for orchestration, or whatever). So the usecase logic is "oh, I have one of these things, I'll make it the problem of the that-type-of-thing handler"

[absoludity]

So are they just services? If so, I think this ADR would be much clearer if they were, from the outset, talked about as a specific type of service. A service that implements a specific protocol handle method returning an Acknowledgement, which would be extended below, rather than written from scratch? Not sure.

[monkeypants]

yes, they are just services (with a specific type of responsibility). Orchestration is done by services (that are responsible for orchestration). So the DI container is responsible for composing orchestration by choosing and injecting the right kind of handlers (services).

[absoludity]

Do they? I can't see a next_action anywhere? I think the polling contrib app had something like that just because it's a re-usable piece, but I don't see any general principle like is described here?

[monkeypants]

oh, sorry - that was true at a point in my branch, when I did it wrong, before I figured out this way was better.

[absoludity]

I don't think this got updated before merging - which means the ADR has an incorrect context which is pretty confusing (at least to me). Can we make this context accurate? Same on the following lines. More info in a comment below about what I've understood was the actual context (whether or not that's correct)

[absoludity]

Just a note that it's not clear in my head the difference between the orchestration we have/had without this Handler pattern (ie. just with temporal workflows / pipelines), versus the orchestration that we get with it. Normally use cases workflows would coordinate between domain entities and external services without knowing implementation detail. I don't really see what this is solving?

[monkeypants]

usecase are part of a bounded context, and they deal in the logic of the bounded context. If they have to know about other bounded contexts we broke abstraction, possibly rather badly. So it's really about orchestrating between usecases, especially if those usecases are in other bounded contexts. The DI container/application is outside bounded contexts, with dependencies pointing inwards (towards usecases, repository and service implementations). That's got to be the thing that knows about orchestation, if we are to preserve dependency rule. So the question becomes "how does it do that", and "by choosing the right service implementations" is the nicest answer I could think of, it's kind of the same answer as every other question about "how does the application do that".

[absoludity]

A temporal workflow's run method can do this though? That is, it can call separate use-cases from different bounded contexts, orchestrating those calls, without any dependency issues - I think? It can even be configured to trigger other pipelines.

That's exactly what the polling app's NewDataDetectionPipeline workflow (which we use) is doing now. The polling app's NewDataDetectionPipeline is run with the downstream pipeline as a string argument:

julee/src/julee/contrib/polling/apps/worker/pipelines.py

Lines 108 to 113 in a157b89

	@workflow.run
	async def run(
	self,
	config: PollingConfig | dict[str, Any],
	downstream_pipeline: str | None = None,
	) -> dict[str, Any]:

which is then run when new data is detected:

julee/src/julee/contrib/polling/apps/worker/pipelines.py

Lines 237 to 241 in a157b89

	downstream_triggered = await self.trigger_downstream_pipeline(
	downstream_pipeline,
	previous_data,
	current_content,
	)

The NewDataDetectionPipeline workflow doesn't need to know anything about the other bounded context, other than it can trigger the supplied pipeline with the previous and current data from the polling result. The downstream pipeline to trigger is configured in the rba's worker when it starts the polling.

Currently in practise, we trigger a bridge pipeline (PollingDataPreparationPipeline) which itself just figures out what changed in the polled data and then triggers the CEAP pipeline with the new data (location). So it is the bridge which understands both bounded contexts (the results of the polling and what the CEAP pipeline requires as input).

Which means, the whole context section and decision section above is, I think, incorrect (it wasn't that the use cases required knowledge of other bounded contexts, unless I'm missing something?)

I'm still not 100%, but I think what you're doing with the handler, is being able to encode that bridge pipeline into a service handler instead, which would be configured to figure out what changed in the polled data and trigger the CEAP pipeline. That is neater I agree, and removes the need for that intermediate pipeline, which is nice, but that's quite different to the context and decision of this ADR.

Let me know if I'm missing anything!

[absoludity]

I think "Unable" is the standard radio communication semantic for what you describe here? "Roger" is, like your parenthetical, "received and understood" but doesn't imply any compliance or lack thereof).

[monkeypants]

that's an improvement! "Unable" is the compliment of Wilco, it means "will not comply" (but not because I hate you, because I am unable to, it's nice like that). Roger is different (it makes no commitment). Maybe we need all 3 responses?

[absoludity]

As above, I don't think "roger" should be used for declining the request. Actually, why use radio phraseology at all? (I mean, it's kind of fun, "unable" is explicit, but "wilco" not necessarily). The Acknowledgement class above looks much like a Response object where your status code is just true or false. But I guess you don't want to call it a Response because that could conflate something else? So if we're using Acknowledgement then we could just acknowledge in the affirmative or negative? Might be simpler.

[monkeypants]

I don't remember exactly TBH, but I think I had a potentially non-empty list of log messages (with different log levels), and I wanted to pass them back FYI to the caller, and I realised that a non-empty list didn't imply success or failure - I needed to be explicit about that. The roger/wilco was shortand method for producing the ACK with the right properties - you don't have to use them, you can explicitly instantiate the ACK exactly how you want it.

[absoludity]

Is that true? I mean, would a use-case not take different action based on Acknowledgement.will_comply. Ah - I think specifically here you mean that, assuming will_comply is true, the use case won't know or care whether it's processed immediately or later or whatever.. So maybe:

Suggested change

	The use case doesn't know or care.
	The use case doesn't know or care about how it is handled, only whether it is handled.

[monkeypants]

yes, that's clearer.

[absoludity]

Ah - perhaps the polling example was the trigger for this ADR?

[monkeypants]

Definitely one of them. Also the need for more compositional ones - where each usecase can serve multiple purposes, and should only know how to do it's job (not tell other usecases how to do their jobs). I was also looking at identity resolution / entity graph creation, estimating identifier types from context, etc. These atomic usecases that made sense in isolation but needed to be wired into larger processes to be really useful, and which we do actually need in multiple situations.

[absoludity]

I went back and had a look at the workflows and how we're using them after our call yesterday. I still think the context and decision here need to be updated (as they confuse why this change was made - basing it on incorrect information, I think).

[absoludity]

I don't think this got updated before merging - which means the ADR has an incorrect context which is pretty confusing (at least to me). Can we make this context accurate? Same on the following lines. More info in a comment below about what I've understood was the actual context (whether or not that's correct)

[absoludity]

A temporal workflow's run method can do this though? That is, it can call separate use-cases from different bounded contexts, orchestrating those calls, without any dependency issues - I think? It can even be configured to trigger other pipelines.

That's exactly what the polling app's NewDataDetectionPipeline workflow (which we use) is doing now. The polling app's NewDataDetectionPipeline is run with the downstream pipeline as a string argument:

julee/src/julee/contrib/polling/apps/worker/pipelines.py

Lines 108 to 113 in a157b89

	@workflow.run
	async def run(
	self,
	config: PollingConfig | dict[str, Any],
	downstream_pipeline: str | None = None,
	) -> dict[str, Any]:

which is then run when new data is detected:

julee/src/julee/contrib/polling/apps/worker/pipelines.py

Lines 237 to 241 in a157b89

	downstream_triggered = await self.trigger_downstream_pipeline(
	downstream_pipeline,
	previous_data,
	current_content,
	)

The NewDataDetectionPipeline workflow doesn't need to know anything about the other bounded context, other than it can trigger the supplied pipeline with the previous and current data from the polling result. The downstream pipeline to trigger is configured in the rba's worker when it starts the polling.

Currently in practise, we trigger a bridge pipeline (PollingDataPreparationPipeline) which itself just figures out what changed in the polled data and then triggers the CEAP pipeline with the new data (location). So it is the bridge which understands both bounded contexts (the results of the polling and what the CEAP pipeline requires as input).

Which means, the whole context section and decision section above is, I think, incorrect (it wasn't that the use cases required knowledge of other bounded contexts, unless I'm missing something?)

I'm still not 100%, but I think what you're doing with the handler, is being able to encode that bridge pipeline into a service handler instead, which would be configured to figure out what changed in the polled data and trigger the CEAP pipeline. That is neater I agree, and removes the need for that intermediate pipeline, which is nice, but that's quite different to the context and decision of this ADR.

Let me know if I'm missing anything!