> Is my understanding correct in that a "Pane" is essentially a UI component?
Yes — it is essentially a UI component (and the only node that represents UI).
> And I am assuming there are tests to catch cases where a new pane is defined in the backend, but not in the SDK (or I suppose, how do you deal with any discrepancy)?
Great question! Because we use proto, the SDK and the backend share the same pane definitions. So we can't have a case where they don't match in the repo. However, SDKs that have already shipped obviously can't be updated with new panes. To handle this, we have a piece of the backend service called the "dispatcher" whose job it is to look at the initial /start request and determine which workflow should be used for that specific client.
Let's say a very old Android SDK is issuing a /start request. The dispatcher knows the version of the SDK requesting a workflow and programmatically can determine which major version of workflow graphs it supports, and pick an appropriate (older) workflow to send back where we know all panes are supported by the SDK.
We also have an escape hatch where the backend can tell an older SDK "actually, you're so old you should fallback on a webview implementation to get the latest and greatest experience".
Good question — leaving out comparison with a traditional state machine wasn't intentional so I'm glad you asked.
A state machine will typically fully describe all the state at any given node in it. i.e. there's no additional state or context not represented by the state machine itself.
In our case, we have (1) the directed graph and (2) the session state. These two are related but don't have a direct correlation. The session state accrues new values as the graph is walked. As an example, theoretically the user could walk through a graph that has a cycle and get back to an earlier node but the session state would be different. In that case the same node would be "current" but there would be a different session state the second time it was visited.
Airflow is an acyclic graph, but our user experience graphs have cycles (intentionally).
Airflow also focuses a bit more on being a multi-process data pipeline with workers rather than a way to drive user experiences with rendering, localization, user interaction, etc.
It's kept as encrypted blobs in redis that times out after some amount of inactivity (to clean up stale sessions)
> Is /event required (as data would be sent in /next?).
Nope! Not necessarily required — although there are some features (client side event tracking, location-in-graph updates on the backend) that benefit from it. We've hooked it up but no feature currently _depends_ on it being called.
> Also as I understand, Processors won't be involved until they are needed. E.g. if we have 2 panes but only the result of the first was required to be processed, would the processor be invoked after just the first pane was complete?
This is true — processors are only invoked as needed. In other words, they are only invoked if the user is traversing an edge that touches a processor. When a processor is invoked depends completely on its location within the graph. The diagram I linked to above in this thread has a stage for "invoke processor" that's only triggered if the backend walks an edge and sees a processor.
There is actually no strict correlation between a pane and a processor. Each can have any pieces of session state they want as input or output. In other words a processor declares "I want Foo ID as my input" not "I want the output of Bar Pane as my input". This helps composability.
> 1. How you implemented this on the backend? Did you consider using something like Temporal, AWS Simple Workflow, or AWS Step Functions?
Given this already a major change in our client<>server interaction model and that we didn't want to _fully_ rewrite the entire world, we didn't consider additional major changes to our existing backend services strategy (Go services + RPC) as part of this project. We had a bunch of backend logic and common code we wanted to reuse across Flex Link and the legacy endpoints, which gives us a stepwise mechanism for future refactors once the old APIs are turned down.
Once this rolls out to 100%, the new API boundaries do allow us to refactor the backend into something more like you suggest — leveraging step functions, lambdas etc if desired.
> 2. Interested in how you're avoiding (waiting) on network calls on each action. You mentioned it was coming in next post.
Yes! Full details of that would take a dedicated post. However, if you look at the "Client Rendering Loop" diagram in the post ( https://images.ctfassets.net/zucqsg1ttqqy/5kp6LibAllW1YW69jT... ) you'll see there is a "render neighbor panes" loop in the flow. At the point where we've walked the graph and encountered a pane, we can do a quick breadth-first-search to render out neighboring panes (that don't have a processor in between) and serve those all to the SDK. We all them "additional panes". The response includes a map from "action" to "additional pane" so the SDK can know whether to hit the API or use an already-provided additional pane. (this is in the diagram as well)
The nice part is, clients don't _have_ to implement this functionality. They can always hit the server each time they want the next pane. But as a performance optimization they can leverage the additional panes to render many things locally.
> 3. How are you managing multiple workflows? Does a common child graph make sense for a common workflow?
We use the expanded semantic versioning in the post for teams to have their own versioned workflow namespace. On the `workflow/start` call, we have a dispatcher that can pick which workflow to serve based on the provided configuration, SDK version, platform, experiments, etc.
We don't yet support subgraphs for common chunks of a workflow but it is definitely in our roadmap and would allow teams to share common pieces of the graph.
There certainly is some conceptual overlap in defining an experience/decision making process as a directional graph.
However, to my limited knowledge of this, BPMN doesn't quite differentiate between its activities in terms of "does this need user interaction" vs "process this thing". In our case, we have pane nodes for the "need user interaction" activity and processor nodes for the "process this thing" activity.
Yes — it is essentially a UI component (and the only node that represents UI).
> And I am assuming there are tests to catch cases where a new pane is defined in the backend, but not in the SDK (or I suppose, how do you deal with any discrepancy)?
Great question! Because we use proto, the SDK and the backend share the same pane definitions. So we can't have a case where they don't match in the repo. However, SDKs that have already shipped obviously can't be updated with new panes. To handle this, we have a piece of the backend service called the "dispatcher" whose job it is to look at the initial /start request and determine which workflow should be used for that specific client.
Let's say a very old Android SDK is issuing a /start request. The dispatcher knows the version of the SDK requesting a workflow and programmatically can determine which major version of workflow graphs it supports, and pick an appropriate (older) workflow to send back where we know all panes are supported by the SDK.
We also have an escape hatch where the backend can tell an older SDK "actually, you're so old you should fallback on a webview implementation to get the latest and greatest experience".