Streaming process logs#
Feature highlight#
A transaction method that takes minutes to complete leaves the user with no feedback until it returns. Combining a long-running transaction method with SAF GLOW events turns that silence into a live stream: the backend pushes each new log line as soon as it is produced, and the frontend displays it without polling or refreshing the page.
In this example, you learn how to:
Write the log file from a long-running transaction method so the UI stays responsive.
Persist the accumulated log content in an entity handle field so it survives a page reload.
Push every new line to the frontend on a custom event stream with
transaction.raise_event.Emit a termination event when the transaction method completes or fails.
Wire callbacks and event listeners that append the streamed lines, refresh the controls, and clear the panel.
Prerequisites#
The long-running transaction and events APIs are provided in the GLOW Engine package, which is available by default in any SAF-based solution.
Coding#
To stream process logs from the backend to the solution UI, work through the following sequence of sections.
Backend#
Create the solution definition.
Key concept — Entity handle field
An EntityHandle field references a blob managed by BDM. It is the way a step persists a file
it produces, so that the frontend can read the content back on any subsequent page render.
Declare the log file field
The log_file field on the step model persists the accumulated log content.
log_file: EntityHandle = NO_ENTITY
The log_file field is an EntityHandle that references the log file content stored via BDM. It defaults to NO_ENTITY
until the first log line is written.
Key concept — Long-running transaction method
A transaction method decorated with @long_running executes asynchronously. The UI stays
responsive while it runs, and a termination event is raised on the stream named after the
method when enable_termination_event=True is set on the @transaction decorator.
Key concept — Event stream
An event stream is a named channel between the backend and the frontend. The backend pushes
messages on it with self.transaction.raise_event(message, stream_name=...), and the frontend
subscribes to it with an event listener component.
Stream the log lines from a long-running transaction method
The generate_process_logs transaction method writes the log file and streams each new line as
it is produced. The clear_logs transaction method discards the persisted content.
@transaction(self=StepSpec(download=["log_file"], upload=["log_file"]), enable_termination_event=True)
@long_running
def generate_process_logs(self, wait_time: float) -> None:
"""
Write the current time to a file every second for a certain amount of time.
The new lines are appended to the existing ``log_file`` content, and each of them
is also sent to the frontend through the ``generate-process-logs-update`` event
stream. The transaction method can be terminated from the UI.
Parameters
----------
wait_time : float
Duration of the process, in seconds.
"""
try:
log_file = self.storage_scope.get_cached(self.log_file)
existing_logs = log_file.read_bytes()
except Exception:
existing_logs = b""
start_time = time.time()
while (time.time() - start_time) < wait_time:
current_time = f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())}\n"
existing_logs += current_time.encode()
self.log_file = self.storage_scope.store_stream(existing_logs)
self.transaction.raise_event(message=current_time, stream_name="generate-process-logs-update")
time.sleep(1)
@transaction(self=StepSpec(upload=["log_file"]))
def clear_logs(self) -> None:
"""Reset the ``log_file`` field so that the log page starts from an empty file."""
self.log_file = NO_ENTITY
The generate_process_logs method is marked @long_running so it executes asynchronously, allowing the UI to remain
responsive while it runs. The transaction decorator sets enable_termination_event=True so that a termination event is
automatically raised on the stream named after the method, generate-process-logs, when the method completes or fails.
On each iteration the method:
appends a new timestamped line to the in-memory log content;
persists the updated content by re-uploading it to storage via
store_streamand assigning the resultingEntityHandletoself.log_file;raises an event carrying only the new line (not the whole log) on the custom
generate-process-logs-updatestream, so that listeners can append it directly to what is already displayed.
The clear_logs transaction simply resets log_file to NO_ENTITY, discarding the previously stored content.
See
Mount the event listeners for how the UI subscribes to both streams;
Append the streamed logs to the panel for the callback that appends streamed lines to the log panel; and
React to the termination event for the callback that reacts to the termination event.
Frontend#
Expose the solution definition in the UI.
Build the layout
The layout function renders the Start button and the scrollable log panel.
NO_LOGS_MESSAGE = "No logs are available yet."
def layout(project: ExamplesSolution) -> html.Div:
"""Build the Process logs page layout.
The page shows a "Start" button that launches the long running
``generate_process_logs`` transaction, and a scrollable card that displays
the log file content. The log content is streamed live from the backend
via an event listener and initially populated from the persisted log file
(if any) when the page is first rendered.
"""
logs_container = dmc.Card(
[
dmc.CardSection(
dmc.Group(
children=[
dmc.Text("Logs", fw=500),
dmc.Tooltip(
dmc.ActionIcon(
DashIconify(icon="mdi:delete-sweep", width=24),
id="clear-logs-button",
color="gray",
variant="transparent",
),
label="Clear Logs",
position="left",
),
],
justify="space-between",
),
withBorder=True,
inheritPadding=True,
py="xs",
),
dmc.Space(h=20),
html.Div(
id="log_container",
children=html.Pre(
id="log_content",
children=get_process_logs_text(project),
style={"whiteSpace": "pre-wrap", "wordBreak": "break-all", "fontSize": "10px"},
),
style={
"height": "600px",
"width": "100%",
"overflowY": "scroll",
},
),
],
withBorder=True,
shadow="sm",
radius="md",
)
return html.Div(
[
html.H1("Process logs", className="display-3", style={"font-size": "40px", "font-weight": "bold"}),
dmc.Blockquote(
"Read the log files for a process and display their content in a solution UI.",
icon=DashIconify(icon="material-symbols:info", width=30),
style={"font-size": "18px", "fontStyle": "italic"},
),
dmc.Space(h=20),
dmc.Button(
"Start",
id="start_button",
variant="filled",
radius="sm",
style={
"font-size": "16px",
"width": "20%",
"background-color": "#2790F1",
},
leftSection=DashIconify(icon="streamline:startup-solid"),
),
dmc.Space(h=20),
dmc.Grid(
[dmc.GridCol(logs_container, span=9)],
grow=True,
gutter="xs",
),
],
style={"paddingLeft": "20px"},
)
The log text is rendered inside a stable html.Pre element identified by id="log_content". Keeping the log text as a
single string on a fixed component (rather than rebuilding the whole log_container tree on every update) makes it possible
for later callbacks to update the displayed logs with a simple string concatenation. The initial content is populated from
the persisted log file, if any, via Restore the log panel on page load.
Key concept — Event listener
An event listener is a frontend component that subscribes to a backend event stream. It fires a callback every time a message is raised on that stream, which removes the need to poll the backend for progress.
Mount the event listeners
The mount_event_listeners callback subscribes the page to the two backend streams.
@callback(
Output("process-logs-event-listeners-container", "children"),
Input("url", "pathname"),
)
def mount_event_listeners(project: ExamplesSolution) -> list[dict[str, Any]] | Any:
"""Mount the backend event listeners used by this page.
Creates two listeners bound to the ``basic_step``: one for the
``generate-process-logs-update`` stream that carries incremental log
lines while the transaction is running, and one for the
``generate-process-logs`` stream that carries the transaction's
termination event. Listeners are (re)mounted whenever the URL changes.
"""
step = project.steps.basic_step
return [
DashClient.create_event_listener(
step, id="generate-process-logs-update-listener", stream_name="generate-process-logs-update"
),
DashClient.create_event_listener(
step, id="generate-process-logs-termination-listener", stream_name="generate-process-logs"
),
]
Two listeners are created: generate-process-logs-update-listener receives every new log line raised on the
generate-process-logs-update stream while the transaction is running, and generate-process-logs-termination-listener
receives the termination event automatically raised on the generate-process-logs stream (named after the transaction
method) when the transaction completes or fails.
Key concept — Callback
A callback is a Dash-decorated function that fires in response to a UI event. In a SAF
solution, callbacks reach the backend through project.steps.<step_name>, read or write
fields, and invoke transaction methods. No manual HTTP calls are needed.
Start the transaction method from the frontend
The start_generate_process_logs_transaction callback starts the generate_process_logs
transaction method.
@callback(
Output("notification-container", "sendNotifications", allow_duplicate=True),
Input("start_button", "n_clicks"),
State("url", "pathname"),
prevent_initial_call=True,
)
def start_generate_process_logs_transaction(n_clicks: int, project: ExamplesSolution) -> list[dict[str, Any]] | Any:
"""Launch the ``generate_process_logs`` long running transaction.
Triggered when the user clicks the "Start" button. Starts the backend
transaction, which writes a timestamped log line every second for
``wait_time`` seconds, and shows a persistent loading notification while
the transaction is in progress.
"""
notification = no_update
if ctx.triggered_id == "start_button" and n_clicks: # pyright: ignore[reportUnknownMemberType]
logger.info("Launch generate_process_logs transaction")
step = project.steps.basic_step
step.generate_process_logs(wait_time=10.0)
notification = [
dict(
title="Info",
id="generate-process-logs-notification",
action="show",
message="Generating logs via long running transaction...",
autoClose=False,
loading=True,
color="blue",
withCloseButton=False,
)
]
return notification
Clicking the Start button calls step.generate_process_logs(wait_time=10.0), which starts the long-running transaction
asynchronously, and immediately shows a persistent, loading notification so the user knows the process is underway.
Append the streamed logs to the panel
The update_logs_on_backend_events callback appends each streamed log line to the log panel.
@callback(
Output("log_content", "children"),
Input("generate-process-logs-update-listener", "message"),
State("log_content", "children"),
prevent_initial_call=True,
)
def update_logs_on_backend_events(message: dict[str, Any], current_logs: str) -> str | Any:
"""Append a newly streamed log line to the displayed log text.
The event payload is JSON-encoded, so it is decoded with ``json.loads``
before being appended. If no logs are currently displayed (or only the
placeholder message is shown), the new line replaces it instead of being
appended, so the placeholder disappears as soon as logs start flowing.
"""
if (
ctx.triggered_id == "generate-process-logs-update-listener" and message
): # pyright: ignore[reportUnknownMemberType]
new_line = json.loads(message["data"])
if not current_logs or current_logs == NO_LOGS_MESSAGE:
return new_line
return current_logs + new_line
return no_update
Each event message payload is JSON-encoded, so it must be decoded with json.loads before being used; otherwise the raw
JSON text (including surrounding quotes and escaped \n sequences) would be displayed instead of an actual line break.
If the log panel currently only shows the placeholder text, the new line replaces it rather than being appended, so the
placeholder disappears as soon as the first log line arrives. Because the log_content element uses whiteSpace:
pre-wrap, the \n terminating each log line causes it to be displayed on its own row.
React to the termination event
The sync_controls and sync_notifications callbacks react to the transaction method’s
termination event.
@callback(
Output("start_button", "disabled", allow_duplicate=True),
Output("start_button", "loading", allow_duplicate=True),
Input("start_button", "n_clicks"),
Input("generate-process-logs-termination-listener", "message"),
prevent_initial_call=True,
)
def sync_controls(n_clicks: int, message: dict[str, Any]) -> tuple[bool, bool]:
"""Keep the "Start" button state in sync with the transaction lifecycle.
Disables and shows a loading spinner on the button as soon as it is
clicked, then re-enables it once the termination event for
``generate_process_logs`` is received from the backend.
"""
disable_start_button, loading_start_button = no_update, no_update
if ctx.triggered_id == "start_button" and n_clicks: # pyright: ignore[reportUnknownMemberType]
disable_start_button = True
loading_start_button = True
elif (
ctx.triggered_id == "generate-process-logs-termination-listener" and message
): # pyright: ignore[reportUnknownMemberType]
disable_start_button = False
loading_start_button = False
return disable_start_button, loading_start_button
@callback(
Output("notification-container", "sendNotifications", allow_duplicate=True),
Input("generate-process-logs-termination-listener", "message"),
prevent_initial_call=True,
)
def sync_notifications(message: dict[str, Any]) -> list[dict[str, Any]] | Any:
"""Show a success or failure notification once the transaction ends.
Parses the termination event's ``MethodState`` payload and replaces the
in-progress notification with a success or failure message depending on
whether ``generate_process_logs`` completed successfully.
"""
notification = no_update
if (
ctx.triggered_id == "generate-process-logs-termination-listener" and message
): # pyright: ignore[reportUnknownMemberType]
method_state = MethodState.model_validate_json(message["data"])
notification = handle_method_event(
method_state,
"generate-process-logs-notification",
"Successfully ran generate_process_logs.",
"Failed to run generate_process_logs. Please check the logs.",
)
return notification
sync_controls disables the Start button and shows a loading spinner as soon as it is clicked, then re-enables it once the
termination event is received. sync_notifications parses the termination event payload into a MethodState and uses the
shared handle_method_event helper to replace the in-progress notification with a success or failure message.
Clear the logs
The clear_process_logs callback is triggered by the Clear Logs icon button.
@callback(
Output("log_content", "children"),
Input("clear-logs-button", "n_clicks"),
State("url", "pathname"),
prevent_initial_call=True,
)
def clear_process_logs(n_clicks: int, project: ExamplesSolution) -> str:
"""Clear the persisted log file and reset the displayed log text.
Triggered by the "Clear Logs" icon button. Deletes the backend log file
reference and resets the log panel to the placeholder message.
"""
if ctx.triggered_id == "clear-logs-button" and n_clicks: # pyright: ignore[reportUnknownMemberType]
step = project.steps.basic_step
step.clear_logs()
return NO_LOGS_MESSAGE
return no_update
This callback calls the clear_logs transaction to discard the persisted log file, and resets the log_content text back
to the placeholder message.
Restore the log panel on page load
The get_process_logs_text helper reads the persisted log file to populate the log panel when
the page is first rendered.
def get_process_logs_text(project: ExamplesSolution) -> str:
"""Read and return the persisted log file content for the initial render.
Returns the placeholder message if no log file exists yet, or an error
message if the log file exists but cannot be read.
"""
try:
step = project.steps.basic_step
log_file = project.storage_scope.get_cached(step.log_file)
except Exception:
return NO_LOGS_MESSAGE
try:
return log_file.read_text()
except Exception as e:
return "Error reading log file: " + str(e)
This ensures that navigating back to the page after logs have already been generated shows the previously accumulated content, rather than starting from the placeholder message.
Now that your implementation is complete, continue to the Testing section.
Testing#
Finally, test your implementation to confirm it works as expected.
Run the solution and compare your results with the results shown in the Feature highlight section.