Show Run Details
Show Run Details is the displays logging data for each time the Flow was run.
View Log Data
To view the Flow log data:
- From the Flow overview page, click the gear icon in the Settings column next to the a currently running Flow.
Click Show run details.
- Hero Platform_ brings you to a new log page for the selected Flow.
The default retention for Flow execution history is 30 days.
The retention period can be modified by opening the hero_flow_default.conf file and configuring the value for history-storage.retention.
The following data is available for each Flow run:
- Run ID
- Number of input records
- Number of output records
- Number of rejected fields
- Start timestamp
- End timestamp
- Flow runtime
- User
- Flow version
View or Download Flow Logs
View or download a .zip file that contains a .log file of the Flow run details.
To view a Flow log:
- Navigate to the Show run details page.
- Click the kabab menu icon on the right side of the Flow run row.
- Click View logs.
To download a Flow log:
- Navigate to the Show run details page.
- Click the kabab menu icon on the right side of the Flow run row.
- Click Download log file.
The log file is downloaded as a .zip file to your local file system.
After unzipping the downloaded file, the following information is available:
Job Information
- Build details:
- Timestamp
- Build revision ID
- Version number
- Build number
- Flow name
- Flow ID
- Schedule ID
Execution ID
Example
Build details: BuildVersionUtils.HeroFlowVersion(_builtTimeStamp=2022-07-11T15:26:32.419+0200, _buildRevision=18d1bf4d1035c510ec4d496ba4a9be97b5541799, _version=6, _buildNumber=2022.8.48, _lastAppliedUpgradeNumber=1) Flow name: 'Flow Demo' Flow ID: 87d41b9a-02a6-4186-b896-e177c6be523d Schedule ID: 48xk72nNkQ9CQTYPN18yWz Execution ID: 6hz919pxUQ40Qks3BooXYG
Execution Plan
- Input names/IDs
- Operations
- Node names/IDs
- Operations
- Output names/IDs
- Operations
Example
------------------------------ Execution plan with 1 segment(s) Segment 0: dependsOn=[], intermediateInputs=0, intermediateOutputs=0 Node: 'User Data' (c6bec2ce-234a-42f4-a3b1-16b8282d4730) Operation: ImmutableExternalTupleInput (id=51hDwQi1LTLeiMEFuxDvtG) Operation: ImmutableTupleMetrics (id=7UDulWShUdWPscDqa7so5u) Node: 'Upper' (43abcd22-3877-4b6b-864d-cadd41a72646) Operation: MapTupleStream(identity) (id=6ml3uld93KYEd9EK0oykYf) Operation: ImmutableTupleMetrics (id=4aC9PNaRwmNKoSlwBhdcDS) Operation: MapTupleStream(FunctionTypeMapOperator(ImmutableSimpleFunctionDefinition)) (id=6nj3u8XEc2YALkjGQOCLT) Operation: ImmutableTupleMetrics (id=5eFl2r2bykIP5CHs1mEuB4) Node: 'Auto Output 2022-07-12 08:57' (674b1d6d-0fb9-4515-9c18-70275a749daf) Operation: MapTupleStream(identity) (id=12eJn6yLJH9bnp2EKvYkL8) Operation: ImmutableTupleMetrics (id=1nBMazpZYjg0tA01NivnMb) Operation: FlatMapTupleStream(TupleStore) (id=7BWzDPuW4I5CqtBJ9S7nNy) Operation: ImmutableTupleMetrics (id=1THwhJxVuJcYLWE7CyG0nN) Operation: ImmutableTupleSink (id=4xPLBe0qdoWxaxCOyUQPl2) ------------------------------
Execution Metrics (JSON)
- Timings:
- Started at
- Stopped at
- Runtime
- Runtime in ms
- Status
- Operators: (All nodes in the Flow including Inputs/functions/Outputs)
- Name
- Node ID
- Input count
- External Input count
- Output count
- External Output count
- Rejected count
- Timings:
- Runtime
- Runtime in ms
- Average runtime
- Average runtime in ms
Example
{
"timings": {
"startedAt": "2022-07-12T09:06:35.237Z",
"stoppedAt": "2022-07-12T09:06:37.292Z",
"runtime": "00:02m",
"runtimeMs": 2055
},
"status": "SUCCEEDED",
"operators": [
{
"name": "Auto Output 2022-07-12 08:57",
"nodeId": "674b1d6d-0fb9-4515-9c18-70275a749daf",
"inputs": 14,
"externalInputs": 0,
"outputs": 14,
"externalOutputs": 14,
"rejected": 0,
"timings": {
"runtime": "\u003c 1 sec",
"runtimeMs": 237,
"averageRuntime": "\u003c 1 sec",
"averageRuntimeMs": 15
}
},
{
"name": "User Data",
"nodeId": "c6bec2ce-234a-42f4-a3b1-16b8282d4730",
"inputs": 0,
"externalInputs": 14,
"outputs": 14,
"externalOutputs": 0,
"rejected": 0,
"timings": {
"runtime": "\u003c 1 sec",
"runtimeMs": 11,
"averageRuntime": "\u003c 1 sec",
"averageRuntimeMs": 0
}
},
{
"name": "Upper",
"nodeId": "43abcd22-3877-4b6b-864d-cadd41a72646",
"inputs": 14,
"externalInputs": 0,
"outputs": 14,
"externalOutputs": 0,
"rejected": 0,
"timings": {
"runtime": "\u003c 1 sec",
"runtimeMs": 0,
"averageRuntime": "\u003c 1 sec",
"averageRuntimeMs": 0
}
}
]
}
Execution Log File
Example
2022-07-12T09:06:35.253Z [ClusterSystem-akka.actor.default-dispatcher-18] INFO a.h.p.c.f.FlowExecutionActor:213 - flowExecution-5: Starting FlowExecutionActor. flowId=87d41b9a-02a6-4186-b896-e177c6be523d, executionId=6hz919pxUQ40Qks3BooXYG, settings=ExecutionSettings{bleedCorruptedData=false, executionMode=STRICT, maxAttempts=3, maxParallelism=1}, recovering=NO, actorRef=Actor[akka://ClusterSystem/system/singletonManagerflowManagementActor/flowManagementActor/flowDeploymentActorV2-5/flowExecutionActorV2-5/flowExecution-5#1240496143]
2022-07-12T09:06:35.259Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.b.a.SequenceNumberPersistentActor:55 - flowExecution-5: Recovery is turned off, we delete all snapshots
2022-07-12T09:06:35.260Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.b.a.SequenceNumberPersistentActor:35 - flowExecution-5: Deleting all snapshots for persistenceId=flowExecution-87d41b9a-02a6-4186-b896-e177c6be523d
2022-07-12T09:06:35.260Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.p.c.f.FlowExecutionActor:301 - flowExecution-5: Flow in status RUNNING. Starting execution.
2022-07-12T09:06:35.263Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.backend.akka.SnapshotPersister:133 - flowExecution-5: Saving snapshot FlowExecutionState{runIndex=1, status=RUNNING, queuedStages=2, inProgressStages=1, completedStages=0}
2022-07-12T09:06:35.270Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.p.c.f.FlowExecutionActor:194 - flowExecution-5: Added Actor[akka://ClusterSystem/system/singletonManagerflowManagementActor/flowManagementActor/flowDeploymentActorV2-5/flowExecutionActorV2-5/$$a-adapter#-665643777] as subscriber.
2022-07-12T09:06:35.276Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.backend.akka.SnapshotPersister:145 - flowExecution-5: Deleting snapshots succeeded.
2022-07-12T09:06:35.276Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.backend.akka.SnapshotPersister:162 - flowExecution-5: Saved snapshot 1 with success.
2022-07-12T09:06:35.548Z [ClusterSystem-akka.actor.default-dispatcher-17] INFO a.h.p.c.f.FlowExecutionActor:377 - flowExecution-5: Stage 0 completed. Running stage will be marked as completed.
2022-07-12T09:06:35.549Z [ClusterSystem-akka.actor.default-dispatcher-17] INFO a.h.backend.akka.SnapshotPersister:133 - flowExecution-5: Saving snapshot FlowExecutionState{runIndex=1, status=RUNNING, queuedStages=1, inProgressStages=1, completedStages=1}
2022-07-12T09:06:35.556Z [ClusterSystem-akka.actor.default-dispatcher-17] INFO a.h.backend.akka.SnapshotPersister:162 - flowExecution-5: Saved snapshot 2 with success.
2022-07-12T09:06:36.313Z [ClusterSystem-akka.actor.default-dispatcher-17] INFO a.h.backend.akka.SnapshotPersister:133 - flowExecution-5: Saving snapshot FlowExecutionState{runIndex=1, status=RUNNING, queuedStages=1, inProgressStages=1, completedStages=1}
2022-07-12T09:06:36.314Z [ClusterSystem-akka.actor.default-dispatcher-18] INFO a.h.backend.akka.SnapshotPersister:162 - flowExecution-5: Saved snapshot 3 with success.
2022-07-12T09:06:36.316Z [ClusterSystem-akka.actor.default-dispatcher-18] INFO a.h.p.c.f.FlowExecutionActor:377 - flowExecution-5: Stage 1 completed. Running stage will be marked as completed.
2022-07-12T09:06:36.316Z [ClusterSystem-akka.actor.default-dispatcher-18] INFO a.h.backend.akka.SnapshotPersister:133 - flowExecution-5: Saving snapshot FlowExecutionState{runIndex=1, status=RUNNING, queuedStages=0, inProgressStages=1, completedStages=2}
2022-07-12T09:06:36.324Z [ClusterSystem-akka.actor.default-dispatcher-6] INFO a.h.backend.akka.SnapshotPersister:162 - flowExecution-5: Saved snapshot 4 with success.
2022-07-12T09:06:36.563Z [ClusterSystem-akka.actor.default-dispatcher-31] INFO a.h.p.c.f.FlowExecutionActor:377 - flowExecution-5: Stage 2 completed. Running stage will be marked as completed.
2022-07-12T09:06:36.564Z [ClusterSystem-akka.actor.default-dispatcher-31] INFO a.h.backend.akka.SnapshotPersister:133 - flowExecution-5: Saving snapshot FlowExecutionState{runIndex=1, status=SUCCEEDED, queuedStages=0, inProgressStages=0, completedStages=3}
2022-07-12T09:06:36.568Z [ClusterSystem-akka.actor.default-dispatcher-16] INFO a.h.backend.akka.SnapshotPersister:162 - flowExecution-5: Saved snapshot 5 with success.
2022-07-12T09:06:37.292Z [ClusterSystem-akka.actor.default-dispatcher-31] INFO a.h.w.e.d.FlowExecutionActorV2:134 - flowExecutionActorV2-5: Shutting down gracefully.
2022-07-12T09:06:37.292Z [ClusterSystem-akka.actor.default-dispatcher-14] INFO a.h.p.c.f.FlowExecutionActor:322 - flowExecution-5: Shutting down gracefully.
2022-07-12T09:06:37.292Z [ClusterSystem-akka.actor.default-dispatcher-14] INFO a.h.b.a.SequenceNumberPersistentActor:35 - flowExecution-5: Deleting all snapshots for persistenceId=flowExecution-87d41b9a-02a6-4186-b896-e177c6be523d
2022-07-12T09:06:37.293Z [ClusterSystem-akka.actor.default-dispatcher-14] INFO a.h.p.c.f.FlowExecutionActor:154 - flowExecution-5: Stopped.
2022-07-12T09:06:37.293Z [ClusterSystem-akka.actor.default-dispatcher-14] INFO a.h.w.e.d.FlowExecutionActorV2:238 - Execution actor delegate has terminated gracefully. Shutting down.
2022-07-12T09:06:37.293Z [ClusterSystem-akka.actor.default-dispatcher-14] INFO a.h.w.e.d.FlowExecutionActorV2:167 - flowExecutionActorV2-5: Stopped
Stage Logs
- Stage ID
- Description
- Type
- Status
- Timings:
- Started at
- Stopped at
- Runtime
- Runtime in ms
- External Inputs
- External Outputs
- Rejected
- Queued tasks
- Running tasks
- Operators:
- Name
- Node ID
- Runtime metrics:
- Inputs
- External Inputs
- Outputs
- External Outputs
- Timings:
- Runtime
- Runtime in ms
- Average runtime
- Average runtime in ms
Example
{
"stageId": "1",
"description": "Processing stage (Segment 0, grouped inputs\u003d0, flat inputs\u003d0)",
"type": "BATCH_PROCESSING",
"status": "COMPLETED",
"timings": {
"startedAt": "2022-07-12T09:06:35.549Z",
"stoppedAt": "2022-07-12T09:06:36.313Z",
"runtime": "\u003c 1 sec",
"runtimeMs": 764
},
"externalInputs": 0,
"externalOutputs": 0,
"rejected": 0,
"queuedTasks": 0,
"runningTasks": 0,
"completedTasks": 1,
"operators": [
{
"name": "Auto Output 2022-07-12 08:57",
"nodeId": "674b1d6d-0fb9-4515-9c18-70275a749daf",
"inputs": 14,
"externalInputs": 0,
"outputs": 14,
"externalOutputs": 14,
"rejected": 0,
"timings": {
"runtime": "\u003c 1 sec",
"runtimeMs": 237,
"averageRuntime": "\u003c 1 sec",
"averageRuntimeMs": 15
}
},
{
"name": "User Data",
"nodeId": "c6bec2ce-234a-42f4-a3b1-16b8282d4730",
"inputs": 0,
"externalInputs": 14,
"outputs": 14,
"externalOutputs": 0,
"rejected": 0,
"timings": {
"runtime": "\u003c 1 sec",
"runtimeMs": 11,
"averageRuntime": "\u003c 1 sec",
"averageRuntimeMs": 0
}
},
{
"name": "Upper",
"nodeId": "43abcd22-3877-4b6b-864d-cadd41a72646",
"inputs": 14,
"externalInputs": 0
"outputs": 14,
"externalOutputs": 0,
"rejected": 0,
"timings": {
"runtime": "\u003c 1 sec",
"runtimeMs": 0,
"averageRuntime": "\u003c 1 sec",
"averageRuntimeMs": 0
}
}
]
}
Containerized Function (Docker)
- Name
- Container consumes:
- CPU time
- Memory usage
- Consumed resources totals
- Uptime
- Consumed CPU time
- Used CPUs average
- Configured CPUs
- Max memory used
Example
--------- Consumed resources ---------- Uptime: 171s Consumed CPU time: 217.5s (185.2s user, 32.2s system) Used CPUs (average): 1.27 (1.08 user, 0.19 system) Configured CPUs: 1.9 (utilized: 66.8% total, 56.9% user, 9.9% system) Max memory used: 1.7 GB/2.0 GB
Execution Statistics
To view execution statistics on a node by node basis:
- Navigate to the Show run details page.
- Click the kabab menu icon on the right side of the Flow run row.
- Click Execution Statistics.
A read-only version of the Flow is displayed.
Under the node name, click the number to view statistics for individual nodes in your Flow.
View:- The number of input records the node processed coming from the previous node.
- The number of external input records the Input processed coming from an external data source.
- The number of output records the node processed coming from the previous node.
- The number of external output records the Output processed that are written to an external data source.
- The number of errors the node recorded.
- The total runtime of the node during execution.
- The average runtime of the node during execution.
Filter Logs Displayed
- Field data can be sorted alphanumerically by clicking the field name at the top of the column.
- Hide all the Flows that have run successfully and had no inputs or outputs by marking the box Hide successful, 0 inputs and 0 outputs.
- This feature is useful when a Flow has been set to run on a schedule but no new data needed to be processed.
- This feature box is marked by default.
