Configure a Flow

You can configure a Flow from the Flow overview page.

To configure a Flow:

  1. From the Flow overview page, click the gear icon in the Settings column next to the Flow you want to configure.
    Click Configuration.
  2. The configuration settings allow you to set:
    1. Flow name - Edit the name that identifies the Flow.
    2. Max parallelism - Enter the number of the top threshold for expected partitions/runners.
    3. Maximum retries - Enter the number of times a node tries to process a record before indicating the Flow has failed.
    4. Execution mode - 
      1. Select Strict if every record must be processed.
      2. Select Forgiving if incomplete records can be skipped.
      3. Select Inline Errors for records to move through the Flow and errors are processed as error messages.
  3. When complete, click SAVE.

Max Parallelism

Partitions/runners are used to logically or physically group data into segments that can be distributed for processing. When you allocate a Flow for processing, you can increase this number if you are processing a large volume of records. For example, if you need to process 1000 records, you could partition it into 2 partitions of 500 or 10 partitions of 100. 

The max number of runners are automatically configured. Less runners can be chosen for reasons including if available resources do not allow that many runners or if the data set cannot be split into multiple chunks (partitions) to allow for parallel processing.

Maximum Retries

Maximum retries indicates the number of times the node will try to process a record before it fails. The default of retries is 3 before a record in a Flow fails. If you are using strict execution mode, the Flow fails. If you are using the forgiving setting, the record is written to a separate file. 

Execution Mode

Strict vs forgiving specifies how strict you are about accepting input data and how strict you are about about every record being processed. Examples of bad input data include values that may be missing, out of range, or malformed. 

  • Strict: Use with invoices or in any case where it is critical that all records get processed.
  • Forgiving: Use this setting if records can be ignored. For example, if you can't generate a list of leads from every record in the source data, you can safely skip the incomplete records and process the completed ones.  Hero Platform_ writes the records that it can't process to a folder.
    • If a value is configured for the maximum retires setting and forgiving mode mode is marked, a retry only occurs when:

      • there is a failure during the partitioning stage on the data source. 

      • a total node failure occurs on a multi-node environment. 

  • Inline Errors: Use this setting to allow the Flow to continue if an error with a tuple occurs. If a processing error occurs, instead of dropping the tuple, the output fields are generated in error mode. Fields in error mode can be handled with functions like ErrorHandler. 
    • This process is similar to the forgiving mode, but errors are processed as error messages rather than skipped. 




Search