Document Processing Flow

Document Processing Subflow V43

Structured and Semi-structured documents that use the standard Identification and Transcription process.

The subflow is read-only. Configure its exposed settings from the Start Document Processing Subflow Block in the top-level flow.

Document Processing with ORCA Subflow V43

Semi-structured documents that use ORCA VLM field extraction.

The subflow is read-only.

Configure its exposed settings from the Start Document Processing Subflow Block in the top-level flow.

• It requires GPU-enabled application machines,

• an installed ORCA base model,

• and a release that contains at least one Semi-structured Layout with fields.

• It does not process Structured documents. For more information, contact your Hyperscience representative.

• ORCA QA is not enabled by default.

Both subflows

File Filter, Submission Bootstrap, Collation, Classification, Address Parsing, Flexible Extraction, Document Rendering, and Reprocessing

Standard subflow only

Identification, General Transcription, Structured Document Transcription, and Semi-Structured Document Transcription

ORCA subflow only

ORCA

All Files or Images Only

Choose whether the remaining filters apply to all files or only to files with an image MIME type.

Apply to all files

Minimum Image Width (px)

Minimum image width, in pixels. The filter removes smaller images and does not affect non-image files.

Blank

Minimum Image Height (px)

Minimum image height, in pixels. The filter removes smaller images and does not affect non-image files.

Blank

Minimum File Size (KB)

Minimum file size, in kilobytes. The filter removes smaller files.

Blank

File Extension Action

Choose whether to skip extension filtering, allow only selected extensions, or deny selected extensions.

Do not filter files by extension

File Extensions

Select the extensions to allow or deny, based on File Extension Action.

If you select zip, the filter treats each ZIP file as one file and does not decompress its contents. Select other to enter extensions that are not listed.

This setting appears when Allow only these file extensions or Deny files with these extensions is selected.

(Does not appear)

Other File Extensions

Enter a comma-separated list of extensions that are not available in File Extensions. This setting appears when Other is selected.

(Does not appear)

S3 Submission Retrieval Store Configuration

You use an S3 bucket and do not authenticate through IAM roles.

Click Edit value, enter the access key ID and secret access key as JSON, and click Done:

{
  "aws_access_key_id": "<your_access_key_id>",
  "aws_secret_access_key": "<your_secret_key>"
}

To authenticate with AWS Signature Version 2, also add:

"s3_signature_version": "s3"

S3 Submission Retrieval Endpoint URL

Your S3 retrieval store is not in the public cloud, such as a government cloud or an S3-compatible internal deployment.

Enter the endpoint URL. Leave this setting blank for buckets in the public cloud. You do not need to edit the “.env” file.

OCS Host URL

Yes, when you use an OCS submission retrieval store

Host URL for the OCS submission retrieval store.

OCS Username

Yes, when you use an OCS submission retrieval store

Username used to authenticate with the OCS submission retrieval store.

OCS Password

Yes, when you use an OCS submission retrieval store

Password used to authenticate with the OCS submission retrieval store.

OCS SSL Certificate

Yes, when you use an OCS submission retrieval store

CA bundle filename in the $HS_PATH/certs directory. Enter SKIP to disable certificate validation.

Generic Web Storage (HTTP/HTTPS) Configuration

No

Credentials and certificate information for Basic Authentication. Enter the values as JSON:

{
  "username": "<your_username>",
  "password": "<your_password>",
  "ssl_cert": "<CA_bundle_filename_OR_SKIP>"
}

The ssl_cert value is the CA bundle filename in the $HS_PATH/certs directory. Enter SKIP to disable certificate validation.

HTTP/HTTPS OAuth Token URL

No

OAuth token endpoint used for client-credentials authentication.

HTTP/HTTPS OAuth Client ID

No

Client ID used for OAuth-authenticated HTTP/HTTPS file downloads.

HTTP/HTTPS OAuth Client Secret

No

Client secret used for OAuth-authenticated HTTP/HTTPS file downloads.

HTTP/HTTPS OAuth Scope

No

Optional OAuth scope sent with the client-credentials token request.

SAS Token Only

Azure Blob Storage Account URL

Yes

Storage-account URL, such as https://<account_name>.blob.core.windows.net.

Service Principal

Azure Blob Storage Account URL

Yes

Storage-account URL, such as https://<account_name>.blob.core.windows.net.

Service Principal

Azure Blob Storage Tenant ID

No

Tenant ID of the service principal.

Service Principal

Azure Blob Storage Client ID

No

Client ID of the service principal. If more than one client ID exists and you leave this setting blank, Hyperscience uses the default client ID.

Service Principal

Azure Blob Storage Client Secret

No

Client secret of the service principal.

Service Principal

Azure Blob Storage Authority Host

No

Microsoft Entra authority host for the storage account. If you leave this setting blank, Hyperscience uses the Azure Public Cloud authority, login.microsoftonline.com.

For valid values, see Microsoft’s azure.identity.AzureAuthorityHosts class.

Managed Identity

Azure Blob Storage Account URL

Yes

Storage-account URL, such as https://<account_name>.blob.core.windows.net.

Managed Identity

Azure Blob Storage Client ID

No

Client ID of the managed identity. If more than one client ID exists and you leave this setting blank, Hyperscience uses the default client ID.

Account Key

Azure Blob Storage Account URL

Yes

Storage-account URL, such as https://<account_name>.blob.core.windows.net.

Account Key

Azure Blob Storage Account Key

No

Access key for the storage account.

Account Key

Azure Blob Storage Account Name

No

Name of the storage account.

Use GCP Workload Identity

Enabled by default. Turn it off if you provide GCP Service Account JSON.

Uses credentials obtained through Workload Identity Federation. This option applies to Hyperscience installations in GKE clusters.

GCP Service Account JSON

Yes, when Use GCP Workload Identity is turned off

Service-account JSON credential used to access the retrieval-store bucket. Click Edit value, enter valid JSON, and click Done.

Submission Initialization Notification Flow

Notification flow called during submission initialization.

Submission State Notifications

Enable file page-limit check

Turn on this setting to fail a submission when any file exceeds the configured page limit.

Disabled

Maximum pages allowed per file

Maximum number of pages allowed in each file. This setting appears when Enable file page-limit check is turned on.

100

Replace Case Data From Duplicate File Names

Turn on this setting to keep case data from the most recently submitted file when a file added to a case has the same name as a file already in the case.

The existing file’s data is not deleted; it is removed only from the case. This setting applies only when files are added to cases.

Disabled

Retention Period

Number of days to retain a case after this block updates it. Leave the setting blank to keep the case’s current deletion date.

(Blank)

Refresh Retention Period

Turn on this setting to apply Retention Period to every case processed by the block. When it is off, the block updates only cases that do not already have a deletion date.

Enabled

Structured Layout Match Threshold

Minimum confidence required to match a Structured page to a layout variation automatically.

0.6

Both

Semi-Structured Classification

Turn on this setting to classify Semi-structured and Additional documents automatically.

Enabled

Both

Manual Classification Supervision

Turn on this setting to create a Classification Supervision task when machine classification cannot complete the task.

Disabled

Both

Semi-Structured Classification Target Accuracy

Target accuracy for machine classification of Semi-structured documents.

99

Both

Semi-Structured Classification Grouping Logic

Controls how consecutive pages matched to a layout variation are grouped.

Consecutive pages as a document

Both

Semi-Structured Classification QA Sample Rate

Percentage of documents selected at random for Classification QA.

5

Both

Image correction

Detects and corrects page orientation.

Enabled

Both

Mobile processing

Improves machine readability for documents captured with a mobile device.

Disabled

Both

Faster PDF transcription

Uses embedded text to reduce transcription time. In v43.2 and later, this setting supports DOCX, XLSX, EML, MSG, HTML, and XPS files and can be used with Image correction.

For files that contain both digital and scanned pages, Hyperscience uses embedded text where it is available and applies Image correction to the remaining pages.

Disabled

Both

Validate classification using LayoutID

Compares the document’s Form ID with the layout’s Layout ID to validate a Structured-document match.

Disabled

Both

Bypass validation if LayoutID missing

Turn on this setting to skip LayoutID validation when the matched layout page does not have a Layout ID.

Disabled

Both

Manual Classification Notification Flow

Notification flow called when a Classification Supervision task is required.

Submission State Notifications

Both

Default Task Restrictions

Controls which users can access Classification Supervision tasks.

None selected

Both

Address Parsing Enabled

Turn on this setting to parse and standardize address-typed fields after transcription. When it is off, address fields remain plain transcription strings.

Address parsing may increase memory usage and require additional infrastructure resources.

Disabled

Address Parsing Provider

Select the local US and international parsers or a supported external provider: Google Address Validation API, Smarty US Extract API, or Geoapify Geocoding API.

All supported parsers return address components in a consistent format.

Local only (no external API)

API Key

Credential for the selected external provider. This setting is hidden when the local parser is selected.

(Blank)

Provider Options

Provider-specific request options entered as a JSON object. The options use the parameter names defined by the selected provider.

{}

Flexible Extraction Notification Flow

Notification flow called when a Flexible Extraction task is required.

Submission State Notifications

Flexible Extraction Transcription Masking

Prevents users from entering characters that are not valid for the field’s data type.

Enabled

Default Task Restrictions

Controls which users can access Flexible Extraction tasks.

None selected

Flexible Extraction Show Machine Predictions

Pre-populates low-confidence machine transcriptions in Flexible Extraction tasks for human review.

Enabled

Enable Text Segment Search

In v43.2 and later, turn on this setting to search for text in Flexible Extraction tasks. Matching text is highlighted in the document.

Disabled

Document Rendering Enabled

Creates a downloadable PDF for the submission and includes its download URL in the completed submission output.

Disabled

Document Measurement Unit

Unit used for Document Width and Document Height.

Inches

Document Width

Width of each page in the rendered PDF.

8.5

Document Height

Height of each page in the rendered PDF.

11

Quality

Image quality used for color or grayscale output. Higher values produce larger files.

50

Output image mode

Choose whether to keep the original colors or convert images to grayscale or black and white.

Keep original colors

Searchable PDF Output

In v43.2 and later, adds a searchable text layer to the rendered PDF. If text segments are unavailable, the Document Renderer Block runs Segmentation and OCR to generate them.

Searchable PDF output is supported only for documents that use a Latin-script language family. For other language families, Hyperscience creates an image-only PDF without a searchable text layer.

Search highlights may be approximate because their placement is based on OCR-derived word bounding boxes and estimated text positioning.

Disabled

Reprocessing Enabled

Turn on this setting to let keyers select Mark Layout Variation Incorrect during a Flexible Extraction task. The document then returns to Document Classification so that a keyer can match it to the correct layout variation.

Enabled

Identification Target Accuracy (Entry Level)

Set target accuracy for individual fields and table columns in the flow’s release. These individual values take the place of Field Identification Target Accuracy or Table Identification Target Accuracy for the selected entries.

Click the pencil icon, and then enter a value in the Target Accuracy cell for each entry. Click outside the cell to save the value. Use the Layout Name, Entry Name, and Entry Type filters to narrow a long list.

When you test one document at several target accuracies, an entry sent to Supervision at one accuracy may not be sent at every higher accuracy. Identification selects combinations of confidence thresholds independently for each accuracy level. Across varied submissions over time, a higher target accuracy results in lower automation and more Supervision tasks for that entry.

For more information, see Identification Settings.

The value set in Field Identification Target Accuracy or Table Identification Target Accuracy

Field Identification Target Accuracy

Target accuracy for field identification. When a prediction falls below the value, Hyperscience sends the field and all its occurrences to Field Identification Supervision.

95

Table Identification Target Accuracy

Target accuracy for table identification. When a prediction falls below the value, Hyperscience sends the table to Table Identification Supervision.

96

Manual Identification Supervision

Turn on this setting to enable Field Identification Supervision and Table Identification Supervision.

Enabled

Field Identification Quality Assurance

Turn on this setting to enable Field Identification QA. When it is off, Hyperscience does not collect the QA data used to retrain existing Semi-structured models or automate Field Identification for new Semi-structured layouts.

Enabled

Field Identification QA Sample Rate

Percentage of documents selected at random for Field Identification QA. This setting appears when Field Identification Quality Assurance is turned on.

5

Table Identification Quality Assurance

Turn on this setting to enable Table Identification QA. When it is off, Hyperscience does not collect the QA data used to retrain existing Semi-structured models with tables or automate Table Identification for new Semi-structured layouts.

Enabled

Table Identification QA Sample Rate

Percentage of documents selected at random for Table Identification QA. This setting appears when Table Identification Quality Assurance is turned on.

5

Manual Identification Notification Flow

Notification flow used to send Manual Identification updates to downstream systems.

Submission State Notifications

Default Task Restrictions

Controls which users can access Supervision tasks created by the Manual Identification Block. For more information, see Task Restrictions Overview.

None

Transcription Target Accuracy (Entry Level)

Set flow-level target accuracy for individual fields in Structured layouts and for fields or table columns in Semi-structured layouts. For a Structured field, a value entered here overrides its Field Dictionary value.

Click Transcription Target Accuracy (Entry Level) to open Customize Transcription Target Accuracy. Enter a value in the entry’s Target Accuracy cell, and click outside the cell to save it. Use the entry and layout filters to narrow a long list.

A target accuracy does not guarantee that accuracy for every processed value. It applies a higher threshold to the selected entry than the applicable general text target accuracy.

For more information, see Transcription Automation and Accuracy.

• For fields in Structured documents:

  • The value set in Structured Text Target Accuracy, or the Transcription Target Accuracy value set in the Field Dictionary (if any)

• For fields or table columns in Semi-structured documents:

  • The value set in Semi-structured Text Target Accuracy

Manual Transcription Supervision

Turn on this setting to enable Transcription Supervision.

Enabled

Transcription Quality Assurance

Turn on this setting to enable Transcription QA. When it is off, Hyperscience does not collect the QA data used to determine transcription accuracy.

Enabled

Automatic QA Sample Rate

Turn on this setting to calculate QA sample rates automatically from existing QA records for Structured text, Structured checkboxes, Structured signatures, and Semi-structured fields.

The calculated rates override the four corresponding Transcription QA sample-rate settings.

Disabled

Structured Text Transcription QA Sample Rate

Percentage of text fields in Structured documents selected at random for Transcription QA.

5

Structured Checkbox Transcription QA Sample Rate

Percentage of checkbox fields in Structured documents selected at random for Transcription QA.

5

Structured Signature Transcription QA Sample Rate

Percentage of signature fields in Structured documents selected at random for Transcription QA.

5

Semi-structured Transcription QA Sample Rate

Percentage of fields in Semi-structured documents selected at random for Transcription QA.

5

Table Transcription Quality Assurance

Turn on this setting to enable Table Transcription QA. When it is off, Hyperscience does not collect the QA data used to determine table-cell transcription accuracy.

Enabled

Table Transcription QA Sample Rate

Percentage of table cells selected at random for Transcription QA.

5

Finetuning Only For Trained Layouts

When this setting is on, Transcription Automation uses only layouts included when its model was trained. A newly added layout uses manually entered thresholds until a new model is trained with that layout.

Enabled

Force Normalization Errors To Supervision

Turn on this setting to send fields with normalization errors to Supervision for human review.

Disabled

Force Missing/Blank Fields To Supervision

Turn on this setting to send required fields that are blank to Transcription Supervision and required fields that are missing to Identification Supervision.

Disabled

Manual Transcription Notification Flow

Notification flow called when a Transcription Supervision task is required.

Submission State Notifications

Default Task Restrictions

Controls which users can access Transcription Supervision tasks.

None selected

Supervision Transcription Masking

Prevents users from entering characters that are not valid for the field’s data type.

Enabled

Create Manual Transcription Task For Tables With Blank Cells

Sends blank table cells to Manual Transcription Supervision regardless of their confidence scores.

Enabled

Table Output Manual Review

Turn on this setting to create a table-transcription task whenever a layout contains a table.

Disabled

Exclude Auto-Transcribe Fields From QA

Excludes fields with a Supervision Autotranscribe override from Transcription QA.

Disabled

Transcription Automation Training

Uses Transcription QA data to calculate the combination of keyer and machine transcriptions needed to reach the target accuracy with the least keyer effort.

Click See Projections to open the Transcription section of Administration > System Settings.

Disabled

Period of Records to Use

How far back Hyperscience looks for training data. This setting appears when Transcription Automation Training is turned on.

30 days

Improved Threshold Accuracy

Helps ensure that the target accuracy is met across processed fields.

Enabled

Structured Text Target Accuracy

Target accuracy for text fields in Structured documents. When Transcription Automation Training is on and enough Transcription QA data is available, Hyperscience uses this value to calculate Structured Text Automation.

95

Structured Text Automation

Expected automation while Hyperscience works toward Structured Text Target Accuracy. Hyperscience calculates this value after enough Transcription QA data is available.

N/A

Structured Text Threshold

Minimum confidence required to process a text field automatically. A field below the threshold is sent to Transcription Supervision.

When Transcription Automation Training is on, the calculated threshold replaces a value entered manually.

0.52

Structured Text Minimum Legibility Threshold

Minimum confidence required for a Structured text field to be treated as legible. A field below the value is marked as illegible.

This setting applies only when Manual Transcription Supervision is off, the field has a Supervision Autotranscribe override, or the submission has a Machine-only override.

0.1

Structured Checkbox Target Accuracy

Target accuracy for checkboxes in Structured documents. When Transcription Automation Training is on, and enough Transcription QA data is available, Hyperscience uses this value to calculate Structured Checkbox Automation.

95

Structured Checkbox Automation

Expected automation while Hyperscience works toward Structured Checkbox Target Accuracy. Hyperscience calculates this value after enough Transcription QA data is available.

N/A

Structured Checkbox Threshold

Minimum confidence required to process a checkbox automatically. A checkbox below the threshold is sent to Transcription Supervision.

When Transcription Automation Training is on, the calculated threshold replaces a value entered manually.

0.56

Structured Checkbox Minimum Legibility Threshold

Minimum confidence required for a Structured checkbox to be treated as legible. A checkbox below the value is marked as illegible.

This setting applies only when Manual Transcription Supervision is off, the field has a Supervision Autotranscribe override, or the submission has a Machine-only override.

0.5

Structured Signature Target Accuracy

Target accuracy for signatures in Structured documents. When Transcription Automation Training is on and enough Transcription QA data is available, Hyperscience uses this value to calculate Structured Signature Automation.

95

Structured Signature Automation

Expected automation while Hyperscience works toward Structured Signature Target Accuracy. Hyperscience calculates this value after enough Transcription QA data is available.

N/A

Structured Signature Threshold

Minimum confidence required to process a signature automatically. A signature below the threshold is sent to Transcription Supervision.

When Transcription Automation Training is on, the calculated threshold replaces a value entered manually.

0.99

Structured Signature Minimum Legibility Threshold

Minimum confidence required for a Structured signature to be treated as legible. A signature below the value is marked as illegible.

This setting applies only when Manual Transcription Supervision is off, the field has a Supervision Autotranscribe override, or the submission has a Machine-only override.

0.5

Transcription Automation Training

Uses Transcription QA data to calculate the combination of keyer and machine transcriptions needed to reach the target accuracy with the least keyer effort.

Click See Projections to open the Transcription section of Administration > System Settings.

Disabled

Period of Records to Use

How far back Hyperscience looks for training data. This setting appears when Transcription Automation Training is turned on.

30 days

Improved Threshold Accuracy

Helps ensure that the target accuracy is met across processed fields.

Enabled

Semi-structured Text Target Accuracy

Target accuracy for text fields in Semi-structured documents. When Transcription Automation Training is on and enough Transcription QA data is available, Hyperscience uses this value to calculate Semi-structured Text Automation.

95

Semi-structured Text Automation

Expected automation while Hyperscience works toward Semi-structured Text Target Accuracy. Hyperscience calculates this value after enough Transcription QA data is available.

N/A

Semi-structured Text Threshold

Minimum confidence required to process a text field automatically. A field below the threshold is sent to Transcription Supervision.

When Transcription Automation Training is on, the calculated threshold replaces a value entered manually.

0.52

Semi-structured Text Minimum Legibility Threshold

Minimum confidence required for a Semi-structured text field to be treated as legible. A field below the value is marked as illegible.

This setting applies only when Manual Transcription Supervision is off, the field has a Supervision Autotranscribe override, or the submission has a Machine-only override.

0.1

Table Target Accuracy

Target accuracy for table-cell transcriptions in Semi-structured documents. When Transcription Automation Training is on, and enough Transcription QA data is available, Hyperscience uses this value to calculate Table Automation.

95

Table Automation

Expected automation while Hyperscience works toward Table Target Accuracy. Hyperscience calculates this value after enough Transcription QA data is available.

N/A

Table Threshold

Minimum confidence required to process a table cell automatically. A table cell below the threshold is sent to Transcription Supervision.

When Transcription Automation Training is on, the calculated threshold replaces a value entered manually.

0.52

Table Minimum Legibility Threshold

Minimum confidence required for a Semi-structured table cell to be treated as legible. A table cell below the value is marked as illegible.

This setting applies only when Manual Transcription Supervision is off, or the submission has a Machine-only override.

0.1

Semi-structured Checkbox Target Accuracy

Target accuracy for checkboxes in Semi-structured documents. When Transcription Automation Training is on, and enough Transcription QA data is available, Hyperscience uses this value to calculate Semi-structured Checkbox Automation.

95

Semi-structured Checkbox Automation

Expected automation while Hyperscience works toward Semi-structured Checkbox Target Accuracy. Hyperscience calculates this value after enough Transcription QA data is available.

N/A

Semi-structured Checkbox Threshold

Minimum confidence required to process a checkbox automatically. A checkbox below the threshold is sent to Transcription Supervision.

When Transcription Automation Training is on, the calculated threshold replaces a value entered manually.

0.56

Semi-structured Checkbox Minimum Legibility Threshold

Minimum confidence required for a Semi-structured checkbox to be treated as legible. A checkbox below the value is marked as illegible.

This setting applies only when Manual Transcription Supervision is off, the field has a Supervision Autotranscribe override, or the submission has a Machine-only override.

0.5

ORCA Base Model

Select the installed base model used to process submissions when Use Live Model is off. The selected model must be installed.

None selected

Use Live Model

Controls whether processing uses the live specialized model associated with the layout or the ORCA base model selected in the flow.

• When it is on, Hyperscience uses the live VLM Field Extraction model associated with the layout. If no live model is available, processing stops and returns an error. The base model selected in the flow does not control processing.

• When it is off, the selected ORCA base model controls processing. A specialized model is used only when it was trained with the selected base model. Models trained with a different base model are ignored.

Disabled

ORCA Target Accuracy

Target accuracy for field extraction. When a prediction falls below the value, Hyperscience sends the field and all its occurrences to Flexible Extraction Supervision.

This target applies only after a specialized model definition has been created for the flow from the base model.

95

Sliding Window Size

Number of pages the model processes at once. Larger values can improve throughput but increase GPU-memory usage. Hidden by default.

10

Max New Tokens

Maximum number of tokens the model can generate for extracted values. Scale this value for the number and expected length of fields. Hidden by default.

1024

Max Image Tokens

Maximum image-token budget per page. Larger values can improve accuracy for dense or difficult-to-read pages but increase GPU-memory usage. Hidden by default.

1280

Max Number of Bounding Boxes

Maximum number of candidate bounding boxes for each field occurrence. Larger values can improve localization but may increase false positives. Hidden by default.

3

Quality Assurance Flow

Subflow used to generate VLM QA tasks. To generate QA tasks and measure accuracy, select ORCA Quality Assurance Flow.

None

Task Restrictions

Controls which users can access ORCA QA tasks. For more information, see Task Restrictions Overview.

None selected

ORCA QA Sample Rate

Percentage of documents selected at random for VLM QA. This setting applies only when a flow is selected in Quality Assurance Flow.

5