Training a Specialized Model

ORCA is a Vision Language Model (VLM) that extracts information from documents. To learn more, see ORCA Vision Language Models

While the base model works out of the box, you can improve its performance by training a specialized model on your specific data using annotated documents. ORCA VLMs can be managed and trained directly in Training Data Management (TDM). This article explains how to create a specialized model on top of an ORCA base model for your use case.

ORCA VLM specialization

The ORCA base model provides general-purpose extraction capabilities, but every customer’s documents are different.

ORCA 2 base model

In v43.2 and later, the ORCA 2 base model is available in addition to ORCA 1.0. ORCA 2 extends ORCA 1.0’s capabilities, and each specialized model inherits the capabilities of the base model selected for its training run. To learn more, see ORCA Vision Language Models and TDM for ORCA VLMs.

Training a specialized model on your documents allows you to:

• Improve extraction Accuracy for your specific layouts. To learn more, see Accuracy.

• Reduce the number of fields that require human review. To learn more, see Automation.

• Specialize the model to your organization’s document formats.

Training creates a specialized model tailored to your use case while still using the capabilities of the ORCA base model. It does not modify the base model itself.

Accuracy and automation tradeoffs

• Creating a specialized model on top of the ORCA base model allows you to define whether you need more accuracy or more automation for your specific use case.

• If you use an ORCA VLM without specialization, you will either fully automate the processing or send everything for human review. To learn more, see Accuracy.

To specialize an ORCA model:

1. Upload and annotate training documents in TDM for VLM Field Extraction.

2. Train the model to create a candidate.

3. Review the candidate model and its Projected Automation.

4. Deploy the candidate model.

5. Validate the model with test documents.

6. Retrain the model with additional data if needed, or keep the Deployed Model live.

Before specializing an ORCA model, make sure that:

• An ORCA base model is installed. You can prepare and annotate Training Data without one, but training requires at least one compatible installed base model. To learn more, see TDM for ORCA VLMs and Installing ORCA VLMs.

• You’ve configured a Semi-structured Layout with the fields you need for your use case. ORCA VLMs extract fields from Semi-structured layouts only.

• The latest Layout Version is locked.

• A Model Definition exists for the layout. To learn more, see Model Definitions.

Follow the steps below to create a specialized model for your use case.

Upload training documents

Before training a specialized model, you must upload and annotate training documents for your layout. Doing so provides the Ground truth values the model will learn from during training.

Dataset requirements

• The uploaded documents should not exceed 5 pages.

• Larger documents may lead to an Out-of-Memory (OOM) Error during specialized model training.

  • Supporting larger documents requires increasing GPU memory. Contact your Hyperscience representative for more information.

• Make sure that all training documents are unique.

  • The system excludes duplicate annotated documents from training.

• If the system excludes too many documents, training may fail because of insufficient data.

• Include documents that represent the different patterns in your Dataset.

• Annotate enough documents to capture these patterns before training the model.

Upload your documents through the Training Data tab in TDM. For the upload steps, see Training Data Management for ORCA VLMs. Once the documents are loaded in the system, you can start the Annotation process.

Annotating training documents

Open each document for annotation from the Training Data table. For the annotation steps and the annotation view’s controls, see Training Data Management for ORCA VLMs.

When annotating documents for a specialized ORCA model, follow these guidelines:

• Enter text exactly as it appears in the document. Do not normalize dates, fix typos, or change the letter case unless specifically instructed.

• Avoid formatting or input validation.

• Review pre-populated values carefully. The application may automatically populate fields by detecting text inside a selected area. Always verify that the captured value matches the document exactly, and correct it if needed.

• Add missing text manually when necessary. If the automated detection misses part of the value, manually enter the missing characters to complete the annotation.

• When a field appears multiple times in a document, annotate the values in natural reading order: top to bottom, left to right, and first page to last page.

• If the same field appears across multiple pages, treat its values as a single continuous sequence for that document.

Train the model

During training, the system uses the annotated document values as ground truth and trains a specialized model tailored to your document format.

Number of required documents

• Make sure that you have at least 120 annotated documents to train the model. Only eligible annotated documents count toward this requirement.

• For more stable and reliable performance, we recommend using 200+ annotated documents, covering the variety of document patterns in your dataset.

• Using a larger and more diverse dataset generally improves model performance.

To initiate training, click Train Model in the Actions drop-down menu. If multiple base models are installed, select an eligible base model when prompted. To learn more, see Training Data Management for ORCA VLMs.

Review the candidate model

Training produces a candidate model. The candidate does not process documents until you deploy it, and you can review its projected performance first.

The Overview tab can show Live model summary, Last live model summary, and Candidate model summary cards. A card appears only when the corresponding model exists.

The Projected Automation card appears when the live or candidate specialized model has projection data. It shows projected automation and its margin of error at the selected Test Target Accuracy. To learn more, see Accuracy and Automation.

• Change Test Target Accuracy to see how the projection changes.

• Expand the card to compare the live and candidate curves. A curve appears only when projection data is available for that model.

• Use the margin of error to understand the uncertainty around the estimate. A smaller margin indicates a more precise projection.

• If the candidate should not be used, you can reject it. You cannot undo the rejection of a candidate. To produce a new candidate, annotate additional documents and run training again.

Deploy the candidate model

To deploy the candidate:

1. On the Model Details page, open the Actions drop-down menu.

2. Select Deploy candidate model and confirm the action.

You can also deploy the candidate from the History tab by finding the candidate model and clicking Deploy.

After deployment, the candidate becomes Live, and documents submitted through the associated flow are processed by the specialized model.

Validate the model with test documents

After deploying the model, validate its performance on realistic data before relying on it in production.

Using testing documents

We recommend setting aside 50–100 representative documents for testing your model’s performance. Doing so allows you to evaluate how the model performs on realistic data.

• These documents should reflect the variety of inputs you expect in production.

• The model must not have seen them during training (i.e., do not include them in the training documents).

Validating

1. Set the flow’s QA sample rate to 100%, so that each processed document is reviewed in a QA task. Follow the steps described in Installing ORCA VLMs to set your sample rate.

2. Run your testing documents through the system. Learn how to perform the VLM QA task in our Vision Language Model Quality Assurance article.

3. Use the QA results to assess whether:

   • the Accuracy rate indicates that the generated transcriptions correctly match the document content, and

   • the Automation rate indicates that documents can be processed without requiring manual transcription review.

4. Compare the metrics between the new model and the previous model.

ORCA VLM transcriptions

Unlike traditional Identification models, ORCA VLMs directly generate transcriptions for each field.

Decide next steps

Based on the validation results, you can:

• Keep the deployed model live if the results meet your requirements.

• Retrain the model by adding more or higher-quality training data, and then review, deploy, and validate the new candidate as described above.

Remember to return the flow’s QA sample rate to its regular value after you finish validating the model.