AI Model & Document Type

The Automate model is trained on 1,079 document types at the sub-artifact level. These are based on the CDISC reference model. Performance varies across document types, however, it is benchmark at approximately 75% accuracy for all document types, including both structured and unstructured documents.

In typical implementations, the CDISC reference model accounts for approximately 85–90% of the required document types. The remaining 10% consists of organization-specific document types, which are incorporated into the model as part of implementation. These additional document types undergo human-in-the-loop reinforcement training to achieve the established benchmark accuracy prior to production use.

Yes, the Automate model is trained on approximately 1,079 document types at the sub-artifact level, based on the CDISC reference model.

Performance varies by document type, with some categories demonstrating stronger accuracy than others; however, the model is benchmarked holistically across all trained document types.

The Automate model is benchmarked at approximately 85% accuracy across all document types, including both structured and unstructured content.

For structured document types, performance is significantly higher, ranging between 95–100% accuracy, due to their standardized format and predictable data patterns.

Document types that are not included in the CDISC reference model can be incorporated into the Automate model as part of implementation.

Typically, the CDISC reference model covers approximately 85–90% of required document types. The remaining ~10%, which are sponsor- or organization-specific, are added through a structured onboarding process. These document types undergo human-in-the-loop reinforcement training to achieve the established benchmark accuracy before being fully deployed.

This is entirely configurable and depends on the customer’s workflow design. Automate can technically push documents through to finalization automatically if desired.

A common configuration approach includes:

• Confidence threshold model (e.g., 95%): Documents scoring above the defined threshold are automatically returned with an “Approved” status and can proceed through the workflow without manual intervention.

• Below-threshold routing: Documents falling below the threshold are returned with a “Needs Review” (or equivalent) status and routed to a human review queue for validation.

Automate can return documents with any status aligned to the statuses configured within the customer’s Veeva environment, enabling full flexibility in workflow orchestration and governance controls.

Yes. The platform is model-agnostic and can accommodate different AI providers, including models from Anthropic or OpenAI, if preferred..

They’ve been open to BYOL; recommend their own because they’ve tested many LLMs and use different ones per task.

There is a solution for this scenario. The AI can accurately identify the document type (e.g., Financial Disclosure Form, CV, medical license) and extract key entities such as the associated individual’s name. However, in cases where document types are highly similar, the model may initially lack sufficient contextual information to determine the individual’s role (e.g., Principal Investigator, Sub-Investigator, or another site role). TransPerfect has methods to work around this limitation by leveraging contextual information from the system.

The information is passed back and forth between the system and your Veeva database.

If document classification requires additional context, such as contact role validation or site matching—the necessary contextual data is retrieved from Veeva to support the decision-making process. For example, this may include distinguishing between a Principal Investigator (PI) and a Sub-Investigator, or performing site name normalization to ensure accurate matching.

It comes back as very low confidence, but it will still pull other data (summary, protocol number if present, site name if present, names/contacts). It shortens indexing time even when doc type is missed.