Cutting Document Review Time From Minutes to Seconds.

Underwriting, claims, audit, and compliance teams rely on long, dense documents every day:

• 30–200+ page cyber and property insurance policies
• Endorsements, exclusions, riders, and cyber addenda
• SOC 2 Type I/II reports
• NIST 800-53 and 800-171 control frameworks
• ESG and regulatory reports
• Audit evidence PDF collections

These documents are technical, cross-referenced, carrier- or vendor-specific, and often stored across multiple systems. Manual lookup can take 20–40 minutes per question, especially around exclusions, deductibles, or control verification, leading to:

• Slower underwriting turnaround and operational delays
• Inconsistent interpretation across reviewers and teams
• Elevated compliance and regulatory risk
• Analyst frustration and cognitive fatigue
• Time lost during submissions, audits, and investigations

What Mattered Most in Practice

• PDF parsing and normalization often mattered more than model size — OCR quality, tables, fonts, and headers heavily influenced recall.
• Chunk size and overlap had a large effect on coverage and cross-referenced sections.
• Embedding quality was critical for insurance and SOC/NIST recall, especially in long, technical documents.
• Grounding-first prompts kept outputs grounded and defensible in the evaluated test set.

Infrastructure & Stability

• In the tested GPU configuration, a clean Linux installation with an RTX 4090 was more stable than the evaluated WSL environment.
• In the tested configuration, approximately 80–85% GPU memory utilization provided a workable balance between throughput and stability.
• Very long answers (>2–3k tokens) increased latency nonlinearly, so answer length and context depth were tuned per workflow.

On-Prem RTX 4090

• Fast, predictable performance for 8B-class models in the tested environment
• Documents and embeddings can remain within client-controlled infrastructure, subject to the surrounding access, logging, backup, and security architecture
• After hardware acquisition, on-prem infrastructure may reduce incremental inference expense, although electricity, depreciation, maintenance, cooling, support, and resilience costs remain
• WSL was less stable than a clean Linux installation in the tested GPU configuration; production suitability depends on the specific infrastructure and workload

Cloud (GCP L4)

• Stable, easy to deploy, runs 24 GB VRAM models comfortably
• Ideal for distributed teams or variable workloads
• Typical configuration: ~8–10 hours/day scheduled uptime, yielding roughly $400–$500/month in compute cost
• At ~40k–50k monthly questions, effective compute cost works out to approximately $0.008–$0.0125 per question ($400/50,000 to $500/40,000). This estimate covers GPU compute only and excludes implementation, integration, storage, networking, support, monitoring, and other operating costs.

Example Hybrid Architecture

• In the tested environment, the on-prem 4090 handled most daily workloads
• Cloud GPUs were used for burst capacity, after-hours usage, or special projects
• The right deployment model depends on each organization's workload, risk profile, and data-residency requirements

API Models & Escalation

The architecture supports a “local-first, escalate-when-needed” strategy:

• A local-first architecture can handle appropriate routine requests and escalate selected complex, ambiguous, or high-risk questions to larger or externally hosted models. Routing thresholds and the expected distribution between local and external models must be established through workload-specific evaluation.
• This approach may keep costs more predictable while preserving access to larger models when warranted.

A key finding from this build and evaluation: the technology works in tests, but production adoption requires intentional change management.

• Analysts must trust the system enough to use it daily
• Managers need clarity on which questions AI should handle vs. when to escalate
• Teams need training on grounded answers vs. free-form AI “guesses”
• Compliance and audit functions need documentation, governance, and clear usage policies
• Integration into existing underwriting, claims, and audit workflows is essential for real ROI

Training, governance, workflow integration, and management support may improve the likelihood of adoption. This internally validated POC did not measure production adoption or organizational outcomes.

Key Lessons Learned

• Python is highly productive for insurance/compliance RAG work.
• In the tested grounded-QA workflow, the evaluated instruction-tuned configuration performed better than the corresponding base-model configuration.
• In the evaluated workflows, retrieval, ranking, and data quality often mattered more than model size alone.
• WSL was less stable than a clean Linux installation in the tested GPU configuration; production suitability should be evaluated for the specific infrastructure and workload.
• In enterprise AI engagements, change management, training, governance, and organizational support can be as important to adoption as model performance.