Cutting Document Review Time From Minutes to Seconds.

Underwriting, claims, audit, and compliance teams rely on long, dense documents every day:

• 30–200+ page cyber and property insurance policies
• Endorsements, exclusions, riders, and cyber addenda
• SOC 2 Type I/II reports
• NIST 800-53 and 800-171 control frameworks
• ESG and regulatory reports
• Audit evidence PDF collections

These documents are technical, cross-referenced, carrier- or vendor-specific, and often stored across multiple systems. Manual lookup can take 20–40 minutes per question, especially around exclusions, deductibles, or control verification, leading to:

• Slower underwriting turnaround and operational delays
• Inconsistent interpretation across reviewers and teams
• Elevated compliance and regulatory risk
• Analyst frustration and cognitive fatigue
• Time lost during submissions, audits, and investigations

“I spend at least a third of my time just searching for where things are in these PDFs.” — Analyst feedback during discovery

What Mattered Most in Practice

• PDF parsing and normalization often mattered more than model size — OCR quality, tables, fonts, and headers heavily influenced recall.
• Chunk size and overlap had a large effect on coverage and cross-referenced sections.
• Embedding quality was critical for insurance and SOC/NIST recall, especially in long, technical documents.
• Grounding-first prompts significantly reduced hallucinations and kept outputs defensible.

Infrastructure & Stability

• Running on a clean Linux install with an RTX 4090 proved far more stable than WSL for heavy inference workloads.
• vLLM required careful VRAM tuning; the sweet spot for both 4090 and L4 (24 GB) was around 80–85% GPU utilization.
• Very long answers (>2–3k tokens) increased latency nonlinearly, so answer length and context depth were tuned per workflow.

On-Prem RTX 4090

• Fast, stable performance for 8B-class models
• Full data control — documents and embeddings stay within internal networks
• Hardware amortization drives incremental compute cost toward near-zero
• Requires a clean Linux install; WSL was not stable enough under sustained load

Cloud (GCP L4)

• Stable, easy to deploy, runs 24 GB VRAM models comfortably
• Ideal for distributed teams or variable workloads
• Typical configuration: ~8–10 hours/day scheduled uptime, yielding roughly $400–$500/month in compute cost
• At ~40k–50k monthly questions, effective compute cost can be on the order of $0.00001–$0.00002 per question, excluding implementation/integration work.

Hybrid (Recommended)

• On-prem 4090 handling most daily workloads
• Cloud GPUs used for burst capacity, after-hours usage, or special projects
• Balanced cost, performance, and privacy for insurance and compliance teams

API Models & Escalation

The architecture supports a “local-first, escalate-when-needed” strategy:

• Use local Llama 3.1 Instruct 8B for 80–90% of routine coverage / control / evidence questions.
• Escalate only the hardest, highest-stakes questions to higher-priced API models (GPT-4-class, Claude Sonnet, etc.).
• This approach keeps costs predictable while preserving access to top-tier models when warranted.

A key finding from deployments: the technology works, but adoption requires intentional change management.

• Analysts must trust the system enough to use it daily
• Managers need clarity on which questions AI should handle vs. when to escalate
• Teams need training on grounded answers vs. free-form AI “guesses”
• Compliance and audit functions need documentation, governance, and clear usage policies
• Integration into existing underwriting, claims, and audit workflows is essential for real ROI

With the right structure in place, adoption tends to accelerate: analysts see immediate time savings, leadership sees improved consistency, and compliance teams gain stronger defensibility.

Key Lessons Learned

• Python is highly productive for insurance/compliance RAG work.
• Instruct models outperform base models for grounded, citation-heavy tasks.
• Retrieval, ranking, and data quality often matter more than model size alone.
• WSL is not suitable for production GPU workloads; clean Linux + GPU is preferred.
• Integration and change management drive the real impact — not just the model choice.