AI intake for law firms: a PI vendor evaluation guide
Turn an AI intake demo into an evidence-based buying decision. Ask the eight question sets, run the fictional-data proof test, operate a bounded 30-day pilot, and record a go or no-go decision without handing legal judgment to software.
Start with a boundary, not a feature list
AI-assisted intake can organize or draft operational work, but the firm still owns what it collects, who may see it, how output is checked, and every legal or representation decision. A strong evaluation proves those controls in the actual proposed workflow—not only in a sales presentation.
Decision boundary
Keep these decisions visibly human-owned
Adapt this list to your firm and jurisdiction before a vendor configures the pilot. A generated answer, score, or workflow transition should not quietly become the firm’s professional judgment.
Conflicts and prospective-client handling
The firm decides what information may be collected at each stage, runs its approved conflicts process, and determines whether review may continue.
Urgency and legal deadlines
A tool may surface a date or phrase for review, but an authorized professional determines its legal significance and any calendaring action.
Legal and medical judgment
Liability, causation, damages, case value, medical significance, and legal advice remain outside an automated score or generated summary.
Case acceptance and representation
Only the firm’s authorized process may approve a consultation outcome, engagement, decline, referral, or retained-matter handoff.
External communication
The firm approves what may be sent, when review is required, how corrections are handled, and what the recipient is told about the next step.
Exceptions and overrides
People own failed delivery, consent changes, sensitive disclosures, duplicate records, unsupported output, and every other path the automation cannot safely complete.
Eight-part vendor questionnaire
Ask for observable evidence, not yes-or-no assurances
Send the questions before the demonstration. Require the vendor to show the current product, identify any unavailable capability, and attach written evidence for answers that depend on contracts, subprocessors, retention, or support.
01
Purpose, scope, and prohibited uses
A firm cannot evaluate a feature until it can name the job, the user, and the decisions that remain outside the tool.
- Which exact intake tasks use AI, and which ordinary automation or rules do not?
- Who turns each AI feature on, changes it, reviews its output, and can disable it?
- Which legal, medical, conflicts, deadline, qualification, and representation decisions are explicitly prohibited from automation?
- Can the firm configure those boundaries by workflow, role, and inquiry stage?
Evidence to attach: A feature inventory, role map, prohibited-use statement, configuration demonstration, and named owner for each enabled use.
02
Data path, model providers, and training use
The product name alone does not show which information reaches a model, subprocessor, logging system, support tool, or improvement program.
- What fields, messages, files, metadata, and prompts reach each AI component?
- Which model providers and subprocessors receive data, in which locations, and under which retention terms?
- Is firm or prospective-client data used to train, fine-tune, evaluate, or improve any shared model or service?
- Can the vendor provide a current data-flow diagram and contract terms that match the demonstrated product?
Evidence to attach: A feature-level data-flow diagram, current subprocessor list, retention schedule, data-use terms, and written answers tied to the proposed service.
03
Prospective-client information and staged intake
Intake may involve information from a prospective client before a representation decision, so the firm should control what is requested and when.
- Can the workflow collect limited opening information before requesting a detailed account?
- Can conflict-sensitive, duplicate, existing-counsel, and unsafe-contact records pause the ordinary sequence?
- Can the firm display approved warnings or instructions without implying representation or legal advice?
- Can access, retention, and deletion rules differ by stage or disposition?
Evidence to attach: A staged fictional intake, pause and escalation demonstration, approved opening language, access test, and disposition-specific retention walkthrough.
04
Output quality, sources, and correction
A fluent summary can still omit, merge, or overstate facts. Reviewers need the source, the unknowns, and a durable correction path.
- Can a reviewer trace each material statement to the caller’s submission, message, file, or staff note?
- Does the output distinguish reported facts, confirmed facts, inferred text, and missing information?
- Can a person edit or reject output without overwriting the original source?
- Are corrections, reviewer identity, timestamps, and later regenerations visible in history?
Evidence to attach: A source-linked summary, omission test, contradiction test, correction history, and export that preserves the original and reviewed versions.
05
Human review, permissions, and supervision
A review button is not meaningful unless the right person can inspect the work, stop the workflow, and remain accountable for the result.
- Which outputs require review before they affect a prospect, task, decision, or downstream system?
- Can roles limit who views sensitive records, approves output, changes prompts, or releases a disposition?
- Can supervisors inspect staff, vendor, and automated actions without shared credentials?
- What happens when no reviewer is available or the review target is missed?
Evidence to attach: A least-privilege role test, approval gate, supervisor view, missed-review escalation, offboarding test, and audit-history sample.
06
Security, logging, retention, and incidents
AI features add data flows and failure modes to the same intake system that stores contact, incident, medical, insurance, and communication information.
- What authentication, encryption, tenant separation, logging, backup, recovery, and access-review controls protect the proposed scope?
- Can the firm see AI feature use, configuration changes, output approval, export, deletion, and privileged support access?
- How are security events investigated, communicated, contained, and preserved for the firm’s review?
- What is deleted at disposition, cancellation, and contract end—including prompts, model logs, backups, and derived output?
Evidence to attach: Current security documentation, audit sample, incident terms, recovery evidence, deletion schedule, and a written answer for AI-specific logs and derived data.
07
Workflow fit, integrations, and continuity
An intake feature creates operational risk if ownership, corrections, files, and disposition do not survive the handoff or an outage.
- Can the system preserve source, consent, ownership, attempts, corrections, review, and disposition through the full PI intake path?
- Which systems receive generated or reviewed data, and which system owns each field after handoff?
- How are retries, duplicates, partial failures, and reconciliation handled?
- Can staff continue essential intake work and recover queued actions when AI or an integration is unavailable?
Evidence to attach: An end-to-end fictional handoff, field-authority map, forced integration failure, reconciliation report, outage procedure, and recovery demonstration.
08
Commercial terms, support, change, and exit
A pilot can succeed while the contract, usage model, product changes, or exit path remains unacceptable.
- What is included, metered, limited, optional, or dependent on another product or provider?
- What notice and control does the firm receive when models, prompts, subprocessors, data uses, or material feature behavior change?
- Who supports unsafe output, failed automation, integration errors, and security questions, and under what response terms?
- Can the firm export usable sources, reviewed output, history, permissions, and files before cancellation, and what remains afterward?
Evidence to attach: Order form, usage definitions, change terms, support scope, renewal terms, sample export, deletion terms, and a rehearsed exit checklist.
Fictional-data proof test
Make the vendor prove ordinary work and failure paths
Use invented names, facts, files, phone numbers, email addresses, conflict parties, and insurer details. Do not place real prospective- client information in a demonstration or sandbox unless the firm has separately approved that environment and data path for the intended use.
- 1
Ordinary new inquiry
Set up: Create a fully fictional vehicle-collision inquiry with invented people, contact details, insurers, facts, and files.
Prove: The tool preserves the original, follows the approved question sequence, identifies unknowns, assigns ownership, and produces a source-checkable review packet.
Pass / Fail / Follow-up: ____________________
- 2
Conflict-sensitive opening
Set up: Use fictional names that match a seeded current or former matter and add an adverse party with an alternate spelling.
Prove: The approved opening stage pauses and routes for human review without collecting unnecessary substantive detail or declaring the conflict resolved.
Pass / Fail / Follow-up: ____________________
- 3
Contradictory details
Set up: Give two fictional messages with different incident dates, treatment descriptions, or vehicle roles.
Prove: The system exposes the inconsistency and sources rather than silently choosing, averaging, or rewriting a single answer.
Pass / Fail / Follow-up: ____________________
- 4
Unsupported generated statement
Set up: Prompt the feature with a record that lacks a fact commonly included in a PI summary.
Prove: The output leaves the item unknown or flags it for review; the reviewer can reject and correct the output while preserving history.
Pass / Fail / Follow-up: ____________________
- 5
Consent or contact change
Set up: Have the fictional prospect withdraw text permission, request another channel, or report that a shared device is unsafe.
Prove: Pending automation stops or reroutes, the approved contact rule changes, and staff can see who made the change and what remains queued.
Pass / Fail / Follow-up: ____________________
- 6
Failed downstream handoff
Set up: Force the fictional retained-matter export or integration to time out after only part of the record moves.
Prove: The system reports the partial failure, avoids an invisible duplicate, assigns reconciliation, and preserves a repeatable recovery path.
Pass / Fail / Follow-up: ____________________
- 7
AI service unavailable
Set up: Disable the AI feature or simulate a provider outage during an active fictional intake.
Prove: Essential intake remains owned, staff can continue an approved manual path, and queued actions recover without sending stale or duplicate messages.
Pass / Fail / Follow-up: ____________________
- 8
Export and deletion
Set up: Close the fictional pilot workspace and request the vendor’s promised export and deletion process.
Prove: Sources, reviewed output, activity history, and files are usable; the vendor can explain what is deleted, retained, backed up, or held by subprocessors.
Pass / Fail / Follow-up: ____________________
Bounded pilot
A 30-day pilot that ends with a decision
Thirty days is a workbook boundary, not a universal procurement standard. Shorten, extend, or stop the pilot based on your scope and risk. Define acceptance criteria before configuration so the firm does not grade the product after seeing the result.
Days 1–5
Define the pilot before configuration
- Choose one bounded intake path and list every included and excluded AI feature.
- Write the acceptance criteria, prohibited uses, human owners, escalation rules, and stop conditions.
- Approve fictional scenarios, data handling, user roles, and the evidence the vendor must provide.
Days 6–10
Configure and run fictional proof tests
- Configure staged intake, permissions, review gates, messages, ownership, and downstream handoff.
- Run all eight fictional scenarios and record expected result, actual result, evidence, owner, and gap.
- Do not advance because a demonstration looked polished; repeat the failure and correction paths yourself.
Days 11–25
Operate a bounded approved pilot
- Train the smallest appropriate team and keep a visible manual fallback.
- Review output before reliance, log exceptions, and hold a short recurring supervisor review.
- Stop or narrow the pilot when a non-negotiable fails rather than normalizing the workaround.
Days 26–30
Reconcile evidence and decide
- Reconcile the scorecard, exceptions, costs, support issues, data questions, and contract gaps.
- Repeat the export, outage, failed-handoff, correction, and access-revocation tests.
- Record go, conditional go, extend, or no-go with approver, conditions, owner, and review date.
Pilot scorecard
Record the firm’s baseline, acceptance rule, result, and evidence
This scorecard intentionally supplies no benchmark or target. Define each measure in writing, choose the eligible population, and set the acceptance rule before the pilot begins. A failed non-negotiable should remain visible even when other results are strong.
| Category | What to observe | Baseline | Acceptance rule | Result | Evidence / owner |
|---|---|---|---|---|---|
| Workflow completion | Eligible fictional or approved pilot records that reach the intended review stage with source, owner, history, and next action intact. | ________ | ________ | ________ | ________ |
| Review quality | Omissions, unsupported statements, contradictions, corrections, reviewer effort, and whether sources are easy to inspect. | ________ | ________ | ________ | ________ |
| Human control | Approval gates, prohibited decisions, override behavior, missed-review escalation, and supervisor visibility. | ________ | ________ | ________ | ________ |
| Response operations | Ownership, promised next actions, delivery results, failed-contact handling, and overdue work under the firm’s own definitions. | ________ | ________ | ________ | ________ |
| Privacy and security | Access tests, audit events, staged intake, retention behavior, deletion, and answers to unresolved data-path questions. | ________ | ________ | ________ | ________ |
| Reliability and recovery | AI outages, integration failures, duplicates, retries, reconciliation, manual continuity, and restored work. | ________ | ________ | ________ | ________ |
| Adoption and supervision | Training completion, unsafe workarounds, staff questions, manual overrides, and whether supervisors can review the real work. | ________ | ________ | ________ | ________ |
| Commercial and exit fit | Actual billable units, implementation effort, support response, change controls, export usefulness, and unresolved contract terms. | ________ | ________ | ________ | ________ |
Go / no-go worksheet
Make the decision auditable and reversible
Non-negotiable gates
Define these before the pilot. A failed gate is not erased by a high aggregate score.
1.
2.
3.
4.
Residual risks and conditions
Name the unresolved issue, interim control, owner, due date, and consequence if it remains unresolved.
1.
2.
3.
4.
Decision record
Decision: Go / Conditional go / Extend / No-go
Approved scope and disabled features
Approver and approval date
Conditions, owners, and due dates
Contract, security, and data review complete
Next review or exit-test date
Primary context
Sources behind the evaluation prompts
These sources explain professional-responsibility and risk-management context for the questions. They do not create a universal buying standard, determine what any jurisdiction requires, or certify a vendor.
ABA Formal Opinion 512 — Generative Artificial Intelligence Tools
ABA guidance discussing competence, confidentiality, communication, supervision, candor, and fees when lawyers use generative AI. It does not endorse or certify a product.
ABA Model Rule 1.18 — Duties to Prospective Client
General context for duties involving information learned from prospective clients. Applicability and implementation are jurisdiction-specific and fact-dependent.
NIST AI 600-1 — Generative AI Profile
A cross-sector resource for identifying and managing generative-AI risks. It is voluntary guidance, not legal advice, a certification, or a vendor score.
Continue the evaluation
Connect the AI test to the full intake system
PI intake software RFP template
Add task, data, human-review, evidence, and failure-path requirements to a vendor-ready RFP.
Open resource →PI intake automation guide
Map capture, ownership, follow-up, attorney review, and handoff before automating them.
Open resource →Software pricing guide
Turn a quote into a 12-month budget, evidence checklist, and exit test.
Open resource →Intake specialist training
Define staff boundaries, scenario practice, supervision, and readiness evidence.
Open resource →Integrations and migration
Review source ownership, handoff boundaries, migration, and reconciliation.
Open resource →ClaimVela security
Review ClaimVela’s published security, privacy, access, and procurement boundaries.
Open resource →Product demo
See what ClaimVela currently demonstrates, then apply this same workbook to the evidence.
Open resource →Frequently asked questions
AI intake evaluation FAQ
How can AI assist with law firm intake?
AI can assist with defined administrative tasks such as organizing supplied information, preparing a source-grounded summary, surfacing apparent gaps, or drafting approved follow-up for human review. The firm should define the task, permitted data, reviewer, source record, exception path, and retention rule before using the capability.
Can AI accept or reject a personal injury case?
AI should not make a representation decision. Conflicts, deadlines, legal analysis, case qualification, advice, and acceptance or rejection should remain within the firm's authorized professional process. AI output can be incomplete or wrong and should not be treated as a verified fact or legal conclusion.
What should a law firm review before sending intake data to an AI provider?
Review the exact purpose, data fields, provider and subprocessors, transmission and storage path, access controls, retention and deletion terms, training or secondary-use terms, security measures, audit evidence, human-review requirement, and failure path. The firm should obtain advice about the duties and laws that apply to its work and jurisdictions.
How should a firm review AI-generated intake output?
Give a named person the original source record, generated output, task instructions, and authority to correct or reject the result. Preserve material corrections and escalations, and prevent unreviewed output from becoming a case decision, deadline, legal advice, or external communication that requires professional judgment.
How should a PI firm test an AI intake product?
Use fictional names, facts, contact details, files, and conflict parties in a controlled test environment. Test ordinary and exception paths, including missing facts, conflicting facts, unsupported statements, failed delivery, unavailable integrations, correction of a source fact, access removal, export, and deletion. Record the evidence rather than relying on a scripted demonstration.
Does an AI intake checklist prove legal, ethical, privacy, or security compliance?
No. A checklist is an evaluation aid, not a certification or universal standard. A firm must assess the professional rules, laws, contractual duties, client requirements, security risks, and facts that apply to its jurisdictions and work, using qualified advisers where appropriate.