Define the role before choosing the tool
“AI” can describe very different functions: drafting a summary, recommending an activity, classifying a message, generating educational content, or holding a conversation. Each creates different risks. Write down what the system may do, what it may not do, and which decisions remain exclusively human.
A narrow, observable workflow is easier to supervise than a vague promise of autonomous care.
Make participation and disclosure meaningful
Show clients when AI is involved, what information it uses, what output reaches the provider, and which non-AI path is available. Place this information where the interaction begins.
Describe the feature around its configured purpose, provider review timing, and urgent-support route.
Design human oversight into the workflow
- Require provider approval for consequential recommendations.
- Make source information and uncertainty visible where possible.
- Allow outputs to be edited, rejected, or disabled.
- Route safety concerns through a documented protocol.
- Keep an audit trail of configuration and important actions.
Minimize and govern data
Collect only what the feature needs. Map where data goes, which vendors process it, how long it is retained, and whether it is used to train models. Apply role-based access, secure transport and storage, deletion processes, and contractual safeguards appropriate to the data and service.
Test performance and failure modes
Evaluate realistic examples, including ambiguous language, spelling differences, crisis statements, cultural variation, and attempts to push the system outside its role. Check for hallucination, overconfidence, stereotyping, unsafe advice, and inconsistent refusal behavior.
Monitor after launch. Models, prompts, surrounding workflows, and user behavior change over time.
Questions to ask a vendor
- What exact models and subprocessors are involved?
- Is customer data used for model training?
- What controls can providers configure?
- How are incidents, model changes, and vulnerabilities communicated?
- What evidence supports the feature’s intended use?
- What happens when the system is uncertain or unavailable?
- Can the practice export or delete its data?
How Habit of Care approaches AI-supported features
Habit of Care positions AI as workflow support under provider direction. Product pages separate suggestions from clinical decisions, state review timing and urgent-support routes, and connect outcome claims to relevant evidence. Providers evaluate configuration, consent, and governance for their own setting.
See the Trust Center for current safeguards and the Evidence & Outcomes page for our evidence standard.