AI Ethics

How we design, guardrail, and govern Otto — and what we'll never do.

Otto always says it's AI

Otto identifies itself as an AI assistant, never claims to be human, and never role-plays being a real person a child might mistake for a friend, therapist, or authority figure — even if asked directly. This isn't just good practice; it's now required by law in states like California and New York for any AI system interacting with minors, and we hold ourselves to it everywhere we operate, not just where it's mandated.

No engagement-maximizing design

We don't design Otto to maximize time-on-app, streaks, or engagement the way social platforms do. There are no infinite-scroll mechanics, no variable-reward notifications, and no dark patterns designed to keep a child chatting longer than they came to. Where continued use runs long, Otto and the Parent Dashboard nudge a break rather than discourage one.

Built-in breaks and honest reminders

During extended conversations, Otto reminds the user — at least every three hours — that it is AI-generated and that it may be a good time to take a break. Parents can also set their own usage limits from the Parent Dashboard.

Safety-first model behavior

• Content filtering: every message is screened in real time for age-inappropriate or harmful content before a response is generated.
• Crisis protocol: if a child expresses suicidal ideation, self-harm, or a similar crisis, Otto follows a maintained protocol that refers them to age-appropriate crisis resources and alerts the parent, consistent with the safety commitments on our Child Safety page.
• No romantic or companion role-play with minors, ever — Otto is an assistant and tutor, not a simulated relationship.
• Zero tolerance for sexual content involving minors, in any form, consistent with our Child Safety policy.

Human oversight, not just automation

Model behavior is reviewed by our team before release and on an ongoing basis — including red-team testing against categories like self-harm, grooming, medical and legal advice, and misinformation. Automated filtering is backed by human review of flagged conversations and by parent-facing alerts, not left to run unsupervised.

What we don't do with your child's data

We don't use your child's conversations to train third-party AI models, and we don't sell or share their data for advertising. See our Trust Center for the technical detail (encryption, zero-retention APIs, PII scrubbing) behind these commitments.

Where this comes from

These commitments reflect our internal AI Governance Policy, current COPPA requirements, state AI-companion-chatbot laws (including California SB 243 and New York's AI Companion Models law), and the EU AI Act's prohibition on AI systems that exploit the vulnerabilities of a group because of their age. We review and update this page as the law — and our product — evolves.

Questions or concerns

Email us at aiethics@heyotto.app, or use our general Contact page. Concerns about a specific conversation or piece of content should go to safety@heyotto.app — see our Child Safety Reporting page.