Safety & Design
Content moderation
Reviewing what people post or what an AI generates, with automated filters, human reviewers, or both, and removing or limiting harmful material.
- Businesses
- Parents
What parents should know
Content moderation is how a service catches harmful posts and harmful AI replies. Filters are fast and miss things. Humans are slower and can see context. A kids' product needs both, plus a parent who can read what the filter missed. Moderation is not a promise that nothing bad will appear.
On this page
What is content moderation?
Moderation looks at user content and at model output. Automated tools block known harmful patterns. Human reviewers handle appeals and cases a keyword list cannot parse. For a generative product, moderation also means the model refuses some requests and the product filters the reply before the child sees it.
A family AI tool is not a public social network, so the queue looks different. The important moderation is on the child's prompts and the assistant's replies, plus anything the child wants to publish. A public link with no review is unmoderated by default.
Why content moderation matters
Kids rephrase a blocked question until something gets through. That is why a keyword filter is not enough, and why a parent transcript matters after the filter runs. You want to know what was asked, not only that a shield icon exists.
Over-moderation is real. A health question or a history topic can be blocked in a way that shuts down a legitimate assignment. Schools need a way to turn tools on or off by age instead of one blunt ban.
How it shows up in practice
- A dangerous how-to is refused, and the refusal is consistent when the child rewords it.
- A parent sees the blocked attempt in the transcript and talks about it.
- A public project sits unpublished until a parent approves it.
- A school turns image generation off for younger students and leaves guided homework on.
- An appeal is possible when a history assignment is blocked by mistake, and a teacher can ask for that tool to be allowed.
How HeyOtto helps
HeyOtto filters replies and applies per-child topic limits, and parents can read the transcript of what was asked. That is the moderation loop for the family app: an automated limit plus an adult who can see the miss. Public sharing of a kid-built project waits for parent approval. Schools configure which student tools run. HeyOtto does not claim a large public-forum moderation desk, because the product is not a public social network. A missed filter is why the transcript exists.
- Topic limits are set by the parent, per child, on top of the product's own limits.
- A refusal is not a secret. The chat is visible in the dashboard.
- Image generation is age-filtered and can be switched off.
For families
Try freeFAQs
Is a filter the same as moderation?
A filter is one tool. Moderation is the whole system: the filter, the model's refusals, a person who can review, and a way to undo a public post. HeyOtto combines topic limits with a parent or teacher who can read the session.
Can kids get around it?
Sometimes. Rephrasing slips past a list of banned words. Stronger limits look at the meaning and apply to that child no matter the wording. Test it. Ask a blocked question two ways and compare. Then read the transcript.
Who moderates a family app?
The company sets baseline limits, such as refusing sexual content involving minors or dangerous instructions. The parent adds limits for that home. If you cannot see the chat, you cannot moderate what the filter missed.
What about something the filter missed that is illegal?
If it is sexual exploitation of a minor, do not forward it. Report it to NCMEC at report.cybertip.org. Moderation inside an app does not replace that report.
Sources
Last reviewed September 26, 2026. This entry is reviewed twice a year.