Moderation is a key part of running your community, but it often takes a lot of time. The AI Moderation Assistant is here to help you detect and prioritize sensitive content automatically.
How to access it?
To use the AI Moderation Assistant, the moderation channel must been already activated on your community and you must have permission to access it.
Go to your moderation channel > AI Moderation Assistant (the tab just between All posts and Questions / Answers).
Presenting the AI Moderation Assistant
The AI Moderation Assistant analyzes posts, comments and images published on the community, highlighting potential risks. Its purpose is to help moderators:
- Reduce the time spent on daily moderation tasks as it detects and highlights sensitive posts faster
- Prioritize contents with the highest risk level
- Limit user exposure to harmful content
- Avoid escalation and potential bad buzz
However, it is not possible to know which element (text or image) triggered a moderation alert.
Understanding how it works
When a post or a comment is published, the AI Moderation Assistant automatically analyzes its content, looking at:
- Text content from a post or a comment such as title, description and custom text fields
- Images attached to the post
For each moderation catefory, the AI Moderation Assistant assigns a confidence percentage.
This indicates how likely the content belongs to that sensitive category.
For example, a 92% hate speech score means that the AI identifies a high probability of hate speech in the content.
If at least one category is above 80%, the post is marked as urgent.
How post are AI identified?
A post is considered identified by the AI Moderation Assistant when at least one moderation category reaches a confidence score of 40 percent or more. In that case, the post appears in the AI Moderation Assistant tab and an AI moderation report is saved.
This means that if no category reaches 40 percent, the post is not AI identified and no AI report is created.
When a post is edited, the AI Moderation Assistant analyze it again. Here are the rules applied:
- If the new analysis is not AI identified, the previous AI report is deleted.
- If the edit does not impact text fields or images, no new analysis is triggered.
- If a video or PDF is added, no new analysis is triggered.
Review AI identified content
For each AI identified content, moderators can quickly understand the reason.
The AI Moderation Assistant tab lists all of them, displaying the identified sensitive category.
In the tab, moderators can:
- Filter themes (seansitive categories) and sort (by highest AI confidence score, most recent or oldest)
- See each post that has been AI identified by the AI Moderation Assistant, displaying the following informations:
- Identified sensitive categories it belongs to
- Username of the member
- Channel in which the content has been published
- Date and time of publication
- The content that has been AI identified by the AI Moderation Assistant
- Take an action:
- Ignore: ignore the AI identified content and remove it from the AI Moderation Assistant tab if the content is not sensitive after checking it.
- Delete: permanently remove the post or comment from the community to ensure a safe place for members.
FAQ
No, the AI Moderation Assistant does not take moderation actions on its own. It identifies potentially sensitive content to help moderators prioritize their reviews. All final decisions remain manual.
Only posts with at least one moderation category reaching a confidence score of 40 percent or more are displayed. Posts below this threshold are considered low risk and are not shown.
The post is re-analyzed if the edit impacts text content or images. If the new analysis is no longer AI identified, the previous AI report is removed and the post disappears from the AI Moderation Assistant tab.
If the post or comment is AI identified but ignored, the member will not receive a notification. If their post or comment is deleted, they will be notified.