How to Prevent AI Lies? The 3-Step Automated Censorship System Practiced by "Hayaoshi Encyclopedia"
When asking an AI to create a quiz, it sometimes blithely tells "impossible lies." In the "Hayaoshi Encyclopedia" tool created by this account's operator, a 3-step censorship system has been integrated to solve this problem.
- 【Step 1】Strict fact-checking by programs
- 【Step 2】Double-checking by employing AI as a "proofreader"
- 【Step 3】Countermeasures for failures and final human intervention

Mechanisms to Technically Seal Out AI "Lies"
When utilizing AI run on your own PC (local AI), the problem of AI providing answers that differ from facts is a very significant challenge. Especially when creating content like quizzes where there is a clear "correct answer," even a small lie can damage the credibility of the entire content.
Therefore, in "Hayaoshi Encyclopedia," rather than believing the AI's output as is, we have built a mechanism to thoroughly check it on the system side. The main mechanisms are as follows:
- 【Automated Censorship by Program】It determines whether the "question text" generated by the AI contains the pre-prepared "correct answer." Additionally, the program instantly checks whether the "evidence" cited by the AI actually exists within the original article text.
- 【Proofreading by AI】Furthermore, we have positioned another AI as a "proofreader." We have this proofreading AI check whether any facts not present in the text have slipped in, or whether the quiz results in a single definitive answer.
- 【Rules for Correction and Limits】If a piece fails these censorship checks, we provide specific feedback to the AI and order it to recreate the content. Recreations are performed up to a maximum of 3 times; if it still cannot create it correctly, the article itself is changed to a different one.
Practical Improvements to Enhance Stability
When integrating AI into a system, countermeasures against unpredictable AI behavior are also important. Hayaoshi Encyclopedia establishes the following practical rules.
First, to prepare for cases where the AI's output goes out of control and won't stop, we have introduced a mechanism to forcibly cut it off at a certain length and proceed to the next process. Additionally, the system remembers answers that have been asked once or "alternative answers (options other than the correct one)" to ensure that articles with the same answer are never selected again, preventing duplicate quizzes.
- 【Operator Intervention】AI is not omnipotent. If it is determined that the AI's judgment itself is incorrect, the host (operator) can manually flip the judgment. When this operation is performed, the score is also automatically corrected.
Summary: Design for Making AI Work "Correctly"
The key point of this mechanism is that instead of leaving everything to the AI, it utilizes a multi-layered structure that combines "fact-checking" by programs with "proofreading" by a separate AI.
By incorporating a design that assumes "AI lies" like this, it becomes possible to automatically generate highly reliable content even with local AI. Next time, I will talk about the techniques for "look-ahead" and "matching" to make these quizzes progress smoothly.
- Explained the 3-step censorship system to prevent "mistakes" in quizzes made by AI.
- People who want to increase the reliability of AI run on their own PC
- People struggling with the "lie" problem in AI-based content creation
- People interested in design methods when integrating AI into a system
- Hayaoshi Encyclopedia
- A browser app where a local AI creates and presents buzzer quizzes from Wikipedia articles. This is a tool created by this account's operator using Claude (AI).
- Proofreading
- Checking whether there are mistakes in the content of a text, whether it matches facts, etc.
- Host
- The person who makes final judgments and manages the operations in this system.
FAQ
Why have AI perform proofreading?
Because it is difficult to judge "contextual correctness," such as the naturalness of the text or whether the answer is definitive, through program-based fact-checking alone, we perform higher-precision checks by having AI do the proofreading.
Is there a possibility that the AI will keep talking and never stop?
Since AI output can sometimes include unexpected behavior, we ensure operational stability by including a process on the system side to cut off the output at a certain length.
To prevent AI lies (hallucinations), Hayaoshi Encyclopedia performs censorship in three stages: "fact-checking by programs," "proofreading by AI," and "final human intervention." By not taking AI output at face value and establishing constraints on the system side, we have built a mechanism to automatically generate highly reliable quizzes.