Designing 'Frames' to Save AI Responses from Monotony: Crafting Constraints in Narratable

During the development of 'Narratable,' a chat tool for progressing stories with AI, a major hurdle was the issue of 'AI responses becoming monotonous.' We will explain the specific techniques used to solve this by using systemic 'frames' to control AI characteristics that cannot be managed by instructions alone.

AI Characteristics: Repeating Once-Established Forms

AI models have a characteristic where they use their own immediate previous output as a template. Because of this, once an AI responds in a specific format, a phenomenon occurs where subsequent interactions continue in that same format for a long time. In actual testing, this phenomenon was confirmed in 11 out of 11 trials.

This prevents variety from emerging in the story and makes it difficult to maintain the quality of the conversation.

• AI has a characteristic of referring to recent outputs for the next output
• Actual measurements showed that once a form was established, it frequently continued

The Limits of 'Character Count' Instructions

Even when imposing constraints via text such as 'Please answer within 20 characters' to the AI, it did not function as expected. In actual testing, despite giving such instructions, the results showed that the AI continued to write an average of 107 characters.

In other words, it is difficult to accurately control the output length of an AI solely through requests in words.

• Instructions in text like 'write shortly' are hard to enforce
• Since there are cases where long outputs are generated while ignoring instructions, a different method is necessary

Solution: 6 Patterns and Individual Generation Limit Settings

To solve this problem, we classified the AI's ways of responding into the following 6 'patterns':

- Normal - Words first - Curt - Show through actions - Rambling - Probing

Then, we introduced a 'Director AI' to act as the role of selecting these patterns according to the situation. More importantly, we set individual limits for the number of generated tokens (num_predict: the maximum number of units of information generated at once) for each pattern.

As a result, short remarks and long ramblings began to mix appropriately within actual conversations, resolving the monotony. However, an exceptional process is applied to the 'Curt' pattern. We found that it is more reliably shortened by letting it write with some leeway and then cutting it at the first completed sentence. This is because if we set it to only pass 26 characters, the sentence ends in the middle.

• The Director AI selects from 6 patterns depending on the situation
• Length is controlled by individually setting the generation token limit for each pattern
• The 'Curt' pattern adopts a method of letting it write with leeway and cutting it at one sentence

Improving Conversation Quality and Experience

Previously, we used weighted random numbers (probability settings) to select patterns. However, there was a concern that this would give the player a mechanical impression of 'it got shorter because of a dice roll.' By having the Director AI make judgments and systemically binding limits as in this current method, it becomes possible to create more natural variations.

We also performed improvements regarding conversation quality. In previous testing, only 7 out of 32 remarks ended with a question. This meant that the burden of creating topics fell entirely on the player side. By adding the 'Probing' pattern, we gave the AI a role in expanding topics from its side as well, achieving smoother dialogue.

• Pursuing naturalness through Director AI selection rather than weighted random numbers
• Reduced the player's burden of topic creation by adding the 'Probing' pattern
• Improved overall conversation quality and diversity