2026-10-28Translated from the Japanese original

Tips for Mastering Small Local AI: Why Specificity in Instructions Determines the Results

Are you struggling with 'not getting the expected answers' even after introducing AI models that run on your own PC, particularly lightweight ones? In fact, the 'way you ask' influences the results far more than the size of the model.

Key points
  • The relationship between model size and responsiveness to instructions
  • Three specific techniques to stabilize answers
  • The powerful effect of providing 'examples' in particular

Why the 'way you ask' is more important for 'smaller models'

There are various types of local AI, ranging from large models that require massive computing resources to 'smaller models' that run smoothly even on home PCs. Generally, the larger the model scale, the better it can read context and perform advanced reasoning even from vague instructions.

On the other hand, smaller models tend to lose sight of 'what to prioritize' when instructions are vague, resulting in off-target answers. However, looking at the flip side, this also means that by providing appropriate instructions (prompts), you can maximize their agility and operational speed.

In the experiments on this account, it has been confirmed that depending on the model size, the quality of output can change dramatically with slight nuances in instructions. Therefore, if you want to utilize lightweight models for work, efforts to prevent the AI from becoming 'confused' are essential.

  • Large models: Infer and supplement the background even with vague instructions
  • Smaller models: Demonstrate their true potential by providing specific and clear instructions

Three tips to increase answer stability

Here are three specific points to raise the accuracy and stability of answers when running small local AI.

First is to 'write specifically what you want it to do.' For example, instead of an abstract instruction like 'Write an email,' clarify the purpose and tone, such as 'Write an apology email to a business partner regarding a delivery delay in polite honorifics.'

Second is to 'specify the output format.' Define in advance whether you want the output in bullet points, within a specific character count, or if you need to separate specific items (Subject, Body, Closing). This prevents unnecessary behaviors, such as the AI adding extra commentary.

And third, the most effective method is to 'show one correct example.' This is an idea close to the method called 'Few-shot prompting.' By giving the AI just one sample of 'this is the input, and I want the output to be like this,' the AI immediately understands the expected pattern, and the accuracy of the answer improves dramatically.

  • [Specificity] Specify purpose, target, and tone in detail
  • [Structure] Decide on the format, such as bullet points, character count, and composition
  • [Example] Present one sample of the expected answer

Summary: To smooth out 'communication' with AI

The biggest advantage of running local AI on your own PC is that you can customize it freely for your own use while protecting your privacy. Smaller models, in particular, are fast, so if you can grasp the tips for instructions, they become very powerful assistants.

It might feel like a bit of extra work at first, but by preparing a few instruction templates, you will be able to make the AI work smoothly from next time. Start by 'inserting just one example' into your usual instructions.

Official information states that these efforts make AI answers more stable. Please try to draw out the potential of AI running in your own PC environment through the 'technique of asking'.

  • Specificity in instructions is the key to drawing out the 'intelligence' of lightweight models
  • Just inserting one example (sample) significantly changes stability
  • Creating your own 'instruction templates' is the shortcut to mastering AI
Useful for
  • People who are running AI on their own PC but feel variability in the quality of answers
  • People who introduced lightweight models but feel they are not working as well as expected
  • People who want to learn more deeply about how to give instructions (prompts) to AI
Glossary
Local AI
Refers to AI that runs using the resources of your own PC or server, rather than using cloud services (such as ChatGPT).
Prompt
A general term for instructions given to AI or providing input.

FAQ

Does it get better the more examples I show?

Basically, accuracy improves by increasing examples, but in the case of smaller models, stuffing in too many examples can sometimes cause the AI to lose sight of the core instruction. It is recommended to start by trying 'one correct example' and adjust as necessary.

Summary

When utilizing small local AI, the key to success is to be conscious of the 'specificity' and 'examples' of instructions rather than relying entirely on the model's performance. By conveying the purpose, output format, and samples as a set, the stability of the answers will improve dramatically.

This article was translated from Japanese by AI; numbers and model names were automatically checked against the original. The original was written with AI from the sources cited there.