2026-10-21Translated from the Japanese original

Practical Guide to Running Local AI on a PC with 8GB RAM: Choosing Models Based on Measured Data

Many people harbor concerns like, 'Will local AI even run on my PC?' Especially when memory capacity is limited, it is crucial to determine which model to choose for smooth operation. In this article, we explain the criteria for practical model selection in detail, based on measured data assuming a PC with 8GB of RAM.

Key points
  • Be mindful of 'practical available memory' after subtracting the amount used by the OS
  • Judge from both 'memory usage' and 'generation speed' of the model
  • Prioritize models that produce 'comfortable' speeds even in CPU-only environments

Thinking about 'Usable Memory' on a PC with 8GB RAM

When running local AI (artificial intelligence that runs directly on your own PC), one of the most important resources is memory. However, you cannot allocate the entire amount of memory installed in the PC to the AI.

For example, when using a PC with 8GB of memory, it is generally assumed that the basic operation of the OS (such as Windows or macOS) consumes about 4GB of memory. In other words, the memory effectively available to run AI models is about 4GB.

Thinking based on this 'practical available memory' is the first step toward smooth operation.

  • Checklist 1: Is the model's memory usage 4GB or less?
  • Checklist 2: Does it produce 'comfortable' speeds even when running on CPU only?
  • Checklist 3: If there is a GPU (a dedicated component for image processing, etc.), does it operate even faster?
  • Checklist 4: Is the model size (capacity) suitable for your intended use?

[Measured Data] List of Models Running in an 8GB Memory Environment

Here, we compare the measured values of major models capable of running on a PC with 8GB of memory (assuming about 4GB is used by the OS). There are significant differences in memory consumption and generation speed depending on the model's capabilities and size.

For generation speed, please refer to the following feel levels:

• 30 tokens/sec or more: Output is produced instantly

• 15–30 tokens/sec: Faster than reading, operates comfortably

• 5–15 tokens/sec: A short wait, but practically usable

• Less than 5 tokens/sec: Wait times may be noticeable and could feel stressful

*Tokens are the minimum units when the AI generates text.

  • gemma3:1b
  • Memory usage: 0.9GB
  • CPU-only speed: instant (61.7 tokens/sec)
  • GPU-equipped speed: instant (216.9 tokens/sec)
  • llama3.2:3b
  • Memory usage: 2.3GB
  • CPU-only speed: comfortable (27.3 tokens/sec)
  • GPU-equipped speed: instant (189.7 tokens/sec)
  • qwen3.5:2b
  • Memory usage: 2.3GB
  • CPU-only speed: comfortable (27.7 tokens/sec)
  • GPU-equipped speed: instant (170.4 tokens/sec)
  • gemma3:4b
  • Memory usage: 2.8GB
  • CPU-only speed: comfortable (21.5 tokens/sec)
  • GPU-equipped speed: instant (142.0 tokens/sec)
  • qwen3.5:4b
  • Memory usage: 3.1GB
  • CPU-only speed: comfortable (19.3 tokens/sec)
  • GPU-equipped speed: instant (132.1 tokens/sec)

Points for Choosing Models Prioritizing Practicality

What we can learn from the measured data is that in an 8GB memory environment, all the models listed above operate at a 'practical speed.' However, if you are looking for a more comfortable experience, it is recommended to choose by focusing on the following points.

First, if you are running in an environment with CPU only, choosing a model that falls within the 15–30 tokens/sec range allows for stress-free interaction. In this data, all of llama3.2:3b, qwen3.5:2b, gemma3:4b, and qwen3.5:4b fall into this 'comfortable' range.

On the other hand, if you want a faster response, lightweight models like gemma3:1b are suitable. These are output in 'instant' time even with CPU only, making them suitable for tasks requiring split-second reactions.

Ultimately, the trick to mastering local AI is to narrow down models not just by 'whether they run' but by 'whether they return answers at a practical speed for what I want to do.'

  • If prioritizing speed (instant): gemma3:1b
  • If prioritizing balance (comfortable): llama3.2:3b, qwen3.5:2b, gemma3:4b, qwen3.5:4b
  • If prioritizing memory savings: gemma3:1b
Useful for
  • Beginners who have a PC with 8GB of RAM and want to try local AI
  • Those who want to know which models run smoothly on their own PC
Glossary
Token
The minimum unit when AI processes or generates text. In the case of Japanese, it corresponds to a part of a character or a word.
Local AI
Artificial intelligence that runs directly in a local environment, such as your own PC, without going through a server in the cloud.
GPU
A processor specialized for image processing and calculation tasks. It plays an important role in improving the inference speed of AI.

FAQ

Can I not run models larger than 4GB with 8GB of memory?

Since we base our recommendations on available memory after subtracting the amount used by the OS, we recommend models of 4GB or less to guarantee practical operation. If you try to run a model exceeding 4GB, it may become extremely slow due to insufficient memory or crash.

Will it run comfortably even if I don't have a GPU?

In this measured data, it has been confirmed that models such as llama3.2:3b and qwen3.5:2b operate at a 'comfortable' speed (15–30 tokens/sec) even with CPU only. Even without a GPU, you can use them at a sufficiently practical speed if you choose the appropriate model.

Summary

When operating local AI on a PC with 8GB of RAM, consider the OS consumption and assume the effectively usable memory to be 4GB. gemma3:1b achieves 'instant' speed, while llama3.2:3b and qwen3.5:2b achieve 'comfortable' speeds; the key to success is selecting the optimal model from these based on your specific use case.

This article was translated from Japanese by AI; numbers and model names were automatically checked against the original. The original was written with AI from the sources cited there.