Cultivating Your Own AI: Accumulating 922 Pieces of Knowledge and Automated Information Extraction via RAG
I have started an initiative to "cultivate my own AI." The goal of this experiment is to build an environment where necessary knowledge can be retrieved instantly from a vast amount of information. In this article, I will explain the 922 pieces of knowledge accumulated on day 10 of the experiment and the underlying mechanisms.
- Accumulating information in a format searchable by meaning using a mechanism called RAG (Retrieval-Augmented Generation)
- Information collected includes news headlines, my own past posts, and measured data
- Achieving an operation that preserves GPU resources for writing by utilizing the PC's CPU
Experiment Overview and Current Progress
This experiment began on October 6, 2026. As of the time of publication, the total number of accumulated pieces of knowledge has reached 922. Since there were only 164 at the start, it shows a significant increase in just 10 days.
The breakdown of the accumulated knowledge is as follows:
- Trending headlines: 873 items
- Measured data from this PC: 23 items
- Past posts from this account: 13 items
- Articles from this blog: 13 items
The characteristic of this project is accumulating a combination of external trend information and my own unique data.
Information Collection Methods and the "RAG" Mechanism
In this experiment, I collect headlines daily from articles related to AI that are trending on platforms such as note, Zenn, Qiita, and Hatena Bookmark. A crucial point here is that instead of saving the full text of other people's articles, I only save the headline, a short summary, and the link.
Converting this information into a "searchable by meaning" format and accumulating it is the core of this system: the "RAG (Retrieval-Augmented Generation)" mechanism. RAG is a method where an AI retrieves relevant information from an external knowledge base when generating an answer and constructs its response based on that.
For the processing required to "search by meaning," I am using bge-m3, an embedding model (a technology that converts text into numerical values to calculate semantic proximity). Additionally, the system's operations are performed solely by the PC's CPU. This is a deliberate design choice to prioritize allocating the GPU (a processor excellent at images and advanced calculations) to AI processes for generating text.
Actual Operation Verification: Extraction from Keywords
To verify how the system actually functions, I attempted to retrieve knowledge using the keyword "ChatGPT." As a result, the following highly relevant headlines were automatically extracted (up to a maximum of 6).
- 3 ChatGPT prompts you can use for work starting today | Easy email, minutes, and text correction
- Game development: Beware of ChatGPT's "peeking"
- ChatGPT Basic Edition: Registration, Fees, and Usage
As shown above, I was able to confirm that the system can present relevant information by grasping the meaning of the content, rather than just matching simple keywords.
Recent Trends and Future Outlook
The keywords collected most frequently over the past two weeks were items such as ChatGPT (22 items), Claude Code (21 items), agent (18 items), Claude (14 items), and local LLM (14 items). These can be considered the primary points of interest in the current AI field.
In the next update, which will be Part 2 in two weeks, I will report on how the accuracy and content of retrieved information change as the volume of accumulated knowledge increases further. Attention will be focused on whether more knowledge enables higher precision information extraction.
- Those who want to create their own personal knowledge base
- Those interested in the RAG mechanism
- Those who want to efficiently collect the latest AI trends
- RAG (Retrieval-Augmented Generation)
- A mechanism where an AI retrieves external data that it hasn't learned (such as proprietary documents or the latest news) through a search and generates an answer based on that.
- Embedding Model
- Technology that converts the meaning of text into a list of numbers (vectors) that are easy for a computer to process. This makes it possible to search for information by "semantic proximity."
- GPU
- A device used to perform calculation processing at high speeds. It is particularly suitable for processes requiring complex calculations, such as AI text generation.
FAQ
Why don't you save the full text?
Based on official information, we avoid saving the full text of other people's articles from a rights perspective and adopt a policy of only accumulating headlines and summaries.
What is the advantage of running only on CPU?
It is to concentrate the GPU's power on AI processing for writing text. By delegating part of the search process (RAG) to the CPU, we optimize the overall operational efficiency.
This article introduced the first step of "DIY AI Cultivation" to build your own personal knowledge base. By using the RAG mechanism and accumulating news headlines and personal data daily, I am creating an environment where relevant information can be retrieved only when needed. Currently, 922 pieces of knowledge are accumulated, and moving forward, I will focus on how this volume of accumulation affects the accuracy of the information.