Search Image Content with 'Text'! Verification Results and Practical Tips for EmbeddingGemma 2

Have you ever thought about using AI running on your own PC to search for image content using text? For example, imagine a situation where you want to find a specific scene like "a cat wearing a red hat" from a massive amount of image data using keywords. In this experiment, we investigated the accuracy and characteristics of "EmbeddingGemma 2," a lightweight model released by Google. In this article, we will explain in detail the "capabilities" and "behaviors to watch out for" that became clear through our experiments.

What kind of model is EmbeddingGemma 2?

The model used in this experiment is called "EmbeddingGemma 2." This model excels at a process called Embedding, which converts the content of sentences and images into "vectors (sequences of numbers)." *Embedding: Converting text or images into multi-dimensional sequences of numbers so that AI can calculate them easily. According to official information, this model is extremely compact, at approximately 330MB in Q8 (a technique called quantization that reduces data size while maintaining accuracy), and has a dimensionality of 768. This size makes it very easy to handle as an AI running on your own PC (local environment), and we confirmed that it operates at a practical level.

• Model size: Approx. 330MB (Q8)
• Dimensionality: 768 dimensions
• Features: Possesses basic capabilities for searching content within images despite being lightweight

Strengths and Weaknesses Found in Experiments

Through several weeks of verification, we discovered that this model has several distinct behavioral characteristics. First, it possesses cross-lingual semantic understanding. For example, it can capture content similarities even when using different expressions in Japanese and English. However, because the model size is compact, the similarity (degree of similarity) values themselves tend to be lower compared to much larger models. *Cosine Similarity: A numerical value representing how similar two pieces of data are (the closer to 1, the more similar).

• Multilingual Support: Possible to understand meanings across different languages such as Japanese, Korean, and English
• Similarity Trends: Due to the model size, similarity values tend to be modest
• Notes for Accuracy: Since it is sensitive to word nuances, differences in spelling such as "blond" and "blonde" can affect the results.

The 'Text Density' That Influences Search Accuracy

What was particularly interesting in this experiment is that the AI sometimes prioritizes "textual information volume" over "visual features. Specifically, we observed a phenomenon where providing too much detail regarding image features actually caused the similarity to decrease. This is thought to be because the information density contained in the text (information with more "unexpectedness") has a stronger influence than visual features within the constraints of the model size. When building a local RAG (*) on your own PC, the "choice of words" becomes a key point that greatly affects search accuracy. *RAG (Retrieval-Augmented Generation): A mechanism where AI searches and refers to external data (images or documents) when generating an answer.

• Impact of Information Density: The more information in the text, the stronger the tendency to heavily influence similarity
• Query Tips: It is necessary to carefully choose appropriate phrasing without stuffing in too many visual features

Summary: Toward Utilization in Local Environments

EmbeddingGemma 2 is an excellent model choice for incorporating image search functionality into local environments due to its lightweight nature. However, if you do not correctly understand its characteristics, you may not get the expected results. The key to mastering it is to adjust search queries while keeping in mind three points: "The model is small but practical," "It understands multiple languages but with modest values," and "Results change significantly based on word choice." While this verification was based on a small dataset, it should serve as a very useful guideline for those who want to efficiently implement image searching.

• Lightweight and practical. Suitable for implementation in local environments
• To improve search accuracy, creating queries that are conscious of word nuances and information density is important
• It is recommended to try appropriate phrasing tailored to your needs, referring to the experimental results