{"id":563,"date":"2026-07-02T12:08:31","date_gmt":"2026-07-02T03:08:31","guid":{"rendered":"https:\/\/conceptminer.ai\/?page_id=563"},"modified":"2026-07-02T23:11:16","modified_gmt":"2026-07-02T14:11:16","slug":"conceptminer-concept-index-2","status":"publish","type":"page","link":"https:\/\/conceptminer.ai\/?page_id=563&lang=en","title":{"rendered":"ConceptMiner Concept Index"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\"><mark style=\"background-color:rgba(0, 0, 0, 0);color:#888888\" class=\"has-inline-color\">Similarity Search and Conceptual Classification Inside Existing RDBs such as PostgreSQL<\/mark><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ConceptMiner Concept Index is a mechanism for adding conceptual similarity search capabilities to existing relational databases such as PostgreSQL.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead of introducing a separate vector database or graph database, Concept Index enables organizations to extract semantically similar groups directly within their existing RDB environment. It can be used for data such as Voice of Customer records, inquiry histories, open-ended survey responses, sales notes, case summaries, and other text-based business records.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">How Concept Index Works<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ConceptMiner\u2019s conceptual structure model is based on GNG+MST technology.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">GNG uses unsupervised machine learning to assign semantically or conceptually similar data records to nodes. These nodes are then connected by edges according to their similarity, forming a topological network.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Concept Index uses this conceptual structure model to assign a corresponding node number to each data record in the database. It also adds a table that represents neighborhood relationships among the nodes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As a result, each record in the RDB is no longer just a row with IDs, attributes, and categories. It also carries information about which conceptual node it belongs to.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"955\" height=\"715\" src=\"https:\/\/conceptminer.ai\/wp-content\/uploads\/2026\/07\/image-3.png\" alt=\"\" class=\"wp-image-564\" srcset=\"https:\/\/conceptminer.ai\/wp-content\/uploads\/2026\/07\/image-3.png 955w, https:\/\/conceptminer.ai\/wp-content\/uploads\/2026\/07\/image-3-300x225.png 300w, https:\/\/conceptminer.ai\/wp-content\/uploads\/2026\/07\/image-3-768x575.png 768w\" sizes=\"auto, (max-width: 955px) 100vw, 955px\" \/><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">How Similar Data Is Retrieved Inside the RDB<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">With ConceptMiner Concept Index, similarity-based retrieval can be performed as follows.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When a user selects any data record in the database, or enters any text as a query, ConceptMiner estimates the corresponding node number. The system then retrieves other data records that have been assigned to the same node. If necessary, it can also retrieve records assigned to neighboring nodes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is the basic mechanism by which ConceptMiner Concept Index enables similarity search inside an ordinary RDB.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead of calculating the distance from the query to every individual data record each time, Concept Index uses a prebuilt conceptual node structure and neighborhood relationships to efficiently retrieve groups of similar records.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Optimized for Conceptual Segment Extraction, Not Just Pinpoint Search<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In principle, it is possible to perform additional distance calculations within the set of records assigned to the same node in order to identify the closest individual record to a selected record or input text.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, the primary purpose of ConceptMiner Concept Index is not to retrieve a single most similar document in the way typical RAG-oriented search systems do.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its main strength lies in extracting semantically similar data groups and conceptually coherent segments.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, it can be used to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Extract groups of customers with similar complaints or requests from VoC data<\/li>\n\n\n\n<li>Identify recurring patterns in inquiry histories<\/li>\n\n\n\n<li>Discover clusters of concerns, interests, or attitudes in open-ended survey responses<\/li>\n\n\n\n<li>Find similar sales cases or negotiation patterns from sales notes<\/li>\n\n\n\n<li>Group related case summaries or project descriptions<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Concept Index makes it possible to handle \u201csemantic closeness\u201d and \u201cconceptual grouping\u201d inside an RDB\u2014something that is difficult to achieve with ordinary keyword search alone.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Differences from Vector Databases and pgvector<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Typical approaches to similarity search include vector databases and pgvector.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A vector database is a dedicated database designed for vector search, separate from a conventional RDB. It is well suited for high-precision similarity search, but it requires an additional search infrastructure, which can increase system complexity and operational cost.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">pgvector is an extension that adds vector search capabilities to PostgreSQL. Its major advantage is that vector search can be performed within an existing PostgreSQL environment. However, its basic principle is still Top-K search based on vector distance calculation, and large-scale use requires careful consideration of search speed, index size, and the load placed on PostgreSQL.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Vector databases and pgvector typically use approximate nearest neighbor indexes to accelerate search. These indexes improve speed, but there is generally a trade-off among search accuracy, speed, and storage size.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ConceptMiner Concept Index takes a different approach. It does not primarily compare every data record directly against every other record. Instead, it uses the relationship between all records and a few hundred to a few thousand conceptual nodes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For this reason, Concept Index is better suited to extracting the conceptual segment to which an input text or selected record belongs, rather than simply retrieving the single nearest record.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As a result, Concept Index aims to provide lightweight, explainable similarity search and conceptual classification inside existing RDBs.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Comparison Table<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Item<\/th><th>Vector Database<\/th><th>pgvector<\/th><th>Concept Index<\/th><\/tr><tr><td>External database required<\/td><td>Yes<\/td><td>No<\/td><td>No<\/td><\/tr><tr><td>Operation inside existing RDB<\/td><td>Weak<\/td><td>Strong<\/td><td>Strong<\/td><\/tr><tr><td>Individual Top-K search<\/td><td>High<\/td><td>High<\/td><td>Coarse without additional distance calculation<\/td><\/tr><tr><td>Segment extraction<\/td><td>Requires separate design<\/td><td>Requires separate design<\/td><td>Strong<\/td><\/tr><tr><td>Explainability<\/td><td>Relatively low<\/td><td>Relatively low<\/td><td>High<\/td><\/tr><tr><td>Storage volume<\/td><td>Can become large<\/td><td>Can become large<\/td><td>Can be kept compact<\/td><\/tr><tr><td>Initial development effort<\/td><td>Medium to high<\/td><td>Medium<\/td><td>Medium<\/td><\/tr><tr><td>Operational cost<\/td><td>Service fees and separate DB management<\/td><td>PostgreSQL load<\/td><td>PostgreSQL load plus update processing<\/td><\/tr><tr><td>Customer-facing explanation<\/td><td>AI search infrastructure<\/td><td>Vector search extension for PostgreSQL<\/td><td>Similarity search plus conceptual classification inside an RDB<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">ConceptMiner Concept Query<\/h1>\n\n\n\n<h2 class=\"wp-block-heading\">Planned Feature: Natural-Language Extraction of Conceptual Segments<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">As a future extension of ConceptMiner Concept Index, we are planning ConceptMiner Concept Query.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Concept Query will use the explainable segmentation obtained through GNG+MST to enable complex data extraction through natural-language instructions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, users will be able to retrieve relevant data groups simply by using expressions such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Customer segments with high profitability but also high churn risk<\/li>\n\n\n\n<li>Groups with low price sensitivity and a strong preference for premium products<\/li>\n\n\n\n<li>Segments with many inquiries but also high satisfaction<\/li>\n\n\n\n<li>Customers who respond well to new products but have low retention rates<\/li>\n\n\n\n<li>Small but unusual customer groups that make a large contribution to revenue<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This is not merely a function where AI automatically generates SQL.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">By using preclassified conceptual segments, Concept Query aims to make it possible to quickly extract groups such as \u201csemantically similar customer segments,\u201d \u201ccases with similar characteristics,\u201d or \u201cdata groups with similar behavioral patterns\u201d\u2014groups that are difficult to express using ordinary SQL alone.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Future Plan<\/h1>\n\n\n\n<h2 class=\"wp-block-heading\">Integration with Data Mining<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In its initial form, ConceptMiner Concept Index is designed to build a conceptual structure model from embeddings of text columns. In other words, it provides similarity search, or conceptual search, for text data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the future, we plan to extend this approach so that conceptual structure models can also be built from quantitative data, including numerical and categorical values.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is closely related to what has traditionally been called data mining.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Mindware Research Institute has been engaged in SOM-based data mining for more than 25 years. Through this experience, we have learned that data mining is, by nature, close to a scientific process of exploration, and that it is extremely difficult to practice effectively in ordinary business settings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ConceptMiner is being developed to use AI to support the mining process itself and to transform useful segmentation obtained through data mining into a form that can be used in everyday business operations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the future, we aim to integrate text data, numerical data, and categorical data, and to develop Concept Index into a practical data utilization platform for handling conceptual segments directly inside RDBs.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\"><a href=\"https:\/\/conceptminer.ai\/?page_id=581&amp;lang=en\" data-type=\"page\" data-id=\"581\">Delivery Format, Installation and Implementation Procedure<\/a><\/h2>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\">Limited Pre-Order Offer<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">ConceptMiner Concept Index is now available as a limited pre-order offer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Standard annual license fee: $2,500.00\/year<br>Pre-order price: $1,625.00, excluding tax<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">*The pre-order price applies only to the first year.<br>*The target database is PostgreSQL. Other databases are available upon consultation.<\/p>\n\n\n<p><script async\n  src=\"https:\/\/js.stripe.com\/v3\/buy-button.js\">\n<\/script><\/p>\n<p><stripe-buy-button\n  buy-button-id=\"buy_btn_1TogeNGGckwXcZG5OYWTchxA\"\n  publishable-key=\"pk_live_51ODHaoGGckwXcZG5lHDpXpLLRNikANIIXoZze14L10hd6BGUdXLdKQZa7PH6XVn7PRatpQzkJj73vQ0uddOS8TNL00VS90Pgxo\"\n><br \/>\n<\/stripe-buy-button><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Similarity Search and Conceptual Classification Inside Existing RDBs such as PostgreSQL ConceptMiner Concept Index is&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-563","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/conceptminer.ai\/index.php?rest_route=\/wp\/v2\/pages\/563","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/conceptminer.ai\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/conceptminer.ai\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/conceptminer.ai\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/conceptminer.ai\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=563"}],"version-history":[{"count":4,"href":"https:\/\/conceptminer.ai\/index.php?rest_route=\/wp\/v2\/pages\/563\/revisions"}],"predecessor-version":[{"id":584,"href":"https:\/\/conceptminer.ai\/index.php?rest_route=\/wp\/v2\/pages\/563\/revisions\/584"}],"wp:attachment":[{"href":"https:\/\/conceptminer.ai\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=563"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}