Home / Delivery Format, Installation and Implementation Procedure

Delivery Format, Installation and Implementation Procedure

Delivery Format

ConceptMiner Concept Index is provided as ConceptMiner Connector.

ConceptMiner Connector is an externally executed software component that connects to the customer’s existing RDB, retrieves the result set of a specified View or SQL query, and performs indexing based on ConceptMiner’s conceptual structure model.

The Connector is provided as a Windows/Linux executable file or as a Docker container. Customers do not need to install Python or set up a machine learning environment separately.

By installing the Connector, configuring the ConceptMiner API key, entering the database connection information, and specifying the target View or SQL query, customers can generate Concept Index tables inside their existing RDB.

ConceptMiner Concept Index does not replace existing business systems or databases. It adds conceptual index information to the existing RDB, making it possible to perform conceptual similarity search and segment extraction.

The standard target database is PostgreSQL. Other RDBs are available upon consultation, depending on the connection method and SQL specifications.

Role of ConceptMiner Connector

ConceptMiner Connector acts as an intermediate component between the existing RDB and the ConceptMiner API.

The Connector retrieves target data from the specified View or SQL query and sends it to the ConceptMiner API. The ConceptMiner API builds a conceptual structure model based on embeddings of the text column and assigns each data record to a corresponding conceptual node.

The results are then written back into the RDB by the Connector. As a result, information such as the relationship between each data record and its conceptual node, node information, and neighborhood relationships among nodes can be used as ordinary RDB tables.

The generated Concept Index tables can be referenced from ordinary SQL, BI tools, business systems, management screens, and other existing applications. This enables users to extract data records that belong to the same conceptual node, or similar data groups that belong to neighboring nodes, directly inside the existing RDB.

Installation and Implementation Procedure

ConceptMiner Concept Index is installed and implemented through the following steps.

1. Install the Connector

First, install ConceptMiner Connector in the customer’s environment.

The Connector is executed in an environment that can access the target RDB. It is provided as a Windows/Linux executable file, and Docker container deployment is also available when required.

The Connector is externally executed software. It is not installed as a PostgreSQL extension. Therefore, it can be introduced without making major changes to the existing database configuration.

2. Configure the API Key

Next, configure the ConceptMiner API key in the Connector.

The Connector uses this API key to access the ConceptMiner API and create a conceptual structure model for the target data.

The API key can be registered through a configuration file or the Connector’s setup screen.

3. Configure Database Connection Information

Enter the connection information for the target RDB into the Connector.

This information includes the database host name, port number, database name, user name, password, and other required connection settings.

During the initial setup, cooperation from the information systems department or database administrator may be required in order to confirm database connection information and access privileges.

However, once the setup is complete, execution and update operations are designed to be performed by business users without programming.

4. Specify the Target View or SQL Query

Specify the target data for Concept Index creation as a View or SQL query.

The target data should include at least the following information:

  • An ID that uniquely identifies each record
  • A text column to be used for similarity search and conceptual classification
  • Optional attributes for display, filtering, or analysis

Examples of target data include inquiry histories, Voice of Customer records, open-ended survey responses, sales notes, case summaries, and similar text-based business records.

In actual operation, it is often safer for the information systems department or database administrator to prepare the target data in advance as a View. In that case, business users can create a Concept Index simply by selecting the prepared View and running the Connector.

5. Run the Connector

After the settings are complete, run the Connector.

The Connector retrieves target data from the specified View or SQL query and sends it to the ConceptMiner API. The ConceptMiner API builds a conceptual structure model from embeddings of the text column and assigns each data record to a corresponding conceptual node.

Users do not need to write any programs. They only need to confirm the settings and run the Connector to start the process.

6. Create Concept Index Tables in the RDB

When processing is complete, the Connector receives the results from the ConceptMiner API and creates Concept Index tables inside the RDB.

The generated tables mainly include the following information:

  • Relationships between data records and conceptual nodes
  • Node IDs
  • Representative information for each node
  • Neighborhood relationships among nodes
  • Segment information, if required
  • Execution history and model information

This makes it possible to extract records that belong to the same node, or records that belong to neighboring nodes, using ordinary SQL inside the existing RDB.

Usage After Installation

Once the Concept Index tables have been created, conceptual similarity search becomes available inside the existing RDB.

For example, a user can select an inquiry record and extract other inquiries that have the same node ID. The user can also expand the search range to neighboring nodes and retrieve a broader group of semantically similar inquiries.

When searching by arbitrary input text, the Connector or ConceptMiner API estimates the node corresponding to the input text. The system then retrieves records belonging to that node or to neighboring nodes from the RDB.

Through this mechanism, users can access conceptually similar data groups through ordinary SQL, existing management screens, BI tools, or internal business systems.

Update Processing

Concept Index can be re-executed according to the update status of the target data.

When data additions or updates are infrequent, the Connector can be executed manually or as a batch process whenever needed. When data is added regularly, the Connector can be scheduled to run daily, weekly, monthly, or at another suitable interval to update the Concept Index tables.

Two types of operation are possible: assigning newly added records to an existing model, or rebuilding the entire conceptual structure model at regular intervals.

The actual operation method is adjusted according to the number of records, update frequency, and intended use.

Information Required for Installation

The following information is required for installation:

  • Type of target database
  • Database connection information
  • Target View or SQL query for creating the Concept Index
  • Column used as the record ID
  • Text column used for similarity search and conceptual classification
  • Number of records
  • Data update frequency
  • Whether the generated Concept Index will be used from business systems, BI tools, or other applications

Based on this information, ConceptMiner Connector is configured and Concept Index tables are created inside the existing RDB.

<< Back