Query Manipulation Server 24.3

# Configure Embeddings in QMS

To use QMS to generate embeddings, you must configure at least one embedding model in the QMS configuration file.

You can configure QMS to use the following methods to generate embeddings:

- a sentence transformer model. You create a sentence transformer model by configuring QMS to use an available third-party model. QMS downloads and caches the model when you first use it.

- a Lua or Python script. You write a script that generates the embeddings, by whatever method you want to use.

## Choose a Transformer Model

The sentence transformer models use vector models from Hugging Face. You can choose any sentence-transformer model.

OpenText recommends the sentence-transformers/sentence-t5-large model (see https://huggingface.co/sentence-transformers/sentence-t5-large).

## Create a Lua Script Model

You can create your own Lua script model, using any method that you choose to generate the embedding data. You can use the Lua model to interact or access third party API embedding generation, such as the Hugging Face Inference API, or Inference Endpoints service.

The script must define a function called `generateembeddings`. This function must accept a single parameter, which is a string that represents the text to generate embeddings for.

The function must return a table of tables, where each inner table contains floating point numbers. Each inner table therefore corresponds to a vector for that text.

You can optionally include a second table of tables, which returns offsets for the embeddings in the format:

    {{embedding_1_start_offset, embedding_1_end_offset}, {embedding_2_start_offset, embedding_2_end_offset}, {embedding_3_start_offset, embedding_3_end_offset}, ...},

NOTE: If you want to use offset information in IDOL (for indexing or querying), you must use UTF-8 byte offsets.

For example:

    function generateembeddings(text)
       return {{1,2,3,4}, {5,6,7,8}, {9,10,11,12}}, {{34,48}, {48, 124}, {124, 156}}
    end

In this example, the function returned three embeddings in the first table (with three nested tables). The second table is the offset values. The vector `{1,2,3,4}` starts at offset 34 and ends at 48. The second vector `{5,6,7,8}` starts at 48 and ends at 124, and so on.

## Create a Python Script Model

You can create your own Python script model, using any method that you choose to generate the embedding data. You can use the Python model to interact or access third party API summary generation.

The script must define a function called `generate_embeddings`. This function must accept a single parameter, which is a UTF-8 string that represents the text to generate embeddings for.

Your script must handle any chunking of the text to generate the embeddings.

The function must return

- a list of lists of floats, which is the generated embeddings

- a tuple that consists of:

  - a list of lists of floats, which is the generated embeddings

  - a list of lists of integers, which are the offset values for the text chunks that were used to create the embeddings

    NOTE: If you want to use offset information in IDOL (for indexing or querying), you must use UTF-8 byte offsets.

For example:

    def generate_embeddings(input_text):
        return [[-0.1, 0.2, -0.3, 0.4, -0.5]]

TIP: You can provide a `requirements.txt` file for any third-party modules that you want to use, by setting the RequirementsFile parameter in your model configuration. See Configure QMS with the Embeddings Model.

## Configure QMS with the Embeddings Model

You configure embeddings in the `[Embeddings]` section of the QMS configuration file. See Embeddings Configuration Parameters.

To configure an embedding model

1. Open your configuration file in a text editor.

2. Find the `[Embeddings]` section, or create one if it does not exist.

3. Add an entry for your embedding model by using a zero-based index number. Set this value to the name of the configuration section to use to define your model. For example:

       [Embeddings]
       0=SentenceTransformer

4. Create a configuration section with the name that you defined.

       [SentenceTransformer]

5. Set the Type parameter to the type of model you want to use. For example:

       [SentenceTransformer]
       Type=Transformer

6. Set additional parameters for your model. The required parameters depend on the type of model:

   - Transformer models require you to set ModelName. You can set AccessToken to use a private model. You can optionally also set CacheDirectory, DataLimit, Device, EmbeddingPrecision, ModelMaxSequenceLength, ModelMinimumFinalSequenceLength, ModelRevision, ModelSequenceOverlap, and OfflineMode.

   - Lua models require you to set Script. You can also optionally set EmbeddingPrecision.

   - Python models require you to set Script. You can also optionally set RequirementsFile and EmbeddingPrecision.

7. Save and close the configuration file.

Copyright 2024 Open Text
Version 24.3 | Last updated June 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Query Manipulation Server 24.3

Topic Title: Configure Embeddings in QMS