Query Manipulation Server 24.2

# Configure Generative Models in QMS

To use QMS to generate summaries, you must configure at least one generative model in the QMS configuration file.

You can configure QMS to use the following methods to generate embeddings:

- a generative LLM. You create a generative LLM model by configuring QMS to use an available third-party model. QMS downloads and caches the model when you first use it.

- a Lua script model. You write a script that generates the summaries, for example by using a HTTP endpoint.

## Choose a Generative Model

The generative models use LLMs from Hugging Face. You can choose generative t5 models.

OpenText recommends the google/flan-t5-small model (see https://huggingface.co/google/flan-t5-small).

## Create a Lua Script Model

You can create your own Lua script model, using any method that you choose to generate the embedding data. You can use the Lua model to interact or access third party API summary generation.

The script must define a function called `summarize`. This function must accept a single parameter, which is a string representing the text to generate summaries for. It must return the output summary as a string.

For example:

    function summarize(text)
       return string.format("Here is your summary: %s, text")
    end

## Configure QMS with the Generative Model

You configure embeddings in the `[Generative]` section of the QMS configuration file. See Generative Configuration Parameters.

To configure a generative model

1. Open your configuration file in a text editor.

2. Find the `[Generative]` section, or create one if it does not exist.

3. Add an entry for your embedding model by using a zero-based index number. Set this value to the name of the configuration section to use to define your model. For example:

       [Generative]
       0=Summarizer

4. Create a configuration section with the name that you defined.

       [Summarizer]

5. Set the Type parameter to the type of model you want to use. For example:

       [Summarizer]
       Type=GenerativeLLM

6. Set additional parameters for your model. The required parameters depend on the type of model:

   - GenerativeLLM models require you to set ModelName. You can also set CacheDirectory, ChunkSize and Device.

   - GenerativeLua models require you to set Script.

7. Save and close the configuration file.

Copyright 2024 Open Text
Version 24.2 | Last updated March 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Query Manipulation Server 24.2

Topic Title: Configure Generative Models in QMS