Query Manipulation Server 24.3

# Configure Generative Models in QMS

To use QMS to generate summaries, you must configure at least one generative model in the QMS configuration file.

You can configure QMS to use the following methods to generate embeddings:

- a generative LLM. You create a generative LLM model by configuring QMS to use an available third-party model. QMS downloads and caches the model when you first use it.

- a Lua or Python script model. You write a script that generates the summaries, for example by using a HTTP endpoint.

## Choose a Generative Model

The generative models use LLMs from Hugging Face. You can choose generative text2text-generation models (see https://huggingface.co/models?pipeline_tag=text2text-generation).

OpenText recommends the google/flan-t5-small model (see https://huggingface.co/google/flan-t5-small).

## Create a Lua Script Model

You can create your own Lua script model, using any method that you choose to generate the summaries. You can use the Lua model to interact or access third-party API summary generation.

The script must define a function called `summarize`. This function must accept a single parameter, which is a string representing the text to generate summaries for. It must return the output summary as a string.

For example:

    function summarize(text)
       return string.format("Here is your summary: %s, text")
    end

## Create a Python Script Model

You can create your own Python script model, using any method that you choose to generate the summaries. You can use the Python model to interact or access third-party API summary generation.

The script must define a function called `summarize`. This function must accept a single parameter, which is a string representing the text to generate summaries for. It must return the output summary as a string.

For example:

    def summarize(text: str) -> str:
        '''
        Sample summarization function.
        '''
        return f'Here is your summary: {text}'

TIP: You can provide a `requirements.txt` file for any third-party modules that you want to use, by setting the RequirementsFile parameter in your model configuration. See Configure QMS with the Generative Model.

## Configure QMS with the Generative Model

You configure embeddings in the `[Generative]` section of the QMS configuration file. See Generative Configuration Parameters.

To configure a generative model

1. Open your configuration file in a text editor.

2. Find the `[Generative]` section, or create one if it does not exist.

3. Add an entry for your embedding model by using a zero-based index number. Set this value to the name of the configuration section to use to define your model. For example:

       [Generative]
       0=Summarizer

4. Create a configuration section with the name that you defined.

       [Summarizer]

5. Set the Type parameter to the type of model you want to use. For example:

       [Summarizer]
       Type=GenerativeLLM

6. Set additional parameters for your model. The required parameters depend on the type of model:

   - GenerativeLLM models require you to set ModelName. You can set AccessToken to use a private model. You can also set CacheDirectory, ChunkSize, Device, GreedySearch, MinFinalSummaryChunkSize, ModelRevision, and OfflineMode.

   - GenerativeLua models require you to set Script.

   - GenerativePython models require you to set Script. You can also optionally set RequirementsFile.

7. Save and close the configuration file.

Copyright 2024 Open Text
Version 24.3 | Last updated June 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Query Manipulation Server 24.3

Topic Title: Configure Generative Models in QMS