Answer Server 24.4
IDOL Answer Server Help

# Configure the Passage Extractor LLM System

The Answer Server configuration file contains information about the subcomponents in your Passage Extractor LLM systems.

For any Passage Extractor LLM system, you must configure the host and port details of your data store, which is an IDOL Content component that contains the documents that Answer Server uses to find answers. You must also configure the LLM module to use to generate or extract answers.

Passage extractor LLM also uses question classifiers, to determine the type of a question. The classifier is required. The Answer Server installation includes classifiers for some languages, but for others you must train a classifier yourself.

The following procedure describes how to configure the Passage Extractor LLM system in Answer Server.

For more details about the configuration parameters for the Passage Extractor LLM system, see Passage Extractor LLM System Configuration Parameters.

To configure the Passage Extractor System

1. Open the Answer Server configuration file in a text editor.

2. Find the `[Systems]` section, or create one if it does not exist. This section contains a list of systems, which refer to the associated configuration sections for each system.

3. After any existing systems, add an entry for your new Passage Extractor system. For example:

       [Systems]
       0=MyAnswerBank
       1=MyFactBank
       2=MyPassageExtractor
       3=MyPassageExtractorLLM

4. Create a configuration section for your Passage Extractor LLM system, with the name that you specified. For example, `[MyPassageExtractor` `LLM``]`.

5. Set Type to `PassageExtractorLLM`.

6. Set IDOLHost and IDOLACIPort to the host name and ACI Port of the IDOL Content component that contains the documents that you want to use to find answers.

   NOTE: If you want to use synonyms to expand queries, set these parameters to the host and port of the Query Manipulation Server (QMS) that provides access to your synonyms. Set the host and port of the Content component in the QMS configuration file instead. For more information about how to enable synonyms, see Use Synonyms to Expand Queries.

7. Set ModuleID to the name of the configuration section that provides details of the LLM module to use. For information about how to configure this, see Configure the LLM Module.

8. Set the ClassifierFile parameter to the path of the question classifier file, and set LabelFile to the path of the label file.

   TIP: The Answer Server installation includes classifier and labels files for English and German. For example, to use the default files for the English language, set ClassifierFile to the location of the `svm_en.dat` file, and set LabelFile to the location of the `labels_en.dat` file.

   If you want to train your own classifier or are configuring a Passage Extractor system for use with another language, set the ClassifierFile and LabelFile parameters to the locations where you want Answer Server to save the question classifier and label files, when you perform training. For information about how to train classifiers, see Train Passage Extractor Classifiers.

9. Save and close the configuration file.

10. Restart Answer Server for your changes to take effect.

For example:

    [MyPassageExtractor]
    Type=PassageExtractorLLM
    // Data store IDOL
    IdolHost=localhost
    IdolAciport=6002
    // Classifier Files
    ClassifierFile=./passageextractor/classifiertraining/svm_en.dat
    LabelFile=./passageextractor/classifiertraining/labels_en.dat

    // Module to use
    ModuleID=LLMExtractiveQuestionAnswering

    [LLMExtractiveQuestionAnswering]
    Type=ExtractiveQuestionAnsweringLLM
    ModelName=distilbert/distilbert-base-uncased

## Change the Passage Extractor Language

The default installation of passage extractor includes example question classifier training files for English. To use passage extractor in another language, you must:

- create a new question classifier in the new language. See Train Passage Extractor Classifiers.
- set the `Language` configuration parameter to the appropriate language, either in the `[Server]` section (to set the language for all of Answer Server), or in the passage extractor LLM system configuration section (to set the language for just passage extractor). You might also want to set the StopList parameter. See Language Configuration.

## Configure the LLM Module

For a passage extractor LLM system, you must have an LLM module that defines the model that Answer Server must use. You can use the following types of models:

- Extractive or Generative LLM. These options use a publicly available model from Hugging Face. See Choose a Question Answering Model.

- Extractive or Generative Lua Script. These options use a Lua script to produce answers. For example, you might want to use your Lua script to generate answers using an HTTP endpoint. For these modules, Answer Server calls the Lua script and processes the response to use as the answers to a question. See Create a Question Answering Lua Script.

- Generative Python Script. This option uses a Python script to produce answers. For example, you might want to use your Python script to generate answers using an HTTP endpoint. For this module, Answer Server calls the Python script and processes the response to use as the answers to a question. See Create a Question Answering Python Script.

The extractive question answering types find the answers in your documents and return the exact text as it occurs in the original text. The generative models can generate new text to answer the question, based on the information that it finds in your documents.

### Choose a Question Answering Model

The extractive and generative models use LLMs from Hugging Face. The types of models that you can use depend on the question answering method.

TIP: To find the type of a particular model, check the value of `model_type` in the `config.json` file on the web page for the model on Hugging Face.

- For Extractive Question Answering, you can use any model with a `model_type` of `question-answering`, except for layoutlmv2, roformer, and lxmert.

- For Generative Question Answering, you can use any model with a `model_type` of `text2text-generation`, except for encoder-decoder.

NOTE: The first time you use a particular model, Answer Server downloads and caches it (for example, when you first send a question to the passage extractor LLM system).

If Answer Server does not have an internet connection, you can use these question answering systems only if you have an existing cached model.

### Create a Question Answering Lua Script

You can configure a Lua script to perform generative or extractive question answering. For example, you might want to use your Lua script to generate answers using an HTTP endpoint.

For extractive question answering, your script must define a function called `perform_extractive_qa`, which must accept two parameters:

- the first parameter is a string that represents the question.

- the second parameter is a string that defines the context from which the answer must be extracted.

The function must return a string that represents the answer to the question that has been extracted from the context. It can optionally also return a float between 0 and 1 corresponding to a score for the answer.

For example:

    function perform_extractive_qa(question, context)
       return context, 0.95
    end

For generative question answering, your script must define a function called `perform_generative_qa`, which must accept two parameters:

- the first parameter is a string that represents the question.

- the second parameter is a string that defines the context to base the answer on.

The function must return a string that represents the answer to the question, based on the provided context.

For example:

    function perform_generative_qa(question, context)
       return string.format("Here is your answer: %s", string.sub(context, 1, 100))
    end

### Create a Question Answering Python Script

You can configure a Python script to perform generative question answering. For example, you might want to use your Python script to generate answers by using an HTTP endpoint.

For generative question answering, your script must define a function called `perform_generative_qa`, which must accept two parameters:

- the first parameter is a string that represents the question.

- the second parameter is a string that defines the context to base the answer on.

The function must return a string that represents the answer to the question, based on the provided context.

For example:

    def perform_generative_qa(question: str, context: str) -> str:
       '''
       Sample generative question answering function.
       '''
       return f'Here is your answer: {context[:100]}'

TIP: You can provide a `requirements.txt` file for any third-party modules that you want to use, by setting the RequirementsFile parameter in your model configuration. See Configure the LLM Module.

### Configure the LLM Module

After you create your model files, or your Lua or Python script, you must configure an LLM module section in your configuration file, which you refer to in your Passage Extractor LLM system configuration.

To configure the LLM Module

1. Open the Answer Server configuration file in a text editor.

2. Create a new configuration section for your LLM module. This is the configuration section name that you use in the ModuleID parameter when you configure the passage extractor LLM system. For example `[LLMExtractiveQA]`

3. In the new configuration section, set the Type parameter to one of the following values:

   - ExtractiveQuestionAnsweringLLM. Use an extractive LLM to extract answers from IDOL documents.

   - GenerativeLLM. Use a generative LLM to generate answers based on information in your IDOL documents.

   - ExtractiveQuestionAnsweringLua. Use a Lua script to extract answers from IDOL documents.

   - GenerativeLua. Use a Lua script to generate answers based on information in your IDOL documents.

   - GenerativePython. Use a Python script to generate answers based on information in your IDOL documents.

4. Set the required parameters for your module type:

   - For a module that uses an LLM (that is, when Type is set to ExtractiveQuestionAnsweringLLM or GenerativeLLM), set ModelName to the model that you want to use. You can set AccessToken to use a private model. You can optionally set CacheDirectory, Device, ModelRevision, and OfflineMode. For GenerativeLLM, you can also set ChunkSize and GreedySearch.

   - For a module that uses a Lua script (that is, when Type is set to ExtractiveQuestionAnsweringLua or GenerativeLua), set Script to the path and file name for your Lua script.

   - For a module that uses a Python script (that is when Type is set to GenerativePython), set Script to the path and file name for your Python script. You can optionally set RequirementsFile to define modules that your script uses.

   For example:

       [LLMExtractiveQuestionAnswering-Small]
       Type=ExtractiveQuestionAnsweringLLM
       ModelName=distilbert/distilbert-base-uncased

       [LLMGenerativeQuestionAnswering-Large]
       Type=GenerativeLLM
       ModelName=google/flan-t5-base

       [ExtractiveQuestionAnsweringLuaScript]
       Type=ExtractiveQuestionAnsweringLua
       Script=LLMscripts/extractive_script.lua

       [GenerativeQuestionAnsweringLuaScript]
       Type=GenerativeLua
       Script=LLMscripts/generative_script.lua

       [GenerativeQuestionAnsweringPythonScript]
       Type=GenerativePython
       Script=LLMscripts/generative_script.py

5. Save and close the configuration file.

Copyright 2016-2024 Open Text
Version 24.4 | Last updated September 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Answer Server 24.4

Topic Title: Configure the Passage Extractor LLM System