Answer Server 24.4
IDOL Answer Server Help

# Configure Candidate Retrieval for Passage Extractor

When you send an Ask action to your Passage Extractor or Passage Extractor LLM system, Answer Server performs the following steps:

1. Question Classification. Answer Server analyzes the input question to classify the key elements of the question.

2. Candidate Retrieval. Answer Server uses the information from the classified question to send a query to the IDOL Content component to retrieve documents that might contain an answer.

3. Answer Generation. Answer Server finds the likely answer from the candidate documents, and returns it.

For the candidate retrieval step, and for the answer generation step you can choose whether to use the standard IDOL conceptual approach, or to use a third-party LLM approach.

To use an LLM approach for the answer generation step, you configure a Passage Extractor LLM system (see Configure the Passage Extractor LLM System). To use the IDOL conceptual approach, you configure a standard Passage Extractor system (see Configure the Passage Extractor System).

To configure candidate retrieval, you can use different settings in your system, depending on the approach you want to use.

## IDOL Conceptual Candidate Retrieval

The default method of candidate retrieval uses IDOL conceptual search to retrieve answers from your IDOL Content component index. You must use this option when you do not have vectors indexed in your IDOL Content component.

Conceptual candidate retrieval does not require any additional configuration, but you can add options to refine your results. For example, you can set ACIMaxResults to set a maximum number of results to retrieve, and you can set CandidateRetrievalDefaults to a configuration section that defines other parameters to use in the IDOL Content component query.

## IDOL Vector Search Candidate Retrieval

You can use IDOL vector search when your IDOL Content component contains vectors in `VectorType` fields. For more information about the IDOL Content component vector search, refer to the IDOL Content component Help.

To configure Answer Server to send vector searches to IDOL, you must do some additional configuration in your Passage Extractor system configuration:

- Set AnswerCandidateEmbeddingsSettings to enable vector search. This parameter points to a configuration section where you define additional settings for the vector search.

- Create a vector settings configuration section. This section must set VectorField to define the `VectorType` field to query in the IDOL Content component index, and settings for the model to use to generate embeddings for the query text that Answer Server creates from your questions.

- Create an embedding generation configuration section. This section provides details of the embedding generation model that you want to use. You can download and use a third-party model from Hugging Face (see Choose an Embedding Model ), or you can use a Lua or Python script for embedding generation (see Create an Embedding Generation Lua Script and Create an Embedding Generation Python Script).

NOTE: When using vector search, you can still use the other candidate retrieval configuration parameters. For example, ACIMaxResults to set a maximum number of results to retrieve, and you can set CandidateRetrievalDefaults to a configuration section that defines other parameters to use in the IDOL Content component query.

### Choose an Embedding Model

The sentence transformer models use vector models from Hugging Face. You can choose any sentence-transformer model.

OpenText recommends the sentence-transformers/sentence-t5-large model (see https://huggingface.co/sentence-transformers/sentence-t5-large).

### Create an Embedding Generation Lua Script

You can create your own Lua script model, using any method that you choose to generate the embedding data. You can use the Lua model to interact or access third party API embedding generation, such as the Hugging Face Inference API, or Inference Endpoints service.

The script must define a function called `generateembeddings`. This function must accept a single parameter, which is a string representing the text to generate embeddings for.

The function must return a table of tables, where each inner table contains floating point numbers. Each inner table therefore corresponds to a vector for that text.

You can optionally include a second table of tables, which returns offsets for the embeddings in the format:

    {{embedding_1_start_offset, embedding_1_end_offset}, {embedding_2_start_offset, embedding_2_end_offset}, {embedding_3_start_offset, embedding_3_end_offset}, ...},

NOTE: If you want to use offset information in IDOL (for indexing or querying), you must use UTF-8 byte offsets.

For example:

    function generateembeddings(text)
       return {{1,2,3,4}, {5,6,7,8}, {9,10,11,12}}, {{34,48}, {48, 124}, {124, 156}}
    end

In this example, the function returned three embeddings in the first table (with three nested tables). The second table is the offset values. The vector `{1,2,3,4}` starts at offset 34 and ends at 48. The second vector `{5,6,7,8}` starts at 48 and ends at 124, and so on.

### Create an Embedding Generation Python Script

You can create your own Python script model, using any method that you choose to generate the embedding data. You can use the Python model to interact or access third party API embedding generation, such as the Hugging Face Inference API, or Inference Endpoints service.

The script must define a function called `generate_embeddings`. This function must accept a single parameter, which is a UTF-8 string that represents the text to generate embeddings for.

Your script must handle any chunking of the text to generate the embeddings.

The function must return

- a list of lists of floats, which is the generated embeddings

- a tuple that consists of:

  - a list of lists of floats, which is the generated embeddings

  - a list of lists of integers, which are the offset values for the text chunks that were used to create the embeddings

    NOTE: If you want to use offset information in IDOL (for indexing or querying), you must use UTF-8 byte offsets.

For example:

    def generate_embeddings(input_text):
        return [[-0.1, 0.2, -0.3, 0.4, -0.5]]

TIP: You can provide a `requirements.txt` file for any third-party modules that you want to use, by setting the RequirementsFile parameter in your model configuration. See Configure Answer Server for Embedding Generation and Search.

### Configure Answer Server for Embedding Generation and Search

You must configure your embedding generation in the Answer Server configuration file, with any suitable configuration section name. You then refer to this from your passage extractor system.

To configure Answer Server to generate embeddings and use vector search

1. Open your configuration file in a text editor.

2. Create a new configuration section for your embedding generation model, for example:

       [EmbeddingsGenerator]

3. Set the Type parameter to the type of model you want to use. For example:

       [EmbeddingsGenerator]
       Type=Transformer

4. Set additional parameters for your model. The required parameters depend on the type of model:

   - Transformer models require you to set ModelName. You can set AccessToken to use a private model. You can optionally also set CacheDirectory, DataLimit, Device, EmbeddingPrecision, ModelMaxSequenceLength, ModelMinimumFinalSequenceLength, ModelRevision, ModelSequenceOverlap, and OfflineMode.

   - Lua models require you to set Script. You can also optionally set EmbeddingPrecision.

   - Python models require you to set Script. You can also optionally set EmbeddingPrecision.

5. Find the Passage Extractor or Passage Extractor LLM system that you want to configure to use vector search.

6. Set the AnswerCandidateEmbeddingsSettings configuration parameter to the name of a configuration section where you define vector search settings. For example:

       AnswerCandidateEmbeddingsSettings=VectorSettings

7. Create a new configuration section with the name you set.

8. In this new section, set EmbeddingsConfig to the name of your embeddings generator configuration section. Set VectorField to the name of the IDOL Content component `VectorType` field that contains the vector values that you want to search. For example:

       [VectorSettings]
       EmbeddingsConfig=EmbeddingsGenerator
       VectorField=VECTORA

9. Save and close the configuration file.

For example:

    [PassageExtractorSystem]
    IDOLHost=localhost
    IDOLACIPort=6002
    Type=PassageExtractor
    ...
    AnswerCandidateEmbeddingsSettings=VectorSettings

    [VectorSettings]
    EmbeddingsConfig=EmbeddingsGenerator
    VectorField=VECTORA

    [EmbeddingsGenerator]
    Type=Transformer
    ModelName=sentence-transformers/sentence-t5-large
    ModelMaxSequenceLength=128

Copyright 2016-2024 Open Text
Version 24.4 | Last updated September 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Answer Server 24.4

Topic Title: Configure Candidate Retrieval for Passage Extractor