IDOL View Component 23.4

# Vector Search

A vector search compares vectors that you index in your IDOL documents with a vector that you use to query. A vector is an array of floating point numbers that represents an object, which might be text or an image. You can generate vectors by using many different models, and index them into IDOL in `VectorType` fields.

## Configure Vector Search

To perform vector comparisons in queries, you must configure `VectorType` and index your documents with the vector values in these fields.

To configure vector fields

1. Open the IDOL Content component configuration file in a text editor.

2. In the `[FieldProcessing]` section of the IDOL Content component configuration file, set up a new field process for your vector fields. For example:

       [FieldProcessing]
       0=VectorFields

3. Create a section for the synonym process you listed.

   - Set `Property` to the name of the property configuration section for the process. The property that you create must not have the same name as the process.

   - Set `PropertyFieldCSVs` to the list of fields that you want to associate with the process.

     To identify the fields, use the format `/FieldName` to match root-level fields, `*/FieldName` to match all fields except root-level, or `/Path/FieldName` to match fields that the specified path points to.

   For example:

       [VectorFields]
       Property=VectorProperties
       PropertyFieldCSVs=*/VECTOR*

   In this example, Content indexes values in the `VECTOR` field as vectors for vector distance comparisons.

4. Create a section for the property. In this section, set `VectorType` to `True`.

       [VectorProperties]
       VectorType=True

5. (Optional) Change the distance metric to use for your vector comparisons, by adding or modifying the value of the `DistanceMetric` configuration parameter. For example:

       [VectorIndex]
       DistanceMetric=L2

   NOTE: The value that you need to use depends on the vector model that you use to generate your vectors. To find out what distance metric to use, refer to the documentation for your vector model.

6. (Optional) If you already have documents in your IDOL index, you can configure IDOL to regenerate the vector index for these documents by using the `RegenerateVectorIndex` configuration parameter.

       [Server]
       ...
       RegenerateVectorIndex=True

   TIP: Alternatively, you can regenerate the vector index by using the `DREREGENERATE` index action.

7. Save and close the configuration file.

8. Restart the IDOL Content component for your changes to take effect.

## Generate Vectors and Index Documents

After you configure vectors in the IDOL Content component, you can index documents that contain these vector fields.

You can generate vectors by using a third-party model. You then add the vector to a field in your IDOL documents.

A vector in a document is a comma-separated list of floating point values. For example, in an IDX field, you might have the following value:

    #DREFIELD VECTOR="0.24275297, 0.6721721, 0.38784778, 0.9476961, 0.7166045, 0.91357803, 0.1593279, 0.8701281, 0.5187642 , 0.6577334 , 0.60528547, 0.83356684, 0.33284906, 0.0672832 , 0.957869  , 0.33899164, 0.8574585 , 0.08680364, 0.16952382, 0.07177012, 0.6952732 , 0.9542396 , 0.62435275, 0.9369151, 0.75291044, 0.6654858, 0.9690549, 0.96977615, 0.93315464, 0.36870354, 0.77450484, 0.8728181"

All vectors in a particular field name must have the same number of dimensions (the previous example has 32 dimensions).

A document can contain multiple vectors with the same field name.

## Send Vector Queries

To use vector search, you use the `VECTOR` operator in the `Query` action `Text` parameter.

The `VECTOR` operator allows you to search for vectors in your documents that are similar to the vector that you provide. The syntax for the `VECTOR` operator is:

    action=Query&text=VECTOR{v1,v2,...,vn:maxdist}[:maxneighbors]:fieldname

For more information about this syntax, see the IDOL Content component Reference.

You can also use vectors for the `Suggest` action, to suggest documents that contain vectors similar to the vectors in the document you provide. In this case, you must set the `UseVectors` action parameter to `True`. For example:

    action=Suggest&ID=200&UseVectors=True

Copyright 2023 Open Text
Version 23.4 | Last updated September 2023

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to swpdl.idoldocsfeedback@microfocus.com.

We appreciate your feedback!

Product: IDOL IDOL View Component 23.4

Topic Title: Vector Search