IDOL Server 24.2

# Vector Search

A vector query returns documents that contain an indexed vector that is similar to a vector that you provide.

A vector is an array of numbers that represents an object, which might be text or an image. You can generate vectors by using many different models, and index them into IDOL in VectorType fields.

The Query action `VECTOR` operator allows you to search for vectors in your documents that are similar to the vector that you provide.

The syntax for the `VECTOR` operator is:

    text=VECTOR{v1,v2,...,vn:maxdist}[:maxneighbors]:fieldname

where,

`v1,v2,...,vn` | is the comma-separated list of values that makes up the vector for which you want to find similar vectors. The vector that you provide must be from the same model, and have the same number of dimensions, as the vectors that you use in your index.
`maxdist` | (optional) is the maximum distance from the search vector that you want to find. IDOL calculates this distance according to the value you set for DistanceMetric, which you set according to the requirements of your vector model. The default value is positive infinity (that is, there is no maximum distance). When DistanceMetric is set to L2 or Cosine, you can also use any negative number to mean there is no maximum distance.
`maxneighbors` | (optional) is the maximum number of similar vectors to return in the results.
`fieldname` | is the name of the field in your indexed documents that contains the vectors that you want to search.

For a vector search, IDOL Server returns the documents where the specified field contains the closest vectors to the vector that you provide, according to the DistanceMetric. If a document contains more than one instance of the vector field, IDOL Server uses the closest one to the search vector.

## Vector Search with Section-Breaking

When you use section breaking in your documents, the vector search can return the document ID of the matching section, as long as source and offset metadata is available in the vector field.

IDOL Server uses the source metadata to determine that the embedding was generated from text in the section-breaking field. It then uses the offset values in your vector field to calculate the section that the vector source text is from. IDOL Server calculates offsets relative to the start of the whole text of the field (before section-breaking), and uses the configured section size to calculate the correct section.

For example, if the `DRECONTENT` field is split into sections of 1000 characters each, `source=DRECONTENT[123:456]` represents an embedding generated from section 0, and `source=DRECONTENT[2234:2345]` represents an embedding generated from section 2.

NOTE: If the source offsets for a single embedding span two or more sections, only the first of these sections returns when this embedding matches a `VECTOR` query.

## Highlight Values in Vector Search

When you use the Highlight parameter with a `VECTOR` query, IDOL Server highlights the appropriate section of the source text for the vector, as long as you print the source field in the query results.

Vector highlighting works for all highlight options. For example, with `Sentences` highlighting, IDOL Server highlights all sentences that overlap with the vector source text.

For `Terms` highlighting, Content highlights only entire words, even if the vector offset is in the middle of a word.

NOTE: `Proximity` highlighting is the same as terms highlighting, as there is no defined proximity between vector and non-vector fields.

Copyright 2024 Open Text
Version 24.2 | Last updated March 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Server 24.2

Topic Title: Vector Search