Improving AI-based Test Object Identification

This topic discusses some of the elements that the UFT Developer AI feature uses to support unique identification of objects in your application.

Associating text with objects

The text associated with an object can help identify an object uniquely. For example:

desktop.findWithAI( "input" , "USER NAME" ).typeKeys( "admin" )

Using the "USER NAME" text in this step to describe the object, ensures that we type in the correct input field.

When detecting objects in an application, if there are multiple labels around the field, UFT Developer uses the one that seems most logical for the object's identification. However, if you decide to use a different label in your object description, UFT Developer still identifies the object.

Note: UFT Developer uses a fuzzy matching algorithm, so you can replace about 20 percent of the characters with for example '*' or blanks and still get the same result.

Example: When detecting objects in an application, a button is associated with the text on the button, but a field is associated with its label, as opposed to its content.

If a field has multiple labels and UFT Developer chooses one for detection, UFT Developer will still identify this field correctly when running a test step that uses a different label to describe the field.

Identifying objects by relative location

UFT Developer must be able to identify an object uniquely to run an action on the object. When multiple objects match your object description, you can add the object location to provide a unique identification. The location can be ordinal, relative to similar objects in the application, or proximal , relative to a different AI object, considered an anchor.

The findWithAI object identification allows to use occurrence and Direction arguments in which you can provide information about the object's ordinal location.

The findWithAIAnchor object identification allows to use Position and anchor arguments in which you can provide information about the object's proximal location.

To describe an object's ordinal location provide the following information:

The object's occurrence, is it first, second, third, and so on.
The orientation, the direction from which to start counting occurrences: left, right, top, bottom

For example, desktop.findWithAI("button", "ON", 2, Direction.LEFT).click() clicks the second "ON" button from the left in your application.

To describe an object's location in proximity to a different AIObject, provide the following information:

The position of the object you are describing compared to the anchor: LEFT_OF, RIGHT_OF, ABOVE, BELOW.
The anchor object. The anchor must be an AIObject that belongs to the same context as the object you are describing.

UFT Developer returns the AIObject that matches the description and is closest and most aligned with the anchor, in the specified direction.

For example, this test clicks on the download button to the right of the Linux text, below the Latest Edition text block:

AIObject latestEdition = desktop.findWithAI("text_block", "Latest Edition")
AIObject linux = desktop.findWithAIAnchor("text_block" , "Linux" , Position.BELOW, latestEdition)
desktop.findWithAIAnchor("button", "download", Position.RIGHT_OF, linux).click()