Text recognition methods enable you to conveniently interact with test applications that contain highly customized controls,
which cannot be identified using object recognition. You can use
text clicks instead of coordinate-based clicks to click on a specified text string within a control.
For example, you can simulate selecting the first cell in the second row of the following table:
Specifying the text of the cell results in the following code:
Text recognition methods are supported for the following technology domains:
- Windows Forms.
- Java SWT and Eclipse.
- Java AWT/Swing.
For Java Applets, and for Swing applications with Java versions prior to version 1.6.10, text recognition is supported out-of-the-box.
For Swing applications with Java version 1.6.10 or later, which do not support Direct3D, you have to add the following command-line
element when starting the application:
javaw.exe -Dsun.java2d.d3d=false -jar mySwingApplication.jar
Text recognition is not supported for Java Applets and Swing applications that support Direct3D.
- Internet Explorer.
- WebDriver-based browsers.
Note: Text recognition does not work with controls that are not visible on the screen. For example, you cannot use text recognition
for a text that is scrolled out of view.
Note: Text recognition might not work if the font that is used in the target text is not installed on the machine on which the test
The text recognition methods can be applied to
Note: Text recognition does not work for text that is drawn in
Note: Text recognition does not work for content added by CSS pseudo-elements like
Text recognition methods
offers the following methods to drive testing through interacting with the text that the AUT renders on the screen:
- Returns the text that is within a control. Also returns text from child controls.
- Clicks on a specified text within a control. Waits until the text is found or the
Object resolve timeout, which you can define in the synchronization options, is over.
- Returns the rectangle of a certain text within a control or a region of a control.
- Determines whether a given text exists within a control or a region of a control.
The text recognition methods prefer whole word matches over partially matched words.
Silk4J recognizes occurrences of whole words previously than partially matched words, even if the partially matched words are displayed
before the whole word matches on the screen. If there is no whole word found, the partly matched words will be used in the
order in which they are displayed on the screen.
to grab the visible text from the application and allow for further processing of that text. The underlying
method is implemented in two different ways.
decides which implementation to use depending on the type of the application under test.
- For native windows applications, including WPF, WinForms, and Java applications, but also
Silk4J hooks into the text rendering functions of the Windows API to extract the text that the application draws on the screen.
Microsoft Edge, and
Note: Because of the different nature of these two implementations,
Silk4J might return different text for the same web application, depending on which browser is used.
The user interface displays the text
the hostname is the name of the host
. The following code clicks on
is displayed before
on the screen:
The following code clicks on the substring
in the word
by specifying the second occurrence: