Web Connector 23.4

# StripQueryParametersRegex

A regular expression to match query parameters that you want to remove from URLs.

You can set this parameter to prevent multiple documents being ingested for the same content. For example, when crawling a web site, the connector might find the following links:

    http://www.example.com/page.php?parameter=something
    http://www.example.com/page.php?parameter=something-else

The connector must ingest these as separate pages, because the web server could use the query parameter value to customize the page content. However, if the query parameter does not affect the page content, you could set `StripQueryParametersRegex=parameter` so that the connector ingests a single document for `http://www.example.com/page.php`.

Type: | Regular expression
Default: |
Required: | No
Configuration Section: | TaskName or FetchTasks
Example: | `StripQueryParametersRegex=parameter|ignore`
See Also: | StripFragmentIdentifiers

Copyright 2023 Open Text
Version 23.4 | Last updated September 2023

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to swpdl.idoldocsfeedback@microfocus.com.

We appreciate your feedback!

Product: IDOL Web Connector 23.4

Topic Title: StripQueryParametersRegex