Open topic with navigation

# Run Custom JavaScript on Web Pages

The connector can run custom JavaScript on web pages before it crawls and ingests them. This allows you to manipulate pages or interact with them before they are processed. For example, if a page has a button to load more content you can configure the connector to run a script that clicks the button. The ingested page will then include the additional content.

TIP:

Running custom JavaScript on pages is usually only necessary in advanced cases, such as processing unconventional sites.

You must write the custom script and save it to a file that can be accessed by the connector.

HPE recommends that you enclose your script in a `try...catch` block to prevent exceptions being thrown out of the script. For example:

    try
    {
      // custom javascript
      true;
    }
    catch(e)
    {
      console.log("Javascript Error: " + e.message) false;
    }

Any messages that you write to the JavaScript console are written to the connector's synchronize log, but only when you set the parameter `LogLevel` to `FULL` and `EnableJavascriptConsoleLogging` to `TRUE`. To troubleshoot your script, set these parameters and include a reasonable amount of logging in your script. The connector also logs the return value of the script.

To run custom JavaScript on web pages

1. Stop the connector and open the configuration file.

2. Modify your fetch task by adding the parameter `CustomJSSections`. This parameter specifies the names of the sections in the connector's configuration file that contain settings for running custom JavaScript on pages. For example:

       [MyTask0]
       ...
       CustomJSSections=MyCustomJS

3. Create a new section in the configuration file to contain the settings for running the script:

       [MyCustomJS]

4. In the new section, set the following parameters:

   `CustomJSUrlRegex` | A Perl-compatible regular expression to match the full URLs of the pages on which you want to run custom JavaScript.
   `CustomJSScriptPath` | The path of the file that contains the custom JavaScript to run. The path must be absolute, or relative to the Web Connector executable file.
   `CustomJSElementSelector` | (Optional) The page elements on which to run the custom JavaScript. Specify the elements using CSS2 selectors.
   `CustomJSAllSelected` | (Optional) A Boolean value that specifies whether to run the custom JavaScript on all elements that match the selector specified by `CustomJSElementSelector` (the default value, `false`, means that the connector runs the JavaScript only on the first matching element).

   For example:

       [MyCustomJS]
       CustomJSUrlRegex=http://www\.hpe\.com/.*
       CustomJSScriptPath=javascript/my_script.js
       CustomJSElementSelector=div#maincontent

5. Save and close the configuration file.

© 2017 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close Send documentation feedback To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. We appreciate your feedback! Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm