Scan Settings: HTTP Parsing

HTTP Parameters Used for State

If your application uses URL rewriting or post data techniques to maintain state within a Web site, you must identify which parameters are used. For example, a PHP4 script can create a constant of the session ID named SID, which is available inside a session. By appending this to the end of a URL, the session ID becomes available to the next page. The actual URL might look something like the following:

 .../page7.php?PHPSESSID=4725a759778d1be9bdb668a236f01e01

Because session IDs change with each connection, an HTTP request containing this URL would create an error when you tried to replay it. However, if you identify the parameter (PHPSESSID in this example), then Fortify WebInspect will replace its assigned value with the new session ID obtained from the server each time the connection is made.

Similarly, some state management techniques use post data to pass information. For example, the HTTP message content may include userid=slbhkelvbkl73dhj. In this case, "userid" is the parameter you would identify.

Fortify WebInspect can identify potential parameters if they occur as posted data or if they exist within the query string of a URL. However, if your application embeds session data in the URL as extended path information, you must provide a regular expression to identify it. In the following example, "1234567" is the session information:

http://www.onlinestore.com/bikes/(1234567)/index.html

The regular expression for identifying the parameter would be: /\([\w\d]+\)/

Enable CSRF

The Enable CSRF option should only be selected if the site you are scanning includes Cross-Site Request Forgery (CSRF) tokens as it adds overhead to the process. For more information, see CSRF.

Determine State from URL Path

If your application determines state from certain components in the URL path, select this check box and add one or more regular expressions that identify those components. Two default regular expressions identify two ASP.NET cookieless session IDs. The third regular expression matches jsessionid cookie.

If your application maintains client state with bearer tokens, select this option and create a rule that will identify the bearer token from the response and add it to the next request automatically.

To add a rule:

1. After selecting the Enable Response State Rules check box, click Add.

   The Rule Search and Replace window appears.

2. In the Rule Name field, type a unique name for the rule. An example is Bearer.

3. Click Add next to the Search in Response field.

   The Search in Response dialog box opens in simple mode.

   Note: If you previously selected Regex Mode, the dialog box opens in Regex mode.

4. Do one of the following:

   • To create a rule in simple mode, type the text that contains the token in the Rule box. As you type, a regular expression is automatically generated in the Regex View box.

     Tip: Click to view a list of predefined tokens.

   • To use a predefined regular expression, select Regex Mode and select a regular expression statement from the Regex list. You can then edit the selected statement.

5. Click OK.

   The regular expression is validated. You must correct any errors that are found before continuing.

6. Click Add next to the Replace in Request field.

   The Replace in Request dialog box opens in simple mode.

   Note: If you previously selected Regex Mode, the dialog box opens in Regex mode.

7. Do one of the following:

   • To create a rule in simple mode, type the text that contains the token in the Rule box. As you type, a regular expression is automatically generated in the Regex View box.

     Tip: Click to view a list of predefined tokens.

   • To use a predefined regular expression, select Regex Mode and select a regular expression statement from the Regex list. You can then edit the selected statement.

8. Click OK.

   The regular expression is validated. You must correct any errors that are found before continuing.

9. Click OK to close the Rule Search and Replace window.

HTTP Parameters Used for Navigation

Some sites contain only one directly accessible resource, and then rely on query strings to deliver the requested information, as in the following examples:

Ex. 1 — http://www.anysite.com?Master.asp?Page=1
Ex. 2 — http://www.anysite.com?Master.asp?Page=2;
Ex. 3 — http://www.anysite.com?Master.asp?Page=13;Subpage=4

Ordinarily, Fortify WebInspect would assume that these three requests refer to identical resources and would conduct a vulnerability scan on only one of them. Therefore, if your target Web site employs this type of architecture, you must identify the specific resource parameters that are used.

Examples 1 and 2 contain one resource parameter: "Page."
Example 3 contains two parameters: "Page" and "Subpage.

To identify resource parameters:

1. Click Add.

2. On the HTTP Parameter window, enter the parameter name and click OK.

   The string you entered appears in the Parameter list.

3. Repeat this procedure for additional parameters.

Most Web pages contain information that tells the browser what character set to use. This is accomplished by using the Content-Type response header (or a META tag with an HTTP-EQUIV attribute) in the HEAD section of the HTML document.

For pages that do not announce their character set, you can specify which language family (and implied character set) Fortify WebInspect should use.

This setting defines how Fortify WebInspect interprets query parameters without values. For example:

http://somehost?param

If this checkbox is selected, Fortify WebInspect will interpret “param” to be a parameter named “param” with an empty value.

If this checkbox is not selected, Fortify WebInspect will interpret “param” to be a nameless parameter with the value “param”.

This setting can influence the way Fortify WebInspect calculates the hit count (see the Limit maximum single URL hits to setting under Scan Settings: General). This setting is useful for scenarios in which a URL contains an anti-caching parameter. These often take the form of a numeric counter or timestamp. For example, the following parameters are numeric counters:

• http://somehost?1234567

• http://somehost?1234568

In such cases, the value is changing for each request. If the value is treated as the parameter name, and the “Include parameters in hit count” setting is selected, the crawl count may inflate artificially, thus increasing the scan time. In these cases, clearing the “Treat query parameter value as parameter name when only value is present” checkbox will prevent these counters from contributing to the hit count and produce a more reasonable scan time.