# HTTP URL Sequence Monitoring

The purpose of URL Sequence monitoring is to monitor multiple web pages of an online application. It checks pages with dynamically generated information, enhancing interactive transactions such as logging into a login page, creating an account using a web form, instructing the application to perform some action, etc.

URL Sequencing starts with a specific URL such as the Login page and then followed by additional links/URLs in it. The URL Sequence monitoring thus performs end-to-end verification of particular transactions, helping you troubleshoot any problems while monitoring.

- [Creating a new HTTP(s) URL Sequence monitor](#creating-a-new-https-url-sequence-monitor)
  - [Import HAR File](#import-har-file)
  - [Use Web Transaction Recorder](#use-web-transaction-recorder)
    - [Getting Started](#getting-started)
    - [Recording a New Transaction](#recording-a-new-transaction)
    - [Modify an existing transaction](#modify-an-existing-transaction)
    - [Monitoring Dynamic URLs](#monitoring-dynamic-urls)
- [Monitored Parameters](#monitored-parameters)

**Note:** To monitor a single URL, refer to [HTTP URL Monitoring](https://www.manageengine.com/products/applications_manager/help/http-url-monitors.html).

## Creating a new HTTP(s) URL Sequence monitor

To create an HTTP(s) URL Sequence monitor, follow the steps given below:

1. Go to **New Monitor** and click **Add New Monitor**.
2. Select **HTTP(s) URL Sequence Monitor** under the **Web Server/Services** category.
3. Here you can create a URL Sequence monitor using any one of the following methods:
   - [Import HAR File](#import-har-file)
   - [Use Web Transaction Recorder](#use-web-transaction-recorder)

To modify an existing URL Sequence monitor, [refer here](#modify-an-existing-transaction).

### Import HAR File

On selecting this option, you will be taken to the page where you have to import HAR file. To create a URL sequence monitor by importing HAR file, the user must generate a HAR file comprising of the required URL Sequence.

#### 1. Import HAR file and view URL(s) recorded

1. Generate a HAR file. [Click here to learn about generating HAR file](https://www.manageengine.com/products/applications_manager/help/har-file.html)
2. Enter a display name for your URL Sequence monitor.
3. Upload the generated HAR file. Maximum file size allowed is *60 MB*.
4. Enter the required [Poll Interval](https://www.manageengine.com/products/applications_manager/help/glossary-applications-manager.html#polling-interval) value in minutes. Default value is *10 minutes*.
5. Click **Upload File**. This will take you to a page where you can see all the URLs recorded and modify them as required.

#### 2. View and modify recorded URL(s)

- **Display Name** field contains the monitor name that was provided while uploading the HAR file.
- **File Name** contains the name of the file which was uploaded.
- **Connection Timeout** is the value of timeout for connection for each URL in the sequence.
- **URL List** section lists all the URLs that have been recorded. To delete URL(s):
  - Select the check boxes corresponding to them in the URL list.
  - Click on the **Delete** button on the right side below the list.
- **URL Details** section contains the information such as URL, display name, headers, HTTP Request method, etc., for the URL currently selected or clicked upon. It has two tabs — **Basic** and **Advanced**, present as a radio button below the URL List.

**Note:** All changes done here will affect only the selected URL, not all URLs listed.

- **Basic:**
  - **URL field** has the URL which is currently selected. You can modify this if required.
  - **Display Name** is the name that will be displayed for the URL in the list of the URLs in the URL Sequence monitor.
  - **Headers** field has the headers that are used by this URL for communication. Applications Manager will also be sending the same URLs while making a request to this URL.
  - **HTTP request method** shows the URL request method, whether it is GET or POST. (PUT, PATCH, HEAD, DELETE URLs are not monitored)
    - If the URL request method is of type **GET**, then the request params used for the URL will be shown in the **Request Parameters** section. The content can be modified to include any extra data if required.
    - If the URL is of type **POST**, then the post data will be shown under the **Payload** section in FORM, TEXT, XML, or JSON format. All the sections can be modified to include any extra data if required.
- **Advanced:**
  - **Alert Configuration** — You can add content check for a URL if required.
    - **Generate Alert if following keywords are present:** If the keywords added here are present in the URL's web page then the monitor's status will be made down and an alert will be generated.
    - **Generate Alert if following keywords are not present:** If the keywords added here are not present in the URL's webpage then the monitor's status will be made down and an alert will be generated.
  - **Basic authentication** — The basic authentication details, if present in a URL will be automatically captured by Applications Manager. If you still wish to add authentication details, you can enter the credential details like **user name** and **password** for authentication, or select the required credentials from the **Credential Manager** list after enabling the **Select from Credential list** option for URL monitor. (To add new credentials under Credential Manager, [refer here](https://www.manageengine.com/products/applications_manager/help/credential-manager.html))
  - **Response Code configuration** — Configures the response code and the condition that you would like to be alerted for. By default you will be alerted for all URLs that return a response code above 200. You can modify them if required.
- **Filter URLs** section filters the URLs based on the following categories:
  - **URL Extension (File Type)** — Filters the captured URLs based on the extension of the file type. By default, URL(s) will be filtered based on default file extension filters like *.css, .js, .tff*, etc.
  - **Domain Components** — Filters the captured URLs based on the domains from which they originate (like *abc.com* or *example.com*).
  - **Content Components** — Filters the captured URLs based on the Content-Type header in the response headers like image, video, audio, text and application.

You can add new filter(s) to each of the above categories (URL Extension, Domain Component or Content Component) using the **Add** button at the bottom of each category list.

- **Associate monitor to a monitor group:** Choose the Monitor Group from the combo box with which you want to associate the URL Sequence monitor (optional). You can choose multiple groups to associate your monitor.

After performing all the above steps, click on **Add Monitor** to create the URL Sequence monitor.

**Note:** Only 25 URLs are allowed to be recorded. If the URLs exceed more than 25, then the monitor will be created with the first 25 URLs. Alternatively, you can also delete the URLs that are unnecessary using the Delete functionality explained earlier.

### Use Web Transaction Recorder

**Warning:** Web Transaction Recorder tool is deprecated. Kindly use [Chrome Extension](https://www.manageengine.com/products/applications_manager/help/rbm-browser-extension.html) to record and add transactions for Real Browser Monitor (RBM) and HAR Import method to add HTTP(s) URL Sequence Monitor.

Applications Manager's [Web Transaction Recorder](https://www.manageengine.com/products/applications_manager/help/web-transaction-recorder.html) tool records all user online transactions in their exact sequence. These transactions will then be replayed at regular intervals of time and notifications will be sent when any error is detected.

Browse through the following topics to use the Transaction Recorder tool:

- [Getting Started](#getting-started)
  - [Hardware/Software Requirements](#hardware-software-requirements)
  - [Download and Installation](#download-and-installation)
- [Recording a New Transaction](#recording-a-new-transaction)
  - Recording a Sequence
  - Editing captured URLs
  - Saving a transaction
- [Modify an existing transaction](#modify-an-existing-transaction)
- [Monitoring URLs with Dynamic Parameters](#monitoring-dynamic-urls)

**Note:** We have set a limit of 25 URLs per URL Sequence monitor.

### Getting Started

You must download and install the recorder tool to record the sequence of user transactions. Every step of the transaction will be captured and recorded. You can also modify or edit your transactions.

### Hardware/Software Requirements

The Transaction Recorder is developed using the Microsoft .NET Platform version 2.0. This is the minimum software requirement. A minimum of 2 MB of disk space is also required for installing the software.

### Download and Installation

Download the [Web Transaction Recorder](https://www.manageengine.com/products/applications_manager/help/web-transaction-recorder.html) and start the installation process. The installation wizard takes you through a four step process after which the Recorder will be started:

1. A welcome screen is displayed, after which you will be asked for confirmation of the license requirements. Click **Yes**.
2. After confirmation, a screen to Choose Destination Location is displayed. The user is allowed to select the drive and directory where they wish to install the Transaction Recorder. Click **Browse** and select the installation destination. Click **Next**.
3. Review the settings in the confirmation dialog and click **Back** if you wish to make changes. If not, click **Next**. The Recorder will be installed in your machine.
4. After the installation is completed, click **Finish**. Make sure the **Start Transaction Recorder** option is checked, if you would like to start the tool automatically after the installation is complete.

### Recording a New Transaction

After installing, the Transaction Recorder can be accessed under All Programs in Windows Start Menu. You can record a new transaction following these steps:

1. Record the Sequence.
   - Click on the **New** button to start a new recording.
   - Enter the URL of the application you wish to monitor then press enter to load the corresponding web page.
   - Follow the sequence of steps that you wish to monitor.
   - After you have completed the sequence, click **Preview & Save**.
   - In the wizard that appears, enter the below details:
     - URL sequence name: The name of the URL sequence monitor.
     - Poll interval: Poll interval is the time in minutes between two consecutive attempts to check for each recorded URL.
     - Connection Timeout: Timeout value to establish a connection with the URL.
   - Then click on **Next**.
2. Edit the captured URLs.
   - Click on **Next** to open the Preview Transactions screen. Here the user can view the recorded URLs and configure monitoring settings for each URL individually.
   - In the Preview Transactions screen, click the **Filter URL** link on the top right corner to open a URL Filter Screen. URLs can be filtered based on the following three categories:
     - File Type / URL Extensions — The captured URLs can be filtered based on their extensions. For example, .gif, .png, .jpg, etc. for images, .js for scripts and more.
     - Content Type / MIME Type — URLs can be filtered based on the MIME type like image, video, audio, text and application.
     - Domain Components — You can also filter URLs based on the domains from which they originate. You have the option to include/exclude websites from a particular domain in the Domain Filter list.

**Note:** These domains are not stored for future references. You have to re-enter such preferences if you re-start the tool.

   - You can add a new Extension, Content Type or Domain Component to the URL filter using the **ADD** button at the bottom of each list.
   - Delete URL: Click on a URL from the list of transactions and click the **Delete URL** link on the top right corner to delete it.
   - Basic Configuration: You can edit basic URL details like URL link, Display name, header and post data by selecting the Basic option at the bottom of the list of recorded URLs.
   - Advanced Configuration:
     - Alert Configuration: You can configure the recorder such that alerts are generated when specific keywords are present/not present. Enter the keywords in the text boxes provided.
     - Basic Authentication Details: Enter Authentication details like username and password.
     - Response Code Configuration: You can configure the recorder to generate alerts if response time is greater than, equal to, not equal to, less than, less than or equal to a specified value.
3. Click **Create URL Sequence Monitor** to create the monitor.

### Modify an existing transaction

Follow the steps given below to modify an existing transaction:

1. Click on the **New** button to start a new recording.
2. Enter the URL of the application you wish to monitor then press enter to load the corresponding web page.
3. Follow the sequence of steps that you wish to monitor.
4. Click **Update Sequence Monitor** and edit the captured URLs, if required.
5. Click **Select Monitor** and select the transaction (monitor) which needs to be modified from the drop-down menu.
6. A side-by-side comparison of the list of old URLs and the list of newly recorded URLs will be shown.
7. Click **Update URL Sequence Monitor** to update the monitor.

**Note:**

- If you would like to set User Agent for monitoring URL sequence, add the following key in AMServer.Properties file located in your Applications Manager installation folder:

```
am.httpclient.useragent=<Browser User Agent which you would like to set>
```

**Example:**

```
am.httpclient.useragent=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1)
```

Save the file and restart Applications Manager for the changes to take effect.

### Monitoring Dynamic URLs

- Suppose your login page has hidden parameters that need to be submitted while logging into your application and the value for the hidden parameter is different for each login request. In this case, the hidden parameter value in the login page needs to be fetched dynamically and passed to the subsequent URL.
- For monitoring these kinds of dynamic URL Sequences, you can use `$<Parameter name>` [eg., $sessionID = 12] in the Request Parameter text box of the URL. So, while monitoring the URL Sequence, `$<Parameter name>` will fetch the dynamic values and pass it to the next URL in the sequence.
- While using the Recorder tool, you cannot specify the parameters in the Request Parameter text box, so instead, after adding the URL Sequence, go to the corresponding URL's **Edit Monitor** page and enter the parameters.
- Consider the following URL Sequence:
  - URL 1:  
    `https://api.appmanager.com/oauth/token?client_id=32faff282a68a03c2eb0f41aeca63a88&grant_type=client_credentials&client_secret=9de7337627ab85f3`
  - URL 2:  
    `https://api.appmanager.com/bestoffer/97232/354992/offers.xml?access_token=fcdee0e94da9b934a4cef6c2dee70384`
  - URL 3:  
    `https://api.appmanager.com/selectoffer/97232/3549/offers.xml?access_token=fcdee0e94da9b934a4cef6c2dee70384`
  - ...
  - URL N: ...

In the above example, `access_token` is a dynamic parameter whose value changes every time we login. In order to monitor the above sequence, follow the steps given below:

- Download the Transaction Recorder tool and record a new transaction sequence.
- The recorder tool will have recorded all the steps along with dynamic request parameter for that session.
- After adding the URL Sequence, click **Edit Monitor** for the corresponding URL to enter the parameters.
- In the Edit Monitor page, for the field **Request Parameters (if any)** provide the dynamic request parameter as:

```
$parameter=TEXT:-:<parameter_name>:-:
```

**For example** — In URL 1: The parameter is `access_token` and `parameter_name` is `access_token=`. So, for the text box corresponding to the **Request Parameters** provide:

```
$access_token=TEXT:-:access_token=:-:
```

and save the page.

- Click **Edit Monitor** for the second URL, remove the dynamic request parameter from the **URL address** field and use the same parameter to fill the **Request Parameters (if any)** field.

**For example** — In URL 2: Edit  
`https://api.appmanager.com/bestoffer/97232/354992/offers.xml?access_token=fcdee0e94da9b934a4cef6c2dee70384`  

Remove the dynamic request parameter (`access_token`) and change the URL address to:  

`https://api.appmanager.com/bestoffer/97232/354992/offers.xml?`  

Provide the Request Parameters as:

```
$access_token=fcdee0e94da9b934a4cef6c2dee70384
```

- Repeat the above steps for the rest of the URLs in the sequence where the dynamic parameter is present.

So for every polling interval Applications Manager will search for this dynamically generated parameter and replace it.

## Monitored Parameters

Applications Manager acts as a continuous URL Sequence monitoring service that keeps a constant watch over the specified URL sequences. URL Sequence monitors verify the availability of multiple URLs in sequence, performing end-to-end verification of transactions.

Go to the **Monitors Category View** by clicking the **Monitors** tab. Click on **HTTPs URLs Sequence** under the *Web Server/Services* table. Displayed is the bulk configuration view distributed into three tabs:

- **Availability** tab gives the Availability history for the past 24 hours or 30 days.
- **Performance** tab gives the Health Status and events for the past 24 hours or 30 days.
- **List view** enables you to perform [bulk admin configurations](https://www.manageengine.com/products/applications_manager/help/bulk-config.html).

Click on the individual URL Sequence monitor listed to go to the monitor overview page. Click on the individual URL listed to view the monitored parameters.

| Parameter | Description |
|---|---|
| **Monitor Information** |  |
| Health | Specifies the health of the monitor based on its dependencies. |
| Type | Type of Monitoring |
| URL address | Specifies the URL being monitored |
| Match Content | The string that is searched in the resulting html page. |
| Request Method | The request method sent to the HTTP/HTTPS URL (Get or Post) |
| **Monitored Parameters** |  |
| Availability | The current status of the URL Sequence - whether it is up or down. |
| Response Time | The total response time of the URL Sequence (takes into account the response time of the individual URLs in the Sequence). |
| Current response time | The response time for accessing the current URL in the Sequence. |
| Average response time | Average response time of the URL Sequence over the last one hour. |
| Current Status | Current status of the URL Sequence. Click on the icon to know its RCA details. |
| **Response Time Split Up** |  |
| DNS Time | The time that it takes to resolve the website's hostname to a certain IP address. If the DNS lookup time is high, it means that you or your hosting provider has a problem with their DNS servers. |
| Connection Time | Once the URL has been resolved to an IP address, the connection time shows how long it takes to establish a connection to your server. The connection time helps in discovering network latency, routing issues, and server bandwidth problems. |
| First Byte Time | The first byte shows how long it takes from the moment the connection was created until the first byte is about to be transferred. The time to perform any negotiations with the server and the time needed for the server to calculate the result are also included. |
| Last Byte Time | Once the first byte of HTML has been received, the Web server continues to send the HTML that represents the layout of the Web page, including CSS and JavaScript. Last byte time includes time till the last byte for all documents with the HTML to load. Also commonly referred to as the "Download Time" or "HTML Content Time". |