Id

String

No

Step identification string.

Before Transformation Debug File

String

No

The downloaded page transformed into XML can be stored in a file for debugging purposes. Only the page for the last row is stored.

After Transformation Debug File

String

No

The same as beforeTransformationDebugFile, except that the transformed page is stored.

Transformation File

String

No

The XSLT transformation file.

Cookies

String

No

Name of the column where to load/store cookies from/to. If this binding is defined together with the urlInitPattern attribute, then urlInitPattern is used to retrieve cookies and these cookies are stored to this column for later usage (in subsequent steps that needs to use same cookies for example). When cookies attribute is set and urlInitPattern is empty, then cookies are read from this column and send to the remote server.

Init Url Pattern

String

No

Address that determines which URL to use to read cookie-related data. It needs to be filled in if the server requires, for example, the session ID to be sent with the HTTP request. This is the case for ASPX, JSP and other page types. Before processing individual rows, the step connects to this address using the GET method and retrieves all cookie-related data. These data are then sent to the server with each HTTP request.

Post Parameters

List of Parameter Definition

No

Defines parameters to be sent to the website as POST parameters.

Use Post Method

Boolean

Yes

Determines whether to use the POST method. It is set to false by default, which represents the GET method.

Url Pattern

String

Yes

The URL where web pages are downloaded from. The URL is constructed dynamically. The column values can be substituted into the pattern as "{column_name}". Example: "http://www.web.com/?id={column_name}".

Timeout

Double

Yes

The time in seconds in which the page must be downloaded for each row. If the downloading time is longer than this, nothing is stored in the output columns, the row is scored and processing continues to the next row.

Columns

List of Column

Yes

Each column maps a text extracted downloaded page (and optionally transformed) to a column in the input row.

When Condition

String

No

Expression that must be satisfied for this step to be executed.

Scorer

String

No

Element which stores basic scoring settings.