This step downloads a web page for each input row. Optionally, it can transform the page using an XSLT transformation and extract information which can be stored in columns. The address of the downloaded web page is constructed using data in the input row. Various problems can be scored, e.g. timeout, no response, wrong transformation, etc.

Parameters to be sent to the server are stored in postParameters (when using the POST method) and depend on the technology used. Some common ones are:

ASPX

Requires posting of the attributes that are involved in the creation of the VIEWSTATE value, the VIEWSTATE value itself and the form element being clicked. The VIEWSTATE value is used by ASPX to keep the state and values of the ASP form. Values that are involved in the VIEWSTATE value are controls having some value (text fields, combo boxes, etc.). To inform the server about the clicked button it also needs to send the respective button ID. Values that need to be sent can be determined by analyzing HTTP data communication between the server and client. This can be done using a specialized tool, such as:

• firefox page info (tab Form) - for version 3 this can be done using, for example, the Page Info Forms and Links plugin (https://addons.mozilla.org/cs/firefox/addon/7978)
• Ethereal network analyzer
• and others

A simple example: a form has one entry text field, idValue, a button, cmdSend, which performs the form's submit action, a button, cmdReset, which performs the form's reset action and labels, label1 and label2. So, the values that indicate that the form was filled in and sent for processing are:

• __VIEWSTATE (can be read from the HTML page containing the form)
• label1, label2
• idValue
• cmdSend (may have any value, since the server only needs to know that cmdSend was clicked)

JSP

Requires sending attributes that determine the result plus the control that was selected (clicked). So, using the same example as for ASP, the values that need to be sent are:

• idValue
• cmdSend (again with any value - the server only needs to know about this control)

<step className="cz.adastra.cif.tasks.io.html.WebLookupAlgorithm" id="algorithm_1">
  <properties initUrlPattern="http://www.someserver.com/init_page.aspx" timeout="5.0" 
              usePostMethod="true" urlPattern="http://www.someserver.com/find_item.aspx" 
              beforeTransformationDebugFile="./server-output.html">
    <columns>
      <column xpathExpression="/html/body/form/div[4]/table/tr[2]/td[2]/span/span" columnName="firma"/>
    </columns>
    <postParameters>
          <!-- This example shows how to configure POST data elements when sending ASPX form data -->
      <!-- VIEWSTATE is an ASPX value that defines the state and values of all FORM controls -->
      <parameterDefinition value="dDwxMzAyNzgzNzI7dDw7bDxpPDE+Oz47bDx0PT4..." name="__VIEWSTATE"/>
      <!-- all attributes contributing to the VIEWSTATE value must be mentioned in the POST parameters -->
      <parameterDefinition value="Some msg" name="msg1"/>
      <!-- input field that will define the data to search for -->
      <parameterDefinition value="{ico}" name="tico"/>
      <!-- search button definition-->
      <parameterDefinition value="Search" name="cmdSearch"/>
    </postParameters>
    <scorer scoreColumn="score" explanationColumn="expl">
      <scoringEntries>
        <scoringEntry key="WLA_RESPONSE_TIMEOUT" explainAs="WLA_RESPONSE_TIMEOUT" score="0" explain="true"/>
        <scoringEntry key="WLA_UNKNOWN_RESPONSE" explainAs="WLA_UNKNOWN_RESPONSE" score="0" explain="true"/>
        <scoringEntry key="WLA_TRANSFORMATION_ERROR" explainAs="WLA_TRANSFORMATION_ERROR" score="0" explain="true"/>
        <scoringEntry key="WLA_XPATH_ERROR" explainAs="WLA_XPATH_ERROR" score="0" explain="true"/>
        <scoringEntry key="WLA_MALFORMED_URL" explainAs="WLA_MALFORMED_URL" score="0" explain="true"/>
        <scoringEntry key="WLA_EMPTY_VALUE_FOUND" explainAs="WLA_EMPTY_VALUE_FOUND" score="0" explain="true"/>
      </scoringEntries>
    </scorer>
  </properties>
</step>