When working with functions and regular expressions, it might be necessary to check their results on a small sample of data without the need to run the plan. This can done by debugging an expression or step as a whole.
Regular expressions or functions can be implemented and debugged in different parts of different steps, for example, the Profiling step (in the Data or Business Rules tab), Column Assigner, or Regex Matching step.
Debugging a function is presented on the example of the Business Rules in the Profiling step.
To debug the function (expression), click Debug. The following window appears:
You can edit the expression on the top, input the data next to the column name, and see the result below after clicking Evaluate. The Expression tree to the right of the Data sources section shows the evaluation and transformation steps the value goes through before reaching the final transformation. It also shows the result of each transformation.
Regular expressions can be debugged in the Regex Matching step.
To debug a regular expression, select it in the navigation panel on the left and click Debug next to the Pattern field, as shown in the following image.
The Regular Expression Debugger window appears, as shown in the following image.
Enter the text into the Input text field and click Evaluate. A few substitution options and their results will appear.
In case you spot an error in your regular expression, you can edit it in the same dialog and repeat the evaluation procedure.
If you have a few transformations (regular expressions or functions) in one step, it is useful to debug the step as a whole to see how they affect the output values of all columns in the step.
To do that, right click on the step and select Debug. The Step Debugger window appears:
The Step Debugger window is divided into two parts: in and out, each listing the same columns, even if some of them are not present in the source data. You can filter out which columns you want to see both in the input and output parts by clicking the Filter Columns button.
The following image shows the Filter feature, which is useful for narrowing down the selection when dealing with many columns.
The difference with debugging a single expression is that you cannot edit any expressions in this dialog, but you can test your expressions on real data.
To test the expressions, right-click anywhere on the in section and select Import Data. Then locate the file with input data and click OK. The table will be populated with the first 500 rows of data from the selected file. The Import Data functionality is not available for database tables.
Important: The imported data should have the same structure (number of columns and column names) as configured in the step. It is best to perform on the actual data that will be used in the step for successful import.