Version 7 Release 7.05

• Data Management Console Ribbon. The DMC ribbon replaces traditional toolbars with a single, rectangular region that spans the width of the application window.
• Improved Diagnostic Messages on Server Connection. If a client (DMC) attempts to connect to a DataMigrator server and the release/gen on the client and server do not match, a warning message appears. It includes the release and gen numbers.
• Restore Defaults Capability Added to DataMigrator Screens. A Restore Defaults option and a corresponding button has been added to some configuration screens for DataMigrator.
• New File Option for Supported File Types. The New context menu, and the New Server Object dialog box, has a new option called File, which lets you create a file on the server. When saving the file, it can now be saved as any supported file type.
• Editing CSV or DAT File Types. Files with a file type of CSV or DAT can now be edited from the Data Management Console.
• New Core Engine Setting. A new Core Engine setting, 2PARTNAME, controls the behavior of stored procedures with SQL statements that include an application directory.
• Using SCD Load Type in a Direct Load Flow. A Direct Load Flow can now have a load type of Slowly Changing Dimension (SCD), which allows using a CDC log synonym as the source.
• Scheduling a Non-Writable Procedure. A user can now schedule a procedure that they have permission to read but not write to. Doing so, creates a new process flow with a schedule that calls the original procedure.
• Files Types in Email Attachments. In previous releases, the file types that could be attached to an email message sent in a DataMigrator process flow were limited. Now, any file type can be used.
• Enabling and Viewing Client Site Traces. The Traces settings window allows you to enable and view client site traces. It is available on the Home tab, in the Tools group.
• Allowing User Entered Tags. DataMigrator automatically generates correlation names, known as tags, for synonyms. You can change the default name (T1, T2, and so on) to one that is more descriptive, to make data flows more understandable. This can be done from the Properties pane of the selected synonym.
• Run a Stress Test for a Procedure. There is a new option in the Run Advanced context menu of the DMC called Run Stress. Selecting this option generates a script for scalability testing.
• Extended Bulk Load Functionality in Sybase ASE. The Extended Bulk Load Utility is now available for both Sybase ASE and Sybase IQ.
• Exporting Flow Report Information to an XML File. If you need to export the flow information into an XML file, you can use the stored procedure flow2xml, which is located in the ibisamp application directory.
• Allowing TRUNCATE on DB2 Tables. There is a new Prior to Load option called Truncate Table, which allows you to remove all rows from the target table without logging.
• Customizing Email Subject. Email options have been added that allow you to include a custom subject, email body, and level of importance. There is now also the ability to include a report in the email body.
• Validation for Specific SCD Fields. A flow with a load type of SCD is not valid if one of the required SCD fields is missing. An error message will appear if a flow is invalid.
• Column Name Display for Alias in the Synonym Editor. When working in the Synonym Editor in the Data Management Console, you can now select Alias, in addition to Name, Title, and Description, from the Column Name Display context menu.
• Using Arrow Keys in Flow Editors and Synonym Editor Modeling View. As an alternative to using the mouse, arrow keys can now be used in the Flow editors, and the Modeling View of the Synonym Editor, to move around the screen.
• New Slowly Changing Dimensions Type Value. There is a new SCD Type value of Change Flag in a flow with a load type of SCD. When this value is used for a field, that field is updated when any Type I field value changes.
• New Index Column Option. In the Synonym Editor, there is a new Customize option named Index. When added to the columns that are displayed, the name of any indexes on the underlying relational database table are shown.
• Default DBMS Error Limit Setting for New Flows. A new option found in the Run Options dialog box allows you to stop processing the number of DBMS errors received if the number of errors specified is exceeded. This setting is the default for new flows. You can override this option for an individual flow by resetting the properties of that flow execution.
• Automatically Add Join Conditions Using Foreign Keys. The Automatically add join conditions option in the Data Flow Designer now uses primary-foreign key relationships, if they exist in the synonym. This is used to determine what conditions to add, before using matching column names.
• Business View Enhancements. In the Business View of the Synonym Editor:
  • When creating a Business View using the default or user created folder structure, the BELONGS_TO_SEGMENT attribute is assigned to each field in the Business View.
  • Fields can now be moved within a folder, as well as from one folder to another.
  • In the Modeling View of the Synonym Editor, clicking a field in a table will highlight all of the places where the same field is used in the Business View.
• Setting Records as Inactive in a Flow With IUD/SCD Load. When using the Change Data Capture log table as a source and a load type of IUD/Slowly Changing Dimensions, rows deleted in the source are flagged as inactive in the target dimension table (ACTIVE=0 and/or END_DATE=today's date).
• Multiple Full Outer Joins in a Data Flow. Multiple Full Outer Joins can now be used in a Data Flow for relational sources using Automatic Pass Through and for NON-APT.

Synonym Enhancements - Version 7 Release 7.05

• Warning for a Synonym with No Key Columns Identified. When a synonym is created without any key columns identified, a warning message is displayed. Keys should be identified because not having keys can cause a table to load slowly.
• ACCEPT Values in a Synonym for Table Log Records. The Sample Data dialog box for a log records table synonym now shows a list of options for the &&CDC_START parameter.
• Show/Modify Data Tools for a Synonym. Show/Modify Data is a new option in the Data Management context menu available from the DMC or Web Console, which allows you to view data for a selected synonym in a grid format, one page at a time.
• Add FTP Connection Name to a Synonym for FIX and DFIX.A synonym that was created based on a remote connection (not local) for a delimited or fixed format file now has a CONNECTION attribute that is displayed in the Synonym Editor.

Version 7 Release 7.04

• Database Default Values. If a database column has a default value assigned in the database, and no value is assigned in Target Transformations, the database will now assign the default value. In previous releases, fields with no value assigned in Target Transformations had a NULL value assigned for NULLABLE columns and a space or zero for NOT NULL columns.
• Making Global Variables Persistent in a Session. Global variables can be used by various users, regardless of the fact that each user has their own profile, by changing a listener configuration property.

  Note: This is available for an HTTP Listener only.

• Change Case. The DMC Text Editor that is used for editing configuration files, and is available in text mode when using synonyms or procedures, has a new context menu option. When a block of text is selected, the Change Case option is available. Invert Case changes all uppercase letters to lowercase letters and all lowercase letters to uppercase letters. Upper Case changes selected text to all uppercase letters. Lower Case changes selected text to lowercase letters.
• New Tree Control Look. The look of the Data Management Console has been updated. This includes the main menu and context menus, as well as the look of property sheets and dialog boxes. Most visible is the way that flows, synonyms, procedures, and other objects are opened in the work area. Previous releases used a Multiple Document Interface where each object had its own window. Now tabbed windows are used, which facilitates switching between open objects. These can be arranged in groups so that multiple objects can be viewed at once.
• Save GUI State. Used for diagnostics, this file saves the current state. This file is only needed if requested by Customer Support Services.
• Undo and Redo Limit. The undo and redo limits specify the maximum number of undo and redo operations allowed in the Data Flow Designer.
• File Listener Enhanced to Monitor Files and Subdirectories. The File Listener has new Monitor capability to detect files of any type that are added, changed, or removed in a specified directory (and subdirectories) without reading them. The action, full path, and name are available in a data flow.
• SQL Calculator List of Values for Input Fields. The SQL calculator shows a list of columns from the selected source synonyms. Normally, when selecting a column and clicking the [+] to expand it, a list of values from the source table is retrieved. Now, when using a source synonym that points to a procedure and selecting an INPUT parameter that has an ACCEPT list, those values are displayed.
• Ability to Add or Modify Core Engine Sets on Web Console. Selected Core Engine Settings can now be set from the Data Management Console or Web Console from Workspace Configuration. This includes settings for SQL compatibility, missing value handling, and error limits.
• Send Multiple Emails on Different Conditions. Previously, the action after an email object was unconditional and there could only be one object immediately following. Now, multiple objects can be added and the action can be based on success or failure of sending the email or any other condition. This allows for sending multiple emails on different conditions or two different addresses.
• Adding Indexes to a Synonym. You can create indexes in a synonym by adding them in the Synonym Editor. When you create a table from the synonym using Data Management, Recreate DBMS table, the indexes are created in the relational data base. When you use New, Synonym to create a synonym for a relational database table with more indexes than just the primary key, they will be included in the synonym.
• Schedule Flows to Run When Server Starts. You can schedule a process flow to run whenever the DataMigrator server is started. This option is useful for flows that need to run all of the time, for example, when using Change Data Capture or File Listener capabilities.
• Improved Printing Capabilities. Printing is improved when displaying a viewed report in a grid and the Print Preview display gives a better picture of the printed report. In addition to printing the entire report, the Print dialog box has an option to select only the desired page. Also, if rows and columns are selected, the Print dialog box has an option named Selection, which allows you to print only the selected columns.
• Parallelization and Partitioning. For faster processing when using Quick ETL Copy for a single table that has one or more numeric keys, the table can be divided into multiple partitions with one job run for each partition. You select a column to use and values are calculated to divide the table by selecting a range of values to partition the source table. When the flow is run, multiple instances are run, one for each partition.
• Function Assist for EXISTS in WHERE and HAVING Conditions. A WHERE condition can contain an EXISTS clause with a SELECT statement to check if one or more rows are returned. A new Function Assist facilitates the building of this condition.
• Stored Procedures Allowed in Parallel Groups. A parallel group in a Data Flow can include stored procedures, in addition to other flows.
• Recently Used Files. The Data Management Console main menu File has a new option named Recent Files that shows a list of the recently used files (server objects). The number of files to show can be set from the General Preferences dialog box in the Tools, Options menu.
• Autosave Feature for Synonym and Data Flow Editors. Synonyms and Data Flows edited in the Data Management Console can now be saved automatically after a user specified number of minutes have elapsed.
• DBMS Recreate Tables. Synonyms created for database tables now include information about DBMS indexes. Using the Recreate DBMS table option for a synonym creates the table with any database indexes in the synonym.

Version 7 Release 7.03

• Creating a Quick ETL Copy. The Quick ETL Copy option generates a simple Data Flow without using the Data Flow designer. For the selected table(s), all rows and columns are copied to new table(s). Although no transformations or WHERE conditions are generated, the Data Flow can still be opened as such and additional criteria can be added.
• Supporting TEXT (TX) Columns. DataMigrator supports columns described with ACTUAL=TX for Text columns. This includes MS SQL Server VARCHAR(MAX) and Text, ORACLE LONG, and DB2 LONG VARCHAR data types. Sample Data displays only the first 50 characters of these columns.
• Clearing User Preferences. The General settings window now has an option to clear user preferences and return them to default values.
• Sending a Target File to an FTP Server. You can use DataMigrator to build a flow that automatically sends the target file to a directory on an FTP server. This can be useful if, for example, a target file(s) has to be created on a remote FTP location. This feature is available for the synonyms of a Flat File and a Delimited Flat File. The synonym can be used in a Direct Load Flow, as well as in a Data Flow.

Version 7 Release 7.02

• Primary Key Analysis Report. Key Analysis provides a report that shows which columns in a data source can be used individually, or in combination, to uniquely identify a row. The columns identified in this report are candidates for key column(s).
• Function Assist. The Function Assist dialog box that is available through DMC calculators has a new drop-down menu that displays lists of column names. Other available information is Format, Description, and Nulls. This menu is configurable using the Column Management preferences found in the Options window.
• Scheduled Events Report. The scheduled events report lets you specify a time range to view all jobs that are specified to start in that time period.
• New XML Target. DataMigrator can create a new XML document with name/value pairs.
• Where Condition With Sub-Select. A where condition with an IN clause and a sub-select can now use multiple columns.
• DBCS Character Functions. DBCS character functions DCTRAN, DEDIT, DPART, DSTRIP, and DSUBSTR have been added to DataMigrator. These functions support operations on double-byte character data.

Version 7 Release 7

• DataMigrator Console. This section describes new features for the DataMigrator Console.
  • Change Data Capture. Detects changes to source tables as they occur.
  • Star Schema Load. Sample files and a tutorial show how to load data into a Star Schema.
  • Iterator. The iterator simplifies processing using multiple tables with the same layout.
  • File Listener. The adapter based File Listener facilitates processing files as they appear in a specified directory or FTP server.
• DataMigrator Flows. This section describes new features for DataMigrator flows.
  • Validate Flow. You can validate a flow by clicking the Validate Flow button. Any errors found will appear in a dialog window with an error message.
  • Direct Load Flow Sources. Direct Load Flow supports any source as long as the structure is flat.
  • Wait After Parallel Group. Proceeds to next object regardless of whether the wait was satisfied.
  • Duplicate Field Names. For hierarchical source tables with repeated field names, the segment names are used to distinguish them.
  • Target Transformation Documentation. You can add a description to a target transformation.
  • Copy and Paste Transformations. Transformations can be copied from a target and pasted to the same, or another, target.
  • Adapter Type Display. The adapter type (such as Microsoft SQL Server) is optionally displayed on the workflow for sources and targets.
  • New Function Assist for COALESCE. The Function Assist has been enhanced to support the SQL function COALESCE, which can take two or more arguments.
  • E-mail Notification. E-mail notification from within a process flow can send both a message and an attached file.
  • Run as Stored Procedure. The option, Execute as RPC, has been renamed to, Run as Stored Procedure, and a new option can be set so that all flows are added with this option checked.
  • Nested Select Statements. Additional SQL objects can be added to a data flow. This allows joining to a select statement and avoids the need to create intermediate tables and use multiple flows.
  • Union. A UNION object can be added to a data flow. This allows concatenating the results of multiple select statements, avoiding the need for multiple flows. This object also supports the SQL operations MINUS and INTERSECT.
  • Join Assist. When you add a synonym to a Join object that already has one table selected, the Select Synonym dialogue can be filtered to show only those synonyms with matching columns. Candidate synonyms may have either matching key column names, or the synonym may contain a primary key/foreign key relationship.
  • Help With Errors. Errors in a transformation can be viewed by selecting Errors details from the transformation. Error message numbers in these and other dialogues are a hot link to the message text.
  • Bulk Load. Bulk load options are available for Teradata and Oracle.
• Data Management Console - Browser. This section describes new features for the Data Management Console browser.
  • Browser Views. Separate tabs for procedures, tabs, and adapters have been replaced by folders in the browser.
  • Workspace Configuration. Server configuration, which previously required using the Web Console, can now be done from a new Workspace folder.
  • Descriptions. Descriptions can be stored and displayed for directories and stored procedures.
  • Home Application Directory. Each user can have a Home application directory, visible only to themselves and the Server Administrator.
  • Nested Application Directories. Application directories can have subdirectories.
  • Foccache Temporary Directory. A temporary directory is added to the application path on connection to the server that exists for the duration of the session.
  • Explorer View. Explorer view shows details of files and directories viewed in the browser window.
  • Schedule and Set Email. You can add schedule and email settings to a stored procedure, and you can edit for procedures and flows without opening the flow.
  • Copy, Cut, and Paste Multiple Server Objects. Using standard Windows shortcut keys, you can select multiple server objects to copy, cut, and then paste in another directory or another server.
  • Source Code Control. Source code control applications ChangeMan and Perforce are supported. Improved support for all source code control systems, including use of private home directories for development and public directories for deployment, as well as a get latest version option. There is also support for operations in Visual SourceSafe, such as creating the folder during association.
• Data Management Console - General. This section describes new features for the Data Management Console.
  • Property Sheets. Property sheets are used instead of dialogues throughout the Data Management Console.
  • Improved Application Path Configuration. The user interface for configuring the application path has more options and allows configuring the path for specific users.
  • Reusable Transformations. A new user interface simplifies creating and editing reusable transformations (functions).
  • Replace in Text Editor. When editing flows, synonyms, or configuration files in the text editor, you can find a text string and replace it.
  • Aerial View. A new aerial view gives an overview of flows and synonyms in the modeling view. For large and complex flows, this simplifies selecting an area of the object to view.
  • Save As Toolbar Button. All editors have a toolbar button, Save as, to save a file under a new name.
  • Close All But Current Window. A main menu option closes all open windows except for the current one.
• Synonym Editor. This section describes new features for the Synonym Editor.
  • Enhanced Modeling View. A concise segment display by default shows only column names. It can be customized to show additional information.
  • Cluster Joins Insert by Reference. A Cluster Join is a synonym that describes multiple tables which previously required a copy of a synonym for each table. Now, synonyms can be included by reference to another synonym, thus ensuring that changes are propagated automatically.
  • Variables in Synonyms. Variables can be added to a synonym to represent table names and connection names.
  • Support Database Generated Auto Increment Columns. Most relational databases support a column that is automatically incremented when a row is inserted. These columns are now identified when synonyms are created. Tables can be created from a synonym that identifies such columns.
  • Filename. For flat file data sources, a field with an ALIAS value of INSTANCE will contain the file name.
• DataMigrator Database Lookup. This section describes new features for the DataMigrator database lookup.
  • DB_LOOKUP for FOCUS Files. FOCUS files can be used with DB_LOOKUP.
  • Cache for DB_LOOKUP. A new setting enables a cache for DB_LOOKUP to improve performance.
• Star Schemas and Slowly Changing Dimensions. This section describes new features for Star Schemas and slowing changing Dimensions.
  • Specify Begin and End Dates. For Slowly Changing Dimension Type II, changes override the default begin and end dates for active and/or inactive records.
  • Support for Auto Increment Columns. Database-generated automatically incremented columns can be used as surrogate keys in dimension tables.
• DataMigrator Reporting. This section describes new features for DataMigrator reporting.
  • View Log and Statistics Reports Enhancements. The View Log and Statistics selection panel has been enhanced so that you can select a specific flow name and view scheduled or immediate flows.
  • Flow Logging and Monitoring. You can control how often and the maximum number of lines that are written to the LOG table for each flow. A flow can now be monitored as log messages are written while the flow is running.
  • Send Warning E-mail for ETLLOG Table Errors. An automatic e-mail message can be sent if an error occurs while writing to the ETLLOG file. Such an error could occur, for example, if the file is full.
  • Maximum Size of Log and Statistics Tables Increased. The ETLLOG and ELTSTATS files were previously limited to two gigabytes. The limit is now 16 gigabytes.
• DataMigrator Scheduler. This section describes new features for the DataMigrator Scheduler.
  • Intraday Scheduling. Flows can be scheduled to run during a specified time period every day.
  • Restart Interval. You can specify the time between restart attempts for all flows with a restart.
  • Scheduler ID. You can explicitly specify the user ID of the scheduler. This will control the application path that is scanned for scheduled flows to run.
• DataMigrator Variables. This section describes new features for DataMigrator variables.
  • Assign Variables. A new variable object in a process flow simplifies assigning values for local and global variables.
  • Use Local Variables in a Data Flow. You can use local variables in a data flow for where conditions and other instances. For running flows, a value can be set in a process flow.
  • Test Local Variables. The DMC now prompts for a local variable if it is needed while testing an SQL or transformation.
  • Use Variables for Target Table Name. A variable can be used for a target table name so that the name can be assigned at run time.
• DataMigrator Adapters. This section describes new features for DataMigrator Adapters.
  • Excel. Now only a single connection is required with a named adapter for Excel that allows creating synonyms for tables in multiple worksheets.
  • IWAF - iWay Adapter Framework Adapters. An iWay adapter enabled for IWAF can be used as a source in a DataMigrator flow.
  • Flat and Delimited Flat Files. A flat file server source can reside on an FTP server.