You can use DataMigrator to build a flow that automatically processes source files as they appear in a designated directory. This can be useful if, for example, source files are transferred by FTP from various locations throughout the day and need to be processed immediately.
Before you create and run the data flows discussed in this section, you must create the sample data and synonyms. See How to Create Sample Procedures and Data for Iteration. This will create source and target synonyms and four source data files used in the example
To implement and run a real time flow:
This flat file synonym corresponds to the layout of the source files that will appear in the designated directory. The flat file source loads the target of your choice in real time.
Refer to the sample flow DMFLORD in the IBISAMP directory for the completed example.
For DataMigrator to automatically process files when they arrive in a directory you need to designate a directory for that purpose. It could be any directory your server can access. For this example, we will create an application directory called “listen.”
In addition, if you want to keep a copy of the files after they are processed you need to designate a directory for that purpose too. For this example, we will create an application directory called “archive.”
The Create New Application Directory window opens.
Click OK.
Next you need a synonym with File Collection Parameters to control how the file listener process files. We will start with a synonym that describes just the record layout of the source files and add the parameters.
This creates a synonym that listens on the "listen" directory for files with names that start with "dmord" and an extension of "dat" polling every 5 seconds until 60 seconds have passed without a new file. As files are processed they are archived to a directory "archive." Files are processed as they arrive until 5 files or 50 records are processed.
A flow with a File Listener source can be either a Data Flow or a Direct Load Flow. For this example, we will use a Direct Load Flow.
In the DMC:
The erties dialog box opens.
The Transformations window opens.
Click the Automap button and click OK.
In this example, we will start the flow and then copy source files to the listen directory.
In the DMC:
dmordbos.dat, dmorddal.dat, dmordorl.dat, dmorstl.da
Tip: For a Windows based server, you can use the Edit and Copy/Paste functions.
Note: As the files are processed, you should see them "disappear" from the listen directory. This could take up to the "timeout" interval specified in the synonym.
With the Discard option set to Archive, after each file is processed it is copied to the archive directory as can be seen in the following image.
These files can be deleted, as they are no longer needed.
iWay Software |