SFTP

In this section:

When you connect to a server using SFTP, SSH encryption is used to protect the connection between your client machine and the server. This protects your password and your data, preventing an eavesdropper from capturing or stealing them as they travel over the network.

Despite the similarity in name and operation, SFTP is a completely different protocol from FTP and does not support all the same features and commands as FTP. Also, while they are both secure file transfer protocols and have similar names, FTPS (FTP with TLS/SSL) should not be confused with SFTP.

To use SFTP for secure connections, the server you are connecting to must also support SFTP. If you try to connect with SFTP to a server that doesn't support it, you will receive an error. Your network administrator or service provider can tell you if your server supports SFTP, and what other information you might need to use SFTP if it does.


Top of page

x
Supported Components

The following components are supported by the SSH File Transfer Protocol:


Top of page

x
Password Authentication Versus Key Pair-based Authentication

All the SFTP components support both password based and keypair based authentication without password.

In conventional password authentication, you prove you are who you claim to be by proving that you know the correct password. The only way to prove you know the password is to tell the server what you think the password is. This means that if the server has been hacked, or spoofed, an attacker can learn your password.

Key Pair authentication solves this problem. You generate a key pair, consisting of a public key (which everybody is allowed to know) and a private key (which you keep secret and do not give to anybody). The private key is able to generate signatures. A signature created using your private key cannot be forged by anybody who does not have that key; but anybody who has your public key can verify that a particular signature is genuine.

So you generate a key pair on your own computer, and you copy the public key to the server under a certain name. Then, when the server asks you to prove who you are, WinSCP can generate a signature using your private key. The server can verify that signature (since it has your public key) and allow you to log in. Now if the server is hacked or spoofed, the attacker does not gain your private key or password; they only gain one signature. And signatures cannot be re-used, so they have gained nothing.

Note: While using keypair authentication, the private key file path has to be populated in the SFTP component that is invoked. If password based authentication is used, the password field has to be populated while the private key file is left blank. This applies for all SFTP components.


Top of page

x
Installing the SFTP Extension

Before you install the SFTP extension, ensure that the following prerequisites are met:

To install SFTP, you must add the sftp extension to your iWay Service Manager instance during the iWay Service Manager installation. For more information on installing iWay Service Manager, see the iWay Installation and Configuration Guide. However, if you have already installed iWay Service Manager, you can also add the sftp extension by modifying the existing iWay Service Manager installation. This section describes how to add the SFTP extension by modifying an existing iWay Service Manager installation.



x
Procedure: How to Add the SFTP Extension

Since there are unique SFTP components required for the Relay and Executor iWay Service Manager instances, the SFTP extension must be added to both for proper operation. To add the SFTP extension:

  1. Open the Windows Control Panel and double-click Add or Remove Programs.
  2. Select iWay 6.0 Service Manager and click the Change/Remove button.

    The iWay 6.0 Service Manager Repair or Remove Installation dialog box opens.

  3. Select Modify and click Next.

    The iWay 6.0 Service Manager Select Features dialog box opens. Select the check box for SFTP, and click Next.

  4. Click Finish.
  5. Stop and restart iWay Service Manager.

    Note: You must perform a hard restart. Using the Restart function in the iWay Service Manager Administration Console is not sufficient.


Top of page

x
Password Authentication versus Key Pair Based Authentication

All the SFTP components support both password based and keypair based authentication without password.

In conventional password authentication, you prove you are who you claim to be by proving that you know the correct password. The only way to prove you know the password is to tell the server what you think the password is. This means that if the server has been hacked, or spoofed, an attacker can learn your password.

Key Pair authentication solves this problem. You generate a key pair, consisting of a public key (which everybody is allowed to know) and a private key (which you keep secret and do not give to anybody). The private key is able to generate signatures. A signature created using your private key cannot be forged by anybody who does not have that key; but anybody who has your public key can verify that a particular signature is genuine.

So you generate a key pair on your own computer, and you copy the public key to the server under a certain name. Then, when the server asks you to prove who you are, WinSCP can generate a signature using your private key. The server can verify that signature (since it has your public key) and allow you to log in. Now if the server is hacked or spoofed, the attacker does not gain your private key or password; they only gain one signature. And signatures cannot be re-used, so they have gained nothing.

Note: While using keypair authentication, the private key file path has to be populated in the SFTP component that is invoked. If password based authentication is used, the password field has to be populated while the private key file is left blank. This applies for all SFTP components listed below.


Top of page

x
Components Supported by iwsftp

This section lists components that are supported by iwsftp.


Top of page

x
Configuring the SFTP Listener

This section describes how to configure the SFTP listener.

The following table lists and describes the procedure for the SFTP Listerner.

Property Name

Property Description

Host Name

Name of host machine where listener will contact service to obtain requests from.

Remote Port

Port to connect to on the SFTP site, blank for default port 22

Input Path

Directory with optional pattern on SFTP host from which to retrieve files. A specific file name or DOS-style pattern) can be used. Do not use suffix in.

Include Symbolic Links

Set to true if you want the sftp listener to process the symbolic links

Include Hidden Files

Set to true if you want the sftp listener to process the hidden files

Destination Directory

Directory on SFTP host to return responses to

Data, Signal or Streaming

Data. Data file will be retrieved from the SSH server and maintained in memory while processed by the listener.

Signal. Data file will be retrieved from the SSH server and stored locally (requires that the 'Local Store Directory' be filled in); a signal document will be generated by the listener.

Streaming. A connection will be opened with the SSH server and data from the file will be retrieved and processed by the listener as needed.

Local Store Directory

Directory on the iWay server to save the files. Required when 'Data, Signal or Streaming' is set to Signal; otherwise this field is not used.

Remove locally stored files

Set to true if you want the SFTP listener to delete the file stored locally after it is processed

Pending Queue

Directory to hold documents which are to be retried later

Suffix In

Limits input files to those with these extensions. For example, enter "XML,in" to accept files with extensions "xml" and "in". Note that this is not case sensitive.Do not use '.'; use - to mean no extension, or * to mean any.

Duration

Maximum time that a document can remain in the retry pending queue

Retry

Interval between retrying pending requests

Do not unzip ZIP files

Pass ZIP files as a single file for processing (requires ACCEPT FLAT turned on)

Bad File List

Maintain a list of files with errors, preventing them from being re-accessed. If set, files will not be retried.

User Name

User ID on the SFTP server

Password

User's password on the SFTP server

Private Key

Path to the private key file for public-key authentication..

Passphrase

Passphrase used to protect the Private Key

Whitespace Normalization

Specifies how the parser treats whitespace in Element content. Choose preserve to turn off all normalization as prescribed by the XML Specification.Choose condense to remove extra whitespace in pretty printed documents and for compatibility with earlier versions.

Accepts non-XML (flat) only

If true, listener expects flat (non-XML). Automatic parsing is not performed.

Optimize Favoring

Selection of memory is useful for large input document

Multithreading

Number of documents that can be processed in parallel

Execution Time Limit

Time limit for document execution(in seconds) before it is cancellation is attempted. (Also see system property "kill interval". This applies to agent stacks and sets a lower limit for process flows.)

Polling Interval

Interval at which to check for new input

Default Java File Encoding

Default encoding if incoming message is not self-declaring (i.e. XML)

Agent Precedence

Changes order by which engine selects agents. Normally Document overrides listener. This is used to manage iWay documents

Always reply to listener default

If true, the default reply definition is used in addition to defined replies

Error Documents treated normally

If true, error documents will get processed by any configured pre-emitters

Listener is Transaction Manager

If true, agents run within a local transaction managed by the listener

Record in Activity Log(s)

If set, activity on this channel will be recorded in the activity logs, else the activity will not be recorded.



x
Procedure: How to Test the SFTP Listerner Channel Using a Private Key File Without a Password
  1. Construct an inlet consisting of an SFTP listener as shown below:

  2. Construct a channel say, mySFTP consisting of the inlet, move Route and a File emitter which writes the output file to a test directory, say c:\test.

  3. Build and deploy the channel. Start the channel.
  4. Place an xml file on the directory /home/org on the SFTP host either using sftp commands or if the server is a windows machine by dropping it onto the folder.
  5. The SFTP channel processes this message and drops the file onto the directory c:\test on the iway client machine.

Top of page

x
Configuring the SFTP Emitter

This section describes how to configure the SFTP emitter.

The following table lists and describes the procedure for the SFTP Emitter.

Property Name

Property Description

Destination

The absolute path of the file which is being emitted @ The name of the SFTP server. For instance, if the file needs to be emitted to a machine sftpsrv on the directory /home/org and the file is to be saved as out[1..9].xml then the value of this field would be /home/org/out*.xml@sftpsrv as shown above.

User Name

The username on the SFTP server that has read and write access to the directory entered in the Input Path field.

Password

Password for user account to use when connecting to protocol host.

Private Key

SSH private Key file used for server authentication (required is password is omitted).

Pass Phrase

SSH Passphrase used when Private Key was generated (optional).

Mode

Mode of File transfer - ASCII/BINARY

Socket Timeout

Timeout in seconds. With this option set to a non-zero timeout, a read() call on the Socket will block for only this amount of time. If the timeout expires, a java.net.SocketTimeoutException is raised. Default timeout is operating system dependent.

Move To

The directory to which the file is to be moved after it is emitted.



x
Procedure: How to Test an SFTP Emitter Test Channel
  1. Construct an outlet consisting of an SFTP emitter, such as, mySftpEmit as explained in Configuring the SFTP Listener.
  2. Construct a channel say, File1 consisting of a file inlet, move Route and a File emitter which writes the output file to a test directory, for example, c:\test.
  3. Build and deploy the channel, the start the channel.
  4. Place an xml file on the directory File1 listener is listening to the iway server.

    The File1 channel processes this message and drops the file out1.xml onto the directory on the SFTP server machine, which is /home/org as per the configuration settings in How to Test an SFTP Emitter Test Channel.


Top of page

x
Configuring the SFTP Agents

This section describes how to configure the SFTP agents.

The following table lists and describes the procedure for the SFTP Agents.

Property Name

Property Description

File Name Tag *

Name of the tag from the input document in which to find the file name.

Enclose Tag

The name of the tag in which to enclose data read. If omitted, no entagging. If used, output is XML.

Base Path

Optional directory to be used if incoming name is not absolute.

Input Data Format

Format of the input data, default is flat.

Transfer Type

For non-XML, this parameter sets the transfer type.

Host Name

The name of the SFTP server to connect.

Remote Port

Port to connect to on the SFTP site, blank for default port 22.

User Name

The username on the SFTP server through which files are emitted.

Password

Password for user account to use when connecting to protocol host.

Private Key

SSH Private Key file used for server authentication (required is password is omitted).

Pass Phrase

SSH Passphrase used when Private Key was generated (optional).

Encoding

Character set encoding to be performed on the input.

Delete After Read

Flag to show whether to delete the file after the read.



x
Procedure: How to Test an Emit Agent Using a Private Key File

The test case is similar to the SFTP emitter in section 6. However, in this case an agent is used to emit the file to the SFTP server using private key file. An SFTPEmitAgent would be used instead of an SFTPEmitter when a process flow which performs a sequence of tasks (for a business process) needs to be used and statuses need to be evaluated.

  1. Construct a Process flow sftp which consists of a service, say sftpservice which refers to the class XDSFTPEmitAgent. Configure the properties of the agent as shown below:

    The properties are explained in the section Configuring the SFTP Emitter.

    As shown in the above example, the value of the HostName can be

    edasol29

    and the username can be

    edasxr

    where:

    edasxr

    Has write access to the Remote Site Folder directory.

    Also, the Private Key file can point to the id_dsa private key file on the iway server. Enter the values of the fields as shown above.

  2. Construct the pflow as shown below:

  3. Add the process sftp to a route say, myRoute.
  4. Construct a channel say, File1 consisting of a file inlet, myRoute and a default outlet.
  5. Build and deploy the channel. Start the channel.
  6. Place an xml file on the directory File1 listener is listening to on the iway server.

The File1 channel processes this message and drops the file out1.xml onto the directory on the SFTP server referred to by the attribute Remote Site Folder on the sftpservice object in the pflow.



x
Procedure: How to Test a Read Agent Using a Private Key File

The SFTP Read agent is used to read files from an SFTP server (drive on Unix or Windows). It can also be used in tandem with a file listener to embed file contents (the file picked up by the listener) into the xml file read from the SFTP drive by specifying the tag.

  1. Construct a Process flow sftp which consists of a service, say sftpservice which refers to the class XDSFTPReadAgent. Configure the properties of the agent as shown below:

    The properties are explained under Configuring the SFTP Agents.

  2. Construct the pflow as shown below:

  3. Add the process sftp to a route say, myRoute. Construct a channel say, File1 consisting of a file inlet, myRoute and a default outlet.
  4. Build and deploy the channel. Start the channel.
  5. Place an xml file, test.xml on the directory File1 listener is listening to on the iway server. Let the xml file be
     <test>c:\test\a.txt </test>
  6. Let the file, a.txt consist of the following text:
    This is a sftp readagent test
  7. The File1 channel processes the xml file test.xml, reading the file a.txt and generates the file testout.xml onto the destination directory specified in the file listener. The file testout.xml is as follows:
    <testout> This is a sftp readagent test </testout>

Top of page

x
SFTPOps Agent Operations

The SFTP Ops agent stands for SFTP operations. Emits via SSH protocol to a given host<:port> using various common SFTP commands. It can be used to perform operations such as Copy, Prepend, Append, Size, Move and so on.



x
Procedure: How to Test the SFTP Ops Agent for a Copy Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.
  3. Set the properties as shown below:

    The From file test.txt is

    This an SFTPServer test

    The to file out6.txt is:-

    <parent><test1>Soumya</test1></parent>
  4. Test run the process flow SFTPOps.

    The To File out6.txt's content is modified as

    This an SFTPServer test


x
Procedure: How to Test the SFTP Ops Agent for a Move Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below:

  3. The From file /users/edasxr/testdir/test.txt is
    This an SFTPServer test
  4. Test run the process flow SFTPOps.

    The From File test.txt is renamed to testmove.txt.



x
Procedure: How to Test the SFTP Ops Agent for a Prepend Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below.

    The From file /users/edasxr/testdir/out6.txt is:

    this is a test

    The to file /users/edasxr/testdir/testmove.txt is:

    for sftpops agent
  3. Test run the process flow SFTPOps.

    The To File testmove.txt's content is modified as

    this is a testfor sftpops agent


x
Procedure: How to Test the SFTP Ops Agent for an Append Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below.

    The From file /users/edasxr/testdir/out6.txt is

    this is a test

    The to file /users/edasxr/testdir/testmove.txt is

    for sftpops agent
  3. Test run the process flow SFTPOps.

    The To File testmove.txt's content is modified as

    this is a testfor sftpops agent


x
Procedure: How to Test the SFTP Ops Agent for a Delete Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below:

  3. Test run the process flow SFTPOps.

    The From file /users/edasxr/testdir/out6.txt is deleted.



x
Procedure: How to Test the SFTP Ops Agent for a Rename Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below:

  3. The From file /users/edasxr/testdir/test.txt is
    This an SFTPServer test
  4. Test run the process flow SFTPOps.

    The From File test.txt is renamed to testmove.txt.



x
Procedure: How to Test the SFTP Ops Agent for a Size Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below:



x
Procedure: How to Test the SFTP Ops Agent for an Exist Operation
  1. Create a process flow having a service of type XDSFTPOpsAgent.

  2. Let the Service object have class name as XDSFTPOpsAgent.

    Set the properties as shown below


Top of page

x
Installing an OpenSSH Server in Windows

Below are steps on how to install an openSSH server in Windows:

  1. Install the typical server version found at http://sourceforge.net/projects/sshwindows/.
  2. Run the installer by clicking on defaults.

    Note: No configuration is required during installation.

  3. Let the install location be C:\Program files.

Top of page

x
Configuring an OpenSSH Server in Windows

Below are steps on how to configure an openSSH server in Windows:

  1. Open a command prompt and change to the installation directory (Program Files\OpenSSH is the default).
  2. CD into the bin directory.
  3. Use mkgroup to create a group permissions file. For local groups, use the "-l" switch. For domain groups, use the "-d" switch.

    For both domain and local, it is best to run the command twice (remember to use >>, not >). If you use both, make sure to edit the file to remove any duplicate entries.

    mkgroup -l >> ..\etc\group      (local groups)
    mkgroup -d >> ..\etc\group      (domain groups)
  4. Use mkpasswd to add authorized users into the passwd file. For local users, use the "-l" switch. For domain users, use the "-d" switch.

    For both domain and local, it is best to run the command twice (remember to use >>, not >). If you use both, make sure to edit the file to remove any duplicate entries.

    mkpasswd -l [-u <username>] >> ..\etc\passwd      (local users)
    mkpasswd -d [-u <username>] >> ..\etc\passwd      (domain users)

    Note:

    • To add users from a domain that is not the primary domain of the machine, add the domain name after the user name.
    • Omitting the username switch adds ALL users from the machine or domain, including service accounts and the Guest account.
  5. Start the OpenSSH server.
    net start opensshd
  6. Test the server. Using a separate machine as the client is best. If you connect but the connection immediately gets dropped, reboot the machine with the server and try connecting again.

    Note: The major rule in using this utility is to only allow trusted users to have login permissions. The cygwin port of OpenSSH uses the full OpenSSH source code and the security of the program is not diluted.



x
Procedure: How to Setup an SSH Login Without a Password Using a Private Key
  1. SSH to your server (in this case, edasol29) using good old user name and password.
  2. Create an .ssh folder under default login directory. In my case it was /users/[myusername]

    Do check permissions on your ~/.ssh folder and make sure to

    chmod 700 .ssh

    if they are wrong.

  3. Generate the keys on the SSH server with something like
    ssh-keygen -t dsa

    or rsa. Read the main pages if your don't know how to use ssh-keygen.

  4. Accept the file names it wants to use and enter a pass phrase if you need to.
  5. Create an empty file authorized_keys under .ssh folder and add public keys. For example:
    mv id_dsa.pub authorized_keys

    The id_dsa, private key generated can be used to login without password to the openSSH server.

  6. Copy the private key (id_dsa) to your local windows machine (use winscp or sftp or some such tool).
  7. Launch puttygen.exe. Under actions, select load and load the id_dsa file.
  8. Enter the pass phrase you set when you generated the key on the server. Puttygen will now convert the key to a format (.ppk) which is used for SFTP connections by most tools such as putty, Winscp and so on.
  9. Save the file as
    privatekey.ppk
  10. Change your putty settings under connection > SSH > auth to use privatekey.ppk.
  11. Try and connect. Enter the pass phrase if you have one.


x
The /home Directory

In the passwd file, you will notice that the home directory of the user is set as /home/username, with username being the name of the account. In the default install, the /home directory is set to the default profile directory for all users. This is usually C:\Documents and Settings on Windows 2000 and XP, and C:\WINNT\Profiles on Windows NT 4.0. The location of /home can be edited to fit your special requirements by editing a registry key.

To change the Windows directory /home corresponds to, you will need to edit a registry entry under HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/home. The value of the key named native is the directory that /home is. If you want all your users to enter in a directory on your machine called F:\Users, change native to read F:\Users. By default, each user will then be placed in the directory F:\Users\username, where username is the name of the user account. To place the user directly under f:\Users, change the home directory in passwd to /home.



x
Firewalls

The OpenSSH server listens for traffic on TCP port 22 by default. If your firewall setup does not allow connections on this port, it can be changed by editing the etc/sshd_config file.

Note: For additional troubleshooting and OPENSSH advanced configuration, refer to readme.txt under C:\Program files\OpenSSH\docs


iWay Software