SFTP Command in Batch Mode Hangs Using java.lang.Runtime.exec() in Java


When working on applications that invoke system calls to execute native commands via java.lang.Runtime.exec(), be careful with invocations that could potentially output an overwhelmingly large number of texts to the standard output. Take for example, the sftp command in Linux. To transfer large number of files to a remote server,  one typical way to do it is in batch mode. “put <local-file>” commands are written to a batch file which is fed to the sftp command.

all_put_cmd.txt contains a series of commands:

If the number of commands in the batch file is overwhelmingly large and executed via Runtime.exec(), the subprocess may become blocked after outputting a few hundred lines of texts and will never complete its operation.

This is actually mentioned in the JDK Javadoc documentation and explains why Runtime.exe() hangs:

Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

It basically means that the buffers that temporarily hold the outputs emitting from the subprocess are limited in sizes and when free spaces run out, the subprocess may become blocked. To avoid this from happening, the standard input and output streams are read from and outputs redirected to a files or the main process’ standard output, e.g., System.out.println().

Software Environment

  • Windows 7 Professional SP1
  • Eclipse – Kepler Release
  • Java 1.7 (1.7.0_67 – Windows x86)

Sample Implementation

Sample Usage