Monday, April 16, 2007

Managed File Transfer using BITS, dark horse is RoboCopy

My current project is considering using Microsoft BITS (Background Intelligent Transfer Service) for managed file transfer between several document management file servers around the world. BITS is already in use in this solution to allow customers to upload very large files in a robust and reliable manner.

As BITS is "designed for C and C++ developers", you will need to create some interop asseblies to be able to use BITS from a .NET application or service. I have not been able to find any official PIA provided by Microsoft. Luckily, the community already provides some very useful stuff:
  • Managed BITS wrapper for v1, 2 and 3 by RodgerB: adjusts to which version that's installed and makes event handling really simple
  • Basic BITS C# wrapper by Eddie Tse: includes support for the SetCredentials method needed for authentication
Most of the articles you'll find about BITS are about downloading files from the remote IIS file server, in addition to being a few years old. BITS is, however, also capable of uploading files to a remote IIS server that have BITS installed and configured. Read Using BITS to Upload Files with .NET by Phil Wilson to learn more.

I wish the new peer file transfer mechanism of v3 was an option for us, but it is limited to Vista and Windows Server "Longhorn". As our use of BITS is for internal managed file transfer between file servers, we have to stick with what's supported on WS2000 and WS2003.

Alas, RoboCopy might be sufficient for getting robust and reliable file transfer, so I'm looking into it's capabilities for bandwidth throttling, background transfers, and for instrumentation and monitoring. The robustness when faced with server restarts, and not just dropped LAN/WAN connections, is on top of my list of issues.

The main reason for considering RoboCopy as an alternative to BITS is that this is internal LAN/WAN managed file transfer and the need for installing IIS at the remote file servers when using BITS. As BITS operates on static files only and have some upload limitations (single file per job, job scalability, target IIS virtual directory, etc), RoboCopy seems to be a good alternative - especially for mass uploading of folders and files to remote locations. Download the Windows Resource Kit Tools to get RoboCopy, including documentation of the multitude of options. TechNet provides a RoboCopy GUI.

Speaking of operations and monitoring, you will need the BitsAdmin tool included in the WinXP Support Tools download. It's just a simple command line tool, but it lets you list and manage BITS jobs.

3 comments:

Anonymous said...

other solution is Spazio MFT

http://www.primeur.com/products/system_integration/managed_secure_file_transfer.php#sf

......

Anonymous said...

Hi,
How did you get on with throttling bandwidth using RoboCopy?

(I'm looking at BITS 3.0 for transferring .WIM images across the network.)

Mark

Kjell-Sverre Jerijærvi said...

the only way afaik is to use /IPG to specify the inter package gap to ensure that not all network packages are consumed by RoboCopy traffic