Shell Scripting: Using FTP to Automatically Create a Mirror of a Specified Directory of Files on a Remote Server
- Working with FTP On the Command Line
- Turning It into a Shell Script
- Where to Go Next
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
One of the greatest challenges of modern computing is keeping files in sync across multiple computers. Whether you have a local copy of your Web site on your PC or (exactly the opposite) have the master copy of your directory on your local Linux/Unix box and want to make sure that a remote directory has the same files available, it's ridiculously difficult to accomplish.
Which is why it's the perfect task for a shell script! And that's what this article will explore: How to write a simple Unix shell script—suitable for Unix, Linux, or even Mac OS X systems—to keep a local and remote directory synchronized automatically.
To start out, a caveat. I will be demonstrating how to accomplish this task using the FTP program, but I nonetheless don't recommend FTP because it's less secure than its SSH-based cousin SFTP. The problem is, most server administrators leave FTP as the de facto method of connectivity. To switch this entire technique to using SFTP is fairly straightforward, however, so the basic solution will nonetheless be helpful and informative.
Working with FTP On the Command Line
The first step is to figure out what commands you'll need to use with the FTP program itself to get to your remote system. First off, let's see the glorious ugliness that is a command line FTP transaction:
$ ftp laptop.intuitive.com Connected to 192.168.1.106. 220 local FTP server (lukemftpd 1.1) ready. Name (192.168.1.106:taylor): taylor 331 Password required for taylor. Password: 230 User taylor logged in. Remote system type is UNIX. Using binary mode to transfer files. ftp> dir 229 Entering Extended Passive Mode (|||49288|) 150 Opening ASCII mode data connection for '/bin/ls'. total 1 drwx------ 10 taylor taylor 340 Mar 17 15:00 Desktop drwx------ 40 taylor taylor 1360 Dec 10 10:26 Documents drwx------ 37 taylor taylor 1258 Feb 20 14:51 Library drwxr-xr-x 3 taylor taylor 102 Jan 26 2003 Movies drwxr-xr-x 4 taylor taylor 136 Sep 22 15:30 Music drwx------ 73 taylor taylor 2482 Mar 5 11:07 Pictures drwxr-xr-x 4 taylor taylor 136 Jan 29 2002 Public drwxr-xr-x 5 taylor taylor 170 Mar 19 17:59 Sites drwxrwxrwx 9 taylor taylor 306 Jan 11 2002 bin drwx------ 6 taylor taylor 204 Jul 7 2002 mail drwxr-xr-x 143 taylor taylor 4862 Sep 26 13:43 scripts 226 Transfer complete. ftp> quit 221 Thank you for using the FTP service $
Here you can see that I connected to a remote system, laptop.intuitive.com, used taylor as my account name, and entered the necessary password. Once connected, I requested a directory listing with dir and then quit.
The same task can be done without as much interactivity by using a here document, denoted with the << sequence. By using this, I can essentially type in all my FTP commands at once; then let them be fed one-by-one to the program:
$ ftp laptop.intuitive.com << "EOF" > taylor > dir > quit > EOF Connected to 192.168.1.106. 220 local FTP server (lukemftpd 1.1) ready. Name (192.168.1.106:taylor): 331 Password required for taylor. Password: 230 User taylor logged in. Remote system type is UNIX. Using binary mode to transfer files. 229 Entering Extended Passive Mode (|||49367|) 150 Opening ASCII mode data connection for '/bin/ls'. total 1 drwx------ 13 taylor taylor 442 Mar 25 15:52 Desktop drwx------ 40 taylor taylor 1360 Dec 10 10:26 Documents drwx------ 37 taylor taylor 1258 Mar 26 09:37 Library drwxr-xr-x 3 taylor taylor 102 Jan 26 2003 Movies drwxr-xr-x 4 taylor taylor 136 Sep 22 2003 Music drwx------ 73 taylor taylor 2482 Mar 5 11:07 Pictures drwxr-xr-x 4 taylor taylor 136 Jan 29 2002 Public drwxr-xr-x 5 taylor taylor 170 Mar 19 17:59 Sites drwxrwxrwx 9 taylor taylor 306 Jan 11 2002 bin drwx------ 6 taylor taylor 204 Jul 7 2002 mail drwxr-xr-x 143 taylor taylor 4862 Sep 26 13:43 scripts 226 Transfer complete. 221 Thank you for using the FTP service $
Pay close attention to what I typed in and where: Everything but the password itself can easily be dropped into a shell script. Let's have a look.