LFTP

Command line based FTP client

LFTP is a fully featured FTP client with a CLI interface:

• Multiple segmented Download
• Automation via Bash script
• Multiple simultaneous downloads
• Directory Navigation

Multiple Segmented Downloads

Clients that have an issue with SFTP download speeds, can take advantage of LFTP's feature to download data in multiple segments. It basically breaks down a file into smaller pieces and downloads them separately to give a boost to your overall download speed.

Each piece is downloaded using it's own connection which means that on paper your speed will be multiplied by the number of segments that are there. However, in the real world that is not the case and you should experiment a little to figure out the number of segments that give you the best performance.

• Connect to your Ultra service via SSH.
• The above is necessary so that your local machine is able to verify the host key of your Ultra.cc service when you use LFTP in the future.
• Exit your service's SSH with the following command: exit
• Use LFTP to access your service's SFTP server.

lftp sftp://username@servername.usbx.me

Example:
lftp sftp://ultradocs@oblivion.usbx.me

• Navigate to the directory on your service that contains the data that you wish to download. By default, data is saved in ~/downloads/<torrent_client_name>. Given below are example commands to change the directory to ~/downloads/rtorrent and then view the data that it contains.

cd ~/downloads/rtorrent
ls

• Navigate to the directory on your local machine where you wish to save the downloaded data. Given below is an example to change the local directory to the Desktop folder and view its contents.

cd C:\\Users\\ultradocs\\Desktop
ls

• Use 5 segments to download a single file.

pget -n 5 <filename>

Example:
pget -n 5 ubuntu.iso

The above will download ~/downloads/rtorrent/ubuntu.iso to the Desktop folder on your local machine.

• Use 5 segments to download a directory.

mirror --use-pget-n=5 <directory_name>

Example:
mirror --use-pget-n=5 MyLinuxISOs

• Use segmented downloads along with parallel downloading for best download speeds. In the example given below, we are using 5 parallel downloads along with 5 segments.

mirror --parallel=5 --use-pget-n=5 MyLinuxISOs

The above will download ~/downloads/rtorrent/MyLinuxISOs to the Desktop folder on your local machine.

As mentioned previously, you can experiment with the number of segments and parallel downloads to get maximum download speeds. We would not recommend setting a number higher than 10.

Push files from your slot using LFTP

This is for those who would like to automate downloading or "pushing" files off your Ultra.cc Slot to another FTP server. For example, you may run a Network Attached Storage device at home which is able to run a simple SFTP/FTP server. This command below can be set up to run automatically and send anything in one folder to your Server at your residence while you sleep. Great for grabbing the latest Nightlight Linux distro's.

lftp -u destinationusername,destinationpassword -e 'mirror --reverse --parallel=3 --continue --only-missing /home/usbdocs/files/LFTP /home/destinationusername/LFTP; quit' sftp://destinationip:port

The Path /home/usbdocs/files/LFTP is used purely as an example. You could change this to any path on your Ultra.cc Slot and LFTP would push that path to the target. You may also add multiple copies of the above line when creating your bash script (details below) to sync multiple folders to multiple target servers and/or folders.

Upload files to your slot using LFTP

Uploading to your slot is just as simple as downloading from your slot "pushing". Using the same command above will work as long as it is run from the location you want your files to arrive at.

lftp -u usbslotusername,usbslotpassword -e 'mirror --reverse --parallel=3 --continue --only-missing /home/destinationusername/LFTP /home/usbdocs/files/LFTP; quit' sftp://destinationip:22

Notice that the main changes are the order of the directory and the address at the end. The function of the command is exactly the same; It is just the direction that has changed.

Automate the Process

• Your first step is to find the full path of your home directory pwd will output your full home path.
• Something like this will be outputted. Be sure to make a note of it.

/home/username

• Next, create the following directories.

mkdir ~/lock && mkdir ~/scripts

• Then, navigate to the ~/scripts directory.

cd ~/scripts

• And create the script file

nano LFTP.sh

• Paste the following lines into the editor:
• Make sure to replace username with your Ultra username.

#!/bin/bash
exec {lock_fd}>/home/username/lock/lftpLock || exit 1
flock -n "$lock_fd" || { echo "ERROR: flock() failed." >&2; exit 1; }
if [ -z "$STY" ]; then exec screen -dm -S lftp /bin/bash "$0"; fi

lftp -u destinationusername,destinationpassword 'mirror --reverse --parallel=3 --continue --use-pget-n=5 --only-missing /home/username/files/LFTP /home/destinationusername/LFTP; quit' sftp://destinationip:port

flock -u "$lock_fd"

• Save the script and exit the nano editor by pressing Ctrl+X then Y then ENTER.
• Now we need to set a few things that will be passed to LFTP automatically ready for automation.
• Create a config directory for lftp.

mkdir ~/.lftp

• Next, create and open the lftp rc file in the nano editor.

nano ~/.lftp/rc

• Paste these lines in and save

set net:limit-total-rate 0:41943040
set ssl:verify-certificate no

• Save and exit the nano editor by pressing Ctrl+X then Y then ENTER.

Test Script

• First, navigate to the ~/scripts folder.

cd ~/scripts

• Next, we need to make the script executable.

chmod +x LFTP.sh

• The script is now ready to be executed, which can be done with the following command.

./LFTP.sh

If the script is running and you were to rerun it, you may see an error message ERROR: flock() failed. This is a file lock to stop multiple script executions running simultaneously. If you are sure the script is not in fact running, you can delete the lock file from the ~/lock directory.

Check Progress

You can also check the progress of the backup script, which is running in a screen session.

• Attach to the screen session with the following command.

screen -rd lftp

• To detach from the screen session and leave the script running in the background, press ctrl+a and then press d.

Automate Script

Once you have concluded the script to be running successfully, you can schedule it to be automatically executed via crontab.

• Open crontab with the following command:

crontab -e

• Inside the crontab add the following line at the bottom of the crontab file.
• Make sure to replace username with your Ultra username.

0 */4 * * * /home/username/scripts/LFTP.sh

• The script will now run every 4 hours, checking for files that have changed and sync them to the target server. Please be aware that anything removed from Target will be synced again. Remove the item from the source or move it to a non synced folder.
• Save and exit crontab by pressing Ctrl+X then Y and ENTER.

If you require further assistance, you can open a support ticket here!

Feedback and suggestions regarding this article are welcome on our Discord server

• LFTP
• Command line based FTP client
• Multiple Segmented Downloads
• Push files from your slot using LFTP
• Upload files to your slot using LFTP
• Automate the Process
• Test Script
• Check Progress
• Automate Script