rclone

rclone is a powerful tool that allows you to interact with a cloud storage provider using a CLI. It supports many cloud providers. With rclone installed and configured properly, you can quickly copy files from your service and vice versa to a remote storage provider such as Google Drive quickly. Not only this, but with more detailed configurations you are able to use cloud storage providers as virtual disks on your service.

Installation, Configuration & Usage of rclone

Ultra.cc support is unable to provide support for rclone, this is due to the large volume of variables and different configurations possible with rclone. The guides found here on the knowledge-base should be able to guide you through using rclone, and any further questions can easily be answered with a quick Google search. You may also be able to find community support for rclone through our community Discord server or the Rclone forums.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders, this creates massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits. Ignorance is not an excuse.

Please do not mount to any of the default directories such as:
files
media
bin
.apps
.config
www
/homexx/username/
or any pre-created directory found on your Ultra.cc Slot

In this guide, we will be going over the installation of rclone. We'll also cover the basic usages such as setting up a remote and how to copy files between your slot and a remote file host.

Installation on your Ultra.cc Service

To begin, make sure you know how to SSH into your slot. All rclone commands are performed via SSH. You can find a guide on SSH here.

To install rclone, run the command below; this command will automatically install rclone to your slot for you.

rclone stable

curl https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/rclone-install-stable.sh | bash

rclone beta

curl https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/rclone-install-beta.sh | bash

Installation on your local PC

Windows

MacOS

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
brew install macfuse
brew install sudo
sudo su
curl https://rclone.org/install.sh | sudo bash 

Linux

curl https://rclone.org/install.sh | sudo bash

Configuration

OAuth (Google Drive)

Ultra_000161.png

Ultra_000160.png

rclone

Now we need to configure a remote to use with rclone. For this guide, we will be configuring Google Drive. This is the most common remote people tend to use as it offers ample storage capacities for a reasonable price. Should you wish to use a different cloud host, feel free to modify the steps, you take.

kbguides@lw914:~$ rclone config
2019/06/15 18:16:33 NOTICE: Config file "/home27/kbguides/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q>
Type of storage to configure.
Enter a string value—Press Enter for the default ("").
Choose a number from below, or type in your own value
...
14 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
15 / Google Drive
   \ "drive"
16 / Hubic
   \ "hubic"
...
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> example12345

Enter a string value. Press Enter for the default ("").
client_secret> example12345
Scope that rclone should use when requesting Access from Drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Full access to all files, excluding Application Data Folder.
   \ "drive"
 2 / Read-only access to file metadata and file contents.
   \ "Drive.readonly"
   / Access to files created by rclone only.
 3 | These are visible on the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ "Drive.file"
   / Allows read and write Access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ "Drive.appfolder"
   / Allows read-only Access to file metadata but
 5 | does not allow any access to read or download file content.
   \ "drive.metadata.readonly"
scope> 1
The ID of the root folder
Leave blank normally.
Fill in to access "Computers" folders. (see docs).
Enter a string value. Press Enter for the default ("").
root_folder_id>

Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Enter a string value. Press **Enter** for the default ("").
service_account_file>

Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n> n
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
        rclone authorize "drive" "eyJjbGllbnRfaWQiOiI5OTIyMjQwOTkzNTYtMWtsZmJrYnFjamE5YWtqdmJiZTZsNGgwdHFhc25pZXEuYXBw"
Then paste the result.
Enter a value.
config_token>
--------------------
[gdrive]
type = drive
client_id = 78311[REDACTED].apps.googleusercontent.com
client_secret = O34[REDACTED]
scope = drive
token = {"access_token":[REDACTED]}
team_drive = 
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> 

Usage

rclone is interacted with purely through SSH. Please ensure you are familiar with the Linux terminal and using SSH prior to using rclone.

General Commands

These commands are helpful to remember. They allow you to interact with rclone and move files around between your local and remote storage or even between two remote destinations.

config - Execute this command to add, modify or remove remote file hosts.
Usage: rclone config

copy - Used to copy files between two locations, remote -> remote, remote -> local, local -> remote
Usage: rclone copy [-P] {origin} {destination}

move - Same as copy however, it does not leave the files at the source
Usage: rclone move [-P] {origin} {destination}

sync - Will make the destination directory identical to the origin. If files exist on the destination that does not exist on origin, they will be deleted. Be careful with the sync command, as it can cause data loss.
Usage: rclone sync [-P] {origin} {destination}

When dealing with remote filesystems, use:
{remote}:{path}

For example, if you wished to copy a file named movie.mkv from your current working directory to a path named Movies in a remote name Drive you'd use this command:
rclone copy movie.mkv Drive:Movies

Rclone VFS and MergerFS Setup

This guide is for advanced users only and it serves as a guide for you to use rclone and mergerFS. The files here are the recommended settings for our slots and will subject to change whenever there are new configurations that are appropriate for the slots. Furthermore, Ultra.cc is not responsible for any data loss or application errors due to this setup should you proceed and will not provide official support for it due to the large volume of variables and different configurations possible with rclone and mergerFS. You may visit the community discord server or the software's respective forums for assistance.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders, this creates massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits. Ignorance is not an excuse.

Please do not mount to any of the default directories such as:
files
downloads
media
bin
.apps
.config
www
/home/username/
or any pre-created directory found on your Ultra.cc Slot

This section will teach you how to set up a rclone VFS mount and MergerFS on Ultra.cc slots, and it assumes the following:

The workflow of the setup is as follows:

The Pros of this setup are as follows:

And, the cons of this setup are as follows:

Before we proceed, it is imperative to stop all rclone/plexdrive processes and stop all the apps that are connected to your rclone mount before proceeding.

Preparation

Install and Configure rclone

Install and configure rclone if you haven't already. Refer here for more information: Installation, Configuration & Usage of rclone

Install mergerfs

bash <(wget -qO- https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/mergerfs-install.sh)
ultradocs@pollux:~$ which mergerfs 
/home/ultradocs/bin/mergerfs

Change Application Settings

Change your application settings according to rclone Optimizations for Apps.

Setup the Work Flow using a Script

What does the script do?

Run the Script

rclone listremotes
bash <(wget -qO- https://scripts.usbx.me/main/MergerFS-Rclone/rclone-mergerfs.sh)
ls ~/MergerFS

Set Sonarr Correctly

Set Root Folder

Remote Path Mapping

Host: {username}.{servername}.usbx.me
Remote Path: /home/your_username/Stuff/Local/Downloads
Local Path: /home/your_username/MergerFS/Downloads

Set Radarr Correctly

Set Root Folder

Remote Path Mapping

Host: {username}.{servername}.usbx.me
Remote Path: /home/your_username/Stuff/Local/Downloads
Local Path: /home/your_username/MergerFS/Downloads

Connect Sonarr & Radarr to Plex / Emby

Plex Media Server

Name : Anything as per your preference.
Notification Triggers: Check `On Download`, `On Upgrade`, `On Rename`.
Host: 172.17.0.1
Port: 16825 (The port of your Plex Media Server, visible in the Control Panel)
Use SSL: Unchecked
Update Library: Checked

Emby

Name : Anything as per your preference.
Notification Triggers: Check `On Download`, `On Upgrade`, `On Rename`.
Host: 172.17.0.1
Port: Emby's port as given in your Ultra Control Panel.
Use SSL: Unchecked
API Key: Paste the one which was created earlier.
Update Library: Checked

Set your Download Clients Correctly

In both Sonarr & Radarr, you must click on the cog that says Show Advanced in Settings. Some required fields for the Download client settings will not be visible otherwise.

Torrent Clients

rTorrent
Deluge
qBittorrent
Default Torrent Management Mode: Automatic
When Torrent Category changed: Relocate torrent
Transmission

Usenet Clients

mkdir -p ~/Stuff/Local/Downloads/usenet/Sonarr
mkdir -p ~/Stuff/Local/Downloads/usenet/Radarr
NZBGet
SABnzbd

Set your Media Server Applications Correctly

Plex Media Server

Emby

JellyFin

Helpful Information for managing this new Work Flow

rclone-vfs mount: ~/scripts/rclone_vfs_mount.log
mergerfs mount: ~/scripts/mergerfs_mount.log
rclone-upload script: ~/scripts/rclone-upload.log
Restart Mounts: systemctl --user restart rclone-vfs.service mergerfs.service
Stop Mounts: systemctl --user stop rclone-vfs.service mergerfs.service
Disable mounts: systemctl --user disable --now rclone-vfs.service mergerfs.service

Understanding the rclone-upload Script

Normal rclone-upload.sh

#!/bin/bash

lock_file="$HOME/scripts/rclone-upload.lock"

trap 'rm -f "$lock_file"; exit 0' SIGINT SIGTERM
if [ -e "$lock_file" ]
then
    echo "Rclone upload script is already running."
    exit
else
    rm "$HOME"/scripts/rclone-upload.log
    touch "$lock_file"
    "$HOME"/bin/rclone move "$HOME"/Stuff/Local/ your_remote_name: \
        --config="$HOME"/.config/rclone/rclone.conf \
        --exclude "Downloads/**" \
        --drive-chunk-size 64M \
        --tpslimit 5 \
        -vvv \
        --drive-stop-on-upload-limit \
        --delete-empty-src-dirs \
        --bwlimit=8M \
        --use-mmap \
        --transfers=2 \
        --checkers=4 \
        --log-file "$HOME"/scripts/rclone-upload.log
    rm -f "$lock_file"
    trap - SIGINT SIGTERM
    exit
fi

rclone-upload.sh with Discord Notification

#!/bin/bash

# Rclone upload script with optional Discord notification upon move completion (if something is moved)
#
# Recommended for use via cron
# For example: */10 * * * * /path/to/rclone-upload.sh >/dev/null 2>&1
# -----------------------------------------------------------------------------

SOURCE_DIR="$HOME/Stuff/Local/"
DESTINATION_DIR="your_remote_name:"

DISCORD_WEBHOOK_URL=""
DISCORD_ICON_OVERRIDE="https://i.imgur.com/MZYwA1I.png"
DISCORD_NAME_OVERRIDE="RCLONE"

LOCK_FILE="$HOME/rclone-upload.lock"
LOG_FILE="$HOME/rclone-upload.log"

# DO NOT EDIT BELOW THIS LINE UNLESS YOU KNOW WHAT YOU'RE DOING
# -----------------------------------------------------------------------------

trap 'rm -f $LOCK_FILE; exit 0' SIGINT SIGTERM
if [ -e "$LOCK_FILE" ]
then
  echo "$0 is already running."
  exit
else
  touch "$LOCK_FILE"
  
  rclone_move() {
    rclone_command=$(
      "$HOME"/bin/rclone move -vP \
      --config="$HOME"/.config/rclone/rclone.conf \
      --exclude "Downloads/**" \
      --drive-chunk-size 64M \
      --use-mmap \
      --delete-empty-src-dirs \
      --log-file="$LOG_FILE" \
      --stats=9999m \
      --tpslimit=5 \
      --transfers=2 \
      --checkers=4 \
      --bwlimit=8M \
      --drive-stop-on-upload-limit \
      "$SOURCE_DIR" "$DESTINATION_DIR" 2>&1
    )
    # "--stats=9999m" mitigates early stats output 
    # "2>&1" ensures error output when running via command line
    echo "$rclone_command"
  }
  rclone_move

  if [ "$DISCORD_WEBHOOK_URL" != "" ]; then
  
    rclone_sani_command="$(echo $rclone_command | sed 's/\x1b\[[0-9;]*[a-zA-Z]//g')" # Remove all escape sequences

    # Notifications assume following rclone ouput: 
    # Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA - Errors: 0 Checks: 0 / 0, - Transferred: 0 / 0, - Elapsed time: 0.0s

    transferred_amount=${rclone_sani_command#*Transferred: }
    transferred_amount=${transferred_amount%% /*}
    
    send_notification() {
      output_transferred_main=${rclone_sani_command#*Transferred: }
      output_transferred_main=${output_transferred_main% Errors*}
      output_errors=${rclone_sani_command#*Errors: }
      output_errors=${output_errors% Checks*}
      output_checks=${rclone_sani_command#*Checks: }
      output_checks=${output_checks% Transferred*}
      output_transferred=${rclone_sani_command##*Transferred: }
      output_transferred=${output_transferred% Elapsed*}
      output_elapsed=${rclone_sani_command##*Elapsed time: }
      
      notification_data='{
        "username": "'"$DISCORD_NAME_OVERRIDE"'",
        "avatar_url": "'"$DISCORD_ICON_OVERRIDE"'",
        "content": null,
        "embeds": [
          {
            "title": "Rclone Upload Task: Success!",
            "color": 4094126,
            "fields": [
              {
                "name": "Transferred",
                "value": "'"$output_transferred_main"'"
              },
              {
                "name": "Errors",
                "value": "'"$output_errors"'"
              },
              {
                "name": "Checks",
                "value": "'"$output_checks"'"
              },
              {
                "name": "Transferred",
                "value": "'"$output_transferred"'"
              },
              {
                "name": "Elapsed time",
                "value": "'"$output_elapsed"'"
              }
            ],
            "thumbnail": {
              "url": null
            }
          }
        ]
      }'
      
      /usr/bin/curl -H "Content-Type: application/json" -d "$notification_data" $DISCORD_WEBHOOK_URL 
    }
    
    if [ "$transferred_amount" != "0 B" ]; then
      send_notification
    fi

  fi

  rm -f "$LOCK_FILE"
  trap - SIGINT SIGTERM
  exit
fi

Changing the time at which rclone-upload.sh is executed (Optional)

Please do not execute the upload script more than once a day. It can lead to the violation of our Terms of Service.

Select an editor.  To change later, run 'select-editor'.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.basic
  3. /usr/bin/mcedit
  4. /usr/bin/vim.tiny

Choose 1-4 [1]: 
0 19 * * * /home/your_username/scripts/rclone-upload.sh > /dev/null 2>&1

Rclone Mount using Systemd

This guide is for advanced users only and it serves as a guide for you to use rclone. The systemd files here are the recommended settings for our slots and will subject to change whenever there are new configurations that are appropriate for the slots. Furthermore, Ultra.cc is not responsible for any data loss or application errors due to this setup should you proceed and will not provide official support for it due to the large volume of variables and different configurations possible with rclone. You may visit the community discord server for help.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders, this creates massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits. Ignorance is not an excuse.

Please do not mount to any of the default directories such as:
files
media
bin
.apps
.config
www
/homexx/username/
or any pre-created directory found on your Ultra.cc Slot

Is this method of rclone mounting suitable for your use case?

This method is not suitable for you if you wish to upload/write data to your rclone mount.

Overview

Rclone's Mount allows you to mount any of your cloud storage accounts as part of your slot's file system using FUSE. In this guide, we will teach you how to run a rclone mount using systemd. Take note that this guide is setup using Google Drive as the cloud storage provider used. Should you use any other cloud storage providers, you may need consult rclone documentation for the appropriate flags for your setup.

There are many ways to mount rclone. You can run the rclone Mount using the screen utility, create a script for running rclone Mount, and checking if the command is still alive, using the --daemon flag, just to mention a few.

We recommend using systemd for several reasons:

rclone mount with no mergerfs

If you don't need to upload files to your mount, follow this guide.

Should you need to upload files or you're planning an automated setup involving your cloud storage, we recommend using Rclone VFS and MergerFS Setup instead.

Preparation

kbguides@lw902:~$ mkdir -p ~/Stuff/Mount
kbguides@lw902:~$
kbguides@lw902:~$ mkdir ~/scripts
kbguides@lw902:~$
$ @lw666:~$ rclone listremotes
Gsuite:

Downloading Rclone Service File

Rclone Mount for Most Remotes
wget -P ~/.config/systemd/user/ https://scripts.usbx.me/main/MergerFS-Rclone/Service%20Files/rclone-normal.service && nano ~/.config/systemd/user/rclone-normal.service
Google Drive Rclone Mount for Media Streaming
wget -P ~/.config/systemd/user/ https://scripts.usbx.me/main/MergerFS-Rclone/Service%20Files/rclone-vfs.service && nano ~/.config/systemd/user/rclone-vfs.service

Editing your service file

Example rclone-normal.service

Rclone.png

Example rclone-vfs.service

Annotation-on-2021-08-11-at-20-19-06.png

Systemd Commands

{mount-name} refers to the service of your choice. Either rclone-normal.service or rclone-vfs.service

Enabling and starting Rclone mount: systemctl --user enable --now {mount-name}.service
Restart Rclone Mount: systemctl --user restart {mount-name}.service
Stop Rclone Mount: systemctl --user stop {mount-name}.service
Stop and disable Rclone mount: systemctl --user disable --now {mount-name}.service (Remove service file after)

rclone Optimizations for Apps

This guide lists optimizations when using rclone mount to host your files to your slot. This is to lessen the API request hits made by rclone to your cloud streaming service and to not hit your cloud streaming's download quotas.

Media Centers

Plex

Settings ⇾ Library
Settings ⇾ Scheduled Tasks

Emby/Jellyfin

Settings ⇾ Advanced ⇾ Scheduled Tasks

Downloaders

Sonarr/Radarr

Settings ⇾ Media Management ⇾ File Management

Rclone Mount on Apple Silicon

Creating an Rclone Mount on Apple Silicon (M1 Chips)

These steps may work on intel based machines but are untested and M1 Chips require extra steps due to the unique architecture.

Installing Homebrew installer

Homebrew installs the stuff you need that Apple (or your Linux system) didn’t. If your familiar with linux you will find a few normal functions are missing or work differently on MacOS “brew” helps bridge this gap.

You can find out more here: https://brew.sh/

Installing Brew is very simple simply run the command bellow and follow the prompts. /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)

Installing MacFuse and Sudo

brew install macfuse brew install sudo

Install rclone under sudo

Installing Rclone is easy but once again Apple will have you jump a few hoops to do so, First you must switch to a root user to allow for rclone to install. For some reason running the command directly as your user even with sudo installed will fail due to permissions.

To switch to root simply run sudo su Then enter your Mac Password You will find your prompt has changed on the left and removed your username You can now install rclone: curl https://rclone.org/install.sh | sudo bash

Once installed successfully you can now exit the root user by typing exit then pressing enter

Now that your back into your main user you need to create a mount folder change username to match your mac username

mkdir /Users/username/Documents/Rclone

Creating a Rclone remote to connect to your Ultra slot

Configuring our Rclone FTP remote The first step is to invoke the rclone config menu with the rclone config command This will present a list of options

e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q>

Field Value
Host Your Ultra Address (servername.usbx.me) or IP (185.123.123.123)
User Your Ultra Username
Port Port 21 is standard. You can press Enter
FTP password Press Y, then enter your FTP Password. You will be asked for this twice.
Use FTPS over TLS (Implicit) False
Use FTP over TLS (Explicit) True
Edit advanced config? (y/n) Press Y
Maximum number of FTP simultaneous connections, 0 for unlimited Press enter
no_check_certificate True
Disable using EPSV even if server advertises support Press Enter
Enter a encoder.MultiEncoder value. Press Enter for the default ("Slash,Del,Ctl,RightSpace,Dot"). Press Enter
y) Yes this is OK (default) Press Y to complete the rclone configuration

Configuring Mac Permissions for FUSE

Upon your first attempt to mount your new remote you will be greeted by a rather angry looking denied message as well as pop up explaining that you will need to change your security settings in order to allow macfuse to run.

To change the security setting on your MacBook with the M1 chip, you need to go into macOS Recovery mode.

  1. Turn on your MacBook holding the power button. Click "Security Policy..." on Startup Security Utility". If you do not see the Startup disk, follow the steps below.
  2. Click "Continue of Options" and "Select a user you know the password for : " will appear. Select your account and click Next. Then enter your password. Select "Utilities" on the top left and select "Startup Security Utility". Click "Unlock", enter your password and click "Security Policy...".
  3. On the screen below, choose "Reduced Security", check "Allow user management of kernel extensions from identified developers" and click "OK".
  4. Reboot and visit System Preferences > Security and Privacy > General. Click the lock icon at the bottom, enter your password and click Allow in front of the message that says System Extension by Benjamin Fleischer was blocked.

Test the mount

Install screen using brew as it will allow you to keep the mount running without leaving a command window open. brew install screen Now enter a new screen screen -S mounttest And attempt to mount rclone cmount Gsuite: /Users/jackalblood/Documents/Rclone If all is well it will appear as if nothing has happened this is fine. Press Command + A then D to detach leaving the mount running Since we are in a command window already we can go ahead and use this to verify it is functional, to do so simply run :

cd /Users/username/Documents/Rclone substitute username with your mac username Then run the command ls you should now see all of the folders you have on your ultra slot, They will also be visible in a folder now called ultra in your documents folder (Mac OS changes the name based on the remote mounted

Ultra.cc External Backup Script using Rclone

This Shell script will allow for an automatic backup of important information related to the configuration of your applications excluding Plex.. It can then be configured to store it or either google drive or a local folder on your Ultras Slot.

Initial Setup and Configuration

create three new folders, one may already exist this is fine mkdir ~/lock mkdir ~/scripts mkdir ~/autobackup

Then you need to enter the new folder called scripts

cd ~/scripts

And create the script file

nano backup.sh

Paste the following lines into it :

#!/bin/bash
exec {lock_fd}>/home/your-username/lock/BackupLock || exit 1
flock -n "$lock_fd" || { echo "ERROR: flock() failed." >&2; exit 1; }
if [ -z "$STY" ]; then exec screen -dm -S autobackup /bin/bash "$0"; fi

DATE=$(date +%Y-%m-%d-%H%M%S)
tar --exclude="$HOME"/.config/plex -czvf "$HOME"/autobackup/MyUSBbackup-$DATE.tar.gz "$HOME"/.apps "$HOME"/.config



rclone copy -v --bwlimit=5M /home/your-username/autobackup/ remotename:/”My USB Backup”


flock -u "$lock_fd"

Save it by pressing Ctrl+X then Y then Enter.

You will need to change the paths home/your-username to match your own home and username. You will also be required to change remotename to match your rclone remote name which can be found with the rclone config command or rclone listremotes. These need to be run via SSH Command line.

Testing

So to test first we navigate to ~/scripts folder we made earlier

cd ~/scripts

Then we need to allow the backup.sh permissions to run

chmod +x backup.sh

And finally, run it

./backup.sh

If all went well * tar -ztvf ~/autobackup/.tar.gz should show a bunch of files starting with /home/username/.config and /home/username/.apps inside.

If the script is running and you were to rerun it, you may see an error message “Flock Failed” this is a file lock to stop multiple downloads running and is normal. If you are sure it isn’t running you can delete the lock file from ~/lock. You can also check the progress of the backup script, which is running in a screen with the command screen -rd autobackup and then to exit the screen while keeping the command running use the keyboard combo CONTROL + A + D

If all is well after the test, we can automate the check via crontab

Open crontab with

crontab -e

You may have a choice of editors. We recommend Nano

Inside the crontab add a single line under everything else in the file that looks like this

0 0 */3 * * /home/your-username/scripts/backup.sh

Save it by pressing Ctrl+X then Y then Enter.

The script will now run every 3 days, checking for files that have changed and syncing them to the destination folder

Using Rclone FTPS

Rclone is a software primarily used with cloud storage providers; however, it is also an easy-to-use Command-line download manager for many protocols, including FTP/FTPS/SFTP. We will show you today how to install rclone onto a Linux machine on your local network and how to set up automatic downloading from your Ultra.cc Slot.

Requirements:

Host machine on your home/local network running a Linux distribution (Mac OS can also be used but is not in the scope of this guide; however, it may still work as a general guide)

Ultra.cc FTP Slot details

Installing Rclone on the Host Machine

Installing rclone is made incredibly easy by the developer. First login to your Host via SSH; this can be under a sudo user, but it is recommended to install rclone as root user and later set up rclone under a sudo/sub-user, never set up under root.

Once logged in, run this curl command

curl https://rclone.org/install.sh | sudo bash

If you receive a message telling you that the “curl command cannot be found,” you will need to install it and run the command above again. To install curl, run apt-get install curl -y

Once rclone is installed, it is recommended you switch to a new user and configure under that user. If you have not created a new user, the steps are simple.

adduser ultra ultra is the username we will use in this guide.

Follow the on-screen prompts for user information and password. Once that is completed, it is an excellent idea to promote this user to sudo; this allows for the execution of higher privileged commands, which are not required for Rclone but are certainly worth setting up.

To add a user to sudo after creation, run this command.

adduser ultra sudo

If, for some odd reason, your distro did not have sudo installed, now might be a good time to run an install just in case:

apt-get install sudo -y

Now we can switch to our ultra user

su ultra

Type cd to jump to the user home folder. This isn’t strictly required but is best practice.

Configuring our Rclone FTP remote

The first step is to invoke the rclone config menu with the rclone config command

This will present a list of options


e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q\>

Press N then enter

Now enter a name: you will need this later when writing our automated script.

Now you will be presented with an extremely long list of options at the time of writing this 11 is FTP, but this may change as updates release

Field Value
Host Your Ultra Address (servername.usbx.me) or IP (185.123.123.123)
User Your Ultra Username
Port Port 21 is standard. You can press Enter
FTP password Press Y, then enter your FTP Password. You will be asked for this twice.
Use FTPS over TLS (Implicit) False
Use FTP over TLS (Explicit) True
Edit advanced config? (y/n) Press Y
Maximum number of FTP simultaneous connections, 0 for unlimited Press enter
no_check_certificate True
Disable using EPSV even if server advertises support Press Enter
Enter a encoder.MultiEncoder value. Press Enter for the default ("Slash,Del,Ctl,RightSpace,Dot"). Press Enter
y) Yes this is OK (default) Press Y to complete the rclone configuration

Testing Remote for access

Once the configuration of the remote is completed, it is a relatively simple job to check if all is configured correctly

One command should list all of the folders in your Ultra Slot Home folder.

rclone lsd ultra: Replace Ultra with your remote name

Automating download from Ultra.cc

Our first step is to write a simple script. We will set up a lock folder for a file lock system so the script will not run if it is already running

Create the lock folder with mkdir -p ~/lock

`if you ever get a flock failed command delete any files in ~/lock with the command rm -rdf ~/lock/*

And create the script file

nano FTP.sh

Paste the following lines into it :

#!/bin/bash

exec {lock_fd}>/home/ultra/lock/FTPLOCK || exit 1

flock -n "$lock_fd" || { echo "ERROR: flock() failed." >&2; exit 1; }

if [ -z "$STY" ]; then exec screen -dm -S FTP /bin/bash "$0"; fi

rclone copy -v remote:/ultraslotfoldertocopy /home/ultra/pathtosavefiles

flock -u "$lock_fd"

Save it by pressing Ctrl+X then Y Enter.

You will need to change the paths ultraslotfoldertocopy to match the folder you wish to download. You will also be required to alter remotename to match your rclone remote name, which can be found with the rclone config command or rclone listremotes. These need to be run via the SSH Command line.

Now we need to allow it to run with the command chmod +x FTP.sh

And finally, run it

./FTP.sh

If the script is running and you were to rerun it, you may see an error message “Flock Failed” this is a file lock to stop multiple downloads running and is normal. If you are sure it isn’t running you can delete the lock file from ~/lock. You can also check the progress of the backup script, which is running in a screen with the command screen -rd FTP

If all is well after the test, we can automate the check via crontab

Open crontab with

crontab -e

You may have a choice of editors. We recommend Nano

Inside the crontab add a single line under everything else in the file that looks like this

* * * * * /home/ultra/FTP.sh

Save it by pressing Ctrl+X then Y then Enter.

The script will now run every 1 minute, checking for files that have changed and syncing them to the destination folder

Troubleshooting Information

Can you help me with rclone?

Rclone is a powerful tool that allows you to create virtual drives or mount points. The following help in this section will assume that you have installed your rclone with the help of our documentation and guidelines.

While it is an excellent tool, it is not very reliable. It should never be used to write data directly to your cloud services with our applications. Use mergerfs for that purpose. It is also essential that those who utilize them will know how to troubleshoot them when needed.

Please do not mount to any of the default directories such as:

This will lead to instabilities within your slot.

Please click here to go to our Rclone setup guide.

Due to these variables involved, official support for rclone or applications utilizing these applications will not be provided.

Job for rclone-vfs.service failed because the control process exited with error code. See "systemctl --user status rclone-vfs.service" and "journalctl --user -xe" for details.

This may be due to the following reasons:

ExecStart=/homexx/yyyyy/bin/rclone mount **gdrive**: /homexx/yyyyy/Stuff/Mount \

Following the advice above, you should find the issue, and when you have, do not forget to execute systemctl --user daemon-reload before trying to enable it again.

ls: cannot access 'SomeDirectory': Transport endpoint is not connected

This means the rclone process/mount is dead, but the mount is still fused. If you’re using systemd following our guide, please stop your mount using systemctl:

systemctl --user stop rclone-vfs
systemctl --user stop mergerfs
mount | grep $USER
fusermount -zu /path/to/SomeDirectory
killall rclone

Now follow the process to restart your mount below.

How do I restart my rclone cloud or mergerfs mount?

Execute the following in your shell terminal (SSH)

  1. Stop all currently running applications that utilize the mount. This applies to Plex, Radarr, Sonarr, etc. (can be done in UCP or ssh using app-[name] stop).
  2. SSH into your slot.
  3. Run systemctl --user stop rclone-vfs and systemctl --user stop mergerfs if using mergerfs
  4. Run fusermount -uq /path/to/mount and fusermount -uq /path/to/mergerfs if using mergerfs
  5. Run systemctl --user start rclone-vfs and systemctl --user start mergerfs if running mergerfs
  6. Verify that the mount is working by going into your mount folder and running ls -la to verify we are connected. If using mergerfs, check the mergerfs folder too.
  7. Start all applications from the UCP or ssh using app-[name] start. (replace [name] with app name, app-plex start.)

I use rclone or a cloud mount, and my xx application is not working or is slow. Why is that?

As stated, you should not use rclone or cloud mount directly with applications that will attempt to write directly to it. It would be best if you used MergerFS for that. Please check our resource for MergerFS here.

My mount is failing. Why?

There may be many things that can cause rclone/mounting to not work correctly, something as simple as a mistype; if you find yourself stuck, it is possible to add debug logging to your rclone systemd file. The steps to do so will be listed below:

  1. First kill the mount if mounted
systemctl --user stop rclone-vfs
systemctl --user stop mergerfs
mount | grep $USER
fusermount -zu /path/to/SomeDirectory
killall rclone
  1. Add debug flag to mounting systemd file

Example

[Unit]
Description=RClone VFS Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
KillMode=none
Environment=GOMAXPROCS=2

ExecStart=%h/bin/rclone mount remote: %h/Mount \
  --config %h/.config/rclone/rclone.conf \
  --use-mmap \
  --dir-cache-time 1000h \
  --poll-interval=15s \
  --vfs-cache-mode writes \
  --log-level DEBUG --log-file "%h/scripts/rclone-debug.log" \
  --tpslimit 10

StandardOutput=file:%h/scripts/rclone_vfs_mount.log
ExecStop=/bin/fusermount -uz %h/Mount
Restart=on-failure

[Install]
WantedBy=default.target

This edit will work for each rclone systemd file, please be sure to edit your log output path accordingly to match your home and user paths.

  1. Reload Systemd file with the new flag

systemctl --user daemon-reload

systemctl --user start rclone-vfs

systemctl --user start mergerfs if running mergerfs

  1. Reading the debug logs

watch tail -n 15 ~/scripts/rclone-debug.log

This will show you the output of the log and refresh its output every 15 seconds. You can change the -n 15 value if you wish to make it more frequent.