rclone

rclone is a powerful tool that allows you to interact with a cloud storage provider using a CLI. It supports many cloud providers. With rclone installed and configured properly, you can quickly copy files from your service and vice versa to a remote storage provider such as Google Drive quickly. Not only this, but with more detailed configurations you are able to use cloud storage providers as virtual disks on your service.

Rclone & MergerFS Explainer

This guide explains the concepts of Rclone and MergerFS to empower you to manage and support your tailored setups. It is not intended to be a detailed “How To” guide. We have a selection of those here.

Overview

You can mount and interact with your cloud storage as if it is a directory (folder) within your home directory. For example, you can create a directory called Gdrive on your server that contains all the files stored on your Google Drive.

To achieve this, we use an open-source tool called Rclone to connect to cloud storage and create mounted directories. The basic steps are:

  1. Download the latest version of Rclone. Your ultra.cc service comes with Rclone pre-installed, but it is only updated occasionally, so it’s better to have your own up-to-date version.
  2. Create an Rclone remote. This means configuring Rclone with details about the cloud storage provider (or server) you want to connect to. Once this step is completed, you can interact with your cloud storage using rclone <command> <remotename:>.
  3. Use the Rclone remote created in step 2 to mount your cloud storage to a directory using the rclone mount <remotename:> command.
  4. Set up MergerFS to union your cloud files and local files into a single directory.

Rclone

Installation (Part 1)

curl https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/rclone-install-stable.sh | bash

*If you are installing rclone to use at home or elsewhere, download rclone from the official site and manually install it.

rclone version
rclone selfupdate

Configuring a Rclone Remote (Part 2)

Now that you have the latest version of Rclone, you can configure a connection to any supported storage provider. The process is similar for all storage providers, but each has specific options that need to be considered.

To create a new Rclone remote:

rclone config
  1. Select n for New Remote
user@server:~$ rclone config
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
  1. Select a name of your choice. It makes sense to select a meaningful name; for example, if you’re creating a connection to Dropbox, select something like dropbox or dropbox1 if you plan on having more than one!
Enter name for new remote.
name> dropbox
  1. Type the number next to the storage provider you want to connect to. When Rclone adds new providers, all the numbers can change, so ensure you select the correct one. At the time of writing, if connecting to Dropbox, I would type 13 and press ENTER.
....
13 / Dropbox
   \ (dropbox)
....
Storage> 13
  1. From this point forward, the options you will be prompted to supply will differ for each storage provider. Jump to https://rclone.org and select the required option from the Storage Systems dropdown menu. This will guide you through the specifics of each provider.

On your local PC:

  1. Install Rclone locally. You can find the installation files here: https://rclone.org/downloads/
  2. Drop into the command shell (Command Prompt on Windows, Terminal on Mac).
  3. Change to the directory (using the cd command) that you installed Rclone in to, for example: cd \rclone.
  4. Type rclone version to ensure it works. The version number should be displayed. If it isn’t, review steps 5 to 7 above.
Microsoft Windows [Version 10.0.22621.2715]
(c) Microsoft Corporation. All rights reserved.

C:\Users\yourname>cd\rclone

C:\rclone>rclone version
rclone v1.64.2
- os/version: Microsoft Windows 11 Pro 22H2 (64 bit)
- os/kernel: 10.0.22621.2715 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.21.3
- go/linking: static
- go/tags: cmount
  1. Now jump back to your Ultra.cc ssh session and copy the Rclone command from there into your Command Prompt/Terminal. For example, for Dropbox, you would copy this line to your local device:
rclone authorize "dropbox"
  1. A web browser session will open. Log in to the cloud storage provider using the credentials you signed up with. If you have switched on Two Factor Authentication with your cloud storage provider, you will need to supply those details too.
  2. You will then be asked to Approve/Accept Rclone. Ensure you are happy to do so, and click on Approve/Accept (or similar).
  3. Now go back to the Command Prompt/Terminal on your local PC/Mac and you will see a token displayed. Copy this to your clipboard (everything between the ---> and <---)
C:\Users\username>cd\rclone

C:\rclone>rclone authorize "dropbox"
2023/11/30 16:01:20 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=m5bBA0FfH8nMZyHmGkP0Tg
2023/11/30 16:01:20 NOTICE: Log in and authorize rclone for access
2023/11/30 16:01:20 NOTICE: Waiting for code...
2023/11/30 16:06:29 NOTICE: Got code
Paste the following into your remote machine --->
{"access_token":"sl.Bq2irrOkj8l6-ngKY6apc6Gkhbxb0tRnJECnNG1iw6PAF9s6TzPuDG4Baegb9da3yBeDCzP1D9ZtQ4_LYuDAJLmOjPuFyFPiw2lnlP_TpUS26oNMX_SZPGSrWHFJHHorvaYe46jVvFFBiaU","token_type":"bearer","refresh_token":"jGekUl1W1UIAAAAAAAAAAd1zHanzbZlMaSMlC8ogY79hyvV1mNUVbFOt6nPJnCUM","expiry":"2023-11-30T20:06:30.642061Z"}
<---End paste

On your Ultra.cc ssh session:

  1. Switch back to your Ultra.cc ssh session and paste the token.
Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.

y) Yes (default)
n) No
y/n> n

Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
        rclone authorize "dropbox"
Then paste the result.
Enter a value.
config_token> {"access_token":"sl.Bq2irrOkj8l6-ngKY6apc6Gkhbxb0tRnJECnNG1iw6PAF9s6TzPuDG4Baegb9da3yBeDCzP1D9ZtQ4_LYuDAJLmOjPuFyFPiw2lnlP_TpUS26oNMX_SZPGSrWHFJHHorvaYe46jVvFFBiaU","token_type":"bearer","refresh_token":"jGekUl1W1UIAAAAAAAAAAd1zHanzbZlMaSMlC8ogY79hyvV1mNUVbFOt6nPJnCUM","expiry":"2023-11-30T20:06:30.642061Z"}
  1. Finish the remaining configuration options.
rclone lsd remotename:
rclone copy yourlocalfile remotename:folder-path
rclone copy remotename:full-path-and-filename localpath

Creating a Mount (Part 3)

  1. Create a new directory that will be used as access your cloud files - this can be called anything you chose and doesn’t need to be the same as your rclone remote name you created above (step 2) but it can be the same if you find that easier.
mkdir DropBoxFiles
  1. Now we want to mount our Rclone remote to that directory:
rclone mount remotename: directoryname
rclone remote dropbox: DropBoxFiles
  1. This command will continue to run until you press CTRL+C. So how do you test? Open up another ssh session, leaving the rclone mount running.
ls -al directoryname
ls -al DropBoxFiles

You now have the knowledge to create a mount to any cloud storage provider that Rclone supports! Next, you should consider MergerFS.

MergerFS (Step 4)

While Ultra.cc services come pre-installed with MergerFS, it is better to download your own version so you can keep it up to date. See the Install MergerFS part of this guide.

MergerFS = Directory_A UNION Directory_B (UNION Directory_C...)

The following visualizes how the union works:

mergerfs.png

mergerfs ~/Directory_A:~/Directory_B ~/MergerFS
mergerfs ~/Directory_A:~/Directory_B:~/Directory_C:~/Directory_D ~/MergerFS
mergerfs ~/Stuff/Local:~/Stuff/Gdrive:~/Stuff/Dropbox ~/MergerFS

where:

[Unit]
Description = MergerFS Service
After=rclone-vfs.service
RequiresMountsFor=%h/Stuff/Local
RequiresMountsFor=%h/Stuff/Mount

[Service]
Type=forking
TimeoutStopSec=60
ExecStart=%h/bin/mergerfs \
    -o use_ino,func.getattr=newest,category.action=all \
    -o category.create=ff,cache.files=auto-full,threads=8 \
    %h/Stuff/Local:%h/Stuff/Mount %h/MergerFS

StandardOutput=file:%h/scripts/mergerfs_mount.log
ExecStop=/bin/fusermount -uz %h/MergerFS
Restart=on-failure

[Install]
WantedBy=default.target
mkdir -p ~/Stuff/Local  for your local files
mkdir -p ~/Stuff/Mount  for your rclone mount
mkdir ~/MergerFS        for your mergerfs unioned mount
mksir ~/scripts         for the log file & upload script
systemctl --user enable --now mergerfs.service
%h/Stuff/Local:%h/Stuff/Mount %h/MergerFS

Multiple Cloud Providers

Using the information you have learned here should mean you can create Rclone mounts for all your cloud providers and add them to your MergerFS mount. Just keep in mind:

  1. Each connection to a cloud provider needs its own Rclone remote (rclone config)
  2. Each cloud provider needs its own directory to display the files (rclone mount)
  3. Each cloud provider needs its own service file to perform the mount (nano ~/.config/systemd/user/rclone-mountname.service)
  4. Each cloud provider mount directory needs to be added to your MergerFS service file (nano ~/.config/systemd/user/mergerfs.service)
  5. Remember, any time you edit a service file, run systemctl --user daemon-reload or you will get an error telling you to do so.
  6. You need to restart your service files for them to take effect after your edits (systemctl —user restart servicefilename)

Troubleshooting

fusermount -uz path-to-mount
	ps ux | grep -i mergerfs         for MergerFS issues

	ps us | grep -i rclone           for Rclone issue 
username@servername:~$ ps ux | grep -i rclone
username   36384  0.1  0.0 1364856 65748 pts/140 Sl+  Sep25 105:16 rclone mount Gdrive: /home/username/Stuff/Mount --cache-db-purge --dir-cache-time 5000h -vvv --log-file /home/username/.config/rclone/rclone.log --poll-interval 15s --vfs-cache-mode write 
username  128007  0.0  0.0 218480   824  pts/77   S+   15:32   0:00 grep -i rclone
kill -9 36384

We hope you find this guide useful and that you now feel confident to experiment with your own set-up.


If you are having trouble with this, you can open a support ticket here!
Feedback and suggestions regarding this article are welcome in our Discord server

Installation, Configuration & Usage of rclone

Ultra.cc support is unable to provide support for rclone, this is due to the large volume of variables and different configurations possible with rclone. The guides found here on the knowledge-base should be able to guide you through using rclone, and any further questions can easily be answered with a quick Google search. You may also be able to find community support for rclone through our community Discord server or the Rclone forums.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders. Do not download directly to a rclone mount from a torrent or nzbget client. Both will create massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits.

Please do not mount to any of the default directories such as:
downloads
files
media
bin
.apps
.config
www
/homexx/username/
or any pre-created directory found on your Ultra.cc Slot

In this guide, we will be going over the installation of rclone. We'll also cover the basic usages such as setting up a remote and how to copy files between your slot and a remote file host.

Installation on your Ultra.cc Service

To begin, make sure you know how to SSH into your slot. All rclone commands are performed via SSH. You can find a guide on SSH here.

To install rclone, run the command below; this command will automatically install rclone to your slot for you.

rclone stable

curl https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/rclone-install-stable.sh | bash

rclone beta

curl https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/rclone-install-beta.sh | bash

Installation on your local PC

Windows

MacOS

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
brew install macfuse
brew install sudo
sudo su
curl https://rclone.org/install.sh | sudo bash 

Linux

curl https://rclone.org/install.sh | sudo bash

Configuration

OAuth (Google Drive)

Ultra_000161.png

Ultra_000160.png

rclone

Now we need to configure a remote to use with rclone. For this guide, we will be configuring Google Drive. This is the most common remote people tend to use as it offers ample storage capacities for a reasonable price. Should you wish to use a different cloud host, feel free to modify the steps, you take.

kbguides@lw914:~$ rclone config
2019/06/15 18:16:33 NOTICE: Config file "/home27/kbguides/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q>
Type of storage to configure.
Enter a string value—Press Enter for the default ("").
Choose a number from below, or type in your own value
...
14 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
15 / Google Drive
   \ "drive"
16 / Hubic
   \ "hubic"
...
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> example12345

Enter a string value. Press Enter for the default ("").
client_secret> example12345
Scope that rclone should use when requesting Access from Drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Full access to all files, excluding Application Data Folder.
   \ "drive"
 2 / Read-only access to file metadata and file contents.
   \ "Drive.readonly"
   / Access to files created by rclone only.
 3 | These are visible on the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ "Drive.file"
   / Allows read and write Access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ "Drive.appfolder"
   / Allows read-only Access to file metadata but
 5 | does not allow any access to read or download file content.
   \ "drive.metadata.readonly"
scope> 1
The ID of the root folder
Leave blank normally.
Fill in to access "Computers" folders. (see docs).
Enter a string value. Press Enter for the default ("").
root_folder_id>

Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Enter a string value. Press **Enter** for the default ("").
service_account_file>

Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n> n
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
        rclone authorize "drive" "eyJjbGllbnRfaWQiOiI5OTIyMjQwOTkzNTYtMWtsZmJrYnFjamE5YWtqdmJiZTZsNGgwdHFhc25pZXEuYXBw"
Then paste the result.
Enter a value.
config_token>
--------------------
[gdrive]
type = drive
client_id = 78311[REDACTED].apps.googleusercontent.com
client_secret = O34[REDACTED]
scope = drive
token = {"access_token":[REDACTED]}
team_drive = 
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> 

Usage

rclone is interacted with purely through SSH. Please ensure you are familiar with the Linux terminal and using SSH prior to using rclone.

General Commands

These commands are helpful to remember. They allow you to interact with rclone and move files around between your local and remote storage or even between two remote destinations.

config - Execute this command to add, modify or remove remote file hosts.
Usage: rclone config

copy - Used to copy files between two locations, remote -> remote, remote -> local, local -> remote
Usage: rclone copy [-P] {origin} {destination}

move - Same as copy however, it does not leave the files at the source
Usage: rclone move [-P] {origin} {destination}

sync - Will make the destination directory identical to the origin. If files exist on the destination that does not exist on origin, they will be deleted. Be careful with the sync command, as it can cause data loss.
Usage: rclone sync [-P] {origin} {destination}

When dealing with remote filesystems, use:
{remote}:{path}

For example, if you wished to copy a file named movie.mkv from your current working directory to a path named Movies in a remote name Drive you'd use this command:
rclone copy movie.mkv Drive:Movies

Rclone VFS and MergerFS Setup

This guide is for advanced users only. Ultra.cc is not responsible for any data loss or application errors due to this setup. We do not provide official support for Rclone or MergerFS. You may visit our #community-support channel on our Discord server for assistance.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders. Do not download directly to a Rclone mount from a torrent or nzbget client. Both will create massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits.

Please do not mount to any of the default directories such as:
downloads
files
media
bin
.apps
.config
www
/homexx/username/
or any pre-created directory found on your Ultra.cc Slot

Introduction

This section will teach you how to set up a Rclone VFS mount and MergerFS on Ultra.cc slots, and it assumes the following:

The workflow of the setup is as follows:

The Pros of this setup are as follows:

And, the cons of this setup are as follows:

Before we proceed, it is imperative to stop all Rclone/plexdrive processes and stop all the apps that are connected to your Rclone mount before proceeding.

Preparation

Install and Configure Rclone

Install and configure Rclone if you haven't already. Refer here for more information: Installation, Configuration & Usage of rclone

Install mergerfs

bash <(wget -qO- https://scripts.usbx.me/main/MergerFS-Rclone/Installer%20Scripts/mergerfs-install.sh)
ultradocs@pollux:~$ which mergerfs 
/home/ultradocs/bin/mergerfs

Change Application Settings

Change your application settings according to rclone Optimizations for Apps.

Setup the Work Flow using a Script

What does the script do?

Run the Script

rclone listremotes
bash <(wget -qO- https://scripts.usbx.me/main/MergerFS-Rclone/rclone-mergerfs.sh)
ls ~/MergerFS

Set Sonarr Correctly

Set Root Folder

Remote Path Mapping

Host: {username}.{servername}.usbx.me
Remote Path: /home/your_username/Stuff/Local/Downloads
Local Path: /home/your_username/MergerFS/Downloads

Set Radarr Correctly

Set Root Folder

Remote Path Mapping

Host: {username}.{servername}.usbx.me
Remote Path: /home/your_username/Stuff/Local/Downloads
Local Path: /home/your_username/MergerFS/Downloads

Connect Sonarr & Radarr to Plex / Emby

Plex Media Server

Name : Anything as per your preference.
Notification Triggers: Check `On Download`, `On Upgrade`, `On Rename`.
Host: 172.17.0.1
Port: 16825 (The port of your Plex Media Server, visible in the Control Panel)
Use SSL: Unchecked
Update Library: Checked

Emby

Name : Anything as per your preference.
Notification Triggers: Check `On Download`, `On Upgrade`, `On Rename`.
Host: 172.17.0.1
Port: Emby's port as given in your Ultra Control Panel.
Use SSL: Unchecked
API Key: Paste the one which was created earlier.
Update Library: Checked

Set your Download Clients Correctly

In both Sonarr & Radarr, you must click on the cog that says Show Advanced in Settings. Some required fields for the Download client settings will not be visible otherwise.

Torrent Clients

rTorrent
Deluge
qBittorrent
Default Torrent Management Mode: Automatic
When Torrent Category changed: Relocate torrent
Transmission

Usenet Clients

mkdir -p ~/Stuff/Local/Downloads/usenet/Sonarr
mkdir -p ~/Stuff/Local/Downloads/usenet/Radarr
NZBGet
SABnzbd

Set your Media Server Applications Correctly

Plex Media Server

Emby

JellyFin

Helpful Information for managing this new Work Flow

rclone-vfs mount: ~/scripts/rclone_vfs_mount.log
mergerfs mount: ~/scripts/mergerfs_mount.log
rclone-upload script: ~/scripts/rclone-upload.log
Restart Mounts: systemctl --user restart rclone-vfs.service mergerfs.service
Stop Mounts: systemctl --user stop rclone-vfs.service mergerfs.service
Disable mounts: systemctl --user disable --now rclone-vfs.service mergerfs.service

Understanding the rclone-upload Script

Normal rclone-upload.sh

#!/bin/bash

lock_file="$HOME/scripts/rclone-upload.lock"

trap 'rm -f "$lock_file"; exit 0' SIGINT SIGTERM
if [ -e "$lock_file" ]
then
    echo "Rclone upload script is already running."
    exit
else
    rm "$HOME"/scripts/rclone-upload.log
    touch "$lock_file"
    "$HOME"/bin/rclone move "$HOME"/Stuff/Local/ your_remote_name: \
        --config="$HOME"/.config/rclone/rclone.conf \
        --exclude "Downloads/**" \
        --drive-chunk-size 64M \
        --tpslimit 5 \
        -vvv \
        --drive-stop-on-upload-limit \
        --delete-empty-src-dirs \
        --bwlimit=8M \
        --use-mmap \
        --transfers=2 \
        --checkers=4 \
        --log-file "$HOME"/scripts/rclone-upload.log
    rm -f "$lock_file"
    trap - SIGINT SIGTERM
    exit
fi

rclone-upload.sh with Discord Notification

#!/bin/bash

# Rclone upload script with optional Discord notification upon move completion (if something is moved)
#
# Recommended for use via cron
# For example: */10 * * * * /path/to/rclone-upload.sh >/dev/null 2>&1
# -----------------------------------------------------------------------------

SOURCE_DIR="$HOME/Stuff/Local/"
DESTINATION_DIR="your_remote_name:"

DISCORD_WEBHOOK_URL=""
DISCORD_ICON_OVERRIDE="https://i.imgur.com/MZYwA1I.png"
DISCORD_NAME_OVERRIDE="RCLONE"

LOCK_FILE="$HOME/rclone-upload.lock"
LOG_FILE="$HOME/rclone-upload.log"

# DO NOT EDIT BELOW THIS LINE UNLESS YOU KNOW WHAT YOU'RE DOING
# -----------------------------------------------------------------------------

trap 'rm -f $LOCK_FILE; exit 0' SIGINT SIGTERM
if [ -e "$LOCK_FILE" ]
then
  echo "$0 is already running."
  exit
else
  touch "$LOCK_FILE"
  
  rclone_move() {
    rclone_command=$(
      "$HOME"/bin/rclone move -vP \
      --config="$HOME"/.config/rclone/rclone.conf \
      --exclude "Downloads/**" \
      --drive-chunk-size 64M \
      --use-mmap \
      --delete-empty-src-dirs \
      --log-file="$LOG_FILE" \
      --stats=9999m \
      --tpslimit=5 \
      --transfers=2 \
      --checkers=4 \
      --bwlimit=8M \
      --drive-stop-on-upload-limit \
      "$SOURCE_DIR" "$DESTINATION_DIR" 2>&1
    )
    # "--stats=9999m" mitigates early stats output 
    # "2>&1" ensures error output when running via command line
    echo "$rclone_command"
  }
  rclone_move

  if [ "$DISCORD_WEBHOOK_URL" != "" ]; then
  
    rclone_sani_command="$(echo $rclone_command | sed 's/\x1b\[[0-9;]*[a-zA-Z]//g')" # Remove all escape sequences

    # Notifications assume following rclone ouput: 
    # Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA - Errors: 0 Checks: 0 / 0, - Transferred: 0 / 0, - Elapsed time: 0.0s

    transferred_amount=${rclone_sani_command#*Transferred: }
    transferred_amount=${transferred_amount%% /*}
    
    send_notification() {
      output_transferred_main=${rclone_sani_command#*Transferred: }
      output_transferred_main=${output_transferred_main% Errors*}
      output_errors=${rclone_sani_command#*Errors: }
      output_errors=${output_errors% Checks*}
      output_checks=${rclone_sani_command#*Checks: }
      output_checks=${output_checks% Transferred*}
      output_transferred=${rclone_sani_command##*Transferred: }
      output_transferred=${output_transferred% Elapsed*}
      output_elapsed=${rclone_sani_command##*Elapsed time: }
      
      notification_data='{
        "username": "'"$DISCORD_NAME_OVERRIDE"'",
        "avatar_url": "'"$DISCORD_ICON_OVERRIDE"'",
        "content": null,
        "embeds": [
          {
            "title": "Rclone Upload Task: Success!",
            "color": 4094126,
            "fields": [
              {
                "name": "Transferred",
                "value": "'"$output_transferred_main"'"
              },
              {
                "name": "Errors",
                "value": "'"$output_errors"'"
              },
              {
                "name": "Checks",
                "value": "'"$output_checks"'"
              },
              {
                "name": "Transferred",
                "value": "'"$output_transferred"'"
              },
              {
                "name": "Elapsed time",
                "value": "'"$output_elapsed"'"
              }
            ],
            "thumbnail": {
              "url": null
            }
          }
        ]
      }'
      
      /usr/bin/curl -H "Content-Type: application/json" -d "$notification_data" $DISCORD_WEBHOOK_URL 
    }
    
    if [ "$transferred_amount" != "0 B" ]; then
      send_notification
    fi

  fi

  rm -f "$LOCK_FILE"
  trap - SIGINT SIGTERM
  exit
fi

Changing the time at which rclone-upload.sh is executed (Optional)

Please do not execute the upload script more than once a day. It can lead to the violation of our Terms of Service.

Select an editor.  To change later, run 'select-editor'.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.basic
  3. /usr/bin/mcedit
  4. /usr/bin/vim.tiny

Choose 1-4 [1]: 
0 19 * * * /home/your_username/scripts/rclone-upload.sh > /dev/null 2>&1

Rclone Mount using Systemd

This guide is for advanced users only and it serves as a guide for you to use rclone. The systemd files here are the recommended settings for our slots and will subject to change whenever there are new configurations that are appropriate for the slots. Furthermore, Ultra.cc is not responsible for any data loss or application errors due to this setup should you proceed and will not provide official support for it due to the large volume of variables and different configurations possible with rclone. You may visit the community discord server for help.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders. Do not download directly to a rclone mount from a torrent or nzbget client. Both will create massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits.

Please do not mount to any of the default directories such as:
downloads
files
media
bin
.apps
.config
www
/homexx/username/
or any pre-created directory found on your Ultra.cc Slot

Is this method of rclone mounting suitable for your use case?

This method is not suitable for you if you wish to upload/write data to your rclone mount.

Overview

Rclone's Mount allows you to mount any of your cloud storage accounts as part of your slot's file system using FUSE. In this guide, we will teach you how to run a rclone mount using systemd. Take note that this guide is setup using Google Drive as the cloud storage provider used. Should you use any other cloud storage providers, you may need consult rclone documentation for the appropriate flags for your setup.

There are many ways to mount rclone. You can run the rclone Mount using the screen utility, create a script for running rclone Mount, and checking if the command is still alive, using the --daemon flag, just to mention a few.

We recommend using systemd for several reasons:

rclone mount with no mergerfs

If you don't need to upload files to your mount, follow this guide.

Should you need to upload files or you're planning an automated setup involving your cloud storage, we recommend using Rclone VFS and MergerFS Setup instead.

Preparation

kbguides@lw902:~$ mkdir -p ~/Stuff/Mount && mkdir ~/.tmp-rclone
kbguides@lw902:~$
kbguides@lw902:~$ mkdir ~/scripts
kbguides@lw902:~$
$ @lw902:~$ rclone listremotes
Gsuite:

Downloading Rclone Service File

Rclone Mount for Most Remotes
wget -P ~/.config/systemd/user/ https://scripts.usbx.me/main/MergerFS-Rclone/Service%20Files/rclone-normal.service && nano ~/.config/systemd/user/rclone-normal.service
Google Drive Rclone Mount for Media Streaming
wget -P ~/.config/systemd/user/ https://scripts.usbx.me/main/MergerFS-Rclone/Service%20Files/rclone-vfs.service && nano ~/.config/systemd/user/rclone-vfs.service

Editing your service file

Example rclone-normal.service

Rclone.png

Example rclone-vfs.service

Annotation-on-2021-08-11-at-20-19-06.png

Systemd Commands

{mount-name} refers to the service of your choice. Either rclone-normal.service or rclone-vfs.service

Enabling and starting Rclone mount: systemctl --user enable --now {mount-name}.service
Restart Rclone Mount: systemctl --user restart {mount-name}.service
Stop Rclone Mount: systemctl --user stop {mount-name}.service
Stop and disable Rclone mount: systemctl --user disable --now {mount-name}.service (Remove service file after)

Rclone Optimizations for Apps

This guide lists optimizations when using rclone mount to host your files to your slot. This is to lessen the API request hits made by rclone to your cloud streaming service and to not hit your cloud streaming's download quotas.

Media Centers

Plex

Settings ⇾ Library
Settings ⇾ Scheduled Tasks

Emby/Jellyfin

Settings ⇾ Advanced ⇾ Scheduled Tasks

Downloaders

Sonarr/Radarr

Settings ⇾ Media Management ⇾ File Management

Rclone Mount on Apple Silicon

Creating an Rclone Mount on Apple Silicon (M1 Chips)

These steps may work on intel based machines but are untested and M1 Chips require extra steps due to the unique architecture.

Installing Homebrew installer

Homebrew installs the stuff you need that Apple (or your Linux system) didn’t. If your familiar with linux you will find a few normal functions are missing or work differently on MacOS “brew” helps bridge this gap.

You can find out more here: https://brew.sh/

Installing Brew is very simple simply run the command bellow and follow the prompts. /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)

Installing MacFuse and Sudo

brew install macfuse brew install sudo

Install rclone under sudo

Installing Rclone is easy but once again Apple will have you jump a few hoops to do so, First you must switch to a root user to allow for rclone to install. For some reason running the command directly as your user even with sudo installed will fail due to permissions.

To switch to root simply run sudo su Then enter your Mac Password You will find your prompt has changed on the left and removed your username You can now install rclone: curl https://rclone.org/install.sh | sudo bash

Once installed successfully you can now exit the root user by typing exit then pressing enter

Now that your back into your main user you need to create a mount folder change username to match your mac username

mkdir /Users/username/Documents/Rclone

Creating a Rclone remote to connect to your Ultra slot

Configuring our Rclone FTP remote The first step is to invoke the rclone config menu with the rclone config command This will present a list of options

e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q>

Field Value
Host Your Ultra Address (servername.usbx.me) or IP (185.123.123.123)
User Your Ultra Username
Port Port 21 is standard. You can press Enter
FTP password Press Y, then enter your FTP Password. You will be asked for this twice.
Use FTPS over TLS (Implicit) False
Use FTP over TLS (Explicit) True
Edit advanced config? (y/n) Press Y
Maximum number of FTP simultaneous connections, 0 for unlimited Press enter
no_check_certificate True
Disable using EPSV even if server advertises support Press Enter
Enter a encoder.MultiEncoder value. Press Enter for the default ("Slash,Del,Ctl,RightSpace,Dot"). Press Enter
y) Yes this is OK (default) Press Y to complete the rclone configuration

Configuring Mac Permissions for FUSE

Upon your first attempt to mount your new remote you will be greeted by a rather angry looking denied message as well as pop up explaining that you will need to change your security settings in order to allow macfuse to run.

To change the security setting on your MacBook with the M1 chip, you need to go into macOS Recovery mode.

  1. Turn on your MacBook holding the power button. Click "Security Policy..." on Startup Security Utility". If you do not see the Startup disk, follow the steps below.
  2. Click "Continue of Options" and "Select a user you know the password for : " will appear. Select your account and click Next. Then enter your password. Select "Utilities" on the top left and select "Startup Security Utility". Click "Unlock", enter your password and click "Security Policy...".
  3. On the screen below, choose "Reduced Security", check "Allow user management of kernel extensions from identified developers" and click "OK".
  4. Reboot and visit System Preferences > Security and Privacy > General. Click the lock icon at the bottom, enter your password and click Allow in front of the message that says System Extension by Benjamin Fleischer was blocked.

Test the mount

Install screen using brew as it will allow you to keep the mount running without leaving a command window open. brew install screen Now enter a new screen screen -S mounttest And attempt to mount rclone cmount Gsuite: /Users/jackalblood/Documents/Rclone If all is well it will appear as if nothing has happened this is fine. Press Command + A then D to detach leaving the mount running Since we are in a command window already we can go ahead and use this to verify it is functional, to do so simply run :

cd /Users/username/Documents/Rclone substitute username with your mac username Then run the command ls you should now see all of the folders you have on your ultra slot, They will also be visible in a folder now called ultra in your documents folder (Mac OS changes the name based on the remote mounted

Using Rclone FTPS

Ultra.cc support is unable to provide support for rclone, this is due to the large volume of variables and different configurations possible with rclone. The guides found here on the knowledge-base should be able to guide you through using rclone, and any further questions can easily be answered with a quick Google search. You may also be able to find community support for rclone through our community Discord server or the Rclone forums.

Rclone is a software primarily used with cloud storage providers; however, it is also an easy-to-use Command-line download manager for many protocols, including FTP/FTPS/SFTP. We will show you today how to install rclone onto a Linux machine on your local network and how to set up automatic downloading from your Ultra.cc Slot.

Requirements:

Host machine on your home/local network running a Linux distribution (Mac OS can also be used but is not in the scope of this guide; however, it may still work as a general guide)

Ultra.cc FTP Slot details

Installing Rclone on the Host Machine

Installing rclone is made incredibly easy by the developer. First login to your Host via SSH; this can be under a sudo user, but it is recommended to install rclone as root user and later set up rclone under a sudo/sub-user, never set up under root.

Once logged in, run this curl command

curl https://rclone.org/install.sh | sudo bash

If you receive a message telling you that the “curl command cannot be found,” you will need to install it and run the command above again. To install curl, run apt-get install curl -y

Once rclone is installed, it is recommended you switch to a new user and configure under that user. If you have not created a new user, the steps are simple.

adduser <username> and make sure to replace <username> with a username of your choice.

Follow the on-screen prompts for user information and password. Once that is completed, it is an excellent idea to promote this user to sudo; this allows for the execution of higher privileged commands, which are not required for Rclone but are certainly worth setting up.

To add a user to sudo after creation, run this command.

adduser <username> sudo

If, for some odd reason, your distro did not have sudo installed, now might be a good time to run an install just in case:

apt-get install sudo -y

Now we can switch to our <username> user

su <username>

Type cd to jump to the user home folder. This isn’t strictly required but is best practice.

Configuring our Rclone FTP remote

The first step is to invoke the rclone config menu with the rclone config command

This will present a list of options


e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q\>

Press N then enter

Now enter a remote name: you will need this later when writing our automated script.

Now you will be presented with an extremely long list of options at the time of writing this 16 is FTP, but this may change as updates release

Field Value
Host Your Ultra Address (servername.usbx.me) or IP (185.123.123.123)
User Your Ultra Username
Port Port 21 is standard. You can press Enter
FTP password Press Y, then enter your FTP Password. You will be asked for this twice.
Use FTPS over TLS (Implicit) False
Use FTP over TLS (Explicit) True
Edit advanced config? (y/n) Press Y
Maximum number of FTP simultaneous connections, 0 for unlimited Press enter
no_check_certificate True
Disable using EPSV even if server advertises support Press Enter
Enter a encoder.MultiEncoder value. Press Enter for the default ("Slash,Del,Ctl,RightSpace,Dot"). Press Enter
y) Yes this is OK (default) Press Y to complete the rclone configuration

Testing Remote for access

Once the configuration of the remote is completed, it is a relatively simple job to check if all is configured correctly

One command should list all of the folders in your Ultra Slot Home folder.

rclone lsd remote-name: Replace remote-name with the remote name you chose previously.

Automating download from Ultra.cc

Our first step is to write a simple script. We will set up a lock folder for a file lock system so the script will not run if it is already running

Create the lock folder with mkdir -p ~/lock

If you ever get a flock failed command, delete any files in ~/lock with the command rm -rdf ~/lock/*

And create the script file

nano FTP.sh

Paste the following lines into it :

#!/bin/bash

exec {lock_fd}>/home/<username>/lock/FTPLOCK || exit 1

flock -n "$lock_fd" || { echo "ERROR: flock() failed." >&2; exit 1; }

if [ -z "$STY" ]; then exec screen -dm -S FTP /bin/bash "$0"; fi

rclone copy -v remote-name:/ultraslotfoldertocopy /home/<username>/pathtosavefiles

flock -u "$lock_fd"

Save it by pressing Ctrl+X then Y Enter.

You will need to change the paths ultraslotfoldertocopy to match the folder you wish to download. You will also be required to alter remote-name to match your rclone remote name that you chose previously. It can be found with the rclone config command or rclone listremotes. These need to be run via the SSH Command line.

Now we need to allow it to run with the command chmod +x FTP.sh

And finally, run it

./FTP.sh

If the script is running and you were to rerun it, you may see an error message “Flock Failed”, this is a file lock to stop multiple downloads running and is normal. If you are sure it isn’t running you can delete the lock file inside ~/lock. You can also check the progress of the backup script, which is running in a screen with the command screen -rd FTP

If all is well after the test, we can automate the check via crontab

Open crontab with

crontab -e

You may have a choice of editors. We recommend Nano

Inside the crontab add a single line under everything else in the file that looks like this

* * * * * /home/ultra/FTP.sh

Save it by pressing Ctrl+X then Y then Enter.

The script will now run every 1 minute, checking for files that have changed and syncing them to the destination folder

Troubleshooting Information

Can you help me with rclone?

Rclone is a powerful tool that allows you to create virtual drives or mount points. The following help in this section will assume that you have installed your rclone with the help of our documentation and guidelines.

While it is an excellent tool, it is not very reliable. It should never be used to write data directly to your cloud services with our applications. Use mergerfs for that purpose. It is also essential that those who utilize them will know how to troubleshoot them when needed.

Please do not mount to any of the default directories such as:

This will lead to instabilities within your slot.

Please click here to go to our Rclone setup guide.

Due to these variables involved, official support for rclone or applications utilizing these applications will not be provided.

Job for rclone-vfs.service failed because the control process exited with error code. See "systemctl --user status rclone-vfs.service" and "journalctl --user -xe" for details.

This may be due to the following reasons:

ExecStart=/homexx/yyyyy/bin/rclone mount **gdrive**: /homexx/yyyyy/Stuff/Mount \

Following the advice above, you should find the issue, and when you have, do not forget to execute systemctl --user daemon-reload before trying to enable it again.

ls: cannot access 'SomeDirectory': Transport endpoint is not connected

This means the rclone process/mount is dead, but the mount is still fused. If you’re using systemd following our guide, please stop your mount using systemctl:

systemctl --user stop rclone-vfs
systemctl --user stop mergerfs
mount | grep $USER
fusermount -zu /path/to/SomeDirectory
killall rclone

Now follow the process to restart your mount below.

How do I restart my rclone cloud or mergerfs mount?

Execute the following in your shell terminal (SSH)

  1. Stop all currently running applications that utilize the mount. This applies to Plex, Radarr, Sonarr, etc. (can be done in UCP or ssh using app-[name] stop).
  2. SSH into your slot.
  3. Run systemctl --user stop rclone-vfs and systemctl --user stop mergerfs if using mergerfs
  4. Run fusermount -uq /path/to/mount and fusermount -uq /path/to/mergerfs if using mergerfs
  5. Run systemctl --user start rclone-vfs and systemctl --user start mergerfs if running mergerfs
  6. Verify that the mount is working by going into your mount folder and running ls -la to verify we are connected. If using mergerfs, check the mergerfs folder too.
  7. Start all applications from the UCP or ssh using app-[name] start. (replace [name] with app name, app-plex start.)

I use rclone or a cloud mount, and my xx application is not working or is slow. Why is that?

As stated, you should not use rclone or cloud mount directly with applications that will attempt to write directly to it. It would be best if you used MergerFS for that. Please check our resource for MergerFS here.

My mount is failing. Why?

There may be many things that can cause rclone/mounting to not work correctly, something as simple as a mistype; if you find yourself stuck, it is possible to add debug logging to your rclone systemd file. The steps to do so will be listed below:

  1. First kill the mount if mounted
systemctl --user stop rclone-vfs
systemctl --user stop mergerfs
mount | grep $USER
fusermount -zu /path/to/SomeDirectory
killall rclone
  1. Add debug flag to mounting systemd file

Example

[Unit]
Description=RClone VFS Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
KillMode=none
Environment=GOMAXPROCS=2

ExecStart=%h/bin/rclone mount remote: %h/Mount \
  --config %h/.config/rclone/rclone.conf \
  --use-mmap \
  --dir-cache-time 1000h \
  --poll-interval=15s \
  --vfs-cache-mode writes \
  --log-level DEBUG --log-file "%h/scripts/rclone-debug.log" \
  --tpslimit 10

StandardOutput=file:%h/scripts/rclone_vfs_mount.log
ExecStop=/bin/fusermount -uz %h/Mount
Restart=on-failure

[Install]
WantedBy=default.target

This edit will work for each rclone systemd file, please be sure to edit your log output path accordingly to match your home and user paths.

  1. Reload Systemd file with the new flag

systemctl --user daemon-reload

systemctl --user start rclone-vfs

systemctl --user start mergerfs if running mergerfs

  1. Reading the debug logs

watch tail -n 15 ~/scripts/rclone-debug.log

This will show you the output of the log and refresh its output every 15 seconds. You can change the -n 15 value if you wish to make it more frequent.