Rclone VFS and MergerFS Setup

This guide is for advanced users only. Ultra.cc is not responsible for any data loss or application errors due to this setup. We do not provide official support for Rclone or MergerFS. You may visit our #community-support channel on our Discord server for assistance.

Please make yourself aware of the Ultra.cc Fair Usage Policy. It is very important not to mount your Cloud storage to any of the premade folders. Do not download directly to a Rclone mount from a torrent or nzbget client. Both will create massive instability for both you and everyone else on your server. Always follow the documentation and create a new folder for mounting. It is your responsibility to ensure usage is within acceptable limits.

Please do not mount to any of the default directories such as:
downloads
files
media
bin
.apps
.config
www
/homexx/username/
or any pre-created directory found on your Ultra.cc Slot

Introduction

This section will teach you how to set up a Rclone VFS mount and MergerFS on Ultra.cc slots, and it assumes the following:

The workflow of the setup is as follows:

The Pros of this setup are as follows:

And, the cons of this setup are as follows:

Before we proceed, it is imperative to stop all Rclone/plexdrive processes and stop all the apps that are connected to your Rclone mount before proceeding.

Preparation

Install and Configure Rclone

Install and configure Rclone if you haven't already. Refer here for more information: Installation, Configuration & Usage of rclone

Install mergerfs

bash <(wget -qO- https://scripts.ultra.cc/main/MergerFS-Rclone/Installer%20Scripts/mergerfs-install.sh)

ultradocs@pollux:~$ which mergerfs 
/home/ultradocs/bin/mergerfs

Change Application Settings

Change your application settings according to rclone Optimizations for Apps.

Setup the Work Flow using a Script

What does the script do?

Run the Script

rclone listremotes

bash <(wget -qO- https://scripts.ultra.cc/main/MergerFS-Rclone/rclone-mergerfs.sh)

ls ~/MergerFS

Set Sonarr Correctly

Set Root Folder

Remote Path Mapping

Host: {username}.{servername}.usbx.me
Remote Path: /home/your_username/Stuff/Local/Downloads
Local Path: /home/your_username/MergerFS/Downloads

Set Radarr Correctly

Set Root Folder

Remote Path Mapping

Host: {username}.{servername}.usbx.me
Remote Path: /home/your_username/Stuff/Local/Downloads
Local Path: /home/your_username/MergerFS/Downloads

Connect Sonarr & Radarr to Plex / Emby

Plex Media Server

Name : Anything as per your preference.
Notification Triggers: Check `On Download`, `On Upgrade`, `On Rename`.
Host: 172.17.0.1
Port: 16825 (The port of your Plex Media Server, visible in the Control Panel)
Use SSL: Unchecked
Update Library: Checked

Emby

Name : Anything as per your preference.
Notification Triggers: Check `On Download`, `On Upgrade`, `On Rename`.
Host: 172.17.0.1
Port: Emby's port as given in your Ultra Control Panel.
Use SSL: Unchecked
API Key: Paste the one which was created earlier.
Update Library: Checked

Set your Download Clients Correctly

In both Sonarr & Radarr, you must click on the cog that says Show Advanced in Settings. Some required fields for the Download client settings will not be visible otherwise.

Torrent Clients

rTorrent
Deluge
qBittorrent
Default Torrent Management Mode: Automatic
When Torrent Category changed: Relocate torrent
Transmission

Usenet Clients

mkdir -p ~/Stuff/Local/Downloads/usenet/Sonarr
mkdir -p ~/Stuff/Local/Downloads/usenet/Radarr
NZBGet
SABnzbd

Set your Media Server Applications Correctly

Plex Media Server

Emby

JellyFin

Helpful Information for managing this new Work Flow

rclone-vfs mount: ~/scripts/rclone_vfs_mount.log
mergerfs mount: ~/scripts/mergerfs_mount.log
rclone-upload script: ~/scripts/rclone-upload.log

Restart Mounts: systemctl --user restart rclone-vfs.service mergerfs.service
Stop Mounts: systemctl --user stop rclone-vfs.service mergerfs.service
Disable mounts: systemctl --user disable --now rclone-vfs.service mergerfs.service

Understanding the rclone-upload Script

Normal rclone-upload.sh

#!/bin/bash

lock_file="$HOME/scripts/rclone-upload.lock"

trap 'rm -f "$lock_file"; exit 0' SIGINT SIGTERM
if [ -e "$lock_file" ]
then
    echo "Rclone upload script is already running."
    exit
else
    rm "$HOME"/scripts/rclone-upload.log
    touch "$lock_file"
    "$HOME"/bin/rclone move "$HOME"/Stuff/Local/ your_remote_name: \
        --config="$HOME"/.config/rclone/rclone.conf \
        --exclude "Downloads/**" \
        --drive-chunk-size 64M \
        --tpslimit 5 \
        -vvv \
        --drive-stop-on-upload-limit \
        --delete-empty-src-dirs \
        --bwlimit=8M \
        --use-mmap \
        --transfers=2 \
        --checkers=4 \
        --log-file "$HOME"/scripts/rclone-upload.log
    rm -f "$lock_file"
    trap - SIGINT SIGTERM
    exit
fi

rclone-upload.sh with Discord Notification

#!/bin/bash

# Rclone upload script with optional Discord notification upon move completion (if something is moved)
#
# Recommended for use via cron
# For example: */10 * * * * /path/to/rclone-upload.sh >/dev/null 2>&1
# -----------------------------------------------------------------------------

SOURCE_DIR="$HOME/Stuff/Local/"
DESTINATION_DIR="your_remote_name:"

DISCORD_WEBHOOK_URL=""
DISCORD_ICON_OVERRIDE="https://i.imgur.com/MZYwA1I.png"
DISCORD_NAME_OVERRIDE="RCLONE"

LOCK_FILE="$HOME/rclone-upload.lock"
LOG_FILE="$HOME/rclone-upload.log"

# DO NOT EDIT BELOW THIS LINE UNLESS YOU KNOW WHAT YOU'RE DOING
# -----------------------------------------------------------------------------

trap 'rm -f $LOCK_FILE; exit 0' SIGINT SIGTERM
if [ -e "$LOCK_FILE" ]
then
  echo "$0 is already running."
  exit
else
  touch "$LOCK_FILE"
  
  rclone_move() {
    rclone_command=$(
      "$HOME"/bin/rclone move -vP \
      --config="$HOME"/.config/rclone/rclone.conf \
      --exclude "Downloads/**" \
      --drive-chunk-size 64M \
      --use-mmap \
      --delete-empty-src-dirs \
      --log-file="$LOG_FILE" \
      --stats=9999m \
      --tpslimit=5 \
      --transfers=2 \
      --checkers=4 \
      --bwlimit=8M \
      --drive-stop-on-upload-limit \
      "$SOURCE_DIR" "$DESTINATION_DIR" 2>&1
    )
    # "--stats=9999m" mitigates early stats output 
    # "2>&1" ensures error output when running via command line
    echo "$rclone_command"
  }
  rclone_move

  if [ "$DISCORD_WEBHOOK_URL" != "" ]; then
  
    rclone_sani_command="$(echo $rclone_command | sed 's/\x1b\[[0-9;]*[a-zA-Z]//g')" # Remove all escape sequences

    # Notifications assume following rclone ouput: 
    # Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA - Errors: 0 Checks: 0 / 0, - Transferred: 0 / 0, - Elapsed time: 0.0s

    transferred_amount=${rclone_sani_command#*Transferred: }
    transferred_amount=${transferred_amount%% /*}
    
    send_notification() {
      output_transferred_main=${rclone_sani_command#*Transferred: }
      output_transferred_main=${output_transferred_main% Errors*}
      output_errors=${rclone_sani_command#*Errors: }
      output_errors=${output_errors% Checks*}
      output_checks=${rclone_sani_command#*Checks: }
      output_checks=${output_checks% Transferred*}
      output_transferred=${rclone_sani_command##*Transferred: }
      output_transferred=${output_transferred% Elapsed*}
      output_elapsed=${rclone_sani_command##*Elapsed time: }
      
      notification_data='{
        "username": "'"$DISCORD_NAME_OVERRIDE"'",
        "avatar_url": "'"$DISCORD_ICON_OVERRIDE"'",
        "content": null,
        "embeds": [
          {
            "title": "Rclone Upload Task: Success!",
            "color": 4094126,
            "fields": [
              {
                "name": "Transferred",
                "value": "'"$output_transferred_main"'"
              },
              {
                "name": "Errors",
                "value": "'"$output_errors"'"
              },
              {
                "name": "Checks",
                "value": "'"$output_checks"'"
              },
              {
                "name": "Transferred",
                "value": "'"$output_transferred"'"
              },
              {
                "name": "Elapsed time",
                "value": "'"$output_elapsed"'"
              }
            ],
            "thumbnail": {
              "url": null
            }
          }
        ]
      }'
      
      /usr/bin/curl -H "Content-Type: application/json" -d "$notification_data" $DISCORD_WEBHOOK_URL 
    }
    
    if [ "$transferred_amount" != "0 B" ]; then
      send_notification
    fi

  fi

  rm -f "$LOCK_FILE"
  trap - SIGINT SIGTERM
  exit
fi

Changing the time at which rclone-upload.sh is executed (Optional)

Please do not execute the upload script more than once a day. It can lead to the violation of our Terms of Service.

Select an editor.  To change later, run 'select-editor'.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.basic
  3. /usr/bin/mcedit
  4. /usr/bin/vim.tiny

Choose 1-4 [1]: 

0 19 * * * /home/your_username/scripts/rclone-upload.sh > /dev/null 2>&1
Revision #49
Created 8 May 2022 12:55:54 by Raikiri
Updated 7 August 2024 07:42:35 by varg