Rclone Installation And Setup

Introduction

Rclone is a command line program to manage files on cloud storage. I used rclone to upload the files my telegram bot receives to Google Drive. I am going to cover only installing and setting up for Google Drive. I also will not cover how to run as a service as I only use Rclone to sync specific files that my Python program generates. I have done this multiple times. If you follow my instructions, you shouldn’t have any problems, but, as we all know, that’s a joke! I tested this all on April 19, 2023 on Ubuntu 22 in Oracle VM. This is an excellent way to try things out because the VMs can so easily be deleted and rebuilt.

Installation

Installation is very simple. First update your machine using the following.

sudo apt update

Then install curl if it isn’t already installed.

sudo apt-get install curl

For Linux/macOS/BSD systems installation, simply run the following code. If you want alternative installation methods, go here.

sudo -v ; curl https://rclone.org/install.sh | sudo bash

Setup On Linux (Ubuntu)

There are a lot of different cloud storages you can use this on. It’s fairly self-explanatory. I will only be covering Google Drive. I think it’s one of the hardest ones to setup due to the API. Now there are also two options for this. You can use Rclone’s client ID, or you can create your own. Rclone recommends using your own. The reason for not using the Rclone client ID, is because it is shared between all Rclone users who don’t have their own. Therefore, it could be slightly slower not to mention you risk running into the problem of reaching the quota. I have covered how to create your own as I found it quite difficult. If you plan to use your own, don’t go any farther until you have created it. Here’s my tutorial.

After Rclone has installed, run,

rclone config

Then type n followed by enter. You will need to then choose a name for the remote. I have chosen GDrive. It doesn’t really matter what you name it but you will need the name later. Next, you will need to choose your storage type. We are going with Google Drive, which in my case is number 18. After you have selected that you will come to the point where you need to decide whether or not you are going to create you own client ID. If you don’t plan to have high usage, just hit enter and skip this step. If you plan to use your own, copy your Client ID in, hit enter, then enter your client secret and enter again. The next step is to choose your scope or how much access you would like to give Rclone. You can give it full read write access, full read-only access, access to all files created by rclone, read write access to Application Data folder, or access to only metadata. I have chosen full access. You are welcome to choose whatever you need. Next you will need to login. It will ask you for service account credentials json file path. Just hit enter to skip. Hit n for no when asked if you want to edit advanced config. Next you will have to do the actual authentication. If you are on a remote machine trying to set this up, it is slightly more complicated.

Here is the exact shell output and input that I got when setting mine up.

digital-coffee-cup@digital-coffee-cup:~$ rclone config
2023/04/20 22:14:49 NOTICE: Config file "/home/digital-coffee-cup/.config/rclone/rclone.conf" not found - using defaults
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
Enter name for new remote.
name> GDrive
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
 1 / 1Fichier
   \ (fichier)
 2 / Akamai NetStorage
   \ (netstorage)
 3 / Alias for an existing remote
   \ (alias)
 4 / Amazon Drive
   \ (amazon cloud drive)
 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, DigitalOcean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, IONOS Cloud, Liara, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS, Qiniu and Wasabi
   \ (s3)
 6 / Backblaze B2
   \ (b2)
 7 / Better checksums for other remotes
   \ (hasher)
 8 / Box
   \ (box)
 9 / Cache a remote
   \ (cache)
10 / Citrix Sharefile
   \ (sharefile)
11 / Combine several remotes into one
   \ (combine)
12 / Compress a remote
   \ (compress)
13 / Dropbox
   \ (dropbox)
14 / Encrypt/Decrypt a remote
   \ (crypt)
15 / Enterprise File Fabric
   \ (filefabric)
16 / FTP
   \ (ftp)
17 / Google Cloud Storage (this is not Google Drive)
   \ (google cloud storage)
18 / Google Drive
   \ (drive)
19 / Google Photos
   \ (google photos)
20 / HTTP
   \ (http)
21 / Hadoop distributed file system
   \ (hdfs)
22 / HiDrive
   \ (hidrive)
23 / In memory object storage system.
   \ (memory)
24 / Internet Archive
   \ (internetarchive)
25 / Jottacloud
   \ (jottacloud)
26 / Koofr, Digi Storage and other Koofr-compatible storage providers
   \ (koofr)
27 / Local Disk
   \ (local)
28 / Mail.ru Cloud
   \ (mailru)
29 / Mega
   \ (mega)
30 / Microsoft Azure Blob Storage
   \ (azureblob)
31 / Microsoft OneDrive
   \ (onedrive)
32 / OpenDrive
   \ (opendrive)
33 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ (swift)
34 / Oracle Cloud Infrastructure Object Storage
   \ (oracleobjectstorage)
35 / Pcloud
   \ (pcloud)
36 / Put.io
   \ (putio)
37 / QingCloud Object Storage
   \ (qingstor)
38 / SMB / CIFS
   \ (smb)
39 / SSH/SFTP
   \ (sftp)
40 / Sia Decentralized Cloud
   \ (sia)
41 / Storj Decentralized Cloud Storage
   \ (storj)
42 / Sugarsync
   \ (sugarsync)
43 / Transparently chunk/split large files
   \ (chunker)
44 / Union merges the contents of several upstream fs
   \ (union)
45 / Uptobox
   \ (uptobox)
46 / WebDAV
   \ (webdav)
47 / Yandex Disk
   \ (yandex)
48 / Zoho
   \ (zoho)
49 / premiumize.me
   \ (premiumizeme)
50 / seafile
   \ (seafile)
Storage> 18
Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a value. Press Enter to leave empty.
client_id> 491250267549-op2406uecf6laockgosgmmq5520ac1so.apps.googleusercontent.com
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret> 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
Option scope.
Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / Full access all files, excluding Application Data Folder.
   \ (drive)
 2 / Read-only access to file metadata and file contents.
   \ (drive.readonly)
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ (drive.file)
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ (drive.appfolder)
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ (drive.metadata.readonly)
scope> 1
Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a value. Press Enter to leave empty.
service_account_file>
Edit advanced config?
y) Yes
n) No (default)
y/n> n
Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes (default)
n) No
y/n>

Now at this point I’m gonna split off to Remote Instructions, and Local Instructions.

Remote Machine Authentication Instructions

If you are ssh’ed into a remote machine, at this point you will type n followed with enter. You will be given something like

rclone authorize "drive" "eyJjbGllbnRfaWQiOiI0OTEyNTAyNjc1NDktb3AyNDA2dWVjZjZsYW9ja2dvc2dtbXE1NTIwYWMxc28uYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJjbGllbnRfc2VjcmV0IjoiR09DU1BYLS1ORkJfUmJLemM0NlVQWkJGbmM3SlZSSi0wZUoiLCJzY29wZSI6ImRyaXZlIn0"

You will need Rclone installed on another computer that has a web browser. Nothing needs to be set up just simply install it on a different computer. Enter what you are given into a terminal window of your local machine. It will have you open a web browser, sign in, and authorize Rclone. When you are done, your terminal will give you a token to paste back into your remote machine. Continue with the regular instructions from this point.

Local Machine Authentication Instructions

This is fairly straightforward. Type y followed by enter. You will be given a link to visit in your browser. Google will make it seem like someone is trying to hack you and steal your entire life but as long as you trust Rclone (which I do) just okay everything and you are good to go.

Continuing on…

Now no matter which previous step you used, it will all be the same from here. Finishing up is quite simple. You will be asked if you want to configure it as a Shared Drive. Type n and hit enter. You will then be given some information regarding your configuration. You will be asked if you want to keep this “GDrive” remote. I keep mine remote as I am only using it to sync specific files with command line. You are welcome to do what you want. At this point we are done. You can close config with q and enter.

Here is the rest of my output.

Configure this as a Shared Drive (Team Drive)?

y) Yes
n) No (default)
y/n> n

Configuration complete.
Options:
- type: drive
- client_id: 491250267549-op2406uecf6laockgosgmmq5520ac1so.apps.googleusercontent.com
- client_secret: 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
- scope: drive
- token: {"access_token":"ya29.a0Ael9sCPpXYmwNpEI-1z0mpLrxnl0ypiicHSGv6yr6jDf-_g9CfvczQ1UPS6vWLKoqDM0T0UZM5inHtbwhiWkTcwThzIhcUsv2_Iz8Wevr4dCp06oADzrjpqN42P-0_uzwlvlsGwreiG68VdGE-T6VRxDcxsOaCgYKAVgSARESFQF4udJh1WL3zPkFCvT5asKtHM3P0g0163","token_type":"Bearer","refresh_token":"1//03p1Beqp63YatCgYIARAAGAMSNwF-L9Ir-9FulMaKOhK9b0Y8_5gTqMtLVb45CxUu_2jFS_0yQZ1vC6M50Zmz8_2I8M0w9650EZc","expiry":"2023-04-20T23:20:05.988475124Z"}
- team_drive: 
Keep this "GDrive" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y

Current remotes:

Name                 Type
====                 ====
GDrive               drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
digital-coffee-cup@digital-coffee-cup:~$ 

Testing

The first test we will run is

rclone about "GDrive:"

This command will show you your quota. You should get some output like this.

digital-coffee-cup@digital-coffee-cup:~$ rclone about "GDrive:"
Total:   15 GiB
Used:    1.731 GiB
Free:    13.269 GiB
Trashed: 24.045 MiB
Other:   204.189 KiB
digital-coffee-cup@digital-coffee-cup:~$

Next we will check to see if it is for sure working. Enter

nano test.txt

This will create a text file we can use to transfer and see if Rclone is working. You can type anything into the nano editor because it is only a test file. Hit Ctr+X, Y, then enter. Next we will try to move that file to our Google Drive using the following command. It may take a few seconds. Navigate to your Google Drive and you should see test.txt in the home folder. Check your machine for the file. It should be deleted. You can delete it also out of Google Drive. We don’t need it anymore.

rclone move "test.txt" "GDrive:"

How It Works

I will now try to explain this. I found it a bit complicated just by reading the documentation out there. I will try to make it more simple by using examples. We will use this code for an example.

rclone move "/root/songs" "GDrive:Uploads/Songs/New Uploads" 

Say you have two songs on your computer or server. They are named song1.mp3 and song2.mp3 and they are located in the /root/songs directory. You want to move them to a directory in your Google Drive called /Uploads/Songs/New Uploads. The above code will do exactly that. It will take all files in the /root/songs directory and copy them to /Uploads/Songs/New Uploads. If you only want a specific file moved then you would send

rclone move "/root/songs/song1.mp3" "GDrive:Uploads/Songs/New Uploads"

Whatever you named your remote clear back at the beginning, that is your home Google Drive folder. It doesn’t matter what you named it, just use it as the beginning of your file path.

Finishing Up

Congrats! You are done! You can now sync files to Google Drive from a remote machine or a local machine using Rclone through the Terminal! If you want more commands, just type

rclone --help

You will be given a list of commands that you can use. You can also read Rclone’s documentation here.

Thanks for Reading!