Skip to content
Snippets Groups Projects
Commit ebb6bad6 authored by Milan's avatar Milan
Browse files

Add rclone config

parent b8a37a31
No related branches found
No related tags found
No related merge requests found
Showing
with 150 additions and 14 deletions
......@@ -16,10 +16,9 @@ Object Storage Services is a portfolio of services allowing to facilitate your a
**S3** is a general service suitable for most of the usecases (archives, backups, special applications...). It also allows to share your data with other users or publicly via link.
[:octicons-arrow-right-24: Overview of S3 Service](./s3-service.md)
[:octicons-arrow-right-24: Bucket Sharing](./rclone.md)
[:octicons-arrow-right-24: Bucket Encryption](./rclone.md)
[:octicons-arrow-right-24: Worm model](./rclone.md)
[:octicons-arrow-right-24: Veeam setup against S3](./rclone.md)
[:octicons-arrow-right-24: Favourite S3 Clients](./rclone.md)
[:octicons-arrow-right-24: Advanced S3 Functions](./rclone.md)
[:octicons-arrow-right-24: Veeam stup against S3](./rclone.md)
<!---
[:octicons-arrow-right-24: Account properties and lifecycle](/account/properties)
--->
......
object-storage/rclone-screenshots/rclone-cmd1.png

65.7 KiB

object-storage/rclone-screenshots/rclone-cmd10.png

45 KiB

object-storage/rclone-screenshots/rclone-cmd11.png

30.2 KiB

object-storage/rclone-screenshots/rclone-cmd12.png

9.77 KiB

object-storage/rclone-screenshots/rclone-cmd13.png

43 KiB

object-storage/rclone-screenshots/rclone-cmd2.png

58 KiB

object-storage/rclone-screenshots/rclone-cmd3.png

35.9 KiB

object-storage/rclone-screenshots/rclone-cmd4.png

48.5 KiB

object-storage/rclone-screenshots/rclone-cmd5.png

69.1 KiB

object-storage/rclone-screenshots/rclone-cmd6.png

41.5 KiB

object-storage/rclone-screenshots/rclone-cmd7.png

25.5 KiB

object-storage/rclone-screenshots/rclone-cmd8.png

33.6 KiB

object-storage/rclone-screenshots/rclone-cmd9.png

154 KiB

object-storage/rclone-screenshots/rclone-gui-s3-1.png

132 KiB

object-storage/rclone-screenshots/rclone-gui-s3-2.png

253 KiB

object-storage/rclone-screenshots/rclone-gui1.png

88.2 KiB

object-storage/rclone-screenshots/rclone-gui4.png

112 KiB

object-storage/rclone-screenshots/rclone-gui_upload.png

126 KiB

......@@ -11,7 +11,8 @@ Firstly you need to download and unzip the desired [rclone](https://rclone.org/d
!!! warning
**We strongly recommend** to use up-to-date versions of rclone tool available at [rclone websites](https://rclone.org/downloads/). Using rclone version from system repositories can cause some errors due to the outdated rclone version. In case of a manual installation into the user profile, see below, you can use for an update following command:
**```rclone selfupdate```**
rclone selfupdate
----
......@@ -19,28 +20,28 @@ Firstly you need to download and unzip the desired [rclone](https://rclone.org/d
We need to unzip the [rclone archive](https://rclone.org/downloads/) after download.
```unzip rclone-v1.59.1-linux-amd64.zip```
unzip rclone-v1.59.1-linux-amd64.zip
Then we need to copy rclone binary file into the pre-prepared bin folder in the user profile/home.
```cp ./rclone-v1.59.1-linux-amd64/rclone /home/user/bin/```
cp ./rclone-v1.59.1-linux-amd64/rclone /home/user/bin/
In the last step, we need to put the path to the rclone binary file into PATH environment variable.
```PATH=/home/user/bin:$PATH```
PATH=/home/user/bin:$PATH
???+ note "Persistent setup"
For **persistent presence** of the rclone binary file path in the PATH variable you can add the following line into **.bashrc file**:
```echo 'PATH=/home/user/bin:$PATH' >> .bashrc```
echo 'PATH=/home/user/bin:$PATH' >> .bashrc
Alternatively you can place the rclone binary file into the system path:
Alternatively you can place the rclone binary file into the system path:
```/usr/local/bin```
/usr/local/bin
In the end, we can make a quick check of PATH variable, whether the desired path is present.
```echo $PATH```
echo $PATH
!!! warning
In case you have installed the rclone using the steps above, you can then use the following command to update rclone:<br/>
......@@ -53,7 +54,7 @@ In the end, we can make a quick check of PATH variable, whether the desired path
Firstly you need to prepare `bin` directory in your user profile, where we will place rclone.exe file. Please prepare the directory in your user profile. You can just put the following command into the file browser:
```%USERPROFILE%```
%USERPROFILE%
In the displayed directory you can click right mouse button -> **New** –> **Directory/Folder**. The directory should be named **bin**. Then move the **rclone.exe** file into it. The file **rclone.exe** is present in the unzipped rclone archive, which you downloaded from the [rclone websites](https://rclone.org/downloads/).
......@@ -71,7 +72,7 @@ In the section User variables for **UserXY (1)** you will select the line with v
You can add the new path by clicking on the **New (1)** button and then you have to insert the path to the pre-prepared **“bin” folder (2)**, see below. The setup is then confirmed by clicking on the **OK (3)** button.
```%USERPROFILE%\bin```
%USERPROFILE%\bin
![](rclone-screenshots/rclone-path-win4.png){ style="display: block; margin: 0 auto" }
......@@ -82,4 +83,140 @@ In the end, you will click **OK** and **Apply**.
**```rclone selfupdate```**<br/>
```2022/08/25 11:54:07 NOTICE: Successfully updated rclone from version v1.59.0 to version v1.59.1```
## Basic configuration of rclone
Below you can find the guide for the elementary configuration of rclone tool. Below are two guides. First describes configuration using the command line and second guide describes configuration using the graphical user interface.
----
### Rclone configuration using the command line
!!! warning
To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](##downloading-and-installation-of-rclone-tool).
Rclone has a configuration wizard, which will guide you step-by-step through the configuration of your S3 data storage.
???+ note "Command line in Windows and Linux"
**Windows users** need to run **Command Prompt** and then run the command below.
**Linux users** can open the **terminal window** and then run the command below.
rclone config
![](rclone-screenshots/rclone-cmd1.png){ style="display: block; margin: 0 auto" }
From the list of options, we will choose **New remote** via inserting **n** letter. Then we need to type the name of our connection/storage, e.g. `cesnet_s3`. Then we will choose the **Option Storage**, here **Amazon S3 Compliant Storage Providers…**
![](rclone-screenshots/rclone-cmd2.png){ style="display: block; margin: 0 auto" }
In the next step, we need to choose **Option provider**, here **Ceph Object Storage**.
![](rclone-screenshots/rclone-cmd3.png){ style="display: block; margin: 0 auto" }
Then it is necessary to choose how to enter S3 credentials. **Here we choose Enter AWS credentials in the next step**.
![](rclone-screenshots/rclone-cmd4.png){ style="display: block; margin: 0 auto" }
In the next steps we enter our credentials **access_key_id** and **secret_access_key** which we obtained from the administrator or which we generated.
![](rclone-screenshots/rclone-cmd5.png){ style="display: block; margin: 0 auto" }
Then we need to select **Option region**. We will leave **this option empty** and then continue with **Enter**.
![](rclone-screenshots/rclone-cmd6.png){ style="display: block; margin: 0 auto" }
Then we need to insert **Option endpoint** according to the data center, where we have generated the credentials, here for example **s3.cl2.du.cesnet.cz**.
![](rclone-screenshots/rclone-cmd7.png){ style="display: block; margin: 0 auto" }
Next is **Option location_constraint**. We will **leave this option empty** and then continue with **Enter**.
![](rclone-screenshots/rclone-cmd8.png){ style="display: block; margin: 0 auto" }
Next point si **Option acl**, here we can either choose **Owner gets FULL_CONTROLL**, or we can leave this option empty and continue with **Enter**.
![](rclone-screenshots/rclone-cmd9.png){ style="display: block; margin: 0 auto" }
In the next step, we need to choose **Option server_side_encryption**, here we pick the option **None**.
![](rclone-screenshots/rclone-cmd10.png){ style="display: block; margin: 0 auto" }
Then we can choose **Option see_kms_key_id**, here we pick the option **None**.
![](rclone-screenshots/rclone-cmd11.png){ style="display: block; margin: 0 auto" }
In the next step we can choose option **Edit advanced config**, here we choose **No**, option **n**.
![](rclone-screenshots/rclone-cmd12.png){ style="display: block; margin: 0 auto" }
In the last step, we check the configuration and we will confirm it by typing **y** letter.
![](rclone-screenshots/rclone-cmd13.png){ style="display: block; margin: 0 auto" }
----
### Rclone configuration using graphical user interface
!!! warning
To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](##downloading-and-installation-of-rclone-tool).
Firstly you need to run the GUI. **Windows users** need to open **Command Prompt** and run the command below. **Linux users** need just open the **terminal window** and run the command below.
rclone rcd --rc-web-gui
The next steps are identical for Windows and Linux.
After starting up the graphical interface we click in the left menu on the **Configs (1)** button and then on the **Create a New Config (2)** button.
![](rclone-screenshots/rclone-gui1.png){ style="display: block; margin: 0 auto" }
In the displayed window, we insert the **connection name** in our example `cesnet_s3cl2` and then we choose the option **Amazon S3 Compliant Storage Providers**.
![](rclone-screenshots/rclone-gui-s3-1.png){ style="display: block; margin: 0 auto" }
In the next step, we need to insert **credentials (1)** which we obtained from the administrator or which we generated. You have to insert **Ceph** into the line denoted **Choose your S3 provider**. Then we need to insert the **S3 endpoint address (2)**. Then we can click **Next**.
![](rclone-screenshots/rclone-gui-s3-2.png){ style="display: block; margin: 0 auto" }
!!! warning
Please be careful during the modification in the **Configs** section. Rclone GUI sometimes **does not save the changes** in the configuration. We strongly recommend to cross-check the **[configuration file](###configuration-file)** after saving.
#### Uploading the data from your local machine
After the configuration, we can start to transfer the data.
In the left menu click on the **Explorer (1)** button. Then select **the name of configuration (2)**, for example `cesnet_s3cl2`. Then you click on the **Open (3)** button. Then there should be a window with the buckets and files from the configured data storage.
![](rclone-screenshots/rclone-gui4.png){ style="display: block; margin: 0 auto" }
!!! warning
Graphical user interface of rclone DOES NOT SUPPORT creation of empty buckets and directories. If you copy your data from the local machine you have to copy the directory with data. Alternatively, you can prepare empty buckets using [command line](##rclone-basic-controls).
If you wish to upload your data then in the displayed window click on **upload icon (1)**. Then you can select the data from your disk or drag and drop them into the window.
![](rclone-screenshots/rclone-gui_upload.png){ style="display: block; margin: 0 auto" }
----
### Configuration file
!!! warning
Configuration file can be found in the location described below. In the configuration file are saved the credentials and all selected options.
???+ note "Windows config file"
C:\Users\DedaLebeda\AppData\Roaming\rclone\rclone.conf<br/>
<br/>[cesnet_s3]
type = s3
provider = Ceph
access_key_id = my-access-key
secret_access_key = my-secret-key
endpoint = s3.cl2.du.cesnet.cz
acl = private
???+ note "Windows config file"
C:\Users\DedaLebeda\AppData\Roaming\rclone\rclone.conf<br/>
<br/>[cesnet_s3]
type = s3
provider = Ceph
access_key_id = my-access-key
secret_access_key = my-secret-key
endpoint = s3.cl2.du.cesnet.cz
acl = private
## Rclone basic controls
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment