diff --git a/object-storage/rclone-screenshots/rclone-path-win1.png b/object-storage/rclone-screenshots/rclone-path-win1.png new file mode 100644 index 0000000000000000000000000000000000000000..d94c207931b553249df8aa0588cc6293720e3b55 Binary files /dev/null and b/object-storage/rclone-screenshots/rclone-path-win1.png differ diff --git a/object-storage/rclone-screenshots/rclone-path-win2.png b/object-storage/rclone-screenshots/rclone-path-win2.png new file mode 100644 index 0000000000000000000000000000000000000000..ec6befc2e64209b9de45a6fbf2dc4ba54ad6f690 Binary files /dev/null and b/object-storage/rclone-screenshots/rclone-path-win2.png differ diff --git a/object-storage/rclone-screenshots/rclone-path-win3.png b/object-storage/rclone-screenshots/rclone-path-win3.png new file mode 100644 index 0000000000000000000000000000000000000000..dfb8cb585d6dfcc149a902bb7582dd1d0505b83d Binary files /dev/null and b/object-storage/rclone-screenshots/rclone-path-win3.png differ diff --git a/object-storage/rclone-screenshots/rclone-path-win4.png b/object-storage/rclone-screenshots/rclone-path-win4.png new file mode 100644 index 0000000000000000000000000000000000000000..e3021259061ea91ac48759fd93ff80a45fb59722 Binary files /dev/null and b/object-storage/rclone-screenshots/rclone-path-win4.png differ diff --git a/object-storage/rclone.md b/object-storage/rclone.md new file mode 100644 index 0000000000000000000000000000000000000000..0e3f7433eaa6d07732bea41683a41e3f75cb484f --- /dev/null +++ b/object-storage/rclone.md @@ -0,0 +1,59 @@ +--- +languages: + - en + - cs +--- +# Rclone +This guide serves for rclone tool configuration. Rclone is the swiss knife for connection to multiple types of storage. However, the following guides are limited only to Object Storage namely S3 service. The following guides are suited for Windows users as well as Linux users. Using rclone tool for S3 enables the creation of the buckets, syncing folders, uploading and downloading files, and much more. + +## Downloading and installation of rclone tool +Firstly you need to download and unzip the desired [rclone](https://rclone.org/downloads/) version according to the system you operate. + +> [color=#F1C232] :warning: **We strongly recommend** to use up-to-date versions of rclone tool available at [rclone websites](https://rclone.org/downloads/). Using rclone version from system repositories can cause some errors due to the outdated rclone version. In case of a manual installation into the user profile, see below, you can use for an update following command: +**```rclone selfupdate```** + +---- + +### Linux - manual installation into the user profile + +We need to unzip the [rclone archive](https://rclone.org/downloads/) after download. + +```unzip rclone-v1.59.1-linux-amd64.zip``` + +Then we need to copy rclone binary file into the pre-prepared bin folder in the user profile/home. + +```cp ./rclone-v1.59.1-linux-amd64/rclone /home/user/bin/``` + +In the last step, we need to put the path to the rclone binary file into PATH environment variable. + +```PATH=/home/user/bin:$PATH``` + + +> [color=#93C47D] :bulb: For **persistent presence** of the rclone binary file path in the PATH variable you can add the following line into **.bashrc file**: +> +> ```echo 'PATH=/home/user/bin:$PATH' >> .bashrc``` +> +> Alternatively you can place the rclone binary file into the system path: +> +> ```/usr/local/bin``` + +In the end, we can make a quick check of PATH variable, whether the desired path is present. + +```echo $PATH``` + +> [color=#F1C232] :warning: In case you have installed the rclone using the steps above, you can then use the following command to update rclone: +> **```rclone selfupdate```** +> ```2022/08/25 11:54:07 NOTICE: Successfully updated rclone from version v1.59.0 to version v1.59.1``` + +---- + +### Windows - manual installation in to user profile + +Firstly you need to prepare `bin` directory in your user profile, where we will place rclone.exe file. Please prepare the directory in your user profile. You can just put the following command into the file browser: + +```%USERPROFILE%``` + +In the displayed directory you can click right mouse button -> **New** –> **Directory/Folder**. The directory should be named **bin**. Then move the **rclone.exe** file into it. The file **rclone.exe** is present in the unzipped rclone archive, which you downloaded from the [rclone websites](https://rclone.org/downloads/). + +In the next step click to **Start (1)** and search for **Edit the system environment variables (2)**. + diff --git a/object-storage/s3-service.md b/object-storage/s3-service.md index 5d0843940c7995ad1716856ef545b6e8fd82a06b..c3210811f6173000abb9f7778861fb0c7aabf961 100644 --- a/object-storage/s3-service.md +++ b/object-storage/s3-service.md @@ -3,7 +3,6 @@ languages: - en - cs --- - # S3 Service S3 service is a general service suited for most of the use cases. S3 service can be used for elementary data storing, automated backups, or various types of data handling applications. @@ -12,7 +11,9 @@ S3 service utilizes similar name convention as AWS S3. The convention is “buck In case your client considers the endpoint as native AWS you have to switch to S3 compatible endpoint. Most of the clients can automatically process both formattings. However, in some cases is necessary to specify the format explicitly. -**To connect to S3 service, you have to contact Data Storage support at: du-support@cesnet.cz** +> [color=#93C47D] :bulb: **How to get S3 service?** +To connect to S3 service you have to contact Data Storage support at: +du-support@cesnet.cz Once you obtain your credentials you can continue to connection itself using one of the following S3 client. @@ -22,5 +23,5 @@ In the following section you can find recommended S3 clients. ### AWS-CLI (Linux, Windows) [AWS CLI](https://aws.amazon.com/cli/) - Amazon Web Services Command Line Interface - is standardized too; supporting S3 interface. Using this tool you can handle your data and set up your S3 data storage. You can used the command line control or you can incorporate AWS CLI into your automated scripts. [Tutorial for AWS CLI](aws-cli.md). - - +### Rclone (Linux, Windows) +The tool [Rclone](https://rclone.org/downloads/) is suitable for data synchronization and data migration between more endpoints (even between different data storage providers). Rclone preserves the time stamps and checks the checksums. It is written in Go language. Rclone is available for multiple platforms (GNU/Linux, Windows, macOS, BSD and Solaris). In the following guide, we will demonstrate the usage in Linux and Windows systems. [Rclone guide](rclone.md).