Skip to content
Snippets Groups Projects
Commit e54e8d33 authored by Milan's avatar Milan
Browse files

RBD setup + cvorrections

parent 7f1e422e
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
index.md
+ 5
− 4
View file @ e54e8d33
...@@ -42,20 +42,21 @@ Data Management Services is a portfolio of services allowing to facilitate the w ...@@ -42,20 +42,21 @@ Data Management Services is a portfolio of services allowing to facilitate the w
Do you need to cooperate with your colleagues, edit documents and share data? Do you need to cooperate with your colleagues, edit documents and share data?
[:octicons-arrow-right-24: Owncloud](https://du.cesnet.cz/en/navody/owncloud/start) [:octicons-arrow-right-24: ownCloud](https://du.cesnet.cz/en/navody/owncloud/start)
[:octicons-arrow-right-24: Onlyoffice](https://du.cesnet.cz/en/navody/onlyoffice/start)
[:octicons-arrow-right-24: ONLYOFFICE](https://du.cesnet.cz/en/navody/onlyoffice/start)
<!--- <!---
[:octicons-arrow-right-24: Account properties and lifecycle](/account/properties) [:octicons-arrow-right-24: Account properties and lifecycle](/account/properties)
---> --->
- :fontawesome-solid-server:{ .lg .middle } __Long Tail Data Preservation__ - :fontawesome-solid-server:{ .lg .middle } __Longterm Data Preservation__
--- ---
Do you need to archive your data in the binary reliable data storage? Do you need to archive your data in the binary reliable data storage?
[:octicons-arrow-right-24: Longtail Preservation - CZ only](https://du.cesnet.cz/cs/navody/ltp/start) [:octicons-arrow-right-24: Longterm Preservation - CZ only](https://du.cesnet.cz/cs/navody/ltp/start)
<!--- <!---
[:octicons-arrow-right-24: Account properties and lifecycle](/account/properties) [:octicons-arrow-right-24: Account properties and lifecycle](/account/properties)
......
object-storage/aws-cli.md
+ 4
− 0
View file @ e54e8d33
...@@ -122,10 +122,14 @@ After successful configuration, the configuration file should be created. You ca ...@@ -122,10 +122,14 @@ After successful configuration, the configuration file should be created. You ca
## Special functions of AWS-CLI ## Special functions of AWS-CLI
There are several advanced functions in AWS-CLI for sharing the data or its versioning.
### Presign URLs ### Presign URLs
For object in S3 service you can generate presign URL to allow your colleagues to download the data. You can find more information the the section dedicated to [advanced S3 features](s3-features.md)
### Bucket policies ### Bucket policies
To share your data you can setup so called bucket policies. You can share specific bucket to a specific group (tenant) or make your bucket publicly readable. You can find more information the the section dedicated to [advanced S3 features](s3-features.md)
### Bucket versioning ### Bucket versioning
You can setup object versioning inside in your buckets. Then you can restore any previous version of the object (file). You can find more information the the section dedicated to [advanced S3 features](s3-features.md)
object-storage/rbd-setup.md
+ 171
− 0
View file @ e54e8d33
...@@ -65,10 +65,181 @@ Ubuntu/Ceph includes all necessary packages natively. So you can just run follow ...@@ -65,10 +65,181 @@ Ubuntu/Ceph includes all necessary packages natively. So you can just run follow
sudo apt install ceph sudo apt install ceph
## RBD configuration and its mapping
Use the credentials which you received from the system administrator to configure and connect the RBD. These are the following:
* pool name: **rbd_vo_poolname**
* image name: **vo_name_username**
* keyring: **[client.rbd_user] key = key_hash ==**
In the directory **/etc/ceph/** create the text file **ceph.conf** with the following content.
???+ note "CL1 Data Storage"
[global]
fsid = 19f6785a-70e1-45e8-a23a-5cff0c39aa54
mon_host = [v2:78.128.244.33:3300,v1:78.128.244.33:6789],[v2:78.128.244.37:3300,v1:78.128.244.37:6789],[v2:78.128.244.41:3300,v1:78.128.244.41:6789]
auth_client_required = cephx
???+ note "CL2 Data Storage"
[global]
fsid = 3ea58563-c8b9-4e63-84b0-a504a5c71f76
mon_host = [v2:78.128.244.65:3300/0,v1:78.128.244.65:6789/0],[v2:78.128.244.69:3300/0,v1:78.128.244.69:6789/0],[v2:78.128.244.71:3300/0,v1:78.128.244.71:6789/0]
auth_client_required = cephx
???+ note "CL3 Data Storage"
[global]
fsid = b16aa2d2-fbe7-4f35-bc2f-3de29100e958
mon_host = [v2:78.128.244.240:3300/0,v1:78.128.244.240:6789/0],[v2:78.128.244.241:3300/0,v1:78.128.244.241:6789/0],[v2:78.128.244.242:3300/0,v1:78.128.244.242:6789/0]
auth_client_required = cephx
???+ note "CL4 Data Storage"
[global]
fsid = c4ad8c6f-7ef3-4b0e-873c-b16b00b5aac4
mon_host = [v2:78.128.245.29:3300/0,v1:78.128.245.29:6789/0] [v2:78.128.245.30:3300/0,v1:78.128.245.30:6789/0] [v2:78.128.245.31:3300/0,v1:78.128.245.31:6789/0]
auth_client_required = cephx
Further in the directory **/etc/ceph/** create the text file **ceph.keyring**. Then save in that file the keyring, see the example below.
[client.rbd_user]
key = sdsaetdfrterp+sfsdM3iKY5teisfsdXoZ5==
!!! warning
If the location of the files `ceph.conf` and `username.keyring` differs from the default directory **/etc/ceph/**, the corresponding paths must be specified during mapping. See below.
sudo rbd -c /home/username/ceph/ceph.conf -k /home/username/ceph/username.keyring --id rbd_user device map name_pool/name_image
Then check the connection in kernel messages.
dmesg
Now check the status of RBD.
sudo rbd device list | grep "name_image"
## Encrypting and creating a file system
The next step is to encrypt the mapped image. Use **cryptsetup-luks** for encryption.
sudo yum install cryptsetup-luks
Then it encrypts the device.
sudo cryptsetup -s 512 luksFormat --type luks2 /dev/rbdX
Finally, check the settings.
sudo cryptsetup luksDump /dev/rbdX
In order to perform further actions on an encrypted device, it must be decrypted first.
sudo cryptsetup luksOpen /dev/rbdX luks_rbdX
???+ note ""
We recommend using XFS instead of EXT4 for larger images or those they will need to be enlarged to more than 200TB over time, because EXT4 has a limit on the number of inodes.
Now create file system on the device, here is an example xfs.
sudo mkfs.xfs -K /dev/mapper/luks_rbdX
!!! warning
If you use XFS, do not use the nobarrier option while mounting, it could cause data loss!
Once the file system is ready, we can mount the device in a pre-created folder in /mnt/.
sudo mount /dev/mapper/luks_rbdX /mnt/rbd
## Ending work with RBD
Unmount the volume.
sudo umount /mnt/rbd/
Close the encrypted volume.
sudo cryptsetup luksClose /dev/mapper/luks_rbdX
Volume unmapping.
sudo rbd --id rbd_user device unmap /dev/rbdX/
???+ note ""
To get better performance choose appropriate size of read_ahead cache depends on your size of memory.
Example for 8GB:
echo 8388608 > /sys/block/rbd0/queue/read_ahead_kb
Example for 512MB:
echo 524288 > /sys/block/rbd0/queue/read_ahead_kb
To apply changes you have to unmap image and map it again.
The approach described above is not persistent (won't survive reboot). To do it persistent you have to add following line into “/etc/udev/rules.d/50-read-ahead-kb.rules” file.
# Setting specific kernel parameters for a subset of block devices (Ceph RBD)
KERNEL=="rbd[0-9]*", ENV{DEVTYPE}=="disk", ACTION=="add|change", ATTR{bdi/read_ahead_kb}="524288"
## Permanently mapping of RBD
Settings for automatic RBD connection, including LUKS encryption and mount filesystems. + proper disconnection (in reverse order) when the machine is switched off in a controlled manner.
### RBD image
Edit configuration file in the path `/etc/ceph/rbdmap` by inserting following lines.
# RbdDevice Parameters
#poolname/imagename id=client,keyring=/etc/ceph/ceph.client.keyring
pool_name/image_name id=rbd_user,keyring=/etc/ceph/ceph.keyring
### LUKS
Edit configuration file in the path `/etc/crypttab` by inserting following lines.
# <target name> <source device> <key file> <options>
rbd_luks_pool /dev/rbd/pool_name/image_name /etc/ceph/luks.keyfile luks,_netdev
where **/etc/ceph/luks.keyfile** is LUKS key.
???+ note ""
path to block device (“<source device>”) is generally `/dev/rbd/$POOL/$IMAGE`
### fstab file
Edit configuration file in the path `/etc/fstab` by inserting following lines.
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/rbd_luks_pool /mnt/rbd_luks_pool btrfs defaults,noatime,auto,_netdev 0 0
???+ note ""
path to LUKS container (“<file system>”) is generally `/dev/mapper/$LUKS_NAME`,
where `$LUKS_NAME` is defined in `/etc/crypttab` (like “<taget name>”)
### systemd unit
Edit configuration file in the path `/etc/systemd/system/systemd-cryptsetup@rbd_luks_pool.service.d/10-deps.conf` by inserting following lines.
[Unit]
After=rbdmap.service
Requires=rbdmap.service
Before=mnt-rbd_luks_pool.mount
???+ note ""
In one case, systemd units were used on Debian 10 for some reason `ceph-rbdmap.service` instead of `rbdmap.service` (must be adjusted to lines `After=` and `Requires=`)
----
### Manual connection
If the dependencies of the systemd units are correct, it performs an RBD map, unlocks LUKS and mounts all the automatic fs dependent on the rbdmap that the specified .mount unit needs (⇒ mounts both images in the described configuration).
systemctl start mnt-rbd_luks_pool.mount
### Manual disconnection
This command should execute if the dependencies are set correctly `umount`, LUKS `close` i RBD unmap.
systemctl stop rbdmap.service
(alternatively `systemctl stop ceph-rbdmap.service`)
### Resize
When resizing an encrypted image, you need to follow the order and the main one is the line with cryptsetup `--verbose resize image_name`.
rbd resize rbd_pool_name/image_name --size 200T
cryptsetup --verbose resize image_name
mount /storage/rbd/image_name
xfs_growfs /dev/mapper/image_name
......
object-storage/rclone.md
+ 15
− 15
View file @ e54e8d33
...@@ -83,12 +83,12 @@ In the end, you will click **OK** and **Apply**. ...@@ -83,12 +83,12 @@ In the end, you will click **OK** and **Apply**.
**```rclone selfupdate```**<br/> **```rclone selfupdate```**<br/>
2022/08/25 11:54:07 NOTICE: Successfully updated rclone from version v1.59.0 to version v1.59.1 2022/08/25 11:54:07 NOTICE: Successfully updated rclone from version v1.59.0 to version v1.59.1
# Basic configuration of rclone ## Basic configuration of rclone
Below you can find the guide for the elementary configuration of rclone tool. Below are two guides. First describes configuration using the command line and second guide describes configuration using the graphical user interface. Below you can find the guide for the elementary configuration of rclone tool. Below are two guides. First describes configuration using the command line and second guide describes configuration using the graphical user interface.
---- ----
## Rclone configuration using the command line ### Rclone configuration using the command line
!!! warning !!! warning
To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool). To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool).
...@@ -153,7 +153,7 @@ In the last step, we check the configuration and we will confirm it by typing ** ...@@ -153,7 +153,7 @@ In the last step, we check the configuration and we will confirm it by typing **
---- ----
## Rclone configuration using graphical user interface ### Rclone configuration using graphical user interface
!!! warning !!! warning
To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool). To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool).
...@@ -196,7 +196,7 @@ If you wish to upload your data then in the displayed window click on **upload i ...@@ -196,7 +196,7 @@ If you wish to upload your data then in the displayed window click on **upload i
![](rclone-screenshots/rclone-gui_upload.png){ style="display: block; margin: 0 auto" } ![](rclone-screenshots/rclone-gui_upload.png){ style="display: block; margin: 0 auto" }
---- ----
## Configuration file ### Configuration file
!!! warning !!! warning
Configuration file can be found in the location described below. In the configuration file are saved the credentials and all selected options. Configuration file can be found in the location described below. In the configuration file are saved the credentials and all selected options.
...@@ -220,7 +220,7 @@ If you wish to upload your data then in the displayed window click on **upload i ...@@ -220,7 +220,7 @@ If you wish to upload your data then in the displayed window click on **upload i
endpoint = s3.cl2.du.cesnet.cz<br/> endpoint = s3.cl2.du.cesnet.cz<br/>
acl = private<br/> acl = private<br/>
# Rclone basic controls ## Rclone basic controls
!!! warning !!! warning
All available commands for rclone can be listed using the command All available commands for rclone can be listed using the command
...@@ -229,7 +229,7 @@ If you wish to upload your data then in the displayed window click on **upload i ...@@ -229,7 +229,7 @@ If you wish to upload your data then in the displayed window click on **upload i
Alternatively you can find rclone guide on the [rclone websites](https://rclone.org/commands/). Below are described the selected commands to control buckets, directories and files. Alternatively you can find rclone guide on the [rclone websites](https://rclone.org/commands/). Below are described the selected commands to control buckets, directories and files.
## Listing buckets and directories ### Listing buckets and directories
**Listing of the available profiles/connections.** **Listing of the available profiles/connections.**
...@@ -245,7 +245,7 @@ If you wish to upload your data then in the displayed window click on **upload i ...@@ -245,7 +245,7 @@ If you wish to upload your data then in the displayed window click on **upload i
-1 2020-11-11 08:53:48 -1 111 -1 2020-11-11 08:53:48 -1 111
-1 2022-07-28 10:03:20 -1 test -1 2022-07-28 10:03:20 -1 test
## Creation of the bucket, copying, deletion... ### Creation of the bucket, copying, deletion...
**Creation of the new bucket.** **Creation of the new bucket.**
...@@ -283,7 +283,7 @@ To delete a particular file, we can use either command **deletefile** or the com ...@@ -283,7 +283,7 @@ To delete a particular file, we can use either command **deletefile** or the com
!!! warning !!! warning
In case you delete the only file (object) in the directory resulting in **empty directories structure** the empty directories will be deleted! Directories are in object technology always represented by the name of a particular object (file), deletion of empty directories is thus expected behavior. In case you delete the only file (object) in the directory resulting in **empty directories structure** the empty directories will be deleted! Directories are in object technology always represented by the name of a particular object (file), deletion of empty directories is thus expected behavior.
## Directory syncing ### Directory syncing
To sync the directories you can use the option `sync`. Synchronization is affecting the content only on the target side, no changes are performed on the source side. To sync the directories you can use the option `sync`. Synchronization is affecting the content only on the target side, no changes are performed on the source side.
...@@ -312,7 +312,7 @@ Option interactive allows interactively deciding which change (on the target dat ...@@ -312,7 +312,7 @@ Option interactive allows interactively deciding which change (on the target dat
--interactive --interactive
## Data integrity checks ### Data integrity checks
???+ note "Enhancing the speed of checking" ???+ note "Enhancing the speed of checking"
All commands related to data integrity check should contain `--fast-list` option, see above. Using the `--fast-list` option will enhance the speed of the integrity checks. All commands related to data integrity check should contain `--fast-list` option, see above. Using the `--fast-list` option will enhance the speed of the integrity checks.
...@@ -328,11 +328,11 @@ The command checks the checksums on the source side as well as on the target sid ...@@ -328,11 +328,11 @@ The command checks the checksums on the source side as well as on the target sid
!!! warning !!! warning
To check data integrity on the encrypted buckets please use the option `cryptcheck` which is described [in the guides related to encrypted buckets](#check-of-encrypted-data-integrity). In the case of using the option check on the encrypted volume, there will occur the forced download of all data in the checked path. Forced downloads are unnecessary and can stall your client. To check data integrity on the encrypted buckets please use the option `cryptcheck` which is described [in the guides related to encrypted buckets](#check-of-encrypted-data-integrity). In the case of using the option check on the encrypted volume, there will occur the forced download of all data in the checked path. Forced downloads are unnecessary and can stall your client.
# Configuration and controls of encryted bucket ## Configuration and controls of encryted bucket
This section describes the configuration and controls of encrypted buckets using rclone tool. It goes about client-side encryption. Below are the guides for setup using the command line and for setup using the graphical user interface. This section describes the configuration and controls of encrypted buckets using rclone tool. It goes about client-side encryption. Below are the guides for setup using the command line and for setup using the graphical user interface.
## Configuration using the command line ### Configuration using the command line
!!! warning !!! warning
To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool). To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool).
...@@ -403,7 +403,7 @@ In the end, we can list the encrypted bucket, where we can see three encrypted f ...@@ -403,7 +403,7 @@ In the end, we can list the encrypted bucket, where we can see three encrypted f
337619 cuqqkkhsklbnf1eegkujfkrcl4 337619 cuqqkkhsklbnf1eegkujfkrcl4
251589 pelqqer8osssa4k8uon95a4o6c 251589 pelqqer8osssa4k8uon95a4o6c
## Configuration of the encrypted bucket using the graphical user interface ### Configuration of the encrypted bucket using the graphical user interface
!!! warning !!! warning
To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool). To be able to configure the rclone tool using this guide **first, you have to download, unzip and install rclone**, the guide can be found in the [first section](#downloading-and-installation-of-rclone-tool).
...@@ -453,7 +453,7 @@ Indeed we can see that our three pictures **(1)** have been encrypted. ...@@ -453,7 +453,7 @@ Indeed we can see that our three pictures **(1)** have been encrypted.
???+ note "Configuration files for encrypted volumes" ???+ note "Configuration files for encrypted volumes"
Configuration file for encrypted volumes can be found in the [previous section](#configuration-file). Configuration file for encrypted volumes can be found in the [previous section](#configuration-file).
## Check of encrypted data integrity ### Check of encrypted data integrity
To check encrypted data integrity it is necessary to use the command **cryptcheck**, see below. Using the common workflow for data integrity checks will cause significant difficulties in the encrypted bucket. It can result in forced downloading of all data from the remote site so it can stall your client. To check encrypted data integrity it is necessary to use the command **cryptcheck**, see below. Using the common workflow for data integrity checks will cause significant difficulties in the encrypted bucket. It can result in forced downloading of all data from the remote site so it can stall your client.
...@@ -465,7 +465,7 @@ To check encrypted data integrity it is necessary to use the command **cryptchec ...@@ -465,7 +465,7 @@ To check encrypted data integrity it is necessary to use the command **cryptchec
???+ note "Enhancing the speed of checking" ???+ note "Enhancing the speed of checking"
While using option cryptcheck we recommend to use option `--fast-list`. It allows cache info about more than 1000 objects within one request, so it rapidly accelerates the checks. While using option cryptcheck we recommend to use option `--fast-list`. It allows cache info about more than 1000 objects within one request, so it rapidly accelerates the checks.
## Sharing of encrypted buckets ### Sharing of encrypted buckets
The buckets can be shared within the mutual space called the tenant or between users using the bucket policy. If you wish to share the buckets equipped with the encrypted volume you need to share the credentials (for encrypted volume in your bucket) with your colleagues. A shared bucket has to have a properly set up [bucket policy](aws-cli.md). The buckets can be shared within the mutual space called the tenant or between users using the bucket policy. If you wish to share the buckets equipped with the encrypted volume you need to share the credentials (for encrypted volume in your bucket) with your colleagues. A shared bucket has to have a properly set up [bucket policy](aws-cli.md).
...@@ -474,7 +474,7 @@ Once you configure the encryption in your bucket you just need to share the encr ...@@ -474,7 +474,7 @@ Once you configure the encryption in your bucket you just need to share the encr
!!! warning !!! warning
Please be aware of the next section describing the need for **change encrypting passwords, or loss of encrypting passwords**. Please be aware of the next section describing the need for **change encrypting passwords, or loss of encrypting passwords**.
## Compromitting of encrypting passwords vs. loss of encrypting passwords ### Compromitting of encrypting passwords vs. loss of encrypting passwords
**In case of compromitting or leakage** of your encrypting passwords or in the situation that you need to change the passwords is only possible to create a new encrypted volume with new encrypting passwords. All data has to be transferred to the new encrypted volume and the old one should be deleted. **In case of compromitting or leakage** of your encrypting passwords or in the situation that you need to change the passwords is only possible to create a new encrypted volume with new encrypting passwords. All data has to be transferred to the new encrypted volume and the old one should be deleted.
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment