Skip to content
Snippets Groups Projects
s3cmd.md 3.4 KiB
Newer Older
Milan's avatar
Milan committed
---
languages:
  - en
  - cs
---

# s3cmd command line tool

[S3cmd](https://s3tools.org/download) is a free command line tool. It allows you to upload and download your data to the S3 object storage. S3cmd is written in Python. S3cmd is an open-source project available under GNU Public License v2 (GPLv2) and it is free for personal as well as commercial usage.

!!! warning
    We recommend you **use preferably [AWS CLI](s3cmd.md)**. We encountered some issues while using s3cmd. For instance, bucket names cannot begin with numbers or capital letters.

## Installation of s3cmd tool

S3cmd is available in the system repositories for CentOS, RHEL and Ubuntu. You can install it via following guide.

**On CentOS/RHEL**

Milan's avatar
Milan committed
    sudo yum install s3cmd 
Milan's avatar
Milan committed

**On Ubuntu/Debian**

Milan's avatar
Milan committed
    sudo apt install s3cmd 
Milan's avatar
Milan committed

## Configuration of s3cmd tool

Please insert the following lines into the config file located at **/home/user/.s3cfg**.

    [default]
    host_base = https://s3.clX.du.cesnet.cz
    use_https = True
    access_key = xxxxxxxxxxxxxxxxxxxxxx
    secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    host_bucket = s3.clX.du.cesnet.cz

Milan's avatar
Milan committed
`host_base` and `host_bucket` is S3 endpoint URL, which you received via email together with `access_key` and `secret_key`. You should receive it via email during the S3 account creation.
Milan's avatar
Milan committed

**Config file with GPG encryption**

    [default]
    host_base = https://s3.clX.du.cesnet.cz
    use_https = True
    access_key = xxxxxxxxxxxxxxxxxxxxxx
    secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    host_bucket = s3.clX.du.cesnet.cz
    gpg_command = /usr/bin/gpg
    gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
    gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
    gpg_passphrase = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

## Basic s3cmd commands

S3cmd commands support elementary operations with buckets - creation, listing, and deletion.

### Bucket operations

???+ note "Bucket name"
    The bucket name should be unique within tenant and should contain only small letters, capital letters, numerals, dashes, and dots. The bucket name must begin only with a letter or numeral and it cannot contain dots next to dashes or multiple dots.

**Listing all s3 buckets**

    s3cmd ls

**Creation of new s3 bucket**

    s3cmd mb s3://newbucket

**Removing s3 bucket**

    s3cmd rb s3://newbucket

_Only emptied bucket can be removed!_

**Listing s3 bucket size**

    s3cmd s3://newbucket/ du 

### Files and directories operation

**Listing of s3 bucket**
    
    s3cmd ls s3://newbucket/ 

**File upload**

Milan's avatar
Milan committed
    s3cmd put file.txt s3://newbucket/
Milan's avatar
Milan committed
 
**Upload of encrypted files**

    s3cmd put -e file.txt s3://newbucket/

**Directory upload**

    s3cmd put -r directory s3://newbucket/ 

_Please make sure, that you didn't forget to remove the trailing slash (e.g. .: directory/), trailing slash denotes uploading only the content of the desired directory._

**Download file from s3 bucket**

    s3cmd get s3://newbucket/file.txt

**Data deletion from s3 bucket**

    s3cmd del s3://newbucket/file.txt

    s3cmd del s3://newbucket/directory 

**Data sync into s3 bucket from local machine**

    s3cmd sync /local/path/ s3://newbucket/backup/ 

**Data sync from s3 bucket to local machine**

    3cmd sync s3://newbucket/backup/ ~/restore/