Ubuntu 22.04 Backup using Amazon S3 and rclone

Home > Ubuntu > Ubuntu 22.04 > Ubuntu 22.04 Backup using Amazon S3 and rclone

1. Creating 7 Daily Buckets with 6-Day Immutability

• Log in to your AWS account and open the S3 console.

• Click Create Bucket, enter a unique name, and confirm creation.

• Open the newly created bucket and go to the Properties tab.

• Under Bucket Versioning, click Enable (required for object lock).

• Scroll to Object Lock, click Edit, and enable it.

• Enable Default Retention:

• Choose Compliance mode.

• Set retention period to 6 days.

• Save changes.

2. Writing Lifecycle Policies for Permanent Deletion after Rclone Deletes

When using rclone to delete objects from an S3 bucket, AWS retains these objects as "deleted objects" if Versioning is enabled. To permanently remove them after 1 day, a Lifecycle Policy must be created.

• Open the AWS S3 Console and your target bucket.

• Go to the Management tab > Lifecycle Rules > Create Lifecycle Rule.

• Set a name and choose to apply the rule to all objects.

• Select the following actions Under Lifecycle Rule Actions:

• Permanently delete noncurrent versions of objects

• Delete expired object delete markers or incomplete multipart uploads

• Set "Days after object deletion" to 1.

• Click Create Rule.

3. Creating an IAM User with Access to a Single S3 Bucket

Create the User

• Go to IAM > Users > Add User.

• Provide a username and select AWS Management Console access (for GUI access).

• Create Custom password

• uncheck the option: Users must create a new password at sign-in - Recommended

• Continue to permissions setup.

Attach a Custom Policy

• Select Attach policies directly > Create Policy.

• Switch to the JSON tab and paste the following:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"s3:GetBucketLocation",
				"s3:ListAllMyBuckets"
			],
			"Resource": "arn:aws:s3:::*"
		},
		{
			"Effect": "Allow",
			"Action": "s3:*",
			"Resource": [
				"arn:aws:s3:::<Bucket_name>",
				"arn:aws:s3:::<Bucket_name>/*"
			]
		}
	]
}

• Replace <Bucket_name> with your actual bucket name.

• Give the policy name

• This policy allows full access to the specified bucket, but not others.

Step 3: Complete User Creation

• Name and create the policy.

• Return to the IAM user setup screen.

• Refresh and attach the new policy.

• Click Next and then Create User.

Note: The user will see names of other buckets but cannot access their data or delete them.

Step 4: Install rclone

Fetch and unpack

# apt intall curl -y
# sudo -v ; curl https://rclone.org/install.sh | sudo bash
# curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
# unzip rclone-current-linux-amd64.zip
# cd rclone-*-linux-amd64

Copy binary file

# sudo cp rclone /usr/bin/
# sudo chown root:root /usr/bin/rclone
# sudo chmod 755 /usr/bin/rclone

Install manpage

# sudo mkdir -p /usr/local/share/man/man1
# sudo cp rclone.1 /usr/local/share/man/man1/
# sudo mandb

Step 5: Access Key for IAM user

• Login to AWS console

• Go to IAM > Users

• Select the user

• Under Summary Select create access key

• Select Command Line Interface (CLI)

• Give description for the key

• click Create access key

• Save the Access key and Secret access key

NOTE: If you lose or forget your secret access key, you cannot retrieve it

• Click Done

Step 6: Configure rclone

Setup rclone

# rclone config

Make new remote

No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

Give a name

Enter name for new remote.
name> S3

Choose S3

Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, ChinaMobile, Cloudflare, DigitalOcean, Dreamhost, GCS, HuaweiOBS, IBMCOS, IDrive, IONOS, LyveCloud, Leviia, Liara, Linode, Magalu, Minio, Netease, Outscale, Petabox, RackCorp, Rclone, Scaleway, SeaweedFS, Selectel, StackPath, Storj, Synology, TencentCOS, Wasabi, Qiniu and others
Storage> 4

Choose the S3 provider

Option provider.
Choose your S3 provider.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
1 / Amazon Web Services (AWS) S3
   \ (AWS)
provider> 1

select the option to enter the credentials in the next step

Option env_auth.
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
Choose a number from below, or type in your own boolean value (true or false).
Press Enter for the default (false).
 1 / Enter AWS credentials in the next step.
   \ (false)
 2 / Get AWS credentials from the environment (env vars or IAM).
   \ (true)
env_auth> 1