What is a S3 Bucket
An Amazon S3 bucket is a public cloud storage resource available in Amazon Web Services (AWS) Simple Storage Service (S3) platform. It provides object-based storage, where data is stored inside S3 buckets in distinct units called objects instead of files.
Amazon S3 buckets are similar to file folders and can be used to store, retrieve, back up and access objects. Each object has three main components — the object’s content or data, a unique identifier for the object and the descriptive metadata, including the object’s name, URL and size.
An object must exist within a bucket, as it can’t exist alone. Each Amazon account could have hundreds of buckets, each containing numerous objects.
A great source of information can be found here for a more indepth examples and such.
Now that we got that out of the way, how can I mount this to my server and use this as a backup or a storage drive?
My focus with this will be on a Ubuntu/Debian system, Fedora systems are 90% the same, just with yum/dnf instead of apt/nala. * I have not touched Arch so I have no examples/howto’s.
**** ASSUMPTIONS **** You already have your bucket name, access key, secret access key, and endpoint.
With my experience I have found I use the S3 Buckets for two reasons.
1. Root level backups * my servers as an example.
2. My Desktop as a storage/backup folder(s).
So how do we do this magic? Well first thing we have to do is connect to our server.
* I am going to use the server 1st as the example, and then move the processes over to my desktop example.*
So lets connect to our server, and run some updates, install the S3 File System, configure, and automate the mount on reboots.
** I am going to assume you are already connected via SSH with your client of choice.
The first thing we need to do is install the S3FS system.
sudo apt install s3fs
** Fedora uses s3fs-fuse so **
sudo dnf install s3fs-fuse
Before we forget, we need to create the folder we want to mount.
mkdir -p /backupsNow we have to create a few files.
** I use nano, but you can use your editor of choice.
sudo nano /etc/passwd-s3fsNow paste your Key and Access Key, in the following format. Access Key : Secret Key NO SPACES
D45tYGyed:wMOx4P6YTqv5A <<—– Fake Key for Demo
So your passwd-s3fs file should look like, and contain nothing more then the following.
DKROVu437n:mINVrHBIjCttFjCNow once saved, we need to change the file permission.
sudo chmod 600 passwd-s3fsNow we have to edit the Fuse config to allow connections.
sudo nano /etc/fuse.confUncomment user_allow_other then save your file.
Now lets make this auto mount on a reboot.
sudo nano /etc/fstabNow add the following line to your fstab, editing for your needs.
s3fs#bucketname /backups/foldername/ fuse _netdev,allow_other,url=https://ee2-16.comNow lets mount the folder.
sudo mount -aNow lets check if things are mounted.
Look at that! We have a bucket mounted and set for data now!
********** DESKTOP USER SETUP **********
Now lets do the same thing, but as your user on your desktop.
Assuming your on your desktop now, we need to fire up our editor of choice, this can be gedit, kate, or nano, vi, vim, you name it. I personally use nano for this.
Now everythingish is the same, except for the location of your password file, and a different way to mount the bucket.
So instead of saving your passwd-s3fs file in /etc/ we will now save it in your /home/$USER folder.
Same content, same file permission.
Now however instead of the above we mount the drive like this.
sudo s3fs YOURBUCKETNAME /home/YOURUSERNAME/MOUNTPOINT -o passwd_file=/home/YOURUSERNAME/.s3fs-passwd -o allow_other -o uid=YOURUID,gid=YOURGID -o url=https://YOURURL
As you can now see the folder is mounted.
Thats It!! You now have your bucket mounted as a backup folder on your server, and then you can either mount that same bucket on your desktop for easy access to that backup, and you can also create another bucket and mount it as your backup, or your shared folder, or your personal folder, what have you.
