Use FTP, FTPS, and SFTP

Rumble Cloud object storage natively supports both S3 and Swift object storage protocols for robust management of your storage needs.

Protocol	Description	Key Features	Common Use Cases
S3 (Simple Storage Service)	Amazon's object storage protocol that has become an industry standard	- RESTful API - Strong consistency - Rich metadata support - Extensive SDK support - Versioning support	- Web applications - Backup and archive - Big data analytics - Content delivery
Swift (OpenStack Object Storage)	OpenStack's object storage protocol	- RESTful API - Eventual consistency - Multi-region support - Built-in CDN capabilities - Container-based organization	- Cloud storage - Content distribution - Data archiving - Media storage

Sometimes when accessing certain projects and legacy systems you may need to use other protocols such as FTP, FTPS, or SFTP.

Protocol	Description	Security	Ports	Best Use with Object Storage
FTP (File Transfer Protocol)	Traditional file transfer protocol	- No encryption - Passwords sent in plain text - Not recommended for public internet use	- 20 (data) - 21 (control)	- Legacy system integration - Internal network transfers - VPC-only deployments
FTPS (FTP over SSL/TLS)	Secure version of FTP using SSL/TLS encryption	- Encrypted data transfer - Encrypted authentication - Certificate-based security	- 989 (data) - 990 (control) - Additional ports for passive mode	- Secure file transfers - When FTP compatibility is required - Public internet transfers
SFTP (SSH File Transfer Protocol)	Secure file transfer protocol built on SSH	- Encrypted data transfer - Encrypted authentication - Key-based authentication support	- 22 (single port)	- Modern secure transfers - Automated transfers - Public internet transfers

In the article you'll learn how to setup a S3 gateway virtual machine (VM) that enables direct transfer of files to S3 through SFTP, FTPS, or FTP. These steps could also be used for other generic remote file protocol (such as NFS, SMB, etc), but with additional limitations.

Because this process mounts the S3 bucket as a directory under Linux using the S3FS utility, your object storage may look a lot like a file system. But since object storage is not designed to be a file system, not all operations will work as expected. The process described here will work well for transfering and storing files, but you should generally avoid more advanced operations like in place editing of files.

In this document we will outline a process using an Ubuntu 24.04 VM with S3FS. Other approaches using other operating systems and tools could be used equally well. For example, the same affect could be achieved with Windows using the rclone mount tool.

Setup a basic server and S3 bucket

1. Create a Rumble Cloud account, and purchase at a plan that includes at least 1 vCPU and 1 TB of Object Storage
2. Build an Ubuntu 24.04 VM using the default Ubuntu 24.04 image available no Rumble Cloud.
   • You can use any flavor. We used s1a.micro in our example.
   • Note that smaller VM sizes have lower network egress limits in Rumble Cloud. Even the smallest VM's can trasmit up to 500 mbps. However, if you need more throughput, please upgrade to a larger VM.
3. You must assign a security group to the VM that allows incoming access for the services that you want to create a gateway for. In these example, we assume you have opened at least port 22 for SSH and SFTP access. If using FTP, TCP ports 20 and 21 at least must be open. Higher TCP ports may also be required for FTP.
4. Create a Bucket on Rumble Cloud.
5. Create a set of access keys on Rumble Cloud.

Setup S3FS on the server, and mount the bucket

1. Install S3FS
   Bash

   sudo apt -y update
   sudo apt -y upgrade
   sudo apt install -y s3fs
   

2. Configure S3FS
   Bash

   echo "ACCESS_KEY:SECRET_KEY" | sudo tee /etc/passwd-s3fs
   sudo chmod 600 /etc/passwd-s3fs
   

3. Create a directory to mount your S3 storage on
   Bash

   sudo mkdir -p /opt/s3storage
   

4. Add the following to your /etc/fstab file, replacing "BUCKET_NAME" with the name of the bucket you created in an earlier step.

   • If your storage is in us-east-1, use:
     Bash

     s3fs#BUCKET_NAME /opt/s3storage fuse _netdev,allow_other,passwd_file=/etc/passwd-s3fs,url=https://object.us-east-1.rumble.cloud/ 0 0
     

   • If your storage is in us-east-2, use:
     Bash

     s3fs#BUCKET_NAME /opt/s3storage fuse _netdev,allow_other,passwd_file=/etc/passwd-s3fs,url=https://object.us-east-2.rumble.cloud/ 0 0
     

   • If your storage is in us-west-1, use:
     Bash

     s3fs#BUCKET_NAME /opt/s3storage fuse _netdev,allow_other,passwd_file=/etc/passwd-s3fs,url=https://object.us-west-1.rumble.cloud/ 0 0
     

5. Reload you systemctl to make sure fstab changes will take affect

   Bash

   sudo systemctl daemon-reload
   

6. Mount the storage

   Bash

   sudo mount /opt/s3storage/
   

Test SFTP access

The default Rumble Cloud Ubuntu image supports SFTP out of the box, so at this point you are ready to use your gateway VM for SFTP access.

1. First verify that you bucket has no files in it.

2. Create a test file

   Bash

   echo "This is a test file" > test.txt
   

3. Transfer the test file to
   Bash

   $ sftp ubuntu@YOUR_IP
   Connected to YOUR_IP
   sftp> cd /opt/s3storage
   sftp> put test.txt
   Uploading test.txt to /opt/s3storage/test.txt
   test.txt                                                                                                                                                      100%   20     0.5KB/s   00:00
   sftp>
   

4. Verify the file has been successfully transferred to your object storage on Rumble Cloud

Using FTP and FTPS

Rumble Cloud strongly recommends against using FTP on a public internet connection. FTP sends your password in plain text. If it is necassary, it should only be used within your own VPC with private IP addresses. Setting up a VPC is out of the scope of this document. However, the S3 gateway VM procedures that we are descring here will work for a VM setup on a public IP address or one within your VCPC. FTPS can be run securely on the internet, but it is more difficult to configure, so we would recommend SFTP or direct S3 for direct access via the public internet.

FTP

1. Install vsftpd
   Bash

   apt install vsftpd
   

2. Update /etc/vsftp to enable writing files by uncommenting the line write_enable=YES
3. Restart vsftpd daemon
   Bash

   systemctl restart vsftpd
   

4. FTP access requires a user with a password. Next create a user for ftp access, and add a password.
   Bash

   useradd -b -s /bin/bash s3gwftp
   passwd s3gwftp
   

5. Test your new FTP server connection. Here is an example from an Ubuntu that uploads a file to the directory with your S3 bucket mounted. This should verify that your FTP access is setup. This assumes that you have the standard ftp command line client available on your command prompt. You can do a similar test with any FTP client available on your system if you don't have the "ftp" command line client.

   Bash

   $ echo "Unencrypted FTP" > test.txt
   $ ftp s3gwftp@YOUR_IP
   Connected to YOUR_IPp..
   220 (vsFTPd 3.0.5)
   331 Please specify the password.
   Password:
   230 Login successful.
   Remote system type is UNIX.
   Using binary mode to transfer files.
   ftp> cd /opt/s3storage
   250 Directory successfully changed.
   ftp> put test.txt
   local: test.txt remote: test.txt
   229 Entering Extended Passive Mode (|||53701|)
   150 Ok to send data.
   100% |*************************|     6       96.05 KiB/s    00:00 ETA
   226 Transfer complete.
   6 bytes sent in 00:00 (8.45 KiB/s)
   ftp> exit
   221 Goodbye.
   $
   

6. Verify that the file was uploaded to your S3 bucket.

FTPS

After following the above instructions for FTP, you can optionally enable FTPS by doing the following.

1. You will need to have a TLS certificate to use FTPS. In this example, we use the default Ubuntu self signed certificate. For additional security, you can purchase one or generate one from a free service such as Let's Encrypt.

2. Edit /etc/vfsftp.conf:

3. Change ssl_enable=NO to ssl_enable=YES

4. Add the line force_local_logins_ssl=YES

5. Restart vsftpd daemon

   Bash

   systemctl restart vsftpd
   

6. At this point, your FTP server should only accept FTPS connections.
7. Test your new FTPS server connection. Here is an example from an Ubuntu machine that uploads a file to the directory with your S3 bucket mounted. This should verify that your FTPS access is setup. Note that this example installs the "lftp" program as the standard ftp client does not support FTPS. You can do a similar test with any client that supports FTPS that is available for your system.
   Bash

   $ sudo apt install lftp
   $ echo "Encrypted FTPS" > test.txt
   $ lftp s3gwftp@YOUR_IP
   Password:
   lftp s3gwftp@YOUR_IP:~> set ssl:verify-certificate no
   lftp s3gwftp@YOUR_IP:~> cd /opt/s3storage
   cd ok, cwd=/opt/s3storage
   lftp s3gwftp@YOUR_IP:/opt/s3storage> put test.txt
   5 bytes transferred
   lftp s3gwftp@YOUR_IP:/opt/s3storage> exit
   $
   

Conclusion

Using the above process you should be able to easily provide SFTP, FTP, or FTPS access to your S3 Buckets for systems that require this access. This process will work well with these protocols since they will normally only upload or download full files. Note that you could also use a similar process to provide access via NFS, SMB, or any other file sharing system. However, in those case you will have to be more cautious to try to limit yourself to operations that add or delete full files. Trying to do more advanced files operations such as online editing directly on the files may result in unexpected results.

Note that we used Ubuntu 24.04 in this example, but a similar process could be used with any other operating system including Windows based OS's, other linux distributions, etc.