How to Mount S3 Storage on Windows, MacOS, and Linux¶
Overview¶
You have options for mounting S3 storage buckets:
- OS-specific clients
- Cross-platform clients
Tip: use rclone
for mounting S3 storage onto local systems because it is:
- Cross-platform
- Well-supported
- MIT licensed
Other clients may be better for your use case. Here are some clients you can use:
RClone
(Cross-platform, open-source, free) (How To)[]Cyberduck
(Cross-platform, open-source, free) (How To)[]Mountain Duck
(Cross-platform, closed source, free trail)s3fs
(Cross-platform, open-source, free)Mountpoint for AWS S3
(Linux only, open-source, free)Expandrive
(Windows/MacOS, closed source, free for personal use/non-free subscription)TntDrive
(Windows only, closed source, non-free/free-trial)S3 Drive
(Cross-platform, closed source, non-free/free with limititations)
Multiple systems accessing a s3 bucket simultaneously¶
We recommend only allowing a single system or user to mount an s3 bucket as writable at one time. If two users or systems can both write to the mounted bucket at the same time, both users will see the file as initially opened but only the changes last user to save the file made will be in the saved object. Enabling versioning on your bucket can help in that the changes made to the object will be in a version but manual intervention will be required to merge the divergent changes. One system writing to a bucket with multiple systems reading from that bucket can provide an efficient method of accessing large amounts of data but careful tuning of your cache options should be considered.
Obtaining your s3 credentials¶
In the Rumble Cloud console interface:
- Select S3 Credentials.
- Select your region in the top of the interface (
us-east-2
is shown in the example) - Enter a name for the set of credentials.
- Copy and securely store the displayed Access Key and Secret Key.
Multiple systems accessing an S3 bucket simultaneously¶
Tip: only allow a single system or user to mount an S3 bucket as writable at one time.
If two users' systems can both write to the mounted bucket at the same time, both users will see the file as initially opened but only the changes last user to save the file made will be in the saved object. Enabling versioning on your bucket can help in that the changes made to the object will be in a version, but manual intervention will be required to merge the divergent changes. One system writing to a bucket with multiple systems reading from that bucket can provide an efficient method of accessing large amounts of data, but careful tuning of your cache options should be considered.
Mounting s3 storage via CLI¶
Configure RClone¶
Use rclone
to create a configuration file.
-
Select
n
for new remote. -
Name your new config with a name of your choice (e.g.
rumble
is used in the examples). -
Type
4
for Option Storag (S3 compliant storage). -
Type
4
for Option Provider (Ceph Object Storage \ (Ceph)). -
Type
1
for Option env_auth (Manually enter credentials). -
Type your access key ID for Option access_key_id.
-
Type your secret key for Option secret_access_key.
-
Leave the region empty for Option Region.
-
Type
object.us-east-1.rumble.cloud
,object.us-east-2.rumble.cloud
, orobject.us-west.rumble.cloud
, based on your rumble cloud location for Option endpoint. -
Leave Option location_constraint empty.
-
Type
1
for Option acl (private). -
Type
1
for Option server_side_encryptioin (None). -
Type
1
for Option sse_kms_key_id (None). -
Type
n
for Edit advanced config. -
Type
y
to keep the config if it looks correct. -
Type
q
to quit configuringrclone
.
At this point you should have a new rclone configuration file stored in the default configuration location.
To see where this is, type: rclone config file
:
[MyBucket]
type = s3
provider = Ceph
access_key_id = *YourAccessKey*
secret_access_key = *YourSecretKey*
endpoint =object.us-east-2.rumble.cloud
acl = private
We recommend using rclone for mounting s3 storage onto local systems as it is cross platform, well supported and is MIT licensed, other options may have advantages for your use case.
Mount an S3 bucket on Windows¶
Tip: use rclone along with winfsp, a Windows file system proxy.
- Install
rclone
.
-
Configure
rclone
as described in the "Configuring Rclone" section. -
In order to use the
rclone
mount functionality, you will need to install winfsp.Winfsp
acts as a filesystem proxy betweenrclone
and the Windows system. -
Mount your S3 bucket as a drive in Windows (see the rclone mount documentation for further options).
-
Mount your S3 bucket.
Mount an S3 bucket on Mac OS-X¶
- Install
rclone
.
Tip: Use the official binary from rclone.org. Tools such as Homebrew
have at times changed the default compilation options which can impact your ability to mount.
- Mount your S3 bucket.
Tip: Use rclone nfsmount
here instead of rclone mount
, as it avoids the dependency of a fuse implementation (macFUSE/fuse-t).
Mount an S3 bucket on Linux¶
- Install
rclone
.
Tip: Use your distro's package repository to install rclone
whenever possible.
-
Configure
rclone
as described in the "Configuring Rclone" section. -
Mount your S3 bucket.
Accessing s3 storage via a GUI client¶
Cyberduck GUI client (Windows / MacOS)¶
Cyberduck is a free nag-ware GUI client for accessing s3 storage. You can register your client for a fee to remove the prompts. Cyber Duck Documentation
Cyberduck can be installed from the Windows Store and Mac App Store or you can download the installer.
Click Install.
The initial Cyberduck interface.
You can use a pre-built profile by double clicking them after download, or manually configure Cyberduck (instructions here)
- Rumble US-East-1 Cyberduck profile
- Rumble US-East-2 Cyberduck profile
- Rumble US-West-1 Cyberduck profile
To manually configure Cyberduck:
- Click the Open Connection button.
- Fill in the Server field with
object.us-east-1.rumble.cloud
,object.us-east-2.rumble.cloud
orobject.us-west-1.rumble.cloud
. - Enter your Access Key in the Access Key ID field.
- Enter your Secret Key in the Secret Access Key field.
If you only want to access a specific bucket instead of all your buckets click the More Options drop-down arrow and fill your bucket into the path field.
Cyberduck drag-and-drop:
If the file already exists, you'll be prompted to decide how to proceed.
Mounting s3 storage via GUI¶
Mountain Duck¶
Mountain Duck is a paid GUI client for mounting s3 storage but has a free 14 Day trial after requesting a trial, without requesting a trail or licensing Mountain Duck will only function for 10 minutes. Mountain Duck Documentation
Mountain Duck can be downloaded and installed for Windows and Mac OS-X.
Run the installer.
Click Install.
Allow the install.
Mountain Duck lives in the status bar on macOS and tray area on Windows. If you have previously used Cyberduck, connection profiles will be automatically imported. Otherwise refer to the Mountain Duck User Interface Documentation.