How to Mount S3 Storage on Windows, MacOS, and Linux¶
Overview¶
You have options for mounting S3 storage buckets:
- OS-specific clients
- Cross-platform clients
Tip: use rclone
for mounting S3 storage onto local systems because it is:
- Cross-platform
- Well-supported
- MIT licensed
Other clients may be better for your use case. Here are some clients you can use:
RClone
(Cross-platform, open-source, free)Cyberduck
(Cross-platform, open-source, free)s3fs
(Cross-platform, open-source, free)Mountpoint for AWS S3
(Linux only, open-source, free)Expandrive
(Windows/MacOS, closed source, free for personal use/non-free subscription)TntDrive
(Windows only, closed source, non-free/free-trial)S3 Drive
(Cross-platform, closed source, non-free/free with limititations)
Mount an S3 bucket on Windows¶
Tip: use rclone along with winfsp, a Windows file system proxy.
- Install
rclone
.
-
Configure
rclone
as described in the "Configuring Rclone" section. -
In order to use the
rclone
mount functionality, you will need to install winfsp.Winfsp
acts as a filesystem proxy betweenrclone
and the Windows system. -
Mount your S3 bucket as a drive in Windows (see the rclone mount documentation for further options).
-
Mount your S3 bucket.
Mount an S3 bucket on Mac OS-X¶
- Install
rclone
.
Tip: Use the official binary from rclone.org. Tools such as Homebrew
have at times changed the default compilation options which can impact your ability to mount.
- Mount your S3 bucket.
Tip: Use rclone nfsmount
here instead of rclone mount
, as it avoids the dependency of a fuse implementation (macFUSE/fuse-t).
Mount an S3 bucket on Linux¶
- Install
rclone
.
Tip: Use your distro's package repository to install rclone
whenever possible.
-
Configure
rclone
as described in the "Configuring Rclone" section. -
Mount your S3 bucket.
Multiple systems accessing an S3 bucket simultaneously¶
Tip: only allow a single system or user to mount an S3 bucket as writable at one time.
If two users' systems can both write to the mounted bucket at the same time, both users will see the file as initially opened but only the changes last user to save the file made will be in the saved object. Enabling versioning on your bucket can help in that the changes made to the object will be in a version, but manual intervention will be required to merge the divergent changes. One system writing to a bucket with multiple systems reading from that bucket can provide an efficient method of accessing large amounts of data, but careful tuning of your cache options should be considered.
Configure RClone¶
Use rclone
to create a configuration file.
-
Select
n
for new remote. -
Name your new config with a name of your choice (e.g.
rumble
is used in the examples). -
Type
4
for Option Storag (S3 compliant storage). -
Type
4
for Option Provider (Ceph Object Storage \ (Ceph)). -
Type
1
for Option env_auth (Manually enter credentials). -
Type your access key ID for Option access_key_id.
-
Type your secret key for Option secret_access_key.
-
Leave the region empty for Option Region.
-
Type
object.us-east-1.rumble.cloud
,object.us-east-2.rumble.cloud
, orobject.us-west.rumble.cloud
, based on your rumble cloud location for Option endpoint. -
Leave Option location_constraint empty.
-
Type
1
for Option acl (private). -
Type
1
for Option server_side_encryptioin (None). -
Type
1
for Option sse_kms_key_id (None). -
Type
n
for Edit advanced config. -
Type
y
to keep the config if it looks correct. -
Type
q
to quit configuringrclone
.
At this point you should have a new rclone configuration file stored in the default configuration location.
To see where this is, type: rclone config file
: