Ultimate Guide to S3 Transfer Acceleration (& the #1 Alternative)

Amazon S3 Transfer Acceleration is a useful feature for companies looking to speed up content transfers to and from S3 buckets. It relies on Amazon CloudFront and the Amazon Web Services (AWS) backbone networks to route traffic and reduce data transfer variability, which makes it a good choice for:

• Applications that use Amazon S3 buckets and have users distributed over the globe.

• Companies that have on-prem apps in distributed locations and use the S3 API to store data in a centralized bucket.

• Any other organizations that regularly transfer gigabytes (or terabytes) of data across continents.

But while useful, this feature also has some downsides. Mainly:

• Its transfer speed is limited due to its point-to-point transfer model. 

• It only works in the context of S3, so you can’t use it to move data between other AWS services or other cloud providers.

• It can make it difficult to estimate your cloud costs (and easy to overspend), especially if you’re already using other transfer, sync, and replication solutions like S3 Replication, Global Accelerator, and so on.

This guide covers everything you need to know to get started with S3 Transfer Acceleration — including its benefits, pricing, configuration, and downsides.

In the second part of this guide, we also show you how Resilio Connect — our high-performance transfer and replication software — can be an ideal alternative to S3 Transfer Acceleration for companies that want to avoid the downsides we just described. 

Specifically, you’ll learn how Resilio Connect can help your organization:

• Accelerate data transfer speeds across any AWS region or service, irrespective of distance or latency. Resilio’s unique P2P architecture and UDP-based WAN optimization technology make it perfect for transferring files of any size across any distance. You may have a few or a thousand endpoints you need to replicate. Either way, Resilio overcomes latency and packet loss associated with TCP to provide predictable performance end-to-end, across multiple endpoints located anywhere.

• Gain location flexibility. Resilio Connect agents can be located anywhere. They can be in remote AWS locations, across multiple regions, or even multiple cloud providers. Resilio enables your users to access current data from any location at predictable speeds. Our software also supports other cloud computing providers (including Azure, GCP, and Backblaze), on-prem storage, and hybrid cloud environments (including NAS, DAS, and SAN).

• Gain granular control and visibility into data transfers from a single place. Regardless if you’re moving data in AWS, other cloud providers, or on-prem storage, Resilio’s Central Management Console lets you control every aspect of the data transfer process from a single place — including file priorities, packet size, data hashing, bandwidth utilization, notifications, and more. 

Companies like 2K Games, Microsoft, and Cisco have used Resilio to achieve high-speed, reliable data transfer, sync, and replication. Click here to book a demo with our team and learn how Resilio can help your organization as well.

S3 Transfer Acceleration: Benefits, Pricing, Configuration, and Limitations

S3 Transfer Acceleration (S3TA) is a bucket-level feature that helps speed up long-distance file transfers. 

When enabled, this feature starts routing data through Amazon CloudFront — AWS’ content delivery network (CDN) that has over 450 Points of Presence in various edge locations over the globe. 

S3TA is designed to use various network protocol optimizations and route traffic through these locations over an optimized network path. However, transfers can only occur between two endpoints at a time due to S3TA’s point-to-point nature. As a result, the transfer speed is always limited by the throughput between each endpoint.

Overall, S3TA helps organizations achieve four main benefits:

1. Faster long-range data transfers. The longer the distance between an app and an S3 bucket, the more useful S3TA is. Again, the speed is still limited because of the point-to-point transfers. 
   
   
2. Shorter distance to AWS S3 buckets. Uploads and downloads are automatically routed to the edge location that’s closest to your app, which reduces physical distance.
   
   
3. Maximized bandwidth utilization. S3TA can fully utilize transfer bandwidth, which reduces the impact distance has on throughput.
   
   
4. Reduced network variability. It’s typical for network speeds to vary wildly, especially when transferring data between regions (e.g. between US East and US West). S3TA helps overcome this issue via AWS’ network optimizations. 

You can estimate how much time S3TA can save you with this speed comparison tool. It uses multipart uploads to transfer a file from your browser to various Amazon S3 regions with and without S3TA.

S3 Transfer Acceleration Pricing

Like most other AWS features, S3TA comes with its own pricing. This pricing differs depending on the S3 region and whether you’re transferring data in or out of S3. 

Here’s S3TA pricing as of August 2023:

• For data transfer into S3 from the Internet:
  • If the transfer is accelerated by an edge location in the US, Europe, or Japan — $0.04 per GB.
  • If the transfer is accelerated by any other edge location — $0.08 per GB.

• For data transfer out from S3 to the Internet — $0.04 per GB, regardless of the edge location that’s accelerating the transfer.

• For data transfers between S3 and other regions — $0.04 per GB, regardless of the edge location that’s accelerating the transfer. Note: In this case, there would also be separate charges for moving data between regions. You can learn more about the topic in our article on reducing AWS cross-region data transfer costs.

It’s also worth noting that these fees apply only for accelerated transfers. If you enable S3TA on a bucket but the transfer of an object or set of objects wasn’t accelerated, there are no additional acceleration charges.  

How to Set Up S3 Transfer Acceleration 

In order to use S3TA, you need to enable transfer acceleration on your S3 bucket. You can do this by opening the S3 console, choosing the bucket, and clicking on “Properties” → “Transfer Acceleration” → “Enable” → “Save Changes”.

Note: The bucket name must be DNS-compliant and not contain any periods (“.”).

You can check out this AWS tutorial if you want more details on how to do this via the AWS CLI, AWS SDKs, or REST API. 

Once enabled, this feature should start accelerating transfers (this may take up to 20 minutes after the setup). 

However, S3TA doesn’t just automatically accelerate all transfers. Instead, after you upload an object to the bucket, S3TA checks if it can accelerate the transfer more than the regular S3 transfer speed. If it determines that it’s not likely to deliver a faster speed, it won’t accelerate the transfer and you won’t be charged a S3TA fee.

Lastly, if you want to access a bucket that’s enabled for transfer acceleration, you have to:

• Use the endpoint bucketname.s3-accelerate.amazonaws.com

• Use the dual-stack endpoint bucketname.s3-accelerate.dualstack.amazonaws.com to connect to the enabled bucket over IPv6

The Limitations of S3 Transfer Acceleration

S3TA has three limitations that can’t be completely avoided as they stem from the broader AWS ecosystem and architecture:

1. Limited transfer rate. S3TA is a point-to-point solution meaning it can only transfer data between two endpoints at a time. As a result, the speed of your transfer is limited by the throughput between each endpoint (the lowest bandwidth endpoint is the limiting factor). Plus, each endpoint is a single point of failure, so one server going down can stop data transfer, sync, or replication across the entire environment.
   
   
2. Complexity. While easy to set up, S3TA is only a small part of the equation when it comes to transferring data quickly and efficiently. There’s a whole range of additional AWS features and services that companies have to employ in their day-to-day — like S3 Replication, AWS Direct Connect, AWS Global Accelerator, Snowball, IAM, AWS Lambda, and many more. This often makes it very difficult to find the right feature or service for your use case and employ it in the most optimal way.
   
   
3. Pricing unpredictability. As we said, S3TA comes with its own pricing on top of the regular AWS egress fees, data transfer charges, and costs for other AWS services. Due to the complexity we just discussed, it’s very common for companies to get unexpectedly large AWS bills as estimating every single charge in advance can be near impossible.
   
   
4. Lack of flexibility. S3TA, like all other proprietary cloud services, is built to keep users within the AWS cloud. As a result, moving data across AWS regions is easy (if expensive) but doing so across other cloud providers is impossible out of the box. This leads to vendor lock-in and forces multi-cloud companies to invest in 3rd-party software or spend resources on building their own tools for multi-cloud data transfers. Also, S3TA is not available in all AWS regions, so you can’t use it if your S3 bucket is outside of the available ones.

In the next sections, we’ll show you how Resilio Connect can help you overcome these limitations while achieving industry-leading data transfer, replication, and sync speeds.

Resilio Connect: The Fast, Reliable, and Flexible Alternative to S3 Transfer Acceleration

Resilio Connect is a software-only file transfer and real-time replication solution utilizing a P2P architecture and proprietary UDP-based WAN optimization technology to deliver the fastest transfer speeds across multiple locations over any distance. 

Brands like Blockhead, MixHits Radio, and Mercedes-Benz use our software to:

• Achieve near-instant transfer speeds across any cloud region, on-prem storage, and hybrid cloud setups.
• Handle massive data workloads and move large volumes of data over many endpoints.
• Manage each aspect of data transfers and syncs from a single place. 

Resilio Connect is a software-only cloud-independent solution that can be deployed anywhere: on your existing IT infrastructure or in any cloud. 

There’s no need to buy new hardware (such as servers or storage). You can deploy Resilio directly on your endpoint devices — from mobile phones to desktops to servers to cloud instances.

Here are five key benefits of using Resilio Connect for your data transfer and replication needs:

1. Automate Data Transfer and Replication at Wire Speeds, Across Any Distance

S3TA, like most conventional file transfer and replication solutions, uses a standard point-to-point model to transfer data. This model comes in one of two flavors:

1. Client-server: Servers in an environment are separated into clients and hubs. The clients can only share data with the hub — while the hub can transfer data across all servers in the environment; it’s also a single point of failure and performance bottleneck because all transfers need to flow through it.
   
   
2. “Follow-the-sun”: Data transfer can only occur sequentially between 2 servers. For example, Server 1 can transfer data to Server 2; when the job completes, Server 2 can then transfer to Server 3, and so on.

In both cases, transfers are limited to at most 2 servers (or endpoints), each of which is a single point of failure and a potential bottleneck. As you add more endpoints or increase the volume of data, transfer speeds take a hit.

Unlike most conventional solutions, Resilio Connect uses a unique P2P (peer-to-peer) replication architecture to ensure the fastest-possible transfer times. 

Combined with UDP-based WAN optimization, Resilio overcomes latency, packet loss, and client-server bottlenecks, as it can rapidly transfer or synchronize files across multiple endpoints located anywhere.  

First, Resilio’s P2P architecture lets all servers share data with each other. In this model, every server can act as a hub, which results in built-in availability, data protection, and data integrity, as there’s no reliance on a single device. And the more endpoints you add, the better our software performs.

And to increase data transfer speeds even more, Resilio Connect uses a process called file chunking to separate files into pieces and transfer them independently of each other. 

The P2P replication and file chunking result in transfer speeds that are 3-10x faster than traditional point-to-point solutions. Plus, it also makes Resilio one of the few solutions that can replicate and sync data in any direction — one-to-one, two-way, one-to-many, many-to-one, and N-way.  

Additionally, Resilio Connect maximizes transfer speeds and overcomes the impact of latency and packet loss across any network.

Thanks to our proprietary UDP-based transfer protocol called Zero Gravity Transport™ (ZGT), Resilio can optimize transfers over WANs (wide area networks). Our software does this by using a congestion control algorithm to calculate the ideal send rate by periodically probing the time it takes for the destination to receive a packet.

The ZGT protocol also:

• Avoids network overload by creating a uniform packet distribution over time.
• Retransmits lost packets once per Round Trip Time to reduce unnecessary retransmissions.
• Sends packet acknowledgements for a group of packets, instead of each individual packet.

Resilio’s WAN optimization technology improves upload speed and replication speed over any distance. 

With Resilio Connect, you get built-in WAN optimization for all of your data transfer needs such as ingest, distribution synchronization, and CRR replication, regardless of your cloud region, cloud provider, and storage solutions.

If you want to see how much time you can save with our technology, check out this calculator on our website.

2. Complete Deployment, Storage, and Cloud Flexibility

Unlike AWS’ proprietary solutions, Resilio Connect doesn’t lock you into any cloud ecosystem. 

Instead, our software builds on open standards, open file formats, and a multi-cloud architecture. You can use it to ingest, move, and replicate data across any cloud provider, service, or region. 

For example, you can:

• Ingest your data into AWS and transfer it across S3 buckets (including different storage classes). Our software overcomes transfer latency for buckets in the same or different regions.

• Transfer and sync data across other cloud providers — including Azure, GCP, Wasabi, Backblaze, and many more — from the Central Management Console.

• Use any mix of hybrid and on-prem storage, like DAS, NAS, or SAN.

And because Resilio Connect is a flexible, software-only solution, it can be deployed on any infrastructure. This means you can continue using your:

• Servers, networks, and desktops. Our software is compatible with all popular operating systems like Mac, iOS, Android, Linux, Unix, and Windows, as well as virtualization platforms like VMWare and Citrix.

• Storage solutions like NAS devices, hard drives, and SSDs. 

In short, Resilio Connect gives you the flexibility to build in any cloud, on-prem, or hybrid cloud environment, without limiting you to specific cloud vendors or other service providers. Besides saving you time and resources, this also makes data management significantly easier in multi and hybrid cloud scenarios.

3. Granular Controls and Efficiency

It’s essential to have a simple, user-friendly way to manage each aspect of your data transfers, especially when dealing with large files and distributed endpoints. This can help you make the process as efficient as possible and avoid big cloud bills due to unnecessary data movement.

As we said, this is an area where AWS users often struggle due to its sheer complexity. To avoid this issue, Resilio Connect gives you a Central Management Console for setting up, managing, and monitoring your data transfer, sync, and replication jobs.

For example, you can use the Central Management Console to configure:

• Replication, transfer, and sync jobs.
• Key parameters like buffer size, disk IO, and more.
• Real-time notifications of file transfer progress (sent via email or webhooks).

The console can be stored on any Amazon EC2 and in any Windows or Linux instance (virtual or physical), in another cloud provider, and on-premises. And because Resilio Connect is a cloud-agnostic solution, you can use the console to manage data transfers between different clouds, e.g., AWS, Azure, and GCP.

The console also gives you plenty of options for monitoring your work. For example, you can use it to review all scheduled and executed jobs, collect transfer logs, troubleshoot errors in real-time, track metrics like the total number of files, bytes, maximum speed, and much more. 

Additionally, Resilio comes with other features that can help you minimize your costs by making data transfers as efficient as possible. 

For example, our software:

• Lets you selectively download, sync, and cache files as needed. You can also store frequently accessed files locally, so you don’t download them from the cloud every time. As a result, you can minimize egress costs, which are among the biggest contributors to many cloud bills.

• Minimizes data transfers during synchronization by only replicating the changed portion of files. This helps you reduce the overall amount of data being moved, which can save you lots of money when transferring files between different regions.

• Allows you to pin traffic to the optimal network. Thanks to our Smart Routing feature, you can keep your traffic on the AWS network or move it to a LAN (Local Area Network), instead of on a more expensive WAN.

• Gives you a single, unified view of data transfer, replication, and sync jobs across any cloud region, cloud provider, or on-prem environment. This makes it much easier to understand exactly what data you’re transferring without having to jump between four or five tools and services in different clouds.

Lastly, Resilio Connect doesn’t require any new hardware or lengthy data migrations. You can start using our solution without spending tons of time and money on buying new machines and training your team on them.

4. Resilience and Fault Tolerance

Because of its P2P architecture, Resilio Connect doesn’t have a single point of failure. 

If one device goes down, Resilio Connect can automatically route around the outage and retrieve data from the nearest available device. End-users can access files from the server closest to them, so data can always be retrieved as fast as possible.

This makes our software suitable for all kinds of disaster recovery (DR) scenarios, including:

• Cold DR 
• Warm-site DR
• Offsite copy
• Hot-site DR

Put simply, Resilio Connect is perfect for organizations that need to meet sub-five-second RPOs (Recovery Point Objectives) and RTOs (Recovery Time Objectives) within minutes of an outage. 

Note: For a deeper dive into this topic, check out our article on the 5 best hot-site disaster recovery solutions.

5. Built-in Security

Security is one of the biggest concerns for companies storing and moving data across cloud regions and providers. However, many transfer and replication solutions don’t have high-level security features out of the box, which forces companies to purchase separate security software.

In contrast, Resilio Connect comes with enterprise-grade security features natively, so there’s no need to invest in 3rd-party solutions. These features include:

• End-to-end AES 256 encryption, which encrypts data at rest and in transit.

• One-time session encryption keys, which protect each individual session.

• Permission controls, which give you granular control over who gets access to specific files and folders. 

• Cryptographic data integrity validation, which guarantees files arrive at their destination uncorrupted.

• Mutual authentication, which ensures our software only delivers files to designated endpoints.

• Data immutability, which means Resilio Connect stores immutable copies of data in the public cloud, protecting you from ransomware and data loss.

Optimize Data Transfer Speeds Across Regions, Services, and Clouds with Resilio Connect

Resilio Connect is an efficient and reliable S3 Transfer Acceleration alternative for enterprise organizations looking to transfer, sync, and replicate large volumes of data across lots of endpoints as fast as possible.

Our software’s unique P2P architecture, WAN optimization technology, and a myriad of useful features make it:

• Extremely fast and efficient. Every server in your environment can take part in the data transfer process.

• Fully flexible. You can deploy it on your existing environment, ingest data into any cloud, quickly move data between cloud providers (like AWS, Azure, GCP, Backblaze, and more), and use any type of storage, including NAS, DAS, and SAN.

• Simple to manage. You can control all aspects of data transfer and access from a single interface (even in a multi or hybrid cloud scenario).

• Cost-efficient. You have full control over data transfers and don’t need to invest in new hardware or expensive data migrations.

• Highly resilient, fault-tolerant, and ideal for disaster recovery scenarios.

• Secure. Thanks to AES 256 encryption among several other security features that have been verified by 3rd-party security experts.

Ready to experience these benefits in your organization? Click here to schedule a demo with our team.