WAN acceleration — or WAN optimization — is a broad category with a long history. There are two types of WAN optimization solutions:
- SD-WAN optimization appliances (or virtual appliances)
- File-based solutions that may be UDP- or UDT-based (UDT is simply an app built on top of UDP.)
Historically, SD-WAN solutions like Silver Peak, Riverbed, and others are designed to keep unnecessary traffic off the network. Chatty apps can be cached and deduplicated or compressed at the edge, keeping traffic off the WAN.
By contrast, UDP-based WAN optimization tools like Aspera, Signiant, and FileCatalyst were designed to replace TCP to obtain full utilization of bandwidth by pushing data as efficiently as possible across the network. Early implementations such as UDP blasters failed due to poor quality of service (QoS). Stories persist about “Aspera taking the entire network down” or some other UDP-based product doing so if not configured correctly.
The bigger issue today is that there’s exponentially more data to cache, move, and synchronize across networks of varying bandwidth and latency. The data flows are more complex. Customers need to both make better use of network capacity and move or synchronize large amounts of data.
Enter Resilio Connect: We have a solution that both reduces network traffic through advanced deduplication and transport efficiency and overcomes latency and loss through a proprietary WAN optimization protocol. Moreover, unlike both SD-WAN and conventional UDP solutions, Resilio has no single point of failure (SPOF). Plus, Resilio can scale out by simply adding nodes (agents) to a cluster (or swarm). The net effect is the ability to reliably and efficiently transfer or synchronize payloads of any size across any network, irrespective of latency and loss.
Want to see Resilio perform blazing-fast transfers over a WAN? Or discuss your use case? We’d be happy to oblige. Schedule a demo here.
Or, feel free to continue reading: We’ll discuss several WAN optimization software solutions, starting with Resilio Connect — our file synchronization software system that utilizes UDP-based WAN optimization.
Resilio Connect is a software-only, file-based WAN optimization solution that uses a proprietary UDP-based WAN acceleration protocol to fully utilize your bandwidth and optimize data transfer over any IP network — including VSAT, cell (3/4/5G), and Wi-Fi.
Unlike other solutions that use UDP-based WAN optimization, Resilio Connect also replicates data in real time using a P2P (peer-to-peer) replication architecture. P2P replication enables Resilio to replicate data in any direction (one-way, two-way, one-to-many, many-to-one, and N-way) so you can rapidly sync multiple endpoints, eliminate single points of failure, and transfer files of any size or number. We’ve used this horizontal scalability to reach speeds of 100+ Gbps over the WAN.
These WAN optimization capabilities make Resilio Connect the perfect solution for:
- Media and VFX production
- Synchronizing large files with public cloud object storage
- Active-Active High Availability for disaster recovery
- Accelerating software build distribution and CI/CD test cycles
- Data ingestion of machine learning datasets into a GPU public cloud
- Migrating data between storage solutions or data centers
- Replicating distributed file systems
- Data center consolidation and transitioning storage to the cloud
- Filling data lakes
Companies like 2K Games, Cisco, Match.com, Exxon, Mercedes-Benz, Blizzard, and more use Resilio Connect to achieve blazing fast synchronization that can reach over 100 Gbps. For a full demonstration of Resilio’s features and how it can help you optimize WAN transfer for your applications, schedule a demo.
Resilio Connect: UDP-Based WAN-Optimized Transport
Resilio Connect uses P2P architecture and proprietary, UDP-based WAN optimization technology to deliver the fastest replication speeds in the industry (we’ve achieved speeds of up to 100 Gbps per server).
While legacy WAN optimization solutions may suffice for some use cases, Resilio is a far superior solution for time-sensitive workflows and organizations that need to quickly and reliably sync big data payloads (large files, large numbers of files) to multiple endpoints.
Resilio is also superior to other replication solutions that use UDP-based WAN optimization, such as Signiant and Aspera, which can only replicate data point-to-point (i.e., only between two endpoints at a time). Resilio’s P2P replication architecture means that every device with a Resilio agent on it can engage in replication simultaneously. This enables you to replicate data in multiple directions, sync multiple endpoints simultaneously (and sync your entire environment much faster than other solutions can), and eliminate single points of failure.
Proprietary WAN Optimization Protocol: Zero Gravity Transport
Resilio Connect optimizes transfer over WAN networks using a proprietary UDP-based protocol called Zero Gravity Transport™ (ZGT).
ZGT optimizes WAN transfer by intelligently analyzing the underlying network performance and measuring packet loss, latency, and throughput over time. It automatically adjusts transfers to maximize utilization of the internet connection and enhance application performance over any network.
To achieve fast, consistent transfer speeds, ZGT uses:
- A congestion control algorithm: ZGT’s congestion control algorithm constantly probes the RTT (Round Trip Time) in order to identify and maintain the ideal data packet send rate.
- Bulk packet transfer: To create a uniform packet distribution over time, the sending device sends file packets periodically with a fixed packet delay.
- Interval acknowledgments: Other transfer protocols require the receiving device to acknowledge every packet receipt. In contrast, ZGT uses interval acknowledgments for groups of packets to enhance transfer speed. Each acknowledgment also contains information about lost packets.
- Delayed retransmission: ZGT retransmits lost packets in groups once per RTT in order to decrease unnecessary retransmissions.
Learn more from our whitepaper on Zero Gravity Transport.
Resilient Transfer
Resilio Connect ensures your files reliably reach their destination in any situation.
In the event that a transfer is interrupted (via a downed network, hardware failure, etc.), Resilio Connect can perform a checksum restart and resume the transfer where it left off once your system is up and running again. This ensures you’re not replicating the same data all over again — saving time and reducing the strain on your network, which is essential when transferring bulk data.
If one of your devices goes down due to a hardware failure or downed network, ZGT can dynamically reroute around the outage to ensure files are still replicated to their destination endpoint.
On disconnected networks, you can communicate with peers on a remote site, then efficiently and intelligently sync changes.
Superior Edge Deployment
Resilio Connect can ingest and sync data from the edge utilizing any type of network connection, including IP connections, VSats, cell (3g, 4g, 5g), and Wi-Fi.
ZGT optimizes data transfers at the edge of networks by seamlessly adjusting to low-capacity links and providing optional data compression methods for metered connections.
This makes Resilio an optimal solution for:
- Syncing branch offices located in low-connectivity areas.
- Deploying security patches and system updates to your fleet of vehicles and vessels.
- Collecting data from a fleet of geographically distributed vehicles or remote sites.
- Quickly distributing time-sensitive operational data (maps, blueprints, manuals, etc.) to your fleet of vehicles or remote sites.
To see how much time Resilio’s WAN optimization technology can save you, try our speed calculator.
Case Study: Northern Marine Group
Northern Marine Group is a ship management and marine services company. Before switching to Resilio Connect, they used CDs in order to synchronize and distribute updates across their fleet of sea vessels — a cumbersome process that took weeks to successfully install updates.
“Being able to use the scripting engine meant that we could essentially have the updates distribute, install, and report back on status all automatically, allowing us to avoid the installation going wrong because of user error. For a few of the ships where we did have issues, all we had to do was create some modified scripts and distribute those using Resilio as well.”
P2P Architecture: Real-Time, Scalable, Multi-Directional Replication
While other sync solutions (like Signiant and Aspera) also offer UDP-based WAN optimization, they rely on point-to-point replication architectures.
Point-to-point replication is inhibited by the fact that transfers can only occur between two endpoints at a time. This creates single points of failure because if any endpoint or network goes down, synchronization across your entire system will be inhibited. Plus, point-to-point replication doesn’t allow for true bidirectional or multidirectional replication.
One of the major keys to Resilio’s superior WAN replication and file sync is the combination of WAN optimization and P2P replication architecture.
With P2P replication, every device in your environment running a Resilio agent can take part in replication simultaneously. This enables you to:
- Sync multiple systems simultaneously.
- Achieve blazing fast replication speeds.
- Utilize the full bandwidth of your entire environment.
- Eliminate single points of failure.
- Organically scale your environment.
- Extend the power of UDP-based WAN acceleration to every endpoint in your system.
Blazing Fast Transfer Speeds
Resilio Connect is capable of replicating data in real-time and synchronizing hundreds of servers 3-10x faster than traditional solutions.
Resilio utilizes optimized checksum calculations (identification markers are assigned to each file that changes when a change is made to the file) and real-time notifications from the host operating system to immediately detect and propagate file changes.
It also uses a process known as file chunking to break files into several chunks that can transfer independently of each other. In concert with its P2P replication architecture, Resilio can utilize every endpoint in your environment to quickly synchronize files and file changes across your entire system.
For example, imagine you need to sync a file across five endpoints. Resilio can split that file into five chunks and begin replicating it. Server 1 can share the first file chunk with Server 2. As soon as it receives that file chunk, Server 2 can immediately share it with Server 3 even before it receives the rest of the file. With every endpoint receiving and sharing file chunks concurrently, synchronization will occur much faster than in a point-to-point scenario.
Sync in Any Direction
Resilio Connect’s P2P replication architecture enables it to replicate files in any direction — one-way, two-way, one-to-many, many-to-one, and N-way — so you can sync files to multiple endpoints simultaneously.
N-way sync is a particularly important capability for use cases that require fast synchronization of large, unstructured data sets across multiple geographically distributed endpoints, such as:
Remote work
Team members in different locations can make changes to files, and have those changes immediately synchronized to every other office. This makes N-way sync ideal for improving collaboration on projects with distributed workforces.
Remote operations
Organizations with geographically distributed branch offices (that may be in areas with poor network connectivity) can ensure that each office stays in sync and everyone has access to the most up-to-date versions of files. And organizations with a geographically distributed fleet of vehicles or vessels can quickly distribute, collect, and synchronize data among their teams in the field.
Disaster recovery
Resilio’s P2P architecture is perfect for disaster recovery and business continuity scenarios that require Active-Active High Availability. Every server in your environment can take part in the disaster recovery process, enabling Resilio to achieve sub-five-second RPOs (Recovery Point Objectives) and RTOs (Recovery Time Objectives) within minutes of an outage.
Organic Scalability
Since every server can take part in replication in a P2P environment, adding more servers only increases replication speed and available bandwidth. In other words, more demand creates more supply — making your environment organically scalable. Each endpoint can utilize Resilio’s WAN optimization technology to overcome latency due to distance.
Resilio Connect also incorporates intelligent automated scale-out technology to perform horizontal scaling with linear performance acceleration across multiple nodes in a cluster. This collectively pools Resilio agents to transfer and sync data in parallel, and has enabled Resilio to:
- Achieve peak speeds of over 100 Gbps.
- Transfer a 1 TB data set between Azure cloud regions in 90 seconds.
- Transfer 500 GB of data in Google Cloud Platform from London to Australia in 50 seconds.
Resilio Connect can replicate files of any size or number. In internal testing, we successfully synced 450+ million files in a single job.
Reliability and Fault Tolerance
Resilio’s features and capabilities enable it to provide bulletproof reliability and fault tolerance.
Its P2P replication environment eliminates single points of failure. If any endpoint goes down (due to network failure, hardware failure, etc.), your data can always be retrieved from any other endpoint in your system. This lets Resilio achieve Active-Active High Availability, making it an excellent solution for disaster recovery strategies.
Resilio utilizes smart routing capabilities to always identify the optimal path for data transfer and ensure your files reach their destination as quickly as possible. And if an endpoint goes down, Resilio can reroute to the next best path.
Plus, with cryptographic data integrity validation, Resilio ensures your files always arrive at their destination intact.
Case Study: VoiceBase
VoiceBase is a speech-to-text software used for audio/video transcriptions, knowledge extraction, speech analytics, and predictive analytics. They use Resilio Connect to quickly distribute their speech model files (50+ GB files) across 400 geographically distributed production servers.
“Resilio Connect enables us to reliably distribute our code, specifically new language models in a fraction of time. These copy jobs now take an hour, down from eight. Best of all, once Resilio Connect was installed, it just works: We never need to manually intervene in any way.”
Automate and Manage Your Entire Environment from One Centralized Location
Resilio Connect is a software-only, vendor-agnostic solution that operates via agents installed on the devices in your replication environment. It’s designed to be incredibly versatile, so you can install it on:
- Any device, including desktops, laptops, mobile devices, and (most) NAS devices.
- Virtual machines, such as Citrix, VMware, and hypervisors.
- Any S3-compatible cloud storage provider, such as AWS S3, Google Object Storage, Azure, Cloudian, Ceph, MinIO, VAST Data, Wasabi, WekaIO, and more.
- Popular operating systems, such as Windows, Mac, Linux, FreeBSD, OpenBSD, and more. (Resilio also offers iOS and Android mobile apps.)
With its flexible BYO storage, you can install Resilio Connect on your existing infrastructure and begin synchronizing files in as little as two hours.
You can use Resilio Connect’s Management Console to manage all of your on-premises and cloud storage endpoints from a single, centralized location.
The Management Console enables you to:
- Automate how synchronization occurs at each endpoint in your entire environment from one location.
- Create bandwidth utilization policies that govern how much bandwidth each endpoint can use during certain times of the day and on certain days of the week.
- Optimize performance and resource utilization by adjusting replication parameters, such as packet size, data hashing, buffer size, file priorities, disk I/O threads, and more.
- Use Resilio’s REST API to script any type of functionality your job requires and generate real-time replication reports.
Because of these features, Resilio consistently provides a straightforward user experience.
Native, State-of-the-Art Data Security Features
Transferring files over WAN networks increases security risks and the need to protect your data. Unlike other file synchronization solutions, Resilio comes equipped with native security features that were reviewed by 3rd-party security experts, eliminating the need to invest in security software or VPNs.
Resilio protects your data using:
- Data encryption: Resilio encrypts data at rest and in transit using AES-256 bit encryption.
- Cryptographic data integrity validation: Resilio validates files to ensure that they arrive at their destination endpoint intact and uncorrupted.
- Forward secrecy: Resilio uses one-time session encryption keys.
- Mutual authentication: Resilio requires each endpoint to provide an authentication key before initiating a transfer, ensuring your data is only delivered to approved devices.
- Permission controls: Resilio provides granular control over who can access specific files and folders, so you can limit access to approved parties.
If you want to learn how WAN-optimized, P2P replication can benefit your organization, schedule a demo.
SD-WAN (Software-Defined Wide Area Network) Solutions
SD-WAN optimization products are hardware and software solutions that enable organizations to centrally control and manage multiple WAN links in order to:
- Monitor packet loss, jitter, and latency.
- Boost WAN performance through application acceleration.
- Minimize network costs by implementing bandwidth optimization techniques (such as deduplication).
- Reduce latency to enhance collaboration and communication between offices.
- Organize network traffic.
SD-WANs are good WAN optimization solutions for simpler use cases — i.e., applications that don’t require large files or large numbers of files to be transferred to multiple endpoints.
By enhancing WAN performance, organizations can improve response times for end-users of business-critical applications.
Aruba Orchestrator
Aruba Orchestrator provides software and physical appliances that enable you to centrally manage, define, and enforce policies to enhance traffic for WAN connections across hundreds or thousands of sites. From a single, easy-to-use interface, you can orchestrate network functions, such as SD-WAN routing, segmentation, firewall, WAN optimization, application visibility, and more. Aruba Orchestrator can be deployed on-premises, in the public cloud, or through an Aruba-managed cloud.
Riverbed SteelHead
Riverbed SteelHead is a cloud-based SD-WAN solution that can manage, monitor, and optimize WAN traffic performance. SteelHead can be quickly and easily deployed on your applications in as little as 15 minutes. It can manage bandwidth utilization, increase transfer speeds through compression of multiple data packets, and provide alerts for latency issues. It also provides a management portal that enables users to get a birds-eye-view of WAN networks across locations.
FatPipe
FatPipe WAN optimization is a product from FatPipe Networks that enables users to boost network performance in order to better utilize their WAN infrastructure and minimize bandwidth costs. FatPipe enhances WAN utilization through router clustering, bandwidth management, load balancing, compression, and caching. FatPipe also includes security features that protect WAN traffic (though many of these must be purchased as add-ons), such as the ability to create routing policy rules that allow or deny traffic.
Use Resilio to Optimize WAN Transfer and Data Sync
Resilio Connect is a file synchronization software system that utilizes UDP-based WAN acceleration technology and P2P replication to dramatically enhance speed and performance over WAN networks and provide:
- File transfer and synchronization that always works over any network (including VSAT, IP connections, cell, and Wi-Fi).
- Blazing fast transfer speeds that can reach up to 100 Gbps.
- Multi-directional replication, so you can synchronize multiple endpoints simultaneously.
- Organic scalability, so your sync speeds and bandwidth grow with your replication environment.
- Versatility that enables you to install Resilio on your existing IT infrastructure, using virtually any device, operating system, or cloud storage provider.
- Native security features that protect your data at rest and in transit.
If you want to learn how WAN-optimized, P2P replication can benefit your organization, schedule a demo.
