How Google Transfer Appliance Works (+ a Software-Only Alternative)

Unlike Google’s Storage Transfer Service, the Google Transfer Appliance is a data storage device. It’s designed to collect and move data one-way from remote sites to the Google cloud.

Transfer Appliance devices are shipped by Google via the postal service. Organizations copy data onto them and send them back to Google where the data is copied into Google cloud storage. 

Conversely, Resilio Connect is a software-only data collection and ingest solution that runs on any device. It’s designed to reliably transfer and sync files from the edge to any other location (other edge locations, core facilities, or any cloud provider). 

It’s a multi-cloud solution for use with GCP, AWS, Azure, Backblaze, Wasabi, and more. Resilio Connect is also purpose-built to reliably and efficiently transfer data across any network. Learn more about Resilio Connect’s edge ingest capabilities here.

Overall, the Google Transfer Appliance can be a good fit for one-time cloud migration to GCP. It’s designed for scenarios with poor or no network connectivity.

However, the Transfer Appliance has some unavoidable drawbacks, such as:

• It only works with Google cloud storage for one-time data migrations.

• It’s not suitable for continuous data ingestion. This can be a big issue if you’re capturing and collecting data at the edge on an ongoing basis. 

• Its regional support is very limited. The Transfer Appliance is currently only available in the US, EU, UK, Singapore, and Japan. 

• There’s uncertainty around data availability. This is caused by the fact that you have to wait days to receive the device, then upload your data to it, send it back to Google, and have their team transfer your data to the cloud. The process can easily take weeks or even months if you’re operating at the extreme edge.

• Manually copying data from your storage systems to the Appliance is a hassle. All four methods of copying data to the Appliance require you to download third-party clients or write custom code, which can take lots of time and effort. 

That’s why, if you’re continuously generating data at the edge and need the flexibility to store and transfer data from the edge to any location on an ongoing basis, Resilio Connect is a superior option.

In this guide, we’ll dive deeper into the strengths and weaknesses of the Google Transfer Appliance. We’ll also explore how Resilio Connect can be a perfect alternative (or, in some cases, an addition) to Google Transfer Appliance. 

Specifically, Resilio Connect is:

• Software-only: Instead of managing new hardware, you can just install Resilio on your existing infrastructure — including servers, desktops, edge devices, and more — and start using it in as little as two hours.

• Resilient and fault-tolerant: Resilio Connect uses a unique peer-to-peer (P2P) architecture and proprietary wide area network (WAN) optimization technology to overcome latency and packet loss regardless of distance. This makes it ideal for collecting and ingesting data from many remote locations to any cloud regions, on-prem storage, or edge locations.

• Natively secure: Resilio Connect protects data end-to-end without reliance on a VPN. Key security features include AES 256 encryption, mutually authenticated endpoints (i.e.,safeguards to ensure your data arrives only at its intended destination), and many more.

• Versatile: You can use Resilio to ingest, transfer, sync, and replicate data across any network, including VSAT, cell, Wi-Fi, broadband, and more. You’re also not limited to any cloud computing provider — Resilio works with Amazon Web Services (AWS), Azure, GCP, Backblaze, Wasabi, and any other S3-compatible cloud storage provider.

• Automated: You easily control every aspect of data ingestion, transfer, sync, and replication from one place. As a result, you don’t need to manage NFS mounts or move data across physical devices, which drastically reduces complexity.

Companies like Exxon Mobil, KFC, and Canadian Railroad use Resilio Connect to securely sync and transfer data across remote sites, cloud regions, and hybrid or multi-cloud environments. To learn how Resilio Connect can help you as well, schedule a demo with our team.

Google Transfer Appliance: How It Works and Its Biggest Strengths

If you request a Transfer Appliance, Google’s team will ship you a device that meets your requirements. These can be either rackable or freestanding with weights ranging from about eight to 75 pounds and data capacities ranging from seven to 300 TB of encrypted data. You can find the complete details about each device type in Google’s documentation.

Once it arrives on-site, you can begin the data upload. There are four options for copying data to the appliance: 

• Secure Copy Protocol (SCP) or Secure File Transfer Protocol (SFTP) for Microsoft Windows, Linux, and macOS. For Windows, you’ll need to download a client that supports UTF-8 filenames. For Linux or macOS, you’ll need to run the commands included in the Google Transfer Appliance supporting documentation.

• Network File System (NFS) share for Linux and macOS. You’ll need to run the commands included in the supporting documentation combined with a copy utility. Optionally, you can also save your Secure Shell (SSH) key for quicker logins and transfer speeds.

• Server Message Block (SMB) share for Microsoft Windows and Linux. First, you’ll need to configure your appliance using the file share protocol flag included in the supporting documentation. Then you’ll need to make changes within your File Manager and input code into your Command Line. In Linux, you’ll also need to install an SMB client.

• Mount on the Appliance for Network File System (NFS) and Common Internet File System (CIFS). To implement your own file shares using a wrapped Linux mount command, you’ll need to run the commands included in the supporting documentation.

For more details and exact instructions, please refer to the supporting documentation for the Transfer Appliance.

If you have access to a network, you can also enable the online transfer option to stream your data to your Cloud Storage Bucket. This requires downloading an Appliance Service Account Key and using it to set the appliance to online mode. Once the data has been transferred, it will be erased from the Appliance. 

When you’re using the Google Transfer Appliance in offline mode, you’ll need to ship the device back to Google. Their team will upload the data to your Cloud Storage Bucket and wipe the appliance. This option makes Google’s Transfer Appliance a good choice for one-time, high-volume, one-way data transfers. For instance, if you need to ship three hundred terabytes of accumulated data from one facility to another, it can be captured and accessed in Cloud Storage in under 25 days, versus 9 months at a typical network bandwidth of 100 Mbps. 

When one of the locations is remote and has no network access, Google Transfer Appliance excels. The device itself is ruggedized, meaning it can withstand the rough conditions often associated with remote locations. In addition, security measures like encryption and tamper-proofing ensure your data isn’t compromised en route.

Cons of Google Transfer Appliance

Despite Google Transfer Appliance’s many strengths, there are three major drawbacks:

• Poor regional support. At the time of this writing, the Transfer Appliance is only available in the US, EU, UK, Singapore, and Japan. If your data is stored in another location, you can’t use the Appliance to transfer it to the cloud.

• Shipping and handling physical devices leads to uncertainty around data availability. Because the device needs to be shipped to you and then back to Google, it can be difficult to estimate when data will be available in the cloud. This in turn impacts any projects reliant on the data to move forward, creating a cascade effect of scheduling uncertainty. The same is true for any other hardware-based transfer solution (like AWS Snowball).

• Manually uploading data to the device is a hassle. As mentioned above, there are four methods of copying data to the Google Transfer Appliance, which you need to understand in order to select one that’s compatible with your infrastructure. Once you’ve made your choice, you’ll need to download third-party software or customize and implement code commands. If you’re transferring data frequently or to multiple locations, this can become a time sink.

In the next sections, you’ll learn how Resilio Connect can help you overcome these limitations.

Resilio Connect: The Top Software-Based Alternative to Google Transfer Appliance

If you need to continuously ingest, replicate, or sync data among multiple edge locations and cloud regions, consider Resilio Connect. It’s a software-only, cloud-agnostic solution with broad compatibility, meaning you won’t need to buy, learn, and manage new hardware to implement it. 

Resilio Connect is:

• Reliable, flexible, fast. Its P2P architecture eliminates the single points of failure (SPOFs) that leave hub-and-spoke architecture susceptible to catastrophe. Allowing data to be transferred in any direction also increases transfer speeds and offers flexibility in how and when data gets shared. Resilio is also extremely efficient as only the changed part of a file is transferred or synchronized across the network. 

• Versatile: Resilio runs on any type of hardware device running your choice of operating system. It can also transfer, sync, and replicate data across any cloud provider, including GCP, Azure, AWS, and more.

• User-friendly. Resilio Connect is controlled from a Central Management Console, which also provides an intuitive interface for file access. Once you complete the quick setup process, it lets you automate data transfer, sync, and replication jobs eliminating the need to manage operations once they’ve begun. This makes Resilio a “set-it-and-forget-it” solution from a management perspective.

Broad Compatibility Means Flexibility

Resilio Connect is a cloud-agnostic agent-based software solution. Whether your cloud service of choice is AWS, Google Cloud Platform, Azure, MinIO, Backblaze, or Wasabi (or any combination of S3-compatible storage solutions), our solution can be deployed. 

Resilio also builds on open standards, open file formats, and a multi-cloud architecture. This means it works with most popular operating systems, such as Windows, Mac, Linux (a variety of distros), iOS, Android, FreeBSD, and Unix, as well as with virtual machines like VMware, Citrix, and hypervisors.

Resilio Connect can be deployed on industry-standard servers, networks, desktops, and laptops. It also enables you to use any storage type, including file, object, block, DAS, NAS, SAN, and more.

This has a few implications. For one, you won’t need to get new hardware for Resilio Connect; you can quickly and inexpensively integrate it with existing systems. The lack of compatibility constraints also gives you greater control over data ingestion, replication, syncing, and storage. 

Among other use cases, you can:

• Freely move data between cloud storage services.

• Sync data among remote sites and cloud regions reliably no matter how far apart they are.

• Sync data reliably between an infinite number of servers, running your operating system of choice — and using any type of storage.

In fact, Resilio Connect can even be used to complement services like Google Transfer Appliance. For instance, let’s say you have a remote location with no network access. You can use the Appliance to get the data into your Cloud Bucket, then use Resilio Connect for all other transfer, sync, and replication uses, e.g., to sync your data across different cloud regions, services, and providers.

Any-Direction Data Transfers for Reliability 

Most data transfer services use one of two types of architectures: hub-and-spoke or follow-the-sun.

• Hub-and-spoke: There is a central location called a hub and all data transfers must go through it. All other locations are spokes. If you want to transfer data from one spoke to another, two separate transfers must take place: one to the hub, and the second to the actual destination.

• Follow-the-sun: Servers must transmit to each other in a specific sequence. For example, data from server A must be sent to server B, B only sends to C, and so on.

Both methods create single points of failure. For example, if a data transfer to the hub is delayed due to a spotty network connection, the transfer to the entire environment is also delayed. More dramatically, if the hub itself was compromised, data transfers would become impossible. But, even if nothing fails, transfer speeds are lower due to data having to be passed only between two devices at a time. 

To avoid these issues, Resilio Connect uses a unique P2P architecture where every device in the network is involved in the data transfer, allowing any-direction data transfer. 

This includes one-way, two-way, one-to-many, many-to-one, and N-way sharing. Combined with our proprietary WAN optimization technology, based on the User Datagram Protocol (UDP), this enables reliable high-speed data replication and sync across any network.

There’s one more method Resilio uses to ensure high transfer speeds: file chunking. Resilio separates files into multiple pieces, or chunks, that are transferred independently. This enables servers to immediately share chunks with each other rather than waiting to receive the full file. This results in speeds 3 to 10 times faster than traditional methods.

For example, here are some results from our recent test:

• 1 TB of data was transferred between Azure regions in 90 seconds.

• 500 GB of data stored in GCP was transferred from London to Australia in 50 seconds.

• Transfer speeds of over 100 Gbps. In tests of our next-generation release, our engineers have reached speeds of over 200 Gbps.

Keep Your Entire Environment Synchronized at All Times 

With our N-way synchronization, you can keep data synced across many distributed endpoints quickly and reliably. Plus, with Resilio Connect, there are no file size limitations and endpoints can be located anywhere. For example, if you want to keep lots of servers across an environment, n-way is perfect because every server can sync across the entire environment.

Another particularly useful application of n-way synchronization is enabling real time collaboration by remote teams. High-speed, reliable data sync ensures everyone has access to the most up-to-date information.

Quickly Bounce Back from Outages, Network Failures, and Other Disasters

Resilio Connect’s P2P architecture doesn’t have a SPOF, so if one device fails, the data is still available from the rest of the devices running a Resilio agent. 

Whether there’s a server crash or a power outage that shuts down a data center, Resilio can leverage the rest of your servers to achieve sub-five-second Recovery Point Objectives (RPOs) and Recovery Time Objectives (RTOs) within minutes of an outage.

Resilio Connect can also assist your Disaster Recovery (DR) efforts by allowing you to set up:

• Hot-Sites. These are duplicates of a primary production facility, including hardware, software, network connectivity, and real-time data backup. Our software can easily keep a hot-site synced with a primary site until it’s needed.

• Cold-sites. These facilities are a bare-bones backup in which only essential functions are replicated, reducing maintenance needs but at the cost of production workload, maximizing downtime, and data integrity. Resilio can handle transfers between the cold-site and the main site during setup and when disaster strikes.

• Warm-sites. A compromise between hot-sites and cold-sites, these facilities are partially equipped but have the same hardware and network connectivity as the primary site. Resilio allows for both replication and transfer of data before and during DR.

• Offsite copies. Data can be stored at a physical location other than your main site (such as a remote server) to prevent data loss during a natural disaster or another similar event. Resilio can maintain this backup so any data that would have been lost can be transferred back.

Zero Gravity Transport™ (ZGT): Transfer Files Reliably and Efficiently over Any Network 

The proprietary, UDP-based transfer protocol Resilio Connect uses for WAN optimization is called Zero Gravity Transport™ (ZGT). 

ZGT optimizes transfers across any network (e.g., VSAT, broadband, Wi-Fi, cell, and so on), making it ideal for companies operating in remote locations with unreliable connections. It’s designed to overcome latency and packet loss by maintaining a uniform rate of packet distribution, reducing redundant retransmissions, and sending interval acknowledgments. 

This enables ZGT to deliver predictable transfer times regardless of the underlying network. In fact, our free calculator can help you determine how much faster your transfer times will be with Resilio Connect. 

For example, Northern Marine Group saved time and money by switching from mailing physical CDs to using Resilio Connect to distribute software updates. The time it took them to bring ships into compliance went from six months to two weeks: a 92% improvement.

If you’d like technical details on how ZGT works, please refer to our whitepaper on WAN optimization.  

Optimize Data Sharing and Access

Resilio Connect lets you choose exactly how you ingest, sync, and access your data, which allows you to optimize data sharing and access. This makes it a flexible and efficient object storage gateway solution.

For example, you can use Resilio to reduce unnecessary data movements and lower egress costs by:

• Using on-prem storage for frequently accessed files. That way users don’t have to constantly download frequently used files, which can drastically reduce your cloud costs.

• Pinning traffic to the most optimal network. This feature is called Smart Routing and it enables you to move parts (or all of your traffic) to a network of your choice. That way, you’re not relying on expensive WAN connections all the time but can also utilize cheaper LANs.

• Having granular control over which files get synced across locations. The Transparent Selective Sync (TSS) feature lets you prevent unnecessary data from being transferred off-network and gives you control of the flow of data, such as only backing up core files on a remote server for DR.

Lastly, Resilio also gives you efficient, low-latency access to files stored in cloud regions, remote sites, and edge locations all over the world. This access is provided via an intuitive interface that operates much like Microsoft OneDrive.

Manage Data Ingest, Sync, Replication, and Access from a Single Place 

Cloud providers often require you to juggle multiple solutions for different ingest, sync, and replication use cases. With Resilio Connect, there’s no need for additional software to handle these various aspects: you can do it all from our Central Management Console. 

The Console lets you set up, debug, monitor, and manage everything from one place. For example, you can use it to allocate bandwidth for endpoints within your environment, as shown below. 

It also lets you adjust key parameters, manage user permissions, set up notifications, and much more. Plus, you can use Resilio’s REST API to integrate tools that your team relies on and configure any type of automation that your workflow requires.

Within the Console, you can monitor key metrics about data transfers, sync, and replication. This includes statistics on past usage, logs, total bytes in a job, transfer status, and more.

Lastly, you can store the Console in any cloud provider (e.g., in GCP, AWS, or Azure), in any virtual physical Windows or Linux instance, or in an on-premise environment.

Keep Your Data Safe by Default

A key part of controlling the flow of data includes preventing unwanted and unauthorized access. That’s why we’ve developed features to protect your data and we had them verified by third-party security experts. 

Resilio Connect’s built-in security features include:

• AES 256 encryption, which protects your data during transfers and within your storage systems.

• Integrity validation, which ensures your data always arrives at its destination uncorrupted.

• Mutually authenticated endpoints. Both the sender and receiver of data must verify their identity, ensuring your transfer arrives at the right place (and that unauthorized sources aren’t inserting malware).

• And more.

As an added bonus, Resilio’s built-in security features mean you don’t need to purchase additional security software.

Resilio Connect: Reliable, Fast, and Flexible Data Ingest, Sync, and Replication across Any Cloud

Resilio is the #1 alternative to Google Transfer Appliance because it’s:

• Software-only. You don’t need to revise your hardware configurations to implement our software-only solution because it’s compatible with most operating systems, cloud storage services, and devices.

• Reliable. Unlike most other methods, there’s no single point of failure during transfers. In addition, we provide multiple safeguards to ensure your data remains intact and up-to-date.

• Fast. ZGT and P2P file sharing ensure that your data migration goes as quickly as possible no matter the distance. 

• Flexible. You can transfer data in any direction, and our Central Management Console allows you to track and manage every aspect of data ingest, sync, and replication from one place.

• Secure. We provide built-in tools like encryption, mutual endpoint authentication, and data immutability.

If you want a fast, reliable, and flexible way to transfer data, visit our site to learn more about pricing and schedule a demo with our team.