Geo Week News

October 3, 2017

Signiant’s Media Shuttle SaaS – Rapid Transfer of Large Data Sets for Power Users

ToPa Submit Interface1

Paul Tice received a free trial of Signiant’s software to test, but was not paid or otherwise compensated by Signiant to write this review.

To date, Signiant has achieved significant penetration in the media and entertainment market, capturing clients that include sportscasting powerhouses such as ESPN and the NFL, major broadcast and cable networks like NBC and Discovery, and film studios such as 20th Century Fox and Universal—or, anyone that needs to move a lot of data fast.

Signiant is exploring our market sector because users in the 3D industry commonly move very large data sets. About a month ago, ToPa 3D was introduced to Signiant so we could test out their Media Shuttle software and compare it to the FTP services we’ve used in the past. When I told them that our 3D scan and photogrammetry projects can range from 50GB to a terabyte (or more) in size, they confidently offered the use of their platform, noting that what we do might be a perfect fit for what they have.

What we have used in the past

At ToPa 3D, we have moved files using Google Drive, Dropbox, Box, and Amazon S3 servers, depending on what our customers have been most accustomed to. These services are not perfect. They require you to divulge some information for their own marketing analytics, enter a log in, and then remember your mother’s favorite dessert or first pet, and what you get in return is a relatively bleak interface that just shows files in a directory. They are functional, and depending on your subscription status, can cost anywhere from “free” to a few thousand dollars a month for large companies moving data on a regular basis.

FTP services also have problems. Often they attempt to reorganize or bundle data you send, which can result in a customer receiving data such as 3D scan files in several dozen individually zipped and unorganized folders (which can happen with Google Drive if there are .zip files involved). That’s a hot mess when you’re sending scan files that are interdependent upon one another in a specific directory structure.

For professionals dealing with massive data sets—such as 3D scanning files or photogrammetric-based mapping image files numbering in the tens of thousands—moving data to the client efficiently is always an issue, especially over long distances. Most of the time, in this field, we just mail an external hard drive and hope the person actually receives it, and on time.

However, by utilizing a new transfer protocol, it is now possible to send data up to 200 times faster than consumer FTP services users may be used to. This magic fire comes from Signiant.

3D data movement redefined

What we’ve seen so far is that the speed of transfers using Signiant Media Shuttle software in a local area is relatively comparable to using any other consumer service such as Dropbox or Google Drive. Where users will start experiencing speeds several orders of magnitude higher is when they transfer large files over large distances, such as between states or countries.

Whether going local or far away, we are sold on Signiant’s simplicity of use and the management options provided by their Media Shuttle software. We can start uploading and forget about it, since through Signiant, we have a few different ways to track our data package transfers and we aren’t limited by file size whatsoever—a huge plus with the type of data we work with.

Oh, and that part about an FTP service reorganizing your data into folders is a thing of the past with Signiant. The way you send is the way your customers will receive the data.

What’s under the hood?

It’s a little complicated to explain how Signiant moves large files over the Internet faster than other transfer methods.

First, Signiant does not use the networking protocol known as Transport Control Protocol (TCP), which is the foundation of Hyper Text Transfer Protocol (HTTP) and File Transfer Protocol (FTP). TCP uses a “sliding window” mechanism. Here’s a simple metaphor to explain how that works (and why it can be so slow).

Imagine you are sending three sets of data in a package: A, B, and C. TCP opens up a connection between the sender and the recipient (think of this as your “window”) and sends one set at a time, in order. TCP will send set A through the open window, wait for acknowledgement that it has been received, and then slide the window over to send set B. If the data doesn’t get through or if the acknowledgement is lost, TCP will time out and try again with a smaller amount of data. This can be quite the bottleneck.

Instead, Signiant utilizes a proprietary solution deployed on top of the User Datagram Protocol (UDP). We might think of UDP as a bit of a “Monte Carlo” (or brute force) method that sends large chunks of data using as much bandwidth as it can, at the cost of some reliability. To return to our metaphor from earlier, UDP uses a dynamic window that it can open as wide as it needs to, even wide enough to blast through all three sets of data (A, B, and C) at the same time. That means UDP sends files much faster, but as mentioned above, it is notorious for its unreliability. Problems include the receiver getting the individual parts of the data package in the wrong order—for instance, as CAB instead of ABC.

Signiant corrects for this, and makes UDP a reliable method for transfer in the following ways:

  • Flow Control ensures that data is transmitted at the optimal rate for the receiver.
  • Congestion Control detects when the network is being overloaded and adapts accordingly.
  • Reliability Mechanisms makes sure that data loss due to congestion or other network factors is compensated for and that the order of the stream of data is maintained.

In short, Signiant has cleverly adapted the UDP protocol, which historically wasn’t as reliable as TCP. They have claimed that, despite the speed gains, their use of UDP is just as reliable and secure as TCP, if not more so. Our experience with this service over the past month or so continues to prove them right.

Signiant’s offerings

Signiant offers three primary file movement and management solutions for Windows and Mac OS. Depending on what your company wishes to accomplish, there may or may not be a need for all three services. Users are expected to provide data storage themselves, but this shouldn’t be a problem for enterprise users, even on the smaller end of the spectrum. Furthermore, users can choose from a number of options for storing their data, whether that be keeping it on-premises, using cloud services like Amazon Web Services and Microsoft Azure, or a mix of all these options. Signiant users also get unlimited transfers and unlimited file sizes.

  • Media Shuttle is a hybrid SaaS that moves data directly between you and a recipient.
  • Flight is a SaaS utility that moves data to and from a cloud object storage service.
  • Lastly, Manager+Agents is Signiant’s on-premises solution for automated delivery of large files within and between geographically dispersed enterprises.

At ToPa 3D, we primarily tested the Media Shuttle service for moving large data sets back and forth with our customers. Yet, since our reality capture customers wish to see our models online more and more, the Flight service could also make sense as an option for bulk unattended uploading of these 3D models to our Amazon S3 server for storage and viewing. We didn’t test Managers+Agents, since that is best suited for power users that are in the business of moving big data on a regular basis, such as broadcasting and media companies, or content suppliers such as Apple iTunes.

For this piece, we’ll focus on Media Shuttle, as that is relevant to most service provider firms working with large scan- and photo-based data sets.

Media Shuttle overview

Media Shuttle is a SaaS solution for transferring files of any size to anywhere in the world quickly and securely.

Signiant’s Media Shuttle service includes unlimited “portals” of any of three types: Send, Share, and Submit. Each portal gives users a brandable interface, back-end control that is intuitive to use, and a drag-and-drop upload interface that makes data transfer accessible to most anyone. Users can select the storage method for each portal individually, meaning you can use cloud storage for one portal and local storage for another, depending on your needs.

Send

The Send portal in the solution gives users a one-way distribution mechanism for sending files to any number of recipients.

When using this portal type, you are prompted to enter the recipient’s email, drag and drop files or folders, and include a message. That’s it. The recipient will receive an email that they have files waiting to be downloaded, and be prompted to install the application to complete the transfer (they only have to download this application once). Once they have successfully downloaded the files, you receive an email notification alerting you that the transfer has been completed. We really like this second email, because it offers a helpful method for staying within delivery schedules, or determining whether data is going to spam.

The opening interface above was branded with ToPa 3D’s scan image and offers a log in prompt.

Share

The Share portal functions much like an FTP server in that it allows you to grant remote access to file folders. When you have files to share, simply add a user and give them upload or download privileges for specific folders.

Submit

Submit allows users to submit files to a predetermined folder on a server. The program organizes these submissions by the email address of the sender. This enables it to avoid overwrites, for instance, if two different users submit two files with the same name to the same folder.

The Submit portal is best used for sending files to a project manager when working as a team, this way a PM can easily track progress from each team member.

Monitoring and management

Media Shuttle provides two different web interfaces for managing portals. Because Signiant hosted my trial, I was unable to play around in the IT interface which provides tools for IT management to perform basic tasks like creating new portals, managing storage and network configuration, setting file size limits, etc. The system also offers more sophisticated controls for managing traffic and permissions. From within the IT interface, IT staff can delegate full administration rights over a portal to another user — typically an operations manager or project manager.

As one of these delegated Administrators of my trial, I did, however, have access to the separate Administration interface. This interface provides for managing internal and external users, and viewing transfer stats such as members, files sizes and history, and the number of transfers per time period. You can also monitor all active transfers. Within the portals, end users are also able to monitor their own transfers and view a predictor on exactly when their files will be delivered (handy for meeting deadlines). Within the Administration interface portal managers can custom brand the portals that their users and customers see, and have the ability to flip back and forth between Send, Share, and Submit. Note that all Send, Share, and Submit portals are independent of one another, so each portal has a separate IT and Administration interface, and each portal can be customized differently.

Secure transfer

Media Shuttle has a number of security features, including the following:

  • User email address authentication and authorization.
  • Encrypted browser sessions and file transfers – up to 256-bit AES encryption.
  • Built-in certificate authority used to manage Public Key Infrastructure (PKI).
  • An option to password-protect individual Send transfers, or all transfers that go through a selected portal.

The price…

This is the part where all these great features might cause one to take pause. Small businesses wishing to use this service more than a few times a month can expect to pay between $6-7k per year, with enterprise solutions starting at $35k per year. It’s important to acknowledge that many small businesses pay less than $100/month for FTP and simply clear out old data to stay under the file sizes necessary to keep their subscription costs down. Each business certainly needs to decide what will work best for them. Signiant may however be a good option for a business that is scaling their data transfer capacities.

Fortunately, Signiant has delineated between what they call “Active Users” and “Members.” You pay for the User only, which is defined as “someone who sends at least one file, and/or receives three or more files in a given month.” A Member, on the other hand is anyone added to a Media Shuttle portal by an administrator, or who has sent or received a file. According to Signiant, there is no charge for Members, and an Administrator can add as many Members as they want. Active Users are defined by floating licenses, meaning which Members count as Active Users can vary from month to month as teams and workloads change.

Overall, there is flexibility in the subscription model. In short, it’s based on the number of Active Users, a stat that Signiant monitors and reports to you each month. That enables you to work with the company to find the package that best fits your needs and modify that package as your needs change.

It’s also worth noting that Signiant doesn’t charge based on bandwidth. They will not throttle your transfer speed or bump you up to a more expensive tier should you upgrade your pipe.

The website paints a broad picture of how pricing works, which means it is certainly worth an in-depth conversation with a Signiant representative to play out specific usage scenarios to get to the bottom line of what this service will cost and what your options are.

Conclusion

Signiant’s accomplishments, and its part in bringing light to new methods for data transfer, are getting the attention of major players in the media industry. With their entry into a new market vertical with AEC, we may be witnessing just the beginnings of a new era of 3D data management.

 

Want more stories like this? Subscribe today!



Read Next

Related Articles

Comments

Join the Discussion