A big part of ensuring the success of science is ensuring that science data can be shared. 

The advanced networking resources that move this data must be performant and ready to support data transfers when, where and no matter how transfers are used. 

This is a complicated and fascinating challenge for many reasons. 

Firstly, Data Transfer Nodes (DTN) and other networking systems are owned by individual institutions or labs. DTNs can be deployed with various equipment, local or remote storage configurations, and within different parts of the network. Second, most institutions have diverse and varied needs to transfer large amounts of data, meaning that DTNs and other resources may not be tested routinely for performance. And third, knowing how “fast a data transfer should be” is not an easy question to answer as capacity, flexibility, and general capabilities of networks are constantly changing and improving.

Unlike home or general internet uses, scientific workflows can require large amounts of burst-transfer capacity. Data being moved may be impossible to duplicate and even one underperforming link can cause the entire transfer to slow down. 

The Energy Sciences Network (ESnet) has long been focused on developing ways to streamline network operational burdens on the scientific programs, researchers, and others they serve. Building on the successful Petascale DTN project and the Science DMZ design pattern, the Data Mobility Exhibition, or DME, was developed to improve the predictability of data movement between research sites and universities of any size. The DME has been included in EPOC’s six main activities (EPOC is a joint effort of Indiana University and ESnet).

DME is a resource that enables the calibration of data transfer performance for a site’s DTNs to ensure that they are performing well by using ESnet’s own test environment, at scale. For instance, a 10 Gbps DTN should be capable of – at a minimum – transferring one terabyte per hour. As part of the DME, network engineers have a wide variety of resources available to analyze network performance against ESnet’s standard DTNs, obtain help from ESnet (or EPOC) to tune equipment, and to share performance data and network designs with the community to help others. 

DME is an easy, effective way to ensure network and storage resources are operating at peak efficiency! Quilt members can use the DME with their partners to better understand the performance that institutions are achieving in practice. DME information can be found at fasterdata.es.net. To date, 47 institutions have used this platform, many of which are associated with The Quilt.