ESnet (Energy Sciences Network) is a high-performance network backbone built to support scientific research. Funded by the U.S. Department of Energy and part of Lawrence Berkeley National Laboratory, ESnet provides fast, reliable connections between national laboratories, supercomputing facilities, and scientific instruments around the globe. Our mission is to allow scientists to collaborate and perform research without worrying about distance or location.
In order to achieve that mission, we need to make sure our network is running as smoothly as possible. A key piece of this is collecting a wide range of telemetry about our network. Since science is an end-to-end process that crosses many networks, we also participate in multiple external collaborations, such as NetSage and perfSONAR, to make sure other networks have the tools they need to perform this collection as well.
Grafana plays an important role in visualizing and understanding the data we collect both internally and as part of our collaborative efforts. Specifically, we use Grafana as the visualization component within our Stardust Environment. Stardust measures network activity and usage giving us a window into network traffic, scientific data transfer activity, and more. This allows us to do things like predict capacity needs and easily view traffic surges on our infrastructure. While Grafana includes a lot of pre-built panel plugins that are great for visualization, it is also an extensible architecture that allows us to create custom visualizations tailored to our network analytic use cases.