Hybrid Memory Cube Powers Galaxy Quest

　　Radio astronomy is getting more data intensive, and Hybrid Memory Cube (HMC) is helping to process the large amounts of information being pulled from the sky.

　　Astronomy is one of the oldest observational sciences, said Simon Ratcliffe, technical lead for scientific computing for the Square Kilometre Array (SKA) in South Africa. It's possible to see only 20,000 stars with the naked eye through an optical telescope. “It's not a lot of data if you're a scientist," he said in a recent webinar. Radio astronomy, however, is turning into a data science.

　　Radio telescopes such as MeerKAT, under construction on the Northern Cape of South Africa, and the biggest and most sensitive in the southern hemisphere until SKA is completed in 2024, are quite data intensive. Ratcliffe said that even though only 16 out of 64 MeerKAT antennae are operational so far, it's been able to detect 1,500 new galaxies in a relatively quiet corner of the universe that astronomers have never been able to see before.

　　As the antennae are moved further and further apart, he added, you gain resolution and see finer details. Compound that with more powerful and larger antennae, and there's a great deal more data. A decade ago, an astronomer could download data to a laptop and work with it, but now there's so much data with current instruments it's become unfeasible to look at it, said Ratcliffe. The challenge is to figure out how to extract knowledge from all of the data. He describes SKA as a "moonshot" type of project. "It's going to be at least 10x bigger than what's gone before," he said. "It's pushing boundaries as a construction project but also as an [high-performance computing] project."

　　Francois Kapp, the sub-system manager for SKA's digital backend, said to illustrate where the challenges come in it's instructive to understand how signals are combined: A front-end digitizer does exactly what its name implies, taking analog signals and converting them to digital. This is done with every each of MeerKAT's 64 antennae, he said, with each stream coming in at 40 Gb per second. “The total input data rate is about 2.5 terabits per second."

　　As astronomers move to more modern, more sophisticated instruments, said Kapp, they need to keep up with this data rate in two ways: it requires increasing depth of memory as well as increasing widths of memory. “The lucky thing is you can actually distribute this memory," he said.

　　The algorithms are also simple, but data from each antenna gets combined and the data keeps coming in 24 hours a day. “We don't need much numerical precision, so that opens up opportunities, but on the other hand, we are severely power-limited," Kapp said.

　　The signal processing is where Micron's HMC comes into play in the form of SKARAB, the digital processing platform for the MeerKAT telescope. It succeeds the ROACH-2 (Reconfigurable Open Architecture), and features 3,600 signal processing elements. The high performance HMC allows the engineers to match processing power to memory size and data bandwidth.

　　SKARAB improves on ROACH with upgraded interconnectivity from a data rate of 10Gbps to 40Gbps expected from each the MeerKAT antenna. Kapp said it wasn't enough to simply add more memory. SKARAB boards are founded on field-programmable gate arrays (FPGAs), with new generations of SKARAB expected to become available every two to three years, matching the release of new FPGA chips. “FPGA is ideally suited for correlation," he said. “The problem we need to solve was memory to match the processing depth and width. Depth is easy as the chips keeping getting bigger, but the width has not." HMC's serial interface combined with FPGA's migration to serial interfaces, combined with the openHMC controller led to selection of HMC as the memory of choice for the SKARAB platform, Kapp added.

　　OpenHMC is critical to meeting to the data processing needs of both MeerKAT and SKA, said Juri Schmidt, a research associate in computer science at the University of Heidelberg. An open-source project developed by the German university's Computer Architecture Group, openHMC is a configurable, vendor-agnostic, AXI-4 compliant HMC controller that can be parameterized to different data-widths, external lane-width requirements, and clock speeds depending on speed and area requirements. “The complexity has shifted to the memory stack so it's up to the manufacturer to deal with that complexity," Schmidt said.

　　Even more important was the ability to build a host controller as buying commercial technology was not financially feasible and researchers would otherwise not have access to the technology, Schmidt added. “We've seen a lot of contributions to the controller. We need the open source community to drive these innovations." He credits the Micron Foundation for its support and access to the technology.

　　Steve Pawlowski, Micron's vice president of advanced computing solutions, said providing access to HMC technology to research projects such as MeerKAT and SKA provides valuable input for future development. “You get tremendous value collaborating with academia and collaborating with researchers," Pawlowski said. "The volumes may not be huge but they believe in the art of the possible."

　　Their feedback helps improve the product, Pawlowski said. “We know eventually that product will come into the mainstream," he said.