About two years ago, the Department of Biochemistry & Biophysics at Stockholm University decided to join the amazing work so many of you are doing with cryo-EM, and we managed to get funding for a new state-of-the art microscope facility. Half of the time in this resources will also be made available nationally in Sweden, through the Science for Life Laboratory national infrastructure for molecular life sciences.
Since we have a history of working with simulations and modeling (in particular the GROMACS package), our team naturally started working with the computational needs of this infrastructure – and quickly realized our hardware budget was at least an order of magnitude too small. However, as we’ve had to spend large efforts enabling our simulation codes to use modern accelerators – in particular graphics processors (GPUs), we decided to try to contribute to this new community by attempting to crack the computational problem instead of just throwing more money at it.
Since early 2015, Björn Forsberg & Dari Kimanius have worked tremendously hard to create a new GPU-accelerated version of RELION – something we could never have done without close collaboration with Sjors Scheres at the MRC Laboratory of Molecular Biology. This has turned into a great collaboration (and I think it’s a wonderful example of the power of open source software): What was initially just a testing hack has turned into RELION-2.0 that will fully support GPU processing out of the box. There are still some rough edges and parts that have not been accelerated, but the parts that account for by far the largest share of computational processing have been accelerated: Both particle autopicking, 2D- and 3D-classification, and autorefinement. There is a preprint available at biorxiv while the work is undergoing peer review.
At this point we have just pushed out the first beta release to a small set of users. We will do our utmost to expand this as quickly as we can (think weeks, at most), but since there are quite a few differences in the code we need to have a smaller set of early test users that help us find things we might have missed during a few months of testing at MRC-LMB and Stockholm University – we simply don’t want this large mass of new code to create bad results for anybody.
So, that brings us to the main part: Lots of you have been asking about what hardware to get for this new code. Since we finally finished the code and an accompanying paper, I’ll add a couple of posts later tonight with suggested hardware ranging from extremely cost-efficient consumer cards you can use either in desktops or servers up to high-end professional cards that your supercomputing center might prefer. The first part is about deciding whether you need consumer or professional cards, the second describes hardware for a low-end workstation (but still dual-card!) optimized for cost, and the third is focused on quad-GPU workstations. There’s now also a post about quad-consumer-GPU rackmount nodes.
In addition to Björn & Dari doing the bulk of GPU programming, we’ve also had great collaborations with Ross Walker at SDSC in the context of MD codes, and in particular to get awesome performance out of consumer NVIDIA hardware – thanks Ross!