The healthcare industry has changed and re-formed more in recent years than ever before. In the face of significantly reduced budgets, this has prompted radical cuts and the introduction of new technologies in order to reshape the industry from top to bottom. One of the most significant technology innovations is the growing utilisation of big data. With cutting-edge technologies allowing researchers and doctors to make new discoveries that just a few years ago would never have been thought possible, the resulting data has the potential to shed previously unprecedented light on medical science.
Who’s looking after the data?
However, as the data generated by today’s pioneering work in the sector continues to grow, many healthcare organisations, including hospitals and research institutions, are finding that storing this vast amount of data in a way that fulfils a range of needs and requirements is much easier said than done. Key criteria that organisations need to consider include how long the data is going to be kept for, whether the format that the data is being stored in will be readable when it is retrieved in the future, and at what rate the data is going to grow. Also meeting compliance of the NHS Information Governance (IG) Toolkit, as well as ensuring the highest standards of data security via the approved N3 network are essential.
Digital pathology is one such area that is significantly benefiting from the application of big data, but equally it is grappling with the issue of finding a suitable home for the sheer quantity of data being created.
20TB of data per year
Traditionally, pathology involves examining tissue slides under a microscope, however many NHS pathology labs are now embracing digital workflows, converting the glass slides into digital images that can be viewed, managed and analysed in a way that transforms digital pathology practices. It is widely believed that digital pathology has significant potential in achieving quicker and cheaper diagnoses and prognoses of life-threatening diseases such as cancer, and the field is seeing significant growth as a result.
Nonetheless, to put into perspective how much of a challenge storing the data generated by digital pathology processes can be, a small slide scanner running 200 slides per day at medium resolution will generate over 20TB of data per year, which has to be stored, managed and kept secure over decade-long timescales, all the while taking into account the compliance, security, cost and data integrity requirements associated with its storage.
Where do you start?
Looking at numbers like this, and when you start to consider what is required to successfully store this data, it quickly becomes clear that digital pathology laboratories, as well as hospitals and various other medical research institutions, have a significant task on their hands. The big question being asked by the majority of specialists who are trying to get to grips with their data long-term storage strategies is “where do I start?”
Many digital pathology managers are turning to NHS IM&T (Information Management & Technology) Managers to solve this problem for them, some of whom are expanding their local infrastructure, while others are implementing cloud-based services such as Amazon Glacier or Google Nearline. However, simply expanding the local infrastructure to provide sufficient space for these data volumes does nothing to ensure that it is protected from corruption or loss.
A compelling business case
An alternative option, which some IM&T Managers are considering, is to bring in a specialist provider of long-term data archiving, such as Arkivum, who can implement a managed service that has been specifically designed from the ground up to provide ultra secure storage for large volumes of data for extended periods of time.
Such specialist services also helps IM&T managers solve their backup problem. In essence, if data is static and no longer likely to change (slide images being a case in point) then this data can be offloaded to much cheaper, and more secure, long-term archive storage. By doing this, IT managers can economise by not needing to provide extensive and expensive infrastructure to maintain an unnecessarily large backup window. While data volumes are very large, using a managed data storage service as described above does help to eliminate many IM&T Management headaches. Predictable costs and long-term contracts, along with significant local IT cost savings, also provide a very compelling business case.
Data management white paper
But it is not just digital pathologists who can employ digital archiving services. Healthcare organisations across the board are looking for ways to preserve their data and I would strongly encourage hospitals, genetics laboratories, fertility clinics and digital pathologists alike to consider the integrity, security and longevity offered by a digital archiving solution, as it is by far the most effective way to ensure that data is protected.
To learn more about how straightforward it is to reap the benefits of efficient digital pathology data management, you can download our new White Paper: Managing Big Data – Reaping the Rewards. In the paper we detail the issues and challenges that are being faced by digital pathology labs when implementing technologies like whole slide imaging and we highlight what a digital pathology data storage strategy needs to include.