A summary of our work in the Archiver project

After more than three years, the ARCHIVER project has drawn to a close. Now is a good time to reflect on the project, its achievements and its impact.   

The ARCHIVER project may have finished, but its legacy is not at an end, rather it’s a new beginning.   

ARCHIVER has successfully kick-started new products and services that are sorely needed to sustain and provide long-term access to internationally important and valuable scientific data resources.   

And there is plenty more to come. 

What was the ARCHIVER Project? 

ARCHIVER was a European Commission supported project that aimed to close the gap between the requirements of Long-Term Digital Preservation (LTDP) of petabyte scale scientific research datasets and the capabilities of commercially available services and solutions.   The gap was clearly identified in the ARCHIVER state of the art review that was done early on in the project.    

As a result, ARCHIVER ran a competitive process, executed in three phases, for Suppliers to design, prototype and pilot new and innovative LTDP services.  These services needed to demonstrably meet the archiving and preservation needs of the Buyer organisations who included CERN, EMBL-EBI, PIC and DESY.    

Arkivum was one of only two organisations that successfully completed all three phases of the competitive process and we are so glad that we did! 

 The ARCHIVER project as a catalyst for digital preservation change 

Before looking at the impact that ARCHIVER has had on Arkivum, it is worth looking at the project as a whole.   

ARCHIVER has successfully raised awareness of the need for, and the benefits of, LTDP for scientific datasets.  This includes as an integral part of the European Open Science Cloud (EOSC).  As a result of ARCHIVER, new products and services are now available for LTDP on the EOSC market place, including Arkivum’s.  Major inroads have been made in connecting the scientific research and digital preservation communities and showing how, in practice, LTDP approaches can enable data to be more Findable, Accessible, Interoperable and Reusable (FAIR).     

For example, reports such as FAIR Forever enumerated the challenges and provided recommendations of how things could be done better when it comes to LTDP of scientific data.    

ARCHIVER has now played its part in showing how some of these gaps can be closed.  Real solutions are now available as result of the R&D done in ARCHIVER.   That’s big progress.   But the job of digital preservation is never done.   

Thankfully, and undoubtedly at least in part as a response to ARCHIVER,  LTDP remains high on the EOSC agenda and is being championed by the new EOSC task force on digital preservation.  Whilst ARCHIVER is just one cog in a large and complex machine, I’m confident that history will show that it has been an important cog in catalysing change – and not least moving the world forward in the LTDP of scientific datasets at a very practical level. 

Part of ARCHIVER’s success comes from its use of the Horizon 2020 PCP programme.  This provides the framework for stimulating new commercial products and services from Suppliers such as Arkivum – and in a way that allows R&D to be done collaboratively and with close cooperation and knowledge sharing with end-users (the Buyers).    

The value of this approach cannot be underestimated.  The model of shared-costs, shared-risks and shared-benefits makes a very refreshing change to more traditional procurement approaches that commercial suppliers of software products and services are more used to (another day, another RFP).   

The impact of ARCHIVER has not gone without recognition: we were very pleased that the ARCHIVER project won the International Council on Archives (ICA) Award for Collaboration and Cooperation at the 2022 Digital Preservation Awards. 

What has ARCHIVER meant for Arkivum? 

The PCP approach has accelerated and de-risked our ability to enter a new market with an innovative product and service that addresses the problem of long-term digital preservation and access to scientific research datasets.   

Without the PCP, we would have been unlikely to target this market on our own due to the level of up-front investment needed, a lack of pre-existing contacts and partnerships, and the cost of diverting time, effort and money away from other aspects of our business.  The approach of sharing costs, risks and benefits in a PCP has enabled to rapidly innovate, test and validate new product and service offerings.   

 The ARCHIVER project has also helped us grow our relationship with partners such as Google and their Google Cloud Platform (GCP).  Arkivum is now an official Google reseller and we have plans for joint promotion of ARCHIVER Results.  Getting the attention of large players such as Google in the domain of LTDP is hard without projects of the scale of ARCHIVER, its funding for R&D, and the involvement of high-profile Buyers such as CERN. 

ARCHIVER has allowed Arkivum to make major improvements to our software solution for LTDP. Using the work done in ARCHIVER, the Arkivum solution allows organisations to provide long-term access to hugely valuable scientific data in a way that:  

• is cost-effective at scale when working with very large datasets;  

• ensures environmental sustainability by minimising the carbon footprint of long-term archiving and data access; and  

• applies good practice in the digital preservation and archiving so that datasets remain Findable Accessible Interoperable and Reusable (FAIR) for decade timescales or more.  Scalability and cost-efficiency for organisations with petabyte datasets is a major feature of the solution, but smaller organisations also benefit through Arkivum’s cloud-hosted and multi-tenanted approach.    

I could wax lyrical about the technical work we’ve done and the new approaches we’ve taken, but that’s covered in detail in our webinar on our ARCHIVER solution and in reports and papers that we’ve shared with the community, for example at iPRES.   

Bigger, better and cleaner digital preservation 

What is worth emphasising is that ARCHIVER has enabled Arkivum to take a broader and longer-term view than might be the norm for a company of our size.   

For example, ARCHIVER enabled us to consider how to measure and reduce the carbon footprint of LTDP in the cloud.  ARCHIVER isn’t just about LTDP that is bigger, better and faster (it certainly does all of those on a grand scale); it has also enabled Arkivum to tackle the issue of how to make LTDP cleaner and more environmentally sustainable.  I think this fits very nicely with the tenet that digital preservation is all about enabling access and reuse of valuable resources (scientific datasets in this case) for many generations to come. 

Overall, Arkivum’s involvement in ARCHIVER has been a hugely enjoyable and beneficial experience.   The project has already had a major impact on the community and there is plenty more to come.   

None of that would have been possible without the support of the European Commission through Horizon 2020, the engagement, openness and collaboration of the Buyers, and the additional support and involvement of Google.   

Hats off to you all!