We were recently in Austin, Texas for this year’s annual Archives*Records conference and hosted a panel discussion on “can you do digital preservation with your eyes closed?”
With a great line up of industry experts on our panel including Annalise Berdini from Princeton University, Corrinne Collett from the Archives of the Episcopal Church and Walker Sampson from the University of Colorado Boulder, and a broad range of experience on the subject, we were looking forward to an interesting discussion. Our goal was for all attendees to take away at least one new interesting thought, regardless of their views on automation going into the session.
Yaron Naor, Arkivum’s VP & GM Americas, kicked off the session by introducing the panellists, and each followed with a short overview of their backgrounds and where their organisations are on their digital preservation journeys. Yaron guided the discussion, starting with the easy question of “do they currently use automation in their archive solution?” This helped to set the scene for the discussion ahead.
The discussion then addressed file formats and any that the panellists had found preservation of couldn’t be easily automated due to their complex content, for example, games, eBooks, social media posts and software applications.
Preservation can sometimes seem more of an art than a science, so we moved the discussion to ideas or practical tips on how preservation can be automated in the real world. Preservation is an imperfect process, so how can it be automated? What happens when things go wrong and errors are thrown? Is there a way that allows automated preservation to be managed by exception?
Some great advice when addressing the factors you need to consider for crafting an efficient and automated preservation workflow is to break the process down into manageable chunks with built-in checks to validate everything is as it should be for that particular chunk. This provides assurance that what you expected to happen has happened, but also makes it easier to identify where any problems have occurred should they arise.
Automation can be an intimidating topic, particularly as it is often hyped in the press as replacing human intervention. So can preservation really be done without people being in the loop and looking in detail at what is happening constantly? Simple answer here is “no”. Ideally, it needs to be a mix of automation and human intervention.
Automation can successfully be used to manage repetitive and labour-intensive tasks as part of a human workflow (rather than vice versa), i.e. there are different checkpoints in the process for humans to provide input and make decisions so the workflow can continue.
With all of that said and done, how do you really know you can trust automation? For example, if you set everything up upfront with rules to test your content, would you then be comfortable that automation would work for your content at large? Do you need to do spot checks and ongoing validation of outputs to make sure everything is running as expected? Can you trust your system to spot any errors? For example, is it accurately checking the outputs of your ongoing normalisation process are correct?
We closed the session with some final thoughts from our experts on where they think automation in preservation is heading to in the near future. Without a doubt, automation in digital preservation is certainly going to remain a hot topic over the coming months.
To hear our digital preservation expert, Dr Matthew Addis’ view on the subject, join our webinar “Digital preservation can sometimes seem more of an art than a science, how can preservation be automated in the real world?”