Research & Development | Tom Lynam| March 23, 2021
Does your company struggle with data preservation? Read stories from three companies who experienced data management failures, and lessons learned, from Arkivum Marketing Manager Tom Lynam.
Data management means different things to different people. On a personal level, most of us have a mix of storage media on various devices, potentially multiple cloud storage accounts and perhaps a handful of old portable hard drives or USB sticks. If I asked you to access certain parts of that data, how certain would you be that you could find it? And if you found the physical location of the data, what state would it be in?
From an organisation perspective, you may assume that that companies have a much better grasp of historical data and would be able to easily access all their old files if required, but this isn’t always the case. I’d like to share a few stories of data loss from high-profile companies that I think we can all learn from.
You probably think that storing data on a hard drive or in the cloud makes it safe, but this is not the case.
Pixar deletes Toy Story 2 … twice
A well-known tale in the data management world is that of how Pixar deleted 90 percent of Toy Story 2 by accident. The Next Web covers the story in more detail but in short, months of work and hours of footage were deleted from the company servers due to a human error.
The data loss was compounded when it was discovered their backup tapes were only 4GB in size leading to the original data being overwritten by new data. Unfortunately, the error log to tell the team of the issue was also located on this full tape and so it hadn’t been noticed.
Luckily, they were saved by a team member who had been working remotely and had a two-week-old backup on a workstation in her house, and after a nervous trip to retrieve it, the files (minus the work conducted in the intervening period) were retrieved.
Ironically, due to initial feedback on the release months later, much of the movie was reworked from the ground up (that would be the “twice” reference in this story’s title) but what this tale does illustrate is a critical point to long-term data management.
While having a secure backup (and not one which is easy to delete) is important, it is also crucial that it is continually checked to ensure it is still working as intended, and the data is secure and useable.
Organisations must think very carefully about the importance of correctly archiving and storing its data. Even if the data may not have much value at the given time, in an ever-changing business environment, things could shift very quickly to a position where that data could be a competitive differentiator.
Doctor Wh …ere?
Another story from the entertainment industry is how 97 episodes of Doctor Who (in addition to a number of other shows) are no longer held by the BBC. Surprisingly, this wasn’t actually in error as the BBC routinely deleted archived programmes for practical reasons such as space, scarcity of materials, a lack of rebroadcast rights. It’s probably also worth mentioning the BBC was not alone in this.
There has been much effort to try and restore these lost episodes, including the discovery that some ardent fans had made audio recordings of the shows when they were broadcast.
So, what lesson does this teach us about data management? Organisations must think very carefully about the importance of correctly archiving and storing its data. Even if the data may not have much value at the given time, in an ever-changing business environment, things could shift very quickly to a position where that data could be a competitive differentiator.
This is part of a wider shift in mindset for archived data from simple storage to effectively leveraging archived data for organisational benefit.
The cost of space travel
In 2020, it was reported that NASA had made a costly blunder in their move to Amazon Web Services (AWS). In an effort to plan for the future, the space agency identified that by 2025 they will need an additional 215 petabytes of data storage (up from the current 32 that it currently needed). And if you’re interested to know how much that is — it is roughly how much space you would need to download 50 million HD movies from Apple TV.
In short, NASA’s decision to move towards cloud storage did not take into account cloud egress costs for users to access the data. The recent audit which flagged this issue concluded that, “Collectively, this presents potential risks that scientific data may become less available to end users if NASA imposes limitations on the amount of data egress for cost control reasons.”
This raises a number of interesting points — while NASA had taken the initiative to actively plan for its long-term data management, it did not fully scope or consult on the full requirements and the solutions available, It illustrates how every organisation needs a robust and well thought-out plan when it comes to its data management.
Although the information NASA collects is fairly unique, this story also illustrates the exponential growth that many organisations face when it comes to the amount of data they are capturing. Big data has been a concept for many moons now, but it is in the coming years that we will really start to see organisations realise the size of the data challenges they face.
These stories feature organisations that have subsequently regretted not having a more considered approach to their long-term data management. They demonstrate how, while it may not seem a pressing issue at the time, a proactive and considered approach will save companies further down the line, whether recovering from a tough situation or taking advantage of a new business opportunity.
Tom Lynam joined Arkivum in 2020 as their Marketing Manager, responsible for all marketing activities across the business. He has more than 10 years of experience in marketing and product management roles across a range of industries including technology, IP and professional services organisations.
Your comment will be reviewed by a moderator for approval.