Australian Government plans data retention mandate. Are we on the cusp of a Data Storage Management "perfect storm"?
The latest plans by the Australian government to retain internet history metadata exemplifies the challenges facing individuals and business across the globe. In the face of the increasing value of data, Australia is doing what it can to capture the data’s potential. But is it sustainable?
Information growth is real, very real. IDC having been calling this out since the research back in 2007 to compare the available storage against the available data. Unsurprisingly available storage fell short showing that if the world’s data were to be stored “off-device” we’d be left wanting. This is though, a single dimensional view of the challenge that looms. The real issue is more likely to be the effective storage management of this data and that challenge is three dimensional.
The first dimension relates to Information growth which is not linear, its exponential. This is because there is growth in both devices and in the desire to collate. Whilst smart phones may be close to critical mass in most western countries, the era of device to device communications and sensor data is only just beginning. Add to this new phone features, televisions, wearable devices, vehicles, satellites and the volume of unstructured data blows out. This is matched with an increasing desire to capture or gather information as business leverage Big Data as a fundamental business tool. Business Intelligence smarts are becoming more accessible through lower cost and ease of use. In turn, inroads are being made into analysing the unstructured data. This creates a simultaneous push and pull on the data footprint.
The second dimension is the changing habits in storing data. Whereas IT managers could once put their hand on the disks and tapes that hold the companies data, the increasing trend of multi-sourcing is placing data with multiple third parties. Data could now not only be sitting in your data centres or in your co-location facility but could also be sitting on a multiple managed environments, clouds, in country or overseas. As different applications, platforms and business functions get farmed off to different providers the challenge of having a unified approach to data storage management becomes increasingly onerous.
The third dimension is the increasing mandates for governance. The Government’s latest data retention order to Telco’s sits on top of numerous existing policies that stipulate retention, disposal and security requirements. These go beyond recommendations, these are legal requirements that are backed by penalties that could include individual or business level fines or in some cases criminal charges. The risk of not adhering to the policies is very real. Take the potential brand damage that could be generated from the leaking of personal data, be it web site visits, purchase history or medical data.
So how will business manage this data effectively in this new world where we have more data, in more places with more governance? There is no easy answer. The perfect storm could be approaching and many companies will simply not have the resources or technology to meet all the requirements.
I recommend you do what the fishermen do when a storm’s on the horizon. Read the clouds and batten down the hatches. Look to your cloud or managed services provider to show leadership in this space – they should have proven experience in solving these problems. Finally, get your Data Storage Management “house" in order quickly – it’s easier to scale a planned solution than trying to build on disorder.