Research has found that dirty data poses a risk to New Zealand businesses as they adopt AI.
New research released by Datacom suggests New Zealand businesses may not have the quality of their data to fully leverage AI and its benefits: Only 9% of New Zealand businesses consider their data to be 100% “clean”, and 30% consider less than half of their extensive data sets to be problem-free.
Almost half (47%) of New Zealand businesses believe that “about 75%” of their data is clean. Unclean, or “dirty,” data can be duplicated, out of date, unsecure, incomplete, inaccurate or inconsistent.
Justin Gray, managing director of Datacom New Zealand, said this kind of dirty data posed a big risk to businesses as AI adoption grew.
“The quality of AI output is entirely dependent on the quality of the data available – garbage in, garbage out. For example, AI can be a powerful predictive tool and can identify patterns and trends within a business to inform strategic decision-making, but if the data is not good, the AI analysis will also be bad.”
The survey also revealed that while many companies have the right intentions when it comes to data, they lack basic data management hygiene practices: 40% of companies admit that they don't regularly cleanse their data, and a further 13% are unsure whether their company regularly cleanses data.
Just under half (48%) of companies perform “regular data cleansing,” but of those that do, 51% aren't clear about how often this process occurs.
Data audit and assurance protocols are also lacking, with 46% of companies saying they do not have these in place and a further 11% saying they are unsure.
Despite poor data management and hygiene practices, the survey found that businesses are leveraging data for several key tasks, including streamlining business operations (80%), strategy and decision-making (79%), enhancing customer experience (77%) and improving processes (76%).
The survey, commissioned by Datacom and conducted among 200 senior data and IT managers, aims to understand whether New Zealand businesses are equipped to use data effectively.
“The results are a clear indication that data and IT managers need to reevaluate their current processes if they want to use data as the foundation for quality, real-time analytics and insights, and if they plan to deploy AI tools and platforms in their businesses that rely on data completeness and accuracy,” Gray said.
When asked about the specific barriers or issues they face when leveraging data, companies cited incomplete data (51%), duplicate data (46%), inaccurate data (43%) and poor data quality (39%) as their top issues. Data-related compliance and regulatory issues were also cited as an issue by 34% of respondents, with privacy risks (30%) and security risks (22%) also cited as data-related issues.
“These issues mean that if businesses are relying on this data to make strategic decisions, they are extremely vulnerable. If you can't trust the data, you can't trust the insights.”
New Data Retention and Protection Requirements
The survey also looked at how and where companies store their data. While some organisations use a combination of locations, the most common data storage location remains on-premise servers (53%), followed by private cloud (35%), New Zealand data centres (34%), public cloud (31%) and offshore data centres (15%).
Gray noted that as emerging technologies evolve, the rate at which data is generated and collected will increase rapidly, and businesses will need to plan how to manage, store and protect that data as volumes grow.
“For example, companies that rely on on-premise servers need to consider implementing backup options for data storage to avoid panicking as data volumes grow exponentially with the expansion of AI usage.”
The research shows that 91 per cent of New Zealand businesses believe their organisation has sufficient data and security protocols in place to protect their information, but only 60 per cent have data governance policies and procedures in place that are managed by trained staff.
The survey was commissioned by Datacom and conducted by Curia Market Research in May 2024 among 200 senior decision makers (CTIs, CIOs, CDOs, senior IT managers and data stewards) working in New Zealand organisations with over 100 employees.
