Big data and mid-sized companies
The collection and analysis of ‘big data’ to empower personalised marketing and product development is no longer the preserve of tech and consumer giants such as Facebook and Amazon. It’s becoming a necessity for medium sized businesses with a large customer wanting to gain a competitive edge. If a company isn’t using big data to understand its customers, it can be assured that its rivals, both domestic and international, will be. Big data can generate better business, speed up decision making, reduce time to market and underpin a rational, data driven culture. However, the mere collection of data is not enough; businesses must understand how to use it to their advantage. Research from Villanova University highlights the issues facing US firms and similar concerns affect the mid-sized sector in Australia.
The amount of data created by users of the internet and social media is staggering and still growing fast as developing countries embrace mobile technology. There are now over 3.8 billion people who use the internet, just over half the world’s population of 7.50 billion. 2.9 billion people use social media and almost 5 billion people have a mobile phone – two thirds of everyone in the world. Mobile phones now account for over half the globe’s internet traffic, with the share generated by computers dropping sharply in recent years to 43%. The average amount of data used by a mobile phone user continues to climb, topping 2 gigabytes a month around the world. Google handled 40,000 searches per second in 2016, answering 1.2 trillion queries over the course of the year.
The average company manages 160 terabytes of data, compared to fewer than 50 terabytes for smaller firms, and that figure is still rising sharply. Villanova’s research finds that almost 60% of American CEOs see big data as vital to their business, and 83% use it to improve the profitability of existing goods and services. However mid-sized firms looking to leverage data to their advantage face the same problems as internet giants such as Google without the same resources to tackle them. Data can be polluted, incomplete, outdated, duplicated, inconsistent data or simply mistyped. Human error, unsurprisingly, is the source of most of these errors, but a lack of internal resources, a poor data strategy and insufficient hardware can also hamper analytical attempts.
Mid-sized firms tend to lack the budget required to hire their own data scientist or perform their own analysis. IT staff may be given responsibility, given their technical know-how, but will lack the grounding in data science required. It is no easy task to justify investment in big data analysis, particularly when its effectiveness may be difficult to prove or audit in conventional terms. Companies are reluctant to take on an ongoing commitment which doesn’t immediately affect their bottom line, particularly when their senior executives may be less than computer literate. External challenges, such as capturing and validating data, ensuring security and proper governance and managing its scale and diversity cannot be addressed without the firm committing to the idea.
Companies with strong data-management strategies naturally tend to get the best results and see it as a major competitive advantage over their peers. Such strategies are a consequence of forward thinking and engaged executive leadership, whose benefits will also tend to manifest in other aspects of the firm. Big data projects which aim to solve a specific problem are more likely to be funded and achieve measureable results as data gathering and analysis must be tailored to an end. A security and compliance will differ greatly from those designed to increase customer loyalty. Firms should consider how data can be gathered and used in different ways across different units, as the more widely big data is used across a business; the more likely it is to be a worthwhile approach. Businesses which use big data tend to be more profitable than those which don’t, but the causality of these factors should not be taken for granted, as profitable firms are more likely to fund a big data approach. Firms which embrace big data will tend to have a closer relationship between their business and technology than their peers, a bond which will grow as data analysis begins to shape internal processes and consumer facing activities.
Firms must also look to hire skilled staff, although many find difficulty in recruiting suitability trained personnel able to sanitise data at its source as well as analyse it. However, while the career of data scientist might appear an attractive one, it could also be one of the first jobs to be replaced by smart algorithms and machines. Indeed a data scientist’s last job might be to design the programme which will replace them. While the role of data science will only grow in importance, the number of firms which need their own data science teams may actually shrink as firms out-source data collection and analysis to Microsoft, Amazon, Google or specialist firms just as they outsource other elements of their computing activities. Those firms which retain their own capacity may well find that non-specialists can use better technology to turn raw information into useful information to shape products and processes. Nobody needs a degree in computer science to complete a spreadsheet or word document today and few people who drive a car would know how to change the spark plugs.
Data scientists with degrees or doctorates in the field and professional business experience are few and far between and tend to seek careers in research or major companies, rather than Australian mid-sized firms. While universities may soon churn out thousands of young graduates with data science degrees seeking work at much more affordable rates, not every company will need to hire one even if they can afford to. Highly profitable firms in high value sectors which require extensive data crunching, from financial trading and fraud analyse to high-end healthcare or aerospace and automotive markets will need a strong team of qualified data scientists to turn staggering amounts of complex data into useful information on a daily basis. These professionals will have the highest levels of statistical, programming and visualization skills and command a commensurate salaries, but most medium sized firms would struggle to keep them fully employed and would be better off employing business analyst firms on a needs must basis.
The fact that few of them will need their own team of data scientists doesn’t mean that mid-sized firms can ignore data science itself. David Styles of DataGravity notes that smaller firms already have the ability to collect transactional, social interaction and customer sentiment data as a consequence of modern digitised business processes, and if they don’t process this data in useful ways, their competitors will overtake them as they use their own data to increase efficiency and customer satisfaction.
He argues that SMBs want ‘data democracy’, in which every employee can use data in ways which were once the province of IT specialists, as naturally as they use a computer and smart phone for other purposes. Firms which embrace a data driven culture will empower all their employees to use data to make better decisions, rather than pigeon hole the task with a specialist team. User friendly technology, rather than data scientists – or even IT professionals – will be the key to unlocking value throughout the company.
Business analysists will increasingly embrace data science as part of their portfolio of skills and major providers will compete to offer cloud based or on premise capabilities. Microsoft’s search engine Bing recently announced the ability to seamlessly incorporate company and employee information into its search results and the ability to merge structured company data with a much greater mass of unstructured information from social media, emails and documents will soon be within every company’s grasp. Tools to analyse and, just as importantly, summarise and present this data in easily understandable, actionable ways will help every firm maximise their value.
