Big data is arguably the most important trend in business today, and the companies that manage to capitalize on their data gain a distinct competitive advantage over their peers.
Unfortunately, the challenges faced in this endeavor are significant, with many organizations struggling to attract the data science skills required to thrive in the modern, data hungry world.
Whilst there are obviously issues surrounding the collection and cleaning of data, I want to focus today on the analysis and interpretation of it.
Smart analysis
The shortage of data science capabilities is well known, and it’s resulted in a number of interesting projects that aim to ‘democratize’ data analysis.
For instance, a team from MIT have taken an AI driven approach to the task. The researchers recently published a couple of papers on the process, including the preparation of data and even the creation of problem specifications.
“The goal of all this is to present the interesting stuff to the data scientists so that they can more quickly address all these new data sets that are coming in,” the authors say. “[Data scientists want to know], ‘Why don’t you show me the top 10 things that I can do the best, and then I’ll dig down into those?’ So [these methods are] shrinking the time between getting a data set and actually producing value out of it.”
Data visualization
Alternatively, I wrote last year about Beautiful Information, a new company that apply visualization techniques to make interpreting data easier.
The company, which was selected as one of Nesta’s ‘New Radicals’ for 2016, has developed an Operational Control Centre app that aims to present data in a more accessible way.
The app aims to transform previously unmanageable data into usable information by displaying it in a visual and real-time way to both managers and clinicians. The app comes with a customizable dashboard so teams and organizations can gain access to the exact data they desire, whether that’s patient waiting time or the throughput of a particular department.
Bringing data to the masses
Very much in this ilk is Count Open, a recent graduate from the Accenture FinTech Innovation Lab. The company, started by a team of Cambridge graduates, aims to make it easy for non-technical employees to examine and analyze data sets.
The system uses natural language processing and aims to allow users to enter in natural language queries, with the system then mining a range of data to not only find the right answer, but to display the answer in the most effective format.
The desktop application is currently in beta mode, but the team are confident that it will open up the magic of data to people without the data science skills organizations crave.
“If we can get people 95% of the way there, it encourages more people to engage with data,” founder Oliver Hughes told me.
What’s more, the system is appealing because it works as well with messy data as it does with structured data. With so much of organizations current investment in data involved in the cleansing of it, this promises to be a particularly prominent selling point.
The ability to effectively interrogate data to make more informed decisions is undoubtedly of huge competitive importance for organizations of all types, and it’s pleasing to see a growing number of tools emerge that bring those kind of capabilities to a wider audience.
