As data has proliferated, a growing number of organizations are emerging to help us make effective use of big data.  The latest of these is the open source BigDataEurope Platform, which aims to help us to solve particular societal challenges by assisting us in getting started with big data.

The platform aims to provide an easy to use environment for running a range of big data tools and frameworks.  These tools have been selected based upon their suitability for tackling a range of complex, societal problems.

The tools range from storage based applications such as Hive and Cassandra, messaging services such as Kafka and Flume, data processing tools such as Apache Hadoop, Apache Spark, Apache Flink and publishing tools such as Geotriples.

The user can follow a simple set of instructions to install, or follow the installation video for getting started with the platform.

Users can then pull ready-to-use images from the BDE repository or create a dataflow pipeline to realize a full data value chain using the components available at the BDE repository. This platform will be refined and expanded collaboratively with the users and the societal challenge (use case) partners. This public release of our platform represents the beginning of the operational phase, with which the use cases will be materialized and tested on the developed platform.

In the future, the BDE platform will introduce the concept of smart data to support a range of processing tools for the semantic web and knowledge graphs. This includes the concept of semantic data lake, a semantic analytics stack engine and a semantic platform for logging and integration for cluster resilience.

Overall, the platform has lowered the barrier to entry for new big data users and scientists from different domains to experiment with a variety of big data tools in a plug-and-play fashion.