Big Data & Machine Learning Juggernaut – II

The Dancing Landscape that I referred to in my previous blog is already on horizon. The Internet of Things (IOT) capabilities have brought in whole new world of data management. New tools, techniques and expertise are on rise. The IOT sensors capture required parameters the values of which get transmitted through various modes of technology, much of it via internet. The received data at the other end needs to be stored in some form and used for drawing inferences.

The Data Lakes and the Object Storage are the new ways of data storage. Such data runs into multiple of petabytes requiring hundreds of servers and their disks. The Big data is coming to reality in true sense with its full force of unstructured characteristic and the size besides velocity of generation and transmission. Multiple vendors have come up with their own technologies of storages and the computational facilities. They include major cloud providers such as Google, AWS and Azure besides many others. Not to be left behind there are hardware companies who offer building on-premises cloud facilities.

The Apache frameworks of Hadoop had seized the Big Data world to the extent that it had become synonymous with Big Data. People understood Hadoop as Big Data. In just few years these are getting replaced by Spark and Object storage facilities. Additionally lot of inferential frameworks for machine learning are progressing such as Tendsorflow and Caffee. Entire Big data computation is being now supported by the hardware evolving from CPU to GPU and now onto TPU by Google. It’s a phenomenal journey and rapidly unfolding technologies and their deployment strategies.

I strongly believe that if we are not quick enough to get aligned to the newer ways of doing things in this tech-world we will not have our rightfully earned place. We must get on with our own upliftment as quickly as possible.