BigData Really…


Hi guys, I was thinking since a long time to write blogs on big data. So, it is my first blog on BIGDATA. As a DBA, I always think where will I be after 5 years or 10 years from now. Should I be a DBA or move to BigData. You may think the same. Guys it is up to you, what you want to be. In IT industry we need to be updated all the time.

As a back-end developer and administrator, I have seen, back-end technology rarely change, new services & tools added on it. But now a days BIGDATA is a burning hot topic. Everyone is talking about it. I was curious about it so I attended so many sessions & conferences about BIGDATA. So what I got from those sessions, BigData is not a back-end technology. It is far away from database or related technologies. It is a concept based on parallel processing. But yes you can say it data management & processing technology.

There are two main components of BigData

  • HDFS (Hadoop Distributed file system) {for storage]
  • MapReduce (Algorithm written in java) {for data or job processing}

Why & What is BigData-

BigData can process structured & unstructured data or data-sets. This means data is not needed to store in table format like traditional RDBMS. But power comes when we have huge amount of data to process. Why & what will be discussed in future blogs about BigData. But yes as data is growing we really need a tool or technology to handle & process it.

What Microsoft is doing about BigData-

I am an active member of DBA community of NCR Region, one day I was attending conference about BigData at Microsoft office. I asked the same question to Microsoft guys. They told me, Microsoft is also working on it. Till date they have created a tool named HDInsight. You can download it from internet with HDInsight emulator. This tool convert unstructured data to table format or structured format. Then they connected this tool with twitter api & shown us twitter’s unstructured data into a table format. It was a good experience.

Guys I need your participation on this too. Please comment what do you think on this- If possible i will tell each and every term about BigData in future. Till then share your experience on this & be happy. bye bye Take care.