10 tips to become a Hadoop developer

hadoop introduction to Big Data

If you work for a cutting edge company that focuses on getting (or experimenting) with newer technologies, you have probably heard already about Big Data or Data Science. All the buzz is in the air. More and more companies are experimenting with Hadoop and its ecosystem to solver their “Big Data” issues. Each company have their own unique use cases for Hadoop. Either for analytics to make better data-driven decisions, or for ETL (Extract, Transform, Load) processing, analyze web logs, or many other use cases.

When companies start experimenting with Hadoop, (most) software developers are introduced into a world that they are not completely used to. Software developers are used to building applications and are given recognition among its peers based on the quality of the code they write or how functional or beautiful an app is based on functionality/beauty. In Big Data (Hadoop) world, the rockstar will always be data regardless of the technologies, beautiful graphs, quality of code, etc.

Data is the center of attention and will always be. Either by its complexity, size, volume, sources and so on. Even Hadoop makes data the core of its ecosystem on HDFS (Hadoop Distributed File System). All the applications and technologies are build on top of HDFS to make data more accessible.

hadoop developer

10 tips to become a Hadoop developer

1. Get out of your comfort zone!

Hadoop will take you to many different/unfamiliar places. Luckily, you are half way there since you have learned so much about code and designing applications already. Now it is time to refresh your math skills, and deal with different kind of use cases that you might not have been exposed before.

2. Take ownership of…data (you guessed right!)

How can you find what your data is hiding if you are not familiar with it? The better you know your data, the better you will be at knowing what to do and what to find with that amount of data. Play with it, analyze it, transform it, decode it, but don’t destroy it 😉

3. Make friends in every department, including yours (IT).

Some people might not talk geek like you do, but they can point you in directions that you have not thought out before. Get different insights and learn how data affects/improves people in your organization. Also, it would be helpful to learn a few things from the Quality Analysts from your team since they might know a few tips about how to see and test data from different angles.

4. Learn new technologies and have fun while doing it.

Here you can find main components of Hadoop, but in reality, there is a vast ecosystem of new technologies on top of Apache Hadoop. Take your time and master them. Apache Spark, for example, is very popular!

5. Embrace change (at a much faster pace).

If you are new to Hadoop/Spark, you will find that things move a way faster. Mainly because organizations are eager to find insights about the data that they possess already. Also, Hadoop technologies are very new and have not gotten to a mature state yet. So new (and improved) technologies are being updated every few months. There is always a new shiny technology ready to be discovered.

6. Learn to see the Big Picture.

You will not be a code monkey anymore. You will be appointed to design new apps based on the data you are collecting, processing, and analyzing. Since you are the one playing with the data, you are responsible to think outside the box and come up with use cases for the company/industry you work for.

7. Become a data-driven problem solver.

You are used to problem solving all the time. It is your job as a code geek. However, Big Data brings a whole new mindset in how we solve problems. Let the data be the focus and vehicle of your decisions.

8. Refresh your math skills.

You are already good at math, just refresh what you learned in college. A good grasp of Statistics should help you with ways to analyze and understand data. Also, if you encounter a problem within your organization that has not been solved elsewhere, you might need to come up with your own algorithms. So stay fresh!

9. Learn Data Science techniques

If you have done all of the steps above, you are on the right track to become a Data Scientist. Now go go improve or learn the skills you are missing and build the next Big Data app!

10. Learn how to visualize your data.

There are many tools like D3.js where you can visualize results from data crunching jobs in Hadoop. This will be very beneficial for infographics and to make your data tell a bigger story.

Conclusion

These tips will direct you in the right path to become a Hadoop developer. It takes time to learn and master each step. However, Big Data and Hadoop jobs are very rewarding in terms of personal knowledge acquisition and value given back to your employer. Not to mention the heftier salaries you will be receiving as a Hadoop developer or related Big Data job.