How To Grow As A Data Scientist

Ready to learn Data Science? Browse courses like Data Science Training and Certification developed by industry thought leaders and Experfy in Harvard Innovation Lab.

What Skills Do You Need To Go From Jr. To Sr. Developer

The role of a data scientist still varies from company to company and even team to team. This makes it much harder for companies to create a standardized growth plan for their data scientists.

Without having a clear growth plan, there is a risk that these talented computer wizards will get stuck. They might provide good insights, but they will never really grow and provide the true ROI they have to offer a business or more importantly themselves.

With this in mind, our team talked to managers around Seattle working at the top tier tech companies to find out what they wanted and expected from their senior data scientists. We wanted to share the information we learned both to help data scientists grow as well as help managers who are trying to challenge their new data scientists to grow.

Based off our discussions we found that it wasn’t about programming, or designing algorithms (that was a baseline for a Jr. data scientist). When we asked these managers what they wanted to see from their more senior data scientists they informed us that they wanted driven individuals who can communicate concisely, who were able to think for themselves, who have a solid understanding of the business and who are capable of managing up.

“Data scientists have the opportunity to sway company decisions.”

In order for a data scientist to grow, they need to be challenged beyond the technical aspects of their jobs. Data scientists have the opportunity to sway company decisions. They have a lot of responsibility on their shoulders. That means they need to take ownership of the work they do. They need to question their data sources, be concise in their insights, know their business and help guide their leaders.

Don’t Just Question Your Findings, Question Your Data

A senior data scientists won’t just trust their data after receiving it. They will poke and prod it for things like bias, missing data, duplicate data, etc.

Data is bound to have quirks. For those who spend hours and hours in data, you know what I am saying. While scrolling, or graphing data you see those strange patterns that make you stop and say “I wonder why x looks like z”. Younger data scientists will often be too focused on finishing the project. They haven’t learned how to stop and really analyze these strange patterns. These patterns can be caused by systems that default specific data outputs like -1 or 1 or maybe even biased data caused by purchasing bots that might skew what customers actually are buying on an e-commerce site, and a thousand other plausible causes of misleading data.

These patterns are not necessarily incorrect or bad data. Even when data is accurate there will always be operational quirks. When designing reports, algorithms and metrics, these need to be considered. An experienced data scientist will not only look for these data quirks, they will expect them.

The term source of truth gets thrown around a lot in data teams. It refers to the original data source that multiple teams have decided is correct. I was very naive when I started out as a data scientist. On one of my first projects, I was informed about a data source that our team had labeled as the source of truth. For months I worked on our “Source of Truth” developing analytics and applications to help over 200 managers and directors have access to that data. Of course, it wasn’t too long until there were consistency issues with other metrics. It was then that I realized I had been working on a data source several ETLs from the source of truth.

Talking to tech managers across Seattle. This is a common issue. Young analysts, data scientists and developers are overly trusting of their data sources. Typically, the younger, less experienced employees will be very eager to get the work done. This will inadvertently lead to less understanding of what the data actually is. Instead of asking why, they spend more time ensuring the product “functions”. Thus, they miss out on noticing data quirks.

In order to grow as a data scientist, you need to step out of just making sure a product or algorithm “meets requirements” and take ownership. You have to take responsibility for understanding the data and its quirks. That way you can fully communicate to your manager or director any and all assumptions you made. A data scientist can’t truly grow if he blames the bad outputs on the data.

Be Able To Concisely State The Value Of Your Findings

To grow as a data scientist you have to expand beyond being a programmer/statistician. You need to learn how to be a communicator and must obtain the ability to concisely state the value of your findings and also state what your director should do with the information.

It can be tempting to provide directors and managers all the graphs, all the data and all the technical information that was gathered during our research to prove we were actually doing work. Especially in data science where sometimes it takes a few months to make valuable headway on a single problem (for good reason). However, at the end of it all. Directors don’t want an excess of information.

All a director wants, are the important points and they want to know what they should do based off of those points. Going on about ROC and why you used one algorithm vs another won’t be very helpful to a manager or director with 8 other teams to manage. Most of the time, we find that managers really just want 2–3 brief points. Sometimes even just, “Yes” or “No” is better than “Maybe, in these conditions…blah blah…possibility 1 has these risks, possibility 2 has these risks…”. An experienced data scientists knows how to help their manager by providing polished insights insights, with distilled action items. If the manager wants to know more, they will ask(and a good data scientist will have an answer). In the end, managers don’t want to be bogged down with extra information that won’t help them make good decisions.

Know Your Business

Any time a data scientist starts a new job, it is understandable that they won’t understand everything about the new business day one. There is a lot to learn besides just data sources, code bases, and other company specific systems. They need to learn about the operational day to day of the data they will be working with. They also need to understand the problems the company is facing. However, an experienced data scientists should be capable of quickly gaining an understanding of a business.

Don’t get so focused on honing your technical skills that you fail to learn the business. Learn how to work with different teams, get involved in projects and allow yourself to be diligently mentored. Data scientists can get thrown into project after project on multiple different types of subject matters and they need to be able to adapt quickly.

Jr. developers will often be much more focused on honing their technical skills vs. their business understanding. Similar to many other trades where the grunt work (in this case coding, data cleansing, etc) is carried out at lower levels. This leaves them no time to gain a in depth understanding of how they can help the business. However, this is an important phase in the early years of a data scientist to ensure they build up a broad set of technical skills. The more experienced data scientists need to focus more on the why of their projects. If a manager doesn’t challenge his more experienced data science team members to grow and learn the business, then it is some what their fault for the lack of growth. Each year or every few months, managers need to help challenge the more experienced data scientists to make sure they are really growing. Otherwise, a business is losing out on maximum ROI.

Manage Up

Managing up is hard in any discipline. In technical disciplines it sometimes can be difficult depending on one’s managers technical background. Business focused managers might not have experience guiding technical teams. This makes it crucial for experienced data scientists to have the skills to manage up. A data scientist that is good at managing up understands people. Managing up requires a data scientists (or anyone at any company for that matter) to take the time to understand the bosses needs. Not just the company needs. What does the boss want from you, and from themselves. Once you understand that, you understand what is driving them. That will allow a data scientist to anticipate his or her bosses needs prior to being asked. This will help develop trust, as well as further investment from your managers and directors. This world is not all about your growth. Managers want to grow too! As an experienced (anybody) you know that helping others grow and meet their goals means you win too.

Summary

Data scientists are expected to do more than just create algorithms and manage large sets of data. Experienced data scientists value comes not only from their technical abilities, but also their soft honed skills. The information and algorithms data scientists create will push upper management decisions. Thus, whatever they provide needs to be understandable by directors and VPs that are managing millions to billions of dollars of people, resources, equipment, projects….and everything else in a company. This means in order for a data scientist to grow in value to a business, they must learn what the business will find valuable.