This is Part Three in a Three-Part Series
Part One: What is Data Science? | Part Two: Learning | Part Three: The Job Market
CHAPTER FOUR: THE JOB MARKET
Time for reflection. If you are ready to gain employment in data science, think how far you have come. You started with an aim, and with a tremendous amount of effort and dedication, you have turned that aim into what you are today: a data scientist in waiting. But don’t uncork the champagne yet, because there is a very large hurdle to conquer: the job market.
As a recruiter, it is easy to forget that most people do not have the exposure to see how this murky world ties together, and so for the inexperienced, it can be fraught with pitfalls. As such, my intention is for this chapter to act as a beacon of light, so you – the employment-seeking data scientist – can approach this challenge equipped with all the know-how you will ever need. But before we dive in, there is a crucially important message to digest:
It isn’t enough to have the capability to do the job in question; others will also be qualified and they will probably have more relevant commercial experience. So you can’t stop at simply being able to do a job – you have to go further.
This chapter will cover how to ‘go further’ and explore why this is necessary, starting with arguably the biggest roadblock affecting those with experience in other disciplines, or the academics out there with industry aspirations.
A brief note before we get into it: gaining employment isn’t the only route to commercial data science, as you might want to leverage your expertise to launch a start-up. While I applaud the enterprising readers who follow this path, this a whole topic in itself, and one I am not qualified to advise on in this post.
The Experience Bias
Even if you possess the technical capability or the potential to learn very quickly, most employers have a strong bias for hiring those with commercial experience. Although the logic is questionable, it is generally because businesses aim to hire individuals who will be up to speed in the quickest time possible, and those who are least likely to be a failed hire. Employing someone with relevant commercial experience is seen as a simple way to mitigate this risk, not just in terms of capability, but also in regards to ‘culture fit’ (more on this later).
There are exceptions of course, and you tend to find the more enlightened data science teams concentrate on core problem solving ability, rather than experience in itself. This makes a lot of sense, because an interview process that effectively filters for problem solving has the benefit of removing human bias associated with an individual’s background. Thus, it is possible to unearth stars that will probably be undervalued, or even ignored by the rest of the market.
Sadly, there are not many teams that operate like this in Australia – where I ply my trade. Perhaps this is different elsewhere, but here, the industry is still developing and there is not a huge start-up scene, so most data scientists are employed in incumbent industries where their work is heavily affected by bureaucracy, politics and legacy infrastructure. Subsequently, there does not appear to be enough market demand to cause a significant talent shortage, which might drive organisations to consider those with different backgrounds *. I cannot comment on the relative levels of supply and demand in other regions (and whether this has an impact on hiring patterns concerning those without commercial data science experience), but if any informed readers have any insights on the matter, I would be fascinated to find out.
* Disclaimer: I wrote previously in Big Data, Data Science and Analytics in Australia that there was in-fact a talent shortage in Australia. While this may well occur in the future, my observations since the time of writing have contradicted my initial research.
Junior Positions
A quick word on junior positions, i.e. hires that require little or no commercial experience: I have had numerous conversations with experienced individuals who have asked me to consider them for junior positions, as a way to break into data science. Fair hiring practices aside; the issue here is teams usually prefer employing younger individuals in junior roles to avoid upsetting the team dynamic, although I doubt anyone will publicly admit to this. As a consequence, the experience bias doesn’t really affect graduates in the same way it does with experienced individuals from academia or other relevant disciplines.
With this in mind then, how can you counteract this bias, if you have the capability, but not the background in industry?
Kaggle / Open-Source / Freelancing
If you read Part Two, you will know we have been here before. And we return for good reason. Competing in Kaggle competitions, freelancing and contributing to open-source projects are not just ways to learn and improve; they also evidence and promote your capability in a way that an online course with Coursera/Udacity/edX never can.
This is especially pertinent if you score highly in Kaggle competitions: people take notice. In short: this is probably the best way to nullify the experience bias, so what are you waiting for??
CV / LinkedIn / GitHub / Blogs
To evidence and promote your capability, add all this to your CV, LinkedIn and GitHub, if you have a profile. When I look at a CV for the first time, I am looking to understand the person’s story: what is their background and how have they developed their expertise? Make it clear. And include a concise profile summary explaining your background, your skill set and your objective. It all helps. Your CV does not need to be two pages, but at the same time, it shouldn’t be an essay, so try and strike a balance. Finally, if your written English is not the best, ask for help – first impressions matter.
On their own, those online courses will not be enough to convince any employer that you are better than other applicants (and everyone completes those courses). So think about your strengths and how you can differentiate yourself. Have you got topics you want to write about? Why not start your own blog? Or you could post on LinkedIn Pulse, or approach Experfy, as I have done here.
The Competition
Thinking about all this stuff is important, because – chances are – you will be up against people with industry experience. To revisit the message we touched on at the start of this chapter:
The fact that you have the capability to carry out the job in question is irrelevant if your competition is selected for interview, and you are not.
And this is a very real risk because you will probably be facing competition from data scientists with a track record in commercial organisations, and very plausibly with pertinent domain/industry experience as well.
To reinforce the point, I have spoken with many aspiring data scientists who are dumbfounded as to why they have not been selected for interview, when they are certain they have the right expertise. But what does it matter if you do not even get through the door? And remember – there might be a recruiter acting as a gatekeeper, who might not understand why an ex-physicist (for example) might actually be appropriate.
Provided you can evidence and promote your expertise effectively, the best way to beat your competition is to simply be better than them. Clearly, this won’t always be possible, but if you aim to be the best you can possibly be, you will give yourself the greatest chance. But this is not to say you have to be a better data scientist than your competition to land that first role – there are many influencing factors in play. For example, you might encounter one of the more enlightened teams, there could be a market shortage of data scientists in your region, or you might just hit it off with the individuals you interview with. Or perhaps you are recommended through your own network, which we will look at in the next section.
To wrap up this discussion with some perspective: there is no value in fearing the competition. Embrace it. Understand what you are up against. By accepting this, you can then focus on what is in your control, i.e. not only getting better as a data scientist, but also evidencing and promoting your capability via every channel possible. And if you do this right, you will help level the playing field. Tell your story!
Networking
Getting. The. Break. Everyone I met for this project got a break, one way or another. Dylan Hogg even got his before he completed his machine learning diploma, because a founder of The Search Party was an ex-colleague and wanted him on-board. So he joined as a software engineer and was given the chance to continue his personal studies with the ultimate goal to transition to data science when he was ready.
It is worth acknowledging: there is always the chance of getting lucky through an online application. But if you have come this far, it would imprudent to risk all that hard work by relying solely on this approach. Networking then, is crucial to landing that first job. And lucky for you, it couldn’t be easier these days. Whether or not you have pre-existing contacts like Dylan, get on Meetup and find your local groups. Not only are they an invaluable source of information, but you will also have the opportunity to meet the community and find out what is going on in the market.
One of the great things about data scientists is their passion for their field (without it, how would they have got to where they are?), and this means they are usually very active in the community. While this tends to foster a close-knit group, I have also observed a welcoming culture towards newbies and non-experts. As such, if you have something you could present, why not approach the organisers? I can’t think of a better way to gain exposure.
Be aware though… Recruiters also attend these events, and considering the role they play in the job market (until algorithms take over), it is only right that we dedicate some time to them here.
RECRUITERS!
I am one of these! And what an interesting bunch we are. It is no shock revelation to say that our profession has a bad name. And here is the thing: it is for good reason. Due to intense competition, low barriers to entry, and an inability to attract and retain intellectually curious people, mediocrity is endemic in the industry.
Of course, good recruiters do exist – but they are few and far between, especially in complex technical fields. But what constitutes a ‘good recruiter’ in the first place? Well – in simple terms – I would characterise it as those with an established network of clients, and a good comprehension of the field in which they recruit. It is these individuals that are worth having in your corner, as they will know which businesses hire your type of skillset, and the reasons why *. This brings advantages through:
- Enhancing the efficiency of your search by eliminating the businesses / positions that are not relevant, which isn’t always possible to deduce from job advertisements / descriptions
- Connecting you with opportunities that you might not have come across on your own
That is the theory anyway. In practice it is tricky, because how are you supposed to distinguish the ‘good’ from the ‘not-so-good’, or from the ‘down-right-terrible’? To clear this up and more, perhaps a full account of how recruiters operate is in order, and what better than this coming directly from one who specialises in data science?
* For the sceptical reader, it might interest you to learn that even I used a recruiter. I know, I know – a recruiter using a recruiter… madness.
1. Identifying Recruiters
Word-of-mouth. Good recruiters will have a reputation so recommendations from the community are simply the best place to start. I wouldn’t worry about what agencies are best; it is more important to identify the individuals.
Failing this, monitor the job sites for well-written advertisements and search for recruiters on LinkedIn. If the content is original and not the usual clichéd drivel, then the recruiter probably knows what they are doing.
Once you get talking to recruiters, it should be obvious if they have a clue or not, simply by what questions they are asking. But at the same time, do not be afraid to pose them some testing questions (it always amazes me how rarely people do this).
2. No Recruiter Will Work With Every Business
Great, you have identified a respected recruiter and you are confident they will represent you to the best of their ability. But is that enough?
Probably not. It is important to realise that however proficient a recruiter is, they will not work with everyone. If you want to work through recruiters to give you good market coverage, or you want to use them to target a specific business or industry, you will need to find out what pre-existing relationships they have.
This is where it gets a little complicated. With so many competing agencies of varying specialities, a business often uses different agencies for different areas. Or they might have a preferred list of agencies (a ‘PSA’ or ‘PSL’ in recruitment speak), but have agreements to use other agencies when the preferred list cannot deliver. It is not important to understand the complexities here, just remember to probe the recruiter with a lot of questions about their clients and the nature of these relationships (asking where they have placed is a good starting point). Once you have this information, you can then attempt to identify a few recruiters with different client bases, which will then give you good coverage.
A quick word on trust though: this should go both ways. Just because a recruiter has discussed their client list with you does not mean they have ‘ownership’ over your future applications to these businesses. But at the same time, if you have no intention of using the recruiter and you are simply trying to gather information, that isn’t right, is it?
3. Recruiters Methods
When you are dealing with recruitment agents, it really is valuable to understand their incentives, most notably: how revenue is actually created. In most cases, recruiters are paid by businesses to fill their open jobs, and usually on a contingent basis, i.e. only after a placement is successfully completed. They are not typically paid by job seekers, and so even the very best recruiters will not place every candidate – they are there to fill jobs, not help every job seeker. However, for ‘highly placeable candidates’, it is in their interest to be pro-active, and it is therefore useful to be aware of the following method that is regularly employed in the industry.
In recruitment-speak, this is referred to as ‘reverse-marketing’. It is the practice of introducing good candidates to relevant businesses, irrespective of whether they have an open hire. I have successfully placed people using this method many, many times, and when done properly, it can be beneficial to all parties. Reason being – it has the potential to unlock opportunities that were not previously known, either because of fortunate timing, or occasionally because the business creates a new position as a result of the introduction.
Quite understandably, job seekers can be wary about their details being sent around without their knowledge, but the recruiter should be transparent with you. To make certain, probe them about their methods and whether they intend on employing this tactic on your behalf (as not all recruiters do this, and even if they do, ‘reverse-marketing’ is typically only conducted for the best and most relevant individuals). If it is their intention and you provide your permission, ensure that you agree the list of target businesses in advance, so you retain control of your search.
Take note that recruiters will often use this technique with organisations they haven’t worked with before, as it is an effective way of developing new business relationships. In this scenario, it might be in your best interest to approach the organisation directly, or use an agency that has a pre-existing relationship in place. However, I have also placed several individuals in this way, and created some very rewarding partnerships in the process. So if you trust your recruiter, why not allow them to try on your behalf? And if they have no luck within an agreed timeframe, you can then explore other avenues *.
* If this is unsuccessful, it is highly unlikely to harm your future chances of employment for two reasons – one, a ‘reverse-market’ of this nature will be conducted on an anonymous basis, i.e. your skills and experience will be summarised but no identifying information will be shared. And two, with no pre-existing contracts in place; there are no terms to dictate that the agency has ‘ownership’ over future applications.
4. Clueless Recruiters
Should you work with them? For a highly technical field like data science, I tend to think you should avoid the fools and partner with recruiters who have a good comprehension of the subject matter. However, like most things in life, it is rarely this simple.
Let’s say that a recruiter who is clearly lacking in the ‘understanding’ department approaches you about an interesting opportunity, and by dumb luck and some matching key words, the position is actually relevant. Etiquette might suggest that you should agree to be represented by the person that first informed you of the opening, but I flatly reject this – your your job search is too important. If you really have no confidence in the recruiter, why not look at other options before giving your approval? You could explore applying directly, or you could find out if other, more trusted consultants are able to help.
Look – if you are the best person for the job and are represented by a clueless recruiter, it might not matter either way. But given the choice, it is preferable to go with someone you rate every time, especially if they have a good relationship with that business. I know from my experience: when I introduce a candidate to my established clients, quite often they will agree to an interview before they have seen a CV. Why? Because they know my track record and expect that the people I introduce will be relevant and worth their time. And a good recruiter doesn’t just act as a filter; using their knowledge of the business, the team and the interview process, they will help prepare you for the application and the subsequent interviews. And if the hiring manager gets the wrong impression from your CV (this happens more than you might think), the recruiter is in a position to push back and correct the misunderstanding. But in a complex field like data science, they will only have this influence if they have the required understanding and credibility.
5. Other Things To Be Aware Of
Giving Your Approval: Whatever happens, if you provide your authority to be represented, stick to it. Do not then give your approval to another recruiter or apply directly – it makes everyone look bad.
Written Confirmation: When you agree to be represented to an open position, request for the recruiter to confirm in writing once they have processed the application on your behalf (as it isn’t unheard of for recruiters to tell job seekers they have processed their application, when in truth, they haven’t)
Honesty: You do not have to tell recruiters where else you have applied (as they might even use this information to approach that business in the attempt to place another candidate of theirs), but be honest in your feedback and keep them informed of your progress elsewhere – it is only fair.
Salary: Considering the higher costs associated with using a recruiter, you might assume that this will negatively affect your remuneration package. However, in reality, it just isn’t that simple. As an example: if you are in demand from multiple businesses, it usually leads to competitive salary offers, irrespective of on-top recruitment fees. Furthermore, it isn’t really a factor in large organisations as it is generally accepted that recruitment comes at a cost (even if third party agencies are avoided, internal recruitment is not exactly cost-free). That said, of course – in particular circumstances involving budget-constrained businesses, higher recruitment costs might result in lower offers. But honestly – there are so many variables; it isn’t worth worrying about. And remember: on the flip-side, a recruiter can actually have a positive impact on your salary by using their knowledge of market rates to negotiate the highest offer possible (after all, recruiters deal with offers all the time). Finally – you might have never known about the opportunity if it weren’t for the recruiter, so in this circumstance, what is better: a (potentially) slightly lower offer, or no offer at all?
Bias: While an informed recruiter can be a great source of knowledge and advice, always be aware that they have a vested interest in you joining an organisation through their representation. As with any perspective lacking objectivity, it doesn’t mean they will outright lie, but just like you should be sceptical of a nutritionist selling a diet plan, be critical and question their claims. My advice: seek guidance from those you trust who have nothing to gain from your decision.
Apply Direct
With the information you now have in your arsenal, you should be able to leverage recruiters to your advantage (exciting right?). However, not all companies use third-party agencies (especially start-ups) so it is advisable to be pro-active with your job hunt. Research businesses that employ data scientists (LinkedIn is great for this), approach them directly, and monitor where businesses advertise including their career pages. You know what to do next.
Advertisements / Position Descriptions
Both agency recruiters and businesses directly advertise open positions. An important message on the content: DO NOT WORRY ABOUT THE DETAILS! They are often generic, clichéd and list every skill and technology under the sun. This is usually because a recruiter with poor understanding has written it, or it has been rehashed from old versions to save time and is therefore not all that relevant.
My advice then: do not worry about ticking every box; try and get an overall sense of the position to determine whether your skillset is suitable. But beware: it is not uncommon for agency recruiters to post fake advertisements as a way of ‘fishing for candidates’. There isn’t really a good way to spot these, however if the same recruiter continues to have mysterious openings that are suddenly filled or cancelled, then this could be an indication. Or they are just really unlucky.
Internal Opportunities
This should be obvious: if you are working for a business in a different role, say as an engineer, your best chance may be an internal move. This is easier if you are working closely with the data science team, but either way, network internally, make sure you communicate your ambitions, and promote the steps you are taking to develop your expertise.
Interviews
You are nearly there. And if you have developed enough capability, interviews are nothing to worry about – your ability will shine through. A couple of tips for you though:
- Communication: As we discussed in Part Two, this is an intrinsic part of data science, and it is one of the most common reasons why data science interviewees are rejected. To repeat then: you will be doing yourself a disservice if you underestimate this skill, however technically advanced you are.
- Preparation: Do your research on the role, the company, the people, and if applicable, have a play around with the product. Take the time to understand the business, and go to the interview prepared with ideas on how you would employ data science to create business value.
The only thing I will add is this: any real data science interview will test your problem solving, most likely through practical exercises. You cannot game these. If you can, I have no doubt that your role will be data science in name only.
Terminology
Overlook this at your peril. It is particularly relevant to the wider science field, as Will Hanninger explained:
“Coming to the bank with lots of pure machine learning guys, I needed to first learn the language; it’s full of different jargon. For example, a variable in physics means a feature in machine learning, and a set of variables means a feature vector”
If you do not have patient interviewers (Will did), you might find yourself without a job due to miscommunication. And this is the precise experience Sean Farrell had with his very first interview – not exactly ideal.
‘Culture Fit’
This is a term that is thrown around a lot, and sometimes used as feedback to explain why a candidate was rejected following an interview. Obtaining honest interview feedback is so useful, and being rejected due to ‘culture fit’ can be frustrating because it doesn’t seem to explain the reasons in sufficient detail.
While it might occasionally be used as a cop-out when an interviewer simply prefers someone else, usually it is legitimate feedback, albeit lacking on the specificities. If you remember Boris Savkovic’s quote towards the end of Part One, it is clear that commercial data science has very different challenges than many other related disciplines. So ‘culture fit’ is normally code for: “we have concerns that this individual might not work well in our environment”. This could be due to a number of reasons: maybe it is your communication, maybe they think you would get frustrated with internal politics, or maybe they feel your interests are better aligned to theory/research than the work they have.
Either way, if you receive this feedback, go back and push for more specific information, so you can improve for future interviews. And try not to get down if you fail a few – unless you are an absolute superstar, it is rare to get job offers every time.
CHAPTER FIVE: CONCLUSION
I set out with the aim of writing something that would cut through the hype and misinformation to help any aspiring data scientist; not only those on the learning curve, but also those embarking on a job hunt. And while it wasn’t my intention, this took me on a journey that questioned the very nature of what we currently mean by ‘data science’.
My hope is that I have done this topic justice, and by doing so, I have achieved what I set out to do. But more than that: I hope it causes you – the reader – to question your motives to ensure you are going down this path for the right reasons. And if you are, I hope you are now equipped with the knowledge to set real goals and begin working towards them.
But above all else, I hope that the key messages are abundantly clear; that to master real data science, it takes time, dedication, persistence, and above all else: passion. Without this, you simply won’t develop into a problem-solving data scientist – however naturally gifted you are. Yes, the market considerations are important, but everything else should be a secondary consideration to the quest of mastering your skills.
I will end on this message: every single person I spoke with worked incredibly hard and ended up getting a break to reach their goal of becoming a data scientist. There are times when it will become tough, but if you have enough passion and persistence, you will get there. And so I leave you with this: GOOD LUCK!
A massive thank you to Will, Dylan, James, Yanir, Boris and the two Sean’s: quite simply, these posts would not have been possible without your input.
