Analytics Interview Questions You Want To Ask Your Future Employer
If you're looking to get a new job in data science, you may want to evaluate the "data driven-ness" of the culture when interviewing. You'll save yourself the potential heartache of watching poorly designed hypothesis tests go out the door after you've informed stakeholders of the pitfalls. We often see tons of articles that discuss what hiring managers are looking in candidates, and candidates are studying and preparing for these questions to put their best foot forward. Interviewing is also a two-way street, there's a ton of information you need to collect for yourself as well.
Let's make sure that candidates are asking the right questions about the state of a company's data, so that they can make a truly informed decision about where they want to work. No one likes surprises. It's not that any of these questions would be a deal breaker necessarily, just make sure you know what you're getting into before you accept an offer.
There are many other questions I'd be looking to ask as well to assess the company culture. As my career has progressed, cultural fit has very much been the deciding factor when considering roles. It is absolutely important to work somewhere that you feel you can be yourself. After all, you spend a hell of a lot of time there. This article is not focused on assessing company culture, but is specifically focused on the questions you might want to ask to assess where a particular company is with their data transformation. By 'data transformation' I'm not talking about taking the natural log of one of your variables. I'm talking about the journey a company goes through while striving to modernize their approach to leveraging their data. Companies often start on a journey to become more data driven, and that's great! But it's also possible that you don't want to take a new job and find that all of their data lives in Excel spreadsheets rather than in a database (this exists folks).
Questions:
Ask: About the relationship with stakeholders. Who has the decision rights on final test design or analysis? Do stakeholders often make decisions that are counter to analysis?
I ask about who has decision right on analytics initiatives. Are analytics and their stakeholders thought partners who collaborate effectively? Or can raising a ticket for analytics feel like asking for food from a short-order cook?Would you like t-tests with that? With work, this relationship dynamic can obviously be changed. There's also the case where a position wouldn't have as much communication with stakeholders as typical data analyst and data scientist roles, there is always room for variability.
It is certainly worth clarifying in the interview whether you can expect your thoughts on methodology to be valued on day 1, or if there is significant work to be done to achieve that. It's probably important you understand the dynamics of this relationship before taking a role.
I also want to note, it's perfectly acceptable for an organization to make strategic decisions. These would be decisions counter to any analysis performed and you decide to do them anyways. Though I do believe that if that is the case your analytics resources should be focused on higher value initiatives and not on items where the final decision has already been made.
I Ask: About the state of self service tools for stakeholders. Are dashboards currently in place? Is there a lot of ad-hoc one off analysis (or are you working on high-value big impact projects)?
Do you love building dashboards? I enjoy making a nice dashboard occasionally, but if the stakeholders don't already have self-service data access, there's a high likelihood that this is what you'd be doing, and it's nice to know this in advance. Although I actually do enjoy building dashboards, the concern I have (and have experienced first hand) is when stakeholders cannot access the level of data they need, these asks now become "ad hoc analysis". These questions can be fun, it feels good to help someone find an answer, but they're typically not higher value questions. The time you help Mary from Marketing look at the price distribution of the cheapest items in the product catalog is not going to earn a spot on your resume. All positions will have ad-hoc queries, but you might want to make sure that ad hoc requests are a piece of the gig and not the majority of the gig. Ensuring that your future stakeholders can already access the data they need to perform their job will reduce the likelihood of constant ad-hoc requests.
Ask: About data governance. Do teams across the organization all have the same understood definition of the same fields or metrics? Or will you get a different answer for certain metrics depending on which department you ask?
Ever complete an analysis only to spend the next week digging into why your number is different from Jimmy's? It's not fun. When there's no governance in place, trying to get your numbers to foot with someone else can also be common. Being a data steward for tables leveraged by analytics was probably not the most exciting part of my job, but I'd take that anytime if the alternative was lacking data governance. When there are too many questions around how things are calculated, it's easier for people to change their mind about how things are calculated (to support their interests). My wish for you is a job where the data is correct and metrics are well defined.
Ask: Are they able to understand (or have data easily accessible) that would allow you to get at the full journey of the customer from first touch points all the way through attrition??
If you don't understand the customer journey, it's possible you're performing analysis on a disjointed view rather than the whole picture. Once you have the full picture of the customer journey, your previous beliefs, stories, and analysis might not hold up. Luckily, performing this analysis would lead to a ton of juicy insights and could be a ton of fun. It's also possible, that the organization you're interviewing with doesn't have the data to make this analysis possible. For instance, I've worked with companies that did not have website click data available. In many scenarios, it's hard to build predictive models without this, so much of what we care about in e-commerce is often tied to this behavior. Asking about the customer journey is my way of starting a dialogue about what data might be missing or hard to access. There's often a number of data sources that need to be integrated to go all the way from acquisition (and the channel that they came in on) all the way to churn. There's sales data, website click data, purchase data, customer service data, lots of data. This is all about having the relevant information you need to make an informed decision.
summary:
You can obviously still join an organization that has less than stellar answers to these questions. No one (or company) is perfect! My hope is you'll at least have a real good idea of what your job might look like and the challenges you might face if you choose to work for that company. The questions here are often the same problems we read about in blogs currently. Many data companies are still working their way through these! Once you take a job, remember to keep asking questions! I've written another post about asking great questions as a data scientist, you can check it out here.
Would love to hear your thoughts on questions that help you assess the current state of data at an organization.