How to Be a Truth-Seeking, Truth-Telling Partner in Data Science

Dave Cole

Welcome to the Data Science Leaders podcast. This is Dave Cole, your host. Today we have the pleasure of having Nate Litton, who is the Vice President of Data and Analytics at Toyota. Nate has a PhD in statistics from Texas A&M. He also happened to go to my alma mater in upstate New York at Cornell, an engineering school major coming with an OR, operations research, specialization. So welcome to the podcast, Nate. Thank you so much.

Nate Litton

Absolutely. Thank you for having me.

Dave Cole

This is great. So I think one of the topics that we were talking about that I think would be unique is the relationship between a data science leader and their stakeholder and their sort of business counterparts. So maybe just if we can talk at a high level, what types of stakeholders… Who have been your sort of business constituents throughout your career? And we can talk Toyota or go beyond that.

Nate Litton

So that’s a great question. The thing I’ll say, I always view data science… We don’t create and solve our own problems, so we always are solving problems on behalf of someone unless we’re in academia doing research. So my team at Toyota, and then this was true previously, but we support the broad organization. We have folks in sales, and in service operations, in accounting and finance, in legal, in compliance. We support a broad cross section of the organization.

The thing I would say, and this is the journey that we’ve been on at Toyota—and I think this will be a theme as we have further conversation—we can build up the greatest technical team anywhere, but if we don’t have the stakeholdership and we don’t have the partnership and we don’t bring the rest of the organization along, it can be for nothing. Because again, we don’t solve, we don’t create and solve our own problems. It really needs to be with our customers and stakeholders. And so this is something that’s definitely near and dear to me because we can’t do it in a bubble. We can’t do it in isolation. We can’t have problems thrown at us and then throw a solution back over the wall. So happy to dig in in any one of these areas because there’s a lot to dig into.

Dave Cole

You talk about how we don’t create and solve our own problems, and that is certainly true, but obviously you partner with your business counterparts. Give us some examples or maybe some tips and tricks in terms of those problems being thrown over the wall, like you say. How do you partner to first identify those problems and then go ahead and solve them? What are some pieces of advice that you have?

Nate Litton

Here’s what I would say. And I think Toyota is a good example, although I’ve seen this in other organizations as well. Toyota is a very relationship and high touch organization. And the thing that I strive to be and this thing that I strive to do, and I try to set the example for the team is, when I have the head of sales or the head of legal or service operations, and they have a business problem, I want to be someone that they want to be sitting at the table with them solving it. Not because I’m technical, not because we can build models or do this or that, but just because I’m a good partner, they trust me, and they want to engage me to help solve their business problems.

For me, that’s incredibly important, right? To be able to then engage and say, “Okay, well, let’s sit down together. Let’s work together to formulate what that problem is.”

Oftentimes, I’d say from a data science team standpoint, we often view ourselves as, “Hey, we’re building a model. We’re building a tool. We’re building a solution.” And many times, the most effective way, the most value that myself or the team brings to the table, is helping to define the right problem. Because oftentimes it’s not well formulated, we’re not solving the right one, or there’s a lot of opportunity there. And so I think that’s where it starts, to engage at that level.

And then, if it’s a technical solution we need to bring to bear, if it’s something more simple, like a dashboard that doesn’t involve any I’d say advanced analytics, but it’s, “Hey, here’s what’s happening in the business. Here are ways of thinking about it. Here are ways of slicing and dicing.” So there’s a range of possible solutions. But for me, it is around first establishing that trust, that partnership, and just being a preferred partner. Like, “Hey, I don’t care that you’re in data science or that you’re over here. I just want you to have a seat at the table when we’re trying to solve important business problems.” And that’s the measure I set and the thing I try to uphold, because if you do that, a lot of the other stuff just follows.

Dave Cole

Sure. And being a good partner, what are some of the characteristics that you see in establishing that great partnership? Clearly understanding the business problem, right, and not just being the tech guy in the corner or the data scientist in the corner, right?

Nate Litton

Yes. The thing I’d say here is, and this is true not just for data science, but for any technical discipline or maybe any discipline in general, is leaders, sometimes they make the mistakes of speaking…when you’re interacting with a broad set of stakeholders, you sort of speak in the language of your discipline. But trying to find ways of meeting them on their terms. Instead of talking about, “Hey, here are the measures of model performance or model accuracy,” trying to translate what that means to them in their terms. When I’ve seen it not go well, there’s too much technical jargon. Or, you know, “Hey, they’re talking above my head.” Our business partners don’t feel like they’re an equal partner in it. And “Oh, that group is just too smart for us.”

So I think that that’s really important. It’s important to spend the time. You have to get to know each other on a personal level, but also to really focus on kind of the “why” before you jump to the “what” or the “how,” really focus on the “why” and spending time and understanding, “Well, why is this a worthwhile problem to solve?” Let me translate what a technical solution looks like but explain it in terms that they can then go to their boss or they can then go to their teams and say, “Hey, here’s why this is important. Here’s why we need to engage with this group.”

Because a lot of times the challenge that I face is we all have ways of framing problems in our own lenses or in our own experiences. And oftentimes the people who are in the forefront, I mean, the functional leaders are owning kind of the business leaders that really have the needy problems, they don’t necessarily think of their problems as ones that data and analytics can help solve in better ways, right? Helping to connect those dots and to translate that is really important. And it just takes time. It takes patience. There’s a little bit of a salesmanship aspect to it, but you don’t want to overdo that. You want to push and engage, but not do it too much to the extent that, “Hey, this team or this person is being pushy and trying to make me do it their way.”

Dave Cole

I think it goes back to what you said about partnership, right? If you’re seen as someone who can hear the problem and is a thought partner…well, what does that mean to be a thought partner? Well, you’re asking questions. You’re trying to get to the actual core question that they’re asking. You’re not just seen as somebody who’s like, “Can you do this analysis? Can you figure out if this campaign was successful? Or can you figure out which customer demographic we should be going after?” And instead it’s like, “Well, what’s the problem?” Well, the problem might be, “Gosh, we’re really struggling to see incremental lift from our marketing campaign and we don’t think that we’re going after the right audience, but maybe it’s something else.” So being brought in at that level and then you can let the weeds of the actual analysis be done by your team and may the jargon be pushed to the side.

Perhaps you can talk about Toyota or wherever, how are you organized in such a way that allows for that strong partnership? Is there something organizationally that you have done or is it really just all about setting up those relationships?

Nate Litton

I’ll answer that in a couple of ways. I’d say in part at Toyota, and again, we’ve done this elsewhere in previous companies I’ve been. We’ve adopted agile in different ways where we bring different skill sets together—data science, data engineering, data visualization—in agile teams that then align with the business, align with products or domains to allow for fast delivery. So we’ve adopted a version of that for Toyota, and we had similar things in my time at Capital One.

The other part to that question is, when we look at the skill sets that we value as a team and as a company, we want people who can do technical work. When we think about data science, we want people who can build models, who can code, some soft skills, but also people who can engage with our customers and stakeholders, and that’s been really important. When we look at our team and how we’ve structured ourselves on the data science side, we have technical managers and leaders leading technical people, but we also have people who we feel comfortable putting in front of our business customers and engaging in ways that foster a strong partnership.

Dave Cole

Right. It seems like you have sort of these tiger teams almost of experts that have one person from data engineering, one person from data science. Kind of reminds me, I’m a big fan of Michael Crichton books. You know, Jurassic Park, you have the mathematician, you have the biologist, and you bring all these different disciplines together all to solve a single problem. You mentioned that you do want to have individual contributors who have the ability, sort of the soft skills, the partner relationship skills. Have you created a dedicated role in those teams as sort of like maybe an engagement manager or anything like that, or is the expectation that they each have the skills and can kind of stand on their own?

Nate Litton

It really depends. You got two parts of the answer. Broadly across the organization, in certain places we have dedicated roles, product owners, or engagement managers. I’d say in other cases, on more of our data science team, we’ve really just relied on ensuring that we’re bringing the right people in the organization. We’ll align them with different products and domains, products and services, but it would be on some of the managers and leaders that would foster those relationships. And it’s gone well, because it’s one of the most important things or the most important thing is people, people, people. That’s where we’ve been very deliberate and thoughtful in bringing in who we bring in in some of those more senior roles on our data science side.

Dave Cole

Absolutely. I think one important thing for any data science leader is to set up the rules of engagement and to understand what success is, right? Do you have any metrics that you worked with your business counterparts that you would like to be held to? I don’t know if it’s a number of projects or if it’s ROI, something along those lines?

Nate Litton

Yeah, we do. I’ll say just a little bit about the journey at Toyota. When I joined four years ago, we didn’t really have a data science team. We had pockets here and there, onesie-twosie, and wover the course of several years, we started bringing some of those together. We said, “Hey, we had success here. Let’s do more of that over here. Okay. What does that look like? We bring these people together. We create additional head count, do more of that… “ And we scaled that way.

While we scaled, and even now, there was more work out there than we have capacity to do. And so naturally, there’s A) how do we prioritize what we work on? How do we define and quantify the value that we bring to the organization? And then how do we communicate that to our stakeholders?

Actually one of the things that we put on our corporate scorecard is the business value that we create through the use of advanced analytics. And so, again, it provides transparency to the entire organization on, “Hey, here’s what this team is doing.” And it’s not just the data science team that I would say is doing advanced analytics. There are other things that you might call advanced that would go into that. But the idea is, “Hey, we want to generate X dollars in the upcoming year through the use of advanced analytics.”

And what that looks like, it’s subjective, right? We have to work with our business partners to help figure out what that is when we work on a project. But the way they think about it is if we were to do a simplified solution, here’s the value that that would drive. Here’s what charge-offs or delinquencies would look like. Here’s what sort of marketing spend would be. Here is our efficiency and incentive dollars, what sales volume we would get, and so forth.

If we did something more advanced, we built a predictive model or had a more sophisticated framework, here’s what that would look like. And then it’s really the delta between the two is the value that we create through the use of advanced analytics.

And we look at that across projects, we look at that across potential projects, and we summarize that throughout the year, and at the end of the year say, “Here’s the value that we helped drive as an organization.”

We’ve done it now for a couple of years. It’s been effective. It drives the right conversations with our stakeholders. It drives the right conversations in terms of priorities. If a project comes along and it’s five people over the course of 18 months, and it’s like, “Well, hey, there’s no value here. What are we doing?” So it prevents us… Sometimes the trap that we fall into in a field like data science where you kind of work on the cool and sexy projects, but it doesn’t drive a whole lot of business value or it’s not very impactful, it helps sort of guard against that as well. Not to say you don’t want to do the proof of concepts at times, but it’s a balance. It helps us to have the right conversations and to have the right level of transparency with the organization.

Dave Cole

I can imagine that dashboard isn’t just something that you show your stakeholders. It was probably also something that you show internally, because I think as a data science leader, or as a data scientist, I can imagine you want to work on those great, cool new projects. You want to do the latest deep learning project and play with the latest cutting edge technology. But if you’re working on a project that’s not moving the needle from an ROI perspective or helping to cut costs, and that project suddenly no longer is funded…If everyone has the same understanding of why you’re doing the work that you’re doing, I think it just makes it that much easier. And hopefully you can use deep learning on impactful projects or whatever it is.

Speaking of which, I’m curious, you mentioned that there’s more projects than you could possibly do. What is your intake mechanism for projects? Do you use JIRA? I don’t know, just get down to brass tacks. How do you collect these projects and then, more importantly, prioritize and figure out what the next project you’re working on is?

Nate Litton

To be quite honest, this is something we’ve had a lot of internal conversation on. What is a better and more effective way to manage intake? There are teams pulling more on the IT side where it’s more, “Hey, fill out this form. It funnels in and then… “ And sometimes that’s needed. I’d say on the data science side, we don’t have a really formalized process. The way that we’ve aligned ourselves is we have leaders that cover various domains, and we want really close integration and partnership with those respective areas. So when we’re coming up with our set of data science priorities, it is those leaders working very closely with those domain leaders to say, “Hey, let’s come up with a joint set of priorities for that area.”

And then once that’s established, then we can take a step back and look across data science. “Hey, are we heavy? Are we light? Do we need to allocate more resources here or there?” And that’s where there’s sort of a step or a layer of, “Okay, like there’s too much here.” Sometimes we have to say “No” or we have to defer, or we have to find a way. Like, “Hey, all these things are value added and it exceeds our capacity, and we want to find a way to be able to achieve that.”

It comes back to the partnerships, but it’s working very closely together across those domains because we want to be equal partners in us having a seat at their table and defining what they’re working on from a business standpoint, and then that feeding directly into the things that we’re choosing to prioritize.

Because oftentimes I see breakdowns where it’s like, “Hey, I think I know what you’re doing over here. I’m supporting you. I’m going to go do this thing over here.” And you come back together. It’s like “You built something that me as a business owner, I don’t want or I don’t need.” And I see this all the time. And so that partnership and that close kind of integration is really important.

Dave Cole

I think I stepped in it a bit. You’d mentioned partnership so many times and if you have like an intake mechanism, something like a JIRA or this ticketing system more generally, you’ve created a process where you’re sort of an order taker…“Fill out this form and then we’ll spit back out a model or a deck that answers your question.” And it doesn’t foster the partnership that you’re looking for. Now, you may internally have your own way of tracking projects, but it should start with that partnership and that conversation. Because I was going to ask, I imagine that some of the ideas on the projects don’t just come from the business side; they also come from your team as well, where your team might see an anomaly or might see something, and just say like, “Hey, did you know this? I want to do some further investigation.” Does this make sense?

Nate Litton

Yeah, absolutely. And that’s something just to add onto that, that’s something that is both fun and exciting about where we are at Toyota. There’s a lot of receptiveness on the business side where either it’s from a data standpoint or a data science standpoint, where we say, “Hey, based off of what we’re seeing or what we’ve seen elsewhere or latest trends in the industry, here are some things that we should be thinking about as a company. And here are some things that we should be thinking about as a business.” And to your point, it goes both ways. There are business priorities that then dictate data science need. But then there are things that we’ll come to the table with like, “Hey, we should be doing this” And help bringing our stakeholders along to understand why that would be a good idea or why it would be worthwhile. That then might cue up something on their agenda or on their side that they want to prioritize.

Dave Cole

Another thing that’s kind of unique about your role is, and this is fairly rare. I have seen it, but you report into the CFO organization. Is that correct?

Nate Litton

That’s right. Yeah.

Dave Cole

I can imagine if you’re reporting into the CFO, measuring ROI and measuring value and being able to justify your existence in some form or fashion has got to be critical. Is that true? Is my assumption correct?

Nate Litton

Yeah, it’s true. There is a, “Well, hey, what are we doing? Why are we doing it? Is it the right set of things that we’re doing? What value does it drive?” So yeah, there is certainly that lens. As I think it should, right? That’s fair.

Dave Cole

Do you have quarterly conference calls and things like that with your stakeholders?

Nate Litton

It’s not that level. Here’s the thing I’ll say.

Dave Cole

Earnings for data science. Sorry. I’m going to move on.

Nate Litton

Here’s the thing I would say that, because when we think around teams and how do we best structure ourselves… our data and our data science team does roll up into the CFO organization. And here’s the thing I would say. One is, we are a service, we’re a financial services company, or at least that’s the side of Toyota that we’re on. And a lot of the data is financial data, to the extent it drives financial reporting or other things. But I’d say more importantly, you could say, objectively, it might make sense for a data science function to sit here or to sit over there.

But I think from a senior leadership standpoint, when we think around where should data science sit, it should sit with the leader or leaders that are the biggest advocates for it. If that’s the CFO, if that’s the Chief Operating Officer, if that’s the CIO…It’s who is willing to spend the time, who is an advocate, who will continue to feed the discipline and grow its impact? And that’s why it landed where it landed.

Dave Cole

That makes sense. Let me ask you maybe a tougher question just in your career, but maybe explain some watch items. So if you were a data science leader today and you don’t have that strong partnership, do you have examples in your past of relationships gone bad, so to speak, that you could talk about?

Nate Litton

Yeah, I do. Because I thought about this a lot, I would categorize it into maybe stakeholders or engagements or partnership into one of three from a data science standpoint. One, the first one are the ones that we want where it’s, hey, you have an engaged partner, you have a trusted partner, and you’re working together to drive towards great outcomes for the company. The second would be you have an open and willing partner, but they don’t quite understand the value of analytics or what it is and isn’t, or the value it can drive. The third one is there is this sort of lack of openness, unwillingness to engage, kind of a demanding customer that no matter what you do, it won’t be right.

And so, we want more of the first one. We want to take that second bucket and try to move them to where they’re open and willing. Not just open and willing, but they are engaged partners and see the value.

And then that third bucket is certainly a challenging one, and that happens for I think a number of reasons. It could be recent failures that are top of mind. I tried data science last year and they weren’t able to solve with me.

Dave Cole

It doesn’t work.

Nate Litton

Yeah, it doesn’t work. You want to spend your time and energy where it matters. I would say that we want to be focusing on that second bucket and moving them to the first.

But the reality is, and this is true at Toyota and this is true where I’ve seen elsewhere, is sometimes those leaders in those groups, there is a ton of value to be unlocked from data science. They’re underserved, or we haven’t invested in analytics in those areas. Staying humble, trying to connect with those leaders and those people on those teams in constructive ways. Sometimes it starts with just setting expectations on what it is and what it isn’t. And sometimes one of the challenges I find is oftentimes there’s unreal expectations put on data science, like, “Well, hey, can I just click this button and it defines the problem and it finds the solution?”

Dave Cole

Right. Input data, output prediction. As simple as that.

Nate Litton

There are lots of reasons why partnerships can be difficult. I’ve seen examples where, “Well, hey, they don’t want to engage because they’re building their own sort of empire or there are walls up.” In companies I’ve seen sometimes, it’s “Hey, don’t worry about what I’m doing. You do your own thing.” And so there can be an unwillingness to share and operate in a transparent way.

People are complex creatures. You can’t influence and change everyone. The best I can do is try to be the example I want to set. One example I set for my team is… Because our team has grown substantially over the last several years, and so we have lots of connections and lots of interactions with a broad cross section of the organization. You know, “Hey, this partner is not… I’m having difficulty here.” And focusing on what we can control, focusing on the levers that we have, trying to influence the ones that we don’t have direct control over. Again, trying to be the partner that we want on the other side. These are just some things that we try to do.

The other thing is, if we’re working for a large company, we as individuals don’t own anything, right? We’re stewards on behalf of the company. So you want to leave the space better than you found it, and a lot of that starts with the partnerships and relationships.

Dave Cole

I would summarize it in a way, you’ve said it a few times, which is sort of kill them with kindness or be the partner that you want them to be if they’re not being the partner that they should be to create that great relationship. But is there anything that is a little bit more sort of prescriptive? Have you put together, I don’t know, a deck of like, “This is what data science is and this is what it isn’t.” And here’s some example problems that you’ve solved in the past? Things like that that sort whet their appetite and help build the rapport?

Nate Litton

Yeah, absolutely. So we’ve definitely put together even just a conceptual representation on here’s what data science is, here’s what business analytics is…trying to define what data science is and isn’t in the context and the language of the organization. I think that’s effective. Showing examples of very concrete, tangible projects that the team has worked on and delivered, and the value that it drove, what that process looked like. That’s pretty effective.

We’ve done this at times, sometimes we’ll go off and tinker. We’ll say, “Hey, we don’t have a willing or engaged stakeholder here, but if you can demonstrate something, a proof of concept…” Or, “Hey, here’s what this looks like and here’s what we’re seeing,’” it can be hard to deny, “Hey, this generates $20 million of incremental value in their areas.” Who’s going to argue with that? So sometimes it’s a combination of those. Influencing through knowledge and education and being transparent about what the discipline is, the projects, but also, again, sometimes it’s just like, “Hey, I’m going to go tinker with this a little bit and see what’s there and see what’s not.”

Dave Cole

You mentioned being transparent a few times, and being a data scientist, it’s all about delivering measurable results and all about data. I imagine there’s also some cases where you thought a model was going to help improve ROI and it didn’t, right? And you probably have had to show those results and said like, “Hey, this didn’t work as we expected.” And that helps. In an odd way, it helps, right, because it’s like this team can be trusted in good times and bad, right?

Nate Litton

Yeah. What when I first came into data science right out of grad school, I joined a team and one of the leaders at the time that I still get to work with at Toyota, but he was at a previous company, the lead there. And just trying to set guiding principles for data science and defining who we want to be as a team. One of the things that really stuck with me is we want to be truth-seekers and truth-tellers. And that’s exactly what you just described there. Even if it’s inconvenient, right? If it’s “Hey, there’s data that is there that points to there is no value in what we’ve done,” or “Hey, we’ve made a mistake,” or “Hey, this is something that we should put on the company’s radar,” we should be willing to talk about it. We should be willing to communicate it.

Sometimes that can be hard. If, as an example, you spend a bit of time on building a model that we thought there was value or a strategy where we thought there was some lift and we did a design of experiments and there’s nothing there, that can be hard to swallow but it does build some credibility and trust that you’re willing to do those things.

Dave Cole

Absolutely. That’s fantastic. This has been a great conversation. I think we’ve learned a lot about building rapport, partnership, delivering measurable results. I think in the back of any good data science leader’s mind, imagine if you did roll into a CFO like yourself and somebody who was really looking at the bottom line and wanted you to be measured. I’ve even talked to data science leaders in the past who’ve wanted to be held to a number, an ROI type number. I don’t know if that’s something that you’ve ever advocated. It’s fairly hand-wavy, and you can run into some problems there, but is that something that has ever come up?

Nate Litton

Going back to what I mentioned before around trying to quantify the value, the dollar impact that we have as a data science team. We have set enterprise targets for this. And so the first year we did it, we set a target and to your point, it was subjective. “Well, hey, this project’s in flight. It hasn’t been delivered or the value hasn’t been verified.” So we had some mechanism by which we took partial credit for things that were in flight, maybe full credit for things that have been delivered and verified, and a range in-between. In the first year we did it, we didn’t meet that target, but it drove the right conversations as an organization. We could have very well said, “Well, hey, we’re going to tweak the framework for how we assess value.” Then what are we doing, right?

But I do want myself and others to be held accountable for what it is we’re choosing to work on and what it is we’re delivering. And we’ve set targets for years. You’re here this year. What does that look like next year, two years from now, three years from now? And we’re going to continue to push ourselves. I would rather set big and bold targets and get 80% or 90% there, than say, “Hey, we’re just going to set something that we can guarantee we will meet.” So yeah, that’s something that we’ve championed as, “Hey, here’s how the organization should think about this and should measure this impact.”

Dave Cole

I think that’s great advice. Well, this has just been absolutely awesome. Are there any parting pearls of wisdom that you have for your fellow data science leaders out there or DSLs in training?

Nate Litton

The thing about data science, it’s a fascinating discipline because it’s so fast changing in terms of talent, skill sets, tools. For me, even when I came out of grad school 11 or 12 years ago, the skill set has changed. A lot of the tools that people are using now have… I came out of grad school using SAS…there was R and there was Python and open source tools that we’ve, as a discipline, have more full heartedly embraced them now versus then. So for me, it’s around always being, no matter your position or how many people you have on your team, always being a student of technology, of trends and methodologies, and areas of application. I find that even finding ways to still carve out time to be hands-on, if you are more well-informed as a data science leader in terms of the trends and the industry, the more effective I think you’re able to be for the team and for the organization.

Dave Cole

That might be our next podcast is how to stay up on trends, Nate. Not for the faint of heart given the pace of change with data science, which makes the field both exciting and challenging all at the same time. But I really appreciate you taking the time. This has been fantastic. Do you have social media, I know you’re on LinkedIn, but is there a way for people to get in touch with you in case they want to reach out and chat with you?

Nate Litton

I’m on LinkedIn so if anyone would like to reach out and to have more conversation, certainly open and willing to do that.

Dave Cole

All right.

Nate Litton

Thank you for the time.

Dave Cole

Well, thank you so much, Nate. Be truth-seekers and truth tellers. I love that phrase. We’re signing off. Have a great rest of your day, folks.