Featured Guest: Melissa Koide

Melissa Koide is the CEO of FinRegLab, a nonprofit research center that tests new technologies and data and facilitates dialogue to inform public policy and drive the financial sector toward a responsible and inclusive financial marketplace. FinRegLab evaluates how technology and data can be safely used to increase financial inclusion and improve financial services for consumers, small businesses, and communities. FinRegLab is currently evaluating the explainability of complex machine learning algorithms in credit underwriting for fairness, model governance, adverse action notices, and inclusion.

Prior to establishing FinRegLab, Melissa served as the U.S. Treasury Department’s Deputy Assistant Secretary for Consumer Policy. In that role, Melissa helped to build the first government offered preretirement savings product, the myRA. She also established the $5 million Innovation Fund to support research and strategies to improve consumers’ financial health and their access to safe and affordable financial products and services. Melissa has testified before the Senate Banking and House Financial Services Committees, and she has spoken extensively to policy, industry, and consumer advocacy audiences.

Episode Transcript

[00:00:00] David Reiling: Welcome to the NextGen Banker podcast, where we explore what’s next in banking and talk with the innovators responsible for creating positive change in the financial sector. I’m your host, David Reiling. I’m very excited to welcome Melissa Koide today as our guest. Melissa, welcome to the show and thank you for being on the NextGen Banker podcast.

[00:00:24] Melissa Koide: Oh, thank you, Dave. This is so much fun. I’m glad to be doing this with you.

[00:00:29] David Reiling: Well, thanks. I’ve looked forward to this, and just for our audience I’ll give you a little background in terms of Melissa. So, Melissa is the founder and CEO of FinRegLab, an innovative center that tests new technologies and data to inform public policy and drive the financial sector towards a responsible and inclusive financial marketplace. She is also the Vice Chair of the Milken FinTech advisory committee, which seeks to educate policy makers and industry stakeholders on the impact of FinTech and its implications on public policy. Melissa has served as the Deputy Assistant Secretary for the US Department of Treasury. She was also the Vice President of policy at the Center for Financial Services Innovation, now known as the Financial Health Network. And so, Melissa, it’s great to have you here today. And I would love to just jump in right away. I’m really curious with my entrepreneurial mind is how did you come up with the idea to start with, for FinRegLab? What, what was the spark that said, “Hey, let’s do this”.

[00:01:30] Melissa Koide: Well, thank you. I’m going to slip in at some point in our conversation, how long you and I have known each other too and where that goes back to. But let me answer that question. I stood up FinRegLab after spending, you and me both my friend, you know, 20 plus years really focused on the needs of underserved communities and individuals who are challenged in terms of accessing safe and affordable financial products and services.

[00:02:06] Melissa Koide: Maybe I will slip it in here now. You and I go back many years, even in looking at prepaid cards. I mean, I think that was early days how you and I got to know each other. I was at CFSI, what’s now the Financial Health Network and we were spending time at conferences together. I was learning a lot from you about what happens in practice in terms of providing accessible products, in that case transaction products, to lots of people who need to function and operate in a financial system. And need accessible products to make that happen.

[00:02:43] Melissa Koide: FinRegLab is an organization I stood up after spending four and a half years in the US Treasury Department. It was at a point in time when we weren’t even using the term FinTech, we were still referring to non-banks as non-banks. But at the same time, sitting in the treasury department and engaging with my regulatory college we could see the potential benefits, but also the uncertainties of using new types of data and emerging technologies in financial services, whether it was offered by a non-bank or a depository.

[00:03:23] Melissa Koide: And the punchline here is that we, in treasury and across government, had a research arm for looking at systemic risk issues. This is the office of financial research under FSOC. We didn’t have any independent source of empirical information to help us policymakers think about what are the potential benefits of a data use case in retail financial services, or a new technology that was emerging, and what importantly are the limitations and what are the risks? And frankly, especially for policymakers, I think it’s really challenging to then determine, how do you advance public policy without that fact-based information? And so FinRegLab is the nonprofit organization that I stood up after leaving the treasury department to serve hopefully not just as the one and only research organization, there are others, but definitely one that could provide fairly insightful, from a market and policy standpoint, timely, empirical research, and then importantly, and you know this as well as I do, then to be able to bring all those different stakeholders around the table, along with policymakers, and say, okay, what did these facts tell us about how we are going to evolve law and regulation so that we’re getting the best and the most benefits and importantly, the most inclusion we can possibly achieve, while making sure that consumer harms are mitigated as best as possible.

[00:05:02] David Reiling: I have to tell you, as soon as I heard that you were doing this and I’m like, this is brilliant because it’s in the data that not sometimes all of the truth lies, but a lot of it does. There’s that empirical data to go back to whether you’re that depository institution or you’re that consumer advocate, we’re all looking at this from a slightly different lens, but the data tells us something and it tells us a lot. And in particularly when it comes to regulation, which people may not understand is that it’s hard to make good regulation, to make it fair and to make it inclusionary and so forth. And so, to have that database sounds great. It’s just such a good basis to have that conversation.

[00:05:48] Melissa Koide: And even getting everybody around the table. I mean, it’s sad, but true, we tend to be so polarized, especially when it comes to public policy and policymaking. And so, if you can get everybody around the table to say, what are those juggernaut questions around data or technology? And you keep everybody at the table, and you keep the dialogue going, you get the research done, and then you keep that conversation going. You build trust and a depth of understanding about where the different parties are coming from. I mean, it doesn’t solve everything, but I think it is an important model, frankly, for how we do policymaking in a larger scale too.

[00:06:28] David Reiling: Yeah, definitely you have to keep that dialogue going, there’s no doubt about it. So, when I think about this, I really kind of, I want to touch bases with the, in terms of AI, Artificial Intelligence. So, I’m really curious as to what you’re seeing out there, what’s piquing your interest? And because you have this data, you have this lens, you’re working with FinTechs of all kinds, what’s of interest today?

[00:06:53] Melissa Koide: Well, and it’s what’s of interest in AI and financial services and where’s there work that would be instructive in financial services, but that might also be instructive even in other sectors. And a couple sort of top line thoughts there. I can talk about this in more detail, but I think we’re all worried about the histories of inequality and how much those realities are baked into the data. And in many respects data that we naturally turn to and for purposes like credit underwriting, how do we make sure that more complex math that may be much harder to understand isn’t perpetuating those inequalities.

[00:07:44] Melissa Koide: And I think probing that question, especially in a high risk, highly sensitive area like credit access, will help us start to understand how we make sure that we’re not baking in those histories of inequality as we move forward. So that’s a big one that’s definitely top of our mind.

[00:08:03] David Reiling: Yeah, that I can believe.

[00:08:05] David Reiling: I’ve heard two different responses to that. So, from more of the market-based players, like you’d say, hey, when the bias is built into the system, you’re not efficiently serving the marketplace and therefore it automatically corrects itself. And I’m like, okay, I can sort of see that. But we’ve also had the system working for years and there’s bias included in that system as well. And so, I can’t see how it’s not baked into the algorithm as well. There is going to have to be some adjudication of the data and so forth as to how it’s being used. So, AI needs some parental oversight. It ultimately needs regulation. Is that where you think it’s headed?

[00:08:48] Melissa Koide: Well, I actually want to take apart what you just said cause we have been thinking about this with the big projects we’ve had underway. Now we’re coming up on the second year of a three-year evaluation of machine learning in credit underwriting and these questions. One is, if, and we know this, we have up to 50 million people who have an insufficient credit history, a thin file or no file is our shorthand for it, and we know that we have higher incidents of minority populations within that sort of un-scorable population. Um, so we know at a minimum, we don’t have sufficient data to actually accurately credit risk assess populations, who we want to make sure, and frankly, who very well may be eligible for credit.

[00:09:44] Melissa Koide: We need more data to really sort of understand the credit risk of populations who were excluded. So, the more that we look at this, and are our partners in the academic community have done some really important research, the AI and machine learning may be especially helpful for better credit risk assessing. But ultimately the AI and machine learning are going to be responsive to the data that they are trained on and if your data are insufficient in terms of really understanding a person or a small business, then the AI isn’t going to solve anything for you. And so, I think underscoring just how important bringing in more inclusive data is, is half of it.

[00:10:32] Melissa Koide: The other half though is answering, I think what are some basic empirical questions that are necessary before we move into “Okay, do we need to regulate and how do we regulate?”. They’re really complex but very basic questions around, given the different types of machine learning algorithms that are being used, or that could be used, from simple XG boost models to much more complex neural nets, how confident can we be? How much can we trust in the output that they generate?

[00:11:09] Melissa Koide: And I don’t want to do too much of the talking here because I love our dialogue, but, but I I’ll just say this. I think what’s really somewhat encouraging in looking at these questions in financial services is the fact that we have laws in place that require people who are using these algorithms to respond to questions of, well how was a credit decision generated, right? You got to explain it to the consumer. Do we see risks of disparate impact? That really is one of a number of ways that you would evaluate fairness. And so, these are laws on the books that force some transparency. The work that we’re doing is asking, empirically, how much confidence can you have in different methods of generating the answers to those questions?

[00:12:00] Melissa Koide: And starting there, we can then, I think, think about, okay, how competent can we be? Do we need changes in our laws and regulations or not?

[00:12:10] David Reiling: Just to get maybe super nerdy for a moment. It almost seems like since you have this AI and this machine learning happening, is there a way to do a feedback loop into test its level of confidence? Meaning we approved 80% of this population, we really could have approved 83%. Why did we miss this difference? If there’s a way to back test it somehow. Again, the more data, theoretically, you feed into the system, and again it’s gotta be good data, which is another kind of question. So, you know, you put bad credit data in a credit report, guess what you get.

[00:12:57] David Reiling: So, it would be really interesting to see if it could, in fact, benchmark itself somehow for accuracy, but let me take you in a different place because it’s really easy, I think, to go down the negatives of AI and in machine learning sometimes, have you seen anything, maybe particularly because the pandemic, maybe accelerate AI for good? Or for machine learning for good? Is there anything is like, oh, it’s refreshing, it’s giving people access?

[00:13:26] Melissa Koide: I might just be making my similar points, but, as painful as the events from the past year and a half have been, I think the reality of people struggling, the stimulus payments have helped, but I think there’s a whole lot of awareness of just how unequal and how challenging it is for households to sort of maintain financial stability and avoid the fragility. And I think those realities plus the increased attention to AI means that there is more of an openness to, “well, how could the data plus more complex analytics actually do good?” and I think there’s even, even among people who want to make sure that we are always mitigating the harms to consumers, I think there’s an openness to “wait, we gotta do this differently.” And do the math plus looking at rental data.

[00:14:35] Melissa Koide: Fannie Mae just announced, for instance, that they’re going to be encouraging and facilitating the use of rental data for mortgage underwriting. I mean, you and I both appreciate how mortgages are sort of the biggest asset for most households. I think we’ve turned a corner and I think there’s recognition pragmatically. We’re not going to put the genie back in the bottle, so we need to figure this part out.

[00:15:00] David Reiling: It’s interesting from my side. I mean, we see a better mouse trap every day, almost, in terms of all the different models out there. There’s a better black box constantly that a FinTech is saying, “well, we’re better because we can do this. And then we have this angle on it.” And so, while you know, that can be fraught with danger, I get it, but the fact is, is all these experiments are happening. And it’s just the part of the testing that has to take place, I think, to hopefully get to a good conclusion with maybe a couple of guard rails on the bowling alley to keep us in the right direction.

[00:15:30] Melissa Koide: Totally. Every time you say that by the way, the last sort of event that we did together, I’m like, oh, I start salivating. Like, Ooh, who’s he hearing from? What do I not know? We definitely need to have an open line.

[00:15:44] David Reiling: Absolutely, cause I need to check with you. Have you heard of these people? Or I need a quick reference.

[00:15:51] David Reiling: I think we actually said that it was an FDIC event and I think we all on the panel were joking, “everybody calls Dave, like Dave, what do you, what do you know?”

[00:16:00] David Reiling: Oh my gosh. I can tell you we have what we know as Stand Up Mondays where we’re going through all the different ideas and opportunities that were presented us from the previous week. And then we have breakup Fridays, which like, okay, this one came out and we’re like, oh, it’s not, you it’s us. You know, it’s that sort of thing. But there’s at least three a week,

[00:16:24] Melissa Koide: Is that right? I mean, cause, I know this is a conversation with me but it’s a conversation with you, but you are really an entrepreneur and in that you are out there trying to keep up with and understand what the latest mousetraps are and how do they work. And so how do you vet them? I mean, for your audience to hear.

[00:16:44] David Reiling: So, for us we have a, I’ll call it a unique process, but we have an opportunity filter that we go through. And for us being a mission-driven organization, that’s really at the forefront. So sometimes it can be easy. If someone comes to us and say, “Hey, we got this great predatory lending product”, it’s like, “okay, no, thank you.” And they call somebody else. And so, there’s just some things that we’re going to map out, in terms of things that are deal killers and things that are very interesting and curious to us. And we do a lot of small dollar lending and so that’s a space that we have expertise in. And credit building is another one. And so, we’re looking at those things that really fit us well and make for a good partnership with the FinTechs.

[00:17:28] Melissa Koide: Well, it totally makes sense.

[00:17:31] David Reiling: It’s art. It’s not science at this point.

[00:17:33] Melissa Koide: Yeah. I mean, I think that about the research projects, we call them experiments, not always technically experiments, but the empirical evaluations we do, and I’d love for there to be sort of a straight and obvious model. But it is, I mean the reality is it’s somewhat bespoken its experience and having a good team and trying to make sure you’re keeping your eye on what’s around the bend.

[00:18:00] Melissa Koide: You know, when COVID hit we wanted to think about where the technology and data potential benefits and risks. When we anticipate there are going to be a lot of families who are distressed borrowers with unsecured credit. And so, it’s getting a little ahead of ourselves.

[00:18:22] Melissa Koide: We haven’t publicly launched this project yet, but we’re going to be doing some work looking at debt repayment plans, and really trying to understand and empirically evaluate how do you think about constructing those plans that are most humane for the households that are in them. But then, importantly, are structured so that the creditors are being paid back. And where is that right sweet spot? And where can data, like cashflow data, really be instructive for understanding? And being frankly, even a little more nimble and dynamic potentially in how that plan works.

[00:19:00] David Reiling: Yeah, and I think that’s really important to understand is, if you’re starting to build that credit history and you can really understand that you have some flexibility, in terms of if they missed a payment or if they can’t make a full payment, but you can give them that extra little levity that helps through that deferment. That’s just without blowing up your credit risk or your loss models. I mean that is so good for the consumer. It’s good for the bank to get regulatorily, the regulators to buy into that. That’s just, it’s healthy for everybody.

[00:19:31] Melissa Koide: Yeah, yeah. I might try to drag you into that project too.

[00:19:36] David Reiling: We got lots of things to do. So let me ask you this though, as I started to think about our opportunity filter process in my team that’s doing that, in terms of the bankers of old, if you will. I mean, we’re learning an innovative process. We’re learning technology day in and day out. Maybe just how tech savvy does a banker have to be today to really be a banker?

[00:20:04] Melissa Koide: Well, I was going to ask you that question, but that is something that we are really probing and thinking about. In fact, we are releasing a paper that captures over 40 interviews and the advisory board that we’ve had with the input from banks, FinTechs, technology companies, technologists, and consumer advocates, what’s the state of machine learning adoption in the financial sector? In credit underwriting in particular? And I was saying this to you at the start before we began that we’d all like it to just be the pure math and the empirical understanding.

[00:20:51] Melissa Koide: It’s so much more than that. There have to be these humans in the middle and frankly, candidly, I think they are going to have to grow in their sophistication and understand the data science, the computer science and making sure that you’ve got the right people in the right positions at the right moment to evaluate your use of more complex math. And, and so we talk about this a fair bit in this paper and the work overall is going to be really honing in on that question of, who are the right people in the right places at the institution as you are deploying more complex technology?

[00:21:33] Melissa Koide: I also, I’m curious, I’d love to hear you react to this. I’ve been talking more to bigger banks than smaller institutions, but I think there’s also realization, at least in the large institutions, that “wait, we might have to readjust who’s in charge and what the governance looks like.” In addition to like what kind of technical capabilities we need to know. I’m curious if you’re picking up any of those kinds of considerations with institutions that you’re talking to too, but it feels like we are in somewhat of a paradigm shift in terms of who needs to be occupying which seats within the banking sector.

[00:22:12] David Reiling: You know, it’s interesting that you bring that up because, you know, the struggle of trying to understand and internalize what the math says and what the statistics say, right? And really understand that. But once you get to that point of understanding and having that knowledge, I think there’s for those on the call who are not so math based, you need a liberal arts, a philosophy, an ethical mindset, in which to take a lens to that data and it’s understanding and really try to understand what impact and effect does it have on different people in different situations? What does it mean? It’s just not a pure equation where it comes out and says, “42” is the answer to life. Right?

[00:23:03] David Reiling: It’s just not that easy. It has all sorts of implications. And again, life experiences, from young to middle-aged to old, will have different views in regard to what that data says to them. And those views have got to be heard. And so, the art and the science come together to, I think, make a sweet cocktail. But you need the right ingredients to make it all happen.

[00:23:24] Melissa Koide: That’s so right. And I know this is said over and over again, but it’s so worthwhile. It’s also the diversity of the people in the positions, right? And we’ll all have those “Aha!” moments. I wish I had a great example where it’s like, “Oh, wow. I didn’t think about that.” I just didn’t, that wasn’t in my experience, in my orbit. And so, making sure you’ve got all of those different, important perspectives, really is key.

[00:23:53] David Reiling: So, I have to tell you from the standpoint of the audience, if you’re looking for more information, FinRegLab.org has got some great resources. So go out there. There’s a bunch of articles. There’s podcasts. I think there’s a recommended reading room out there.

[00:24:07] David Reiling: And so, don’t overwhelm yourself with all the tech, just digest it and hit start. I think that is my biggest piece of advice for people listening is, this can get complicated, but look for things that can simplify it, because that’s just the way to start.

[00:24:25] David Reiling: Wow. Melissa our time is flying!

[00:24:28] Melissa Koide: I just wanted to join you in making that pitch to your audience and, truly I mean this, I would love to hear feedback from your audience. This piece that we’re putting out, it really is intended to take the data science and translate it into a text that is accessible for people who didn’t come out of school with a PhD in computer science. So, we hope it does that because I do, you’ve said it, I mean, it’s going to take lots of different people to sort of make sure we get the adoption of tech and data right. But at the same time, we’re all going to have to learn a little more about how these approaches and the math works. This is an effort to provide a resource that we hope is helpful.

[00:25:20] David Reiling: Yeah, definitely. Well, thank you for doing that. And I just have a question, just, it’s a little tangential to this, I really think when, to get your opinion on the generations of bankers. At times when I tell my kids of things and I’m like, “I see this wrong in the system” and they’re like, “just change it, dad.” And I’m like, “well, you know, there’s a system and a process” and they’re all like, “just change it. Why don’t you just change it?” And I’m like, “well, it just doesn’t happen that way.”

[00:25:47] David Reiling: And they’re like, “oh, you boomer, you never want to change anything.” I’m like, “first of all, I am not, I’m not a boomer.” Yeah, I got a little more experienced in terms of how this works, but I mean, their mindset is one of change and change fast. And again, all the information has been at their fingertips, their entire life.

[00:26:07] David Reiling: So, I can see where we need a little bit more youth and urgency around the change process. But along with that comes a bit of caution in terms of making sure you do it right. So, a little wisdom of the generations that have come before them, but I don’t know. Do you see a younger banker as pushing banking towards a positive direct?

[00:26:30] Melissa Koide: I definitely do. Without question. Many, many thoughts come to mind. I think it is, in fact we just had somebody who’s with CUNA actually stay at our house. He’s a young gentleman he’s working his way up at CUNA and I love to see he’s straddling the world of Venmo and PayPal but yet sitting in the position of engaged with the depository institutions.

[00:27:03] Melissa Koide: And so, we need more younger, motivated people to come in and think about how do we prudently evolve our financial system so that all the benefits and perks of technology and data come to rest, but we still have a safe and sound financial system. Like that’s a big deal. It’s been a while since we’ve had a crisis that captured our attention about that, but I can’t help but channel my old treasury days. Like, yeah, we’ve got to walk the walk to evolve, but do it prudently.

[00:27:38] David Reiling: Absolutely. And so, for those in the audience who don’t know CUNA, if you’re from more of a global audience. So, CUNA is the National Credit Union Association. They are the regulator for credit unions or cooperatives that are regulated institutions here in the US. So, most of the, you’ve answered this for the most part, but I’m going to give it to you anyways, just to see if there’s anything else, any other nugget.

[00:28:01] David Reiling: And so, what do you think the next gen banker looks like? What are those unique skills or qualities or abilities that are needed in order to make positive change?

[00:28:13] Melissa Koide: Right as you’re saying this, my dog is like popping up. The next gen banker. The one thing, I don’t want to end it on a downer note, but I do think being aware of just how important a safe financial system is.

[00:28:30] Melissa Koide: And so, I’ve got a 16-year-old who is ready to sort of see the world move to crypto. But I, and he will be a brilliant banker if he so chooses to be sure, but he also needs to really understand the financial system and the financial infrastructure. And so, finding those kids, young people, not kids, who are ultimately going to understand both worlds and be able to help us sort of forge the path is an important piece of it.

[00:29:09] Melissa Koide: Yeah, I’m afraid I’m getting a slightly distracted with the dog. Who’s like calling at me now, but this is really good.

[00:29:18] David Reiling: Great to chat with you. Thank you so much for being on the NextGen Banker podcast. It is always great to talk to you. And as we always say, gosh, I need to talk to you more often at what’s going on.

[00:29:29] David Reiling: So, in any event, thanks for your time. Appreciate you. Thanks.

[00:29:33] Melissa Koide: Thank you. It’s been a pleasure.