EPISODE 063 – API Security for Everyone

Catch the latest Agent of Influence episode with Buchi Reddy, Founder and CEO at Levo.ai, where he dives deep into the current state of API security.

NetSPI Field CISO, Nabil Hannan, sits down with Buchi to explore API security from the developer’s lens, including the importance of proactive security measures, and the challenges of API discovery and inventory. Hear about Buchi’s motivations for starting Levo.ai and gain insights into why today’s applications urgently need robust API security. 

Show Notes: 

• 01:04 – State of API Security Today 
• 02:35 – Why Focus on API Security  
• 05:34 – Special Considerations Based on Type of API 
• 08:02 – Motivation for Starting Levo.ai 
• 10:56 – API-Security Related Breaches 
• 12:30 – Building Security into APIs Earlier 
• 14:44 – Clear Communications between Security and Development 
• 24:17 – Security vs Efficiency when Building APIs 
• 27:11 – The Challenges with Discovery and Inventory of APIs 

Transcript between Nabil and Buchi 

Best practices for API security, building API security into the development life cycle, and overcoming challenges with API discovery and inventory 

This transcript has been edited for clarity and readability. 

Nabil Hannan: Hi everyone, I’m Nabil Hannan, Field CISO at NetSPI, and this is Agent of Influence. Today, Buchi Reddy joins us to talk about API security, and in particular, proactive API security practices. Buchi, really excited to have you with us today. 

Buchi Reddy: Thank you, Nabil, super excited to be here. Thanks for the opportunity. 

Nabil: Let’s start by telling us a little bit about where you are today, professionally. 

Buchi: I’ve been in the Valley for last 10+ years. My background is engineering. I started as an engineer and built several products, especially in the last dozen years, I’ve been in the space of building application performance monitoring systems, distributed racing systems, and last five places in API security. I started Levo.ai a co-founder and CEO about three years ago.  

We are at a phase where we got some initial great logos, and we are at the scaling phase. So, I’m an engineer who turned into a Founder and CEO and going through the learning journey of building the first company and trying to sell. 

01:04: Where are we truly in the API security journey?  

Nabil: You’re braver than me, but that’s why you’re probably going to do better than me in life. So  that’s great.  

Let’s start by talking about where we are from the current state of API security today. It’s a concept and term we hear about all the time. Everyone seems to be trying to figure out what to do. So where are we truly in that API security journey? 

Buchi: Great question, and I can definitely talk a lot about it, because it’s been three years since Levo began, and when we started, we were very early, especially with the proactive security. I know you guys talk a lot about proactive API security. We were very early. I think the way I see some of the early adopters who have been adopting API security, now is reaching that critical mass.

So, we’ve crossed the education barrier and proactively, especially, how do you even do proactive API security? I mean, we can get into some of those details. But customers used to have all of those questions. Today we are past all of that. Most of the customers are very well aware of, what does API security even mean? What are all the use cases? Like, if I’m evaluating the product, what are the criteria on which I need to evaluate a basic product? All of that they are very well aware of. I think we are reaching that critical mass, which is one of the reasons why I’m very excited about for the next phase of Levo and my journey. 

02:35: What are the main reasons to focus on API security? And why do people still have API security on the back burner?  

Nabil: I have a basic understanding of why APIs are critical, especially given how software development processes and also software architecture has evolved over the years.  

From your perspective, what are the main reasons why focusing on APIs and using APIs have become so integral today? And then from there, we would love to dive into understanding, given that this is such a critical part of software development and software architecture, why do people still have security for APIs on the back burner?  

Buchi: I love talking about data day-in, day-out, because it’s such an important topic. First, why is API security important?

Usually, you know, when you’re interacting with a mobile application, you’re looking at the UI. You’re not seeing the actual API calls that are carrying your personally identifiable information, all the sensitive data, your bank, phone number, account balance, all of that is going through APIs, but you don’t see that in the UI. You only see the UI element. That’s generally one of the reasons why people don’t immediately connect to that and see that as an important thing, because it’s on the back end.  

But it’s important because all the data that we are seeing and interacting with is going through that nervous system, through APIs, and all the architectural changes that have come. Whether it’s microservices, Kubernetes, or continuous deployment, continuous integration, whatever helps enterprises to innovate and move much faster, it has only increased the number of APIs, especially for teams to be independent and move much faster.  

APIs have become an integral part of all that architectural movement, and that just made the APIs explode. API probably is the word that everybody is using. But still, because it is not what you are seeing and interacting with directly, they tend to take a backseat. And it’s only the recent data breaches and realization that, oh my god, all the data could be exfiltrated through a vulnerable API. Only when that that started hitting people, then they started realizing, yeah, they are back-end technology. We may not see them, but they are so powerful and they are such a big attack surface that they could completely cause your data loss like and data is the crown jewel for every enterprise, and you don’t want to lose that. So that is the realization, or aha moment for people to start realizing API security is really important, even though I may not be touching and feeling it in the user interface. 

05:34: Are there special or unique considerations to think about based on the type of API you may be using or implementing? 

Nabil: I understand that there are common application security concepts controls that have to go into APIs themselves, right? Authentication, authorization, input, validation, output, and we know all, there’s a whole bunch of those that we have to do that are just basics of application security and software development today.  

Now, there are different types of APIs out there. Are there any special or unique considerations that you have to think about based on the type of API that you may be using or implementing, and what are they? 

Buchi: Absolutely. First, for the audience, what are the types of APIs there might be? There is generally REST APIs, even REST APIs, there are several ways people implement them – there is no one way to do it.  

Then there is, recently the spec-based APIs, generally whether it is GraphQL or gRPC or some other kind of RPC. There is traditional SOAP.  

And how they take the data and how the validations are done. Change from API to API. So, for example, like gRPC or any RPC physique, or any spec-driven APIs physique, the input validation probably is little less important. Or maybe some of the input validation is already happening, at least on the structure of the data that is coming in these APIs, compared to REST, for example, where you could be taking the entire blob of JSON and trying to feed it into the back-end server.  

So yes, there are definitely differences. And even authentication. For example, gRPC provides even MTLS or much stronger authentication. It has provisions whether you use it or not. Again, it depends. But versus some of the other HTTPS they some of them purely depend on just session tokens.  

So it’s a very wide topic. The spectrum is very broad, the API to API, because just depending on how it actually works, there are a lot of differences: authentication, input validation, or in general, like the rate limiting, and even those APIs, even if it is REST API, whether you implemented REST API with an API gateway versus without it, again, what you have to do changes a lot.  

There’s no one line of answers. Unfortunately, it changes from API type. But it is also not completely tied only to the API type. There are several other factors, like how you deployed it, what technologies you are using, all of that also impacts it. 

08:02: Are there unique challenges in the API security space that motivated you to start Levo.ai?  

Nabil: Is there something that’s unique about the challenges in the API security space that motivated you to start your company? And why did you choose that over so many other things that also need attention beyond just API security? 

Buchi: Yes, absolutely. I was a founding engineer in couple of companies prior to this; I was definitely already interested in, and excited about working in the companies. But what motivated me? A safer digital world for future generations, and I would add myself.  

If you think about it, you want your kids and the next generation to be in a safer world, whether it is digitally or physically. To my earlier point, APIs are the nervous system through which your entire data is getting transmitted, and digitization wave is only picking up, and every day we are only moving more and more things to digitization. The thought itself scares me sometimes. There could be insecurities or vulnerabilities in these APIs, which are dealing with the entire data and everything could be at risk.  

I landed on this problem firsthand, in my experience. I was at AppDynamics. Cisco acquired AppDynamics, and I was leading a couple of engineering teams, and Cisco team came and said, ‘Hey, you’re now Cisco, your software better be secure, because you know, you’re not a small company anymore.’ You are a big company, and our brand and reputation will be at stake.  

I volunteered to be the first team to go under the Cisco security scrutiny. I was learning several things, but I quickly started understanding that the security domain is lagging behind by five years, and some of the things seem very basic, why are they not done yet? And API security was one of them.  

I still remember the security team coming and asking, do you use authentication for these and how strong is authentication? I learned as I saw that as a first time.  

And then when I started working in the API security domain, I saw that the problem was not solved proactively. It is reactive. Most of the solutions were just promising that, okay, I’m gonna sit in production and I’ll try to block the attacks. When you can actually do a lot of things before you even put in put the application in production, why wait until last minute to give that a try? And the worst thing is, that won’t address the root cause of some of these reactive solutions.  

So that’s what prompted me, basically, hey, there’s such a big gap. When I dig deeper, there were some technical challenges, so I understood that there is a reason why it was not solved already. And when I had the confidence that me and my team can solve some of these technical challenges, because of which this wasn’t made a reality, I jumped on it. I know the gap. I know we can solve it. What else do you need? 

10:56: Do you have a favorite API-security related breach? 

Nabil: Do you have a favorite API-security related breach, and why is it your favorite?  

Buchi: Well, there’s quite a lot. I mean, AT&T is frequently in the news. I generally don’t tend to say any company names, because generally, ambulance chasing is a word people say, right?  

I don’t want to be, but most of the consumer-facing companies, because the number that immediately catches attention is like, 300 million people’s information is out there. Or if it is any Telecom, you say has 100 million people’s phone numbers and PI and addresses out there.  

So typically those catch my attention, but generally I am a big believer, I mean even this, if you take the latest CrowdStrike instance, you can say that they messed up and all these things. But you have to be practical. CrowdStrike is a respected company. They try to do their best in spite of what happened, and so, I’m in that bucket where, when some data breach happens, obviously, you have to learn and you have to do the best, but I don’t try to make that a show. So I don’t have one favorite. Every time I see it and it bothers me, then it’s like, okay, one more reiteration for the mission that we are on.  

So every time I see it’s reiteration that okay, we are on the right mission, and we have to solve it. But I don’t have one particular favorite to answer to your question. 

12:30: What advice would you give an organization on how to start building security into APIs earlier?  

Nabil: What advice would you give engineering organizations or a development organization on how to start building security into APIs earlier? And actually build security in, versus trying to bolt security on at a later time?  

Buchi: I actually have advice to the security counterparts on how they communicate the importance to developers. A lot of times, people say developers don’t care about security. I’ve heard that several times. I’m like, no, that’s not true. I was a developer myself, even before starting Levo, even when I was an engineering leader, if you come to me and say, boss, if there is weaker authentication or authorization here, and our customers’ data could be in the wrong hands, there is no freaking way I would take it lightly.  

I don’t think any reasonable developer or engineer would take that lightly as well. So it’s oftentimes, it’s not the technology challenge. Of course, we can, I can give the best practices, like the pointers, 1,2,3, that you can do, but oftentimes what I have seen is the communication and conveying the importance of it and the ramifications of what happens if you don’t take care of it, I would say that’s what there needs to be, a healthy dialogue between these counterparts to communicate that.  

But when it comes to just the tactical advice, developers do this, like even testing. We automate testing, right? The basic functional testing is automated. To a large extent, they’re not waiting; they figure out ways to do it.  

Of course, I can slide in Levo and say, yeah, put Levo in the CICD, and we’ll take care of it. That’s one way. But there are other ways, usually design time itself, do threat modeling, think about all the possible ways it can go wrong, and then also automate everything.  

That’s the only way we can scale, especially going back to the APIs problem, an average enterprise has tens of thousands of API endpoints. If you combine internal and external, there’s no freaking way you can do it all manually, so you’ve got to automate. These are all tactical advice. But in my opinion, any smart enough engineering team is already aware of a lot of these things. What they don’t know is why this is super important, and why you have to take enough time to do it. 

14:44: Is the challenge that the security team is not communicating the importance to the development teams? Or is the challenge that they’re not speaking with shared terminology?  

Nabil: You bring up a really good point that I want to explore a little bit deeper. Go a little bit under the surface here, you started by saying that the security team needs to articulate the importance of this.  

Development teams, and I’m with you on this one, I think development teams actually want to do the right thing; it’s not that they don’t care about security. They may be unaware or they’re not thinking from a security mindset, because typically, they’re hyper focused on some feature set, or some requirement that they’re building out.  

So I get that piece and I understand that piece where I want to explore a little deeper and pick your brain on: Is the challenge that the security team is not communicating the importance to the development teams? Or is it that the two teams are speaking, for lack of a better term, they’re speaking different languages, and we haven’t figured out how to bridge and become an effective translator between those languages. What do you think about that concept? 

Buchi: Yeah, great question. I keep thinking about it a lot, because even when I’m trying to sell to companies, like I come across companies where security, sometimes they think that this is important, but some of the teams, I don’t see them moving and closing the deal with us. So I keep thinking about this topic a lot.  

I think there is a prioritization, and the number of things that both the parties have to do is also playing a role, and that’s where, again, it goes back to your earlier question of, why say API security, or in general, any security, could be taking a back burner?  

The language comes into the picture only when this is called out as a priority, and then they started actually communicating to a large extent. What I have noticed is, there is a mindset where things related to compliance are non-negotiable, and compliance, governance and some of these things, those automatically get prioritized and added to the list of things. When it comes to really keeping things secure, or when you know nobody might be forcing you to do it, but you had to do because it is the right thing to do, there is a prioritization problem. So I didn’t see the problem in actual communication, but when it comes to communication, also, I’ve seen sometimes they assume that developers don’t care on this side or the other side, also assuming, like one. I can tell from my own experience, the engineering side, what I’ve security team said, the problem that I’ve seen is they sometimes assume that developers don’t care about it, which I already said, please don’t.  

And the developer said, what I’ve seen is some of the compliance and governance side of things, when they come, developers don’t see a reason to do it. Sometimes, let’s say I’ve seen cases where you go and introduce a SAST tool. There are a lot of false positives, and some of these things, and developers are not even convinced that fixing every one of those is even needed, like, some of this code is not even reachable. Why are you asking me to do it? 

When such things happen, slowly, they get into your mindset where, like, oh, come on, man, these guys, because they have the mandate to push us, they’re gonna push. So they start taking even the real things lightly. You do that at some point they know that, okay, now I know this is coming as a mandate, and just because it is the security team, I have to do it. But like, is there a real reason this is happening? So it’s more of you know that feeling based on what is built from some other narratives, slowly they develop that mindset.  

That’s why the prioritization, and you know, both teams agreeing why it is really important is super important, but especially the SAST trend does a lot of damage when it comes to the development team taking something seriously, is from my experience, because there’s, like, way too many of them blindly saying, like, you got to fix every one of these. You’re gonna lose their trust as well.  

Like, okay, do I? Should I really take you seriously? Because you’re telling me to you’re asking me to fix everything, even though I know that some of this code is not even reachable. 

Nabil:  So when it comes to SAST and DAST, it’s almost become a running joke now, because everybody understands that they’re extremely noisy when it comes to false positives, but because of regulatory and compliance requirements, you have to run those tools regularly.  

And to your point, that’s where a lot of the frustration comes from the dev teams, because they understand that it’s a requirement that they have to fulfill, but they don’t understand the rationale behind why, especially if it’s not providing them any value.  

So is this a problem, then, in our regulatory frameworks and requirements that’s causing this friction? Or is this a problem of the security team not correctly interpreting the spirit of the requirement and not helping fine-tune the static analysis tools and dynamic analysis tools to focus on the key things with high confidence and take away all the noise so that the dev teams actually find value in what’s happening? 

Which problem is it? And do we need to start thinking differently about these problems? 

Buchi: I love this question, and I think it is the first one that is happening more, especially in 2024. What I mean by that is some of the compliances and regulations are a bit outdated. Again, I’m not trying to throw somebody under the bus or finger point only at one particular persona here, but I’m just talking from my experience. I’ve seen cases where auditors don’t even understand how some of these applications are built, how the cloud is provisioned, and they’re just following some rules that were crafted a while ago. 

Nabil: They’re going through the checklist to see if something was done.  

Buchi: Yeah. When it comes to the security team, they sometimes cannot say no to a few things because they have to be followed. So that gets transferred to the development team also as is. So, I think it’s more of a first one that you pointed out, which is the governance. 

I think that will also change. As an industry, one good thing is we are always taking feedback, right? That’s how the new startups, new products keep coming, and we keep innovating. We keep changing things. So it may change, but at least as of today, I definitely see some outdated governance and compliance rules being followed, where some of the check boxes are checked just because we had to.  

The second one you pointed out, which is the security team not being aware of, I don’t see that as a big problem. Maybe it was a problem a few years ago, but especially with some of the new innovations, like, if you see the same SAST trend this year, new tools we are talking about, like reachability, exploitability, and all these things, and we are trying to get ahead of them. Again, there might be old tools which don’t have that capability, but there is innovation. 

And you know, one or two years from now, every SAST trend I see, the customers will be holding these tools accountable to say, don’t throw all the noise on me, like, just tell me which ones are really reachable and exploitable, so that I only focus on it, so that I see it’s heading in the right direction. Even, like, you know, some of the tech that we built is actually useful for that exploitability.  

For example, like one of the things that we promised to our customers, like every vulnerability that we report is already exploited, so you are getting the exploit also, along with the vulnerability that we report. So, I see similar sentiment with some of the new age SCA tools and all. 

Nabil: There’s evidence that this is exploitable, and this is why we’re not just guessing, or we’re not looking at a pattern and just throwing up a flag. We’re actually testing. 

Buchi: I don’t see a problem with the security teams not doing the job at all, like they are definitely doing a pretty good job. In some cases, when they don’t have the right tooling given to them, or not enough time, probably they were trying to just follow the compliance.  

So this is, in fact, API security, like in my own journey, many times I keep telling my team like that, maybe at some point when we grow enough and we have enough muscle power, I want to go back and change the thinking of these auditors and some of these people and tell them the importance of API security. The change has to start there.  

Some of them don’t even understand that there are APIs behind apps. I’ve spoken to a few people who say that I don’t have APIs. Like, what do you mean you don’t have APIs? Like, you have web applications. Do you have mobile applications? Yeah, a lot of them. What are they doing? How are they even working?  

They think that just because they don’t have a clearly documented API that is given for somebody else to build on top, they don’t have APIs. This is completely wrong. Technically, every one of your client-facing web applications and mobile applications is calling APIs. That, in itself, is a big problem. And if you go to auditors, they’re like even behind in some of these things. So great point, and I personally want to invest a lot in that area in the future. 

24:17: What advice would you give on how to balance security versus efficiency, especially when it comes to building out APIs? 

Nabil: When I think of security and things like efficiency or usability, etc., they’re often, if not always, opposing forces. They are actually fighting against each other. So, what advice would you have on how to balance security versus efficiency, especially when it comes to building out APIs? 

Buchi: That’s where some of the tooling matters. We should not be too rigid. On one hand, if you want practices to be followed by everybody, it’s easier if you say, this is how it works, zero or one. But on the other hand, that’s what introduces rigidity and slows down things.  

I think a better middle ground is to come up with tooling that is smart enough about some of these things. Let’s take a simple example. If you have a tool that’s going to break a build when there are vulnerabilities, if the tool is not smart enough to detect if it is a newly introduced vulnerability – the severity, the exploitability – then you can actually implement it in a much more practical way, where you break the build only when it is enough severity and all these things.  

What I have noticed is sometimes the tools lack some of that capability, and teams who are trying to adopt it, they obviously at the end of the day, when we are in a very large enterprise, we need some practices. We cannot just leave it open ended or gray area, right? So they are trying to do the right thing. But the tool, if it is not sophisticated enough to give that flexibility, what ends up happening is, I break the build, now you are breaking the build unnecessarily more times than you should be, and that’s where the frustration is.  

Or if you want to go to the other side, where you are not breaking when you are supposed to, that’s where the frustration on the other side of the team is. So, to a large extent, tooling is the suggestion that I would give. Evaluate your tools correctly, make the requirements to vendors correctly, and if you get the good tooling.  

Otherwise, I think the culture of every company is a little snowflake in itself. There is no one size. You have to do what works. But most of the times, tools that are giving enough information, but passive, are not going to work in only one way.  

Applying this to API security, for example, when we are running security tests, we give the flexibility to the customer. We are not dictating one approach where I’m always going to block or not. They can define all these things on their own, depending on where you land in the spectrum as a company, depending on the maturity, you can choose which fits best for you. 

27:11: Why is API discovery and inventory so challenging? And why has it taken so long to solve this?  

Nabil: A common problem a lot of organizations I talk to today around APIs, is less about security, but more about discovery and inventory. Why is that so challenging? And are there certain steps we’ve taken that’s making that easier today, and and more importantly, why has it taken so long to build this? 

Buchi: Oh my God, this is exactly what I was telling myself when I started, when I was about to start this company. I have been building APIs for almost 10 plus years by the time I started this company, and I was like, this can’t be true. Don’t tell me there is no API security solution now, because I, myself, have been building APIs for 10 plus years.  

But indeed, that was the case. Why? Because it touches on some of the things that I said as you mentioned about API types.  

One, there is diversity with respect to what kind of frameworks, what kind of tools are used by development teams to build APIs, how they are implemented and all, and more importantly, the documentation. When we moved away from the waterfall model to the agile model, the emphasis was on delivering and innovating faster, and we were not documenting, like you used to do in the waterfall model.  

Mostly, in the API world, you say there is a spec-first or design-first process, and there is code-first process. In the code-first process, you are just implementing the API. You are not even writing API documentation. 

Nabil: I have a mentor who used to joke about that, and he used to say that if you don’t have documentation, then you can’t be wrong. So we started writing code before we started documenting it, and that’s getting us in trouble today.  

Buchi: Basically, that is the crux of the problem. That’s what happens a lot, and as developers, not just developers, but in general, people are not super interested in doing something that may not be consumed, whether it is API documentation or anything.  

So to be very frank, now, because API security has become important, we are saying API documentation is needed, but there were not many other use cases for API documentation, to a large extent.  

Also the way the teams happen is there’s only a certain set of people who are working on these APIs, especially on the development side. Oftentimes they just fire up the ID and see the code so they don’t need to ask. Because of all these reasons, there is no documentation.  

I’ll quote one of our customers who said, APIs grow like mushrooms. In my company, it is exactly how they grow, like mushrooms – that crazily. So now you are dealing with a problem where nothing is documented and they’re growing like mushrooms. That’s what causes the API problem or the API discovery problem. Nobody knows the full spectrum of how many APIs we have.  

And I know the answer to this question from every customer, but I still ask this. We open the call with, do you know how many APIs you have? Not, because I don’t know the answer. I just want them to admit that they don’t know it 100%. Nobody knows how many they have, and then what kind of data they are dealing with, and all these things.  

Discovery is a huge part of API security problem in general. And combining this with how you write your APIs, how you run your applications, where do you deploy them, how do you deploy them, this is different from company to company. That’s what makes it a very challenging and interesting problem, that nobody has API documentation, and that has given opportunity, or created a space where somebody had to come up with a solution, and we came up with such a solution, and several API security products came up with solutions to solve this problem. 

Nabil: I think there’s one other thing that you didn’t mention, but it’s equally important. We talked about how many APIs you have, what type of data is it touching? But also, is the API externally accessible or not? I think the internal versus external conversation is another big challenge that many people have. Understandably, if you don’t have an inventory, it’s even harder to have an inventory that is categorized in that sort of fine granularity as well. 

Buchi: Totally, there are several other things. Not just the external entity, like what kind of authentication they use, how strong that authentication becomes, authorization? Who is accessing this data, what permissions do they have? There are several other things, I think these are all core parts of what we offer, in general in our solution, or in some of the other API security solutions.  

Nabil: Thank you so much. This was a very fun conversation, and I’m so excited to learn all this from you, and hopefully we will get to do this again soon. 

Buchi: Thank you for the opportunity. Hopefully the audience finds it useful, and feel free to ping me if you have any questions. I love these kinds of conversations, and I’m happy to chat with anybody about this.

Find more episodes on YouTube or wherever you listen to podcasts, as well as at netspi.com/agentofinfluence. If you want to be a guest or want to recommend someone, please reach out to us at podcast@netspi.com.