How FedEx Delivers Zero Trust Security at Scale

Ryan Rudnitsky:
Good morning, afternoon, evening, or wherever you may be. My name is Ryan and I'm a senior customer success manager with Okta. As a success manager, my role is to be the liaison between Okta and our customers to ensure they're getting the most out of Okta. This includes understanding key use cases, updating teams on new functionality and features, as well as supporting large implementations. Today's discussion is going to be a truly amazing story that focuses on FedEx's journey towards Zero Trust. As you will hear, it's a carefully coordinated effort between Okta and FedEx to modernize identity and access management for the next wave of FedEx employees.

Ryan Rudnitsky:
You will hear firsthand from the true innovators behind the success, as well as how this conversation of Zero Trust came to fruition. This large-scale deployment involved many stakeholders at FedEx, as well as the folks from training, support, customer success, and professional services at Okta, the four pillars of the customer-first organization. As you know, Okta's a public company, so any forward-looking statements we share are subject to change. Please read the safe harbor statement, and in depth, at your leisure. Additionally, please feel free to submit all questions in the chat window throughout the presentation and we'll be happy to answer them. Without further delay, I would like to introduce the wizards behind FedEx, Trey Ray.

Trey Ray:
Thank you, Ryan. I've got a couple of colleagues with me I'd like to introduce. First of all, Pat O'Neil. He's a cybersecurity fellow in our identity and access management organization. And I've also got Prashanth Karne with me, who's a cybersecurity principal, my lead engineer and my right hand. So I think most of you have probably heard of FedEx. We're the company that's become a verb. So if you want to ship something and make sure it gets there on time, you're probably going to FedEx it. FedEx has been around since 1973. We're based in Memphis, Tennessee and we've got half a million team members. We connect 99% of the global GDP and every month 65 million visitors visit FedEx.com and track 250 million packages a month.

Trey Ray:
Overall, we ship about 15 million packages a day. In terms of our workforce identity space, we've got about 350 software service applications. We also have a legacy WAM that protects about 1,500 URLs and that makes up somewhere around five to 600 additional applications. We've also got, and this slide's a little bit dated, shows 250 cloud-native applications. I suspect that number is probably reaching close to 400. So this is really our workforce identity, you know, space by the numbers.

Trey Ray:
So I've attended a lot of these conferences, mostly in person, this is the first virtual one. But you always hear a lot of talk about digital transformation. Well, the thing about it, FedEx has always been a digital company. So we don't really need a digital transformation per se. We've always been digital. If you go back to 1979, we launched COSMOS, and that was basically a digital system that managed people's packages, vehicles, and weather. In 1980, we installed these DADS terminals, which you see here on the photo, which is basically a small display in a keyboard in every courier van. And it connected to a private wireless network that we had. And this allowed our dispatchers to actually contact couriers for pickup and delivery without actually having to call them on the telephone.

Trey Ray:
In 1984, we launched PowerShip and this was a PC-based shipping system where if you were a small business you could actually generate shipping labels with a Windows PC in your office. In 1986, we introduced the SuperTracker and it was basically a wand-type device that scanned barcodes for pickup and delivery and captured detailed information about the package. In 1994, we launched FedEx.com and it became the first transportation website to offer online package status and tracking. And then, moving ahead several years, in 2019 we introduced Roxo the SameDay Bot. You may have seen Roxo on the Tonight Show and we're testing it right now, and it's an autonomous delivery service to do same-day deliveries.

Trey Ray:
And our founder, Fred Smith, who you see in the photo in the center, well, as I've said, we've always been a digital company. He had a quote several years ago that said, “The information about the package is as important as the package itself.” So although we didn't have to have a digital transformation, since we've been digital since the '70s, we did need a digital modernization. We had a lot of legacy systems, we have a lot of mainframe applications and that type of thing. So our CIO, Rob Carter, several years ago, set out on this task for IT renewal. And, and it's been a decade-long journey. And it's really about these nine renewal tenets, you know, being service oriented, being API first, working with a new cloud-native architecture, implementing DevSecOps, continuous delivery, trying to retire a lot of that legacy that I talked about with mainframe applications, and that type of thing.

Trey Ray:
So as I mentioned, we didn't necessarily need a digital transformation, but we certainly needed a digital modernization, and renewal was all about that. So one of the things that was spawned out of the IT renewal journey a couple of years ago was this team called the Cloud DOJO. It's basically a cross-organizational team of experts that are teaching other internal development teams modern development techniques. Teaching them how to take their applications and make them cloud-native so that they can deploy them, you know, private cloud, public cloud, and using new development tools like Spring Boot, Spring Security, Angular, also taking a new framework that we made an investment in from Pivotal called Cloud Foundry. One of the other things that DOJO does is peer development sessions. So if you're new to this area and maybe you're not familiar with some of these cloud-native techniques, we’ll pair you up with an expert and do some peer development, so you can learn.

Trey Ray:
As you can see, in 2019 the Cloud DOJO was a winner of the CIO100 Award. So we're pretty proud about that. So the thing about it, as our developers were adopting these new cloud-native techniques and getting their hands on these things like Spring and Angular, they were really expecting to jump in from a security perspective and implement things like OAuth 2 and OpenID Connect for the security side of things. Unfortunately, they were presented with this. And this is really our workforce identity and access management space that we've had, over the last 20 years ago and it's really just a matter of us, you know, standing up point solutions to handle a particular need at the time. As you can see on the diagram, the spaghetti diagram, as I call it, we've got a VPN solution.

Trey Ray:
We've got an on-prem multi-factor authentication solution. We've got an on-prem federation solution, an on-prem web access management solution. And although we've made these things work together with baling wire and duct tape, it presents a lot of friction for our software developers who are trying to do things in a modern fashion and having to marry up with this legacy world. You'll notice on the right-hand side, there's a couple of clouds that represent our cloud-native space and this is the area that those guys going through the Cloud DOJO are working in. So Pat, if you would, can you kind of elaborate on how this legacy IAM has been a roadblock to the developers that are trying to do modern development techniques in cloud-native apps.

Pat O’Neil:
That's right. One of the things that we do in the Cloud DOJO as a part of our control shift and the journey that these development teams are on, is we want to achieve the sort of DevSecOps experience. We want them to be able to concentrate on deploying their application and not have to worry about where their application is deployed. Now, today, as you can see, or at least in the past, the near past, the picture that you see before you, you don't have that luxury. If you deploy on prem, even if you're using cloud-native technologies like Cloud Foundry, if you're deploying on prem, you have to worry about which identity source you're connecting up to and how you configure that. And as you can see from the diagram, there are several point solutions that we've configured both in public cloud and in on-prem clouds and in CoLo and for SaaS that are all distinct.

Pat O’Neil:
Now what we want to achieve to improve that of course, is what we've done with Okta and we'll talk a little bit more about that in the next few slides. But we want that experience to be one. We want them to be able to say, "Okay, it doesn't matter where I'm running or which cloud I'm running in." So that we can flex their deployments into public clouds or into hybrid and CoLo clouds, you know, and they don't have to worry about it. It's transparent to them. To do that, we've got to be able to allow them to consume from a centralized identity provider and that's exactly what we have with Okta. Back to you, Trey.

Trey Ray:
Thank you, Pat. So not only is this legacy identity and access management workforce infrastructure high friction for our developers that are trying to deploy cloud-native applications, it's also high friction for our end users. You notice there are, well, five or six small numbers next to some of these flows, and what that represents is how many times you'll have to sign in to get to one of these applications that are using these various identity solutions. Fortunately, we've got a good identity-governance solution from our partners at SellPoint, so we synchronize our password. But if you're a FedEx sales guy, you may have to enter that password five times to get productive in the morning. If he's working remotely, he may have to log in once with that credential through VPN.

Trey Ray:
If he hits a header-based app that's using our legacy WAM, he may have to enter it a second time. If he needs to access something that's using Cerberus or AD he may have to log in a third time, and then a fourth time for something using direct LDAP auth. You get the picture. So really high friction in terms of our software developers, as well as our end users. So, what did we do about it? So a couple of years ago, we set out on a journey as part of the IT renewal process that I talked about earlier on scoping out an identity-as-a-service solution. And we read a lot of white papers. We watched a lot of YouTube videos. We talked to a lot of pundits about identity as a service. And we ended up narrowing the field down to three vendors and we actually put out an RFI to get a little bit more information. And then we narrowed that down to just a couple and brought those in for an onsite bake off.

Trey Ray:
And if you've ever heard about FedEx bake offs for technology selection and security, they're notorious for being very brutal. And you can ask a couple of the Okta SEs about our bake offs. After the gloves came off and the referee called it, Okta came out on top. And these are some of the reasons, really, why we went with Okta. And one of the big ones is vendor interoperability. Okta is one of those companies that integrates with everything. We're a big Workspace ONE shop, you know, we use their watch as our MDM. So there was really tight integration between Okta and Workspace ONE. Ease of use was really key. The ability to use a single admin console to do your work as a security professional instead of having to bounce around in four or five different websites was really important to us.

Trey Ray:
We mentioned that API was one of our first IT renewal tenets. Well, pretty much anything you can do with the admin console with Okta you can also do with the APIs. Okta also had a wide range with multi-factor authentication methods. We're really focusing on using Okta Verify, although we do use some of the older auth hard tokens for some use cases, and I'll get into that a little more later. And then another thing that was really key and that's part of the Universal Directory feature, was this ability to aggregate identities from multiple user stores. So we're a big company. We buy a lot of companies, so that means we've got a lot of directories.

Trey Ray:
Being able to take all those directories in and pull them into Universal Directory under that single pane of glass. And then, you know, one of the things that makes Pat happy, and the rest of the developers in the DOJO, is there's turnkey compatibility between Okta and some of the tools we're using, like Spring Boot, Spring Security, and Pivotal Cloud Foundry. And we think, well, we don't think, we know because we're making good progress that in the spirit of IT renewal and the decommissioned legacy tenant, we're going to be able to decommission a lot of this legacy. Now, I always put an asterisk next to VPN. I don't know if we'll ever completely get rid of VPN, but I do think we can eliminate it for a lot of our business workforce who don't necessarily need a full tunnel.

Trey Ray:
But the things on the bottom,some of that legacy identity stack, we're certain that we're going to get rid of a lot of that. And really, this is the endgame. And what you'll see here is that we've taken a lot of that legacy and moved it into the Okta Identity Cloud. We still have the SSO capability. We still have MFA capability, but now we're going to be able to do things like verify the user, verify the device before we allow sign in. You can see moving from top to bottom, those 350 SaaS apps I mentioned will be integrated with Okta using SAML 2.0 in most cases. Pat, if you don't mind, kind of give me a rundown on how implementing Okta is going to help in the cloud-native space with the developers that are working in the Cloud DOJO.

Pat O’Neil:
Sure, Trey. As you can see from this diagram how it changed from the last one, there's one cloud-native cloud, if you will, with the cloud-native apps in it. And then we mentioned that in that cloud there's public, private, and CoLo. And as I mentioned earlier, the nirvana for us is being able to flex our applications into consumption and hybrid situations like CoLo or even public clouds to be able to handle volume surges, which is a big problem in our business. Well, as you can see, before, we had a sort of a Whack-A-Mole game going with different one-off solutions for several different identity providers that may all have been using some modern auth technologies like OpenID Connect or OAuth 2, but each one of those separate ones that is no longer in this diagram was a separate opportunity to get the configuration wrong.

Pat O’Neil:
So it really was a Whack-A-Mole game from the security perspective, because then we had to run around to all these different places and make sure that that configuration was securable across all of them. Now, as you can see with this model, we've got one place that we can validate our security posture. The dev teams themselves are much happier, because they've got one kind of token to worry about and they do authentication and authorization in a consistent way no matter where they're deployed. Back to you, Trey.

Trey Ray:
Thank you, Pat. As I mentioned in one of the earlier slides, we have a fair amount of legacy apps that are still using header-based authentication that are behind legacy WAM, so I'm going to ask Prashanth to talk about how we're going to deal with those applications that may not be able to modernize necessarily. So Prashanth, can you help us out?

Prashanth Karne:
We use technologies from F5, like Application Policy Manager, APM, and Application Security Manager, ASM. To perform the protocol transformation and still authenticate with Okta using our modern tech, modern protocol such as SAML 2.0, but still transform the protocol back to the application if it requires headers or any special cookies. So the F5 APM can control all those transformations, still authenticating the user with Okta, using modem authentication protocols but still send the user back to the legacy application with all the headers or cookies, whatever the application requires. Back to you, Trey.

Trey Ray:
Thank you, Prashanth. One last thing I want to mention about this endgame diagram is, you know, I told you earlier, we have multiple disparate user repositories as we've acquired companies over time, and you can see we're able to aggregate those into the Universal Directory under that single pane of glass. Another thing to note, this is a reduced-friction environment for the FedEx workforce. So that sales guy I mentioned earlier, instead of having to sign in four or five times with the same credential to start his day, he's now going to see a single Okta sign in to access all of his applications, whether they're SaaS, cloud-native, or legacy.

Trey Ray:
We like to brand things at FedEx. So we worked with our employee communications team and our brand marketing folks, and we've branded the FedEx Okta experience as PurpleID. And we've done this fantastic website, program website, which has great information, FAQs, and that type of thing. We've done promotional videos that show the end users how to enroll in Okta Verify, which has been very popular. So PurpleID is our tagline. And so far, it's been adopted very well by our workforce users.

Trey Ray:
So you might ask yourself, that sure is a whole lot of trouble just to make your software developers happy and make your end users have less sign ons. Well, you're right. So, we are a security organization and we did have a bit of an ulterior motive with all this, in that we had in our back pocket, this entire time, that we wanted to roll out a Zero Trust security model. It's obvious that passwords have failed us over time. You know, there's a couple of news articles here that talk about breaches and account takeovers and that type of thing. And you'll see some of the methods that the attackers used. And you know, the thing about it is, phishing still works.

Trey Ray:
It's still one of the main ways that credentials are compromised and accounts are taken over. So passwords really just aren't going to cut it for us any longer, and we realized that a couple of years ago. And it turns out, this notion of Zero Trust is not really a new thing. This fellow, John Kindervag from Forrester Research, wrote a couple of white papers back in 2010 on this notion of Zero Trust security. Basically, he says, you know, instead of trust but verify: verify everything. You've got to stop treating people like they’re packets. All networks are considered untrusted and you must assume breach.

Trey Ray:
And then in 2014, Google really built their own version of Zero Trust and, to be honest, I think they're still on that journey, and they published another set of white papers as part of their BeyondCorp initiative. And actually, if you go out to BeyondCorp.com, you can learn about the Google Zero Trust framework and hear some of the tenants that that's based on. Now, what I tell people, you know, Google's got probably 75, 80% engineers (or propeller-heads, as we call them), so it's a lot easier for them to do a build versus buy. FedEx doesn't have that luxury, so we're more of a buy type of model in terms of implementing Zero Trust.

Trey Ray:
So really, if, from a high level, this is kind of our 10,000-feet diagram of how we envision Zero Trust security at FedEx, and it's really about doing user validation and marrying that up with device validation. Instead of using a username and password, take that a step further and validate the user with, you know, push notification, maybe a YubiKey, or even WebAuthn with biometric authenticator on your MacBook, or something like that. And then device validation. You know, is the device joined to our main active directory domain? Is it managed by our Workspace ONE MDM? And then using that to make a decision on how to tailor the sign-in experience when they log into FedEx applications and resources.

Trey Ray:
So next, I want to take a look at what I consider the six building blocks of Zero Trust security at FedEx. So the first one, and what I consider the cornerstone, is an identity provider. And for us, that's the Okta Identity Cloud, with the identity as a service model using Universal Directory, SSO and support for, you know, SaaS, cloud-native and legacy applications. Second is this notion of adaptive and textural MFA, using Okta Adaptive MFA. Now, is it MFA all the time? No, it's MFA based on the situation and context. As I mentioned, we're definitely using Okta Verify. We also still hold onto the hardware authenticators for certain use cases where you may not be able to take a phone into the operation, and that type of thing.

Trey Ray:
We're also piloting the more modern authenticators, like FIDO2 and WebAuthn. As a matter of fact, when I log into the Okta admin interface, I'm able to do touch ID on my MacBook and it's very low friction, very cool. Next is endpoint security and monitoring. That's about antivirus, things like that. Making sure the device has good posture and marrying that up with the things we're doing on the identity space, and trying to support personal devices where possible. I understand Okta will be introducing some things at this virtual conference that will help us in that space, so I hear. And then the access proxy, Access Gateway. Prashanth touched on that. That's the F5 APM that we're using as a bit of glue to, you know, bring legacy applications that may be using header authentication into the Okta fold.

Trey Ray:
And then we're also doing a lot with network segmentation. We've got a huge project to segment our data centers, our sort facilities, and our hubs, to keep our network from being flat. That involves macro segmentation, as well as micro segmentation. Then access policy engine, and this is really the brains of the thing. This is what helps us make decisions on how to tailor the sign-in experience. You know, is it password only? Is it no password at all? Is it password and MFA? The engine lets us build those policies and rules to make those decisions.

Trey Ray:
And then, UBA. We've got some things going on with Splunk and some machine learning that we're doing on our own with a data lake, and basically just trying to take all of this rich identity data and make decisions about it and be proactive from a security perspective. Okay, I want to switch gears a little bit, and kind of talk about a real-world use case in terms of Zero Trust at FedEx. And this is a partnership-type thing where three companies have worked together and that's FedEx, Okta, and VMware to deliver this Zero Trust use case. So Prashanth, you want to take this one and describe what we've done here?

Prashanth Karne:
As Trey mentioned, FedEx mobile device management is done via AirWatch Workspace ONE. So what we did is, well, when Workday announced that they're going to provide a feature like where we can limit access to self-service or provide full-fledged access based upon the device type, we spoke to you after we spoke to VMware and we came to a conclusion that we could cook something up where we can provide a conditional access based upon whether a particular mobile device is managed or a personal device.

Prashanth Karne:
The way we did it is, we have registered Workspace ONE as our IDP within Okta and we have created specific routing rules, where the device trust with both iOS and Android platforms. And also, we spoke to Workday and then we are kind of passing a SAML attribute if the user is actually using a managed mobile device, managed by FedEx IT. If the user is not using a FedEx-managed mobile device, then we flag that particular SAML attribute as a personal device. Based upon that, Workday will take it further and they will restrict the access to a full-blown access if the user is coming from a FedEx-managed mobile device or just to self-service or, in some cases, read-only if the user is coming from a personal device.

Prashanth Karne:
So the flow of this particular policy works like this. When a user opens a mobile app, then, in this case like Workday, so the user will go to Workday and present that, ”Hey, I want to access this particular Workday tenant.” Workday will sense that it is a federated tenant and it will send the authentication request to Okta. Now, Okta detects that the user is coming from a mobile device and then it will fire another authentication request back to VMware. So in this case, Okta and VMware kind of switch the roles. Now, Okta becomes the service provider and VMware becomes the identity provider.

Prashanth Karne:
Now, when VMware receives the request from Okta about this particular user, then VMware further checks whether that particular device has the identity branded to that particular user and it will also further check whether the particular device is in a compliant status, and then whether it has all sorts of security policies and configurations in place. So if everything goes well, VMware will reply back to Okta saying that, “Hey, the user is found on that particular device and the user identity is so and so.” And will also tell Okta that the device is in compliant status, with all the patches and configurations in place. So at this time, again, there will be another switch.

Prashanth Karne:
Now Okta becomes IDP and then VMware becomes service provider after receiving the response back from VMware. Now, Okta gets back as an IDP and then rewrites the SAML authentication response and sends it back to Workday. So when it sends that, it will put a special flag, a SAML flag, based upon whether it's a managed device, or a personal device, or whether that particular device is found in FedEx IT, FedEx managed tenant from VMware. Once it does that, Workday will receive the SAML authentication response and then reads whether it's a managed device or personal device. So based upon that, Workday will further apply policies on their side and then provides a full unrestricted experience for the user if it is a managed device. Otherwise it will lock it down to self-service, so that the user can only use certain features from Workday in that particular mobile app. Back to you, Trey.

Trey Ray:
Thank you, Prashanth. I wanted to take a minute and cover some of the accomplishments that we've got with the Okta program. So that we've been really busy the last three or four months, and these are some of the SaaS applications that we've migrated to Okta SSO, some of the heavy hitters on top with the larger logos. I wanted to call out that, due to the increased work-at-home environment that we've had the last few weeks, we actually had to accelerate some of this, and over a 36-hour period, we actually moved Workday, Office 365, WebEx, Check Point VPN, and Zoom to Okta SSO in about 36 hours. And I'll tell you, that was trying, but we did it, and excellent job. My folks stepped up and we got that done. Now, we’ve probably got another 100, 150 or so SaaS applications to go, but we've put a pretty good dent in it.

Trey Ray:
I'm really proud of the team. I wanted to wrap up with a bit of a ‘lessons learned’ about, you know, setting out on a large program like this. There's some key takeaways. You've got to have executive sponsorship. If you don't have that all the way up to the CIO and the CSO, you might as well pack it up and stay in the truck. You're not going to get anywhere. Fortunately, we did have that and it's made us successful. You’ve got to have a great communications plan. We worked closely with our communications team and they helped us draft emails and communications that helped us through the migration journey, letting people know this type of thing was coming.

Trey Ray:
As I mentioned, we produced videos and things like that to help people with Okta verify enrollments. So communication's definitely a key. Also, you’ve got to do a phased migration if you're an enterprise of any size. As I said, we divided it up into two or three main chunks, with SaaS apps, then cloud-native apps, and then wrapping it up with legacy apps. You can't do it all at once. Put yourself a schedule together. Try to stick to it and then, finally, and this goes back to the customer-first notion that Ryan our CSM mentioned: you’ve got to establish a partnership with Okta. In addition to that, you need a good third-party integrator, which we have.

Trey Ray:
You know, we partnered with Okta. They came in after they won the business and we did onsite Okta administrator training and things like that. And then we worked with a third-party Okta integrator with some of the heavy lifting. I've got some pretty smart folks on my team, though we did need a little help with some of the more gnarly things. So Ryan, before we wrap up, did you have anything you wanted to add on the customer-first side of the house?

Ryan Rudnitsky:
Sure, Trey. A lot of the time it's really just based on that partnership. It's making sure that our customers understand the features and functionality, and what's going on, and if they need more information on that particular use case then we can dig in and maybe talk to some people at Okta, whether it's the product team, some of our engineers, and making sure that we're all in alignment with what your migration goes for. And then, secondly, is making sure that support is on call, in case any issues or anything pops up, we're able to handle that super quickly. So I think it's a great testament to show how, within 36 hours, we could get Okta up and running with FedEx. But I think that moving forward, we are going to take that in stride and get the remaining apps, as Trey was talking about, as well as other new functionality, that come in the future.

Trey Ray:
Thanks, Ryan. Appreciate that, Ryan. So with that, that concludes our presentation. We will be online to handle the online question and answer, and we appreciate your attendance in this virtual conference. Thank you.