Amway Builds Global Identity in the Cloud from the Ground Up

Lorraine C.:  Welcome to the second breakout session in the customer success track. My name is Lorraine Costello, I run the strategic programs team for customer success at Okta. I'm really excited to welcome Amway to the stage, they've got a really cool success story to share with you today. I've got Brian Hibma and Peter Richardson from Amway. They're going to talk to you about building a global authentication platform for millions of customers as well as their employees as well. Without further delay, I'm going to hand it over to Pete and Brian. Thank you.

Brian Hibma:  All right, thank you.

Peter R.:  Thank you.

Brian Hibma:  All right. Thank you, everyone, and good morning. My name is Brian Hibma. As she said, I'm a solution architect at Amway and this is my colleague Peter Richardson who's a development team lead and worked on the solution that we'll be showing you this morning. We're excited to be here to share with you a few of the details of Amway's path towards building a global identity in the cloud and built on top of Okta as a platform. When I say global identity, I'm referring to this external use case, one that's customer-facing and B2C. It's been our experience that this is very different and unique from the more traditional internal employee, sort of B2B style use cases that you'll, I'm sure, hear about while you're here at Oktane. Our goal over the next 45 minutes is to walk you through what this type of use case can look like and how that's going to look for your company. Our hope is that you can exit this talk better equipped with some of our successes and our failures as we've tried to approach the solution as you can apply it to your implementations.

Before we get started, I'll tell you a little bit about Amway, the company. Amway is the number one direct selling business worldwide. We operate in over 100 different countries and territories. We empower a network of independent business owners, millions of them around the world who earn a primary or secondary income selling Amway products in the home, beauty and nutrition categories. Amway was founded in 1959 so it's coming up around 60 years old, and that's relevant to our discussion today just because it's an established company. It's a global company and as you can imagine, that provides us with a lot of opportunities but also some certain challenges as we try and work past some of the legacy and older implementations and move forward with identity.

All right, so our talk today, we're going to go through three different sections. The first, we'll start out by describing what this business climate was that we were in, what was the rationale and the reasons for starting down this path in the first place. We'll move from there over into the solution that we created and we'll talk about some of the techniques that we applied towards solving this identity problem at Amway and also we'll take a close look at a few different technologies that were helpful for us. Then we'll wrap things up by covering a few of the lessons that we learned along the way and some of the key takeaways that we found. To run this through some of the business case and why we started this path in the first place is Peter Richardson.

Peter R.:  As Brian had mentioned, Amway has been around for 60 years. They've been in all these different markets and they grew these solutions organically. Each market would have its own business rules, its own implementation on how they did authentication and so that caused a lot of fragment implementation. We had a different implementation in the Americas versus Europe versus Asia. That also had varying username and password policies. When we took this on, we had all these different markets with all these different rules and different implementations of what they were doing.

Another thing a while back is that the credentials were at the business level so you might have a business with five different people but you only had one login so that login had to be shared amongst all those five different people. Each business had its own login so we knew we had to fix that because we want to make sure that going forward, if we have a global identity, that it was going to be at the individual level and not at the business level so we could do roles and other types of things. We knew we had to fix that also.

Now, they had one login for business but businesses especially the big hitters would actually be in different markets. Business A might have a business in Asia and also a business in Europe. What happened in that situation was that those distributors with multiple businesses across the globe had separate credentials for each of those regions so not only would they have a shared credential for the business in that region but they would have multiple logins for each different region of their business and we want to fix that too. We want to have just one login for them and it would expand their whole global business. Along with that, I do want to mention that within a region, there's also multiple applications that we had and then they would have different logins for those different applications so we really want to clean that up.

The last thing that I want to mention here is that as these things grew over years and as the markets grew up, they had different user experiences. Person A might have a login in Asia and the user experience for their Europe login was totally different. It was kind of aggravating for them to have these different user experiences but also, for us, presented some problems on maintenance of those so we had all these different user interfaces. Also, the security had to be managed for all those so we had to make sure the security for that login and that other login were the same things or that they met the security provision that we had set up. With that, those are the issues that we came to solve.

Brian Hibma:  This isn't where we wanted to be as a company and our leadership recognize that if they were going to implement some of these key initiatives that they had going forward, it depended on being able to uniquely identify our users. They recognize that identity management was a critical component to some of their future plans.

As we look at what we wanted to do going forward, there are a few categories and topics that we wanted to solve with an identity solution like this. First of all, we wanted to be able to provide a unified identity platform both in terms of the experience our users would go through so the flows of log in or changing a password, resetting a password, we wanted to standardize that. Also on a technical level, for application development teams, giving them a platform that they could build from and rather than having this inefficient and error prone way of having each application take care of identity themselves, we wanted to provide them sort of the standard baseline that they could build from. We want to improve some of our security practices. There were varying degrees of adherence to basic best practices in security and so we wanted to bring all these applications up to at least a minimal level of adherence to those best practices.

Also forward-looking and what we wanted to do in the future. Security is not a stagnant thing, it moves and it evolves and so if we wanted to be able to rapidly react to changing security needs, we have to have something that was consistent among these applications. For example, if we wanted to introduce a multifactor authentication step in our login, we needed this platform to be able to do that consistently across all these apps.

One of the key benefits for our end users and ones that our business identified as a benefit was being able to reduce and eliminate the friction of users moving from one application to the next. Being able to provide the single sign-on experience was something that we really recognize as a key benefit to our end user.

Making that credential, getting it down from that business level down to the individual like Pete explained, we knew that our solution would have to account for this change management and migrating that credential. In the past, when we had rolled out identity systems like this, we were able to migrate that credential for the user behind the scenes but because we were shifting the level, we knew that we had to account for this kind of change management and this will be a key piece of what we would build.

Then finally, we knew from customer service data that just getting in the front door of our applications, getting past login was often a difficult step for our users and was often reported as a key pain point. We thought that by creating a global user experience, one that's easily recognizable, that this might help with some of that pain so that when users are going from one application to the next, instead of encountering a varying degree of different experiences and patterns, we can make that consistent for our users so that this was less of a burden on them.

What we built is what we call the Amway ConnectID, that's the brand we gave for this new identity solution. We have a little teaser video here to explain a little bit of this so let's give that a watch.

Speaker 4:  Introducing the new Amway ConnectID. The Amway ConnectID will make it easier to access multiple applications with one password. This ID provides an easy and secure single sign-on experience for ABOs worldwide or anyone registered with Amway. The secure and up-to-date Amway ConnectID is unique to you and the basis for your Amway digital identity. This ID will provide you a more personalized experience that will better accommodate your specific needs across all Amway applications. Your personalized Amway ConnectID will eliminate past confusion caused by others in your group or business changing or resetting passwords and allows you to manage your own identity with one username and password on multiple software programs and apps. You don't need to be concerned about your online security, your new Amway ConnectID provides a safe, secure and consistent experience no matter where you do business.

Brian Hibma:  All right, so there that is, that's the ConnectID that we built. I wish that I could stand here and tell you that that was exactly what we built right out of the gate, that was our first attempt and we hit it out of the park but, unfortunately, that's just not the case. This is actually our third attempt at solving this identity problem at Amway. I think it's instructive for us to look backward and see what we did on those first two attempts and why those didn't work.

Our first attempt, we had a project that was underway that needed an identity and we had this idea taking form of what we wanted this global identity to be and so we thought, "Okay, it'll be great, we just merge these two things together and we'll tackle them in this one project." We built a custom user interface to handle these identity concerns but blended it in with this ongoing project. This did result in a working solution and it's actually in production today still being used but it never made it anywhere past that, it stopped at that one implementation. The reason it did is because what we recognize later, once you built the solution, you're about halfway done, there's a whole element of change management and strategy planning on how you're going to roll this identity out to the rest of your applications. That was lacking when we combine the projects this way, there wasn't enough time and focus given on how we were going to do that and what our strategy will be.

We went from there and we corrected for that in our second attempt where we did dedicate a team to the solution so we have a team focused on this and planning that strategy. One of the questions that we get most often, even still today in what we've built now is why can't you just give me a series of APIs and I'll take care of the user experience around this. I don't want you handling all that, I got the user experience covered, you just take care of that back end stuff and we'll take care of the experience. This is a legitimate request and, I think, expresses the need that application owners have to be in control of that overall experience, they want to know and control what that's going to be for their customer.

This is what we try with that second attempt. We have the team and we built a series of APIs that application was going to integrate with. Again, this can work, it did produce an application that runs in production today; however, what stopped that one application was the fact that the integration is actually quite difficult. The degree of effort it was going to take to move that and scale that out to other applications we saw to be prohibitively high.

The reason for that is that the user experience around identity management is actually a main piece, it makes up a large amount of the work that's involved in deploying the solution. So much of it is flow-based, if you think about the process of changing a password or resetting, you start with probably a screen that takes in some piece of their identity, triggers an email or SMS, comes back to the application, verifies that, then presents some of the screen that they can change that and so that whole sequence of events that flows, the user experience flow. Even more so this migration that we have to do, we knew that was going to be a multi-step workflow we would have to take the user through. If that was owned by each application, building and integrating with this identity, it just wouldn't scale. It also meant that we wouldn't be able to move forward on things like that multifactor authentication, it's more than just a series of APIs we found.

Our ConnectID attempt is actually correcting for both of those. We dedicated a team and made the strategy change management plans but then also made a custom interface that our users would use.

Peter R.:  As Brian had mentioned, we tried a couple of different approaches. We found that we wanted to use identity provider as a framework piece. Because of the different approaches and just doing APIs or just doing one application, we needed to control the branding, the messaging, the user experience of that so we built ConnectID in the middle of the user and Okta. Normally, you guys might have done or what we've done in the past is use Okta as a Sass and we use all the emails out of the box, we use Okta out of the box, all the screens from them. We found that with the things that we wanted to tackle, we needed to put some user interface in the middle of that that we had the brand and we had the control.

We end up using Okta as an identity provider and Okta was a real good fit. We use them for our corporate login and they were trusted and we knew that we could accomplish those with Okta. We also use the customer success manager from Okta a lot and helped us integrate these user flows, OIDC flows, all these different things that we wanted to do with ConnectID. They helped us a lot with architecture and putting that together.

This also had some different things that we had to accomplish because if were using Okta out of the box and using our emails, there are some things that we wouldn't have to do. What we did have to do is make sure the security. Since we're writing code and building this in the middle, we had to make sure that it was secure in all the security requirements that we needed to do because passwords and stuff are flowing through that. We also had to do some appliqued and provisioning so we had to provision the applications that were using our ConnectID to do their login and their authentication and we also had to set up API keys for that. All in all, this worked out really well and having Okta as an identity framework that we built upon really worked out really well for us.

With that, having a dedicated team, we wanted to make sure that we were structured for success. To do this and to build this, we want to make sure we had all the different roles that we needed. We did build this project using agile methodology, we had a scrum master, we had two-week sprints that we were able to, then, tailor what we were doing in two weeks to fit the needs of this changing landscape as we build this application and that worked out really well. We had Brian as architect in architecture so we had enterprise architecture involved from the start. We had business analysts that knew all the different markets and all the different places of where this was going to be used and they helped us to make sure that the rules and the business rules that we were building here were correct. Development, we had three to five developers building out the solution along the project.

I think where I really wanted to stress is two roles that really helped us make this a success in infrastructure and the first one is user experience. I don't know if you guys, with your projects, have user experience involved or somebody who's dedicated to user experience but we had a dedicated person who all their job was to do was to look at the user experience of logins. When we had to take this from one login for a business down to having an individual login, how that flow went, how that would do in different countries and different languages, we had some guy that all he did was that. He looked at that, he ran tests around that with user experience and that was key to that because we really need to hit a home run with this.

Then the last one was QA. We had a set of QA testers that all they did was test our solution and make sure that it met all the standards and it met all the requirements that we laid out and met all the user experience stuff that we did. From there, we had everything in place and the team was in place for success for that.

Brian Hibma:  All right. From a technology standpoint, how we built this. We knew that we had a couple of constraints technically that we had to operate within. The first was that we knew that as soon as we had a solution available, the demand that was going to be coming at us was going to come, really, from all directions and all countries and continents so whatever we build had to be able to scale out rapidly to meet that kind of demand.

We knew that availability and quality are essential to, really, any project that you're delivering but, certainly, this one as well and not just in a production environment. What we found is that as soon as applications were dependent on you, let's say that they're giving a demo to an executive or they are going through a user acceptance testing phase of their project, this kind of things that take place in lower environments which, traditionally at Amway, had been accepted as lesser quality or lesser stability, all of a sudden we were required and had to be up for those, there will be no accommodation for disrupting an executive demo like that. We needed to ensure that we were highly available and of high quality.

Performance, obviously, we needed to account for that but the nature of the application is a little different than your typical app. Users don't come to this application to hang out a while and do some shopping, it's just an identity solution where they're having a task that needs to get done so we needed to tailor our application for that type of posture of getting the user in and out and getting their job done. As Pete mention, we only had between three and five developers working on this at any time and so we had a rather small team that had quite a large application to support. Like any business, we're instructed to keep our costs under control. This was the technical boundaries that we had drawn around this.

Few of the decisions we made technically to address some of those concerns. First and probably the most important, we decided to go fully to the cloud and build a cloud-native application. We've been coming to Oktane now for a few years and we've been pretty inspired by some of the architecture that Okta has shown as well as some of your companies and presentations we've seen. We felt at Amway, it was the right time for us to move into a cloud-native type app.

We settled on AWS as our cloud provider and we were hoping to achieve these benefits out of it. First, that global deployment, being able to rely on AWS' global infrastructure to deploy this application meant that we could locate close to our users this solution to give them the best latency possible. Then also with availability, any good cloud application that's going to be deployed has to have at least two regional instances for failover and high reliability. We took an approach of having different locales, our Americas, Europe and Asia we call them and within those, we could use multiple AWS regions for this fault tolerance.

There was a third thing, though, that we realized was a benefit and that was cost control. It's an indirect way but by having actual metrics on each infrastructure element and what it's costing, gave us the power to be able to fine-tune our architecture and minimize that cost. We could see a detailed metric for each piece of our application stack and so this also help contribute to cost control.

Now, as for why using the cloud, I mean, this is pretty well-documented elsewhere and I will cover that but a few of the key benefits that we wanted to realize through using the cloud is, first, Amway's traffic pattern is not linear, we get extraordinary amount of traffic towards the end of our month and so we knew that scaling and getting a correct scaling policy was going to be critically important for us especially as multiple applications are hitting us at once.

There were other some lesser known benefits or ones that we didn't really think of initially when we made this decision that came to be, actually, very important to us. The first is that by using the cloud, we could give our developers an environment that closely mimics the production environment as well. This took care of a whole category of issues for us where a developer could develop with confidence that the infrastructure he's working on matches what's in production and there's no surprises as you transition that application into a production environment. Furthermore, we always had problems as we moved our applications through our upper environments where configurations will be slightly different or the infrastructure might be totally different, in fact. For us, to be able to have a reliable way of moving this through those environments where it was the exact infrastructure and infrastructure as code meant that we could have that kind of confidence that we wouldn't encounter those surprises as we move through.

How we use the cloud, Amway's background was probably pretty traditional where we started infrastructure as a service with EC2s and raw servers and then we moved unto more of a platform as a service with Elastic Beanstalk. That give us a nice abstraction over top of these servers where we didn't have to manage as much of the direct, sort of bare metal configurations.

We wanted to take that further and that's what led us to our second decision which was to go to a serverless deployment model. Like the name implies, essentially, you don't have servers, there's nothing you're SSH-ing into to go get access to that machine, you give that control up to AWS.

There's a few reasons for doing this. First, from a maintenance standpoint, for us that was a large win. Having these servers that we would have to manage, we realized, especially for giving the exact same copy to our developers, there was no way between patches and upgrades and operating systems that we would have to keep control over, that we could scale that. Serverless being a function as a service, we use AWS Lambda to do this. You upload your code and then give it a couple of traits about how you want the memory profile to be, the timeout environment variables and so on and then AWS handles that for you. There's still service, of course, running but AWS is managing that and scaling that for you.

A second attribute to this was that it was cost-efficient as well. A serverless technology is you're paying only for what you use. If you think about as many developers as we had and as many environments as we were planning on, those environments that weren't being used aren't incurring cost as they sit there dormant and so for us, this actually became very cost-efficient. In our lower environments, many times we didn't exit the Free Tier on AWS just because they weren't used much.

This was a little bit more of a risky decision for us, it's a newer technology and so I don't think that we can leave this slide without saying that there's definitely some downsides to consider if this is something interesting to you. It's a newer technology, like I said, and so the tooling and the models for architecture are not quite as mature. It just is something where you're going to have to be a little bit more creative in your solutions. An example of this, we have a case where we needed what would typically be a reverse proxy sitting in front of our user interface and you don't have that Apache or IS server to rely on. What we had to do is use AWS' Lambda Edge capability which sits on CloudFront CDN edge nodes and gave us that opportunity to solution for that similar type of problem. It does require, I think, a little bit more creative way of looking at some of these problems and that's something to be aware of before you go down this path.

Our overall simplified architecture here looks something like this where we had on our user interface side, we stored static assets and S3 buckets and we surface that through CloudFront CDNs regionally, Route 53 for DNS and then the application itself as a single page app that runs in the user's device. From the backing services then, like I said, Lambda is that layer that's giving us that serverless capability and that scales horizontally to meet demand. Each Lambda service is its own piece of the application, its own function so we can scale login functions independently from other functions on the site. Those talk to Okta in the API configuration so it's using Okta APIs extensively, we really don't use much of the Okta user interface and those also talk to our own back end integrations as well. We front that by Amazon's AWS API Gateway and CloudFront is in play with that as well. Then Route 53 again for DNS, Akamai is what we use to geolocate our traffic and so we can route our traffic based on user's regional proximity to a given node.

There's a couple of other supporting services too that we use. CloudFormation, again, is one key service that we relied on quite heavily, that gave us that infrastructure as code and ability to deploy each of the infrastructure elements in a reliable way.

CloudWatch for that monitoring and quality assurance. Being able to set up custom metrics and alarms that watch our application in each of the different regions is what gives us that assurance of when we do a deployment, that we're doing it correctly, that we're not encountering errors. We have some other monitoring tools as well but from AWS, we really relied on this one piece.

We do use Kinesis as well, we're starting to use that for an events feed and eventing out together other applications. That's something we found other applications really want is if you have this centralized hub whether it's customer support applications or whatever else it might be, they want to know what activity is happening in your identity solution so that they can ... for instance, when an end user calls up a customer service agent, they can say, "I see that you've had this failed login attempts and done these other things," so we're publishing those events through a Kinesis stream. Then sending out emails through the Simple Email Service as well.

Peter R.:  Being cloud, we can't have a talk without DevOps, right? DevOps, for us, was extremely important to the success of what we were doing. If you look at our application, it really has just the user interface. We have built Angular 2, we use Angular 2 for our user interface. We had a set of internal services and those internal services where what's our internal applications were calling to get information about user logins, user accounts and stuff like that. Then we had a set of external services and that's where our user interface was calling our external services.

Overall, this seems pretty doable. I mean, there's three applications here, I mean, why would we need all this DevOps, this automation to maintain all of this. When you start breaking these things down, okay, well, our user interface now has DNS, it has policies, it has a CDN, our internal services have DNS, CDN, policies, monitoring and providers, all of this. As you start breaking these things down especially in a serverless application because we don't have EC2 servers, all of these ... built off services whether it's a CDN, whether it's the internal services with an API Gateway, all that got to be configured and deployed to do that. We knew that, okay, these are just the three services and each one there goes in with that but you even go a little bit further down the road because, then, we got multiple deployments.

The scariest time is when you do a deployment, can we make sure that it's up during that time and we couldn't go down. We had blue, green deployments or we had ... Let's just say we have our green solution in load so that's in load serving traffic, what we would do is we would bill out to a blue to another whole stage, stage that environment, test the code, test what was happening there, making sure that everything was good in that environment so then we could switch. Then once we switch, it took less than a minute and then all of a sudden the traffic that was going to the green stage was going to the blue stage. For us, that was extremely important, that we had no downtime. This being a global application, there really wasn't ... It's 5:00 everywhere, right? 12:00 here or 12:00 at midnight here was 9:00 in China and Japan and stuff like that so we knew that we had to be up 24/7.

Along with this, now we got our user interface, our internal service, our external services with all those specific services that we had to do, then we had blue, green deployments, then on top of that, we also had it across the world. This is why we pick AWS because they have a global infrastructure in place. Through our services here, we've got it deployed in six different regions so we've got it in Virginia, Oregon for the Americas, in Europe, we have it in Ireland and Frankfurt and then within Asia, we have it in Tokyo and Seoul so we have six different deployments. On top of all of this, our user interface in a production mode has blue, green deployments for each of the different regions so we've got 12 of those. In our internal services, we got 12 of those deployments. That's just production, that's not our QA environment and our test environment and our dev environment. All in all, we got probably around 50 stacks that we have to maintain.

We could never have done that if we're going to manually do this so we took a lot of time upfront to build out our continuous integration and build out and automate all these deployments. Everything from the user interface, from a CDN, internal services, we use CloudFormation and so we automate all that. The infrastructure as code, we got all that automate. With a push of a button, we can build out one of those stacks without doing anything and have it all be automated from that. We use Jenkins and CloudFormation to do all that and that really help to make sure that what we were doing here, we could automate and maintain with the three to five developers that we had.

To automate this, I did want to bring up a few more things and that is the other stuff that we spend a lot of time is automate the testing. All our code, to do this, we wrote unit test. We spent a lot of time making sure from the user interface to the internal services that we had unit test being written for that.

I think in our services, we have about 90% code coverage on the unit test. On the UI, for other reasons, we weren't as high on the unit test coverage with Angular. Angular 2 had come out and they had changed a lot of different ways on how they want to do the unit test and stuff like that so we end up being about 35%. Then we look at all the issues and look at what we've seen, our services that have the 90% unit test coverage are actually a lot more stable, there's actually a lot less bugs and a lot less going on there whereas the user interface, they're still very good but we notice more issues around that. we attribute that to the upfront time that we spent on the unit testing to have that there.

The other thing that I did want to mention too is that we have ... because of the unit test on a user interface, we also spent a lot of times writing automated test using Selenium and Protractor and some of those kinds of tools to actually automate the testing of the front end or the end to end testing for that. We have a whole suite of tests that we do. When we check code into our development branch, actually, automatically the CI process kicks off and build and deploys that into our dev environment. From that, we, actually, then kick off a set of integration testing that click through and drive the browser and go through a bunch of different tests to validate that that particular deployment is good and it works. It doesn't do everything, it's really [inaudible 00:36:01], is this particular deployment good enough for that. From there, if it doesn't, if it fails any of that kind of stuff, we do get notified.

The idea is to catch these issues quicker up and not have it to go all the way to QA environment to have our test review it, we wanted to find those types of things quicker in the process and that worked out really really well for us.

All in all, we spent a lot of time upfront making sure that we have all of our infrastructure automated so that we can build out these stacks without doing a lot of manual maintenance. We spent a lot of upfront time creating the unit test and the automated test so that when we do these deployments, we can find these issues up quicker. All in all, this really has helped us, with a smaller team, be able to build out a global infrastructure to do that which is no small task.

With that, kind of want to go through some lessons, just a recap of what we talked about through the slides. The first thing from the project, as a team lead, I have the project learnings. Having the Okta customer success manager help was beneficial to what we were doing. If you have those guys, use them. We ask a lot of questions especially on an OIDC flow, they helped us with a lot of the architecture and how to do a couple of things, it was really helpful.

The time spent upfront was really worthwhile. As I mention in the previous slide, the time that we spent up putting together all of this automate and all this infrastructure so that we can build a stack with CloudFormation with one click was really beneficial. We're really seeing the benefits of that now, when we do a deployment, it's basically just scheduling all these things to kick off and it's worked out really well and save us a lot of time. The third one is dedicated team and executive buy-in. Like I said, Amway put together a really good team and we had buy-in that we're going to do this correctly and we're going to put all the right pieces in place.

UX wins over API approach. Having that UX there and having all these different applications be able to use one user experience and having the control over that worked out really well us. The last thing I want to mention is change management. It wasn't easy having all these different markets that had their own autonomous way of doing this, They came in with, "We want to work like this, we want to work like this," and we had to look across globally. This was no small task to look globally across everything and decide that this solution is going to work for all that and not let small requirements from these other markets kind of erode what we we're trying to do which was build a global solution and not a solution for just Asia or just Europe. In the end, that worked out really well to hold that line from a global ... that this is going to work globally and not just for one market.

Brian Hibma:  Then on the technical side, just recapping some of those, that, for us ... one of the best decisions we made in producing this application. It enabled our developers to be productive and produce a quality output in supporting DevOps, we could rely on its security and fine-grained, we use IM extensively to fine-tune that application to ensure that we're only giving access to those pieces of the app that needed it and getting that quality, of being able to have reliable build all the way through our environment. Serverless, like we said, there are some things to be aware of with that but if you're interested in that technology, it's a good way of making a maintainable product and containing costs. Then finally from DevOps, like Pete said, automating. For a solution like this, that's essential to keep secure and up all the time. Relying on automation and testing like that is going to be key. I think that wraps it up, we're at a point where we can take any questions that you might have on this.

Speaker 5:  That was, obviously, a very mature organization. Do you have any insights or more [inaudible 00:41:15] earlier stage in their development process? The directional [inaudible 00:41:22] so any advice to folks that are just getting started or are very early on in the process.

Speaker 6:  Can you please repeat the question?

Brian Hibma:  Sure. The question is for businesses that are maybe not quite as large or are just starting out and don't have, maybe, as much legacy behind them, is there any advice to those kinds of companies? I think I would say yes, I would say still pursue this. In fact, I think you'll probably find that it's going to be a little bit easier for you just because you don't have some of these competing concerns. You're going to find, hopefully, a little less friction in getting that off the ground. I think, though, that no matter how large or small the deployment is going to be, some of those key principles still remain the same and so I wouldn't shortcut any of those in what you're building, I would still put in just the same amount of effort in producing that quality output. As we were describing there, some of the things that we did in trying to automate in all that, each project, no matter how large, is going to rely on those pieces. Do you have anything, Pete?

Peter R.:  Yeah. Like Brian said, I think it will be easier because you don't have a lot of legacy stuff but make sure to take that upfront time to really hammer down and make sure that what you're building is what will meet the needs of what you have, I mean, that is really important. Especially in a solution like this, security is very important to the needs of the organization.

Jessica:  Hi, this is Jessica. We are new to Okta, we just signed in May. Our CEO has come to me and said, "Jessica, you need to make sure that when Okta goes down, you have a plan to maintain that high availability," and that is central to him and central to us because we've seen that as a three-bucket issue on the East Coast, Azure had an issue recently. What strategies do you employ to maintain that global high availability in the event that Okta, that does a really incredible job, in case they have disruptions?

Brian Hibma:  That's an excellent question. That's actually something that we're currently working through as well with Okta in trying to improve some of that. In our infrastructure, we have that regional failover and that's the same kind of model that we were trying to work with Okta on building and obtaining. Okta does have other regional instances, they have one in Europe, for example. I think that our goal over the long term is to be able to do that same kind of configuration where something goes down in the North American cell, that we can go back to a Europe cell and have that as a fallback. That's an ongoing conversation and for right now, that's not currently what's in place. For the time being, basically, what we have is a situation where we react quickly with them, we try and get these things taken care of as quickly as possible but there is no alternate cell that we're failing over to in that condition.

Jessica:  [crosstalk 00:44:31]. I mean, I'm not even talking about provisioning, I'm just talking about initial login for single sign-on as well as the refresh token.

Brian Hibma:  I'm sorry, could you repeat that piece?

Jessica:  Letting provisioning go aside, they're down 20 minutes, right? The two use cases were most interested in pursuing our initial login as well as issuing the refresh token, those are the two ...

Peter R.:  Kind of read only type of activities.

Jessica:  [inaudible 00:45:10]. You've got an hour on your token window, you've got a ... Okta is down, then what, right? It sounds like you're wrestling with that which I get and a number of other folks I've spoken to here have said the same thing. I think it's an honest question for us all to consider.

Brian Hibma:  Yeah, definitely. I think one thing that gives us a bit of assurance is that Okta does have a difference in its read only mode versus its full functionality and so when they do encounter issues, we have seen a fallback into this read only mode where certain functions continue to operate as normal. For Amway, a big piece of our company is registering new business owners and so during those times, yes, we do have a challenge with adding in new people into that org.

Speaker 6:  I think we have time for one more question or two if you guys would like to chat afterwards.

Speaker 8:  Yes. Just tell me, when did you execute this project and would you do anything different given the state of the Okta technology today? Because we saw a lot of exciting things this morning and you seem to have eliminated the entire Okta interface in favor of the API approach.

Brian Hibma:  Yeah. We started this project about ... I think it was a year ago now so we've been operating for about a year. At that time, they still have the widgets, for example, they have a sign-in widget where you could get a headstart on the user interface. We did take a hard look at that and try and see if that would fit our model of what we wanted to do. I don't think that we would change anything right now based on what we've seen. For example, with login, we needed to do enough customization with that process, that even the widget didn't quite apply to what we need at that time. Beyond that, we wanted to take this solution forward even in the future, we want to get more into account management and profile management, roles management, those kinds of things. We saw this as more of a wholistic solution beyond just the identity piece and so with that kind of strategy in mind, I think that we were pretty set on building this interface over their APIs instead of trying to make use of some of those UI elements.

Peter R.:  What we're trying to solve was so Amway specific in some of these like the login that whatever widgets, they wouldn't have that in there anyway going forward. I agree with Brian, I think we would continue down the path we're going down.

Brian Hibma:  Definitely come up afterwards if you have more questions. Thank you.