Part Two: Bug Bounty Programs — Is Your Organization Ready?
This piece is the second in a series of three blog posts on bug bounty programs and what are some considerations to think about when investing in or launching the program.
In my last post, I discussed the benefits of experimenting with a private bug bounty program before launching a public bounty. Today I’ll share which teams you should involve in the decision and program, as well as a few additional considerations.
Get buy-in from the engineering teams: For security teams that don’t fix bugs, it is crucial that engineering teams are on board with the proposed launch. Typically, there is an onslaught of submissions in the initial weeks after a launch before the counts start tapering off. Difficult to believe for security teams, but developers also have their own priorities and it is not only fixing bugs. Engineering teams need to be on-board and aware of the potential onslaught of submissions. If not, your SLAs and external perceptions of your responsiveness can take a hit.
Don’t forget to talk to DevOps: Imagine 100+ individuals running multi-threaded authenticated scans of your product. That is considerable addition to the typical load. Talk to the DevOps team and identify a joint approach to dealing with this. Always let DevOps follow their internal procedure to protect your infrastructure, but don’t start blocking each IP that sends you <script>alert(1)</script> or you will end up being a support team helping unlock researcher accounts.
Keep your legal team in the loop: Your end goal is to protect customer data and your company’s intellectual property. Typical bounty standard disclosure terms might not account for all concerns that might be applicable to your business. Export control, ownership to fully use submissions or indemnity can be things to consider. Talk to your legal team to figure out if any supplemental legal language is necessary to protect your company’s interests. Though useful to have ready, keep in mind that legal provisions are difficult if not impossible to enforce in this case due to lack of attribution.
Talk to marketing and PR: This will be useful not just to leverage the launch as a PR tool, but leverage the PR to get more researcher visibility for your program. With 500+ bug bounty program out there in some shape or form, sooner rather than later the big question you are going to ask yourself is how to motivate researchers into finding more and higher quality bugs. Don’t keep the payout ranges as your only variable.
Define metrics you want to track as early as possible: In financial terms, bug bounties have demonstrated tremendous ROI. Identify your qualitative and quantitative metrics and know how you plan to record them before you start the program. Typical metrics like submissions over time, sensitivity in submissions of a particular type (for e.g. XSS) due to changes in payout ranges or counts of submission per internal product team are easier to capture. Metrics like the amount of time researchers spend testing a part of your product are harder to gauge.
Know your budget: T-shirts are a dime a dozen. Unless you are a brand researchers want, get ready to shell out some money. Costs of utilizing a Bug Bounty platform are known and upfront. Amount of high value submissions you are going to get is difficult to know. Know the upper end of your budget, keep a buffer and then curate your payout ranges to allow you to pay at least a few very high value submissions before going back to your finance team. Costs of man hours spent by internal teams is also usually unknown but can turn out to be significant. Unless you have dedicated engineers working on the program, expect to re-prioritize internal goals mid-way.
Cover all your bases: Regardless of the scope you define in your rules_of_game, assume that all researchers are not going to follow your guidelines. Most product security teams are focused on the security of your core revenue generating products and researchers might focus on out of scope tier-2/tier-3 web properties like marketing or support portals. You don’t want to be left with a defaced static www site. Schedule and execute security deep dives on these peripheral properties and uncover critical bugs before the researchers do.
Setup the “hall of fame”: Did I mention that not all researchers are motivated by money? If your company has a good brand, a hall of fame mention adds more to a researcher's resume than a one-time payout. It is not only a good tool to recognize the researcher’s efforts but it can also signal of your company’s commitment to security.
Formalize your thoughts and approach to responsible disclosure: Again, not all researchers are motivated by money. Be ready for the eventual disclosure. Bug bounty programs are a partnership to improve security for your customers. Remember to be transparent and always choose to collaborate with the researcher. A joint blog post that showcases your collaboration with the security community is far better than a post by an angry researcher that talks about how you messed it up. Generally, you will find that researchers are accommodating to any constraints you have such as waiting till the bugs are fixed as per your SLAs. However, responsible disclosure is quickly becoming the wild wild west so know when to hold your ground. But more on that later.
Figure out how you want to treat submissions that are duplicates of internal findings: This is a difficult one. When you launch your bug bounty, your internal teams don’t go on vacation and will keep finding vulnerabilities. Often you will end up with cases of high payout submissions that are already internally known and about to be fixed. You will have to either pay the researcher for something you already know about, or annoy them. Not recommended but if you use an external bug bounty platform, figure out a way to export non-discrete information about your submissions to the platform’s analysts. Paying the researcher is the best option.
Implement an internal process to discuss scope/impact/payout: This is crucial. Regardless of how well you have defined your rules_of_game, you will get submissions that fall in the grey area. These need to be frequently discussed within the team and consensus reached on the payout amounts.
Still have questions? I’ll share my third and final installment on considerations for a bug bounty program on the Okta blog next week.
Disclaimer: This article only illustrate some of the things an organization needs to think about when considering a bug bounty program. Each environment and situation is different. The article should not be taken as an exhaustive list of diligence items, recommendations or a set of best practices.
