Will a crowd-sourcing approach to web security testing work for your corporate applications? asks Ilia Kolochenko, CEO of High-Tech Bridge.
A recent incident with the Facebook Bug Bounty program has led to many different reactions supporting both Facebook and the security researcher. Regardless of who is right in this whole story, the one fact is clear: the researcher went far beyond what Facebook had initially expected, and got access to the sensitive data Facebook didn’t really want to share with anybody including the researchers’ community.
These days Bug Bounties become very popular, raising more and more questions about their efficiency and effectiveness. We will try to understand how and if Bug Bounties can be used to test your corporate web applications. I intentionally omit bug bounties for stand-alone software (e.g. Chrome or various IoT applications) as it’s a different topic.
On the one side, the crowd-sourcing approach to vulnerability testing used in Bug Bounties makes a lot of sense. As the more security researchers participate in the audit, the more vulnerabilities and weaknesses they will find. Bug Bounties’ “pay per result” approach also may look very attractive for many CISOs who are constantly trying to optimize their budgets to cover all the risks and threats in a very hostile and dynamically changing cyber environment.
On the other side, Bug Bounties have some pitfalls we need to clearly understand:
1. Do not expect researchers to take into consideration your risk strategy when reporting bugs
A vulnerability you may be perfectly aware of, and that you do not plan to remediate immediately for an economic reason, compatibility problem or due to an agreement with your supplier, will probably be quickly reported to you by several researchers at the same time claiming the bounty. A refusal will certainly provoke the rage of the community that will not report any more vulnerabilities to you in the best-case scenario. Such behaviour is perfectly reasonable with the “pay per result” conditions of complicated work.
2. Few researchers will carefully read your Bug Bounty guidelines and conditions
Yes, you may specify any conditions and criteria for vulnerability submission, however few researchers will fully read them and even fewer will practically respect them. Again, don’t be surprised with such comportment when you pay unknown people per results.
3. Bug Bounty requires very serious technical, human and thus financial resources
Some people tend to think that Bug Bounty can seriously reduce vulnerability testing costs as you pay only for results. For the majority of cases, this assumption is totally wrong, as a poorly-implemented Bug Bounty will just spoil your relations with the security community and create a bad reputation for your company.
Some researchers will even intentionally proceed to Full Disclosure in revenge for lack of responsiveness or award refusal. A good example is the recent mess around Oracle CSO Mary Ann Davidson’s blog post, which berated the constantly growing number of garbage submissions, including copy-pasted reports of automated scanners. If you launch a Bug Bounty – make sure you have enough technical and human resources to handle, analyze and properly follow-up the avalanche of submissions that you are likely to receive – every single submission should be properly answered in a timely manner.
4. You will hardly distinguish Black Hats and legitimate researchers in your logs
As soon as a Bug Bounty is announced, many legitimate researchers from all over the world will start probing your systems. Quite often they will go beyond the initially defined perimeter of testing, trying to compromise a secondary system to get access to the target (e.g. to get source codes of your web application stored on SVN server). Unexperienced researchers and newbies will just start automated scanning with all open-source and free software they will find. Such environment is an El Dorado for Black Hats to hide in the traffic noise, and a nightmare for your SIEM.
5. Unexpected testing methodologies and techniques will regularly appear on your horizon
I personally know two companies that received “Bug Submission Reports” from researchers who proudly claimed bounty payment for DoS attacks against the company’s infrastructure. I omit the huge arsenal of free, often buggy and thus dangerous, tools that [unexperienced] researchers will launch all at once against your systems without any caution.
As a response to some of the above-mentioned challenges, several Bug Bounty management companies recently entered the cybersecurity market, offering a sort of intermediary services between your company’s Bug Bounty program and the researchers. They definitely do great work in helping both businesses and the researcher’s community. However, they have also brought some confusion to the market: some security officers try to compare [managed] Bug Bounties with traditional web security instruments and solutions.
Let’s try to figure out the limits of the Bug Bounties for web applications in comparison to the traditional web security:
1. Bug Bounty cannot serve as an additional protection layer to your web infrastructure
Similar to classic penetration testing or vulnerability scanning, Bug Bounty aims to detect vulnerabilities, and not to patch them or prevent their exploitation (virtual patching). Therefore, even a well-managed bounty will not replace your Web Application Firewall.
2. Bug Bounty cannot replace continuous monitoring of your web infrastructure
Continuous monitoring designed to constantly detect newly appearing vulnerabilities has become a vital part of almost any information security strategy. However, you cannot really force researchers to respect any sort of SLA – they test whenever they want and however they want. So, don’t expect a bounty program to replace your 24/7 monitoring system.
3. Bug Bounty is not suitable to test private systems
Will your risk department feel comfortable sending a 2FA token for your production e-baking web portal to Siberia or Bangalore? Many talented security researchers live there and are listed as top security researchers in the Honor Lists of some of the largest US companies. However, even acting via a managed Bug Bounty program you will never achieve the same level of financial insurance, compliance, liability and personnel clearance as you may expect from a professional penetration testing company.
Some Bug Bounty management companies can limit your bounty project to researchers with high reputation, specific skills or even location – however they just shift their business model towards classical penetration testing with unclear profitability, as the more limits you have – the more difficult is to find a qualified researchers who will agree to work remotely and get paid by result. In this case, you’d better hire penetration testers yourself: many small companies in developing countries will offer you very attractive prices and guaranteed scope of work.
Testing methodologies shall be proportional to your web infrastructure and its expected usage in production. Therefore, if you are a large company with numerous publicly facing web applications open to anyone and designed for everyone – a Bug Bounty program can definitely be a great added-value for your existing security arsenal. Otherwise, you’d be better off concentrating on the main risks to web applications, and spending your security budget on traditional web security solutions and services that are more suitable to your business needs and technical requirements.
Bug Bounty is like hitchhiking – a great way to travel when you are on holidays or have plenty on time, but totally unsuitable for the majority of business trips, which require assurance, quality and accuracy.
