Production Testing Policy
Why we have this policy
Production environments hold real customer data and support real business operations. Testing them therefore introduces risks that do not apply, or apply to a lesser extent, in a non-production environment (such as staging, testing, or UAT).
We’ve published this policy so you understand our approach, and can make a confident, informed decision about how your testing should be scoped and undertaken.
Our preference
Our default position is not to perform application penetration testing against production systems.
Wherever practicable, we recommend that testing is carried out against a non-production environment that is as representative of production as possible. This allows our testers to work with the freedom needed to properly assess the application’s security, utilising real-world exploits and invasive techniques without the risk of affecting real users or live data.
When production testing makes sense
We recognise that a representative non-production environment is not always available, generally for budget reasons, and that some forms of testing are only meaningful when carried out against a production target.
In these circumstances, phew will discuss the implications with you and agree the approach in writing before testing begins, as part of our standard scoping process.
What you should know before agreeing
We believe in being upfront about the trade-offs. If production testing is being proposed, we’d like you to be aware of the following caveats:
- Reduced assurance: To protect your live systems, we would only use non-disruptive methods. In some cases, this may prevent us from being able to confirm or rule out a vulnerability.
- Risk to the target itself: Regardless of the guardrails, and due to its offensive nature, there is always a non-zero risk that testing breaks the target, on top of the information risk to live customer data. The target should be subject to adequate (at least daily) backup and reliable recovery processes with adequate RPO and RTO.
We’ll ask you to weigh these availability and data integrity risks against the value of testing the live environment and to confirm in writing that we can proceed with testing.
Controls we apply during production testing
If we do agree to testing on a production system, we will apply a heightened set of controls to reduce the likelihood and impact of any disruption. Here’s what it looks like in practice:
We stick to non-invasive techniques
We use non-invasive scanning and non-disruptive testing methods only. Any active exploitation with a real chance of affecting stability, such as denial of service, is off the table unless you specifically agree to it, and understand the associated risks.
A real person is doing the testing
Our work involves the use of technical tooling, but is manual and tester-led rather than fully automated. Where we do use AI, it is to augment the planning, orchestration, and speed of authenticated black-box testing, with LLM agents allowing our testers to run our existing, proven tooling, just at AI speed. This means an experienced human is making the calls, and can pause or change approach at any time.
We agree the rules upfront
Before testing starts, we set out clear rules of engagement with you, covering what’s in and out of bounds, when we’ll be testing, and anything you want kept off limits entirely.
Approved access and tooling only
All of our testing runs through our public IP address, using tools we’ve vetted. Any activity against your systems is traceable and can be distinguished from malicious attempts.
Comms are open
We stay in touch with your team throughout, so if anything looks unusual, either you or we can raise it straight away, and you’re never left wondering what we’re up to.
If in doubt, we stop
If there’s any ambiguity about whether something is approved for testing, or if we think our testing might be having an effect on your system, we pause and check with you before continuing. Anything that does affect your production system gets escalated internally and communicated to you promptly.
But ultimately, you accept the risk
We take all of the above precautions, but ultimately you accept the fully disclosed risks associated with offensive security testing against your production target and the information it contains, and with recovery from an unwanted outcome from our testing.
Getting the approach right for you
Every engagement, including whether and how production testing happens, is scoped collaboratively and confirmed in writing before we start. Nothing gets tested in production until that’s agreed and you’ve told us you’re comfortable with the risks above.
Questions?
If you have any questions about this policy, or would like to discuss the right testing approach for your environment, please get in touch at [email protected].
