How to measure security alert noise
The two most popular New Years resolutions are to exercise more and to lose weight. People prefer these to more abstract goals because they can’t measure personal fulfillment on a bathroom scale.
When it comes to your security analytics, how do you measure success? While there is no one single metric, teams that do good security analytics seek to minimize the noise from their threat detection. The motivation is that constant alerts wear out analysts and make it more likely that an actual breach will be missed.
This is especially true for fast-growing cloud-based environments where today’s “manageable” number of daily detections will turn into a flood of alarms as the business scales up. As you build security analytics for the cloud, measuring how well your detections scale will keep your security team from becoming a drag on the overall business’s growth.
What’s a good way to actually measure alert noise? For an organization that expects to rapidly scale its cloud server footprint, a simple metric is the Server to Alert (STA) ratio. By dividing the number of running servers by the number of threat detections generated that day, you can get a sense for how noisy are your alerts.
For example, consider a security team responsible for 1,000 virtual machines in Azure. If this team has to somehow deal with 200 alerts per day, their server to alert ratio is 5:1. When Engineering spins up another 1,000 servers, the security team will need to spread itself as thin as butter on toast to try and investigate 400 alerts every day.
If, however, that same team has been tracking their STA and over time have improved their analytics such that for the same 1,000 servers they have only 10 alerts per day then they will have achieved a ratio of 100:1. With an STA of 100, the company can grow to 2,000 servers in Azure and the security team will still have a manageable 20 alerts to review each day.
This simple metric is calculated daily at Snowflake and we track it closely. STA cannot be the only security analytics metric (turning off your alerting does wonders for this ratio) but it helps security teams to scale securely without burning out.
