Learning from Incidents

It's only lunchtime on the first day of the FIRST Conference 2018, and already two talks have stressed the importance and value of reviewing incidents over both the short and long terms. In the very different contexts of an open science research lab (LBNL) and an online IPR-based business on IPR (Netflix), a common message applies: "don't have the same incident twice".

After you have detected, mitigated and recovered from an incident, make the time to understand the sequence of events and how they were (finally) discovered. If an attack was quickly detected and blocked then that won't take long. But where an attack moves undetected through several phases it may well be useful to adopt what Netflix described as a "purple-team" approach, looking at each stage from both the attacker (red) and defender (blue) perspectives. What options did the attacker have to make progress, and what options might the defenders have used to stop those.

Sometimes this will highlight the need for new defensive tools and techniques. But if you have a long-term record of how past incidents were detected, that may contain existing approaches that could be reused against new types of attack. Such a record can also inform when to retire tools and techniques that aren’t proving useful in detecting or preventing attacks.

And the first talk after lunch added another point. DON’T do this while the incident is in progress. The right time to ask “how could we have stopped this?” is afterwards. So maybe the question should be “how could we have stopped this sooner?”.