Discovery: IT'S not Darts and Dart Boards
“The unseen enemy is always the most fearsome.”
― George R.R. Martin, A Clash of Kings
I never thought I would be quoting George R.R. Martin, but I also never thought I would be writing a blog, so here we are.
Discovery isn’t for everyone. Discovery done right can help reshape how an organisation manages its business through finding anomalies with their data that has so far remained unseen by its controls.
What indicators do you use to find the unseen? Or how do you baseline the weird?
To baseline the “weird” is like trying to understand how long a piece of string is. Or throwing a dart at a dart board and wherever you hit, that’s your anomaly.
If you “know” an indicator, that is looking for knowns which means it’s no longer Discovery. Messes with your brain a little bit? Welcome to my everyday!
How do you find anomalies or weird?
Imagine you are travelling with a giant suitcase. Is it easier for you to find something that does not belong in your suitcase if you had proper organisation vs having everything just thrown in?
For me, it is about understanding what is normal and how things are supposed to work.
Then, if anything does not fall within the pattern of “normal,” I go and understand why.
What tools do we use for this?
A super magic tool that costs you a gazillion dollars in magic beans. HA!
No, let’s start with Excel, or Power BI. Any tool that is capable of some basic visualisation and can do basic maths like max/min/averages. Maths is my worst subject ever; if I can get meaning out of this data, you can too.
How should we do this and where do we start?
You’re gonna need some log data. Proxy logs, Active Directory logs are good candidates.
You can start with a single log source and build on top of that. I’m a huge advocate of walking before running so as to avoid face planting, big time.
Here are some initial questions you can ask of your data:
What are my top 10 most active devices or user accounts?
What functions do these top 10 perform?
Is this what I would expect?
If anything is unexpected, who is this, what could have caused this surge or drop in the activity?
What business context can I gather that will help me determine whether this activity is:
Normal - accepted business process
Risky - done by well meaning an employee but introducing a risk to the organisation.
Malicious - activity done by individuals with the intent to cause harm to the organisation.
Is this activity seen by existing controls. If yes, was it investigated? If no, should this type of activity be detected as a part of organisation’s ongoing monitoring process?
If all my top 10 noisy accounts are normal and accepted business process, what processes are they running? How critical are they? Do existing controls ensure any changes to their activities are looked at by analysts?
It is a good idea to begin with some in the moment analyses to understand the current state. You may also wish to expand your time frames and see how this activity changes depending on if you were looking at data across a day, a week, a month or several months.
Discovery work is a process, not magic silver bullet. When we are talking about large, and noisy environments, it is important to first manage the noise to reveal the unseen.
Before the next post, have a go at grabbing some log data, import it into Excel or similar and ask the above series of questions.
In the next post I will take you through another Discovery question, steps I take to narrow down large datasets and how I distill knowledge from the data.
If you have a question you would like me to go through, drop me a line via dm on LinkedIn or Twitter.
I am still working on a contact me form so you can all subscribe and get in contact directly through this site.
Until next time!
Skye
Disclaimer: All statements and comments are my own. They do not reflect the views of any past or present employers.