SIEM Next-Genness (2.0)

I previously wrote about SIEM next-genness, but I mixed and matched between features and architectures. It’s a problem because a product can have a lot of features but be a sluggish clunky product that’s a nightmare to manage, or it can be super well architected but there’s not much you can do with it.

So I now give you a matrix of capabilities in rough order of next-genness, from 😱 legacy to 😵‍💫 too-next gen.

As the big asterisk notes, take this as an illustration rather than an assessment. If you want proper peer-reviewed and fact-checked assessments, please read my research.

Achitecturally

This is a list in order of how SIEM products are deployed and designed to store and process data.

• Hardware deployment - Box in closet

• 𝐇𝐨𝐫𝐢𝐳𝐨𝐧𝐭𝐚𝐥 𝐬𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲 - More boxes in closet

• Virtualized deployment - a virtual machine that can either run on a box in your closet, or as a box in somebody else’s closet (like AWS’s). If you have unused boxes in your closet, you can deploy additional virtual machines to scale your SIEM.

• Containerized deployment - a container that can either run on a box in your closet that also has an operating system, or in somebody else’s box in a closet that has an operating system. Cool thing is that you can manage the containers rather flexibly with the Greek word for pilot.

• 𝐂𝐥𝐨𝐮𝐝 𝐧𝐚𝐭𝐢𝐯𝐞 - these solutions are built on top of hyperscaler infrastructures and may even be available in cloud marketplaces. Cloud-native products can either be deployed in your own hyperscaler environment or consumed as SaaS, where the vendor managed the environment and you just give your web UI to your security analysts.

• 𝐌𝐢𝐜𝐫𝐨𝐬𝐞𝐫𝐯𝐢𝐜𝐞𝐬 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 - with the risk of sparking the monolith vs microservices conundrum, this indicates “newer” SIEMs that are deployed in containers, use K8s for scaling, and run components independently of each other to help mitigate knock-on effects. Most (if not all) microservices-based SIEMs are consumed as-a-service, because otherwise the customer needs to inherit or the complexity of managing the monstrosity.

• Native data pipeline management - This refers to applying filtering, normalization, routing, and correlation of data prior to ingestion/storage. It’s a way to reduce both costs and volumes. There’s a whole market on data pipelines for SIEMs (look at Cribl and co). Some SIEMs are starting to develop these features natively

• Decoupled/Distributed SIEM - This refers to having data management and the threat analysis separate. For example, you can use a security data lake to store your logs at an arguably cheaper price, and the “analysis” module pulls data as it is queried. I link here Anton Chuvakin’s blog post as I agree with his take and I only reference things I agree with.

Featurally

This is list in order of newer and supposedly more useful features to detect and respond to threats.

• Machine Leaning - some good old pre-GenAI statistical analysis to identify deviations from the baseline.

• Deterministic 𝐑𝐞𝐬𝐩𝐨𝐧𝐬𝐞 𝐚𝐧𝐝 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐢𝐨𝐧 - workflows or scripts that are usually the result of a SOAR acquisition or rarely by in-house developments. This boils down to doing a bunch of API work using workflow logic. But mind you, good workflow engines are very comprehensive and most of these natively available in SIEMs are not that advanced.

• 𝐃𝐞𝐭𝐞𝐜𝐭𝐢𝐨𝐧-𝐚𝐬-𝐜𝐨𝐝𝐞 - allowing your engineers to write their detection rules in YAML and using SIGMA rules.

• 𝐒𝐞𝐥𝐟-𝐭𝐮𝐧𝐢𝐧𝐠 𝐝𝐞𝐭𝐞𝐜𝐭𝐢𝐨𝐧 - your engineers may be too busy writing their detection rules in YAML, so the tool can optimize these rules by itself based on investigation results and flagged false positives

• Copilots - using LLMs to provide a natural language interface for the product. Analyst says what they want, and the copilot can spew out an SQL query or navigate the product to perform some actions. I think copilots are on the low end of LLM productivity gains.

• Self-writing detection - using LLMs to write detection rules. Particularly suitable if the product supports detection-as-code.

• 𝐀𝐈-based response and automation - architecting LLMs with memory, retrieval, and guardrails to investigate threats, propose remediation, carrying them out, and perhaps do all that autonomously.