Your organization implemented an anomaly-based Intrusion Detection System (IDS) designed to optimize false-negative probability and minimize potential losses. However, the IDS triggered too many false alarms. As the IDS administrator, you are evaluating the configuration change to reduce the number of false alarms. Which of the following is the least likely ramification of the change? (Wentz QOTD)
A. Zero-day exploits might increase
B. Investigation workload might decrease
C. The sensitivity of the IDS might be lowered
D. The detection threshold of IDS might be enlarged

Kindly be reminded that the suggested answer is for your reference only. It doesn’t matter whether you have the right or wrong answer. What really matters is your reasoning process and justifications.

My suggested answer is D. The detection threshold of IDS might be enlarged.

Rakhi shared a question about IDS threshold with me, and it motivated me to think more about how the sensitivity and threshold of an IDS works. I wrote this question as research to learn more about IDS and machine learning. Your feedback is warmly welcome.

Generally, all four options make sense. However, thresholds are implementation-specific. Some algorithms may not employ a threshold or may use one or more thresholds in different ways. Sensitivity in a confusion matrix is a common way to evaluate the performance of an IDS.

• Once the IDS sends an alert, it should be investigated and validated, and the workload accrues. Decreasing the number of false alarms reduces the investigation workload.
• However, decreasing false-positive alarms may increase the false-negative cases, leading to more zero-day exploits, attacks exploiting vulnerabilities unknown to the organization or stakeholders.
• “Generally speaking, increasing the sensitivity of an intrusion-detection system results in a higher false positive rate, while decreasing the sensitivity lowers the false positive rate.” (Chapple) Lowering the “sensitivity” here may refer to configuring the IDS to detect less aggressively and responsively and send fewer alerts. However, what exactly is the sensitivity of an IDS system? The operational definition of sensitivity in the binary classification is TP / (TP + FN). Lowering sensitivity means FN (False Negative) increases.
• Option D makes sense in a binary classifier as the following diagram shows. The detection threshold of IDS is a parameter of implementation that affects sensitivity, while implementations vary. An anomaly-based IDS may employ a binary or multi-label classifier/algorithm to classify events, e.g., {Attack, Non-attack}, or { Normal, Suspicious, Attack }. As there are various types of anomaly-based IDS, thresholds may or may not be used in every algorithm. Moreover, raising the threshold in different algorithms may affect sensitivity in different ways.

S-IDS and A-IDS

Intrusion detection systems can be divided into signature-based (S-IDS) and anomaly-based (A-IDS) in terms of the detection method. S-IDS works fine in detecting known attacks, while A-IDS is good at unknown attacks. Anomaly, aka outlier, novelty, noise, deviation, and exception, refers to “something that deviates from what is standard, normal, or expected.” (Google)

The following diagram summarizes anomaly detection approaches quite well. However, please be noticed that it has a typo (signature IDS should be “S-IDS”), and most of the CISSP study guides treat Knowledge-based IDS (K-IDS) the same as Signature IDS (S-IDS).

Classification

An IDS may employ a model (classifier) to determine an event or traffic if it’s an attack using a binary classifier or classify it, e.g., Normal, Suspicious, or Attack, using a multi-label classifier. Thresholds as model parameters may be used in the classification process.

The decision for converting a predicted probability or scoring into a class label is governed by a parameter referred to as the “decision threshold,” “discrimination threshold,” or simply the “threshold.” The default value for the threshold is 0.5 for normalized predicted probabilities or scores in the range between 0 or 1. (Brownlee)

ROC Curve

An ROC curve (receiver operating characteristic curve) is a graph showing the performance of a classification model at all classification thresholds. This curve plots two parameters: True Positive Rate and False Positive Rate. (Google)

Confusion Matrix

Confusion Matrix is a tool to evaluate the performance of the model used by an IDS. It contains information about actual and predicted classifications using binary or multi-label classifiers, e.g., {Attack, Non-attack}, or { Normal, Suspicious, Attack }.

Reference

• Precision and recall
• Sensitivity and specificity
• Precision-Recall
• Optimal thresholds for intrusion detection systems
• Intrusion Detection Systems – IDS performance tuning (YouTube)
• Network intrusion detection system: A systematic study of machine learning and deep learning approaches
• Intrusion detection model using machine learning algorithm on Big Data environment
• Classification Algorithms in Machine Learning: How They Work
• 7 Types of Classification Algorithms
• Statistical Anomaly Detection
• Anomaly detection
• Sensitivity, Specificity, Accuracy, Associated Confidence Interval and ROC Analysis with Practical SAS® Implementations
• Evaluating and tuning an intrusion-detection system
• What Is an Intrusion Detection System? Latest Types and Tools
• Tour of Evaluation Metrics for Imbalanced Classification
• A Gentle Introduction to Threshold-Moving for Imbalanced Classification
• A Study on Threshold Selection for Multi-label Classification
• Classification: Thresholding (Google)
• Multi-label classification
• Multiclass Classification (Amazon)
• Binary Classification (Amazon)
• Fine tuning a classifier in scikit-learn
• Machine Learning: Neural Network

A BLUEPRINT FOR YOUR SUCCESS IN CISSP

My new book, The Effective CISSP: Security and Risk Management, helps CISSP aspirants build a solid conceptual security model. It is not only a tutorial for information security but also a study guide for the CISSP exam and an informative reference for security professionals.

• It is available on Amazon.
• Readers from countries or regions not supported by Amazon can get your copy from the author’s web site.

您的組織實施了基於異常(anomaly-based)的入侵偵測系統（IDS），該系統旨在優化假陰性(false-negative)概率並最大程度地減少潛在損失。 但是，IDS觸發了太多的錯誤警報(false alarm)。 作為IDS管理員，您正在評估配置更改，以減少錯誤警報的數量。 以下哪項最不可能是該變更所帶來的影響？(Wentz QOTD)
A. 零日利用(zero-day exploits)可能會增加
B. 調查工作量可能會減少
C. IDS的敏感性可能會降低
D. IDS的檢測閾值可能會加大