Making a specific pattern in a data that do not follow any pattern is quite difficult for the customers who work on real-time data. The new feature of Azure Stream Analytics, machine-based anomaly detection offers the best solution to this problem which allows the customer to monitor the streaming data without using expensive custom machine learning models. Everyone can’t use custom machine learning models as it requires adequate knowledge of the use case and problem domain.
How Does the Monitoring become Easy?
The complex models of machine learning didn’t allow the anomaly detection in the streamlining irrespective of associated value for many industrial IoT sites. The major barrier has solved with the help of machine-based anomaly detection as you can use service monitoring by tracking KPI over time. Apart from the service monitoring, the feature has made the checking of usage monitoring possible with the help of different metrics which can be numbers or searches. Performance monitoring is enabled through counters like file read and CPU. Numerical series data is the main focus of this new feature that detects positive and negative trends while the changes are detected in the dynamic range of values. These trends can be used to generate alerts like security alerts in the login failure.
The Capability of Machine Learning
in an Azure Stream Analytics query, finding out an anomaly in the input data is effortless in a function call due to the presence of general purpose machine learning model. Powerful machine learning detectors easily track the changes in the given values and generate a report as anomaly scores. It never requires the assistance of ad-hoc threshold tuning and function calls directly return anomaly scores for each point in time.
How to Enable Anomaly Detection with Declarative SQL
The detection of anomalies is possible with the help of declarative SQL and the below-mentioned example depicts how SQL declaratives detect anomalies over one hour of the time series.
select id, val, ANOMALYDETECTION(val) OVER(LIMIT DURATION(hour, 1)) FROM input
Usage with portioning
select id, val, ANOMALYDETECTION(val) OVER(PARTITION BY id LIMIT DURATION(hour, 1)) FROM input
Usage with partitioning and “when”
select id, val, ANOMALYDETECTION(val) OVER(PARTITION BY id LIMIT DURATION(hour, 1) WHEN id != 2) FROM input
Usage showing the extraction of scores:
select id, val FROM input WHERE (GetRecordPropertyValue(ANOMALYDETECTION(val) OVER(LIMIT DURATION(hour, 1)), ‘BiLevelChangeScore’)) < -1.0 Three score fields are exposed: BiLevelChangeScore, SlowPosTrendScore, SlowNegTrendScore
