Popular Post

Wednesday, October 10, 2018

#Opmantek product has implemented my SEDS method (MASF based #AnomalyDetection) and referenced my work positively

After I have published my previous post

#Opmantek product has implemented my Statistical Exception Detection System (SEDS) - MASF based #AnomalyDetection

Opmantek CTO has reached me out, confirmed they used my methodology and provided a very positive feedback:

"I am glad you contacted Opmantek about this blog, and we have updated it to include your name and a reference to one of the key blog articles.

I have been meaning to reach out to you to let you know about what we had done, technically the product is still in Beta but it seems the marketing team have pushed forward with making it generally available.

We found your work of great value as we looked through various methodologies for trending, in the end we implemented something based on SEDS with a few changes/additions, to be honest I would have to review the code to see what the differences are.

At the moment the product opTrend is working well enough, but we need to make some refinements and enhancements before it will be the first release.

Your research and publications are of great value and highly appreciated."


Thursday, October 4, 2018

#Opmantek product has implemented my Statistical Exception Detection System (SEDS) - MASF based #AnomalyDetection

Looks like one of the Opmantek product has implemented my dynamic thresholding method - SEDS. I found the reference to that in the following blog post at their site: 

System Automation Through Integration

"...These solutions were then complimented by the addition of opTrend, which expands on Opmantek’s already expansive thresholding and alerting system by implementing a highly flexible Statistical Exception Detection System (SEDS) that learns what’s normal behavior on the client’s network and adjusts thresholding dynamically based on historical usage for every hour of each day of the week..."

The description is limited, but apparently it is my SEDS method (MASF based Anomaly Detection) published in several white papers and blog posts.

I am happy except there is no reference to my name, papers or at least this blog. 

Thursday, September 27, 2018

#CMGnews: My talk, "Catching #Anomaly and Normality in Cloud by #NeuralNet and Entropy Calculation", has been selected for #CMGimpact 2019


Part 1. The Neural Network (NN) is not a new machine learning method. About 12 years ago I was involved as a Capacity Planning resource for the project of building an infrastructure (servers) to run NN for the fraud detection application. Now NN got much more attention and popularity as a part of AI, mostly because the computing power is increased dramatically and respectively more tasks can be done by using NN.
The goal of the presentation is to demystify the technique in some simple terms and examples to show what it actually is and how that could be used for Capacity and Demand management. That is done by developing R code to recognize typical workload pasterns, like OLTP, or others in the time series performance data daily profiles.
Part 2. It is the typical concern to detect anomalies for short living objects or for the object with very small amount of measurements. Why? Number of those objects could be thousands and thousands so it is important to separate exceptional ones with anomalies for further investigation. That could be servers or customers that have just started being monitored or public cloud objects (EC2s, ASGs) that usually have very short lifespan. Suggested approach to detect anomalous behavior of this type of objects is to estimate the Entropy of the each object. If the entropy is low, everything should be in order and most likely OK. If not - there is a possible disorder there or mess and someone needs to check what is going on with the object. The method is implemented in the cloud based application written on R that scans every hour all cloud Auto Scaling Groups (ASG) to detect imbalanced ones in term of number of EC2 instances in the group. That allows to separate a couple hundreds ASGs out of hundreds thousands of them.
This entropy based method is well known and it described in details in the post: “Quantifying Imbalance in Computer Systems” which is written based on CMG’12 paper.

Tuesday, August 28, 2018

Catching Anomaly and Normality in Cloud by Neural Net and Entropy Calculation - #CMGnews

- I have submitted my next CMG (https://cmgimpact.com/) presentation and waiting for the acceptance. BTW I used some past CMG and this blog posts as a material for my presentation.

E.G. You may check it out using below links:

  Quantifying Imbalance in Computer Systems

UPDATE: The presentation is accepted. See abstract HERE.

Monday, June 18, 2018

#CMGnews: #CLOUDXCHANGE virtual conference is tomorrow! Check the agenda here

CMG will be streaming live sessions from 7:00 AM to 7:00 PM EDT at www.cmg.org/cloudxchange. Check out our confirmed speakers below and share your participation with #cloudXchange2018.

·     7:00 AM EDT - The Economics of Cloud Computing - Owen Rogers, 451 Research

·     8:00 AM EDT - An Introduction to Open Source Application Performance Monitoring (APM) - Dr. Andreas Brunnert, RETIT GmbH

·     9:00 AM EDT- Business Process with Cloud Native Computing: Overcoming Challenges of Traceability in Micro Services-based Architecture - Yuri Shkuro, Uber Technologies

·     10:00 AM EDT - Customer Experience and the Journey to Digital Experience Optimization - Dan Boutin, Blue Triangle

·     10:30 AM EDT: Utilizing software defined parallel IO to enable secured, cloud work-spaces  - Andrew Caldwell, R3SOLV

·     11:00 AM EDT - Let’s Get Real About Self-Driven IT Ops - Jeff Henry, CA Technologies

·     12:00 PM EDT - Performance Unit Testing - Why and When They Work - Erik Squires

·     12:30 PM EDT - The World of Private and Hybrid Cloud - Anthony Maiello, Tier4

·     1:00 PM EDT - Exposing the Cost of Performance Hidden in the Cloud - Neil J. Gunther, Performance Dynamics

·     1:30 PM EDT - Continuous Load Testing Using Real Browsers from the Cloud - Tim Koopmans, TIRCENTIS, FLOOD IO

·     2:00 PM EDT - Measuring Client Performance & Inventing Teleportation - Edward Hunter, Netflix

·     3:00 PM EDT - How Linux Containers Work - Sasha Goldshtein, SELA Group

·     3:30 PM EDT - Business By The Numbers: Embedded Analytics for Devices, Edges and Servers - Alan Clark, Telchemy

·     4:00 PM EDT - The Machines are Talking: The Ride of Anomaly Detection - Shanti Subramanyam, Orzota, Inc.

·     4:30 PM EDT - Serverless In Production, An Experience Report - Yan Cui, DAZN

·     5:00 PM EDT - The Journey to Cloud Through the Eyes of a Chief Architect - Kim Eckert, IBM Services

·     6:00 PM EDT - Lessons from Automating Cloud Clients Installation and Testing - Wilson Mar, JetBloom

·     6:30 PM EDT - How to Effectively Implement and Manage Blockchain Performance Engineering - Zak Cole, Whiteblock