Popular Post

_

Thursday, June 29, 2017

My CMG'05 papers was cited in PhD Thesis "Finding External Indicators of Load on a Web Server via Analysis of Black-Box Performance Measurements"

Author:
  from  MarkLogic Corporation
Thesis for: PhD, Advisor: Dr. Alva Couch

ABSTRACT:

Traditional methods for system performance analysis have long relied on a mix of queuing theory, detailed system knowledge, intuition, and trial-and-error. These approaches often require construction of incomplete gray-box models that can be costly to build and difficult to scale or generalize. In this thesis, we present a black-box analysis method to discover the amount of load on a web server with minimal knowledge of its internal mechanisms. In contrast to white-box analysis, where a system's internal mechanisms can help to explain its behavior, black-box analysis relies on external measurements of a system's reactions to well-understood inputs. The primary advantages of black-box analysis are its relative independence from specific architectures,its applicability to opaque environments (e.g., closed-source systems), and its scalability. In this thesis, we show that statistical analyses of web server response times can be used to discover which server resources are stressed by particular workloads. We also show that under certain conditions, the settling period of server response times after resource perturbation correlates positively with the degree of perturbation. Finally, we use the two-sample Kolmogorov-Smirnov (KS) test to measure statistical equality of multiple samples drawn from response times of a server under various steady-state load conditions. We show that in specific circumstances, the number of samples that test as statistically equal can serve as an imprecise indicator of the amount of load on a server. All of these contributions will aid performance analysis in new environments such as cloud computing, where internal server mechanisms and configurations change dynamically and structural information is hidden from users.

Finding External Indicators of Load on a Web Server via Analysis of Black-Box Performance Measurements. Available from: https://www.researchgate.net/publication/230707525_Finding_External_Indicators_of_Load_on_a_Web_Server_via_Analysis_of_Black-Box_Performance_Measurements [accessed Jun 29, 2017].

Cited:  CMG'2005  paper:


Friday, June 16, 2017

The #DynamicThreshold is common art by now...

I have got the following feedback on the previous post about some capacity management tool from one of the this blog posts author: 

"As far as any similarity to my own work, I think my methods for dynamic thresholds are common art by now… and most certainly derived from Igor’s own presentations that I attended 😊"

So I am proud of making  some influence!

Tuesday, June 6, 2017

Re-posting #CMGamplify - "#DataScience Tools for Infrastructure Operational Intelligence"

The following  CMG Amplify blog post written by Tim Browning is interesting as it underlines what this "System Management by Exception" blog is always about:

"...In order for the performance analyst to attend to troubled systems that may number in the thousands, it is imperative that we filter out of this vast ocean of time series metrics only those events that are anomalous and/or troubling to operational stability. It is too overwhelming to sit and look at thousands of hourly charts and tables. In addition, there is a need for continuous monitoring capability that detects problems immediately or, better yet, predicts them in the near term.  Increasingly, we need self-managing systems that learn and adapt to complex continuous activities and quickly identify the causal reconstruction of threatening conditions as well as recommend solutions (or even automatically deploy remediation events).  Out of necessity, this is where we are heading..."

and in order to  achieve that:

"..In data mining, anomaly detection (also known as outlier detection) is the search for data items in a dataset which do not conform to an expected pattern. Anomalies are also referred to as outliers, change, deviation, surprise, aberrant, peculiarity, intrusion, etc. Most performance problems are anomalies. Probably the most successful techniques (so far) would be Multivariate Adaptive Statistical Filtering (MASF) for detecting statistically extreme conditions and Relative Entropy Monitoring for detecting unusual changes in patterns of activity..."

See entire post here: https://www.cmg.org/2017/06/data-science-tools-infrastructure-operational-intelligence/