Popular Post

Search This Blog

Monday, August 13, 2007

CMG'06: Performance Data Statistical Exceptions Analysis (Review) and my paper there...

2016 UPDATE. My paper from that year CMG conference can be found under the following link now:
SYSTEM MANAGEMENT BY EXCEPTION, PART 6 

I'm going to CMG'07 in San Diego California - December 2nd through 7th, 2007

Here is the list of CMG2006 (http://www.cmg.org/) papers that discussed statistical exception detection technique:

A Priori Evaluation of Data and Selection of Forecasting Model, Alexander Gilgur, Michael Perka MonoSphere, Inc.

LINK: http://www.daschmelzer.com/cmg2006/PDFs/038.pdf
The paper shows how important to capture “OutLaers” to produce meaningful forecasts. To do that they use some SEDS-like algorithm “Detection of Outlier Events”.


• 2006 Best Paper Award paper: Did Something Change? Using Statistical Techniques to Interpret Service and Resource Metrics. Frank M. Bereznay, Kaiser Permanente

LINK: http://cmg.org/conference/cmg2006/awards/6139.pdf
Author has some references to my papers:
“…Statistical techniques are not new to CMG. Starting in the early 1990’s there have been numerous papers addressing this subject, [Brey90], [Chu92], [Lipner92] and [Schwartz93]. This body of work seemed to cumulate with Jeff Buzen and Annie Schum’s 1995 CMG Paper introducing Multivariate Adaptive Statistical Filtering (MASF) as a new statistical technique [Buzen95]. Interest in the subject seemed to decline from that point on, with the notable exception of Igor Trubin’s set of papers on the application of MASF to many measurement and management areas [Trubin01], [Trubin02], [Trubin03], [Trubin04], and [Trubin05]. All of these papers are excellent treatments of the subject and are recommended reading…”
During discussions at this presentation some questions were asked (e.g. Sean Meidhan from BEN, who implemented some MASF ideas in there tool) about “fault positive” (fault alerts) situation sometimes generated out this technique. I had to step up and give some clarifications how SEDS handles that.

(08/2007 UPDATE:
Frank M. Bereznay have recently gave the interview to CMG MeasureIT: MeasureIT - Issue 5.08 - Getting to Know Mullen Award Winner ...
"...I also noticed that the number of papers in this area seemed to be declining since the mid to late 1990s. There were a number of papers leading up to Jeff Buzen and Annie Shum's MASF (multivariate adaptive statistical filtering) paper in 1995, and since then the trend seemed to decline with the exception of Igor Trubin's work, so I wanted to give the statistical methods some additional visibility.."
He also will be presenting at this fall SCMG meetings September 27 in Richmond and September 28 in Raleigh :
Using Statistical Techniques to Interpret Service and Resource Metrics )

ACTIVE BASELINING IN PASSIVE DATA ENVIRONMENTS, Mike Tsykin, Fujitsu Australia, Ltd.

LINK: http://www.fujitsu.com/downloads/AU/active_baselining_in_passive_data_environments.pdf
Author in this paper uses the SPC approach for baselining. I have met with him in the previous CMG conferences discussing my SEDS technique, he picked up SPC idea and implemented that for alerting part of some Fujitsu performance tool.


Dials for a PM Dashboard: Velocity’s Missing Twin, and Quantifying Surprise, Rich Olcott, IBM Information Technology Services

LINK: http://www.daschmelzer.com/cmg2006/PDFs/102.pdf
This paper has also some discussion how SPC should be used. E.g. how Simple Average cold be replaced by EWMA. He has references on my paper and M.Tsykin’s paper.


• My paper: SYSTEM MANAGEMENT BY EXCEPTION, PART 6, Igor Trubin, PhD

LINK: http://www.daschmelzer.com/cmg2006/PDFs/021.pdf
ABSTRACT: Statistical Exception Detection System (SEDS) has been successfully used for more than six years to automatically produce web-based exception reports against the performance data warehouse for a large, multi-platform environment. Adding some application specific metrics including middleware traffic and response times made SEDS an excellent tool for application performance management. This paper also describes how to create statistical control charts using a spreadsheet in order to capture a performance issue without using expensive tools with built-in SPC procedure.


_________________________________________________________________
CONCLUSION: There is still a big interest to MASF, SPC and SixSigma methods applied to system performance data. This year CMG'07 conference has already announced some papers related to this subject as well, including my next paper:

System Management by Exception, Part Final, Dr. Igor A. Trubin, IBM; Ray White, IBM

LINK: http://cmg.org/cgi-bin/agenda_2007.pl?start=&perpage=&action=search&token=Trubin&listBy=&sortBy=Author_Last,%20Author_First&print=yes&grid
ABSTRACT: Statistical Exception Detection System (SEDS) has been successfully used for more than seven years to automatically produce web-based exception reports and smart alerts against the performance data warehouse for a large, multi-platform environment. This paper starts with an overview of how SEDS uses SPC and MASF techniques and how SEDS could be used as a part of Lean/Six Sigma. Then it focuses on the memory usage exceptions that SEDS captures to proactively identify server and application performance issues.
(08/2007 UPDATE: This paper is presented also September 27 in Richmond SCMG: meting: System Management by Exception, Part Final)
I'm going to CMG'07 in San Diego California - December 2nd through 7th, 2007

Thursday, June 7, 2007

System Management by Exception

Greetings!

To keep the discussion about how to Manage computer Systems by Exception (e.g. by  using SPC, APC, MASF, 6-SIGMA, SETDS and other techniques), I run this blog and also publish/present white papers at the www.CMG.org.  Please take a look at the following set of CMG papers related to Statistical Exception Detection System (SEDS or SETDS):

2017 -  The Model Factory - Correlating Server and Database Utilization with Customer Activity"

2016 - Is your Capacity available? 

2012 - SEDS-Lite:  Using Open Source Tools (R, BIRT and MySQL) to Report and Analyze Performance Data 

2008 Exception Based Modeling and Forecasting

2005 - Capturing Workload Pathology by Statistical Exception Detection System 

2004 - Mainframe Global and Workload Level Statistical Exception Detection System Based on MASF

2003 - Disk Subsystem Capacity Management Based on Business Drivers I/O Performance Metrics and MASF

2002 - Global and Application Levels Exception Detection System, Based on MASF Technique

2001 - Exception Detection System, Based on the Statistical Process Control Concept