Popular Post


Tuesday, December 11, 2018

CMG IMPACT conference discount #cmgnews

All -

I have promotional code for you in case you want to attend. Just respond and I'll give you a good discount!

Thank you!

Igor Trubin 


Wednesday, November 21, 2018

Great Customer Feedback about the #CapacityManagement Service I Provide

I have just got an internal feedback which has impressed me so I have decided to share that here as a good indication of making difference by Capacity Management service I provide to my customers:

..."Igor has helped us on a couple of instances, prevent potential disasters within our platform by the historical trends he’s tracking.  We know this since there was one occasion where our server crashed when we didn't react fast enough to Igor’s email warnings..."

Wednesday, November 14, 2018

AIXCHANGE (Virtual #CMGnews Event about #AI) November 27 @ 10:00 am - 3:00 pm EST

Attend this virtual conference all about artificial intelligence!
Conference rooms and office stand-ups are all buzzing. Artificial intelligence (AI) is emerging to represent one of the largest technological shifts across countless industries in recent history.
From driving manufacturing to supporting marketing to improving customer retention to improving ITops, AI is become the most in-demand technology of IT execs and quickly rising to the top of the list for IT investment in 2019.
According to recent reports, the machine learning market alone is anticipated to grow from $1.4B in 2017 to $8.8B by 2022.
CMG wants to help you to navigate this technological shift and keep you the smartest team member at the table. On November 27th, join CMG and its partners for AIXCHANGE. The latest of CMG’s virtual conference program will feature live presentations from companies and individuals leading in the AI space.
Scheduled Sessions
  • 10:00 AM - The History and Future of AI with Bryan Krouse
  • 11:00 AM - The Machines are Talking - The Future of AI and Chat for the Enterprise with Stephen Mallik
  • 12:00 PM - A Practical Guide for Information Discovery Using Machine Learning and Visualization
  • 1:00 PM - To Be Announced!
  • 2:00 PM - Soon AI will Test Everything with Jason Arbon

Tuesday, November 13, 2018

"Implementation and Interpretation of Control Charts in R" - qcc package

I was pointed to the nice resent on-line publication: https://datascienceplus.com/implementation-and-interpretation-of-control-charts-in-r/ 

"...Control charts are used during the Control phase of DMAIC methodology. Control charts, also known as Shewhart charts or process-behavior charts, are a statistical process control tool used to determine if a manufacturing or business process is in a state of control. If analysis of the control chart indicates that the process is currently under control, then no corrections or changes to process control parameters are needed. Moreover, data from the method can be used to predict the future performance of the process. If the control chart indicates that the process is not in control, analysis of the chart can help determine the sources of variation, as this will result in degradation of process performance..."

My comment:
When I had been developing SEDS (Performance Anomaly Detection System) long ago (years ago) I looked at that package (and referenced the link to my early CMG papers, BTW) even not knowing how to write R programs (now I can!)... They might improved that, but I did not find that time the way to do MASF type of control charts. I have even dreamed to build SETDS charts (IT-Control Charts)  package on a open source way. So my approach is the same but different, please read details in my paper:

Friday, October 19, 2018

#CMGnews: The training course Performance Anomaly Detection is now free for CMG Members. Visit the Membership Benefits Page for Access.

Be a CMG member and get the following benefits:
  • Access to a repository of more than over 1,000 research papers, presentations, and white papers from industry experts for the information you need for delivering improved results.
  • Free 1 hour webinars featuring industry experts and technology demos and recorded webinar access.
  • Free newsletters that feature the latest informative CMG news.
  • Quarterly issues of the CMG Journal
  • Discounted registration for CMG International-hosted events.
  • Access to CMG imPACt conference presentations and papers.
  • Access to our members-only videos channel with event videos, interviews and more.
  • Discounted registration for CMG’s international annual imPACt conference bringing together the best in the industry for four days of research, training, workshops, networking and practice sharing.
  • Access to our international Member Directory.
  • Access to CMG's Slack channel and LinkedIn to take part in performance and capacity discussions, hot topics, evaluations, information, networking, stay up to date with company news, and more!

Wednesday, October 17, 2018

#CMGnews: Registration is opened for #IMPACT2019 conference! Check discounts!

  • IMPACT 2019 will be an action-packed, 3-day conference filled with information and collaboration. Register today for #IMPACT2019 and take advantage of $100 off your conference pass with code FOB2019:  https://cmgimpact.com/ #cmgnews 

  • Act today and take $100 off your registration fee with code “FOB2019”. To find out more about content, sessions, and activities and what makes CMG’s IMPACT the best #technology conference on the planet, click here:  https://cmgimpact.com/ #cmgnews #IMPACT2019

  • Join hundreds of #industryleaders for CMG's 44th International Conference! #IMPACT2019 promises to be an exciting conference with great learning and networking opportunities. Discount code (ASK ME!) available to save $100 off a conference pass. Act today and save!  https://cmgimpact.com/ #cmgnews 

  • IMPACT 2019 #Conference sessions will educate and enlighten, enabling attendees to take a #leadership role in their own companies’ #digitaltransformations. Register today for #IMPACT2019 and take advantage of $100 off your conference pass with “FOB2019”: https://cmgimpact.com/ #cmgnews 

Tuesday, October 16, 2018

www.CMG.org Announces Launch Of Dynamic New Brand And Communications Platform

Computer Management Group (www.CMG.org), one of the worlds most influential organizations of IT professionals committed to digital transformation initiatives and best practices, is delighted to announce the launch of its new brand and communications platform—a platform built to showcase its measurement and management of computer systems and networks from a performance and capacity ... Read More »

Wednesday, October 10, 2018

#Opmantek product has implemented my SEDS method (MASF based #AnomalyDetection) and referenced my work positively

After I have published my previous post

#Opmantek product has implemented my Statistical Exception Detection System (SEDS) - MASF based #AnomalyDetection

Opmantek CTO has reached me out, confirmed they used my methodology and provided a very positive feedback:

"I am glad you contacted Opmantek about this blog, and we have updated it to include your name and a reference to one of the key blog articles.

I have been meaning to reach out to you to let you know about what we had done, technically the product is still in Beta but it seems the marketing team have pushed forward with making it generally available.

We found your work of great value as we looked through various methodologies for trending, in the end we implemented something based on SEDS with a few changes/additions, to be honest I would have to review the code to see what the differences are.

At the moment the product opTrend is working well enough, but we need to make some refinements and enhancements before it will be the first release.

Your research and publications are of great value and highly appreciated."


Friday, October 5, 2018

"#CapitalOne on #AWS" dedicated landing page

Thursday, October 4, 2018

#Opmantek product has implemented my Statistical Exception Detection System (SEDS) - MASF based #AnomalyDetection

Looks like one of the Opmantek product has implemented my dynamic thresholding method - SEDS. I found the reference to that in the following blog post at their site: 

System Automation Through Integration

"...These solutions were then complimented by the addition of opTrend, which expands on Opmantek’s already expansive thresholding and alerting system by implementing a highly flexible Statistical Exception Detection System (SEDS) that learns what’s normal behavior on the client’s network and adjusts thresholding dynamically based on historical usage for every hour of each day of the week..."

The description is limited, but apparently it is my SEDS method (MASF based Anomaly Detection) published in several white papers and blog posts.

I am happy except there is no reference to my name, papers or at least this blog. 

Thursday, September 27, 2018

#CMGnews: My talk, "Catching #Anomaly and Normality in Cloud by #NeuralNet and Entropy Calculation", has been selected for #CMGimpact 2019


Part 1. The Neural Network (NN) is not a new machine learning method. About 12 years ago I was involved as a Capacity Planning resource for the project of building an infrastructure (servers) to run NN for the fraud detection application. Now NN got much more attention and popularity as a part of AI, mostly because the computing power is increased dramatically and respectively more tasks can be done by using NN.
The goal of the presentation is to demystify the technique in some simple terms and examples to show what it actually is and how that could be used for Capacity and Demand management. That is done by developing R code to recognize typical workload pasterns, like OLTP, or others in the time series performance data daily profiles.
Part 2. It is the typical concern to detect anomalies for short living objects or for the object with very small amount of measurements. Why? Number of those objects could be thousands and thousands so it is important to separate exceptional ones with anomalies for further investigation. That could be servers or customers that have just started being monitored or public cloud objects (EC2s, ASGs) that usually have very short lifespan. Suggested approach to detect anomalous behavior of this type of objects is to estimate the Entropy of the each object. If the entropy is low, everything should be in order and most likely OK. If not - there is a possible disorder there or mess and someone needs to check what is going on with the object. The method is implemented in the cloud based application written on R that scans every hour all cloud Auto Scaling Groups (ASG) to detect imbalanced ones in term of number of EC2 instances in the group. That allows to separate a couple hundreds ASGs out of hundreds thousands of them.
This entropy based method is well known and it described in details in the post: “Quantifying Imbalance in Computer Systems” which is written based on CMG’12 paper.

Tuesday, August 28, 2018

Catching Anomaly and Normality in Cloud by Neural Net and Entropy Calculation - #CMGnews

- I have submitted my next CMG (https://cmgimpact.com/) presentation and waiting for the acceptance. BTW I used some past CMG and this blog posts as a material for my presentation.

E.G. You may check it out using below links:

  Quantifying Imbalance in Computer Systems

UPDATE: The presentation is accepted. See abstract HERE.

Friday, July 20, 2018

#CMGnews - You can now register to #imPACt Conference in Seattle FEB19-21 2018

Thursday, June 21, 2018

Performance #AnomalyDetection on-line class - my presentation at #cloudXchange virtual conference

Tuesday, June 19, 2018

"The Ride of #AnomalyDetection" -#cloudXchange2018 video is here going now

Self-Driven IT Ops. starts with Anomaly Detection! Check the #cloudXchange presentation about that made by CA SME

Monday, June 18, 2018

#CMGnews: #CLOUDXCHANGE virtual conference is tomorrow! Check the agenda here

CMG will be streaming live sessions from 7:00 AM to 7:00 PM EDT at www.cmg.org/cloudxchange. Check out our confirmed speakers below and share your participation with #cloudXchange2018.

·     7:00 AM EDT - The Economics of Cloud Computing - Owen Rogers, 451 Research

·     8:00 AM EDT - An Introduction to Open Source Application Performance Monitoring (APM) - Dr. Andreas Brunnert, RETIT GmbH

·     9:00 AM EDT- Business Process with Cloud Native Computing: Overcoming Challenges of Traceability in Micro Services-based Architecture - Yuri Shkuro, Uber Technologies

·     10:00 AM EDT - Customer Experience and the Journey to Digital Experience Optimization - Dan Boutin, Blue Triangle

·     10:30 AM EDT: Utilizing software defined parallel IO to enable secured, cloud work-spaces  - Andrew Caldwell, R3SOLV

·     11:00 AM EDT - Let’s Get Real About Self-Driven IT Ops - Jeff Henry, CA Technologies

·     12:00 PM EDT - Performance Unit Testing - Why and When They Work - Erik Squires

·     12:30 PM EDT - The World of Private and Hybrid Cloud - Anthony Maiello, Tier4

·     1:00 PM EDT - Exposing the Cost of Performance Hidden in the Cloud - Neil J. Gunther, Performance Dynamics

·     1:30 PM EDT - Continuous Load Testing Using Real Browsers from the Cloud - Tim Koopmans, TIRCENTIS, FLOOD IO

·     2:00 PM EDT - Measuring Client Performance & Inventing Teleportation - Edward Hunter, Netflix

·     3:00 PM EDT - How Linux Containers Work - Sasha Goldshtein, SELA Group

·     3:30 PM EDT - Business By The Numbers: Embedded Analytics for Devices, Edges and Servers - Alan Clark, Telchemy

·     4:00 PM EDT - The Machines are Talking: The Ride of Anomaly Detection - Shanti Subramanyam, Orzota, Inc.

·     4:30 PM EDT - Serverless In Production, An Experience Report - Yan Cui, DAZN

·     5:00 PM EDT - The Journey to Cloud Through the Eyes of a Chief Architect - Kim Eckert, IBM Services

·     6:00 PM EDT - Lessons from Automating Cloud Clients Installation and Testing - Wilson Mar, JetBloom

·     6:30 PM EDT - How to Effectively Implement and Manage Blockchain Performance Engineering - Zak Cole, Whiteblock

Friday, June 15, 2018

#CMGnews: The #Compuware expert speaks about #MachineLearning, #DevOps and products (#zAdviser)

That was one of the session during the National Capital Area Computer Management Group (www.CMG.org) meetup meeting "Performance Engineering for/on Mainframe and Cloud" See agenda is here: https://www.meetup.com/NCACMG/events/251111371/

This session was for members and NOVA college students.

Tuesday, June 12, 2018

#CMGnews - I am presenting at #CLOUDXCHANGE #VirtualConference the CMG Professional Development Offerings

That will be an announcement with some explanations of the on-line classes that are honored by CMG experts.

Among others that includes my class:

The short presentation is scheduled on the next Tuesday 6/19 at about 5:30. Please preregister to the virtual conference HERE  and connect to my presentation! 

To enroll go HERE or find CMG school at https://cmg1.teachable.com/

Sunday, June 10, 2018

#CMGnews in #DC- #CloudCapacity Management - fragment of the presentation

Full version is available for www.CMG.org members
NEXT MEETING ON 6/14/18 RSVP - https://www.meetup.com/NCACMG/

Thursday, May 31, 2018

Hey! If you live close to DC - We meet again to discuss Mainframe and Cloud related stuff in our NCA CMG meetup. Check the agenda!

Hey! If you live close to DC - We meet again to discuss Mainframe related stuff in our meetup: 

LINK https://www.meetup.com/NCACMG/events/251111371/

When: Thursday, June 14, 2018
10:00 AM to 3:00 PM
Add to calendar



1. "Preparing Today’s Infrastructure for Tomorrow’s Workloads", Kofi Hayford, DataCore

2. "Flash, Flash and more Flash: What’s in it for me?", John Baker from Data Kinetics

ABSTRACT: Flash. Is this the light on my camera, a plugin for my browser, or a superhero that runs really fast? If you’re in IT, flash likely refers to something else: flash memory of some sort. But even then, the term is both ubiquitous and vague. The solid-state drive in your laptop; the new memory in your z14; your new “all flash” storage array. These all boast “flash” technology that promises to make everything faster.
Flash technology is not new but reductions in price and advancements in capacity are exploding adoption. There are many compelling arguments for flash – but is it the solution to all storage bottlenecks? Take a stroll with John as we slow down the hype to explore the various implementations of flash technology and which, if any, can best speed up your datacenter. And don’t worry; I won’t be wearing a red, spandex suit.

3. "Dave’s not here: R4HA, AWLC, SCRT and other 4-letter software pricing words", John Baker from Data Kinetics

ABSTRACT: David Chase has left the building. Which leaves many of us scratching our heads about the whirlwind of available pricing options. IBM Z continues to improve efficiency and options like Country-Wide Multiplex (CMP), Mobile and Container pricing, and soft capping options, improve ROI but how does all this stuff work? The R4HA remains key but even that requires context. Its measured on each LPAR but often combined depending on what, and where, various subsystems are running. Which value, average, or total determines your software bill? And what if some LPARs are capped? This session is for everyone. Performance, Capacity, or Finance. Costs are driving more and more of our decisions. Lets start by understanding how these costs are determined.

PRESENTER: John Baker has over 25 years in the IT industry as both a customer and consultant. For the last 20 years, John has focused on mainframe performance and cost optimization. As a customer, John designed, implemented and maintained many critical projects such as WLM Goal Mode, GDPS/Data Mirroring, and merging datacenters.

Light LUNCH is provided by Compuware

4. Compuware presentations by Kelly Vogt:

- "The Practical and Wholesale Application of Strobe to Reduce Cost, Improve Scalability and Performance",

ABSTRACT: Tuning is often a reactive activity. A crisis presents and we turn out in force to stamp out the fire; then return to the myriad other things we have to do… But what if we flip that around and make it a proactive activity for fun and profit? See how a common sense approach to proactive treasure hunting in your system can save large amounts of money in your installation and improve service for your customers.

- "The Necessity of, and the Value Proposition for, Total Batch Automation with ThruPut Manager"

ABSTRACT: What is the future of your batch strategy? Is it nimble? Fluid? For all the automation brought to the mainframe, why is it that batch is still manually operated? All that tracking jobs and endless commands fiddling about with initiators and job classes. Learn why we cannot continue on like we are… and what we can do about it.

PRESENTER: Kelly Vogt joined Compuware in February as an Field Tech in support of ThruPut Manager. He has 38 years in the mainframe arena, with 24 years in systems programming and performance management. His last 14 years were spent in management leading Large Systems Engineering for Humana. He has extensive experience buying and renewing IBM and ISV hardware/software; knowledge of software license models and contracts; performance and capacity planning experience; and the day in, day out of data center operation.

5. "Performance Anomaly Detection and more at CMG Training Program" - 
https://www.cmg.org/professional-development/ Igor Trubin, IT Manager at Capital One, CMG director.

Wednesday, May 16, 2018

NCA CMG meetup meeting 12/6/2017 -roll call of participants

Tuesday, May 15, 2018

Got caught on the www.CMG.org official video: "41st International Performance & Capacity Conference"

See me started at 1:05!

Subscribed to the CMG YT channel. You?

Wednesday, April 18, 2018

Performance #AnomalyDetection Online Course Info and #Perfomalist Tool DEMO


What is covered:

  • Machine learning based Anomaly Detection technique
  • Classical (SPC) and MASF (For system performance data) Control Chartirting
  • Where is the Control Chart Used?
  • What are the types of Control Charts?
  • Reading, building, and interpreting Control Charts
  • Typical cases of real world issues captured by anomaly detection system (VMs, Mainframes, Middleware, E2E response and more)
  • How to build free AWS cloud server with R and build there control charts
  • Performance anomaly (Perfomaly) detection system R implementation example (SEDS-lite - open source based tool)

"Perfomaly"=Performance Anomaly:

Control Chart as a Machine Learning Tool to Detect Anomalies

Tuesday, April 17, 2018

Perfomalist is a #ChangeDetection system based on Exception Value (EV) data analysis

2022 update:
Now the same method is used for free online tool - Perfomalist https://www.perfomalist.com/
More detailed has been added to the method and published here "

CPD - Change Point Detection is planed to be implemented in the free web tool Perfomalist


The right term for SETDS methodology is Anomaly and Change (point) detection.

The second part of SETDS - trend detection recently implemented on R and got the "TrendieR" name - is actually a Change Point Detection tool.

See more details how it works in the previous post:

AnomalyDetection vs. NoveltyDetection. SETDS Method Detects and Separates both

But in short the change points are the the following equation roots (solutions)


Where t is time and EV is Exception Value (magnitude of exception calculated as the difference between actual data and baseline statistical limits UCL and LCL). 

CMG.org paper there was the attempt to describe that calculation more formally by the following formulas:

Friday, March 2, 2018

#NemtsovPlaza opening culmination and me arguing with famous Russian journalists...

Wednesday, February 7, 2018

More about #CloudCapacityPlanning from #CMGnews

Visiting #CapitalOneCafe

What is Capacity Management?

What is Capacity Management? [Webinar Recap]: Capacity management is the practice of making sure IT resources meet business demands today and down the road—without over-provisioning. But the role of capacity management has changed as IT environments have evolved.

Tuesday, February 6, 2018

"Machine Learning for Predictive Performance Monitoring" - interesting #CMGJournal article (#CMGnews)

Tim Browning has a lot of good publications about Capacity Management in www.CMG.org  and also in this blog:

"Entropy-Based Anomaly Detection for SAP z/OS Systems"

#CMGamplify - "#DataScience Tools for Infrastructure Operational Intelligence"

"the review of cloud computing article "Optimal Density of Workload Placement"

He has just published his new paper in the CMG Journal:  
            "Machine Learning for Predictive Performance Monitoring",
which is available for CMG members

I have enjoyed reading the paper, below is the abstract:

I like especially his following very true saying: 

"...Machines don’t actually “learn” nor do statistical algorithms represent some mechanistic disembodied intelligence. However, human learning and intelligence is greatly assisted by statistical modeling in much the same way that optics technology assists vision..."

I appreciate he referenced two my CMG papers under his "Useful Related Materials" section:

- Trubin, Igor, “Exception Based Modeling and Forecasting”, CMG2008 Proceedings
- Trubin, Igor, “Capturing Workload Pathology by Statistical Exception Detection System”,
CMG2005 Proceedings.

Friday, February 2, 2018

YouTube playlist "My Work"