"Change Point Detection (#ChangeDetection) for MongoDB Time Series Performance Regression" paper for ACM/SPEC ICPE 2022 Data Challenge Track

The ACM/SPEC ICPE 2022 - Data Challenge Track Committee has decided to ACCEPT our article:

TITLE: Change Point Detection for MongoDB Time Series Performance Regression
AUTHORS: Md Shahriar Iqbal, Mark Leznik, Igor Trubin, Arne Lochner, Pooyan Jamshidi and André Bauer

https://icpe2022.spec.org/tracks-and-submissions/data-challenge-track/

ABSTRACT

Commits to the MongoDB software repository trigger a collection

of automatically run tests. Here, the identification of commits 

responsible for performance regressions is paramount. Previously, the

process relied on manual inspection of time series graphs to identify

signi￿cant changes, later replaced with a threshold-based detection

system. However, neither system was sufficient for finding changes

in performance in a timely manner. This work describes our recent

implementation of a change point detection system built upon the

Perfomalist approach in combination with XGBoost algorithm. The

algorithm produces a list of change points representing significant

changes from a given history of performance results. We are able

to automatically detect change points and achieve an 83% accuracy,

all while reducing the human effort in the process.

More Perfomalist's  approach details can be found in this blog post:

CPD - Change Point Detection (#ChangeDetection) is implemented in the free web tool Perfomalist

The result of initial usage of Perfomalist CPD API against MongoDB data is published HERE:

Perfomalist #ChangeDetection API was used against #MongoDB #perfomanceTesting dataset