Modellgetriebene Methoden und Werkzeuge zur Performanz-Vorhersage und Kapazitätsplanung komponentenbasierter Software-Systeme (Ferdinand)
Final Report Abstract
The Ferdinand project was initiated as a heterogeneous, multi-domain research project as a combination of software engineering, performance engineering, and model-driven software development. The focus of the Ferdinand project and its main contribution was to provide and develop engineering approaches to performance-aware software development by software architecture-level design and analysis that is coupled to the software realization through well-defined transformations. The contributions of the project can be subdivided into the following four categories: Parametrizable performance model completions using higher-order transformations, model transformation maintenance, performance analysis and prediction for virtualized execution environments, and performance analysis and prediction for message-oriented communication. Overall, the contributions include case studies and empirical validation with state-of-the-art technology, such as QVT transformations, Xen Server virtualization, and ActiveMQ messaging middleware, as well as real-world system environments, such as Sun Fire and IBM System z server hardware. Our first contribution facilitates design-time predictions of software quality attributes by introducing completion transformations, a model-driven technique for automatically refining architectural models with low-level details. For optimal reusability, completion transformations must be highly parametrizable. We extend ideas from product line engineering and tailor them to the problem at hand: a completion transformation is compiled from transformation fragments following a particular feature configuration. Implementation-wise, our approach relies on higher-order transformations, i.e., transformations that process transformations and transformation fragments. We built several highly configurable completions including a connector completion to weave in messaging-related performance influences, and an I/O completion that automatically integrates performance costs of modern virtualized storage environments. A second contribution of the project aimed to alleviate maintenance efforts of such completion transformations as well as model transformations in general. We defined a set of code metrics to assess the quality of model transformations, which help to quickly identify potential design issues of model transformation programs. Since most transformation languages do not provide for modular programming, we developed information-hiding modularity for transformation languages. We were able to demonstrate that it helps to better structure transformation programs and to encapsulate design details behind well-defined interfaces. With regard to legacy transformation programs, benefits of a modular structure do not always justify manual re-modularization costs. For that matter, we suggest to apply visual analytics to cope with the inherent complexity of transformations on larger models. The visual analytics methodology helps to carry out typical maintenance tasks with a significantly reduced effort, according to an empirical study. The third contribution comprises performance analysis and prediction techniques tailored for applications in virtualized execution environments. Since virtualization technology is getting increasingly popular, the application deployed in virtualized environments require special attention to avoid performance bottlenecks and allow to use resources efficiently. To this end, the techniques developed as part of the project include regression analysis-based as well as queueing theory-based performance models. Further, by using the Palladio Component Model (PCM) as example, software architecture modeling approaches were enhanced and extended by concepts to allow practical performance predictions and optimize software designs at the architecture level. Due to their high abstraction level and high degree of automation, the modeling approaches allow to address typical capacity planning questions that were either very difficult or even impossible to answer with previously existing and related approaches. The final contribution comprises performance analysis and predictions for message-oriented communication. In enterprise systems, message-oriented middleware platforms are often used as a building block for distributed software applications. A systematic performance analysis methodology coupled with fine-grained, in-depth analysis to reveal important performance effects and their causes are required and usually abstracted or carefully sidestepped in comparable work. Here, new modeling approaches of the middleware core based on queueing theory are developed as a basis that can be included in performance models of the overall distributed software system. The Ferdinand project benefited from various collaborations and discussions with academic partners, especially the Charles University in Prague, and industry partners, specifically IBM and SUSE Linux in the context of the Informatics Innovation Center (IIC), as well as the SPEC Research Group as a collaborative organization for knowledge transfer between academia and industry. These collaborations emphasize the academic and practical relevance of the project and were a key contributor to its success leading to overall approximately 50 research publications in the scope of the Ferdinand project as well as multiple tools and demonstrators. The publications of the project include leading, highly ranked scientific platforms such as the Int. Conference on Software Engineering (ICSE), the Int. Conference on Modularity (AOSD), the Int. Symposium on Component-based Software Engineering (CBSE), the Int. Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), the Int. Conference on the Quality of Software Architectures (QoSA), the Int. Conference on Model Transformation (ICMT), and the Int. Conference on Performance Engineering (ICPE) as well as the Software and Systems Modeling (SoSyM) journal. Today, the members of the project either have finished, or are about to finish their PhD. Multiple of the bachelor's and master's students involved in the Ferdinand project have started their PhD in the group of the grant holder at the Karlsruhe Institute of Technology (KIT). Furthermore, the existing collaborations with academia and industry as well as new collaborations, for example with the King's College London, are based on the results of the project and will be pursued in the future.
Publications
- Can Linear Approximation Improve Performance Prediction?
In: European Performance Engineering Workshop EPEW 2011: Computer Performance Engineering, ed. by N. Thomas. Lecture Notes
in Computer Science, Vol. 6977. 2011, pp. 250-264.
V. Babka, P. Tuma
(See online at https://doi.org/10.1007/978-3-642-24749-1_19) - Java Bytecode Instrumentation Made Easy: The DiSL Framework for Dynamic Program Analysis. In: Asian Symposium on Programming Languages and Systems APLAS 2012: Programming Languages and Systems.
Lecture Notes in Computer Science, Vol. 7705. 2012, pp. 256-263.
L. Marek, Y. Zheng, D. Ansaloni, A. Sarimbekov, W. Binder, P. Tuma, Z. Qi
(See online at https://doi.org/10.1007/978-3-642-35182-2_18) - Providing Dependability and Resilience in the Cloud: Challenges
and Opportunities. In: Resilience Assessment and Evaluation of Computing Systems, ed. by K. Wolter, A. Avritzer, M. Vieira, A. P. A. van Moorsel. Springer, 2012, pp. 65-81.
S. Kounev, P. Reinecke, F. Brosig, J. T. Bradley, K. Joshi, V. Babka, A. Stefanek, S. Gilmore
(See online at https://doi.org/10.1007/978-3-642-29032-9_4) - Interactive Visual Analytics for Ecient Maintenance of Model Transformations. In: International Conference on Theory and Practice of Model Transformations ICMT 2013: Theory and Practice of Model Transformations. Lecture Notes in Computer Science (LNCS), Vol. 7909. 2013, pp. 141-157.
A. Rentschler, Q. Noorshams, L. Happe, R. Reussner
(See online at https://doi.org/10.1007/978-3-642-38883-5_14) - Supporting swift reaction: automatically uncovering performance problems by systematic experiments. In: Proceedings of the 2013
International Conference on Software Engineering. ICSE '13. San Francisco, CA, USA: IEEE Press, 2013, pp. 552-561.
A. Wert, J. Happe, L. Happe
(See online at https://dx.doi.org/10.1109/ICSE.2013.6606601) - Using Queuing Models for Large System Migration Scenarios - An Industrial Case Study with IBM System z. In: European Workshop on Performance Engineering, EPEW 2013: Computer Performance Engineering, Venice, Italy, Sept. 16-17, 2013. Proceedings. Ed. by M. S. Balsamo, W. J. Knottenbelt, and A. Marin. Lecture Notes in Computer Science, Vol. 8168. 2013, pp 263-275.
R. Vaupel, Q. Noorshams, S. Kounev, R. Reussner
(See online at https://doi.org/10.1007/978-3-642-40725-3_20) - Completion and Extension Techniques for Enterprise Software Performance Engineering. In: Future Business Software { Current Trends in Business Software Development. Ed. by G. Brunetti, T. Feld, J. Schnitter, L. Heuser, and C. Webel. Progress in IS, 2014, pp. 117-131.
L. Happe, E. Burger, M. Kramer, A. Rentschler, R. Reussner
(See online at https://doi.org/10.1007/978-3-319-04144-5_11) - Designing Information Hiding Modularity for Model Transformation Languages. In: Proceedings of the 13th International Conference on Modularity (AOSD '14), Lugano, Switzerland, April 22 - 26, 2014, 2014, pp. 217-228.
A. Rentschler, D. Werle, Q. Noorshams, L. Happe, R. Reussner
(See online at https://dx.doi.org/10.1145/2577080.2577094) - Interactive Dependency Graphs for Model Transformation Analysis. In: Joint Proceedings of MODELS'13 Invited Talks, Demonstration
Session, Poster Session, and ACM Student Research Competition co-located with the 16th International Conference on Model Driven Engineering Languages and Systems (MODELS '13), Miami, USA, September 29 - October 4, 2013. Ed. by Y. Liu and S. Zschaler. CEUR Workshop Proceedings, Vol. 1115. 2014, pp. 36-40.
A. Rentschler, P. Sterner
- Stateful component-based performance models. Software & Systems Modeling, Vol. 13. 2014, Issue 4, pp. 1319–1343.
L. Happe, B. Buhnova, R. Reussner
(See online at https://doi.org/10.1007/s10270-013-0336-6) - The Storage Performance Analyzer: Measuring, Monitoring, and Modeling of I/O Performance in Virtualized Environments. In: Proceedings of the 6th ACM/SPEC International Conference on
Performance Engineering. ICPE '15, Austin, Texas, USA, 2015, pp. 107-108.
Q. Noorshams, A. Busch, S. Kounev, R. Reussner
(See online at https://dx.doi.org/10.1145/2668930.2693845)
