Empirical Evaluation of Diagrams of the Run-time Structure for Coding Tasks

Abstract

With object-oriented design, it is at least as important—possibly more important—to understand the runtime structure, in terms of objects and their relations, as to understand the code structure dealing with source files, classes and packages. Today, many tools and diagrams help developers understand the code structure. Diagrams of the run-time structure, however, are much less mature. One diagram of the run-time structure is a statically extracted, global, hierarchical Ownership Object Graph (OOG). The OOG conveys architectural abstraction by ownership hierarchy by showing architecturally significant objects near the top of the hierarchy and data structures further down. In an OOG, objects are also organized into named, conceptual groups called domains. We posit that types are not enough for object-oriented code comprehension, and that the OOG improves comprehension by giving developers the ability to distinguish the role that an object plays, not only by type, but also by named groups (domains) or by position in the run-time structure (ownership), i.e.,types + ownership + domains. We evaluate, in a controlled experiment, whether an OOG, as a diagram of the run-time structure, improves object-oriented code comprehension by giving developers the ability to distinguish the role that an object plays, not only by type, but also by named groups (domains) or by position in the run-time structure (ownership). We observed 10 participants, for 3 hours each, perform three feature implementation tasks on a framework application. Our experiment identified that, on average, the participants who used information obtained from OOGs, in addition to information obtained from the code structure, i.e., class diagrams and browsing or reading the code in Eclipse, always outperformed the participants who used only information obtained from the code structure, i.e., class diagrams and browsing or reading the code in Eclipse. Our results indicate that, on average, the OOG had a positive effect of varying extents on comprehension that reduced the time spent by 22%-60% and irrelevant code explored by 10%-60%. The difference was significant (p < 0.05) for two of the tasks