Usability Analysis of Visual Programming Environments: a 'Cognitive Dimensions' Framework

Reviewed by Neil Ernst / 2012-06-25
Keywords: Usability, Visual Programming Environments

Green1996 T.R.G. Green and M. Petre: "Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework". Journal of Visual Languages & Computing, 7(2), 1996, 10.1006/jvlc.1996.0009.

The cognitive dimensions framework is a broad-brush evaluation technique for interactive devices and for non-interactive notations. It sets out a small vocabulary of terms designed to capture the cognitively-relevant aspects of structure, and shows how they can be traded off against each other. The purpose of this paper is to propose the framework as an evaluation technique for visual programming environments. We apply it to two commercially-available dataflow languages (with further examples from other systems) and conclude that it is effective and insightful; other HCI-based evaluation techniques focus on different aspects and would make good complements. Insofar as the examples we used are representative, current VPLs are successful in achieving a good 'closeness of match', but designers need to consider the 'viscosity' (resistance to local change) and the 'secondary notation' (possibility of conveying extra meaning by choice of layout, colour, etc.).

It must be a major coup as a researcher for your research to make in onto Wikipedia (and not get deleted for "notability"). The reason the Cognitive Dimensions framework has had such impact is that it is a great way to assess user interfaces without conducting user studies (which are costly and often of questionable value). Green's Cognitive Dimensions framework proposes a set of heuristic criteria which evaluate how well the interface supports certain tasks. These criteria include consistency, hard mental operations and error-proneness.

In this paper Green and Petre apply the framework to visual programming environments (e.g., Simulink). The research question they investigate is the degree to which these tools help in writing software compared to a text editor for Basic programming, using a yardstick problem of calculating rocket trajectory.

Some of their findings:

  • Visual programming languages can easily grow incomprehensible, placing "severe demands on working memory" as concepts become intertwined.
  • Abstraction is easier to manage in visual languages, since low-level items can literally be hidden away.
  • Adding new code to existing visual programs was very slow: from 508 seconds for LabView, 194 seconds for Prograph and only 63 seconds in Basic, which they note is "an astonishing ratio of 8:1 between extremes".
  • These languages, like spreadsheets, make long-range data dependencies difficult to track down.
  • Textual languages (like Pascal or Basic) support secondary notations such as the use of whitespace or statement ordering to communicate other information. In the two visual languages they studied, these notations are meagre. Visual layout is not terribly useful for communicating information. For example, "I quite often spend an hour or two just moving boxes and wires around, with no change in functionality, to make it that much more comprehensible when I come back to it."

Many of these findings are even more relevant today in the model-driven design (MDD) field, and show the power of a simple set of heuristics in uncovering potential difficulties. Designers of modern visual language tools (like Metacase) should be able to explain how they overcome these challenges.