The Anatomy of a Design Science Paper: A Research Note

(A Suggested Structure for a Design Science Article or Thesis)

Shirley Gregor and David Jones

This note is a supplement to Gregor and Jones (2007). It is a “work-in-progress” and the authors welcome input.

When we prepared our 2007 paper it originally had an appendix that showed a suggested structure for a journal article or thesis reporting an information systems design theory (“design-science-type knowledge” using more general terms). When writing a design science article it is necessary to include additional material that is not strictly part of the design theory/knowledge. That is, it is necessary to add the components that “position the article so that it is worthy of publication, showing why the topic is significant, that the work is credible, and that it is an advance on prior work (demonstrating novelty and a contribution to knowledge).

The reviewers were not comfortable with the appendix, in part because it was thought that it might appear too formulaic and be used inappropriately without reflection. Thus, the appendix did not appear in the published article. Subsequently, we have come to believe that colleagues might find it useful to have such a structure presented, at least for the purpose of discussion. Each of us has struggled with presenting design work in articles and having these articles accepted for publication. Portions of our work over a number of years remain unpublished, partly we think because we did not understand how to position and present the work satisfactorily. There has also of course been the problem that design-type work was seen by some as not being legitimate research in Information Systems. This problem has motivated our papers about theory and design theory.

Table 1 is an update of the original appendix and shows an view of the idealized structure of a paper (article or thesis) that presents design knowledge, with the eight components of a design theory as in Gregor and Jones (2007). Variations on this idealized structure would be expected in practice, both in content and the order of presentation. Following the table is an analysis of an existing article in a leading journal to examine whether it conforms to this structure.

It should be noted that this structure does not vary greatly from a traditional article or thesis (for example, see Perry 1998). The difference is that Chapters 5, 6 and 7 (or similar) in a design science paper show how an artifact was (or is to be) constructed, reflect on the principles underlying its construction and possibly advance hypotheses about its behaviour. In contrast, in a non-design article, these chapters would describe the results of an experiment, survey or case study in a standard empirical study. In both cases, these chapters are presenting the “evidence” that is the basis for the knowledge claims in the article.

In our discussion above we have indicated that what we term “design theory” may not be generally recognized as “theory”, but may be given some other label such as “design-type knowledge”. We appreciate that many in our community are uncomfortable with using the word “theory” in connection with artifact design. While this issue remains open, we ourselves prefer to use the word “theory”. In this we follow Gregor (2006), who distinguishes theory for design and action as a specific and recognizable category of theory (Type V theory).

It can be seen from the example given in a following section that the structural elements of a design theory can be detected in an article that does not even claim to be “design science”. We would argue that well-developed design papers that have been accepted in leading journals are likely to have many of the aspects of design theory (in our terms), rather than presenting only the description of a single artifact. The paper by Chiang and Mookeerjee (2004) does not just describe the artifact construction, but gives underlying background theory to show why it works and reflects on how applicable the design might be across a number of contexts. Thus, the generalization required of a theory is provided, even though it does still have a relatively narrow scope. These elements of reflection on underlying justificatory knowledge and the limits of applicability of the artifact need to be present to show that the research is in fact research, and not “consulting” or routine industry problem solving. Of course the demonstration of novelty, significance and credibility is also required for the paper to be accepted as a research contribution.

Table 1 Idealized structure of a design science paper

Section/Chapter Contents Notes
1. Introduction The purpose or goal and scope of the theory. ISDT component
  Definition of constructs ISDT component (2)
  Motivation, significance, outline of article. Similar to conventional articles
2. Literature review/background What was known about these systems before the work in the article was begun (that is, prior design theory), including similar existing systems. Similar to conventional articles
  Problems, gaps in knowledge and reasons for a new theory being needed. Similar to conventional articles
3. Research methodology Description of design science/constructive research approach. Many existing articles in Software Engineering and Computer Science omit this component. See Nunamaker et al., 1991; Walls et al. 1992); March and Smith 1995); Hevner at al. 2003; Gregor and Jones 2007.
4. Justificatory knowledge Theory from IS and natural and social science that informs the design theory. The placing of this component is debatable. It could be in Section 2 or 5. ISDT component (7)
5. Specification of the Designed artifact Meta-requirements – the purpose or goal of the class of artifacts addressed by the theory ISDT component (1)

  Process by which the designers arrived at their solution. This component could be optional. However, it might assist in demonstrating credibility. For example,  it could describe an iterative design with intermediate test stages and so on (the “search” process).
  Principles of form and function ISDT component (3)
  Consideration of artifact mutability ISDT component (4)
  Principles of implementation ISDT component (6)
  Testable design propositions ISDT component (5)
6. Instantiation Description of any working system or method in use
  ISDT component (8) This material could be partly combined with the previous section for exposition.
7. Evaluation Describe tests of system/ method in use. The tests could show performance, functionality, user acceptance and so on.
  Evaluate against criteria for design science/constructive research. See Burstein and Gregor (1999), March and Smith (1995), Hevner et al., (2004).
8. Discussion and conclusions Summarize work and findings, discuss limitations, establish significance. Similar to conventional articles

Example Design Article Structure (Chiang and Mookeerjee, 2004)

This paper was used as an example in Gregor and Jones (2007, p. 324) as presenting a design theory for a method, although the article did not explicitly present itself as being “design science”.

Analysing the Chiang and Mookeerjee paper shows a structure as follows.

1. Introduction (3 pages)

The significance and importance of incremental development is established.

It is claimed that existing research does not give much guidance on how to manage incremental development (that is, a “knowledge gap is identified).

States that the paper advances a “fault threshold policy for incremental improvement” (p. 4) (that is, the purpose of the design theory).

The term used in the paper are also explained (constructs).

Gives an overview of the policy problem, with a diagram (showing the scope of the problem addressed). Our experience with other articles of this type suggests that this is an important aid to readability, as giving a practical, easily understandable example of the nature of the artifact being dealt with helps to “ground the problem for the reader early on.

2. Literature Review (1.5 pages)

The problem is shown as connected to work on coordination and team integration (theory base and justificatory knowledge).

Prior research on improving software team productivity and related work is reviewed.

This section concludes with a statement that methods to manage software faults in an evolving system have not yet been prescribed.

Missing – Methods Section

There is no description of what methodology was employed in the study. This omission is observed in many design science type papers.

3. Fault Threshold Policy (5 pages)

This section provides an analytic description of the new software fault threshold policy.

It gives a detailed description of the way the policy operates (principles of form and function). An example is provided (a hypothetical instantiation).

Missing – principles pf implementation?

Not a great deal of detail is given on how to build a concrete version of this abstract policy in specific projects. An example is given where the formulae in the policy are applied to an imaginary scenario. It is stated that it might be necessary to build some randomness into the model in a real-life project and this is left for further work.

4. Learning effects (3 pages)

The behaviour of the policy in the context of multiple construction cycles, as learning benefits occur, is investigated (artifact mutability).

This section also includes a sub-section with a comparison of the new policy with another method, a “fixed-release method. It could be questioned whether this sub-section might have been placed differently (eg in the evaluation section), as it appears more to do with a comparative evaluation than learning effects.

5. Simulation experiments (3 pages)

This section provides an evaluation of the new method.

It is stated that this section has three purposes:

  1. that the new method/model makes accurate predictions;
  2. that it is robust with respect to underlying assumptions;
  3. that it is applicable to heterogeneous systems (that is, it has generalizability).

An overall assessment of the policy is provided and future directions for research are identified.

6. Summary and conclusions (.75 page)

A summary of the paper is given and it is speculated that most organizations would benefit from frequent system testing and use of the model, as it has economic benefits ( a testable hypothesis).

Note that there are 18 pages in the body of the paper in total. Almost a third of the paper describes the software fault threshold policy itself.

References

Burstein, F. and S. Gregor (1999) “The Systems Development or Engineering Approach to Research in Information Systems: An Action Research Perspective”, in Proceedings of the 10th Australasian Conference on Information Systems, B. Hope and P. Yoong, (eds.), Victoria University of Wellington, pp.
122-134.

Chiang, I. R. and V. S. Mookerjee (2004) “A Fault Threshold Policy to Manage Software Development Projects”, Information Systems Research, (15)1, pp. 3-21.

Gregor, S. and Jones, D. (2007). The anatomy of a design theory. Journal of the Association of Information Systems, 8, 5, Article 2, 312-335.

Hevner, A. et al. (2004) “Design Science in Information Systems Research”, MIS
Quarterly, (28)1, pp. 75-105.

March, S. T. and G. F. Smith (1995) “Design and Natural Science Research on Information Technology”, Decision Support Systems, (15), pp. 251-266.

Perry, C. (1998). A Structured Approach for presenting theses. Australasian Journal of Marketing, 6(1), 63-85.

2 thoughts on “The Anatomy of a Design Science Paper: A Research Note

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php