Principle and Methods of Development Research

Jan Van Den Akker, Principle and Methods of Development Research

Reviewed by: Evangelista L.W. Palupi

Source taken from:

This is a review of a chapter of a Akker’s book about design research. In this chapter “Principle and Methods of Development Research” Akker discusses the role of research in relation to educational design and development activities.

In the first part of Chapter 1#, Akker focuses on describing the rationale and basic principle of development research by outlining motive for conducting development research, definition and aim of developmental research and its key characteristic. Furthermore, methods of development research, its problem, and its major challenges will be described in the second part of this chapter. In this section I will try to review the first part only.

1. Motives for Development Research

Akker stated that there are various motives for initiating and conducting development research. The basic motive come from the experience that ‘traditional’ research approaches like experiment, surveys, correlation analyses and so on which focus on descriptive knowledge, cannot provide prescriptions with useful solution for variety of design and development problems in education. Another motives stem from the highly ambitious and complex nature of many policies in education worldwide; rather dubious reputation of educational research in general; and a distinct scientific interest at stake.

2. Definition and Aim

There are a lot of labels of development research such as design studies, design experiment design research, developmental/development research, Formative research, formative inquiry, formative experiment, formative evaluation, action research, and engineering research. Thus various labels are rather confusing.

‘Development research’ was ever used by Walker in discussing methodological issues in curriculum research (Walker & Bresler, 1993). Its goal is to inform the decision making process during the development of a product/program in order to improve the product/program being developed and the developers’ capabilities to create things of this kind in future situations. While Akker and Plomp (1993) defined ‘development research’ by its twofold purpose: (i) supporting the development of prototypical products (including providing empirical evidence for their effectiveness), and (ii) generating methodological directions for the design and evaluation of such products. In this approach, the scientific contribution (knowledge growth) is seen as equally important as the practical contribution (product improvement).

Besides having a prominent role/place in curriculum research, development research also has a prominent place in area of educational media and technology. Richey and Nelson (1996) mention as its aim: “improving the processes of instructional design, development, and evaluation … based on either situation-specific problem solving or generalized inquiry procedures” (o.c., p. 1213).

In the board of learning and instruction invest more in’design experiment’, Greeno, Collin and Resnick (1996) highlight the “kind of research that includes developmental work in designing learning environments, formulating curricula, and assessing achievements of cognition and learning and, simultaneously, on efforts to contribute to fundamental scientific understanding” (o.c., p. 41).

In the teacher education area the concept of ‘action research’ is rather popular. It refers to practical inquiries where teachers (often in collaboration with others) investigate and reflect on their own teaching and students’ learning. The primary goal is usually to contribute to the teachers’ professional learning and/or bringing about change in a specific educational setting (Elliott, 1991; Hollingsworth, 1997).

In the area of didactics the emphasis tends to be on ‘developmental research’ as an interactive, cyclic process of development and research in which theoretical ideas of the designer feed the development of products that are tested in classroom settings, eventually leading to theoretically and empirically founded products, learning processes of the developers, and (local) instructional theories.

3. Key Characteristics of Development Research

In this part, van Den Akker delineate the key of characteristics of development research by exploring two things, they are: the difference between developmental research and professional design and development; and development research compared to other research approaches.

In the case of the difference between developmental research and typical for design and development approaches in professional practices, Akker focuses is on additional elements that are more prominent in development research than in common design and development practices which are preliminary investigation, theoretical embedding, empirical testing and documentation, analysis and reflection on process outcomes.

In Development research, more intensive and systematic preliminary investigation of tasks, problems, and contexts is made. More systematic efforts are also made to apply state-of-the-art knowledge in articulating the theoretical rationale for design choices. In addition, in development research, clear empirical evidence is delivered about the practicality and effectiveness of the intervention for the intended target group in real user setting. Much attention is also given to systematic documentation, analysis and reflection on the entire design, development, evaluation and implementation process and on its outcomes in order to contribute to the expansion and specification of the methodology of design and development.

If we compare development research to other research approaches then their differences are best perceived from their aims and context. Akker wrote that the development research aims at making both practical and scientific contributions. Interaction with practitioners like teachers, policy makers etc. is also needed for searching innovative solution. Those kind of interaction can help to clarify the problems and the characteristics of its potential solution. In additon, to do interaction with practitioners gives benefits in both social and technical which is to improve their fitness for survival in real life context.

Related to its context or its nature of knowledge, Akker states that the major knowledge to be gained from development research is in the form of (both substantive and methodological) ‘design principles’ to support designers in their task. Those principles are usually heuristic statements of a format such as: “If you want to design intervention X [for the purpose/function Y in context Z], then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis], and to do that via procedures K, L, and M [procedural emphasis], because of arguments P, Q, and R.” Obviously those principles cannot guarantee success, but they are intended to select and apply the most appropriate (substantive and procedural) knowledge for specific design and development tasks.

This is a review of a chapter of a Akker’s book about design research. In this chapter “Principle and Methods of Development Research” Akker discusses the role of research in relation to educational design and development activities.

In the first part of Capter 1#, Akker focuses on describing the rationale and basic principle of development research by outlining motive for conducting development research, defenition and aim of developmental research and its key characteristic. Furthermore, methods of development research, its problem, and its major challenges will be described in the second part of this chapter which will be described in this section.

5.   Methods of Development Research

Methods of development research are not necessarily different from those in other research approaches. However, there are some specific features that are worth discussing here to further clarify the image of development research. The first one has to do with the central role of formative evaluation procedures in formative research. The second aspect refers to several typical methodological problems and dilemmas for development researchers.

5.1 Formative evaluation as key activity

formative evaluation holds a prominent place in development research, especially in formative research. The main reason for this central role is that formative evaluation provides the information that feeds the cyclic learning process of developers during the subsequent loops of a design and development trajectory. It is most useful when fully integrated in a cycle of analysis, design, evaluation, revision, et cetera, and when contributing to improvement of the intervention. However, a few typical characteristics of formative evaluation within the context of development research approaches deserve some elaboration.

Formative evaluation within development research should not only concentrate on locating shortcomings of the intervention in its current (draft) version, but especially generate suggestions in how to improve those weak points. Richness of information, notably salience and meaningfulness of suggestions in how to make an intervention stronger, is therefore more productive than standardization of methods to collect and analyze data. Also, efficiency of procedures is crucial. The lower the costs in time and energy for data collection, processing, analysis and communication, the bigger the chances on actual use and impact on the development process.

The basic contribution of formative evaluation is to quality improvement of the intervention under development. During development processes, the emphasis in criteria for quality usually shifts from validity, to practicality, to effectiveness (cf. Nieveen’s chapter 10 in this book). Validity refers to the extent that the design of the intervention is based on state-of-the-art knowledge (‘content validity’) and that the various components of the intervention are consistently linked to each other (‘construct validity’). Practicality refers to the extent that users (and other experts) consider the intervention as appealing and usable in ‘normal’ conditions. Effectiveness refers to the extent that the experiences and outcomes with the intervention are consistent with the intended aims.

5.2 Problems and Dilemmas in development research

In this section, Van Den Akker briefly describing some typical problems and dilemmas faced by researchers when doing development research. Some of that problems are:

  • Tension in role division between development and research. A tension can easily arise between designer who are eager to pursue their ideals in creating innovative interventions and researchers who tend to critically seek for correctness of decisions and empirical proof of outcomes.
  • Isolating ‘critical’ variables versus comprehensive and complex design. A typical difference between formative research and many other sorts of research is that one can hardly isolate, manipulate and measure separate variables in the same study. On the contrary, it is the very nature of formative research to investigate comprehensive interventions that deal with many interrelated elements at the same time which makes it very hard to apply.
  • Generalization of findings. Since data collection in formative research is usually limited to small (and purposive) samples, efforts to generalize findings cannot be based on statistical techniques, focusing on generalizations from sample to population. Instead one has to invest in ‘analytical’ forms of generalization: readers need to be supported to make their own attempts to explore the potential transfer of the research findings to theoretical propositions in relation to their own context.

6.   Major Challenges for Development Research

As a relatively new and upcoming research approach, Development research has its potentials , limitations and challenges for those who are interested in further exploration and improvement of its methodology.

A challenging trend for designers is the increasing prominence of prototyping approaches. Various questions arise: What does (rapid/evolutionary) prototyping imply for efficiency of the development process? Will it affect the balance between creative and systematic features of the approach? Does it reduce the relevance of preliminary investigations? To what extent does it influence the relationship between methodology (as prescribed in literature) and actual design activities in professional practices (can ‘theory’ keep up with ‘practice’, or will the gap even widen)?

Not only that, many challenges are also apparent with respect to evaluation methodology. What are appropriate tactics for increasing the information richness and efficiency of data collection procedures and instruments? How may the linkages between data collection, processing, and analysis be optimized? How can the communication about evaluation findings and the subsequent utilization for improvement of interventions be furthered? What are the most relevant indicators of quality, success and impact of interventions? What are promising approaches to further the generalizibility of research findings? How can the utilization of evaluation findings to design tasks in other settings be facilitated? Many useful suggestions and examples of such tactics are already offered in various publications (e.g. Miles and Huberman, 1994; Walker, 1992), but additional support is quite welcome.

An overall reflection is that research-based progress to expand and sharpen our knowledge on design and development is greatly enhanced through interdisciplinary approaches with purposive cross-fertilization between the many specialized subdomains in educational science and technology. Moreover, it is our own experience that joint development research efforts of professionals in various roles offer fine opportunities for professional learning and capacity building. Such activities have the potential to sometimes produce outcomes with ‘interocular’ significance: results that hit you between the eyes (Scriven, 1996).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s