Context
This blog pot is an assignment I wrote for my subject at University of Queensland. Since it is never likely to be published anywhere, I did the next best thing – turn it into a serious looking blog post. Somewhat more of an experiment than anything else.
Author: Micheal Axelsen
1 Introduction
This paper presents the result of a qualitative methodology literature review regarding the effect of the use of different business technologies upon user knowledge. This topic is of interest as the use of business technologies in carrying out duties and undertaking workplace learning generally enhances an individual’s workplace effectiveness (Dugas, Green, & Leckie, 1999; Sutton, Young, & McKenzie, 1994), but over time this use can ‘deskill’ the user by diminishing their expertise (Alavi & Liedner, 2001; Arnold & Sutton, 1998; Mitroff & Mason, 1989).
Section 2 of this review provides background to the topic. Section 3 selects papers addressing this topic with qualitative methodologies, and considers these papers in terms of their ontological and epistemological position, research aims, project design, and research method. Section 3 also explores the implications of these findings for research design. Section 4 identifies opportunities for potential contributions to theory development through the consideration of gaps in research, and provides concluding comments regarding this review.
2 Background
Technology is increasingly used in business in response to legal liability concerns that demand consistency from the business professional (Sutton, et al., 1994). Business technologies are increasingly used to improve the reliability of decision making (Dowling & Leech, 2007; Mascha, 2001; Sutton, et al., 1994), and to deliver workplace learning, as well technology is something for taking your business to a new level. (Dugas, et al., 1999).
By relying upon business technologies such as intelligent decision aids and expert systems, it is theorised (Alavi & Liedner, 2001; Arnold & Sutton, 1998; Mitroff & Mason, 1989) that the professional’s expertise is decreased. There is a role for technology in increasing the workplace effectiveness of the professional, but the use of such technologies may decrease their expertise over time. This process, and strategies to mitigate its impact, requires further research (Dowling & Leech, 2007).
3 Qualitative methodology literature review
3.1 Selected papers
For this review, a literature search identified peer-reviewed, published papers from a variety of sources. The selected papers examined the role of business technologies in the development of knowledge or expertise, also needed have adopted a qualitative research method in addressing the topic.
The six selected papers focussed on differing branches of the topic. Broadly, these branches of the topic are the effectiveness of technology’s use in continuing professional development programs and learning (Lainema & Lainema, 2007; Zhou, Varnhagen, Sears, Kasprzak, & Shervey, 2007), the use of information systems to distribute and share knowledge (Franco & Mariano, 2007; Hyvonen, Jarvinen, & Pellinen, 2006; McHenry & Stronen, 2008), and the differing use of technology by novices and experts (Schenk, Vitalari, & Davis, 1998).
3.2 Ontological and epistemological positions
Each selected paper addresses the same broadly-defined topic. However, the ontological and epistemological positions do differ between the papers. The ontological position, being ‘what we may know’ (Grix, 2002), can be either objectivist, whereby social phenomena have an existence independently of social actors, or constructivist, whereby social phenomena are produced by social interaction and continually revised (Grix, 2002).
The epistemological position, being ‘how we come to know what we know’ (Grix, 2002), particularly affects the selection of the research method. The epistemological position is generally aligned with the ontological position of a research paper, such that an objectivist researcher generally adopts a positive approach, whilst the constructivist adopts an interpretive approach.
Epistemologically, some of the papers selected are positivist in their approach rather than interpretive, despite the selection of qualitative research methods. Franco and Mariano (2007), Hyvonen et al (2006), and McHenry and Stronen (2008) tend to the interpretivist tradition, whereas Lainema and Lainema (2007), Schenk et al (1998) and Zhou et al (2007) tend to positivism. In each paper, this epistemological position is a primary consideration in the selection and application of the research method, such that the researchers from the interpretivist position have adopted purely qualitative methods rather than adopting the mixed method approach favoured by positivist researchers.
3.3 Research aims
The research aims of the papers – and the branch of the literature within which the papers operate – are related to the ontological position of the researchers.
Both Lainema and Lainema (2007) and Zhou et al (2007) examine the role of technology in continuing professional development and learning. Lainema and Lainema (2007) examine the development of knowledge specific to the business (‘business know-how’), particularly the elements that contribute to the acquisition of business know-how. Zhou et al (2007) investigate whether online delivery of professional development is a successful alternative to other forms of professional development.
Both papers are focussed upon the individual actors within the business and increasing their effectiveness and contribution through technology. The research aim in both cases is focussed upon a cause-and-effect relationship. This area of research has a long history (see, for example, Porter 1957). It is a more mature area of research and perhaps due to this maturity and history there is a general tendency towards positivism in research carried out in this area.
In contrast, Franco and Mariano (2007), Hyvonen et al (2006), and McHenry and Stronen (2008) examine the role of technology in distributing and sharing knowledge. Franco and Mariano (2007) sought to propose potential solutions and tactics to ensure the success of information technology knowledge repositories. Hyvonen et al (2006) focused upon the mediation of knowledge through the transfer of knowledge amongst the organisation’s staff through the implementation of ‘best practice’ standards enforced by standardised cost accounting systems, and McHenry and Stronen (2008) investigated the role of a competency management information system in the development of competency and knowledge within businesses.
This area of research is emergent, and the selected papers examine the development of a shared understanding of the factors at play rather than undertake an investigation of positivist ‘cause-and-effect’ relationships.
Schenk et al (1998) undertook a comparison of the behaviour of novice system analysts with the behaviours of expert system analysts. The research aim here belies an ontological position of objectivism, and particularly with the aim of comparing ‘novices’ with ‘experts’, the research question in some respects calls for a quantitative research method. However, the researchers have here selected a qualitative research method (protocol analysis), and applied the method in a positivist manner rather than from an interpretivist perspective.
3.4 Project design
The project design of each of the selected papers relies upon research subjects. The selected papers, excluding Lainema and Lainema (2007) and Schenk et al (1998), adopt a case study perspective by examining the research topic within a single organisation. Lainema and Lainema (2007) run a simulation game with participants from two organisations, whilst Schenk et al (1998) uses subjects from a university course and experienced professionals in the field.
In Franco and Mariano (2007), Hyvonen et al (2006), and McHenry and Stronen (2008) a representational sample strategy (J. Mason, 2002) is adopted within a single organisation, in that the sample selected is considered to represent the organisation as a whole. The sampling strategy used by Schenk et al (1998) is deliberately illustrative in seeking out extreme novices and extreme experts. Participants in the Lainema and Lainema (2007) simulation game were from two different organisations. Franco and Mariano (2007) adopted a representational case selection strategy (Miles & Huberman, 1994), which is an organic selection strategy whereby subjects are selected based upon recommendations made by key informants within the organisation.
It is not clear from the latter two papers whether the sampling strategy selected is intended to be a strategic sampling approach, or an illustrative sample of convenience. On the basis of the evidence presented, the sample in both cases is more opportunistic than strategic, and is intended to take advantage of opportunities representing themselves to the researchers in examining the research question. This is not to say, however, that the research is flawed. Fundamentally, so long as the researchers recognise any limitations in their conclusions, an opportunistic sample that explores legitimate and compelling questions within an emergent area of research offers potential contributions to its theory development. A refusal to consider a worthwhile opportunity as it arose would be counter to the aim of theory development.
Project design reflects the researcher’s ontological position. Interpretivist researchers tended to adopt qualitative methods requiring long-term commitments to the research. Positivist researchers adopting qualitative approaches, on the other hand, tended to adopt qualitative methods that did not require such extensive and long-term involvement.
3.5 Research method
Mason (2002) identifies three broad categories of qualitative research methods. The selected papers utilise these methods, including ‘interviewing’ (Franco & Mariano, 2007; Hyvonen, et al., 2006; Lainema & Lainema, 2007; McHenry & Stronen, 2008; Schenk, et al., 1998; Zhou, et al., 2007), ‘observation’ (Franco & Mariano, 2007), and the use of ‘documents for discussion’ (Franco & Mariano, 2007; Hyvonen, et al., 2006). None of the selected papers utilise a ‘visual methods for discussion’ approach in their analysis.
Some papers utilise a survey in addition to their qualitative research methods (Lainema & Lainema, 2007; Zhou, et al., 2007). Schenk et al (1998) present descriptive statistics in support of their qualitative results. Schenk et al (1998) used protocol analysis, which is based upon ‘talking aloud’ and is generally considered to be an ‘empiricist qualitative method’ (O. J. Mason, 2003).
The emphasis in the papers selected is definitely upon semi-structured interviews and focus groups. This is consistent with the observation that such an approach is respectful of the subjects’ time and less of an imposition than, say, an experiment or extended participation or observational techniques. Interviews nevertheless allow the researcher to generate significant insight and understanding, and are generally effective as a qualitative research method. Some papers (for example, Zhou et al 1998) used interviews together with unstructured, or open-ended, survey questions in order to assist with triangulation of their data (J. Mason, 2002).
Interestingly, although some of the selected papers are from a positivist tradition, and some papers adopt a mixed-methods approach and thus have quantitative results available for analysis, none of the selected papers emphasise the findings arising from the quantitative methods used. Instead the authors emphasise qualitative methods in their discussion of their findings. As Zhou et al (2007) note, these methods allow the authors to achieve ‘an in-depth understanding of … attitudes and experiences’. For those papers where quantitative results were reported and discussed, the quantitative results were illustrated with the qualitative data generated from interviews.
This approach elicited a rich understanding of the underlying factors in the context of the researchers’ research question. The selected papers drew their conclusions principally from data generated from the qualitative research methods adopted.
3.6 Implications
Each paper reflects the authors’ differing research aims and ontological position. From a positivist perspective, some papers appear inherently opportunistic and lacking ‘scientific’ rigour (for example, Hyvonen et al 2006, p.147: ‘Opportunity to research the processes of new system implementation surfaced in Autumn 2002’). However, the purpose of such papers is to build understanding of the area of research. So long as the authors do not purport to present scientific conclusions in the manner of the natural sciences, but rather aim for building a common understanding of the issues involved and contributing to theory development, such criticisms are meaningless (Chalmers, 1982).
The topic selected for review, being the effect of the use of business technologies upon user knowledge, may be inherently empiricist in its formulation. The topic has usually been examined within areas traditionally quantitative in approach. It seems that few papers utilising qualitative methods were available for consideration in this literature review.
The selected papers drew from the rich context provided by qualitative research methods for the significant bulk of their findings, rather than from the quantitative findings. Whilst quantitative approaches provided some insight, these methods generally did not allow researchers to draw significant conclusions. The researchers instead developed their knowledge, and reached important conclusions, on the basis of qualitative data.
These findings graphically illustrate one role for qualitative research methods. A qualitative approach enhances the understanding of the topic in a manner that quantitative research is generally unable to (Dubin, 1978), and this is particularly so for emergent research where the relevant issues are unidentified (and perhaps, in some cases, never will be).
Conversely, quantitative methods allow this understanding – expressed as theory models – to be tested in a wider context to see whether the relationships and findings hold outside of the specific context of the qualitative research. At their core, qualitative and quantitative research methods are complementary rather than competing (Fitzgerald & Howcroft, 1998), and to rely upon one approach to the exclusion of the other would be counter-productive to the aims of research (Weber, 2004).
4 Conclusion
4.1 Gaps in the research
A specific gap in the research regarding the use of the ‘visual methods for discussion’ approach (J. Mason, 2002) can be demonstrated. Past research has emphasised the use of semi-structured interviews, observation, and documentary analysis. Although documentary analysis is one form of a visual method, it is likely that researchers have found practical limitations and low applicability of the ‘purely visual’ (for example, video and photographs) methods for analysis.
Nonetheless, future research may find it advantageous to adopt this qualitative method. For example, a researcher might ask new graduates working in an audit firm to document their experiences and learning with a video camera, and present and discuss an edited video in an online forum such as YouTube. This allows documentary analysis (transcripts, comment interactions) to be carried out, but the researcher is also able to undertake visual analysis through identifying the setting, visual context (for example, user interfaces and interactions), and editing choices by the graduate. A contribution to understanding of the research topic might be possible in this manner, although of course pragmatically this approach may be difficult.
The major gap in the research, however, is less that of a singular qualitative method. There is a general lack of qualitative research addressing this topic. It may be that, although qualitative research has been undertaken, it remains unpublished. Given that this is an emerging topic, particularly in the context of rapid technological change (Cetron, 2009), it is important that the potential contributions of qualitative research methods, and an interpretivist position, be explored through future research.
4.2 Concluding comments
This literature review has identified a lack of published qualitative research into the effect of business technologies upon user knowledge. No area of research, particularly an area yet in its infancy, can be addressed exclusively with either approach (Fitzgerald & Howcroft, 1998). It seems likely that qualitative and quantitative approaches are best considered to be complementary to building understanding.
Echoing Weber (2004), it is unlikely that the extreme positivist and interpretivist epistemological positions are truly held by any researcher. The dichotomy is artificial, and the common concern is rather that researchers be able to justify the knowledge claims made in the context of the research method. However, this does not imply that ‘anything goes’ in the context of research (Chalmers, 1982). Quality research meeting a necessary high standard is needed for development of the body of knowledge.
Evaluation of the value of research requires less of an emphasis upon its epistemological position, and more upon its standard of quality and rigour. As identified in this review, there is a potentially significant contribution to be made by qualitative and quantitative methods in a research program addressing the topic of the effect of business technologies upon user knowledge.
References
Alavi, M., & Liedner, D. L. (2001). Review: Knowledge Management and knowledge management systems: Conceptual foundations and research issues. MIS Quarterly, 25(1), 107-137.
Arnold, V., & Sutton, S. (1998). The theory of technology dominance: Understanding the impact of intelligent decision aids on decision makers’ judgments. Advances in Accounting Behavioral Research, 1(3), 175-194.
Cetron, M. J. (2009). Timeline for the Future: Potential Developments and Likely Impacts. The Futurist, 43(2), 33-37.
Chalmers, A. F. (1982). What is this thing called Science? Brisbane: University of Queensland Press.
Dowling, C., & Leech, S. (2007). Audit support systems and decision aids: Current practice and opportunities for future research. International Journal of Accounting Information Systems, 8(2), 92-116.
Dubin, R. (1978). Theory Building (Revised Edition ed.). New York, New York: The Free Press.
Dugas, T., Green, L., & Leckie, N. (1999). Impact of Technologies on Learning in the Workplace. Final Report: Human Resources Development Canada, Hull (Quebec). Office of Learning Technologies.
Fitzgerald, B., & Howcroft, D. (1998). Competing Dichotomies in IS Research and Possible Strategies for Resolution. Paper presented at the International Conference on Information systems, Helsinki, Finland.
Franco, M., & Mariano, S. (2007). Information technology repositories and knowledge management processes; A qualitative analysis. VINE, 37(4), 440-451.
Grix, J. (2002). Introducing Students to the Generic Terminology of Social Research. Politics, 22(3), 175-186.
Hyvonen, T., Jarvinen, J., & Pellinen, J. (2006). The role of standard software packages in mediating management accounting knowledge. Qualitative Research in Accounting and Management, 3(2), 145-160.
Lainema, T., & Lainema, K. (2007). Advancing Acquisition of Business Know-How: Critical Learning Elements. Journal of Research on Technology in Education, 40(2), 183-198.
Mascha, M. (2001). The effect of task complexity and expert system type on the acquisition of procedural knowledge: some new evidence. International Journal of Accounting Information Systems, 2(2), 103-124.
Mason, J. (2002). Qualitative Researching. London: SAGE Publications Ltd.
Mason, O. J. (2003). The Application of Mindfulness Meditation in Mental Health: Can Protocol Analysis Help Triangulate a Grounded Theory Approach? Forum: Qualitative Social Research, 3(1).
McHenry, J. E. H., & Stronen, F. H. (2008). The trickiness of IT enhanced competence management. Journal of Workplace Learning, 20(2), 114-132.
Miles, M. B., & Huberman, A. M. (1994). Qualitative Data Analysis: A Sourcebook of New Methods (2nd ed.). Newbury Park, California: Sage.
Mitroff, K., & Mason, R. (1989). Deep ethical and epistemological issues in the design of information systems. Expert System Review(Fall), 21-25.
Porter, D. (1957). A critical review of a portion of the literature on teaching devices. Harvard Educational Review, 27(2).
Schenk, K. D., Vitalari, N. P., & Davis, K. S. (1998). Differences between novice and expert systems analysts: what do we know and what do we do? Journal of Management Information Systems, 15(1), 9-50.
Sutton, S. G., Young, R., & McKenzie, P. (1994). An analysis of potential legal liability incurred through audit expert systems. International Journal of Intelligent Systems in Accounting, Finance and Management, 4, 191-204.
Weber, R. (2004). The Rhetoric of Positivism Versus Interpretivism: A Personal View. MIS Quarterly, 28(1), iii-xii.
Zhou, G., Varnhagen, S., Sears, M. R., Kasprzak, S., & Shervey, G. (2007). Online professional development for inservice teachers in Information and Communication Technology: Potentials and challenges. Canadian Journal of Learning and Technology, 33(2).
