Developing a change management measurement instrument for effective use of examination administration system

Abstract

The quantitative change management measurement for effective use, which consists of operationalized change drivers, dimensions and sub-dimensions for effective use of Examination Administration System (EAS) in the South African context, has been scarcely discussed in the literature. This paper intends to develop and validate a change management measurement instrument (CHAMI) to measure the effective use of EAS in South Africa’s (SA) Technical and Vocational Education and Training (TVET) colleges. The CHAMI was developed drawing on the survey measurement instruments literature for change management and effective use and assessed quantitatively using data collected from 215 EAS users across all 318 TVET colleges from nine provinces of SA. The empirical results found the existence of construct validity of the CHAMI with 11 dimensions (i.e. user involvement and change recognition, user satisfaction, performance measurement, technology use, EAS adaptive use, EAS verification, user learning, transparent interaction, representational fidelity, informed action, and effective use) and 63 questions. Another contribution this study stems from the way the research constructs in a measurement model are operationalized as they incorporate measurement class (i.e. reflective or formative). This paper presents a CHAMI for effective use of EAS. In the future, management, researchers and practitioners can use the CHAMI to measure effective use, espouse or adapt instrument as an additional use measurement dimension on United Nations digital transformation programmes index or test it in different settings to advance its generalizability.

Keywords: {'name': 'Examination Administration System'}, {'name': 'Change management measurement model'}, {'name': 'Effective use'}, {'name': 'Technical and Vocational Education and Training colleges'}, {'name': 'Effective Use Theory'}

Authors & affiliations

Ngoanamosadi Stanford Mphahlele
Affiliation not supplied

Raymond Mompoloki Kekwaletswe
Affiliation not supplied

Tshinakaho Relebogile Seaba
Affiliation not supplied

Deposited references

Supplied references, not a list of publications citing this article.

DOI: 10.23919/ICITST.2017.8356370

DOI: 10.1108/JEIM-06-2016-0119

Afyonluo?lu, M., Aydin, A., Sevil, S., G., Yüksel, E., & Güngör, M. K. (2014). An e-government project management approach with e-transformation perspective. International Journal of eBusiness and eGovernment Studies, 6(1), 21–33.

Alamir Nasser Salim, A. (2017). Holistic Approach to the Factors Affecting Individual Investor’s Decision Making in the GCC Markets: Evidence from Oman and Saudi Arabia [Unpublished PhD thesis]. Plymouth University.

Al-Mamary, Y. H., Shamsuddin, A., & Aziati, N. (2014). The role of different Types of Information Systems in business Organizations: A Review. International Journal of Research (IJR), 2(7).

DOI: 10.1016/j.procir.2018.03.141

DOI: 10.1016/j.techsoc.2019.101209

Bonaretti, D. (2019). Effective Use of Information Systems for Emergency Management: A Representation Theory Perspective [Doctoral Dissertations, Louisiana State University]. https://repository.lsu.edu/gradschool_dissertations/5028

Bryman, A., & Bell, E. (2011). Business research methods (3rd ed.). Oxford University Press, Oxford.

Burton-Jones, A., & Grange, C. (2008). Using Information Systems Effectively: A Representational Perspective. Proceeding> Proceedings of JAIS Theory Development Workshop, 8. http://sprouts.aisnet.org/8-21.

DOI: 10.1287/isre.1120.0444

DOI: 10.1287/isre.1060.0096

DOI: 10.1287/isre.2017.0702

DOI: 10.1080/23311975.2023.2264001

DOI: 10.1108/SJME-08-2018-0036

DOI: 10.1080/10919392.2019.1571756

DOI: 10.1063/1.5005370

Cohen, J. (1988). Statistical power analysis for the behavioral sciences, 2nd ed, (2nd ed.). Lawrence Erlbaum Associates.

Creasey, T. (2019). Using the ADKAR model as a structured change management approach [Prosci]. Using the ADKAR Model as a Structured Change Management Approach. https://blog.prosci.com/the-adkar-model-a-structured approach

Creswell, J. W., & Creswell, J. D. (2018). Quantitative, and Mixed Methods Approaches (5th ed.). Sage Publications.

Department of Education (DOE). (2008). National Plan for Further Education and Training Colleges in South Africa [Plan]. https://www.gov.za/sites/default/files/gcis_document/201409/317121361.pdf

Department of Higher Education and Training (DHET). (2018). The Statistics on Post-School Education and Training in South Africa: 2018. Department of Higher Education and Training (DHET). https://www.dhet.gov.za/SiteAssets/Statistics%20on%20Post-School%20Education%20and%20Training%20in%20South%20Africa%2C%202018.pdf. Accessed: 08 September 2019

Department of Higher Education and Training (DHETb). (2023). The Statistics on Post-School Education and Training in South Africa 2021. Department of Higher Education and Training (DHET). www.dhet.gov.za/sitePages/DocPulictions.aspx

DHET News Article. (2017, October 13). Department of Higher Education and Training announces Electronic query system for TVET college students. Department of Higher Education and Training Website. https://www.dhet.gov.za/SiteAssets/Media/Statements/2017/191013%20EQuerry%20Launch%20Statement.pdf

DOI: 10.1108/17465660610667775

DOI: 10.2478/nispa-2021-0003

Easterby-Smith, M., Thorpe, R., & Jackson, P. R. (2015). Management and Business Research (5th ed.). Sage Publications Ltd.

DOI: 10.17705/2msqe.00005

Eden, R., Fielt, E., & Murphy, G. (2020, June 15). Advancing the Theory of Effective Use through Operationalization. Proceedings of the 28th European Conference on Information Systems (ECIS). https://aisel.aisnet.org/ecis2020_rp/116 or https://eprints.qut.edu.au/205069/1/68718174.pdf

Elbaz. (2013). Actor-network Theory, Tourism Organizations and the Development of Sustainable Community Livelihoods [PhD Thesis, Plymouth University]. http://hdl.handle.net/10026.1/2899

DOI: 10.1007/978-3-319-46152-6_13

DOI: 10.1108/14637150710752272

DOI: 10.1177/002224378101800104

Further Education and Training Colleges Act 16 of 2006, 16 (2006). http://www.saqa.org.za

DOI: 10.9790/487X-1123645

DOI: 10.2307/249689

DOI: 10.17705/1atrr.00024

Haake, P., Schacht, S., Mueller, B., Lauterbach, J., & Maedche, A. (2018, November 28). Toward an operationalization of effective use”, European Conference on Information Systems. Twenty-Sixth European Conference on Information Systems (ECIS2018).

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Prentice Hall.

Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modelling (PLS-SEM). Thousand Oaks: Sage.

DOI: 10.1007/978-3-030-80519-7

DOI: 10.1080/10696679.2011.11046435

DOI: 10.1016/j.lrp.2012.09.008

DOI: 10.1037/pag0000133

DOI: 10.1108/IMDS-09-2015-0382

DOI: 10.1177/00187267211070769

DOI: 10.1287/isre.2013.0501

Kalema, B. M. (2012). Critical Factors Influencing the Use of Integrated Tertiary Software [Unpblished Doctoral Degree Thesis]. Tshwane University of Technology.

Kapupu, R., & Mignerat, M. (n.d.). Change Management Strategies in ERP Implementations: A Case Study. Conference on Information Resources Management (CONF-IRM) 2015, 41. http://aisel.aisnet.org/confirm2015/41

Kasunic, M. (2005). Designing an effective survey (CMU/SEI-2005-HB-004). Carnegie Mellon Software Engineering Institute.

DOI: 10.1057/palgrave.jit.2000072

DOI: 10.1016/j.ijinfomgt.2009.06.003

DOI: 10.4018/ijec.2014010101

Kock, N. (2022). WarpPLS User Manual (8.0). Laredo, Texas.

DOI: 10.17705/1jais.00302

DOI: 10.1080/08874417.2019.1571457

DOI: 10.1108/ITP-04-2013-0063

DOI: 10.1080/01449290301782

DOI: 10.1007/s10447-017-9317-8

DOI: 10.24251/HICSS.2017.610

DOI: 10.4102/sajim.v25i1.1602

DOI: 10.1016/j.procs.2018.10.109

DOI: 10.20525/ijrbs.v13i1.3044

Mrini, Y. E., Belaissaoui, M., & Taqafi, I. (2019). Difficulties Related to the Implementation of ERPs in Morocco: Focus on Change Management. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS), 51(1), 47–55. https://asrjetsjournal.org/index.php/American_Scientific_Journal/article/view/4582

Musungwini, S., & Mono, L. (2019). Developing a Change Management Model for Managing Information Systems Initiated Organisational Change: A Case of the Banking Sector in Zimbabwe. https://doi.org/10.20470/jsi.v10i1.362

Mutiro, N. S. (2020). An evaluation of a computer information system in technical vocational education and training colleges [Unpblished Masters Degree Thesis]. Cape Peninsula University of Technology.

DOI: 10.19173/irrodl.v19i1.2886

DOI: 10.5539/ijbm.v7n19p125

OBbrien, J. A., & Marakas, G. M. (2007). Management information systems (10th ed.). McGraw-Hill/Irwin, a business unit of The McGrawHill Companies.

DOI: 10.1016/j.procs.2018.10.074

DOI: 10.1007/s10799-011-0097-2

DOI: 10.2307/25148814

DOI: 10.1109/ICTKE.2018.8612401

Piccoli, G., & Pigni, F. (2018). Information systems for managers: With cases (4.0). Prospect Press.

Protection of Personal Information Act, Pub. L. No. 4 of 2013 (2013). https://www.gov.za/documents/protection-personal-information-act

Ramaphosa, C. (2018). Ramaphosa, C. (2018). State of the Nation Address. Accessible on: Https://www.gov.za/speeches/president-cyril-ramaphosa-2018-state-nation-address-16-feb-2018-0000. https://www.gov.za/speeches/president-cyril-ramaphosa-2018-state-nation-address-16-feb-2018-0000

Raosoft Incorporated. (2004). Raosoft Sample size calculator [Raosoft Sample size calculator]. http://www.raosoft.com/samplesize.html

Rasoolimanesh, S. M. (2022). Discriminant validity assessment in PLS-SEM: A comprehensive composite-based approach. Data Analysis Perspectives Journal, 2(3), 1–8.

Rasoolimanesh, S. M., Nejati, M., Lei Mee, M., Ramayah, T., Shafaei, & ABD Razak, N. (2017). Full collinearity as a new criterion to assess discriminant validity of composite (formative) and reflective measurement models. 9th International Conference on PLS and Related Methods (PLS’17), 17–19.

DOI: 10.17705/1jais.00550

Republic of South Africa Continuing Education and Training Act 16 (CET Act 16) (2006). https://www.gov.za/documents/further-education-and-training-colleges-act.

Republic of South Africa National E-Government Strategy and Roadmap (Notice 886 of 2017), Pub. L. No. 25, Government Gazette No. 41241 (2017).

Rogan, M. (2019). The post-school education and training landscape in South Africa: ‘Massification’ amidst inequality’ In Post-School Education and the Labour Market in South Africa. (Ed. Rogan, M.). HSRC Press.

DOI: 10.21512/tw.v23i1.7170

DOI: 10.1108/02635570910939371

Sachdeva, S. (2008). Change Management for e-Governance (Working Paper). https://www.yashada.org/yash/egovcii/static_pgs/TC/CM_for_eGov.pdf

DOI: 10.1108/JEIM-09-2018-0206

DOI: 10.3390/su142315942

DOI: 10.1016/j.chb.2015.03.059

DOI: 10.1080/0144929X.2012.671851

DOI: 10.1016/j.mayocpiqo.2022.09.004

Sun, H., & Zhang, P. (2008). Adaptive System Use

An Investigation at the System Feature Level'. International Conference on Information Systems (ICIS) 2008 Proceedings. http://aisel.aisnet.org/icis2008/170

Susanta, T. D. (2016). Manajemen Layanan Teknologi Informasi. Surabaya: AISINDO.

DOI: 10.1016/j.compedu.2013.05.010

Tantua, E., & Godwin-Biragbara, F. (2020). Management Information System and Office Productivity of the Print Media in Rivers State Nigeria. International Journal of Business & Law Research, 8(1), 74–84.

DOI: 10.25300/MISQ/2022/14880

United Nations. (2022). United Nations Report, E-Government Survey: The Future of Digital Government. United Nations. https://desapublications.un.org/publications/un-e-government-survey-2022

Urus, T. B. S. (2013). Living with Enterprise Resource Planning (ERP): An investigation of End User Problems and Coping Mechanisms [Doctoral Degree thesis]. RMIT University.

DOI: 10.4102/sajems.v24i1.3581

DOI: 10.2307/30036540

DOI: 10.2307/41410412

DOI: 10.17705/1jais.00428

Visser, M. M. (2017). A methodology for the evaluation of management information systems at public technical and vocational education and training colleges in South Africa [Doctoral dissertation, University of South Africa]. https://uir.unisa.ac.za/handle/10500/23836

DOI: 10.1145/2389836.2389881

DOI: 10.4102/sajim.v15i1.531

DOI: 10.1007/s11747-015-0455-4

Wand, Y., & Weber, R. (1990). Toward a theory of the deep structure of information systems. ICIS 1990 Proceedings. International Conference on Information Systems,. https://aisel.aisnet.org/icis1990/3

DOI: 10.1111/j.1365-2575.1995.tb00108.x

DOI: 10.1109/ICACSIS.2018.8618253

DOI: 10.1016/j.heliyon.2022.e10044

DOI: 10.1016/j.ipm.2020.102461

DOI: 10.1016/j.aci.2019.11.007

DOI: 10.4236/jss.2014.210012

DOI: 10.1016/j.chb.2010.01.013

DOI: 10.28945/2185

Zikmund, W. G., & Babin, B. J. (2010). Exploring marketing research. Mason: Thomson Higher Education.

Zuboff, S. (1988). In the Age of the Smart Machine: The Future of Work. Basic Books Inc.