Examining stakeholder involvement in the evaluation process for program improvement

Abstract

The results of an evaluation should be used for the envisioned goal and the evaluation process and/or outcomes should be used in practice and decision making. This article presents research whose objective was to establish the extent to which stakeholder involvement in evaluations impacts the utilization of evaluation findings for program improvement. Guided by the pragmatic paradigm and supported by the Utilization-Focused Evaluation Model and Knowledge Use Theory, the researchers assumed a descriptive and correlational design using mixed methods. The sample size for this study was 232 project staff from Non-Governmental Organizations (NGOs) in Kisumu Central Sub-County, Kenya. To analyze qualitative data, the open-ended responses from key informant interviews were recorded and coded appropriately for further analysis for themes through content analysis and comparative analysis. Frequencies and percentages were calculated to describe the basic characteristics of the quantitative data. To ensure the validity and reliability of the research instruments, pilot testing was conducted. Cronbach’s alpha at ? = 0.908 was attained as the reliability coefficient of the pre-test instruments. Tests of statistical assumptions were carried out before data analysis to avoid invalidation. A hypothesis was tested at the ? = .05 level of significance and was rejected. The findings demonstrate that there is a significant relationship between stakeholder involvement in evaluations and the utilization of evaluation results. This research, therefore, reinforces literature and helps to understand the ways in which stakeholder involvement in evaluations influences the utilization of evaluation results. It informs the evaluation field of study, fills gaps in the evaluation use literature, and contributes to the appreciation of factors that predict and enhance the utilization of evaluation results

Keywords: {'name': 'Project Evaluation, Evaluation Utilization, Stakeholder Involvement.'}

Authors & affiliations

Edwin Ochieng Okul
Affiliation not supplied

Raphael Ondeko Nyonje
Affiliation not supplied

Deposited references

Supplied references, not a list of publications citing this article.

African Water Facility. (2014). Project Appraisal Report, Innovative Sanitation Value Chain for the Un-sewered Urban Poor in Kisumu, Kenya. https://www.africanwaterfacility.org/fileadmin/uploads/awf/Projects/AWF Project-Appraisal-Report-KENYA-KIWASCO.pdf.

Alkin, M. C., Daillak, R., White, P., & White, P. (1979). Using evaluations: Does evaluation make a difference?

DOI: 10.4135/9781412984157.n2

Tall, O. K. B. (2009). The role of national, regional and international evaluation organizations in strengthening country-led monitoring and evaluation systems. Country-led monitoring and evaluation systems, 119.

Burr, E. M. (2009). Evaluation use and influence among project directors of state GEAR UP grants. Doctoral Dissertations, 15.

DOI: 10.3138/cjpe.43179

DOI: 10.1016/j.stueduc.2008.07.001

Concern Worldwide. (2014) Squeac Report, Otp Program Kisumu East District, Kisumu County, Kenya Samuel Kirichu, February 2013. http://www.coverage-monitoring.org/wp-content/uploads/2015/07/Kisumu-East-OTP_-Feb-2013_-Kenya_-SQUEAC.pdf.

DOI: 10.3102/01623737014004397

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions.

DOI: 10.1177/001316446202200401

European Policy Evaluation Consortium (EPEC) (2005). Study on the Use of Evaluation Results in the Commission. Final Report, Dossier no.1: Synthesis Report and Annexes

DOI: 10.1177/1098214008331009

Freedman, D., Pisani, R., & Purves, R. (2007), Statistics, 4th edition, New York: Norton Publications

DOI: 10.1016/S1098-2140(03)00056-0

DOI: 10.1016/S0263-2373(97)00036-4

DOI: 10.4102/aej.v3i1.107

DOI: 10.1177/0193841X8801200201

Guba, E. G. (1990). The Paradigm Dialog (edited by Egon G. Guba).

Guskey, T. R. (2000). Evaluating professional development. Corwin press.

DOI: 10.1016/j.ameval.2003.10.001

DOI: 10.1016/S1098-2140(03)00056-0

DOI: 10.1177/1356389014550562

DOI: 10.1177/1098214009341660

Khan, M. A. (1992). Monitoring and evaluation of development projects in South East Asia: The experience of Indonesia, Malaysia, the Philippines and Thailand (No. 19697, p. 1). The World Bank.

King, J. A. (2005). Participatory evaluation. In S. Mathison (ed.), Encyclopaedia of Evaluation. Thousand Oaks, CA: Sage.

Kirori, G. N., (2015). Rural Development Policies in Kenya: A Descriptive Analysis Over 1970-2001 Period. European Journal of Business, Economics and Accountancy 3(1).

Kiumbe, P., Wambugu, L. and Luketero, S. (2018). Influence of Stakeholder Participation in Utilization of Monitoring and Evaluation Results on the Performance of Fish Farming Projects in Kenya. Journal of Humanities and Social Science (IOSR-JHSS) Volume 23, Issue 9, Ver. 5 65-70 https://doi.org/10.9790/0837-2309056570

DOI: 10.1177/001316447003000308

Lowden, C. 2005. Evaluating the impact of professional development. Journal of Research in Professional Learning, (National Staff Development Council). Available online at http://institute.nsta.org/learningcenter/pdp/NSDC_Evaluating_Impact_PD.pdf (accessed 3 December 2019)

Magno, C., & Gonzales, R. D.L.C. (2011). Measurement and evaluation in the Philippine higher education: Trends and development. In E. Valenzuela (Ed) UNESCO Policy Studies: Trends and development in Philippine Education. Manila: UNESCO.

Mark, M. M., Henry, G. T. & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding and improving policies and programmes. San Francisco: Jossey-Bass.

McCormick, E. R. (1997). Factors influencing the use of evaluation results. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 58.

DOI: 10.1177/104973200129118183

Mouton, C. 2010, ‘The history of programme evaluation in South Africa’, MPhil thesis, Faculty of Arts and Social Sciences. Sociology and Social Anthropology Department, University of Stellenbosch.

Mugenda, O. M. and Mugenda, A.G. (2003).Research methods, qualitative and quantitative approaches: Nairobi: ACTS Press.

Najera, J. A., Gonzalez-Silva, M., & Alonso, P. L. (2011). Some Lessons for the

Future from the Global Malaria Eradication Programme (1955–1969). PLoS

Medicine, 8(1)

Nyonje, R. O., Ndunge, K. D., & Mulwa, A. S. (2012). Monitoring and Evaluation of Projects and Programs-A Handbook for Students and Practitioners. Nairobi, Kenya: Aura Publishers, 4(12), 45-99.

Ochanda, J. (2005). A study of strategic management practices in the Kenya public sector. Nairobi: Unpublished MBA project, School of Business, University of Nairobi.

Orodho, J. A. (2009). Techniques of writing research proposals and reports in education and social sciences. Nairobi. Kanezja publishers.

Owen, John M (2013). Evaluation Utilization and Communication' Program

Evaluation: Forms and Approaches. Australia. Allen & Unwin.

Patton, M. Q. (1997). Utilization-Focused Evaluation: The New Century Text (3rd Ed.). Thousand Oaks, CA: Sage.

Patton, M. Q. (2008). Utilization Focused Evaluation (4thed.). Thousand Oaks: Sage

Palys, T., & Atchison, C. (2008). Research decisions: Quantitative and qualitative perspectives. Toronto, Canada: Thomson Nelson.

DOI: 10.56645/jmde.v6i12.228

DOI: 10.1080/19439342.2014.966453

DOI: 10.4102/aej.v1i1.25

DOI: 10.1177/109821400302400402

DOI: 10.56645/jmde.v11i24.422

Republic of Kenya, (2007). Master plan for Implementation of NIMES, Nairobi: Government Printer.

Republic of Kenya. (2014). Second Annual Progress Report on the Implementation of the First Medium Term Plan (2008-2012). Nairobi: Republic of Kenya.

Republic of Kenya. (2017). Performance Management Framework for County Governments. Council of Governors. Nairobi: Republic of Kenya.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage Publications, Inc.

Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.

DOI: 10.3138/cjpe.010.012

DOI: 10.1016/S0886-1633(97)90027-1

Stufflebeam, D.L., Madaus, G.F., & Kellaghan, T. (2000). Evaluation models: Viewpoints on educational and human services evaluation. (2nd Eds.). Boston: Kluwer Academic Publishers.

Stufflebeam, D. L. & Shinkfield, A. J. (2007). Evaluation theory, models, & applications. San Francisco, CA: Jossey-Bass.

United States Agency for International Development- USAID (2014). Capacity in Kenya. End of Project Evaluation Final Report. http://pdf.usaid.gov/pdf_docs/pa00jrd2.pdf. Accessed on 15th March, 2017

Warah, R. (2013). UNDP’s Shortcoming a Reflection of a Wider Failure with the UN System. Daily Nation, P. 12