Giving to RU
Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 1: Develop a High Level Monitoring and Evaluation Framework for SETAs

This project will develop a high level M&E framework for all SETAs and support all other components of the Chair. The high level framework is to be used by DHET, SETAs themselves and parties appointed, to continuously monitor and regularly evaluate the functioning of SETAs and their impact on social development and economic participation. The framework must enable SETAs to be accountable to their stakeholders and in equal measure, to learn from M&E findings in order to increase their positive impact within their sphere of influence.

Last Modified: Tue, 23 Oct 2018 09:49:26 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Chair of Monitoring & Evaluation in a SETA environment

In March 2018 Rhodes University (ELRC, NALSU and Community Engagement) were awarded a Research Chair to focus on Monitoring and Evaluation (M&E) in a SETA Environment. To be rolled out over three years (August 2018 – March 2020), this research programme is an initiative of South Africa’s 21 Sector Education and Training Authorities (SETAs) and strongly supported by the Department of Higher Education and Training (DHET). It is funded in the first instance by the BANKSETA and ServicesSETA.

The research programme consists of nine individual projects, aimed at developing frameworks, methods, guidelines, tools and capacity for M&E of Sector Education and Training Authorities’ work and impacts.  These projects are:

Project 1:  Overall M&E framework for SETAs

Project 2:  Tools for evaluating skills for enterprise development in a township economy

Project 3:  Performance standards for SETAs

Project 4:  Cost-benefit analysis tool for evaluating work-based learning

Project 5:  Tracer study guidelines for work-based learning

Project 6:  Framework for M&E in relation to the Discretionary Grant

Project 7:  Framework for M&E in relation to the Mandatory Grant

Project 8:  Evaluating SETA governance

Project 9:  Capacity building 

The programme will address the fact that despite extensive monitoring and reporting already taking place in the national post-school education and training system, some important evaluative questions about our post-school education and training system and how to strengthen it, remain unanswered.

It will explore innovative methods to address the need for evaluation at multiple levels: from single initiatives by individual SETAs, to a composite national picture. Both conceptual depth and practical feasibility are important, with due consideration of the kinds of M&E that SETAs and their research partners can realistically undertake.

A key consideration is to ensure that M&E processes in the national system actually support transformation agendas. Currently, drives to improve performance and accountability through managerialism and compliance control, threaten to swamp the transformative intent of the post-school education and training system. Performance management should serve transformation, not inundate it. Is a balance possible and if not, what is the alternative?

The answers to such fundamental questions need to inform M&E tools and design. M&E is not just a technical activity, it is deeply strategic, normative and ideological. Making sure that processes are aligned with intentions, may well be where the system’s capacity needs to grow, and where university-based research can play a significant role.

The Chair in M&E in a SETA Environment is held by Prof Eureta Rosenberg of the ELRC. A number of project leads and researchers have already been appointed, although staffing has not yet been completed. Prof Mike Rogan  (NALSU) leads Project 5 and Dr Glenda Raven, a post-doc scholar at the ELRC, leads Project 4. Mike Ward, PhD student at the ELRC, is the first researcher to be appointed in the programme, and Danel Janse van Rensburg is the project manager.  Consultations, interviews and policy reviews are underway. The National Skills Authority has asked that the interim findings of this research be shared in one or more colloquia during 2019. Until then, interested parties are welcome to contact us at Rhodes University.

Last Modified: Tue, 23 Oct 2018 10:09:04 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 3: Develop Standards for SETA Performance

This project will develop Performance Standards to be used in the assessment of SETA Performance.  It will be guided by the M&E Framework (Project 1), and by an evaluation of the existing performance assessment framework(s) in place, in particular the MPAT (Management Performance Assessment Tool). The Standards should be useful to the SETAs themselves, the Department of Planning, Monitoring and Evaluation (DPME), DHET (Skills Branch) and parties appointed, to assess the performance of SETAs in relation to the given and agreed upon frameworks. These assessments must enable SETAs and other roleplayers in the system to be accountable to stakeholders and in equal measure, to learn from the findings in order to increase their positive impact within their sphere of influence. This project is an opportunity to develop an aligned, implementable M&E framework that sees post-school education and training (PSET) as a system and supports systemic and institutional learning as much as accountability. Previous initiatives aimed at improving SETA performance and the assessment of SETA performance will be considered, and efforts made to overcome their limitations. This may need to include early on-boarding of all key roleplayers, and change management processes possible within the scope of the research chair. The standards could be broad enough to be also used in a future PSET landscape

Last Modified: Tue, 23 Oct 2018 09:52:34 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 4: Develop a Cost Benefit Analysis Tool

This project aims to develop a cost benefit analysis tool to support SETAs in the evaluation of skills planning activities.  It seeks to take a systems approach to considering costs and benefits to include also indirect and non-financial costs and benefits in addition to direct and financial costs and benefits which are often easier to identify and quantify.  The project is proposed as a collaboration with SETAS, drawing on current practices to explore direct, indirect, financial and non-financial costs and benefits for the individual, for the economy through specific economic sectors and for society as a whole.  The project plan involves the evaluation of 3-4 WBL activities to surface costs and benefits – envisaged and incidental - associated with WBL and use this as a basis for developing the tool.  The tool will then be piloted in 3-4 cases.  The key output is an online tool, with a guidelines resource, allowing SETAs to adapt the tool in different contexts of skills development.    

Last Modified: Tue, 23 Oct 2018 09:53:19 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 5: Tracer Study Protocol

To investigate the destinations of learners who complete workplace based learning (WBL) programmes that are funded by SETAs. An additional objective is to develop a standardised research methodology which can be used to track the completers of WBL programmes on a regular basis, across all 21 SETAs.

Last Modified: Tue, 23 Oct 2018 09:54:01 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 6: Develop a Framework for Monitoring and Evaluating the Implementation of the Discretionary Grant

This project will develop a specific framework for the monitoring and evaluation of the implementation of the Discretionary Grant, for all SETAs. It will be aligned with the high level M&E framework (Project 1), which will in turn align with other relevant M&E frameworks including DHET’s. It aims to support the goals of the Post-School Education & Training (PSET) system, i.e. to contribute to social development and economic participation. This framework must support SETAs in evaluating the impact of the Discretionary Grant, which has a wide scope. The purpose is to develop more effective and innovative methodological approaches for evaluating DG implementation. These approaches can be used between SETAs (i.e. some SETAs can evaluate some aspects and others can evaluate other aspects) and over time (evaluating some aspects one year and other aspects the following year). This will allow SETAs to collectively (systemically and systematically) build up a comprehensive understanding of what works, in what context and why; as well as what has not worked, and why? (the focus of realist evaluation approaches) to inform future improvements in the system.The expansive learning – activity system approach is a form of participatory evaluation/evaluative stakeholder engagement that allows for immediate learning and improvements in particular contexts.

Last Modified: Tue, 23 Oct 2018 09:54:43 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 7: Develop a Monitoring and Evaluation Framework for the Mandatory Grant

This project will develop a specific framework for the monitoring and evaluation of the implementation of the Mandatory Grant, for all SETAs. It will be aligned with the high level M&E framework (Project 1), which will in turn align with other relevant M&E frameworks including DHET’s. It aims to support the goals of the Post-School Education & Training (PSET) system.

Known challenges in implementing the Mandatory Grant (DHET Guidelines on SETA Regulations 2015) include: Lack of clear policy and strategic intent against which SETA impacts can be measured; The inadequate quality and accuracy of employer data on scarce and critical skills and skills gaps; Factors affecting SETA effectiveness and efficiency including compliance measures and other policies; Quality and relevance of training provided by accredited providers; Availability of training and qualifications to meet actual sector needs.

While the high level M&E framework (Project 1) and the Performance Standards (Project 3) will address more general evaluation foci, this specific framework for the Mandatory Grant will assist SETAs and their stakeholders to (1) gain deeper understanding specifically into these challenges and (2) how they can be addressed, and (3) assess the success of measures to address them. The results should (4) inform the annual reviews of Policy, Strategic Plans, Sector Skills Plans and Annual Performance Plans. This would introduce into these reviews a strong learning focus and a more transformative strategic focus, beyond numerical targets. It would also (5) ensure that M&E findings are actually used for improvements in the system.

Last Modified: Tue, 23 Oct 2018 09:55:28 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 8: Evaluate SETA Governance

This project will undertake an evaluation of SETA governance. It will follow up on previous evaluations of SETA governance and fill knowledge gaps in existing knowledge gaps about SETA governance. It will also test a methodology for evaluating governance which will contribute to the high level M&E framework (Project 1) and Capacity Development Programme (Project 6). By focussing on governance, this evaluation will look at an important consideration in the functioning, effectiveness and impact of SETAs, and their sustainability. Recommendations arising from the evaluation should be broad enough to inform a future PSET landscape.

Last Modified: Tue, 23 Oct 2018 09:56:06 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

Project 9: Develop a capacity development programme

This project will develop and implement a capacity development programme for SETA representatives (research and M&E staff). The capacity building programme, in the form of an accredited ‘long short course’, will complement and finish off the M&E tools and frameworks to be developed in the rest of the SETA programme. The purpose is to:

  • Involve SETA representatives in the ongoing development of M&E resources in a way that builds the participating individuals’ and the SETA’s capacity (praxis)
  • Familiarise SETA representatives with the M&E frameworks developed in the SETA M&E Chair Research Programme so that they are confident to implement them and guide others in their implementation (e.g. write terms of reference for service providers) and if possible, make adjustments to proposed frameworks in their own contexts
  • Serve as a form of ‘change management’ to allow SETA representatives and the SETAs themselves to work through the challenges of taking on board new frameworks and processes (this may mean, from time to time, broader participation in the programme)
  • Share the training material content and implementation lessons learnt with wider audiences
  • Promote SETAs as learning organisations in the national PSET and international contexts.

Last Modified: Tue, 23 Oct 2018 09:56:57 SAST

Rhodes>NALSU>M&E in a SETA Environment>SETA

M&E About

In March 2018 Rhodes University (ELRC, NALSU and Community Engagement) were awarded a Research Chair to focus on Monitoring and Evaluation (M&E) in a SETA Environment. To be rolled out over three years (August 2018 – March 2020), this research programme is an initiative of South Africa’s 21 Sector Education and Training Authorities (SETAs) and strongly supported by the Department of Higher Education and Training (DHET). It is funded in the first instance by the BANKSETA and ServicesSETA.

...read more

Last Modified: Tue, 23 Oct 2018 10:13:51 SAST