Full Version (470Kb ~ 3 min)
Section A: Introduction to the study
|HEQC Evaluative Study of Institutional Audits 2006
|Researchers: Sharman Wickham, Gonda Coetzee, Barbara Jones, Anthea Metcalfe
|Sharman Wickham, Gonda Coetzee, Barbara Jones, Anthea Metcalfe
This introductory section explains the purpose of this study, describes the institutions it covers, outlines its research approach and methodology, introduces the research team, and finally notes the limitations of the study.
A.1 The study and its purpose
The focus for the data collection and analysis and the structure of this report were outlined by the HEQC. Consistent with its commitment to reflective practice, the HEQC has been conducting a number of focused evaluations of its systems that are geared to improving and eventually reviewing the methodologies used in the four areas of quality assurance that constitute its main responsibilities: programme accreditation, national reviews, institutional audits, and quality promotion and capacity development.
The overall aim of this study is to evaluate the extent to which the institutional audits of higher education institutions conducted by the HEQC since 2004 have taken place in an efficient and effective manner in each of their phases and in relation to the individuals and structures involved in the process. A key question is how far the audits have advanced the objectives set out by the HEQC in its Framework for institutional audits (June 2004)
More explicitly, the HEQC wanted to know:
A.2 The institutions included in the study
How did the audited institutions experience each phase of the audit process?
What effects did each of the phases have on the institutions?
How did the institutions (in all their various layers) perceive the audit?
What were the positive and negative outcomes of the audit for the institutions (again in all their layers)?
On-site visits and face-to-face interviews took place at three public institutions between April and June 2005 – the Central University of Technology (audited in 2004), Rhodes University and Stellenbosch University (both audited in 2005), and two private providers – the Management College of South Africa and CityVarsity Film and Television Multimedia School (both audited in 2004). Each of these institutions constituted a case for this study. In addition, permission was granted to use information from the internal evaluation report on the institutional audit at the University of Cape Town (audited in 2005). These six institutions are referred to in this report as CUT, Rhodes, SU, Mancosa, CityVarsity and UCT. It is important to note that while this study covers six distinct institutions the sample does not represent all the higher education institutions in South Africa, since some institutional types (such as historically disadvantaged institutions and large private institutions) were not included.
It needs to be noted that, while useful in further developing and confirming patterns and trends noted in the data collected in the other five institutions, the findings documented in the UCT internal evaluation report do not address all the questions and issues raised in the interviews at the other five institutions where face-to-face interviews took place. (The UCT internal evaluation report was based on data collected through questionnaires sent to staff who had participated in the audit. A total of 258 questionnaires was sent and there were 45 responses.) Besides the internal evaluation report, a paper written by a senior staff member at UCT, based on this report and providing further interpretation of its findings, was given to the research team.
Each of the five case study reports developed for this study begins with a brief profile of the institution, highlighting its unique features and helping develop an understanding of the variety of responses to the experience of audit that this study found.
While there are clear distinctions between the larger, well-established public institutions and the smaller, relatively new, private providers, there are also important differences between the institutions which fall into these two categories. For example, CUT had been granted university status around the time of the audit, while Rhodes, SU and UCT have enjoyed this status since their inception. While the latter may be categorised as traditional universities, their different
histories and locations have affected their structures, cultures and policies and, as will be seen in this report, their responses to and experiences of audit. Differences were also noted between the two private providers. CityVarsity focuses on film, television and multimedia, Mancosa on management studies. This, too, resulted in different perceptions and understandings of audit and different experiences of the phases and processes.
Staff members themselves, interviewed in this study, held the view that their institutions have distinctive characteristics which might not be easily understood by ‘outsiders’. For example, the researcher who visited Rhodes, herself an old Rhodian, was often told by interviewees that she would understand a particular point being made because of her previous knowledge and experience of this institution.
It was also noted that the interviewees at the participating institutions differed in the extent of their knowledge about quality assurance and institutional audit. While all had certain views of audit prior to the audit itself, staff at the three traditional universities also displayed what might be termed a deeper knowledge of the debates and contestations relating to audit. The researchers who visited Rhodes and SU noted that certain interviewees here were able to refer
to a body of literature on the topic and were likely to have developed stronger positions and perspectives on approaches to quality assurance than those at the university of technology and the private institutions. The internal evaluation report from UCT also refers to recent international literature on audit, the review of which may have contributed to their positions and
perspectives and the processes and procedures followed. There was less evidence of such knowledge among staff at the small private providers and, for the most part, at the university of technology.
Finally, as will be highlighted in Section B of the report, besides differences across and within categories of institutions, there were also considerable differences in the responses to questions posed in the interviews within the individual institutions
. The absence of an institutional voice suggests that these institutions are not homogeneous entities but composed of complex layers that cannot easily be characterised. In some cases there were contradictions between the various
layers of the institutions (e.g. senior managers and academics, or main campus and satellite campus personnel) while in other cases responses were more dependent on the primary identities of the interviewees (e.g. researchers versus teachers). The most contradictory responses were those from interviewees at Rhodes, where it seemed that ideological positions and views of quality and quality assurance may have been influential. In highlighting the lack of institutional voices, this report attempts to make sense of the variety of responses across and within institutions.