A Systematic Assessment Framework for Higher-Education Institutions

The multiple criteria of ranking, rating and accrediting the higher education institutions in India and across the globe have de facto become the lens for viewing the institutions. There is an increasing number of assessment systems used by multiple agencies. With the public policy emphasis on assessing higher education institutions, it is necessary to use a systematic and systemic framework for assessment. For this purpose, this paper proposes an ontological framework for assessment of higher education institutions. The ontology’s dimensions, sub-dimensions, and their constituent elements are derived from higher-education assessment systems that are well known globally and in India. The framework can help higher-education institutions: (a) assess themselves systemically and systematically; (b) highlight the bright, light, blind, and blank spots in their performance; and (c) correct their trajectory to fulfil their vision. Such a framework would assist in identifying the gaps in and pathways to improve their position in assessments.


Introduction
The exponential growth of higher education institutions (HEIs) in India and globally has increased the complexity of assessing the institutions holistically. The complexity has been further compounded by the proliferation of ranking, rating, and accrediting agencies of HEIs in the past decade. The scores institutions obtain on these assessments play an increasingly important role in determining their reputation, the quality of students and faculty members they can attract, the resources they can generate, the autonomy they have, and their brand value. Similar to the national and global trend amongst HEIs to become a world-class university (Altbach, 2004) rating, ranking and accreditation is gaining traction globally. There is a paradigm shift in policies related to higher education, which are considering a position in global rankings, ratings and accreditation as a criterion for giving funds and autonomy to HEIs. The guidelines to categorize HEIs in India is one such illustration. The Ministry of Human Resource Development (MHRD), Government of India is using the Quacquarelli Symonds (QS) world university rankings, Times Higher Education Ranking (THER) world university rankings, National Institutional Ranking Framework (NIRF) and National Assessment and Accreditation Council (NAAC) assessment for categorizing HEIs for granting autonomy (Government of India, 2018). This shift in policies of the government and the globally emerging trend have influenced the institutions to align their management policies along with the strategy of governance to such assessment systems.
Globally there are multiple assessment systems to rank, rate, and give accreditation to HEIs based on their defined criteria. The criteria of each assessment system do not systematically encompass multiple aspects of HEIs for a comprehensive assessment, which if done, could lead to the holistic development of the institution. There is an absence of a single assessment system that could capture the entire vision of HEIs. The bias in assessments would only reinforce the institutions' skewed approach to governance, management, and delivery of education.
The absence of a comprehensive assessment system can undermine higher-education policymaking, governance, and regulation. It could also exacerbate conflicts and prove to be a hindrance to HEIs to improve their position in the assessment. In this context, it would be appropriate and pertinent to have a framework that could give a big picture of the existing unwieldy assessment systems. Such a framework would avoid the fragmentation engendered by the present system. The comprehensiveness of the framework would complement the efforts of HEIs for assessment. It would further ease the process of getting assessed by multiple assessment systems. This paper systematically develops an ontological framework of higher-education assessment, by deriving its elements from well-known global and national ranking, rating, and accreditation systems and organizing them. It also discusses the validation of the framework and its application to the analysis and advancement of HEIs.

Ontological Framework for Higher-Education Assessment
The multiple assessment criteria of different agencies are varied in nature. For instance, some might focus more on the research dimension and others less on research but more on teaching and reputation. For instance, neither Shanghai nor THER emphasize the quality of teaching (Marginson & van der Wende, 2007). Further, there is a variance in assessment systems at national and global levels (Çakır, Acartürk, Alaşehir, & Çilingir, `2015). It is challenging to bring these varied criteria under a comprehensive framework to make the metaphorical assessment elephant visible. In this context, an ontological framework can help conceptualize the domain (Gruber, 2008) and systematize its complexity (Cimino, 2006). Similar frameworks have been used in higher-education to envision world-class universities (Ramaprasad, 2011), and to analyse the state of aspiration of HEIs in Karnataka (Hasan, Ramaprasad, & Singai, 2014) and Chile (Coronado, La Paz, Ramaprasad, & Syn, 2015). This method has also been used in the meta-analysis of higher education policy (Ramaprasad, Singai, Hasan, Syn, & Thirumalai, 2016). The method is applied to meta-analyse and synthesize a research domain systemically and systematically .
In the ontology, the terminologies and taxonomies of the domain are systematically organized (Cameron, Ramaprasad, & Syn, 2017), which is explained below. The ontological framework for higher-education assessment ( Figure 1) includes all the elements derived from the following ranking, rating, and accreditation systems criteria: Academic Ranking of World Universities (ARWU); Bloomberg; Leiden; National Assessment and Accreditation Council (NAAC); National Institutional Ranking Framework (NIRF); Quacquarelli Symonds World University Rankings (QS); Times Higher Education World University Rankings (THER); U-Multirank World University Rankings; and, US News & World Report. Using the criteria of all the nine assessment systems, the ontology was constructed through an iterative process amongst authors, as explained by Ramaprasad andSyn (2014, 2015). The ontological framework of higher education assessment constructed is logical and parsimonious. It is logical as it deconstructs and organizes all the assessment criteria; it is parsimonious in that it encapsulates the elements and the logic of assessment in one concise figure.
The Ontology of Higher Education Assessment (Figure 1) has three parts: Inputs to the HEIs system, Processes of the system, and Outputs of the system. Each part has three dimensions, each represented by a column of the framework. Each dimension has been articulated by a one-, two-, or three-level taxonomy of elements. Thus, the Outputs of HEIs is defined by three dimensions, i.e., HEIs Reputation, Research, and Students. They are represented in the three rightmost columns. The HEIs Reputation can be assessed among Peers, Employers, Research Investors, Public, and Competitors.  All the issues affect all the three categories of Inputs, Processes and Outputs of HEIs. For holistic development, they need to be considered together. In the outputs-based issues, the students and research outputs determine the reputation of the institution. A positive reputation would trigger a virtuous cycle of better inputs and improved processes. A symbiotic relationship, as opposed to a siloed one, between research, education, and service processes, will be critical to the success of student, research, and reputational output for the institution. The simultaneously complementary and contradictory societal pressures of equity, expansion, and excellence are reshaping the contours of higher education in the 21 st century. It is affecting and will continue to affect the recruitment of faculty and students and the allocation of other resources, and consequently affect the processes and outputs of higher education institutions.

A Systematic Assessment Framework for Higher-Education Institutions
The above framework may be extended by adding categories or reduced by eliminating categories, refined by adding sub-categories and coarsened by combining categories/subcategories. Thus, one can make and study the assessments of HEIs at different levels of granularity. The present framework encapsulates 7*14*9*10*6*3*9*12*6=10,28,76,480 possible first-order potential components of measures on higher education. It provides a comprehensive picture of the assessment of HEIs.

Validation of the Ontological Framework
The above ontological framework underwent face, semantic and systemic, and external validation following traditional construct of validation (Brennan, Voros, & Brady, 2011;Horn & Lee, 1989). The face validity is high as it made sense on 'its face' when the framework was presented to education experts and formed the basis of a roundtable discussion on higher education. The design of the roundtable discussion was structured, wherein, the participating stakeholders of higher education domain were informed about the discussion with a concept note and framework. The stakeholders during the discussion agreed with the structure, dimensions, sub-dimensions, and the elements of the framework. The discussion which took place in a natural English language was coded onto the framework, so it has a high semantic validation. It is a comprehensive framework deriving its dimensions and elements from nine assessment systems of higher education, so its systemic validity is high. Its external validation is high, as its application was tested by mapping the discussion on higher education by a group of external experts and its dimension and elements are not external to higher education domain. Subsequently, feedback by the stakeholders established the validity through confirming the utility and relevance of the ontological framework in assessing and understanding the HEIs.

Application
The comprehensiveness and logical taxonomies of elements in the framework would assist in applying the framework for systematic identification and addressing the gaps in HEIs. The mapping of data on HEIs onto the framework would give us a visual image and help in ascertaining high or medium or low emphasis given by the HEIs. The theme map based on cluster analysis tool would show the cross-dimensional thematic representation of elements. This would help in identifying the gaps in the functioning of the HEIs and identify the logical pathways to improve their position in assessment systems. The assessment could raise questions and make recommendations on Inputs, Processes and Outputs based issues. For instance, the ontological framework could bring out outputs issues such as institutions training students to be employable, knowledgeable, or both; institutions aligning their education with employment opportunities; institutions balancing the quality and quantity of research, and balancing the market need for qualifications, competencies and vocational skills. Assessment could raise issues related to the processes such as the need for institutions to balance research, education and service; balancing levels of research, and funding different types of researches; mixing face-to-face, online and blended education, and balancing quality and quantity of credit hours for the various degrees. Concerning inputs, the assessment might raise issues related to challenges of equity, expansion and excellence; balancing the regional diversity of faculty members and students; the issue of educating the educator, and cost and quality of education.

Conclusion
The HEIs could use the ontological framework for a comprehensive assessment of their performance, evaluating their database management systems, policies and so on. Its comprehensiveness could assist the management and policymakers in avoiding the blind spots and in facilitating systematic discourse. It can be used for mapping the state of practice in higher education institutions, discover the gaps in aspiration and practice, identify the pathways and formulate a strategy. By making the 'elephant visible', the ontology of higher education assessment will help in framing the policy problems. The framework would prevent the policymakers of and stakeholders in higher education from getting trapped in the policy fallacies as they endeavor to analyze the problem, predict the future, and make decisions.