Validation of a Rubric to Evaluate Universities’ Practices on Student Diversity and Inclusion in Higher Education
Abstract
Purpose: Students are considered the main beneficiaries of education, and diversity and inclusion are integral components of the education system. Countries have made several commitments to strengthen student diversity and inclusion in higher education as part of their transition towards Sustainable Development Goal 4 (SDG4). This highlights the need develop a valid tool to assess the institutionalisation of student diversity and inclusion in higher education. Methods: This study describes the use of a rubric of 20-indicators on diversity and inclusion, each with three categories, to assess the content. The rubric was developed by conducting a literature review on these topics. Fifteen experts were consulted for their judgments and ratings. The experts not only ranked the items in order of importance but also suggested revisions. Findings: Based on their feedback, content validity ratio (CVR) was calculated, resulting in 79 validated items. Improvement/Novelty: The validation process helped to determine the validity of the rubric and allowed the identification of items that required modifications. The validated rubric will assist the universities in self-assessing their progress and finding out areas of attention related to student diversity and inclusion in higher education.
Keywords: Evaluation, Diversity, Inclusion, Students, Higher education, Rubric, Validation.
Full Text:
PDFReferences
EUROPEAN UNIVERSITY ASSOCIATION. Universities’ Strategies and Approaches towards Diversity, Equity and Inclusion: Examples from across Europe. In: ANNA-LENA CLAEYS-KULIK and THOMAS EKMAN JØRGENSEN (eds.). EUA, Belgium, 2019. https://www.eua.eu/downloads/publications/universities-39-strategies-and-approaches-towards-diversity-equity-and-inclusion.pdf
EUROPEAN UNION. Promoting citizenship and the common values of freedom, tolerance and non-discrimination through education. European Union, France, 2015. https://op.europa.eu/en/publication-detail/-/publication/ebbab0bb-ef2f-11e5-8529-01aa75ed71a1
EUROPEAN HIGHER EDUCATION ASSOCIATION. Yerevan Communiqué: EHEA ministerial meeting, 2015, Yerevan, Armenia. https://ehea.info/media.ehea.info/file/2015_ Yerevan/70/7/YerevanCommuniqueFinal_613707.pdf
BOLGER, M. What’s the difference between Diversity, Inclusion and Equity? 2017, https://medium.com/the-ga-index/whats-the-difference-between-diversity-inclusion-and-equity-f012abda3894
MESSIOU, K., BUI, L. T., AINSCOW, M., GASTEIGER-KLICPERA, B., BEŠIĆ, E., PALECZEK, L., HEDEGAARD-SØRENSEN, L., ULVSETH, H., VITORINO, T., SANTOS, J., SIMON, C., SANDOVAL, M., and ECHEITA, G. Student diversity and student voice conceptualisations in five European countries: Implications for including all students in schools. European Educational Research Journal, 2022, 21(2): 355- 376. https://doi.org/10.1177/1474904120953241.
CHANG, M. Perservation or Transformation: Where's the Real Educational Discourse on Diversity? The Review of Higher Education, 2002, 25(2): 125-140. https://doi.org/10.1353/rhe.2002.0003/
AHMAD, S. On being included: Racism and diversity in institutional life. Duke University Press, Durham, 2012. https://doi.org/10.1215/9780822395324
AMERICAN ASSOCIATION OF COLLEGES AND UNIVERSITIES. Committing to equity and inclusive excellence. A Campus Guide for Self-Study and Planning. Association of American Colleges and Universities, Washington, 2015. https://tinyurl.com/y8tq8bbz
BUENESTADO-FERNÁNDEZ M, ÁLVAREZ-CASTILLO JL, GONZÁLEZ-GONZÁLEZ H, and ESPINO-DÍAZ L. Evaluating the institutionalisation of diversity outreach in top universities worldwide. PLoS One, 2019, 14(7):e0219525. https://doi.org/10.1371/journal.pone.0219525
DEPARTMENT OF EDUCATION. Advancing Diversity and Inclusion in Higher Education. Key Data Highlights Focusing on Race and Ethnicity and Promising Practices, 2016. https://www2.ed.gov/rschstat/research/pubs/advancing-diversity-inclusion.pdf0Ahttp://files/519/advancing-diversity-inclusion.pdf
FERREIRA, C., VIDAL, J. and VIEIRA, M. J. Student guidance and attention to diversity in the processes of quality assurance in higher education. European Journal of Education, 2014, 49(4): 575-589. https://doi.org/10.1111/ejed.12098
GAUSE, C. P., DENNISON, S., and PERRIN, D. H. Equity, Inclusiveness, and Diversifying the Faculty: Transforming the University in the 21st Century. Quest, 2019, 62(1): 61–75. https://doi.org/10.1080/00336297.2010.10483632
HITCH, D., MACFARLANE, S., and NIHILL, C. Inclusive pedagogy in Australian universities: A review of current policies and professional development activities. International journal of the first year in higher education, 2015, 6(1): 135-145. https://doi.org/10.5204/intjfyhe.v6i1.254
LLORENT, V. J., ZYCH, I., and VARO-MILLÁN, J. C. (2020). University academic personnel’s vision of inclusive education in Spanish universities (Visión del profesorado sobre la educación inclusiva en la universidad en España). Culture and Education, 2020, 32(1): 147-181. https://doi.org/10.1080/11356405.2019.1705
MAY, H. and BRIDGER, K. Developing and embedding inclusive policy and practice in higher education. The Higher Education Academy, York, 2010. https://www.advance-he.ac.uk/knowledge-hub/developing-and-embedding-inclusive-policy-and-practice-higher-education
MORIÑA, A. Inclusive education in higher education: Challenges and opportunities. European Journal of Special Needs Education, 2017, 32(1): 3-17. https://doi.org/10.1080/08856257.2016.1254964
MORIÑA, A., SANDOVAL, M., and CARNERERO, F. Higher education inclusivity: when the disability enriches the university. Higher Education Research & Development, 2020, 39(6): 1202–1216. https://doi.org/10.1080/07294360.2020.1712676
NEW ENGLAND RESOURCE CENTER FOR HIGHER EDUCATION (NERCHE). Self-Assessment Rubric for the Institutionalization of Diversity, Equity and Inclusion in Higher Education. New England Resource Center for Higher Education, 2016. https://www.wpi.edu/sites/default/files/Project_Inclusion_NERCHE_Rubric-Self-Assessment-2016.pdf
Sandoval, M., Simón, C. y Márquez, C. Inclusive or exclusionary classrooms? Barriers to learning and participation in university settings [¿Aulas inclusivas o excluyentes? Barreras para el aprendizaje y la participación en contextos universitarios]. Revista Complutense de Educación, 2019, 30(1): 261-276. https://doi.org/10.5209/RCED.57266
RAMOS SANTANA, G., PÉREZ CARBONELL, A., CHIVA SANCHIS, I., and MORAL MORA, A. Validation of a scale of attention to diversity for university teachers. Educación XX1, 2021, 24(2): 121-142. https://doi.org/10.5944/educXX1.28518
TOBOSO, M., FELTERO, R., and APARICIO, M. Inclusive education and education for inclusion: transformations analyzed from the ideas of justice and universal accessibility (Educación inclusiva y educación para la inclusión: transformaciones pendientes analizadas desde las ideas de justicia y accesibilidad universal en Espana). In: M. LAGARRETA, I. CEBALLOS, and S. CARRASCAL (eds.). Educación y transformación social y cultural, 2019, (pp.185-196). Universitas. https://dialnet.unirioja.es/servlet/articulo?codigo=6933638
ONU (2018). Informe de la investigación relacionada con España bajo el artículo 6 del Protocolo Facultativo (CRPD/C/20/3). https://bit.ly/3t4hu07
ECHEITA, G. Inclusive education: Smiles and tears (Educación inclusive: Sonrisas y lágrimas en España). Aula Abierta, 2017, 46(2): 17-24. https://doi.org/10.17811/rifie.46.2.2017.17-24
MCNAIR, T. B. The time is now: Committing to equity and inclusive excellence. Diversity & Democracy, 2016, 19(1): 4-7. https://dgmg81phhvh63.cloudfront.net/content/user-photos/Publications/Archives/Diversity-Democracy/DD_19-1_WI16.pdf
ESCOBAR-PÉREZ, J. and CUERVO-MARTÍNEZ, A. Content validity and expert judgment: An approach to their use. Advances in Measurement (Validez de contenido y juicio de expertos: Una aproximación a su utilización en España). Avances en Medición, 2008, 6, 27–36. https://www.humanas.unal.edu.co/lab_psicometria/application/files/9416/0463/ 3548/Vol_6._Articulo3_Juicio_de_expertos_27-36.pdf
ROSA GÓMEZ, A., MEZA CANO, J. M., and MIRANDA DÍAZ, G. A. Validation of a Rubric to Evaluate Open Educational Resources for Learning. Behavioral Sciences, 2019, 9(12): 126. https://doi.org/10.3390/bs9120126
LAWSHE, C. H. A quantitative approach to content validity. Personnel Psychology, 1975, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
LÓPEZ OBANDO, P. (2009). Construction and validation of a test to measure mathematical knowledge (Construccion y validacion de una prueba para medir conocimientos matematicos en España). Horiz Pedagógico, 2009, 11(1): 29-37. https://horizontespedagogicos.ibero. edu.co/article/view/333/299
MISTRY, H. S. Construction and standardization of an inclusive education teaching aptitude test. Postdoctoral Research Report, The Maharaja Sayajirao University of Baroda, 2018.
ALMANASREH, E., MOLES, R., and CHEN, T. F. Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 2019, 15(2), 214–21. https://doi.org/10.1016/j.sapharm.2018.03.066
BENSIMON, E. M. The diversity scorecard: A learning approach to institutional change. Change: The Magazine of Higher Education, 2004, 36(1): 44–52. https://doi.org/10.1080/00091380409605083
WILLIAMNS, D. A., JOSEPH, B. B. and SHEREDICK, A. M. Toward a Model of Inclusive Excellence and Change in Postsecondary Institutions. Association of American Colleges and Universities, Washington, DC, 2005. https://operations.du.edu/sites/default/files/2020-04/model-of-inclusive-excellence.pdf
FURCO, A. Self-assessment rubric for the institutionalization of service-learning in higher education. Service Learning, General, 1999, 127. https://digitalcommons.unomaha. edu/slceslgen/127/
DAVIS, L. L. Instrument review: getting the most from a panel of experts. Applied Nursing Research, 1992, 5(4), 194–7. https://doi.org/10.1016/s0897-1897(05)80008-4
LYNN, M. R. Determination and quantification of content validity. Nursing Research, 1986, 35(6), 382–386. https://doi.org/10.1097/00006199-198611000-00017
CLAEYS-KULIK, A-L., JORGENSEN, T. E. & STOBER, H. Diversity, equity and inclusion in European higher education institutions: Results from the INVITED project. European University Association, 2019. https://www.eua.eu/images/web_diversity_equity_and_inclusion_in_european_higher_education_institutions.pdf
MISTRY, H. S. and SANCHEZ-GOMEZ, M. C. Development and validation of a rubric to assess universities policies and practices on student diversity and inclusion in higher education [pp. 131-148]. In Tankersley, M. & Hebebrand, B. (Eds.). Advancing Special Education: The Kent State University Florence International Summit on Learning and Behavioral Health, 2025. Italy: Viella. https://www.viella.it/libro/9791254699881
YUAN, M. and RECKER, M.M. Does audience matter? Comparing teachers’ and non-teachers’ application and perception of quality rubrics for evaluating open educational resources. Educational Technology Research and Development, 2018, 6(5): 1-23. https://doi.org/10.1007/s11423-018-9605-y
ZAMANZADEH, V., GHAHRAMANIAN, A., RASSOULI, M., ABBASZADEH, A., ALAVI-MAJD, H. & NIKANFAR, A. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. Journal of Caring Science, 2015, 4(2): 165-178. https://doi.org/10.15171/jcs.2015.017
TOURON, J., MARTIN, D., NAVARO, A., PRADAS, S., & INVIGO, V. Construct validation of a questionnaire to measure teachers’ digital competence (TDC). Revista Española De Pedagogía, 2018, 76(269): 25–54. https://doi.org.10.22550/rep76-1-2018-10.
Brancaccio-Taras L., Awong-Taylor J., Linden, M., Marley K., Reiness, C. G., & Uzman, J. A. The PULSE Diversity Equity and Inclusion (DEI) Rubric: a Tool To Help Assess Departmental DEI Efforts. Journal of Microbiology & Biology Education, 2022, 23(3):e00057-22. https://doi.org/10.1128/jmbe.00057-22
Refbacks
- There are currently no refbacks.
