Factor analysis of research culture: A comparative study of 3-point and 5-point Likert Scales

April M. Obon; Jolly S. Balila; Edwin A. Balila

doi:10.53730/ijhs.v9n1.15375

Authors

April M. Obon
amobon@aup.edu.ph
Adventist University of the Philippines, Silang, Cavite, Philippines
Jolly S. Balila Adventist University of the Philippines, Silang, Cavite, Philippines
Edwin A. Balila Adventist University of the Philippines, Silang, Cavite, Philippines

Keywords:

attitude, competence, dimensionality, model fit, motivation

Abstract

Understanding factors that foster a strong research culture requires a reliable measurement tool. The 3-point and 5-point Likert scales are widely used in surveys to gauge attitudes. However, the choice between these scales may influence data quality, reliability, and interpretation, potentially affecting the actor's analysis. Despite its widespread use, there is limited research comparing the scales’ dimensionality, model fit, and validity in contributing to research culture assessments. Thus, the study aims to compare the effectiveness of 3-point and 5-point Likert scales using confirmatory factor analysis (CFA). This descriptive-comparative study was conducted among 1,139 allied health students in selected sectarian institutions in the Philippines, using purposive and random sampling techniques. The respondents completed a self-constructed research culture questionnaire. Data analysis using SPSS and AMOS confirmed three factors: motivation, attitude, and competence in research with CFA indicating a good fit. But the 5-point Likert scale has better reliability and validity results; it showed higher composite reliability, better average variance extracted (AVE) coefficients, and better construct validity using the Fornell-Larcker criterion, while the 3-point has a better fit using the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC).

Downloads

Download data is not yet available.

References

Abal, F. J. P., Auné, S. E., & Attorresi, H. F. (2018). Variación de la escala Likert en el test de Utilidad de la Matemática. Interacciones. Revista de Avances en Psicología, 4(3), 177-189. DOI: https://doi.org/10.24016/2018.v4n3.134

Adelson, J. L., & McCoach, D. B. (2010). Measuring the mathematical attitudes of elementary students: The effects of a 4-point or 5-point Likert-type scale. Educational and Psychological measurement, 70(5), 796-807. DOI: https://doi.org/10.1177/0013164410366694

Ahadzadeh, A. S., Sharif, S. P., Ong, F. S., & Khong, K. W. (2015). Integrating health belief model and technology acceptance model: an investigation of health-related internet use. Journal of medical Internet research, 17(2), e3564. DOI: https://doi.org/10.2196/jmir.3564

Akerjordet, K., Lode, K., & Severinsson, E. (2012). Clinical nurses' attitudes towards research, management and organisational resources in a university hospital: part 1. Journal of Nursing Management, 20(6), 814-823. DOI: https://doi.org/10.1111/j.1365-2834.2012.01477.x

Alan, Ü., & Kabasakal, K. A. (2020). Effect of number of response options on the psychometric properties of Likert-type scales used with children. Studies in Educational Evaluation, 66, 100895. https://doi.org/10.1016/j.stueduc.2020.100895 DOI: https://doi.org/10.1016/j.stueduc.2020.100895

Alumran, A., Hou, X. Y., Sun, J., Yousef, A. A., & Hurst, C. (2014). Assessing the construct validity and reliability of the parental perception on antibiotics (PAPA) scales. BMC public health, 14, 1-9. DOI: https://doi.org/10.1186/1471-2458-14-73

Aybek, E. C., & Toraman, C. (2022). How many response categories are sufficient for Likert type scales? An empirical study based on the Item Response Theory. International Journal of Assessment Tools in Education, 9(2), 534-547. DOI: https://doi.org/10.21449/ijate.1132931

Barrios, E., & Ulises, D. (2020). Design and validation of the questionnaire" Undergraduate students' attitude towards research”.

Barroso, C. J., & Egar, M. M. (2017). Development and Validation of a Research Culture Index in the Academe. Asia Pacific Journal of Social and Behavioral Sciences, 14, 16-16. DOI: https://doi.org/10.57200/apjsbs.v14i0.87

Böttcher, F., & Thiel, F. (2018). Evaluating research-oriented teaching: a new instrument to assess university students’ research competences. Higher Education, 75, 91-110. DOI: https://doi.org/10.1007/s10734-017-0128-y

Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. routledge. DOI: https://doi.org/10.4324/9780203807644

Canivez, G. L., Watkins, M. W., & McGill, R. J. (2019). Construct validity of the Wechsler Intelligence Scale For Children–Fifth UK Edition: Exploratory and confirmatory factor analyses of the 16 primary and secondary subtests. British Journal of Educational Psychology, 89(2), 195-224. DOI: https://doi.org/10.1111/bjep.12230

Canti, L., Chrzanowska, A., Doglio, M. G., Martina, L., & Van Den Bossche, T. (2021). Research culture: science from bench to society. Biology open, 10(8), bio058919. DOI: https://doi.org/10.1242/bio.058919

Cappelleri, J. C., Lundy, J. J., & Hays, R. D. (2014). Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical therapeutics, 36(5), 648-662. https://doi.org/10.1016/j.clinthera.2014.04.006 DOI: https://doi.org/10.1016/j.clinthera.2014.04.006

Chakrabartty, S. N. (2023). Optimum number of Response categories. Current Psychology, 42(7), 5590-5598. DOI: https://doi.org/10.1007/s12144-021-01866-6

Chen, Q., Sun, M., Tang, S., & Castro, A. R. (2019). Research capacity in nursing: a concept analysis based on a scoping review. BMJ open, 9(11), e032356. DOI: https://doi.org/10.1136/bmjopen-2019-032356

Cheng, W., & Zhang, G. (2012). Empirical Research on the Competency Model of Senior Researchers. In 2012 Second International Conference on Business Computing and Global Informatization (pp. 832-835). IEEE. DOI: https://doi.org/10.1109/BCGIN.2012.222

Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024). Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pacific Journal of Management, 41(2), 745-783. DOI: https://doi.org/10.1007/s10490-023-09871-y

Clark, D. A., & Bowles, R. P. (2018). Model fit and item factor analysis: Overfactoring, underfactoring, and a program to guide interpretation. Multivariate behavioral research, 53(4), 544-558. DOI: https://doi.org/10.1080/00273171.2018.1461058

Coker, J. L., Catlin, D., Ray-Griffith, S., Knight, B., & Stowe, Z. N. (2018). Buprenorphine medication-assisted treatment during pregnancy: an exploratory factor analysis associated with adherence. Drug and alcohol dependence, 192, 146-149. https://doi.org/10.1016/j.drugalcdep.2018.07.042 DOI: https://doi.org/10.1016/j.drugalcdep.2018.07.042

Corchon, S., Watson, R., Arantzamendi, M., & Saracíbar, M. (2010). Design and validation of an instrument to measure nursing research culture: the Nursing Research Questionnaire (NRQ). Journal of Clinical Nursing, 19(1‐2), 217-226. DOI: https://doi.org/10.1111/j.1365-2702.2009.03049.x

Coronel-Santos, M. A., & Ramírez-Montoya, M. S. (2020). An instrument to assess the research culture in formative processes: The validation of the instrument. In Eighth International Conference on Technological Ecosystems for Enhancing Multiculturality (pp. 803-809). DOI: https://doi.org/10.1145/3434780.3436582

Dalmoro, M., & Vieira, K. M. (2013). Dilemas na construção de escalas Tipo Likert: o número de itens e a disposição influenciam nos resultados?. Revista gestão organizacional, 6(3). DOI: https://doi.org/10.22277/rgo.v6i3.1386

Davidson, Z. E., & Palermo, C. (2015). Developing research competence in undergraduate students through hands on learning. Journal of Biomedical Education, 2015(1), 306380. DOI: https://doi.org/10.1155/2015/306380

De Ayala, R. J. (2022). The theory and practice of item response theory. The Guilford Press.

DeCleene Huber, K. E., Nichols, A., Bowman, K., Hershberger, J., Marquis, J., Murphy, T., ... & Sanders, C. (2015). The correlation between confidence and knowledge of evidence-based practice among occupational therapy students. The Open Journal of Occupational Therapy, 3(1), 5. DOI: https://doi.org/10.15453/2168-6408.1142

Deicke, W., Gess, C., & Rueß, J. (2014). Increasing Students' Research Interests Through Research-Based Learning at Humboldt University. Council on Undergraduate Research Quarterly, 35(1).

Dennis, B. K. (2014). Understanding participant experiences: Reflections of a novice research participant. International Journal of Qualitative Methods, 13(1), 395-410. DOI: https://doi.org/10.1177/160940691401300121

Desbrow, B., Leveritt, M., Palmer, M., & Hughes, R. (2014). Evaluation of a curriculum initiative designed to enhance the research training of dietetics graduates. Nutrition & Dietetics, 71(1), 57-63. DOI: https://doi.org/10.1111/1747-0080.12039

Esteban, R. F. C., Mamani-Benito, O., Huancahuire-Vega, S., & Lingan, S. K. (2022, April). Design and validation of a research motivation scale for Peruvian university students (MoINV-U). In Frontiers in Education (Vol. 7, p. 791102). Frontiers Media SA. DOI: https://doi.org/10.3389/feduc.2022.791102

Fang, J., Fleck, M. P., Green, A., McVilly, K., Hao, Y., Tan, W., ... & Power, M. (2011). The response scale for the intellectual disability module of the WHOQOL: 5‐point or 3‐point?. Journal of Intellectual Disability Research, 55(6), 537-549. DOI: https://doi.org/10.1111/j.1365-2788.2011.01401.x

Finch, W. H. (2020). Using fit statistic differences to determine the optimal number of factors to retain in an exploratory factor analysis. Educational and psychological measurement, 80(2), 217-241. DOI: https://doi.org/10.1177/0013164419865769

Flinkman, M., Leino‐Kilpi, H., Numminen, O., Jeon, Y., Kuokkanen, L., & Meretoja, R. (2017). Nurse Competence Scale: a systematic and psychometric review. Journal of Advanced Nursing, 73(5), 1035-1050. DOI: https://doi.org/10.1111/jan.13183

Fouad, S., El Araby, S., Abed, R. A. R. O., Hefny, M., & Fouad, M. (2020). Using Item Response Theory (IRT) to Assess Psychometric Properties of Undergraduate Clinical Education Environment Measure (UCEEM) among Medical Students at the Faculty of Medicine, Suez Canal University. Education in Medicine Journal, 12(1). DOI: https://doi.org/10.21315/eimj2020.12.1.3

Garrido, L. E., Abad, F. J., & Ponsoda, V. (2016). Are fit indices really fit to estimate the number of factors with categorical variables? Some cautionary findings via Monte Carlo simulation. Psychological methods, 21(1), 93. DOI: https://doi.org/10.1037/met0000064

Gefen, D., Straub, D., & Boudreau, M. C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communications of the association for information systems, 4(1), 7. DOI: https://doi.org/10.17705/1CAIS.00407

Gerish, K., & Lacey, A. (2010). The research process in nursing.

González-Betanzos, F., Leenen, I., Lira-Mandujano, J., & Vega-Valero, Z. (2012). El efecto del número de opciones de respuesta sobre las propiedades psicométricas de la medida de estrés con un instrumento aplicado a niños. Revista Evaluar, 12(1). DOI: https://doi.org/10.35670/1667-4545.v12.n1.4694

Grande, R. A. N., Berdida, D. J. E., Villagracia, H. N., Ablao, J. N., & Garcia, P. R. B. (2022). Multi-university assessment of Biggs's constructive alignment as an index of nursing research competencies among Saudi students. Teaching and Learning in Nursing, 17(1), 68-76. https://doi.org/10.1016/j.teln.2021.09.004 DOI: https://doi.org/10.1016/j.teln.2021.09.004

Greene, A. L., Eaton, N. R., Forbes, M. K., Fried, E. I., Watts, A. L., Kotov, R., & Krueger, R. F. (2022). Model fit is a fallible indicator of model quality in quantitative psychopathology research: A reply to Bader and Moshagen. DOI: https://doi.org/10.31234/osf.io/e7pcd

Gullick, J. G., & West, S. H. (2016). Building research capacity and productivity among advanced practice nurses: an evaluation of the Community of Practice model. Journal of advanced nursing, 72(3), 605-619. DOI: https://doi.org/10.1111/jan.12850

Hair Jr., J. F. Hult, G. T., Ringle, C. M., Sarstedt M., Danks, N. P. & Ray, S. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: A workbook. Springer. DOI: https://doi.org/10.1007/978-3-030-80519-7

Han, J., & Yin, H. (2016). Teacher motivation: Definition, research development and implications for teachers. Cogent education, 3(1), 1217819. DOI: https://doi.org/10.1080/2331186X.2016.1217819

Helgøy, K. V., Smeby, J. C., Bonsaksen, T., & Rydland Olsen, N. (2020). based occupational therapy education: an exploration of students’ and faculty members’ experiences and perceptions. PLoS One, 15(12), e0243544. DOI: https://doi.org/10.1371/journal.pone.0243544

Hosseini, M., & Bahrami, V. (2022). Adaptation and validation of the research motivation scale for language teachers. The Journal of Experimental Education, 90(1), 229-248. DOI: https://doi.org/10.1080/00220973.2019.1709036

Hu, B. Y., Zhou, Y., Li, K., & Killingsworth Roberts, S. (2014). Examining program quality disparities between urban and rural kindergartens in China: Evidence from Zhejiang. Journal of Research in Childhood Education, 28(4), 461-483. DOI: https://doi.org/10.1080/02568543.2014.944720

Irwing, P., Booth, T., & Hughes, D. J. (Eds.). (2018). The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118489772

Ivanenko, N. A., Mustafina, G. M., Sagitova, V. R., Akhmetzyanov, I. G., Khazratova, F. V., Salakhova, I. T., & Mokeyeva, E. V. (2015). Basic Components of Developing Teachers' Research Competence as a Condition to Improve Their Competitveness. Rev. Eur. Stud., 7, 221. DOI: https://doi.org/10.5539/res.v7n4p221

Jansson B., Olsen, K., Erixon, R. M., & Archer, T. (2020). Predictive validity estimations for jobmatchtalent instrument. International Journal of Psychology and Behavior Analysis, 6(165), 1-3. DOI: https://doi.org/10.15344/2455-3867/2020/165

Jayachandran, J., & Chandrasenan, D. (2021). Institutional Research Culture Scale (IRCS): Development and Validation in the context of Universities in Kerala, India. In 8 Th International Conference on Research in Behavioral and Social Science (pp. 47-53). DOI: https://doi.org/10.33422/8th.icrbs.2021.07.72

Jeong, H. J., & Lee, W. C. (2016). The level of collapse we are allowed: Comparison of different response scales in Safety Attitudes Questionnaire. Biom Biostat Int J, 4(4), 00100. DOI: https://doi.org/10.15406/bbij.2016.04.00100

Kakupa, P., & Xue, H. (2019). Students' Attitudes towards Research: A Study of Graduate Education Students at a Chinese Normal University. Educational Process: International Journal, 8(2), 97-110. DOI: https://doi.org/10.22521/edupij.2019.82.1

Kandell, G. V. (2019). Medical Students' Understanding of the Nature of Science.

Kean, J., Bisson, E. F., Brodke, D. S., Biber, J., & Gross, P. H. (2018). An introduction to item response theory and Rasch analysis: Application using the eating assessment tool (EAT-10). Brain Impairment, 19(1), 91-102. DOI: https://doi.org/10.1017/BrImp.2017.31

Kemp, T., Finlayson, L., Chan, J., Lackey, G., Richards, D., Rupnik, C., ... & Low, S. (2020). Exploring the research culture in the health information management profession in Australia. Health Information & Libraries Journal, 37(1), 60-69. DOI: https://doi.org/10.1111/hir.12281

Kline, R. B. (2023). Principles and practice of structural equation modeling. Guilford publications.

Kyvik, S., Vågan, A., Prøitz, T. S., & Aamodt, P. O. (2014). based education in undergraduate professional programmes. In From vocational to professional education (pp. 105-123). Routledge. DOI: https://doi.org/10.4324/9781315757131-7

Lange, T., Kopkow, C., Lützner, J., Günther, K. P., Gravius, S., Scharf, H. P., ... & Schmitt, J. (2020). Comparison of different rating scales for the use in Delphi studies: different scales lead to different consensus and show different test-retest reliability. BMC medical research methodology, 20, 1-11. DOI: https://doi.org/10.1186/s12874-020-0912-8

Loginov, Y., & Kovalev, I. (2017). Formation of research competence in university project-oriented training. In SHS Web of Conferences (Vol. 37, p. 01027). EDP Sciences. DOI: https://doi.org/10.1051/shsconf/20173701027

Lopes, F. O. D. A., Hurtado-Puerto, A. M., Moreno, H., Fregni, F., Falcão, D. P., & Amorim, R. F. B. D. (2016). Creating a research idea: Steps and challenges. Geriatrics, Gerontology and Aging, 10(3), 118-125. DOI: https://doi.org/10.5327/Z2447-211520161600034

Marrs, S. A., Quesada-Pallarès, C., Nicolai, K. D., Severson-Irby, E. A., & Martínez-Fernández, J. R. (2022). Measuring perceived research competence of junior researchers. Frontiers in psychology, 13, 834843. DOI: https://doi.org/10.3389/fpsyg.2022.834843

Matus, J., Tearne, J. E., Blyth, K., Coates, S., Pearson, S., & Cavalheri, V. (2020). An evaluation of research capacity and culture in a sample of Western Australian Allied Health professionals. Tasman Medical Journal, 3(1), 23-29.

McKee, G., Codd, M., Dempsey, O., Gallagher, P., & Comiskey, C. (2017). Describing the implementation of an innovative intervention and evaluating its effectiveness in increasing research capacity of advanced clinical nurses: using the consolidated framework for implementation research. BMC nursing, 16, 1-13. DOI: https://doi.org/10.1186/s12912-017-0214-6

Mellenbergh, G. J., & Mellenbergh, G. J. (2019). Test dimensionality. Counteracting Methodological Errors in Behavioral Research, 135-156. DOI: https://doi.org/10.1007/978-3-030-12272-0_10

Miyejav, I., Otgonbaatar, K., & Ochir, P. (2023). Development and validation of a competence model for educational researcher in the Mongolian context. Int J Eval & Res Educ, 12(2), 601-612. DOI: https://doi.org/10.11591/ijere.v12i2.24319

Moore, J., Crozier, K., & Kite, K. (2012). An action research approach for developing research and innovation in nursing and midwifery practice: Building research capacity in one NHS foundation trust. Nurse education today, 32(1), 39-45. https://doi.org/10.1016/j.nedt.2011.01.014 DOI: https://doi.org/10.1016/j.nedt.2011.01.014

Moss, T. P., Lawson, V., & White, P. (2015). Identification of the underlying factor structure of the Derriford Appearance Scale 24. PeerJ, 3, e1070. DOI: https://doi.org/10.7717/peerj.1070

Muhamad Safiih, L. & Nor Azreen, M. A. (2016). Confirmatory factor analysis approach: A case study of mathematics students’ achievement in TIMSS. Malaysian Journal of Mathematical Sciences, 10(S), 41-51.

Nicholson, P., Griffin, P., Gillis, S., Wu, M., & Dunning, T. (2013). Measuring nursing competencies in the operating theatre: Instrument development and psychometric analysis using Item Response Theory. Nurse education today, 33(9), 1088-1093. https://doi.org/10.1016/j.nedt.2012.04.008 DOI: https://doi.org/10.1016/j.nedt.2012.04.008

Nilsson Kajermo, K., Alinaghizadeh, H., Falk, U., Wändell, P., & Törnkvist, L. (2014). Psychometric evaluation of a questionnaire and primary healthcare nurses' attitudes towards research and use of research findings. Scandinavian Journal of Caring Sciences, 28(1), 173-185. DOI: https://doi.org/10.1111/scs.12037

Noguez, J., & Neri, L. (2019). based learning: a case study for engineering students. International Journal on Interactive Design and Manufacturing (IJIDeM), 13(4), 1283-1295. DOI: https://doi.org/10.1007/s12008-019-00570-x

Nyangau, J. Z. (2018). Motivations of faculty engagement in internationalization: An agenda for future research. In FIRE: Forum for International Research in Education (Vol. 4, No. 3). DOI: https://doi.org/10.32865/fire2018431

Ommering, B. W., Wijnen-Meijer, M., Dolmans, D. H., Dekker, F. W., & van Blankenstein, F. M. (2020). Promoting positive perceptions of and motivation for research among undergraduate medical students to stimulate future research involvement: a grounded theory study. BMC medical education, 20, 1-12. DOI: https://doi.org/10.1186/s12909-020-02112-6

Paek, I., & Cole, K. (2019). Using R for item response theory model applications. Routledge. DOI: https://doi.org/10.4324/9781351008167

Pahlevan Sharif, S., Naghavi, N., & Sharif Nia, H. (2022). Concerns regarding the validity of nutrition self-efficacy questionnaire among Iranian elderly population. Journal of Health, Population and Nutrition, 41(1), 3. DOI: https://doi.org/10.1186/s41043-022-00282-1

Palermo, C., Fitzgerald, A. & Nestel, D. (2014). Student and faculty perceptions of research in nutrition and dietetics: A qualitative investigation. Critical Public Health, 2(2), 34–44. DOI: https://doi.org/10.32920/cd.v2i1.777

Pan, Y. H., Cheng, J. L., Kou, L. H., & Sun, Y. M. (2011). Investigation on scientific research capacity of nursing staff in Shanxi Province. China Nursing Research 25(22), 1989-1992.

Parahoo, K. (2014). Nursing research: principles, process and issues. Bloomsbury Publishing. DOI: https://doi.org/10.1007/978-1-137-28127-2

Polit, D. F., & Beck, C. T. (2008). Nursing research: Generating and assessing evidence for nursing practice. Lippincott Williams & Wilkins.

Preacher, K. J., Zhang, G., Kim, C., & Mels, G. (2013). Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate behavioral research, 48(1), 28-56. DOI: https://doi.org/10.1080/00273171.2012.710386

Qiu, C., Feng, X., Reinhardt, J. D., & Li, J. (2019). Development and psychometric testing of the Research Competency Scale for Nursing Students: An instrument design study. Nurse education today, 79, 198-203. https://doi.org/10.1016/j.nedt.2019.05.039 DOI: https://doi.org/10.1016/j.nedt.2019.05.039

Rahmatpour, P., Peyrovi, H., & Sharif Nia, H. (2021). Development and psychometric evaluation of postgraduate nursing student academic satisfaction scale. Nursing Open, 8(3), 1145-1156. DOI: https://doi.org/10.1002/nop2.727

Robinson, G. F., Moore, C. G., McTigue, K. M., Rubio, D. M., & Kapoor, W. N. (2015). Assessing competencies in a master of science in clinical research program: the comprehensive competency review. Clinical and Translational Science, 8(6), 770-775. DOI: https://doi.org/10.1111/cts.12322

Ryan, R. M. (2012). Motivation and the organization of human behavior: Three reasons for the reemergence of a field. The Oxford handbook of human motivation, 3-10. DOI: https://doi.org/10.1093/oxfordhb/9780195399820.013.0001

Schweizer, K. (2014). On the ways of investigating the discriminant validity of a scale in giving special emphasis to estimation problems when investigating multitrait-multimethod matrices. Psychological Test and Assessment Modeling, 56(1), 45.

Shi, D., DiStefano, C., Maydeu-Olivares, A., & Lee, T. (2022). Evaluating SEM model fit with small degrees of freedom. Multivariate behavioral research, 57(2-3), 179-207. DOI: https://doi.org/10.1080/00273171.2020.1868965

Shrestha, N. (2021). Factor analysis as a tool for survey analysis. American journal of Applied Mathematics and statistics, 9(1), 4-11. DOI: https://doi.org/10.12691/ajams-9-1-2

Sismondo, G. (2020) An Assessment on the level of research competencies of grade 12 senior high school students in a Parochial School. The Asian Conference on Education & International Development. DOI: https://doi.org/10.22492/issn.2189-101X.2020.6

Smith, S. A. (2012). Nurse competence: A concept analysis. International journal of nursing knowledge, 23(3), 172-182. DOI: https://doi.org/10.1111/j.2047-3095.2012.01225.x

Smyth, L., Davila, F., Sloan, T., Rykers, E., Backwell, S., & Jones, S. B. (2016). How science really works: the student experience of research-led education. Higher Education, 72, 191-207. DOI: https://doi.org/10.1007/s10734-015-9945-z

Solinger, O. N., Heusinkveld, S., & Cornelissen, J. P. (2024). Redefining concepts to build theory: A repertoire for conceptual innovation. Human Resource Management Review, 34(1), 100988. https://doi.org/10.1016/j.hrmr.2023.100988 DOI: https://doi.org/10.1016/j.hrmr.2023.100988

Sözer, E., & Kahraman, N. (2021). Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models. Journal of Measurement and Evaluation in Education and Psychology, 12(2), 129-146. DOI: https://doi.org/10.21031/epod.819927

Taherdoost, H. (2019). What is the best response scale for survey and questionnaire design; review of different lengths of rating scale/attitude scale/Likert scale. Hamed Taherdoost, 1-10.

Tavakol, M., & Wetzel, A. (2020). Factor Analysis: a means for theory and instrument development in support of construct validity. International journal of medical education, 11, 245. DOI: https://doi.org/10.5116/ijme.5f96.0f4a

Thomas, A., Han, L., Osler, B. P., Turnbull, E. A., & Douglas, E. (2017). Students’ attitudes and perceptions of teaching and assessment of evidence-based practice in an occupational therapy professional Master’s curriculum: a mixed methods study. BMC Medical Education, 17, 1-11. DOI: https://doi.org/10.1186/s12909-017-0895-2

Thorpe, G. L., & Favia, A. (2012). Data Analysis Using Item Response Theory Methodology: An Introduction to Selected Programs and Applications.

Van Blankenstein, F. M., Saab, N., Van der Rijst, R. M., Danel, M. S., Bakker-van den Berg, A. S., & Van den Broek, P. W. (2019). How do self-efficacy beliefs for academic writing and collaboration and intrinsic motivation for academic writing and research develop during an undergraduate research project?. Educational Studies, 45(2), 209-225. DOI: https://doi.org/10.1080/03055698.2018.1446326

Van Nunen, K., Li, J., Reniers, G., & Ponnet, K. (2018). Bibliometric analysis of safety culture research. Safety science, 108, 248-258. https://doi.org/10.1016/j.ssci.2017.08.011 DOI: https://doi.org/10.1016/j.ssci.2017.08.011

Vasetskaya, N. O., Glukhov, V. V., & Burdakov, S. F. (2018). The elaboration of the model of competences of the research and teaching university staff. In 2018 XVII Russian Scientific and Practical Conference on Planning and Teaching Engineering Staff for the Industrial and Economic Complex of the Region (PTES) (pp. 98-101). IEEE. DOI: https://doi.org/10.1109/PTES.2018.8604215

Vereijken, M. W., van der Rijst, R. M., van Driel, J. H., & Dekker, F. W. (2018). Student learning outcomes, perceptions and beliefs in the context of strengthening research integration into the first year of medical school. Advances in Health Sciences Education, 23, 371-385. DOI: https://doi.org/10.1007/s10459-017-9803-0

Vuković, M., Dašić, D., & Vuković, A. (2024). Initial Steps In Preparing A Scientific Concept Outline–Formulating The Problem And Determining The Research Subject. Sport mediji i biznis, 10(1), 75-90. DOI: https://doi.org/10.58984/smb2401075v

Whelan, K., & Markless, S. (2012). Factors that influence research involvement among registered dietitians working as university faculty: a qualitative interview study. Journal of the Academy of Nutrition and Dietetics, 112(7), 1021-1028. https://doi.org/10.1016/j.jand.2012.03.002 DOI: https://doi.org/10.1016/j.jand.2012.03.002

Zanon, C., Hutz, C. S., Yoo, H. H., & Hambleton, R. K. (2016). An application of item response theory to psychological test development. Psicologia: Reflexão e Crítica, 29. DOI: https://doi.org/10.1186/s41155-016-0040-x

Zhao, L., Han, S., Lu, B., Wang, M., & Zhao, X., 2011. Study on nursing research capacity of nursing undergraduates and its influencing factors. J. Nurs. Sci. 26(4), 64–66.