Purpose:The adequacy of research performance of universities or research institutes have often been evaluated and understood in two axes:“quantity”(i.e.size or volume)and“quality”(i.e.what we define here as a meas...Purpose:The adequacy of research performance of universities or research institutes have often been evaluated and understood in two axes:“quantity”(i.e.size or volume)and“quality”(i.e.what we define here as a measure of excellence that is considered theoretically independent of size or volume,such as clarity in diamond grading).The purpose of this article is,however,to introduce a third construct named“substantiality”(“ATSUMI”in Japanese)of research performance and to demonstrate its importance in evaluating/understanding research universities.Design/methodology/approach:We take a two-step approach to demonstrate the effectiveness of the proposed construct by showing that(1)some characteristics of research universities are not well captured by the conventional constructs(“quantity”and“quality”)-based indicators,and(2)the“substantiality”indicators can capture them.Furthermore,by suggesting that“substantiality”indicators appear linked to the reputation that appeared in university reputation rankings by simple statistical analysis,we reveal additional benefits of the construct.Findings:We propose a new construct named“substantiality”for measuring research performance.We show that indicators based on“substantiality”can capture important characteristics of research institutes.“Substantiality”indicators demonstrate their“predictive powers”on research reputation.Research limitations:The concept of“substantiality”originated from IGO game;therefore the ease/difficulty of accepting the concept is culturally dependent.In other words,while it is easily accepted by people from Japan and other East Asian countries and regions,it might be difficult for researchers from other cultural regions to accept it.Practical implications:There is no simple solution to the challenge of evaluating research universities’research performance.It is vital to combine different types of indicators to understand the excellence of research institutes.Substantiality indicators could be part of such a combination of indicators.Originality/value:The authors propose a new construct named substantiality for measuring research performance.They show that indicators based on this construct can capture the important characteristics of research institutes.展开更多
Taking the scholarly activities of 73 doctoral program mentors working at the Chinese Academy of Medical Sciences & Peking Union Medical College(the CAMS &PUMC) as a sample of our investigative survey, we trie...Taking the scholarly activities of 73 doctoral program mentors working at the Chinese Academy of Medical Sciences & Peking Union Medical College(the CAMS &PUMC) as a sample of our investigative survey, we tried using such statistical methods as the analysis of variance(ANOVA), factor analysis and correlation analysis to compare the different characteristics of scholarship assessment of Chinese medical scholars as exhibited in their published papers in domestic and foreign journals. Our research findings show that citations per paper and A-index are more suitable for assessing the highly accomplished senior Chinese medical professionals(e.g. academicians) for their domestic and international scholarship attainment. In contrast, the m-quotient is not deemed appropriate to assess their academic influence both at home and abroad. Upon our further analysis of 6 evaluative indicators, we noticed that these indicators might be applied in two different aspects: One is from the viewpoint of Chinese scholars' academic influence at home, which has been evaluated mainly from the perspective of 'total' amount and 'average' amount of both publications and citations. The other is from their academic impact embodied by the means of documents retrieved from the Web of Science, which is mainly assessed from the two viewpoints of publications and citations. It is suggested that the accumulated time-length of a given scholar's active engagement in professional practice in a specific subject area be taken into consideration while assessing a researcher's performance at home and abroad.展开更多
基金The authors would like to thank Elsevier and its Japanese team,especially,Kana Takasaka,for their bibliometric data provision and relevant support.This research was partly supported by JSPS KAKENHI(16H06580,17K01173)JST/RISTEX research funding program“Science of Science,Technology and Innovation Policy”(JPMJRX19B3).
文摘Purpose:The adequacy of research performance of universities or research institutes have often been evaluated and understood in two axes:“quantity”(i.e.size or volume)and“quality”(i.e.what we define here as a measure of excellence that is considered theoretically independent of size or volume,such as clarity in diamond grading).The purpose of this article is,however,to introduce a third construct named“substantiality”(“ATSUMI”in Japanese)of research performance and to demonstrate its importance in evaluating/understanding research universities.Design/methodology/approach:We take a two-step approach to demonstrate the effectiveness of the proposed construct by showing that(1)some characteristics of research universities are not well captured by the conventional constructs(“quantity”and“quality”)-based indicators,and(2)the“substantiality”indicators can capture them.Furthermore,by suggesting that“substantiality”indicators appear linked to the reputation that appeared in university reputation rankings by simple statistical analysis,we reveal additional benefits of the construct.Findings:We propose a new construct named“substantiality”for measuring research performance.We show that indicators based on“substantiality”can capture important characteristics of research institutes.“Substantiality”indicators demonstrate their“predictive powers”on research reputation.Research limitations:The concept of“substantiality”originated from IGO game;therefore the ease/difficulty of accepting the concept is culturally dependent.In other words,while it is easily accepted by people from Japan and other East Asian countries and regions,it might be difficult for researchers from other cultural regions to accept it.Practical implications:There is no simple solution to the challenge of evaluating research universities’research performance.It is vital to combine different types of indicators to understand the excellence of research institutes.Substantiality indicators could be part of such a combination of indicators.Originality/value:The authors propose a new construct named substantiality for measuring research performance.They show that indicators based on this construct can capture the important characteristics of research institutes.
基金supported by the Institute of Medical Information,Library of Chinese Academy of MedicalSciences(Grant No.09R0216)
文摘Taking the scholarly activities of 73 doctoral program mentors working at the Chinese Academy of Medical Sciences & Peking Union Medical College(the CAMS &PUMC) as a sample of our investigative survey, we tried using such statistical methods as the analysis of variance(ANOVA), factor analysis and correlation analysis to compare the different characteristics of scholarship assessment of Chinese medical scholars as exhibited in their published papers in domestic and foreign journals. Our research findings show that citations per paper and A-index are more suitable for assessing the highly accomplished senior Chinese medical professionals(e.g. academicians) for their domestic and international scholarship attainment. In contrast, the m-quotient is not deemed appropriate to assess their academic influence both at home and abroad. Upon our further analysis of 6 evaluative indicators, we noticed that these indicators might be applied in two different aspects: One is from the viewpoint of Chinese scholars' academic influence at home, which has been evaluated mainly from the perspective of 'total' amount and 'average' amount of both publications and citations. The other is from their academic impact embodied by the means of documents retrieved from the Web of Science, which is mainly assessed from the two viewpoints of publications and citations. It is suggested that the accumulated time-length of a given scholar's active engagement in professional practice in a specific subject area be taken into consideration while assessing a researcher's performance at home and abroad.
