Empirical Study ARTICLE
Different Perspectives: A Study of Probabilities and Likelihoods of Brazilian Universities Upgrading in the National Evaluation System
- 1Department of Applied Health Research, University College London, London, UK
- 2Post-Graduate Program in Business Administration, Universidade Nove de Julho, Sao Paulo, Brazil
In the past few decades, the interaction between science and society has changed its nature. With this change, the reputation of academic institutions became important in all countries around the globe. In Brazil, the Coordination of Betterment of People in Higher Education (CAPES) is the governmental organ responsible for the process of evaluation of higher education, which produces a report and attributes grades to all Post-Graduate Programs around the nation. In this paper, we have analyzed the universe of attributed grades from 2004 to 2013. Our goal was to find patterns and possible biases in the evaluation process. We have found two important patterns. One refers to the higher likelihood of obtaining a higher grade if you are located in the Southeast. The other has to do with the administrative nature of universities. Public institutions are privileged by CAPES in the process of evaluation. Particularly, federal universities are many times more likely to achieve higher grades in some cases. This produces a disincentive to newer and private institutions, which have been fulfilling the role of expanding higher education in Brazil.
• The current process of evaluation of higher education favors public universities.
• Post-Graduate Programs in the Southeast are more likely to be better evaluated.
• The evaluation process increases the gap between public and private universities.
In the past few decades, one could observe an important transformation in the scope of interaction between science and society. Whereas before, scientific knowledge was considered relevant per se, nowadays its dissemination and applicability are also taken into account, which brings light to the social responsibility of research (Moreira, 2006). In this context, universities and higher education institutions play an important role. It is known that such institutions are mainly responsible for scientific and technological development in Brazil, just as they are in countries such as the USA. In the latter, roughly 73% of the information used for patent registration comes from publications in the academic sector, whereas only 27% come from an industrial research background (Narin et al., 1997).
As universities become more important in the dissemination of knowledge, so does their reputation, both in teaching and research. The Coordination of Betterment of People in Higher Education (CAPES in Portuguese), an organ from the Ministry of Education (MEC) in Brazil performs an evaluation of every Post-Graduate Program within the nation, as a means to establish good (or bad) reputation. It attributes grades varying from 1 to 7, taking into consideration the academic production of the program, the number of dissertations and thesis defended, and the student per faculty ratio, among others. If a university obtains a Grade 3 (the starting grade for programs to receive the stamp of approval by the MEC), it is considered to be either new or not to possess such a high reputation. Grade 7, on the other hand, consists of the highest possible grade and represents excellence (CAPES, 2013). This paper is devoted to estimate probabilities and likelihoods of several programs changing their grade, relying on historical data. It looks at Masters and PhD Programs by academic area, region of the country, and administrative nature, once the programs of health sciences and engineering are considered strategic in the development of the nation and there is a high concentration of programs in the Southeast (circa 50% according to the CAPES database). We estimate the underlying probability of change, using the latent probability variable of observed past change, and use national data, which reports the grading for all Post-Graduate Programs since the early 2000s. For the purpose of this analysis, we focus on the period between 2004 and 2013, as this consists of the latest grading of Post-Graduate Programs in Brazil (CAPES, 2013).
Apart from this introduction, this paper is divided into five sections. The first reviews the relevant theoretical background, including the process of evaluation of Post-Graduate Programs, its description, the importance of such in the national context, and the possible tools used to disseminate this information, which should be available to the public easily. A methodological section follows, explaining the treatment of the data, some descriptive statistics, and the basic formula used to calculate the probabilities and likelihoods. Section “Results” presents the main findings of this paper, which demonstrate the difficulty of some programs in obtaining a high(er) grade, as well as the existing bias, concerning the attribution of a higher grade to a public university. Section “Discussion” reflects on potential problems that may arise from CAPES policy of evaluation of Post-Graduate Programs, whereas Section “Conclusion” brings light to the external validity as well as the existing limitations of this study.
The Process of Evaluation of Post-Graduate Programs in Brazil
Compared to other places across the globe, Brazil does not have a long tradition in higher education, as only 12% of its population has obtained some form of a higher degree (IBGE, 2012). As a consequence, there is also a lack of consistency in evaluating the higher degree programs across the nation, although it clearly consists of a matter of great importance for the long-term development of the country. The manner through which higher education programs, including Post-Graduate Programs, are evaluated has evolved considerably in the past recent decades. According to Maccari (2008), currently the process of evaluation takes into account mechanisms for the maintenance of quality in building up human resources who contribute to scientific and technological development.
Back in 2006, the National Institute of Educational Studies and Research (INEP in Portuguese) postulated that evaluating higher education was crucial to stimulate changes both in behavior inside educational institutions, as well as restructuring of the management models within those institutions (INEP, 2006). All in all, the process of evaluation of higher education institutions is necessary to determine the strengths, weaknesses, and potential limitations of such institutions, which in turn allows for adaptation of those in their academic formation (INEP, 2006). The evaluation is without a doubt the greatest control mechanism that governmental organs can exercise over higher education. Once they face regular and recurrent evaluations, universities and other higher education institutes must keep a minimal standard of quality (Durham, 1992). In developed nations, such as the UK or the USA, the process of evaluation of academic standards is fairly thorough. Ivy league faculty members, for instance, have strict publication goals and their maintenance in those schools depends mostly on high-quality peer-reviewed journal publications (Nelson, 2015).
In Brazil, CAPES is the organ responsible for the national process of evaluation of the Post-Graduate Programs. By taking into account a series of complex criteria, the organ attributes a final grade to each of the existing Programs, where research and teaching are the most important elements. In special circumstances, ad hoc consultants visit the institutions to observe the number of faculty members, their degree, as well as the student per faculty ratio. Other observed criteria are the number of hours devoted by faculty members to the institution and the scientific and technological production (Leite et al., 2010). The institutions and programs that achieve the highest grades, namely, Grades 6 and 7, are entitled to preferential treatment when biding for funds from national organisms.
In turn, the process of evaluation takes place centrally and makes use of a data collection software. Previously known as Coleta CAPES, the Sucupira system follows two steps, one takes place yearly and the other originally only every 3 years. Since 2014, the tri-annual part of the evaluation has changed its periodicity to every 4 years, with effect on the current evaluation, that is, the one that accounts for the period 2013–2016 (CAPES, 2013).
The formal evaluation from CAPES, and later analyzed in this paper, has been used in policy making and consists of an explicit criteria for granting scholarships to deprived families with the aim to subsidize higher education and improve equality of access (de Souza Rossetto and de Oliveira Gonçalves, 2015). The program FIES–SISU was set up in the late 2000s and explicitly favored granting funds to private universities located in the North, Northeast, and Center-West (Sinaes, 2014). More recently in 2016, the program also made explicit that it favors programs achieving Grades 4 and 5, and it started taking into account other development indicators, such as micro-regional Human Development Indices (MEC, 2015).
The use of the evaluation performed by CAPES into funding opportunities and public policy highlights the importance of analyzing potential biases in the process of evaluation of Post-Graduate Programs and higher education institutions.
Having produced reports, after both parts of the evaluation, a final grade is attributed to each Post-Graduation Program. Such grade is of public knowledge and serves as reference of quality. The grades vary between 1 and 7. Grades 1 and 2 are very low, and thus programs graded as 1 and 2 are not recognized by the MEC, hindering their capacity to issue higher institution degrees. If an institution only reaches a grade of one or two, it loses the right of teaching, and ultimately opening its door. On the other hand, Grades 6 and 7 correspond to institutions of international excellence (Leite et al., 2010). Efficacy and efficiency are important characteristics of Coleta CAPES, which is always looking for ways of improving both the collection of data process, as well as the process of evaluation per se (Maccari et al., 2008).
As it is currently designed, the evaluation procedure focuses on hard outputs, that is, publication record attributable to the institution and number of alumni at Master and PhD levels. Being quantitative in nature, the procedure focuses on mean levels of publication and successful Master and PhD students over the period of evaluation, although it also takes into consideration the distribution around those measures.
In spite of the fact that many different criteria are taken into consideration, the literature points toward two as the most relevant, particularly in social sciences and business administration. According to Ferraz et al. (2014), more than 50% of the grade in Business, Accounting, and Tourism Programs rely on the evaluation of student and faculty body. The authors state that the first accounts for 20% of the final grade, whereas the latter 35%. The items evaluated are (i) faculty members profile; (ii) adherence and dedication of permanent faculty to the research activities of the Post-Graduate Program; (iii) distribution of research activity and extra training for faculty members; (iv) contribution of faculty member in teaching or research at an undergraduate level; and (v) participation in academic events in line with the faculty member’s research avenue (CAPES, 2013).
As more than a third of the final grade depends upon the evaluation of faculty members, a well-distributed, qualified, and robust academic production is of utmost importance (Maccari et al., 2009). That is particularly true if one considers the influence of publications in journals well-classified among Sistema Qualis/Capes over the tri-annual (now quadri-annual) part of the evaluation. Sistema Qualis/Capes attributes points for most relevant academic journals of an area of interest and is, therefore, defined by areas. In all areas, however, the journals receive a ranking, where A1 is the highest quality publication possible, or the equivalent of 100 points, going down to A2 (80 points), than B1 (60 points) to B5 (5 points) and C (null points).
Also, the analysis of the student body is highly mingled with the evaluation of faculty, once it takes into account: (i) the number of defended thesis and dissertations as a ratio of students per faculty member; (ii) distribution of the supervision of defended thesis and dissertations per permanent faculty in the evaluation period; (iii) quality of the defended thesis and dissertations at port-graduate level and scientific production at undergraduate level; (iv) program’s efficiency in forming PhDs and Masters awarded alumni; and (v) participation of students and alumni in events in line with the research of the program (CAPES, 2013).
It is important to realize that the process of evaluation consists of an addition of several relevant elements, in which the faculty appears to be the most crucial. Nonetheless, the evaluation is a complex process, which explains why its periodicity is fairly big (every 3 years, now 4 years). The complexity can also be perceived in the process of data collection for the elaboration of the evaluation reports. Since 2013, a new software was developed to improve the process of data collection for Coleta CAPES, such as described in the following section. Although the evaluation reports are mainly a result of internal members of CAPES, the institution does have committees for each area of knowledge composed by senior academics of reputable institutions, whose job is to guarantee some degree of transparency and accountability in the evaluation process (Nascimento, 2010).
The Post-Graduate Programs in Brazil are evaluated by CAPES using information extracted by a specific software, designed, and kept by CAPES itself. It is the responsibility of the programs themselves to maintain all information up-to-date in this software, including personal information of staff and students, professional information of staff and students (including former students for up to 3 years), as well as publication records, conference attendances and presentations, and patents and property rights of academic output.
Sucupira system is the name given to a computational system that performs the extraction from Plataforma Lattes, in order to obtain information on scientific production of researchers of Graduate Programs (Alves et al., 2012). Plataforma Lattes, in turn, consists of a circular basis created by National Board of Scientific and Technological Development (CNPq in Portuguese), which aggregates informational systems, databases, portals, and knowledge systems, where all academic research results are stored. Plataforma Lattes integrates the dataset of curricula, Research Groups, and Institutions in one unique Informational System. In 2012, it had approximately two million registered curricula (Alves et al., 2012), which shows the dimension of Brazilian academia. As mentioned earlier, it is the performance of faculty that count the most for the evaluation of Post-Graduate courses in Brazil; therefore, the information from Plataforma Lattes and the systematization from Sucupira system are fundamental.
Data and Methods
For this study, we have used the universe of information regarding grades of Post-Graduate Programs in Brazil between 2004 and 2013, that is, all 26,741 accredited programs were analyzed and patterns of behavior were researched, and ultimately explained. Table 1 shows the regional distribution of all Post-Graduate Programs, and it shows that most are concentrated in the Southeast, nearly half of them, followed by the South and Northeast regions.
Another important feature of the data relates to the division of academic areas. Brazil divides its academia into 9 great areas, which in turn are subdivided into 280 subareas. Table 2 demonstrates the distribution of the great areas.
Here, the distribution is more even, although Health Sciences appear to be slightly more frequent than all other areas. Health Sciences alongside Engineering are considered strategic for the development of any country (MCTi, 2014). Thus, it could be argued that, from an educational perspective, Brazil is trying to narrow the gap in health sciences by disseminating and producing knowledge. The same, however, cannot be said for Engineering, that although strategic, it is less frequent than most other great areas.
Some descriptive statistics may help the understanding of patterns of behavior of the probabilities and likelihoods later calculated in this paper. First, one must look at the total distribution of the grades of all Post-Graduate Programs in Brazil (Table 3). It shows, as expected, that a certain pattern of y = (1/x)n exists, where y is the frequency of a certain grade, x is the grade itself, and n in the number of times a grade repeats itself. This means that the higher the grade, the smaller its frequency.
Second, one must look at the percentage distribution of the grades by region (Table 4). This allows for the perception that higher grades are hugely concentrated in the Southeast. Even considering that an agglomeration of higher institutions exists in this region of the country, the percentage of Grades 6 and 7 is more than proportional, showing the existence of potential of bias in the grading process. Alternatively, one could think that the observed results reflect the high concentration of resources in that region. However, although this may be reasonable if one thinks that there is increased competition as a larger concentration of trained faculty can be found in the Southeast, from a policy perspective, particularly resources from FIES have been allocated elsewhere (Sinaes, 2014), strengthening the possibility of bias. For the purpose of calculation of the predicted probabilities, such bias was used; hence, the reality as observed was considered. An alternative would have been treating the bias in order to produce equality of opportunity (Roemer, 1998, 2002), so each region would have its equivalent in grade.
Finally, even if one considers that no deliberate bias exists, one could speculate a circular problem, in the lines of the chicken and egg problem. Due to historical reasons, the Southeast is the most populated and most affluent region of the country. Not surprisingly, it is also the regions where the largest number of universities is found, both private and public. This region has some of the oldest universities established in Brazil, and one may potentially think that seniority and tradition in research are clearly influencing the observed distribution. This may well be true but could pose a problem to new universities entering the higher education market. As they lack seniority and tradition, it could be that the evaluation as performed by CAPES disfavors them by design.
On another aspect of the evaluation, the nature of the institution appears to be important. Whereas more than 80% of all Brazilian universities are stately owned, leaving only roughly 20% to the private sector, more than 55% of those, i.e., 44% of the total, are controlled by the federal government. This expresses a highly centralized education policy, initiated in the 1950s, where municipalities should be responsible for primary education, states should be responsible for secondary education, and the federal government should be responsible for higher education (Rodrigues, 1986). Table 5 demonstrates the existing bias related to the administrative nature of the university. Clearly, private universities are less likely to receive higher grades, which appear far more frequently in federal- and state-level owned universities. Once again, this will be taken into account when calculating estimated probabilities, as it is our goal to reflect the reality as closely as possible.
All statistical treatment was performed in R, using the method of maximum likelihood estimation (Bierens, 2004). The estimated values took into account the actual distribution of grades per region and nature of institution as opposed to assuming a normal distribution with unknown mean and variance.
The calculation of probabilities was defined empirically, that is, the probability of an event occurring was given by: be A an event of a random experiment (for example, of a university changing its grade in the national evaluation process). Let the experiment be repeated n number of times out of which A occurs f times (in our main analysis, n is the number of possible years, while f is the number of actual changes in the grades). Then, the probability of A is given by the limit into infinity of the frequency ratio (f/n). Mathematically, we have
Thus, the value of the frequency ratio imposes a limit as the number of repetitions becomes infinitely large, and this defines the probability of the event A. Note that the main interest of this paper consisted in calculating the probability of a given university improving its grade in the national evaluation process. This is relevant as universities aim at acquiring better reputation and compete internally to be the most recognized in their area of knowledge. It is also policy relevant, once it provides an outlook of the higher education market (or academia) in Brazil.
We now turn our attention to some results. First and most importantly, Table 6 presents the calculated probability of a university upgrading for federal, state, and privately owned institutions. Unfortunately, due to the low variation in municipal universities, this analysis was not possible at this level.
Some features of the table are interesting. First, the different magnitude of the probabilities could be highlighted. Clearly, the probability of improving and obtaining a higher grade in all areas is more likely to happen for Federal. That is particularly true for Engineering, which has the highest probabilities of upgrading at all level for Federal universities, opposed to very low probabilities of upgrading in private universities. Health Sciences also presents an interesting pattern. Not only it appears to be monotonic as we move from Grades 3 to 7 but also the percentages appear to be twice as big when comparing federal and state universities, with the exception of the chance from Grades 6 to 7, and three times as big, when comparing private universities and state ones. One may also highlight that the probability is only higher from private universities, when compared to state universities, in the case of Humanities. This may be policy relevant, once it expresses the clear bias of CAPES, where private universities are not given the same opportunity of improving their grade as are publicly owned universities.
Another interesting aspect to analyze is the deviation from the mean for each of the grades by region. A skim over Table 7 demonstrates that all regions apart from the Southeast follow the same pattern. They possess a higher percentage of lower grades than the mean, resulting in positive gaps, and as we move to higher grades, the gaps become negative, which means that the regions have percentages lower than the mean at this level. The exact opposite is true for the Southeast, demonstrating a possible bias from CAPES in attributing higher grades for institutions localized in this area of the country, although it is possible that this results from a higher concentration of financial resources in the region, which enables higher education institutions to establish partnerships with the private sector more frequently and produce more societal impact from the commissioned research.
Finally, we turn our attention once again to the calculated probabilities, only this time to look at how many times more likely is a given university to improve its grade at the national evaluation system. Here, also as before, the nature of the establishment appears to be the most relevant aspect of the analysis. Table 8 presents the odds ratios for changing grades using private universities as a base. Therefore, the data exposed should be interpreted in terms of how many times more likely is a given institution of achieving an improvement of grade.
From a fairness perspective, some of the information from Table 8 raises concern. For instance, in Social Sciences, a Federal University is 149 times more likely to change from Grades 5 to 6. That shows a massive disadvantage of private universities in this area of knowledge. In Engineering, a strategic area of knowledge, changing from the high Grade 6 to the top Grade 7 is nearly 13,500% easier for federal universities. Private universities only appear to be in advantage when compared to State universities in Humanities, where the odds ratios are smaller than one. Even so, at the highest level, state universities possess an advantage of circa 30%.
When considering the process of evaluation of higher institutions, one expects fairness to be one of the guiding principles of any evaluator. In Brazil, higher education institutions are evaluated by a complex process referred to as Coleta CAPES.
This paper has used statistical treatment and probability calculation to look at possible evaluation biases. Two clear ones have been found. First, there exists a regional bias, that is, the Southeast region institutions appear to possess an advantage in the evaluation process, thus obtaining higher grades than their counterparts in other areas of the nation. This bias may be explainable due to historical and funding reasons. On the one hand, the Southeast is the most populated area of the country and the financial center of Brazil. Thus, there is greater competition in most industries, including academia. The increased competition may consist of an incentive for local faculty to improve its performance and this would be reflected in the grade obtained by local institutions. On the other hand, the universities established in the Southeast are among the most reputable and most traditional of the country. This seniority could imply better academic standards, justifying the higher grade. Indeed, one could argue that indeed establishments in the Southeast are higher in quality; nonetheless, the number of existing Programs and some statistical properties say otherwise. A very large number of Post-Graduate Programs (26,741) were evaluated; so, a normal distribution of the grades should be expected.
Another important potential bias relates to the administrative nature of the institutions. Our analysis has clearly shown that federal universities are privileged over all others. In Social Sciences, this fact is expressed in an extreme odds ratio for changing from Grades 5 to 6 (using private universities as base). Once again, the seniority and funding arguments apply. Even if one does not believe that a deliberate bias exists in the process of evaluation performed by CAPES, those remain relevant findings, as one could think of these as a “chicken and egg” problem. More senior and better funded universities are more likely to be highly graded, and as traditionally federal universities in the Southeast are more senior and better funded, a status quo persists. This could be potentially unwanted from a policy perspective, particularly if it is in the interest of the government in improving access to higher education, once it could be perceived by new (private) universities as a disincentive.
Both these potential biases are policy relevant, as they express a preference from CAPES to Southeastern federal universities. To a certain extent, the clear guideline from FIES to prefer programs graded 4 and 5 and in the regions North, Northeast, and Center-West could be a way to reduce the observed pattern. As the guideline was only put in place in 2016, further research will be necessary to evaluate whether this new policy has affected the likelihood of regrading.
Finally, the finding from this paper are a cause for concern, as Brazil is still a long way from being highly educated—only 12% of its adult population have some form of higher degree—and much market space has been opened in recent years by private institutions. These, however, cannot compete with public ones, particularly federal establishments, that not only possess a stronger background, historical strength to fall upon, but also are looked at with better eyes by the evaluation board.
In the past few decades, the interaction between science and society has changed its nature. Whereas before, science was considered relevant on its own, it nowadays must be applicable and well disseminated. It is known that academia is the principal responsible for the production of knowledge in current days. Therefore, the reputation of academic institutions becomes important in all countries around the globe.
In Brazil, the Coordination of Betterment of People in Higher Education (CAPES) is the governmental organ responsible for the process of evaluation of higher education in Brazil produces a report and attributes grades to all Post-Graduate Programs around the nation. These grades serve as reference of quality.
In this paper, we have analyzed the universe of attributed grades for each Post-Graduate Program over the period of 10 years, from 2004 to 2013. Our goal was to find patterns and possible biases in the evaluation process. We have found two important deviations. One refers to the higher likelihood of obtaining a higher grade if you are located in the Southeast. The other has to do with the administrative nature of universities. Public institutions are privileged by CAPES in the process of evaluation. Particularly, federal universities are hundreds of times more likely to achieve higher grades in some cases. This produces a disincentive to newer and private institutions, which have been fulfilling the role of expanding higher education in Brazil.
This study can be easily replicated using the methods explained above. Some limitations do exist, nonetheless. First, the authors have considered that large samples, such as the one used in the statistical work of this paper, are normally distributed and have known mean and variance. Furthermore, the distributions were considered as they are observed for the calculation of the probabilities, another possibility would have been correcting the biases in the distributions and latter calculating the probabilities. This would not be helpful in our case, as we are interested exactly in the biases.
Finally, this is the first of many possible studies. We have here been interested in the macro environment of Brazilian academia. However, with the data available, several other micro studies are possible, looking at specific courses or areas of knowledge.
LQ and EM have designed the study. EB has conducted the statistical analysis. EB and RF have written the first draft. All the authors have approved the final manuscript.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
EB is currently supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart’s Health NHS Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health.
Alves, A. D., Yanasse, H. H., and Soma, N. Y. (2012). SUCUPIRA: a system for information extraction of the lattes platform to identify academic social networks. Paper Presented at the Proceedings of 6th Iberian Conference on Information Systems and Technologies (CISTI), Chaves.
CAPES. (2013). Avaliação da pós-graduação. Available at: http://www.capes.gov.br/36-noticias/6908-capes-divulga-resultado-final-da-avaliacao-trienal-2013-apos-analise-de-recursos
de Souza Rossetto, C. B., and de Oliveira Gonçalves, F. (2015). Equidade na educação superior no Brasil: uma análise multinomial das políticas públicas de acesso. Dados-Revista de Ciências Sociais 58, 791–824. doi: 10.1590/00115258201559
Ferraz, R. R. N., Quoniam, L. M., and Maccari, E. A. (2014). A utilização da ferramenta Scriptlattes para extração e disponibilização on-line da produção acadêmica de um programa de pós-graduação stricto sensu em Administração. RBPG 11, 361–389. doi:10.21713/2358-2332.2014.v11.445
IBGE. (2012). Pesquisa Nacional por Amostra de Domicílios. Available at: http://www.ibge.gov.br/home/estatistica/populacao/trabalhoerendimento/pnad2012/default.shtm
INEP. (ed.). (2006). “Modelos Institucionais de Ensino Superior,” in Educação Superior em Debate, Vol. 7. Brasília: INEP. Available at: http://www.publicacoes.inep.gov.br/portal/download/465
Maccari, E. A. (2008). Contribuições à gestão dos programas de pósgraduação stricto sensu em administração no Brasil com base nos sistemas de avaliação norte americano e brasileiro. (Doctorate). São Paulo: Universidade de São Paulo.
Maccari, E. A., Lima, M., and Riccio, E. (2009). Uso do sistema de avaliação da Capes por programas de pós-graduação em Administração no Brasil. Revistade Ciências da Administração 11, 68–82. doi:10.5007/2175-8077.2009v11n25p68
Maccari, E. A., Rodrigues, L. C., Alessio, E. M., and Quoniam, L. M. (2008). Sistema de Avaliação da Pós-Graduação da CAPES: pesquisa-ação em um Programa de Pós-Graduação em Administração. RBPG 5, 171–205. doi:10.21713/2358-2332.2008.v5.147
MCTi. (2014). Ministério da Ciência, Tecnologia e Inovação. Available at: http://www.mcti.gov.br/
Keywords: Post-graduate Programs, evaluation, CAPES, education, Brazil
Citation: Barbosa EC, Quoniam L, Maccari E and Nogueira Ferraz RR (2017) Different Perspectives: A Study of Probabilities and Likelihoods of Brazilian Universities Upgrading in the National Evaluation System. Front. Educ. 2:9. doi: 10.3389/feduc.2017.00009
Received: 10 January 2017; Accepted: 13 March 2017;
Published: 30 March 2017
Edited by:Claudia Fahrenwald, University of Education Upper Austria, Austria
Reviewed by:Rosangela Malachias, Rio de Janeiro State University, Brazil
Diana G. Pounder, University Council for Educational Administration - INSPIRE, USA
Copyright: © 2017 Barbosa, Quoniam, Maccari and Nogueira Ferraz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Estela Capelas Barbosa, email@example.com