Bibliometric and Altmetric Analysis of Three Social Science Disciplines

Introduction

In academia, the publication of social science and humanities (SSH) research findings is known to differ from that of other areas of knowledge. As the object of study in such disciplines is often local or national, the results in a given context may not necessarily be useful to researchers in other countries (Nederhof, 2006). The type of vehicle chosen to disseminate the findings also tends to vary. Book chapters and monographs are researchers' options of choice both when publishing their own findings and when seeking those of others in the literature (Hicks, 2004; Nederhof, 2006; Kulczycki et al., 2018). SSH collaboration also deviates from the general pattern, with a prevalence of singly authored texts and scant inter-institutional partnering. SSH researchers have likewise been observed to write not only for specialized readers, but also for the lay public (Hicks, 2004). Citation patterns differ in SSH disciplines as well: references have much earlier dates (Glänzel and Schoepflin, 1999) and such particulars also affect impact. That state of affairs is exacerbated by the small proportion of co-authored social sciences and humanities papers for, as some authors have reported, scientific partnering and number of citations are closely related parameters (Hsu and Huang, 2011; Puuska et al., 2014).

Such circumstances necessitate the pursuit of other variables that would provide information on SSH research. Other publication policies must be furthered to raise the visibility of the area in society in general and the scientific community in particular, so its findings can be applied to generate new knowledge. Some authors (Sivertsen, 2016) contend that SSH research should be internationalized. Others recommend publication on open access platforms, a strategy that has been found to raise the number of citations in several areas, social science among them (Antelman, 2004; Atchison and Bull, 2015).

Whilst scientometrics and bibliometrics have been consolidated as the primary tools for analysing scientific output and its impact on the research community, widespread access to the internet has driven change in the scientific communication paradigm. The advent of Web 2.0 has ushered in radical change that has enhanced both communication and collaboration among academics and their interaction with social agents. Today many content sharing sites such as fora, blogs, and social networks (including Facebook and Twitter) have not only proven very popular among the public at large, but have also seduced members of the scientific community, giving rise to the so-called “academic social web.” This new space enables researchers to share and validate their projects through tools tailored to the academic environment such as reference management software (Mendeley, CiteULike), professional networks (ResearchGate, ScienceOpen), and digital identity applications (Web of Science's ResearcherID or ORCID).

Van Noorden (2014) notes that scientific communication has changed unpredictably, for academics obviously wish to share their knowledge and experiences “openly,” as the use of multiple digital identity tools attests. One example of the impact of these new tools is their use by many researchers since, as Kramer and Bosman report (2016), ResearchGate and institutional repositories have become today's vehicles of choice for conveying research findings. Martín-Martín et al. (2018) show that Google Scholar and Research Gate are the two most widely used tools in bibliometrics research at this time.

That set of tools is reinforced by the open access to science movement and online publications and repositories (PLos ONE, ArXiv, CiteSearch, PubMed, and RePEc). Mohammadi and Thelwal (2013) note that this proliferation of informal communication channels constitutes a new challenge for the analysis of scientific activity. New models to address and analyse scientific communication must therefore be proposed. Traditional bibliometric-based studies may be supplemented with new altmetric indicators that afford a measure of the interest roused in society by research in a given area and that have had a particular impact since their advent in 2010 (Priem and Hemminger, 2010).

One of the main advantages of such indicators is that as data are classified at the article level, the impact of studies can be appraised with no regard to the quality or visibility of the publishing journal (Neylon and Wu, 2009). That has given rise to what some authors (Orduña-Malea et al., 2016; Martín-Martín et al., 2018) contend is a new branch of bibliometrics, ALMetrics (author level metrics), which analyses the performance of authors by measuring all the dimensions of their intellectual activity. Further to that approach the analysis of mentions in the social media affords new opportunities to obtain information on the characteristics of SSH scientific activity.

Given the enormous heterogeneity of research in these fields and its possible impact on society, ascertaining its effect on the scientific and social domains may be of significant use in determining its actual role. Hammarfelt (2014) acknowledges that “Many of the problems of applying bibliometrics to the humanities are also relevant for altmetric approaches; the importance of non-journal publications, the reliance on print as well the limited coverage of non-English language publications.”

A combined analysis of both types of indicators would provide considerable insight into social sciences and humanities research. Further to Gorraiz (2018), two types of indicators might be defined: on the one hand long-term indicators such as citations, with a protracted half-life that can only furnish significant information after a given period of time, depending on the discipline; and on the other, short-term or short-lived indicators, such as measures of use, gathered almost prior to the official publication date and in the following few years.

Interest in the use of alternative metrics has spawned a number of studies addressing the scope of different platforms and indicators (Torres-Salinas et al., 2013; Robinson-García et al., 2014), among others. The advantages and limitations of altmetric indicators have also been widely described in the literature (Gumpenberger et al., 2016; Orduña-Malea et al., 2016; Moed, 2017; Martín-Martín et al., 2018). Other researchers have focused on the characteristics of documents that may affect their social impact (Haustein et al., 2015; Robinson-Garcia et al., 2017; De Filippo and Serrano-López, 2018) or the relationship between bibliometric and altmetric indicator- measured impact (Cabezas-Clavijo and Torres-Salinas, 2010; Schloegl and Gorraiz, 2010; Eysenbach, 2011; Costas et al., 2014; Serrano-Lopez et al., 2017).

In light of the methodological limitations to the use of certain altmetric tools (such as synthetic indices to measure the score) identified in the scientific literature, this article, in an attempt to carry the research beyond specific case studies (a single institution's, journal's, discipline's or country's output), analyses worldwide production in three social science disciplines to:

• detect patterns of scientific activity in terms of scientific output, collaboration, impact and visibility

• study impact and visibility in greater depth, both through bibliometric and altmetric indicators to identify similarities and differences

• ascertain the possible existence of relationships between scientific impact and social reaction (bibliometric vs. altmetric indicators)

• explore the possibility of using social media indicators to analyse document visibility.

The disciplines chosen for analysis were communication (COM), economics (ECON) and sociology (SOCIOL), for as their activity profiles differ their dynamics could be analyzed.

Information Sources

Clarivate Analytics' Web of Science database, deemed to be a model reference for determining visibility and prestige in international output, was the source of information. It presently carries bibliographic information on over 18,000 scientific journals (Mangan, 2017), books and scientific, social sciences, art, and humanities congress proceedings.

Despite the scant representation of SSH in international databases, as mentioned earlier, the analysis drew from the Social Science Citation Index (SSCI) for several reasons. The first was that despite inter-area differences in output dynamics and the communication of scientific findings, SSH publication habits have been changing recently. They have been driven, for instance, by the institutional and assessment policies in place in many countries that reward publication in journals listed in databases of prestige (Ainsworth and Russel, 2018). Another game-changer has been editors' efforts to improve the quality of national scientific journals and the concomitant inclusion of many such publications in international databases (Tavares de Matos Cardoso, 2011; De Filippo, 2013).

Speedy and convenient communication has also favored greater international collaboration among researchers in the area and the resulting publication in international journals. Attesting to that trend is the growing number of SSH articles in international databases such as the WoS Social Science Citation Index, with a rise (165% in social sciences from 2006 to 2015) in both the volume of papers listed and their proportion in the WoS total. Partnering and co-authorship rates are also gradually rising and English is increasingly the language used (Sanz-Casado et al., 2017).

Web of Science was also chosen because it contains the information on document impact and visibility needed to conduct the analysis proposed and because the data can be unbundled into very specific disciplines. Output as listed in the SSCI was chosen to analyse the papers published in journals listed under that major area of knowledge. Information on impact was drawn from Journal Citation Reports.

The Altmetric.com application was used to gather information on social media (blog posts, Twitter, Wikipedia citations, mainstream media, Google+, Facebook, RSS feeds, and videos). This is one of the most thorough tools for studying visibility in such media, although it is also subject to limitations such as the sources used to obtain indicators and its language bias (Robinson-García et al., 2014; Haustein et al., 2015) or the scant transparency of its methodology and obsolescence of the results included (Gumpenberger et al., 2016). For those reasons, “score,” the synthetic indicator provided by Altmetric.com, was not used. Rather, the mentions in each source were reviewed separately.

Methodology

The procedure followed to conduct this study is outlined below.

(1) Information retrieval

Papers were retrieved on the grounds of the WoS category assigned to the respective journals.

The three disciplines analyzed, which differ in object studied and output, are included in JCR for the social sciences. and defined in the database itself as follows:

Communication: Covers resources on the study of the verbal and non-verbal exchange of ideas and information. Included here are communication theory, practice and policy, media studies (journalism, broadcasting, advertising, etc.), mass communication, public opinion, speech, business and technical writing as well as public relations.

Economics: Covers resources on all aspects, both theoretical and applied, of the production, distribution, and consumption of goods and services. These include generalist as well as specialist resources, such as political economy, agricultural economics, macroeconomics, microeconomics, econometrics, trade, and planning.

Sociology: Covers resources that focus on the study of human society, social structures, and social change as well as human behavior as it is shaped by social forces. Areas covered in this category include community studies, socio-ethnic problems, rural sociology, socio-biology, social deviance, gender studies, the sociology of law, the sociology of religion, and comparative sociology (Clarivate Analytics, in press).

The publications in each discipline were collected and analyzed independently. All type of documents were included gather information on the social impact of the overall output.

The 3 year period analyzed, 2013–2015, was chosen because the documents retrieved exhibited a sufficient number of citations and altmetric indicators. Including more recent papers would have hampered the collection of significant information on citations, for whereas in areas such as biomedical science, the life sciences or physics a 2 year citation window has been observed to suffice (Adams, 2005; Campanario, 2011), a longer period is needed in social sciences and humanities (Vanclay, 2008; Campanario, 2011; Waltman and Van Eck, 2012; Dorta-González and Dorta-González, 2013). Conversely, enlarging the analysis to include papers published prior to 2013 would have distorted the compilation of the altmetric indicators, the recent implementation of which limits their validity to more recent years.

The search strategy used was as follows:

(2) Data processing

The papers identified in each discipline were downloaded and independent relational databases, containing all the bibliographic information on every document, were built using MySQL. The fields to which particular attention was paid included language, type of document, year of publication, journal, citations received, DOI, and document sourcing.

The Journal Citation Reports for the years analyzed (2013–2015) were likewise downloaded and integrated into the aforementioned bases.

(3) Bibliometric indicators

The cleansed data were used to define the bibliometric output indicators in each discipline, grouped under four dimensions: activity, visibility, impact, and collaboration. Both simple (describing data with a single dimension) and relational (relating different dimensions, especially impact, and visibility) indicators were collected. All the indicators studied are listed in Table 1.

(4) Altmetric indicators

A script developed by the Carlos III University of Madrid's Laboratory of Metric Information Studies was applied to find the altmetric indicators for each document based on its DOI (digital object identifier) and the Altmetric.com API (application programming interface). The indicators obtained were included in the “presence in social media” dimension, for they area all based on mentions in the social media. This analysis excluded academic network (such as Mendeley) indicators, for Altmetric.com only counts readers when a paper has some other altmetric indicator, which might lead to under-representation of readers. Both simple and relational indicators were obtained (the latter by relating traditional impact and visibility to mentions in social networks).

After obtaining all the indicators, the number of citations (traditional impact indicator) was compared to the number of mentions in the social media to ascertain whether they were related. Pearson's correlation coefficient and similar statistical tests were conducted and the respective p values (probability of statistical significance) calculated.

TABLE 1

Table 1. Dimensions and indicators studied in the analysis.

Results

The findings for each of the dimensions analyzed are set out below.

Bibliometric Indicators

Scientific Activity

Of the 112,300 papers retrieved with publication dates in the 3 years studied, over half (76,400) were in economics, whilst 1,670 were classified under more than one of the WoS categories analyzed. The steepest percentage rise from 2013 to 2015 was in sociology and communication at 9%. Those values were higher than growth in the SSCI database overall in the period (5.8%). Although the highest proportion of papers relative to total SSCI output was found for economics, growth in that parameter in the period analyzed was negative (−3.5%). In contrast, in percentage of the whole, communication and sociology rose by 3% (Table 2).

TABLE 2

Table 2. Yearly output by discipline.

The U.S. headed the output by country in all three disciplines, with the highest percentages in communication and sociology. Output was also high in the England, especially in economics and sociology.

English was by far the most prevalent language, ranging from 89% in sociology to 99% in economics. A moderate number of papers were found in Spanish, particularly in economics, and Russian, in sociology.

The types of document most widely found in communication and sociology were journal articles and book reviews, whilst economics was characterized by greater variety. In that discipline, although articles accounted for 73% of the total, other typologies (such as meetings, biographical material, retracted publications, and letters) together came to 16% (Figure 1).

FIGURE 1

Figure 1. Publications by document type and discipline.

Visibility

The visibility analysis was based on the papers published in JCR-listed journals. As Figure 2 shows, most of those journals had a high impact factor (94% in communication and around 97% in the other two). In economics the papers were clustered in Q1, whilst the highest percentage of communication and sociology papers were in Q2 and Q3.

FIGURE 2

Figure 2. Publications by quartile and discipline.

The percentage of open access papers was substantially higher in economics than in the other two disciplines and somewhat higher than the SSCI mean (Figure 3).

FIGURE 3

Figure 3. Percentage of open access papers per discipline.

Impact

The number of citations received did not vary substantially among the three disciplines (4.04 citations/paper in sociology and 5.47 in economics). The percentage of NON-cited papers ranged from 43% in sociology to 33% in communication and economics (Table 3). In sociology 0.36% of the papers were highly cited, in communication 0.41% and in economics 0.75%.

TABLE 3

Table 3. Citations per paper, total number of citations by discipline.

Collaboration

Co-authorship rates were observed to rise in all three disciplines in the 3 years analyzed. Economics, with the highest values, had a mean of around four authors per paper. The papers lacking inter-institutional collaboration outnumbered the others. The highest percentage of papers involving international collaboration was found in the disciplines with the highest overall co-authorship rates (Figure 4). Despite the low percentages of international partnering, the proportion rose by around at least two percentage points in all four disciplines in the period studied.

FIGURE 4

Figure 4. Yearly co-authorship rates (Left) and institutional collaboration by discipline (right).

Crossing the information on visibility and impact with the data on collaboration revealed that in all three disciplines, impact was highest in the papers involving international co-authorship, whilst the lowest number of citations per paper was observed for those with no inter-institutional collaboration. Co-authored papers were also published in higher (Q1 and Q2) quartile journals than those with no institutional partnering. In sociology, papers involving international collaboration exhibited the highest percentage of Q1 articles, whereas in the other two disciplines the articles co-authored nationally had the highest percentage of Q1 papers (Table 4).

TABLE 4

Table 4. Impact and visibility by type of collaboration and discipline.

Altmetric Indicators

Presence in the Social Media

As 85 to 92% of the papers in the three disciplines analyzed had a DOI, their altmetric indicators could be calculated. The percentage has grew in all three in the 3-year period analyzed, especially in communication, where it climbed by over three percentage points. Over half of the communication papers with DOI had mentions in the social media and nearly half of the sociology articles also had such mentions. Despite the large proportion of economics papers with DOI, in contrast, less than one-third were mentioned in the social media (Figure 5). The proportion of documents with mentions also rose, by four percentage points in communication and economics and eight in sociology.

FIGURE 5

Figure 5. Papers with DOI and mentions in the social media by discipline.

Figure 6 shows the percentage of DOI documents with mentions in each type of source. While the values differed slightly, the profile was similar for the three disciplines. The two most common types of mentions were tweets and blog posts, particularly in communication. Facebook was much less significant, especially in economics, where it had little more than a token presence. By maximum number of mentions received, however, economics had 150 in Facebook and very large numbers of blog posts and tweets, attesting to the significant variation between the mean and the outliers.

FIGURE 6

Figure 6. Social media mentions by type of source and discipline.

The relationship between traditional and social media impact was found by applying the Pearson correlation coefficient to the data on citations and mentions. The coefficients for the relationship between traditional media and two most frequently used social networks (blog posts and Twitter) are given in Table 5. The relationship between the two was observed to be very low, for the most cited papers were not necessarily the ones most widely mentioned or vice-versa. Nonetheless, most of the highly cited papers were mentioned in social media. Where citations and mentions for those documents only were analyzed, the correlation values were somewhat higher, except in economics.

TABLE 5

Table 5. Bibliometric and altmetric correlation.

Other indicators, listed in Table 6, were consequently used to analyse the correlation between citations and presence in the social media in greater depth. The first was the percentage of papers cited and not cited in each discipline. Communication and economics exhibited the highest proportion of the former (66.8%). In all disciplines the percentage of papers receiving citations was much higher in those with than those without a DOI, perhaps logically, inasmuch as the identifier affords access to papers and their timely reuse to generate new knowledge. A large majority (from 79 to 87%) of the papers with mentions in the social media received citations. Those percentages were much higher than found for the papers without such mentions.

TABLE 6

Table 6. Percentage of papers cited with and without DOI and with and without mentions in the social media by discipline.

The number of citations per paper with and without mentions in the social media was also compared. As Figure 7 shows, the number was consistently higher in papers mentioned in the social media than those that were not. The difference between the means for the two sets were statistically significant in all disciplines (p-value < 0.0001).

FIGURE 7

Figure 7. Citations per paper with vs. those without mentions in the social media by discipline.

The impact of the papers with mentions in the social media was determined by analysing the number of citations by paper broken down by source. As Table 7 shows, the percentage of citations received by source was higher than the overall mean in all three disciplines. The papers with mentions in Google+, MSM, videos, and Wikipedia were the ones most widely cited, although those were not the predominant media.

TABLE 7

Table 7. Impact (citations/paper) of papers mentioned in the social media by type of source and discipline.

Social media mentions of open access papers were also analyzed. Further to the data in Table 8, the proportion of open access papers with mentions in the social media was higher than the percentage of open access papers in the respective discipline in three of the four. The exception was economics.

TABLE 8

Table 8. Documents with open access and mentions in social media per discipline.

In the three disciplines, review papers exhibited the greatest impact in terms of citations/document. In all types of documents, those mentioned in social networks received more citations than those that were not (Table 9).

TABLE 9

Table 9. Impact (citation/paper) per documental type and presence in social media.

Discussion

This study aimed to: identify internationally prestigious scientific publications in three social sciences, analysing activity, collaboration, impact and visibility; determine their repercussion in the social media; and explore the possible inter-relationships. One of the main limitations of the study is that it focused only on three disciplines, using the data indexed in the SSCI and analysing new metrics with the “Altmetric.com” tool. The objective, however, was not to analyse the results from different sources, but rather to compare the activity profiles of disciplines with different dynamics. The findings showed that even within a given subject area such as the social sciences, output dynamics may differ from one discipline to the next. The nature of production in the disciplines analyzed was observed to be dissimilar, for the objects of study have a bearing on the approach to research and communication of the findings.

Economics is the most stable of the three disciplines, with a more deeply rooted tradition of publication in journals listed in international databases, along with more highly profiled partnering, impact and visibility. Although journal articles predominate in that science meetings abstracts constitute a fairly high proportion of the total. Of the three fields studied, economics is the one with the highest percentage of documents in Q1 (48%). Moreover, 25% are open access publications, two percentage points above the mean for the social sciences. The maximum number of citations received is also higher than in the other disciplines, a finding closely linked to the average number of authors per document and international collaboration, which is likewise higher. The other two disciplines, communication and sociology, exhibit profiles closer to those of traditional SSH. In both cases book reviews account for a significant proportion of the total, while no more than 11% of the documents are openly accessible. Output distribution by quartile and collaboration patterns are also similar in these two areas of knowledge.

Perhaps one of the most pertinent findings is the presence of SSH papers in the social media. Altmetric indicator analysis is highly dependent upon the existence of a DOI. The present findings show that the percentage of documents with this identifier grew in the period studied, a trend consistent with previous reports (Gorraiz et al., 2018). The number of documents with mentions in social media also grew yearly in the three disciplines, at rates higher than observed for the percentage of documents with DOI.

A significant proportion of papers in all three disciplines studied were mentioned in social media (over 55% in communication). Earlier studies showed that the percentage of papers with such mentions ranged from 15 to 24%, with social sciences and humanities papers on the high end of the scale (Haustein et al., 2015). One explanation for that discrepancy may be the relationship between the object of study and its social impact, for topics relating to the communication of science appear to elicit the greatest reaction (De Filippo et al., in press).

The number of tweets is one of the most frequent altmetric indicators, a constant in most studies using Altmetric.com (Costas et al., 2014; Robinson-García et al., 2014; Haustein et al., 2015) or PlumX (Gorraiz et al., 2018). Some authors contend that since Twitter is widely used outside academia it may be an especially promising source of evidence for social interest in science. Impact measured by tweets about articles differs from that measured by traditional citation platforms or Mendeley reader counts. As Twitter is used by academics, however, tweets may also attest to academic impact (Haustein et al., 2014).

No close correlation appears to exist between the number of citations and the number of social media mentions, for the papers most widely cited do not necessarily receive more mentions or vice-versa. Similar results were found in country-scale (Gorraiz et al., 2018) or discipline-wide (Sanz-Casado et al., 2016) studies of the correlation between mentions on Twitter and citations as well as by earlier authors (Cabezas-Clavijo and Torres-Salinas, 2010; Torres-Salinas et al., 2013; Costas et al., 2014). According to Holmberg and Thelwall (2013), the studies that confirm a positive relationship between traditional citations and mentions in social media are based on elite journals. They showed that 6 of the 11 altmetric indicators studied could be associated with traditional citations in medicine and biology, although their study revealed no correlation between them.

Despite the disconnect found in the literature between the number of citations and the number of mentions, the present findings appear to identify a link. They show that 78% (sociology) to 87% (economics) of the papers with mentions in the social media are also cited, whilst the percentage of cited papers is much lower among those without mentions. The mean number of citations per paper is also higher (and with statistically significant differences) in those with than those without mentions in the social media. Moreover, a significant proportion of highly cited papers (77% in economics and around 95% in the other disciplines) is mentioned in social media.

In two of the three disciplines analyzed (the exception being economics), the percentage of open access documents with mentions in the social media is higher than the percentage of open access papers relative to the total number of papers. Those two variables were also reported to be related by authors such as Bruns and Stieglitz (2012).

Journal articles and reviews are the types of documents with greatest social media presence, as studies analysing the relationships between formal aspects of documents and that presence have shown (Haustein et al., 2015; De Filippo and Serrano-López, 2018). Despite the predominance of journal articles, other types of SSH publications frequently found in SSH, such as book reviews, are more widely cited if present in social media.

Although the inclusion of altmetric indicators such as used here constitute a new perspective for judging research impact, both the advantages and drawbacks of the approach must be borne in mind. The use of the new metrics for the purposes of assessment is still highly controversial and indisputably confronted by many challenges. Unresolved issues such as reliability, completeness, interrelationship, standardization, stability, scalability and normalization of the data collected are will have to be tackled. Nor should the use of new metrics be confined to assessment only (Gorraiz et al., 2018). As Moed (2017) explains, one of the advantages of mentions in the social media is that they are accessible immediately after publication of the research findings. They may also reveal the impact on non-academic audiences and constitute tools to link scientific experience to social needs. They cannot be used to measure scientific or academic impact, however, for the number can be manipulated and the interdependence of the various social media may flatter the figures.

Whilst serious doubts have been raised about the value of altmetric indicators, presence in the social media obviously shortens the scientific community-to-society dissemination time and broadens the scope and diversity of the target audience. They appear to be suited not only for assessment purposes, however, but to ascertain the interest and impact of scientific output and to monitor its visibility over time.

Such perspectives, attendant upon the possibility of deploying additional metrics, should serve as a starting point to re-think the criteria for assessing SSH. From the standpoint of traditional communication certain features such as enhanced international collaboration would raise visibility and impact. Furthering improvements in the quality of national journals would have a similar effect. As new communication tools also reach wider audiences, disseminating findings in the social media initially published in other formats such as books, chapters, and conference papers would favor access to knowledge by a broader public.

All the foregoing should ultimately spur debate around the determination of the primary objective of enhancing visibility of scientific knowledge: should it be to raise one's individual position in academic rankings? to disseminate the knowledge generated more widely? to transfer that knowledge from the scientific domain for reuse in the generation of new knowledge? or to propose possible solutions to specific social problems? Many other similar questions might well be posed, all very likely with affirmative answers and able to co-exist in an ever more (geographically and socially) globalized and inter-related world, favored by fluent communication between the scientific community and the public at large. That may be the starting point to chart the best course for raising the long pursued visibility of the social sciences and humanities: areas of research, as noted at the beginning of this article, characterized by their direct and significant social implications.

Conclusions

The conclusions that can be drawn from this paper and their relevance to the objectives sought are discussed below,

Activity Patterns

The identification of different patterns of activity among the disciplines studied is one of the most prominent findings of this study. Economics exhibits a more international and consolidated profile in international databases, while parallels can be found in the activity of communication and sociology (with features closer to the SSH disciplines as a whole).

Impact and Visibility

When these dimensions are analyzed with bibliometric indicators, economics also appears to be better placed than the other two fields, with higher citation/document values and a higher proportion of documents in Q1 and in open access. Sociology and communication have higher rates of social media mentions, however, with Twitter and blog posts the vehicles most frequently involved.

Relationships Between Metrics

Statistical correlations between bibliometric impact (citations per document) and presence in social media (number of mentions) are low. In all three disciplines, however, documents with a social media presence, especially in Google+, MSM, videos, and Wikipedia, have a statistically significant larger number of citations/document than those without such a presence. Almost all of the highly cited papers have mentions in social media (Twitter and blog posts).

Validity of the Use of Altmetric Indicators

Given the disparities between traditional citation and network presence and the existing methodological problems, altmetric indicators do not appear to be apt for assessment purposes. They do, however, supplement the information on the dissemination of publications, especially in SSH, where they may enhance the visibility and scope of documents (especially those that are neither journal articles nor reviews).

Author Contributions

DD conceptualization, data retrieval, methodology, indicator development, formal analysis, write-up–original draft. ES-C conceptualization, methodology, formal analysis, write-up-review, and editing.

Funding

Altmetrics indicator methodology was developed as part of the project entitled Detection of new research and innovation fronts. Analysis of knowledge flows in the scientific domain, industry and society (ref.: CSO2014-58889-JIN), funded by Spain's Ministry of the Economy and Competitiveness.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Andres Pandiella-Dominique's (Carlos III University of Madrid) support in the preliminary data collection is gratefully acknowledged. The authors thank Margaret Clark for her collaboration in translating the text and valuable comments.

References

Adams, J. (2005). Early citation counts correlate with accumulated impact. Scientometrics 63, 567–581. doi: 10.1007/s11192-005-0228-9

CrossRef Full Text | Google Scholar

Ainsworth, S., and Russel, J. M. (2018). Has hosting on science direct improved the visibility of Latin American scholarly journals? A preliminary analysis of data quality. Scientometrics 115, 1463–1484. doi: 10.1007/s11192-018-2725-7

CrossRef Full Text

Antelman, K. (2004). Do open-access articles have a greater research impact? College Res. Libraries 65, 372–382. doi: 10.5860/crl.65.5.372

CrossRef Full Text | Google Scholar

Atchison, A., and Bull, J. (2015). Will open access get me cited? An analysis of the efficacy of open access publishing in political science. PS Political Sci. Politics 48, 129–137. doi: 10.1017/S1049096514001668

CrossRef Full Text

Bruns, A., and Stieglitz, S. (2012). Quantitative approaches to comparing communication patterns on twitter. J. Technol. Human Ser. 30, 3–4. doi: 10.1080/15228835.2012.744249

CrossRef Full Text | Google Scholar

Cabezas-Clavijo, A., and Torres-Salinas, D. (2010). Indicadores de uso y participación en las revistas Científicas 2.0: el caso de PLoS One. El Profesional Información 19, 431–434. doi: 10.3145/epi.2010.jul.14

CrossRef Full Text | Google Scholar

Campanario, J. M. (2011). Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window. Scientometrics 87, 189–204. doi: 10.1007/s11192-010-0334-1

CrossRef Full Text | Google Scholar

Clarivate Analytics (in press). Web of Science Categories List.

Costas, R., Zahedi, Z., and Wouters, P. (2014). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. J. Assoc. Inform. Sci. Technol.66, 2003–2019. doi: 10.1002/asi.23309

CrossRef Full Text | Google Scholar

De Filippo, D. (2013). Spanish scientific output in communication sciences in WOS. The Scientific Journals in SSCI (2007-12). Comunicar 41, 25–34. doi: 10.3916/C41-2013-02

CrossRef Full Text | Google Scholar

De Filippo, D., and Serrano-López, A. (2018). From academia to citizenry. Study of the flow of scientific information from projects to scientific journals and social media in the field of “Energy saving”. J. Clean. Product. 199, 248–256. doi: 10.1016/j.jclepro.2018.07.177

CrossRef Full Text

De Filippo, D., Silva, P., and Borges, M. M. (in press). Caracterización de las publicaciones de España y Portugal sobre open science y análisis de su presencia en las redes sociales. Revista Española de Documentación

Dorta-González, P., and Dorta-González, M. I. (2013). Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor. Scientometrics 95, 645–672. doi: 10.1007/s11192-012-0929-9

CrossRef Full Text | Google Scholar

Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. J. Med. Internet Res. 13:e123. doi: 10.2196/jmir.2012

CrossRef Full Text

Glänzel, W., and Schoepflin, U. (1999). A bibliometric study of reference literature in the sciences and social sciences. Information Process. Manage. 35, 31–44. doi: 10.1016/S0306-4573(98)00028-4

CrossRef Full Text | Google Scholar

Gorraiz, J. (2018). A Thousand and one reflections of the publications in the mirrors'labyrinth of the new new metrics. El Professional Información 27, 231–236. doi: 10.3145/epi.2018.mar.01

CrossRef Full Text | Google Scholar

Gorraiz, J., Blahous, B., and Wieland, M. (2018). Monitoring the broader impact of the journal publication output on country level: a case study for Austria, in Altmetrics for Research Outputs Measurement and Scholarly Information Management, AROSIM 2018. Communications in Computer and Information Science, eds M. Erdt, A. Sesagiri Raamkumar, E. Rasmussen and Y. L. Theng (Singapore: Springer), 39–62. Available online at https://link.springer.com/chapter/10.1007/978-981-13-1053-9_4

Gumpenberger, C., Glänzel, W., and Gorraiz, J. (2016). The ecstasy and the apgony of the altmetric score. Scientometrics 108, 977–82. doi: 10.1007/s11192-016-1991-5

CrossRef Full Text | Google Scholar

Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics 101, 1419–1430. doi: 10.1007/s11192-014-1261-3

CrossRef Full Text | Google Scholar

Haustein, S., Costas, R., and Lariviere, V. (2015). Characterizing social media metrics of scholarly papers: the effect of document properties and collaboration patterns. PLoS ONE 10:e0120495. doi: 10.1371/journal.pone.0120495

CrossRef Full Text | Google Scholar

Haustein, S., Lariviere, V., Thelwall, M., Amyot, D., and Peters, I. (2014). Tweets vs. Mendeley readers: how do these two social media metrics differ? IT Inf. Technol. 56:207e215. doi: 10.1515/itit-2014-1048

CrossRef Full Text | Google Scholar

Hicks, D. (2004). “The four literatures of social science,” in Handbook of Quantitative Science and Technology Research, eds H. Moed (Dordrecht: Kluwer Academic), 473–496.

Google Scholar

Holmberg, K., and Thelwall, M. (2013). Disciplinary differences in twitter scholarly communication, in 14th International Society of Scientometrics and Informetrics Conference, 15-19 July (Viena).

Hsu, J., and Huang, D. (2011). Correlation between impact and collaboration. Scientometrics 86, 317–324. doi: 10.1007/s11192-010-0265-x

CrossRef Full Text | Google Scholar

Kramer, B., and Bosman, J. (2016). Innovations in Scholarly Communication. Available online at: https://101innovations.wordpress.com/

PubMed Abstract

Kulczycki, E., Engels, T. C. E., Polonen, J., Bruun, K., Duskova, M., Guns, R., et al. (2018). Publication patterns in the social sciences and humanities: evidence from eight European countries. Scientometrics 116, 463–486. doi: 10.1007/s11192-018-2711-0

CrossRef Full Text | Google Scholar

Mangan, R. (2017). Web of Science: Manual de uso. Claryvate Analytics.

Martín-Martín, A., Orduña-Malea, E., and Delgado-López-Cózar, E. (2018). Author level metrics in the new academics profile platforms: the online behaviour of bibliometrics community. J. Informetrics 12, 494–509. doi: 10.1016/j.joi.2018.04.001

CrossRef Full Text | Google Scholar

Moed, H. F. (2017). Applied Avaluative Informetrics. Amsterdam: Springer International Publishing.

Google Scholar

Mohammadi, E., and Thelwal, M. (2013). Assessing the Mendeley readership of social sciences and humanities research, in 14th International Society of Scientometrics and Informetrics Conference, 15-19 July (Viena).

Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: a review. Scientometrics 66, 81–100. doi: 10.1007/s11192-006-0007-2

CrossRef Full Text | Google Scholar

Neylon, C., and Wu, S. (2009). Article-level metrics and the evolution of scientific impact. PLoS Biol. 7:e1000242. doi: 10.1371/journal.pbio.1000242

PubMed Abstract | CrossRef Full Text | Google Scholar

Orduña-Malea, E., Martín-Martín, A., and amd Delgado-López-Cózar, E. (2016). The next bibliometrics: ALMetrics (Author Level Metrics) and the multiple faces of autor impact. El profesional Información 25, 485–496. doi: 10.3145/epi.2016.may.18

CrossRef Full Text | Google Scholar

Priem, J., and Hemminger, M. (2010). Scientometrics 2.0: Toward new metrics of scholarly impacto n the social web. First Monday. doi: 10.5210/fm.v15i7.2874. Available online at: https://ojphi.org/ojs/index.php/fm/article/view/2874/2570

CrossRef Full Text

Puuska, H.-M., Muhonen, R., and Leino, Y. (2014). International and domestic co-publishing and their citation impact in different disciplines. Scientometrics 98, 823–839. doi: 10.1007/s11192-013-1181-7

CrossRef Full Text | Google Scholar

Robinson-García, N., Torres-Salinas, D., Zahedi, Z., and Costas, R. (2014). New Data, New Possibilities: Exploring the Insides of Altmetric.Com. arXiv preprint arXiv 1408.0135.

Robinson-Garcia, N., Trivedi, R., Costas, R., Isett, K., Melkers, J., and Hicks, D. (2017). Tweeting about journal articles: engagement, marketing or just gibberish? arXiv preprint arXiv:1707.06675.

Sanz-Casado, E., De Filippo, D., and Alexandre-Benavent, R. (2017). “Classification model of Spanish scientific journals in social sciences and humanities,” in 22nd Nordic Workshop on Bibliometrics and Research Policy, 9-10 November 2017 (Helsinski).

Google Scholar

Sanz-Casado, E., García-Zorita, C., Serrano-Lópeza, A. E., Filippo, D., and Vanti, N. (2016). Desarrollo de indicadores para los nuevos hábitos de información y comunicación científica. Educ Med. 17, 45–50. Available online at: https://medes.com/publication/116134

Google Scholar

Schloegl, C., and Gorraiz, J. (2010). Comparison of citation and usage indicators: the case of oncology journals. Scientometrics 82, 567–580. doi: 10.1007/s11192-010-0172-1

CrossRef Full Text | Google Scholar

Serrano-Lopez, A. E., Ingwersen, P., and Sanz-Casado, E. (2017). Wind power research in Wikipedia: does Wikipedia demonstrate direct influence of research publications and can it be used as adequate source in research evaluation? Scientometrics 112:1471e1488. doi: 10.1007/s11192-017-2447-2

CrossRef Full Text | Google Scholar

Sivertsen, G. (2016). Patterns of internationalization and criteria for research assessment in the social sciences and humanities. Scientometrics 107, 357–368. doi: 10.1007/s11192-016-1845-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Tavares de Matos Cardoso, M. M. (2011). El peer review de las revistas científicas en Humanidades y Ciencias Sociales: políticas y prácticas editoriales declaradas. Revista Española de Documentación Científica, 34, 141–164. doi: 10.3989/redc.2011.2.796

CrossRef Full Text | Google Scholar

Torres-Salinas, D., Cabezas-Clavijo, A., and Jiménez-Contreras, E. (2013). Altmetrics: new indicators for scientific communication in Web 2.0. Comunicar 41, 53–60. doi: 10.3916/C41-2013-05

CrossRef Full Text | Google Scholar

Van Noorden, R. (2014). Online collaboration: scientists and the social network. Nat News 512, 126–129. doi: 10.1038/512126a

PubMed Abstract | CrossRef Full Text | Google Scholar

Vanclay, J. K. (2008). Impact factor: outdated artefact or stepping-stone to journal certification? Scientometrics 92, 211–238. doi: 10.1007/s11192-011-0561-0

CrossRef Full Text | Google Scholar

Waltman, L., and Van Eck, N. J. (2012). A new methodology for constructing a publication-level classification system of science. J. Am. Soc. Information Sci. Technol. 63, 2378–2392. doi: 10.1002/asi.22748

CrossRef Full Text | Google Scholar