Student participation in STEM is often reported at a broad level, with information detailing the counts of students who are enrolled in each subject, and how this differs across demographic groups. In Aotearoa New Zealand, data is readily available by sex (male or female) and Socio-economic status (SES) from 2004 to 2018 (Ministry of Education, 2018). Exploring these data can provide a surface level description of what the field of STEM education looks like in Aotearoa New Zealand.

As shown in Figure 1, in Year 13 (final year of high school), female students in Aotearoa New Zealand are less likely to take physics, with this under-representation being steady across years. The same figure shows that female students are more likely to take biology, and more recently chemistry, with this over-representation becoming increasingly more pronounced over time.

Figure 2 shows that female students continue to be underrepresented in mathematics subjects, such as accounting and calculus. However, female students have higher levels of representation in statistics than male students in recent years (Ministry of Education, 2018). The same data shows that, in technology subjects, the computer and engineering subjects have continually been male-dominated, with this becoming more pronounced over time (Ministry of Education, 2018). Food technology and textiles are the only female dominated technology domains.

Data from Ministry of Education (2018) also allows us to see trends in STEM participation by school decile, a proxy measure of SES. In Aotearoa New Zealand, school decile refers to the affluence of the neighborhood in which a school is located. High decile schools are located in more affluent areas, while low decile schools are located in less affluent areas. As shown in Figure 3, students who attended higher decile schools had greater rates of participation in science subjects, and this pattern was also evident for calculus and statistics. The relationship between student enrollment in technology learning domains and decile has no discernible pattern.

Broad level data, such as those discussed above, allow us to interpret trends in subject enrolments over time. However, they provide only a surface level understanding of STEM participation. Beneath the aggregation of counts per subject label hides important information that is useful for policy makers and researchers. Each subject consists of many different assessments, each covering unique content and following different assessment criteria. The following section will provide a brief introduction to Aotearoa New Zealand’s main high school assessment system, the National Certificate of Educational Achievement (NCEA).

The National Certificate of Educational Achievement (NCEA) is the main form of secondary school assessment in Aotearoa New Zealand. First introduced to students in 2002, NCEA was designed to replace norm-referenced assessment. In norm-referenced assessments student achievement is judged against the average achievement of the student population (Mahoney, 2005). Instead, achievement in NCEA is based on the competencies of individual students (Hipkins et al., 2016), meaning that achievement is an indicator of what a student knows, and not just how they rank among their peers. Therefore, it is possible for all students to pass if they all meet the assessment criteria. Assessment operates at the level of specific skills, or standards, that comprise a subject discipline. For example, instead of just receiving an overall grade for biology, students take several standards in the subject discipline of biology that demonstrate their competence in particular areas (e.g., “Demonstrate understanding of biological ideas relating to micro-organisms”). By successfully completing standards, students accumulate credits, the value of which is linked to the amount of work needed to fulfill a standard. The three levels of the NCEA typically correspond to the final three years of high school. NCEA Level 1 is typically taken in Year 11 (age ∼15), NCEA Level 2 in Year 12, and NCEA Level 3 in Year 13.

What makes the NCEA a unique assessment system is its flexibility. Compared to the systems it replaced (School Certificate, Sixth Form Certificate, and Bursary), there is more variety in the assessments/standards that students may be enrolled in (Mahoney, 2005). In providing increased choice to students and their educators, and more flexible pathways through high school, it was hoped that the NCEA would benefit students from a range of backgrounds. As stated in the New Zealand curriculum (Ministry of Education, 2007, p. 41):

Schools recognize and provide for the diverse abilities and aspirations of their senior students in ways that enable them to appreciate and keep open a range of options for future study and work. Students can specialize within learning areas or take courses across or outside learning areas, depending on the choices that their schools are able to offer.

The NCEA meets these goals by providing students with more learning pathways through high school, which aims to serve both students who wish to progress to tertiary study, and those who want to enter into vocational careers. These two pathways are reflected in the two main types of assessment offered: unit and achievement standards.

We make use of Statistics NZ’s Integrated Data Infrastructure (IDI) to access administrative data pertaining to students’ high school and census information (Statistics New Zealand, 2018). The IDI is a collection of government data sets, containing micro-data on student enrollment and demographics, linked at the level of individuals for the population of Aotearoa New Zealand. We focus on students taking NCEA Level 3 from 2010 to 2016, as this is the most up to date data available at the time of writing. We focus on NCEA Level 3 as this level is the most highly specialized, and precedes entrance to university and employment. Years prior to 2010 are available, but were omitted due to processing constraints. The years spanning 2010 to 2016 were also of specific interest, due to education policy reforms introduced around 2012 and 2013 (Hipkins et al., 2016).

We apply several rules when selecting student cohorts to be included, in order to minimize the risk of adding statistical noise to our analysis. In order to focus our analysis on students who have had the majority of their education in Aotearoa New Zealand, we only select individuals who are identified as having tax, birth or visa records present in the IDI. We also only include students who had NCEA records when they were 15 or 16 and during NCEA Level 1. These filters help focus our sample on the resident population of Aotearoa New Zealand, and minimize the chances of including visitors or foreign exchange students. We also limit our sample to students who attended state schools in Aotearoa New Zealand. This is because private schools in Aotearoa New Zealand are more likely to offer a combination of the NCEA and other formal qualifications (such as Cambridge or International Baccalaureate), introducing additional layers of complexity. For the purposes of our analysis we also assign each student a single cohort year based on the most frequent year in which they took standards. This is because students are able to take NCEA Level 3 standards over multiple years. For example, if a student took two NCEA Level 3 standards during 2015, and ten NCEA Level 3 standards during 2016, we would assign the student to the 2016 cohort. We choose not to exclude Level 3 standards taken in a different year from the overall cohort year, as these standards would still contribute to the student’s qualification.

We include the following variables in our analysis:

• Students’ sex (male or female). Due to limitations in the administrative data used, we are not able to include gender (and non-binary classifications of gender) in our analysis.

We employ network analysis to understand STEM enrollment at NCEA Level 3 at a fine-grained level. At its fundamental level, a network is a collection of nodes and edges. Nodes can represent an agent (e.g., a student) or an object (e.g., a standard); edges link two nodes together to indicate some form of relationship. Networks can be used to represent anything from human relationships and transport networks, to biological and computer systems (Barabási, 2003). In education research, network analysis has tended to focus on the relationships shared between students in the classroom (Tranmer et al., 2014), or communication between staff at educational institutions (Daly, 2010). There are few examples of education research that use network analysis to investigate non-social relationships. We seek to expand this area of research by applying network analysis to high school assessment enrollment data. As we will outline in the following section, network analysis can help us identify patterns in NCEA standard co-enrolments.

In our analysis, nodes take the form of students and standards. Edges in our network represent any recorded instance where a student was enrolled in a NCEA Level 3 STEM standard during high school. This creates a bipartite network (also commonly referred to as a two-mode network). A bipartite network is any network where there are two types of node, and nodes can only connect to a node of a different type. In our case, a standard cannot be connected directly to another standard, and a student cannot be connected to another student. For example, in Figure 4A, standards may be represented by nodes in set U, and students may be represented by nodes in set V.

We create a network of students and the standards they were enrolled in for the whole of our student population. We structure this network so that it is multidimensional. Each student node belongs to a specific year, region, and decile, while standards can exist across multiple years, regions, and school deciles. In order to analyze the properties of our network, we are required to ‘project’ onto one set of nodes. This means that we take the node set belonging to a single node type, and generate edges between these nodes when they are linked to a common node of the other node set. For example, in Figures 4B and 4C shows the projection of the network in Figure 4. In the projections, standards represented in set U are now connected to one another (B), and students in set V are also now connected (C).

As we are interested in the patterns of standards that students took, we project onto the standard nodes (Figure 4B). This results in a network of standards that are connected by edges indicating that students took those two standards together within their NCEA Level qualification. The edges of the projected standard network can also take on a weighting that corresponds to the frequency that two standards were taken together by students.

Our goal is to use the co-enrolment network to understand the standards that tend to be taken together, and by which students. To do this, we employ community detection. Community detection is a process in which we identify sets of nodes that are clustered together by the edges in the network. Previous research by Ferral (2005) has employed similar clustering techniques to investigate communities of subjects that tend to be taken together in the NCEA, but this was limited by the number of high schools sampled, the response rate of schools, and the availability of demographic and standard information. Usually, community detection methods identify communities by maximizing the modularity score within communities. Modularity refers to the tendency of nodes to connect to other nodes within the same community relative to nodes that are outside the community. While there are many different community detection algorithms, the current study makes use of the infomap algorithm (Rosvall et al., 2009).

In order for our communities to more truly represent the standards that tend to be taken together, we need to normalize our edges so that weights do not refer to the raw counts of students’ co-enrolments. The raw weighting does not consider the fact that standards have different populations of students. As a result, community detection may group two standards together simply because one standard has a large number of students. To explain more clearly, we can use the hypothetical case of English Standard A, Physics Standard A and Physics Standard B. If English Standard A has a population of 1,000 students, and 10% of those students take Physics Standard B, the raw weight is 100 students. If 100 students took Physics Standard A, and 90% of those students took Physics Standard B, the raw weight is 90 students. While we would expect the two physics standards to be grouped together, using raw counts of students as edge weights may not result in grouping that meet these expectations. Instead, we make use of a normalization technique called Revealed Comparative Preference (RCP) to provide more consistent communities. RCP is a ratio of ratios that measures the fraction of students from standard j who also took a second standard i, relative to the overall fraction of students taking standard i, across all other standards. More specifically: RCP ( i , j ) = x i j / ∑ j x i j ∑ i x i j / ∑ i j x i j = x i j / x i x j / x = x i j ⋅ x x i ⋅ x j where x _i,j is the number of students taking both standard i and j, x j (or x _i ) is the total number of students taking standard j (respectively, standard i), and x is the total number of unique students enrolled in any standard. This RCP metric is based on the measure Revealed Comparative Advantage, used in economics (Balassa, 1965), and was calculated using the EconGeog package (Balland, 2017) in R (R Core Team, 2013). The RCP calculation returns a value where anything greater than one indicates a ‘preference’ for two standards being taken together. Conversely, a value below one indicates that, given the number of students in either standard, there was a dispreference for the two standards being taken together.

We remove any edge in the network where the RCP value is below 1, and subsequently any node that no longer has any edges (isolated nodes with a degree of 0). This results in removal of around 31% of edges. Following this pruning stage, the network now consists of standards connected by edges with a weighting relative to the preference for each standard being taken together with its neighbors in NCEA Level 3. We then identify communities of standards that are grouped together in our network using the infomap community detection algorithm (Rosvall et al., 2009). In simple terms, the infomap algorithm partitions the network in a way that maximizes the number of edges within a community, relative to the edges between communities.

To compare student participation across the educational fields detected, we can consider the relative proportion of students from particular years, and across school deciles and social groups. One of our goals is to establish an idea of how each network is structured. Are the enrolments for a specified demographic group spread more evenly across a network, or are they focused in particular areas? To answer this question, we employ the metric of entropy. Entropy is a concept originating from the field of thermodynamics, and provides an indication of how organized or disorganized a system is. In the case of the current study, we use entropy to assess how participation is spread across the network. Using the measures of entropy as signals of disparities, we then explore the rates of participation across communities and standards in finer detail. The following section will outline our measure of entropy.

Overall, there were small differences in the entropy in the network by sex, with entropy being slightly higher for the male sub-population (see Figure 7). This finding suggests that the male sub-population of the network had more enrolments spread across the network, while the female sub-population were more focused in specific areas. Further investigation of communities in the network showed clear examples of disparities in subject enrolments by sex which may explain the difference in entropy. Male students tended to dominate communities defined by standards in the agriculture, engineering, and practical technology (welding, furniture making etc.) domains, while female students had greater rates of enrolments in standards related to life sciences and textiles.

Corroborating the broad level trends outlined previously in the current study, and the trends detailed across other international contexts (Else-Quest et al., 2013; Institute of Physics, 2013; Sheldrake et al., 2015; National Science Foundation, 2017), female students were more likely to enroll in biology standards, and less likely to enroll in physics and calculus standards. The majority of biology standards had around 60–70% female students across years, while female students were also more likely to have enrolled in standards in the Core Science domain. This domain includes standards such as Research a current scientific controversy (61.5% female) and Describe genetic processes (67.3% female). Female students were less likely to be represented in calculus and physics standards than male students. Investigating these disparities at the standard-level provides additional insights (see Table 1).

The rates of enrollment for female students in the physics standards were low, with the proportion of female students in externally assessed physics standards being around 35% overall. The participation of female students in the standards related to calculus were also low compared to male students, with the proportion of female students being around 35–38% in the main externally assessed standards. Interestingly, the internally assessed unit standard equivalents of the calculus standards, which were available to students prior to 2013, had an increased proportion of female students (around 42–46%). Much research has been dedicated to understanding why disparities persist in physics and calculus by sex, with research often suggesting that female students tend to be less confident in mathematics and calculus compared to male students (Heilbronner, 2012; Simon et al., 2015; Hofer and Stern, 2016). It may be that the calculus unit standards, which are assessed internally, in a familiar space with the opportunity to resit, offers a safer assessment environment where female students are more comfortable (see Cheryan et al. (2017) for a comprehensive review of the issues impacting on gender differences in STEM choice).

We report the results for ethnicity and school decile together, given that they are inextricably linked; Māori and Pacific students are over-represented in low decile schools. As can be seen in Figure 8, entropy differed across groups, across deciles, and changed significantly over time. With that being said, the Asian sub-population tended to have lower entropy overall. The Māori and Pacific sub-populations had higher entropy prior to 2013, but this pattern changed in later years with the entropy of Pacific sub-population decreasing to the same level as Asian. Overall, we observed how ethnic group differences in entropy were greater prior to 2013, and more similar in more recent years. The reduced difference between the entropy of each ethnic group sub-population can be explained in terms of the pivotal reform in the NCEA that took place around 2013 (Hipkins et al., 2016). These reforms saw curriculum-linked unit standards phased out of operation, and the system was made to be more standardized and less flexible. Māori and Pacific Island students, who were over-represented in these curriculum-linked unit standards, saw the greatest drops in entropy following the removal of these standards.

Looking across deciles, Figure 8 shows that the top decile school sub-population tended to have lower entropy compared to the other deciles across years. The observed drop in entropy following the policy reform around 2013 is also much smaller than (approximately half) the drop seen for middle and low decile schools. In the years following 2013, the differences in entropy between each ethnic group sub-population at the top decile schools appears to be more compressed, with the entropy for Māori, Europeans and Pacific sub-populations more closely representing the level of Asian. In contrast, lower decile schools had higher entropy, and a greater drop following the policy reform.

The higher entropy for lower decile schools may be a consequence of increased enrolments in internally assessed standards, and fewer enrolments in standardized, externally assessed standards which have historically been a stronger focus for top decile schools (Hipkins et al., 2016). Previous research has commented on this pattern, with Wilson et al. (2017) observing that lower decile schools were less likely to have students enrolled in Subject Literacy Achievement Standards, which are achievement standards that can be used as indicators of subject-specific literacy. After exploring the rates of enrollment, we also confirm that lower decile schools are less likely to have students enrolled in key externally assessed science and mathematics standards.

The lower entropy for top decile and Asian students is likely related to a focused participation in science and mathematics achievement standards required for university entrance. In contrast, enrollment for Māori and Pacific sub-populations has been less focused in these standards and more balanced across other domains (including communities of unit standards; see Figure 9). We explore this further by comparing the rates of participation in key externally assessed science and calculus standards. Table 2 shows how enrollment differed for each ethnic group sub-population, split by school decile and comparing 2010 to 2016.

While Table 2 shows that Asian and European had higher rates of participation in key externally assessed science and calculus standards, the differences between low and high school deciles appears to be considerable for the Asian sub-population compared to other groups. For example, the difference in participation for Asian students by decile in the calculus standard on differentiation offered in 2016 is 22%, compared to 6.7% for European, 6.4% for Māori, and 4.7% for Pacific Islands. This may be a consequence of the categorical grouping of “Asian”, which contains an extremely diverse population of students. This categorization ranges from Pakistan and Bangladesh to China, and also some Pacific Island nations (e.g., Fijian Indians). The diversity of the population, including the cultures and social backgrounds, may have been reflected in an increased diversity of enrolments. The different trends we observe in these key standard enrolments may explain why the Asian sub-population in low decile schools had the higher entropy than other groups in 2015 and 2016 (see Figure 8).

The fact that low decile and Māori and Pacific sub-populations had fewer enrolments in key science and mathematics standards provides evidence that the pathway to university science is dominated by higher decile schools, and especially Asian and European students at these schools. In contrast, students from lower decile schools, and also Māori and Pacific students in higher decile schools, had relatively more enrolments in a larger and more disparate pool of internally assessed unit standards. Unit standards provide a valuable type of assessment that prepares students for vocational careers, and it may be that a higher proportion of students from less affluent areas seek vocational careers after high school. However, this does not necessarily explain why Māori and Pacific sub-populations attending higher decile schools are less likely to participate in science and mathematics standards.

The differing patterns of enrollment for Māori and Pacific Island sub-populations and Asian and Pakēha is complex, but may be explained in several ways. Firstly, higher decile schools primarily serving Māori and Pacific students may choose to offer more internal assessments that provide increased opportunity to assess in culturally appropriate way (e.g., less competition, more formative feedback). Secondly, and less optimistically, it may be that teachers hold lower expectations for Māori and Pacific students (Turner et al., 2015), and are less likely to place them in the pathway toward university science. This idea was reflected by a participant in a study by (Graham et al., 2010): The teachers decide where the class is at in terms of choosing which standards [Unit vs. Achievement]. It’s a disadvantage on you because it depends on what the teacher thinks you can do and what the kids in your class can do.

Our results suggest that policy reforms introduced around 2013 did not result in a decrease in participation in science and mathematics for Māori and Pacific Island sub-populations, who were originally over-represented in the curriculum-linked unit standard in earlier years. Instead, as can be seen in Table 2 enrollment often increased at a greater rate than other ethnic groups, especially in higher decile schools. For example, in high decile schools the Pacific Island sub-population saw an larger increase of around 10.6% in external biology standards relating to evolution, compared to 0.9% for Asian, 2.8% for European, and 4% for Māori. Although we cannot comment on how this educational reform impacted on students’ outcomes in science and mathematics, the reduced flexibility may actually help students by making NCEA less complex. Previous research has found that the complexity of NCEA can be confusing for students and parents to navigate (Graham et al., 2010; Jensen et al., 2010).

The current study fills a gap in the previous literature by investigating patterns of co-enrolments in NCEA Level 3 STEM standards by students’ sex, ethnicity, and a proxy measure of SES. We believe that this study is the first of its kind to use bipartite networks to represent high school assessment data. Through our methodological approach, we are able to take into account a wealth of information related to students and the standards that they enrolled in. This includes demographic information (such as sex and ethnicity) and specific NCEA Level 3 standard information, such as the manner in which standards were assessed (externally or internally), and whether the standard was an achievement standard (traditional curriculum based subjects, such as English or science) or a unit standard (more vocational subjects, such as farming or practical technology).

The NCEA is very complex, but our method of analysis allows us to consider the different pathways that students follow based on the assessments they enrolled in. The communities of standards highlighted through our analysis reflect two main pathways, either toward vocations and careers from science, or the pathway toward university and careers in science. Despite growing discussion regarding the outcomes of different types of standards in the Aotearoa New Zealand context (Hipkins et al., 2016; Lipson, 2017), there has been a lack of research into how this information relates to student background. The methodology and results outlined in the current study enables us to represent the NCEA as a complex education system, and this can provide detailed insights into what science participation looks like.

A limitation of our analysis is the fact that we do not have access to students’ level of achievement in the standards they enrolled in Level 3, or in previous years. As detailed by Jensen et al. (2010), achievement outcomes in standards would be highly influential in shaping the pathways that open up or close off for students as they go through NCEA. Furthermore, the disparities seen in participation in key science standards may be tied to the development of academic identity (Bolstad and Hipkins, 2008) which we are also unable to quantify. (Archer et al., 2014, p. 216) argue that ‘cleverness’ [can be viewed] as a racialized, gendered, and classed discourse, such that the identity of the ‘ideal’ or ‘clever’ student is not equally open to all students as a viable and authentic identity. This notion of ‘cleverness’ may explain the disparities found in the current study. More specifically, it may be that the ‘clever’ pathway through NCEA is not open to all students. As described by Hipkins et al. (2016), NCEA informally developed into a two-tiered system, with curriculum-linked unit standards commonly being viewed an easy pathway, and achievement standards, and especially externally assessed achievement standards, being viewed as a tougher pathway. Students who identify as less academic may purposefully seek easy pathways through NCEA, without fully understanding that doing so can reduce educational opportunities later on (Jensen et al., 2010).

Students with a family background of success in education may be more likely to view the academic pathway as normal or even expected. This idea is described in a related study of high school science pathways in the United Kingdom, where Archer et al. (2017) found that students from more affluent backgrounds were more likely to see the science-orientated pathway as an ‘obvious’ choice. Students from less affluent backgrounds may also be more motivated to seek full time employment, rather than pursue a pathway toward university study and the debt it may entail. However, the question remains as to the extent to which student from less affluent backgrounds knowingly choose vocational pathways and are not channeled down this pathway by simply attended a school in a low SES area.