Smart rogaining for computer science orientation

1. Introduction

1.1. Background and motivations

Computer science specialists are needed all over the world. Despite the wide range of available professional roles, the number of graduates does not match the demand of the labor market (Kori et al., 2018). Education systems are dealing with the impact of technology and the fast changing need for new skills and knowledge. Furthermore, in the last years, education systems are dealing with the impact of technology and the fast changing need for new skills and knowledge. For all these reasons, the problem of preparing young people for careers in computer science has attracted considerable attention. Many important initiatives have been proposed by non profit international organizations, some of which gained worldwide popularity such as the Code.org initiative (Partovi and Sahami, 2013; Partovi, 2014)¹ and the joint Informatics Europe and ACM Europe Working Group on Informatics Education (Gander et al., 2013; Caspersen et al., 2018, 2019). In April 2022, the project Programma il Futuro² inspired by Code.org and by the Computational Thinking principles (Wing, 2006) received the first prize for promoting digital competences in Italy.

To support computer science education, a wide range of new technologies and applications have been developed both in industry and academy. For instance, the Scratch Foundation guided by Mitchel Resnick, Professor of Learning Research at the MIT Media Lab, in collaboration with Google, developed Scratch (Scratch, 2021) a very popular visual programming language for beginners (Resnick et al., 2009; Maloney et al., 2010). The Catrobat organization headed by Professor Wolfgang Slany worked for more than ten years to a mobile version of the Scratch language called PocketCode (Müller et al., 2019; Catrobat Pocketcode, 2021). The Micro:bit Foundation designed a series of programmable and wearable devices for beginners (Sentance et al., 2017).

More in general, initiatives for promoting computer science are nowadays organized by most of the colleges and universities around the world (Vegas et al., 2021). In the meantime, formats such as scavenger hunt (Talton et al., 2006; Hutzler et al., 2017; Manzoor et al., 2020), role games (Blecic et al., 2007), online polls (Glencross and Elsom, 2019) are becoming more and more popular for student orientation, especially when combined with the use of emerging technology, i.e., mobile applications, wearable sensors, virtual, and augmented reality, etc.

1.2. Research goal

Our overall goal is to increase the number of computer science students at the University of Genoa, and to enroll informed students, since our degree (as most Computer Science Bachelor's degrees in Italy) is characterized by a high dropout rate. In this respect, internships for high school students play a crucial role: they represent the last chance to capture the attention of undecided prospective students, and to provide an exposure to the topics that will be studied in the degree. Internships have a limited duration (<1 week) and combine structured activities, i.e., seminars, meetings with instructors, hands-on labs, mini-projects, etc., with half-day slots in which it is possible to experiment new formats.

Our specific research goal is to design reproducible formats for half-day activities that could help prospective students in appreciating the richness and professional potential of our discipline. All activities shall be organized by academic teachers in collaboration with University students and they shall be offered to high school students with different background, and potentially completely novices to computer science.

Despite the large number of existing computer science dissemination initiatives, our research question has to face some important issues:

• Topic Coverage: A first challenge is to provide, in a short time, a sufficiently broad overview of the different topics covered in a computer science degree, with a right balance between methodological and technological aspects. To appreciate the difficulty, the 2012 Computing Classification of the Association for Computing Machinery³ gives a comprehensive list of subject categories of the discipline. They range from the Theory of Computation to Applied Computing, i.e., education, law, economy, healthcare, etc.

• Soft-skills: A second challenge is to convince prospective students that computer science also requires soft skills (e.g., teamwork and good interaction skills; Lingard and Barkataki, 2011), combined with the ability to adapt to the rapid evolution of technology. In addition, as an orientation activity for university studies, we aim at stressing the importance of soft skills that are crucial for higher education, such as planning, time management, goal prioritization, work under stress, reaction to unexpected events.

• Gender-gap: A closely related challenge is finding strategies for the mitigation of gender gap, a problem for the vast majority of computer science degrees (Happe et al., 2021).

1.3. Contribution

To face the above mentioned challenges (topic coverage, soft-skills, and gender-gap), in this paper we present a novel teamwork format for internships of limited duration (half day) inspired by rogaining outdoor activities and, more in general, by strategic team reasoning.

Rogaining (Phillips and Phillips, 2000) is an outdoor orienteering sport involving both route planning and navigation between checkpoints using a variety of map types. In a rogaine, teams usually consisting of two to five members choose which checkpoints to visit within a time limit with the intent of maximizing their score. Although endurance and competition are important aspects, teamwork is probably the central feature of this sport. Rogaining is indeed one of the favorite activities in corporate events organized by companies. Differently from scavenger hunt, rogaining requires some form of strategic reasoning. Teams are forced to select a strict subset of checkpoints to visit during the competition. Indeed, by design, the total duration of all activities is larger than the duration of the rogaine.

Our format is aimed at improving students' engagement with respect to traditional internships activities. It is mainly designed as an indoor navigational activity with a limited time duration (4 h). Gamification drives the design and implementation of our approach: participants, divided into teams, are involved in a rogaine consisting of several checkpoints organized as a series of practical activities of the same duration. The activities proposed in the checkpoints cover different subject categories of computer science and combine soft-skills and computational thinking tasks.

Students are exposed to computer science both with learning goals (e.g., through labs and hands-on activities on basic coding concepts) and dissemination goals, to let them approach more advanced and challenging topics through short talks, demos, and exhibits. Since 2019 and before the COVID-19 pandemic, the format has been applied in orientation events for high school students of the Computer Science Bachelor degree of the University of Genoa.

The proposed format is the result of a joint design effort between Computer Science instructors, team-building experts (Edutainment Formula) and psychologists. Teamwork turned out to be quite effective for stimulating students' engagement in hands-on activities and projects, especially considering that our interns are part of a large heterogeneous group of students. More in general, the format lets the participants exercise different soft skills, ranging from intra-personal skills like adaptability and flexibility, to inter-personal skills like negotiation, to methodological skills like problem solving.

The management of the activity is supported by a mobile and web application, namely the SR (Smart Rogaining App), developed in collaboration with Edutainment Formula. Its functionalities have been tested following the orchestrated crowdsourced testing approach described in Leotta et al. (2019). SR allows teams to explore a map with information on checkpoints (description, scores, etc.), book a checkpoint for the team, and check whether a checkpoint is currently locked by another team or not.

An experimental evaluation of the format is then presented, based on data collected in the 2019 and 2020 editions at the University of Genoa with 135 and 62 participants, respectively. We believe that a discussion of our experience could be helpful to evaluate pros and cons for reproducing a similar format in other institutes and, perhaps, in other disciplines. Since bachelor, master and Ph.D. students are involved as active parts of the organization and as mentors for the checkpoint activities, our format also offers the opportunity of improving soft skills of university students.

The paper presents an integrated and extended version of the preliminary works discussed in Delzanno et al. (2020a), Delzanno et al. (2020b), Noceti et al. (2020), and Ballestin et al. (2020).

1.4. Plan of the paper

The remainder of the paper is organized as follows. In Section 2, we introduce our teamwork rogaining format as well as learning goals for both participants and university students. We also discuss the advantages in using a web and mobile application in conducting the event. In Section 3, we describe the challenges used in the two editions. In Section 4, we discuss the experimental evaluation of the format based on relevant data collected in the selected editions. In Section 5, we discuss and compare our approach with related work. Finally, in Section 6, we address conclusions and future directions for our research.

2. Smart rogaining for orientation to computer science

In this section, we present our format, discuss the expected learning outcomes for participants and staff members (i.e., University instructors and students), and the relevance of having the activity supported by a web application, that makes the rogaining smart.

2.1. Rogaining format

Our teamwork format is based on a rogaining activity with a duration of 4 h. The rogaine consists of a collection of checkpoints distributed in different areas of the campus of the Faculty of Science of the University of Genoa. In the two reported editions we selected rooms from different buildings, floors, departments in the campus of the Science Faculty of our University in order to get closer to traditional navigational competitions. A score is assigned to each checkpoint based on the distance from the home base and on the difficulty of the proposed exercise. As we will describe later in this section, we proposed different types of challenges ranging from problem solving to tinkering, coding, and programming labs. In general, it would be desirable to assign different roles or tasks to group members in each challenge. Prior to starting with the activity, participants are required to fill in a questionnaire to identify their background, skills, and aspects of individual personality that are used to form the teams. The goal is to group students in homogeneously heterogeneous teams with respect to different axes (e.g., different schools, background, skills) so as to balance the competition during the game. Specific attention is paid to gender balancing and to avoiding putting interns from the same school or already knowing each other in the same team.

During an initial briefing, a map of the event location and a short description of the checkpoint catalog is assigned to each team. The number of checkpoints must be greater than the number of teams (at least 20%). Furthermore, teams are forced to select a subset of checkpoints checkpoints (i.e., the total duration of the event must be much less than the total time required by all checkpoints). In the two editions we proposed 15 checkpoints for 12 teams and 12 checkpoints for 9 teams, respectively. The duration of the activity at each checkpoint is 30 min, so the target number of checkpoint to complete is 6–7. Teams have 20 min to select an initial strategy (i.e., to select the checkpoint list to visit during the challenge). Before starting the game each team has to communicate to the game staff a final score prediction based on their strategy. The strategy itself is kept secret. The score prediction will be useful in the post-game briefing. Two mentors are assigned to each checkpoint. They are in charge of the following tasks: explain the activity and the exercise, assign the points to the team, evaluate the behavior of team members, and, finally, notify that the lock has been released to all other teams. Staff members have to provide support for implementing this kind of non trivial interaction between different teams and checkpoints.

Checkpoints are locked by teams via the SR. Teams are required to lock the next checkpoint right after the conclusion of the current one. Therefore, teams may have to dynamically modify their strategy and navigation plan.

When the rogaine time expires, teams return to the home base for the conclusion of the game. The organization staff collects score and timings of each group and presents the final ranking of the game comparing expected and achieved results. Participants are then required to fill in a peer observation form (in the spirit of Bales Interaction Process Analysis; Bales, 1950) for each member of their team.

2.2. Workflow specification of the rogaining activity

Figure 1 shows an UML-like diagram of the workflow associated to the team-based rogaining activity described in the paper.

FIGURE 1

Figure 1. Smart Rogaining architecture.

In the UML scheme the rogaining scenario is divided in two workspaces: the outdoor workspace where the on-the-field team members move through checkpoints and where domain experts organize their challenges, and the control room for organizers and team coordinators, the latter ready to advise and interact with their teammates.

The team formation is a fundamental stage and is supposed to take place in a preliminary preparation phase. Each team is equipped with a map of the area. The map can be either static (can be read by anyone and shows where challenges are located and their peculiar information) or dynamic, providing dynamic and updated information on where the teams are located and the status of each challenge (free or occupied). The dynamic map is an extension of the static one, but it can be accessed only by people in the control room workspace. Challenges are created by domain experts who define their features such as the topic of the challenge, the points that can be earned by solving it and the difficulty. Each challenge takes place in a different location of the event area—inside the outdoor workspace. Team members can face the challenge only when they physically reach its location. The general scheme also considers the possibility of using the SR to submit quizzes to the different teams during the rogaine, e.g., to provide additional information on the proposed laboratories or to simply to increase the difficulty level of the challenge.

The domain expert is responsible for creating the various challenges according to the theme of the event. Each team is uniquely identified via the team credentials used to access SR. The idea behind splitting inside a team is not a physical, but only a logical, role-based one, for better shaping “who does what” based on the participants attitudes. Sub-roles of “on-the-field” members can be for example the team leader, the challenge leader, and the query leader. The team leader emerges during the game progress according to how the members behave and interact with others. The other two roles are also assigned at runtime and can be adopted, possibly by different members, each time a team reaches a challenge. The on_the_field team members other than the current challenge/query leaders will support either the resolution of the challenge or query answering. The last role is the team coordinator. She/he has a complete view of the situation on the field and can suggest the next move when replanning is needed. The main stages of the process, which are depicted in Figures 2, 3. We do not include briefing and the debriefing stages. The organizers set up the rogaining event by exploiting some team formation algorithm and by communicating to each participant the team they belong to. Once everything is set, each team has some time to consult the map, discuss and exploit some strategy creation algorithm in order to choose jointly the best strategy to follow. At that time, in each team an initial internal structure, with roles associated with participants, is arranged. Each team waits for the three-two-one-go command of the organizer to then move to the first challenge chosen according to its strategy. In the event that such challenge is occupied the team moves to one of those not faced yet.

FIGURE 2

Figure 2. Smart Rogaining team preparation.

FIGURE 3

Figure 3. Smart Rogaining game.

2.3. Learning goals for teams and university students

Checkpoint activities are proposed by volunteering bachelor, master, and Ph.D. students in collaboration with staff members. This feature has the nice effect of introducing a novel type of soft skills activities in our degrees. Indeed, in most of the cases the design of a checkpoint lab requires a software artifacts specifically designed for computer science education. This task turns out to be a quite non standard, but still interesting, exercise for our students. Mentors have to deal with organizational issues in order to manage teams and have to strictly interact with staff members before, during and after the game. This creates new communication channels between students and staff members that go beyond what is required in a standard study program (teamwork in teamwork!). In both editions, mentors at checkpoints running the activities are students. Mentors are required to evaluate, via simple questionnaires, the level of engagement and the degree of success in completing the activity of each team, thus providing feedback to the entire organization process.

The game structure itself embeds non trivial computer science concepts. For instance, rogaining is built on top of an important concurrent programming pattern (i.e., synchronization). Finding the initial plan is an instance of a variation of the well known traveling salesman problem with additional constraints induced by team members background and skills.

During the game at any given time, each checkpoint is assigned to a unique team. Since it is not reasonable to lock all checkpoints in advance, teams are required to lock the next checkpoint right after the conclusion of the current one. Race conditions are admitted in this phase. They are solved by using a FIFO discipline with no pending queues (i.e., if the planned checkpoint is locked), teams have to select another free checkpoint and retry in the next round.

Teams have no tutors during the game. Instead they had to follow instructions, maps and signs, understand the rules, ask questions to other teams and mentors. The event is an anticipation of their future student life, in which they will be required to move from one lesson to another, schedule their time for labs and exams, etc. Most important, they experience that sometimes is not possible to complete everything in due time, that they may need to give up something and take decisions, and that plans need to be realistic. Carefully planning is crucial, but unexpected events may happen (e.g., the checkpoint we wanted to book is not available) so teams have to dynamically modify their navigation plan, a frequent situation in a student career.

Technology enables a diversification of activities, different perspectives for common concepts (e.g. coding via visual languages, modal interfaces, etc.), animate the checkpoint activities, provide interesting links for possible insights on theoretical aspects. We decided to introduce the rogaining model only after the adoption of a smart app (SR) to manage the entire event. The app is used by teams and mentors and provide functionalities to lock (teams) and unlock (mentors) checkpoints, and to notify these operations and the current score of each team to all participants. All these information are shared among all participants via a map embedded in the user interface (see Figure 4). The current game state is displayed on a large screen in the home base so that game manager can monitor both teams and checkpoints. Telegram is used for fast communication among staff members. Each team is equipped with at least one tablet used to monitors the entire game using the SR as shown in Figure 4. The mobile app is used for navigational purposes (to find the checkpoints), to lock and release a checkpoint.

FIGURE 4

Figure 4. SR-App Campus map with challenges locations (left), and a team leader with the SR-App (right).

The introduction of new game features was enabled by the supporting SR. Specifically, the possibility of dynamically assigning checkpoints to teams (i.e., locking/unlocking of checkpoint etc.) is an easy task for a centralized management of the event via the SR, while it could be a very hard task to implement via other communication tools such as Whatsapp and Telegram without resorting to both or similar artifacts. On another level, the fact that the web app itself had been developed by our students, was helpful to demonstrate to interns what a computer science student can realize in a Bachelor final project.

3. Checkpoint activities

To give a better idea of the variety of activities that can be included in our format, in this section we briefly discuss the challenges organized by staff members and computer science students in the two reported editions (2019 and 2020 editions during a 1 week internship for high school students at the University of Genova). Specifically, we first summarize the entire set of challenges and then focus on and discuss in greater detail three of them, designed by our students.

The activities offered in the checkpoints give an overview of foundational and applied aspects of computer science. Technology is used as a vehicle to show the importance of algorithms and programming concepts, they are the engine behind most of the technology we used today. Checkpoints combine computational thinking aspects with mathematics, physics, and engineering concepts. At the same time, they show different application domains such as simulation and serious games, automation and IoT, data analysis, smart applications, etc. A summary of the proposed challenges is described in Table 1, while Table 2 describes the topic and knowledge elements of each activity, according to the ACM/IEEE Computing Curricula 2020 CC Task Force (2020), and reports the internship edition in which the activity was proposed.

TABLE 1

Table 1. Checkpoint descriptions.

TABLE 2

Table 2. Knowledge elements from the ACM/IEEE Computing Curricula 2020.

3.1. Checkpoints designed by university students

In this subsection, we provide some additional details on three checkpoints that were specifically designed for their use in the smart rogaining competition by University students as projects of bachelor/master courses. The examples show a concrete example of learning outcomes for computer science university students: designing application that can be tested on the field with large number of users.

The first example is Tangible Coding, a programming activity based on physical shapes used as instructions (Cooper et al., 2003; Horn and Jacob, 2007; Futschek and Moschitz, 2011). On the back-end of the tangible coding activity, there was an interpreter of the objects sequences composed by different artificial intelligence modules (see Figure 5), going from the acquisition of an image depicting the sequence of objects, to the localization and recognition of the shapes on the image, to the generation of a fantasy short story using Natural Language Processing principles. The shape recognition task has been addressed with a Convolutional Neural Network⁴ we designed and trained from scratch, collecting a set of shape images under different environmental conditions (~7,300 images, the 80% of which has been used for training, the remaining samples for validation). We also included a data augmentation procedure to increase the robustness against shape orientations variability. The model has been assessed directly on the field during lab sessions, achieving the 89% of recognition rate. The shape recognition returns an ordered list of labels which is fed to the following text generation module to produce a sentence in a natural language (Italian), following a simple rule based approach. Participants were presented the technical details of the back-end technology, and then they were asked to test the system.

FIGURE 5

Figure 5. The main steps of our software components, on an example (bambina = girl, scimmia = monkey, bambino = boy, bosco = woods, casa = house).

The second example is the Codinji challenge. The webcam blocks available in Scratch 3.0 are used to detect video motion in the current position of a sprite in order to simulate the interaction between the real player and the sprites in the game. This feature can be used to create games involving groups of participants. The Scratch application with the webcam video as a background was projected in a large screen in front of participants filmed by a webcam. Two teams can then play against each other in order to catch different types of sprites floating in the game board.

Finally, in the Alien vs. Terminator challenge, one student (the VR player, see Figure 6B) wears an HMD⁵ for VR and is immersed in a virtual environment, where some sphere-shaped enemies attack him from every direction. The remaining participants are split into two teams: defenders and attackers. Defenders monitored the virtual scene from a top viewpoint, and their mission was to help the person immersed in VR to dodge the enemies. Attackers' objective was to outsmart the defenders' team communication and hit the VR player by creating custom enemies with special features (e.g., different mesh and collider size, velocity, or transparency of the texture), using the Unity editor. Specifically, the attackers' team had to use the Unity editor in play mode (the debug tool built in the editor is shown in Figure 6A) as if they were real developers. We created a reusable asset (prefab) of a standard enemy, which the students could instantiate at run time, and several scripts. By default, the enemy prefab was disabled in the scene. The students had to instantiate an enemy, modify its parameters and behaviors, and finally enable it to finalize the spawning process. The scripts we provided can be use to modify the mesh and collider size, the speed, the spawning position, and the direction of movement. In the beginning, a tutor briefly explains the basics of the Unity editor interface: how to assemble custom enemies using pre-built scripts, tweak their parameters and activate the created game objects (~5 min). After the game starts, the attackers have about 10 min to defeat the VR player. The defenders win if the VR player survives. After one match, the two teams switch roles, and a new VR player is chosen.

FIGURE 6

Figure 6. (A) The Unity interface presented to the attackers' team: top view of the game and interface to create custom enemies. (B) The VR player tries to dodge the enemies created by the group in the foreground (the attackers).

4. Experimental evaluation

In this section, we discuss the evaluation of the proposed format in terms of data collected from the 2019 and 2020 internship editions (Computer Science Bachelor degree of the University of Genoa). After shortly describing the setup (i.e., providing some details about the editions), we discuss the results in terms of teamwork evaluation, rogaine evaluation, effectiveness for recruitment and orientation, and effectiveness for ensuring informed enrollment and contrasting the dropout and reducing the gender gap.

4.1. Experimental setup: Analyzed editions

Interns are high school students enrolled in 12-th grade (majority) and 13-th grade. The period was early February. Table 3 summarizes basic facts about the editions.

TABLE 3

Table 3. Basic facts about the two reported editions.

4.2. Teamwork evaluation

Teams were asked to declare their scores before starting the rogaine. Table 4 reports declared (before game start) and obtained scores for the teams in the two editions. Note that all the teams quite relevantly underestimated their performances. This is quite typical in this kind of activities. Teams were overall quite cautious in predicting their scores. This could be due to the fact that teams are formed by students that did not know each other before starting the activity.

TABLE 4

Table 4. Declared (decl) and obtained (obt) scores for teams in the two editions.

At the end of the activities, a peer evaluation is performed (Dochy et al., 1999; Ohland et al., 2012). Figure 7 summarizes the results of the peer evaluation questionnaire aggregating student responses according to the three relevant dimensions: climate, productivity, and process. The final questionnaire contained eighteen questions, six per dimension. Students are required to observe teams with respect to different behaviors. Possible answers are: behavior observed, opposite behavior observed, no observation for this behavior. Examples of questions are “Encourage, support and help others” (climate), “Quickly acquire information, learn from others” (productivity), “Work out analysis and evaluation on costs/benefits on multiple possibilities” (process). Note that an ideal team should have a perfect balance (33%) of the three dimensions. The fact we got a light unbalance toward climate is coherent both with the age of the participants and with the joyful atmosphere during the activity.

FIGURE 7

Figure 7. Climate, productivity, and process dimensions in teams, according to the peer evaluation questionnaire, in the two editions.

We also analyzed the results of the evaluation of each team made by the mentors during the entire activity. The mean scores turned out be: 4.5 for engagement and 4.7 for the activity completion (in both cases in a 1–5 scale). As a positive outcome we observed a very low variance among different teams and different checkpoints: all computed average values by team and by checkpoints are comprised between 4 and 5. This outcome seems to validate the criteria that we adopted for team formation and for the selection of the checkpoint activities.

4.3. Rogaine evaluation and student appreciation

Figure 8 reports the appreciation for the activity by the interns, obtained from an anonymous post-internship questionnaire. The figure reports the overall evaluation for the rogaining activity, the average evaluation for all the other activities proposed during the 1 week internship, and average, minimum, and maximum evaluation of the individual checkpoints. All evaluations are on a scale 1 (I did not like it at all) to 5 (I liked it a lot). All the internship activities were very well received, but the rogaining was one of the most appreciated ones (in the top-2 for both the editions). All checkpoints received very positive evaluations.

FIGURE 8

Figure 8. Student appreciation for the internship activities and individual checkpoints in the two editions.

We also collected data during the rogaining contest via the SR. More precisely, the sequence of visited checkpoints for each team and the time required to complete each task. These data turned out to be quite useful in order to integrate the parameters selected via the agent-based simulation of the rogaine with a finer tuning of duration and physical location of each checkpoint. We also exploit the large number and high frequency of the requests to the SR as a stress test for the SR itself. Logged data related to use of the SR user interface have been employed in order to test and improve the usability of the SR itself using the crowd-sourced testing methodology presented in Leotta et al. (2019).

4.4. Rogaine effectiveness for recruitment and orientation

The main motivation for organizing such activities is ensuring informed enrollment and contrasting dropout. We thus measure the effectiveness of the proposed events in terms of (i) percentage of the participants that changed their mind in terms of enrolling/not enrolling in the Computer Science BSc after participation to the internship (collected through a pre/post questionnaire) (ii) analysis of the careers of students that participated to the events and then actually enrolled to our degree.

Before presenting the data, we need to point out two limitations. First of all, as already discussed, the rogaine events are part of a longer internship and what we are discussing here is the effectiveness of the entire internship, not of the rogaine alone. Second, in order to attract students from different geographical area, the team participants are from different areas. After analysing our data, we realize that almost only students from close areas actually enrolled to our university. We are able to monitor the careers of local students only, thus the analysis of careers is partial (and on a limited sample).

In terms of effectiveness for orientation and recruitment:

• 39.4% of the participants that then actually enrolled to our degree declared an effect of the internship on the decision of enrolling in the Computer Science BSc (from mildly interested to strongly interested),

• 15.15% of the participants that then actually enrolled to our degree declared a strong effect of the internship on the decision of enrolling in the Computer Science BSc (from not interested to strongly interested).

Information are collected by pre- and post-interviews and we restrict to participants then are currently actually enrolled in our degree. Note that there is also a positive orientation effect of the internship in terms of discouraging enrollment of participants with a misconception of and computer science.

In terms of career analysis, Table 5 reports a comparison between the overall population (enrolled students) and the enrolled interns, i.e., participants to one of the editions of the internship that enrolled to one of the possible cohorts⁶.

TABLE 5

Table 5. Comparison of the number of enrollments, drop out rate, and careers of students and internship participants.

Specifically, for the three observed cohorts the table reports, for the overall population vs enrolled interns subset:

• the number of students that enrolled as freshmen;

• the number of students that are still enrolled;

• the dropout rate;

• (for interns only) the average percentage of acquired ECTS over the total ECTS that can be acquired.

Note that the dropout rate is obviously decreasing for the three observed cohorts, since we are observing just one semester for 2021, while students of cohort 2019 currently are now in the 6th and last semester of the Bachelor. For all the cohorts, the intern dropout rate is below or around one half the general one⁷. The percentage of acquired ECTS is detailed for interns only, and is much higher than the average on the entire enrolled students, which is 27.8%. The lower percentage of acquired ECTS for cohort 2020 compared to 2019 and 2021 can be likely due to the effect of distance learning and COVID-19 pandemic, a similar effect can be observed in the entire student population.

Finally, Table 6 reports a comparison between the overall population and the enrolled interns in term of gender. Specifically, for the three observed cohorts, the table reports the absolute female numbers and female percentages of initial and current enrollments.

TABLE 6

Table 6. Comparison of female students vs. female internship participants.

Overall, the much higher percentages of female students and of acquired ECTS seem to give further strength to orientations initiatives aimed at increasing both engagement and coverage of the different areas of computer science.

5. Related work

This section discusses most relevant related work. It starts discussing general studies on Computer Science orientation, including gender issues. We then focus on gamified and app-supported activities, most notably scavenger hunts, that are close to the format we propose. We then discuss work related to two specific challenges of the proposed activity, namely topic coverage (including elements of novelty of the checkpoint activities designed by our students) and soft-skills.

5.1. Orientation to computer science and gender issues

Recruiting and retaining STEAM majors is an ongoing challenge for colleges and universities (Vegas et al., 2021). The issue is even more relevant for Computer Science and IT higher education, given the shortage of qualified workforce (Cook, 1996; Rosson et al., 2011; see Marzolla and Mirandola, 2019) for a report in the Italian education system).

The issue of how to attract students have been investigated, with specific attention to gender issues (Cheryan et al., 2017; Main and Schimpf, 2017), attributing the reasons of gender unbalance mainly to insufficient early experience, lack of role model, and stereotypes. More in general, gender issues in STEM are attributed to the importance of communal or other-oriented goals for female students. In this respect, our format fights gender stereotypes by putting emphasis on teamwork and communication, and promotes self-efficacy by the solution of simple assignments associated with checkpoints and is well suited to covering diverse topics related to application domains. Offering an internship or participation to a Summer camp earlier in a student's undergraduate career (Aritajati et al., 2015) not only allows participants to gain confidence in their ability to apply their skills to real world problems, but have been demonstrated useful in piquing interest, exposing students to different topics.

The longer the activity, the highest the effectiveness, our challenge is to design an activity that does not require too much time (e.g., Lang et al., 2016 that lasts a whole semester).

A relevant related problem is the high dropout rate in IT higher education studies (Kori et al., 2015, 2018; Giannakos et al., 2017). Among the most frequently proposed solution to mitigate this problem we found tutoring and early assistantship programs.

5.2. Gamification, rogaining, and scavenger hunts

The advantages of gamification in (computer science) education have been thoroughly investigated (Narasareddy Gari et al., 2018; Barianos et al., 2022; Lazarinis et al., 2022). For instance, gamification has been frequently applied in order to improve student retention (Talton et al., 2006). In this context, Augmented/Virtual Reality (AR/VR) and Internet of Things (IoT) technology can substantially improve the student experience especially when the activities are organized in the academic environment (Hutzler et al., 2017; Manzoor et al., 2020). Game-development approaches have also been proposed in introducing novices to coding (Papadakis, 2020), bringing advantages in terms of motivation, fun, commitment, and enthusiasm. Gamification is also being used for orientation. Scavenger hunts (Talton et al., 2006; Hutzler et al., 2017; Manzoor et al., 2020) are very common for university orientation as well as role games (Blecic et al., 2007) especially when combined, e.g., with the virtual reality (Fitz-Walter et al., 2012; Glencross and Elsom, 2019).

A unique feature of our format is the inclusion of computer-related hands-on activities in the rogaine checkpoints.

Rogaining provides a perfect context in which to embed a heterogeneous set of activities covering basic topics of the bachelor degree (i.e., programming, complexity, mathematics, etc.) as well as topics and applications related to advanced courses (i.e., Artificial Intelligence, Computer Security, IoT, etc.). In other words, the rogaine competition replaces more traditional orientation activities with a learning by doing experience in a direct contact with other students (i.e., early assistantships), and instructors. The only other similar format that we found in the literature is the Run and Solve competition focused on mathematics as a part of the Caucasus Mathematical Olympiad (Musatov et al., 2019).

5.3. Topic coverage

The checkpoints included in our rogaining events range from coding and computational thinking to artificial intelligence and virtual reality. For what concerns recruitment activities based on computational thinking and coding, in the literature we can find several important initiatives based on visual languages and human interaction tools (Resnick et al., 2009; Partovi and Sahami, 2013; Partovi, 2014; Sentance et al., 2017). Since our activities assume no prior programming background, some checkpoints propose coding activities relying on visual languages, one of the most difficult points is the selection of an adequate set of building blocks that each player can use in order to specify interactions between sprites in the virtual world or between the human player and the virtual sprites. Scratch (Resnick et al., 2009; Maloney et al., 2010) and Pocketcode (Müller et al., 2019) are perfect tools for stimulating creativity while learning coding by examples and experiments. Indeed, Scratch was created to foster computational creativity. Scratch 3.0 has introduced the motion capture blocks that revealed to be a perfect means for collaborative activities for group of students and that we used in the Codinji checkpoint to create competitions between teams. Concerning the language adopted in the Tangible Coding checkpoint, we took inspiration by approaches on teaching principles of coding to very young children (Cooper et al., 2003; Horn and Jacob, 2007; Futschek and Moschitz, 2011). In the Virtual Reality checkpoint, the activity has been designed in a simplified virtual world (a single room) in order to avoid problems such as simulation sickness (LaViola, 2000; Zhang et al., 2017).

Although each checkpoint activity has innovative aspects with respect to coding laboratories that we are aware of, we remark that the novelty of our proposal is the format based on rogaining, a perfect means to integrate very different computer science areas and applications that students will encounter in their future career.

5.4. Soft skills

The importance of teamwork for computer science orientation had been recognized very soon (Cook, 1996), and, in recent years activities based on the development of teamwork and social interaction skills are gaining more and more attention (Lingard and Barkataki, 2011). The proposed format, differently from scavenger hunts (Talton et al., 2006), requires some forms of planning and goal prioritization, since teams are forced to select a strict subset of checkpoints to visit during the competition, and of dynamic re-planning (reaction to unexpected events) in case the planned checkpoint activity is occupied by another team. Thus, our format allows participants to train planning, goal prioritization, time management, and negotiation skills. The use of gamification for soft skills such as planning and goals prioritization has been recently proposed for to-do list management in Consul et al. (2022).

6. Conclusions and future directions

In this paper, we presented an innovative teamwork format for half day activities offered during internships for high school students at the University of Genoa. The format has been applied in many different occasions and with different configurations, starting from 2019. The format has been designed considering important challenges in computer science orientation and education such as coverage of the wide range of topics of our discipline (using the different activities proposed in the checkpoints), the need of acquiring both technical and soft skill (e.g., strategy and teamwork during the rogaine), and the need of reducing gender gap and improving inclusion (i.e., combining soft skills and activities related to several different application domains). The small number of members of each teams allowed us to optimize the use of specific hardware resources such as AR/VR visors, tablets, and devices. Some of our students were involved in the design of the software and hardware used in the activities. This provides an additional clear example of technical skills that could be acquired by studying computer science. Our students were also involved in the design of checkpoints, achieving a further meta-goal besides the orientation one: showing students how to convey computer science concepts and methods in a limited amount of time and in an engaging way. In principle our format (and related software tools) can be applied to other disciplines by changing the contents (or the goals) of the checkpoints.

The activity turned out to be appreciated by the participants, and, together with the internship program it is part of, to be effective in promoting informed enrollment to our degree, as shown by the analysis reported in Section 4.

After the COVID-19 pandemic, the challenge we are currently facing is the preparation of activities in blended learning with both students in presence and in remote. We have recently organized a first experiment in this direction dividing 120 high school interns in two equally distributed groups. Both groups attended the same presentations and the same laboratories proposed in the two versions. We have collected data for both groups and we plan to carry out a detailed evaluation of the experience in order to compare the performance of the different activities using the same model adopted for the rogaining events.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation. Additional data are available on Dropbox at the URL: https://bit.ly/smartrogaining.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin. Written informed consent was obtained from the individual(s) and minor(s') next of kin for the publication of any potentially identifiable images or data included in this article.

Author contributions

GD, GG, LG, and VM are the main designers of the rogaine format. VM worked on the format workflow specification. FV supported the team formation process. AF is the main author of the SR and related back-end software. MC, GD, GG, FO, and NN have designed the challenges used in the 2019 and 2020 rogaine editions. All the authors contributed to writing and editing. All authors contributed to the article and approved the submitted version.

Funding

GD, GG, and VM were supported by the Boosting Computational Thinking with Pervasive and Collaborative Technologies 2017–2019 Grant funded by the University of Genoa.

Acknowledgments

The authors would like to thank all students who helped the authors in conducting the different editions of the team-based activity.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^Code.org. Hour of Code. Available online at: https://code.org.

2. ^MUR and CINI. Programma il Futuro. Available online at: https://programmailfuturo.it/.

3. ^https://dl.acm.org/ccs

4. ^https://keras.io/

5. ^HTC Vive Pro headset.

6. ^Since our internship involved students in 12th and 13th grade, 2020 cohort collected students from both the editions, while 2019 cohort includes only students from the 2019 edition (13th grade), and 2021 cohort only students from the 2020 edition (12th grade).

7. ^The early dropout rate (dropout at the end of the first year) is around 23% on average for our Bachelor.

References

Aritajati, C., Rosson, M. B., Pena, J., Cinque, D., and Segura, A. (2015). “A socio-cognitive analysis of summer camp outcomes and experiences,” in Proceedings of the 46th ACM Technical Symposium on Computer Science Education, SIGCSE 2015 (Kansas City, MO), 581–586. doi: 10.1145/2676723.2677292

CrossRef Full Text | Google Scholar

Bales, R. F. (1950). Interaction Process Analysis; A Method for the Study of Small Groups. Boston, MA: Addison-Wesley.

Google Scholar

Ballestin, G., Bassano, C., Solari, F., and Chessa, M. (2020). “A virtual reality game design for collaborative team-building: a proof of concept,” in Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020, eds T. Kuflik, I. Torre, R. Burke, and C. Gena (Genoa: New York, NY), 159–162. doi: 10.1145/3386392.3399328

CrossRef Full Text | Google Scholar

Barianos, A. K., Papadakis, A., and Vidakis, N. (2022). Content manager for serious games: theoretical framework and digital platform. Adv. Mobile Learn. Educ. Res. 2, 251–262. doi: 10.25082/AMLER.2022.01.009

CrossRef Full Text | Google Scholar

Blecic, I., Cecchini, A., and Colorni, V. A. (2007). “University explorer: a web-based role-play game for university orientation,” in Learning With Games International Workshop (Sophia Antipolis, France), 337–342.

Google Scholar

Caspersen, M. E., Gal-Ezer, J., McGettrick, A. D., and Nardelli, E. (2018). Informatics for all the strategy. ACM Europe & Informatics Europe. New York, NY: Association for Computing Machinery. doi: 10.1145/3185594. Available online at: https://www.acm.org/binaries/content/assets/public-policy/acm-europe-ie-i4all-strategy-2018.pdf

CrossRef Full Text | Google Scholar

Caspersen, M. E., Gal-Ezer, J., McGettrick, A. D., and Nardelli, E. (2019). Informatics as a fundamental discipline for the 21st century. Commun. ACM 62:58. doi: 10.1145/3310330

PubMed Abstract | CrossRef Full Text | Google Scholar

Catrobat Pocketcode (2021). Catrobat Pocketcode. Available online at: https://share.catrob.at/app/.

CC Task Force (2020). ACM/IEEE Computing Curricula. Available online at: https://www.acm.org/binaries/content/assets/education/curricula-recommendations/cc2020.pdf. doi: 10.1145/3467967

CrossRef Full Text | Google Scholar

Cheryan, S., Ziegler, S. A., Montoya, A. K., and Jiang, L. (2017). Why are some stem fields more gender balanced than others? Psychol. Bull. 143:1. doi: 10.1037/bul0000052

PubMed Abstract | CrossRef Full Text | Google Scholar

Consul, S., Stojcheski, J., Felso, V., Lieder, F., He, R., Mehta, A., et al. (2022). “Leveraging AI for effective to-do list gamification,” in Proceedings of the 5th Int'l workshop “GAM-R ? Gamification Reloaded” in conjunct with the ACM MuC Conference 2022 (Darmstadt).

Google Scholar

Cook, C. R. (1996). A computer science freshman orientation course. SIGSCE Bull. 28, 49–61.

Google Scholar

Cooper, S., Dann, W., and Pausch, R. (2003). “Teaching objects-first in introductory computer science,” in Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (Reno, NV), 191–195. doi: 10.1145/611892.611966

CrossRef Full Text | Google Scholar

Delzanno, G., Guerrini, G., Mascardi, V., Gelati, L., Petito, V., Vitali, F., et al. (2020a). “Slow rogaining: an innovative teamwork model for computer science education,” in Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020, eds T. Kuflik, I. Torre, R. Burke, and C. Gena (Genoa; New York, NY: ACM), 119–126. doi: 10.1145/3386392.3399310

CrossRef Full Text | Google Scholar

Delzanno, G., Guerrini, G., Perotto, S., Traverso, F., Mammoliti, S., and Staglianó, S. (2020b). “Codinji: welcome to the coding jungle!,” in Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020, eds T. Kuflik, I. Torre, R. Burke, and C. Gena (Genoa; New York, NY: ACM), 151–154. doi: 10.1145/3386392.3399326

CrossRef Full Text | Google Scholar

Dochy, F., Segers, M., and Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: a review. Stud. High. Educ. 24, 331–350. doi: 10.1080/03075079912331379935

CrossRef Full Text | Google Scholar

Fitz-Walter, Z., Tjondronegoro, D., and Wyeth, P. (2012). “A gamified mobile application for engaging new students at university orientation,” in Proceedings of the 24th Australian Computer-Human Interaction Conference, OzCHI '12 (Melbourne, Australia), 138–141. doi: 10.1145/2414536.2414560

CrossRef Full Text | Google Scholar

Futschek, G., and Moschitz, J. (2011). “Learning algorithmic thinking with tangible objects eases transition to computer programming,” in Contributing to 21st Century Education, Proceedings of the 5th International Conference on Informatics in Schools: Situation, Evolution and Perspectives, ISSEP 2011, Lecture Notes in Computer Science 7013 (Bratislava, Slovakia: Springer), 155–164. doi: 10.1007/978-3-642-24722-4_14

CrossRef Full Text | Google Scholar

Gander, W., Petit, A., Berry, G., Demo, B., Vahrenhold, J., McGettrick, A., et al. (2013). Informatics Education: Europe Cannot Afford to Miss the Boat, Report of the Joint Informatics Europe and ACM Europe Working Group on Informatics Education. Available online at: https://www.informatics-europe.org/images/documents/informatics-education-europe-report.pdf

Google Scholar

Giannakos, M. N., Pappas, I. O., Jaccheri, L., and Sampson, D. G. (2017). Understanding student retention in computer science education: the role of environment, gains, barriers and usefulness. Educ. Inform. Technol. 22, 2365–2382. doi: 10.1007/s10639-016-9538-1

CrossRef Full Text | Google Scholar

Glencross, S., and Elsom, S. (2019). Using an alternate reality game to facilitate student engagement during orientation. Stud. Success 10, 13–23. doi: 10.5204/ssj.v10i2.1298

CrossRef Full Text | Google Scholar

Happe, L., Buhnova, B., Koziolek, A., and Wagner, I. (2021). Effective measures to foster girls' interest in secondary computer science education. Educ. Inf. Technol. 26, 2811–2829. doi: 10.1007/s10639-020-10379-x

CrossRef Full Text | Google Scholar

Horn, M., and Jacob, R. (2007). “Designing tangible programming languages for classroom use,” in Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI 2007 (Baton Rouge, LA), 159–162. doi: 10.1145/1226969.1227003

CrossRef Full Text | Google Scholar

Hutzler, A., Wagner, R., Pirker, J., and Gütl, C. (2017). “Mythhunter: gamification in an educational location-based scavenger hunt,” in Immersive Learning Research Network - Third International Conference, iLRN 2017, Proceedings, Vol. 725 of Communications in Computer and Information Science, eds D. Beck, C. Allison, L. Morgado, J. Pirker, F. Khosmood, J. Richter, and C. Gütl (Coimbra: Heidelberg; Springer), 155–169. doi: 10.1007/978-3-319-60633-0_13

CrossRef Full Text | Google Scholar

Kori, K., Pedaste, M., and Must, O. (2018). The academic, social, and professional integration profiles of information technology students. ACM Trans. Comput. Educ. 18, 20:1–20:19. doi: 10.1145/3183343

CrossRef Full Text | Google Scholar

Kori, K., Pedaste, M., Tonisson, E., Palts, T., Altin, H., Rantsus, R., et al. (2015). “First-year dropout in ICT studies,” in IEEE Global Engineering Education Conference, EDUCON 2015 (Tallinn), 437–445. doi: 10.1109/EDUCON.2015.7096008

CrossRef Full Text | Google Scholar

Lang, C., Craig, A., and Egan, M. (2016). The importance of outreach programs to unblock the pipeline and broaden diversity in ICT education. Int. J. Inform. Commun. Technol. Educ. 12, 38–49. doi: 10.4018/IJICTE.2016010104

CrossRef Full Text | Google Scholar

LaViola, J. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bull. 32, 47–56. doi: 10.1145/333329.333344

CrossRef Full Text | Google Scholar

Lazarinis, F., Boididis, I., Kozanidis, L., and Kanellopoulos, D. (2022). An adaptable multi-learner serious game for learning cultural heritage. Adv. Mobile Learn. Educ. Res. 2, 201–215. doi: 10.25082/AMLER.2022.01.004

CrossRef Full Text | Google Scholar

Leotta, M., Petito, V., Gelati, L., Delzanno, G., Guerrini, G., and Mascardi, V. (2019). “Orchestrated crowdsourced testing of a mobile web application: a case study,” in Conference Companion of the 3rd International Conference on Art, Science, and Engineering of Programming (Genova; New York, NY: ACM) 17:1–17:6. doi: 10.1145/3328433.3328451

CrossRef Full Text | Google Scholar

Lingard, R., and Barkataki, S. (2011). “Teaching teamwork in engineering and computer science,” in 2011 Frontiers in Education Conference (FIE) (Rapid City, SD). doi: 10.1109/FIE.2011.6143000

CrossRef Full Text | Google Scholar

Main, J. B., and Schimpf, C. (2017). The underrepresentation of women in computing fields: a synthesis of literature using a life course perspective. IEEE Trans. Educ. 60, 296–304. doi: 10.1109/TE.2017.2704060

CrossRef Full Text | Google Scholar

Maloney, J., Resnick, M., Rusk, N., Silverman, B., and Eastmond, E. (2010). The scratch programming language and environment. ACM Trans. Comput. Educ. 10, 16:1–16:15. doi: 10.1145/1868358.1868363

CrossRef Full Text | Google Scholar

Manzoor, A., Samarin, M., Mason, D., and Ylianttila, M. (2020). Scavenger hunt: utilization of blockchain and iot for a location-based game. IEEE Access 8, 204863–204879. doi: 10.1109/ACCESS.2020.3037182

CrossRef Full Text | Google Scholar

Marzolla, M., and Mirandola, R. (2019). “Gender balance in computer science and engineering in Italian universities,” in Proceedings of the 13th European Conference on Software Architecture - Vol. 2 (Paris, France), 82–87. doi: 10.1145/3344948.3344966

CrossRef Full Text | Google Scholar

Müller, M., Schindler, C., and Slany, W. (2019). “Pocket code: a mobile visual programming framework for app development,” in Proceedings of the 6th International Conference on Mobile Software Engineering and Systems, MOBILESoft@ICSE 2019, ed E. Tilevich (Montreal, QC), 140–143. doi: 10.1109/MOBILESoft.2019.00027

CrossRef Full Text | Google Scholar

Musatov, D. V., Kalina, M. I., Malkhozheva, O. N., Yurov, A. V., and Mamiy, D. K. (2019). Run and solve: Competitions in mathematical rogaining. Russ. Digit. Libr. J. 22, 672–685. doi: 10.26907/1562-5419-2019-22-6-672-685

CrossRef Full Text

Narasareddy Gari, M. R., Walia, G. S., and Radermacher, A. D. (2018). “Gamification in computer science education: a systematic literature review,” in Proceedings of the 2018 ASEE Annual Conference & Exposition (Salt Lake City, UT). doi: 10.18260/1-2-30554

CrossRef Full Text | Google Scholar

Noceti, N., Odone, F., Marsella, A., Moro, M., and Nicora, E. (2020). “Tangible coding for kids with AI inside,” in Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020, eds T. Kuflik, I. Torre, R. Burke, and C. Gena (Genoa; New York, NY: ACM), 163–166. doi: 10.1145/3386392.3399329

CrossRef Full Text | Google Scholar

Ohland, M., Loughry, M., Woehr, D. J., Bullard, L. G., Felder, R. M., Finelli, C. J., et al. (2012). The comprehensive assessment of team member effectiveness: development of a behaviorally anchored rating scale for self-and peer evaluation. Acad. Manage. Learn. Educ. 11, 609–630. doi: 10.5465/amle.2010.0177

CrossRef Full Text | Google Scholar

Papadakis, S. (2020). Evaluating a game-development approach to teach introductory programming concepts in secondary education. Int. J. Technol. Enhanced Learn. 12, 127–145. doi: 10.1504/IJTEL.2020.106282

CrossRef Full Text | Google Scholar

Partovi, H. (2014). “Transforming US education with computer science,” in The 45th ACM Technical Symposium on Computer Science Education, SIGCSE '14, eds J. D. Dougherty, K. Nagel, A. Decker, and K. Eiselt (Atlanta, GA; New York, NY: ACM). doi: 10.1145/2538862.2554793

CrossRef Full Text | Google Scholar

Partovi, H., and Sahami, M. (2013). The hour of code is coming! SIGCSE Bull. 45:5. doi: 10.1145/2553042.2553045

CrossRef Full Text | Google Scholar

Phillips, G. N., and Phillips, R. (2000). Rogaining: Cross-Country Navigation. Outdoor Recreation in Australia, Melbourne, QLD.

Resnick, M., Malone, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., et al. (2009). Scratch: programming for all. Commun. ACM 52, 60–67. doi: 10.1145/1592761.1592779

CrossRef Full Text | Google Scholar

Rosson, M. B., Carroll, J. M., and Sinha, H. (2011). Orientation of undergraduates toward careers in the computer and information sciences: gender, self-efficacy and social support. ACM Trans. Comput. Educ. 11, 1–23. doi: 10.1145/2037276.2037278

CrossRef Full Text | Google Scholar

Scratch 3.0 (2021). Scratch 3.0. Available online at: http://scratch.mit.edu/ (accessed 2021).

Sentance, S., Waite, J., Hodges, S., MacLeod, E., and Yeomans, L. (2017). “Creating cool stuff: pupils' experience of the BBC micro: bit,” in Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, eds M. E. Caspersen, S. H. Edwards, T. Barnes, and D. D. Garcia (Seattle, WA; New York, NY: ACM), 531–536. doi: 10.1145/3017680.3017749

CrossRef Full Text | Google Scholar

Talton, J. O., Peterson, D. L., Kamin, S., Israel, D., and Al-Muhtadi, J. (2006). “Scavenger hunt: computer science retention through orientation,” in Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education, SIGCSE 2006, eds D. Baldwin, P. T. Tymann, S. M. Haller, and I. Russell (Houston, TX; New York, NY: ACM), 443–447. doi: 10.1145/1121341.1121478

CrossRef Full Text | Google Scholar

Vegas, E., Hansen, M., and Fowler, B. (2021). Building skills for life: How to expand and improve computer science education around the world, centre FDOR universal education at brookings. District of Columbia: Brookings Institution. Available online at: https://www.brookings.edu/essay/building-skills-for-life-how-to-expand-and-improve-computer-science-education-around-the-world/Washington.

Google Scholar

Wing, J. M. (2006). Computational thinking. Commun. ACM 49, 33–35. doi: 10.1145/1118178.1118215

CrossRef Full Text | Google Scholar

Zhang, F., Chu, S., Pan, R., Ji, N., and Xi, L. (2017). “Double hand-gesture interaction for walk-through in VR environment,” in Proceedings of the 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS) (Wuhan, China), 539–544. doi: 10.1109/ICIS.2017.7960051

CrossRef Full Text | Google Scholar