Addressing Study Skills, Learning Theory and Critical Thinking Skills in Principles of Economics Courses

The Wall Street Journal recently noted that United States “employers say too many schools aren’t teaching students the skills they need—or even basic critical thinking”. This paper 1) motives the need for economics and other college educators to i) address study habits in the classroom, (ii) explain the importance of critical thinking skills, and (iii) define what critical thinking is to introductory students; and 2) provides an overview of how the authors accomplish this in their classrooms. The authors encourage readers to borrow this information to create a study skills and critical thinking module for their own students when teaching principles courses.


INTRODUCTION
The Wall Street Journal recently noted that United States "employers say too many schools aren't teaching students the skills they need-or even basic critical thinking" (Belkin, 2018). While a recent meta-analysis found "that students make substantial gains in critical thinking during college," the study's authors went on to note that "observed gains in critical thinking appear to have deteriorated over time. . ." (Huber and Kuncel, 2016, p. 456). A survey of 202 departments of economics ranked "the ability to use critical thinking skills" first in a list of the ten most important student learning outcomes (Allgood et al., 2015, p. 290). In a recent survey of the literature on 21st century skill frameworks, sixteen articles were identified with "problem solving" or "critical thinking" skills being prioritized ten or nine times respectively out of the 16 frameworks reviewed (Rios et al., 2020). The study's authors went on to analyze 142,000 job advertisements and found "problem solving" to be the fourth most requested skill, behind oral communication, written communication, and collaboration (Rios et al., 2020). If "critical economic thinking" is defined as "effective, high-quality problem solving" (Leyden, 2012, p. 2), how can economics educators promote this skill development?
The goal of this paper is twofold: First, to motive the need for a study habits and critical thinking skills intervention by economics educators; second, to provide set of resources and research to present to students the first week of class to aid in the development of study habit and critical thinking skills.
The paper is divided into four additional sections. Section 2 provides broader motivation by reviewing the larger literature on the need, in fact, the demand among employers for graduates better equipped in critical thinking and problem-solving skills. Section 3 discuss the results of a survey on study habits conducted by the authors. Section 4 outlines the materials presented to students to promoted better studying and critical thinking skills. Finally, Section 5 provides concluding remarks.

STUDY SKILLS, LEARNING, AND THE DEMAND FOR CRITICAL THINKING
A meta-analysis of studies (N = 72,431, k = 344) examining the relationship between study habits, skills, and attitude inventories and constructs and college outcomes, found that "study habits and attitude inventories are the most predictive of performance," even when compared to a student's high school grades or standardized test scores (Credé and Kuncel, 2008). A more recent comprehensive review of the literature on study skills and their efficacy revealed that distributed practice, selfexplanation, and interleaved practice, techniques often used by instructors when developing critical thinking skills, demonstrated either "high" or "moderate utility" in improving student learning across a wide range of class and student factors (Dunlosky et al., 2013). On the other hand, "summarization, highlighting, the keyword mnemonic, imagery use for text learning, and rereading" exhibited the lowest utility and were found to not consistently promote student learning (Dunlosky et al., 2013, p. 5). In other words, when study habits move away from memorization toward active problem solving, content connections, and application, learning efficacy and the development of critical thinking skills improve.
What types of study skills do students actually employ while in college? Course Hero, an online learning platform, has recently provided some informal data on their users' study habits. More than half of Course Hero users do most of their studying Monday through Wednesday and approximately 25-28 percent of their users studying takes place between 9 PM and 5 AM (Course Hero, 2019). Research has consistently shown that sleep deprivation leads to poorer learning outcomes (Doyle and Zakrajsek, 2018;Ro, 2018). Several recent studies have found that students are likely to use ineffective study habits (Nonis and Hudson, 2010;Hora and Oleson, 2017;Miyatsu et al., 2017). The National Survey of Student Engagement (NSSE), based at the Indiana University Bloomington School of Education, provides one of the most comprehensive surveys, including 531 colleges and university in the United States The NSSE survey finds that while students in 2019 are spending more time in academic preparation compared to students in 2004, there has been a decline in this preparation the last few years.
Elaborating on Huber and Kuncel's previously mentioned work, which found that students make substantial gains in critical thinking during college, but that "the observed gains in critical thinking appear to have deteriorated over time despite increased interest in fostering critical thinking skills" (2016, p. 456), Huber and Kuncel offer three potential explanations for the deterioration in the observed gains among students: 1), if "students have increasingly learned more critical thinking skills before entering college. . .(then). . .overall gain scores should be reduced"; 2), with college enrollment increasing over time "many new students (who were previously not in the college cohort) may not be sufficiently prepared to learn more complex reasoning skills (pulling gain scores down)"; and 3), "students have become less willing or able to learn critical thinking skills over time" (2016, p. 457). Even if 1) or 2), the most encouraging potential explanations, is the reason for the college gains decline, neither resolves the fact the employers consistently report a lack of critical thinking skills in college graduates.
A recent study by the Educational Testing Service (ETS) identified what they have described as a "paradox" among Millennials, those born between 1981-1996 (Dimock, 2019), regarding their educational attainment and their corresponding achievement on several international assessments. While the Millennials are "on track to be our most educated generation ever. . . they consistently score below many of their international peers in literacy, numeracy, and problem solving in technology-rich environments" (Goodman et al., 2015, p. 4). Using data from the Organization for Economic Co-operation and Development (OECD) and their Programme for the International Assessment of Adult Competencies (PIAAC) survey "of nationally representative samples of adults age 16 to 65", the ETS study reported "[i]n literacy, United States millennials scored lower than 15 of the 22 participating countries" (Goodman et al., 2015, p. 4, 5, Figure 1). The ETS report closes their executive summary by noting "despite investments and reforms in K-12 education over the past decades, America continues to lose ground in terms of the developed skills of its adult population and workforce" (Goodman et al., 2015, p. 5).
Evidence of the issues highlighted in the ETS report are already making themselves felt among some sectors of the American labor market. A recent Wall Street Journal article found that while ever increasing numbers of students are entering and completing four-year degrees, incentivized by federal education policy and assisted with student loan subsidization, however, "technology is changing faster than colleges can keep up and employers say too many schools aren't teaching students the skills they need-or even basic critical thinking" (Belkin, 2018). This has led to an increasing demand among United States employers for specific skills over four-year degrees and to heavier investments by employers, such as AT&T which spent over $1 billion dollars to retrain approximately 100,000 employees, to meet the increasing demands of the labor market, especially in the manufacturing, technology and health-care industries (Belkin, 2018). And even more recently Inside Higher Ed reported that "75 percent of employers claim the students they hire after 12, 16 or more years of formal education lack the ability to think critically and solve problems" (Haber, 2020).
Acknowledging the importance of critical thinking skills to the labor market and the larger economy is one thing but defining exactly what is meant by "critical thinking" is quite another. As one group of economists have noted, critical thinking is often "not well defined" and in economics there has been "relatively little" published on this topic, despite the fact that we often strive to emphasize "the economic way of thinking" in many of our principles courses (Greenlaw and DeLoach, 2003, p. 36). Leyden (2012), in his book dedicated to developing critical thinking in economics, describes it as "all about effective, high-quality problem solving" (Leyden, 2012, p. 2). Mulnix (2012) provides a review of several definitions of critical thinking and then arrives at the following for what she feels is the preferred description of the "critical thinker": This suggests that the fundamental skill to be acquired by a critical thinker is the ability to recognize inferential connections holding between statements, where this would include the ability to understand the possibility that what we believe might be false and the ability to identify the sorts of evidence that would undermine our beliefs (Mulnix, 2012, p. 474).
Students that have an adequate understanding of critical thinking and can successfully employ those skills can improve their learning outcomes (Abrami, et al., 2015;Huber and Kuncel, 2016). However, as just reviewed, the larger literature indicates that students are not always ready and/or prepared to develop those skills at the post-secondary level (Pithers and Soden, 2000;Huber and Kuncel, 2016). Further, while economists recognize the importance of critical thinking skills for student learning (Allgood et al., 2015, p. 290), there is relatively little literature on the topic among economists. A recent review of the literature only identified 13 citations (see Allgood et al., 2015) dealing with "active learning" in economics (Boyle and Goffe, 2018).
While many students believe to overcome poor performance they just need to study more, research suggests that study time alone will not necessarily lead to improved learning (Nonis and Hudson, 2010;Chew, 2011;Doyle and Zakrajsek, 2018). Furthermore, many students leave exams believing they did well when this is not the case. Ambrose et al. (2010, pp. 4-6) identifies seven research-based principles to improve learning in their book How Learning Works: 1. Prior knowledge 2. How students organize knowledge 3. Motivation 4. Skill acquisition, interpretive practice, and appropriate application 5. Goal-directed practice 6. Level of student development and climate of the course 7. Self-directed learning, including an emphasis on metacognition Without students, or educators, understanding how learning works, what critical thinking is, and what study skills are most effective, it is rather fanciful to believe students will successfully develop the critical thinking skills economics educators rank as the most important learning outcome, and that employers demand. As some recent research has demonstrated, "elements of critical thinking need to be taught explicitly, rather than assumed to come along for the ride when thoughtful teachers run through complex material with students" (emphasis original; Haber, 2020). The next sections of this paper provide results from study habits questions posed to students in the authors classes; followed by an example of a set of resources the authors go over in class and assign to students to help prepare them to be able to learn more effectively and think more critically.

STUDENT STUDY HABITS QUESTIONNAIRE
At the beginning of the Spring 2020 semester, the authors administered a survey to students (n~285) enrolled in their principles of micro and macroeconomics courses asking about their study habits. Some good news bad news patterns emerged. While 62% of the students reported taking notes when reading the textbook half of the time or more, only 8% reported that they always read the textbook. While 95% of the students reported taking notes in class half of the time or more, only 5% reported that they always rewrite their notes at some point after class. While 76% of the students reported studying "really hard" the night before an exam always or most of the time, only 19% reported always studying at least three days prior to an exam.
Lastly, when asked what the most important factor in successful learning is, 87% of the students reported "learning in a way that matches your learning style" or "the intention and desire to learn", while only 13% reported "Paying close attention to the material as you study" or "the time you spend studying", and less than 0.5% reported "What you think about while you study". The response to this question is of particular importance because a wide body of literature has shown that matching learning style does not correlate with gains in learning (Pashler et al., 2008;Willingham et al., 2015). And while having an intention and desire to learn is a commendable goal, it is of limited value unless time is spent studying and paying attention while doing so. In fact, of the answer choices, the single most important determinant of learning is "what you think about while studying" (Hyde and Jenkins, 1969; see also ;Chew, 2011). Table 1 provides further details of the survey results.
The conclusion from the survey is that a majority of students in the authors classes may be unaware of, or discount, behaviors that translate into successful learning and the development of critical thinking skills.

A CLASSROOM STUDY SKILLS AND LEARNING MODULE
The authors propose educators of introductory economics classes address the topic of study skills, learning theory, and critical thinking head on inside their own classrooms. What follow is a description of the resources, content, and approach the authors take.
The first week of classes, a class meeting is devoted to the topics of studying, learning, and the value of critical thinking. This not only brings awareness of the issue to students, but helps the instructor create a framework for understanding and allows students to organize the information into new learning habits to draw from (Ambrose et al., 2010). During the class a presentation of the following materials is given: • What does it mean to study and how should you study? A casual conversation with the class to see what they think and to give students an opportunity to hear from and discuss with their peers. • Some facts about studying and learning o If you want deeper comprehension of materials print might be better than digital (Alexander and Singer, 2017). o If you take longhand notes, review them, and focus on core concepts, you learn more and perform better on exams (Carter et al., 2017;Morehead et al., 2019).
o If you cram and don't get an adequate amount of sleep you impede your ability learn, manipulate, and even recall information (Doyle and Zakrajsek, 2018;Ro, 2018). o If you incorporate effect study skills you can spend less time studying and get better results, while matching your learning style doesn't help but what you think about while studying certainly does (Chew, 2011;Willingham et al., 2015). o If you use critical thinking skills in college you will learn more and have better grades (Huber and Kuncel, 2016). • How learning works -A quick walk through and explanation of Ambrose's seven points on how learning works is provided, with a particular focus on how the elements of the course relate to these principles of learning (Ambrose et al., 2010;Chew, 2011). • What is critical thinking?-The meaning of the definitions from Leyden and Mulnix are explored, followed by a conversation that asks students why they think critical thinking is important (Leyden, 2012;Mulnix, 2012). • What are cognitive and non-cognitive skills? The authors explain to students that "30-40 percent of the explained variance in achievement test scores across students is due to personality traits and not IQ" (Heckman et al., 2013(Heckman et al., , p. 2056. Soft skills matter (Jackson, 2014;Jackson, 2018). • Why does all of this matter? The students are engaged in a discussion of The WSJ quote (Belkin, 2018), the ETS study on skill development (Goodman et al., 2015), and that employers want critical thinking and problem-solving skills (Chanler, et al., 2019;Heckman et al., 2019;Johanns, 2019;Rios, et al., 2020).
Prior to the next class meeting, students are directed to watch a series of six videos on "How to Study" (~36 min in total) developed by Samford University cognitive psychologist Stephen Chew (see Chew, 2011)  Chew's videos review much of the material introduced in the classroom discussion and further expand the material by practically demonstrating how students can apply the findings from cognitive science to their class through best practices regarding note take, reading, studying for exams and how to move forward when not performing poorly on tests. Upon completing the video series, students are assigned an unlimited attempts multiple-choice assessment to enable them to correctly uncover the key point(s) of each video. The scores on the assessment were close to 100%, suggesting nearly every student took the attempts necessary to identify the main points. Table 2 lists the questions that were asked. The next class, a brief review of the videos occurs and students are encouraged to integrate what they have learned into their own study plans. While the authors do not formally assess the impact of their module, research has consistently demonstrated that targeted instruction can be effective in changing student study skills as long as the material is in context, within the content domain, and promotes active learning and metacognitive awareness (Hattie et al., 1996;Dunlosky et al., 2013;Miyatsu et al., 2017). Video 1: Beliefs That Make You Fail. . .Or Succeed (6:53) 1. All students base their study behavior on _____.
a. the best available research b. their beliefs about how they learn best * c. their peers' study habits d. All the above are correct 2. To truly comprehend course material, you should _____.
a. Highlight and memorize b. Quickly read and highlight c. Skim and memorize d. Carefully read and review 3. The problem with focusing on isolated facts, such as definitions of key terms, is that: a. You may not cover all of the course material b. You may get the definition incorrect c. You may not be able to apply the concept a * d. You may neglect reading all the course material 4. Academic success is much more a matter of _____.
a. Hard work and time * b. Inborn talent c. Natural gifting d. Identifying your learning style 5. The research evidence is overwhelming that: a. Learning styles improve learning b. Most students can multitask to improve learning if they utilize technology to study c. The amount of time you spend studying depends on the subject d. We are bad at multitasking, especially for tasks requiring concentration and effort * 6. One of the key differences between successful and struggling students is: a. Learning styles, how you receive the content you are trying to learn b. Metacognition, your awareness of your level of understanding of the content * c. Velocity, the speed with which you can process and memorize the content d. None of the answers are correct Video 2, Video 2: What Students Should Understand About How People Learn (7:15) 1. What is the most important factor in successful learning? a. The intention and desire to learn b. Paying close attention to the material as you study c. Learning in a way that matches your own learning style d. The time you spend studying e. What you think about while you study * 2. In the video Professor Stephen Chew discussed the research by psychologists Thomas S. Hyde andJames J. Jenkins (1923-2012) which examined learning through testing subjects' ability to recall a series of 24 words. What was the main finding of the study according to Professor Chew? a. The group looking for the letter "e" had the best word recall rates b. The group evaluating how pleasant a word was to them had the best word recall rates * c. The control group, which tried to learn the words as best they could, had the best word recall rates d. The subjects that were intentional, regardless of their group, had the best word recall rates 3. All of the following do not help learning except: a.  a. is relatively easy for most students and takes little time b. is impossible for most students to achieve c. is a process of ongoing improvement over time * d. is a function of your dominant learning style 6. The advantages of overlearning information include: a. Prevent forgetting and making recall fast and easy * b. Aiding memorization and intentionality c. Prevent studying material in a learning style that is not your dominant one d. Aiding multi-tasking and your ability to skim texts better Video 4: Putting the Principles for Optimizing Learning into Practice (9:16) 1. Good study strategies focus on _______.
a. Comprehension and implications * b. Memorization and intentionality c. Specific information and memorization d. Highlighting and definitions 2. Professor Chew outlined three research-based study strategies that include all of the following except: a. Question generation b. Creating a concept map c. Practice retrieving the information in the way the teacher expects d. Memorization of definitions and key concepts * 3. Taking notes in class _____.
a. provides a key summary of concepts, creates a set of memory cues, and engages you in the class * b. should be done via laptop or tablet device so you can capture all of the information, minimizes distractions, and maximizes student engagement c. only engages the student in shallow processing so students should use laptops and tablet devices d. provides a comprehensive summary of concepts, increases intentionality, and engages you in the class only if the student takes notes electronically 4. Keys to effectively taking notes include all of the following except: a. Get missed information right away b. Consider recording the lecture c. Borrowing notes in place of attending class * d. Actively organize and review notes 5. The key to deep processing while reading a textbook includes _____.
a. Highlighting portions of the text for definitions and specific information b. Highlighting portions of the text for connections, distinctions and applications * c. Highlighting portions of the text as completely as possible so you can review it later d. Highlighting portions of the text using multiple colors to differentiate key concepts making sure you have most of the text covered in at least one color 6. Good highlighting requires _____.
a. rereading and reviewing, which can be slow and effortful * b. extensive highlighting to minimize review time and effort c. selective highlighting of key terms and definitions to reduce effort and time (Continued on following page)

CONCLUSION
There is a growing body of literature in economics where scholars are attempting to incorporate the findings from the cognitive and learning sciences to improve learning outcomes. This paper attempted to identify the issues involved, provide an overview of the broader literature, and describe how the authors have attempted to develop a classroom response. While it recognized that the introduction of a study skill module takes up valuable classroom time, how can a student become a better critical thinker if he or she does not have a basic understanding of effective study skills, how learning works, and what critical thinking is? On these grounds, the introduction of such a module is a most valuable use of classroom time. The authors encourage other educators to develop their own study skills and learning theory module based in part, or in whole, on the information and resources discussed in this paper.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

AUTHOR CONTRIBUTIONS
JS-Survey design, analysis, writing (50%) EH-Literature review, writing (50%) d. including as much text as possible and if done correctly will not require reviewing 7. Rules of effective group study include all of the following except: a. Set a goal and agenda b. Set conditions for participation c. Keep ultimate goal of learning in mind d. Everyone can ask/answer questions e. The group facilitator can express the group's understanding * Video 5: "I Blew the Exam, Now What?" (7:28) 1. According to Professor Chew if students do poorly on an exam, they should first panic and then go into denial.
a. True b. False * 2. Professor Chew identified five things that students should do if they "blew the exam". These include all the following except: a. Reviewing your preparation b. Comparing your errors on the exam with your note taking c. Recopying notes without considering connections * d. Discuss your performance with your professor e. Develop a plan for improving your performance 3. Helpful strategies to raise your grade include all the following except: a. Commit time and effort b. Minimize distractions c. Attend class d. Space out study time, avoid cramming, and maximize review time e. Don't give away points by failing to follow directions or skipping assignments f. Develop a program of memorization of key definitions *