problem solving formalisms for information technology

Search form

Critical thinking and problem solving with technology.

Brief Summary: Critical thinking and problem solving is a crucial skill in a technical world that can immediately be applied to academics and careers. A highly skilled individual in this competency can choose the appropriate tool to accomplish a task, easily switch between tools, has a basic understanding of different file types, and can troubleshoot technology when it’s not working properly. They can also differentiate between true information and falsified information online and has basic proficiency in data gathering, processing and interpretation. 

Learners with proficient skills in critical thinking and problem solving should be able to: 

• Troubleshoot computers and mobile devices when issues arise, like restarting the device and checking if it requires a software or operating system update 
• Move across tools to complete a task (for example, adding PowerPoint slides into a note taking app for annotation) 
• Differentiate between legitimate and falsified information online 
• Understand basic file types and know when to use them (for example, the difference between .doc and .pdf files) 

Market/Employer Trends: Employers indicate value in employee ability to problem solve using technology, particularly related to drawing information from data to identify and solve challenges. Further, knowing how to leverage technology tools to see a problem, break it down into manageable pieces, and work toward solving is of important value. Employers expect new employees to be able to navigate across common toolsets, making decisions to use the right tool for the right task.  

Self-Evaluation: 

Key questions for reflection: 

• How comfortable are you when technology doesn’t work the way you expect?  
• Do you know basic troubleshooting skills to solve tech issues?  
• Do you know the key indicators of whether information you read online is reliable? 

Strong digital skills in this area could appear as: 

• Updating your computer after encountering a problem and resolving the issue 
• Discerning legitimate news sources from illegitimate ones to successfully meet goals 
• Converting a PowerPoint presentation into a PDF for easy access for peers who can’t use PowerPoint 
• Taking notes on a phone and seamlessly completing them on a computer

Ways to Upskill: 

Ready to grow your strength in this competency? Try: 

• Reviewing University Libraries’ resources on research and information literacy  
• Read about troubleshooting in college in the Learner Technology Handbook 
• Registering for ESEPSY 1359: Critical Thinking and Collaboration in Online Learning  

Educator Tips to Support Digital Skills: 

• Create an assignment in Carmen prompting students to find legitimate peer-reviewed research  
• Provide links to information literacy resources on research-related assignments or projects for student review 
• Develop assignments that require using more than one tech tool to accomplish a single task 

What Is Problem Solving? How Software Engineers Approach Complex Challenges

From debugging an existing system to designing an entirely new software application, a day in the life of a software engineer is filled with various challenges and complexities. The one skill that glues these disparate tasks together and makes them manageable? Problem solving . 

Throughout this blog post, we’ll explore why problem-solving skills are so critical for software engineers, delve into the techniques they use to address complex challenges, and discuss how hiring managers can identify these skills during the hiring process. 

What Is Problem Solving?

But what exactly is problem solving in the context of software engineering? How does it work, and why is it so important?

Problem solving, in the simplest terms, is the process of identifying a problem, analyzing it, and finding the most effective solution to overcome it. For software engineers, this process is deeply embedded in their daily workflow. It could be something as simple as figuring out why a piece of code isn’t working as expected, or something as complex as designing the architecture for a new software system. 

In a world where technology is evolving at a blistering pace, the complexity and volume of problems that software engineers face are also growing. As such, the ability to tackle these issues head-on and find innovative solutions is not only a handy skill — it’s a necessity. 

The Importance of Problem-Solving Skills for Software Engineers

Problem-solving isn’t just another ability that software engineers pull out of their toolkits when they encounter a bug or a system failure. It’s a constant, ongoing process that’s intrinsic to every aspect of their work. Let’s break down why this skill is so critical.

Driving Development Forward

Without problem solving, software development would hit a standstill. Every new feature, every optimization, and every bug fix is a problem that needs solving. Whether it’s a performance issue that needs diagnosing or a user interface that needs improving, the capacity to tackle and solve these problems is what keeps the wheels of development turning.

It’s estimated that 60% of software development lifecycle costs are related to maintenance tasks, including debugging and problem solving. This highlights how pivotal this skill is to the everyday functioning and advancement of software systems.

Innovation and Optimization

The importance of problem solving isn’t confined to reactive scenarios; it also plays a major role in proactive, innovative initiatives . Software engineers often need to think outside the box to come up with creative solutions, whether it’s optimizing an algorithm to run faster or designing a new feature to meet customer needs. These are all forms of problem solving.

Consider the development of the modern smartphone. It wasn’t born out of a pre-existing issue but was a solution to a problem people didn’t realize they had — a device that combined communication, entertainment, and productivity into one handheld tool.

Increasing Efficiency and Productivity

Good problem-solving skills can save a lot of time and resources. Effective problem-solvers are adept at dissecting an issue to understand its root cause, thus reducing the time spent on trial and error. This efficiency means projects move faster, releases happen sooner, and businesses stay ahead of their competition.

Improving Software Quality

Problem solving also plays a significant role in enhancing the quality of the end product. By tackling the root causes of bugs and system failures, software engineers can deliver reliable, high-performing software. This is critical because, according to the Consortium for Information and Software Quality, poor quality software in the U.S. in 2022 cost at least $2.41 trillion in operational issues, wasted developer time, and other related problems.

Problem-Solving Techniques in Software Engineering

So how do software engineers go about tackling these complex challenges? Let’s explore some of the key problem-solving techniques, theories, and processes they commonly use.

Decomposition

Breaking down a problem into smaller, manageable parts is one of the first steps in the problem-solving process. It’s like dealing with a complicated puzzle. You don’t try to solve it all at once. Instead, you separate the pieces, group them based on similarities, and then start working on the smaller sets. This method allows software engineers to handle complex issues without being overwhelmed and makes it easier to identify where things might be going wrong.

Abstraction

In the realm of software engineering, abstraction means focusing on the necessary information only and ignoring irrelevant details. It is a way of simplifying complex systems to make them easier to understand and manage. For instance, a software engineer might ignore the details of how a database works to focus on the information it holds and how to retrieve or modify that information.

Algorithmic Thinking

At its core, software engineering is about creating algorithms — step-by-step procedures to solve a problem or accomplish a goal. Algorithmic thinking involves conceiving and expressing these procedures clearly and accurately and viewing every problem through an algorithmic lens. A well-designed algorithm not only solves the problem at hand but also does so efficiently, saving computational resources.

Parallel Thinking

Parallel thinking is a structured process where team members think in the same direction at the same time, allowing for more organized discussion and collaboration. It’s an approach popularized by Edward de Bono with the “ Six Thinking Hats ” technique, where each “hat” represents a different style of thinking.

In the context of software engineering, parallel thinking can be highly effective for problem solving. For instance, when dealing with a complex issue, the team can use the “White Hat” to focus solely on the data and facts about the problem, then the “Black Hat” to consider potential problems with a proposed solution, and so on. This structured approach can lead to more comprehensive analysis and more effective solutions, and it ensures that everyone’s perspectives are considered.

This is the process of identifying and fixing errors in code . Debugging involves carefully reviewing the code, reproducing and analyzing the error, and then making necessary modifications to rectify the problem. It’s a key part of maintaining and improving software quality.

Testing and Validation

Testing is an essential part of problem solving in software engineering. Engineers use a variety of tests to verify that their code works as expected and to uncover any potential issues. These range from unit tests that check individual components of the code to integration tests that ensure the pieces work well together. Validation, on the other hand, ensures that the solution not only works but also fulfills the intended requirements and objectives.

Explore verified tech roles & skills.

The definitive directory of tech roles, backed by machine learning and skills intelligence.

Explore all roles

Evaluating Problem-Solving Skills

We’ve examined the importance of problem-solving in the work of a software engineer and explored various techniques software engineers employ to approach complex challenges. Now, let’s delve into how hiring teams can identify and evaluate problem-solving skills during the hiring process.

Recognizing Problem-Solving Skills in Candidates

How can you tell if a candidate is a good problem solver? Look for these indicators:

• Previous Experience: A history of dealing with complex, challenging projects is often a good sign. Ask the candidate to discuss a difficult problem they faced in a previous role and how they solved it.
• Problem-Solving Questions: During interviews, pose hypothetical scenarios or present real problems your company has faced. Ask candidates to explain how they would tackle these issues. You’re not just looking for a correct solution but the thought process that led them there.
• Technical Tests: Coding challenges and other technical tests can provide insight into a candidate’s problem-solving abilities. Consider leveraging a platform for assessing these skills in a realistic, job-related context.

Assessing Problem-Solving Skills

Once you’ve identified potential problem solvers, here are a few ways you can assess their skills:

• Solution Effectiveness: Did the candidate solve the problem? How efficient and effective is their solution?
• Approach and Process: Go beyond whether or not they solved the problem and examine how they arrived at their solution. Did they break the problem down into manageable parts? Did they consider different perspectives and possibilities?
• Communication: A good problem solver can explain their thought process clearly. Can the candidate effectively communicate how they arrived at their solution and why they chose it?
• Adaptability: Problem-solving often involves a degree of trial and error. How does the candidate handle roadblocks? Do they adapt their approach based on new information or feedback?

Hiring managers play a crucial role in identifying and fostering problem-solving skills within their teams. By focusing on these abilities during the hiring process, companies can build teams that are more capable, innovative, and resilient.

Key Takeaways

As you can see, problem solving plays a pivotal role in software engineering. Far from being an occasional requirement, it is the lifeblood that drives development forward, catalyzes innovation, and delivers of quality software. 

By leveraging problem-solving techniques, software engineers employ a powerful suite of strategies to overcome complex challenges. But mastering these techniques isn’t simple feat. It requires a learning mindset, regular practice, collaboration, reflective thinking, resilience, and a commitment to staying updated with industry trends. 

For hiring managers and team leads, recognizing these skills and fostering a culture that values and nurtures problem solving is key. It’s this emphasis on problem solving that can differentiate an average team from a high-performing one and an ordinary product from an industry-leading one.

At the end of the day, software engineering is fundamentally about solving problems — problems that matter to businesses, to users, and to the wider society. And it’s the proficient problem solvers who stand at the forefront of this dynamic field, turning challenges into opportunities, and ideas into reality.

This article was written with the help of AI. Can you tell which parts?

Get started with HackerRank

Over 2,500 companies and 40% of developers worldwide use HackerRank to hire tech talent and sharpen their skills.

Recommended topics

• Hire Developers
• Problem Solving

Does a College Degree Still Matter for Developers in 2024?

Formalizing Problem Solving in Computational Thinking : an Ontology approach

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Evolution
• Language Reference
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Religion
• Music and Media
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Strategy
• Business Ethics
• Business History
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Systems
• Economic History
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Political Theory
• Politics and Law
• Public Administration
• Public Policy
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

4 “Sticky Information” and the Locus of Problem Solving: Implications for Innovation

• Published: September 1999
• Cite Icon Cite
• Permissions Icon Permissions

The problem of ‘sticky’ information in the context of firms—the fact that information needed for technical problem solving tends to be costly to acquire, transfer, and put to use in a new location—is highlighted. When the requisite sticky information resides at only one location, problem solving tends to take place at that location. When more sites collectively serve as a repository of the demanded sticky information, problem solving is iterated between these sites or the problem will be broken down so as to simulate the first case. The final avenue is to make investments that reduce the stickiness, and thus the costs, of applying such information at other sites. The findings have significant implications for more general issues such as patterns in the diffusion of information, the specialization of firms and the locus of innovation.

Signed in as

Institutional accounts.

• Google Scholar Indexing
• GoogleCrawler [DO NOT DELETE]

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code

Institutional access

• Sign in with a library card Sign in with username/password Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Sign in through your institution

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Sign in with a library card

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms

We introduce a large-scale dataset of math word problems and an interpretable neural math problem solver that learns to map problems to operation programs. Due to annotation challenges, current datasets in this domain have been either relatively small in scale or did not offer precise operational annotations over diverse problem types. We introduce a new representation language to model precise operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models. Using this representation language, our new dataset, MathQA, significantly enhances the AQuA dataset with fully-specified operational programs. We additionally introduce a neural sequence-to-program model enhanced with automatic problem categorization. Our experiments show improvements over competitive baselines in our MathQA as well as the AQuA datasets. The results are still significantly lower than human performance indicating that the dataset poses new challenges for future research. Our dataset is available at: https://math-qa.github.io/math-QA/ .

1 Introduction

Answering math word problems poses unique challenges for logical reasoning over implicit or explicit quantities expressed in text. Math word-problem solving requires extraction of salient information from natural language narratives. Automatic solvers must transform the textual narratives into executable meaning representations, a process that requires both high precision and, in the case of story problems, significant world knowledge.

As shown by the geometry question in Figure 1 , math word problems are generally narratives describing the progress of actions and relations over some entities and quantities. The operation program underlying the problem in Figure 1 highlights the complexity of the problem-solving task. Here, we need the ability to deduce implied constants (pi) and knowledge of domain-specific formulas (area of the square).

In this paper, we introduce a new operation-based representation language for solving math word problems. We use this representation language to construct MathQA 1 1 1 The dataset is available at: https://math-qa.github.io/math-QA/ , a new large-scale, diverse dataset of 37k English multiple-choice math word problems covering multiple math domain categories by modeling operation programs corresponding to word problems in the AQuA dataset Ling et al. ( 2017 ) . We introduce a neural model for mapping problems to operation programs with domain categorization.

Most current datasets in this domain are small in scale  Kushman et al. ( 2014 ) or do not offer precise operational annotations over diverse problem types  Ling et al. ( 2017 ) . This is mainly due to the fact that annotating math word problems precisely across diverse problem categories is challenging even for humans, requiring background math knowledge for annotators. Our representation language facilitates the annotation task for crowd-sourcing and increases the interpretability of the proposed model.

Our sequence-to-program model with categorization trained on our MathQA dataset outperforms previous state-of-the-art on the AQuA test set in spite of the smaller training size. These results indicate the superiority of our representation language and the quality of the formal annotations in our dataset. Our model achieves competitive results on MathQA, but is still lower than human performance indicating that the dataset poses new challenges for future research. Our contributions are as follows:

We introduce a large-scale dataset of math word problems that are densely annotated with operation programs

We introduce a new representation language to model operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models.

We introduce a neural architecture leveraging a sequence-to-program model with automatic problem categorization, achieving competitive results on our dataset as well as the AQuA dataset

2 Background and Related Work

Large-Scale Datasets Several large-scale math word problem datasets have been released in recent years. These include Dolphin18K Huang et al. ( 2016 ) , Math23K Wang et al. ( 2017 ) and AQuA. We choose the 2017 AQUA-RAT dataset to demonstrate use of our representation language on an existing large-scale math word problem solving dataset. The AQuA provides over 100K GRE- and GMAT-level math word problems. The problems are multiple choice and come from a wide range of domains.

The scale and diversity of this dataset makes it particularly suited for use in training deep-learning models to solve word problems. However there is a significant amount of unwanted noise in the dataset, including problems with incorrect solutions, problems that are unsolvable without brute-force enumeration of solutions, and rationales that contain few or none of the steps required to solve the corresponding problem. The motivation for our dataset comes from the fact we want to maintain the challenging nature of the problems included in the AQuA dataset, while removing noise that hinders the ability of neuralized models to learn the types of signal neccessary for problem-solving by logical reasoning.

Additional Datasets Several smaller datasets have been compiled in recent years. Most of these works have focused on algebra word problems, including MaWPS Koncel-Kedziorski et al. ( 2016 ) , Alg514 Kushman et al. ( 2014 ) , and DRAW-1K Upadhyay and Chang ( 2017 ) . Many of these datasets have sought to align underlying equations or systems of equations with word problem text. While recent works like Liang et al. ( 2018 ); Locascio et al. ( 2016 ) have explored representing math word problems with logical formalisms and regular expressions, our work is the first to provide well-defined formalisms for representing intermediate problem-solving steps that are shown to be generalizable beyond algebra problems.

Solving with Handcrafted Features Due to sparsity of suitable data, early work on math word problem solving used pattern-matching to map word problems to mathematical expressions Bobrow ( 1964 ); Charniak ( 1968 , 1969 ) , as well as non-neural statistical modeling and semantic parsing approaches Liguda and Pfeiffer ( 2012 ) .

Some effort has been made on parsing the problems to extract salient entities Hosseini et al. ( 2017 ) . This approach views entities as containers, which can be composed into an equation tree representation. The equation tree representation is changed over time by operations implied by the problem text.

Many early works focused on solving addition and subtraction problems Briars and Larkin ( 1984 ); Dellarosa ( 1986 ); Bakman ( 2007 ) . As word problems become more diverse and complex, we require models capable of solving simultaneous equation systems. This has led to an increasing focus on finding semantic alignment of math word problems and mentions of numbers Roy and Roth ( 2018 ) . The main idea behind those work is to find all possible patterns of equations and rank them based on the problem.

Neural Word Problem Solvers Following the increasing availability of large-scale datasets like AQuA, several recent works have explored deep neural approaches to math word problem solving Wang et al. ( 2017 ) . Our representation language is motivated by exploration of using intermediate formalisms in the training of deep neural problem-solving networks, as is done in the work of Huang et al. ( 2018b ) to solve problems with sequence to sequence models. While this work focused on single-variable arithmetic problems, our work introduces a formal language of operations for covering more complex multivariate problems and systems of equations.

Interpretability of Solvers While the statistical models with handcrafted features introduced by prior work are arguably “interpretable” due to the relative sparsity of features as well as the clear alignments between inputs and outputs, new neuralized approaches present new challenges to model interpretability of math word problem solvers Huang et al. ( 2018a ) . While this area is relatively unexplored, a prior approach to increasing robustness and interpretability of math word problem-solving models looks at using an adversarial dataset to determine if models are learning logical reasoning or exploiting dataset biases through pattern-matching Liang et al. ( 2018 ) .

3 Representing Math Word Problems

A math word problem consists of a narrative that grounds mathematical formalisms in real-world concepts. Solving these problems is a challenge for both humans and automatic methods like neural network-based solvers, since it requires logical reasoning about implied actions and relations between entities. For example, in Figure 2 , operations like addition and division are not explicitly mentioned in the word problem text, but they are implied by the question.

As we examine the context of a math word problem, we have to select arguments for operations based on which values are unimportant for solving the problem and which are salient. In Figure 2 , the numeric value “100” appears in the context but does not appear in the underlying equation.

By selecting implied operations and arguments, we can generate a program of intermediate steps for solving a math word problem. Each step involves a mathematical operation and its related arguments. In Figure 2, there are three addition operations and one division. As illustrated in the figure, operations can be dependant to the previous ones by the values they use as arguments. Every math word problem can be solved by sequentially executing these programs of dependent operations and arguments.

We define formalisms for expressing these sequential operation programs with a domain-aware representation language. An operation program in our representation language is a sequence with n 𝑛 n operations. The general form is shown below. Each operation o i subscript 𝑜 𝑖 o_{i} takes in a list of arguments a of length i 𝑖 i :

Given this general definition, the problem in Figure 2 has the following representation 2 2 2 Here the arguments 174 174 174 , 254 254 254 and 349 349 349 are the outputs of operations 1, 2 and 3 respectively. :

Our representation language consists of 58 operations and is designed considering the following objectives.

Correctness → → \rightarrow Operation programs should result in the correct solution when all operations are executed.

Domain-awareness → → \rightarrow Operation problems should make use of both math knowledge and domain knowledge associated with subfields like geometry and probability to determine which operations and arguments to use.

Human interpretability → → \rightarrow Each operation and argument used to obtain the correct solution should relate to part of the input word problem context or a previous step in the operation program.

Learning logical forms has led to success in other areas of semantic parsing Cheng et al. ( 2017 ); Zelle and Mooney ( 1996 ); Zettlemoyer and Collins ( 2007 , 2005 ) and is a natural representation for math word problem-solving steps. By augmenting our dataset with these formalisms, we are able to cover most types of math word problems 3 3 3 We omit high-order polynomials and problems where the solutions are entirely nonnumeric. . In contrast to other representations like simultaneous equations, our formalisms ensure that every problem-solving step is aligned to a previous one. There are three advantages to this approach. First, we use this representation language to provide human annotators with clear steps for how a particular problem should be solved with math and domain knowledge. Second, our formalisms provide neural models with a continuous path to execute operations for problems with systems of equations, instead of forcing models to align equations before problem solving. This reduces the possibility of intermediate errors being propagated and leading to a incorrect solution. Finally, by having neural models generate a solution path in our representation language before computing the final solution, we are able to reconstruct the logical hops inferred by the model output, increasing model interpretability.

Our dataset (called MathQA) consists of 37,200 math word problems, corresponding lists of multiple-choice options and aligned operation programs. We use problems in the AQuA dataset and carefully annotate those problems with formal operation programs.

Math problems are first categorized into math domains using term frequencies (more details in Section  5.2 ). These domains are used to prune the search space of possible operations to align with the word problem text. Figure 3 shows the category-based hierarchies for operation formalisms.

We use crowdsourcing to carefully align problems with operation programs (Section  4.1 ). Table  1 shows overall statistics of the dataset. 4 4 4 We also experimented with an automatic dynamic programming approach to annotation that generates operation programs for problems using numbers in the AQuA rationales. Due to the noise in the rationales, only 61 % percent 61 61\% of those problems pass our human validation. This is mainly due to the fact that the rationales are not complete programs and fail to explicitly describe all important numbers and operations required to solve the problem. To maintain interpretability of operation paths, we did not include automatic annotations from our dataset and focus on operation programs derived by crowdsourcing.

4.1 Annotation using Crowd Workers

Annotating GRE level math problems can be a challenging and time consuming task for humans. We design a dynamic annotation platform to annotate math word problems with formal operation programs. Our annotation platform has the following properties: (a) it provides basic math knowledge to annotators, (b) it is dynamic by iteratively calculating intermediate results after an operation submission, and (c) it employs quality control strategies.

Dynamic Annotation Platform

The annotators are provided with a problem description, a list of operations related to the problem category, and a list of valid arguments. They iteratively select operations and arguments until the problem is solved.

Operation Selection The annotators are instructed to sequentially select an operation from the list of operations in the problem category. Annotators are provided with math knowledge by hovering over every operation and getting the related hint that consists of arguments, formula and a short explanation of the operation.

Argument Selection After selecting the operation the list of valid arguments are presented to the annotators to choose from. Valid arguments consist of numbers in the problem, constants in the problem category, and the previous calculations. The annotators are restricted to select only from these valid arguments to prevent having noisy and dangling numbers. After submission of an operation and the corresponding arguments, the result of the operation is automatically calculated and will be added as a new valid argument to the argument list.

Program Submission To prevent annotators from submitting arbitrary programs, we enforce restrictions to the final submission. Our platform only accepts programs which include some numbers from the problem, and whose final calculation is very close to the correct numeric solution.

High Quality Crowd Workers

We dynamically evaluate and employ high-quality annotators through a collection of quality-control questions. We take advantage of the annotation platform in Figure Eight . 5 5 5 https://www.figure-eight.com The annotators are randomly evaluated through a pre-defined set of test questions, and they have to maintain an accuracy threshold to be able to continue their annotations. If an annotator’s accuracy drops below a threshold, their previous annotations are labeled as untrusted and will be added to the pool of annotations again.

Alignment Validation

To further evaluate the quality of the annotated programs, we leverage a validation strategy to check whether the problems and annotated programs are aligned or not. According to this strategy, at least 2 out of 3 validators should rank the operation program as valid for it to be selected. The validation accuracy is 94.64 % percent 94.64 94.64\% across categories.

We develop encoder-decoder neural models to map word problems to a set of feasible operation programs. We match the result of the executed operation program against the list of multiple-choice options given for a particular problem. The matching solution is the final model output.

We frame the problem of aligning an operation program with a math word problem as a neural machine translation (NMT) task, where the word problem 𝐱 𝐱 \mathbf{x} and gold operation program 𝐲 𝐲 \mathbf{y} form a parallel text pair. The vocabulary of 𝐲 𝐲 \mathbf{y} includes all possible operations and arguments in our representation language.

5.1 Sequence-to-Program

For our initial sequence-to-program model, we follow the attention-based NMT paradigm of Bahdanau et al. ( 2015 ); Cho et al. ( 2014 ) . We encode the source word problem text 𝐱 = ( x 1 , x 2 , … , x M ) 𝐱 subscript 𝑥 1 subscript 𝑥 2 … subscript 𝑥 𝑀 \mathbf{x}=(x_{1},x_{2},...,x_{M}) using a bidirectional RNN encoder θ e ​ n ​ c superscript 𝜃 𝑒 𝑛 𝑐 \theta^{enc} . The decoder θ d ​ e ​ c superscript 𝜃 𝑑 𝑒 𝑐 \theta^{dec} predicts a distribution over the vocabulary and input tokens to generate each operation or argument in the target operation program. For our sequence-to-program model vocabulary, we use informed generation, in which the program tokens are generated separately from the vocabulary of operations O 𝑂 O or arguments A 𝐴 A .

We compute the d 𝑑 d -dimensional decoder hidden state h i d ​ e ​ c subscript superscript ℎ 𝑑 𝑒 𝑐 𝑖 h^{dec}_{i} using a LSTM recurrent layer:

At each timestep, we make a prediction for an operator o ​ p i 𝑜 subscript 𝑝 𝑖 op_{i} or argument a ​ r ​ g i ​ k 𝑎 𝑟 subscript 𝑔 𝑖 𝑘 arg_{ik} , where k 𝑘 k corresponds to the index of the argument in operator i 𝑖 i ’s argument list. This prediction is conditioned on the previous tokens ( y 1 , … , y i − 1 ) subscript 𝑦 1 … subscript 𝑦 𝑖 1 (y_{1},...,y_{i-1}) and the input 𝐱 𝐱 \mathbf{x} to decode an entire operation program 𝐲 = ( y 1 , y 2 , … , y N ) 𝐲 subscript 𝑦 1 subscript 𝑦 2 … subscript 𝑦 𝑁 \mathbf{y}=(y_{1},y_{2},...,y_{N}) of length N 𝑁 N :

Here f 𝑓 f is a 1-layer feed-forward neural network and g 𝑔 g is the softmax function. During training time, we minimize the negative log-likelihood (NLL) using the following objective:

At test time, we only observe the input text when predicting operation programs:

5.2 Categorized Sequence-to-Program Model

We extend our base sequence-to-program model to integrate knowledge of math word problem domain categories. We modify the RNN decoder layers that compute the decoder hidden state to be category-aware. Here, the category label c 𝑐 c is deterministically computed by the category extractor (explained below). It functions as a hard decision switch that determines which set of parameters to use for the hidden state computation:

The updated objective function from equation ( 7 ) is shown below:

The full model architecture is shown in Figure 4 .

Domain-Specific Category Extraction

We first construct a lexicon of n-grams relating to a specific domain. The lexicon is a list consisting of domain-specific categories and associated n-grams. For each domain category c 𝑐 c in the lexicon, we select associated n-grams n c subscript n 𝑐 \textbf{n}_{c} that occur frequently in word problems belonging to domain category c 𝑐 c , but rarely appear in other domain categories. We compute n-gram frequency f p ​ c subscript 𝑓 𝑝 𝑐 f_{pc} as the number of n-grams associated with a category c 𝑐 c appearing in the text of a word problem p 𝑝 p . We obtain a list of potential categories for p 𝑝 p by choosing all categories for which f p ​ c > 0 subscript 𝑓 𝑝 𝑐 0 f_{pc}>0 , and then assign a category label to p 𝑝 p based on which category has the highest n-gram frequency.

5.3 Solving Operation Programs

Once a complete operation program has been decoded, each operator in the program is executed sequentially along with its predicted set of arguments to obtain a possible solution. For each word problem p 𝑝 p and options o 𝑜 o , we generate a beam of the top n 𝑛 n decoded operation programs. We execute each decoded program g 𝑔 g to find the solution from the list of options o of the problem. We first choose options that are within a threshold of the executed value of g 𝑔 g . We select g 𝑔 g as the predicted solution by checking the number of selected options and the minimum distance between the executed value of g 𝑔 g and a possible option for p 𝑝 p . For the problems in AQuA that do not belong in any category of MathQA, we randomly choose an option.

6 Experimental Setup

6.1 datasets.

Our dataset consists of 37 ​ k 37 𝑘 37k problems which are randomly split in ( 80 / 12 / 8 ) % percent 80 12 8 (80/12/8)\% training/dev/test problems. Our dataset significantly enhances the AQuA dataset by fully annotating a portion of solvable problems in the AQuA dataset into formal operation programs.

We carefully study the AQuA dataset. Many of the problems are near-duplicates with slight changes to the math word problem stories or numerical values since they are expanded from a set of 30,000 seed problems through crowdsourcing Ling et al. ( 2017 ) . These changes are not always reflected in the rationales, leading to incorrect solutions. There are also some problems that are not solvable given current math word problem solving frameworks because they require a level of reasoning not yet modeled by neural networks. Sequence problems, for example, require understanding of patterns that are difficult to intuit without domain knowledge like sequence formulas, and can only be solved automatically through brute-force or guessing. Table 2 shows a full breakdown of the AQuA dataset by solvability. 6 6 6 There is overlap between unsolvable subsets. For example, a sequence problem may also be a duplicate of another problem in the AQuA dataset.

6.2 Annotation Details

We follow the annotation strategy described in Section  4 to formally annotate problems with operation programs. 7 7 7 We tried two other strategies of showing extra information (rationales or end solutions) to annotators to facilitate solving problems. However, our manual validation showed that annotators mostly used those extra information to artificially build an operation program without reading the problem.

Annotator Agreements and Evaluations

Our expert evaluation of the annotation procedure for a collection of 500 problems shows that 92% of the annotations are valid. Additionally, it has 87 % percent 87 87\% agreement between the expert validation and the crowd sourcing validation task.

Annotation Expansion

The AQuA dataset consists of a group of problems which share similar characteristics. These problems can be solved with similar operation programs. We find closely similar problems, replace numeric values with generic numbers, and expand annotations to cover more problems from the AQuA dataset. For similarity, we use Levenshtein distance with a threshold of 4 words in edit distance.

6.3 Model and Training Details

We use the official python implementation of OpenNMT Klein et al. . We choose a LSTM-based encoder-decoder architecture. We use Adam optimizer  Kingma and Ba ( 2015 ) , and the learning rate for training is 0.001 0.001 0.001 . The hidden size for the encoder and decoder is set to d = 100 𝑑 100 d=100 . Both the encoder and decoder have 2 2 2 layers. The word embedding vectors are randomly initialized. At inference time, we implemented a beam search with beam size of 200 for AQuA and 100 for MathQA.

The program vocabulary consists of the operations O 𝑂 O in our representation language and valid arguments A 𝐴 A . For valid arguments, we do not use their actual values since the space is very large. Instead, we keep a list of numbers according to their source. Constants are predefined numbers that are available to all problems. Problem numbers are added to the list according to their order in the problem text. Calculated numbers in the intermediate steps are added to the list according to the operation order.

7 Experimental Results

7.1 results.

Table 3 compares the performance of our sequence-to-program models trained on MathQA with baselines on MathQA and AQuA test sets. The base model is referred to as “Seq2prog,” while our model with categorization is “Seq2prog + cat.” For accuracy, the performance was measured in terms of how well the model would perform on an actual math test.

We observe improvement for our “Seq2prog + cat” model despite the fact that our training data is proportionally smaller than the AQuA dataset, and our model is much simpler than the state-of-the-art model on this dataset. This indicates the effectiveness of our formal representation language to incorporate domain knowledge as well as the quality of the annotations in our dataset.

7.2 Analysis

Qualitative analysis.

Table 5 and Figure  5 show some examples of problems solved by our method. We analyzed 50 problems that are solved wrongly by our system on the MathQA dataset. Table  4 summarizes four major categories of errors.

The most common type of errors are problems that need complicated or long chain of mathematical reasoning. For example, the first problem in Table  4 requires reasoning that goes beyond one sentence. Other errors are due to limitations in our representation language. For example, the second problem in Table  4 requires the factorization operation which is not defined in our representation language. Future work can investigate more domains of mathematics such as logic, number factors, etc. Some errors are due to the slightly noisy nature of our categorization strategy. For example, the third problem in Table  4 is mistakenly categorized as belonging to physics domain due to the presence of words m, cm, liter in the problem text, while the correct category for the problem is geometry . The final category of errors are due to problems that do not have enough textual context or erroneous problems (e.g., fourth problem in Table  4 ).

Impact of Categorization

Table  3 indicates that our category-aware model outperforms the base model on both AQuA and MathQA datasets. The gain is relatively small because the current model only uses categorization decisions as hard constraints at decoding time. Moreover, the problem categorization might be noisy due to our use of only one mathematical interpretation for each domain-specific n-gram. For example, the presence of the words “square” or “cube” in the text of a math word problem indicate that the word problem is related to the geometry domain, but these unigrams can also refer to an exponential operation ( n 2 superscript 𝑛 2 n^{2} or n 3 superscript 𝑛 3 n^{3} ).

To measure the effectiveness of our categorization strategy, we used human annotation over 100 problems. The agreement between human annotators is 84 % percent 84 84\% and their agreement with our model is 74.5 % percent 74.5 74.5\% . As a future extension of this work, we would like to also consider the context in which domain-specific n-grams appear.

Discussions

As we mentioned in section 3 , the continuous nature of our formalism allows us to solve problems requiring systems of equations. However, there are other types of word problems that are currently unsolvable or have multiple interpretations leading to multiple correct solutions. While problems that can only be solved by brute-force instead of logical reasoning and non-narrative problems that do not fit the definition of a math word problem (in Table 2 these appear as “no word”) are removed from consideration, there are other problems that are beyond the scope of current models but could pose an interesting challenge for future work. One example is the domain of sequence problems. Unlike past word problem-solving models, our models incorporate domain-specific math knowledge, which is potentially extensible to common sequence and series formulas.

8 Conclusion

In this work, we introduced a representation language and annotation system for large-scale math word problem-solving datasets that addresses unwanted noise in these datasets and lack of formal operation-based representations. We demonstrated the effectiveness of our representation language by transforming solvable AQuA word problems into operation formalisms. Experimental results show that both our base and category-aware sequence-to-program models outperform baselines and previous results on the AQuA dataset when trained on data aligned with our representation language. Our representation language provides an extra layer of supervision that can be used to reduce the influence of statistical bias in datasets like AQuA. Additionally, generated operation programs like the examples in figure 5 demonstrate the effectiveness of these operation formalisms for representing math word problems in a human interpretable form.

The gap between the performance of our models and human performance indicates that our MathQA  still maintains the challenging nature of AQuA problems. In future work, we plan to extend our representation language and models to cover currently unsolvable problems, including sequence and high-order polynomial problems.

Acknowledgements

This research was supported by ONR (N00014-18-1-2826), NSF (IIS 1616112), Allen Distinguished Investigator Award, and gifts from Google, Allen Institute for AI, Amazon, and Bloomberg. We thank Marti A. Hearst, Katie Stasaski, and the anonymous reviewers for their helpful comments.

• Bahdanau et al. (2015) D. Bahdanau, K. Cho, and Y. Bengio. 2015. Machine translation by jointly learning to align and translate. In ICLR .
• Bakman (2007) Yefim Bakman. 2007. Robust understanding of word problems with extraneous information. In arXiv preprint math/0701393 .
• Bobrow (1964) Daniel G Bobrow. 1964. Natural language input for a computer problem solving system.
• Briars and Larkin (1984) Diane J Briars and Jill H Larkin. 1984. An integrated model of skill in solving elementary word problems. In Cognition and instruction 1(3) , pages 245–296.
• Charniak (1968) Eugene Charniak. 1968. Calculus word problems. Ph.D. thesis, Massachusetts Institute of Technology.
• Charniak (1969) Eugene Charniak. 1969. Computer solution of calculus word problems. In Proceedings of the 1st international joint conference on Artificial intelligence , pages 303–316. Morgan Kaufmann Publishers Inc.
• Cheng et al. (2017) Jianpeng Cheng, Siva Reddy, Vijay Saraswat, and Mirella Lapata. 2017. Learning structured natural language representations for semantic parsing . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 44–55. Association for Computational Linguistics.
• Cho et al. (2014) Kyunghyun Cho, Bart van Merri e ¨ ¨ e \ddot{\text{e}} nboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder–decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation , pages 103–111.
• Dellarosa (1986) Denise Dellarosa. 1986. A computer simulation of childrens arithmetic word-problem solving. In Behavior Research Methods, Instruments, Computers 18(2) , pages 147–154.
• Hosseini et al. (2017) Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, and Nate Kushman. 2017. Learning to solve arithmetic word problems with verb categorization. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) .
• Huang et al. (2018a) Danqing Huang, Jing Liu, Chin-Yew Lin, and Jian Yin. 2018a. Neural math word problem solver with reinforcement learning. In Proceedings of the 27th International Conference on Computational Linguistics , pages 213–223.
• Huang et al. (2016) Danqing Huang, Shuming Shi, Chin-Yew Lin, Jian Yin, and Wei-Ying Ma. 2016. How well do computers solve math word problems? large-scale dataset construction and evaluation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) .
• Huang et al. (2018b) Danqing Huang, Jin-Ge Yao, Chin-Yew Lin, Qingyu Zhou, and Jian Yin. 2018b. Using intermediate representations to solve math word problems. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics .
• Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In the 3rd International Conference for Learning Representations (ICLR) .
• (15) G. Klein, Y. Kim, Y. Deng, J. Senellart, and A. M. Rush. Opennmt: Open-source toolkit for neural machine translation. ArXiv e-prints .
• Koncel-Kedziorski et al. (2016) Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016. Mawps: A math word problem repository. In NAACL .
• Kushman et al. (2014) Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014. Learning to automatically solve algebra word problems. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL) .
• Liang et al. (2018) Chao-Chun Liang, Yu-Shiang Wong, Yi-Chung Lin, and Keh-Yih Su. 2018. A meaning-based statistical english math word problem solver. In Proceedings of NAACL-HLT 2018 .
• Liguda and Pfeiffer (2012) Christian Liguda and Thies Pfeiffer. 2012. Modeling math word problems with augmented semantic networks. In International Conference on Application of Natural Language to Information Systems , pages 247–252. Springer.
• Ling et al. (2017) Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of ACL .
• Locascio et al. (2016) Nicholas Locascio, Karthik Narasimhan, Eduardo DeLeon, Nate Kushman, and Regina Barzilay. 2016. Neural generation of regular expressions from natural language with minimal domain knowledge. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 247–252. Association for Computational Linguistics.
• Roy and Roth (2018) Subhro Roy and Dan Roth. 2018. Mapping to declarative knowledge for word problem solving. In Proceedings of NAACL-HLT 2018 .
• Upadhyay and Chang (2017) Shyam Upadhyay and Ming-Wei Chang. 2017. Annotating derivations: A new evaluation strategy and dataset for algebra word problems. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics .
• Wang et al. (2017) Yan Wang, Xiaojiang Liu, and Shuming Shi. 2017. Deep neural solver for math word problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP) .
• Zelle and Mooney (1996) John M. Zelle and Raymond J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In AAAI/IAAI , pages 1050–1055. AAAI Press/MIT Press.
• Zettlemoyer and Collins (2005) Luke Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In UAI ’05, Proceedings of the 21st Conference in Uncertainty in Artificial Intelligence , pages 658–666.
• Zettlemoyer and Collins (2007) Luke Zettlemoyer and Michael Collins. 2007. Online learning of relaxed ccg grammars for parsing to logical form. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLPCoNLL) , pages 678–687. Association for Computational Linguistics.

Bridging Formal and Informal Learning Through Technology in the Twenty-First Century: Issues and Challenges

• Reference work entry
• First Online: 10 August 2018
• Cite this reference work entry

• Cathy Lewin 5 &
• Amina Charania 6  

Part of the book series: Springer International Handbooks of Education ((SIHE))

4460 Accesses

6 Citations

This chapter presents a comprehensive review of the current debates surrounding bridging informal and formal learning, from the perspective of improving the learner’s experience in formal educational provision. Firstly, the chapter reviews the literature defining informal and formal learning, noting the complexity and the lack of consensus. Secondly, it discusses how technology can be used to bridge learning through harnessing the digital practices that young people engage with informally such as social networking, game-based learning, and digital making. The authors then outline some pedagogical issues which need to be considered to maximize the potential of bridging formal and informal learning. Next, the pedagogical strategies needed to enhance learners’ opportunities for autonomy, collaboration, and authentic learning are discussed. The chapter also explores the divides, cultural tensions, and ethical concerns that shape practices such as the constraints of a performativity culture and the invasion of young people’s private space. A vignette of a project in India is presented as an illustration of good practice. Here, despite limited access to technology, young people have been supported to engage in authentic learning projects involving the creation of digital artifacts, both in- and out-of-school. The chapter concludes by arguing that there must be a shift from transmissive to collaborative pedagogical strategies; school cultures need to change. In order to do so, teachers need professional development and support to take risks and experiment. More research is needed so that the interrelationship between technology-enabled formal and informal learning can be better understood but also because good models of practice need to be identified and shared.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Adams Becker, S., Freeman, A., Giesinger Hall, C., Cummins, M., & Yuhnke, B. (2016). NMC/CoSN horizon report: 2016 K-12 edition . Austin: The New Media Consortium.

Google Scholar  

Adhikari, S. (2014, August 10). M-learning, the way to go. The Hindu . Retrieved from http://www.thehindu.com/features/education/mlearning-the-way-to-go/article6298965.ece .

Arnesen, T., Elstad, E., Salomon, G., & Vavik, L. (2016). Educational technology and polycontextual bridging: An introduction. In E. Elstad (Ed.), Educational technology and Polycontextual bridging (pp. 3–14). Rotterdam/Boston/Taipei: Sense Publishers.

Chapter   Google Scholar  

Banks, J. A., Au, K. H., Ball, A. F., Bell, P., Gordon, E. W., Gutiérrez, K. D., & Zhou, M. (2007). Learning in and out of school in diverse environments: Life-long, life-wide, life-deep . Seattle: LIFE Center and Center for Multicultural Education (University of Washington).

Beland, L.-P., & Murphy, R. (2015). Ill Communication: Technology, Distraction & Student Performance . CEP Discussion Paper No 1350, May 2015. London: Centre for Economic Performance, London School of Economics and Political Science.

Birdwell, J., Scott, R., & Koninckx, D. (2015). Non-formal learning could help to build character and close attainment gap: Learning by doing . London: DEMOS.

Boticki, I., Baksa, J., Seow, P., & Looi, C. (2015). Usage of a mobile social learning platform with virtual badges in a primary school. Computers & Education, 86 , 120–136. https://doi.org/10.1016/j.compedu.2015.02.015 .

Article   Google Scholar  

Brevik, L. M. (2016). The gaming outliers: Does out-of-school gaming improve boys’ reading skills in English as a second language? In E. Elstad (Ed.), Educational technology and polycontextual bridging (pp. 389–361). Dordrecht: Sense Publishers.

Brewer, E., Demmer, M., Du, B., Ho, M., Kam, M., Nedevschi, S., Pal, J., Patra, R., Surana, S., & Fall, K. (2005). The case for technology in developing regions. IEEE Computer, 38 (6), 25–38. https://doi.org/10.1109/MC.2005.204 .

Chan, T.-W., Roschelle, J., Hsi, S., Kinshuk, M. S., Brown, T., Patton, C., Cherniavsky, J., Pea, R., Norris, C., Soloway, S., Balacheff, N., Scardamalia, M., Dillenbourg, P., Looi, C. K., Milrad, M., & Hoppe, U. (2006). One-to-one technology-enhanced learning: An opportunity for global research collaboration. Research and Practice in Technology Enhanced Learning, 1 (1), 3–29. https://doi.org/10.1142/S1793206806000032 .

Charania, A. (2015). India: Integrated ICT school supplements in community centres. In P. Twining, N. E. Davis, & A. Charania (Eds.), Developing new indicators to describe digital technology infrastructure in primary and secondary education (pp. 64–67). Montreal: UNESCO Institute for Statistics.

Chen, B., & Bryer, T. (2012). Investigating instructional strategies for using social media in formal and informal learning. The International Review of Research in Open and Distributed Learning, 13 (1), 87–104. https://doi.org/10.19173/irrodl.v13i1.1027 .

Clark, W., Logan, K., Luckin, R., Mee, A., & Oliver, M. (2009). Beyond web 2.0: Mapping the technology landscapes of young learners. Journal of Computer Assisted Learning, 25 (1), 56–69. https://doi.org/10.1111/j.1365-2729.2008.00305.x .

Colley, H., Hodkinson, P., & Malcolm, J. (2003). Informality and formality in learning: A report for the learning and skills research Centre . London: LSRC.

Cox, M. (2013). Formal to informal learning with IT: Research challenges and issues for e-learning. Journal of Computer Assisted Learning, 29 (1), 85–105. https://doi.org/10.1111/j.1365-2729.2012.00483.x .

Crook, C. (2012). The ‘digital native’ in context: Tensions associated with importing web 2.0 practices into the school setting. Oxford Review of Education, 38 , 63–80. https://doi.org/10.1080/03054985.2011.577946 .

Dabbagh, N., & Kitsantas, A. (2012). Personal learning environments, social media, and self-regulated learning: A natural formula for connecting formal and informal learning. Internet and Higher Education, 15 (1), 3–8. https://doi.org/10.1016/j.iheduc.2011.06.002 .

Davies, C., & Eynon, R. (2013). Studies of the internet in learning and education: Broadening the disciplinary landscape of research. In W. H. Dutton (Ed.), The Oxford handbook of internet studies (pp. 328–349). Oxford: Oxford University Press.

Deng, L., Connelly, J., & Lau, M. (2016). Interest-driven digital practices of secondary students: Cases of connected learning. Learning, Culture and Social Interaction, 9 , 45–54. https://doi.org/10.1016/j.lcsi.2016.01.004 .

Erstad, O. (2012). The learning lives of digital youth – Beyond the formal and informal. Oxford Review of Education, 38 , 25–43. https://doi.org/10.1080/03054985.2011.577940 .

Erstad, O., & Sefton-Green, J. (2013). Digital disconnect? The ‘digital learner’ and the school. In O. Erstad & J. Sefton-Green (Eds.), Identity, community, and learning lives in the digital age (pp. 87–104). New York: Cambridge University Press.

Erstad, O., Gilje, Ø., & Arnseth, H. C. (2013). Learning lives connected: Digital youth across school and community spaces. Comunicar, 40 , 89–98. https://doi.org/10.3916/C40-2013-02-09 .

Eshach, H. (2007). Bridging in-school and out-of-school learning: Formal, non-formal, and informal education. Journal of Science Education and Technology, 16 (2), 171–190. https://doi.org/10.1007/s10956-006-9027-1 .

Freina, L., & Ott, M. (2015). A literature review on immersive virtual reality in education: State of the art and perspectives. Proceedings of eLearning and Software for Education (eLSE), Bucharest, April 23–24, 2015.

Greenhow, C., & Lewin, C. (2016). Social media and education: Reconceptualizing the boundaries of formal and informal learning. Learning, Media and Technology, 41 (1), 6–30. https://doi.org/10.1080/17439884.2015.1064954 .

Hoppers, W. (2006). Non-formal education and basic education reform: A conceptual review . Paris: IIEP, UNESCO.

Hsi, S. (2007). Conceptualizing learning from the everyday activities of digital kids. International Journal of Science Education, 29 (12), 1509–1529. https://doi.org/10.1080/09500690701494076 .

Hung, D., Lee, S. S., & Lim, K. Y. T. (2012). Authenticity in learning for the twenty first century: Bridging the formal and the informal. Educational Technology Research & Development, 60 (6), 1071–1091. https://doi.org/10.1007/s11423-012-9272-3 .

Illeris, K. (2009). Transfer of learning in the learning society: How can the barriers between different learning spaces be surmounted, and how can the gap between learning inside and outside schools be bridged? International Journal of Lifelong Education, 28 (2), 137–148. https://doi.org/10.1080/02601370902756986 .

Ito, M., Gutierrez, K., Livingstone, S., Penuel, B., Rhodes, J., Salen, K., Schor, J., Sefton-Green, J., & Watkins, S. (2013). Connected learning: An agenda for research and design . Irvine: Digital Media and Learning Research Hub.

Khaddage, F., Müller, W., & Flintoff, K. (2016). Advancing mobile learning in formal and informal settings via mobile app technology: Where to from here, and how? Educational Technology & Society, 19 (3), 16–26.

Kluge, A. (2016). I am connected, therefore I am: Polycontextual bridging in education. In E. Elstad (Ed.), Educational technology and Polycontextual bridging (pp. 129–148). Rotterdam/Boston/Taipei: Sense Publishers.

Kumar, A., Tewari, A., Shroff, G., Chittamuru, D., Kam, M., and Canny, J. (2010). An exploratory study of unsupervised mobile learning in rural India. In CHI 2010, April 10–15, 2010, Atlanta.

Kumpulainen, K., & Mikkola, A. (2016). Toward hybrid learning: Educational engagement and learning in the digital age. In E. Elstad (Ed.), Educational technology and Polycontextual bridging (pp. 15–38). Rotterdam/Boston/Taipei: Sense Publishers.

Kumpulainen, K., & Sefton-Green, J. (2014). What is connected learning and how to research it? International Journal of Learning and Media, 4 (2), 7–18. https://doi.org/10.1162/IJLM_a_00091 .

Lai, K. W., Khaddage, F., & Knezek, G. (2013). Blending student technology experiences in formal and informal learning. Journal of Computer Assisted Learning, 29 , 414–425. https://doi.org/10.1111/jcal.12030 .

Laru, J., & Järvelä, S. (2015). Seamless learning despite context. In L.-H. Wong, M. Specht, & M. Milrad (Eds.), Seamless learning in the age of mobile connectivity (pp. 471–484). Singapore: Springer.

Lemke, J. L., Lecusay, R., Cole, M., & Michalchik, V. (2015). Documenting and assessing learning in informal and media-rich environments . Cambridge, MA: MIT Press.

Lopez, M. E., & Caspe, M. (2014). Family engagement in anywhere, anytime learning . Cambridge, MA: Harvard Family Research Project.

Mao, J. (2014). Social media for learning: A mixed methods study of high school students’ technology affordances and perspectives. Computers in Human Behavior, 33 , 213–223. https://doi.org/10.1016/j.chb.2014.01.002 .

McKay, C. & Peppler, K. (2013). MakerCart: A mobile fab lab for the classroom. Position Paper at the Interaction Design for Children Conference (IDC), New York. Retrieved from http://kpeppler.com/Docs/2013_Peppler_Maker_Cart.pdf .

Merchant, G. (2012). Unravelling the social network: Theory and research. Learning, Media and Technology, 37 (1), 4–19. https://doi.org/10.1080/17439884.2011.567992 .

National Research Council (NRC). (2015). Identifying and supporting productive STEM programs in out-of-school settings . Committee on successful out-of-school STEM learning. Board on science education. In Division of behavioral and social sciences and education . Washington, DC: The National Academies Press.

Peppler, K., & Bender, S. (2013). Maker movement spreads innovation one project at a time. Phi Delta Kappan, 95 (3), 22–27. https://doi.org/10.1177/003172171309500306 .

Quinlan, O. (2015). Young digital makers . London: Nesta.

Rajala, A., Kumpulainen, K., Hilppö, J., Paananen, A., & Lipponen, L. (2016). Connecting learning across school and out-of-school contexts: A review of pedagogical approaches. In O. Erstad, K. Kumpulainen, Å. Mäkitalo, K. C. Schrøder, P. Pruulmann-Vegerfeldt, & T. Jóhannsdóttir (Eds.), Learning across contexts in the knowledge society (pp. 15–38). Rotterdam/Boston/Taipei: Sense Publishers.

Raman, A. (2014, June 5). Mobile learning: Smart education system for India. Forbes India . Retrieved from http://www.forbesindia.com/blog/business-strategy/mobile-learning-smart-education-system-for-india/ .

Rogers, A. (2014). The base of the iceberg: Informal learning and its impact on formal and non-formal learning . Opladen/Berlin/Toronto: Barbara Budrich Publishers.

Book   Google Scholar  

Schuck, S., Kearney, M., & Burden, K. (2017). Exploring mobile learning in the third space. Technology, Pedagogy and Education, 26 (2), 121–137. https://doi.org/10.1080/1475939X.2016.1230555 .

Sefton-Green, J. (2004). Report 7: Literature review in informal learning with technology outside school . Bristol: Futurelab.

Sefton-Green, J. (2013). Learning at not-school: A review of study, theory, and advocacy for education in non-formal settings . Cambridge, MA: MIT Press.

Sefton-Green, J., & Erstad, O. (2016). Researching ‘learning lives’ – A new agenda for learning, media and technology. Learning, Media and Technology, 42 (2), 1–5. https://doi.org/10.1080/17439884.2016.1170034 .

Sharples, M. (2015). Seamless learning despite context. In L.-H. Wong, M. Milrad, & M. Specht (Eds.), Seamless learning in the age of mobile connectivity (pp. 41–55). Singapore: Springer.

Srivastava, M. (2015, August 20). 80% of Indian employees find mobile learning useful, says study. Livemint . Retrieved from http://www.livemint.com/Consumer/Jx5FFXAsYA6nB4hob1f0gJ/80-Indian-employees-find-mobile-learning-useful-says-study.html .

Stocklmayer, S. M., Rennie, L. J., & Gilbert, J. K. (2010). The roles of the formal and informal sectors in the provision of effective science education. Studies in Science Education, 46 (1), 1–44. https://doi.org/10.1080/03057260903562284 .

The World Bank. (2003). Lifelong learning in the global knowledge economy: Challenges for developing countries . Washington, DC: The World Bank. Retrieved from http://siteresources.worldbank.org/INTLL/Resources/Lifelong-Learning-inthe-GlobalKnowledge-Economy/lifelonglearning_GKE.pdf .

UNESCO. (2012). UNESCO guidelines for the recognition, validation and accreditation of the outcomes of non-formal and informal learning . Hamburg: UNESCO Institute for Lifelong Learning. Retrieved from http://unesdoc.unesco.org/images/0021/002163/216360e.pdf .

Weigel, M., James, C., & Gardner, H. (2009). Learning: Peering backward and looking forward in the digital era. International Journal of Learning and Media., 1 (1), 1–18. https://doi.org/10.1162/ijlm.2009.0005 .

Werquin, P. (2010). Recognising non-formal and informal learning: Outcomes, policies and practices . Paris: OECD Publishing.

Yang, J. (2015). Recognition, validation and accreditation of non-formal and non-formal learning in UNESCO member states . Hamburg: UNESCO Institute for Lifelong Learning.

Download references

Author information

Authors and affiliations.

Manchester Metropolitan University, Manchester, UK

Cathy Lewin

Tata Institute of Social Sciences, Mumbai, India

Amina Charania

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Cathy Lewin or Amina Charania .

Editor information

Editors and affiliations.

University of Amsterdam, Amsterdam, The Netherlands

University of North Texas, Denton, Texas, USA

Gerald Knezek

Rhonda Christensen

University of Otago, Dunedin, New Zealand

Kwok-Wing Lai

Section Editor information

University of Otago College of Education, Dunedin, New Zealand

University of Otago College of Education, North Dunedin, New Zealand

Keryn Pratt

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this entry

Cite this entry.

Lewin, C., Charania, A. (2018). Bridging Formal and Informal Learning Through Technology in the Twenty-First Century: Issues and Challenges. In: Voogt, J., Knezek, G., Christensen, R., Lai, KW. (eds) Second Handbook of Information Technology in Primary and Secondary Education . Springer International Handbooks of Education. Springer, Cham. https://doi.org/10.1007/978-3-319-71054-9_13

Download citation

DOI : https://doi.org/10.1007/978-3-319-71054-9_13

Published : 10 August 2018

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-71053-2

Online ISBN : 978-3-319-71054-9

eBook Packages : Education Reference Module Humanities and Social Sciences Reference Module Education

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research