annotated research context

Annotated Research Context

Arcs are fair digital objects (fdos).

As such, they come along with metadata, code for operations, and a persistent identifier. ARCs are compliant to the FAIR Principles since they are​

Real-world ARC

An ARC is intended to capture complete (meta)data, including raw and processed data, but also workflow descriptions as well as essential external files. ARCs cover scenarios ranging from single experimental setups to complex experimental designs. The full ARC-specifications are provided here .

ARCs are RO-Crate certified

ARCs are a profile implementation of RO-Crates for basic plant research.

The ARC is packaged with its digital object identifier and its RO Metadata File (ro-crate-metadata.json).

This RO metadata file is automatically generated using a converter, which is able to read and transport information out of the ARC.

Tools for creating ISA model coherent ARCs

Although ARCs can be generated completely manually, we encourage you to use our convenient tools that assist you in the process. The ARC Commander offers machine-aided creation and completion of the ARC folder and file structure.

SWATE and our templates assists you in the process of creating your assay files.

ARCs offer a single point of entry logic for data management and computation

Isa investigation file.

A mandatory central registry for studies, assays, persons, … is saved as XLSX format, which follows the ISA model specification (v1.0).

Using the ARC Commander allows you to automatically fill and update the investigation file.

Every worksheet needs to contain one table object storing the metadata. Comments or additional information can be stored alongside with table objects in a worksheet.

Isa assay file

Assay metadata must be annotated in the file isa.assay.xlsx at the root of the assay's subdirectory. This workbook must contain a single assay organized in one or many worksheets. A worksheet named “assay” must store the STUDY ASSAYS section of the ISA model and is not required in the isa.investigation.xlsx. Additional worksheets must contain a table object organized on a per-row basis with the first row as column headers. Table objects must contain at least one source. Source sample relations must follow a unique path in a directed acyclic graph. Sources must be indicated by the column header Source Name, Samples accordingly by the header Sample Name.

Assays correspond to outcomes of experimental assays or analytical measurements and are treated as immutable data. Each assay is a collection of files stored in a single directory, including a mandatory metadata file in ISA-XLSX format.

Assay data files, as well as protocols, must be placed in a subdirectory individually.

We advise you to use SWATE for completing the study and assay metadata files, as it assists you in following the annotation principles and comes along with a broad range of prepared templates.

Studies are collections of material and resources used within the investigation. You need to place each study in a unique subdirectory. Material or experimental samples, as well as external data files, can be stored as virtual sample files (containing unique identifiers) in the resources directory. To describe the sample or material creation process, you can store protocols in the designated sub-folder.

For each study, an isa.study.xlsx file following the ISA study model needs to be present to specify the characteristics of all material and resources, such as a certain strain. Resources might include external data (e.g., knowledge files or result files) that need to be included and cannot be referenced due to external limitations. Resources described in a study file can be the input for one or multiple assays.

Workflows in ARCs represent processing steps used in computational analyses and other data transformations of assays to generate run results. Typical examples include data cleaning and preprocessing, computational analysis, or visualization.

All files belonging to a specific workflow need to be stored in a single sub-directory. This also includes a mandatory per-workflow executable CWL description (v1.2 or higher), which can contain a tool or workflow description.

We highly recommend to include a reproducible execution environment description in form of a Docker container description for tool descriptions.

Runs in an ARC represent all artefacts that derive from computations on assay and external data. Plots, tables, or similar results, specific to certain run need to be saved in a subdirectory of the top-level runs directory.

A run.cwl (v1.2 or higher) is mandatory for each of these subdirectories to cover workflow description and reproducibility. These files need to be executable without additional payload files or files outside the ARC.

You can speficy input parameters for each run with a run.yml parameter file.

Research Strategies

• Reference Resources
• News Articles
• Scholarly Sources
• Search Strategy
• OneSearch Tips
• Evaluating Information
• Revising & Polishing
• Presentations & Media
• MLA 9th Citation Style
• APA 7th Citation Style
• Other Citation Styles
• Citation Managers
• Annotated Bibliography
• Literature Review How to

What is An Annotated Bibliography?

An annotated bibliography is a list of sources (books, articles, websites, etc.) with short paragraph about each source. An annotated bibliography is sometimes a useful step before drafting a research paper, or it can stand alone as an overview of the research available on a topic.

Each source in the annotated bibliography has a citation - the information a reader needs to find the original source, in a consistent format to make that easier. These consistent formats are called citation styles.  The most common citation styles are MLA (Modern Language Association) for humanities, and APA (American Psychological Association) for social sciences.

Annotations are about 4 to 6 sentences long (roughly 150 words), and address:

•     Main focus or purpose of the work
•     Usefulness or relevance to your research topic 
•     Special features of the work that were unique or helpful
•     Background and credibility of the author
•     Conclusions or observations reached by the author
•     Conclusions or observations reached by you

Annotations versus Abstracts

Many scholarly articles start with an abstract, which is the author's summary of the article to help you decide whether you should read the entire article.  This abstract is not the same thing as an annotation.  The annotation needs to be in your own words, to explain the relevance of the source to your particular assignment or research question.

Annotated Bibliography video

MLA 9th Annotated Bibliography Examples

Ontiveros, Randy J.  In the Spirit of a New People: The Cultural Politics of the Chicano Movement . New York UP, 2014.

This book analyzes the journalism, visual arts, theater, and novels of the Chicano movement from 1960 to the present as articulations of personal and collective values. Chapter 3 grounds the theater of El Teatro Campesino in the labor and immigrant organizing of the period, while Chapter 4 situates Sandra Cisneros’s novel  Caramelo  in the struggles of Chicana feminists to be heard in the traditional and nationalist elements of the Chicano movement. Ontiveros provides a powerful and illuminating historical context for the literary and political texts of the movement.

Journal article

Alvarez, Nadia, and Jack Mearns. “The Benefits of Writing and Performing in the Spoken Word Poetry Community.”  The Arts in Psychotherapy , vol. 41, no. 3, July 2014, pp. 263-268.  ScienceDirect ,  https://doi.org/10.1016/j.aip.2014.03.004 .

Spoken word poetry is distinctive because it is written to be performed out loud, in person, by the poet. The ten poets interviewed by these authors describe “a reciprocal relationship between the audience and the poet” created by that practice of performance. To build community, spoken word poets keep metaphor and diction relatively simple and accessible. Richness is instead built through fragmented stories that coalesce into emotional narratives about personal and community concerns.  This understanding of poets’ intentions illuminates their recorded performances.

*Note, citations have a .5 hanging indent and the annotations have a 1 inch indent. 

• MLA 9th Sample Annotated Bibliography

APA 7th Annotated Bibliography Examples

Alvarez, N. & Mearns, J. (2014). The benefits of writing and performing in the spoken word poetry community.  The Arts in Psychotherapy, 41 (3), 263-268.  https://doi.org/10.1016/j.aip.2014.03.004 Prior research has shown narrative writing to help with making meaning out of trauma. This article uses grounded theory to analyze semi-structured interviews with ten spoken word poets.  Because spoken word poetry is performed live, it creates personal and community connections that enhance the emotional development and resolution offered by the practice of writing. The findings are limited by the small, nonrandom sample (all the participants were from the same community).

• APA 7th Sample Annotated Bibliography
• << Previous: Citation Managers
• Next: Literature Review How to >>
• Last Updated: Apr 8, 2024 9:46 AM
• URL: https://libguides.csun.edu/research-strategies

Report ADA Problems with Library Services and Resources

Annotated Research Context

The importance of Research Data Management (RDM) systems steadily increases, as scientists and institutions generate big, heterogeneous data nowadays. According to the FAIR principles such data should be available in open, public archives, stored in contextualized, non-proprietary formats. With the Annotated Research Context (ARC), DataPLANT provides an RDM platform to answer these requirements and to support machine-readability, as the system is entirely based on established standards. ARCs will be tagged with a persistent and unique identifier and contain, besides raw data, the entire information (metadata, external files, and code for computations) needed to reproduce the mapped studies. These can range from a single publication up to a multi-lab, long-term project. Thus, ARCs are FAIR Digital Objects (FDOs) and drive FAIRification of research data for fundamental plant research.

The data-centric model is based on an architecture in which data is the primary and permanent asset and applications are interchangeable. Thus, the data model precedes the implementation of any given service and application, which are in a state of constant change to meet user requirements and experiences or functionality extensions. To realize such a data centric approach for RDM in fundamental plant research, we propose the Annotated Research Context to capture and structure the complete research cycle meeting the FAIR requirements with low friction for the individual researcher. ARCs are self-contained and include assay or measurement data, workflow and computation results accompanied by metadata in one package. Their structure allows full user control over all metadata and facilitates usability, access, publication and sharing of the research. Thereby, ARCs are a practical implementation of existing standards encompassing the benefits of the ISA model , Common Workflow Language (CWL) , and Research Object Crates (RO-Crate) .

The ARC concept relies on a structure that partitions studies, assays, workflows, and runs for a granular reuse and development. Studies cover biological data, while assays cover experimental, and instrumental data, including their self-contained description using the ISA model. Similarly, workflows cover all computational steps of a study and contain application code, scripts, or any other executable description of an analysis ensuring highest flexibility for the scientists. To ensure persistence and reproducibility, these workflows comprise their own containerized running environment. The resulting data (runs) is linked to the workflows by a minimal Common Workflow Language file specifying the input and output of the process. The suggested structure for ARCs is a starting point for individual research projects and defines a framework for the organization, sharing, reuse, and evolution of research projects in a fashion familiar from open-source software development (see also Data Sharing ).

This results in standardized RDM procedures being process-oriented, meaning that each tool realizes or supports the researcher in a distinct task within the RDM cycle. As a consequence, this enables the desired mixed mode of application, in which both human and machine can operate processes simultaneously or asynchronously while avoiding technological barriers.

Every ARC follows a distinct scheme with a specific directory and file structure, including the sub-directories "studies", "assays", "workflows", and "runs".

Studies are collections of material and resources used within the investigation. You need to place each study in a unique subdirectory. Material or experimental samples, as well as external data files, can be stored as virtual sample files (containing unique identifiers) in the "resources" directory. To describe the sample or material creation process, protocols can be stored in the designated sub-directory.

Assays correspond to outcomes of experimental assays or analytical measurements and are treated as immutable data. Each assay is a collection of files stored in a single directory, including corresponding metadata files. Assay data files, as well as protocols, are placed in a subdirectory individually.

Workflows in ARCs represent processing steps used in computational analyses and other data transformations of studies and assays to generate run results. Typical examples include data cleaning and preprocessing, computational analysis, or visualization. We highly recommend to include a reproducible execution environment description in form of a Docker container description for tool descriptions.

Runs in an ARC represent all artefacts that derive from computations on assay and external data. Plots, tables, or similar results, specific to certain runs need to be saved in a subdirectory of the top-level "runs" directory.

Detailed information can be found in the official ARC Specification .

ARCs are based on the lightweight and decentralized version-control system git . Thereby, every ARC contains its entire previous versions and is shipped together with them. Distribution and management of access rights is accomplished via DataPLANT’s GitLab instance functioning as DataHUB . The git extension Git LFS takes care of files with a size of >100 Mb, as it stores the files in a separate location to avoid long synchronization times and creates a pointer file in the repository to establish a connection. Git (LFS) also ensures integrity of your data, as all objects in git are accessed using the SHA-1 hash function. Changing the content of a file thereby also results in a change of the checksum. This prevents corruption of files, as Git will return an error if some data cannot be found based on its checksum (see also git ).

Once an ARC is filled, it should be able to target and interact with multiple platforms and services for research data management and computation. ARCs are stored on DataPLANTs DataHUB (GitLab), which enables collaborative work and version control immediately after initiation. Currently developed converters will be able to extract, transform, and validate metadata within an ARC to established standards like the Research Object Crate (RO-Crate) and facilitate an automatic upload to desired domain-specific endpoint repositories (such as GEO, MetaboLights, or PRIDE). Reproducible and transparent analyses are furthermore supported by connectivity to the workflow management system Galaxy.

With our platform, data can be stored and jointly used prior to final submission. This allows wet lab scientists, (measurement) facilities, and bioinformaticians to add and logically connect their contributions to existing projects. Git's versioning capability allows to trace each step at any time, preserving the provenance of each contribution. This could result in a shift from project-oriented to scientist-centered publications, while the context of the investigation remains the same.

ARCs will receive a unique and persistent identifier, which enables referencing in publications and grants or on service platforms. Furthermore, with the use of a metadata registry, researchers will be able to query databases for distinct terms for all ARCs they have access to, including public ARCs, their own ARCs and ARCs they were invited to for collaboration. These queries can range from project names and involved scientists down to single methods and even single datasets present in an ARC.

Today, investigations are oftentimes published with the minimal information needed, such as protocols or raw datasets. Researchers interested in said studies carefully need to comprehend carried out processes and manually set up computations trying to reproduce and reuse published results. We envision to facilitate and extend data publications with the possibility to automatically replicate results by shipping the code with the respective data. In the future, ARCs should contain a blueprint in form of an executable workflow and run description that allows for auto-processing of data to generate results present in a distinct publication.

The following table gives an overview about DataPLANT tools and services supporting you in creating your own ARC. Follow the link in the first column for details.

• Common Workflow Language
• Research Object Crate

DataPLANT Support

Annotated Bibliographies

What is an annotated bibliography, creating an annotated bibliography, what about formatting, sample papers.

An annotated bibliography is a list of citations to sources, such as books and articles. Each citation is followed by an annotation, a brief descriptive and evaluative paragraph about 150 words long, that analyzes the source. An annotated bibliography usually looks like any other bibliography with alphabetized citations of sources, except that here each source is followed by an explanatory paragraph. The purpose of the annotation is to inform on the relevance, accuracy, and quality of the sources cited, and this work can form the basis of a literature review later in the writing process.

Types of annotations 

• Descriptive: states the topic of the source only
• Evaluative: evaluates the source, which may include placing the work in context of other research or evaluating its usefulness. This is the type expected for most research assignments. 
• Summary: summarizes the source but does not take a stance or make an argument about the source.

Select Topic

Your topic should be neither too broad nor too narrow, but engage with a specific research question. You may not have a thesis but will form one in the course of reading sources. Consider some strategies for selecting and refining a topic.

Locate Sources 

This is a time-consuming process when writing an annotated bibliography. You are looking for sources that work together to support or refute your research question, not just the first few sources available. You should also consider a variety of sources, including books, articles, primary sources, and reference materials. Check the Research Guide in your discipline for suggestions. 

Read and Evaluate Sources

Evaluating a source is about more than reading the abstract. As part of the annotation, you should provide the following information: a summary of the source, the intended audience, a critical evaluation of the argument, and a contextual analysis of how it fits in your own research. 

Create Citations 

The citation is the first piece of information a reader will see, and should conform to one of the major citation style guides . Most guides require a "hanging first line," whereby the first line of the citation sits further to the right on the page with subsequent lines indented. This is a special indentation feature offered in the paragraph formatting section of Word (or other word-processing software). Don't try to indent by just adding spaces.

Write Annotations

Each annotation immediately follows the citation, and consists of a short, evaluative paragraph. It can include a very brief summary of the source, along with information about the author(s) and intended audience, followed by a critical analysis of the source in relation to your topic and research question. 

Ask yourself: Does it cover my topic? Is it a good representation of the sources available on the topic?  An annotated bibliography isn't only a list of sources; the annotations should indicate some relationship between the sources and how they work together in the context of your research. 

Style and Format 

As a final check, be sure all the citations are formatted in accordance with your chosen style guide. Also make sure the overall organization of the bibliography makes sense in the context of the research question. 

Most of the major citation styles call for a hanging first line on annotated bibliographies. This means the first line of the citation will align with the left margin of the page, and all subsequent lines of the citation and annotation will indent to the right. 

Annotated Bibliography Samples

Stoll, J. S., Leslie, H. M., Britsch, M. L. & Cleaver, C. M. (2019). Evaluating aquaculture as a diversification strategy for Maine's commercial fishing sector in the face of change. Marine Policy , 107(103583), 1-10. https://doi.org/10.1016/j.marpol.2019.103583

This article discusses the growth of the aquaculture sector in Maine and analyzes the overlap between commercial fishing licenses and aquaculture leases and licenses to determine the extent to which commercial fishers are diversifying into aquaculture. A small percentage of commercial fishers are licensed to do aquaculture and instead, those getting into aquaculture appear to be a new group of ocean users. This has implications for efforts to diversify commercial fishing-dependent livelihoods as a response to climate change impacts. This is relevant to my research in which I am exploring the potential for aquaculture to be a diversification strategy for commercial fishermen. I will likely use this source as supporting evidence that such a transition does not currently appear to be taking place other than for a minority of commercial fishermen.

Chicago Notes & Bibliography

Stoll, Joshua S., Heather M. Leslie, Melissa L. Britsch, and Caitlin M. Cleaver. "Evaluating Aquaculture as a Diversification Strategy for Maine's Commercial Fishing Sector in the Face of Change." Marine Policy 107, (2019): 103583.

The annotation should be indented one inch from the start of the citation, while the hanging indent should be indented half an inch.

Stoll, Joshua S., et al. "Evaluating Aquaculture as a Diversification Strategy for Maine's Commercial Fishing Sector in the Face of Change." Marine Policy , vol. 107, 2019, pp. 103583.

How to create this format

In microsoft word.

• Highlight your citations and annotations.
• Open Format  > Paragraph.  Alternatively, highlight the text, right-click and select Paragraph .
• Under Indentation , there is a drop down menu for Special options. This includes the Hanging First Line .

In Google Docs

• In the  menu, click on Format , then go down to Align & indent , then click on Indentation options .
• In the Indentation options menu, under  Special,  select  Hanging.
• Norton: Sample Annotated Bibliography (PDF) A sample bibliography from the Norton Field Guide to Writing.
• Purdue OWL Annotated Bibliography Samples Samples of citations and annotations in various style guides from Purdue OWL.
• Updated: Oct 15, 2021 12:41 PM
• URL: https://libguides.bates.edu/annotatedbib

Loading metrics

Open Access

Community Page

The Community Page is a forum for organizations and societies to highlight their efforts to enhance the dissemination and value of scientific knowledge.

See all article types »

Annotated primary scientific literature: A pedagogical tool for undergraduate courses

Affiliations Department of Biological Sciences, Florida International University, Miami, Florida, United States of America, STEM Transformation Institute, Florida International University, Miami, Florida, United States of America

* E-mail: [email protected]

• Matthew Kararo, 
• Melissa McCartney

Published: January 9, 2019

• https://doi.org/10.1371/journal.pbio.3000103
• Reader Comments

Annotated primary scientific literature is a teaching and learning resource that provides scaffolding for undergraduate students acculturating to the authentic scientific practice of obtaining and evaluating information through the medium of primary scientific literature. Utilizing annotated primary scientific literature as an integrated pedagogical tool could enable more widespread use of primary scientific literature in undergraduate science classrooms with minimal disruption to existing syllabi. Research is ongoing to determine an optimal implementation protocol, with these preliminary iterations presented here serving as a first look at how students respond to annotated primary scientific literature. The undergraduate biology student participants in our study did not, in general, have an abundance of experience reading primary scientific literature; however, they found the annotations useful, especially for vocabulary and graph interpretation. We present here an implementation protocol for using annotated primary literature in the classroom that minimizes the use of valuable classroom time and requires no additional pedagogical training for instructors.

Citation: Kararo M, McCartney M (2019) Annotated primary scientific literature: A pedagogical tool for undergraduate courses. PLoS Biol 17(1): e3000103. https://doi.org/10.1371/journal.pbio.3000103

Copyright: © 2019 Kararo, McCartney. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This research was supported through National Science Foundation Division of Undergraduate Education 1525596 (MM) and Florida International University College of Arts, Sciences & Education Postdoctoral Fellowship (MM and MK). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: CREATE, consider, read, elucidate the hypotheses, Analyze and interpret the data, and think of the next experiment; FIU, Florida International University; SitC, Science in the Classroom; STEM, science, technology, engineering, and math; TOSLS, Test of Scientific Literacy Skills

Provenance: Not commissioned; externally peer reviewed.

A major output of public research universities is primary scientific literature, in addition to educating students and conferring degrees. It is imperative for researchers and universities to increase the transparency and outreach of the primary research literature they produce. However, most primary scientific literature remains unknown and/or inaccessible to the public, because it is published in journals targeting academics in the same field and is often placed behind journal paywalls [ 1 ].

Public research universities also have a responsibility to produce scientifically literate graduates [ 2 , 3 ]. Many students graduate without an understanding of scientific practices and an acculturation to interpreting scientific communication, especially primary scientific literature [ 4 , 5 ]. One way to potentially improve scientific literacy overall and develop specific skills, such as interpreting scientific communication, is to incorporate primary scientific literature into the undergraduate curricula and provide pedagogical tools that may help bridge the divide between everyday language and the language used by experts [ 6 – 11 ].

The study of primary scientific literature as a pedagogical tool in undergraduate biology courses has led to innovative approaches. The most well-known of these may be the Consider, Read, Elucidate the hypotheses, Analyze and interpret the data, and Think of the next Experiment (CREATE) method, in which faculty redesign their existing courses around primary scientific literature in order to provide an intensive and comprehensive analysis of primary scientific literature for undergraduates [ 6 , 12 – 14 ]. Although this type of a semester-long innovative elective course provided student benefits, adding an entire course to a degree sequence may prove difficult and by definition, does not impact students that choose not to include them in an already credit-crunched plan of study. This credit-crunch is especially prevalent at institutions such as the one in this study, Florida International University (FIU), where any additional credit hours are charged at out-of-state tuition rates. Therefore, it would benefit biology education, and biology as a field of study, to develop innovative ways to utilize primary scientific literature as a pedagogical tool, ideally with a minimal impact to existing plans of study and time investment from course instructors.

A growing body of research shows that less-intensive interventions using primary scientific literature can be valuable and useful in science, technology, engineering, and math (STEM) education, with the greatest amount of research happening at the undergraduate level. Programs include journal clubs, data and figure exploration, and tutorials on how to read primary scientific literature [ 15 – 17 ]. Assessment tools used to evaluate these interventions are equally as diverse, ranging from rubrics to validated surveys [ 18 , 19 ].

Annotated primary scientific literature

Annotated primary scientific literature is designed to help readers interpret complex science by overlaying additional information on a scientific research article. Preserving the original text and its context is what makes annotated primary scientific research literature unique from other genres that modify or rewrite the original text. This preservation is the key difference between annotated primary scientific literature and adapted primary literature, an approach that takes portions of primary scientific literature and rewrites the original content to turn them into pedagogical tools [ 20 ]. Science in the Classroom (SitC; www.scienceintheclassroom.org ) is a highly developed and sophisticated example of annotated primary scientific literature that we decided has potential for classroom pedagogical use.

SitC, a collection of freely available annotated papers, aims to make primary scientific research literature more accessible to students and educators. The repository of annotated primary scientific literature articles is accessible to educators and searchable by keyword, classified by topics, and grouped in collections. The process of reading and deconstructing scientific literature in undergraduate courses has been shown to result in students potentially gaining an understanding of scientific practices, such as how scientists design their experiments and present their results, essentially allowing students to experience the logic behind drawing conclusions from a set of data [ 6 , 7 , 12 – 14 ].

Annotated primary scientific literature uses the original text of research articles along with a “Learning Lens” overlay, designed to provide students tools to use for interpretation. The “Learning Lens” is used to selectively highlight different parts of the text and is composed of seven headings: Glossary, Previous work, Author's experiments, Conclusions, News and policy links, Connect to learning standards, and References and notes, which are color-coded to match the corresponding text of the research article. For example, an annotated glossary term, when clicked on, will produce a pop-up box containing the definition of the word ( Fig 1 ). Annotations contained within the “Learning Lens” have been designed to be at the reading comprehension level of a first-year undergraduate student, and ongoing evaluation efforts have provided evidence that this goal is being met [ 21 ].

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pbio.3000103.g001

Annotated primary literature as a pedagogical tool

Annotations provide an educational scaffold that could help students become more comfortable with reading scientific papers. We propose annotated primary scientific literature as an example of a resource that can be incorporated into existing courses and provide scaffolding that may help undergraduate students develop skills necessary to read primary scientific literature while requiring a minimal time investment from instructors. Using annotated primary scientific literature as a pedagogical tool not only could potentially help universities develop scientifically literate graduates, but it may also broaden the impact of primary scientific research literature produced by faculty.

The previously mentioned pedagogical tools and curriculum transformations can require a substantial investment of time and effort from the university, faculty, and staff. Therefore, additional tools and opportunities should be considered in order to achieve a wider variety of complementary opportunities for teaching with authentic scientific practices and engaging students in reading primary scientific literature [ 22 ]. We hypothesize that the incorporation of annotated primary scientific literature in the classroom represents one of these opportunities.

In this pilot study, we had a goal of developing an implementation protocol that could incorporate annotated primary scientific literature into undergraduate courses with a minimal time investment for instructors and minimal disruption and alteration to existing courses and plans of study.

Implementation of annotated primary scientific literature

All data were collected in accordance with an approved FIU Institutional Review Board protocol #17–0398 and #17–0105. Our initial attempts to develop an implementation protocol for using annotated primary scientific literature as a pedagogical tool had the educational goal of introducing students to the “Learning Lens” annotations and observing how instructors and students used the tool. Initial attempts to incorporate annotated primary scientific literature focused on undergraduate biology courses at FIU, including General Biology II, Ecology, and Plant Life History. The implementation sessions were run iteratively during the same semester, ensuring that students did not overlap, and each class had only one implementation session. We describe two variations of our implementations here.

Students involved in the study self-reported their major, with 76% being biology majors. We did not collect any data on students’ prior knowledge of biology, but the majority of students in these classes are first- or second-year students.

We used the same annotated piece of primary scientific literature for all in-class activities described in this study: “Caffeine in floral nectar enhances a pollinator's memory of reward” ( https://tinyurl.com/k7m329g ). We chose an article that incorporated many different aspects of biology, including evolution, ecosystem interactions, basic botany, learning and memory, and animal behavior in a single study, making this paper applicable in a wide variety of undergraduate courses.

The objectives were to introduce undergraduate students to annotated primary scientific literature and collect baseline data on how students interacted with the annotations themselves. The first implementation involved a one-time intervention, connected to the student’s coursework, conducted by the researchers and began with an approximately 5-minute orientation to annotated primary scientific literature. This orientation included how to use the “Learning Lens” and a brief overview of the importance of primary scientific literature. Students were then given 20 minutes to read the selected piece of annotated primary scientific literature. At the 20-minute time point, a Qualtrics (online survey software; Provo, Utah and Seattle, Washington) link was provided, and if they were done reading, students could begin answering the feedback questionnaire. Students were given an additional 20 minutes to complete the questionnaire. Collecting and analyzing this first round of pilot data allowed for reflection on opportunities for iterative improvement.

In addition to the questionnaire data, feedback was collected through in-class activity observations conducted by the researchers. We kept detailed field notes indicating when students appeared on task, i.e., independently interacting with annotated primary scientific literature. We also noted when alternative tasks were observed, i.e., students checking email or social media, and when task completion appeared to have occurred. During the implementation, our in-class observations estimated an average time on task, i.e., interacting with annotated primary scientific literature, to be 10 minutes, because there was a noticeable increase in classroom noise after this time point. We confirmed this by using Adobe Analytics (Adobe, San Jose, California), which measures the time spent on a website by each user. We measured an average time spent on annotated primary scientific literature of 13 minutes. Due to limitations of Adobe Analytics, we are unable to collect individual data points and were limited to an aggregate average for the entire class. Note that the difference between the observed time spent on the activity and the digital measure can be explained by Adobe Analytics averaging all participants’ time spent on the article page.

The main student feedback was collected through a questionnaire containing both quantitative (content questions) and qualitative items (i.e., “what did you like about this activity?”). One of the key ideas we garnered from the qualitative data was that a one-time intervention was perceived by students as somewhat discordant when a connection between the article they read and the content they were covering at the time in their course was not made explicit by their course instructor ( Table 1 ).

https://doi.org/10.1371/journal.pbio.3000103.t001

When asked if the topic of the paper related to their course, students in this iteration gave feedback such as this activity was “only slightly relevant to the course,” and “no, we[‘re] studying plants” despite the article being explicitly about caffeine production by plants in order to attract pollinators. Additionally, we were uncertain that we had connected with the students as researchers in the same way as the instructor with whom the students had built a relationship.

Although some students may have not perceived a connection between the article content and their course content, in general, students found the annotations useful, especially regarding graphs and vocabulary interpretation. Examples of student responses can be seen in Table 2 .

https://doi.org/10.1371/journal.pbio.3000103.t002

For our second iteration, we decided to address the issues of students feeling discordant by having the course instructors introduce the article and annotated primary scientific literature activity themselves. Additionally, we asked instructors to explicitly connect the annotated paper to current course content. With both of these procedures in place, the average time students engaged with the annotated article, as measured by Adobe Analytics, increased to 19 minutes ( Fig 2 ).

https://doi.org/10.1371/journal.pbio.3000103.g002

This new implementation, in which the instructor introduced the piece of annotated primary scientific literature and annotated primary scientific literature activity, not only appeared to increase the time that students engaged with the material, but it also removed the manpower requirement for the researchers to be present in every classroom in order to describe and implement the activity. This could allow for a more widespread implementation of annotated primary scientific literature as a pedagogical tool. It was also apparent that students introduced to the activity by their course instructor were more readily able to recognize the connections between reading primary scientific research literature and their course content, which can be seen in student responses in Table 3 .

https://doi.org/10.1371/journal.pbio.3000103.t003

When asked if the topic of the paper related to their course, students in this iteration stated “This article related to 3 different courses I am taking this semester,” “yes it most certainly did,” “yes! We’re learning about pollination,” and that “…scientific papers on new experiments …are important.”

During the initial iterations of the implementation protocol, students read the annotated articles and completed an assessment during class time. However, a growing concern was feasibility of an in-class assignment due to the time requirement and allowing for instructor flexibility in scheduling. While observing a senior lecturer at FIU, who was not involved in this current study, and his existing implementation method of students reading primary scientific literature as homework and answering iClicker questions at the beginning of the following class, the researchers noticed an increased enthusiasm among the students during the class discussion. Supporting this observation, the history of research on the use of clickers in the classroom shows an increase in feelings of class involvement [ 23 ] and learning gains in students [ 24 ]. Because of the observations and support from instructors, the decision was made to adopt the homework protocol moving forward with future implementations. The homework protocol allows for more instructor freedom in selecting articles relevant to course content, reduces the class time required for implementation, and separates content questions from a pre–post attitude and motivation questionnaire. Using articles as homework also allows for instructors to utilize as many articles as they wish, but for this project moving forward, in future implementations, we will require a minimum of three articles over the course of a semester. We are currently piloting an implementation protocol using annotated primary scientific literature as a homework assignment and are excited to see how instructors and students use annotated primary scientific literature moving forward.

Advice to others

In the ongoing iterative development of an implementation protocol for annotated primary scientific literature, the most fruitful exercise has been reflection. This is great practice for any educator or educational researcher during the curriculum or pedagogical tool development process. Reflection on early classroom implementations helped us identify the opportunities for improvement in our subsequent protocol iterations and allowed us to make modifications based upon quantitative, qualitative, and observational data. One example of changes coming from reflection was noticing that during an implementation, students were opening the assessment without reading the article and using the “find” feature within the article to find answers to assessment questions. This led to preventing entry into the assessment until the time for reading had elapsed. Our subsequent classroom observations showed us that this forced students to interact with the article and be more thoughtful about their answers to the assessment, i.e., answers were not cut-and-pasted from the article text. We advise others to continue this practice of thoughtful reflection when using annotated primary scientific literature as a pedagogical tool. We also welcome any feedback or alternative uses of annotated primary scientific literature.

Future steps

The latest annotated primary scientific literature implementation protocol iteration is being pilot tested during fall 2018. Focusing more on robust evaluation now that implementation obstacles have been overcome will allow us to determine the effectiveness of annotated primary scientific literature as a pedagogical tool in undergraduate biology classrooms. Future studies are being designed to examine students’ scientific literacy before and after completing the annotated article activities using a previously validated scientific literacy instrument (Test of Scientific Literacy Skills [TOSLS]) [ 2 ]. Additionally, we aim to measure students’ subjective task values with regards to reading primary scientific research literature [ 25 – 28 ], as well as their primary scientific literature reading self-efficacy [ 29 – 32 ].

We hope to spread the word about annotated primary scientific literature and investigate its potential impacts on student learning and motivation as we further refine our implementation protocol and propagate beyond our department and institution.

Acknowledgments

We thank Beth Ruedi and Shelby Lake at AAAS, and Rebecca Vieyra for help editing this manuscript, our FIU colleagues Richard Brinn, Ligia Collado-Vides, Sat Gavassa, John Geiger, Camila Granados-Cifuentes, Zahra Hazari, Suzanne Koptur, and Sparkle Malone for providing us with class time, and all the participating students at FIU.

• View Article
• PubMed/NCBI
• Google Scholar
• 22. National Academy of Sciences. Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. Washington, DC: National Academies Press; 2012.
• 25. Eccles J, Adler TF, Futterman R, Goff SB, Kaczala CM, Meece JL, et al. Expectancies, values, and academic behaviors. In: Spence JT, editor. Achievement and achievement motives: Psychological and sociological approaches. San Francisco, CA: W.H. Freeman; 1983. pp. 75–146.
• 29. Bandura A. Social learning theory. Englewood Cliffs, NJ: Prentice Hall; 1977.
• 30. Bandura A. Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall; 1986.
• 32. Bandura A. Self-efficacy: The exercise of control. New York: Freeman; 1997.

John Gray Park '28 Library

• How Do I...
• Library Map
• Library Databases
• NoodleTools
• Research Guides
• Newspapers & Magazines
• Academic Resource Center (ARC)
• ARC Appointments
• Faculty & Staff
• Peer Tutoring

Academic Resource Center

• Introductions and Conclusions
• Body Paragraphs
• Paraphrasing
• Grammar and Punctuation
• Fiction and Poetry
• College Essays
• Résumé Basics
• Time Management
• Organization
• Note-taking
• Test and Exam Prep
• Subject-Specific Tips
• Assistive Tech & Audio
• Email Basics
• How to format In-Text Citations
• How to make a Reference Page
• How to Paraphrase
• How to integrate Quotations and Paraphrases
• How to format Footnotes and Endnotes
• How to make a Bibliography
• How to write an Annotated Bibliography
• How to cite Primary Sources
• How to cite specific sources
• How to make a Works Cited

Helpful Resources

• Annotated Bibliographies Examples

What is an Annotated Bibliography?

An annotated bibliography is an organized list of sources (like a reference list). It differs from a straightforward bibliography in that each reference is followed by a paragraph length annotation, usually 100–200 words in length.

Creating an Annotated Bibliography

What it is: Chicago-formatted source citations, accompanied by a summary and/or evaluation of each of the sources.

What it does (adapted from Purdue’s OWL site): An annotated bibliography helps you learn about your topic during the research stage, because when you have to write annotations for each source, you're forced to read each source more carefully. You begin to read more critically instead of just collecting information.

An annotated bibliography helps you see what research has already been done, so you have context for your own research, and it helps you develop or refine your thesis. By reading and responding to a variety of sources on a topic, you'll start to see what the issues are, what people are arguing about, and you'll then be able to develop your own point of view.

An annotated bibliography is also invaluable to other historians and historical researchers doing work in your field.

Details on each of the steps , as detailed on the Purdue OWL site:

• Summarize : What are the main arguments? What is the point of this book or article? What topics are covered? If someone asked what this article/book is about, what would you say?
• Assess : After summarizing a source, to evaluate it. Is it a useful source? How does it compare with other sources in your bibliography? Is the information reliable? Is this source biased or objective? What is the goal of this source?
• Reflect : Once you've summarized and assessed a source, you need to ask how it fits into your research. Was this source helpful to you? How does it help you shape your argument? How can you use this source in your research project? Has it changed how you think about your topic?

Suggestions from the History Department

The History Department at Kent suggest that you include the following information in your annotated bibliography: 1. An explanation of the main purpose of the source 2. A short summary of key findings or arguments of the source 3. The academic/intellectual credentials of the source: Does it appear in a peer-reviewed journal? Is the author someone who has expertise in the area? 4. Any shortcomings or biases you notice 5. The value of this work as a contribution to the topic you’re exploring

• << Previous: How to make a Bibliography
• Next: How to cite Primary Sources >>
• Last Updated: Mar 28, 2024 3:53 PM
• URL: https://libguides.kent-school.edu/ARC

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

VII. Researched Writing

7.6 Writing an Annotated Bibliography

Emilie Zickel; Melanie Gagich; and Terri Pantuso

As you are gathering sources in your research, you will want to keep track of which information comes from what source. While other strategies have been discussed such as note taking, some researchers use an annotated bibliography for long term reference purposes. As the name implies, an annotated bibliography is the bibliographical reference of a given source along with key information from that source that you may use for future reference. As assignment parameters will vary by instructor, generally speaking the annotations are 150-200 words in length per source and do not include quoted material. The purpose of the annotations is to summarize the material within the context of your thesis statement.

Annotated Bibliographies follow a common structure and format. Below is an explanation of the elements and format of an annotated bibliography.

Components of an Annotated Bibliography

An annotation often offers a summary of a source that you intend to use for a research project as well as some assessment of the source’s relevance to your project or quality and credibility. There are two key components for each source: the citation and the annotation.

The Annotated Bibliography Samples page [1]  on the Purdue OWL offers examples of general formatting guidelines for both an MLA and an APA Annotated Bibliography.

You will provide the full bibliographic reference for the source: author, title, source title, and other required information depending on the type of source. This will be formatted just as it would be in a typical Works Cited for an MLA paper or a References page for an APA paper.

Tone and Style

Some elements can vary depending on the style you are using (e.g., APA or MLA). Be sure to review your style guide along with your assignment sheet. Generally speaking, use the following as a guide:

• Use signal phrases to refer to the author(s).
• Always maintain a neutral tone and use the third-person point of view and correct tense according to style guide (present tense for MLA, past tense for APA) (i.e., Tompkins asserts… ).
• Keep the focus of the summary on the text, not on what you think of it, and try to put as most of the summary as you can in your own words. If you must use exact phrases from the source that you are summarizing, you must quote and cite them.
• Annotations should not be a replication of the abstract provided by the source.

What to Include in Annotations

• After the bibliographical information, begin to discuss the source. Begin with a general summary of the source. Describe the key sections of the text and their corresponding main points. Try to avoid focusing on details; a summary covers the essential points and typically does not include quoted material.
• Evaluate the source’s credibility or relevance. Is the author an expert on the topic? How do you know? Is the source peer-reviewed or otherwise credible in nature? How do you know? What makes this source a good one to use?
• Discuss how you plan to integrate the source in your paper. Do you need to point out similarities or differences with other sources in the annotated bibliography? How does it support (or refute) your intended thesis?

Review your Annotated Bibliography assignment sheet for additional content requirements . Instructors often require more than a simple summary of each source, and specific requirements may vary. Any (or all) of these aspects may be required in an annotated bibliography, depending on how or if your instructor has designed this assignment as part of a larger research project.

This section contains material from:

Gagich, Melanie, and Emilie Zickel. “Keeping Track of Your Sources and Writing an Annotated Bibliography.” In A Guide to Rhetoric, Genre, and Success in First-Year Writing . Cleveland: MSL Academic Endeavors. Accessed July 2019. https://pressbooks.ulib.csuohio.edu/csu-fyw-rhetoric/chapter/annotated-bibliography/ . Licensed under a Creative Commons Attribution 4.0 International License .

OER credited in the text above includes:

Jeffrey, Robin. About Writing: A Guide . Portland, OR: Open Oregon Educational Resources. Accessed December 18, 2020. https://openoregon.pressbooks.pub/aboutwriting/ . Licensed under a Creative Commons Attribution 4.0 International License .

• "Annotated Bibliography Samples," Purdue Online Writing Lab, accessed December 20, 2021, https://owl.purdue.edu/owl/general_writing/common_writing_assignments/annotated_bibliographies/annotated_bibliography_samples.html . ↵

A statement, usually one sentence, that summarizes an argument that will later be explained, expanded upon, and developed in a longer essay or research paper. In undergraduate writing, a thesis statement is often found in the introductory paragraph of an essay. The plural of thesis is theses .

7.6 Writing an Annotated Bibliography Copyright © 2022 by Emilie Zickel; Melanie Gagich; and Terri Pantuso is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Creating an Annotated Bibliography

Organize your research efforts and extend your thinking on a research topic by creating an annotated bibliography.

An annotated bibliography is a list of reference sources and critical summaries/evaluations of the citations. Typically, researchers will:

• Provide the citation information for each source following the rules of a particular bibliography style (e.g., MLA Style, APA Style, Chicago Style). Logically, you want to use the citation style in your bibliography that you will use in your research report. Examples of citation sources include books, articles, Internet sites, newspapers, and audiovisual materials.
• List each reference source in alphabetical order. Occasionally researchers will introduce themes to their annotated bibliographies, essentially introducing headings for each theme and then organizing citations and summaries according to the themes that are emerging.
• Provide a brief (100- to 200-word) descriptive and evaluative summary of each source. Researchers may address the relevance of the reference source, summarize the unique findings or arguments, include judgments regarding the quality of the source, and critique the methods employed by the source to generate knowledge.

https://player.vimeo.com/video/13016416

What is an Annotated Bibliography? from Kimbel Library on Vimeo .

Sample Annotated Bibliography Format

Begin the Annotated Bibliography after the body of the paper and at the top of a new page. The title should be centered and presented in plain type.

• Bibliographic Reference: [Author: Last Name, First Name.] [Name of book, article, document] [Publisher Information] [Date of Publication] [Page Numbers, if appropriate]
• 100- to 200-Word Summary:
• Audience, purpose, voice, tone, persona, media. Conduct a rhetorical analysis, evaluating the source’s intended audience, purpose, scope, and so on.
• Relevance? Importance? Is the source timely, controversial, and/or focused on matters related to your research project? Are the results significant? Is the argument persuasive?
• Authority of the researchers. What universities or corporations support the research? Is the researcher or research team frequently cited by others? Is the source published by a credible publishing company? Is it peer-reviewed?
• Significant findings and arguments.
• Research methods: Are the researchers employing appropriate research methodologies? Are the methodologies fairly standard, i.e., have the researcher’s methods been used by past researchers?
• Quality of the research or article. Is it thorough?

Who Reads Annotated Bibliographies?

There tend to be three major audiences for annotated bibliographies: the authors of the annotations, instructors, and other researchers.

Self as Audience

Many people find it useful to craft an annotated bibliography while researching topics. Writing brief summaries of the research you consult, whether you’re researching newspapers, journals, books, or videos, helps you to remember these sources over time. More than that, by writing critical evaluations of the research you consult, you will identify common themes and methods. You will find what research is commonly cited on a topic, what methods are employed, and what a community of scholars believes needs additional inquiry.

Instructors as Audience

In college and university contexts, instructors often require students to craft annotated bibliographies as a preliminary step to writing a formal research paper. Asking students to construct an annotated bibliography enables instructors to ensure that students understand the bibliography style for citing references. It helps ensure the student has consulted a variety of timely and reputable sources.

Other Researchers

Occasionally, professionals will actually publish their annotated bibliographies. This happens in research fields where a lot of information is being published. Professional researchers often begin their survey of research by finding annotated bibliographies on a topic that interests them.

Tips for Constructing Annotated Bibliographies

Compiling an annotated bibliography enables you to carefully keep track of the sources you use while conducting research. It can be quite annoying and frustrating to be forced, after completing your research and writing, to return to the library or Internet to retrieve the information you need to properly cite these works. In the case of books, this frustration intensifies when you discover that other people have subsequently checked out your materials. Regarding Internet research, there isn’t necessarily any guarantee that the site you consulted for your research/writing will still be accessible (or even exist at all) when you need to consult it for bibliographic information for your in-text citations/Works Cited page, etc.

When asked to write an annotated bibliography, check to ensure that you understand the form of documentation that you will need to follow. Then, be sure that you record on the copy of the photocopied material all of the bibliographical facts that you will need to cite in the bibliography—such as the author’s name, the journal name and volume number, or the book title and publisher; the city and date of publication (if available) and the page numbers. Fortunately, in most scholarly journals and some commercial magazines, this information is already printed on the title page of each essay. If it isn’t, you had better record it now; otherwise, you may need to retrace your steps and, as explained above, it would be unfortunate if you were unable to locate your source/s, especially if a deadline is imminent.

(Consider using software to construct your annotated bibliography. Experienced researchers sometimes use bibliography software to compile their annotated bibliography.)

Review this sample annotated bibliography: Cinderella Romance Novels

Adapted from Annotated Bibliography  by Joseph M. Moxley ( Writing Commons ) licensed under CC BY-NC-ND 4.0.

Writing as Critical Inquiry Copyright © by Keri Sanburn Behre, Ph.D. and Kate Comer, Ph.D. is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Citing sources
• What Is an Annotated Bibliography? | Examples & Format

What Is an Annotated Bibliography? | Examples & Format

Published on March 9, 2021 by Jack Caulfield . Revised on August 23, 2022.

An annotated bibliography is a list of source references that includes a short descriptive text (an annotation) for each source. It may be assigned as part of the research process for a paper , or as an individual assignment to gather and read relevant sources on a topic.

Scribbr’s free Citation Generator allows you to easily create and manage your annotated bibliography in APA or MLA style. To generate a perfectly formatted annotated bibliography, select the source type, fill out the relevant fields, and add your annotation.

An example of an annotated source is shown below:

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

Annotated bibliography format: apa, mla, chicago, how to write an annotated bibliography, descriptive annotation example, evaluative annotation example, reflective annotation example, finding sources for your annotated bibliography, frequently asked questions about annotated bibliographies.

Make sure your annotated bibliography is formatted according to the guidelines of the style guide you’re working with. Three common styles are covered below:

In APA Style , both the reference entry and the annotation should be double-spaced and left-aligned.

The reference entry itself should have a hanging indent . The annotation follows on the next line, and the whole annotation should be indented to match the hanging indent. The first line of any additional paragraphs should be indented an additional time.

In an MLA style annotated bibliography , the Works Cited entry and the annotation are both double-spaced and left-aligned.

The Works Cited entry has a hanging indent. The annotation itself is indented 1 inch (twice as far as the hanging indent). If there are two or more paragraphs in the annotation, the first line of each paragraph is indented an additional half-inch, but not if there is only one paragraph.

Chicago style

In a  Chicago style annotated bibliography , the bibliography entry itself should be single-spaced and feature a hanging indent.

The annotation should be indented, double-spaced, and left-aligned. The first line of any additional paragraphs should be indented an additional time.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

For each source, start by writing (or generating ) a full reference entry that gives the author, title, date, and other information. The annotated bibliography format varies based on the citation style you’re using.

The annotations themselves are usually between 50 and 200 words in length, typically formatted as a single paragraph. This can vary depending on the word count of the assignment, the relative length and importance of different sources, and the number of sources you include.

Consider the instructions you’ve been given or consult your instructor to determine what kind of annotations they’re looking for:

• Descriptive annotations : When the assignment is just about gathering and summarizing information, focus on the key arguments and methods of each source.
• Evaluative annotations : When the assignment is about evaluating the sources , you should also assess the validity and effectiveness of these arguments and methods.
• Reflective annotations : When the assignment is part of a larger research process, you need to consider the relevance and usefulness of the sources to your own research.

These specific terms won’t necessarily be used. The important thing is to understand the purpose of your assignment and pick the approach that matches it best. Interactive examples of the different styles of annotation are shown below.

A descriptive annotation summarizes the approach and arguments of a source in an objective way, without attempting to assess their validity.

In this way, it resembles an abstract , but you should never just copy text from a source’s abstract, as this would be considered plagiarism . You’ll naturally cover similar ground, but you should also consider whether the abstract omits any important points from the full text.

The interactive example shown below describes an article about the relationship between business regulations and CO 2 emissions.

Rieger, A. (2019). Doing business and increasing emissions? An exploratory analysis of the impact of business regulation on CO 2 emissions. Human Ecology Review , 25 (1), 69–86. https://www.jstor.org/stable/26964340

An evaluative annotation also describes the content of a source, but it goes on to evaluate elements like the validity of the source’s arguments and the appropriateness of its methods .

For example, the following annotation describes, and evaluates the effectiveness of, a book about the history of Western philosophy.

Kenny, A. (2010). A new history of Western philosophy: In four parts . Oxford University Press.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

A reflective annotation is similar to an evaluative one, but it focuses on the source’s usefulness or relevance to your own research.

Reflective annotations are often required when the point is to gather sources for a future research project, or to assess how they were used in a project you already completed.

The annotation below assesses the usefulness of a particular article for the author’s own research in the field of media studies.

Manovich, Lev. (2009). The practice of everyday (media) life: From mass consumption to mass cultural production? Critical Inquiry , 35 (2), 319–331. https://www.jstor.org/stable/10.1086/596645

Manovich’s article assesses the shift from a consumption-based media culture (in which media content is produced by a small number of professionals and consumed by a mass audience) to a production-based media culture (in which this mass audience is just as active in producing content as in consuming it). He is skeptical of some of the claims made about this cultural shift; specifically, he argues that the shift towards user-made content must be regarded as more reliant upon commercial media production than it is typically acknowledged to be. However, he regards web 2.0 as an exciting ongoing development for art and media production, citing its innovation and unpredictability.

The article is outdated in certain ways (it dates from 2009, before the launch of Instagram, to give just one example). Nevertheless, its critical engagement with the possibilities opened up for media production by the growth of social media is valuable in a general sense, and its conceptualization of these changes frequently applies just as well to more current social media platforms as it does to Myspace. Conceptually, I intend to draw on this article in my own analysis of the social dynamics of Twitter and Instagram.

Before you can write your annotations, you’ll need to find sources . If the annotated bibliography is part of the research process for a paper, your sources will be those you consult and cite as you prepare the paper. Otherwise, your assignment and your choice of topic will guide you in what kind of sources to look for.

Make sure that you’ve clearly defined your topic , and then consider what keywords are relevant to it, including variants of the terms. Use these keywords to search databases (e.g., Google Scholar ), using Boolean operators to refine your search.

Sources can include journal articles, books, and other source types , depending on the scope of the assignment. Read the abstracts or blurbs of the sources you find to see whether they’re relevant, and try exploring their bibliographies to discover more. If a particular source keeps showing up, it’s probably important.

Once you’ve selected an appropriate range of sources, read through them, taking notes that you can use to build up your annotations. You may even prefer to write your annotations as you go, while each source is fresh in your mind.

An annotated bibliography is an assignment where you collect sources on a specific topic and write an annotation for each source. An annotation is a short text that describes and sometimes evaluates the source.

Any credible sources on your topic can be included in an annotated bibliography . The exact sources you cover will vary depending on the assignment, but you should usually focus on collecting journal articles and scholarly books . When in doubt, utilize the CRAAP test !

Each annotation in an annotated bibliography is usually between 50 and 200 words long. Longer annotations may be divided into paragraphs .

The content of the annotation varies according to your assignment. An annotation can be descriptive, meaning it just describes the source objectively; evaluative, meaning it assesses its usefulness; or reflective, meaning it explains how the source will be used in your own research .

A source annotation in an annotated bibliography fulfills a similar purpose to an abstract : they’re both intended to summarize the approach and key points of a source.

However, an annotation may also evaluate the source , discussing the validity and effectiveness of its arguments. Even if your annotation is purely descriptive , you may have a different perspective on the source from the author and highlight different key points.

You should never just copy text from the abstract for your annotation, as doing so constitutes plagiarism .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Caulfield, J. (2022, August 23). What Is an Annotated Bibliography? | Examples & Format. Scribbr. Retrieved April 13, 2024, from https://www.scribbr.com/citing-sources/annotated-bibliography/

Is this article helpful?

Jack Caulfield

Other students also liked, evaluating sources | methods & examples, how to find sources | scholarly articles, books, etc., hanging indent | word & google docs instructions, scribbr apa citation checker.

An innovative new tool that checks your APA citations with AI software. Say goodbye to inaccurate citations!

You likely read, and perhaps also write, annotation every day. Annotation influences how we interact with texts across everyday contexts. Annotation provides information, shares commentary, sparks conversation, expresses power, and aids learning. This is why annotation matters.

1: Introduction

And if you have managed to graduate from college without ever having written ‘Man vs. Nature’ in a margin, perhaps now is the time to take one step forward. —Billy Collins, Marginalia

All the News that’s Fit to Annotate

On April 18, 2019, a redacted version of the Report On The Investigation Into Russian Interference In The 2016 Presidential Election , by Special Counsel Robert Mueller, was released to the public by the U.S. Department of Justice. Annotation shaped how the report was shared and interpreted.

Approximately a tenth of the entire report was redacted, or blacked out . Redaction is a type of annotation. To provide a rationale for extensive redaction, the Department of Justice also annotated each redaction according to one of four color-coded categories. According to an analysis by The New York Times , about 70 percent of the line-by-line redactions concerned ongoing investigations (white annotation), almost 20 percent related to grand jury material (red), and the remaining redactions concerned either classified information, such as investigative techniques (yellow), or personal privacy (green).

Figure 1: Mueller Report

In addition to the report’s redaction-as-annotation and annotation-of-redactions, media reporting of the report’s conclusions about possible coordination (popularly referred to as “collusion”) and obstruction of justice did more than offer a summary of key findings. Journalists annotated the report to provide their readers with information, analysis, and commentary. The Washington Post published a page-by-page analysis titled “The Mueller report, annotated,” NPR offered “Highlights from the Mueller Report, Annotated,” and Politico reporters contributed “An annotated guide to the redacted Mueller report,” among other examples. Annotation across these publications varied in detail, scope, visual style, and interactive features.

Over the past few years, leading media organizations have embraced an annotated approach to journalism. Annotation by reporters frequently accompanies political speeches, debate and interview transcripts, the release of legal documents like the Mueller report, and analysis of news conferences. Why the trend? On the one hand, annotation is easy to feature because it’s similar to established journalism practices, like quoting experts, hyperlinking to supporting resources, and presenting media content. On the other hand, annotation goes a step further by illustrating both granular detail and germane context. Annotation allows journalists to comment more transparently, to more informally share behind-the-scenes or insider perspectives. Annotation is also proving to be an effective fact-checking strategy. 1

Journalistic interest in annotation is not confined to politics. Following Ta-Nehisi Coates’ successful turn authoring Marvel’s Black Panther comic, The New York Times published “Captain America No. 1, by Ta-Nehisi Coates, Annotated.” This online article featured exclusive previews of the comic, insider commentary by Mr. Coates as head writer, and a few spoilers for good measure. 2 In 2017, the Times also featured the author Margaret Atwood annotating key episodes and scenes from the TV adaption of her celebrated novel The Handmaid’s Tale . Yet in both form and function, how different is annotation in The New York Times Magazine’s “Talk” interviews, a featured introduced in 2019, from Listrius’ annotation of Erasmus’ Moriae encomium , “a standard appendage to the work” since the 1515 Basel edition? 3

That which is fit to print - be it the news, or social commentary, or religious doctrine - has for centuries been fit for annotation, too. While it’s not newsworthy to observe that journalism is changing rapidly in our digital era, it is distinctive to note how annotation traditions and conventions are reinvigorating journalism as more connected, more interactive, and more relevant.

The ways in which, and the reasons why, journalists annotate represents but one small set of practices within the broader genre of annotation. Marginalia thrived in England during the sixteenth century, as studies of book culture during the rule of Elizabeth I and James I demonstrate. 4 Annotated books were routinely exchanged among scholars and friends as “social activity” throughout the Victorian era. 5 Some of the most significant commentary about the Talmud, first written in the eleventh century, has been featured prominently as annotation in print editions since the early 1500s. Today, scientists’ annotation of the human genome and proteome for large-scale biomedical research relies upon techniques that are both similar to and also very different from linguists and historians who have translated, annotated, and digitally archived Babylonian and Assyrian clay tablets. 6 From the annotatio of Roman imperial law to the medieval gloss , annotation nowadays helps people to write computer code, evaluate chess games, and interpret rap lyrics.

Perhaps annotation has already appeared in this book, too. Annotation is a form of self-expression, a way to document and curate new knowledge, and is a powerful means of civic engagement and political agency.

Annotation provides information, making knowledge more accessible. Annotation shares commentary, making both expert opinion and everyday perspective more transparent. Annotation sparks conversation, making our dialogue - about art, religion, culture, politics, and research - more interactive. Annotation expresses power, making civic life more robust and participatory. And annotation aids learning, augmenting our intellect, cognition, and collaboration. This is why annotation matters.

You likely read, and perhaps also write, annotation every day. Whether handwritten or digital, this book will help you to define, identify, and author annotation. More importantly, this book will discuss five essential purposes of annotation that contribute to cultural, professional, civic, and educational activities. Annotation is written within the warp and weft of our texts, patterning the fabric of daily life. This book will help you to understand how that happens and why that matters. We’ll start by introducing a key idea that appears throughout this book - annotation is an everyday activity.

An Everyday Activity

The Scottish author Kenneth Grahame, best known for his novel The Wind in the Willows , observed in an 1892 essay, “The child’s scribbling on the margin of his school-books is really worth more to him than all he gets out of them.” 7 Imagine your high school literature course. Or picture those classic scenes from Dead Poets Society . Maybe your English teacher assigned Toni Morrison’s Beloved . While reading, perhaps you highlighted key passages, noted plot devices, and commented on structure and dialogue. How else to comprehend two chapters, back-to-back, famous and famously unconventional, both of which begin “I am Beloved and she is mine.”? You might have shown these annotations to your teacher - “you see, I did the reading!” - or used them as references when writing a final paper. Or do you recall jotting down formulae for molecules and compounds in the margins of your chemistry textbook? Writing to students in 1940, the American philosopher and educator Mortimer Adler declared, “Marking up a book is not an act of mutilation but of love.” 8

What is everyday about annotation in school? Annotation may have been an expected or required academic practice. You’ve likely annotated as a part of many different courses and inside many different texts. Maybe annotation in school helped you to develop an idiosyncratic notation system that you still use today. Or maybe you were taught a more formal convention. Annotation happens every day in school and is an everyday activity for students, for “at every stage, students working with books have used the tool of annotation.” 9 And graduation from high school or college probably didn’t get you off the hook; as we’ll discuss, annotation is an everyday activity for many professionals, like journalists, programmers, scientists, and scholars.

But we’re not all scientists or scholars who annotate as a part of our job. Let’s bring this idea of everyday annotation a bit closer to home. Some of us may fondly recall measuring the height of a child against a doorframe, making a small pencil mark, and then writing down your child’s name and the date. Or maybe you were that child and this family ritual also helped you to practice writing your name while growing up. As you, or your child, aged, so too did these measurement marks travel upwards, edging ever closer to the upper frame. This, too, is an act of everyday annotation, stretched over time and etched with love. While this type of annotated measurement might not have happened every single day, it did record an aggregate of day-to-day changes. And the annotation - the marks, the names, the dates - served as a visual and daily reminder of growth added to the text and texture of life.

Annotation is an everyday activity whether it makes journalism more viable or schooling more valuable. Annotation is an everyday activity whether it contributes to scientific discovery or catalogues a child’s growth. Annotation is an everyday activity because different types of notes, whether political commentary or a child’s name, are added by many different types of people - journalists, programmers, parents and children - to a variety of texts, like transcripts, and code, and even a door jamb. In these cases, and in many others discussed throughout this book, it isn’t our prerogative to suggest which examples of everyday annotation are more or less remarkable (and yes, pun intended). Rather, it is our job to highlight how these various types of annotation share five common purposes - to provide information, to share commentary, to spark conversation, to express power, and to aid learning. Annotation makes knowledge more accessible, perspective more transparent, discourse more interactive, authority more contested and complex, and education more vibrant.

Defining Annotation

As print culture developed in Renaissance Europe and books become more widely available to a reading public, so, too, did “marginal material” flourish and serve various purposes. During the 1500s, annotation added details and examples to books, and provided references, corrected or objected to an author’s statement, emphasized importance, evaluated arguments, provided justification and translation, and even parodied the text, among other functions. 10 It wasn’t until 1819 that Samuel Taylor Coleridge first used the term “marginalia,” from the Latin marginalis (or “in the margin”), when, as a literary critic, he wrote about another author’s work for Blackwood’s Magazine . 11 Were you to ask more contemporary scholars about annotation, they might suggest annotation facilitates reading and later writing, the ability to “eavesdrop” on other readers, that annotation provides feedback and opportunities for collaboration, and “call[s] attention to topics and important passages.” 12 And when Sam Anderson wrote his 2011 essay What I Really Want Is Someone Rolling Around in the Text , he recalled experiencing annotation as additive, useful, social, a means to collaborate with a text, and as “meta-conversation running in the margins.” 13 Then again, a computer scientist will tell you that annotation means labeling data - images, text, or audio - for the purposes of identifying, categorizing, and training machine learning systems.

So how to define annotation? Merriam-Webster defines annotation as “a note added by way of comment or explanation.” And the Oxford English Dictionary echoes with: “A note by way of explanation or comment added to a text or diagram.” In this book, we’ll take an even simpler approach and define annotation as:

A note added to a text.

We’ve settled upon this definition because it gives us flexibility to explore the broad genre of annotation, both handwritten and digital, textual and visual, from different periods of time, and that serve different cultural and civic purposes. You might notice that unlike the standard-bearing dictionaries, our definition does not include the terms “comment” and “explanation,” denoting two annotation purposes. And that’s because, in our assessment, annotation serves five equally important, and sometimes overlapping purposes: Providing information, sharing commentary, expressing power, sparking conversation, and aiding learning. Our definition of annotation will allow us to explore wide-ranging issues of authorship, intent, and expression.

Most immediately, our definition of annotation requires us to ask - and answer - three questions. First, what is a note? Second, what does it mean to add? And third, what is a text?

Figure 2: A note added to text

What is a Note?

Notes , according to literary scholar Andrew Piper, are “records of the quotidian.” 14 In our earlier examples of everyday annotation, we featured various types of notes. A note can be a word, phrase, sentence, or even extended prose written in a textbook or cookbook. Some scholars have argued that signs and nonverbal codes are not notes, that notes must be discursive and responsive. 15 However, to embrace the full repertoire of annotation we also suggest that a note can be a symbol, like a question mark, exclamation point, or an asterisk. Copyediting marks are notes, and so too tick or tally marks like those added to the Lee Resolution, or The Resolution for Independence, passed by the Second Continental Congress on July 2, 1776. The Lee Resolution features 12 marks tallying the “united colonies” that voted for American independence.

Figure 3: Lee Resolution

Notes help mediate the relationship between reading and writing. For Piper, notes are “silent embers,” indicating “where the often mind-numbing, repetitive mundaneness of our daily lives bump into the high-flying acrobatics of human intellect.” 16 In 1947, a high-flying moth bumped into Harvard’s Mark II Aiken Relay Calculator, was removed by computer operator William Burke, and then taped to the computer log - popularizing the existing term “debugging.”

Figure 4: debugging

To discuss what counts as a note and how notes work, we need to introduce a new idea - multimodality .

With roots in rhetoric and semiotics, the concept of multimodality has long influenced how we understand and participate in acts of communication. A full theoretical review of multimodality is beyond the scope of this book. For our purposes, it will be sufficient to start by explaining how different forms of media are characterized by different types of modes . Earlier, we mentioned Toni Morrison’s Beloved . Beloved is a book, and books are a category of media. In the case of Beloved , this media artifact uses a textual mode of communication between Toni Morrison, the author, and you, the reader. Of course, media in the same category can feature different modes. Children’s picture books and your coffee table book featuring landscape photography exemplify a visual modality, whereas books written in braille demonstrate the importance of a tactile mode.

To further understand the relationship between media and mode, let’s briefly revisit two examples that we’ve already mentioned. Margaret Atwood’s novel The Handmaid’s Tale has been narrated by the actress Claire Danes to create an audiobook, another type of media, which allows for communication with listeners via an aural modality. Ta-Nehisi Coates, the lead writer of Black Panther , worked alongside a team of illustrators and colorists who, together, created a series of comic books (yet another type of media) that communicated with readers through both textual and visual modalities. Furthermore, both The Handmaid’s Tale and Black Panther demonstrate how a specific text can be adapted from one type of media to another; the former from a book into a television show, the latter from comic books into a feature film. And when that happened, the primary modality of both texts also changed to favor a visual mode.

Just like different types of media communicate through various modes, so, too, do notes communicate through multiple modes. As records of the quotidian, words, symbols, images, and even animated GIFs may all be notes that communicate through a textual mode, or a visual mode, or an aural model. When we discuss what counts as a note in this book, and when we describe how notes function in relation to a text, we do so with multimodality in mind. Annotating Beloved in literature class likely meant adding textual notes to a book that also communicated through a textual mode. Alternatively, adding textual notes to an image, or vice versa when adding a doodle or an image atop a text, suggests that the concept of multimodality - or what Piper describes as the “multidimensional” qualities of notes - will help us to examine notes and their relationship to annotation.

Consider all the various media you interact with every day - books, newspapers, magazines, and comics (including these texts’ digital versions), as well as movies, video games, podcasts, and social media. Your daily media diet likely features various modalities: Textual, for the media you read; visual, for media you watch; tactile, for media you touch, like touch screens at a restaurant or museum; and aural, for your favorite podcasts. Have you ever noticed annotation associated with this media? If so, what do these notes say and through what modalities do they speak? If, as Piper suggests, notes are “technologies of oversight,” then as multiple notes are added to a text over time, it’s likely that a group of notes will come to demonstrate the multimodality and multidimensionality of annotation. Importantly, notes are not only written; notes, in our view, may be more than words.

What Does it Mean to Add?

We now understand what notes are, how notes function, and that notes are multimodal. Let’s survey a few records of the quotidian in a variety of everyday scenarios. You’re reading a favorite book, jotting words and symbols in the margins; this is your “private exchange” with the author, a means of “talking back to” the text. 17 An educator reads a student’s essay and returns it covered in red-inked copyediting marks, or does the same using a digital annotation application. You and a colleague use collaborative word processing software to write a report for colleagues in your organization, a version of augmented intellectual work anticipated by, among others, the inventor and Internet pioneer Douglas Engelbart in the early 1960s. 18 In these instances, we recognize the importance of idiosyncratic meaning-making, how a student’s first draft is improved through expert feedback, and how a report is co-authored thanks to professional processes. The addition of notes while reading a book, revising a school assignment, or authoring a report communicates an important message: To add a note is to act with agency.

The word agency is traced back to the Latin verb agō meaning to act, to do, or to make. In our discussion of annotation we’re not referring to the type of organizational agency that makes stuff, like an advertising firm or a government bureaucracy. Rather, we’re interested in how an individual or a group acts and makes stuff, as when copy editing an essay or collaboratively writing research. And when doing so, it’s likely that people will annotate, that they will add some type of note to a text, that they will author a “responsive kind of writing permanently anchored to preexisting written words.” 19 Our expansive view of annotation suggests notes are not only responsive writing (notes might be GIFs), and also that more than written words are annotated (buildings can be annotated, too). What really matters, in this discussion of agency, is the fact that when a note is added - when people exercise agency in different contexts, under a variety of circumstances, and for many purposes - an annotation is permanently anchored to a text.

What does it mean to add? Adding is to act with agency. When we discuss annotation in this book, agency means that someone has permanently anchored a note to a text.

What is a Text?

This book you’re reading - whether printed on paper and bound together, or as a digital epub - is a text . We’ve mentioned a lot of texts so far: Beloved , The Handmaid’s Tale , Black Panther . We began our introduction by referencing multiple news articles written by journalists and published by media organizations; those articles are also texts. If you’ve made it this far into a book about annotation, then you’re likely familiar with a diversity of material and digital texts. In forthcoming chapters we’ll discuss medieval manuscripts, religious scripture, works of art, hashtags, computer code, legislation, and all manner of images - they’re all texts, too.

What makes something a text? First, a text has an author. Someone, or maybe a group of someones, authors a text by writing, composing, speaking, drawing, or through photography. Second, a text is defined by its content. A text conveys a “main message.” Some would say that texts have a “body,” and that texts are distinguished by a given style or subject. As we’ve discussed, texts are also defined by different forms of media; one text is a comic book, another a film. And texts communicate messages through different modes including, as with comic books, multiple modalities at the same time. While different types of annotation like marginalia, glosses, and rubrication have historically appeared as notes within books (as we’ll discuss in Chapter 2), the breadth of what we count as a text is a reminder that annotation may be permanently anchored to much more than books.

The features of a text should resonate as familiar given something you’ve likely authored today, if not quite recently - a text message. You’re the author of the text message. To author your text message, perhaps you wrote words, or used an emoji, or included a photograph or GIF. The content, or the body, of your text quite literally conveys a message: “Here’s the book I recommended,” or “Let’s go see a movie this weekend.” And depending upon whether you composed with words, or with emojis, or with images, your text message communicated through a textual mode, via a visual mode, or maybe with multiple modalities.

Why discuss the qualities of texts and this example of text messages? For two reasons. First, the defining features of a text - an author, message, structure, and style - are similar to the features of notes - which are quotidian, include both words and signs, and are multimodal. Both texts and notes can take the form of different media and communicate through different modalities. The second reason is this: the interplay of a note added to a text can best be understood by introducing the concept of intertextuality .

Intertextuality

Intertextuality is essential to further articulating annotation as a note added to a text.

Simply put, intertextuality describes the relationship between texts. A relationship between two (or more) texts might be established for the purposes of comparison, or alliteration, or interpretation, or as a means of fact-checking or critique. In some cases, an intertextual relationship may be explicit. This book, so far, has referenced Beloved and Black Panther on multiple occasions to help illustrate key ideas. In doing so, we’ve begun building a series of intertextual dialogues among these texts. In other instances, an intertextual relationship may be implicit. For example, there is both implicit and interpretive intertextuality among James Joyce’s Ulysses and Homer’s Odyssey , as well as between the Odyssey and the Coen brothers’ film O Brother, Where Art Thou?

Of course, the idea of intertextuality is a bit more complex than just explicit or implicit references among texts. The Russian philosopher and literary theorist Mikhail Bakhtin, writing throughout the 1930s and 40s, suggested that the nature of language is dialogical . He argued that both written and spoken language is always in dialogue with other texts and authors. We agree with Bakhtin’s view. What you write and what you say is dialogical because it’s responsive to other people (like a teacher or colleague), other texts (like the Odyssey ), as well as other ideas (as when, for example, you hold a sign or chant during a protest). And because we embrace the idea that written and spoken language is dialogical, we can now suggest that annotation - the addition and permanent anchoring of a note to a text - is dialogical, too.

Not only is annotation dialogical, we can also observe that all annotation is intertextual. And rather than take our word for it, let’s be expressly dialogical and put our book and ideas into dialogue with another text and set of ideas. Figure 4 includes a quote by the French philosopher Jacques Derrida. In addition to this quote, we’ve exercised our agency - to express our authorial power, to spark a response from you - by annotating Derrida. 20 Here, in dialogue with one another, is what we have to say about intertextuality and annotation. And maybe you, too, have something to say.

Figure 5: Derrida

So what does Derrida mean by annotation helping to prop up one discourse on another? Let’s revisit a few of the examples we’ve already discussed. When journalists annotated the Mueller report, annotation established an intertextual relationship between the report (as one “discourse”) and other discourses or dialogues, such as referenced evidence, an expert’s analysis, or established fact. When you annotated Beloved in a literature course, you created an intertextual relationship between Toni Morrison’s book (her discourse) and your own discourse comprised of reactions and wonderings. And your annotation of Beloved also added marks of evidence for when, subsequently, you were in dialogue with peers and your teacher. Just as annotation is multimodal, so too is it intertextual.

Annotation in Action

We’ve now established that annotation is an everyday activity. It’s likely that you read and write annotation regularly, perhaps on a daily basis. We’ve also introduced the ideas of multimodality, agency, and intertextuality, and suggested that the act of adding notes to a text is both multimodal and intertextual. As we close this introductory chapter, we’ll do so by sharing one more example of annotation in action. And rather than reference other texts or present hypothetical scenarios, we’ll turn our gaze inward and describe this very book.

Annotation was integral to how we wrote and received feedback about Annotation , how The MIT Press published the book, and how, perhaps, you’re reading and responding to the book right now.

The first full draft of our manuscript was shared publicly for the purposes of open peer review by The MIT Press using the online publication platform PubPub. Throughout the summer of 2019… [ADD SUMMARY OF OPEN PEER REVIEW PROCESS, INCLUDING TOTAL PARTICIPANTS AND ANNOTATION, EXAMPLES, ETC.].

This book, in its published form, also features various forms of annotation. The MIT Press has established a number of structural conventions that are consistent across all the books published as part of the Essential Knowledge series. As you may have already noticed, this book includes a Notes section just before the Bibliography (it starts on page XXX). This section of Notes is a collection of annotations. Organized by chapter, Notes presents to you, the reader, a total of 208 endnotes that we felt were important to add to the body of our text (even if, at times, it’s difficult flipping back to these endnotes rather than reading more proximal footnotes 21 ). In addition to Notes, this book includes a Glossary of key terms (see page XXX). We’ll discuss the origin and purpose of glosses and glossaries next, in Chapter 2. All glossaries are a curated list of annotations, and that goes for the Glossary in this book, too. Finally, just before the Index, this book includes suggested Further Readings. As researchers of literacy and learning, we’ve read and written quite a bit about annotation. We’ve selected a list of readings that we hope you might put into further dialogue with our book as you continue to explore annotation. In appending these Further Readings to this book, we’ve added yet another note to this text. Together, the Glossary, Notes, and Further Readings provide three different types of annotation that are integral to this text, adding both structure and insight for you, our reader.

And speaking of our readers, what of your annotation? Have you underlined or circled a word or phrase? Have you written an interlinear annotation, such as a word or symbol between the lines of text? Or have you added marginalia, a responsive type of discourse? And have you cursed us out, or called us names, or disagreed vehemently with our ideas? If you’ve yet to do so, now might be the time! Maybe you’ve borrowed this book from a friend. Marking up this book makes your thinking visible, allowing your friend to someday see what you thought about annotation and how you responded to our ideas. Or maybe you’re reading this book for a class or as part of a research project. Annotating this introductory chapter about annotation may help you to experience multimodality, exercise agency, establish intertextual relationships, and expand dialogical language among other texts, people, and ideas.

We don’t believe a published book is meant to live as a pristine artifact unadulterated in perpetuity. As Mortimer Adler wrote 80 years ago, the marked up book is the “thought-through book,” for it is “a conversation between you and the author.” And we certainly haven’t written the final word about annotation. We’ve hopefully written some useful words about an important topic so as to start a dialogue. We welcome your words and annotation throughout, about, within, and atop this text, too.

Figure 6: Dialogue

Why is Annotation Essential?

Annotation is essential because we interact with a variety of texts everyday, texts that are both digital and material, and texts that communicate through various modalities and may also be multimodal. We interact with these texts dialogically by adding our own thoughts, questions, reactions, and reminders. Annotation is also important because our intertextual interactions with texts cross multiple contexts - the personal, the academic, the professional, and even the commercial and civic: “Printed marginalia, functioning at their most creative level, open doorways specifically, insistently for the purpose of crossing the text-context threshold.” 22 Annotation follows us into, and then changes because of, the different ways in which we interact with texts across everyday contexts.

Annotation matters because it provides information; knowledge becomes more accessible because of annotation. Annotation matters because it shares commentary; perspectives become more transparent because of annotation. Annotation matters because it sparks conversation; dialogue becomes more interactive because of annotation. Annotation matters because it expresses power; authority becomes more contested and complex because of annotation. And annotation matters because it aids learning; education becomes more vibrant because of annotation. We’ll be discussing all of this, and more, as we explore how and why notes are added to texts.

One might also introduce the term “intratextuality” as distinguished from “intertextuality.” The former refers to cross-references within a single text, whereas the latter involves references between or among different texts. Annotation could involve both, I think. A note as such is intratextual … it is anchored to one specific text. But that note could itself contain a reference to another text, thereby introducing intertextuality. And, of course, that further text might be made accessible through a link, giving us the category of hypertextuality.

Uses of annotation in journalism

dfsdfasdfsdf

Let me start this part for you.

“For example, the annotated contributions of Dr. Jeremy Dean, to whom we dedicate this book…”

While I use this line myself all the time (as salesman essentially), your repetition of the idea has finally given even me pause: there as so many assumptions in this idea that everyone annotates. While it’s likely true for the readers of an MIT Press book and maybe fine as such, there are obviously many people who don’t and for reasons that often have to do with power. Just one example is that some public school teachers often don’t allow students to write in the margins of their books because they will be collected and reused.

Should this section be above the one on note and add?

I think it’s interesting to bring in social media behavior here. What is Instagram but “everyday” visual notes on our lives?

I wonder if the above three paragraphs are really necessary to get here.

Great quote!

is document a strong enough word here? create?

Footnote needed?

I wish I could annotate directly on top of the images you have created but I don’t seem to be able to do that (no fault of yours .. just an observation). If I want to digitally annotate your images (like the one below), I need to move outside of the text and then share back into the text …

It’s strange to read this in the present, with echoes of the future, reaching back to the past. Of course, it has to be written that way, but it feels, here in the margins, like some odd time ripple. I am here, writing about what you say has already been written (and I may even be too late in the process). Time for more coffee …

This seems important — that the reader has a role in the text with annotation … either by themselves or with others.

Ah, yes, here’s mention of the orality component. Could be cool to circle back to the historical side of things to this end. When it comes to discussions of Western spiritual texts (particularly the Torah in Jewish tradition), “oral” annotation, so to speak, was considered far more sacred than written annotation (especially since you can’t write on Talmudic scrolls)

Maybe this will come later in the book too, but I’m wondering whether there is room for discussion here on how often annotation is explicitly taught in schools? To what extent do teachers actively ask their students to engage in annotation activity? Or do students pick up annotation in a more idiosyncratic way?

More accessible to whom exactly? And where does the concept of accessibility, from a disability perspective, fit in here when we talk about annotation?

Great question …

I wonder whether perhaps this section or at least your proposed definition of annotation might not be more useful if it occurred earlier in a reader’s experience? Shortly before getting here I went back and scoured the first sections of chapter 1 and the preface in search of a foundational definitional of annotation and wasn’t quite sure I could offer a working definition for your crucial term as you wanted me to understand it!

It also serves as an outlet for dissent & disagreement, even if only privately!

Great visual example from Wikimedia :

Babylonian Talmud, Seder Zera'im, Venice: Daniel Bomberg [1543-44].

Great visual example from the Babylonian Talmud here: https://upload.wikimedia.org/wikipedia/commons/thumb/3/3b/Babylonian_Talmud%2C_Seder_Zera%27im.jpg/800px-Babylonian_Talmud%2C_Seder_Zera%27im.jpg

This feels a bit more optimistic that I think is warranted. Annotation in journalism seems to me to trending toward very limited, expert-only voices interacting with texts. It might be helpful to balance this trend against a reflection on the rise and decline of ‘comment sections’ on news and other public websites (lots of sites have killed public comments/annotation in the last 5 years), and the relationship between bottom of the page comments (‘page notes’) and anchored annotation/marginalia? See https://www.niemanlab.org/2015/09/what-happened-after-7-news-sites-got-rid-of-reader-comments/ + https://www.americanpressinstitute.org/publications/dropping-comment-sections/ + https://www.kqed.org/lowdown/29720/no-comment-why-a-growing-number-of-news-sites-are-dumping-their-comment-sections + https://www.nytimes.com/roomfordebate/2016/04/18/have-comment-sections-on-news-media-websites-failed + https://www.salon.com/2018/11/17/why-comments-sections-must-die/ + https://www.theatlantic.com/letters/archive/2018/02/letters-comments-on-the-end-of-comments/552392/ + https://www.wired.com/2015/10/brief-history-of-the-demise-of-the-comments-timeline/ + http://audreywatters.com/2017/04/26/no-annotations-thanks-bye

https://www.nytimes.com/2017/06/14/watching/the-handmaids-tale-tv-finale-margaret-atwood.html

https://www.politico.com/story/2019/04/18/mueller-report-summary-key-findings-1280879

https://www.npr.org/2019/04/18/708965026/highlights-from-the-mueller-report

https://www.washingtonpost.com/graphics/2019/politics/read-the-mueller-report/

In particular, the technologies that we have at our disposal for accessing — and annotating — texts matter a great deal. Even if we have robust tools, we may not always know how to use them in the most productive ways, or to communicate our annotations to a wider audience.

Yet, in many ways, they still do… even in our e-book, editable, and constantly interactive world of texts, eventually a book has to stand alone in the world.

One reader’s interpretation of that book (via annotation) is a unique and beautiful act, and many readers can discuss the book, but the book (which could be revised later) still stands alone.

Going to your point above that some works of literature/film can have “implicit and interpretive intertextuality,” I don’t know that my current practice of annotation on your manuscript is something I would put into this intellectual register.

That is, I am annotating, but I don’t know that I am really creating the kinds of intertextuality described by literary theorists. I am making some links, adding some images, and sharing my own ideas, yes. But, I am not writing another book here, making tacit or overt references to your ideas, or mimicking your structure or style.

All this is to say, I think that you might want to make your argument on intertextuality a little more nuanced… are there different “levels” of intertextuality that happen, depending on the quality and type of annotation?

>But, I am not writing another book here… Why not? How do you know!?

Keeping with my thread from above, if students are being told that they must annotate… I wonder if they are truly acting with agency, and engaging in genuine work of annotation. Or, are they merely fulfilling their assignment?

In short, I am afraid that the practice of having students use reading strategies to approach texts, while useful in many ways, can become a mindless exercise filled with many, many sticky notes and few genuine interactions with the text itself.

Agree with the above.

I wonder though if just complicating our understanding of agency and power in these annotation contexts might allows the focus on agency to remain, just complicated.

Another complication: the various platforms in which annotation can happen are themselves not neutral. Even the margin of a page of text has certain limitations and expectations…

Perhaps an additional pair of questions: By whom? For what purpose?

Yet, for decades, students have been explicitly told not to damage or otherwise deface their textbooks (see page 17 in this PDF of a district policy ).

While I can agree that annotations in school are, for some students, common practice, I would encourage you to be more nuanced here.

For some students, who are willing and able to take the teacher’s shared notes, outlines, or slides (or, go so far as to make photocopies of their textbooks, they might engage in the kinds of personal and useful annotation practices you describe.

For the vast majority of students, however, I would argue that annotation (if done at all) is perfunctory. They are given specific texts and tasks, and required to make so many notations in trade for a grade.

The “required” part of your definition, then, could use some elaboration and, to extend the idea, some clarification on whether or not the process of annotating is, ultimately, useful for these students.

Moreover, connecting to your next idea, how can we help students learn to annotate in a way that honors and extends their own “idiosyncratic notation system[s]” in productive, engaging ways?

Seconding Troy’s thoughts here. It would also be interesting to consider the role of medium here and how that’s considered in schools (if at all). That is, annotation itself as a practice may be very different in print and in digital spaces. Is education a space where that nuance is approached? (Not sure if this is necessarily the right place for this particular conversation, but I think there’s more nuance for the school-based context as Troy suggests).

Another good example is the built up genealogy of family bibles inscribed with the names of owners and their family tree which are passed from one generation to the next. To some extent this is highlighted by the passages of the bible in which W begat X begat Y begat Z begat... (Genesis chapters 5 & 11).

These sorts of ancient and modern genealogies also heavily underpinned personal, familial, tribal, and governmental power structures through the ages. Paternity was power.

A well known popular culture version of this appears in the title of the book and film *Harry Potter and the Half-blood Prince* as well as a primary plot point in which Potter actively eschews a beaten up copy of a potions textbook, but to his pleasant surprise find a heavily annotated text that helps him significantly in his studies.

https://harrypotter.fandom.com/wiki/Severus_Snape%27s_copy_of_Advanced_Potion-Making

Yup, see Chapter 6 and our seventh note ;)

I can't help but think of one of the biggest and longest standing puzzles in mathematics in Fermat's Last Theorem. He famously wrote in the margin of a book that he had a proof. but that it was too large to fit in the margin.

https://en.wikipedia.org/wiki/Fermat%27s_Last_Theorem

Yes! We really like this example, too, and wondered where to strategically include in the book. Ultimately, we're still searching for the right place or moment to mention Fermat… as you read the book, perhaps you can suggest where it may be best to include this example.

Google has accelerated this by using search to better link pieces of knowledge in the modern world, but scholars have been linking thoughts manually for centuries.

Surprisingly, these have only been recently aggregated online at [Sefaria](https://www.sefaria.org/texts) a story delineated here:

https://www.washingtonpost.com/religion/2018/09/18/quest-put-talmud-online/

Yes, we mention this and link to Sefaria in Chapter 3 when discussing the Talmud and commentary.

Other great examples include teaching and scientific progress. Owen Gingerich details annotations in all the extant copies of Copernicus in his text [The Book Nobody Read: Chasing the Revolutions of Nicolaus Copernicus](https://www.amazon.com/Book-Nobody-Read-Revolutions-Copernicus/dp/B000BNPG8C). There it seemed obvious that the moving state-of-the art of science and teaching was reflected in the annotations made by professors who handed those annotations down to students who also copied them into their textbooks.

Wonderful, we'll include this book in Further Readings .

In some sense this is a textual equivalent of the directors commentary tracks on DVDs from the 1990's in which one could watch films with overdubbed running commentary of the film's director (and often cast, producers, et al. as appropriate).

The first time I recall seeing such journalistic annotations was on the web in The Smoking Gun ( http://www.thesmokinggun.com/ ) which generally annotated court documents that were the source of newsworthy tidbits—generally relating to celebrities or gossip pages.

Great example, and perhaps one we should explicitly mention.

copyediting, will do

remove space

So, let’s say I cringe when writing on the paper and use post-it notes instead, would that not count under your definition? Because it can be removed so isn’t permanent? Pencil annotations can be erased.

This is very much the case for me. And this has also evolved over time, as the purpose of my reading has shifted.

I’m sure that you will also discuss the value of annotation for personal/private use also.

Maybe even the next paragraph. Ha!

It would be fantastic to have the links to these items here in the text. Do you plan to add that later?

I agree with Heather. The link to the NYTimes annotated/searchable version of the Mueller Report would be a good example.

And other texts: the doorjamb with a child’s growth marks, William Burke’s computer log, a building annotated with graffiti, etc.

Loving the stress on agency here. In this context, thinking how notes become a physical manifestation of agency that may occur otherwise without record, like that voice in my head while reading that keeps saying things like “WTF?” or “Exactly!!!!!!).

Maybe there’s a connection here to your earlier concern regarding the “deterministic effects attributed to annotation.” People have agency. People are exercising their agency when adding notes to texts. And when that happens, annotation serves five purposes (as we suggest here in Chapter 1 and explore throughout the book) and it is those annotation purposes that consequently have certain effects… is that helpful?

This moth should become the mascot of multimodal annotation.

?!* couldn’t resist!

Again: maybe “enable” would be a better choice?

Yes, as noted above, we can easily revise throughout.

Just putting a +1 on all of these comments from Nate, Chris, and Troy!

Related to my note above about power in annotation, I feel I need to post a concern here that I’m on the watchout for deterministic effects attributed to annotation as a general technology/practice — rather than to specific social deployments of annotation practices. Each of these outcomes seems like a _possible_, but not _required_ outcome of annotation in specific contexts.

This slightly negative characterization of annotation is a bit jarring as so far we readers have not been presented with a negative view of annotation.

Helpful, thank you.

This sentence made me pause. I certainly think the first clause is worthy, but I’m not sure the the second must always follow. I expect you’ll get into power more later in the book, but based on what I’ve read so far, this seems like a very strong statement to make.

Yes, a strong statement to make. Can we revisit this once you’ve read Chapter 5?

Is the idea that this sentence might link to a way to see possible annotations on this book?

Antero and I have thought a lot about how to create an annotation experience with this manuscript, in both digital and print form, and how that experience can *enable* ongoing conversation. This open review helps to check the box for digital interactions, trails, and spaces. The custom illustrations in every chapter ideally invite reader interaction with the print text (once the book is in hand). And the dedicated hashtag #AnnoConvo will hopefully become another “place” to archive some of that activity. For example, perhaps a future reader annotates one of the custom illustrations - like Fig 5 in this chapter - and then photographs their book/annotation and shares via social media with the #AnnoConvo tag.

For other examples of annotation being used in the sciences, see ClinGen (https://www.clinicalgenome.org/working-groups/biocurators/), NIF (https://neuinfo.org/about/organization), the Qualitative Data Repository (https://qdr.syr.edu/), and SciBot (https://web.hypothes.is/blog/annotation-with-scibot/). I could connect more dots to these or intro folks how know more.

I didn’t know about ClinGen, thank you! QDR is featured in Chapter 2 in our section “Information among Knowledge Communities.” And SciBot is featured in Chapter 7 when we ask, “How should we read human-machine annotation?”

Starting to seem like I only care about em dashes ;) but I think this would read better set apart with em-dashes ;)

Consider replacing throughout with real em-dashes: — ;)

Copyediting - will do ;)

Maybe addressed elsewhere in the text, but it would be nice to see some other examples of fact-checking here. https://climatefeedback.org/ comes to mind, or for a meta-example, Poynter’s “What to expect from fact-checking in 2018”, annotated later to evaluate their predictions (https://www.poynter.org/fact-checking/2017/what-to-expect-from-fact-checking-in-2018/#annotations:16039949). There are likely more…

Thanks for these suggestions, Nate. Climate Feedback is featured in the final section of Chapter 3 (and I hope you appreciate the particular example of peer review that we highlight!). The Poynter resource is great, perhaps that becomes an endnote here?

This sentence is tripping me up with the comma and no “and”. Maybe the comma is more like a colon or em-dash, something like: “Annotation enables journalists to comment more transparently — to share behind-the-scenes or insider perspectives more informally.”?

Yup, a change for clarity. Or maybe also including the word “and” between clauses?

Whenever I see “allows”, I always wonder if “enables” might be a better word choice. Here I think so…

Yes, thank you. We can adjust throughout. That’s a very important distinction related to your broader comments regarding agency.

Love this. In my work, I view “Text” as being very broad. And “notes” on text…or annotation should/could add value to that intersection.

One of the features that I wish I had in a tool like Hypothesis (I guess I have it in Vialogues/VideoANT), but I would love to annotate these different texts, and connect the dots across those spaces. Connect a note on a video to a note on a wikipage to a note on a tweet.

Using annotation to connect across multiple texts and multimedia texts/compositions would make more visible thinking, engagement, and agency of the annotator. Perhaps this is yet to come …

Making me think about affordances of different forms of text (images, hyperlinks, GIFs, video, text) and what that adds or detracts from the text.

This is interesting… if we take an image (or GIF, or other item) that someone else has created, and insert it into our own annotation, have we made it “our own?” Remixed or repurposed it? Can that be considered an annotation, in the sense that we are adding value to the text, or really just a comment of little consequence?

Loving multimodality in here…also the connection to “embers.”

I, too, appreciate that you are already layering in multimodality, even in some print-centric examples of annotation.

In addition to the ideas you are offering about multimodality here, I would also encourage you to look at the seminal work of Gunter Kress, for instance Multimodality: A Social Semiotic Approach to Contemporary Communication .

I’ve viewed these annotation practices (e.g., Hypothesis) as having “discussion about the text baked into the text.” This has the potential to provide a third space for not only dialogue, but growth in a variety of areas.

I’d be curious to know more about the concept of annotation as a “third space” — is the idea that what unfolds in the margins becomes its own, distinct text that can be separated from the original but still stand on its own?

Along with these trends, we do see some that do not value the use/inclusion of annotation on their publishing spaces as they view it as another form of commentary about their work that may modify/limit points made.

This is also making me think about power, access, and digital literacy/savvy. When I first introduce people to Hypothesis, some of their responses are about the fact that “anyone can annotate online” and “on the spaces they already read.” Some view this “invisible layer” of annotation on the Internet as questionable/problematic.

Interesting thought. I’ve always viewed annotation as “additive” or generally positive/beneficial for all. I’ve thought (perhaps it’s my own bias) that redaction is a negative…but now that I’m typing this I realize (I think) that redaction is a type of annotation…and annotation/redaction always benefits someone…it just might not be you. :)

In this respect, you may be very interested to see our discussion about power and redaction in Chapter 5.

!!! I cannot add a GIF here?!?! :)

Exactly…having discussion about the text…baked into the text.

I hope you’re including a mention of Jasper Fforde’s use of the footnoterphone as a tool for conversation across the book world he created in his Thursday Next series.

Might you share a specific link/resource, Bud? Thanks!

But what really, <i>really</i> matters is that the act of making the note is frequently an act of talking back to the text, of recognizing and/or remembering that the author of the text you’re reading isn’t the only person with something to say, and that what they’re saying may not be the last word on the subject.

Bud’s point here about “talking back to the text” is important, and I think that we could employ other prepositions as well.

What does it mean to talk to the text? About the text? Beyond the text? Within the text? Through the text?

Annotation can serve all of these purposes, when approached strategically.

I was gonna say that if I didn’t see a discussion of Marginalia in this book somewhere, I’d be disappointed.

So you lead with a poem ‘bout it. Well played.

If you’re looking for a formal definition of marginalia, as a particular type of annotation, check out Chapter 2 where as also discuss rubrics and rubrication, scholia, and other “forms that inform.”

Librarian approved

Hmm. Me, too, but I didn’t even think about Director Cuts/Commentary DVDs as annotation until you mentioned it

The first annotation I really remember interacting with was all A/V based: VH1 Pop Up Videos and Director Commentaries on the DVD Extras section for movies I liked.

Analyzing Concordances

• First Online: 05 May 2021

Cite this chapter

• Stefanie Wulff 3 &
• Paul Baker 4  

1872 Accesses

1 Citations

In its simplest form, a concordance is a list of all attestations (or hits) of a particular search word or phrase, presented with a user-defined amount of context to the left and right of the search word or phrase. In this chapter, we describe how to generate and manipulate concordances, and we discuss how they can be employed in research and teaching. We describe how to generate, sort, and prune concordances prior to further analysis or use. In a section devoted to qualitative analysis, we detail how a discourse-analytical approach, either on the basis of unannotated concordance lines or on the basis of output generated by a prior quantitative examination of the data, can help describe and, crucially, explain the observable patterns, for instance by recourse to concepts such as semantic prosody. In a section devoted to quantitative analysis, we discuss how concordance lines can be scrutinized for various properties of the search term and annotated accordingly. Annotated concordance data enable the researcher to perform statistical analyses over hundreds or thousands of data points, identifying distributional patterns that might otherwise escape the researcher’s attention. In a third section, we turn to pedagogical applications of concordances. We close with a critical assessment of contemporary use of concordances as well as some suggestions for the adequate use of concordances in both research and teaching contexts, and give pointers to tools and resources.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

For a list of web-based concordancers (and many other corpus-linguistic resources), see http://martinweisser.org/corpora_site/CBLLinks.html . Accessed 31 May 2019.

Baker, P. (2006). Using corpora in discourse analysis . London/New York: Continuum.

Google Scholar  

Baker, P., & McEnery, A. (2014). Find the doctors of death: the UK press and the issue of foreign doctors working in the NHS, a corpus-based approach. In A. Jaworski & N. Coupland (Eds.), The Discourse Reader (pp. 465–480). London: Routledge.

Boulton, A. (2008). DDL: reaching the parts other teaching can’t reach? In A. Frankenberg-Garcia (Ed.), Proceedings of the 8th Teaching and Language Corpora conference (pp. 38–44). Lisbon: Associação de Estudos e de Investigação Cientifíca do ISLA-Lisboa.

Boulton, A. (2010). Data-driven learning: taking the computer out of the equation. Language Learning, 60 (3), 534–572.

Article   Google Scholar  

Boulton, A., & Cobb, T. (2017). Corpus use in language learning: a meta-analysis. Language Learning, 67 (2), 348–393.

Cobb, T. (1997). Is there any measurable learning from hands on concordancing? System, 25 (3), 301–315.

Frankenberg-Garcia, A., Aston, G., & Flowerdew, L. (Eds.). (2011). New trends in corpora and language learning . New York: Bloomsbury.

Gries, S. T. (2016). Quantitative corpus linguistics with R: a practical introduction (2nd ed.). London/New York: Routledge.

Book   Google Scholar  

Gries, S. T., & Wulff, S. (2013). The genitive alternation in Chinese and German ESL learners: towards a multifactorial notion of context in learner corpus research. International Journal of Corpus Linguistics, 18 (3), 327–356.

Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: an overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8 (1), 23–34.

Huang, Z. (2014). The effects of paper-based DDL on the acquisition of lexico-grammatical patterns in L2 writing. ReCALL, 26 (2), 163–183.

Jaeger, T. F. (2010). Redundancy and reduction: speakers manage syntactic information density. Cognitive Psychology, 61 , 23–62.

Lee, H., Warschauer, M., & Lee, J. H. (2018). The effects of corpus use on second language vocabulary learning: a multilevel meta-analysis. Applied Linguistics . published online first.

Louw, W. E. (1993). Irony in the text or insincerity in the writer? The diagnostic potential of semantic prosodies. In M. Baker, G. Francis, & E. Tognini-Bonelli (Eds.), Text and technology: in honour of John Sinclair (pp. 157–176). Amsterdam: John Benjamins.

Chapter   Google Scholar  

McGlashan, M. (2016). The representation of same-sex parents in children’s picturebooks: A corpus-assisted multimodal critical discourse analysis . Dissertation, Lancaster University.

Partington, A. (1998). Patterns and meaning: using corpora for English language research and teaching . Amsterdam/Philadelphia: John Benjamins.

Partington, A. (2004). “Utterly content in each other’s company”: semantic prosody and semantic preference. International Journal of Corpus Linguistics, 9 (1), 131–136.

Römer, U. (2008). Corpora and language teaching. In A. Lüdeling & M. Kyto (Eds.), Corpus linguistics: an international handbook (Vol. 1, pp. 112–130). Berlin: Mouton de Gruyter.

Simpson, P. (1993). Language, ideology and point of view . London: Routledge.

Sinclair, J. (1991). Corpus, concordance, collocation . Oxford: Oxford University Press.

Sinclair, J. (2004). Trust the text: language, corpus and discourse . London: Routledge.

Stevens, V. (1991). Concordance-based vocabulary exercises: a viable alternative to gap-fillers. English Language Research Journal, 4 , 47–63.

Stubbs, M. (1995). Collocations and semantic profiles: on the cause of the trouble with quantitative studies. Functions of Language, 2 (1), 23–55.

Stubbs, M. (2001). Words and phrases: corpus studies of lexical semantics . Oxford: Blackwell.

Wulff, S., Gries, S. T., & Lester, N. A. (2018). Optional that in complementation by German and Spanish learners. In A. Tyler & C. Moder (Eds.), What is applied cognitive linguistics? Answers from current SLA research (pp. 99–120). New York: De Gruyter Mouton.

Wulff, S., Lester, N. A., & Martinez-Garcia, M. (2014). That -variation in German and Spanish L2 English. Language and Cognition, 6 , 271–299.

Download references

Author information

Authors and affiliations.

University of Florida, Gainesville, Florida, USA

Stefanie Wulff

Lancaster University, Lancaster, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stefanie Wulff .

Editor information

Editors and affiliations.

FNRS Centre for English Corpus Linguistics, Language and Communication Institute, UCLouvain, Louvain-la-Neuve, Belgium

Magali Paquot

Department of Linguistics, University of California, Santa Barbara, CA, USA

Stefan Th. Gries

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Wulff, S., Baker, P. (2020). Analyzing Concordances. In: Paquot, M., Gries, S.T. (eds) A Practical Handbook of Corpus Linguistics. Springer, Cham. https://doi.org/10.1007/978-3-030-46216-1_8

Download citation

DOI : https://doi.org/10.1007/978-3-030-46216-1_8

Published : 05 May 2021

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-46215-4

Online ISBN : 978-3-030-46216-1

eBook Packages : Religion and Philosophy Philosophy and Religion (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Writing Resources

• Student Paper Template
• Grammar Guidelines
• Punctuation Guidelines
• Writing Guidelines
• Creating a Title
• Outlining and Annotating
• Using Generative AI (Chat GPT and others)
• Introduction, Thesis, and Conclusion
• Strategies for Citations
• Determining the Resource This link opens in a new window
• Citation Examples
• Paragraph Development
• Paraphrasing
• Inclusive Language
• International Center for Academic Integrity
• How to Synthesize and Analyze
• Synthesis and Analysis Practice
• Synthesis and Analysis Group Sessions
• Decoding the Assignment Prompt
• Annotated Bibliography
• Comparative Analysis
• Conducting an Interview
• Infographics
• Office Memo
• Policy Brief
• Poster Presentations
• PowerPoint Presentation
• White Paper
• Writing a Blog
• Research Writing: The 5 Step Approach
• Step 1: Seek Out Evidence
• Step 2: Explain
• Step 3: The Big Picture
• Step 4: Own It
• Step 5: Illustrate
• MLA Resources
• Time Management

ASC Chat Hours

ASC Chat is usually available at the following times ( Pacific Time):

If there is not a coach on duty, submit your question via one of the below methods:

  928-440-1325

  Ask a Coach

  [email protected]

Search our FAQs on the Academic Success Center's  Ask a Coach   page.

Outlining and Annotating Resources

Writing an Annotated Bibliography

An annotated bibliography is a summary and evaluation of a resource. Writing an annotated bibliography will help you gain an in-depth understanding of your topics and is useful for organizing and cataloging resources for use when developing an argument. An annotated bibliography begins with an APA formatted reference followed by one or two paragraphs of text that summarizes the study, evaluates the reliability of the information, and evaluates how the information relates to previous and future research. 

This table provides a high-level outline of the structure of a research article and how each section relates to important information for developing an annotated bibliography.

Annotated Bibliography Sample Outline

Author, S. A. (date of publication). Title of the article.  Title of Periodical, vol.  (issue), page-page.  https://doi.org/XXXXXX

Write one or two paragraphs that focus on the study and its findings.

• Two or more sentences that outline the thesis, hypothesis, and population of the study.
• Two or more sentences that discuss the methodology.
• Two or more sentences that discuss the study findings.  
• One or more sentences evaluating the study and its relationship to other studies.

Outlining (Scholarly Writing) - Group Session

Tuesday 4:00 p.m. 

Outlining is a way of organizing ideas and is a helpful strategy for academic success. There are multiple ways to outline and doing so before and after composing a paper can help with the paper's arrangement and help ensure alignment with assignment prompts. This group session will include general organization techniques, creating an outline from an assignment prompt, creating an outline from a thesis, outlining for larger projects, and reverse outlining. 

Was this resource helpful?

• << Previous: Creating a Title
• Next: Using Generative AI (Chat GPT and others) >>
• Last Updated: Apr 10, 2024 2:12 PM
• URL: https://resources.nu.edu/writingresources

Annotated Bibliographies

What Is An Annotated Bibliography?

What is an annotated bibliography.

An annotated bibliography is a list of citations (references) to books, articles, and documents followed by a brief summary, analysis or evaluation, usually between 100-300 words, of the sources that are cited in the paper.  This summary provides a description of the contents of the source and may also include evaluative comments, such as the relevance, accuracy and quality of the source.  These summaries are known as annotations. 

• Annotated bibliographies are completed before a paper is written
• They can be stand-along assignments
• They can be used as a reference tool as a person works on their paper

Annotations vs. Abstracts

Abstracts are the descriptive summaries of article contents found at the beginning of scholarly journal articles that are written by the article author(s) or editor. Their purpose is to inform a reader about the topic, methodology, results and conclusion of the research of the article's author(s).  The summaries are provided so that a researcher can determine whether or not the article may have information of interest to them.  Abstracts do not serve an evaluative purpose.

Annotations found in bibliographies are evaluations of sources cited in a paper.  They describe a work, but also critique the source by examining the author’s point of view, the strengths and weakness of the research or article hypothesis or how well the author presented their research or findings.

How to write an annotated bibliography

The creation of an annotated bibliography is a three-step process. It starts with finding and evaluating sources for your paper. Next is choosing the type or category of annotation, then writing the annotation for each different source. The final step is to choose a citation style for the bibliography.

Types of Annotated Bibliographies

Types of Annotations

Annotations come in different types, the one to use depends on the instructor’s assignment.  Annotations can be descriptive, a summary, or an  evaluation or a combination of descriptive and evaluation.

Descriptive/Summarizing Annotations

There are two kinds of descriptive or summarizing annotations, informative or indicative, depending on what is most important for a reader to learn about a source.  Descriptive/summarizing annotations provide a brief overview or summary of the source. This can include a description of the contents and a statement of the main argument or position of the article as well as a summary of the main points.  It may also describe why the source would be useful for the paper’s topic or question. 

Indicative annotations provide a quick overview of the source, the kinds of questions/topics/issues or main points that are addressed by the source, but do not include information from the argument or position itself.

Informative annotations, like indicative annotations, provide a brief summary of the source.   In addition, an informative annotation identifies the hypothesis, results, and conclusions presented by the source.  When appropriate, they describe the author’s methodology or approach to the topic under discussion.  However, they do not provide information about the sources usefulness to the paper or contains analytical or critical information about the source’s quality. 

Evaluative Annotations (also known as critical or analytical)

Evaluative annotations go beyond just summarizing the source and listing out it’s key points, but also analyzes the content. It looks at the strengths and weaknesses of the article’s argument, the reliability of the presented information as well as any biases of the author. It talks about how the source may be useful to a particular field of study or the person’s research project.

Combination Annotations

Combination annotations “combine” aspects from indicative/informative and evaluative annotations and are the most common category of annotated bibliography.  Combination annotations include one to two sentences summarizing or describing content, in addition to one or more sentences providing an critical evaluation.

Writing Style for Annotations

Annotations typically follow three specific formats depending on how long they are.

• Phrases – Short phrases providing the information in a quick, concise manner.
• Sentences – Complete sentences with proper punctuation and grammar, but are short and concise.
• Paragraphs – Longer annotations break the information out into different paragraphs. This format is very effective for combination annotations.

To sum it up:

An annotation may include the following information:

• A brief summary or overview of the source content
• The source’s strengths and weaknesses in presenting the argument or position
• Its conclusions
• Why the source is relevant in to field of study of the paper
• Its relationships to other studies in the field
• An evaluation of the research methodology (if applicable)
• Information about the author’s background and potential biases
• Conclusions about the usefulness of the source for the paper

Critically Analyzing Articles

In order to write an annotation for a paper source, you need to first read and then critically analyze it:

• Try to identify the topic of the source -- what is it about and is it clearly stated.
• See if you can identify the purpose of the author(s) in doing the research or writing about the topic. Is it to survey and summarize research on a topic?  Is the author(s) presenting an argument based on previous research, or refuting previously published research?
• Identify the research methods used and try to identify whether they appear to be suitable or not for the stated purpose of the research.  
• Was the research reported in a consistent or clear manner?  Or, was the author's argument/position presented in a consistent or convincing manner? Did the author(s) fail to acknowledge and explain any limitations?
• Was the logic of the research/argument claims properly supported with convincing evidence/analysis/data? Did you spot any fallacies?
• Check whether the author(s) refers to other research and if similar studies have been done. 
• If illustrations or charts are used, are they effective in presenting information?
• Analyze the sources that were used by the author(s). Did the author(s) miss any important studies they should have considered?
• Your opinion of the source -- do you agree with or are convinced of the findings?  
• Your estimation of the source’s contribution to knowledge and its implications or applications to the field of study.

Worksheet for Taking Notes for Critical Analysis of Sources/Articles

Additional Resources:

Hofmann, B., Magelssen, M. In pursuit of goodness in bioethics: analysis of an exemplary article. BMC Med Ethics 19, 60 (2018). https://doi.org/10.1186/s12910-018-0299-9

Jansen, M., & Ellerton, P. (2018). How to read an ethics paper. Journal of Medical Ethics, 44(12), 810-813.  http://dx.doi.org/10.1136/medethics-2018-104997

Research & Learning Services, Olin Library, Cornell University Library  Critically Analyzing Information Sources: Critical Appraisal and Analysis

Formatting An Annotated Bibliography

How do I format my annotated bibliography?

An annotated bibliography entry consists of two components: the Citation and the Annotation.

The citation should be formatted in the bibliographic style that your instructor has requested for the paper. Some common citation styles include APA, MLA, and Chicago. For more information on citation styles, see Writing Guides, Style Manuals and the Publication Process in the Biological & Health Sciences .

Many databases (e.g., PubMed, Academic Search Premier, Library Search on library homepage, and Google Scholar) offer the option of creating your references in various citation styles. 

Look for the "cite" link -- see examples for the following resources:

University of Minnesota Library Search

Google Scholar

Sample Annotated Bibliography Entries

An example of an Evaluative Annotation , APA style (7th ed). (sample from University Libraries, University of Nevada ).

APA does not have specific formatting rules for annotations, just for the citation and bibliography.

Maak, T. (2007). Responsible leadership, stakeholder engagement, and the emergence of social capital. Journal of Business Ethics, 74, 329-343.  https://doi.org/10.1007/s10551-007-9510-5

This article focuses on the role of social capital in responsible leadership. It looks at both the social networks that a leader builds within an organization, and the links that a leader creates with external stakeholders. Maak’s main aim with this article seems to be to persuade people of the importance of continued research into the abilities that a leader requires and how they can be acquired. The focus on the world of multinational business means that for readers outside this world many of the conclusions seem rather obvious (be part of the solution not part of the problem). In spite of this, the article provides useful background information on the topic of responsible leadership and definitions of social capital which are relevant to an analysis of a public servant.

An example of an Evaluative Annotation , MLA Style (10th ed), (sample from Columbia College, Vancouver, Canada )

MLA style requires double-spacing (not shown here) and paragraph indentations.

London, Herbert. “Five Myths of the Television Age.” Television Quarterly, vol. 10, no. 1, Mar. 1982, pp. 81-69.

     Herbert London, the Dean of Journalism at New York University and author of several books and articles, explains how television contradicts five commonly believed ideas. He uses specific examples of events seen on television, such as the assassination of John Kennedy, to illustrate his points. His examples have been selected to contradict such truisms as: “seeing is believing”; “a picture is worth a thousand words”; and “satisfaction is its own reward.” London uses logical arguments to support his ideas which are his personal opinion. He does not refer to any previous works on the topic. London’s style and vocabulary would make the article of interest to any reader. The article clearly illustrates London’s points, but does not explore their implications leaving the reader with many unanswered questions.

Additional Resources

University Libraries Tutorial --  Tutorial: What are citations?  Completing this tutorial you will:

• Understand what citations are
• Recognize why they are important
• Create and use citations in your papers and other scholarly work

University of Minnesota Resources

Beatty, L., & Cochran, C. (2020). Writing the annotated bibliography : A guide for students & researchers . New York, NY: Routledge. [ebook] 

Efron, S., Ravid, R., & ProQuest. (2019). Writing the literature review : A practical guide . New York: The Guilford Press. [ebook -- see Chapter 6 on Evaluating Research Articles] 

Center for Writing: Student Writing Support

• Critical reading strategies
• Common Writing Projects (includes resources for literature reviews & analyzing research articles)

Resources from Other Libraries

Annotated Bibliographies (The Writing Center, University of North Carolina at Chapel Hill)

Writing An Annotated Bibliography (University of Toronto)

Annotated Bibliographies (Purdue Writing Lab, Purdue University)

Annotated Bibliography (UNSW Sydney)

What is an annotated bibliography? (Santiago Canyon College Library): Oct 17, 2017. 3:47 min.

Writing an annotated bibliography (EasyBib.com) Oct 22, 2020. 4:53 min.

Creating an annotated bibliography (Laurier University Library, Waterloo, Ontario)/ Apr 3, 2019, 3:32 min.

How to create an annotated bibliography: MLA (JamesTheDLC) Oct 23, 2019. 3:03 min.

Citing Sources

Introduction

Citations are brief notations in the body of a research paper that point to a source in the bibliography or references cited section.

If your paper quotes, paraphrases, summarizes the work of someone else, you need to use citations.

Citation style guides such as APA, Chicago and MLA provide detailed instructions on how citations and bibliographies should be formatted.

Health Sciences Research Toolkit

Resources, tips, and guidelines to help you through the research process., finding information.

Library Research Checklist Helpful hints for starting a library research project.

Search Strategy Checklist and Tips Helpful tips on how to develop a literature search strategy.

Boolean Operators: A Cheat Sheet Boolean logic (named after mathematician George Boole) is a system of logic to designed to yield optimal search results. The Boolean operators, AND, OR, and NOT, help you construct a logical search. Boolean operators act on sets -- groups of records containing a particular word or concept.

Literature Searching Overview and tips on how to conduct a literature search.

Health Statistics and Data Sources Health related statistics and data sources are increasingly available on the Internet. They can be found already neatly packaged, or as raw data sets. The most reliable data comes from governmental sources or health-care professional organizations.

Evaluating Information

Primary, Secondary and Tertiary Sources in the Health Sciences Understand what are considered primary, secondary and tertiary sources.

Scholarly vs Popular Journals/Magazines How to determine what are scholarly journals vs trade or popular magazines.

Identifying Peer-Reviewed Journals A “peer-reviewed” or “refereed” journal is one in which the articles it contains have been examined by people with credentials in the article’s field of study before it is published.

Evaluating Web  Resources When searching for information on the Internet, it is important to be aware of the quality of the information being presented to you. Keep in mind that anyone can host a web site. To be sure that the information you are looking at is credible and of value.

Conducting Research Through An Anti-Racism Lens This guide is for students, staff, and faculty who are incorporating an anti-racist lens at all stages of the research life cycle.

Understanding Research Study Designs Covers case studies, randomized control trials, systematic reviews and meta-analysis.

Qualitative Studies Overview of what is a qualitative study and how to recognize, find and critically appraise.

Writing and Publishing

Citing Sources Citations are brief notations in the body of a research paper that point to a source in the bibliography or references cited section.

Structure of a Research Paper Reports of research studies usually follow the IMRAD format. IMRAD (Introduction, Methods, Results, [and] Discussion) is a mnemonic for the major components of a scientific paper. These elements are included in the overall structure of a research paper.

Top Reasons for Non-Acceptance of Scientific Articles Avoid these mistakes when preparing an article for publication.

Annotated Bibliographies Guide on how to create an annotated bibliography.

Writing guides, Style Manuals and the Publication Process in the Biological and Health Sciences Style manuals, citation guides as well as information on public access policies, copyright and plagiarism.

13.4 Annotated Student Sample: Research Log

Learning outcomes.

By the end of this section, you will be able to:

• Demonstrate the ability to inquire, learn, think critically, and communicate when reading in varying rhetorical and cultural contexts.
• Identify and analyze relationships between ideas, patterns of organization, and interplay between verbal and nonverbal elements in written texts.
• Practice and apply strategies such as interpretation, synthesis, response, and critique to compose texts that integrate the writer’s ideas with those from appropriate sources.

Introduction

Lily Tran created this log entry during the research process for an argumentative research paper assigned in her first-year composition class, as shown in this Annotated Student Sample .

Living by Their Own Words

Planning to write.

public domain text Freewrite: I found this photograph in an article I was reading about food insecurity during the COVID-19 pandemic. I copied and pasted it here as inspiration for my argumentative research paper. end public domain text

annotated text Lily Tran includes a visual in the freewrite section of her research log. The visual may or may not appear in the final paper, but here, it serves to stimulate her writing and thinking about her topic and possibly connect to other information she finds. end annotated text

public domain text For a sustainable future, food production and processing have to change. So does global distribution. end public domain text

annotated text Tran begins to establish problem-and-solution reasoning, recognizing that there are different stages to food production and that all will be affected by any proposed solution. end annotated text

public domain text The necessary changes will affect nearly all aspects of life, including world hunger, health and welfare, use of land resources, habitats, water, energy use and production, greenhouse gas emissions and climate change, and economics, as well as cultural and social values. end public domain text

annotated text Tran also employs cause-and-effect reasoning in beginning to think about the effects of any proposed change. end annotated text

public domain text These needed changes may not be popular, but people will have to accept them. end public domain text

annotated text She recognizes potential counterarguments to address if the paper is to be persuasive. end annotated text

Discussion Questions

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/writing-guide/pages/1-unit-introduction

• Authors: Michelle Bachelor Robinson, Maria Jerskey, featuring Toby Fulwiler
• Publisher/website: OpenStax
• Book title: Writing Guide with Handbook
• Publication date: Dec 21, 2021
• Location: Houston, Texas
• Book URL: https://openstax.org/books/writing-guide/pages/1-unit-introduction
• Section URL: https://openstax.org/books/writing-guide/pages/13-4-annotated-student-sample-research-log

© Dec 19, 2023 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

Help | Advanced Search

Computer Science > Computation and Language

Title: leave no context behind: efficient infinite context transformers with infini-attention.

Abstract: This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M sequence length passkey context block retrieval and 500K length book summarization tasks with 1B and 8B LLMs. Our approach introduces minimal bounded memory parameters and enables fast streaming inference for LLMs.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Anthropic researchers wear down AI ethics with repeated questions

How do you get an AI to answer a question it’s not supposed to? There are many such “jailbreak” techniques, and Anthropic researchers just found a new one, in which a large language model (LLM) can be convinced to tell you how to build a bomb if you prime it with a few dozen less-harmful questions first.

They call the approach “many-shot jailbreaking” and have both written a paper about it and also informed their peers in the AI community about it so it can be mitigated.

The vulnerability is a new one, resulting from the increased “context window” of the latest generation of LLMs. This is the amount of data they can hold in what you might call short-term memory, once only a few sentences but now thousands of words and even entire books.

What Anthropic’s researchers found was that these models with large context windows tend to perform better on many tasks if there are lots of examples of that task within the prompt. So if there are lots of trivia questions in the prompt (or priming document, like a big list of trivia that the model has in context), the answers actually get better over time. So a fact that it might have gotten wrong if it was the first question, it may get right if it’s the hundredth question.

But in an unexpected extension of this “in-context learning,” as it’s called, the models also get “better” at replying to inappropriate questions. So if you ask it to build a bomb right away, it will refuse. But if the prompt shows it answering 99 other questions of lesser harmfulness and then asks it to build a bomb … it’s a lot more likely to comply.

( Update : I misunderstood the research initially as actually having the model answer the series of priming prompts, but the questions and answers are written into the prompt itself. This makes more sense, and I’ve updated the post to reflect it.)

Image Credits: Anthropic

Why does this work? No one really understands what goes on in the tangled mess of weights that is an LLM, but clearly there is some mechanism that allows it to home in on what the user wants, as evidenced by the content in the context window or prompt itself. If the user wants trivia, it seems to gradually activate more latent trivia power as you ask dozens of questions. And for whatever reason, the same thing happens with users asking for dozens of inappropriate answers — though you have to supply the answers as well as the questions in order to create the effect.

The team already informed its peers and indeed competitors about this attack, something it hopes will “foster a culture where exploits like this are openly shared among LLM providers and researchers.”

For their own mitigation, they found that although limiting the context window helps, it also has a negative effect on the model’s performance. Can’t have that — so they are working on classifying and contextualizing queries before they go to the model. Of course, that just makes it so you have a different model to fool … but at this stage, goalpost-moving in AI security is to be expected.

Age of AI: Everything you need to know about artificial intelligence