how to find research papers reddit

Reference management. Clean and simple.

The top list of academic search engines

1. Google Scholar

4. science.gov, 5. semantic scholar, 6. baidu scholar, get the most out of academic search engines, frequently asked questions about academic search engines, related articles.

Academic search engines have become the number one resource to turn to in order to find research papers and other scholarly sources. While classic academic databases like Web of Science and Scopus are locked behind paywalls, Google Scholar and others can be accessed free of charge. In order to help you get your research done fast, we have compiled the top list of free academic search engines.

Google Scholar is the clear number one when it comes to academic search engines. It's the power of Google searches applied to research papers and patents. It not only lets you find research papers for all academic disciplines for free but also often provides links to full-text PDF files.

• Coverage: approx. 200 million articles
• Abstracts: only a snippet of the abstract is available
• Related articles: ✔
• References: ✔
• Cited by: ✔
• Links to full text: ✔
• Export formats: APA, MLA, Chicago, Harvard, Vancouver, RIS, BibTeX

BASE is hosted at Bielefeld University in Germany. That is also where its name stems from (Bielefeld Academic Search Engine).

• Coverage: approx. 136 million articles (contains duplicates)
• Abstracts: ✔
• Related articles: ✘
• References: ✘
• Cited by: ✘
• Export formats: RIS, BibTeX

CORE is an academic search engine dedicated to open-access research papers. For each search result, a link to the full-text PDF or full-text web page is provided.

• Coverage: approx. 136 million articles
• Links to full text: ✔ (all articles in CORE are open access)
• Export formats: BibTeX

Science.gov is a fantastic resource as it bundles and offers free access to search results from more than 15 U.S. federal agencies. There is no need anymore to query all those resources separately!

• Coverage: approx. 200 million articles and reports
• Links to full text: ✔ (available for some databases)
• Export formats: APA, MLA, RIS, BibTeX (available for some databases)

Semantic Scholar is the new kid on the block. Its mission is to provide more relevant and impactful search results using AI-powered algorithms that find hidden connections and links between research topics.

• Coverage: approx. 40 million articles
• Export formats: APA, MLA, Chicago, BibTeX

Although Baidu Scholar's interface is in Chinese, its index contains research papers in English as well as Chinese.

• Coverage: no detailed statistics available, approx. 100 million articles
• Abstracts: only snippets of the abstract are available
• Export formats: APA, MLA, RIS, BibTeX

RefSeek searches more than one billion documents from academic and organizational websites. Its clean interface makes it especially easy to use for students and new researchers.

• Coverage: no detailed statistics available, approx. 1 billion documents
• Abstracts: only snippets of the article are available
• Export formats: not available

Consider using a reference manager like Paperpile to save, organize, and cite your references. Paperpile integrates with Google Scholar and many popular databases, so you can save references and PDFs directly to your library using the Paperpile buttons:

Google Scholar is an academic search engine, and it is the clear number one when it comes to academic search engines. It's the power of Google searches applied to research papers and patents. It not only let's you find research papers for all academic disciplines for free, but also often provides links to full text PDF file.

Semantic Scholar is a free, AI-powered research tool for scientific literature developed at the Allen Institute for AI. Sematic Scholar was publicly released in 2015 and uses advances in natural language processing to provide summaries for scholarly papers.

BASE , as its name suggest is an academic search engine. It is hosted at Bielefeld University in Germany and that's where it name stems from (Bielefeld Academic Search Engine).

CORE is an academic search engine dedicated to open access research papers. For each search result a link to the full text PDF or full text web page is provided.

Science.gov is a fantastic resource as it bundles and offers free access to search results from more than 15 U.S. federal agencies. There is no need any more to query all those resources separately!

• Getting Published
• Open Research
• Communicating Research
• Life in Research
• For Editors
• For Peer Reviewers
• Research Integrity

Reddit 101 for Scientists

Author: Penny Freedman

When it comes to utilizing social media in the science community, you might not automatically think about including Reddit into your activities. While Reddit threads may have a reputation for being controversial, there is another side to Reddit that is both important and useful to the scientific world.

What exactly is Reddit?

You can think of Reddit as one giant virtual conference for every discipline and subject you could imagine where people break off into smaller groups -called subreddits- to talk about topics interesting to them. When you first enter Reddit it looks like one large message board. When you register for an account you can post content and vote posts up or down the page, helping to determine what will receive the most attention.

What subreddits do I even begin with?

If you’re interested in a more general discussion on science, start with  http://reddit.com/r/EverythingScience . It’s a place for people to talk about anything and everything having to do with science. You can filter by field, add your thoughts to discussions already taking place, or start a new discussion by submitting a link to something you are interested in – a blog post, video, news article, editorial, etc.

If you’re looking for a more defined discussion on peer-reviewed science, head on over to The New Reddit Journal of Science at  http://reddit.com/r/science . There you may only submit links to published peer-reviewed research. Get the conversation started on your work or a peer’s work!

What is an AMA?

An AMA is short for “Ask Me Anything.” A scientist arranges a time with Reddit moderators to discuss a specific topic related to their research or interests. You submit a brief bio and summary of what you would like to discuss, and the Reddit community is given the chance to submit questions before the AMA start time. There is a submission guide  with detailed information on how to get started with setting one up. An AMA is a great way to get a conversation started on items that are of particular interest to you, and a way to share your expertise with people interested in studying or working in the same field, or just interested in learning something new.

Springer editors and authors have hosted a few AMAs, including:

• An AMA on rare and neglected diseases
• An AMA on American politics
• An AMA on realistic robots

How do I establish myself as a qualified scientist in my field to the Reddit community?

Reddit uses something called flair to designate who is a trained scientist, doctor, or engineer. The flair will present as a small bar next to your user name, noting your title and/or education level (such a Professor of Biology, PhD, etc.). When you add this bit of information people will understand that the comments you provide are knowledgeable and valuable. Once you have created your account reference  these instructions  to get your flair.

How is using Reddit any different than posting on other social media sites?

Reddit gives you the opportunity to share your knowledge and expertise in a more detailed, conversational way. You can find people discussing topics at length that you are interested in and can contribute meaningfully to. Unlike social media platforms that are centered around creating a personalized profile that is all about you, Reddit prides itself on being a community. The things you share should not be overly promotional, but should contribute to the discussion as a whole. Joining the discussion can help serve to expand your network and reach.

Penny Freedman is a Marketing Manager on the Author Experience & Services team based in the New York office. She works closely on sharing insight and guidance on the benefits and services available to our editors, reviewers, and authors.

• social media
• research communication
• Open research
• Tools & Services
• Account Development
• Sales and account contacts
• Professional
• Press office
• Locations & Contact

We are a world leading research, educational and professional publisher. Visit our main website for more information.

• © 2023 Springer Nature
• General terms and conditions
• Your US State Privacy Rights
• Your Privacy Choices / Manage Cookies
• Accessibility
• Legal notice
• Help us to improve this site, send feedback.

• Corrections

Search Help

Get the most out of Google Scholar with some helpful tips on searches, email alerts, citation export, and more.

Finding recent papers

Your search results are normally sorted by relevance, not by date. To find newer articles, try the following options in the left sidebar:

• click "Since Year" to show only recently published papers, sorted by relevance;
• click "Sort by date" to show just the new additions, sorted by date;
• click the envelope icon to have new results periodically delivered by email.

Locating the full text of an article

Abstracts are freely available for most of the articles. Alas, reading the entire article may require a subscription. Here're a few things to try:

• click a library link, e.g., "FindIt@Harvard", to the right of the search result;
• click a link labeled [PDF] to the right of the search result;
• click "All versions" under the search result and check out the alternative sources;
• click "Related articles" or "Cited by" under the search result to explore similar articles.

If you're affiliated with a university, but don't see links such as "FindIt@Harvard", please check with your local library about the best way to access their online subscriptions. You may need to do search from a computer on campus, or to configure your browser to use a library proxy.

Getting better answers

If you're new to the subject, it may be helpful to pick up the terminology from secondary sources. E.g., a Wikipedia article for "overweight" might suggest a Scholar search for "pediatric hyperalimentation".

If the search results are too specific for your needs, check out what they're citing in their "References" sections. Referenced works are often more general in nature.

Similarly, if the search results are too basic for you, click "Cited by" to see newer papers that referenced them. These newer papers will often be more specific.

Explore! There's rarely a single answer to a research question. Click "Related articles" or "Cited by" to see closely related work, or search for author's name and see what else they have written.

Searching Google Scholar

Use the "author:" operator, e.g., author:"d knuth" or author:"donald e knuth".

Put the paper's title in quotations: "A History of the China Sea".

You'll often get better results if you search only recent articles, but still sort them by relevance, not by date. E.g., click "Since 2018" in the left sidebar of the search results page.

To see the absolutely newest articles first, click "Sort by date" in the sidebar. If you use this feature a lot, you may also find it useful to setup email alerts to have new results automatically sent to you.

Note: On smaller screens that don't show the sidebar, these options are available in the dropdown menu labelled "Year" right below the search button.

Select the "Case law" option on the homepage or in the side drawer on the search results page.

It finds documents similar to the given search result.

It's in the side drawer. The advanced search window lets you search in the author, title, and publication fields, as well as limit your search results by date.

Select the "Case law" option and do a keyword search over all jurisdictions. Then, click the "Select courts" link in the left sidebar on the search results page.

Tip: To quickly search a frequently used selection of courts, bookmark a search results page with the desired selection.

Access to articles

For each Scholar search result, we try to find a version of the article that you can read. These access links are labelled [PDF] or [HTML] and appear to the right of the search result. For example:

A paper that you need to read

Access links cover a wide variety of ways in which articles may be available to you - articles that your library subscribes to, open access articles, free-to-read articles from publishers, preprints, articles in repositories, etc.

When you are on a campus network, access links automatically include your library subscriptions and direct you to subscribed versions of articles. On-campus access links cover subscriptions from primary publishers as well as aggregators.

Off-campus access

Off-campus access links let you take your library subscriptions with you when you are at home or traveling. You can read subscribed articles when you are off-campus just as easily as when you are on-campus. Off-campus access links work by recording your subscriptions when you visit Scholar while on-campus, and looking up the recorded subscriptions later when you are off-campus.

We use the recorded subscriptions to provide you with the same subscribed access links as you see on campus. We also indicate your subscription access to participating publishers so that they can allow you to read the full-text of these articles without logging in or using a proxy. The recorded subscription information expires after 30 days and is automatically deleted.

In addition to Google Scholar search results, off-campus access links can also appear on articles from publishers participating in the off-campus subscription access program. Look for links labeled [PDF] or [HTML] on the right hand side of article pages.

Anne Author , John Doe , Jane Smith , Someone Else

In this fascinating paper, we investigate various topics that would be of interest to you. We also describe new methods relevant to your project, and attempt to address several questions which you would also like to know the answer to. Lastly, we analyze …

You can disable off-campus access links on the Scholar settings page . Disabling off-campus access links will turn off recording of your library subscriptions. It will also turn off indicating subscription access to participating publishers. Once off-campus access links are disabled, you may need to identify and configure an alternate mechanism (e.g., an institutional proxy or VPN) to access your library subscriptions while off-campus.

Email Alerts

Do a search for the topic of interest, e.g., "M Theory"; click the envelope icon in the sidebar of the search results page; enter your email address, and click "Create alert". We'll then periodically email you newly published papers that match your search criteria.

No, you can enter any email address of your choice. If the email address isn't a Google account or doesn't match your Google account, then we'll email you a verification link, which you'll need to click to start receiving alerts.

This works best if you create a public profile , which is free and quick to do. Once you get to the homepage with your photo, click "Follow" next to your name, select "New citations to my articles", and click "Done". We will then email you when we find new articles that cite yours.

Search for the title of your paper, e.g., "Anti de Sitter space and holography"; click on the "Cited by" link at the bottom of the search result; and then click on the envelope icon in the left sidebar of the search results page.

First, do a search for your colleague's name, and see if they have a Scholar profile. If they do, click on it, click the "Follow" button next to their name, select "New articles by this author", and click "Done".

If they don't have a profile, do a search by author, e.g., [author:s-hawking], and click on the mighty envelope in the left sidebar of the search results page. If you find that several different people share the same name, you may need to add co-author names or topical keywords to limit results to the author you wish to follow.

We send the alerts right after we add new papers to Google Scholar. This usually happens several times a week, except that our search robots meticulously observe holidays.

There's a link to cancel the alert at the bottom of every notification email.

If you created alerts using a Google account, you can manage them all here . If you're not using a Google account, you'll need to unsubscribe from the individual alerts and subscribe to the new ones.

Google Scholar library

Google Scholar library is your personal collection of articles. You can save articles right off the search page, organize them by adding labels, and use the power of Scholar search to quickly find just the one you want - at any time and from anywhere. You decide what goes into your library, and we’ll keep the links up to date.

You get all the goodies that come with Scholar search results - links to PDF and to your university's subscriptions, formatted citations, citing articles, and more!

Library help

Find the article you want to add in Google Scholar and click the “Save” button under the search result.

Click “My library” at the top of the page or in the side drawer to view all articles in your library. To search the full text of these articles, enter your query as usual in the search box.

Find the article you want to remove, and then click the “Delete” button under it.

• To add a label to an article, find the article in your library, click the “Label” button under it, select the label you want to apply, and click “Done”.
• To view all the articles with a specific label, click the label name in the left sidebar of your library page.
• To remove a label from an article, click the “Label” button under it, deselect the label you want to remove, and click “Done”.
• To add, edit, or delete labels, click “Manage labels” in the left column of your library page.

Only you can see the articles in your library. If you create a Scholar profile and make it public, then the articles in your public profile (and only those articles) will be visible to everyone.

Your profile contains all the articles you have written yourself. It’s a way to present your work to others, as well as to keep track of citations to it. Your library is a way to organize the articles that you’d like to read or cite, not necessarily the ones you’ve written.

Citation Export

Click the "Cite" button under the search result and then select your bibliography manager at the bottom of the popup. We currently support BibTeX, EndNote, RefMan, and RefWorks.

Err, no, please respect our robots.txt when you access Google Scholar using automated software. As the wearers of crawler's shoes and webmaster's hat, we cannot recommend adherence to web standards highly enough.

Sorry, we're unable to provide bulk access. You'll need to make an arrangement directly with the source of the data you're interested in. Keep in mind that a lot of the records in Google Scholar come from commercial subscription services.

Sorry, we can only show up to 1,000 results for any particular search query. Try a different query to get more results.

Content Coverage

Google Scholar includes journal and conference papers, theses and dissertations, academic books, pre-prints, abstracts, technical reports and other scholarly literature from all broad areas of research. You'll find works from a wide variety of academic publishers, professional societies and university repositories, as well as scholarly articles available anywhere across the web. Google Scholar also includes court opinions and patents.

We index research articles and abstracts from most major academic publishers and repositories worldwide, including both free and subscription sources. To check current coverage of a specific source in Google Scholar, search for a sample of their article titles in quotes.

While we try to be comprehensive, it isn't possible to guarantee uninterrupted coverage of any particular source. We index articles from sources all over the web and link to these websites in our search results. If one of these websites becomes unavailable to our search robots or to a large number of web users, we have to remove it from Google Scholar until it becomes available again.

Our meticulous search robots generally try to index every paper from every website they visit, including most major sources and also many lesser known ones.

That said, Google Scholar is primarily a search of academic papers. Shorter articles, such as book reviews, news sections, editorials, announcements and letters, may or may not be included. Untitled documents and documents without authors are usually not included. Website URLs that aren't available to our search robots or to the majority of web users are, obviously, not included either. Nor do we include websites that require you to sign up for an account, install a browser plugin, watch four colorful ads, and turn around three times and say coo-coo before you can read the listing of titles scanned at 10 DPI... You get the idea, we cover academic papers from sensible websites.

That's usually because we index many of these papers from other websites, such as the websites of their primary publishers. The "site:" operator currently only searches the primary version of each paper.

It could also be that the papers are located on examplejournals.gov, not on example.gov. Please make sure you're searching for the "right" website.

That said, the best way to check coverage of a specific source is to search for a sample of their papers using the title of the paper.

Ahem, we index papers, not journals. You should also ask about our coverage of universities, research groups, proteins, seminal breakthroughs, and other dimensions that are of interest to users. All such questions are best answered by searching for a statistical sample of papers that has the property of interest - journal, author, protein, etc. Many coverage comparisons are available if you search for [allintitle:"google scholar"], but some of them are more statistically valid than others.

Currently, Google Scholar allows you to search and read published opinions of US state appellate and supreme court cases since 1950, US federal district, appellate, tax and bankruptcy courts since 1923 and US Supreme Court cases since 1791. In addition, it includes citations for cases cited by indexed opinions or journal articles which allows you to find influential cases (usually older or international) which are not yet online or publicly available.

Legal opinions in Google Scholar are provided for informational purposes only and should not be relied on as a substitute for legal advice from a licensed lawyer. Google does not warrant that the information is complete or accurate.

We normally add new papers several times a week. However, updates to existing records take 6-9 months to a year or longer, because in order to update our records, we need to first recrawl them from the source website. For many larger websites, the speed at which we can update their records is limited by the crawl rate that they allow.

Inclusion and Corrections

We apologize, and we assure you the error was unintentional. Automated extraction of information from articles in diverse fields can be tricky, so an error sometimes sneaks through.

Please write to the owner of the website where the erroneous search result is coming from, and encourage them to provide correct bibliographic data to us, as described in the technical guidelines . Once the data is corrected on their website, it usually takes 6-9 months to a year or longer for it to be updated in Google Scholar. We appreciate your help and your patience.

If you can't find your papers when you search for them by title and by author, please refer your publisher to our technical guidelines .

You can also deposit your papers into your institutional repository or put their PDF versions on your personal website, but please follow your publisher's requirements when you do so. See our technical guidelines for more details on the inclusion process.

We normally add new papers several times a week; however, it might take us some time to crawl larger websites, and corrections to already included papers can take 6-9 months to a year or longer.

Google Scholar generally reflects the state of the web as it is currently visible to our search robots and to the majority of users. When you're searching for relevant papers to read, you wouldn't want it any other way!

If your citation counts have gone down, chances are that either your paper or papers that cite it have either disappeared from the web entirely, or have become unavailable to our search robots, or, perhaps, have been reformatted in a way that made it difficult for our automated software to identify their bibliographic data and references. If you wish to correct this, you'll need to identify the specific documents with indexing problems and ask your publisher to fix them. Please refer to the technical guidelines .

Please do let us know . Please include the URL for the opinion, the corrected information and a source where we can verify the correction.

We're only able to make corrections to court opinions that are hosted on our own website. For corrections to academic papers, books, dissertations and other third-party material, click on the search result in question and contact the owner of the website where the document came from. For corrections to books from Google Book Search, click on the book's title and locate the link to provide feedback at the bottom of the book's page.

General Questions

These are articles which other scholarly articles have referred to, but which we haven't found online. To exclude them from your search results, uncheck the "include citations" box on the left sidebar.

First, click on links labeled [PDF] or [HTML] to the right of the search result's title. Also, check out the "All versions" link at the bottom of the search result.

Second, if you're affiliated with a university, using a computer on campus will often let you access your library's online subscriptions. Look for links labeled with your library's name to the right of the search result's title. Also, see if there's a link to the full text on the publisher's page with the abstract.

Keep in mind that final published versions are often only available to subscribers, and that some articles are not available online at all. Good luck!

Technically, your web browser remembers your settings in a "cookie" on your computer's disk, and sends this cookie to our website along with every search. Check that your browser isn't configured to discard our cookies. Also, check if disabling various proxies or overly helpful privacy settings does the trick. Either way, your settings are stored on your computer, not on our servers, so a long hard look at your browser's preferences or internet options should help cure the machine's forgetfulness.

Not even close. That phrase is our acknowledgement that much of scholarly research involves building on what others have already discovered. It's taken from Sir Isaac Newton's famous quote, "If I have seen further, it is by standing on the shoulders of giants."

• Privacy & Terms

How to find Research Papers: A Cheat Sheet for Graduate Students

• July 23, 2022
• PRODUCTIVITY

“I will read this paper later.” I thought to myself before adding another paper to my overflowing internet browser.

Of course, I didn’t read it later.

Since my workflow was unorganized, I missed out on reading many important papers.

This was a crucial period in my undergraduate career. I had been working with a company for my final year project and knew success would require a solid intellectual foundation. For many hours, I read papers, determined to master the literature in my field.

“How to find research papers quickly?” has been a never-ending question for me.

However, I was unable to succeed despite my best intentions, largely due to inefficiency. In addition, I did not have a system in place for keeping track of new papers being published daily in my topic area or checking if I had missed key studies.

Nothing is worse than forgetting where you saved an important research paper. If I couldn’t find that specific paper, I couldn’t do anything else, and sometimes a day would pass before I found it.

As I was about to begin my PhD, I convinced myself that I should be more organized.

This is the first post of the four-part blog series:  The Bulletproof Literature Management System . Follow the links below to read the other posts in the series:

• How to How to find Research Papers (You are here)
• How to Manage Research Papers
• How to Read Research Papers
• How to Organize Research Papers

My workflow has evolved through many iterations, and I have finally found a system that suits my needs after lots of trial and error.

These tips will help you how to find research papers quickly and more efficiently.

Get recommendations from your supervisor

You may have already received a folder of information from your supervisor regarding your thesis topic. Your supervisor should have already been working on the proposal before you were hired for a funded project.

My supervisor, for example, has a folder named “Literature” for each project folder that contains all the important papers one might need to complete that project.

Therefore, asking your supervisor is one of the most straightforward ways to find research papers.

Even though your supervisor has not put up a folder like that, you can still ask them for recommendations, and they can point out a couple of pertinent articles. From there, you can find the references in the papers they recommended.

Use feed aggregators

Feed aggregators, such as Feedly , Inoreader , and NewsBlur , help me organize my feeds. In the morning, I dedicate five minutes to scanning my feed. For most papers, I just glance at the title and scroll past. Whenever I come across something interesting, I add it to my ‘Read Later’ folder.

Instead of storing papers in an unsecured location, my papers are more secure. As a result, it is much easier for me to look at that folder later on.

Use literature mapping tools

ResearchRabbit , Inciteful , Litmaps , and Connected Papers are literature-mapping tools you can use to dig deeper into a topic. It lets you see which papers are the most groundbreaking in a given field based on their citation networks.

This might not be very helpful if you’re doing research in a relatively new area. Finding relevant research papers in such cases may be more challenging.

This is why checking research databases would be a better option.

Use standard research databases

Scopus has strong searching capabilities and publishes metrics that can measure the relative importance of papers in their fields. However, it may take up to 2 years before an article is included in Scopus.

It has more features for sorting and filtering, so you might not feel overwhelmed when searching.

Therefore, if you are just starting your research, SCOPUS might be an excellent option for finding research papers.

ResearchGate

In addition to traditional searching for publications, ResearchGate offers the following features:

• Follow researchers in your field, so you can keep up with their work.
• Keep up-to-date with the research projects of other researchers by following their research projects, and
• Comment on publications, ask questions, and send direct messages to interact with others.

As most of the comments on ResearchGate are coming from experts in their respective fields, the QnA section may be a great resource for finding the right paper for your research.

An RSS(Really Simple Syndication) feed, as the name implies, is a straightforward solution. By subscribing to RSS, users can access content from specific websites.

You can find RSS feeds for nearly every major journal and preprint server on their home pages – just look for the orange icon. As new articles are added to PubMed or Google Scholar, you can even subscribe to specific keywords.

Use academic textbooks the right way

If you are new to a particular research area, it would be best to start by reading textbooks to understand the topic better.

Despite the lack of depth and detail in a textbook, it can provide you with the basic concepts you need to read further. Furthermore, textbooks often include extensive lists of references as well as this information to get you started . Download the relevant articles from these references.

You might feel overwhelmed if you try to read an academic textbook from beginning to end. For this reason, read only the sections which contain the information you need for your project.

Review papers are game changers

A review paper on your topic is a great starting point for finding good references and getting a broad overview of your research topic.

After reading the review paper, you can read the references cited therein.

You are reading a much more comprehensive summary of the topic than you would have found reading ten individual research papers on the same topic if you found a highly relevant review paper for your research.

Look for technical reports and theses

Make sure you don’t limit yourself to research papers when looking for references. A technical report or code document on your topic may contain important citations (as well as practical information).

There is nothing that compares to a PhD thesis when it comes to the depth and extent of analytical work. See which references students have cited in their theses on your topic.

If you find a relevant thesis for your literature review, you will have extensive information about the research topic in one place, saving you a ton of time.

Google Scholar

The best for the last!

Due to its versatility and efficiency in finding academic papers, I decided to include Google Scholar separately from the database section.

I enjoy using Google Scholar among all the fancy databases available. One drawback to Google Scholar is that it lacks the ability to search for keywords and filter results.

Therefore, if you are just starting your research and aren’t sure what “keywords” to search for, Google Scholar might not be your first choice.

The advantage of Google Scholar is that if you are already familiar with your field of study and already know what you are doing, you will be able to find relevant research papers more quickly.

Use Google Scholar’s search function to locate relevant articles. Furthermore, you can subscribe to updates from colleagues in your field to access the latest references. The publisher of a journal paper may also report an article faster to Google Scholar than another database, which can take up to two years to include an article.

Images courtesy: Internet marketing vector created by jcomp – www.freepik.com

Aruna Kumarasiri

Founder at Proactive Grad, Materials Engineer, Researcher, and turned author. In 2019, he started his professional carrier as a materials engineer with the continuation of his research studies. His exposure to both academic and industrial worlds has provided many opportunities for him to give back to young professionals.

Did You Enjoy This?

Then consider getting the ProactiveGrad newsletter. It's a collection of useful ideas, fresh links, and high-spirited shenanigans delivered to your inbox every two weeks.

I accept the Privacy Policy

Hand-picked related articles

Why do graduate students struggle to establish a productive morning routine? And how to handle it?

• March 17, 2024

How to stick to a schedule as a graduate student?

• October 10, 2023

The best note-taking apps for graduate students: How to choose the right note-taking app

• September 20, 2022

Leave a Reply Cancel Reply

Your email address will not be published. Required fields are marked *

Name  *

Email  *

Add Comment  *

Notify me of follow-up comments by email.

Notify me of new posts by email.

Post Comment

• Follow us on Facebook
• Follow us on Twitter
• Criminal Justice
• Environment
• Politics & Government
• Race & Gender

Expert Commentary

How to find an academic research paper

Looking for research on a particular topic? We’ll walk you through the steps we use here at Journalist's Resource.

Republish this article

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License .

by David Trilling, The Journalist's Resource October 18, 2017

This <a target="_blank" href="https://journalistsresource.org/home/find-academic-research-paper-for-journalists/">article</a> first appeared on <a target="_blank" href="https://journalistsresource.org">The Journalist's Resource</a> and is republished here under a Creative Commons license.<img src="https://journalistsresource.org/wp-content/uploads/2020/11/cropped-jr-favicon-150x150.png" style="width:1em;height:1em;margin-left:10px;">

Journalists frequently contact us looking for research on a specific topic. While we have published a number of resources on how to understand an academic study and how to pick a good one — and why using social science research enriches journalism and public debate — we have little on the mechanics of how to search. This tip sheet will briefly discuss the resources we use.

Google Scholar

Let’s say we’re looking for papers on the opioid crisis. We often start with Google Scholar, a free service from Google that searches scholarly articles, books and documents rather than the entire web: scholar.google.com .

But a search for the keyword “opioids” returns almost half a million results, some from the 1980s. Let’s narrow down our search. On the left, you see options “anytime” (the default), “since 2013,” “since 2016,” etc. Try “since 2017” and the results are now about 17,000. You can also insert a custom range to search for specific years. And you can include patents or citations, if you like (unchecking these will slightly decrease the number of results).

Still too many results. To narrow the search further, try any trick you’d use with Google. (Here are some tips from MIT on how to supercharge your Google searches.) Let’s look for papers on opioids published in 2015 that look at race and exclude fentanyl (Google: “opioids +race -fentanyl”). Now we’re down to 2,750 results. Better.

Unless you tell Google to “sort by date,” the search engine will generally weight the papers that have been cited most often so you will see them first.

Try different keywords. If you’re looking for a paper that studies existing research, include the term “meta-analysis.” Try searching by the author’s name, if you know it, or title of the paper. Look at the endnotes in papers you like for other papers. And look at the papers that cited the paper you like; they’ll probably be useful for your project.

If you locate a study and it’s behind a paywall, try these steps:

• Click on “all versions.” Some may be available for free. (Though check the date, as this may include earlier drafts of a paper.)
• Reach out to the journal and the scholar. (The scholar’s email is often on the abstract page. Also, scholars generally have an easy-to-find webpage.) One is likely to give you a free copy of the paper, especially if you are a member of the press.
• In regular Google, search for the study by title and you might find a free version.

More tips on using Google Scholar from MIT and Google .

Other databases

• PubMed Central at the National Library of Medicine: If you are working on a topic that has a relationship to health, try this database run by the National Institutes of Health. This free site hosts articles or abstracts and links to free versions of a paper if they are available. Often Google Scholar will point you here.
• If you have online access to a university library or a local library, try that.
• Directory of Open Access Journals .
• Digital Public Library of America .
• Subscription services include org and Web of Science .

For more on efforts to make scholarly research open and accessible for all, check out SPARC , a coalition of university libraries.

Citations as a measure of impact

How do you know if a paper is impactful? Some scholars use the number of times the paper has been cited by other scholars. But that can be problematic: Some papers cite papers that are flawed simply to debunk them. Some topics will be cited more often than others. And new research, even if it’s high-quality, may not be cited yet.

The impact factor measures how frequently a journal, not a paper, is cited.

This guide from the University of Illinois, Chicago, has more on metrics.

Here’s a useful source of new papers curated by Boston Globe columnist Kevin Lewis for National Affairs.

Another way to monitor journals for new research is to set up an RSS reader like Feedly . Most journals have a media page where you can sign up for press releases or newsletters featuring the latest research.

Relevant tip sheets from Journalist’s Resource:

• 10 things we wish we’d known earlier about research
• How to tell good research from bad: 13 questions journalists should ask  (This post also discusses how to determine if a journal is good.)
• Lessons on online search techniques, reading studies, understanding data and methods
• Guide to critical thinking, research, data and theory: Overview for journalists

About The Author

David Trilling

Social Media Research

• Considerations when researching social media
• Collected tools
• Additional resources

Reddit TOS and API

• Reddit's TOS The Reddit TOS is more permissive of research use than Meta's platforms, partially by omission. It does not directly address research on Reddit, but it does allow for automated capturing of posts via the API.
• Reddit's API documentation The Reddit API exposes most of the site's content to automated collection. Some rules are linked on this page as well, which are fairly straightforward and permissive to research uses.

Reddit's TOS and API rules do not contain the sort of blanket bans on automated data collection that Meta's TOSes do, but they do not contain any specific provisions for research use either. Like Twitter, the API helps collect posts as they happen rather than archiving all posts on the site.

Tools for Reddit research

• Netlytic Netlytic is a browser-based social media research tool that has text mining and network visualization features. Works with Twitter, YouTube, RSS feeds, and Reddit. Free accounts are sufficient for most student purposes. Netlytic has a YouTube channel with demonstrations for a variety of types of project.
• Mozdeh Mozdeh is a social media quantitative analysis FOSS software that can also collect tweets, like Netlytic or Chorus. It works with the same things as Netlyltic: Tweets, YouTube comments, Reddit comments, and manually imported data. Unlike Netlytic, it is a desktop app. It also has a YouTube channel where you can find guides to collecting and analyzing data.
• Reaper Reaper, built on the socialreaper Python library, is a desktop app with no coding required. While it calls what it does "scraping", it makes use of site APIs and the user will need to register for an API key for any site they want to use Reaper on. This includes Facebook, Twitter, Reddit, YouTube, Tumblr, and Pinterest. It outputs all data as .csv tabular files.
• PRAW: the Python Reddit API Wrapper PRAW is a Python library for working with the Reddit API.
• 4CAT 4CAT is a relatively advanced tool for the collection and analysis of social media data - it's best run on a UNIX server and has dependencies that it does not automatically install itself - but with the upside that it has modules built to work with important but niche platforms like 4chan, 8kun, Parler, and more, as well as Twitter and Reddit.
• pushshift.io Pushshift is a popular wrapper for the Reddit API used with the requests package in Python. Documentation on pushshift.io is there, but tutorials must be found elsewhere.
• Here's one tutorial on how to use the pushshift.io wrapper in Python.

Example publications in Reddit research

• Using Data from Reddit, Public Deliberation, and Surveys to Measure Public Opinion about Autonomous Vehicles ABSTRACT: When and how can researchers synthesize survey data with analyses of social media content to study public opinion, and when and how can social media data complement surveys to better inform researchers and policymakers? This paper explores how public opinions might differ between survey and social media platforms in terms of content and audience, focusing on the test case of opinions about autonomous vehicles. more... less... The paper first extends previous overviews comparing surveys and social media as measurement tools to include a broader range of survey types, including surveys that result from public deliberation, considering the dialogic characteristics of different social media, and the range of issue publics and marginalized voices that different surveys and social media forums can attract. It then compares findings and implications from analyses of public opinion about autonomous vehicles from traditional surveys, results of public deliberation, and analyses of Reddit posts, applying a newly developed computational text analysis tool. Findings demonstrate that social media analyses can both help researchers learn more about issues that are uncovered by surveys and also uncover opinions from subpopulations with specialized knowledge and unique orientations toward a subject. In light of these findings, we point to future directions on how researchers and policymakers can synthesize survey and social media data, and the corresponding data integration techniques, to study public opinion.
• Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics ABSTRACT: This article offers a systematic analysis of 727 manuscripts that used Reddit as a data source, published between 2010 and 2020. Our analysis reveals the increasing growth in use of Reddit as a data source, the range of disciplines this research is occurring in, how researchers are getting access to Reddit data, the characteristics of the datasets researchers are using, the subreddits and topics being studied, the kinds of analysis and methods researchers are engaging in, and the emerging ethical questions of research in this space. more... less... We discuss how researchers need to consider the impact of Reddit’s algorithms, affordances, and generalizability of the scientific knowledge produced using Reddit data, as well as the potential ethical dimensions of research that draws data from subreddits with potentially sensitive populations.
• << Previous: Facebook
• Next: YouTube >>
• Last Updated: Dec 12, 2023 12:23 PM
• URL: https://subjectguides.library.american.edu/socialmediaresearch

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Disguising Reddit sources and the efficacy of ethical research

Joseph reagle.

Communication Studies, Northeastern University, 215 Holmes Hall, 43 Leon St, 02115 Boston, MA USA

Associated Data

Not applicable. Research data are: (1) research reports, (2) phrases from those reports taken from Reddit, and (3) interviews with the authors of the reports. The first two datasets are confidential so as not to embarrass researchers. The third set is confidential because they were obtained via a consent form that sated: “The confidentiality of your research records will be strictly maintained. Records of our discussion will be (1) kept separate from this consent form, (2) not shared with others, and (3) kept on an encrypted file system.”

Not applicable.

Concerned researchers of online forums might implement what Bruckman ( 2002 ) referred to as disguise . Heavy disguise, for example, elides usernames and rewords quoted prose so that sources are difficult to locate via search engines. This can protect users (who might be members of vulnerable populations, including minors) from additional harms (such as harassment or additional identification). But does disguise work? I analyze 22 Reddit research reports: 3 of light disguise, using verbatim quotes, and 19 of heavier disguise, using reworded phrases. I test if their sources can be located via three different search services (i.e., Reddit, Google, and RedditSearch). I also interview 10 of the reports’ authors about their sourcing practices, influences, and experiences. Disguising sources is effective only if done and tested rigorously; I was able to locate all of the verbatim sources (3/3) and many of the reworded sources (11/19). There is a lack of understanding, among users and researchers, about how online messages can be located, especially after deletion. Researchers should conduct similar site-specific investigations and develop practical guidelines and tools for improving the ethical use of online sources.

Introduction

Reddit is known as the “front page of the web,” claiming “52 M + daily active users” and “100K + communities” (Reddit, 2021 ). Millions of Redditors, including minors and other vulnerable populations, have thousands of subreddits to discuss extraordinarily specific and sometimes sensitive topics, including sexuality, health, violence, and drug use.

Given the public prominence, breadth, and depth of Reddit’s content, researchers use it as a data source. Proferes et al., ( 2021 ) identified 727 such studies published between 2010 and 2020-May. They found that only 2.5% of their studies claimed to paraphrase compared to the 28.5% of the studies that used exact quotes. Researchers who do paraphrase write of limiting the locatability of sources and possible consequent harm. (The studied reports are not quoted or cited directly, see the “Ethics” section below.) I am not aware of cases, fortunately, of online users coming to harm because of information in research reports. That doesn’t mean it hasn’t happened, and given the sensitivity of topics and vulnerability of sources, additional scrutiny could be consequential to users’ health, relationships, employment, and legal standing. (For a more general discussion of potential harm, sensitive topics, and sources, see Franzke et al., ( 2020 ), § 3.2.5.) Additionally, I note that users need not be personally identified to feel embarrassed, to be harassed, or to be forced to abandon a long-held pseudonym. And researchers themselves, whose use of public sources is thought to be outside of human subjects review, might face embarrassment or repercussions nonetheless if a source complains. Paraphrasing sources’ prose might mitigate such outcomes.

Verbatim quoting and paraphrasing are two practices within a spectrum of what Bruckman ( 2002 ) identified as disguise , which can range from none to heavy. Disguise can also include altering usernames and the context of a message, such as the time and forum of posting.

I analyze 22 Reddit research reports: 3 of light disguise, using verbatim quotes, and 19 of heavier disguise, claiming to reword phrases. I test if their sources can be located via three different search indexes (i.e., Google, Reddit, and RedditSearch). I was able to locate all of the verbatim sources (3/3) and many of the reworded sources (11/19). I also interview 10 of the reports’ authors about their sourcing practices, influences, and experiences. These conversations reveal that there is a lack of coherent practice and guidance on effective disguise, the importance of search and archival affordances, and how errors can arise amidst multi-author collaborations and during the review and revision process. Most importantly, these interviews identify exemplary practices, such as researchers testing their own disguises.

The present work does not address whether researchers should disguise their sources. This decision depends on the type of research, the sensitivity of the topic, the vulnerability of the sources, and the attributes of the venue. Rather, my concern is empirical: when researchers choose to employ disguise, does it work? And what, then, can we do to improve the practice?

Background and terminology

Reddit and sensitive topics.

Reddit was founded in June 2005 as a pseudonymous-friendly site for users to share and vote for links they had read (i.e., “I read it.”) Reddit’s development as a forum of forums, where users could trivially create subreddits, each with its. own moderators, led the website to succeed over its link-sharing peers.

Like Twitter and Wikipedia, Reddit serves an extraordinary corpus of mostly public data. That is, while there are private and quarantined subreddits, the vast majority of content is public : transparently accessible to any web browser or search engine. More so than Wikipedia and much of Twitter, Reddit hosts discussions of a personal character. Subreddits on sexuality, health (including mental health and eating disorders), interpersonal abuse and violence, and drug use and cessation have been topics of research. Reddit is a compelling and accessible venue, but with sensitive – even if public – information.

Ethics and Online Research

The practice of online research has been accompanied by discussion of how to do so ethically (Eysenbach & Till, 2001 ; Flicker et al., 2004 ; Mann & Stuart, 2000 ; Reid, 1996 ; Smith & Murray, 2001 ; Waskul & Douglas, 1996 ). And the issues noted by Siang ( 1999 ) over two decades ago remain salient today: of “the blurred distinction between public and private domains,” “the ease of anonymity or pseudonymity,” “the suspension of temporal barriers with the recording and archiving of communications,” and “the relatively low cost, easy to use, technological means to facilitate tracking of participants.”

An intuitive approach to these early concerns was to apply existing research guidelines to the online context, such as those from the American Psychological Association (King 1996 ) and other disciplinary and national societies. At the same time, the Association of Internet Researchers (AoIR) was constituted as a transdisciplinary endeavor, which created and maintains guidelines for online research (Ess & Committee 2002 ; Franzke et al., 2020 ).

Institutional review boards (IRBs) can also be a source of guidance and review. Like much of the disciplinary guidelines, however, their focus tends to be on human subjects research, where there is an interaction between researchers and subjects. Most Reddit research (86.1%) makes no mention of “IRB” or “ethics review.” Of those that do, the majority (77.2%) note an exempt status, though it’s unclear if this is “an official designation given by a review board or whether the authors made this judgment themselves” (Proferes et al., 2021 , p. 14).

What is clear is that there is no widely accepted and consistent practice when it comes to reporting excerpts of public content. Systemic literature reviews show this (Ayers et al., 2018 ; Proferes et al., 2021 ), as will the present work. For those concerned with disguising public data, there’s little guidance on how to do so effectively.

Online Sites, subjects, and sources

The incoherent approach to public data is related to a lack of agreement about terminology and substance. What should researchers call those whom they research online? I distinguish between subjects , those with whom researchers interact, and sources , authors of public content with whom researchers do not interact. (Recall that I use public to mean content that is transparently accessible to any web browser or search engine.) There is also the question of Reddit terminology. Following the architecture of Reddit, I distinguish between posts and their subsequent comments within a thread . I refer to posts and comments, generically, as messages .

Beyond terminology, what should researchers do? There is substantive disagreement compounded by different understandings of terms. Sharf ( 1999 , p. 253), for example, argued that researchers should contact public sources “in order to seek consent” and “implied consent should not be presumed if the writer does not respond.” Rodham & Gavin ( 2006 ) responded “that this is an unnecessarily extreme position to take” and wrote, “messages which are posted on such open forums are public acts, deliberately intended for public consumption.” Presently, I analyze published research reports and seek their sources, without consent. Unlike Sharf’s study of a breast cancer email list (“public” because the list is “open” for anyone to join), published reports are closer to Rodham and Gavin’s sense of the term (i.e., “intended for public consumption”).

Increasingly, researchers are engaging in site-specific considerations, which requires contextual ethical reasoning, be it at Wikipedia (Pentzold 2017 ), at sites where “we are studying people who deserve credit for their work” (Bruckman et al., 2015 ), or public sites where people, nonetheless, discuss sensitive topics or share images (Andalibi et al., 2017 ; Ayers et al., 2018 ; Chen et al., 2021 ; Dym & Fiesler, 2020 ; Haimson et al., 2016 ). For example, on Twitter, Fiesler & Proferes ( 2018 ) found that “few users were previously aware that their public tweets could be used by researchers, and the majority felt that researchers should not be able to use tweets without consent. However, we find that these attitudes are highly contextual, depending on factors such as how the research is conducted or disseminated, who is conducting it, and what the study is about.” Additionally, as I will show, specific websites have affordances that affect how sources can be located (e.g., novel search capabilities or external archives).

De-Identifying, Anonymizing, fabricating, and Disguising

Researchers who attempt disguise note that their sources might be struggling with health, sexuality, or drug use, and additional scrutiny might cause them harm. For the reasons that follow, I speak of disguising public sources to prevent them from being located .

Bruckman ( 2002 ) identified a spectrum of disguise, from none to heavy. Under light disguise, for example, “an outsider could probably figure out who is who with a little investigation.” The forum is named, usernames and other details are changed, yet “verbatim quotes may be used, even if they could be used to identify an individual.” Under heavy disguise , some false details are introduced and verbatim quotes are avoided if a “search mechanism could link those quotes to the person in question.” If the heavy disguise is successful, “someone deliberately seeking to find a subject’s identity would likely be unable to do so.” Moderate disguise is “a compromise position … incorporating some features of light disguise and some of heavy disguise, as appropriate to the situation.” Kozinets ( 2015 , p. 3473) adopted this notion in his discussion of ethical netnography though he used the term cloaking “to emphasize the researcher’s protective actions rather than the state of the participant.” This is a good point, but disguise is the more common term in the literature.

In commercial contexts, enterprises use sanitization to remove sensitive information such as “credit card numbers, email addresses and Social Security Number (SSN)” (Nguyen & Cavallari 2020 , pp. 37–38). In human subjects research, such as healthcare, de-identification “involves the removal of personally identifying information in order to protect personal privacy” (Guidelines for Data de-Identification or Anonymization, 2015). Anonymized is sometimes used synonymously with de-identified, or can have a stronger connotation of data being rendered incapable of being re-identified. I avoid anonymized because it is far too an assured word given the known cases of failure (Ohm 2010 ). And in public data contexts, there might not be personally identifiable information to speak of given the use of pseudonyms. Even so, users need not be personally identified to feel exposed or embarrassed, to be harassed, or to be forced to abandon a long-held pseudonym.

Introducing false or combined details about a source has been referred to as fabrication , a tactic of heavy disguise. The practice is not limited to prose and can include visual content, such as a profile picture in a screenshot (Haimson et al., 2016 ). This practice can conflict with traditional notions of research rigor and integrity. Markham ( 2012 ) argues that if done with care, fabrication can be the most ethical approach. If not done with care, however, fabrication can lead to suspicions of fraud (Singal, 2016 ).

UnGoogling has been used for “obscuring published data and analysis from index, search, and retrieval for ethical purposes” (Shklovski & Vertesi, 2013 , p. 2172). And obfuscating has been used to speak of the “deliberate addition of ambiguous, confusing, or misleading information to interfere with surveillance and data collection” (Brunton & Nissenbaum 2015 , p. 1). UnGoogling is too service-specific and is more often used to describe users removing themselves from the Google ecosystem, such as abandoning Android for iOS. Obfuscation ’s most common use is to describe users protecting their privacy rather than as a research practice.

I examine research reports who disguise their public sources to keep them from being located .

Locating sources

What of the substance, the process, of locating research subjects or sources? Sometimes the ethnographic subjects themselves, of a small town, for example, can recognize themselves and their neighbors. Sometimes real-world events, such as the occurrence of a murder, provide a clue to the public (Reyes, 2017 , n. 9; Singal, 2015 ). And when a researcher from a top-tier New England university describes their research using undergrads from a top-tier New England university, the subjects are probably their students. Online, messages’ prose style (Narayanan et al., 2012 ), timing (Johansson et al., 2015 ), and network relationships (Zhou et al., 2016 ) serve as digital fingerprints (Brown & Abramson, 2015 ), amendable to digital forensics (Guarino, 2013 ), which can lead to online accounts and even personal identities being linked together (Backes et al., 2016 ). For example, Narayanan & Shmatikov ( 2009 ) were able to re-identify a third of users in their “anonymous” Twitter graph who also had a Flickr account “with only a 12% error rate … even when the overlap between the target network and the adversary’s auxiliary information is small.”

As far back as the 1990s, King ( 1996 ) faulted Finn & Lavitt ( 1994 ) for disguising sources’ names, but not that of the sexual abuse forum or the date and time of posts. More recently, Zimmer ( 2010 ) critiqued researchers from a top-tier New England university for creating a “Tastes, Ties, and Time” Facebook dataset that was improperly – perhaps impossibly – “anonymized.” The data was obtained by Harvard Resident Advisors acting as research assistants and scraping their Facebook friends lists. And once the school and cohort were known, other aspects of students’ tastes, ties, and activity made re-identification possible (e.g., being the only student from a specific country in the dataset). Journalists, too, sometimes participate. At the New York Times , Barbaro & Zeller ( 2006 ) reported on – and confirmed – the potential to locate sources in an AOL dataset. A decade later, in the same newspaper, Singer ( 2015 ), wanting to speak to a source in a research study, was able to identify, contact, and interview the subject.

Concerned researchers have started to assess how often usernames, quotations, and media are included in research reports. Ayers et al., ( 2018 ) analyzed 112 health-related papers discussing Twitter and found 72% quoted a tweet, “of these, we identified at least one quoted account holder, representing 84%.” When usernames were disclosed, in 21% of the papers, all were trivially located. Ayers et al. wrote that these practices violate International Committee of Medical Journal Editors (ICMJE) ethics standards because (1) Twitter users might protect or delete messages after collection, and (2) revealing this information has no scientific value.

Proferes et al., ( 2021 ) performed a systematic overview of 727 research studies that used Reddit data and were published between 2010 and 2020-May. They found “Sixty eight manuscripts (9.4%) explicitly mentioned identifiable Reddit usernames in their paper and 659 (90.7%) did not. Two hundred and seven papers (28.5%) used direct quotes from users as part of their publications, 18 papers used paraphrased quotes, noting they were paraphrased (2.5%) and 502 (69.1%) did not include direct quotes” (Proferes et al., 2021 , p. 14).

I make no claim as to whether sources should be disguised. Rather, I ask if a researcher chooses disguise, does it work? Can the original message used by a researcher be located? If so, the full message, associated username, and context (i.e., subreddit, thread, and posting history) are then available. This, itself, could be revealing or linked with other information, including personally-identifying information.

I collected two sets of research reports. In 2020, I sought ethnographic-type research reports since 2015 that included Reddit messages. I searched via Google using keywords such as “AoIR guidelines,” “privacy,” “verbatim,” and “fabrication.” I found three reports using light-disguise with verbatim phrases and three claiming heavier disguise with reworded phrases. In 2021, as part of a panel proposal, I discussed this work with two of the authors of a systematic review of Reddit (Proferes et al., 2021 ), and they kindly shared their list of reports that “paraphrased” Reddit messages, adding 16 new reports to my initial set. Because paraphrase can connote significant change, I use the term reword , which can be as minimal as inserting an adjective or altering a place or name. The final corpus, then, included 22 reports, with 19 claiming to reword.

From each report, I collected phrases of more than ten words because any less than that is too short for meaningful searches. I excluded phrases from subreddit documentation such as sidebars, wikis, or FAQs; these have multiple authors and are informative rather than personal disclosures.

The process of locating Redditors’ original messages was idiosyncratic: intensive, manual, and subjective. I performed exact searches (using quotation marks) and inexact searches across the whole phrase and fragments of novel-seeming prose. Near the end of this work, and hoping to share a method of scrambling phrases and testing disguises, I wrote a script that automated the invocation and opening of search query results (Reagle & Gaur 2022 ). Even so, I had to use discretion with how many search results to review, usually no more than the first page or 20 results – each search service returns results differently. I made no effort to personally identify Reddit users. However, locating sources, as I attempted, could be the first step in the distinct process of identifying users.

After my initial analysis, I emailed the research reports’ authors and asked if they would speak with me. If so, and they completed the consent form, I began with five questions about their practice, rationale, influences, and thoughts about my efforts. We worked to identify weaknesses to avoid and strengths to emulate as part of research and publication. One interview was a ~ 30-minute voice communication, others were via email exchanges with each subject. Interviewees were allowed to review my characterization of their work and our discussion in this report.

Though I used public research reports and their own Reddit sources in my analysis, they are not identified, cited, or quoted. I wanted candid interviews with researchers free of possible embarrassment. I hope that “someone deliberately seeking to find a [subject’s or source’s] identity would likely be unable to do so” (Bruckman 2002 ). That said, other Reddit researchers who are conversant with the literature could make guesses about the identity of research sources. Should this happen, I believe my sources have plausible deniability.

This method was specified as part of Institutional Review Board application #20-08-30 and “approved” as DHHS Review Category #2: “Exempt… No further action or IRB oversight is required as long as the project remains the same.”

Analysis and discussion

Table  1 describes the reports’ approaches to phrases, number of sources, and how many were located. The rightmost column has strengths (bold Ⓢ) to emulate and weaknesses (Ⓦ) to avoid in creating effective disguise relative to reports’ stated policy, actual practice, and ease of location. Importantly, all reports articulated a policy of disguise in their approach to sources, even if weak (i.e., removed usernames but included verbatim quotes).

Research reports and results (“i” = interview)

Searching reddit and the meaning of deletion

Authors V1 and V2 both relied on the fact that Redditors are typically pseudonymous. They included verbatim quotes without the authors’ usernames (i.e., light disguise).

V2 claimed that because pseudonyms are encouraged, the quoted Redditors could not be traced. This claim is highly probable, but digital forensics can sometimes link pseudonyms with other identities, especially as it is easy to peruse all of a user’s posts. Additionally, users who maintain multiple accounts can mistakenly post a message with the wrong account. Even though such users can edit or delete mistaken messages, it’s likely the original will survive elsewhere.

V1 was more cognizant of these concerns and stated they only used posts wherein Redditors explicitly declared they were using a throwaway (single-use) account. However, oddly, V1 did include verbatim quotes from a few Redditors who wrote why they chose not to use a throwaway . A researcher might inadvertently collect posts with the term “throwaway” even if the Redditor was explaining why they did not do so.

The research reports of V1 and V2 each had about twenty phrases (of ten or more words), and I was able to locate almost all of them using three indexes of Reddit content.

Table  2 represents the relative usefulness of the three search services across all 22 research reports. Oddly, Google under-performed (“†”) in verbatim searches because it did not return any of V1’s 18 sources from Reddit. Google’s search algorithms are opaque and ever-changing, so I do not know why it missed these posts, but they could become locatable in the future. Indeed, much could change, and search engines’ capabilities are likely to improve. When removing V1 from the calculation, Google’s verbatim rate is 45%.

Percent of sources found (non-exclusively) at service

RedditSearch (using the Pushshift service) was the most generative search engine because it permits accurate time and subreddit searches. In practice, winnowing away misses is as important as roughly matching hits. It also returned some posts that had since been deleted by their authors, including from V1’s users who did not use throwaways – and perhaps regretted that decision and deleted their posts. Similarly, I was able to locate phrases from deleted posts in the reports of V1, R6, R14, and R18.

The deletion of messages by Redditors suggests that users can feel exposed even when using pseudonymous or throwaway accounts. Users should appreciate that deleted messages on Reddit can be archived and indexed off-site. Researchers should appreciate that they could inadvertently publicize such messages.

Additionally, the Pushshift data originally contained public and private subreddits (determined by moderators) and can include quarantined subreddits (determined by Reddit for problematic but not yet banned) (Stuck_In_the_Matrix, 2019 , 2015 ). Pushshift data has also been packaged in common “big data” frameworks, permitting even more powerful queries and analysis. For example, BigQuery (Balamuta, 2018 ) was used by R5, R6, and R17; ConvoKit ( 2018 ) was used by R9. Locating sources via these resources would add additional capabilities beyond the human-facing searching engines I limited myself to.

Making mistakes and the need for a system

V3 argued that because the site is premised on Redditors competing for upvoted visibility, the site can be taken as public. Even so, V3 elided all usernames, except for two central characters in their report. They quoted phrases from a couple of posts and a handful of comments. This made it easy to find their sources. I was also able to (redundantly) find a post by looking for V3’s description of a meme via a Google image search.

Upon reading V3’s report I was confused by the positioning of Redditors as authors deserving credit in a public venue (hence no consent was obtained), yet, also of a need to elide most Redditors’ usernames (while quoting their prose verbatim). V3 responded that the approach to sources and its description changed during the reports’ review and editing: “originally each of the pseudonyms was formally cited, but this was removed in an earlier stage of peer review.” The confusion in the description was the likely result of this change “and not picked up during the copy-editing stage of the journal.”

R12 also reflected on the likely cause of mistakenly including verbatim phrases. Because of the massive size of their data, “we only paraphrase those we would actually use in the paper.” The process of managing the manuscript and sources then became a problem: “We initially inserted the original quotes into the draft and did one round of paraphrasing. But writing was an iterative process, especially when review & resubmit was involved, during which we might switch in and out quotes as appropriate.” Having multiple authors work on this no doubt contributed: “We probably thought one person on the research team did the paraphrasing.”

Similarly, R16 intended to change all the quotes and believed they had: “I obviously didn’t do a thorough job at it, and I don’t know why – was I aiming to keep the authenticity of the quotes, or was I simply running out of time and did not work diligently? Probably both.” Ethical disguise had been at the forefront of their mind at the start, but perhaps not later: “Was I weighing up the risk of [sources] being identified in this context of technologies used by parents? Certainly in my ethics application, but probably not as much in the reporting.” R5, R6, R10, and R19 similarly included verbatim phrases contrary to their stated policy, perhaps because of similar reasons as the researchers above.

Balancing Fidelity and disguise

Many of the interviewees spoke of the challenge of balancing fidelity to sources’ sentiments with the ethical concern of limiting sources’ exposure.

With respect to identities, V3 shared that “The intention here was to not explicitly name Redditors (using their pseudonym) unless there was a reason to do so.” That is, “My ethical practice defaulted to anonymity, but when necessary for the discussion I used the pseudonyms that the user provides to the public forum.” Two prominent Redditors “are named because of how identifiable their content is and how widely it has been shared across platforms (including sites such as KnowYourMeme).” Additionally, one “username itself was worthy of comment as a signifier of geek humor.” And, once published, the “study gave them significant appreciated notoriety on Reddit and beyond,” something they welcomed.

With respect to verbatim phrases, V3 recognized that phrases can be searched for. However, “What you can find this way is a user’s publicly available (shared) content and pseudonym, not their ‘real name’.” In any case, “As researchers we understand that ethics is a process, not something that is achieved once it is rubberstamped by an institution.” As part of V3’s process, “I considered the trade-off between potential tracking back to a pseudonym and fair representation. The expectation of users, popularity of content, and lack of real names also fed into this calculation.”

R2 attempted to disguise sources and this was a shift in practice from earlier work, where they “included the usernames and preserved quotations.” The earlier work had been influenced by an AoIR presentation about a site wherein the Redditors saw themselves as creative developers worthy of and preferring attribution. “And, because I believed part of my argument about Reddit hinged on the creative play that Redditors engaged in, I wanted to preserve usernames (as this is one of the places where this kind of play occurred).” However, “given the nature of the online sphere these days (this was pre-Gamergate), I would likely not have made the same choice.”

Additionally, the “AOIR guidelines have been hugely influential” in R2’s practice. The guidelines respect that research practices “are not one-and-done decisions, but that things like anonymizing [online] identities/quotes are ongoing decisions that we need to make. IRB guidelines are pretty much worthless in this regard, as they would consider any public forum ‘public’ and their understanding of how easy it is to find out information based on usernames or quotes is limited in my experience.”

The AoIR guidelines were influential to R3, R16, and R18 as well. Other noted influences included boyd ( 2007 ), Kozinets ( 2015 ), and especially Markham ( 2012 ).

Changing practice and changing context

Unlike R2’s past work, in their present report, usernames were elided and phrases from posts and comments were lightly reworded. Though Google can be astoundingly good at finding similar phrases when the field is sufficiently narrow, the modest rewording was sufficient to frustrate my efforts with Reddit, Google, and RedditSearch. However, those messages appeared in threads whose titles were included verbatim in the report, and this leaked information was useful in locating sources. Once in the right thread, it was trivial to locate phrases from the report. Not only did verbatim titles become avenues for locating messages, but they can also be sensitive disclosures.

Like V3, R2 changed their level of disguise during the report’s review: “This piece was a content analysis, and so in my first draft of the article I actually preserved this material as-is, because I wanted to be transparent and make my study potentially replicable.” However, reviewers found this to be problematic because it could open the Redditors to trolling. R2’s forums were not sensitive per se : “I would absolutely have issues with someone using usernames/direct quotes from a health or relationship subreddit for obvious reasons.” Yet, personal disclosures were made in the studied forums and its users are sometimes targets of harassment. R2 agreed with the concern and altered the quoted phrases: “the outlet and the reviewers made a difference in this case.”

R2’s experiences speak to the importance of site-specific context and the larger zeitgeist. A practice on one subreddit might not be appropriate to another, especially after larger events increase the likelihood of trolling and harassment. Similarly, R15 noted that “I think ethical use of social media posts for research has to take context into consideration – it’s a different thing to quote someone posting from a public Twitter account than it is to burrow into an obscure subreddit and identify one comment on a long thread to surface.” And the social media context is dynamic: “With more and more news-style platforms grabbing social media posts without permission to use as comments in news articles we might even see a shifting culture around what people think is permissible once they’ve posted something publicly. Or this practice might result in pushback in which people demand to be asked permission or credited for their posts!”

The world that researchers seek to understand is ever-changing, as are the technical affordances of media platforms and search services. It can be difficult to match ethical policy to the quickly shifting online world, as can implementing that policy with consistent practice, especially given the time and changes needed during a reports’ publication.

Effective tactics of disguise

R1’s report is a detailed ethnography of a few identified subreddits that is well-grounded with descriptions of community concerns and quotes from Redditors. Yet there were only two phrases (of more than 10 words) to attempt to locate. Most of everything else was from subreddits’ documentation (not sensitive) and interviews (not indexed by search engines). Confidential interviews of public Redditors can enable a surprising degree of richness, disclosure, and confidentiality.

Otherwise, searching for 150 + sources in the 22 research papers reveals that the metaphor of finding a needle in a haystack (of returned search results) is apt. Reports that focus on a single subreddit (as stated or inferred) in a single year winnow away much of the hay. Additionally, changes of punctuation, switching to or from contractions, single word insertions or removals, and retaining novel words are usually insufficient disguise.

Larger datasets – or less specific descriptions – and more substantive changes are more effective. R9, for example, included 11 effectively disguised phrases. Their dataset included over a million posts over a dozen subreddits – their haystack was large. Additionally, their technique for disguising the phrases might be an effective consequence of an analytic technique used to normalize text into a canonical form: “normalization is an approach to finding long English phrases in online vernacular and replace them with concepts in clinical knowledge sources. For instance, ‘shortness of breath’ in a post is replaced by the term ‘edema’ or ‘dyspnea.’”. Though this technique was not created to disguise sources, it seemingly serves that purpose.

The rigor of testing and disciplinary differences

R3 used about ten reworded phrases from Reddit; I was not able to locate their sources.

Two influences at the start of R3’s career – as well as the sensitive topics they tend to study – led to a rigorous process for disguising sources. Today, their process is an iterative one, of swapping in synonyms or larger rephrasing “in a way that doesn’t change the meaning and yet would be untraceable. If someone were to put that quote in Google and try to find it there … they wouldn’t be able to do that.” To accomplish this, R3 performs the task themselves. That is, they seek to locate their own disguised sources – though, as seen above, Google is not the only index of Reddit messages. And their method is akin to my method here: using exact searches (in quote marks), near searches (without quote marks), and focusing on portions of a phrase, while conceding the process is “pretty subjective.” Just as with my method, they have to choose how to specify the search and how many returned results to review.

R3’s germinal influences were an event and a scholar. First, as a doctoral student, they saw another researcher’s source publicly disclosed because of the inclusion of a verbatim quotation. This was “an example of how [verbatim quotes] can be a problem.” Second, subsequently, R3 learned of Markham ( 2012 )’s coining and explication of “ethical fabrication,” giving R3 a name and rationale for something similar to what R3 was already doing.

Today, in their role as an editor and reviewer, R3 sometimes asks researchers to reflect on their sourcing practice and rationale, with those in their discipline tending to be thoughtful about this issue. Elsewhere, though, R3 has experienced pushback against fabrication, such as in a presentation before a group of network-analytic sociologists. The audience was upset when they learned they were seeing fabricated, rather than authentic, quotes and images in the presentation.

R4 employed similar tactics as R3, changing “gender, location, specific details of an incident etc. so that, while they convey the original thought of the author, they cannot be traced back to them.” They tested these disguised phrases using a Pushshift-related service and “shared the two snippets with others in the team in order to see if the rephrase is too far off.”

R14 was the third interviewee to test their disguises. Though R14 used Pushshift in other work, they did not test their disguises against RedditSearch/Pushshift. Instead, they pursued the tactic of change-and-test “until I couldn’t find the quote + reddit in Google.” I was able to locate many of their sources because I limited my queries to the Reddit website (i.e., site:reddit.com) in Google. This extra specificity plus the year led to many of R14’s sources.

R16, too, has tested their disguises in the past via Google, “but I don’t know if I did it with this paper. And I certainly did not try other search engines.”

If a researcher wants to use disguised public data, rather than interviews, then the best disguise is tested disguise. This means investigating where their data sources are likely to be archived, how it all can be searched, and using as many facets of search as possible to test their efforts.

Limitations and Future Work

The present work is idiosyncratic and relatively small in scale; nonetheless, it shows that the practice of disguise is often haphazard and ineffective. The next step is to investigate automated methods for managing and disguising sources. That is, can automated programs and services alter phrases for inclusion in reports with more or less efficacy than humans? Managing sources could be easy as keeping quotes, their source, and their disguise in a spreadsheet shared among collaborators – and this could then facilitate automatic testing. The next phase of the current work tests the feasibility and efficacy of this approach (Reagle & Gaur, 2022 ) .

The web and the services that index it are dynamic. Google routinely updates its search algorithms, and in April of 2022 – after the data and collection in the present report – Reddit announced they had extended their search facility to comments and made their searches less literal: “100% of a query doesn’t have to match the text of a post to return relevant results. By doing this we saw a 60% increase in results for queries that previously didn’t receive results” (Staff, 2022 ). Such changes will affect how easy it is to locate a source. Though such changes could make location more difficult by crowding out the true source, it is clearly the intention of services to improve their search efficacy.

Another practical follow-up is to increase the understanding of risks and options among researchers. King ( 1996 ) faulted Finn & Lavitt ( 1994 ) for leaking information about source context decades ago, and yet it still happens. A guide that builds on Bruckman ( 2002 )’s categories of disguise and identifies risks and options available to the researcher could help authors, reviewers, and editors. The guide could include a checklist of things to attend to for a given category of risk and disguise. And it could be complemented by site-specific information about conventions and norms, and affordances around user identity and message persistence and locatability.

Finally, messages appeared in research reports that were subsequently deleted by their authors – even if from pseudonymous and throw-away accounts. This merits more attention and work-in-progress indicates throwaway accounts regularly – routinely even – delete posts in advice subreddits.

Conclusions

There is no single research policy appropriate to disguising online sources. For example, community members might expect or appreciate recognition as creative authors. Like V3’s two Redditors who gained additional notice for appearing in her report, R18 noted that “One of the moderators of a subreddit I used reached out to me on ResearchGate and thanked me for my research and steps toward harm reduction; they were happy that I used material from their subreddit.”

If researchers chose to use disguise, however, their practice ought to effectively match their policy. I found descriptions of ethical policy that were confusing or inconsistent with actual practice. In a few cases, this was the result of changes made during the review and editing process. In another case, I suspect it was an oversight in data collecting and reporting. Many others simply made mistakes or failed to appreciate the affordances of Reddit and web searching.

The RedditSearch interface to the Pushshift repository proved especially useful in locating sources. And such data can be repackaged in ways that permit even more powerful searching capabilities (e.g., BigQuery and ConvoKit). While some researchers might use these resources in large-scale analyses, other researchers were unfamiliar with them. In addition to advanced search capabilities, these resources also mean that researchers who use them might include data since deleted by users in research analyses and reports.

The highest level of disguise, eliding usernames and rewording prose, can be effective, especially when the reworded phrases are tested against search engines – the practice of a few interviewed researchers. However, concerned researchers should be as specific as possible in their test queries, taking advantage of site, date, and subreddit facets.

My interviewees shared how their practices changed relative to their research sites, the larger cultural context, and their influences and experiences. The different approaches we see in reports, however, are not necessarily the result of a consistent policy (i.e., from conception to publication), fully cognizant of technical affordances (e.g., Google’s site: facet and RedditSearch/Pushshift existence and abilities), and users’ wishes (e.g., when users delete posts from throw-away accounts). The research community can improve on this, though, via similar site-specific investigations and practical guidelines that inform the conception, execution, and review of research. We also need additional work on automating, managing, and testing research disguise.

Acknowledgements

I thank the other panelists of the AoIR 2021 panel on Reddit and ethics, especially Michael Zimmer (Marquette University) and Nicholas Proferes (Arizona State University), who shared information with me about their large-scale survey of Reddit research. I thank the anonymous reviewers for their helpful insights and suggestions. Finally, I am grateful to my interviewees, especially as their attention and time were under stress from COVID-19.

Availability of data and material (data transparency)

Code availability (software application or custom code), declarations.

Northeastern Univervisty Institutional Review Board application #20-08-30 and “approved” as DHHS Review Category #2: “Exempt… No further action or IRB oversight is required as long as the project remains the same.”

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Andalibi, N., Ozturk, P., & Forte, A. (2017). Sensitive self-disclosures, responses, and social support on Instagram. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing . 10.1145/2998181.2998243
• Reagle, J., & Gaur, M. (2022). Spinning words as disguise: Shady services for ethical research? First Monday . 10.5210/fm.v27i1.12350
• Ayers, J. W., Caputi, T. L., Nebeker, C., & Dredze, M. (2018). Don’t quote me: Reverse identification of research participants in social media studies. NPJ Digital Medicine , 1 (1), 10.1038/s41746-018-0036-2 [ PMC free article ] [ PubMed ]
• Backes, M., Berrang, P., Goga, O., Gummadi, K. P., & Manoharan, P. (2016). On profile linkability despite anonymity in social media systems. Proceedings of the 2016 ACM on Workshop on Privacy in the Electronic Society - WPES’16 . 10.1145/2994620.2994629
• Balamuta, J. (2018, November 13). Using Google BigQuery to obtain Reddit comment phrase counts. The Coatless Professor . https://thecoatlessprofessor.com/programming/sql/using-google-bigquery-to-obtain-reddit-comment-phrase-counts/
• Barbaro, M., & Zeller, T. Jr. (2006, August 9). A face is exposed for AOL searcher no. 4417749. The New York Times . https://www.nytimes.com/2006/08/09/technology/09aol.html
• Baumgartner, J. (2016, September 19). pushshift.io: API documentation: List of endpoints . pushshift.io. https://pushshift.io/api-parameters/
• Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020). The Pushshift Reddit dataset. Proceedings of The International AAAI Conference on Web and Social Media , 14 (1), 830–839. https://ojs.aaai.org/index.php/ICWSM/article/view/7347
• boyd, danah. (2007). Why youth heart social network sites. In D. Buckingham (Ed.), Youth, identity, anddigital media. MIT Press. 10.1162/dmal.9780262524834.119
• Buckingham, D. (Ed.). (2007). Why youth heart social network sites Youth, identity, and digital media . MIT Press. 10.1162/dmal.9780262524834.119. boyd
• Brown, A., & Abramson, M. (2015). Twitter fingerprints as active authenticators. 2015 IEEE International Conference on Data Mining Workshop (ICDMW) . 10.1109/ICDMW.2015.223
• Bruckman, A. (2002). Studying the amateur artist: a perspective on disguising data collected in human subjects research on the Internet. Ethics and Information Technology , 4 (3). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.432 .1591&rep=rep1&type=pdf
• Bruckman, A., Luther, K., & Fiesler, and C (2015). When should we use real names in published accounts of internet research?. In E. Hargittai, & C. Sandvig (Eds.), Digital research confidential: The secrets of studying behavior online . MIT Press
• Brunton, F., & Nissenbaum, H. (2015). Obfuscation: A user’s guide for privacy and protest . MIT Press. https://we.riseup.net/assets/355198/Obfuscation.pdf
• Chen, Y., Sherren, K., Smit, M., & Lee, K. Y. (2021). Using social media images as data in social science research. New Media & Society , 146144482110387. 10.1177/14614448211038761
• ConvoKit (2018, October 31). Reddit corpus (by subreddit) . Cornell. https://convokit.cornell.edu/documentation/subreddit.html
• Dym, B., & Fiesler, C. (2020). Ethical and privacy considerations for research using online fandom data. Transformative Works and Cultures , 33 . 10.3983/twc.2020.1733
• Ess, C., & Committee, A. E. W. (2002). Ethical decision-making and Internet research: recommendations from the AOIR Ethics Working Committee . http://aoir.org/reports/ethics.pdf
• Eysenbach, G., & Till, J. E. (2001). Ethical issues in qualitative research on internet communities. BMJ ,1103–1105. http://bmj.bmjjournals.com/cgi/content/full/323/7321/1103 [ PMC free article ] [ PubMed ]
• Fiesler, C., & Proferes, N. (2018). “Participant” perceptions of Twitter research ethics. Social Media + Society , 4 (1), 10.1177/2056305118763366
• Finn J, Lavitt M. Computer-based self-help groups for sexual abuse survivors. Social Work With Groups. 1994; 17 (1–2):21–46. doi: 10.1300/J009v17n01_03. [ CrossRef ] [ Google Scholar ]
• Flicker S, Haans D, Skinner H. Ethical dilemmas in research on internet communities. Qualitative Health Research. 2004; 14 (1):124–134. doi: 10.1177/1049732303259842. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Franzke, A. S., Bechmann, A., Zimmer, M., Ess, C., & AoIR (2020). and. Internet research: Ethical guidelines 3.0 . AoIR. https://aoir.org/reports/ethics3.pdf
• Gaffney, D., & Matias, J. N. (2018). Caveat emptor, computational social science: Large-scale missing data in a widely-published Reddit corpus. PLOS ONE , 13 (7), 10.1371/journal.pone.0200162 [ PMC free article ] [ PubMed ]
• Guarino, A. (2013). Digital forensics as a big data challenge. ISSE 2013 Securing Electronic Business Processes , 197–203. 10.1007/978-3-658-03371-2_17
• Guidelines for data de-identification or anonymization . EDUCAUSE. https://www.educause.edu/focus-areas-and-initiatives/policy-and-security/cybersecurity-program/resources/information-security-guide/toolkits/guidelines-for-data-deidentification-or-anonymization
• Haimson, O. L., Andalibi, N., & Pater, J. (2016, December 20). Ethical use of visual social media content in research publications . AHRECS. https://ahrecs.com/ethical-use-visual-social-media-content-research-publications/
• Johansson, F., Kaati, L., & Shrestha, A. (2015). Timeprints for identifying social media users with multiple aliases. Security Informatics , 4 (1), 10.1186/s13388-015-0022-z
• King, S. A. (1996). Researching internet communities: Proposed ethical guidelines for the reporting of results. The Information Society , 12 (2), 10.1080/713856145
• Kozinets RV. Netnography: Redefined (Kindle) Limited: SAGE Publications; 2015. [ Google Scholar ]
• Mann, C., & Stuart, F. (2000). Internet communication and qualitative research: a handbook for researching online . Sage
• Markham, A. (2012). Fabrication as ethical practice: Qualitative inquiry in ambiguous Internet contexts. Information Communication & Society , 15 (3), 10.1080/1369118x.2011.641993
• Narayanan, A., Paskov, H., Gong, N. Z., Bethencourt, J., Stefanov, E., Shin, E. C. R., & Song, D. (2012). On the feasibility of internet-scale author identification. 2012 IEEE Symposium on Security and Privacy . 10.1109/SP.2012.46
• Narayanan, A., & Shmatikov, V. (2009). De-anonymizing social networks. 2009 30th IEEE Symposium on Security and Privacy . 10.1109/SP.2009.22
• Nguyen, H., & Cavallari, S. (2020). Neural multi-task text normalization and sanitization with pointer-generator. Proceedings of the First Workshop on Natural Language Interfaces . 10.18653/v1/2020.nli-1.5
• Ohm, P. (2010). Broken promises of privacy: Responding to the surprising failure of anonymization. UCLA Law Review , 58 (2). https://www.uclalawreview.org/broken-promises-of-privacy-responding-to-the-surprising-failure-of-anonymization-2/
• Pentzold C. “What are these researchers doing in my Wikipedia?”: Ethical premises and practical judgment in internet-based ethnography. Ethics and Information Technology. 2017; 19 (2):143–155. doi: 10.1007/s10676-017-9423-7. [ CrossRef ] [ Google Scholar ]
• Proferes, N., Jones, N., Gilbert, S., Fiesler, C., & Zimmer, M. (2021). Studying Reddit: A systematic overview of disciplines, approaches, methods, and ethics. Social Media + Society , 7 (2), 10.1177/20563051211019004
• Reddit (2021, January 27). Reddit by the numbers . RedditInc. https://www.redditinc.com/press
• Reddit Search . (2021, January 14). Reddit. https://www.reddit.com/wiki/search
• Reid, E. (1996). Informed consent in the study of online communities: A reflection on the effects of computer-mediated social research. Information Science , 12 (2)
• Reyes, V. (2017). Three models of transparency in ethnographic research: Naming places, naming people, and sharing data. Ethnography , 19 (2), 10.1177/1466138117733754
• Rodham K, Gavin J. The ethics of using the internet to collect qualitative research data. Research Ethics. 2006; 2 (3):92–97. doi: 10.1177/174701610600200303. [ CrossRef ] [ Google Scholar ]
• Sharf, B. (1999). Beyond netiquette: The ethics of doing naturalistic discourse research on the Internet. In S. Jones (Ed.), Doing internet research: Critical issues and methods for examining the net . Sage
• Shklovski, I., & Vertesi, J. (2013, April 27). “UnGoogling” publications: The ethics and problems of anonymization. Proceedings of CHI 2013 . https://pure.itu.dk/portal/files/80190129/p2169_shklovski.pdf
• Siang, S. (1999). Researching ethically with human subjects in cyberspace. Professional Ethics Report , 22 (4). http://www.aaas.org/spp/sfrl/per/per19.htm [ PubMed ]
• Singal, J. (2016, March 9). 3 lingering questions from the Alice Goffman controversy. The Cut . https://www.thecut.com/2016/01/3-lingering-questions-about-alice-goffman.html
• Singal, J. (2015, June 18). The internet accused Alice Goffman of faking details in her study of a black neighborhood. I went to Philadelphia to check. The Cut . https://www.thecut.com/2015/06/i-fact-checked-alice-goffman-with-her-subjects.html
• Singer, N. (2015, February 14). Love in the time of Twitter. The New York Times . https://web.archive.org/web/20190412053116/https://bits.blogs.nytimes.com/2015/02/13/love-in-the-times-of-twitter/
• Smith, J. S., & Murray, C. D. (2001). Pearls, pith, and provocation: Ethical issues in the documentary data analysis of internet posts and archives. Qualitative Health Research , 11 (3) [ PubMed ]
• Staff, R. (2022, April 14). New on Reddit: Comment search, improved search results relevance, updated search design . Reddit Inc. https://www.redditinc.com/blog/new-on-reddit-comment-search-improved-search-results-relevance-updated-search-design
• Stuck_In_the_Matrix (2019, April 8). Pushshift will now be opting in by default to quarantined subreddits . r/pushshift. https://www.reddit.com/r/pushshift/comments/bazctc/pushshift_will_now_be_opting_in_by_default_to/ .
• Stuck_In_the_Matrix (2015, September 8). Reddit data for ~ 900,000 subreddits (includes both public and private subreddits) . r/datasets. https://www.reddit.com/r/datasets/comments/3k3mr9/reddit_data_for_900000_subreddits_includes_both/
• Waskul D, Douglas M. Considering the electronic participant: polemical observations on the ethics of online research. The Information Society. 1996; 12 :129–139. doi: 10.1080/713856142. [ CrossRef ] [ Google Scholar ]
• Zhou X, Liang X, Zhang H, Ma Y. Cross-platform identification of anonymous identical users in multiple social media networks. IEEE Transactions on Knowledge and Data Engineering. 2016; 28 (2):411–424. doi: 10.1109/TKDE.2015.2485222. [ CrossRef ] [ Google Scholar ]
• Zimmer, M. (2010). “But the data is already public”: On the ethics of research in Facebook. Ethics and Information Technology , 12 (4), 10.1007/s10676-010-9227-5

How to Find Relevant Papers for your Research Project

Keeping up to date with new literature and reviewing the existing literature helps you to build deep knowledge and understanding in your field. Developing effective reading strategies and systems to retain and evolve your knowledge over time is a key part of your research project. With millions of papers published every year in more than ten thousand different journals, the challenge lies in finding relevant papers for your work, deciding which ones to read in-depth and working out how to retain and build an expansive knowledge over a long time frame, while balancing this alongside all your other research activities.  

In this multi-part Academic Reading Series we’ll focus on systems for reading and learning from academic papers, pooling information from our experiences in academia and advice from current researchers. In this first part we’ll cover searching for papers.

Why read Academic Papers?

Academic papers share the details of recent developments and discoveries, and are the most up to date records of advances in a field.  When starting a new project, you’ll typically first spend a lot of time reading.  This reading helps you to build a solid working knowledge, and learn about current areas of investigation and interest from researchers in your field.  

Every paper you read can also help you to:

• learn standard techniques and methods for your project
• discover emerging or new techniques and theories 
• find and access relevant data sets for analysis
• identify other relevant papers in reference sections
• spot gaps for your exploration
• notice opportunities for new projects and collaborations
• save yourself time so you don’t repeat what’s already been done

Papers are an invaluable source of information that you can use to build your foundational knowledge, determine interesting directions and to generate novel ideas for your projects.

Papers and your Research Project

Publishing papers is the academic standard for disseminating ideas, discoveries and findings.  Through your research projects, you’ll add to the body of academic knowledge, ideally contributing something novel to your field.  Depending on your research stage your project may involve writing a thesis or dissertation or publishing a journal articles to share your ideas within the research community. 

Typically academic writing involves referring to previously conducted work in the field to provide the context for your own work and sharing the information and evidence that supports your approach or methodology, as well as your findings and interpretations.  So the papers you are reading not only form a critical part in shaping your research, they also needed to back up your work when you write.  

While reading, you can start to create a personal catalogue of information and ideas that you can refer to when writing. Depending on how you build your catalogue and organise notes from your reading, you can then save yourself time searching for relevant knowledge when you sit down to write or want to plan out an investigation in future.  

Finding Relevant Papers when starting a Project

When you first start a project, you’ll likely be pointed in the direction of some relevant literature by your supervisor or PI.  There are several ways to find other relevant literature to read and would strongly recommend speaking with the experts around you, such as asking other researchers within your group, (if you are joining a group) or looking at your PI’s other publications, and exploring the publications of any collaborators on those papers.  Your PI may have varied research interests so keep in mind not all of their publications will be relevant for your project.

If you are continuing a project from a previous researcher, then their thesis or publications can hold relevant information, as well as point you towards other relevant literature from their references and bibliography.  

Recommendations from your PI and other researchers may point you in the direction of Seminal papers – the work that initiated or presents key ideas and theories in your field. These are the papers synonymous with your field so as you are searching for relevant papers, don’t disregard some of the older papers in your searches. 

Another way to identify relevant literature is finding relevant conferences in the field and exploring the presentations, posters and keynote speakers listed on the websites.  Again asking other researchers in your group and your supervisors will help you find conferences and meetings.  

It’s tempting to go search and uncover everything through searching online, but don’t forget to take advantage of the specific knowledge and expertise available from the people around you.

Finding Relevant Papers Online

As you deepen your familiarity with your research, you’ll start to notice and identify the most relevant recurring keywords in the papers you’re finding.  Take note of them for use in your future searches.  These keywords and combinations of those keywords can be used to help you with finding further relevant literature through literature databases like Google Scholar, PubMed or Web of Science.  You could even use these keywords to set up notifications to alert you about newly published papers related to these keywords.

Literature databases operate on keyword searches and have additional filters available to apply further parameters to your search, such as publication date or publication type as well as citation count. Generally high citation counts, when you consider the publication date, can indicate key papers and researchers in the field.  These high citation papers, and their authors offer another avenue for finding relevant papers for your project, from the references section of the paper and looking at other publications from these authors.   

Review papers can be a particularly good source of other relevant papers.  Reviews assess and analyse the published work in a field to date, and also highlight potential areas for further investigation.  The references section of these papers can point you towards several further relevant papers for your project. 

You might have come across the idea of a ‘Seed Paper’ and this is a paper you find that is very relevant to your work, which can act to point you in the direction of other relevant work. Tools like LitMaps can be useful here, as they show you the prior and derivative work based on the references in the paper, and which papers have since cited the paper. 

Ongoing Literature Search

Searching the literature is an expansive and iterative activity.  Finding relevant papers to read should be a regular part of your research activities. As you identify your keywords, you can take advantage of automated notifications when new papers are published. You might also identify particular journals that publish relevant research and you can monitor RSS feeds, using tools like Feedly to see new publications.

There are also lots of AI tools to help with your academic research from identifying relevant papers, generating summaries and ‘talking’ with collections of your papers.  These tools can definitely speed up your search and reading process. Keep in mind AI can be an incredibly useful tool, but you’ll need to analyse, interpret and apply the knowledge to direct your project in new and interesting directions.

Literature Searches are an iterative process, you might need to try several different combinations of keywords to find the most relevant results.  And as your project evolves or you follow different lines of inquiry in your work, you should search with updated and/or new keywords to unearth relevant literature.

It can be beneficial to save these searches, as well as the papers you want to look at in more detail. Some tools keep a log of all your searches performed, others you might need to keep track using your own system.  

Organising Relevant Papers

As you start finding relevant papers, it’s beneficial to make use of Reference Managers.  These tools capture the important information about a paper and contain a link or copy of the paper for you to access in future.  Adding your papers in you’ll start to build your own personal literature library, which will be an invaluable resource as you continue your project.

With a growing literature collection, which will only get larger over time, Reference Managers are a much better option than having a load of open Tabs, saved bookmarks in your browser or a Folder of PDFs on your desktop.   These Managers let you develop organisational systems to group related papers together and help you to add references into your academic writing. Proper referencing is a critical part of all academic writing and is made much quicker with a reference manager.  You can use popular reference management tools like Mendeley and Zotero , and Protolyst also has built-in referencing .

In the next part of this series, we’ll cover deciding which papers to read and different approaches to reading papers. 

• There are many ways to identify relevant literature to build your knowledge and inform your research project direction.  You can search through literature databases, search using AI tools, and don’t forget to seek suggestions from your PI and other researchers in your group.
• Once you’ve identified a relevant paper, look at the references in that paper and explore other publications from the Authors.
• Reading papers is an ongoing activity, consider using systems to organise your searches and the papers you discover
• Keep track of the most relevant Keywords for your project to iterate on your searches for relevant Literature
• Make use of Reference Managers to keep track of all the papers you are discovering to read and refer to in future
• Keep track of all relevant papers, you can decide which ones you should read and how to read them later

This is not an exhaustive list of suggestions to find relevant papers for your research! What tips and advice would you add? We’d love it if you let us know !

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

You're wasting 1 day a week refinding the golden nuggets of info buried in your files and folders. Protolyst is a networked note taking web app where you can collect and categorise each individual highlight to find and use your most valuable info fast.

• Cookie Policy
• Privacy Policy
• Writing Referenced Articles Fast
• Writing a Literature Review
• Taking Notes on Research Papers
• Building a Literature Database
• Zettelkasten Notetaking System
• Connecting Cornell Notes

Privacy Overview

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process

A Beginner's Guide to Starting the Research Process

When you have to write a thesis or dissertation , it can be hard to know where to begin, but there are some clear steps you can follow.

The research process often begins with a very broad idea for a topic you’d like to know more about. You do some preliminary research to identify a  problem . After refining your research questions , you can lay out the foundations of your research design , leading to a proposal that outlines your ideas and plans.

This article takes you through the first steps of the research process, helping you narrow down your ideas and build up a strong foundation for your research project.

Table of contents

Step 1: choose your topic, step 2: identify a problem, step 3: formulate research questions, step 4: create a research design, step 5: write a research proposal, other interesting articles.

First you have to come up with some ideas. Your thesis or dissertation topic can start out very broad. Think about the general area or field you’re interested in—maybe you already have specific research interests based on classes you’ve taken, or maybe you had to consider your topic when applying to graduate school and writing a statement of purpose .

Even if you already have a good sense of your topic, you’ll need to read widely to build background knowledge and begin narrowing down your ideas. Conduct an initial literature review to begin gathering relevant sources. As you read, take notes and try to identify problems, questions, debates, contradictions and gaps. Your aim is to narrow down from a broad area of interest to a specific niche.

Make sure to consider the practicalities: the requirements of your programme, the amount of time you have to complete the research, and how difficult it will be to access sources and data on the topic. Before moving onto the next stage, it’s a good idea to discuss the topic with your thesis supervisor.

>>Read more about narrowing down a research topic

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

So you’ve settled on a topic and found a niche—but what exactly will your research investigate, and why does it matter? To give your project focus and purpose, you have to define a research problem .

The problem might be a practical issue—for example, a process or practice that isn’t working well, an area of concern in an organization’s performance, or a difficulty faced by a specific group of people in society.

Alternatively, you might choose to investigate a theoretical problem—for example, an underexplored phenomenon or relationship, a contradiction between different models or theories, or an unresolved debate among scholars.

To put the problem in context and set your objectives, you can write a problem statement . This describes who the problem affects, why research is needed, and how your research project will contribute to solving it.

>>Read more about defining a research problem

Next, based on the problem statement, you need to write one or more research questions . These target exactly what you want to find out. They might focus on describing, comparing, evaluating, or explaining the research problem.

A strong research question should be specific enough that you can answer it thoroughly using appropriate qualitative or quantitative research methods. It should also be complex enough to require in-depth investigation, analysis, and argument. Questions that can be answered with “yes/no” or with easily available facts are not complex enough for a thesis or dissertation.

In some types of research, at this stage you might also have to develop a conceptual framework and testable hypotheses .

>>See research question examples

The research design is a practical framework for answering your research questions. It involves making decisions about the type of data you need, the methods you’ll use to collect and analyze it, and the location and timescale of your research.

There are often many possible paths you can take to answering your questions. The decisions you make will partly be based on your priorities. For example, do you want to determine causes and effects, draw generalizable conclusions, or understand the details of a specific context?

You need to decide whether you will use primary or secondary data and qualitative or quantitative methods . You also need to determine the specific tools, procedures, and materials you’ll use to collect and analyze your data, as well as your criteria for selecting participants or sources.

>>Read more about creating a research design

Finally, after completing these steps, you are ready to complete a research proposal . The proposal outlines the context, relevance, purpose, and plan of your research.

As well as outlining the background, problem statement, and research questions, the proposal should also include a literature review that shows how your project will fit into existing work on the topic. The research design section describes your approach and explains exactly what you will do.

You might have to get the proposal approved by your supervisor before you get started, and it will guide the process of writing your thesis or dissertation.

>>Read more about writing a research proposal

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

Is this article helpful?

Other students also liked.

• Writing Strong Research Questions | Criteria & Examples

What Is a Research Design | Types, Guide & Examples

• How to Write a Research Proposal | Examples & Templates

More interesting articles

• 10 Research Question Examples to Guide Your Research Project
• How to Choose a Dissertation Topic | 8 Steps to Follow
• How to Define a Research Problem | Ideas & Examples
• How to Write a Problem Statement | Guide & Examples
• Relevance of Your Dissertation Topic | Criteria & Tips
• Research Objectives | Definition & Examples
• What Is a Fishbone Diagram? | Templates & Examples
• What Is Root Cause Analysis? | Definition & Examples

"I thought AI Proofreading was useless but.."

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”