n8n Twitter/X Scraper: Automate Tweets to Google Sheets

In this tutorial you will learn how to use an n8n workflow template to scrape tweets from Twitter/X and store them automatically in Google Sheets. We will walk through each node, explain how pagination and counters work, and show you how to run the workflow safely within rate limits.

Learning goals

By the end of this guide, you will be able to:

• Import and configure a ready-made n8n Twitter/X scraper template.
• Connect Twitter/X (via HTTP) and Google Sheets credentials.
• Customize the tweet search query for your own keywords or accounts.
• Understand and adjust pagination, counters, and loop limits.
• Map tweet data into a Google Sheet with structured columns.
• Handle common errors and respect API rate limits.

Why use n8n for Twitter/X scraping?

n8n is an open-source automation platform that lets you connect APIs and services using a visual workflow editor. It is well suited for building a Twitter/X scraper because you can:

• Design workflows visually and reuse them as templates.
• Control pagination and rate limiting with dedicated nodes.
• Send scraped tweets directly to Google Sheets, databases, or other tools.
• Use code nodes to clean, transform, and enrich tweet data.

This template takes advantage of those strengths so you do not have to write a full integration from scratch.

How the template works at a high level

The workflow uses the Twitter API (through twitterapi.io or another compatible endpoint) to run an advanced search, then writes each tweet to a Google Sheet. The core logic includes:

• A starting counter to control how many pages of tweets to fetch.
• Cursor-based pagination using the API’s next_cursor value.
• A transformation step to normalize the API response into clean JSON items.
• A Google Sheets step that appends each tweet as a new row.
• A loop with an If condition that stops after a set number of iterations.

In practical terms, the workflow repeats this cycle:

1. Fetch a page of tweets for your search query.
2. Extract and format the tweet data.
3. Append the tweets to Google Sheets.
4. Update the counter and cursor to get the next page.
5. Check if the loop should continue or stop.

Overview of the key nodes

Before we configure anything, it helps to understand what each node does in the loop.

Core control and loop nodes

• Set Count – Initializes the starting counter value for testing and controls how many iterations you run.
• Counter – Holds the current counter and cursor values and passes them into each loop cycle.
• If – Checks whether the counter has reached your configured maximum. If the condition is met, the loop stops; otherwise, it continues.
• Limit – Used to throttle or control the flow between iterations. This is helpful for staying within Twitter/X rate limits.
• Increase Count – A code node that increments the counter and updates the cursor for the next API request.

Data fetching and transformation nodes

• Get Tweets – An HTTP Request node that calls the advanced search endpoint. It sends parameters such as query=OpenAI and the cursor for pagination.
• Extract Info – A code node that converts the raw API response into uniform JSON items. It outputs fields like tweetId, url, content, engagement counts, and createdAt.
• Add to Sheet – A Google Sheets node that appends each tweet as a new row in your spreadsheet, using mapped columns.

Once you understand these roles, the configuration steps will make more sense.

Step-by-step setup in n8n

Step 1: Import the workflow template

Start in your n8n instance:

1. Open n8n and go to the workflow import area.
2. Import the provided template JSON file.

If the import screen shows a loading spinner for a long time, try:

• Waiting a short while, then refreshing the browser.
• Doing a hard refresh or opening n8n in a different browser.
• Confirming that the n8n server is running and accessible.

Step 2: Configure credentials

The workflow uses two main credentials that you must set up before running it:

• Twitter API (HTTP Header Auth)
  
  • Create or edit an HTTP Header auth credential in n8n named something like Twitter API.
  • Provide the required header token or API key for your chosen Twitter/X API provider.
  • If you use twitterapi.io or Twitter API v2, check their documentation to match the exact header names and values.
• Google Sheets (OAuth2)
  
  • Set up a Google Sheets OAuth2 credential in n8n.
  • Use a Google account that has access to the spreadsheet where you will store tweets.
  • Authorize n8n to read and write to Google Sheets when prompted.

Step 3: Customize the Twitter/X search query

Next, decide what tweets you want to scrape. Open the Get Tweets HTTP Request node:

• Locate the query parameter in the node settings.
• Replace the default value (for example, OpenAI) with your own search expression.

You can use simple keywords, hashtags, or advanced search syntax. For example:

query=from:elonmusk -is:retweet lang:en

This would search for English tweets from a specific account, excluding retweets. Adjust the query to match your monitoring or research needs.

Step 4: Configure Google Sheets mapping

Open the Add to Sheet node to connect it to your spreadsheet:

• Select the correct documentId (the spreadsheet) and sheetName.
• Review the field mapping. The template typically maps:

• Tweet ID
• URL
• Content
• Likes
• Retweets
• Replies
• Quotes
• Views
• Date

Adjust the mapping if your sheet has different column names or if you want to add or remove fields.

Step 5: Tune pagination and loop limits

The workflow uses cursor-based pagination. The Twitter/X API returns a next_cursor value, which points to the next page of results. The loop logic uses this cursor along with a simple counter to avoid infinite runs.

Key elements to adjust:

• Counter node – Holds the current iteration count.
• If node – Contains a condition like counter == 3.

By default, the template stops after 3 iterations. To change this:

1. Open the If node.
2. Update the comparison value (for example, from 3 to 10) to allow more pages.

Alternatively, you can extend the logic to stop when next_cursor is null or missing, which means there are no more pages. That approach prevents unnecessary requests when the API has no additional data.

Important code nodes explained

Extract Info node (formatting tweet data)

The Extract Info code node is responsible for turning the raw HTTP response into a clean list of tweets that n8n can send to Google Sheets. It typically:

• Checks that the response contains a tweets array.
• Maps each tweet into a separate n8n item.
• Normalizes field names and sets default values.
• Formats createdAt into a human-readable date string.

A simplified version of the logic looks like this:

// Simplified logic
if (input.json.tweets && Array.isArray(input.json.tweets)) {  return input.json.tweets.map(tweet => ({  json: {  tweetId: tweet.id || '',  url: tweet.url || '',  content: tweet.text || '',  likeCount: tweet.likeCount || 0,  retweetCount: tweet.retweetCount || 0,  replyCount: tweet.replyCount || 0,  quoteCount: tweet.quoteCount || 0,  viewCount: tweet.viewCount || 0,  createdAt: formattedDate  }  }));
}

You can customize this node to:

• Filter out unwanted tweets (for example, by language or minimum engagement).
• Remove duplicates before they reach Google Sheets.
• Add enrichment, such as sentiment scores, keyword extraction, or tags.

Increase Count node (managing the loop)

The Increase Count code node updates the loop state after each iteration. It usually does two things:

• Reads the current counter value and increments it by 1.
• Pulls the next_cursor from the latest API response and stores it so the next Get Tweets call can use it.

This is what allows the workflow to move from page 1 to page 2, and so on, while also keeping track of how many times it has looped.

Handling rate limits and improving reliability

Twitter/X APIs typically enforce rate limits, so your n8n workflow should be configured to respect them and recover gracefully from errors.

• Add delays between requests
  Insert a Wait node or use the Limit node to slow down the loop. Increasing the delay between calls reduces the risk of hitting 429 errors.
• Use error handling
  Wrap custom code in try/catch blocks where appropriate and consider using n8n’s On Error workflow to retry failed requests or log them separately.
• Log raw responses
  Before the Extract Info node, add a Set node or similar step to store the raw HTTP response. This makes debugging easier if the structure changes or unexpected data appears.

Troubleshooting common issues

Issue: Template import shows a persistent loading spinner

If the import screen or the n8n interface keeps showing a loading spinner:

1. Refresh the browser and clear the cache.
2. Verify that your n8n server is running and that you can reach its URL.
3. Try a different browser or a private/incognito window.
4. Open the browser’s Developer Tools and check the Console for JavaScript errors.

Issue: Empty results or API errors from Twitter/X

If the workflow runs but no tweets appear, or you see errors in the Get Tweets node:

• Confirm that your Twitter/X API keys or headers are correct and not expired.
• Check the execution log of the Get Tweets node for HTTP status codes and error messages.
• If you see status code 429 (too many requests), increase delays or reduce the number of loop iterations.
• Review your query parameter to ensure it is valid and returns results when tested directly with your provider.

Extending and customizing the workflow

Once the basic scraper is working, you can adapt it to more advanced use cases.

• Store data in a database
  Instead of (or in addition to) Google Sheets, connect nodes for Postgres, MySQL, or another database to store JSON output for analytics or production use.
• Deduplicate tweets
  Maintain a list of tweet IDs already saved and skip any that match existing ones. This prevents duplicate rows in Sheets or your database.
• Add NLP and enrichment
  Insert nodes for sentiment analysis, keyword extraction, or entity recognition to analyze tweet content before saving it.
• Send alerts
  Connect Slack, email, or another notification service to alert you when a tweet matches high-priority criteria, such as specific keywords or high engagement.

Security, compliance, and best practices

When scraping or collecting data from Twitter/X:

• Follow Twitter/X terms of service and API usage policies.
• Respect rate limits to avoid suspension or throttling.
• Be mindful of personal data and local privacy regulations.
• If you plan to run scraping at scale or distribute the workflow, make sure you have the appropriate API access level and permissions.

Quick checklist before your first run

Before clicking “Execute Workflow” in n8n, verify that:

• Twitter/X API credentials (HTTP Header Auth) are configured and tested.
• Google Sheets OAuth2 is connected, and the correct spreadsheet ID and sheet name are set.
• The search query in the Get Tweets node matches your target tweets.
• The loop limit and counter values are set to a safe number for initial testing (for example, a small number of pages).
• You have done at least one test run and reviewed each node’s output in the execution log.

FAQ

Can I change the columns stored in Google Sheets?

Yes. Open the Add to Sheet node and adjust the field mapping. You can remove fields you do not need or add new columns if you modify the Extract Info node to output additional data.

Do I have to use twitterapi.io?

No. The template is designed around a generic HTTP Request node. You can use any compatible Twitter/X API provider, as long as you update the URL, headers, and query parameters to match that provider’s requirements.

How can I stop the loop when there are no more tweets?

Extend the logic in the Increase Count or If node to check whether next_cursor is null or missing. If it is, you can stop the loop instead of relying only on a fixed counter.

What if I want to run this on a schedule?

Add a Cron node at the start of the workflow to trigger it at regular intervals, such as every hour or every day. Make sure your rate limiting and deduplication logic are suitable for repeated runs.

If you would like help tailoring this template, you can:

• Ask for a version customized for a specific search query or account.
• Add deduplication logic so your Google Sheet never receives duplicate tweet rows.
• Convert the workflow to store results in a database instead of, or in addition to, Google Sheets.

Once you are ready, import the template, connect your credentials, run a small test, and then gradually increase the loop limits as you confirm everything works correctly. If you encounter errors, check the node execution logs and adjust the configuration or rate limits as needed.

Call to action: