March 5, 2026

"How to Get Twitter/X Data Without the API: A Practical Guide"

Learn how to extract Twitter/X data without the official API using structured endpoints, with ready-to-use code examples.

Twitter's API used to be one of the most developer-friendly in the world. Then Elon Musk acquired the platform, rebranded it to X, and put a price tag on almost everything.

If you need X data for research, monitoring, or building a product, you now have a real problem. Let's walk through your options — and the one that actually works without draining your budget.

The X API Pricing Problem

In early 2023, Twitter shut down free API access. Here's what replaced it:

Free tier: Write-only. You can post tweets, but you can't read data. Useless for extraction.
Basic tier ($100/month): 10,000 tweet reads per month. That's roughly 300 tweets per day — enough for a toy project, not a real product.
Pro tier ($5,000/month): 1 million tweet reads. Better, but that's $60,000/year before you've written a line of product code.
Enterprise tier: Custom pricing. If you have to ask, you probably can't afford it.

For most teams doing sentiment analysis, competitor monitoring, or trend detection, the official API is either too expensive or too limited. And even if you pay, the data access restrictions have tightened — many fields that were once free now require elevated access.

Why Scraping X Directly Is Risky

The natural next thought: "I'll just scrape it myself."

X has some of the most aggressive anti-bot measures of any social platform. Here's what you're up against:

Authentication walls: Most useful data requires a logged-in session. You need auth tokens, which expire and rotate unpredictably.
Shadow bans and account suspensions: Automated access patterns get flagged fast. X actively monitors for scraping behavior.
IP-level blocking: Even with rotating proxies, X's fingerprinting can detect automated browsers.
Constant frontend changes: X ships UI updates frequently. A scraper that works today might break tomorrow when they rename a CSS class or restructure their GraphQL queries.
Legal gray area: X's terms of service explicitly prohibit scraping. While the legal landscape is evolving (see the hiQ v. LinkedIn ruling), it's a risk many teams don't want to carry.

Building and maintaining your own X scraper is a full-time job. And it's a job that has nothing to do with your actual product.

The Third Option: Unified Data APIs

There's a middle ground between paying $5,000/month for the official API and building brittle scrapers yourself: services that handle the extraction complexity and return clean, structured JSON.

These APIs abstract away the session management, proxy rotation, anti-bot evasion, and data parsing. You make a simple HTTP request, and you get back the data you need in a consistent format.

ByCrawl is one of these services. It covers X/Twitter along with 9 other platforms through a single API. Let's look at how it works in practice.

Getting X Data with ByCrawl

ByCrawl's X endpoints follow a simple pattern. All requests go to https://api.bycrawl.com with your API key in the header.

Fetching a User Profile

Get any public user's profile data — follower counts, bio, verification status, and more.

curl -s "https://api.bycrawl.com/x/user?username=elonmusk" \
  -H "x-api-key: your-api-key-here" | jq

The same request in Python:

import requests

response = requests.get(
    "https://api.bycrawl.com/x/user",
    params={"username": "elonmusk"},
    headers={"x-api-key": "your-api-key-here"},
)

user = response.json()
print(user)

Example response:

{
  "username": "elonmusk",
  "name": "Elon Musk",
  "bio": "Read @WallStreetSilv",
  "followers": 214500000,
  "following": 865,
  "posts_count": 52400,
  "verified": true,
  "profile_image_url": "https://pbs.twimg.com/profile_images/..."
}

Searching Posts by Keyword

Search across X for posts matching any query. Useful for brand monitoring, trend tracking, or research.

curl -s "https://api.bycrawl.com/x/search/posts?query=AI%20agents" \
  -H "x-api-key: your-api-key-here" | jq

import requests

response = requests.get(
    "https://api.bycrawl.com/x/search/posts",
    params={"query": "AI agents"},
    headers={"x-api-key": "your-api-key-here"},
)

results = response.json()
for post in results:
    print(f"@{post['author']['username']}: {post['text'][:80]}...")
    print(f"  Likes: {post['likes']} | Reposts: {post['reposts']}")

Example response:

[
  {
    "id": "1893847261045382",
    "text": "AI agents are going to change how we interact with software entirely...",
    "author": {
      "username": "techfounder",
      "name": "Sarah Chen"
    },
    "likes": 342,
    "reposts": 87,
    "replies": 45,
    "created_at": "2026-03-04T14:23:00Z"
  }
]

Getting a Specific Post's Engagement Data

If you have a post URL, you can pull its full details — text, engagement metrics, media attachments, and reply counts.

curl -s "https://api.bycrawl.com/x/post?url=https://x.com/elonmusk/status/1893847261045382" \
  -H "x-api-key: your-api-key-here" | jq

import requests

response = requests.get(
    "https://api.bycrawl.com/x/post",
    params={"url": "https://x.com/elonmusk/status/1893847261045382"},
    headers={"x-api-key": "your-api-key-here"},
)

post = response.json()
print(f"Text: {post['text']}")
print(f"Likes: {post['likes']} | Reposts: {post['reposts']} | Views: {post['views']}")

Pulling a User's Recent Posts

Get the latest posts from any account. Great for building feeds, tracking influencers, or competitor analysis.

curl -s "https://api.bycrawl.com/x/user/posts?username=openai" \
  -H "x-api-key: your-api-key-here" | jq

import requests

response = requests.get(
    "https://api.bycrawl.com/x/user/posts",
    params={"username": "openai"},
    headers={"x-api-key": "your-api-key-here"},
)

posts = response.json()
for post in posts:
    print(f"{post['created_at']}: {post['text'][:100]}...")

When to Use What

Here's a quick comparison of your three options:

	Official X API	DIY Scraping	Unified API (ByCrawl)
Cost	$100-$5,000+/mo	Free (+ engineering time)	Pay-per-request
Setup time	Hours (approval process)	Days to weeks	Minutes
Maintenance	Low (API is stable)	High (breaks constantly)	None (handled for you)
Rate limits	Strict (tied to tier)	Risk of bans	Managed for you
Data format	Structured JSON	Raw HTML to parse	Structured JSON
Legal risk	None	Moderate	None (provider's responsibility)
Multi-platform	X only	Per-platform effort	10 platforms, one API

The official API makes sense if you have the budget and only need X data. DIY scraping can work for one-off research tasks where reliability doesn't matter. For everything in between — production workloads, multi-platform data needs, teams that want to focus on their product — a unified API is the practical choice.