Twitter advanced scraper

Problem

Collect historical tweets on chosen keywords, for sentiment analysis or research, without paying for the official X Enterprise API.

Approach

A Python script that joins the keywords into one OR query, sends it to the TwitterAPI.io advanced search API and follows the result cursor page by page until it reaches the target count. When the API answers 429 it waits 15 seconds and tries again, and it pauses 6 seconds between pages. The tweets go into a pandas DataFrame and out as an Excel file.

Impact

• 7fields per tweet: author, name, date, text, likes, retweets, link
• 429rate-limit responses waited out instead of ending the run
• .xlsxsaved straight to Excel, with no JSON to convert