apunkt/ShopExtract
1
0
Fork 0
mirror of https://github.com/Coding-Doctor-Omar/ShopExtract.git synced 2026-06-07 07:55:18 +02:00
Code Issues Projects Releases Packages Wiki Activity
The only tool you need to generate Shopify import-compatible CSVs from ANY Shopify store on the internet.
12 commits 1 branch 0 tags 268 KiB
Find a file
Exact
Omar cda660d240 Update scraper name in the about screen.
2026-05-31 09:03:15 +03:00
images Initial commit 2026-05-28 09:44:49 +03:00
.gitignore Initial commit 2026-05-28 09:44:49 +03:00
main.py Update scraper name in the about screen. 2026-05-31 09:03:15 +03:00
README.md Add full stops to sentences 2 and 3 in Features. 2026-05-30 08:58:05 +03:00
requirements.txt Replace curl_cffi with wreq. 2026-05-30 08:19:48 +03:00
shopify_csv_extractor_logo.svg Initial commit 2026-05-28 09:44:49 +03:00

README.md

ShopExtract — The Only Tool You Need to Extract Full Shopify Product Catalogs

Repo Logo

Changelog

May 30, 2026

  1. Migrated from curl_cffi to wreq.
  2. Upgraded the collections aggregation strategy to have concurrency at the collections level, resulting in a tremendously faster collections aggregation strategy for stores with more than 25k products.

Features

  1. Interactive menu-based text-user-interface (TUI) with live on-screen scraping progress.
  2. Very fast scraping (~ up to 3,000 products/sec).
  3. Bypasses Cloudflare's anti-bot protections.
  4. Handles timeouts via auto-retries and exponential back-off.
  5. Bypasses /products.json endpoint blocks by auto-detecting a store's myshopify.com domain.
  6. Produces ready-to-import CSVs (with proper column and row-formatting) to allow the user to immediately use the CSVs in Shopify.
  7. Respects the 15-MB-size and 50,000-row Shopify limits per CSV. For large catalogs, it auto-splits the data into multiple CSVs.

Outputs

For any Shopify store, the scraper produces a JSON Lines (.jsonl) file that contains the entire product catalog and one or more CSV files with proper formatting for immediate Shopify product import.

Limits

For stores with product catalogs of more than 25,000 products, the scraper falls back to the collections aggregation strategy, which makes it slower (mitigated significantly in the May 30, 2026 update).

Setup

Make sure you navigate to the project folder, then write the commands below.

Create new virtual environment

MacOS/Linux

python3 -m venv venv

Windows

python -m venv venv

Activate virtual environment

MacOS/Linux

source venv/bin/activate

Windows CMD

.\venv\Scripts\activate

Windows Bash

source venv/Scripts/activate

Install dependencies

MacOS/Linux

pip3 install -r requirements.txt

Windows

pip install -r requirements.txt

Run the tool

MacOS/Linux

python3 main.py

Windows

python main.py

Usage

  1. Press '1' in the main menu screen and press ENTER.
  2. Type your target store URL (e.g. https://www.gymshark.com/) and press ENTER.
  3. Type your output CSV name and press ENTER.
  4. Wait until scraping is complete.
  5. Enjoy your CSVs.

Screenshots

Menu

Menu

Menu

Scraping

Scraping

Output

Output

CSV