Python for SEO: 7 practical scripts for in-house teams (2026)

Python for SEO means using short scripts to pull, join and check search data at a scale your tools' screens can't reach. For an in-house team, the jobs that pay back first are Search Console API pulls, bulk index checks and redirect tests after a migration. Next come Googlebot log checks, GA4 audits, bot-spike alerts and raw-versus-rendered HTML comparisons. You don't need to be a developer. You need read-only access, a safe home for credentials and one person who owns the code.

Here at Searchmaxxed, we're an SEO and AI SEO team for law firms, clinics and premium builders, and these are the checks we lean on. Two of the faults below came from client accounts, where GA4's standard reports hid them. If you'd rather have the rankings and the pipeline than look after the scripts, we're the team to call.

Should you learn Python for SEO?

Learn it if you manage a large site, several properties or frequent releases. Skip it if your site has a few hundred pages and your crawler and Search Console already answer every question you have.

Volume is the tipping point. Google's own Search Console data guide puts the interface export at 1,000 rows. A B2B SaaS site with docs, integrations and a blog can rank for tens of thousands of queries. Most of that picture never reaches the screen.

Python also fits the jobs you repeat. A redirect map with thousands of lines eats a morning by hand. In code, minutes. The same goes for rechecking every integration page after a template change.

You don't need a computer science degree. Ahrefs makes the same point in its beginner's guide: Google Colab runs notebooks in the browser, and a chatbot can explain most error messages. What you do need is the habit of checking the output before you trust it.

What can Python do for SEO that your tools can't?

Python joins data your tools keep apart. It also runs checks they don't offer. Here are the seven jobs we'd start with, the library each one needs and the official limit that shapes it.

Every script runs on your own data. None needs a paid SEO tool.

How do you pull Search Console data with Python?

Use the Search Analytics endpoint of the Search Console API with the google-api-python-client library. Ask for one day at a time, 25,000 rows per request, and page through with startRow until a request comes back empty.

Google's query reference sets rowLimit between 1 and 25,000, with 1,000 as the default. Its guide to getting all your data recommends one daily query for one day's data. Expect a lag. Fresh data usually lands two to three days later.

from google.oauth2 import service_account
from googleapiclient.discovery import build
import pandas as pd
SCOPES = ["https://www.googleapis.com/auth/webmasters.readonly"]
creds = service_account.Credentials.from_service_account_file("gsc-reader.json", scopes=SCOPES)
gsc = build("searchconsole", "v1", credentials=creds)
def pull_day(site, day):
    rows, start = [], 0
    while True:
        body = {"startDate": day, "endDate": day, "dimensions": ["page", "query"],
                "type": "web", "rowLimit": 25000, "startRow": start}
        batch = gsc.searchanalytics().query(siteUrl=site, body=body).execute().get("rows", [])
        if not batch:
            return pd.DataFrame(rows)
        rows += [{"page": r["keys"][0], "query": r["keys"][1], "clicks": r["clicks"],
                  "impressions": r["impressions"], "position": r["position"]} for r in batch]
        start += 25000
df = pull_day("sc-domain:example.com", "2026-10-28")
df.to_csv("gsc-2026-10-28.csv", index=False)

If you write only one script, make it this one. Three things trip people up.

• The totals won't match. Rows leave out anonymised queries, so they add up to less than the chart total, as Google's data guide explains.
• Quotas apply per site and per user. Google's usage limits allow 1,200 Search Analytics queries a minute for each. Heavy page-and-query requests can hit a separate load limit first; it clears after about 15 minutes.
• The service account needs a seat. Add its email address as a user on the property, at the lowest level that works.

For the biggest sites, Search Console's bulk data export sends a daily copy to BigQuery instead. It holds all the performance data Search Console has, except anonymised queries. Choose the export if a data team already runs BigQuery and wants every row in one warehouse. If one analyst needs answers by Friday, the API gets there faster.

How do you check index status in bulk?

Call the URL Inspection API for your priority URLs and save the verdict, coverage state, Google's chosen canonical and last crawl time. It returns the same data as the Inspect URL box in Search Console, one URL per call.

The quota sets the plan. Google allows 2,000 inspections a day and 600 a minute per site, so a 40,000-URL site needs a shortlist. Start with the pages that earn demos and sign-ups, then add new pages after each release.

def inspect(site, url):
    res = gsc.urlInspection().index().inspect(body={"inspectionUrl": url, "siteUrl": site}).execute()
    idx = res["inspectionResult"]["indexStatusResult"]
    return {"url": url, "verdict": idx.get("verdict"), "coverage": idx.get("coverageState"),
            "google_canonical": idx.get("googleCanonical"), "user_canonical": idx.get("userCanonical"),
            "last_crawl": idx.get("lastCrawlTime")}
results = pd.DataFrame([inspect("sc-domain:example.com", u) for u in priority_urls])
results[results.google_canonical != results.user_canonical].to_csv("canonical-mismatch.csv", index=False)

The read-only scope works here too. Watch the last line. It saves every page where Google picked a different canonical from the one you declared, which is often the first sign of duplicate templates.

We used URL Inspection to watch a brand-new site of ours get indexed. Here's what it showed:

We launched a new site with 161 guides, each built for one search from the live results and People Also Ask. We submitted the sitemap and pinged IndexNow. No paid indexer. About 25 hours after Google's first crawl, 203 of 228 URLs were indexed. Indexed isn't ranked, though: 5,198 impressions in ten days and 5 clicks.

That makes indexing step two, not the finish line. Join the inspection results to your Search Console rows, and the indexed pages that still earn nothing show up at once.

How do you test redirects with Python after a migration?

Load your redirect map into pandas and request every old URL with requests, recording each hop. Flag three failures: more than one hop, a final URL that isn't the mapped target, or a final status other than 200.

Chains cost you. Google says its crawlers follow up to 10 redirect hops by default, and each extra hop is one more place for the move to break. The requests library keeps every hop in `Response.history`, so the check takes a few lines.

import requests
HEADERS = {"User-Agent": "example-seo-redirect-check (seo@example.com)"}
def check(old, expected):
    r = requests.get(old, headers=HEADERS, timeout=15)
    codes = [h.status_code for h in r.history]
    ok = len(codes) == 1 and codes[0] in (301, 308) and r.url == expected and r.status_code == 200
    return {"old": old, "hops": len(codes), "codes": codes, "final": r.url, "status": r.status_code, "ok": ok}
redirect_map = pd.read_csv("redirect-map.csv")
report = pd.DataFrame([check(o, n) for o, n in zip(redirect_map.old, redirect_map.new)])
report[~report.ok].to_csv("redirect-failures.csv", index=False)

Set a timeout on every request. The requests docs say nearly all production code should, because without one a script can hang indefinitely. Run the check on staging before launch, again on launch day, then weekly until the old URLs fade from Search Console. Our site migration SEO checklist covers the rest.

How do you parse server logs for Googlebot with Python?

Read the log into pandas and keep the lines whose user agent says Googlebot. Then verify each IP with a reverse DNS lookup and a forward lookup on the hostname it returns. Fakes fail that test.

Google's verification steps take four moves. Look up the IP's hostname and check it ends in googlebot.com, google.com or googleusercontent.com. Then look up that hostname and confirm it returns the same IP.

import socket
from functools import lru_cache
@lru_cache(maxsize=None)
def is_googlebot(ip):
    try:
        host = socket.gethostbyaddr(ip)[0]
        return host.endswith((".googlebot.com", ".google.com", ".googleusercontent.com")) \
            and ip in socket.gethostbyname_ex(host)[2]
    except OSError:
        return False

Why the cache? The same few hundred IPs can repeat millions of times, and each DNS lookup takes time. This version checks IPv4 addresses; for IPv6, swap in socket.getaddrinfo. For huge logs, match IPs against Google's published JSON ranges instead. The advertools library adds a logs_to_df() function that turns raw logs into a table.

Want the full method? Our log file analysis guide covers crawl waste, orphan pages and AI crawlers.

How do you audit GA4 with Python?

Use the GA4 Admin API to list your key events and flag any name over 40 characters. Then use the GA4 Data API to break traffic down by hour, screen size and browser, so bot bursts stand out from real growth.

Find key events that never count

GA4's collection limits cap event names at 40 characters. Google adds that a key event over 40 characters won't report as a key event. The interface gives you no warning at all. It happened to a client:

A client's four enquiry types never counted as conversions. Their event names ran 41 to 50 characters; GA4 cuts at 40. The 40-character "career" event counted every time. Nobody had noticed, because the events still showed up.

Finding it takes a short script, and fixing it means shorter names in your tag manager:

from google.analytics.admin_v1beta import AnalyticsAdminServiceClient
admin = AnalyticsAdminServiceClient()
names = [k.event_name for k in admin.list_key_events(parent="properties/123456789")]
names += open("tracking-plan-events.txt").read().split()
for name in sorted(set(names)):
    if len(name) > 40:
        print(len(name), "characters, won't count as a key event:", name)

Run it against your tracking plan too. The broken names may never reach the key event list at all.

Spot bot bursts that look like growth

Pull sessions by hour with the GA4 Data API, then check the busiest hours for one screen size, one browser and only new users. Real visitors don't arrive like that.

A client's GA4 showed AI-assistant visits jump from 354 to 6,050 in a month. Almost none of it was real. About 5,700 came in three one-hour bursts: same 1920x1080 desktop Chrome, all new users, no referrer, about 20 seconds each, landing on two pages. Real AI traffic was 385 sessions against 354 the month before, roughly flat.

from google.analytics.data_v1beta import BetaAnalyticsDataClient
from google.analytics.data_v1beta.types import DateRange, Dimension, Metric, RunReportRequest
ga = BetaAnalyticsDataClient()
dims = ["dateHour", "sessionDefaultChannelGroup", "screenResolution", "browser"]
req = RunReportRequest(property="properties/123456789",
    dimensions=[Dimension(name=d) for d in dims],
    metrics=[Metric(name="sessions"), Metric(name="newUsers")],
    date_ranges=[DateRange(start_date="2026-09-01", end_date="2026-09-30")], limit=100000)
df = pd.DataFrame([[v.value for v in r.dimension_values] + [float(m.value) for m in r.metric_values]
                   for r in ga.run_report(req).rows], columns=dims + ["sessions", "new_users"])
hourly = df.groupby("dateHour").sessions.sum()
spikes = hourly[hourly > hourly.median() * 10].index
print(df[df.dateHour.isin(spikes)].groupby(["screenResolution", "browser"]).sessions.sum().nlargest(5))

If one screen size and one browser own a spike, filter it out before anyone calls it growth. For the other gaps between the two tools, see our guide to why Search Console and GA4 numbers differ.

How do you catch JavaScript rewriting your canonicals?

Fetch each page twice. First grab the raw HTML with requests, then render it in a headless browser with Playwright. Compare the title, canonical and robots tags. A mismatch means the browser changes what the server sent.

View-source audits miss this. Google runs JavaScript before it indexes a page. Its JavaScript SEO basics warn against using JavaScript to change the canonical from the one in the original HTML. A rendered check proves yours doesn't.

from bs4 import BeautifulSoup
from playwright.sync_api import sync_playwright
def tags(html):
    soup = BeautifulSoup(html, "html.parser")
    link = soup.find("link", rel="canonical")
    return {"title": soup.title.string if soup.title else None, "canonical": link.get("href") if link else None}
def compare(url):
    raw = tags(requests.get(url, headers=HEADERS, timeout=15).text)
    with sync_playwright() as p:
        browser = p.chromium.launch()
        page = browser.new_page()
        page.goto(url, wait_until="networkidle")
        rendered = tags(page.content())
        browser.close()
    return {"url": url, "raw": raw, "rendered": rendered, "match": raw == rendered}

Install Playwright's browsers once with playwright install, as its Python docs show. After every release, run the check on one URL per template, because ten templates can sit behind many thousands of pages.

Which Python libraries should SEO teams use?

Four libraries cover most SEO scripts. Use requests to fetch pages, Beautiful Soup to read HTML, pandas to join data, and Google's client libraries for Search Console and GA4. Add others as needed.

Scrapy suits full custom crawlers. Once the basics run, scikit-learn or spaCy can handle keyword clustering. Where the code runs matters less. Google Colab or a local Jupyter notebook is fine for learning. Anything you'll rely on belongs in a scheduled job on a company server, under version control.

How do you run Python for SEO safely?

Give every script the least access it needs, keep credentials out of code, and name an owner. Your security team will ask about all three, so settle them first.

Here's the checklist we'd hand to IT:

1. Read-only scope. Search Console offers a `webmasters.readonly` scope, and reporting or inspection scripts need nothing more.
2. The lowest Search Console permission. Google's permission levels run from owner to full user to restricted user, which has view rights on most data.
3. No keys in notebooks or Git. Google Cloud's service account guidance says to use keys only when there's no viable alternative. If you must, keep them in a secret manager and rotate them.
4. Respect the quotas. When the API says slow down, back off and retry.
5. Crawl politely. Name your script in the user agent, set timeouts and throttle requests, even on your own site.
6. Treat logs as personal data. Server logs hold IP addresses, so store and delete them under your privacy policy.
7. Review every script, chatbot-written ones included. A second person reads the code before it runs against production.
8. Name an owner and keep history. Someone has to fix the script when an API changes, and the raw exports need a safe home.

None of it slows a good team. It's what keeps the scripts running after the person who wrote them moves on.

Should you build scripts, buy a tool or hand it to a team?

Build scripts when someone on the team codes and your questions are specific to your site. Buy a crawler for routine audits. Hand the work to a team when you want fixes and rankings, rather than one more dataset to maintain.

We run Searchmaxxed, so we've put it first in this table.

Most teams end up with a mix. Scripts answer the odd questions and a crawler covers the routine, while a team turns the findings into rankings and pipeline.

Is Searchmaxxed right for you?

We're a strong fit if you're a law firm, clinic or premium builder losing ground in Google or missing from AI answers. You may already know what's broken. What's missing is a team to fix it and prove it moved pipeline.

Here's what you get. You talk to the people doing the work, with no account manager in between. We wire up and check Search Console and GA4, the kind of checking that found the two GA4 faults above. We also track which searches and AI answers turn into revenue, through your CRM as well as GA4. You own your site, content and data from day one. On the technical side, our technical SEO service covers crawling, indexing, rendering and site speed, from the first audit to the finished fixes.

One client came to us with a website that wasn't working. Three months later he was on page one for one of the most competitive searches in his industry. Now he has more high-intent leads than he can handle, and we're building the systems to handle them.

Across the client book:

Searchmaxxed's work has generated more than $15 million in revenue through search for its clients.

"I can't believe this worked." That's what we hear. Clients tell us they finally found a growth partner in search that keeps its word and gets things done.

We're the wrong call in two cases. If you want to learn Python and enjoy building the scripts yourself, this article and a free Colab notebook will take you a long way. And if you need leads this month just to survive, search takes longer than that.

Clean data is the foundation. Getting named by ChatGPT, Gemini, Perplexity and AI Overviews takes more, and our guide to showing up in AI search sets out the steps.

What should you do this week?

Run the Search Console script for a single day, three days back, and count the rows. More than 1,000? Then you've found search demand your team has never seen.

Want us to look with you? Get your free search analysis. We'll benchmark you against your top competitors across Google, ChatGPT and the other AI engines. We'll show you where they're named instead of you, and the few moves that will matter most.

Frequently asked questions