Loading page content.
Loading page content.
Search visibility
A ranking is not a property of a keyword; it is a property of a keyword, a location, a device and a moment. Tracking it honestly means issuing each query from the place it is supposed to represent.
The problem
Search results are localised at a granularity below the country. Two queries issued from opposite ends of the same city can return different local packs, different map results and a different organic order underneath them. A tracker that runs from one datacenter reports the ranking for that datacenter's location and calls it the national position.
Search engines also defend aggressively against automated querying, and reasonably so — it is the most-scraped surface on the web. Sustained query volume from one address earns a challenge page within minutes, and the challenge is served with a 200 status, so an unchecked tracker records a page of nothing as a page with no rankings. That produces the worst possible artefact: a chart showing a client dropping out of the index on a day when nothing happened.
Personalisation compounds it. Prior queries, prior clicks and any signed-in state reshape the result set. A tracking rig that reuses a browser profile across thousands of queries is measuring a search history it created itself.
Why proxies
Geography is the measurement condition. If a client sells in Manchester, a Manchester exit is the only vantage point whose result is worth reporting, and city-level targeting is what makes that repeatable rather than approximate. Country targeting alone is enough for national queries and nothing else.
Rotation is the throughput condition. Every exit carries a query budget before it is challenged; spreading a keyword set across many residential addresses keeps each one well under it. Residential addresses specifically, because query defences weigh ASN reputation heavily and hosting ranges reach the challenge threshold far sooner.
Recommended product
A fresh residential IP on every request, or one held for 30 minutes
from $1.20/GB
Tracking rankings on smaller vertical search engines and site-search endpoints, which rarely weigh ASN reputation, is materially cheaper on the datacenter pool.
from $3.40/GB
Auditing the mobile SERP as a phone actually sees it — including features that only appear on carrier connections — needs a mobile ASN rather than a residential one.
Worked example
Two things make a rank check trustworthy: the exit is in the location being reported, and the parser refuses to interpret a challenge page as an empty result. The second is one condition and it prevents most of the bad data a tracker produces.
import requests
USER = "fp_8s2k4d19"
PASSWORD = "Xk7mQ2pTz9vRn4Ls"
def proxy_for(country: str, city: str | None = None) -> str:
segments = [USER, "country", country]
if city:
# City targeting narrows the exit to that metro, which is what a
# local-pack position is actually measuring.
segments += ["city", city]
return f"http://{'-'.join(segments)}:{PASSWORD}@gate.fleetproxy.com:8080"
class ChallengedError(RuntimeError):
"""The engine served an interrogation page, not a result set."""
def serp(query: str, country: str, city: str | None = None, device: str = "desktop"):
proxy = proxy_for(country, city)
response = requests.get(
"https://search.example/search",
params={"q": query, "num": 20},
proxies={"http": proxy, "https": proxy},
headers={"User-Agent": USER_AGENTS[device]},
timeout=(10, 30),
)
response.raise_for_status()
results = parse_results(response.text)
# Zero results for a commercial head term is not a ranking, it is a block.
# Recording it as position 0 is how a tracker invents a client's outage.
if not results and looks_like_challenge(response.text):
raise ChallengedError(query)
return results
print(serp("plumber near me", country="gb", city="manchester", device="mobile"))Replace the credential with the one in your dashboard. The gateway host, port and username format are the same across every product.
Pitfalls
Each of these is common, cheap to fix, and expensive to leave in place. They are listed in roughly the order teams hit them.
A challenge page parses to zero results, and a tracker that trusts it draws a cliff on the client's chart. Distinguish 'not ranking' from 'not measured', store them as different states, and alert on the measurement failure rate per engine.
For queries with local intent there is no national ranking to report. Pick the metros that matter commercially, track each explicitly with city targeting, and report them separately rather than blending them into one number.
Cookies and search history personalise later queries in the run, so results drift as the batch progresses. Use a fresh context per query and never sign in — the signed-in SERP is a different product.
Large result-count parameters are both an unusual pattern and a heavier page. Request the depth you actually report on, usually the first two pages, and paginate only for the keywords where deeper positions are tracked.
Questions
Start on 50MB of free residential bandwidth, measure your own targets, and scale into volume tiers that step the rate down as the job grows. Unused bandwidth never expires.
No card required for the trial. Cancel or downgrade at any time.