Menu

How We Built an AI-Powered Web Scraping Bot Without Knowing How to Code

AI-assisted Lighthouse web scraping for GreyFox Ads Green Score

In 2024, when GreyFox Ads was just getting started, we wanted to build something that would make our small French programmatic ad network stand out. The problem? Our idea required collecting technical data from hundreds of websites. And we did not know how to code. 😅

The short version: With ChatGPT as a coding assistant, we built a first-generation bot to retrieve desktop and mobile Lighthouse scores for roughly 500 French websites. It took about three hours, thirty rounds of troubleshooting and plenty of coffee. It was not a sophisticated carbon-measurement system, but it turned an otherwise manual task into a repeatable workflow.

1. Why we needed our own Green Score

At the start of the GreyFox Ads adventure in 2024, we were looking for a way to differentiate our programmatic offering beyond the usual targeting and inventory selection promises.

We looked at solutions addressing the environmental impact of digital advertising. Many focused on variables such as connection type, creative weight, delivery conditions and supply-chain characteristics. We wanted to ask another question: how efficiently is the publisher's website itself built and delivered?

That became the starting point for our proprietary Green Score—a practical signal designed to complement, not replace, other quality and sustainability criteria.

2. Why we started with Google Lighthouse

Google Lighthouse is a widely used auditing tool for web performance and other aspects of website quality. It can produce separate results for desktop and mobile conditions, making it a useful starting point for comparing technical efficiency across sites (provided the test configuration and methodology remain consistent).

Our hypothesis was simple: a site with a more efficient technical implementation might be a better candidate for digital sobriety than a similarly situated site with heavy, poorly optimized pages. That is a hypothesis about technical efficiency, not a direct measurement of carbon emissions.

But there was a logistical problem: manually auditing the top 500 French websites would have taken days. We needed automation.

3. Three hours, thirty errors and one AI coding assistant

There was just one tiny detail: we did not know how to code. So we asked ChatGPT to help us build a scraping and automation bot.

What followed was not a magical one-prompt success story. We spent roughly three hours working through around thirty errors, often sharing screenshots of error messages and asking the assistant to explain what had gone wrong and how to fix it.

Eventually, we had a working first-generation tool capable of collecting desktop and mobile Lighthouse scores for approximately 500 French sites. Not bad for an afternoon of trial, error and coffee. ☕

The important lesson was not that AI had replaced software engineers. It was that an industry specialist could use an AI assistant to prototype a useful technical workflow without starting from a traditional development background.

4. What the workflow actually does

At a high level, our workflow turns a list of websites into a structured set of technical indicators:

StepActionWhy it matters
1. InputPrepare the list of French publisher domains or URLs.Establish a consistent analysis perimeter.
2. AuditRun or retrieve Lighthouse results for mobile and desktop.Capture comparable technical performance signals.
3. CollectExtract the scores into a structured file.Avoid hundreds of manual copy-and-paste operations.
4. ReviewCheck failures, outliers and inconsistent measurements.Prevent a temporary error from becoming a misleading ranking.
5. UseFeed the results into a broader Green Score methodology.Turn technical information into a usable curation signal.

This is an explanation of the workflow, not a claim that our original prototype used a particular software library or API. There are several ways to automate Lighthouse, and the appropriate implementation depends on scale, access rules and available infrastructure.

5. A Lighthouse score is not a carbon footprint

This distinction matters. Lighthouse does not directly measure the CO₂ emitted by a website or an advertising impression. Its performance metrics can serve as indirect indicators of technical efficiency, but they do not capture the entire environmental impact of hosting, networks, devices or ad delivery.

Scores can also change with test conditions, page selection, network simulation and website updates. For that reason, a meaningful methodology needs repeatable tests and clear assumptions (a single homepage score is not necessarily representative of every article page).

For an advertiser, a sustainability-oriented score should therefore be treated as one decision signal among several, alongside brand safety, attention, supply-path quality and actual campaign outcomes.

6. From Lighthouse scores to a broader Green Score

Our original bot was only the beginning. We subsequently evolved the GreyFox Ads Green Score to incorporate the bid requests per page-view ratio, helping us identify potentially excessive ad-auction activity and flag inventory that may warrant closer examination for Made-for-Advertising (MFA) characteristics.

A high bid-request-to-page-view ratio is a useful warning signal, but it is not proof that a website is MFA (the figure depends on refresh behavior, header bidding configuration, ad slots and measurement scope).

This work fits a wider question we regularly explore: how much of the Open Web actually meets demanding quality criteria? Better signals can help us be more selective without confusing quantity with value.

7. What non-developers can learn from this experiment

If you work in media, product or AdTech and have a repetitive data task, you do not necessarily need to wait for a fully staffed engineering project to test your idea.

And no, the lesson is not “anyone can build anything in three hours.” Our experience was a relatively simple, first-generation automation task. Production-grade tooling still requires maintenance, monitoring and technical judgment.

8. At GreyFox Ads, ingenuity is part of the product

We are a small French programmatic ad network, and that makes resourcefulness especially important. Our approach combines local knowledge of the French Open Web with hands-on curation, practical quality signals and transparent supply selection.

We also explore how Curator Seats can make these choices actionable for media buyers, and how better inventory signals complement more meaningful campaign KPIs than viewability alone.

For media traders outside France, our Master Curated Deal brings French-market expertise into a single activation route. And yes, we still like building clever little tools when they solve a real problem. 🤘

About the author

Max Legrand is the founder of GreyFox Ads, a French programmatic ad network specializing in curated Open Web inventory, transparent supply paths and quality-driven activation.

Activate quality French Open Web inventory

Looking for a partner combining French publisher knowledge, practical technology and curated programmatic activation?

Discover the Master DealContact GreyFox Ads

Original story: Read the original LinkedIn post. The figures about development time, debugging and site coverage reflect the author's firsthand account.