gabriele pieretti

ARTICLE

A Free Audit Is a Server That Fetches Strangers' URLs: Building a Lead Magnet in Laravel

LaravelsecuritySaaS

For LocalPro AI, the local marketing service I run for restaurants and pizzerias in Italy, I wanted the classic lead generation tool: type in your website address, get a score out of 100 and a list of things to fix in a few seconds, and leave your email to receive the full report. The visible part took an afternoon. The rest, meaning everything needed so that a public form that fetches URLs chosen by strangers does not become a liability, took much longer. This post is about that rest, because it applies to any lead magnet.

What it measures, and what I chose not to measure

The audit downloads the home page of the site you give it and runs nine checks, each with a fixed weight:

public const WEIGHTS = [
    'reachable'       => 25, // the site opens
    'https'           => 15,
    'speed'           => 10, // full marks within 1.5 s, half within 3 s
    'mobile'          => 10, // meta viewport with width=device-width
    'meta'            => 10, // title 10-70 characters, description 50-170
    'structured_data' => 10, // JSON-LD or microdata of a LocalBusiness type
    'nap'             => 10, // name, address, phone readable on the page
    'maps'            => 5,  // link or map pointing to the Google listing
    'crawl'           => 5,  // robots.txt and sitemap
];

The score is the sum, capped at 100. No mysterious weighted average: whoever reads the report should be able to work out on their own why they got 62.

What it does not do is more interesting than what it does. It does not query Google: no Places API, no scraping of search results, which Google's terms forbid. As for the Google listing, I only check whether the site links to it. That is a deliberate limit: an audit that starts by breaking the terms of your customer's most important supplier is not a great business card.

There is also a bit of AI, and it is off by default. When enabled it only rewrites the advice in the email into friendlier Italian, and the prompt contains the business type, the score and the advice: never the business name, the email or the URL. It has a cap of 100 calls a day on top of the general AI budget, and if it fails or hits the cap the static texts are used. The report works the same without it.

The real problem: a server that fetches whatever you tell it

A form that accepts a URL and goes to fetch it is, by definition, an HTTP request made by my server on behalf of anyone. If you are not careful, that anyone asks for http://169.254.169.254/ (the cloud metadata service), http://localhost:6379 or the database IP on the internal network. It is called SSRF, and a lead magnet is the perfect place to introduce it without noticing.

The fetcher I wrote, SafeHttpFetcher, does the following, in order:

The core is the set of options passed to curl through Laravel's HTTP client:

Http::withHeaders([
    'Accept-Encoding' => 'identity',
])->withOptions([
    'allow_redirects' => false,
    'decode_content'  => false,
    'timeout'         => $timeout,
    'curl' => [
        CURLOPT_RESOLVE   => ["{$host}:{$port}:{$pinned}"],
        CURLOPT_PROTOCOLS => CURLPROTO_HTTP | CURLPROTO_HTTPS,
        CURLOPT_PROXY     => '',
        CURLOPT_MAXFILESIZE => $maxBytes,
        CURLOPT_NOPROGRESS  => false,
        CURLOPT_PROGRESSFUNCTION =>
            fn ($ch, $dlTotal, $dlNow) => $dlNow > $maxBytes ? 1 : 0,
    ],
])->get($url);

Three lines deserve a comment.

CURLOPT_RESOLVE is the defence against DNS rebinding: the attacker's domain answers with a public IP when I check it and with 127.0.0.1 half a second later, when curl connects. If I decide the IP, there is no second lookup.

CURLOPT_PROXY => '' looks pointless until you find out that curl honours the HTTP_PROXY and HTTPS_PROXY environment variables. With a proxy in the middle, the proxy resolves the host and the pin is gone. The empty string turns it off even if someone sets those variables in the container one day.

decode_content => false together with Accept-Encoding: identity: otherwise Guzzle asks curl to decompress without limits, and a gzip response of a few KB can turn into a few GB in memory. I do the decompression myself, with the same size cap. The progress function aborts the download as soon as it goes over, even when the server does not send a Content-Length.

One last detail, found while writing the tests: the timeout applies to the whole redirect chain, not to each hop. Otherwise a slow site with three redirects keeps a worker busy for four times the expected time. On top of that there is a time budget for the entire audit, home page plus robots.txt and sitemap.

Rate limits: per IP, and per address

Every analysis fetches a third-party site, and every report sends an email. Those are two different resources, with separate limits: analyses are limited per IP, per minute and per day; reports are limited per IP and per email address, and the address only enters the limiter key as a SHA-256 hash.

The per-address limit is the one that matters: someone rotating IPs cannot use the form to flood another person's inbox. The address only reaches the rate limiter key as a hash, so Redis does not hold a plain list of emails. There is no captcha: for a form aimed at a pizza maker opening it on their phone between two services, I prefer a honeypot (a hidden field called fax_aziendale, "company fax", that no human fills in) and tight limits.

A cached page cannot carry a CSRF token

This is the trade-off I thought about the longest. The form page is served from the public cache, like the rest of the marketing site, because it has to be fast. But a cached page is the same for everyone, so it cannot contain a session CSRF token. The two audit POST routes are therefore excluded from CSRF verification in bootstrap/app.php.

I accepted the exclusion after looking at what a third-party site could do by making your browser submit that form: start an analysis (limited per IP) or have a report sent to an address (limited per address). Neither acts on an account, because there are no accounts. The limits do the job the token usually does. If one day the audit had to touch data of a logged-in user, that decision would have to be made again from scratch.

The result pages, on the other hand, go out with Cache-Control: no-store, private and X-Robots-Tag: noindex, nofollow: someone's pizzeria score must not end up in a shared cache or on Google.

The lead only exists once there is consent

The analysis creates nothing in the database. The result stays in the cache for 60 minutes under a random 40 character token, and the lead is written only when the person asks for the report, inside a transaction together with the consent records. Then the token is deleted: submitting the same form twice creates no duplicates and sends no second email.

On consent:

The audit sits behind a switch, off by default: when off, the form and the analysis answer 404 and the page disappears from the sitemap and the privacy notice. The unsubscribe routes are deliberately left outside the switch. Anyone who received an email must be able to unsubscribe even on the day I decide to turn everything off.

The tests

There are 73 Pest tests in seven files, and more than half are not about the score. They cover redirects to private IPs, hosts resolving to several IPs of which one is internal, compressed responses over the limit, slow redirect chains, the token expiring, the same form submitted twice, consent given by a GET that must not count. Computing the score is the easy part to test and the one that breaks least.

The point, if you are building one

A lead magnet looks like marketing, but from the server's side it is a public, anonymous endpoint that makes network requests and sends email to addresses supplied by anyone. Treating it as a side feature is the quickest way to have your first security incident on the very page meant to bring in customers. The rule I set myself: everything the form does on behalf of the visitor needs a limit, and every limit needs a test.

If you need a tool like this for your product, or want someone to look at the one you already have, this is how I work. On Laravel security I have also written about path traversal in Storage::path().

← All articles