Anubis is a joke

2025-04-16

Over the past few months, a lot of people have turned to Anubis by Xe Iaso for trying to protect their sites, primarily Git forges and alternative frontends, against AI scraping.

Anubis is a new PoW captcha "solution" that (allegedly) holds out scrapers by slowing down your browsing and forcing you to enable JavaScript to pass a challenge to view the site. Once it's wasted a few seconds of your time and made you reevaluate the worth of whatever you were visiting, the stupid anime girl (previously AI generated) it shows you give a smile and you're on your way. This challenge only will work on Chromium and its Google-funded controlled opposition, Firefox. Basilisk does seem to work, though with broken CSS. It doesn't even work on Safari (allegedly, I don't own an iToy to test this with) and no other browser (until you read the next section) works on this.

There's one small problem to Anubis though. By default (which no installation I've checked changes), Anubis will only present a challenge to User-Agents with "Mozilla" and some obvious scraper agents, at the time of me writing this. You can check this in /data/botPolicies.json.

This means all one of those evil scrapers Anubis is supposed to protect against have to do to bypass Anubis is not use one of these User-Agents. It also means that you too can completely bypass this as I know it's been annoying a lot of people lately. You can curl a site using the default config (most of them), and it won't give an Anubis challenge, it'll just show you the site in its original form. No special options, no custom User-Agent, just curl http://domain.name and it'll let you through. This is applicable to your normal browser as well, just give it a user agent that doesn't contain "Mozilla" or any of the other terms in the file and you won't have any problems.

I was expecting a much more involved workaround to dealing with this piece of shit but no, all you have to do is give it a UA not containing some keywords.