post: six months reading every Italian municipality

src/content/blog/six-months-reading-italian-municipalities.mdx

@@ -0,0 +1,146 @@
+---
+title: "Six months reading every Italian municipality"
+description: "It started as a way to learn Rust with one town near my house. Now it's albopretorio.io, Piazza in Festa, RadarPA and a new role at Welance."
+pubDate: 2026-09-29
+heroImage: "../../assets/photos/six-months-reading-italian-municipalities/cover.jpg"
+tags: ["rust", "side-projects", "open-data", "public-administration", "ai", "welance"]
+---
+
+On April 1st I opened a new Rust workspace.
+<br/>
+No, it was not a joke, even if the date was suspicious 😄
+
+The plan was very small: learn Rust by scraping the public notice board of **one** municipality, Scicli, a town 15 minutes from my house. One town, one website, one crate.
+
+Six months later the numbers are these:
+
+- **7,896** Italian municipalities, all 20 regions, plus unions of towns, health agencies, parks, provinces... **9,617** public bodies in total
+- more than **5.1 million** administrative acts, 4.6 million of them with the PDF downloaded
+- **17** Rust crates in the workspace
+- **2,161** commits
+- three products and one new job title
+
+So... something went a bit out of control. This post is the recap, mostly for future-me, because I already forget the details of things I did in May.
+
+## What an "albo pretorio" is
+
+In Italy every public body has an *albo pretorio*: an online notice board where they must publish their acts. Council decisions, spending approvals, tenders, public works, the money for the town festival, everything.
+
+It's public by law. Anyone can read it.
+<br/>
+In theory.
+
+In practice every municipality buys its website from a vendor and every vendor built its own notice board, in its own way, with its own bugs. There is no standard, no API, no feed. You want to know what happened in 7,896 towns? Open 7,896 websites, one by one, every day.
+
+So I built [albopretorio.io](https://albopretorio.io): a crawler that does it for you and one place where you can search everything.
+
+## The scraper that became a zoo
+
+The first scraper was for one vendor platform. Easy. Then I added a second town and it was on a different platform. Then a third one.
+
+At some point I stopped counting towns and started counting **vendors**. Today there are about **100** of them in the codebase, and each one has a personality:
+
+- old Java server pages with a hidden "ViewState" that you must send back exactly as you received it, or nothing works
+- sites that only work inside a real browser, so the crawler drives a headless Chromium
+- PDFs that say they are PDFs but they are signed `.p7m` files
+- pages that lie about their encoding
+- "search" pages that return everything if you pass `limit=0` (thank you, I'll take it 🙏)
+
+One number surprised me: **67.6%** of Italian municipalities publish their notice board on a domain they don't own. It's the vendor's domain. The official record of what a town decided lives on a `.com` of a private company.
+
+And then there is my favorite discovery. Some public bodies were closed years ago (mountain communities, old consortiums) and nobody renewed their domain. Somebody else bought it. One of them, still listed as "official website" in the national registry of public administrations, today is an **online pharmacy selling Viagra without prescription**. In Italian. 🙃
+
+Also funny: I tried to add the 20 Regions too, before understanding that most Regions **don't have an albo pretorio** at all. Lesson learned: read the law before writing the crawler.
+
+## About Rust (and about honesty)
+
+The original idea was a strict "learning mode": I asked Claude to be my Rust tutor, explain the compiler errors and **not** write code for me.
+<br/>
+That rule lasted less than I want to admit.
+
+When the project became bigger than a toy, I switched to the same way of working I described in the [Astro migration post](/blog/from-angular-to-astro): I decide what to build, the agent writes a lot of it, I review, I test, I fix what it doesn't see.
+
+Did I learn Rust? Yes, but in a different way than I planned. I learned to *read* it very well, I learned ownership by fighting with it on real problems, I learned why `Result` and `?` are better than try/catch. I didn't learn it by writing every line.
+<br/>
+I'm fine with that. The goal changed on the way.
+
+What I would say to anyone doing the same: **the agent is great at adding, you must be the one who checks.** Every time I trusted a number without looking at the data myself, the number was wrong. Every time.
+
+## PDFs, LLMs and privacy
+
+The notice board gives you a title and a PDF. The interesting part is inside the PDF: who gets the money, how much, for what.
+
+So the pipeline got longer: download the PDF, extract text (OCR when it's a scan, and it's a scan more often than you think), then ask an LLM to pull out the fields. I started with a local model with Ollama, then moved to cheaper hosted models, then I learned that you can do a lot with boring regex before calling any model, and that's much cheaper.
+
+There is also a serious part: these acts are full of personal data. Names, addresses, health situations of people who asked the town for help. Public for 15 days by law doesn't mean "republish forever on a search engine". So nothing goes online before a privacy filter checks it, and whole categories of acts are never shown at all.
+
+## Piazza in Festa: the side effect
+
+Here is a thing I didn't expect: when a town organizes a festival, a *sagra*, a concert in the square, there is almost always an act about it. Money approved, road closed, stage rented.
+
+So the events are already there, hidden in the bureaucracy. 🎉
+
+I started to extract them and that became [Piazza in Festa](https://piazzainfesta.it): around **67,000** events across Italy, found in municipal acts.
+
+Some honest numbers from the last month:
+
+- about **10,000 people** per month
+- **ChatGPT brings around 3 times more visitors than Google**. I measured it twice because the first time I was wrong (the ChatGPT tag stays in the URL and gets counted on every page after the first one, and "bard" is inside "lom**bard**ia"... yes, really)
+- **62%** of people arrive on an event that is **already over**, because the AI assistant read that page months ago
+
+That last one changed the product: instead of a dead page, a past event now says "we'll tell you when it's back next year".
+
+I also asked myself the classic question: should I put ads? I did the math: around 18,000 page views per month would make 20 to 50 euros. Not worth it, not worth the cookie banner, not worth the image.
+<br/>
+The real value is the data, so now I sell the event feed to portals and tourism operators, per region. And I discovered that some websites were already copying our data without asking, which is also a kind of market validation, I guess 😅
+
+## RadarPA: selling is harder than building
+
+Acts talk **before** tenders. A town approves a project to digitalize its services, and the public tender arrives months later. If you are a company, knowing it early is gold.
+
+That's [RadarPA](https://radarpa.it). I built it fast: signals from the acts, a "heat" score, a shop, payments, a nice UI.
+<br/>
+And nobody bought it.
+
+On September 3rd I stopped. Not the product, the way I was using it. I realized I was trying to sell something to strangers without having closed a full loop even once, with one real customer.
+
+So the first customer became... us. Welance. One sector turned on (digitalization), and every day I use RadarPA to find leads for the agency. If it works for us, then I can sell it with a straight face.
+
+And using it every day, yesterday I built what I think is the real killer feature: follow a signal from the first act until the tender is out, with the documents in the same place and an assistant that reads them with you. From "there is an interesting decision in this town" to "this is the tender, this is where you download the documents, this is what's missing". A broad question to the assistant costs me about 14 cents. Less than a coffee, even in Sicily ☕
+
+## Welance, as co-CTO
+
+And then Welance.
+
+[Welance](https://welance.com) is an international collective that has been around for about 15 years, and today I'm its co-CTO.
+
+It's a different job. Less code, more decisions. More "should we do this at all?" and less "how do I do this?".
+
+A good example: last week I measured our non-production Kubernetes cluster. 9 nodes, and a big part of the RAM was not used by our apps but by the platform itself and by reservations. The proposal I wrote brings it to 2 nodes, around half of the cost.
+<br/>
+Same philosophy of [removing code](/blog/learning-to-remove-code), just with servers. 🔥
+
+The other side of the job is bringing the products inside the company story: albopretorio.io, Piazza in Festa and RadarPA are three ways of reading the same public data and I want them to become something bigger than my side projects. That's also why RadarPA's first customer is Welance itself. How exactly all of this fits together is still work in progress, and I'm ok to say it.
+
+## What I learned (so far)
+
+- **Start with one town.** Scicli was the right size to learn. Then the second town teaches you more than the first.
+- **There is no standard, only vendors.** If you want to read the public sector, you must understand who sells software to it.
+- **Measure twice.** Every number I shared too fast was wrong: visitors, empty municipalities, costs.
+- **One real customer beats a nice shop.** Even if the customer is you.
+- **AI agents make you fast, not right.** Being right is still your job.
+- **Boring infra is still good.** Rust binaries, Postgres and a couple of machines on OVH. Nothing fancy, and it holds 5 million acts.
+
+## Closing
+
+Six months ago I wanted to learn a programming language.
+<br/>
+Now I know which vendor runs the notice board of a town of 300 people in the Alps, how many *sagre* there are in Puglia in August and which public domains are selling pills.
+
+Not what I planned, but I'm happy it went this way.
+
+Next post, maybe, less numbers and more Rust. Maybe. 🙂
+
+---
+
+<em class="text-sm">Cover: art from *Book Burning* (Magic: The Gathering, Judgment, 2002) by **Dave Dorman**. Magic: The Gathering is © Wizards of the Coast.</em>