diff --git a/src/content/blog/festivals-hidden-in-the-bureaucracy.mdx b/src/content/blog/festivals-hidden-in-the-bureaucracy.mdx new file mode 100644 index 0000000..120f38b --- /dev/null +++ b/src/content/blog/festivals-hidden-in-the-bureaucracy.mdx @@ -0,0 +1,59 @@ +--- +title: "Festivals hidden in the bureaucracy" +description: "How 67,000 Italian events came out of municipal acts, why ChatGPT sends me 3 times more people than Google and why I said no to ads." +pubDate: 2026-10-06 +tags: ["side-projects", "open-data", "ai", "seo", "piazza-in-festa"] +--- + +This is the third post of the series that started with [six months reading every Italian municipality](/blog/six-months-reading-italian-municipalities). + +And this is the part I didn't plan at all. + +## The side effect + +When a town organizes a festival, a *sagra*, a concert in the square, there is almost always an act about it. Money approved, road closed, stage rented, electricity for the lights. + +So the events are already there, hidden in the bureaucracy. 🎉 + +I started to extract them and that became [Piazza in Festa](https://piazzainfesta.it): around **67,000** events across Italy, found in municipal acts. Many of them are small things you never find on the big event websites: the patron saint of a village of 800 people, the *sagra* of a vegetable you didn't know existed. + +## Some honest numbers + +From the last month: + +- about **10,000 people** per month +- **ChatGPT brings around 3 times more visitors than Google** +- **62%** of people arrive on an event that is **already over** + +The ChatGPT number I measured twice, because the first time I was wrong. Twice wrong, actually: + +1. ChatGPT adds a tag to the links it shows. That tag stays in the URL, so it was counted again on every page after the first one. The first result was **5 times** bigger than the real one. +2. I was looking for Gemini with the word "bard". And "bard" is inside "lom**bard**ia". Every visit to a Lombardy page was counted as Gemini. Real Gemini visitors: zero. Yes, really. + +Measure twice. Especially when the number makes you happy. + +## Landing on the past + +The 62% changed the product. + +It makes sense: the AI assistant read that page months ago and it links it today. For me it was a dead page, for the person arriving it was the only page. + +So now a past event doesn't say "it's over, bye". It says "we'll tell you when it's back next year". A festival next year is usually the same festival, same town, same name, only the date changes. So the same page can wait for it. + +## Should I put ads? + +The classic question. I did the math: around 18,000 page views per month would make **20 to 50 euros**. +
+Not worth it, not worth the cookie banner, not worth the image with the towns I talk to. + +The real value is the data. So now I sell the event feed to portals and tourism operators, per region. + +And I discovered that some websites were already copying our data without asking. Which is also a kind of market validation, I guess 😅 + +## What I learned + +- **AI assistants are already a real channel.** For a small site like this one, bigger than Google. +- **People land where the assistant remembers,** not where you want. Make old pages useful. +- **Small traffic doesn't pay with ads.** Look for who needs the data, not the eyeballs. + +Next time: a product that nobody bought, and what I did about it. diff --git a/src/content/blog/italian-notice-boards-are-a-zoo.mdx b/src/content/blog/italian-notice-boards-are-a-zoo.mdx new file mode 100644 index 0000000..945e441 --- /dev/null +++ b/src/content/blog/italian-notice-boards-are-a-zoo.mdx @@ -0,0 +1,77 @@ +--- +title: "Italian public notice boards are a zoo" +description: "About 100 vendor platforms, PDFs that are not PDFs and a municipal domain that now sells Viagra. What I found crawling every albo pretorio in Italy." +pubDate: 2026-10-02 +tags: ["scraping", "rust", "open-data", "public-administration", "privacy"] +--- + +In the [first post of this series](/blog/six-months-reading-italian-municipalities) I told how a small Rust exercise on one town became a crawler for every Italian municipality. + +This one is about the part nobody sees: **how** you read 9,617 public notice boards when there is no standard, no API and no feed. + +Short answer: one vendor at a time. 😅 + +## The scraper that became a zoo + +The first scraper was for one vendor platform. Easy. Then I added a second town and it was on a different platform. Then a third one. + +At some point I stopped counting towns and started counting **vendors**. Today there are about **100** of them in the codebase, and each one has a personality: + +- old Java server pages with a hidden "ViewState" that you must send back exactly as you received it, or nothing works +- sites that only work inside a real browser, so the crawler drives a headless Chromium +- PDFs that say they are PDFs but they are signed `.p7m` files +- pages that lie about their encoding +- "search" pages that return everything if you pass `limit=0` (thank you, I'll take it 🙏) + +The good news is that vendors are few compared to towns. When you understand one platform, you often get hundreds of municipalities for free, because they all bought the same software. So the real map is not "7,896 towns", it's "who sells software to the public sector". + +## Who owns the notice board? + +One number surprised me: **67.6%** of Italian municipalities publish their notice board on a domain they don't own. It's the vendor's domain. +
+The official record of what a town decided lives on a `.com` of a private company. + +It works, most of the time. But if the vendor changes, closes or forgets to renew something, the history of that town moves or disappears with it. + +## The Viagra domain + +And then there is my favorite discovery. + +Some public bodies were closed years ago (mountain communities, old consortiums) and nobody renewed their domain. Somebody else bought it. +
+One of them, still listed as "official website" in the national registry of public administrations, today is an **online pharmacy selling Viagra without prescription**. In Italian. 🙃 + +For a crawler this is a real problem, not only a funny story: the domain answers `200 OK`, the page loads, everything looks alive. You must look at *what* the page says, not only *if* it answers. + +Also funny: I tried to add the 20 Regions too, before understanding that most Regions **don't have an albo pretorio** at all. Lesson learned: read the law before writing the crawler. + +## PDFs, LLMs and regex + +The notice board gives you a title and a PDF. The interesting part is inside the PDF: who gets the money, how much, for what. + +So the pipeline got longer: + +1. download the PDF (4.6 million so far) +2. extract the text, with OCR when it's a scan, and it's a scan more often than you think +3. pull out the fields: amounts, suppliers, dates, offices + +For step 3 I started with a local model with Ollama, then moved to cheaper hosted models. Then I learned the boring lesson: you can do **a lot** with regex before calling any model. A spending approval in Italian has a very recognizable shape. The LLM is for what the regex can't read, not for everything. + +Cheaper, faster and, surprise, easier to test. + +## The serious part: privacy + +These acts are full of personal data. Names, addresses, health situations of people who asked the town for help. + +The law says they must stay public for 15 days. That doesn't mean "republish forever on a search engine". +
+So nothing goes online before a privacy filter checks it, and whole categories of acts are never shown at all. Crawling everything is fine. Showing everything is not. + +## What I learned + +- **There is no standard, only vendors.** If you want to read the public sector, you must understand who sells software to it. +- **A 200 is not a proof of life.** Check what the page says. +- **Regex first, LLM after.** It's not sexy, but it pays the bills. +- **Collect everything, publish carefully.** Two different decisions. + +Next time: the festivals hidden inside all these acts. 🎉 diff --git a/src/content/blog/removing-servers-as-co-cto.mdx b/src/content/blog/removing-servers-as-co-cto.mdx new file mode 100644 index 0000000..1ad0685 --- /dev/null +++ b/src/content/blog/removing-servers-as-co-cto.mdx @@ -0,0 +1,52 @@ +--- +title: "Removing servers, now as co-CTO" +description: "My new role at Welance: less code, more decisions, and a Kubernetes cluster that can go from 9 nodes to 2." +pubDate: 2026-10-13 +tags: ["welance", "kubernetes", "infrastructure", "thoughts", "engineering"] +--- + +Last post of the series that started with [six months reading every Italian municipality](/blog/six-months-reading-italian-municipalities). The code part is done, this one is about the job. + +## Welance + +[Welance](https://welance.com) is an international collective that has been around for about 15 years, and today I'm its co-CTO. + +It's a different job. Less code, more decisions. +
+More "should we do this at all?" and less "how do I do this?". + +## 9 nodes, 2 needed + +A good example. + +We have a Kubernetes cluster for everything that is not production: development and staging environments of all the projects. 9 nodes. It worked, nobody complained, so nobody looked at it. + +I looked at it. Some things I found: + +- almost **a quarter of the RAM** was reserved by the system on each node, before any app could use it. More nodes, more reservation. Pay 9 times for the same overhead. +- the platform itself (monitoring, logging, operators) was using more memory than the real apps +- the apps **asked** for about 3 times the CPU they really used. Kubernetes believes what the apps ask, so the nodes looked full while they were mostly sleeping + +The proposal I wrote brings it to **2 bigger nodes**, around half of the cost. +
+Same philosophy of [removing code](/blog/learning-to-remove-code), just with servers. 🔥 + +The funny part: before removing nodes you must fix the fake CPU requests. If you don't, the pods on the new small cluster just stay "Pending", because on paper they don't fit. First you tell the truth, then you cut. + +## The other side of the job + +The other side is bringing the products inside the company story. + +The albo pretorio crawler, [Piazza in Festa](https://piazzainfesta.it) and [RadarPA](https://radarpa.it) are three ways of reading the same public data, and I want them to become something bigger than my side projects. That's also why RadarPA's first customer is Welance itself. +
+How exactly all of this fits together is still work in progress, and I'm ok to say it. + +## What I learned (so far) + +- **Nobody looks at what works.** That's exactly where the waste hides. +- **Measure before cutting.** The numbers told a different story than the dashboards. +- **Being CTO is mostly choosing what not to do.** And then explaining why. + +That's the end of the series. Six months, one town that became all of them, three products and a new job title. + +Now I need to keep writing, without a series to hide behind. 🙂 diff --git a/src/content/blog/selling-is-harder-than-building.mdx b/src/content/blog/selling-is-harder-than-building.mdx new file mode 100644 index 0000000..fbfdf30 --- /dev/null +++ b/src/content/blog/selling-is-harder-than-building.mdx @@ -0,0 +1,56 @@ +--- +title: "Selling is harder than building" +description: "I built RadarPA fast and nobody bought it. So the first customer became my own company." +pubDate: 2026-10-09 +tags: ["side-projects", "product", "public-administration", "ai", "radarpa"] +--- + +Fourth post of the series about [reading every Italian municipality](/blog/six-months-reading-italian-municipalities). This one is less about code and more about a mistake. + +## Acts talk before tenders + +A town approves a project to digitalize its services. A council decides to renovate a school. Months later, the public tender arrives. + +If you are a company that works with the public sector, knowing it **early** is gold: you can prepare, find partners, understand what the town really needs before everybody else reads the tender. + +All of this is written in the acts I was already reading. So I built [RadarPA](https://radarpa.it). + +## Building was the easy part + +I built it fast: signals from the acts, a "heat" score for how close a tender is, a shop, payments, a nice UI. +
+And nobody bought it. + +Not "few people". Nobody. The site had no traffic, the UI didn't make clear what you do with it, and keeping the LLM running on every sector was costing money while the data got old. + +## What I changed + +On September 3rd I stopped. Not the product, the way I was using it. + +I realized I was trying to sell something to strangers without having closed a full loop even once, with one real customer. + +So the first customer became... us. Welance. One sector turned on (digitalization, what we actually do), and every day I use RadarPA to find leads for the agency. +
+If it works for us, then I can sell it with a straight face. + +A nice side effect: with one sector instead of all of them, the queue of acts to analyze went from about 800,000 to 184,000. Less cost, fresher data. + +## The feature that came from using it + +Using it every day, I found what I think is the real killer feature. + +A signal is not useful by itself. What you want is to **follow** it: from the first act until the tender is out, with the documents in the same place and an assistant that reads them with you. +
+From "there is an interesting decision in this town" to "this is the tender, this is where you download the documents, this is what's missing". + +I would never have designed this from a whiteboard. It came from being annoyed as a user. + +A broad question to the assistant costs me about 14 cents. Less than a coffee, even in Sicily ☕ + +## What I learned + +- **One real customer beats a nice shop.** Even if the customer is you. +- **Use your own product every day.** The annoying parts are the roadmap. +- **Turn things off.** One sector done well is better than twenty sectors that nobody reads. + +Next and last one of the series: Welance, and my new role there. diff --git a/src/content/blog/six-months-reading-italian-municipalities.mdx b/src/content/blog/six-months-reading-italian-municipalities.mdx index 21dcc4d..715cfcd 100644 --- a/src/content/blog/six-months-reading-italian-municipalities.mdx +++ b/src/content/blog/six-months-reading-italian-municipalities.mdx @@ -1,9 +1,9 @@ --- title: "Six months reading every Italian municipality" -description: "It started as a way to learn Rust with one town near my house. Now it's albopretorio.io, Piazza in Festa, RadarPA and a new role at Welance." +description: "It started as a way to learn Rust with one town near my house. Six months later it reads 5 million acts from every Italian municipality." pubDate: 2026-09-29 heroImage: "../../assets/photos/six-months-reading-italian-municipalities/cover.jpg" -tags: ["rust", "side-projects", "open-data", "public-administration", "ai", "welance"] +tags: ["rust", "side-projects", "open-data", "public-administration", "ai"] --- On April 1st I opened a new Rust workspace. @@ -20,7 +20,7 @@ Six months later the numbers are these: - **2,161** commits - three products and one new job title -So... something went a bit out of control. This post is the recap, mostly for future-me, because I already forget the details of things I did in May. +So... something went a bit out of control. Too many things for one post, so this is the first of a small series, mostly for future-me, because I already forget the details of things I did in May. ## What an "albo pretorio" is @@ -34,24 +34,6 @@ In practice every municipality buys its website from a vendor and every vendor b So I built albopretorio.io: a crawler that does it for you and one place where you can search everything. -## The scraper that became a zoo - -The first scraper was for one vendor platform. Easy. Then I added a second town and it was on a different platform. Then a third one. - -At some point I stopped counting towns and started counting **vendors**. Today there are about **100** of them in the codebase, and each one has a personality: - -- old Java server pages with a hidden "ViewState" that you must send back exactly as you received it, or nothing works -- sites that only work inside a real browser, so the crawler drives a headless Chromium -- PDFs that say they are PDFs but they are signed `.p7m` files -- pages that lie about their encoding -- "search" pages that return everything if you pass `limit=0` (thank you, I'll take it 🙏) - -One number surprised me: **67.6%** of Italian municipalities publish their notice board on a domain they don't own. It's the vendor's domain. The official record of what a town decided lives on a `.com` of a private company. - -And then there is my favorite discovery. Some public bodies were closed years ago (mountain communities, old consortiums) and nobody renewed their domain. Somebody else bought it. One of them, still listed as "official website" in the national registry of public administrations, today is an **online pharmacy selling Viagra without prescription**. In Italian. 🙃 - -Also funny: I tried to add the 20 Regions too, before understanding that most Regions **don't have an albo pretorio** at all. Lesson learned: read the law before writing the crawler. - ## About Rust (and about honesty) The original idea was a strict "learning mode": I asked Claude to be my Rust tutor, explain the compiler errors and **not** write code for me. @@ -66,68 +48,10 @@ I'm fine with that. The goal changed on the way. What I would say to anyone doing the same: **the agent is great at adding, you must be the one who checks.** Every time I trusted a number without looking at the data myself, the number was wrong. Every time. -## PDFs, LLMs and privacy - -The notice board gives you a title and a PDF. The interesting part is inside the PDF: who gets the money, how much, for what. - -So the pipeline got longer: download the PDF, extract text (OCR when it's a scan, and it's a scan more often than you think), then ask an LLM to pull out the fields. I started with a local model with Ollama, then moved to cheaper hosted models, then I learned that you can do a lot with boring regex before calling any model, and that's much cheaper. - -There is also a serious part: these acts are full of personal data. Names, addresses, health situations of people who asked the town for help. Public for 15 days by law doesn't mean "republish forever on a search engine". So nothing goes online before a privacy filter checks it, and whole categories of acts are never shown at all. - -## Piazza in Festa: the side effect - -Here is a thing I didn't expect: when a town organizes a festival, a *sagra*, a concert in the square, there is almost always an act about it. Money approved, road closed, stage rented. - -So the events are already there, hidden in the bureaucracy. 🎉 - -I started to extract them and that became [Piazza in Festa](https://piazzainfesta.it): around **67,000** events across Italy, found in municipal acts. - -Some honest numbers from the last month: - -- about **10,000 people** per month -- **ChatGPT brings around 3 times more visitors than Google**. I measured it twice because the first time I was wrong (the ChatGPT tag stays in the URL and gets counted on every page after the first one, and "bard" is inside "lom**bard**ia"... yes, really) -- **62%** of people arrive on an event that is **already over**, because the AI assistant read that page months ago - -That last one changed the product: instead of a dead page, a past event now says "we'll tell you when it's back next year". - -I also asked myself the classic question: should I put ads? I did the math: around 18,000 page views per month would make 20 to 50 euros. Not worth it, not worth the cookie banner, not worth the image. -
-The real value is the data, so now I sell the event feed to portals and tourism operators, per region. And I discovered that some websites were already copying our data without asking, which is also a kind of market validation, I guess 😅 - -## RadarPA: selling is harder than building - -Acts talk **before** tenders. A town approves a project to digitalize its services, and the public tender arrives months later. If you are a company, knowing it early is gold. - -That's [RadarPA](https://radarpa.it). I built it fast: signals from the acts, a "heat" score, a shop, payments, a nice UI. -
-And nobody bought it. - -On September 3rd I stopped. Not the product, the way I was using it. I realized I was trying to sell something to strangers without having closed a full loop even once, with one real customer. - -So the first customer became... us. Welance. One sector turned on (digitalization), and every day I use RadarPA to find leads for the agency. If it works for us, then I can sell it with a straight face. - -And using it every day, yesterday I built what I think is the real killer feature: follow a signal from the first act until the tender is out, with the documents in the same place and an assistant that reads them with you. From "there is an interesting decision in this town" to "this is the tender, this is where you download the documents, this is what's missing". A broad question to the assistant costs me about 14 cents. Less than a coffee, even in Sicily ☕ - -## Welance, as co-CTO - -And then Welance. - -[Welance](https://welance.com) is an international collective that has been around for about 15 years, and today I'm its co-CTO. - -It's a different job. Less code, more decisions. More "should we do this at all?" and less "how do I do this?". - -A good example: last week I measured our non-production Kubernetes cluster. 9 nodes, and a big part of the RAM was not used by our apps but by the platform itself and by reservations. The proposal I wrote brings it to 2 nodes, around half of the cost. -
-Same philosophy of [removing code](/blog/learning-to-remove-code), just with servers. 🔥 - -The other side of the job is bringing the products inside the company story: albopretorio.io, Piazza in Festa and RadarPA are three ways of reading the same public data and I want them to become something bigger than my side projects. That's also why RadarPA's first customer is Welance itself. How exactly all of this fits together is still work in progress, and I'm ok to say it. - ## What I learned (so far) - **Start with one town.** Scicli was the right size to learn. Then the second town teaches you more than the first. -- **There is no standard, only vendors.** If you want to read the public sector, you must understand who sells software to it. - **Measure twice.** Every number I shared too fast was wrong: visitors, empty municipalities, costs. -- **One real customer beats a nice shop.** Even if the customer is you. - **AI agents make you fast, not right.** Being right is still your job. - **Boring infra is still good.** Rust binaries, Postgres and a couple of machines on OVH. Nothing fancy, and it holds 5 million acts. @@ -139,7 +63,7 @@ Now I know which vendor runs the notice board of a town of 300 people in the Alp Not what I planned, but I'm happy it went this way. -Next post, maybe, less numbers and more Rust. Maybe. 🙂 +In the next posts: the zoo of vendor platforms behind the notice boards, the festivals hidden in the bureaucracy, a product that nobody bought (yet) and my new role at Welance. One at a time. 🙂 ---