diff --git a/src/content/blog/italian-notice-boards-are-a-zoo.mdx b/src/content/blog/italian-notice-boards-are-a-zoo.mdx index 5eab5fa..f004a16 100644 --- a/src/content/blog/italian-notice-boards-are-a-zoo.mdx +++ b/src/content/blog/italian-notice-boards-are-a-zoo.mdx @@ -52,7 +52,7 @@ The notice board gives you a title and a PDF. The interesting part is inside the So the pipeline got longer: -1. download the PDF (4.6 million so far) +1. download the PDF 2. extract the text, with OCR when it's a scan (and it's a scan more often than you think) 3. pull out the fields: amounts, suppliers, dates, offices @@ -66,14 +66,14 @@ These acts are full of personal data. Names, addresses, health situations of peo The law says they must stay public for 15 days. That doesn't mean "republish forever on a search engine".
-So nothing goes online before a privacy filter checks it. Whole categories of acts are never shown at all. Crawling everything is fine. Showing everything is not. +So nothing goes online before a privacy filter checks it. Whole categories of acts are never shown at all. Personal data is the part to handle with the most care, not the least. ## What I learned - **There is no standard, only vendors.** If you want to read the public sector, you must understand who sells software to it. - **A 200 is not a proof of life.** Check what the page says. - **Regex first, LLM after.** It's not sexy, but it pays the bills. -- **Collect everything, publish carefully.** Two different decisions. +- **Privacy is a design choice.** It goes in the pipeline from day one, not in a disclaimer at the end. Next time: the festivals hidden inside all these acts. 🎉 diff --git a/src/content/blog/six-months-reading-italian-municipalities.mdx b/src/content/blog/six-months-reading-italian-municipalities.mdx index 7a7c20b..3fea708 100644 --- a/src/content/blog/six-months-reading-italian-municipalities.mdx +++ b/src/content/blog/six-months-reading-italian-municipalities.mdx @@ -15,7 +15,7 @@ The plan was very small: learn Rust by scraping the public notice board of **one Six months later the numbers are these: - **7,896** Italian municipalities, all 20 regions, plus unions of towns, health agencies, parks, provinces... **9,617** public bodies in total -- more than **5.1 million** administrative acts, 4.6 million of them with the PDF downloaded +- more than **5.1 million** administrative acts - **17** Rust crates in the workspace - **2,161** commits - three products and one new job title