loke/lorenzoiovino.com
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

post: soften the data collection wording in the series
All checks were successful
Deploy to OVH VPS / deploy (push) Successful in 1m15s
Details

Browse source
This commit is contained in:
Lorenzo Iovino 2026-09-29 00:44:51 +02:00
parent d69d38e708
commit 2251a3e938
2 changed files with 4 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

6
src/content/blog/italian-notice-boards-are-a-zoo.mdx
View file

@ -52,7 +52,7 @@ The notice board gives you a title and a PDF. The interesting part is inside the
So the pipeline got longer: So the pipeline got longer:
1. download the PDF (4.6 million so far) 1. download the PDF
2. extract the text, with OCR when it's a scan (and it's a scan more often than you think) 2. extract the text, with OCR when it's a scan (and it's a scan more often than you think)
3. pull out the fields: amounts, suppliers, dates, offices 3. pull out the fields: amounts, suppliers, dates, offices
@ -66,14 +66,14 @@ These acts are full of personal data. Names, addresses, health situations of peo
The law says they must stay public for 15 days. That doesn't mean "republish forever on a search engine". The law says they must stay public for 15 days. That doesn't mean "republish forever on a search engine".
<br/> <br/>
So nothing goes online before a privacy filter checks it. Whole categories of acts are never shown at all. Crawling everything is fine. Showing everything is not. So nothing goes online before a privacy filter checks it. Whole categories of acts are never shown at all. Personal data is the part to handle with the most care, not the least.
## What I learned ## What I learned
- **There is no standard, only vendors.** If you want to read the public sector, you must understand who sells software to it. - **There is no standard, only vendors.** If you want to read the public sector, you must understand who sells software to it.
- **A 200 is not a proof of life.** Check what the page says. - **A 200 is not a proof of life.** Check what the page says.
- **Regex first, LLM after.** It's not sexy, but it pays the bills. - **Regex first, LLM after.** It's not sexy, but it pays the bills.
- **Collect everything, publish carefully.** Two different decisions. - **Privacy is a design choice.** It goes in the pipeline from day one, not in a disclaimer at the end.
Next time: the festivals hidden inside all these acts. 🎉 Next time: the festivals hidden inside all these acts. 🎉

2
src/content/blog/six-months-reading-italian-municipalities.mdx
View file

@ -15,7 +15,7 @@ The plan was very small: learn Rust by scraping the public notice board of **one
Six months later the numbers are these: Six months later the numbers are these:
- **7,896** Italian municipalities, all 20 regions, plus unions of towns, health agencies, parks, provinces... **9,617** public bodies in total - **7,896** Italian municipalities, all 20 regions, plus unions of towns, health agencies, parks, provinces... **9,617** public bodies in total
- more than **5.1 million** administrative acts, 4.6 million of them with the PDF downloaded - more than **5.1 million** administrative acts
- **17** Rust crates in the workspace - **17** Rust crates in the workspace
- **2,161** commits - **2,161** commits
- three products and one new job title - three products and one new job title