Lorenzo Iovino
7cf2e858a2
BREAKING CHANGE: Complete rewrite from Angular SPA to Astro - Replace Angular 18 with Astro 5.16.7 + Tailwind CSS - Convert all Angular components to Astro components - Add content collections for blog with markdown support - Setup S3 deployment with CloudFront invalidation - Add RSS feed and sitemap generation - Configure Prettier and Biome for code formatting - Switch from npm to pnpm - Remove Amplify backend (now fully static) - Improve SEO and performance with static generation
32 lines
516 B
Text
32 lines
516 B
Text
# robots.txt for lorenzoiovino.com
|
|
|
|
User-agent: *
|
|
Allow: /
|
|
|
|
# Sitemaps
|
|
Sitemap: https://lorenzoiovino.com/sitemap-index.xml
|
|
Sitemap: https://lorenzoiovino.com/rss.xml
|
|
|
|
# Crawl-delay (optional, adjust if needed)
|
|
# Crawl-delay: 1
|
|
|
|
# Disallow paths (if any)
|
|
# Disallow: /admin/
|
|
# Disallow: /private/
|
|
|
|
# Allow specific bots
|
|
User-agent: Googlebot
|
|
Allow: /
|
|
|
|
User-agent: Bingbot
|
|
Allow: /
|
|
|
|
User-agent: Slurp
|
|
Allow: /
|
|
|
|
# Block bad bots (optional)
|
|
User-agent: AhrefsBot
|
|
Crawl-delay: 10
|
|
|
|
User-agent: SemrushBot
|
|
Crawl-delay: 10
|