docs: add robots.txt to allow all crawlers

[brad-dow]

Description of changes
Adds robots.txt to docs/public so the file is served at rafiki.dev/robots.txt after build and deploy

Currently, the site returns a 404 for /robots.txt. While crawlers interpret a missing robots.txt as "allow everything", having the file in place is a much better practice and resolves the 404 error.

Checklist

• PR title is prefixed with docs:
• PR title or description includes a fixes #
• You've run pnpm format and pnpm lint
• You've reviewed Vale errors and made changes where appropriate

[netlify]

✅ Deploy Preview for brilliant-pasca-3e80ec ready!

Name	Link
🔨 Latest commit	5868712
🔍 Latest deploy log	https://app.netlify.com/projects/brilliant-pasca-3e80ec/deploys/6a46c6fe04014d00089f6725
😎 Deploy Preview	https://deploy-preview-3949--brilliant-pasca-3e80ec.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[brad-dow]

I was worried that might not work, but you can see the robots.txt in the preview environment as well: https://deploy-preview-3949--brilliant-pasca-3e80ec.netlify.app/robots.txt

[github-actions]

🚀 Performance Test Results

Test Configuration:

• VUs: 4
• Duration: 1m0s

Test Metrics:

• Requests/s: 42.40
• Iterations/s: 14.13
• Failed Requests: 0.00% (0 of 2551)

📜 Logs

> performance@1.0.0 run-tests:testenv /home/runner/work/rafiki/rafiki/test/performance
> ./scripts/run-tests.sh -e test -k -q --vus 4 --duration 1m

Cloud Nine GraphQL API is up: http://localhost:3101/graphql
Cloud Nine Wallet Address is up: http://localhost:3100/
Happy Life Bank Address is up: http://localhost:4100/
cloud-nine-wallet-test-backend already set
cloud-nine-wallet-test-auth already set
happy-life-bank-test-backend already set
happy-life-bank-test-auth already set
     data_received..................: 921 kB 15 kB/s
     data_sent......................: 2.0 MB 33 kB/s
     http_req_blocked...............: avg=7.07µs   min=2.14µs   med=5.31µs   max=1.68ms   p(90)=6.38µs   p(95)=7µs     
     http_req_connecting............: avg=279ns    min=0s       med=0s       max=238.04µs p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=93.58ms  min=7.52ms   med=75.39ms  max=454.87ms p(90)=162.71ms p(95)=184.97ms
       { expected_response:true }...: avg=93.58ms  min=7.52ms   med=75.39ms  max=454.87ms p(90)=162.71ms p(95)=184.97ms
     http_req_failed................: 0.00%  ✓ 0         ✗ 2551
     http_req_receiving.............: avg=101.26µs min=27.16µs  med=85.69µs  max=2.04ms   p(90)=127.21µs p(95)=163.47µs
     http_req_sending...............: avg=40.33µs  min=9.28µs   med=25.87µs  max=2.63ms   p(90)=47.58µs  p(95)=68.57µs 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=93.44ms  min=7.38ms   med=75.19ms  max=454.75ms p(90)=162.6ms  p(95)=184.81ms
     http_reqs......................: 2551   42.40498/s
     iteration_duration.............: avg=282.61ms min=156.27ms med=268.39ms max=960.89ms p(90)=345.71ms p(95)=388.39ms
     iterations.....................: 850    14.129452/s
     vus............................: 4      min=4       max=4 
     vus_max........................: 4      min=4       max=4