Skip to content

[Feature] (Deferred) GitHub App push-webhook accelerator for instant source sync #1179

Description

@chronoai-shining

Background

Related to #1176, #1177, #1178. Deferred / optional — do not implement until latency is a proven complaint. Filed for tracking (issue-first), not scheduled.

#1175#1178 deliver full automatic sync via a pull-based scheduled poller (sourceSyncScheduler), which is the correct baseline because Ornn links to arbitrary public repos the skill author usually does not administer — and both GitHub Apps and per-repo webhooks require repo-admin. The poller's only weakness is latency: a change is picked up on the next cron tick, not instantly.

This issue is the optional push-based accelerator for the minority of repos whose author can install an Ornn GitHub App: receive push webhook events and trigger an immediate re-check, with the #1176 poll retained as the reconciliation backstop.

Verified facts (GitHub docs): a GitHub App gives one centralized webhook endpoint across all installed repos (no per-repo wiring). Delivery is at-least-once with no automatic redelivery; if the endpoint doesn't ack within 10s the delivery is marked failed. So the handler must ack fast, dedupe, and rely on the poll as a safety net.

Purpose

Cut auto-sync latency from "next poll" to near-instant for authors who opt in by installing the Ornn GitHub App on their source repo — without changing the guarantee for everyone else (polling still covers 100%).

Affected Files

  • ornn-api/src/domains/skills/sourceWebhook/routes.tsnew: POST /webhooks/github/source — HMAC-verify, ack 202 within 10s, enqueue.
  • ornn-api/src/domains/skills/sourceWebhook/verify.tsnew: HMAC-SHA256 signature verification against a configured webhook secret (from settings/env, never hardcoded).
  • ornn-api/src/domains/skills/crud/sourceSyncScheduler.ts — modify: add an Agenda queue/job to process a webhook-triggered re-check for the affected (repo, ref) (reuse the drift+auto-publish path from [Feature] Scheduled GitHub source drift-check job (auto-detect upstream changes) #1176/[Feature] Unattended auto-publish of drifted GitHub-sourced skills #1177).
  • ornn-api/src/domains/settings/sections/sourceSync.ts — modify: add githubAppId, githubAppInstallation* (if using an App) and/or webhookSecret (encrypted at rest + masked).
  • ornn-api/src/bootstrap.ts — modify: mount the webhook route.
  • Colocated tests: verify.test.ts, routes.test.ts.
  • .changeset/*.md — new changeset.

Implementation Instructions

  1. Endpoint: POST /webhooks/github/source parses the push event, verifies the X-Hub-Signature-256 HMAC against the configured secret (constant-time compare), and responds 202 immediately after pushing { deliveryId, repo, ref, headSha } onto an Agenda queue. Do all real work async (GitHub's 10s ack rule).
  2. Dedupe / idempotency: key on X-GitHub-Delivery (delivery id) and on (repo, ref, headSha) so at-least-once redelivery does not double-publish. Reuse the [Feature] Unattended auto-publish of drifted GitHub-sourced skills #1177 re-read + sha-compare guard.
  3. Fan-out: map the pushed (repo, ref) to all skills whose source matches and run the [Feature] Scheduled GitHub source drift-check job (auto-detect upstream changes) #1176 checkSourceDrift[Feature] Unattended auto-publish of drifted GitHub-sourced skills #1177 autoPublishFromSource path for each. Ignore pushes to refs no skill tracks.
  4. Backstop: leave the [Feature] Scheduled GitHub source drift-check job (auto-detect upstream changes) #1176 poller running (perhaps at a longer cadence) as reconciliation for dropped deliveries — GitHub never auto-redelivers, and a missed webhook must still be caught within one poll interval.
  5. Security: reject unsigned/mis-signed requests with 401; the secret is loaded from settings/env only; never log the raw payload or secret. Rate-limit / size-cap the endpoint (GitHub payloads up to 25 MB).
  6. Constraints / non-goals: this is an accelerator, not a replacement — do not remove or weaken the poller. Only build after [Feature] Scheduled GitHub source drift-check job (auto-detect upstream changes) #1176[Feature] Surface auto-sync status: drift badges + notifications #1178 are merged and latency is an actual complaint.

Definition of Done

  • Every step in Implementation Instructions is implemented (no partial scope)
  • Behavior exercised locally: send a signed sample push payload → 202 within 10s → the affected skill re-checks and (if drifted + version bumped) auto-publishes; an unsigned payload → 401
  • Redelivery/dedupe verified: same delivery id twice → single publish
  • Poller backstop still runs and catches a simulated dropped delivery
  • Secret + payload never logged in plaintext
  • Tests run locally and green: cd ornn-api && bun test src/domains/skills/sourceWebhook
  • bun run typecheck and bun run lint clean
  • Docs updated (webhook setup + the App install steps)
  • Changeset added
  • gitleaks clean; conventional, atomic, buildable-per-commit
  • No Co-Authored-By lines in any commit
  • PR targets develop-auto; CI 100% green

Priority

P3 — deferred; build only if latency becomes a real complaint.

Size

L

Metadata

Metadata

Labels

apiAPI design & endpointsinfraInfrastructure & deploymentpriority:P3Low / nice-to-have. Picked up only when the queue is otherwise empty.sa:gh-auto-syncsa-design context: automatic sync of GitHub-sourced skillssize:LLarge: cross-cutting, multiple modules. Size is informational.status:triageNewly filed. Awaiting prioritization/sizing; not yet auto-pickable.type:featureNew user-facing capability.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions