2 commits

Author	SHA1	Message	Date
Danny Avila	990763cbee	🧠 feat: Enable GitNexus Embeddings for Dev Branch and PR Indexes (#12620) ... * feat: auto-enable embeddings for dev and PR indexes too Previously only main branch pushes got --embeddings; dev and contributor PRs ran graph-only and relied on BM25 search. Semantic search on those indexes silently returned empty, which defeats the whole point of serving them to MCP clients. New logic: every automatic trigger (push to main/dev, pull_request from contributors) enables --embeddings. Only workflow_dispatch still respects the explicit input toggle, so operators can run a fast graph-only re-index when they don't need fresh vectors. Cost: adds ~3-5 minutes per index run. Acceptable tradeoff for having semantic search work across all served branches + open PRs instead of just main. * refine: gate PR embeddings on unit-test path relevance Previous version auto-enabled --embeddings on every contributor PR, which cost ~3-5 min of extra CI per index even on PRs that couldn't benefit from semantic code search (docs, config, workflow files, i18n strings, etc.). New logic mirrors the backend-review.yml and frontend-review.yml path filters — if a PR doesn't touch api/, client/, or packages/ it won't trigger unit tests and it doesn't need embeddings. The check queries the GitHub API for the PR's changed file list via `gh api repos/.../pulls/<N>/files` (paginated for very large PRs) and enables embeddings only when at least one path matches. main/dev pushes still always embed. workflow_dispatch still respects the explicit input toggle, which also covers the /gitnexus index [embeddings] PR command. The contributor gate at the job level is unchanged — non-contributor PRs are still skipped entirely regardless of paths. * feat: /gitnexus command works for non-contributor and fork PRs The command workflow already gated on the commenter's author association (not the PR author's), so a contributor commenting /gitnexus index on an outside contributor's PR passes the auth check. But the downstream index workflow checked out the PR's raw head SHA, which only exists in the fork for cross-repo PRs — actions/checkout fetches from the base repo's origin and fails. Switch the command workflow to dispatch with refs/pull/<N>/head instead of the SHA. GitHub mirrors every PR's head into the base repo as this ref regardless of whether the PR is from a fork, so the checkout always resolves. End result: a contributor can type `/gitnexus index embeddings` on any PR — including one opened by a first-time contributor from a fork — and the index (with embeddings, if requested) is built and served. The contributor takes responsibility for the trust boundary by typing the command. Updated the relevant header/inline comments in both workflows so the next maintainer understands the refs/pull/<N>/head choice and the commenter-based gating. * refine: /gitnexus index defaults to embeddings on A contributor typing the command has already chosen to spend ~5 minutes of CI on a full re-index; they wouldn't invoke the command just to get a BM25-only result. Flip the default so the short form `/gitnexus index` produces an embeddings-enabled index. Modifier semantics: /gitnexus index -> embeddings ON (new default) /gitnexus index embeddings -> embeddings ON (explicit, no-op alias) /gitnexus index fast -> embeddings OFF (opt-out) /gitnexus index graph-only -> embeddings OFF (alias) /gitnexus index no-embeddings-> embeddings OFF (alias) The previous `embeddings` modifier is preserved as a no-op alias so anyone who learned the earlier form still gets what they expected.	2026-04-11 13:35:29 -04:00
Danny Avila	8eab39bc8f	🌊 feat: Add GitNexus DigitalOcean Pipeline with PR Index Serving (#12612) ... * feat: migrate GitNexus deployment from Fly.io to DigitalOcean droplet Fly.io's 1GB machine was pegged at ~900MB memory with load spiking to 2.7 under even modest query load. Moving to a 2GB+ DO droplet that can take advantage of existing credits. Architecture change: indexes no longer baked into the image. Instead, a long-lived image (built only when .do/gitnexus/ changes) is pulled from GHCR, and the deploy workflow rsyncs .gitnexus/ data into /opt/gitnexus/indexes/<name>/ on the droplet and restarts only the gitnexus container. Caddy stays running so TLS certs don't churn. - Add .do/gitnexus/Dockerfile (same native-addon + extension patch layers as the Fly variant, but no COPY indexes/ step) - Add .do/gitnexus/docker-compose.yml with gitnexus + caddy services on an internal bridge network, 1.8GB memory limit, healthcheck - Add .do/gitnexus/Caddyfile with automatic HTTPS for the configured subdomain and bearer token auth for all routes except /health - Add .do/gitnexus/entrypoint.sh that registers every index mounted at /indexes/<name>/.gitnexus at container start, then runs gitnexus serve bound to 0.0.0.0 (internal docker network only) - Add .do/gitnexus/install-extensions.js for LadybugDB FTS/vector extension pre-install (workaround for upstream bug) - Add .github/workflows/gitnexus-deploy-do.yml that builds the image only on Dockerfile/entrypoint changes, pushes to GHCR, rsyncs the index artifacts to the droplet, and restarts the gitnexus container - Remove .fly/gitnexus/ and .github/workflows/gitnexus-deploy.yml — Fly app will be destroyed after DO deploy is verified working Required new secrets: DO_HOST, DO_USER, DO_SSH_KEY. GITNEXUS_DOMAIN and API_TOKEN live in /opt/gitnexus/.env on the droplet itself. * refactor: prefix deploy secrets with GITNEXUS_ for namespace isolation Rename DO_HOST -> GITNEXUS_DO_HOST, DO_USER -> GITNEXUS_DO_USER, and DO_SSH_KEY -> GITNEXUS_DO_SSH_KEY so the secrets are clearly scoped to the gitnexus deploy and don't collide with any other DigitalOcean secrets LibreChat might add later. * feat: serve PR indexes alongside main/dev and add /gitnexus command The index workflow was already building and uploading per-PR indexes (gitnexus-index-pr-<N>) for contributor PRs, but the deploy workflow only consumed main and dev artifacts. PR indexes were sitting in storage doing nothing. This wires them all the way through to the live MCP server, with proper cleanup when PRs close. Deploy workflow changes: - Drop the branches filter on workflow_run so PR index completions also trigger deploys (PR indexes are already contributor-gated upstream in gitnexus-index.yml via author_association) - Resolve all open PRs via the GitHub API, look up each one's latest non-expired gitnexus-index-pr-<N> artifact, and serve whichever ones exist. PRs without an index artifact are skipped — we don't retroactively index anything. - Per-ref concurrency group so rapid pushes to the same PR coalesce but different refs still deploy in parallel - After rsyncing active indexes, prune any /opt/gitnexus/indexes/ folder that isn't in the active set. Safety net for missed PR close events. New workflow: gitnexus-cleanup-pr.yml - Fires on pull_request closed (merged or not) - SSHs to the droplet, removes /opt/gitnexus/indexes/LibreChat-pr-<N>, restarts the gitnexus container New workflow: gitnexus-pr-command.yml - Listens for issue_comment events where body starts with /gitnexus - Contributor gated via author_association - Supports: /gitnexus index — index with defaults /gitnexus index embeddings — index with --embeddings - Dispatches gitnexus-index.yml with the PR number and head SHA, reacts to the comment with a rocket emoji Index workflow changes: - New dispatch inputs pr_number and pr_ref for command-driven runs - Checkout step uses inputs.pr_ref when set so the PR's head commit is analyzed instead of the default branch - Artifact naming falls back through pr_number -> pull_request number -> ref_name, keeping existing behavior for push/PR events - Concurrency group switches to pr-<N> when dispatched by the command so re-runs on the same PR debounce correctly * chore: remove Fly variant reference from Dockerfile header The Fly variant no longer exists in the repo, so the comparison comment is meaningless. Rewritten as a standalone description of the image's design. * review: resolve 15 findings from review audit Critical - Drop the unused caddy binary from the gitnexus image. Caddy runs in its own container in this architecture; installing it inside the gitnexus image added ~40-60MB for no reason and contradicted the Dockerfile header comment. Major - Replace ssh-keyscan TOFU with a GITNEXUS_DO_KNOWN_HOST secret. Both deploy and cleanup workflows now pin the droplet's host key from a stored value instead of silently trusting whatever the host presents at deploy time. Fails the workflow if the secret is empty so no one accidentally regresses to TOFU. - Gate gitnexus-cleanup-pr.yml to same-repo PRs via github.event.pull_request.head.repo.full_name == github.repository. Fork PR closes no longer produce failed runs when secrets are withheld by GitHub. The deploy workflow's stale-folder prune step remains the safety net for any fork-contributor indexes. - Fail fast in entrypoint.sh when main/dev index registration errors. Previously `|| echo WARN` swallowed failures so a broken index passed the docker healthcheck and the deploy was marked green while queries returned empty. PR indexes stay best-effort (a corrupt PR index shouldn't take the whole server down). - Authenticate the droplet with GHCR on every deploy using GITHUB_TOKEN, so private GHCR packages work without documentation detours or manual docker login on the host. Bootstrap comments explain the flow. - Switch docker-compose caddy.depends_on from the short-form (service_started) to service_healthy so Caddy doesn't route to a gitnexus container that's still starting (500ms-60s window after recreation). Minor - Guard the HEAD~1 diff with `git rev-parse --verify HEAD~1` so the first-commit and workflow_run-from-PR cases default to rebuild instead of silently skipping a legitimately-changed image. - Move `packages: write` off the workflow-level permissions and onto the build-image job. deploy no longer inherits unnecessary GHCR write access. - Skip the SSH session in cleanup-pr.yml when no gitnexus-index-pr-<N> artifact ever existed for the PR. Eliminates ~95% of no-op SSH round-trips on a busy repo (docs-only PRs, paths-ignored PRs, etc). - Reload Caddy in-place after config upload with `caddy reload`, falling back to force-recreate on reload failure and `compose up` on first-time bootstrap. Picks up Caddyfile or env changes without losing TLS certs. - Replace `sleep 5` post-deploy with a real readiness poll against docker's health status. Fails the workflow if gitnexus doesn't report healthy within 120s, so a broken startup surfaces in CI instead of being silently marked green. - Warn when listPulls hits the 100-item per_page ceiling so a future growth spurt past 100 open PRs doesn't silently drop indexes. Nit - Tighten NODE_OPTIONS --max-old-space-size from 1536 to 1280MB, giving KuzuDB's C++ heap ~512MB of room under the 1792MB cgroup limit instead of ~256MB. - Rewrite the stale "headroom for Caddy" comment in entrypoint.sh (Caddy lives in a separate container now). - Restore load-bearing comments in install-extensions.js explaining the @ladybugdb/core path layout and the throwaway-db cache-priming pattern. - Parameterize the docker-compose image reference as ${GITNEXUS_IMAGE:-ghcr.io/danny-avila/librechat-gitnexus:latest} so forks or pinned version tags can override via /opt/gitnexus/.env. Deferred - Finding 12 (memory headroom) addressed partially via the heap cap reduction; full profiling of KuzuDB C++ allocations under query load deferred to post-deploy monitoring. * review: resolve 8 follow-up findings from second review pass Security - F1: pipe GHCR token via SSH stdin instead of expanding it into the remote command string. Previously `"echo '$GH_TOKEN' | docker login"` expanded the token locally before SSH sent it as an argument, so the live token was briefly visible in /proc/<pid>/cmdline on the droplet to any process running as deploy or root. New form uses `printf '%s' "$GH_TOKEN" | ssh ... "docker login --password-stdin"` so the token only travels through the encrypted SSH stdin pipe. Reliability - F2: add json-file log rotation (50m x 3 files) via a YAML anchor shared by both services. Default Docker logging is unbounded and would eventually fill the 60GB droplet disk. - F4: set memswap_limit=1792m to match mem_limit. Without this, Docker lets the container silently spill onto host swap when KuzuDB's C++ heap overruns the 1792m RAM budget, turning sub-second graph queries into multi-second ones with no alert. Hard OOM-kill is preferable — unless-stopped restarts the container, the deploy health poll catches it, the failure is explicit. - F5: extend the post-deploy health poll from 24 iterations (120s) to 36 iterations (180s) so it clears Docker's own unhealthy-detection ceiling (start_period 60s + retries 3 * interval 30s = 150s). A container that legitimately takes 125s to warm up would previously fail the deploy at 120s while Docker would still report it as "starting". Operability - F3: document the `--no-deps` escape hatch in the compose file header so an operator can restart Caddy during a gitnexus outage without being trapped by the service_healthy dependency (e.g. emergency Caddyfile fix while gitnexus is thrashing). - F7: rewrite the misleading service_healthy comment. The old text said it prevents 502s "after a restart", implying continuous protection. Clarified that depends_on only governs initial compose up ordering — during force-recreates Caddy briefly routes to a starting gitnexus and the deploy's health poll is the actual guard. - F6: add `shopt -s nullglob` before the prune loop so an empty /opt/gitnexus/indexes directory is an explicit no-op instead of relying on the quirk that `rm -rf "*"` (with literal "*") silently succeeds. Next reader won't have to recognize the bash default. - F8: soft-fail PR artifact downloads when the artifact disappeared between resolve and download. Main/dev artifact failures stay fatal because a missing main/dev index is a real deploy failure, but a deleted PR artifact no longer aborts the whole deploy. * review: resolve 3 NITs from third review pass - F1: rewrite the printf '%s' comment. The previous version claimed docker login --password-stdin rejects trailing newlines, which is inaccurate — docker login strips whitespace. The real reason for printf over echo is byte-exact output and portability, and the token-in-process-table security rationale is already documented in the preceding sentences. - F2: when a PR artifact download soft-fails, the PR's name stays in active_names so the prune step keeps the droplet's existing copy instead of wiping it (stale > empty). Make this transition visible by spelling it out in the :⚠️: message. - F3: fencepost fix in the health poll. The previous loop ran 36 iterations and claimed "180s" in the comment, but the final iteration exits without a trailing sleep, so the real ceiling was 35 * 5s = 175s. Extended to 37 iterations (36 sleeps * 5s = 180s) so the comment matches reality.	2026-04-11 13:04:46 -04:00