24 commits

Author	SHA1	Message	Date
Danny Avila	03ecac8ac1	🧪 ci: Resolve DataTable test infinite re-render (#13947) ... DataTable.spec failed with "Too many re-renders" (35 tests). Root cause: @tanstack/react-virtual is measurement-driven, and jsdom has no real layout, so its re-render loop never converges. This went unnoticed because packages/client had no jest CI job (only the client workspace runs jest in frontend-review.yml). - DataTable: only read the virtualizer (getVirtualItems/getTotalSize) when virtualization is active; the non-virtualized branch renders rows directly, so engaging it for small tables was wasted render-phase work. - Spec: mock @tanstack/react-virtual, since jsdom can't exercise real virtualization layout. - Add a test:ci script to @librechat/client and a Tests: @librechat/client CI job so packages/client specs run on every frontend PR.	2026-06-24 23:40:18 -04:00
Danny Avila	70f7450bab	🪟 ci: Shard Windows Frontend Unit Tests (#13651) ... * 🪟 ci: Shard Windows Frontend Unit Tests Mirror the 4-way jest sharding the Ubuntu frontend test job already uses onto the Windows job, which currently runs the whole client suite in a single 20-minute job. Also drops the `--verbose` flag, which npm consumed itself (it preceded `--`) and only raised npm's own log level. * 🪟 ci: Trigger Frontend Tests on Workflow Changes	2026-06-10 09:00:28 -04:00
Danny Avila	cb6dbc8f60	⚡ refactor: Migrate @librechat/client build to tsdown (#13596) ... * ⚡ refactor: Migrate @librechat/client build from Rollup to tsdown Mirrors the data-schemas migration. Replaces Rollup (rpt2 + postcss) with tsdown (rolldown + oxc); the package build drops from tens of seconds to ~0.3s. - Emit isolated-declaration .d.ts via oxc (dts.oxc) and enforce isolatedDeclarations in tsconfig for editor DX (source made clean: explicit export type annotations added across src, no `any`). - Extract component CSS to dist/style.css so the CJS output stays valid CommonJS (the prior postcss runtime-injection produced an ESM import in the CJS bundle that breaks jest/require). Imported once in the client app entry; Vite bundles it for the app. - Repoint package.json to dual .mjs/.cjs + .d.mts/.d.cts and add ./style.css and ./package.json exports. - Update CI build-cache keys to hash tsdown.config.mjs; remove rollup.config.js. * 🔧 chore: address Codex review on client tsdown migration - Add tsdown.config.mjs to turbo.json build `inputs` so changes to the new bundler config invalidate the Turbo cache (the shared inputs only listed the rollup configs). Also covers the already-migrated data-schemas. - Name the memoized default export (ControlComboboxMemo) instead of the codefix-generated `_default_1`, for clearer stack traces / grepping.	2026-06-08 11:20:03 -04:00
Danny Avila	fbc51f6e66	⚡ refactor: Migrate data-provider Build to tsdown (split tsc dts) (#13597) ... Replace the Rollup + `rollup-plugin-typescript2` build with a split pipeline: tsdown (rolldown) bundles the JS in ~0.2s, and plain `tsc` emits the declarations to `dist/types` (~2s). Full cold build drops from ~9.2s to ~2.5s (~3.6x) with zero source changes. Unlike data-schemas, the fast oxc/isolated-declarations dts path isn't viable here: the package's 78 exported zod schemas produce 374 `isolatedDeclarations` errors (TS9013/TS9038) and a `z.ZodType<T>` annotation would break the 76 downstream `.extend`/`.shape`/`.pick` usages. Plain `tsc` keeps the rich zod types intact, and since dts was never the bottleneck (rollup-plugin-typescript2 was), the win stands. - dts stays unbundled in `dist/types/` — identical to the prior output, so the existing deep `dist/types` imports and the exports `types` paths are unchanged. - ESM output renamed `index.es.js` -> `index.mjs` (via the exports map; no consumer hardcodes the old path). cjs/types paths unchanged. - `./react-query` now emits a real cjs build + types — the exports map already promised them, but Rollup only ever built the esm file. - Kept `rollup` + the plugins used by `server-rollup.config.js` (the `rollup:api` server-bundle smoke test in backend-review.yml); removed only the deps used solely by the deleted `rollup.config.js`. - Repointed CI build-cache keys from `rollup.config.js` to `tsdown.config.mjs`.	2026-06-08 11:09:16 -04:00
Danny Avila	265d660076	👷 ci: Type-check the Client Workspace (#13560) ... The `client/` workspace was never type-checked: the existing typecheck job only covered `packages/` and `api/`, and Vite/esbuild transpiles without type-checking, so type errors shipped through every CI gate. - Add a `typecheck` job to frontend-review.yml running `tsc --noEmit` over `client/` (zero tolerance), reusing the data-provider + client-package build artifacts. Triggers on `client/**`, `packages/client/**`, `packages/data-provider/**`. - Fix all 168 pre-existing client type errors this surfaced (source + tests), including genuine latent bugs: - `getFileConfig()` was typed as merged `FileConfig`, but the server returns the raw config that `mergeFileConfig()` consumes (`TFileConfig`). - SidePanel/Agents `Retrieval`/`ImageVision` were bound to `AgentForm` but use the assistants `Capabilities` enum → `AssistantForm`. - `useSearchResultsByTurn` read a `sources` field its type lacked. - Removed orphaned dead code: `Artifacts/Mermaid.tsx` (imported a never-installed dep) and dead barrel re-exports (`./Plugins`, `./MCPAuth`). - Narrow `client/tsconfig.json` to the client app (drop `../e2e` and `../config/translations`, which reference backend/tooling modules) so the gate's scope matches its trigger. No `any`/`@ts-ignore`/`as unknown as`. Localized newly-surfaced strings.	2026-06-06 18:40:31 -04:00
Danny Avila	bfb6b224d2	🔧 chore: Update ESLint config, Import Sorting script, Test Sharding, Bump @librechat/agents (#13552) ... * 🔧 chore: Update ESLint config, add import sorting script, Test Sharding, Bump `@librechat/agents` * Change 'no-nested-ternary' rule from 'warn' to 'error' in ESLint config * Add new scripts for sorting imports in the project * Update lint-staged configuration to include import sorting * Modify GitHub Actions workflows to support sharding for unit tests * chore: remove nested ternary expressions * refactor: Extract scale multiplier logic into a separate function in CircleRender component * refactor: Simplify auto-refill rendering logic in Balance component for better readability * refactor: Improve width style handling in DataTable components for clarity and maintainability * chore: remove CircleRender component * delete: Remove CircleRender component as it is no longer needed in the project * chore: Bump @librechat/agents to version 3.2.31 and update Node.js engine requirement * Update @librechat/agents dependency from 3.2.2 to 3.2.31 in package-lock.json, api/package.json, and packages/api/package.json * Change Node.js engine requirement from >=20.0.0 to >=24.0.0 in @librechat/agents * chore: Add import sorting check to ESLint CI workflow * Implement a new job in the GitHub Actions workflow to verify import ordering on changed files. * The job checks for changes in specific file types and reports any import order drift, providing instructions for local fixes.	2026-06-06 12:31:55 -04:00
Danny Avila	fb282a2afa	🐳 chore: Upgrade Docker Builds To Node 24 (#13448) ... * chore: upgrade docker builds to node 24 * test: avoid array at in telemetry spec	2026-06-01 10:03:18 -04:00
Danny Avila	21574f02ca	🛡️ chore: Harden CI Supply Chain Workflows (#13090) ... * chore: harden CI supply chain workflows * chore: address CI hardening review feedback * chore: tighten GitNexus dispatch hardening * chore: use app token for Locize PR automation * chore: use dedicated token for Locize PR automation	2026-05-18 16:55:25 -04:00
Marco Beretta	f6ee2ea0ee	📜 feat: Skills UI + Initial E2E CRUD / Sharing (#12580) ... * 🎨 feat: Skills UI — Create/Edit/Share/List with Conditional File Tree First-pass UI on top of the CRUD API scaffolding (#12613). Ships the full user-facing flow for inline, single-SKILL.md skills and leaves a clean drop-in for phase-2 multi-file support. - Create a skill from /skills/new with name (kebab-case, validated), description, and SKILL.md body — wired to the real `useCreateSkillMutation` and `TCreateSkill` payload. - List skills in a sidebar (SkillsSidePanel) via `useListSkillsQuery` with live search filtering. - Edit any skill the caller has EDIT permission on — `useUpdateSkillMutation` passes `expectedVersion` for optimistic concurrency and surfaces 409 conflicts as a warning toast + cache refetch. - Non-blocking `TSkillWarning[]` (e.g. "description too short") are shown inline above the form after a successful create/patch. - Read-only mode when the current user lacks EDIT — the form still renders but inputs are marked `readOnly` and the save/reset buttons are hidden. - Share via ACL using the existing `GenericGrantAccessDialog` — the `ShareSkill` button is gated on the SHARE permission. - Delete with confirmation, driven by `useDeleteSkillMutation({ id })`. - Conditional file tree: only rendered when `useListSkillFilesQuery` returns > 0 files. The tree groups flat `relativePath` strings into a nested view (no `react-arborist` dependency) and supports per-file deletion via `useDeleteSkillFileMutation`. Upload is intentionally deferred — the backend stubs it at 501 in phase 1. - New routes: `/skills`, `/skills/new`, `/skills/:skillId`. - Sidebar accordion (`SkillsAccordion` wrapping `SkillsSidePanel`) added to `useSideNavLinks` gated on `PermissionTypes.SKILLS` USE. The initial UI branch (#12580) shipped a lot of exploration code on top of a now-superseded placeholder backend. Kept as complementary: the `Skills/` component tree, translation keys, role descriptions, `PublicSharingToggle` SKILL mapping, `resources.ts` SKILL config, `useCanSharePublic` SKILL mapping, and `data-provider/roles.ts` `useUpdateSkillPermissionsMutation`. Deferred out of this first pass: - Skill favorites (`useSkillFavorites`, `getSkillFavorites` endpoint) — the backend route doesn't exist yet; saving for a follow-up. - AgentConfig `SkillSelectDialog` integration — the UI branch had this gated behind `false &&`; rolled back with the config. - `InvocationMode` / `CategorySelector` / `parseSkillMd` / tree-node mutations — not in the Anthropic skill spec and not in the CRUD API. - `react-arborist` dependency — replaced with a hand-rolled recursive tree built from flat `TSkillFile[]`. - 38 data-schemas skill model tests: pass - 25 api skill route tests: pass - 16 user-controller cleanup tests: pass * 🔐 feat: Default-On Skills in Interface Config and Role Seeder The skills accordion was registered in the side nav gated on `PermissionTypes.SKILLS` USE, but no one was actually seeding that permission on startup, so a fresh install had the USER role with zero skill permissions and the accordion never rendered. Fixes three gaps: 1. `interfaceSchema` in data-provider's `config.ts` had no `skills` field at all. Added it alongside the existing agents/prompts shape (boolean | { use, create, share, public }) and a default of `{ use: true, create: true, share: false, public: false }`. 2. `loadDefaultInterface` in data-schemas passed every interface key through to the loaded config EXCEPT `skills`. Added the one-line passthrough so `appConfig.interfaceConfig.skills` is actually populated on boot. 3. `updateInterfacePermissions` in packages/api/src/app/permissions.ts seeds role permissions from the interface config on every restart. Added: - `SKILLS` case to `hasExplicitConfig` - `skillsDefaultUse/Create/Share/Public` extraction (mirrors prompts/agents) - `PermissionTypes.SKILLS` block in `allPermissions` that falls through config → roleDefaults → schema default, same pattern as AGENTS and PROMPTS - `SKILLS` entry in the share-backfill array so that pre-existing SKILL role docs missing SHARE/SHARE_PUBLIC get them filled on the next restart Test expectations updated: seven `expectedPermissionsFor(User|Admin)` blocks in `permissions.spec.ts` now include SKILLS, matching the role-default values (USER: use+create true, share/public false; ADMIN: all true). Result: on a fresh install, a regular USER gets skill USE/CREATE and the "Skills" accordion shows up in the chat side panel without any yaml config. Admins can lock it down per role or per tenant via `interface.skills` in librechat.yaml. Tests: - 34 packages/api permissions.spec.ts: pass - 151 packages/api app tests: pass - 38 data-schemas skill.spec.ts: pass - 928 data-provider tests: pass - 25 api skills.test.js: pass * ♻️ fix: Resolve Skills UI Review Findings Addresses the 13 findings from the PR review against the prior commit. 1. **canEdit consistency** — extracted `useSkillPermissions(skill)` as the single source of truth for owner/admin/ACL gating. `SkillsView`, `SkillForm`, `ShareSkill` all consume it; `SkillFileTree`'s per-file delete button now honors admin + EDIT-bit permissions instead of just ownership. Unit tests cover owner, admin, editor-ACL, viewer-ACL, owner-ACL, loading, and undefined-skill cases. 2. **Disabled submit buttons** — create/edit form submit buttons now set native `disabled` (not just `aria-disabled`) during `isLoading`. `onSubmit` also guards with an early return when the mutation is still in-flight so a duplicate enter-key submit can't create two skills. 3. **Wrong maxLength error message** — description/name `maxLength` rules no longer re-use `com_ui_skill_*_required`. Added dedicated `com_ui_skill_name_too_long` and `com_ui_skill_description_too_long` keys with the literal limit interpolated (`{{0}}`). 4. **Search debouncing** — `SkillsSidePanel` now threads the filter input through the existing `useDebounce` hook (250ms) so typing "skills" no longer fires six separate list queries. 5. **Frontend test coverage** — added: - `tree.test.ts` (9 tests) covering `buildTree` / `nodeKey` edge cases: empty input, single root file, multiple roots, nested folders, deeply-nested trees, lexicographic sort, empty paths, stable keys - `useSkillPermissions.test.ts` (7 tests) covering every precedence branch (owner / admin / EDIT / VIEW / owner-ACL / loading / undef) Form integration tests proved flaky against react-hook-form's async `isValid` with our jest-dom mock setup; deferred to a follow-up PR with a proper `@librechat/client` test harness. 6. **Shared `SKILL_NAME_PATTERN`** — promoted the regex plus the four length constants (`SKILL_NAME_MAX_LENGTH`, `SKILL_DESCRIPTION_MAX_LENGTH`, `SKILL_DESCRIPTION_SHORT_THRESHOLD`, `SKILL_DISPLAY_TITLE_MAX_LENGTH`, `SKILL_BODY_MAX_LENGTH`) out of `packages/data-schemas/src/methods/skill.ts` and into `packages/data-provider/src/types/skills.ts`. The data-schemas module now aliases the shared exports so the backend validator and the frontend form share one source of truth. Also fixed a latent bug: the client regex was stricter than the backend (`^[a-z0-9]+(?:-[a-z0-9]+)*$` vs. the real `^[a-z0-9][a-z0-9-]*$`), which would have rejected valid names like `foo--bar` client-side. 7. **Removed hardcoded "Claude"** — replaced `com_ui_skill_description_help` ("Claude uses this to...") with a new `com_ui_skill_create_subtitle` for the form header and `com_ui_skill_description_field_hint` ("This is what the model reads to decide...") for the inline hint. LibreChat is LLM-agnostic; the old copy misled GPT/Gemini users. 8. **Lifted tree mutation hook** — `useDeleteSkillFileMutation` is now instantiated once in `SkillFileTree` (not per `TreeRow`). A `TreeContext` provides `onDeleteFile` + `isDeleting` + `canEdit` to rows. A 60-node tree used to instantiate 60 mutation hooks; it now instantiates one. 9. **List O(n) re-render** — `SkillListItem` no longer reads `useParams()` directly. `SkillList` reads the active id once and passes `isActive` as a prop, so navigation only re-renders the two items whose `isActive` flipped (memo'd), not all N items. 10. **Deduped help text** — the field-level hint and form-level subtitle now use different translation keys with distinct copy instead of showing the same sentence twice on the same page. 11. **Removed ineffective `useCallback`** — `DeleteSkill.handleDelete`, `CreateSkillForm.onSubmit` / `.handleCancel`, `SkillForm.onSubmit`, and `SkillFileTree.handleDeleteFile` all wrapped closures around React Query `mutation` refs, whose identities change every render. Their dep arrays invalidated every render, making the memo a no-op with extra overhead. `SkillFileTree` now destructures the stable `mutate` function and inlines the arrow inside the memoized `contextValue` — one stable reference per deps change. 12. **Import order** — fixed shortest→longest package ordering and longest→shortest local ordering across all touched skill files per AGENTS.md. `react` always first where imported. 13. **Memoization principle** — documented the rule with inline comments: `memo` on components that appear in repeated contexts (`TreeRow`, `SkillListItem`) or as children of frequently-re-rendering parents (`ShareSkill` / `DeleteSkill` under `SkillForm`'s per-keystroke form-state updates). Removed `memo` from `SkillFileTree` since its parent `SkillDetailPanel` only re-renders on query-data changes. - 38 data-schemas skill.spec.ts - 34 packages/api permissions.spec.ts - 25 api skills.test.js - 16 client unit tests (9 buildTree + 7 useSkillPermissions) - All type-checks + eslint clean on touched files * 🧹 fix: Skills Duplication, Input Styling, Remove LLM-specific Copy Three UI fixes from an in-chat review pass: 1. **Sidebar duplication** — `SkillsView` was rendering its own `SkillsSidePanel` aside alongside the chat side panel's `SkillsAccordion`, so on `/skills` the user saw the skill list twice. Fixed by mirroring the `InlinePromptsView` pattern: the route content is now just the detail / create panel and the chat side panel is the sole list. Added `/skills → /skills/new` redirect and a `/skills/new` literal route so `useParams().skillId` is `undefined` for "new" (matches prompts). 2. **Name Input styling** — the big floating-label pattern used by prompts/agents for the primary name field was replaced with a conventional `<Label>` + `<Input>` above it, diverging from the rest of the app. Restored the prompts-style `text-2xl` input with the peer-focus animated label on both `CreateSkillForm` and `SkillForm`. Kept the conventional pattern for description and body since they're textareas. 3. **Remove LLM-specific copy from skill translations** — dropped `com_ui_skill_description_help` ("Claude uses this to...") and the transitional "This is what the model reads..." phrasing. Field hint is now a neutral "Be specific about when this skill should apply." and the create-page subtitle is a neutral "Author a new skill your agents can invoke." LibreChat is LLM-agnostic; baking product names into user-facing copy is wrong outside the `com_endpoint_anthropic_*` keys where the setting actually only applies to Claude models. Side-effect: the `SkillDetailView` wrapper in `SkillsView` now only renders the file-tree aside when the skill has > 0 files — same conditional-tree behavior as before, just scoped to this route instead of also trying to also render a list sidebar. - 16 client skill tests still pass - Type-check + eslint clean on touched files * 🎁 feat: Restore Skills UI from PR #12580 Brings back everything the original UI PR (#12580, commit da039917c) shipped that my earlier rebase dropped. Verbatim restores where possible; adapts the new hooks/types where the backend contract has shifted. **Scoped-out / gated-off (now restored as inert UI scaffolding):** - `hooks/useSkillFavorites.ts` + `utils/favoritesError.ts` + the `useGetSkillFavoritesQuery` / `useUpdateSkillFavoritesMutation` additions in `data-provider/Favorites.ts`. The backend route doesn't exist yet — the data-service functions resolve with empty arrays so the Star UI is a visual-only no-op until phase 2. - `dialogs/SkillSelectDialog.tsx` + the "Add Skills" section in `SidePanel/Agents/AgentConfig.tsx` (still gated behind the original `false &&`) + `skills?: string[]` on `AgentForm` / `Agent` / `AgentCreateParams` / `AgentUpdateParams` + the `skills: []` entry in `defaultAgentFormValues`. - `TUserFavorite.skillId` reserved on the shared favorites type. **Concept-is-gone / deleted-types (restored as UI-only types + stubs):** - `InvocationMode` enum and `TSkillNode`, `TSkillTreeResponse`, `TCreateSkillNodeRequest`, `TUpdateSkillNodeRequest` types in `packages/data-provider/src/types.ts`. UI-facing only; the backend flat `TSkillFile[]` contract is unchanged. - `TSkill.invocationMode?: InvocationMode` as an optional field. Forms read/write it in local state and deliberately drop it from the PATCH payload until the backend column lands. - `tree/SkillFileTree.tsx` (`react-arborist`-based), `SkillTreeNode.tsx`, `TreeToolbar.tsx`, `SkillFileEditor.tsx`, `SkillFilePreview.tsx` — full filesystem-style browser UI restored verbatim. - `data-provider/Skills/tree-queries.ts` + `tree-mutations.ts` hooks (`useGetSkillTreeQuery`, `useCreateSkillNodeMutation`, etc.). The `data-service` stubs them: `getSkillTree` returns `{ nodes: [] }`, `createSkillNode` / `updateSkillNode` / `updateSkillNodeContent` return synthetic node shapes, `deleteSkillNode` resolves void. Hooks compile and run; tree is empty until phase 2 wires a real backend. - `MutationKeys.createSkillNode` / `updateSkillNode` / `deleteSkillNode` / `updateSkillNodeContent` + `CreateSkillNodeBody` / `UpdateSkillNodeVariables` / `DeleteSkillNodeBody` / `UpdateSkillNodeContentVariables` types. - `QueryKeys.skillTree` / `skillNodeContent` / `skillFavorites` / `favorites` and the `skillTree()` endpoint helper. **Scope-simplified (restored with minimal adaptation):** - `display/SkillDetailHeader.tsx` + `display/SkillDetail.tsx`. Header now falls back to `InvocationMode.auto` when `skill.invocationMode` is undefined. - `forms/SkillContentEditor.tsx` — click-to-edit markdown preview toggle for the SKILL.md body field. Wired into both `CreateSkillForm` and `SkillForm` replacing the plain `<TextareaAutosize>`. (Needed `@ts-ignore` on `remarkPlugins` / `rehypePlugins` for the same `PluggableList` vs `Pluggable[]` shape drift `MarkdownLite.tsx` already works around.) - `forms/InvocationModePicker.tsx` + `forms/CategorySelector.tsx` — the auto/manual/both dropdown and the skill category selector. Wired into both forms inside a `FormProvider` so the Controller-based widgets can read `useFormContext`. `category` flows to the PATCH / POST payload as before; `invocationMode` is UI-only per the type note above. - `buttons/CreateSkillMenu.tsx` + `utils/parseSkillMd.ts` — dropdown with AI / Manual / Upload SKILL.md entries + the YAML frontmatter parser for the upload path. `CreateSkillForm.defaultValues` now accepts the parsed shape, so the upload → redirect → pre-populated form flow works again. - `buttons/AdminSettings.tsx` — admin permissions dialog. Uses the existing `useUpdateSkillPermissionsMutation` which was already wired. - `sidebar/FilterSkills.tsx` — restored filter + AdminSettings + CreateSkillMenu wrapper. `SkillsSidePanel.tsx` is back to the original `FilterSkills`-based layout. - `lists/SkillList.tsx` + `lists/SkillListItem.tsx` — restored verbatim. - `layouts/SkillsView.tsx` — restored the full tree + file editor + file preview layout. The chat side panel keeps its own accordion list; this view is the inline detail experience. - `hooks/Generic/useUnsavedChangesPrompt.ts` — route-leave guard hook. - `useGetSkillByIdQuery` is aliased to `useGetSkillQuery` so restored components (`SkillsView`, `SkillForm`) that import the old name resolve to the new hook. - `SkillSelectDialog` + `AgentConfig` coerce `skillsData?.skills` instead of `.data` (list response shape drift from the CRUD PR). - `CreateSkillForm` / `SkillForm` wrap their JSX in `FormProvider` so the restored `CategorySelector` and `SkillContentEditor` components — which read `useFormContext` — work inside the existing forms without another refactor. - `CreateSkillForm.defaultValues` prop accepts `Partial<Values> & { invocationMode?: unknown }` so the upload flow's `{ name, description, invocationMode }` shape passes through cleanly. - `SkillsView` route map gains `/skills/:skillId/edit` and `/skills/:skillId/file/:nodeId` so the tree-navigation URLs the original view produces actually resolve. - `client/package.json` gains `react-arborist@^3.4.3`. - ~60 translation keys the restored files reference — invocation labels, edit/create page titles, file editor chrome, tree toolbar tooltips, favorites, admin allow-settings, unknown-file-type, sr_public_skill, delete/rename _var variants — all added to `en/translation.json`. - Prompts-style floating-label name input — kept from my earlier commit so it matches the rest of the app (user reviewed and approved that styling). Hidden skill-body textarea is replaced by `SkillContentEditor` in both forms. - 38 data-schemas skill.spec.ts - 34 packages/api permissions.spec.ts - 25 api skills.test.js - 7 client useSkillPermissions.test.ts - Type-check: pre-existing error count (188) dropped to 120 because my restorations fixed some previously-broken field types. * chore: Update package-lock.json to include react-arborist and memoize-one * feat: Add support for react-arborist in Vite configuration This update introduces a new condition in the Vite configuration to handle the 'react-arborist' package, ensuring it is properly recognized during the build process. This change enhances compatibility with the recently added 'react-arborist' dependency in the project. * 🩹 fix: Hide InvocationMode, Fix SkillContentEditor Click-to-Edit 1. Hide InvocationModePicker from both CreateSkillForm and SkillForm. Component stays on disk for when the backend lands the column. 2. Fix "Click to edit" doing nothing on SkillContentEditor. The `onBlur={() => setIsEditing(false)}` on the TextareaAutosize was racing with `autoFocus` — React renders the textarea, autoFocus fires, then a layout/reconciliation blur fires immediately, bouncing back to preview mode before the user can interact. Removed onBlur; users toggle via the header button or Escape key. * 🎨 feat: Reader-First Skills UI — Match Claude.ai Layout Reworks the Skills UI from form-first to reader-first, matching Claude.ai's skill detail pattern. **Default view is now read-only.** Clicking a skill in the sidebar navigates to `/skills/:id` which renders `SkillDetail` — a clean content view with: - Skill name as the primary heading - Metadata row: "Added by" + "Last updated" (formatted date) - Description block - Rendered SKILL.md body in a bordered card with a source/rendered toggle (eye + code icons, matching Claude.ai's segmented control) No form fields, no save/cancel buttons. The user reads the skill first and takes action deliberately. **Create is now a dialog.** The `/skills/new` route is gone. `CreateSkillMenu` (the + dropdown in the sidebar) now opens `CreateSkillDialog` — a minimal modal with name, description, and instructions fields. Upload-from-file still works: parse → populate dialog → create. Matches Claude.ai's "Write skill instructions" modal. **Edit is behind an action.** The detail view shows an "Edit" button (permission-gated) that navigates to `/skills/:id/edit`, rendering the existing `SkillForm`. The edit route is preserved for direct linking. **Navigation goes to detail, not edit.** `SkillListItem` now navigates to `/skills/:id` (detail) instead of `/skills/:id/edit`. - `display/SkillMarkdownRenderer.tsx` — shared ReactMarkdown component extracted from `SkillContentEditor`. Same remark/rehype plugins, no form dependency. - `display/SkillDetail.tsx` — the reader-first view (replaces the old thin wrapper). - `dialogs/CreateSkillDialog.tsx` — OGDialog modal for skill creation. - `layouts/SkillsView.tsx` — gutted and rebuilt. Three states: no-skill (empty state), skillId (SkillDetail), skillId+edit (SkillForm). Removed full-page CreateSkillForm, removed TreeView. - `buttons/CreateSkillMenu.tsx` — opens dialog instead of navigating to `/skills/new`. Upload flow: parse → set dialog defaults → open. - `lists/SkillListItem.tsx` — navigate to detail, not edit. - `routes/index.tsx` — removed `/skills/new` and file/nodeId routes; `/skills` renders SkillsView directly (empty state). - `display/index.ts`, `dialogs/index.ts` — added new exports. - `locales/en/translation.json` — added ~10 new keys for metadata, toggle labels, dialog title, empty state. * 🩹 fix: SkillContentEditor click-to-edit z-index — button was z-0 behind rendered content * 🩹 fix: Align Edit button size with Share/Delete (size-9) * 🎨 feat: Claude.ai-Style Skill List Panel Rewrites the skills sidebar to match Claude.ai's panel layout: - Header: "Skills" title + search icon (toggles input) + add icon (opens CreateSkillDialog directly, no dropdown menu) - Collapsible "Skills" section with chevron toggle - Skill items: 24px icon badge (rounded square with ScrollText icon) + name only. No description text in the list — that lives in the detail view. Active item gets highlighted bg + bold font. - Removed AdminSettings button from sidebar header — admin config is accessible via the admin dashboard, not cluttering every user's skill list. - Removed FilterSkills wrapper (was Filter + AdminSettings + CreateSkillMenu). The search + create are now inline in the panel header. Files changed: - sidebar/SkillsSidePanel.tsx — full rewrite - sidebar/SkillsAccordion.tsx — simplified wrapper - lists/SkillList.tsx — collapsible section, no description - lists/SkillListItem.tsx — icon badge + name, memo'd * 🎨 fix: Align Skills UI Styling with Prompts Patterns Style alignment pass based on direct comparison with claude.ai and the existing prompts preview dialog. SkillsSidePanel search now replaces the title in the header row when toggled (search icon + input + X close), matching Claude.ai's pattern. Previously it pushed a separate input below the header, wasting vertical space. Close button clears the search term. Replaced `text-text-tertiary` with `text-text-secondary` across SkillDetail, SkillList, SkillForm, CreateSkillForm, CreateSkillDialog, SkillContentEditor. Tertiary was too dark / low contrast. SkillList section chevron label now reads "Personal skills" (matching Claude.ai) via the existing `com_ui_my_skills` key, instead of the generic "Skills" which duplicated the header. Aligned with `PromptDetailHeader` styling: - 48px round icon (ScrollText in bg-surface-secondary circle) - Name + public badge in the icon row - Metadata below the icon: User icon + author, Calendar icon + date (text-xs text-text-secondary with gap-3, matching prompts exactly) - Description uses the same label-above-text pattern as prompts - Content card uses `bg-transparent` border (not bg-surface-primary-alt) - Toggle buttons use size-5 icons and text-text-secondary for inactive Changed from `max-w-lg p-0` to `max-w-5xl` with the same max-height and padding pattern as the prompts PreviewPrompt dialog: `max-h-[80vh] p-1 sm:p-2 gap-3 sm:gap-4`. Close button now renders via default OGDialogContent behavior (removed showCloseButton=false). * 🩹 fix: SkillDetail fills parent height, tighter spacing (px-6 pb-6 gap-2) * 🩹 fix: Align Skills panel header padding (px-4) with list content below * 🩹 fix: Reduce Skills header top padding (pt-2) to align with sidebar icon strip * 🩹 fix: Tighten Skills header (py-2) and detail top (py-2) to align with sidebar icons and match edit view * 🩹 fix: Offset SidePanel Nav pt-2 with -mt-2 on SkillsAccordion so Skills header aligns with icon strip * 🛠️ fix: Increase Node memory limit for production build in package.json * 🩹 fix: Remove top padding from SkillDetail header row (py-2 → pb-2) * 🏗️ refactor: Move pt-2 from SidePanel/Nav wrapper to each panel Removed the global `pt-2` from `SidePanel/Nav.tsx` and pushed it into each panel's own top-level wrapper. This lets each panel own its vertical alignment independently — Skills can sit flush at the top to align with the sidebar icon strip, while other panels keep their original spacing. Panels updated with `pt-2`: - PromptsAccordion (via className on PromptSidePanel) - BookmarkPanel - FilesPanel - MemoryPanel - MCPBuilderPanel - AgentPanel (form wrapper) - AssistantPanel (form wrapper) - ParametersPanel (already had pt-2) SkillsAccordion: removed the -mt-2 hack, now naturally flush. * 🧹 fix: Align CreateSkillDialog field styling + remove 19 unused i18n keys Dialog fields: all three inputs now use consistent `rounded-xl border-border-medium px-3 py-2 text-sm` styling. Replaced the `<Input>` component with a plain `<input>` to avoid the component's built-in `rounded-lg border-border-light` overriding the dialog's border style. Labels use `font-medium` for consistency. Removed 19 unused translation keys from translation.json: com_ui_skill_body, com_ui_skill_body_placeholder, com_ui_skill_create_subtitle, com_ui_skill_file_delete_confirm, com_ui_skill_file_delete_error, com_ui_skill_file_deleted, com_ui_skill_files_empty, com_ui_skill_files_multi_hint, com_ui_skill_list, com_ui_skill_load_error, com_ui_skill_resize_file_tree, com_ui_skill_select_file, com_ui_skill_select_file_desc, com_ui_skills_load_error, com_ui_add_first_skill, com_ui_create_skill_page, com_ui_edit_skill_page, com_ui_save_skill, com_ui_no_skills_title * 🎁 feat: Upload Skill Dialog + Simplified Create Menu New `UploadSkillDialog` matching Claude.ai's upload modal: - Dashed drop zone with drag-and-drop support - Accepts .md, .zip, .skill files - Phase 1: processes .md files (parses YAML frontmatter → creates skill with body as the full file content) - Shows file requirements below the drop zone - On success: navigates to the new skill's detail view `CreateSkillMenu` now has two flat options (no sub-menu): - "Write skill instructions" → opens `CreateSkillDialog` - "Upload a skill" → opens `UploadSkillDialog` Removed the disabled "Create with AI" option and the old file input hidden-element approach. The sidebar `+` button now renders `CreateSkillMenu` directly instead of a standalone create dialog. - Removed 5 unused i18n keys (com_ui_skill_added_by, com_ui_skill_last_updated, com_ui_skills_add_first, com_ui_skills_filter_placeholder, com_ui_skills_new) - Tightened metadata gap in SkillDetail (mt-1 → mt-0.5) - Added 7 new upload-related i18n keys * 🔒 feat: Zip/Skill File Upload Support with Safety Limits Rewrites UploadSkillDialog to properly handle all three accepted file types: - `.md` — reads as text, parses YAML frontmatter, creates skill - `.zip` / `.skill` — reads as ArrayBuffer, extracts with JSZip, finds SKILL.md (at root or one level deep), parses its content, creates skill. Shows spinner during processing. Security guards against zip bombs: - MAX_ZIP_SIZE: 50MB compressed file limit - MAX_ENTRIES: 500 file limit inside the archive - Path traversal rejection: skips entries with `..` or leading `/` - SKILL.md search limited to depth ≤ 2 segments Added `jszip@^3.10.1` to client dependencies (already in the monorepo's node_modules from backend usage). The name is inferred from the zip filename if SKILL.md frontmatter doesn't have one (e.g. `skills-autofix.zip` → `skills-autofix`). * 🚀 feat: Backend Skill Import + Live File Upload Endpoints New endpoint that accepts a single multipart file (.md, .zip, .skill) and creates a skill with all its files in one request: - **.md**: parse YAML frontmatter → create skill with body - **.zip / .skill**: extract with JSZip, find SKILL.md (root or one level deep), create skill from its content, then persist every additional file via `upsertSkillFile` + local file storage strategy. Returns the created skill + an `_importSummary` with per-file results. Security: - 50MB compressed file size limit (multer) - 500 max entries in archive - 10MB per individual file - Path traversal rejection (no `..`, no absolute, validated charset) - File type filter: only .md/.zip/.skill accepted - Rate limited via existing `fileUploadIpLimiter` + `fileUploadUserLimiter` Handler lives in `packages/api/src/skills/import.ts` with injectable deps (`createSkill`, `upsertSkillFile`, `saveBuffer`) for testability. Replaced the 501 stub with a real handler: - Accepts multipart FormData with `file` + `relativePath` - Saves file via local storage strategy - Calls `upsertSkillFile` to persist the SkillFile record - Returns the upserted document - Rate limited, ACL-gated (EDIT permission required) - 10MB per file limit `UploadSkillDialog` now sends the file to `/api/skills/import` via `dataService.importSkill(formData)` — no more client-side JSZip. Removed `jszip` from client dependencies (only backend needs it). Added `importSkill()` in data-service + `importSkill()` endpoint builder in api-endpoints. Updated the file upload test from expecting 501 stub to expecting 400 "no file provided" (live validation). All 25 skill route tests pass. * 🔒 fix: Complete Import Handler — Validation, Ownership, Error Surfacing Fixes several gaps in the skill import flow: 1. **Skill validation now runs and surfaces properly.** The import handler calls the real `createSkill(CreateSkillInput)` which runs `validateSkillName`, `validateSkillDescription`, `validateSkillBody`. Validation errors (SKILL_VALIDATION_FAILED) are caught and returned as 400 with the issue messages. Duplicate-key errors return 409. Previously all errors were swallowed into a generic 500. 2. **`authorName` is now populated.** The `CreateSkillInput` requires `authorName` which was missing — resolved from `req.user.name ?? req.user.username ?? 'Unknown'`, matching the existing create handler. 3. **SKILL_OWNER permission is granted after import.** Calls `grantPermission` with `AccessRoleIds.SKILL_OWNER` so the uploader can edit/delete/share the imported skill. This was entirely missing — imported skills would have been ownerless. 4. **`tenantId` propagated.** Both the skill and each SkillFile record receive `req.user.tenantId` for multi-tenant deployments. 5. **SkillFile records are created in the DB.** Each non-SKILL.md file in the zip is saved to file storage via `saveBuffer` and recorded via `upsertSkillFile`, which validates the relativePath, infers the category from the path prefix, and atomically bumps the skill's `fileCount` and `version`. Import deps now include `grantPermission` from PermissionService, injected in `api/server/routes/skills.js`. * 🐛 fix: Import grant uses accessRoleId (not roleId) — fixes skill not appearing in list * 🎨 fix: Cache invalidation, file tree, frontmatter rendering Three fixes for the skill detail view: 1. **Cache invalidation after import.** UploadSkillDialog now calls `queryClient.invalidateQueries([QueryKeys.skills])` after a successful import so the sidebar list picks up the new skill without requiring a page refresh. 2. **File tree in detail view.** When a skill has `fileCount > 0`, the detail view now queries `useListSkillFilesQuery` and renders a file list below the body card — SKILL.md first, then folders and root files. Icons: Folder for directories, FileText for files. 3. **Frontmatter stripped and rendered as metadata.** YAML frontmatter (`---\nversion: 0.1.0\ntriggers: ...\n---`) is now parsed out of the body before markdown rendering. The `name` and `description` fields are skipped (already shown in the header). Remaining fields (version, triggers, dependencies, etc.) are displayed in a Claude.ai–style grid: label on the left, value on the right, above the rendered markdown content. Source view still shows the full raw body including frontmatter. * 🩹 fix: Always fetch skill files — fileCount may be stale in cached skill object * 🌳 feat: Inline File Tree in Sidebar Skill List Moves the file tree from the bottom of SkillDetail into the sidebar list, matching Claude.ai's pattern: - Multi-file skills show a chevron toggle on the right side of the skill list item - Clicking the chevron expands an inline file tree below the skill name: SKILL.md first, then folders (with folder icon + right chevron) and root files - File list is fetched lazily (only when expanded) via useListSkillFilesQuery - Clicking a file navigates to the skill detail view - Files section removed from SkillDetail — the sidebar is now the sole file tree location, keeping the detail panel clean SkillDetail cleaned up: removed groupFiles helper, file-related state, useListSkillFilesQuery import, FileText/Folder icon imports. * 🌲 feat: Virtualized inline file tree with react-vtree Replace hand-rolled recursive FolderRow/FileRow buttons with a proper virtualized FixedSizeTree from react-vtree for the sidebar skill list. Dynamic height tracks open folders; capped at 350px with smooth expand/collapse transitions. * chore: Remove no longer used SkillFileTree and SkillTreeNode components * chore: Update Vite config to replace 'react-arborist' with 'react-vtree' for module resolution * feat: Skill file content viewing with lazy DB caching - Add `skills` field to `fileStrategiesSchema` so operators can configure a dedicated storage backend for skill files. Falls back by type (image/document) when unset. - Fix hardcoded `FileSources.local` in skill save/import — now uses the resolved strategy via `getFileStrategy(req.config, { context })`. - Replace 501 download stub with real handler that streams from any storage backend and returns JSON `{ content, mimeType, isBinary }`. - Binary detection (null-byte + non-printable ratio on first 8 KB) flags files on first read so they're never re-fetched. - Text content ≤ 512 KB is cached in the SkillFile MongoDB document; subsequent reads skip storage entirely. - Clicking a skill row now expands inline files (not just chevron). - Clicking a file navigates to `?file=<path>` and renders content in a new SkillFileViewer (markdown, code, images, binary placeholder). * chore: Remove react-window and its type definitions from package.json and package-lock.json - Deleted `react-window` and `@types/react-window` dependencies from both `package.json` and `package-lock.json` to streamline the project and reduce unnecessary bloat. * fix: Build errors — remove endpoints import, fix Uint8Array cast - Replace `import { endpoints }` (not public) with inline URL in SkillFileViewer - Remove `as Uint8Array` cast in stream chunk handling - Extend getSkillFileByPath return type with content/isBinary to decouple from data-schemas build artifact resolution * chore: Remove 8 unused i18next keys com_ui_create_skill_ai, com_ui_create_skill_manual, com_ui_delete_folder_confirm_var, com_ui_delete_skill, com_ui_delete_skill_confirm_var, com_ui_delete_var, com_ui_rename_var, com_ui_skill_files * fix: Add configMiddleware to skills router, handle SKILL.md in viewer - Add configMiddleware to skills router so req.config is populated when getLocalFileStream (or any strategy) reads file paths. - Handle SKILL.md in download handler — serves skill.body directly from the Skill document instead of looking for a SkillFile record. - Clicking SKILL.md in sidebar tree now opens the file viewer (matching Claude.ai behavior: file view vs default detail view). * ci: Run unit tests on PRs to any branch Remove the branches filter from both test workflows so contributor PRs targeting feature branches (not just main/dev) get CI coverage. Path filters are kept so tests only run when relevant files change. * fix: Update skills route tests for download handler changes - Mock configMiddleware (sets req.config for file storage access) - Mock getStrategyFunctions and getFileStrategy (storage strategy deps) - Replace 501 stub test with SKILL.md content test + 404 test * fix: Auto-expand files, frontmatter parsing, select-none, prefetch - Auto-expand file tree when navigating directly to a skill URL - Prefetch files for the active skill (eliminates first-expand lag) - Fix frontmatter parser to handle multi-line YAML list values (triggers field was missing because it uses list syntax) - SkillFileViewer now parses frontmatter for .md files — shows structured grid + rendered body (matching SkillDetail's display) with source/rendered toggle - Add select-none to all sidebar skill and file tree buttons * refactor: Derive expanded state from isActive instead of useEffect Replace useEffect sync with deterministic derivation: expanded = hasFiles && (isActive || !collapsed) Active skill is always open. collapsed is a manual toggle that only takes effect on non-active items. * fix: Remove empty space above body card — overlay view toggle Move the rendered/source toggle from a dedicated row (40px of empty space) to an absolute-positioned overlay in the card's top-right corner, matching Claude.ai's layout. * fix: Remove header bars from content editors — overlay action buttons Collapse the full-width header bars ("Skill Content", "Text") in SkillContentEditor, PromptTextCard, and PromptEditor. Action buttons (edit/save toggle, copy, variables) are now absolute-positioned in the card's top-right corner, reclaiming ~46px of vertical space. * fix: Spinner visibility in file viewer — use text-text-secondary * fix: Address review findings — security, correctness, code quality Codex P1: Use $unset instead of undefined to clear cached content and isBinary fields on file re-upload (Mongoose strips undefined). Codex P2: Match skill-file validation errors by error.code instead of error.message substring. F1: Zip bomb defense — track cumulative decompressed bytes (500 MB cap), check declared uncompressed size before buffering each entry. F2: Remove misleading "atomically" from import handler JSDoc. F3: Static import for isBinaryBuffer instead of dynamic import(). F4: Replace console.error with logger in upload handler. F6: Add multer error handler middleware to skills router. F7: Move React import to top of SkillDetail.tsx. F9: Fix variable shadowing (trimmed → item) in parseFrontmatter. F11: Replace JSON.parse(JSON.stringify()) with toJSON() for Mongoose document serialization. F12: Remove dead dynamic import('fs') fallback (memoryStorage always provides file.buffer). F13: Hoist MIME_MAP to module scope to avoid per-call allocation. F16: Share single multer.memoryStorage() instance. * fix: Follow-up review — close zip bomb gap, fix error handler F1: Add post-decompression cumulative byte check with break (the pre-decompression check relies on undocumented JSZip internals that may be absent; this closes the gap unconditionally). F2+F3: Multer error handler now forwards non-multer errors via next(err) instead of swallowing them. Also catches file filter rejections (plain Error, not MulterError) by message prefix. F4: Move isBinaryBuffer import to local imports section per CLAUDE.md import order rules. F5: Simplify dead toJSON branch — createSkill returns a POJO. * nit: Link filter error message to handler prefix check * feat: Accordion expansion + active file highlight in sidebar - Only one skill's file tree can be expanded at a time (accordion). Expansion state lifted from SkillListItem to SkillList. - Selected file gets bg-surface-active highlight in the tree. Skill row uses subtle style (no background) when a file is active, matching Claude.ai's pattern where the file — not the skill — carries the selection state. * style: Adjust margin for file tree in SkillListItem component - Reduced left margin from 10 to 5 for improved layout consistency in the file tree display. * fix: TS error on FileTreeNode, nested ternary, chevron collapse - Make style prop optional to match react-vtree's NodeComponentProps - Flatten nested ternary for skill row active styles - Skill row click expands (but doesn't collapse) files + navigates - Chevron click explicitly toggles collapse (matching Claude.ai where clicking the chevron is how you collapse files) * fix: Upload basePath, reject SKILL.md uploads, add skills permission route - Pass basePath: 'uploads' in per-file upload handler (was defaulting to 'images' path, inconsistent with the import flow). - Reject uploads targeting SKILL.md (reserved path — download handler special-cases it to return skill.body, making an uploaded file unreachable via the API). - Add skills entry to roles router permissionConfigs so PUT /api/roles/:roleName/skills actually reaches a handler instead of returning 404. * feat: Expand content area, move controls to header, reduce padding Default detail view: - Remove rounded-xl bordered card wrapper — content flows directly into the article, capitalizing on full screen width - Move eye/code toggle inline with the divider row - Reduce px-6/pb-6 to px-4/pb-4 File viewer: - Move eye/code toggle from card overlay to the header bar - Add copy-to-clipboard button for text files in the header bar - Remove rounded-xl bordered card wrapper for markdown content - Remove bordered pre wrapper for non-markdown text - Reduce px-6/py-4 to px-4/py-3 Both views maximize content space over decorative chrome. * fix: Stable header height, restore some padding - Fix layout shift in file viewer header: use fixed h-10 so the bar height stays constant whether the eye/code toggle renders (markdown) or not (plain text). - Bump content padding from px-4/py-3 back to px-5/py-4 in both views — the previous reduction was too aggressive. * fix: Grant rollback, path validation, error format, dead code cleanup F2: grantOwnership now rolls back (compensating delete) on failure, matching the create handler. Both markdown and zip import paths check the result and return 500 on grant failure. F4: Upload handler validates relativePath with regex + traversal check before calling downstream upsertSkillFile. F5: Document JSZip _data.uncompressedSize as best-effort; the post-decompression cumulative check is the real safety net. F10: Standardize all upload handler error responses to { error } (was { message }, inconsistent with handlers.ts). F13: Single-pass fileResults accumulation in import handler. F1-5: Remove dead uploadFileStubHandler (no route references it). Codex P2: Fix delete nav from /skills/new to /skills. F12: Use cn() in UploadSkillDialog instead of template literals. * perf: Stream-first binary detection + O(1) public skill check F1: Download handler now reads only the first 8 KB for binary detection. If binary, the stream is destroyed immediately without buffering the remaining file. Text files continue reading for caching. Eliminates buffering up to 10 MB per request for binary files under concurrent load. F7: Single-skill GET and PATCH now use hasPublicPermission (O(1) ACL lookup) instead of getPublicSkillIdSet (queries ALL public skill IDs). The list handler still uses the Set approach since it serializes multiple skills. serializeSkill/serializeSkillSummary now accept boolean | Set for flexibility. * fix: Update test to match { error } response format * fix: Critical stream truncation bug, grantedBy, error format NF-1 (CRITICAL): Rewrite binary detection to single for-await loop. Breaking out of for-await-of destroys the stream via iterator.return(), so the previous two-loop approach silently truncated text files > 8KB. Now: one loop collects chunks, checks binary after 8KB accumulated, and either destroys+returns (binary) or continues reading (text). NF-2: Add grantedBy to import handler's grantPermission call and interface (was missing, inconsistent with create handler). NF-3: Standardize all import handler error responses from { message } to { error }, matching handlers.ts convention. Update client's UploadSkillDialog to read response.data.error accordingly. * fix: Prefer specific validation message over generic error field * fix: YAML quote stripping, saveBuffer null guard, dot segment rejection - Strip surrounding YAML quotes from frontmatter values so name: "my-skill" parses as my-skill (not "my-skill" with quotes that fails the name validator). - Guard resolveSkillStorage against backends with saveBuffer: null (e.g. OpenAI/vector strategies) — throws a descriptive error caught by the handler's try/catch instead of a TypeError. - Tighten upload path validation to reject . segments (e.g. docs/./a.md) matching the model-layer validator, preventing storage writes for paths the DB will reject. * fix: Orphan cleanup, stream errors, malformed zip, cache latency F1: Upload handler now deletes the stored blob if the subsequent DB upsert fails, preventing orphaned files on disk/cloud. F2: Multer error handler returns { error } (was { message }). F3: Wrap JSZip.loadAsync in try/catch — malformed zip returns 400 instead of falling through to 500. F4: Raw download stream gets an error handler — logs the error and destroys the response if headers were already sent. F8: Strip leading hyphens from inferred skill name so filenames like _my-skill.zip don't produce -my-skill (invalid name pattern). F9: Fire-and-forget all updateSkillFileContent cache writes so the response is sent immediately. Cache failures are logged but don't block or fail the read. * fix: Import orphan cleanup + Content-Disposition sanitization Finding A: Add deleteFile dep to ImportSkillDeps. The per-file loop in handleZip now cleans up stored blobs when upsertSkillFile fails, closing the second half of the F1 orphan fix (upload handler was already fixed). Finding B: Sanitize filename in Content-Disposition header for raw downloads — strip quotes, backslashes, and newlines to prevent header injection from user-uploaded filenames. * security: Prevent stored XSS via raw file downloads Non-image files served via ?raw=true now use Content-Disposition: attachment (force download) instead of inline. An uploaded .html or .svg file served inline from the LibreChat origin could execute scripts with access to the user's session — this closes that vector. Images stay inline (needed for <img> rendering in SkillFileViewer). X-Content-Type-Options: nosniff added to prevent MIME sniffing. * security: Block SVG XSS — allowlist safe raster MIME types for inline SVG (image/svg+xml) passed the startsWith('image/') check and was served inline, but SVG is a scriptable format — embedded <script> tags execute in the LibreChat origin. Replace the prefix match with a Set of safe raster-only MIME types (png, jpeg, gif, webp, avif, bmp). SVGs and any future scriptable image/* subtypes now get Content-Disposition: attachment (forced download). * fix: Cap JSON text response at 1MB, consistent md name inference F3: Text files > 1MB now return { isBinary: false } with no content field, forcing the client to use ?raw=true for download. Prevents buffering 10MB files into heap for JSON serialization. Frontend shows a download fallback when content is absent. F4: handleMarkdown now infers skill name from filename (same as handleZip) when frontmatter has no name, instead of rejecting with 400. Consistent behavior across import paths. F1 (reviewer concern): upsertSkillFile is NOT affected — it uses { new: false } for insert-vs-replace detection but does a follow-up findOne (lines 855-859) to return the post-upsert document. * fix: deleteFile arg shape, raw URL base path, hoist SAFE_INLINE_MIMES Codex P2: deleteFile expects { filepath } object, not a raw string. Both upload handler cleanup and import handler cleanup now pass { filepath } to match the strategy contract (deleteLocalFile, deleteFileFromS3 all expect a file object). Codex P2: Raw download URL in SkillFileViewer now uses apiBaseUrl prefix so subpath deployments (/chat, etc.) resolve correctly. NIT: Hoist SAFE_INLINE_MIMES Set to factory scope — was re-allocated per raw download request inside the if block. * fix: Remove inert cache write for large text files, localize aria-label N2: The { isBinary: false } cache write for text files > 1MB had no effect — subsequent requests still fell through to stream read since neither isBinary nor content provided a fast-path short-circuit. Removed the pointless DB updateOne per request. N4: Replace hardcoded "Back to skill" aria-label with localize(). * refactor: Extract shared parseFrontmatter, widen deleteFile type N3: Extract parseFrontmatter into Skills/utils/frontmatter.ts — single implementation shared by SkillDetail and SkillFileViewer. Accepts optional skipKeys set so callers control which frontmatter fields are excluded (SkillDetail skips name/description since they're shown in the header; other .md files show all fields). N5: Widen ImportSkillDeps.deleteFile file param from { filepath } to { filepath; [key: string]: unknown } to signal extensibility if strategies start accessing additional file properties. * fix: Advance i past list items for skipped keys, DRY parseSkillMd Finding A: parseFrontmatter now consumes multi-line YAML list items before checking skipKeys — prevents list lines from leaking into subsequent key parsing as spurious fields. Finding B: parseSkillMd now delegates to the shared parseFrontmatter instead of re-implementing the same frontmatter scanning loop. Reduces client-side parseFrontmatter implementations from 3 to 1. * fix: Call apiBaseUrl(), delete storage blob on file removal - apiBaseUrl is a function, not a string — call it in the template literal so raw download URLs resolve correctly. - deleteFileHandler now looks up the file record before deleting, then fire-and-forget deletes the storage blob via the strategy's deleteFile. Previously only the DB record was removed, leaving orphaned blobs in local/S3/Firebase/Azure storage. * fix: Clean up storage blobs when deleting an entire skill deleteHandler now lists all files for the skill before calling deleteSkill, then fire-and-forget deletes each blob via the storage strategy. Previously only per-file deletion cleaned up blobs — deleting a whole skill left all associated files orphaned in local/S3/Firebase/Azure storage. * refactor: useImportSkillMutation hook, fix TSkill[] unsafe cast - Create useImportSkillMutation in mutations.ts + ImportSkillOptions type. UploadSkillDialog now uses the mutation hook instead of calling dataService.importSkill directly with manual useState loading management. Eliminates unmounted-component state update risk and aligns with the React Query mutation pattern used by every other mutation in the codebase. - SkillSelectDialog: replace as unknown as TSkill[] with proper TSkillSummary typing. SkillCard props updated to TSkillSummary. The dialog only uses summary-level fields (name, description, category, author) — the cast was hiding a type mismatch. * fix: Use saved source for import cleanup, delete old blob on replace Codex P2: Import cleanup now uses file.source (the backend the file was actually saved to) instead of re-resolving from config. In mixed strategy setups, the previous approach could target the wrong backend. Codex P2: When re-uploading a file to an existing relativePath, the old blob is now deleted after successful upsert. Previously only the DB record was replaced, leaving the old storage object orphaned. * fix: Register PUT /:roleName/skills route in roles router * fix: Re-read skill after zip file processing for fresh metadata The import response was built from the skill object created before the file loop, but each upsertSkillFile bumps version and fileCount. Clients caching the stale response would get 409 conflicts on first edit and see incorrect file counts. Now re-reads the skill via getSkillById after the loop so the response reflects the current version, fileCount, and updatedAt. * fix: Size-check SKILL.md before decompression, don't gate on fileCount P1: SKILL.md was decompressed before any size accounting. A crafted archive could expand SKILL.md past 10MB before validation ran. Now checks declared size pre-decompression and actual size post, both against MAX_SINGLE_FILE_BYTES. P2: File list query was gated on cached fileCount which can be stale after mutations. Now fetches files for the active skill regardless of fileCount. hasFiles derived from fetched data with fileCount as fallback, so newly uploaded files appear without hard refresh. * fix: Move files declaration before hasFiles to avoid TDZ error * security: Stream-decompress zip entries with enforced byte cap Replace zipEntry.async('nodebuffer') (buffers entire entry before checking limits) with zipEntry.nodeStream('nodebuffer') piped through a byte counter that destroys the stream when the per-file or cumulative limit is exceeded. Previously, when JSZip's _data.uncompressedSize was absent (the common case), a high-ratio entry could allocate hundreds of MB before the post-decompression check caught it. Now decompression is aborted mid-stream at the exact byte threshold — no entry can exceed its limit regardless of compression ratio. * refactor: Reorganize access check for prompts in useSideNavLinks hook Moved the prompts access check to a new position in the code to improve readability and maintainability. This change ensures that the prompts link is added to the navigation only if the user has the appropriate access, without altering the existing functionality. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-04-25 04:02:00 -04:00
Danny Avila	c324a8d9e4	⚡ refactor: Parallelize CI Workflows with Isolated Caching and Fan-Out Test Jobs (#12088) ... * refactor: CI Workflow for Backend with Build and Test Jobs - Updated the GitHub Actions workflow to include a new build job that compiles packages and uploads build artifacts. - Added separate test jobs for each package (`api`, `data-provider`, and `data-schemas`) to run unit tests after the build process. - Introduced caching for build artifacts to optimize build times. - Configured Jest to utilize 50% of available workers for improved test performance across all Jest configurations in the `api`, `data-schemas`, and `packages/api` directories. * refactor: Update CI Workflow for Backend with Enhanced Build and Cache Management - Modified the GitHub Actions workflow to improve the build process by separating build and cache steps for `data-provider`, `data-schemas`, and `api` packages. - Updated artifact upload and download steps to reflect the new naming conventions for better clarity. - Enhanced caching strategies to optimize build times and ensure efficient artifact management. * chore: Node Modules Caching in CI Workflow - Updated the GitHub Actions workflow to implement caching for the `node_modules` directory, improving build efficiency by restoring cached dependencies. - Adjusted the installation step to conditionally run based on cache availability, optimizing the overall CI process. * refactor: Enhance CI Workflow for Frontend with Build and Test Jobs - Updated the GitHub Actions workflow to introduce a structured build process for frontend packages, including separate jobs for building and testing on both Ubuntu and Windows environments. - Implemented caching strategies for `node_modules` and build artifacts to optimize build times and improve efficiency. - Added artifact upload and download steps for `data-provider` and `client-package` builds, ensuring that builds are reused across jobs. - Adjusted Node.js version specification for consistency and reliability across different jobs. * refactor: Update CI Workflows for Backend and Frontend with Node.js 20.19 and Enhanced Caching - Updated Node.js version to 20.19 across all jobs in both backend and frontend workflows for consistency. - Enhanced caching strategies for build artifacts and `node_modules`, increasing retention days from 1 to 2 for better efficiency. - Adjusted cache keys to include additional files for improved cache hit rates during builds. - Added conditional installation of dependencies to optimize the CI process. * chore: Configure Jest to Use 50% of Available Workers Across Client and Data Provider - Added `maxWorkers: '50%'` setting to Jest configuration files for the client and data provider packages to optimize test performance by utilizing half of the available CPU cores during test execution. * chore: Enhance Node Modules Caching in CI Workflows - Updated caching paths in both backend and frontend GitHub Actions workflows to include additional `node_modules` directories for improved dependency management. - This change optimizes the caching strategy, ensuring that all relevant modules are cached, which can lead to faster build times and more efficient CI processes. * chore: Update Node Modules Cache Keys in CI Workflows - Modified cache keys in both backend and frontend GitHub Actions workflows to include the Node.js version (20.19) for improved cache management. - This change ensures that the caching mechanism is more specific, potentially enhancing cache hit rates and build efficiency. * chore: Refactor Node Modules Cache Keys in CI Workflows - Updated cache keys in backend and frontend GitHub Actions workflows to be more specific, distinguishing between frontend and backend caches. - Removed references to `client/node_modules` in backend workflows to streamline caching paths and improve cache management.	2026-03-05 13:56:07 -05:00
Doyle	b9792160e2	💾 feat: Add Memory Configuration Options for CI unit tests (#10567) ... * 💾 feat: Add Memory Configuration Options for CI unit tests - configured GitHub Actions workflows with configurable Node.js heap allocation, defaults to 6144 MiB - added heap usage logging for memory monitoring and debugging - increased Docker frontend build memory allocation to ensure consistent memory limits - optimized Jest timeout for tokenSplit test * 💾 feat: Add Memory Configuration Options for CI unit tests - responding to PR feedback from Copilot	2025-12-25 16:17:17 -05:00
Atef Bellaaj	ef1b7f0157	🧩 refactor: Decouple MCP Config from Startup Config (#10689) ... * Decouple mcp config from start up config * Chore: Work on AI Review and Copilot Comments - setRawConfig is not needed since the private raw config is not needed any more - !!serversLoading bug fixed - added unit tests for route /api/mcp/servers - copilot comments addressed * chore: remove comments * chore: rename data-provider dir for MCP * chore: reorganize mcp specific query hooks * fix: consolidate imports for MCP server manager * chore: add dev-staging branch to frontend review workflow triggers * feat: add GitHub Actions workflow for building and pushing Docker images to GitHub Container Registry and Docker Hub * fix: update label for tag input in BookmarkForm tests to improve clarity --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai>	2025-12-11 16:36:34 -05:00
Danny Avila	14660d75ae	🆕 feat: Enhanced Title Generation Config Options (#8580) ... * 🏗️ refactor: Extract reasoning key logic into separate function * refactor: Ensure `overrideProvider` is always defined in `getProviderConfig` result, and only used in `initializeAgent` if different from `agent.provider` * feat: new title configuration options across services - titlePrompt - titleEndpoint - titlePromptTemplate - new "completion" titleMethod (new default) * chore: update @librechat/agents and conform openai version to prevent SDK errors * chore: add form-data package as a dependency and override to v4.0.4 to address CVE-2025-7783 * feat: add support for 'all' endpoint configuration in AppService and corresponding tests * refactor: replace HttpsProxyAgent with ProxyAgent from undici for improved proxy handling in assistant initialization * chore: update frontend review workflow to limit package paths to data-provider * chore: update backend review workflow to include all package paths	2025-07-21 17:37:37 -04:00
Danny Avila	9ec665dd2c	🪟 fix: Windows Build (npm) (#3889) ... * chore: package-lock.json * chore: remove shadcn files (temp) * refactor: language comparisons script * fix: resolve package-lock file for windows compatibility * chore: Enable Windows unit tests for frontend * refactor: move shadcn components to data-provider	2024-09-02 10:01:09 -04:00
Danny Avila	3b61322459	🚀 feat: Enhance PWA and asset caching strategy (#3822) ... * chore: Update VitePWA registerType to 'autoUpdate', add cache busting to static file outputs * chore: disable windows frontend workflow for now	2024-08-28 12:13:14 -04:00
Danny Avila	27e7621b6a	🪟 fix: Windows Vite Build Issue (#2663) ... * fix: Windows CSS VITE build issue: peer-focus:dark => dark:peer-focus * ci: add windows env step for frontend ci/cd	2024-05-10 12:44:43 -04:00
Ed Burnette	49571ac635	chore: Get the latest of all github actions (#1335)	2023-12-14 07:44:38 -05:00
Marco Beretta	b1a96ecedc	feat: auto-scroll to the bottom of the conversation (#1049) ... * added button for autoscroll * fix(General) removed bold * fix(General) typescript error with checked={autoScroll} * added return condition for new conversations * refactor(Message) limit nesting * fix(settings) used effects * fix(Message) disabled autoscroll when search * test(AutoScrollSwitch) * fix(AutoScrollSwitch) test * fix(ci): attempt to debug workflow * refactor: move AutoScrollSwitch from General file, don't use cache for npm * fix(ci): add test config to avoid redirects and silentRefresh * chore: add back workflow caching * chore(AutoScrollSwitch): remove comments, fix type issues, clarify switch intent * refactor(Message): remove unnecessary message prop form scrolling condition * fix(AutoScrollSwitch.spec): do not get by text --------- Co-authored-by: Danny Avila <messagedaniel@protonmail.com>	2023-10-16 11:01:38 -04:00
Danny Avila	c6f5d5d65c	ci: add e2e workflow, optimize client code for testing (#771) ... * refactor(e2e): fix tests with latest changes, convert to TS, use test Ids * chore(EndpointMenu.jsx): add data-testid attribute to new-conversation-menu button * refactor(EndpointItem): add data-testid attr., convert to TS * refactor(e2e): remove unnecessary awaits and convert to TS * chore(playwright.config.local.ts): add absolute path to server index.js file chore(playwright.config.local.ts): add dotenv configuration chore(playwright.config.local.ts): change webServer command to use absolute path chore(playwright.config.local.ts): add NODE_ENV and process.env to webServer env chore(playwright.config.local.ts): remove unused import chore(login.spec.js): delete login.spec.js file * chore(.gitignore): add 'my.secrets' to the list of ignored files fix(Registration.tsx): add 'data-testid' attribute to the error message div fix(Registration.spec.tsx): comment out test case that calls 'registerUser.mutate' * chore(ConvoIcon.tsx): add data-testid attribute to svg element chore(messages.spec.ts): refactor conversation navigation logic * chore(playwright.config.ts): add support for absolute path to server index.js file feat(playwright.config.ts): add support for dotenv configuration feat(playwright.config.ts): set NODE_ENV to 'production' in webServer environment variables * chore(workflows): comment out push event and specify paths for pull_request event in backend-review.yml chore(workflows): comment out push event and specify paths for pull_request event in frontend-review.yml * chore(install.js): add check to skip install script in CI environment * chore: complete playwright workflow * chore(Landing.tsx): add data-testid attribute to landing title element chore(authenticate.ts): update selector to wait for landing title element by test id instead of text content * chore(playwright.yml): add step to upload screenshot artifact on failure fix(authenticate.ts): capture screenshot before waiting for landing title and increase timeout due to GH Actions load time * chore(playwright.yml): rename artifact name from 'screenshot' to 'login-screenshot' feat(LoginForm.tsx): add data-testid attribute to login button fix(authenticate.ts): change screenshot name to 'login-screenshot.png' and conditionally take screenshot only in CI environment * chore(playwright.yml): add CI environment variable and set it to true * chore(playwright.yml): update Playwright installation command chore(playwright.config.ts): update storageState path to use process.cwd() * fix(playwright.yml): update node version to 18 in setup-node action fix(playwright.yml): update actions/cache to v3 in Cache Node.js modules step fix(playwright.yml): update actions/cache to v3 in Cache Playwright installations step fix(authenticate.ts): change login button click to press 'Enter' on password input * chore(playwright.yml): update E2E_USER_EMAIL and E2E_USER_PASSWORD values for testing purposes chore(authenticate.ts): add console.dir to log user object for debugging * chore(playwright.yml): add step to upload storageState artifact The storageState artifact is now uploaded as part of the workflow. This artifact contains the state of the storage used during the end-to-end tests. It will be retained for 2 days. * chore(playwright.yml): comment out upload screenshot step chore(playwright.config.ts): change NODE_ENV to development chore(authenticate.ts): comment out screenshot related code * chore(playwright.config.ts): add SESSION_EXPIRY environment variable with value 86400000 * chore(playwright.yml): update environment variables in Playwright workflow fix(General.tsx): add data-testid attributes to clear conversations buttons test(messages.spec.ts): add setup and teardown steps for clearing conversations before and after tests * fix(messages.spec.ts): fix clearing conversations before and after message tests feat(messages.spec.ts): add beforeEach and afterEach hooks to create and close new page for each test * chore: remove storageStage upload artifact	2023-08-08 11:17:15 -04:00
Danny Avila	600a0d15b1	fix(backend-review.yml): update Node.js version from 19.x to 20.x (#767) ... fix(frontend-review.yml): update Node.js version from 19.x to 20.x	2023-08-06 13:08:23 -04:00
Dan Orlando	04e4259005	Move data provider to shared package (#582) ... * create data-provider package and move code from data-provider folder to be shared between apps * fix type issues * add packages to ignore * add new data-provider package to apps * refactor: change client imports to use @librechat/data-provider package * include data-provider build script in frontend build * fix type issue after rebasing * delete admin/package.json from this branch * update test ci script to include building of data-provider package * Try using regular build for test action * Switch frontend-review back to build:ci * Remove loginRedirect from Login.tsx * Add ChatGPT back to EModelEndpoint	2023-07-04 15:47:41 -04:00
Dan Orlando	2003480fed	Change prepare script to not run in CI mode and remove --ignore-scripts flag from workflow (#491) ... * Change prepare script to not run in CI mode and remove --ignore-scripts flag from workflow * add release/* to branches for workflows	2023-06-13 09:20:47 -04:00
Daniel Avila	297291c541	feat(frontend-review.yml): add support for dev branch in frontend unit tests workflow	2023-06-11 16:45:54 -04:00
Danny Avila	3dadedaf69	test: frontend jest ci/cd & minor fixes post-release (#478) ... * improve final reply for gpt-4, gpt-3.5 needs a more stable approach * fix: better context output for gpt-3.5 * fix: added clarification for better context output for gpt-3.5 * feat(PluginsOptions): add advanced mode to show/hide options style(PluginsOptions): add styles for advanced mode and show/hide options * minor changes to styling * refactor(langchain): add support for custom GPT-4 agent This commit adds support for a custom GPT-4 agent in the langchain module. The `CustomGpt4Agent` class extends the `ZeroShotAgent` class and includes a new `createPrompt` method that generates a prompt template for the agent. The `initializeCustomAgent` function has been updated to use the `CustomGpt4Agent` class when the model is not GPT-3. The `instructions.js` file has also been updated to include new instructions for the GPT-4 agent. The `formatInstructions` method has been removed and replaced with `gpt4Instructions` and `prefix2` and `suffix2` have been added to include the new instructions. feat(langchain): add custom output parser for langchain agents This commit adds a custom output parser for langchain agents. The new parser is called CustomOutputParser and it extends ZeroShotAgentOutputParser. It takes a fields object as a parameter and sets the tools and longestToolName properties. It also sets the finishToolNameRegex property to match the final answer. The parse method of the CustomOutputParser class takes a text parameter and returns an object with returnValues, log, and toolInput properties. This commit also adds a Gpt4OutputParser class that extends ZeroShotAgentOutputParser. It takes a fields object as a parameter and sets the tools and longestToolName properties. It also sets the finishToolNameRegex property to match the final answer. The parse method of the Gpt4OutputParser class takes a text parameter and returns an object with returnValues, log, and toolInput properties. feat(langchain): add isGpt3 parameter to * Stable Diffusion Plugin (#204) * Added stable diffusion plugin * Added example prompt * Fixed naming * Removed brackets in the prompt * fix: improved agent for gpt-3.5 * fix: outparser, gpt3 instructions, and wolfram error handling * chore: update langchain to 0.0.71 * fix: long parsing action input fix * fix: make plugin select close on clicking label/button * fix: make plugin select close on clicking label/button * fix: wolfram input formatting and gpt-3 payload without plugins * chore(api): update axios package version to 1.3.4 feat(api): add requireJwtAuth middleware to askGPTPlugins endpoint fix(api): replace session user with user id in askGPTPlugins endpoint docs(LOCAL_INSTALL.md): update guide for local installation and testing This commit updates the guide for local installation and testing of the ChatGPT-Clone app. It includes instructions for locally running the app, updating the app version, and running tests. It also includes a new option for running the app using Docker. The commit also fixes some typos and formatting issues. * add reverseProxy to plugins client * chore(Dockerfile-app): add Dockerfile for building and running the app in a container docs: remove outdated guides on Google search and Bing jailbreak mode docs(LOCAL_INSTALL.md): remove outdated Windows installation instructions and update MeiliSearch configuration file * fix: handle n/a parsing error better, reduce token waste if no agentic behavior is needed * style: fix formatting and add parentheses around arrow function parameter style: change hover background color to white and dark hover background color to gray-700 * chore: re-organize agent dir and files * feat(ChatAgent.js): add support for PlanAndExecuteAgentExecutor feat(PlanAndExecuteAgentExecutor.js): add PlanAndExecuteAgentExecutor class feat(planExecutor.js): add demo for PlanAndExecuteAgentExecutor * feat: add azure support to plugins * refactor(utils): add basePath endpoint for genAzureEndpoint feat(api): add support for Azure OpenAI API in various modules and tools * feat: add plugin api for fetching available tools * feat: add data service for getting available plugins * feat: first iteration plugin store UI * refactor: rename files to follow proper naming convention * feat: Plugin store UI components * feat: create separate user routes, service, controller, and add plugins to user model * feat: create data service for adding and removing plugins per user * feat: UI for adding and removing plugins, displaying plugins in dropdown based on what user has installed * fix: merge conflicts from main * fix: fix plugin items titles * fix: tool.value -> tool.pluginKey * fix: testing returnDirect for self-reflection * fix: add browser tool to manifest * refactor(outputParser.js): remove commented out code feat(outputParser.js): add support for thought input when there is no action input * handling 'use tool' edge case * merge main to langchain * fix(User.js, auth.service.js, localStrategy.js): change deprecated Joi.validate() to schema.validate() method (#322) * fix(auth.service.js): fixes deprecated error callback in mongoose save method (#323) * chore: run formatting script with new rules * refactor: add requiresAuth to manifest, fix uninstall button * version with plugin auth as dialog modal * feat: Complete frontend for plugin auth * frontend styling updates * feat: api for plugin auth * feat: Add tooltip with field description to plugin auth form * fix: issue with plugin that has no auth * feat(tools): add support for user-specific API keys This commit adds support for user-specific API keys for the following tools: - Google Search API - Web Browser - SerpAPI - Zapier - DALL-E - Wolfram Alpha API It also adds support for OpenAI API key for the Web Browser tool. The `validateTools` function now takes a `user` parameter and checks for user-specific API keys before falling back to environment variables. The `loadTools` function now takes a `user` parameter and initializes the tools with user-specific API keys if available. The `manifest.json` file has been updated to include the new `authConfig` fields for the tools that support user-specific API keys. The `askGPTPlugins.js` file has been updated to use the `validateTools` function with the `user` parameter. refactor(ChatAgent.js): add user parameter to initialize function and pass it to loadTools function refactor(tools/index.js): set default value for tools parameter in validateTools function refactor(askGPTPlugins.js): remove duplicate user variable declaration and use the one from req object * refactor(ChatAgent.js): await validTool() before pushing to this.tools array refactor(tools/index.js): use Map instead of Set to store valid tools refactor(tools/index.js): filter availableTools to only validate tools passed in refactor(PluginController.js): filter out duplicate plugins by pluginKey refactor(crypto.js): use environment variables for encryption key and initialization vector feat(PluginService.js): add null check for pluginAuth in getUserPluginAuthValue() * feat(api): add credentials key and IV to .env.example for securely storing credentials * Adds testing for handling tools, introducing a test env to the backend Fixes bugs & optimizes code as revealed through testing, including: - wolfram.js: fixes bug where wolfram was not handling authentication - ChatAgent.js: ChatAgent modified to reflect 'handleTools' changes - handleTools.js: Moves logic out of index file - handleTools.js: loadTools: returns only requested tools - handleTools.js: validTools: correctly returns tools based on authentication * test(index.test.js): add test to validate a tool from an environment variable * test(tools): add test for initializing an authenticated tool through Environment Variables * refactor(ChatAgent.js): remove commented out code and unused imports * refactor(ChatAgent.js): move instructions to a separate file and import them fix(ChatAgent.js): replace hardcoded instructions with imported ones * refactor(ChatAgent.js): change import path for TextStream refactor(stream.js): remove unused TextStream class * chore(.gitignore): add .env.test to gitignore refactor(ChatAgent.js): rename CustomChatAgent to ChatAgent test(ChatAgent.test.js): add tests for ChatAgent class refactor(outputParser.js): remove OldOutputParser class refactor(outputParser.js): rename CustomOutputParser to OutputParser docs(.env.test.example): add comment explaining how to use OPENAI_API_KEY refactor(jestSetup.js): use dotenv to load environment variables from .env.test file * Various optimizations and config, add tests for PluginStoreDialog * test(ChatAgent.test.js): add test to check if chat history is returned correctly * test: unit tests for plugin store * test: add frontend-test script to root package.json * feat(ChatAgent.js, askGPTPlugins.js): add support for aborting chat requests (in progress) * test: add more client tests * feat(ChatAgent): allow plugin requests to be cancelled * feat(ChatAgent): allow message regeneration * feat(ChatAgent): remember last selected tools * Remove plugins we don't yet have from manifest.json * fix(ChatAgent.js): increase maxAttempts from 1 to 2 fix(ChatAgent.js): change error message to 'Cancelled.' if message was aborted mid-generation fix(openaiCreateImage.js): replace unwanted characters in input string fix(handlers.js): compare action.tool in lowercase to 'self-reflection' * fix(ChatAgent): Fix up plugin I/O formatting for n/a actions * refactor(Plugin.jsx): remove unused import statement feat(Plugin.jsx): add Plugin component with svg paths and styles * refactor: simplify credential encryption/decryption by using a single key and IV for all environments. Update crypto.js and .env.example files accordingly. * fix(ChatAgent.js): reduce maxAttempts from 2 to 1 feat(ChatAgent.js): add model information to responseMessage object feat(Message.js): add model field to messageSchema feat(Message.js): add model field to message object feat(Message.jsx): pass model information to getIcon function feat(getIcon.jsx): add Plugin component and handle plugin messages differently * feat(askGPTPlugins.js): add model property to the ask function response object feat(EndpointItem.jsx): add message property to the EndpointItem component feat(MessageHeader.jsx): add Plugin icon to the plugins section feat(MessageHeader.jsx): change alpha to beta in the plugins section feat(svg): add Plugin, GPTIcon, and BingIcon components to the svg folder refactor(EndpointItems.jsx): remove unused import statement * refactor(googleSearch.js, wolfram.js): change error handling to return a message instead of throwing an error * refactor(CustomAgent): remove commented code and change return object to include returnValues property * feat(CustomAgent.js): add currentDateString to createPrompt method options deps(api/package.json): update langchain to v0.0.81 * fix: do not show pagination if the maxPage is 1 * Add Zapier back to manifest (accidentally removed) * chore(api): update langchain dependency to version 0.0.84 * feat(DALL-E.js): add DALL-E tool for generating images using OpenAI's DALL-E API refactor(handleTools.js): update import for DALL-E tool refactor(index.test.js): update import for DALL-E tool refactor(stablediffusion.js): add check for image directory existence before saving image * refactor(CustomAgent): rename instructions prefix variable to gpt3 and add gpt4 instructions feat(CustomAgent): add support for gpt-4 model fix(initializeCustomAgent.js): pass model name to createPrompt method fix(outputParser.js): set selectedTool to 'self-reflection' when tool parsing fails * style(langchain/tools): update guidelines for image creation in DALL-E and StableDiffusion - Update guidelines for image creation in DALL-E and StableDiffusion tools - Emphasize the importance of "showing" and not "telling" the imagery in crafting input - Update formatting for the example prompt for generating a realistic portrait photo of a man - Generate images only once per human query unless explicitly requested by the user * docs(tools): update tool descriptions for DALL-E and Stable Diffusion - Update the description for DALL-E tool to indicate that it is exclusively for visual content and provide guidelines for generating images with a focus on visual attributes. - Update the description for Stable Diffusion tool to indicate that it is exclusively for visual content and provide guidelines for generating images with a focus on visual attributes. * chore(api): update "@waylaidwanderer/chatgpt-api" dependency to version "^1.36.3" * refactor(ChatAgent.js): use environment variable for reverse proxy url refactor(ChatAgent.js): use environment variable for openai base path refactor(instructions.js): update gpt3 and gpt3-v2 instructions refactor(outputParser.js): update finishToolNameRegex in CustomOutputParser class * refactor(DALL-E.js): change apiKey and azureKey fields to uppercase refactor(googleSearch.js): change cx and apiKey fields to uppercase feat(manifest.json): add authConfig field for Stable Diffusion WebUI API URL refactor(stablediffusion.js): add url field to constructor and change getServerURL() to this.url refactor(wolfram.js): change apiKey field to uppercase WOLFRAM_APP_ID * refactor(handleTools.js): simplify tool loading and add support for custom tool constructors and options * refactor(handleTools.js): remove commented out code and unused imports * refactor(handleTools.js, index.js): change file name from wolfram.js to Wolfram.js and selfReflection.js to SelfReflection.js to follow PascalCase convention * refactor(outputParser.js, askGPTPlugins.js): improve code readability and remove unnecessary comments * feat(GoogleSearch.js): add GoogleSearchAPI tool to allow agents to use the Google Custom Search API feat(SelfReflection.js): add SelfReflectionTool to allow agents to reflect on their thoughts and actions feat(StableDiffusion.js): add StableDiffusionAPI tool to allow agents to generate images using stable diffusion webui's api feat(Wolfram.js): add WolframAlphaAPI tool for computation, math, curated knowledge & real-time data through WolframAlpha. * testing openai specs * doc: fix link in .env.example * package-update * fix(MultiSelectDropDown.jsx): handle null or undefined values in availableValues array * refactor(DALL-E.js, StableDiffusion.js): remove 'dist/' from image path feat(docker-compose.yml): add comments for reverse proxy configuration * chore(.gitignore): ignore client/public/images/ fix(DALL-E.js, StableDiffusion.js): change image path from dist/ to public/ feat(index.js): add support for serving static files from client/public/ directory * fix: remove selected tool when uninstalled * plugin options in progress * fix: fix issue with uninstalling a plugin that is in use and typescript errors * feat(gptPlugins): add Preset support for GPT Plugins endpoint feat(ChatAgent.js): add support for agentOptions object feat(convoSchema.js): add agentOptions field to conversation schema feat(defaults.js): add agentOptions object to defaults feat(presetSchema.js): add agentOptions field to preset schema feat(askGPTPlugins.js): add support for agentOptions object in request body feat(EditPresetDialog.jsx): add support for showing/hiding GPT Plugins agent settings feat(EditPresetDialog.jsx): add support for setting GPT Plugins agent options fix(EndpointOptionsDialog.jsx): change endpoint name from 'gptPlugins' to 'Plugins' feat(AgentSettings.jsx): add AgentSettings component for GPT plugins configuration feat(client): add GPT Plugins settings component and endpoint to Settings component fix(client): remove unused imports in GoogleOptions component feat(PluginsOptions): add support for agent settings and refactor code feat(PluginsOptions): add GPTIcon to show/hide agent settings button feat(index.ts): export SVG components feat(GPTIcon.jsx): add className prop to GPTIcon component feat(GPTIcon.jsx): import cn function from utils feat(BingIcon.tsx): export BingIcon component feat(index.ts): export BingIcon component feat(index.ts): export MessagesSquared component refactor(cleanupPreset.js): add default values for agentOptions in gptPlugins endpoint feat(getDefaultConversation.js, handleSubmit.js): add agentOptions object to conversation object for GPT plugins endpoint. Update default temperature value to 0.8. Add chatGptLabel and promptPrefix properties to conversation object. * fix: set default convo back to null * refactor(ChatAgent.js, askGPTPlugins.js, AgentSettings.jsx): change variable names for better readability and remove redundant code * test: add RecoilRoot to layout-test-utils * refactor(askGPTPlugins.js): remove redundant code and use endpointOption directly feat(askGPTPlugins.js): add validation for tools in endpointOption before using it * chore(ChatAgent.js, Settings.jsx): add agentOptions to saveConvo function and adjust Settings component height The ChatAgent.js file was modified to include the agentOptions object in the saveConvo function. The Settings.jsx file was modified to adjust the height of the component to ensure that all content is visible. * refactor(ChatAgent.js): extract reverseProxyUrl option to a class property and add support for it feat(ChatAgent.js): add support for completionMode option in sendApiMessage method feat(ChatAgent.js): add support for user-provided promptPrefix in buildPrompt method * feat(plugins): allow preset change mid conversation * refactor: playwright config * build: update configs * test: add automated setup process for playwright * test: update messages spec to delete message after its been created * build: add github action for e2e to workflows * make husky install conditional * build: try making husky install exit if running in ci * ignore husky on ci * chore: update OPENAI_KEY to OPENAI_API_KEY in .github/playwright.yml and api/.env.example refactor(chatgpt-client.js): update OPENAI_KEY to OPENAI_API_KEY feat(langchain): add demo-aiplugin.js and demo-yaml.js, remove test2.js, test3.js, and test4.js chore: remove unused test files fix(titleConvo.js): fix typo in environment variable name fix(askGPTPlugins.js): fix typo in environment variable name fix(endpoints.js): fix typo in environment variable name docs: update installation guide to use OPENAI_API_KEY instead of OPENAI_KEY in .env file * fix(index.test.js): change import of GoogleSearchAPI to use uppercase G in GoogleSearch * backtrack and redo * add jwt secret * try manually installing vite * refactor workflow again * move vite to root and add frontend:build script * chore(api): bump langchain version * feat(PluginController.js): authenticate plugins from environment variables if they are set feat(PluginStoreDialog.tsx): show plugin auth form only if plugin is not authenticated by env var and require authentication feat(types.ts): add authenticated field to TPlugin type definition * docs: update google_search.md and add stable_diffusion.md * Update stable_diffusion.md * refactor(Wolfram.js): remove newline characters from query before encoding docs(wolfram.md): add instructions for setting WOLFRAM_APP_ID in api.env to bypass prompt for AppID in plugin * refactor(Wolfram.js): replace deprecated replaceAll method with replace method * Update wolfram.md * fix(askGPTPlugins): error message will reference correct Parent Message * refactor(chatgpt-client.js, ChatAgent.js): simplify maxContextTokens calculation and add promptPrefix parameter to buildPrompt method * docs: initial draft of intro to plugins * Update introduction.md * Update introduction.md * Feature: User/Reg cleanup + Install / Upgrade script for langchain (#427) * test: login tests * test: finish login tests * test: initial tests for registration * test: registration specs * feature: Init a app config file - Simplifies the ENV vars too - Legacy fallbacks for older builds * refactor(auth): Refactor log in/out controllers - Moves both login and logout controllers to their own file * chore(jwt): Throw warning if secret is default * feature(frontend): Ability to disable registration * feature(env): Env in the root + version support ie .env.prod, .env.dev, .env.test * feature: Upgrade .env script for users * chore(config): Refactor and remove legacy env refs * feature(upgrade): Upgrade script for .env changes * feature: Install script and upgrade script * bugfix: Uncomment line to remove old .env file * chore: rename OPENAI_KEY to OPENAI_API_KEY * chore: Cleanup config changes/bugs * bugfix: Fix config and node env issues * bugfix: Config validation logic * bugfix: Handle unusual env configs gracefully * bugfix: Revert route changes and fix registration disable calling * bugfix: Fix env issues in frontend * bugfix: Fix login * bugfix: Fix frontend envs * bugfix: Fix frontend jest tests * bugfix: Fix upgrade scripts * bugfix: Allow install in non-tty envs * bugfix(windows): Use cross-env to set for windows * bugfix(env): Handle .env being incorrect to begin with for client domain * chore(merge-conflict): Update to LibreChat * chore(merge-conflict): Update to package-lock --------- Co-authored-by: Daniel D Orlando <dan@danorlando.com> * chore: comment out unused agent options * refactor playwright.yml again * try adding .npmrc * delete playwright yml * create new action * test new playwright action * Update langchain plugins docs (#461) * Update: install docs (LibreChat) (#458) * Release: rename project from ChatGPT Clone to LibreChat Release: rename project from ChatGPT Clone to LibreChat * Release: rename project from ChatGPT Clone to LibreChat Release: rename project from ChatGPT Clone to LibreChat * Release: rename project from ChatGPT Clone to LibreChat Release: rename project from ChatGPT Clone to LibreChat * Release: rename project from ChatGPT Clone to LibreChat Release: rename project from ChatGPT Clone to LibreChat * Update documentation_guidelines.md * Update introduction.md add link to readme * Update stable_diffusion.md add link back to readme * Update wolfram.md add link back to readme * Update README.md add Plugins to ToC * feat(ChatAgent.js): add support for langchainProxy configuration option Add a new configuration option `langchainProxy` to the ChatAgent class. If the option is set, the `basePath` configuration option of the `ChatOpenAI` instance is set to the base path of `langchainProxy`. * bugfix(errors): Possible workaround for error flashing (#463) * Test/user auth system client tests (#462) * test: login tests * test: finish login tests * test: initial tests for registration * test: registration specs * modify for new secrets * modify for new secrets * fix corrupted package-lock * remove env vars for testing * chore(api): update langchain dependency to version 0.0.91 * Update introduction.md * Update introduction.md * Update introduction.md * run it again * try externalizing uuid package in rollupoptions to fix build error * update authenticate script for LibreChat * add npm install playwright/test to run * try adding langchain dependency to root package.json to resolve zod error * add missing env vars * fix: no longer renders html in markdown content fix: patch XSS vulnerability completely by handling cursor on the frontend without css/html * fix(Content.jsx): fix cursor logic so it never shows for static messages * bugfix(langchain): Upgrade script, docker, env and docs (#465) * bugfix(errors): Remove incorrect manual fix from misunderstanding * chore(env): Lets not make a .env.prod and use the prod values in the default root .env - .env.dev will still be created * chore(upgrade.js): Lets tell the user about .env.dev if we create it * bugfix(env): Move to full name environments for vite - .env.prod => .env.production - .env.dev => .env.development * chore(env-example): Explain how to get google login working in production * bugfix(oauth): Minor fix to point isProduction to a correct value * bugfix: Typo in public * chore(docs): Update docs to note the changes to .env * chore(docs): Include note on how to get google auth working in dev and how to disable registration * bugfix: Fix missing env changes * bugfix: Fix up docker to work with new env / npm changes * Update .env.example Cleanup the env of the palm2 instruction and fix to formating * chore(docker): Simplify Docker deployments - Needs work to support dev env/hotreload * bugfix: Remove volume map for client dir * chore(env-example): Change instructions to be more user centric --------- Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com> * update: install docs (#466) * Add files via upload * Update apis-and-tokens.md * Update apis-and-tokens.md * Update docker_install.md * Update linux_install.md * Rename apis-and-tokens.md to apis_and_tokens.md * Update docker_install.md * Update linux_install.md * Update mac_install.md * Update linux_install.md * Update docker_install.md * Update windows_install.md * Update apis_and_tokens.md * Update mac_install.md * Update linux_install.md * Update docker_install.md * Update README.md * Update README.md : Breaking Changes --------- Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com> * Update README.md (#468) add new API/Token docs to Toc * docs: guide on how to create your own plugin * Update make_your_own.md * Update make_your_own.md * feat(docker): add build args for frontend variables in Dockerfile feat(docker-compose): add build args for frontend variables in docker-compose.yml * Update docker_install.md * Update docker_install.md * Update docker_install.md * Update docker_install.md * docs: update (#469) * Update: make_your_own.md * Update README.md add `make_your_own.md` to ToC * Update linux_install.md * Update mac_install.md * Update windows_install.md * Update apis_and_tokens.md * Update docker_install.md * Update docker_install.md * Update linux_install.md * Update mac_install.md * Update windows_install.md * Update apis_and_tokens.md * Update user_auth_system.md * Update docker_install.md clean up of repeated information * Update docker_install.md * Update docker_install.md typo * fix: fix issue with pluginstore next and prev buttons going out of bounds * fix: add icon for web browser plugin * docs(GoogleSearch.js): update description of GoogleSearchAPI class to be more descriptive of its functionality * feat(ask/handlers.js): add cursor to indicate ongoing progress of a long-running task fix(Content.jsx): handle null content in the message stream by replacing it with an empty string (with a space so a text space is rendered) * Update README.md * Update README.md * fix: plugin option stacking order * update: web browser icon (#470) * Delete web-browser.png * update: web browser icon * Update readme (#472) * Update README.md Discord badge now displays the number of online users Project description has been updated to reflect current status Feature section has been updated to reflect current capabilities Sponsors section is now located just above the contributors section Roadmap has been removed as it was outdated. * Delete roadmap.md Roadmap has been removed to streamline document maintenance. * Update README.md * Update README.md * Delete CHANGELOG.md * add frontend unit test action * action for backend unit test run * add env vars for backend tests * add step for install api deps * use test:ci script * make test:ci run from api folder * add script for installing linux x64 sharp * move jest setup from root to /api * fix: pluginstore in mobile view getting clipped and not scrolling * change scripts to test:client and test:api in root package.json * test api pipeline sharp installation * remove sharp installation * last attempt * revert * docs(linux_install.md): remove duplicate git clone command * chore(Dockerfile): comment out nginx-client build stage docs(README.md): update installation instructions and mention docker-compose changes docs(features/plugins/introduction.md): bold plugin names and add emphasis to notes * feat: add superscript and subscript support to markdown rendering refactor: support markdown citations for BingAI * refactor: support markdown citations for BingAI * chore(client): add cross-env package to dependencies refactor(client): comment out uuid from external rollup options in vite.config.ts * feat(package-lock.json): add Jest as a devDependency and remove uuidv4 dependency * feat(workflows): add jest branch to backend and frontend unit tests on push and pull_request events * chore(workflows): update Node.js version to 19.x in backend and frontend review workflows * test(PluginStoreDialog.spec.tsx): update test to reflect new pagination The test was updated to reflect the new pagination. The test now expects to see Plugin 6 and Plugin 7 on the second page instead of Plugin 3, Plugin 4, and Plugin 5. The previous expectations were commented out. * chore: update package versions to 0.5.0 chore: remove jest branch from backend-review.yml feat: add support for feat/playwright-jest-cicd branch in backend-review.yml fix(api): fix sampleTools variable in index.test.js * fix(Content.jsx): replace escaped whitespace with invisible character --------- Co-authored-by: David Shin <42793498+dncc89@users.noreply.github.com> Co-authored-by: Daniel D Orlando <dan@danorlando.com> Co-authored-by: LaraClara <2524209+ClaraLeigh@users.noreply.github.com> Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com>	2023-06-11 00:00:48 -04:00