Commit graph

10 commits

Author SHA1 Message Date
Danny Avila
2414e9c7d2
🛟 refactor: Gracefully Skip Unavailable Web Search Rerankers (#13191)
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
GitNexus Index / index (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
GitNexus Index / post-index (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
2026-05-19 09:48:12 -04:00
Danny Avila
f5dd053128
🛡️ refactor: Restrict User Tavily Endpoint URLs (#12946) 2026-05-05 05:09:11 +09:00
Danny Avila
2c4a78094a
🛂 refactor: Avoid Default Tavily Safe Search (#12939) 2026-05-04 12:01:23 +09:00
Yashwanth Alapati
3da1d8c961
🔍 feat: add Tavily as Search and Scraper Provider (#12581)
* feat: add Tavily integration as search provider and scraper provider

* chore:update tavily web search parameters

* chore:tavily paramer update

* chore:update data-schemas test for tavily

* fix: allow Tavily string option modes

* fix: align Tavily config options

* fix: scope Tavily scraper timeout

* fix: use resolved scraper provider timeout

* fix: widen Tavily search provider types

* fix: harden Tavily web search config

* fix: cap Tavily option timeouts

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-05-04 11:29:13 +09:00
Daniel Lew
2503365c44
🚫 feat: Add Support for none Reranker Type in Web Search Config (#12765)
Most of the codebase already supports the concept of *not* using a reranker w/
web search, except there was no way to initially setup an absent reranker
component.

Now there's a special path for skipping the reranker auth when loading web
search config, which allows for skipping the reranker when using web search.
2026-04-29 09:17:04 +09:00
Danny Avila
1312cd757c
🛡️ fix: Validate User-provided URLs for Web Search (#12247)
* 🛡️ fix: SSRF-validate user-provided URLs in web search auth

User-controlled URL fields (jinaApiUrl, firecrawlApiUrl, searxngInstanceUrl)
flow from plugin auth into outbound HTTP requests without validation.
Reuse existing isSSRFTarget/resolveHostnameSSRF to block private/internal
targets while preserving admin-configured (env var) internal URLs.

* 🛡️ fix: Harden web search SSRF validation

- Reject non-HTTP(S) schemes (file://, ftp://, etc.) in isSSRFUrl
- Conditional write: only assign to authResult after SSRF check passes
- Move isUserProvided tracking after SSRF gate to avoid false positives
- Add authenticated assertions for optional-field SSRF blocks in tests
- Add file:// scheme rejection test
- Wrap process.env mutation in try/finally guard
- Add JSDoc + sync-obligation comment on WEB_SEARCH_URL_KEYS

* 🛡️ fix: Correct auth-type reporting for SSRF-stripped optional URLs

SSRF-stripped optional URL fields no longer pollute isUserProvided.
Track whether the field actually contributed to authResult before
crediting it as user-provided, so categories report SYSTEM_DEFINED
when all surviving values match env vars.
2026-03-15 18:05:08 -04:00
Danny Avila
31a283a4fe
🔍 feat: Add Serper as Scraper Provider and Firecrawl Version Support (#9984)
* 🔧 chore: Update @librechat/agents to v2.4.84 in package.json and package-lock.json

* feat: Serper as new scraperProvider for Web Search and add firecrawlVersion support

* fix: TWebSearchKeys and ensure unique API keys extraction

* chore: Add build:packages script to streamline package builds
2025-10-05 20:34:05 -04:00
Danny Avila
838fb53208
🔃 refactor: Decouple Effects from AppService, move to data-schemas (#9974)
* chore: linting for `loadCustomConfig`

* refactor: decouple CDN init and variable/health checks from AppService

* refactor: move AppService to packages/data-schemas

* chore: update AppConfig import path to use data-schemas

* chore: update JsonSchemaType import path to use data-schemas

* refactor: update UserController to import webSearchKeys and redefine FunctionTool typedef

* chore: remove AppService.js

* refactor: update AppConfig interface to use Partial<TCustomConfig> and make paths and fileStrategies optional

* refactor: update checkConfig function to accept Partial<TCustomConfig>

* chore: fix types

* refactor: move handleRateLimits to startup checks as is an effect

* test: remove outdated rate limit tests from AppService.spec and add new handleRateLimits tests in checks.spec
2025-10-05 06:37:57 -04:00
Daniel Andersen
6f6a34d126
🔗 feat: Custom Jina API URL for Web Search Reranking (#9236)
* feat: added support for custom JINA_API_URL

* fixed tests

* chore: Update @librechat/agents dependency to version 2.4.77 in package-lock.json and package.json files

* fix: Update Jina API URL to use environment variable in configuration files

* Refactor AppService, web.ts, and config.ts to replace hardcoded Jina API URL with an environment variable placeholder.
* Ensure consistency across tests and configuration for Jina API URL.

* chore: alphabetical order translation.json

* fix: alphabetical order

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2025-09-06 08:39:20 -04:00
Dustin Healy
d0c958ba33
🔥 feat: Add Firecrawl Scraper Configurability (#8495)
- Added firecrawlOptions configuration field to librechat.yaml
- Refactored web.ts to live in packages/api rather than data-provider
- Updated imports from web.ts to reflect new location
- Added firecrawlOptions to FirecrawlConfig interface
- Added firecrawlOptions to authResult of loadWebSearchAuth so it gets properly passed to agents to be built into firecrawl payload
- Added tests for firecrawlOptions to web.spec.ts
2025-07-18 22:37:57 -04:00