From f81b2f42c6209aa8cb209fbd038ec5e368a943c3 Mon Sep 17 00:00:00 2001 From: Na'aman Hirschfeld Date: Wed, 1 Jul 2026 10:46:49 +0200 Subject: [PATCH] Add crawlberg, html-to-markdown, liter-llm --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 04ecbac..4ffd80f 100644 --- a/README.md +++ b/README.md @@ -192,6 +192,7 @@ _Libraries for Machine Learning. Also see [awesome-machine-learning](https://git - [feature_engine](https://github.com/feature-engine/feature_engine) - sklearn compatible API with the widest toolset for feature engineering and selection. - [h2o](https://github.com/h2oai/h2o-3) - Open Source Fast Scalable Machine Learning Platform. - [lightgbm](https://github.com/lightgbm-org/LightGBM) - A fast, distributed, high performance gradient boosting framework. +- [liter-llm](https://github.com/xberg-io/liter-llm) - Universal LLM API client for 142+ providers with a unified interface and streaming (Rust core with Python bindings). - [mindsdb](https://github.com/mindsdb/minds) - MindsDB is an open source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning models using standard queries. - [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for probabilistic graphical models and Bayesian networks. - [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for Machine Learning with extensive documentation and community support. @@ -380,6 +381,7 @@ _Libraries to automate web scraping and extract web content._ - Frameworks - [browser-use](https://github.com/browser-use/browser-use) - Make websites accessible for AI agents with easy browser automation. - [crawl4ai](https://github.com/unclecode/crawl4ai) - An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents. +- [crawlberg](https://github.com/xberg-io/crawlberg) - High-performance web crawling and scraping engine with HTML-to-Markdown conversion and a headless-Chrome fallback (Rust core with Python bindings). - [mechanicalsoup](https://github.com/MechanicalSoup/MechanicalSoup) - A Python library for automating interaction with websites. - [scrapy](https://github.com/scrapy/scrapy) - A fast high-level screen scraping and web crawling framework. - Content Extraction @@ -899,6 +901,7 @@ _Libraries for parsing and manipulating plain texts._ _Libraries for working with HTML and XML._ - [beautifulsoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) - Providing Pythonic idioms for iterating, searching, and modifying HTML or XML. +- [html-to-markdown](https://github.com/xberg-io/html-to-markdown) - Fast, CommonMark-compliant HTML to Markdown converter (Rust core with Python bindings). - [justhtml](https://github.com/EmilStenstrom/justhtml/) - A pure Python HTML5 parser that just works. - [lxml](https://github.com/lxml/lxml) - A very fast, easy-to-use and versatile library for handling HTML and XML. - [markupsafe](https://github.com/pallets/markupsafe) - Implements a XML/HTML/XHTML Markup safe string for Python.