Commit graph

7 commits

Author SHA1 Message Date
Kovid Goyal
fd22b77241
Much faster SIMD UTF-8 decoding of valid UTF-8
Decode 64 byte blocks of valid UTF-8 containing no ESC using the
approach of simdutf: blocks are validated with the lookup tables of
Keiser and Lemire and converted twelve bytes at a time using shuffles
looked up from a mask of the bytes that end code points, with fast
paths for runs of ASCII, two byte and three byte sequences. Anything
else, ESC, invalid UTF-8 and the end of the input, is decoded as
before.

The shuffle tables are generated by gen/utf8_decode_tables.py.

This decodes the text of the unicode benchmark 2.6 times faster, faster
than simdutf's own conversion with error checking, and speeds up the
unicode benchmark by ~8%.

The new AVX2 decoder is faster than the AVX-512 one, so remove the
AVX-512 decoder and use the AVX2 one on CPUs with AVX-512 as well.

Also add a test comparing the SIMD decoders with the scalar one on long
inputs, which exercise the decoding of blocks, as the existing tests use
short inputs.
2026-11-19 19:57:35 +05:30
Kovid Goyal
7f0b072f0d
autoformat 2026-08-05 05:32:56 +05:30
Kovid Goyal
573bfb688a
Move parsing of colors fully into C
Uses a perfect hash function for color name lookup
2025-12-31 14:21:45 +05:30
Kovid Goyal
2b3f2258ff
More pyugrade to 3.9 2024-08-05 11:00:51 +05:30
Kovid Goyal
4f1971c480
Rationalize mouse cursor shape handling
Now can use the full range of standard mouse cursor shapes similar to
those supported by browsers via the CSS cursor property.

Still needs to be fully implemented for cocoa backend.
2023-10-15 09:17:31 +05:30
Kovid Goyal
e6ef2fceea
py3.8 support 2023-10-14 07:57:03 +05:30
Kovid Goyal
56063b96fd
Move gen scripts into their own package 2023-10-14 07:44:18 +05:30