Font collection files contain multiple faces, which can have different
variable data, such as different named styles. Both the C font selection
code and the Python variable data cache were keyed on the file path alone,
so once the data for one face of a collection was cached, other faces in
the same file got that data. For example, selecting a named style that
only exists in the second face of a collection failed if the first face
was looked at first, falling back to a different family.
Key the caches on the face index for fontconfig and the postscript name
for CoreText, which does not expose the face index. Do not share cache
entries for fonts without a path in the C code, matching the Python code.
Expose the face index on the FreeType Face object for the Python cache.
- Convert OpenType axis tags to Python as four Latin-1 characters instead
of UTF-8, so that fonts with axis tags that are not valid UTF-8 no longer
cause an exception. On macOS this made font selection fail entirely if
the selected variable font had such a tag. Convert axis_map keys back to
tags losslessly, validating them, instead of reading four bytes from the
UTF-8 encoding, which could read past the end of short keys.
- variable_data_from_font_file() now raises ValueError for files that are
not valid fonts, instead of an OSError with a stale or zero errno.
- Avoid quadratic work when reading a crafted font with many axes, named
instances and name records: de-duplicate axis tags once rather than per
instance, and binary search name records by name id. A 64MB crafted font
that took over a minute to parse now takes about a second.
HarfBuzz returns the glyphs of a backward (RTL) buffer in visual order,
with decreasing clusters, but cells were consumed from the logical front
of the run, so pointed Hebrew and Arabic lam-alef ligatures were grouped
with the wrong cells.
Replace the per codepoint cell cursor with a table, built while loading
the HarfBuzz buffer, mapping codepoint offsets to cells. The clusters
grouped so far always cover a prefix (LTR) or a suffix (RTL) of the text,
so the cells they complete come straight from the table, with no
direction specific code. The codepoint used to detect special glyphs is
now that of the glyph's own cluster.
Add DejaVu Sans Mono to the test fonts for Arabic shaping tests.
Closes#10891
Lines scrolled onto the screen no longer have their render data cleared,
instead they are marked dirty so that rendering recomputes it, except when
there is a marker, as marks are render data used without rendering. This
reduces the memory written to clear a line by 40%.
The ASCII fast path in draw_text no longer checks for multicell cells in
a line that was just cleared by a line feed scrolling the screen, until
something that can create multicell cells happens.
Together these make the ascii benchmark ~14% and the scrollback and
render benchmarks ~8% and ~14% faster. Also update the throughput
numbers in the performance docs, using the median of several runs.
Loading the builtin NERD font, which includes a fontconfig match for its
hinting settings, was done on the main thread when the results of font
selection were first used. Do the slow parts of it in the font selection
thread instead. On Linux the PostScript name is read from the name table
of the font file and the fontconfig match is done in the thread, and the
FreeType face, which is only used for fallback, is created when first
needed. On macOS the font descriptor is created in the thread.
This saves ~0.6ms on the main thread during startup on Linux.
Font selection was done in Python on the main thread, taking ~15ms at
startup on Linux and much more on macOS, where all installed fonts were
listed from CoreText and converted to Python, taking ~165ms.
Port the selection algorithm to pure C in font-select.c, with platform
backends in fontconfig.c and core_text.m, so that it can run in a thread
started by set_font_family(). The results are used when the first font
group is created, typically when the first OS Window is created. The
data for variable fonts is read from the font tables by C code that does
not use Python or FreeType.
The fontconfig backend lists all monospace fonts once and filters them
in C, as listing fonts is dominated by scanning all fonts. The CoreText
backend queries only the needed font families and PostScript names
instead of all installed fonts.
On Linux, the selected fonts are identical to those selected by the
previous Python implementation for ~1400 font specifications, including
variable fonts. The CoreText backend has not yet been compiled or tested.
The HarfBuzz font was created before the named instance or axis
coordinates were applied to the FreeType face and was only resynced if
the face had a transform matrix. So shaping used the default variation
coordinates, which broke fonts that substitute glyphs based on axis
values via GSUB FeatureVariations, such as the cursive (CRSV) axis of
Annotation Mono.
Adds a test using a small subset of Annotation Mono (SIL OFL 1.1).
Fixes#10897
Reduce the allowance for cropping glyphs wider than their cell, rather
than rescaling them, to 2/10 of the cell width, since cropping removes
ink, and test that cropped glyphs fill their cell.
FreeType: Move combining marks with their base glyph when it is rescaled
and moved into its cell, so that they stay in place relative to it.
CoreText: Rescale glyphs individually, as with FreeType, so that a
combining mark wider than the cell does not cause its base glyph to be
shrunk. Measure the width of glyphs in whole pixels, as rasterized, as
with FreeType, so that both renderers make the same decision. Never crop
colored glyphs, as with FreeType.
concat_cells: Produce RGBA bytes for 8 bit sprites, as for 32 bit ones,
and handle empty cells.
As with the CoreText renderer, a glyph that is rescaled to fit its cell is
moved so that its ink starts at the left edge of the cell, so that marks that
hang to the left of it, as in polytonic Greek capitals, are no longer cut off.
Port the overhang allowance from the FreeType renderer: a glyph wider than
its single cell by at most 2/9 of the cell width, with a minimum of 2px, is
cropped rather than rescaled. Previously any glyph whose right edge was more
than 1px past the cell was rescaled, shrinking glyphs such as the d with
caron of Cascadia Code and the texture healing alternates of Monaspace.
Widths are measured per glyph, as with FreeType, so that combining marks do
not cause their base glyph to be shrunk.
The left overhang of a glyph is now taken into account, and a glyph that is
rescaled is moved so that its ink starts at the left edge of the cell, so
that marks that hang to the left of it, as in polytonic Greek capitals, are
no longer cut off. Glyphs that extend past the right edge of the cell are
moved left as far as their left bearing allows, as with FreeType.
Also fix the tests for this to run on macOS: face_from_path() ignored the
font size on macOS and the tests treated pixels with only color bits set as
ink.
Fix concat_cells() for 32 bit sprites, broken since the SIMD compositing
change, which treated the 0xRRGGBBAA sprite pixels as RGBA bytes, producing
an opaque white image.
A glyph whose bitmap is wider than its cell is rendered again at a
smaller font size so that it fits, unless it is wider by no more than
two pixels, in which case it is cropped. That allowance is absolute, so
it covers less of the cell as the cell grows, and fonts with glyphs
that overhang their advance by design end up on the wrong side of it:
the texture healing alternates of Monaspace and 0xProto, or the d with
caron of Cascadia Code. Such a glyph is drawn visibly smaller than its
neighbors, at one font size and not at the next.
Make the allowance proportional to the cell width: 2/9 of the cell,
which is the 2px the rule was originally tuned for at the 9px cell of a
typical monospace font at 11pt and 96 dpi, with a minimum of 2px for
smaller cells. That keeps the widest Monaspace alternate, 14% wider
than its cell, at full size. A glyph that overhangs by more, such as a
Greek capital whose breathing mark and accent hang well outside the
cell, is still rescaled, since cropping would remove the marks.
Closes#10879
Lines whose thickness is specified in pts are now scaled by the ratio of
the current font size to the configured font_size, so they no longer look
too thin when zoomed in or too thick when zoomed out. At the configured
font size nothing changes. This combines with the existing multicell
scale for the text sizing protocol. Sprites are already cached per font
size, so no cache changes are needed.
For the window title bar, the drawn border is kept at least half a pixel
inside the pixels reserved for it, since the rounded rect shader
anti-aliases over a one pixel band on either side of each edge and would
otherwise bleed onto the rendered title.
Fixes#10362
The glyph for a private use codepoint depends on the font, not the style.
Looking for a bold or italic font covering it tends to find an unrelated
text font that happens to cover it, for example Liberation Serif Italic for
the Nerd Font check mark U+f00c, which rendered as a thin line in the bold
italic title of the active tab.
Restrict the style-ignoring fallback lookup to the Private Use Area ranges
of the Nerd Fonts glyph sets, so that other private use codepoints, which
may be used by fonts that do have styled variants, keep their style.
- Back off on all accept() errors that leave the connection pending, not
just resource exhaustion, so the talk thread cannot spin on a readable
listener
- Use an atomic for the test only accept failure injection instead of
taking the talk mutex on every accept
- Encrypt disk cache entries in chunks of up to 1MB to reduce the number
of write syscalls for large images
- Have GlobalWatchers.set_extra() invalidate the cached watchers
fill_key() sizes its scratch buffer assuming every cell holds at most
MAX_NUM_CODEPOINTS_PER_CELL codepoints. The draw paths in screen.c
enforce that, but Line.add_combining_char() does not, so a line built via
the Python API with more codepoints in a cell overflows the key buffer
once the run is long enough. Bail out of caching such runs instead.
The coverage threshold (0.5 in fbd4e5ad1, lowered to 0.3 in d4106ef2d to
get CI green) is environment-dependent: a correctly sized emoji covers
~0.84 of its cell on one box but ~0.39 in CI, while the buggy render is
~0.28, so the margin is thin and font/cell dependent.
Render the same font two ways at one size instead: via its fontconfig
descriptor (which carries the size-fixup matrix) and via its file path
(which does not). They must come out the same size; the bug shrinks the
descriptor one. Only the matrix differs, so the check no longer depends
on the environment or the emoji artwork.
ee937bdd1b routed FC_MATRIX through the cairo font matrix so synthetic
slant reaches color glyphs. But FC_MATRIX is also how fontconfig encodes
the pixel-size fixup of fixed-size faces. Noto Color Emoji is a ~109px
bitmap strike and fontconfig hands consumers a matrix scaling it to the
requested size (factor = requested_px / strike_px). cairo_set_font_size()
already brings the strike to the requested size, so feeding that matrix
into cairo_set_font_matrix() in apply_cairo_font_size() scales it down
again by the fixup factor. At terminal cell sizes that is a large shrink,
up to ~9x for small cells (easing to 1x as the cell nears the strike), so
color emoji render as a dot; fit_cairo_glyph() only shrinks, so it never
grows them back.
Only the shear carries synthetic slant; the diagonal is the size, which
cairo_set_font_size() and fit_cairo_glyph() already handle. Apply only
the shear and fall through to cairo_set_font_size() when there is no
shear. Pure-slant matrices are unchanged. This carries only the shear to
color glyphs, so a non-uniform diagonal scale from a hand-built FC_MATRIX
is dropped on that path; the stock fixup is uniform, so dropping it is
the intended behaviour.
Add a regression test that renders a color emoji and checks it fills its
cells, skipped unless a fixed-size color font with a fontconfig fixup
matrix is present.
Fixes#10144
fontconfig's FcFontList omits FC_MATRIX from its object set
(kitty/fontconfig.c), so a roman font that find_best_match finds there
(e.g. Fira Code, which ships no italic, in both its static and variable
builds) carries no synthetic-italic shear and its "italic" renders upright.
A family that is not found is substituted, and when the substitute
resolves through the listed faces those descriptors are equally
matrix-less, so this attach covers them too. Only raw fc_match
descriptors (runtime glyph-fallback faces via create_fallback_face, and
find_best_match's last-resort return) already carry the matrix from
substitution.
The italic intent for the configured faces exists only during selection,
not at face construction, so attach the matrix at the end of
get_font_files: for an italic slot whose chosen face is upright and has no
matrix, ask fc_match what fontconfig would do. fc_match returns a synthetic
matrix only when there is no real italic to use (no italic face and no
slanted named instance or variable slant axis), so a font that is already
italic, static or variable, is never double-slanted. Face construction
applies the matrix via FT_Set_Transform; the previous commit makes it
survive the size specialization step the render path builds faces from.
Only the matrix is taken, so selection is unchanged.
FontConfigPattern declared matrix as a required key, but pattern_as_dict
sets it only when the pattern has one, so declare it NotRequired. With
that and narrowing on descriptor_type the attach needs no cast.
Add a regression test (test_synthetic_italic_matrix): a roman no-italic
font gets a non-identity matrix on its italic slot while a real-italic
control does not, and the matrix survives specialize_font_descriptor. It
asserts the invariant rather than the exact shear (the value is
fontconfig's, version-dependent) and skips when the synthetic rule is
inactive.
Covers the four configured faces. Limitation: fc_match re-matches by family
name, so under an uncommon config (a multi-face family key plus a user
per-font FC_MATRIX rule keyed on width/style) it can attach a matrix
computed for a different face; the 90-synthetic shear this targets is
weight-independent and unaffected. A production version should re-match the
selected face by path+index+slant.
Store in a separate VAO instead more performant as it uses less VRAM and
allows shader lookup co-ords to be passed to fragment shaders without
calculation in the fragment shader.
Will allow sprites to point to where their decorations should be read
from, for instance. Needed for scaled text and also if we want to
implement decoration avoidance.
Frees up two bytes in GPUCell. Doesn't require a minimum texture row
size. Makes a bunch of code faster. Index uses 31 bits which gives us
2,147,483,647 aka ~ 2 billion sprites.