neuroplast.ioHuman-machine interface research

Blitz

github.com/DioxusLabs/blitz · MIT OR Apache-2.0

For hotty-blitz

32 patches on 674d7d2. hotty-blitz is HOTTY's renderer: it lays out and paints a terminal program's HTML surfaces with Blitz, behind the C ABI that hottyterm links and in the polyfill for other terminals. A HOTTY program changes its documents by small deltas many times a second, and expects of them what a browser does: layout that costs the change rather than the document, form controls in the page's colour scheme, srcset, password inputs and selects, a user stylesheet, and SVG that reads no files. We mod Blitz rather than fork it so that each of those stays one change on a pinned commit, written as a browser behaviour Blitz could take as it is.

The patches, each with its upstream fit:

  1. A delta costs the change, not the document — maybe
  2. Inline SVG follows its preserveAspectRatio — yes
  3. A control in the disabled state acts disabled — yes
  4. Outlines take outline-offset and every border style — yes
  5. A box whose layout changed gets its overflow and transform again — yes
  6. An SVG's image loads a data: URL and no file — yes
  7. A request says what it is for: image, style, font, iframe or document — yes
  8. An img shows the source that srcset, sizes and picture choose — yes
  9. An inline SVG's image draws what the document fetches for it — yes
  10. An img chooses its source again when the device changes — yes
  11. Form controls follow the used color scheme — yes
  12. Text controls are as wide as size and cols, and as tall as rows — yes
  13. A scroll container's client rect doesn't move with its own scroll — yes
  14. A document can lock scrolling along an axis — maybe
  15. A scroll container is hit only within its box — yes
  16. Stylesheets of the user origin — yes
  17. :focus-visible matches, and a click shows the ring only on a text field — yes
  18. Selected text takes the colours of its ::selection — yes
  19. Backspace edits a text input on macOS too — maybe
  20. An embedder can hide the caret, to blink it — maybe
  21. A password input shows a mask and edits its value — yes
  22. An embedder can edit a text input: its value, selection and rows — maybe
  23. A select has a selected option, submits it, and draws closed — yes
  24. A table emptied of its rows forgets them — yes
  25. A box gone display: none forgets the boxes built under it — yes
  26. Speculative relayout leaves nodes out of the layout tree alone — maybe
  27. A mutation's queued work skips the nodes it freed — yes
  28. An inline svg is built again when its source changes — maybe
  29. An inline svg's currentColor is written as rgb() — yes
  30. A row move can extend the selection — yes
  31. A double click selects a word, and a triple click its block — yes
  32. A double click in a password input selects all of it — yes

A delta costs the change, not the document

Upstream fit: maybe · patch 0001

What it changes: one changed node cost every phase of resolve a walk of the whole document. Each phase now stops where the change stops. Construction skips elements with no damage below them. Each node's layout cache grows past taffy's 9 entries, up to 64, since a content-sized container k levels deep is measured under about 3 + 5k constraints a pass and the entries evicted each other. A flex or grid container keeps its cache when every changed item, asked again what the container asked it last time, answers the same (layout/speculate.rs). Rounding visits only the nodes whose layout changed and the subtrees that moved, and everything when more than a quarter changed. Debug builds compare every incremental rounding with taffy::round_layout, and Blitz's 299 tests pass.

Why we carry it: HOTTY programs send deltas many times a second; a one-cell change to a large dashboard cost the whole document.

For upstream: maybe. The cache and the rounding are generic, but the speculation is a large change to how layout settles, and upstream's own damage tracking has moved since our base, so it would need rework there.

Inline SVG follows its preserveAspectRatio

Upstream fit: yes · patch 0002

What it changes: an inline <svg> was always painted as object-fit: contain. A browser fits the viewBox to the element's box as preserveAspectRatio says (SVG 2, 8.6): none stretches it, slice covers the box, and the default fits inside. 14 lines in blitz-paint.

Why we carry it: charts drawn in a fixed viewBox and stretched to their box came out narrow and centred.

For upstream: yes. It is what browsers do, and small.

A control in the disabled state acts disabled

Upstream fit: yes · patch 0003

What it changes: pointer handling and form-control painting asked for the disabled attribute. Blitz already keeps a control's disabled state in its ElementState, which :disabled matches; they now ask ElementData::is_disabled, the attribute or the state. Nothing changes for a document whose state comes from its attributes. An embedder that disables a control with Node::disable, leaving the attributes alone, gets what the attribute gives: no focus, no toggle, no submit, no selection, and a grey checkbox.

Why we carry it: a detached HOTTY surface's controls act disabled (SPEC §5.5) while the document keeps the program's attributes.

For upstream: yes. It is small, and makes the state and the attribute agree everywhere.

Outlines take outline-offset and every border style

Upstream fit: yes · patch 0004

What it changes: an outline was drawn solid, just outside the border box: outline-offset was ignored, and dashed, dotted, double, groove, ridge, inset and outset outlines came out solid. The outline is now the border of a box of its own, offset from the border box (inside it when the offset is negative), drawn with the border's edge code, so it takes every border style. It is painted after the element's content, as CSS 2.1 Appendix E does, so the element's background doesn't cover one inside it. Dotted outlines have round dots, as Blitz's dotted borders do.

Why we carry it: a surface's root fills the surface, so an outline on it has to come inside with a negative offset, and was drawn off the surface.

For upstream: yes. It is CSS as browsers draw it.

A box whose layout changed gets its overflow and transform again

Upstream fit: yes · patch 0005

What it changes: Blitz computed a node's transform and scrollable overflow again only when its own style damage asked for it. Damage flows up the tree, so a box resized only because an ancestor changed kept its old overflow, and paint, which culls a box by its overflow, could drop it on every later paint. Every node whose final layout changed is now recorded (set_final_layout), with the boxes that reach it in resolve_transforms, and those are computed again too.

Why we carry it: a delta to a custom property that set a ring's height and translateY left its stretched child culled off the surface.

For upstream: yes. It is a bug in incremental updates that any embedder changing a document can hit.

An SVG's image loads a data: URL and no file

Upstream fit: yes · patch 0006

What it changes: Blitz parses every SVG (an <img>, a CSS image and an inline <svg> alike) with usvg's default options, whose image resolver reads any <image href> that isn't a data: URL as a path on the disk. A page could draw any image the user's files hold, without asking the NetProvider. The resolver now loads data: URLs, and (with a later patch) what the document fetched for an inline SVG's <image>, and nothing else. A test in blitz-dom checks it.

Why we carry it: HOTTY documents come from programs, and a surface must never read the user's files (SPEC §12).

For upstream: yes. It closes a file read that browsers never allow.

A request says what it is for: image, style, font, iframe or document

Upstream fit: yes · patch 0007

What it changes: a NetProvider got a URL and nothing about what it was for. Request now has fetch's destination, and every request a document makes sets it: Image for <img> and CSS images, Style for stylesheets, their reload and @import, Font for @font-face, Iframe for an iframe's document, and Document for a navigation. Request::get leaves it Empty. The struct was non_exhaustive already.

Why we carry it: HOTTY applies a fetch policy per kind of resource, as CSP's img-src, style-src and font-src do (SPEC §7.2).

For upstream: yes. It is fetch's own field, and any embedder with a content policy needs it.

An img shows the source that srcset, sizes and picture choose

Upstream fit: yes · patch 0008

What it changes: Blitz loaded an <img>'s src and nothing else. BaseDocument::image_source now selects a source as HTML does: the first <source> of its <picture> whose type this build decodes and whose media matches the device, else its own srcset, else its src. Of a srcset, it takes the candidate with the smallest density at or above the device's, else the densest; a w candidate's density is its width over the size sizes gives it, with Stylo's SourceSizeList. Changing the attributes loads the image again.

Why we carry it: HOTTY surfaces are drawn at the terminal's scale and theme, and programs send images for each.

For upstream: yes. It is HTML's image source selection.

An inline SVG's image draws what the document fetches for it

Upstream fit: yes · patch 0009

What it changes: usvg resolves an <image>'s href synchronously while it parses, and Blitz parses an inline <svg> while it constructs boxes, so nothing could be fetched for it. An SVG <image> is now fetched as an <img> is, through the NetProvider as an Image request, when it joins the document or its href changes. The bytes are kept by URL; when they arrive, the outermost <svg> is built again, and the resolver hands usvg those bytes. An SVG that is itself an image still loads only data:, as in a browser.

Why we carry it: cid: resources in an SVG href (HOTTY SPEC §7.1), and img-src reaching images in SVG (§7.2).

For upstream: yes. Inline SVG images then load as in a browser.

An img chooses its source again when the device changes

Upstream fit: yes · patch 0010

What it changes: when the viewport's size, scale or colour scheme, or the media type, changes, every <img> whose source the device chose chooses again, and loads what it chose if that differs. The old image shows until the new one arrives, and one chosen for an old device that arrives late isn't shown.

Why we carry it: a terminal's scale and theme change under a live document, as a browser window's do.

For upstream: yes, with "An img shows the source that srcset, sizes and picture choose".

Form controls follow the used color scheme

Upstream fit: yes · patch 0011

What it changes: the UA stylesheet made fields white and buttons light grey, and checkboxes and radio buttons were painted white, so a page with color-scheme: dark showed light controls. Fields now take Field and FieldText, and buttons ButtonFace and ButtonText, system colours that Stylo already resolves in the element's used colour scheme, with Chromium's borders in each scheme (light-dark()). Checkboxes and radio buttons are drawn in the used scheme (BaseDocument::used_color_scheme), with Chromium's colours for accent-color: auto and its 13px geometry. A document's own colours still win.

Why we carry it: surfaces on a dark terminal are dark (HOTTY SPEC §8).

For upstream: yes. Browsers draw controls this way; the shades are Stylo's system colours, a little off Chromium's.

Text controls are as wide as size and cols, and as tall as rows

Upstream fit: yes · patch 0012

What it changes: a text input was 300px wide (or the available width) whatever its size, and a textarea 300px unless it had cols. Controls now count characters as Chromium does: one character is the font's average width (OS/2 xAvgCharWidth), or the advance of 0; an input is size characters wide (20 by default) plus the widest glyph less one character; a textarea is cols characters (20) with no scrollbar gutter, since Blitz's scrollbars are overlays; a row is the line height, the font's own for line-height: normal. For Noto Sans and Noto Sans Mono at 16px, every width matches Chromium's to the pixel.

Why we carry it: forms in surfaces are laid out by their attributes.

For upstream: yes. It is HTML's sizing.

A scroll container's client rect doesn't move with its own scroll

Upstream fit: yes · patch 0013

What it changes: get_client_bounding_rect is the border box, which a scroll container's own scroll offset doesn't move, but the position walk subtracted that offset, so a scrolled box reported itself moved. scroll_into_view already made up for it.

Why we carry it: HOTTY's area and bringing a focused element into view went wrong inside a scrolled box.

For upstream: yes. It is a bug fix.

A document can lock scrolling along an axis

Upstream fit: maybe · patch 0014

What it changes: BaseDocument::set_scroll_axes locks an axis: every scroll target's range along it is empty, whoever scrolls (the user, a fragment link, a fling, a thumb), locking puts the offsets back to zero, and no scrollbar is drawn or hit there. A text input still scrolls its own text to follow the caret.

Why we carry it: HOTTY scrolls a document only along the axes it asks for (SPEC §5.1, §5.3).

For upstream: maybe. It is an embedder API shaped by HOTTY's model; another embedder might want it, but nothing in the web platform asks for it.

A scroll container is hit only within its box

Upstream fit: yes · patch 0015

What it changes: hit testing added a node's scroll offset to the point before testing it against the node's box, so a box scrolled down took points above itself and handed them to the children it had scrolled out of view. The point is now tested against the box first, the scroll offset moves only the content, and content a box clips isn't hit outside its padding box, where it isn't drawn.

Why we carry it: a press above a scrolled list landed on the items it had scrolled away.

For upstream: yes. It is a bug fix.

Stylesheets of the user origin

Upstream fit: yes · patch 0016

What it changes: DocumentConfig::user_stylesheets, and BaseDocument::add_user_stylesheet and remove_user_stylesheet. A user stylesheet wins over the user agent's and loses to the document's own, whatever the selectors' specificity, as the CSS cascade orders its origins.

Why we carry it: HOTTY's host stylesheet (SPEC §8) dresses controls, focus, links and selection in the terminal's colours with zero-specificity :where() rules, which must beat the UA sheet and lose to the document. As a second UA sheet it lost to input { background-color: Field }.

For upstream: yes. User stylesheets are part of CSS, and any embedder with a theme can use them.

:focus-visible matches, and a click shows the ring only on a text field

Upstream fit: yes · patch 0017

What it changes: :focus-visible matched nothing. It now matches the focus ring (ElementState::FOCUSRING), which focus from the keyboard or a script sets. Focus moved by the pointer (set_focus_by_pointer, which Blitz's own click handling now calls) keeps the ring only on a text field, as browsers do.

Why we carry it: HOTTY's host stylesheet draws focus rings with :focus-visible (SPEC §8).

For upstream: yes. It is the selector as browsers match it.

Selected text takes the colours of its ::selection

Upstream fit: yes · patch 0018

What it changes: a selection was a fixed light blue under text in its own colours. The highlight is now the inline root's, or the text field's, ::selection background, or that blue when it sets none, and the selected text is drawn again over it, clipped to it, in each run's ::selection colour.

Why we carry it: HOTTY's host stylesheet colours selections with the terminal's palette (SPEC §8).

For upstream: yes. It is ::selection as CSS has it.

Backspace edits a text input on macOS too

Upstream fit: maybe · patch 0019

What it changes: the text input's Backspace arm was compiled out on macOS, where Cocoa's text system turns the key into a deleteBackward: keybinding that does the deleting. The arm now applies on every platform. It is the only key the text input left to Cocoa: Delete, the arrows, Home and End take the same path everywhere.

Why we carry it: a terminal has no Cocoa text system and hotty-blitz sends no keybindings, so on macOS Backspace did nothing in a field.

For upstream: maybe. An embedder without Cocoa needs it, but upstream's own macOS shell would then delete twice unless it is made conditional on the embedder.

Upstream fit: maybe · patch 0020

What it changes: a focused text input's caret was painted solid. BaseDocument::set_caret_shown turns it off and on, and the painter honours it.

Why we carry it: hotty-blitz blinks the caret on the frame clock that plays animated images, repainting only the input, as a browser blinks it.

For upstream: maybe. Blinking is wanted, but upstream may prefer to blink the caret itself rather than leave it to the embedder.

A password input shows a mask and edits its value

Upstream fit: yes · patch 0021

What it changes: an <input type=password> was edited as a text field and showed the password. Its value is now kept beside the editor (TextInputData::password), and the editor holds a bullet, U+2022 as in Chrome, for each character, so the caret, selection and hit testing work on what is shown. Each edit of the mask is made to the value too. The input event, form submission and TextInputData::value carry the value, and the value attribute sets it. As in a browser, the value isn't copied or cut, the field composes nothing, and changing type to or from password masks or shows the value, which is how a page makes a reveal toggle.

Why we carry it: HOTTY forms ask for passwords.

For upstream: yes. It is what browsers do, with tests in blitz-dom.

An embedder can edit a text input: its value, selection and rows

Upstream fit: maybe · patch 0022

What it changes: on BaseDocument: text_input_selection reads a text input's value and selection; set_text_input sets both (a password field's value too, its mask following) and redraws; move_text_input_rows moves the caret by laid-out rows from Parley's selection; and text_input_rows_shown counts the rows the field shows, for a page.

Why we carry it: a text field's keys are its program's keymap (HOTTY SPEC §10.2), so hotty-blitz does the editing itself rather than send keys Blitz's editor would read its own way.

For upstream: maybe. An embedder with its own keymap needs this, but its shape follows hotty-blitz's.

A select has a selected option, submits it, and draws closed

Upstream fit: yes · patch 0023

What it changes: every <option> was hidden and nothing kept selectedness, so a select drew as an empty box and never reached form data. An option's selectedness is now its ElementState::CHECKED bit, as in Gecko, so option:checked matches it, and the mutator settles it with HTML's selectedness algorithm as options come, go, or have selected set or removed. Form data has the selected options. A select without multiple draws closed, as a browser draws it: the selected option and an arrow, as wide as its widest option. BaseDocument::pick_option picks. Its list isn't drawn, nor is a multiple select or one with size above 1.

Why we carry it: HOTTY lets a host show no list, and hotty-blitz picks with the keys (SPEC §10.2).

For upstream: yes. Selectedness and the closed select are HTML's, and a list could come later.

A table emptied of its rows forgets them

Upstream fit: yes · patch 0024

What it changes: construction returned before the table branch for a node with no children, so a table emptied of its rows kept its table context, and paint read the freed rows from it ("invalid SlotMap key used"). The context now goes with the inline layout, first, and is built again while the node is a table with children.

Why we carry it: a delta that emptied a table panicked, and in hottyterm that turned HOTTY off for the terminal. hotty-blitz's fuzz test found it.

For upstream: yes. It is a crash fix.

A box gone display: none forgets the boxes built under it

Upstream fit: yes · patch 0025

What it changes: a box that goes display: none isn't constructed again, and its unstyled subtree reports no damage, not even a removal. It kept its layout children, anonymous blocks, inline layout, hoisted out-of-flow boxes and paint tree, as did the boxes under it, and Taffy and paint still visited it. Damage propagation now has a hidden box forget all of it, for its former subtree too; shown again, the subtree is built afresh. Blitz's own assert_layout_parents_consistent, which failed after such a delta, holds again.

Why we carry it: a delta under a box that went display: none panicked on freed nodes. hotty-blitz's fuzz test found it.

For upstream: yes. It is a crash fix.

Speculative relayout leaves nodes out of the layout tree alone

Upstream fit: maybe · patch 0026

What it changes: the speculative relayout of "A delta costs the change, not the document" asked nodes that had just left the layout tree about their layout parent, and laid out their children, both possibly freed. It now passes over a node unless each layout parent up to the root element lists it; its old container lays out again, as for any unsettled child.

Why we carry it: a crash on freed nodes, found by hotty-blitz's fuzz test.

For upstream: maybe, as part of "A delta costs the change, not the document": it fixes that patch, and goes wherever it goes.

A mutation's queued work skips the nodes it freed

Upstream fit: yes · patch 0027

What it changes: the mutator queues work for its flush (a form control's form owner, a style element's sheet, a select's selectedness), and a control inserted and freed by the same mutation stayed queued, so the flush touched a freed node. The flush now skips nodes no longer there.

Why we carry it: a delta that inserted a form control and removed it again panicked. hotty-blitz's fuzz test found it.

For upstream: yes. It is a crash fix.

An inline svg is built again when its source changes

Upstream fit: maybe · patch 0028

What it changes: an inline <svg> is drawn from one usvg parse of its markup, made while boxes are constructed, with currentColor resolved to each element's colour then. A colour change later is only style damage, so an icon drawn with fill="currentColor" kept its first colour through a parent's class, a theme, :hover and :focus. A hash of the source each inline svg was built from is now kept, and when an svg or anything under it is damaged and its source differs, that svg alone is constructed again.

Why we carry it: the HOTTY UI kit draws its icons in currentColor.

For upstream: maybe. The bug is real, but Blitz has since rebuilt inline SVG when its styles change in its own way, after our base, which may make this patch unnecessary once the base moves.

An inline svg's currentColor is written as rgb()

Upstream fit: yes · patch 0029

What it changes: outer_html wrote an inline svg's currentColor with to_css_string, so a colour that computes to anything but legacy sRGB (color-mix(), oklab, display-p3) came out as color(srgb …), which usvg doesn't parse, and the icon drew black. It is now written as rgb() or rgba() (into_srgb_legacy).

Why we carry it: every icon in the HOTTY UI kit's muted text, a color-mix(), drew black.

For upstream: yes. It is small, and any page with modern colours hits it.

A row move can extend the selection

Upstream fit: yes · patch 0030

What it changes: TextInputData::move_rows, and move_text_input_rows, take extend: with it, the selection's anchor stays and the caret moves, as Shift+ArrowDown does in a browser's textarea. Without it, a selection collapses first, as before. Parley's move_lines already extends; this passes the flag through.

Why we carry it: HOTTY text fields select by rows with the keymap.

For upstream: yes. It is small and generic.

A double click selects a word, and a triple click its block

Upstream fit: yes · patch 0031

What it changes: in a page's text, as a browser does: a double click selects the word under the pointer (Selection::word_from_point), and a triple click the inline root's whole text. A text input already did both.

Why we carry it: HOTTY documents are read and copied from like pages (SPEC §9).

For upstream: yes. It is what browsers do.

A double click in a password input selects all of it

Upstream fit: yes · patch 0032

What it changes: a password input's mask has no words: to UAX #29 each bullet is a word of its own, so a double click selected one. As in a browser, a double click selects the whole value, as a triple click does.

Why we carry it: password inputs in HOTTY forms.

For upstream: yes, with "A password input shows a mask and edits its value".


neuroplast.io · Modding · github.com/neuroplastio

Starting the terminal…