What should a native DOM templating API look like?
If you read my previous post, The time is right for a DOM templating API, you might be wondering what such an API would look like.
Let's dive into that question now.
What are we building?
First, let's clarify what we're trying to design here, because when people hear the abstract template API idea described, before there's a concrete proposal or examples, they can sometimes think of very different things.
In webcomponents/1069 I propose that we add a "declarative JavaScript templating API"
Declarative means that the developer is describing what they want, not how to achieve it. What exactly constitutes "declarative" is often debatable, and under every declarative API is an imperative implementation, but hopefully people can align on the spirit of the word.
By "templating" we mean the ability to describe the DOM that the developer wants to create and update in a form that resembles the output. This includes syntaxes like JSX and lit-html, which for some people count as something other than templating. For our purposes, they are.
Taken together, "declarative DOM templating" is something that is very, very common. All major frameworks today have declarative templating at their core.
But we're going to narrow our focus a bit JavaScript APIs.
By "JavaScript API" we mean methods, classes, etc. that are part of the overall DOM JavaScript API surface. Something that's a sibling and alternative to innerHTML, importNode(), and a bunch of other DOM APIs that web developers and frameworks use to create and update DOM. But also, we mean that script-like abilities will be left to JavaScript, not added to markup.
It's also important to not that we're not talking about components at all, just templated DOM creation and updating. Most frameworks intertwine the two concepts, so some people aren't aware that there can be a distinction. One of the use cases for this proposal is to be a primitive that frameworks can use, so it's necessary that components can be added on top of templates, but any such feature to do that should be as generic and minimal as possible.
What's guiding us?
In my experience, we can derive the general API shape for this feature from its constraints and requirements, with a dash of opinion about good APIs on top. Listing these helps us see what things must be a certain way vs which things can flex as part of the decision space for the problem.
Constraints
Because we're designing a native DOM API, and not a framework, we have more constraints to deal with. A native API has to work within the current platform, can't break existing sites, and has to be viable within the standards process.
Web compatibility: The proposal can't introduce any breaking changes to the platform.
Standard file types: While a proposal could introduce a new file type just for templates, that's unlikely to make it through the web standards process without incredibly compelling reasons. We'll take using the existing file types as a hard constraint.
Standard JavaScript: It's also possible to try to simultaneously introduce changes to both the DOM and JavaScript, but this is also less likely to succeed because we would be dealing with two separate standards bodies and a much larger overall addition to the platform.
JavaScript is not just a web language anymore, and the JavaScript side would likely have to be useful for non-web use-cases to move forward, dramatically raising the complexity of a proposal.
So we'll also take it as a hard constraint that this API works with today's standard JavaScript.
Low implementation security risk: The proposal should not require drastic and potentially-unsafe HTML parser changes. Implementors are just extremely reluctant to make large HTML parser changes, since they are prime targets for attackers and previous changes (like
) did introduce security bugs.No performance regressions on existing sites. Slowing down existing sites, or benchmarks like Speedometer, will sink a proposal when it's implemented.
Security: New APIs have a much higher security bar to clear than previous platform changes. They need to be secure by default.
No required compiler: This is not universal, but many web platform maintainers insist that web APIs should be directly usable by developers and not require a compiler to be useful. This does not preclude an API from being a good compile target for alternate syntaxes.
Coherence with the platform. Also not universal, but it will be drastically easier to get a proposal accepted if it's coherent with the existing platform. This would prevent us from introducing a new "better DOM", for instance.
Requirements
Next we need to look at our requirements. What does this proposal have to do to be useful and successful? The actual proposal will need to detail its target use cases, but we can list some requirements now:
Ergonomic API: This API should be nice to use directly for developers.
Usable from JavaScript libraries: Libraries should be able to adopt this API as an internal implementation helper.
JavaScript for control flow and expressions: We don't want to introduce a new JavaScript-like expression language. That would increase the size of the proposal, be less likely to get agreement on, and introduce security concerns.
It would also introduce a different lexical environment that developers would have to get their data into. It's much easier on developers and tooling if template expressions exist within the same lexical scope as their data.
Ability to set attributes, properties, and event listeners: DOM elements have three major key/value style APIs that developers are used to setting declaratively from templates. Because these APIs are truly independent in the DOM, we require the ability to set them each unambiguously.
Support full HTML syntax: elements, attributes, children, comments, etc. We need to be able to create any structure that can be created with HTML.
Security: More specific than the platform constraint, we need templates to be safe from XSS and gadget attacks, etc.
Performance: The API must be fast for initial rendering and updates with DOM stability.
Composability: Developers should be able to build templates from pieces, and abstract over templates and rendering.
Extensibility: One template API is not going to be able to have every feature that all developers need. It needs to have hooks for extending the system with new behavior.
Open to evolution and extension of HTML and the DOM: The system should assume that the platform will evolve and that elements, attributes, and events can be added in userland. It should avoid special casing specific DOM elements and attributes.
Opinions
Then we have opinions, which people might or might not agree with. Things that make the API good, elegant, maximally general and flexible. These are my personal opinions, but I hope they're very reasonable and shared by a lot of other people.
The APIs should work with functional-style programming. One of the motivating reasons to have a declarative template API is to reduce the amount of imperative code need to build UIs. An API compatible with function-style programming can still be used in imperative contexts.
Composition should be a core feature that other features are built on. For example, instead of specific and special constructs for conditionals and loops, we should be able to rely on JavaScript and composition to dynamically construct a template.
The underlying DOM creation approach should be HTML template cloning. HTML templates - the
tag has a few performance advantages over other DOM creation approaches.s can use a simpler HTML parser, and cloning nodes is faster than re-parsing HTML as needed withinnerHTML.The API should support support multiple models of reactivity and DOM updates, possibly through userland extension.
- Template re-rendering should be a core supported feature because it is extremely fast and generalizes to many types of data.
- Fine-grained reactivity should be supported for native observable data types, which will hopefully soon include Signals.
- DOM diffing with various algorithms should be possible to add with userland utilities.
The API should have good layering. DOM update mechanisms should utilize lower-level platform APIs, like DOM Parts and a tree-aware task scheduler, that can be used by libraries that aren't using the templating API.
The API should not introduce any new component abstractions, but it should have stateful hooks that allow a framework to easily attach their own component instances to the DOM.
Template syntax
In my previous post I made the claim that we know what good template syntax looks like. While some people took exception with that claim, I stand by it because of how fundamentally similar all the popular template syntaxes are. The claim is even stronger when considering JavaScript-based APIs.
The core similarity is that HTML templates are all markup with interpolations and control flow.
Compare:
- React
<> <h1>Hello {name}h1> <button onClick={handleClick}>Click Mebutton> <ul> {items.map((i) => (<li>{i}li>))} ul> > - Vue
<template> <h1>{{ name }}h1> <button @click="handleClick">Click Mebutton> <ul> <li v-for="item in items"> {{ item.message }} li> ul> template> - Angular
<h1>{{ name }}h1> <button (click)="handleClick()">Click Mebutton> <ul> @for (item of items) { <li> {{ item.message }} li> } ul> - Lit
html`Hello
${name} `)}
<template>
<h1>{{ name }}h1>
<button on-click="handleClick">Click Mebutton>
<ul>
<template is="dom-repeat" items="{{ items }}"">
<li>
{{ item.message }}
li>
template>
ul>
template>html<MyElement>`
Hello
${x => name}
${repeat(x => x.items, html<string>`${i => i}To me, these are in fact very similar:
- The HTML portion is described in markup
- Bindings can appear in text and attribute value positions
- Bindings expressions are (usually) delimited by some kind of separator
- Control flow can conditionally render a subtree/sub-template
Yes, there are some differences.
- Different expression delimiters
- Some systems delimit all expressions, some don't in certain cases (bound attributes)
- Expression language differences
- Property and event listener delineation
- Control flow: attributes (directives), templates, non-markup syntax, control-flow-is-JS-expressions
- Some template-literal-based systems require bindings to contain closures.
I personally don't think these differences are actually all that big. Most of them are surface syntax.
The biggest difference in my mind are:
- Whether the syntax is HTML-based or JavaScript-based
- Control flow syntax and placement among the HTML-based syntaxes
- Whether bindings require closures in JavaScript-based syntaxes
When we narrow our choices based on the fact that we and proposing a JavaScript API, most of the differences fall away. Expressions are JS, control flow is just expressions, expression delimiters are fixed.
Template expressions
One of my strongly held opinions in this area is that templates should be side-effect free expressions that return a value that describes the intended DOM. This is a functional-style API that's extremely flexible and simple. This is what most JSX transforms (ie, React's) do, what Lit does, and most other templates-in-JS systems do.
In addition, I believe that template expressions should be able to be re-evaluated to generate a new description of DOM, also how React and Lit work. This ensures that any data available in the lexical scope of the template can be consumed by templates, and any trigger that indicates that data has changed can be used to initiate a template re-evaluation.
The approach of expressions generating descriptions of DOM lends itself to a wide variety of powerful techniques.
- Abstraction: template results can be evaluated and returned by functions, accepted by functions, and computed over. It enables patterns like higher-order templates and render props.
- Transformation: Utilities can walk over the DOM description and transform it into a different description.
- Lazy evaluation: template expressions naturally work with lazily evaluated JavaScript constructs like ternaries, short-circuiting operators, defaults for parameters and object destructuring.
- Alternate render functions: The process of rendering a DOM description isn't locked off to platform code. Alternate ways of applying a description to live DOM, like DOM morphing libraries, can take these DOM descriptions and render them in userland. This can even be a means of polyfill future new features of the template system.
Alternatives to template expressions
- Static template definitions. This is how FAST, Polymer, Angular, and some others work. A template is a property of a component class, and is interpreted by the component instances. I believe this ties templates and components too closely together for a native API, and instead of just accessing variables in a JavaScript lexical scope (ie, function scope or class instance scope in a method) expressions have to be bound to some scope object, which make this not play very well with plain functions and functional style programming.
- Side-effectful statements. These mutate DOM as they are evaluated. Google's internal framework, Wiz, worked this way when I left the company. It seemed to me to cause a lot friction for abstraction, as there was no way to programmatically access the template definition or result. The best you could do is pass around an opaque closure with the instructions in it.
Template container syntax.
Aside from the internal template syntax, like bindings and such, we need a container for templates. Let's look at the relevant constraints and requirements:
- No compiler required, only standard syntax
- Templates resemble output, ie markup
- Full support for HTML
This first constraint rules out using JSX (until and unless JSX becomes a standard). And without JSX, the next two items pretty much require that our templates be contained in strings. Nested function calls (like React's createElement() or Hyperscript's h()) or object-based builder-style APIs just don't look enough like HTML.
So we want the HTML portion of a template to be strings, like:
'Hello World
'But we also need a way to include expressions, and one of our requirements is that we use JavaScript instead of a new expression language.
This rules out something like:
'Hello {{ name }}
'because this {{ name }} bit is inside the string and can't reference variables in the containing JavaScript scope. You could try to do a format-string style of API with a separate data bag of named parameters:
render('Hello {{ name }}
', {name: 'World'})but that's pretty ugly, and JavaScript has much a nicer syntax for doing this same thing already!
Tagged template literals
The most straightforward way to embed markup in standard JavaScript is with tagged template literals. In fact, embedding a DSL like HTML in JavaScript is exactly what tagged template literals were designed for - and they have some special features to make doing so efficient and safe.
We'll call our tag the most obvious thing: html. Templates expressions are then template literals tagged with the html tag:
html`Hello World`This ends up looking a lot like JSX, but with the markup inside of a string instead of as part of the outer language grammar.
With tagged template literals, our expression delimiter syntax is chosen for us: ${}. So expressions look like:
html`Hello ${name}`Benefits of tagged template literals
It's worth taking a moment to talk about tagged template literals, because I think they're one of the least appreciated and understood parts of ES2015.
Like I said above, tagged template literals were specifically designed to enable embedding of external DSLs (like HTML, SQL, or GraphQL) into JavaScript, and they have a few features that make this possible.
A tagged template literal is prefixed by a reference to a tag function, and a tag function has this signature:
(strings: TemplateStringsArray, ...values: Array<unknown>) => R;(where R is the return type of the specific tag function)
The important features for DSLs:
The tag function receives the static strings and values separately.
This means that we can process the static strings and values separately, which is great for both performance and security. We can skip most work for the static strings, and use them to make a cloneable
element. We trust the static strings as developer-authored and allow them to create HTML; we distrust the values as potentially user-controlled and make the browser escape them before inserting into the DOM.The tag function can return any type of value, not only strings.
This means that while template expressions may look a lot of string interpolation, they are quite a bit more powerful. Our
htmltag will simply return an object holding the static strings and the values.The static strings array passed to the tag function is the same array instance every time a tagged template literal is evaluated.
This is a little subtle. If you have a tagged template literal expression that can be evaluated multiple time, say if it's in a function that's called multiple times, that the tag function will receive the same strings array on every evaluation.
Here's a simple example:
// A template tag function that simply returns the strings array const tag = (strings) => string; // A function that evaluates returns the result of tag const run = (x) => tag`abc ${x} def`; run(1) === run(2); // trueThis means that we can perform one-time setup work based on the strings, and then use the strings array as a cache key to retrieve that work on future evaluations. We can also use the strings array as a key to check whether a template has already been been rendered to a DOM location so that we only update the changed values.
The tag can identify the embedded language. Any template literal using the tag
htmlcontains HTML. Lots of IDEs and other tools key off of the tag name to automatically give developers syntax highlighting and other checks.
Downsides of tagged template literals
Of course, tagged template literals aren't perfect, otherwise there's a good chance that the React team would have used them instead of inventing JSX. Two things that may or may not matter much depending on your personal preferences are related to verbosity:
More symbols than JSX for templates and bindings. The
htmltag and backticks are 6 more characters per template expression than JSX with one root element, though only one more character for a JSX expression wrapped in a fragment. The binding delimiter of${}is one more character than{}.Verbose for userland components. This one is more significant. In tagged template literals, element names are strings in the static side of the template. This means they can't hold JavaScript references to component definitions. String tage names work well with custom elements because components, like
<${MyComponent}>.This is why it's important that this template API can be a good compile target for JSX. If we want userland component systems to be able to use the API, they will need to allow their users to write templates in their most preferred syntax, and can then compile to something that may require more characters.
By the way, JSX's convention of capitalized tag names being JS references and lower-cased names being string literals is one of my biggest worries about the standardization of JSX. I haven't heard anyone on TC39 opine on this yet, but it seems like the kind of odd rule that works well enough in practice, but that some people will object to. It's entirely possible that to make it through standardization a proposal would need a more explicit differentiator between strings and references anyway.
Aren't strings bad?
Some people critique lit-html and similar APIs as being too "string based". The complaint being that a string can contain anything and isn't checked for correctness by JavaScript parsers. I think this complaint is usually overblown. All programming languages are strings. JavaScript, HTML, and CSS are strings. What makes them structured languages are their grammars and tools that understand them.
This holds true for these tagged template literals. There is a syntax and grammar to them, which is just standard HTML and can be checked by tools and runtimes. Developers who use these style of templates today usually add editor and compiler extensions to their toolchain to give them all the benefits of good PL tools support: syntax highlighting, syntax checking, type-checking, intellisense, linting, and formatting.
The biggest difference between an embedded syntax like this and an extension syntax like JSX is that in the actual browser runtime syntax errors will happen at template expression evaluation time rather than module parse time. This difference should be mitigated by edit and built-time tools that check template syntax.
Bindings and template parts
Because we have static template vs dynamic expression separation, we can mark exactly which portions of the DOM will change and which won't. Expressions in templates - really the gaps where expressions go - create DOM Parts that we can update with new values.
DOM Parts are a proposal for a new DOM object that can be attached to a specific location in the DOM and updated over time. It's a lower-level templating feature that will need to be worked on as part of any proposal here. A goal with DOM Parts is being usable by frameworks and template libraries. If a framework can't take advantage of the template API for some reason, hopefully it can use the DOM Parts APIs directly.
Attributes, properties, and events
After we have our container syntax, we have another bit of template system to figure out: how to differentiate between attribute, properties, and events.
Attribute, properties, and events are three separate key/value-style namespaces on elements that developers expect to be able to set declaratively from templates.
Frameworks and templating libraries take different approaches to this. Some try to merge them into a single namespace and differentiate them with runtime introspection, conventions, and/or special case lists. Others disambiguate with prefixes or sigils on the attribute name to signify properties and events.
Some examples:
- React: special case lists, runtime introspection, and
onprefix for events. - Vue: runtime introspection and specific syntax.
- Angular: specific syntax
- Lit: specific syntax
Runtime introspection is potentially error-prone, and there an element could potentially have an attribute, property, and event of the same name and require specific disambiguation to setup properly.
Vue and Lit both use a similar shorthand syntax for properties and events:
.prefix for setting properties.@prefix for setting event listeners
In addition Lit has a useful ? prefix for opting into boolean attribute behavior.
I think this is a nice, concise, and pretty intuitive syntax. A proposal will go into more detail on the rationale for this choice, but here's what it would look like in a template:
html`
${v}
?required=${isRequired}
@input=${handleInput}>
`Isn't this not "just HTML" anymore?
These sigils do give special meaning to attribute that plain HTML doesn't have, but they entirely valid standard HTML. So the syntax is standard while the semantics have been augmented for our DOM templating purposes.
Composition, conditionals, and looping
Any template system needs control flow and composition capabilities. One thing that React and JSX taught us is that control flow can be implemented in terms of composition, leaving all control flow features to JavaScript.
Start with simple composition: a template expression can contain a reference to another template:
const hello = html`Hello`;
const message = () => html`${hello} World!`;And now you can use JavaScript for conditionals:
const hello = html`Hello`;
const goodbye = html`Goodbye`;
const message = (isLeaving) => html`${isLeaving ? goodbye : hello} World!`;Like JSX, ternaries aren't the only conditional you can use. Any conditional expression or function works.
Then we need to support nesting lists of nested-templates as well:
const renderList = () => html`
${[
html`One `,
html`Two `
]}
`;And we get looping:
const renderList = () => html`
${items.map((i) => html`${i}`)}
`;Updating and reactivity
Once we have a way to express templates we need a way to actually render them to the DOM, and like my last post pointed out, it's critical that we have ways to efficiently update that DOM from the same template later.
The React and Lit render() API for this is pretty simple, and would work well for a built-in API.
Those APIs take a renderable value and a container to render into. But since we're imagining a native API, we can introduce a new render() method on Element that takes a template result:
const go = (name) => containerEl.render(html`Hello
${name}!`);Updates can work in two ways: template re-rendering, and fine-grained updates.
Template re-rendering
Template re-rendering works by recording what template was rendered to a DOM location, and checking any new template to be rendered there against the previous template. If they're different, then the DOM is cleared and the new template is rendered. If they're the same, then just the dynamic values that changed are updated.
This is quite often the same result you get from VDOM diffing, especially if you key your top-level elements in a JSX fragment. It's essentially keying the fragment by the template identity.
This approach is a lot more efficient than VDOM diffing because it's a single identity check for the template instance, then a dirty check for each value. We're diffing on the value side, and by running through a linear list of new and old values rather than traversing a VDOM tree.
Fine-grained reactivity
Fine-grained reactivity approaches, like signals, can be even more efficient in some cases. And they can have ergonomic benefits if your shared data is inside of observable containers as far-flung uses of that data will all update when the data changes.
There's a proposal to add Signals to JavaScript. If that moves forward, signals could be easily supported within templates for fine-grained reactivity:
const name = new Signal.State('Fred');
containerEl.render(html`Hello
${name}!`);
// DOM: Hello Fred!
name.set('Ambrose');
// DOM: Hello Ambrose!
There are a lot of important details to work out around batching and scheduling of updates and efficient list updating, but I think it's important to have a path forward to built-in fine-grained reactivity. I've seen a lot of web developers asking for something like this.
Why not JSX again?
The big reason not to pursue standardizing JSX as part of this proposal is that it would make the proposal much larger and much harder to move forward.
I do hope that TC39 tackles some addition to JavaScript that meets the JSX demand, and in that case the templating API should accept whatever data type JSX expressions produce. I hope that if a templating API is added to the platform before JSX that it would really help inform the capabilities we need out of a JSX-like addition.
In the meantime, people can develop and use JSX transforms for the template API, and that's an important goal of this API idea. This has already been done for Lit, and Preact's html library should that you can make the JSX->template literal association in the other direction too.
Why not templates in HTML itself?
We already have the element, but only supports cloning its inert contents, and a lot of developers want something more powerful like data-binding support.
I also think that we can do this, but it's a larger task than a JavaScript API - we'd have to specify binding syntax, expressions, and control flow that are already done for us in JavaScript.
Isn't this just standardizing lit-html?
Honestly? Yes, and no. Though I think it's more accurate to say that this proposal and lit-html look similar because they share similar constraints and goals, and this API shape is the most obvious way to accomplish them.
The reason why lit-html looks the way it does is precisely because the Polymer team had these same constraints and requirements that the web platform does. This was partly the disposition of the team members at the time, but largely because as part of the Chrome organization, that was our mission. A large part of our job was to polyfill and prototype upcoming standards, and experiment with things that potentially could be standards proposals one day.
Once Polymer moved from using HTML Imports to JavaScript modules, its HTML-based template system became a big drag. Compared to a more JavaScript-oriented solution, it was more code to implement, had more custom concepts and cognitive overhead, slower in many cases, and created a division between JavaScript and template lexical environments that added friction to development.
lit-html was our response to those issues, still working within our constraints of no required compiler, no forking the web's core languages, and potentially standardizable features and API shapes. This simply led us to the same place that this proposal is going.
And we weren't the only ones. Preact's htm library, Microsoft's FAST, and HyperHtml look extremely similar, for similar reasons.
In fact, I'm not aware of a client-side templating library that meets these constraints and requirements and doesn't look a lot like lit-html (or HyperHTML or htm or FAST). I would love to see examples if there are any.
So it seems like just a natural way to do things given the way JavaScript and the DOM work. I think that's a strong indicator of the style of API being ready for standardization.
Not everything in lit-html is standardizable in my opinion, and any proposal here should be derived on its own from its own requirements, not copied from lit-html, so I don't think this proposal would look exactly like lit-html. But if this API were added to the platform I do think it would largely obsolete lit-html. But that's also one of the goals of the Lit née Polymer team: to eventually be obsoleted by the platform itself.
Summary
I think that's a pretty good overview of the type of API I think that the web should standardize on. The end result is something that looks and feels like the templating portions of React and other popular frameworks in a lot of ways, but works with the standard JavaScript we have today.
This can be a controversial area of web development to wade into, so any proposal here is going to have to be extremely well researched, argued, socialized, and detailed early to have any hope of success. I hope to have time to push these ideas forward over the next few months.
If you're interested in collaborating, please let me know by commenting on webcomponents#1069, or reach out to me on Bluesky!
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0
