reflow_tokenize.c File Reference

No-heap streaming XHTML tokenizer for reflow (replaces the former heap-backed DOM shim). More...

#include <stddef.h>
#include <stdint.h>
#include <string.h>
#include "ra8_attributes.h"
#include "ra8_err.h"
#include "reflow.h"
#include "reflow_tokenize_internal.h"

Include dependency graph for reflow_tokenize.c:

Go to the source code of this file.

Data Structures
struct	tok_ctx_t
	Mutable tokenizer state threaded through the scan helpers. More...

Functions
static bool	internal_suppressed (const tok_ctx_t *ctx)
	Test whether the tokenizer is inside a display:none subtree.
static ra8_err_t	internal_handle_cdata (tok_ctx_t *ctx, const uint8_t *buf, size_t *pi, size_t len)
	Handle a CDATA section, emitting its inner text verbatim.
static ra8_err_t	internal_handle_end (tok_ctx_t *ctx, const uint8_t *buf, size_t *pi, size_t len)
	Handle a closing tag, popping the stack and emitting block-end.
static reflow_html_tag_t	internal_parse_start (const uint8_t *buf, size_t *pi, size_t len, bool *selfclose)
	Parse a start tag's name and self-close flag, advancing past '>'.
static ra8_err_t	internal_open_attrs (tok_ctx_t *ctx, const uint8_t *tag, size_t span, reflow_html_tag_t kind, bool block, const ra8_css_style_t *comp, uint16_t css_font_px)
	Capture a just-opened tag's id (block) or href (<a>).
static void	internal_handle_link (tok_ctx_t *ctx, const uint8_t *buf, size_t tag_lt, size_t end)
	Resolve a <link rel="stylesheet" href> via the bound CSS loader.
static bool	internal_handle_void (tok_ctx_t *ctx, const uint8_t *buf, size_t tag_lt, size_t end, reflow_html_tag_t tag, ra8_err_t *out_err)
	Emit the single token for a void tag (<br> / <hr> / <img>), or resolve a <link> stylesheet.
static uint16_t	internal_css_font_px (const tok_ctx_t *ctx, const ra8_css_style_t *comp)
	Resolve an element's effective CSS font px from the cascade + inherited.
static ra8_err_t	internal_open_styled (tok_ctx_t *ctx, const uint8_t *tagbuf, size_t span, reflow_html_tag_t tag, bool block)
	Run the CSS cascade for a just-pushed element and apply the result.
static ra8_err_t	internal_handle_start (tok_ctx_t *ctx, const uint8_t *buf, size_t *pi, size_t len)
	Handle an opening, void, or self-closing start tag.
static bool	internal_tag_is (const uint8_t *buf, size_t i, size_t len, const char *name)
	True iff <name starts at buf[i] and is followed by a tag delimiter.
static void	internal_handle_raw_text (tok_ctx_t *ctx, const uint8_t *buf, size_t *pi, size_t len, const char *close_lit, bool is_style)
	Consume a raw-text element (<style> / <script>) without emitting.
static ra8_err_t	internal_handle_lt (tok_ctx_t *ctx, const uint8_t *buf, size_t *pi, size_t len)
	Dispatch the markup construct beginning at buf[*pi] == '<'.
static bool	internal_emit_text_run (tok_ctx_t *ctx, const uint8_t *buf, size_t run, size_t end)
	Stash + emit a character-data run, tagging it with the active link id.
ra8_err_t	priv_reflow_xml_walk (reflow_t *engine, const uint8_t *xhtml_buf, size_t xhtml_len)
	Tokenize the XHTML buffer and populate engine->tokens[].

Detailed Description

No-heap streaming XHTML tokenizer for reflow (replaces the former heap-backed DOM shim).

Scans the XHTML byte buffer in a single forward pass and populates the engine's fixed token / text pools (engine->tokens[], engine->text_pool[]) directly – no DOM, no heap. This is the on-target parse path: the firmware traps _sbrk, and the former DOM pool grew via new, so the DOM walk could not run on the device.

Output is byte-for-byte equivalent to the former DOM walk for the v1 conformance subset: a pre-order traversal that emits block-start / block-end tokens for block elements, single tokens for the void elements (br/hr/img), no token for inline-style elements (their style bit is OR'd into descendant text), and pass-through for unknown tags. Text runs are entity-decoded and whitespace-collapsed per text node (HTML rules: runs of ASCII whitespace collapse to one space; a node's leading whitespace is dropped). Inline style (bold/italic/underline) propagates down a bounded explicit stack – no recursion (NASA P10 Rule 1) and bounded loops (Rule 2).

[Ring 4 / Reflow] {World: NS}

Copyright

Copyright (c) 2026 Brighton Sikarskie SPDX-License-Identifier: MIT

Since

0.1.0

Definition in file reflow_tokenize.c.

Function Documentation

internal_css_font_px()

uint16_t internal_css_font_px ( const tok_ctx_t * ctx, const ra8_css_style_t * comp )

static

Resolve an element's effective CSS font px from the cascade + inherited.

Returns the inherited size (0 = none -> UA default) when no font-size is declared, else resolves the declared value: an absolute px, or a %/em of the parent size (the inherited px, or the body size when no ancestor set one). The result clamps to [k_reflow_min_font_px, k_reflow_max_font_px].

Parameters

[in]	ctx	Tokenizer context (carries the inherited font px + body size).
[in]	comp	The element's cascaded style.

Returns

The element's effective CSS font px, or 0 for "use the UA default".

Return values

0	No font-size declared for this element and no ancestor has set one.

Precondition

ctx and comp are non-null.

ctx->engine->font_px is the body font size.

Postcondition

No state mutated.

Return value is clamped to [k_reflow_min_font_px, k_reflow_max_font_px] when non-zero.

Note

Pure aside from reading ctx.

Since

0.1.0

Definition at line 482 of file reflow_tokenize.c.

References tok_ctx_t::css_font_px, tok_ctx_t::engine, reflow_t::font_px, ra8_css_style_t::font_unit, ra8_css_style_t::font_val, k_ra8_css_font_pct, k_ra8_css_set_fontsize, k_reflow_max_font_px, k_reflow_min_font_px, k_reflow_pct_full, and ra8_css_style_t::set.

Referenced by internal_open_styled().

internal_emit_text_run()

bool internal_emit_text_run ( tok_ctx_t * ctx, const uint8_t * buf, size_t run, size_t end )

static

Stash + emit a character-data run, tagging it with the active link id.

Text outside any open element is dropped. Otherwise the run is entity-decoded + whitespace-collapsed into the text pool and emitted as a text token whose reserved byte carries the active <a> link id (0 = none).

Parameters

[in,out]	ctx	Tokenizer context.
[in]	buf	Source buffer.
[in]	run	Run start offset (inclusive).
[in]	end	Run end offset (exclusive).

Returns

true on success, false on a pool overflow.

Return values

false

Text or token pool full.

Precondition

ctx and buf are non-null; run <= end.

ctx->engine->token_count is consistent.

Postcondition

On a non-empty stored run a text token is appended + link-tagged.

On ctx->sp == 0 or inside display:none the pools are unchanged.

Note

Not thread-safe.

Since

0.1.0

Definition at line 795 of file reflow_tokenize.c.

References tok_ctx_t::active_link, reflow_token_t::color, tok_ctx_t::color, tok_ctx_t::engine, tok_ctx_t::face_slot, internal_suppressed(), k_reflow_tag_unknown, k_reflow_tok_text, priv_reflow_tok_emit(), priv_reflow_tok_stash_run(), reflow_token_t::reserved, reflow_token_t::reserved16, tok_ctx_t::sp, tok_ctx_t::style, reflow_t::token_count, and reflow_t::tokens.

Referenced by priv_reflow_xml_walk().

internal_handle_cdata()

ra8_err_t internal_handle_cdata ( tok_ctx_t * ctx, const uint8_t * buf, size_t * pi, size_t len )

static

Handle a CDATA section, emitting its inner text verbatim.

Inner bytes are whitespace-collapsed but NOT entity-decoded (matching XML CDATA semantics). Emits a text token when inside an element and the collapsed run is non-empty.

Parameters

[in,out]	ctx	Tokenizer context.
[in]	buf	Source buffer.
[in,out]	pi	Cursor at "<![CDATA["; advanced past "]]>".
[in]	len	Total buffer length.

Returns

k_ra8_ok, or k_ra8_err_no_mem on pool overflow.

Return values

k_ra8_ok

Section consumed.

Precondition

ctx, buf, pi are non-null.

buf[*pi] begins "<![CDATA[".

Postcondition

*pi advances past the section (or to len).

Token / text pools may grow by one text run.

Note

Not thread-safe.

Since

0.1.0

Definition at line 131 of file reflow_tokenize.c.

References tok_ctx_t::engine, tok_ctx_t::face_slot, internal_suppressed(), k_ra8_err_no_mem, k_ra8_ok, k_reflow_tag_unknown, k_reflow_tok_text, priv_reflow_tok_emit(), priv_reflow_tok_feed(), reflow_token_t::reserved16, tok_ctx_t::sp, strlen(), tok_ctx_t::style, reflow_t::text_pool_used, reflow_t::token_count, and reflow_t::tokens.

Referenced by internal_handle_lt().

internal_handle_end()

ra8_err_t internal_handle_end ( tok_ctx_t * ctx, const uint8_t * buf, size_t * pi, size_t len )

static

Handle a closing tag, popping the stack and emitting block-end.

Restores the style saved at the matching open and, if that element was a block, emits its block-end token.

Parameters

[in,out]	ctx	Tokenizer context.
[in]	buf	Source buffer.
[in,out]	pi	Cursor at "</"; advanced past '>'.
[in]	len	Total buffer length.

Returns

k_ra8_ok, k_ra8_err_no_mem, or k_ra8_err_validation_failed.

Return values

k_ra8_err_validation_failed

Stray end tag (empty stack).

Precondition

ctx, buf, pi are non-null.

buf[*pi..*pi+1] are "</".

Postcondition

*pi advances past the tag (or to len).

The element stack shrinks by one on success.

Note

Not thread-safe.

Since

0.1.0

Definition at line 185 of file reflow_tokenize.c.

References tok_ctx_t::active_link, tok_ctx_t::color, tok_ctx_t::css_font_px, tok_ctx_t::engine, tok_ctx_t::face_slot, tok_ctx_t::family_len, tok_ctx_t::family_off, internal_suppressed(), k_ra8_err_no_mem, k_ra8_err_validation_failed, k_ra8_ok, k_reflow_tok_block_end, priv_reflow_tok_emit(), priv_reflow_tok_is_block(), tok_ctx_t::sp, tok_ctx_t::stack_color, tok_ctx_t::stack_el, tok_ctx_t::stack_face, tok_ctx_t::stack_fam_len, tok_ctx_t::stack_fam_off, tok_ctx_t::stack_font, tok_ctx_t::stack_link, tok_ctx_t::stack_style, tok_ctx_t::style, tok_ctx_t::suppress_sp, and ra8_css_element_t::tag.

Referenced by internal_handle_lt().

internal_handle_link()

void internal_handle_link ( tok_ctx_t * ctx, const uint8_t * buf, size_t tag_lt, size_t end )

static

Resolve a <link rel="stylesheet" href> via the bound CSS loader.

Best-effort. When a CSS loader is bound and the tag is a stylesheet link carrying an href, fetches the bytes and parses them into the chapter sheet at this document position (so a later inline <style> / style= overrides them). Any missing attribute / loader failure is silently ignored – the chapter renders with whatever rules are present.

Parameters

[in,out]	ctx	Tokenizer context (carries the engine + its CSS sheet).
[in]	buf	Source buffer.
[in]	tag_lt	Index of the tag's '<'.
[in]	end	One past the tag's '>'.

Returns

Nothing.

Precondition

ctx, buf non-null; end >= tag_lt.

ctx->engine is non-null and its CSS sheet is initialised.

Postcondition

On a resolved stylesheet, its rules are appended to ctx->engine->css.

When no loader is bound or attributes are absent, the sheet is unchanged.

Note

Not thread-safe.

Since

0.1.0

Definition at line 358 of file reflow_tokenize.c.

References reflow_t::css, reflow_t::css_loader, reflow_t::css_loader_ctx, tok_ctx_t::engine, k_ra8_ok, priv_reflow_tok_find_attr(), priv_reflow_tok_rel_is_stylesheet(), and ra8_css_parse().

Referenced by internal_handle_void().

internal_handle_lt()

ra8_err_t internal_handle_lt ( tok_ctx_t * ctx, const uint8_t * buf, size_t * pi, size_t len )

static

Dispatch the markup construct beginning at buf[*pi] == '<'.

Routes to the comment / CDATA / processing-instruction / declaration skip, or to the end-tag / start-tag handlers.

Parameters

[in,out]	ctx	Tokenizer context.
[in]	buf	Source buffer.
[in,out]	pi	Cursor at '<'; advanced past the construct.
[in]	len	Total buffer length.

Returns

Result of the selected handler (k_ra8_ok on a skip).

Return values

k_ra8_ok

Comment / PI / declaration consumed.

Precondition

ctx, buf, pi are non-null.

buf[*pi] == '<'.

Postcondition

*pi advances past the construct.

Token / text pools may grow per the selected handler.

Note

Not thread-safe.

Since

0.1.0

Definition at line 743 of file reflow_tokenize.c.

References internal_handle_cdata(), internal_handle_end(), internal_handle_raw_text(), internal_handle_start(), internal_tag_is(), k_ra8_ok, priv_reflow_tok_skip_past(), and priv_reflow_tok_starts_with().

Referenced by priv_reflow_xml_walk().

internal_handle_raw_text()

void internal_handle_raw_text ( tok_ctx_t * ctx, const uint8_t * buf, size_t * pi, size_t len, const char * close_lit, bool is_style )

static

Consume a raw-text element (<style> / <script>) without emitting.

Per HTML raw-text rules the content runs verbatim (no nested elements) to the matching lowercase close tag. <style> content is parsed into the engine stylesheet (#111); <script> content is discarded. A missing close tag consumes to end-of-buffer. Lowercase close literals match valid XHTML (XML is case-sensitive and requires lowercase element names).

Parameters

[in,out]	ctx	Tokenizer context (engine stylesheet).
[in]	buf	Source buffer.
[in,out]	pi	Cursor at '<'; advanced past the close tag.
[in]	len	Total buffer length.
[in]	close_lit	Lowercase close tag literal (e.g. "</style>").
[in]	is_style	True to parse the content as CSS, false to discard.

Precondition

ctx, buf, pi, close_lit are non-null.

buf[*pi] == '<'.

Postcondition

*pi advances past the element (or to len).

For <style> any parsed rules are appended to ctx->engine->css.

Note

Not thread-safe.

Since

0.1.0

Definition at line 705 of file reflow_tokenize.c.

References reflow_t::css, tok_ctx_t::engine, priv_reflow_tok_find_lit(), priv_reflow_tok_skip_past(), ra8_css_parse(), tok_ctx_t::saw_element, and strlen().

Referenced by internal_handle_lt().

internal_handle_start()

ra8_err_t internal_handle_start ( tok_ctx_t * ctx, const uint8_t * buf, size_t * pi, size_t len )

static

Handle an opening, void, or self-closing start tag.

Parses the tag name and self-close flag, then dispatches: void tags (br / hr / img) emit a single token and return; self-closing block tags emit an empty block-start / block-end pair; ordinary tags push the element onto the style stack (after bounds-checking depth) and call internal_open_styled() to cascade + apply CSS and emit any block-start token.

Parameters

[in,out]	ctx	Tokenizer context.
[in]	buf	Source buffer.
[in,out]	pi	Cursor at '<'; advanced past '>'.
[in]	len	Total buffer length.

Returns

k_ra8_ok, k_ra8_err_no_mem, or k_ra8_err_validation_failed.

Return values

k_ra8_ok	Tag processed successfully.
k_ra8_err_no_mem	Token pool or nesting-depth bound exceeded.

Precondition

ctx, buf, pi are non-null.

buf[*pi] == '<' and the tag is not an end-tag, comment, or declaration.

Postcondition

*pi advances past the tag (or to len on truncation).

The element stack and token pool grow as required by the tag kind.

Note

Not thread-safe.

Since

0.1.0

Definition at line 604 of file reflow_tokenize.c.

References tok_ctx_t::active_link, tok_ctx_t::color, tok_ctx_t::css_font_px, tok_ctx_t::engine, tok_ctx_t::face_slot, tok_ctx_t::family_len, tok_ctx_t::family_off, internal_handle_void(), internal_open_styled(), internal_parse_start(), internal_suppressed(), k_priv_max_depth, k_ra8_err_no_mem, k_ra8_ok, k_reflow_tok_block_end, k_reflow_tok_block_start, priv_reflow_tok_css_element(), priv_reflow_tok_emit(), priv_reflow_tok_is_block(), tok_ctx_t::saw_element, tok_ctx_t::sp, tok_ctx_t::stack_color, tok_ctx_t::stack_el, tok_ctx_t::stack_face, tok_ctx_t::stack_fam_len, tok_ctx_t::stack_fam_off, tok_ctx_t::stack_font, tok_ctx_t::stack_link, tok_ctx_t::stack_style, and tok_ctx_t::style.

Referenced by internal_handle_lt().

internal_handle_void()

bool internal_handle_void ( tok_ctx_t * ctx, const uint8_t * buf, size_t tag_lt, size_t end, reflow_html_tag_t tag, ra8_err_t * out_err )

static

Emit the single token for a void tag (<br> / <hr> / <img>), or resolve a <link> stylesheet.

<br> -> break, <hr> -> rule, <img> -> image (capturing its src), <link rel=stylesheet> -> external CSS load (no token). Returns whether tag was a void tag so the caller can fall through to ordinary open-tag handling otherwise.

Parameters

[in,out]	ctx	Tokenizer context.
[in]	buf	Source buffer.
[in]	tag_lt	Index of the tag's '<'.
[in]	end	One past the tag's '>'.
[in]	tag	Classified tag.
[out]	out_err	Receives the emit result when handled.

Returns

true iff tag was a void tag (then *out_err is set).

Return values

true	Void tag handled; check *out_err.
false	Not a void tag; caller continues.

Precondition

ctx, buf, out_err are non-null.

end >= tag_lt.

Postcondition

On true a single token was emitted (or the pool overflowed).

On false neither the token pool nor *out_err are modified.

Note

Not thread-safe.

Since

0.1.0

Definition at line 412 of file reflow_tokenize.c.

References tok_ctx_t::engine, internal_handle_link(), internal_suppressed(), k_ra8_err_no_mem, k_ra8_ok, k_reflow_tag_br, k_reflow_tag_hr, k_reflow_tag_img, k_reflow_tag_link, k_reflow_tok_break, k_reflow_tok_image, k_reflow_tok_rule, priv_reflow_tok_capture_attr(), priv_reflow_tok_emit(), and tok_ctx_t::style.

Referenced by internal_handle_start().

internal_open_attrs()

ra8_err_t internal_open_attrs ( tok_ctx_t * ctx, const uint8_t * tag, size_t span, reflow_html_tag_t kind, bool block, const ra8_css_style_t * comp, uint16_t css_font_px )

static

Capture a just-opened tag's id (block) or href (<a>).

For a block tag, captures id="..." and emits the block-start token carrying the id slice (so layout can record an anchor position). For an <a>, captures href="...", interns it, and sets the active link id on ctx.

Parameters

[in,out]	ctx	Tokenizer context (active link + engine pools).
[in]	tag	Raw tag span starting at '<'.
[in]	span	Length of tag, bytes.
[in]	kind	Classified tag.
[in]	block	True iff kind is a block element.
[in]	comp	Cascaded style (supplies the block alignment).
[in]	css_font_px	Block CSS font px to stamp (0 = UA default).

Returns

k_ra8_ok, or k_ra8_err_no_mem on a token-pool overflow.

Return values

k_ra8_err_no_mem

Block-start token pool full.

Precondition

ctx and tag are non-null.

ctx->sp > 0 (the tag was already pushed).

Postcondition

For a block tag a block-start token is emitted (id slice or 0,0).

For <a> ctx->active_link reflects the interned href (0 if none).

Note

Not thread-safe.

Since

0.1.0

Definition at line 288 of file reflow_tokenize.c.

References tok_ctx_t::active_link, ra8_css_style_t::align, reflow_token_t::css_font_px, tok_ctx_t::engine, internal_suppressed(), k_ra8_css_set_align, k_ra8_err_no_mem, k_ra8_ok, k_reflow_align_left, k_reflow_tag_a, k_reflow_tok_block_start, priv_reflow_tok_capture_attr(), priv_reflow_tok_emit(), priv_reflow_tok_intern_link(), reflow_token_t::reserved, ra8_css_style_t::set, tok_ctx_t::style, reflow_t::token_count, and reflow_t::tokens.

Referenced by internal_open_styled().

internal_open_styled()

ra8_err_t internal_open_styled ( tok_ctx_t * ctx, const uint8_t * tagbuf, size_t span, reflow_html_tag_t tag, bool block )

static

Run the CSS cascade for a just-pushed element and apply the result.

Builds the element's CSS identity and inherited run style, runs the author <style> rules + inline style attribute through ra8_css_cascade_ctx() (#111 / #140), emits the block-start via internal_open_attrs() carrying the cascaded alignment + font size, and updates ctx->style, ctx->color, ctx->css_font_px, ctx->family_off/len, and ctx->face_slot for descendant content. Unstyled content lays out byte-identically to the pre-CSS engine. Factored out of internal_handle_start() to stay within the NASA Rule 4 function-length budget.

Parameters

[in,out]	ctx	Tokenizer context (element already pushed onto stack).
[in]	tagbuf	Raw tag span starting at '<'.
[in]	span	Length of tagbuf, bytes.
[in]	tag	Classified tag.
[in]	block	True iff tag is a block element.

Returns

k_ra8_ok, or k_ra8_err_no_mem on a block-start token-pool overflow.

Return values

k_ra8_ok	Cascade applied; context updated.
k_ra8_err_no_mem	Token pool full when emitting the block-start token.

Precondition

ctx and tagbuf are non-null.

ctx->sp > 0 (the element has already been pushed by the caller).

Postcondition

ctx->style / ctx->color / ctx->css_font_px reflect the cascaded style.

A block element emitted its block-start token with cascaded alignment.

Note

Not thread-safe.

Since

0.1.0

Definition at line 528 of file reflow_tokenize.c.

References ra8_css_style_t::color, tok_ctx_t::color, reflow_t::css, tok_ctx_t::css_font_px, ra8_css_style_t::display, tok_ctx_t::engine, tok_ctx_t::face_slot, ra8_css_style_t::family_len, tok_ctx_t::family_len, ra8_css_style_t::family_off, tok_ctx_t::family_off, internal_css_font_px(), internal_open_attrs(), k_priv_style_mask, k_ra8_css_set_color, k_ra8_css_set_display, k_ra8_css_set_family, k_ra8_ok, k_reflow_color_inherit, priv_reflow_tok_css_inline(), priv_reflow_tok_resolve_face_slot(), priv_reflow_tok_style_for(), ra8_css_cascade_ctx(), ra8_css_style_t::set, tok_ctx_t::sp, tok_ctx_t::stack_el, ra8_css_style_t::style, tok_ctx_t::style, and tok_ctx_t::suppress_sp.

Referenced by internal_handle_start().

internal_parse_start()

reflow_html_tag_t internal_parse_start ( const uint8_t * buf, size_t * pi, size_t len, bool * selfclose )

static

Parse a start tag's name and self-close flag, advancing past '>'.

Skips attributes quote-aware so a '>' inside an attribute value does not end the tag.

Parameters

[in]	buf	Source buffer.
[in,out]	pi	Cursor at '<'; advanced past '>'.
[in]	len	Total buffer length.
[out]	selfclose	Set true for a "/>"-terminated tag.

Returns

The classified tag.

Return values

k_reflow_tag_unknown

Unrecognised element name.

Precondition

buf, pi, selfclose are non-null.

buf[*pi] == '<'.

Postcondition

*pi advances past the tag (or to len).

*selfclose reflects the "/>" terminator.

Note

Not thread-safe.

Since

0.1.0

Definition at line 236 of file reflow_tokenize.c.

References priv_reflow_tok_classify(), and priv_reflow_tok_is_xml_whitespace().

Referenced by internal_handle_start().

internal_suppressed()

bool internal_suppressed ( const tok_ctx_t * ctx )

static

Test whether the tokenizer is inside a display:none subtree.

Returns true when ctx->suppress_sp is nonzero, meaning the scan entered an element whose cascaded style declared display:none (#140). All token and text-pool emits are suppressed until the matching close tag pops the stack back above the recorded depth.

Parameters

[in]

ctx

Tokenizer context to query.

Returns

true while inside a display:none subtree; false otherwise.

Return values

true	Emission is suppressed for the current position.
false	Emission proceeds normally.

Precondition

ctx is non-null.

ctx->suppress_sp is a valid depth value (0 or a previous sp).

Postcondition

No state is modified (pure read of ctx).

ctx->suppress_sp is unchanged.

Note

Pure function.

Since

0.1.0

Definition at line 100 of file reflow_tokenize.c.

References tok_ctx_t::suppress_sp.

Referenced by internal_emit_text_run(), internal_handle_cdata(), internal_handle_end(), internal_handle_start(), internal_handle_void(), and internal_open_attrs().

internal_tag_is()

bool internal_tag_is ( const uint8_t * buf, size_t i, size_t len, const char * name )

static

True iff <name starts at buf[i] and is followed by a tag delimiter.

Distinguishes <style ...> / <script> from look-alikes such as a hypothetical <styled> by requiring >, /, whitespace, or end-of-buffer after the name.

Parameters

[in]	buf	Source buffer.
[in]	i	Offset of the leading '<'.
[in]	len	Total buffer length.
[in]	name	Literal element open including '<' (e.g. "<style").

Returns

true iff the element name matches with a following delimiter.

Return values

false

No match, or a longer name continues past name.

Precondition

buf and name are non-null.

i <= len (caller guarantees the cursor is within bounds).

Postcondition

No state is modified (pure).

Return value depends solely on the inputs.

Note

Pure function.

Since

0.1.0

Definition at line 669 of file reflow_tokenize.c.

References priv_reflow_tok_is_xml_whitespace(), priv_reflow_tok_starts_with(), and strlen().

Referenced by internal_handle_lt().

priv_reflow_xml_walk()

ra8_err_t priv_reflow_xml_walk	(	reflow_t *	engine,
		const uint8_t *	xhtml_buf,
		size_t	xhtml_len )

Tokenize the XHTML buffer and populate engine->tokens[].

The tokenizer entry point. A single forward pass over the XHTML source with no heap and no DOM. Defined in reflow_tokenize.c and consumed by the layout driver in reflow_parse.c; the per-decision helpers above are the individually-MC/DC-able primitives it drives.

Parameters

[in,out]	engine	Engine whose token / text pools are populated.
[in]	xhtml_buf	XHTML source bytes.
[in]	xhtml_len	Length of xhtml_buf, in bytes.

Returns

ra8_err_t Error code.

Return values

k_ra8_ok	Tokenization succeeded.
k_ra8_err_null_ptr	engine or xhtml_buf is null.
k_ra8_err_invalid_size	xhtml_len is zero.

Precondition

engine and xhtml_buf are non-null.

xhtml_len is non-zero.

Postcondition

On success engine->tokens[] is populated.

On failure the engine token / text pools are left unmodified.

Note

Not thread-safe; single-threaded, non-recursive.

Since

0.1.0

Definition at line 827 of file reflow_tokenize.c.

References tok_ctx_t::color, reflow_t::css, tok_ctx_t::engine, internal_emit_text_run(), internal_handle_lt(), k_ra8_err_invalid_size, k_ra8_err_no_mem, k_ra8_err_null_ptr, k_ra8_err_validation_failed, k_ra8_ok, k_reflow_color_inherit, k_reflow_style_normal, ra8_css_sheet_reset(), tok_ctx_t::saw_element, tok_ctx_t::sp, and tok_ctx_t::style.

Referenced by reflow_parse_xhtml().