mdl_extract_hits.c File Reference

Bounded titled-anchor extraction and deduplication. More...

#include <string.h>
#include "mdl_extract.h"
#include "mdl_extract_internal.h"

Include dependency graph for mdl_extract_hits.c:

Go to the source code of this file.

Data Structures
struct	mdl_entity_t
	One recognised HTML entity mapped to its single replacement byte. More...

Functions
static bool	internal_is_ws (char c)
	ASCII whitespace test (locale-independent).
static bool	internal_decode_entity (const char *s, size_t len, size_t *i, char *out_ch, bool *space)
	Decode an entity at &, returning its byte and advancing *i.
static void	internal_append_out_char (char *out, size_t *n, size_t cap, bool *fits, char ch)
	Append one character to bounded output, tracking overflow.
static bool	internal_clean_inner_text (const char *text, size_t len, char *out, size_t cap)
	Copy the anchor inner text from text into out, cleaned for display.
static void	internal_url_slug (const char *url, char *out, size_t cap)
	Fill out with the URL's last non-empty path segment (slug fallback).
static bool	internal_anchor_title (const char *html, size_t html_len, const char *tag, size_t tag_len, size_t gt_off, char *out, size_t cap)
	Read the title for the anchor whose > sits at gt_off.
static size_t	internal_hit_index_of (const mdl_hit_list_t *out, const char *url)
	Index of an existing hit with URL url, or count when absent.
static ra8_err_t	internal_merge_hit (mdl_hit_list_t *out, bool *real, const char *url, const char *title, bool title_real)
	Merge one resolved (url,title) hit; upgrade a slug with a real title.
static ra8_err_t	internal_emit_hit (const char *html, size_t html_len, const char *base_url, const char *keep, const char *tag, size_t tag_len, size_t gt_off, mdl_hit_list_t *out, bool *real)
	Resolve one anchor's href, count it, and emit a filtered hit.
ra8_err_t	mdl_extract_hits (const char *html, size_t html_len, const char *base_url, const char *url_contains, mdl_hit_list_t *out)
	Scan html for <a href> links, keeping each hit's title and URL.
static bool	internal_class_has_token (const char *classes, const char *value)
	Find one whitespace-delimited class token.
static ra8_err_t	internal_selector_copy (const char *value, char *out, size_t cap)
	Copy one non-empty literal selector value.
static ra8_err_t	internal_extract_meta (const char *html, size_t len, const char *key, char *out, size_t cap)
	Extract one matching HTML meta-content value.
static ra8_err_t	internal_extract_class (const char *html, size_t len, const char *token, char *out, size_t cap)
	Extract visible text from the first matching class token.
static ra8_err_t	internal_extract_label (const char *html, size_t len, const char *label, char *out, size_t cap)
	Extract the first anchor text following a visible label.
ra8_err_t	mdl_extract_selector (const char *html, size_t html_len, const char *selector, char *out, size_t out_cap)
	Extract one bounded metadata value using a site-descriptor selector.

Detailed Description

Bounded titled-anchor extraction and deduplication.

Scans anchor tags, resolves URLs through the shared public resolver, cleans bounded display titles, and publishes only complete unique hits into caller-owned storage without allocation.

Copyright

Copyright (c) 2026 Brighton Sikarskie SPDX-License-Identifier: MIT

Since

Version 0.1.0

Definition in file mdl_extract_hits.c.

Function Documentation

internal_anchor_title()

bool internal_anchor_title ( const char * html, size_t html_len, const char * tag, size_t tag_len, size_t gt_off, char * out, size_t cap )

static

Read the title for the anchor whose > sits at gt_off.

Returns

true when a real title (attribute or inner text) was found; false when the caller must fall back to the URL slug.

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[in]	html	Readable markup bytes.
[in]	html_len	Readable markup length.
[in]	tag	Readable tag byte span.
[in]	tag_len	Readable tag length.
[in]	gt_off	Offset of the tag-closing greater-than byte.
[out]	out	Caller-owned result storage.
[in]	cap	Destination capacity including any terminator.

Return values

true	The documented predicate holds or the requested operation completed.
false	The predicate does not hold or validation rejected the operation.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 229 of file mdl_extract_hits.c.

References internal_clean_inner_text(), internal_find_attr_value(), internal_find_ci(), and RA8_INTERNAL.

Referenced by internal_emit_hit().

internal_append_out_char()

void internal_append_out_char ( char * out, size_t * n, size_t cap, bool * fits, char ch )

static

Append one character to bounded output, tracking overflow.

Writes ch at out[*n] when capacity remains (reserving room for the trailing NUL), advancing *n; otherwise leaves out unchanged and marks fits false.

Parameters

[out]	out	Destination buffer.
[in,out]	n	Current write offset, advanced on success.
[in]	cap	Destination capacity including the trailing NUL.
[in,out]	fits	Cleared when a character could not be appended.
[in]	ch	Character to append.

Returns

Nothing.

Precondition

out addresses cap writable bytes.

n and fits are non-NULL and already initialized.

Postcondition

*n always leaves room for a trailing NUL at out[*n].

fits is only ever cleared, never set back to true.

Note

Pure bookkeeping; never itself writes the trailing NUL.

Since

0.1.0

Definition at line 106 of file mdl_extract_hits.c.

Referenced by internal_clean_inner_text().

internal_class_has_token()

bool internal_class_has_token ( const char * classes, const char * value )

static

Find one whitespace-delimited class token.

Uses exact token boundaries so substrings of longer class names do not match.

Parameters

[in]	classes	NUL-terminated HTML class attribute.
[in]	value	Non-empty class token to find.

Returns

Whether value occurs as one complete token.

Return values

true	A boundary-delimited token matched.
false	No complete token matched.

Precondition

classes and value are non-NULL and NUL-terminated.

value is non-empty.

Postcondition

Both inputs are unchanged.

No locale or global state is accessed.

Note

Thread-safe: reads only caller storage.

Since

0.1.0

Definition at line 433 of file mdl_extract_hits.c.

References internal_is_ws(), RA8_INTERNAL, strlen(), and strstr().

Referenced by internal_extract_class().

internal_clean_inner_text()

bool internal_clean_inner_text ( const char * text, size_t len, char * out, size_t cap )

static

Copy the anchor inner text from text into out, cleaned for display.

Strips nested <...> tags, collapses runs of whitespace to one space, decodes the common HTML entities, and trims the ends.

Parameters

[in]	text	Source HTML fragment; need not be NUL-terminated.
[in]	len	Number of readable bytes at text.
[out]	out	Destination for cleaned display text.
[in]	cap	Destination capacity including NUL.

Returns

Whether the complete cleaned text fit.

Return values

true	out contains the complete cleaned text.
false	out contains a NUL-terminated truncated prefix.

Precondition

text and out are non-NULL.

cap is greater than zero and describes writable out storage.

Postcondition

out is always NUL-terminated.

No source bytes are modified.

Note

Thread-safe: uses only caller-owned storage.

Since

0.1.0

Definition at line 136 of file mdl_extract_hits.c.

References internal_append_out_char(), internal_decode_entity(), and internal_is_ws().

Referenced by internal_anchor_title(), internal_extract_class(), internal_extract_label(), and internal_extract_meta().

internal_decode_entity()

bool internal_decode_entity ( const char * s, size_t len, size_t * i, char * out_ch, bool * space )

static

Decode an entity at &, returning its byte and advancing *i.

Returns

true when a known entity was consumed; *space is set when the entity is whitespace-like (&nbsp;) so the caller collapses it.

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[in]	s	Readable text or byte sequence.
[in]	len	Readable byte length.
[in,out]	i	Current bounded scan offset.
[out]	out_ch	Receives a decoded character.
[out]	space	Receives the whitespace classification.

Return values

true	The documented predicate holds or the requested operation completed.
false	The predicate does not hold or validation rejected the operation.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 63 of file mdl_extract_hits.c.

References mdl_entity_t::ch, memcmp(), and strlen().

Referenced by internal_clean_inner_text().

internal_emit_hit()

ra8_err_t internal_emit_hit ( const char * html, size_t html_len, const char * base_url, const char * keep, const char * tag, size_t tag_len, size_t gt_off, mdl_hit_list_t * out, bool * real )

static

Resolve one anchor's href, count it, and emit a filtered hit.

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[in]	html	Readable markup bytes.
[in]	html_len	Readable markup length.
[in]	base_url	NUL-terminated base URL.
[in]	keep	Optional substring filter.
[in]	tag	Readable tag byte span.
[in]	tag_len	Readable tag length.
[in]	gt_off	Offset of the tag-closing greater-than byte.
[out]	out	Caller-owned result storage.
[in,out]	real	Per-hit real-title flags.

Returns

Operation status.

Return values

k_ra8_ok	The operation completed successfully.
other	The originating validation, storage, stream, or network error.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 340 of file mdl_extract_hits.c.

References abs(), mdl_hit_list_t::anchors_seen, internal_anchor_title(), internal_contains_ok(), internal_find_attr_value(), internal_merge_hit(), internal_url_slug(), k_mdl_hit_title_max, k_mdl_url_max, k_ra8_ok, mdl_extract_resolve_url(), and RA8_INTERNAL.

Referenced by mdl_extract_hits().

internal_extract_class()

ra8_err_t internal_extract_class ( const char * html, size_t len, const char * token, char * out, size_t cap )

static

Extract visible text from the first matching class token.

Scans bounded start tags, matches one exact class token, and cleans the following element text without truncation.

Parameters

[in]	html	Bounded HTML bytes.
[in]	len	Number of readable HTML bytes.
[in]	token	Non-empty class token to match.
[out]	out	Destination for cleaned visible text.
[in]	cap	Destination capacity including NUL.

Returns

An ra8_err_t extraction result.

Return values

k_ra8_ok	A non-empty value was extracted.
k_ra8_err_not_found	No matching non-empty element exists.
k_ra8_err_invalid_size	The cleaned value did not fit.

Precondition

html, token, and out are non-NULL.

token is non-empty and len bounds readable html bytes.

Postcondition

On success, out is non-empty and NUL-terminated.

No input bytes are modified.

Note

Thread-safe: uses only caller storage.

Since

0.1.0

Definition at line 549 of file mdl_extract_hits.c.

References internal_class_has_token(), internal_clean_inner_text(), internal_find_attr_value(), internal_find_ci(), k_mdl_hit_title_max, k_ra8_err_invalid_size, k_ra8_err_not_found, k_ra8_ok, and memchr().

Referenced by mdl_extract_selector().

internal_extract_label()

ra8_err_t internal_extract_label ( const char * html, size_t len, const char * label, char * out, size_t cap )

static

Extract the first anchor text following a visible label.

Finds label case-insensitively, locates the next anchor, and cleans its bounded inner text without truncation.

Parameters

[in]	html	Bounded HTML bytes.
[in]	len	Number of readable HTML bytes.
[in]	label	Non-empty visible label to find.
[out]	out	Destination for cleaned anchor text.
[in]	cap	Destination capacity including NUL.

Returns

An ra8_err_t extraction result.

Return values

k_ra8_ok	A non-empty anchor value was extracted.
k_ra8_err_not_found	The label/anchor/value was absent.
k_ra8_err_invalid_size	The cleaned value did not fit.

Precondition

html, label, and out are non-NULL.

label is NUL-terminated and len bounds readable html bytes.

Postcondition

On success, out is non-empty and NUL-terminated.

No input bytes are modified.

Note

Thread-safe: uses only caller storage.

Since

0.1.0

Definition at line 601 of file mdl_extract_hits.c.

References internal_clean_inner_text(), internal_find_ci(), k_ra8_err_invalid_size, k_ra8_err_not_found, k_ra8_ok, memchr(), and strlen().

Referenced by mdl_extract_selector().

internal_extract_meta()

ra8_err_t internal_extract_meta ( const char * html, size_t len, const char * key, char * out, size_t cap )

static

Extract one matching HTML meta-content value.

Scans bounded markup for a property or name equal to key, then cleans its content value without truncation.

Parameters

[in]	html	Bounded HTML bytes.
[in]	len	Number of readable HTML bytes.
[in]	key	Meta property/name to match.
[out]	out	Destination for cleaned content.
[in]	cap	Destination capacity including NUL.

Returns

An ra8_err_t extraction result.

Return values

k_ra8_ok	A non-empty value was extracted.
k_ra8_err_not_found	No matching non-empty meta value exists.
k_ra8_err_invalid_size	The cleaned value did not fit.

Precondition

html, key, and out are non-NULL.

cap is greater than zero and len bounds readable html bytes.

Postcondition

On success, out is non-empty and NUL-terminated.

No input bytes are modified.

Note

Thread-safe: uses only caller storage.

Since

0.1.0

Definition at line 498 of file mdl_extract_hits.c.

References internal_clean_inner_text(), internal_find_attr_value(), internal_find_ci(), k_mdl_hit_title_max, k_mdl_url_max, k_ra8_err_invalid_size, k_ra8_err_not_found, k_ra8_ok, memchr(), strcmp(), and strlen().

Referenced by mdl_extract_selector().

internal_hit_index_of()

size_t internal_hit_index_of ( const mdl_hit_list_t * out, const char * url )

static

Index of an existing hit with URL url, or count when absent.

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[in]	out	Existing bounded hit list.
[in]	url	NUL-terminated URL input.

Returns

Index of url, or out->count when it is absent.

Return values

out->count	No existing hit has the requested URL.
other	Zero-based index of the matching hit.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 265 of file mdl_extract_hits.c.

References mdl_hit_list_t::count, mdl_hit_list_t::hits, RA8_INTERNAL, strcmp(), and mdl_hit_t::url.

Referenced by internal_merge_hit().

internal_is_ws()

bool internal_is_ws ( char c )

static

ASCII whitespace test (locale-independent).

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[in]

c

Input ASCII byte.

Returns

True when c is supported ASCII whitespace.

Return values

true	The documented predicate holds or the requested operation completed.
false	The predicate does not hold or validation rejected the operation.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 36 of file mdl_extract_hits.c.

References RA8_INTERNAL.

Referenced by internal_class_has_token(), and internal_clean_inner_text().

internal_merge_hit()

ra8_err_t internal_merge_hit ( mdl_hit_list_t * out, bool * real, const char * url, const char * title, bool title_real )

static

Merge one resolved (url,title) hit; upgrade a slug with a real title.

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[out]	out	Caller-owned result storage.
[in,out]	real	Per-hit real-title flags.
[in]	url	NUL-terminated URL input.
[in]	title	NUL-terminated display title.
[in]	title_real	Whether the title came from markup rather than a slug.

Returns

Operation status.

Return values

k_ra8_ok	The operation completed successfully.
other	The originating validation, storage, stream, or network error.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 294 of file mdl_extract_hits.c.

References mdl_hit_list_t::count, mdl_hit_list_t::hits, internal_copy_fits(), internal_hit_index_of(), k_mdl_hit_title_max, k_mdl_max_hits, k_mdl_url_max, k_ra8_err_no_mem, k_ra8_ok, RA8_INTERNAL, mdl_hit_t::title, and mdl_hit_t::url.

Referenced by internal_emit_hit().

internal_selector_copy()

ra8_err_t internal_selector_copy ( const char * value, char * out, size_t cap )

static

Copy one non-empty literal selector value.

Rejects empty and over-capacity values without reporting success.

Parameters

[in]	value	NUL-terminated literal value.
[out]	out	Destination character buffer.
[in]	cap	Destination capacity including NUL.

Returns

An ra8_err_t extraction result.

Return values

k_ra8_ok	The complete value was copied.
k_ra8_err_not_found	The value was empty.
k_ra8_err_invalid_size	The complete value did not fit.

Precondition

value and out are non-NULL.

cap is greater than zero and describes writable out storage.

Postcondition

On success, out is non-empty and NUL-terminated.

value is unchanged.

Note

Thread-safe: uses only caller storage.

Since

0.1.0

Definition at line 466 of file mdl_extract_hits.c.

References internal_copy_fits(), k_ra8_err_invalid_size, k_ra8_err_not_found, k_ra8_ok, and RA8_INTERNAL.

Referenced by mdl_extract_selector().

internal_url_slug()

void internal_url_slug ( const char * url, char * out, size_t cap )

static

Fill out with the URL's last non-empty path segment (slug fallback).

Scans bounded anchor bytes and merges duplicate resolved URLs in place. Only complete titles and URLs fitting caller result storage publish.

Parameters

[in]	url	NUL-terminated URL input.
[out]	out	Caller-owned result storage.
[in]	cap	Destination capacity including any terminator.

Precondition

Every required pointer is non-null and remains valid for the call.

Lengths and capacities describe complete referenced objects without overflow.

Postcondition

Documented outputs and the return value describe the same outcome.

A rejected or failed operation is never reported as successful.

Note

Thread safety follows ownership of the supplied context; no synchronization is added.

Since

Version 0.1.0

Definition at line 185 of file mdl_extract_hits.c.

References RA8_INTERNAL, and strlen().

Referenced by internal_emit_hit().

mdl_extract_hits()

ra8_err_t mdl_extract_hits	(	const char *	html,
		size_t	html_len,
		const char *	base_url,
		const char *	url_contains,
		mdl_hit_list_t *	out )

Scan html for <a href> links, keeping each hit's title and URL.

The discovery counterpart to mdl_extract_anchors: instead of a bare URL list it yields (title, URL) pairs suitable for a numbered search/browse listing. For every <a> whose resolved absolute href contains url_contains it records the URL and a best-effort title – the anchor's title= attribute, else its inner text with nested tags stripped, whitespace collapsed and the common HTML entities decoded, else the URL's last path segment. Duplicate URLs are merged, and a later occurrence carrying a real title upgrades an earlier slug-only fallback (a results card is often a thumbnail link followed by a titled text link to the same series). The total number of resolvable anchors scanned – before the filter – is reported in out->anchors_seen so the caller can distinguish "no match" from "no links".

Parameters

[in]	html	HTML bytes (need not be NUL-terminated).
[in]	html_len	Length of html in bytes.
[in]	base_url	Absolute URL of the page (for relative resolution).
[in]	url_contains	If non-NULL and non-empty, keep only hits whose absolute URL contains this substring.
[out]	out	List to fill; out->count and out->anchors_seen are reset first.

Returns

An ra8_err_t scan result.

Return values

k_ra8_ok	Scan complete (count may be 0).
k_ra8_err_invalid_arg	A NULL html, base_url or out.
k_ra8_err_no_mem	Reached k_mdl_max_hits; the rest were skipped but anchors_seen still counts them.

Precondition

html, base_url and out are non-NULL.

out points to writable mdl_hit_list_t storage.

Postcondition

out->count hits are unique by URL and each has a non-empty title.

out->anchors_seen >= out->count.

Note

Not thread-safe: writes caller storage.

See also

mdl_extract_anchors

mdl_search_classify

Since

0.1.0

Definition at line 372 of file mdl_extract_hits.c.

References mdl_hit_list_t::anchors_seen, mdl_hit_list_t::count, internal_emit_hit(), internal_find_ci(), internal_is_name_end(), k_mdl_max_hits, k_ra8_err_invalid_arg, k_ra8_err_no_mem, k_ra8_ok, memchr(), and memset().

Referenced by mdl_discover_run().

mdl_extract_selector()

ra8_err_t mdl_extract_selector	(	const char *	html,
		size_t	html_len,
		const char *	selector,
		char *	out,
		size_t	out_cap )

Extract one bounded metadata value using a site-descriptor selector.

The deliberately small selector grammar is data-driven and portable: meta:og:title reads a matching <meta property|name> content value, class:post-title cleans the matching element's visible text, label:Author(s): cleans the next anchor's text, and literal:en copies a descriptor-provided constant. Markup is scanned without allocation.

Parameters

[in]	html	HTML bytes; need not be NUL-terminated.
[in]	html_len	Number of readable bytes at html.
[in]	selector	Selector in the grammar above.
[out]	out	Destination for cleaned UTF-8 bytes.
[in]	out_cap	Capacity of out, including the terminator.

Returns

An ra8_err_t extraction result.

Return values

k_ra8_ok	A non-empty value was written.
k_ra8_err_not_found	No matching non-empty value exists.
k_ra8_err_invalid_arg	A pointer was NULL, capacity was zero, or the selector was invalid.
k_ra8_err_invalid_size	The extracted value did not fit completely.

Precondition

html, selector, and out are non-NULL.

out points to out_cap writable bytes.

Postcondition

On success, out is NUL-terminated and non-empty.

On failure, out[0] is the NUL byte when out is writable.

Note

Thread-safe: uses caller-owned storage only.

Since

0.1.0

Definition at line 627 of file mdl_extract_hits.c.

References internal_extract_class(), internal_extract_label(), internal_extract_meta(), internal_selector_copy(), k_ra8_err_invalid_arg, strchr(), and strncmp().

Referenced by internal_extract_cover(), internal_extract_optional_metadata(), internal_extract_series_metadata(), internal_mdl_fetch_select_chapter_number(), and internal_mdl_fetch_select_chapter_title().