mdl_discover.h File Reference

Search / browse discovery: governed fetch of a results page, parse, honest presentation, and optional select-for-download. More...

#include <stddef.h>
#include <stdint.h>
#include "mdl_config.h"
#include "mdl_extract.h"
#include "mdl_politeness.h"
#include "mdl_session.h"

Include dependency graph for mdl_discover.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures
struct	mdl_discover_req_t
	Injected dependencies and inputs for one discovery run. More...

Enumerations
enum	mdl_discover_mode_t : uint8_t {   k_mdl_discover_search = 0 ,   k_mdl_discover_browse = 1 }
	Which discovery endpoint of a site descriptor to query. More...

Functions
int	mdl_discover_run (const mdl_discover_req_t *req, size_t pick, char *out_url, size_t out_cap)
	Run one search/browse discovery: fetch, parse, present, and select.

Detailed Description

Search / browse discovery: governed fetch of a results page, parse, honest presentation, and optional select-for-download.

The network-facing half of #304, kept out of main.c so the entry point stays a thin dispatcher and so the pure parse policy (::mdl_search) is tested on its own. Given an already-built session (honest User-Agent + robots cache) and politeness governor, mdl_discover_run builds the query URL from the site descriptor, gates it against robots.txt, fetches it through the same governor every other request uses (search is never a rate-limit bypass), parses the results with mdl_extract_hits, and prints one of three honest outcomes: a numbered list, a genuine "no results", or "the markup changed".

When a 1-based pick index is supplied it also copies the chosen series URL out for the caller to feed straight into a download – the combined search-and-select flow that removes the copy-paste step.

Copyright

Copyright (c) 2026 Brighton Sikarskie SPDX-License-Identifier: MIT

Definition in file mdl_discover.h.

Enumeration Type Documentation

mdl_discover_mode_t

enum mdl_discover_mode_t : uint8_t

Which discovery endpoint of a site descriptor to query.

See also

mdl_discover_run()

Since

0.1.0

Enumerator
k_mdl_discover_search	search_url with a {q} term substitution.
k_mdl_discover_browse	browse_url latest-updates page, no term.

Definition at line 38 of file mdl_discover.h.

Function Documentation

mdl_discover_run()

int mdl_discover_run	(	const mdl_discover_req_t *	req,
		size_t	pick,
		char *	out_url,
		size_t	out_cap )

Run one search/browse discovery: fetch, parse, present, and select.

Builds the results URL for req->mode from the site descriptor (encoding the term and expanding {q} for search; using browse_url verbatim for browse), refuses when the descriptor supplies no such endpoint, gates the URL against robots.txt, and fetches it through req->gov with bounded governed retries. It then parses the response with mdl_extract_hits and classifies it with mdl_search_classify, printing a distinct message for each honest outcome so an empty result is never dressed up as a successful search. When pick is non-zero it names the pick-th hit (1-based) and copies its series URL into out_url for a follow-on download.

Parameters

[in]	req	Injected dependencies and inputs (never NULL).
[in]	pick	1-based hit to select, or 0 to only list results.
[out]	out_url	Receives the selected series URL, or "" when nothing was selected. May be NULL only when pick is 0.
[in]	out_cap	Capacity of out_url in bytes (>= 1 when out_url set).

Returns

A process exit code.

Return values

0	Listed results (possibly zero) or selected a hit successfully.
1	The request failed, or the page could not be read as results, or a pick was requested with no results to choose from.
2	A capability/usage error: no such discovery endpoint configured, an empty/oversized term, or a pick index out of range.

Precondition

req and its session/gov/site/page_buf/hits are non-NULL.

out_url has room for out_cap bytes when pick is non-zero.

Postcondition

req->hits holds the parsed hits for the run.

*out_url is a selected URL exactly on a ::0 return with pick != 0.

Note

Not thread-safe: mutates the session cache, governor and buffers.

See also

mdl_extract_hits

mdl_search_classify

Since

0.1.0

Definition at line 471 of file mdl_discover.c.

References mdl_site_t::chapter_url_contains, mdl_discover_req_t::diagnostic, mdl_discover_req_t::gov, mdl_discover_req_t::hits, internal_discover_fetch(), internal_discover_text3(), internal_discover_url(), internal_mode_word(), internal_present_and_pick(), mdl_discover_req_t::io_error, k_disc_url_max, k_mdl_reason_max, k_ra8_ok, mdl_extract_hits(), mdl_search_classify(), mdl_search_filter_series_hits(), mdl_discover_req_t::mode, mdl_discover_req_t::output, mdl_discover_req_t::page_buf, priv_mdl_fetch_reason(), priv_mdl_stream_text(), mdl_site_t::search_result_contains, mdl_discover_req_t::session, and mdl_discover_req_t::site.

Referenced by mdl_app_run_discover().