mdl_fetch_ctx_t Struct Reference

Everything mdl_fetch_run needs, injected for testability. More...

#include <mdl_fetch.h>

Collaboration diagram for mdl_fetch_ctx_t:

Data Fields
mdl_session_t *	session
	Identity + robots + net backend.
mdl_storage_t *	storage
	Portable storage binding + workspaces.
mdl_state_t *	state
	Persistent state, read and updated.
mdl_cache_t *	cache
	Per-host HTML cache binding.
const char *	state_path
	Atomic checkpoint target, or NULL.
const char *	series_abs_dir
	Absolute series dir (paths resolve here).
const char *	series_url
	Series URL, sent as the chapter Referer.
const mdl_site_t *	site
	Selectors + politeness bounds.
mdl_governor_t *	gov
	Per-host rate/backoff governor, or NULL.
uint32_t	timeout_ms
	Per-request time budget.
char *	page_buf
	Chapter-HTML scratch (caller-owned).
size_t	page_cap
	Capacity of page_buf.
mdl_url_list_t *	images
	Extracted-image scratch (caller-owned).
bool	update_only
	Skip chapters already recorded complete.
bool	refetch
	Ignore local page reuse and hit network.
mdl_fetch_faillog_t *	faillog
	Caller-cleared failure log, or NULL.
ra8_io_stream_t *	diagnostic
	Borrowed diagnostic sink.
mdl_progress_fn	progress_fn
	Per-page progress sink, or NULL (silent).
void *	progress_ctx
	Context passed to progress_fn.
ra8_err_t	progress_error
	First callback failure from this run.

Detailed Description

Everything mdl_fetch_run needs, injected for testability.

The network is reached only through session (a ::mdl_net vtable behind it), and state is the injected mdl_state_t, so a test drives the whole loop against a scripted fake backend and a temp directory.

Invariant

page_buf/page_cap describe a scratch buffer large enough for a chapter HTML page.

series_abs_dir is an existing, resolved absolute directory.

See also

mdl_fetch_run

Since

0.1.0

Definition at line 164 of file mdl_fetch.h.

Field Documentation

cache

mdl_cache_t* mdl_fetch_ctx_t::cache

Per-host HTML cache binding.

Definition at line 168 of file mdl_fetch.h.

Referenced by internal_cover_cached(), internal_mdl_fetch_chapter_html(), and priv_mdl_app_ensure_series_cover().

diagnostic

ra8_io_stream_t* mdl_fetch_ctx_t::diagnostic

Borrowed diagnostic sink.

Definition at line 181 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_diag3(), internal_mdl_fetch_select_chapter_number(), and internal_mdl_fetch_select_chapter_title().

faillog

mdl_fetch_faillog_t* mdl_fetch_ctx_t::faillog

Caller-cleared failure log, or NULL.

Definition at line 180 of file mdl_fetch.h.

Referenced by priv_mdl_fetch_record_fail().

gov

mdl_governor_t* mdl_fetch_ctx_t::gov

Per-host rate/backoff governor, or NULL.

Definition at line 173 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_governed_get_body(), and priv_mdl_fetch_cache_get_buf().

images

mdl_url_list_t* mdl_fetch_ctx_t::images

Extracted-image scratch (caller-owned).

Definition at line 177 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_chapter_html(), internal_mdl_fetch_chapter_pages_and_checkpoint(), internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_emit_progress(), internal_mdl_fetch_mark_complete(), internal_mdl_fetch_one_page(), and priv_mdl_fetch_chapter_pages().

page_buf

char* mdl_fetch_ctx_t::page_buf

Chapter-HTML scratch (caller-owned).

Definition at line 175 of file mdl_fetch.h.

Referenced by internal_cover_cached(), internal_cover_publish_cached(), internal_mdl_fetch_chapter_html(), internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_select_chapter_number(), and internal_mdl_fetch_select_chapter_title().

page_cap

size_t mdl_fetch_ctx_t::page_cap

Capacity of page_buf.

Definition at line 176 of file mdl_fetch.h.

Referenced by internal_cover_cached(), and internal_mdl_fetch_chapter_html().

progress_ctx

void* mdl_fetch_ctx_t::progress_ctx

Context passed to progress_fn.

Definition at line 183 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_emit_progress().

progress_error

ra8_err_t mdl_fetch_ctx_t::progress_error

First callback failure from this run.

Definition at line 184 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_diag3(), internal_mdl_fetch_select_chapter_number(), internal_mdl_fetch_select_chapter_title(), internal_run_prepared(), mdl_fetch_run(), and priv_mdl_fetch_chapter_pages().

progress_fn

mdl_progress_fn mdl_fetch_ctx_t::progress_fn

Per-page progress sink, or NULL (silent).

Definition at line 182 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_emit_progress(), and internal_mdl_fetch_mono_ms().

refetch

bool mdl_fetch_ctx_t::refetch

Ignore local page reuse and hit network.

Definition at line 179 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_one_page(), internal_mdl_fetch_prepare_page(), and priv_mdl_app_ensure_series_cover().

series_abs_dir

const char* mdl_fetch_ctx_t::series_abs_dir

Absolute series dir (paths resolve here).

Definition at line 170 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_resolve_dest(), internal_mdl_fetch_try_reuse(), and mdl_fetch_run().

series_url

const char* mdl_fetch_ctx_t::series_url

Series URL, sent as the chapter Referer.

Definition at line 171 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_chapter_html(), and internal_mdl_fetch_ctx_ready().

session

mdl_session_t* mdl_fetch_ctx_t::session

Identity + robots + net backend.

Definition at line 165 of file mdl_fetch.h.

Referenced by internal_cover_cached(), internal_cover_stream(), internal_mdl_fetch_chapter_html(), internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_governed_get_body(), internal_mdl_fetch_prepare_page(), mdl_fetch_asset(), priv_mdl_app_ensure_series_cover(), and priv_mdl_fetch_cache_get_buf().

site

const mdl_site_t* mdl_fetch_ctx_t::site

Selectors + politeness bounds.

Definition at line 172 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_chapter_html(), internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_prepare_page(), internal_mdl_fetch_select_chapter_number(), internal_mdl_fetch_select_chapter_title(), mdl_fetch_asset(), and priv_mdl_app_ensure_series_cover().

state

mdl_state_t* mdl_fetch_ctx_t::state

Persistent state, read and updated.

Definition at line 167 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_one_page(), internal_mdl_fetch_prepare_page(), internal_mdl_fetch_process_chapter(), internal_mdl_fetch_publish_page(), internal_mdl_fetch_resolve_not_modified(), internal_mdl_fetch_try_reuse(), and priv_mdl_fetch_checkpoint().

state_path

const char* mdl_fetch_ctx_t::state_path

Atomic checkpoint target, or NULL.

Definition at line 169 of file mdl_fetch.h.

Referenced by priv_mdl_fetch_checkpoint().

storage

mdl_storage_t* mdl_fetch_ctx_t::storage

Portable storage binding + workspaces.

Definition at line 166 of file mdl_fetch.h.

Referenced by internal_cover_current(), internal_cover_publish_cached(), internal_cover_stream(), internal_fetch_asset_execute(), internal_mdl_fetch_ctx_ready(), internal_mdl_fetch_prepare_page(), internal_mdl_fetch_publish_page(), internal_mdl_fetch_resolve_dest(), internal_mdl_fetch_try_reuse(), mdl_fetch_run(), and priv_mdl_fetch_checkpoint().

timeout_ms

uint32_t mdl_fetch_ctx_t::timeout_ms

Per-request time budget.

Definition at line 174 of file mdl_fetch.h.

Referenced by internal_cover_cached(), internal_cover_stream(), internal_mdl_fetch_chapter_html(), internal_mdl_fetch_prepare_page(), and mdl_fetch_asset().

update_only

bool mdl_fetch_ctx_t::update_only

Skip chapters already recorded complete.

Definition at line 178 of file mdl_fetch.h.

Referenced by internal_mdl_fetch_process_chapter().

---

The documentation for this struct was generated from the following file:

• apps/shared_libs/mdl/inc/mdl_fetch.h