Cross-TU seam between the RAR5 entropy front-end and the LZ decoder. More...

#include <stddef.h>
#include <stdint.h>
#include "ra8_attributes.h"
#include "ra8_err.h"
#include "ra8_rar5.h"

Include dependency graph for ra8_rar5_internal.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures
struct	r5_block_t
	Decoded fields of one RAR5 compressed-block header. More...

Enumerations
enum	r5_mask_t : uint32_t { k_r5_bf_mask = 0xFFFEU , k_r5_nibble_mask = 0x0FU , k_r5_byte_mask = 0xFFU }
	Wide bit masks the decoder applies to accumulated fields. More...
enum	r5_bits_t : uint8_t { k_r5_byte_bits = 8U , k_r5_bf_bits = 16U , k_r5_maxbits = 15U , k_r5_len_escape = 15U , k_r5_low3_mask = 7U , k_r5_acc_bits = 64U }
	Fixed field widths and code-length limits of the RAR5 bitstream. More...
enum	r5_blockflag_t : uint8_t { k_r5_bf_bitsize_mask = 0x07U , k_r5_bf_bcount_shift = 3U , k_r5_bf_bcount_mask = 3U , k_r5_bf_last = 0x40U , k_r5_bf_tables = 0x80U , k_r5_hdr_chk_seed = 0x5AU }
	RAR5 compressed-block header flag layout. More...
enum	r5_mainsym_t : uint16_t { k_r5_sym_filter = 256U , k_r5_sym_replast = 257U , k_r5_sym_repdist0 = 258U , k_r5_sym_lenbase = 262U }
	Reserved main-alphabet symbol boundaries above the 256 literals. More...
enum	r5_tblcode_t : uint8_t { k_r5_tbl_copy_long = 17U , k_r5_tbl_zero_short = 18U , k_r5_tbl_zero_long = 19U , k_r5_run_long_bits = 7U , k_r5_run_long_add = 11U , k_r5_zeros_extra = 2U }
	Continuation codes and run bases of the length-table encoding. More...
enum	r5_distth_t : uint32_t { k_r5_dist_th1 = 0x100U , k_r5_dist_th2 = 0x2000U , k_r5_dist_th3 = 0x40000U }
	Distance thresholds that add 1..3 to a decoded match length. More...
enum	r5_filter_t : uint8_t { k_r5_ftype_bits = 3U , k_r5_fchan_bits = 5U }
	Field widths of the RAR5 in-stream filter descriptor. More...
enum	r5_filterop_t : uint8_t { k_r5_x86_call = 0xE8U , k_r5_x86_jmp = 0xE9U , k_r5_x86_ilen = 5U , k_r5_arm_bl = 0xEBU }
	Opcode bytes and instruction widths of the executable filters. More...
enum	r5_armmask_t : uint32_t { k_r5_arm_off_mask = 0xFFFFFFU }
	The 24-bit branch-offset mask of the ARM filter. More...
enum	r5_pad_t : uint16_t { k_r5_max_pad_bits = 64U }
	Slack, past the packed length, tolerated while draining the bit stream. More...

Functions
uint32_t	priv_rar5_get (ra8_rar5_state_t *st, uint32_t n)
	Peek and consume `n` bits from the streaming reader in one step.
uint32_t	priv_rar5_decode_num (ra8_rar5_state_t st, const ra8_rar5_dtab_t d)
	Decode one Huffman symbol from `d`, consuming its code bits.
ra8_err_t	priv_rar5_read_block_header (ra8_rar5_state_t st, r5_block_t b)
	Read and validate one RAR5 block header at the current bit position.
ra8_err_t	priv_rar5_read_tables (ra8_rar5_state_t *st)
	Parse a table block: build the BD pre-table then the four LZ tables.
bool	priv_rar5_copy_match (uint8_t out, size_t out_pos, size_t unp, uint32_t length, uint64_t dist)
	Copy an LZ match of `length` bytes at back-distance `dist` into `out`.
void	priv_rar5_filter_delta (ra8_rar5_state_t st, uint8_t d, uint32_t len, uint32_t channels)
	Apply the per-channel byte-delta filter over `d`.
uint32_t	priv_rar5_fill_zeros (uint8_t *out, uint32_t start, uint32_t count, uint32_t max)
	Extend `out` with `count` zero bit-lengths, bounded by `max`.
ra8_err_t	priv_rar5_apply_run (ra8_rar5_state_t st, uint8_t tbl, uint32_t *idx, uint32_t num)
	Append one run (copy-previous or zero) to the length table.

Detailed Description

Cross-TU seam between the RAR5 entropy front-end and the LZ decoder.

Tag: [Ring 4 / Domain] {World: NS}

The clean-room RAR 5.0 decompressor is split across two translation units so neither exceeds the file-size maintainability cap:

ra8_rar5_tables.c – the entropy front-end: the streaming MSB-first bit reader, the canonical-Huffman decode-table builder, and the per-block header plus length-table parser.
ra8_rar5.c – the LZ decoder driver: the length/distance token decode, the post-decode data filters, the block/stream loop, and the public entry ra8_rar5_decompress.

The dependency is one-directional: the driver calls into the front-end for a raw bit read (priv_rar5_get), one Huffman symbol (priv_rar5_decode_num), a block header (priv_rar5_read_block_header), and a table block (priv_rar5_read_tables). The front-end never calls back into the driver. This header is the entire contract between them plus the shared bit-field / symbol / filter enum vocabulary and the decoded block-header struct; it is not part of the public ra8_rar5_decompress surface (ra8_rar5.h).

Note: Not thread-safe; the single-threaded reader loop serialises access.

Copyright: Copyright (c) 2026 Brighton Sikarskie SPDX-License-Identifier: MIT

Definition in file ra8_rar5_internal.h.

Enumeration Type Documentation

◆ r5_armmask_t

enum r5_armmask_t : uint32_t

The 24-bit branch-offset mask of the ARM filter.

Since: Version 0.1.0

Enumerator
k_r5_arm_off_mask	ARM BL 24-bit word offset.

Definition at line 157 of file ra8_rar5_internal.h.

◆ r5_bits_t

enum r5_bits_t : uint8_t

Fixed field widths and code-length limits of the RAR5 bitstream.

Since: Version 0.1.0

Enumerator
k_r5_byte_bits	Bits per packed byte.
k_r5_bf_bits	DecodeNumber look-ahead window width.
k_r5_maxbits	Longest Huffman code length.
k_r5_len_escape	4-bit escape in the BD length list.
k_r5_low3_mask	Low three bits (byte-alignment residue).
k_r5_acc_bits	Bit-accumulator width (mask guard).

Definition at line 63 of file ra8_rar5_internal.h.

◆ r5_blockflag_t

enum r5_blockflag_t : uint8_t

RAR5 compressed-block header flag layout.

Bits 0-2 give the last-byte valid-bit count minus one; bits 3-4 the block-size byte count minus one; bit 6 marks the last block in the file; bit 7 marks a block that carries fresh Huffman tables.

Since: Version 0.1.0

Enumerator
k_r5_bf_bitsize_mask	Last-byte valid-bit count minus one.
k_r5_bf_bcount_shift	Shift to the block-size byte count.
k_r5_bf_bcount_mask	Mask of the block-size byte count.
k_r5_bf_last	Last block in the file.
k_r5_bf_tables	Block carries new Huffman tables.
k_r5_hdr_chk_seed	Header-checksum seed constant.

Definition at line 80 of file ra8_rar5_internal.h.

◆ r5_distth_t

enum r5_distth_t : uint32_t

Distance thresholds that add 1..3 to a decoded match length.

Since: Version 0.1.0

Enumerator
k_r5_dist_th1	+1 length above this distance.
k_r5_dist_th2	+1 more above this distance.
k_r5_dist_th3	+1 more above this distance.

Definition at line 124 of file ra8_rar5_internal.h.

◆ r5_filter_t

enum r5_filter_t : uint8_t

Field widths of the RAR5 in-stream filter descriptor.

Since: Version 0.1.0

Enumerator
k_r5_ftype_bits	Filter-type field width.
k_r5_fchan_bits	Delta channel-count field width.

Definition at line 135 of file ra8_rar5_internal.h.

◆ r5_filterop_t

enum r5_filterop_t : uint8_t

Opcode bytes and instruction widths of the executable filters.

Since: Version 0.1.0

Enumerator
k_r5_x86_call	x86 near CALL opcode.
k_r5_x86_jmp	x86 near JMP opcode.
k_r5_x86_ilen	CALL/JMP instruction width.
k_r5_arm_bl	ARM BL opcode byte.

Definition at line 145 of file ra8_rar5_internal.h.

◆ r5_mainsym_t

enum r5_mainsym_t : uint16_t

Reserved main-alphabet symbol boundaries above the 256 literals.

Symbol 256 reads a filter, 257 repeats the last match, 258-261 select a remembered distance, and 262+ are LZ length slots.

Since: Version 0.1.0

Enumerator
k_r5_sym_filter	Read one data filter.
k_r5_sym_replast	Repeat the last match.
k_r5_sym_repdist0	First remembered distance.
k_r5_sym_lenbase	First LZ length slot.

Definition at line 96 of file ra8_rar5_internal.h.

◆ r5_mask_t

enum r5_mask_t : uint32_t

Wide bit masks the decoder applies to accumulated fields.

The DecodeNumber 16-bit window (low bit cleared), the low-nibble mask for a transmitted bit length, and the single-byte mask.

Since: Version 0.1.0

Enumerator
k_r5_bf_mask	DecodeNumber bit-field, low bit cleared.
k_r5_nibble_mask	Low nibble of a bit-length byte.
k_r5_byte_mask	Single-byte mask.

Definition at line 52 of file ra8_rar5_internal.h.

◆ r5_pad_t

enum r5_pad_t : uint16_t

Slack, past the packed length, tolerated while draining the bit stream.

The token loop is bounded by consumed reaching the packed size plus this many padding bits, so a hostile stream cannot spin forever on zero-output symbols.

Since: Version 0.1.0

Enumerator
k_r5_max_pad_bits	Bit slack past the packed member end.

Definition at line 169 of file ra8_rar5_internal.h.

◆ r5_tblcode_t

enum r5_tblcode_t : uint8_t

Continuation codes and run bases of the length-table encoding.

Values 0-15 are literal bit lengths; 16/17 copy the previous length for a short/long run; 18/19 emit a short/long run of zero lengths.

Since: Version 0.1.0

Enumerator
k_r5_tbl_copy_long	Copy previous length, 7-bit run.
k_r5_tbl_zero_short	Zero length, 3-bit run.
k_r5_tbl_zero_long	Zero length, 7-bit run.
k_r5_run_long_bits	Long-run extra-bit width.
k_r5_run_long_add	Long-run length bias.
k_r5_zeros_extra	BD zero-run length bias.

Definition at line 110 of file ra8_rar5_internal.h.

Function Documentation

◆ priv_rar5_apply_run()

ra8_err_t priv_rar5_apply_run	(	ra8_rar5_state_t *	st,
		uint8_t *	tbl,
		uint32_t *	idx,
		uint32_t	num )

Append one run (copy-previous or zero) to the length table.

Reads the run length for the continuation code num and repeats either the previous bit length (codes 16/17) or zero (codes 18/19). Defined in ra8_rar5_tables.c; declared here for direct MC/DC test reach.

Parameters

[in,out]	st	Decoder state (non-NULL).
[in,out]	tbl	Length table being filled (non-NULL).
[in,out]	idx	Current fill index; advanced past the run (non-NULL).
[in]	num	Continuation code (16..19).

Returns: ra8_err_t status.

Return values

k_ra8_ok	The run was appended.
k_ra8_err_validation_failed	A copy-previous run with no previous entry.

Precondition: tbl holds k_ra8_rar5_huff_total writable bytes.; *idx <= k_ra8_rar5_huff_total.

Postcondition: *idx advanced by the (clamped) run length.; A copy run repeats tbl[*idx-1]; a zero run writes zeros.

Note: Not thread-safe.

MC/DC:: Promoted from file-static so the host suite can reach the !is_zero && *idx == 0 copy-with-no-previous leg and the i < k_ra8_rar5_huff_total clamp leg of c < count && i < k_ra8_rar5_huff_total directly. The two || decisions (is_long / is_zero) stay covered by the priv_rar5_read_tables round-trip. Production caller remains internal_read_full_table in ra8_rar5_tables.c.

Since: Version 0.1.0

Append one run (copy-previous or zero) to the length table.

Definition at line 357 of file ra8_rar5_tables.c.

References k_r5_run_long_add, k_r5_run_long_bits, k_r5_tbl_copy_long, k_r5_tbl_zero_long, k_r5_tbl_zero_short, k_ra8_err_validation_failed, k_ra8_ok, k_ra8_rar5_huff_total, priv_rar5_get(), and RA8_PRIV.

Referenced by internal_read_full_table().

◆ priv_rar5_copy_match()

bool priv_rar5_copy_match	(	uint8_t *	out,
		size_t *	out_pos,
		size_t	unp,
		uint32_t	length,
		uint64_t	dist )

Copy an LZ match of length bytes at back-distance dist into out.

Byte-by-byte self-overlapping copy (so a short distance repeats), clamped to unp. A distance reaching before the member start (a solid archive) or of zero is rejected. Defined in ra8_rar5.c; declared here so the host tests can drive its guard/clamp legs directly (see the MC/DC note).

Parameters

[in,out]	out	Output/window buffer (non-NULL).
[in,out]	out_pos	Current output length; advanced by the copy (non-NULL).
[in]	unp	Target unpacked length (copy clamp).
[in]	length	Match length in bytes.
[in]	dist	Back-distance in bytes.

Returns: Whether the match was valid and copied.

Return values

true	The match copied (possibly clamped at `unp`).
false	`dist` is zero or reaches before the member start.

Precondition: out holds at least unp writable bytes.; *out_pos <= unp.

Postcondition: On true, *out_pos advanced by up to length.; On false no byte is written.

Note: Not thread-safe.

MC/DC:: Promoted from file-static so the host suite can reach both guard legs of dist == 0 || dist > pos (a valid stream never decodes distance 0, so that condition is only reachable by a direct call) and the pos < unp clamp leg of k < length && pos < unp. Production callers remain the LZ token helpers in ra8_rar5.c.

Since: Version 0.1.0

Copy an LZ match of length bytes at back-distance dist into out.

Definition at line 225 of file ra8_rar5.c.

Referenced by internal_do_match(), internal_do_repdist(), and internal_do_replast().

◆ priv_rar5_decode_num()

uint32_t priv_rar5_decode_num	(	ra8_rar5_state_t *	st,
		const ra8_rar5_dtab_t *	d )

Decode one Huffman symbol from d, consuming its code bits.

Reads the 16-bit look-ahead window, finds the matching bit length by comparing against the upper limits, then indexes the symbol slot. An out-of-range slot (malformed stream) clamps to slot 0 rather than reading out of bounds. The LZ driver uses it for the main / distance / low-distance / repeat-length alphabets.

Parameters

[in,out]	st	Decoder state (non-NULL).
[in]	d	Decode table built by the front-end (non-NULL).

Returns: The decoded symbol.

Return values

0	On a clamped (out-of-range) or genuinely-zero code.

Precondition: st is a bound decoder state.; d was built by the front-end table builder.

Postcondition: st->consumed advanced by the code's bit length (>= 1).; The returned symbol is < d->max.

Note: Not thread-safe.

See also: priv_rar5_get()

Since: Version 0.1.0

Decode one Huffman symbol from d, consuming its code bits.

Definition at line 241 of file ra8_rar5_tables.c.

References internal_drop(), internal_peek(), k_r5_bf_bits, k_r5_bf_mask, k_r5_maxbits, ra8_rar5_dtab_t::len, ra8_rar5_dtab_t::max, ra8_rar5_dtab_t::num, ra8_rar5_dtab_t::pos, and RA8_PRIV.

Referenced by internal_decode_distance(), internal_decode_token(), internal_do_repdist(), and internal_read_full_table().

◆ priv_rar5_fill_zeros()

uint32_t priv_rar5_fill_zeros	(	uint8_t *	out,
		uint32_t	start,
		uint32_t	count,
		uint32_t	max )

Extend out with count zero bit-lengths, bounded by max.

Appends up to count zero entries from start, stopping at max so a hostile run length can never overflow the array. Defined in ra8_rar5_tables.c; declared here for direct MC/DC test reach.

Parameters

[in,out]	out	Bit-length array being filled.
[in]	start	First index to write.
[in]	count	Zero entries to append.
[in]	max	Array capacity (no write at/after it).

Returns: The next write index after the appended zeros.

Return values

start When start is already at max.

Precondition: out holds max writable bytes.; start <= max.

Postcondition: out[start .. min(start+count, max)) are zero.; The return value is <= max.

Note: Not thread-safe.

MC/DC:: Promoted from file-static so the host suite can reach the i < max clamp leg of c < count && i < max directly, without crafting a BD length list whose zero run happens to overshoot the 20-entry array. Production caller remains internal_read_bd_lengths in ra8_rar5_tables.c.

Since: Version 0.1.0

Extend out with count zero bit-lengths, bounded by max.

Definition at line 310 of file ra8_rar5_tables.c.

References RA8_PRIV.

Referenced by internal_read_bd_lengths().

◆ priv_rar5_filter_delta()

void priv_rar5_filter_delta	(	ra8_rar5_state_t *	st,
		uint8_t *	d,
		uint32_t	len,
		uint32_t	channels )

Apply the per-channel byte-delta filter over d.

De-interleaves channels streams and runs a running byte-sum per channel. Bounded by k_ra8_rar5_delta_scratch; a longer range or a zero channel count is left untouched (comic pages never carry a delta filter). Defined in ra8_rar5.c; declared here for direct MC/DC test reach.

Parameters

[in,out]	st	Decoder state (delta scratch, non-NULL).
[in,out]	d	Output range to transform (non-NULL).
[in]	len	Range length in bytes.
[in]	channels	Delta channel count (1..32 from a stream; 0 rejected).

Returns: Nothing.

Precondition: d holds len writable bytes.; st::delta holds k_ra8_rar5_delta_scratch bytes.

Postcondition: On a supported length, d holds the delta-decoded range.; On an over-long range or zero channels, d is unchanged.

Note: Not thread-safe.

MC/DC:: Promoted from file-static so the host suite can reach the channels == 0 leg of len > k_ra8_rar5_delta_scratch || channels == 0: the descriptor parser only ever yields channels >= 1, so a zero channel count is unreachable through the public decode path and can only be exercised by a direct call. Production callers remain internal_apply_one_filter in ra8_rar5.c.

Since: Version 0.1.0

Apply the per-channel byte-delta filter over d.

Definition at line 307 of file ra8_rar5.c.

References k_ra8_rar5_delta_scratch, and memcpy().

Referenced by internal_apply_one_filter().

◆ priv_rar5_get()

uint32_t priv_rar5_get	(	ra8_rar5_state_t *	st,
		uint32_t	n )

Peek and consume n bits from the streaming reader in one step.

The front-end's raw bit read: ensures the bits are buffered, returns them oldest-bit-first, and advances the consumed-bit total. The LZ driver uses it to read length/distance extra bits and filter fields.

Parameters

[in,out]	st	Decoder state (non-NULL).
[in]	n	Bits to read (1..32).

Returns: The consumed n bits, oldest bit most significant.

Return values

0	When the bits are all zero (or all past-end padding).

Precondition: st is a bound decoder state.; 1 <= n <= 32.

Postcondition: st->consumed increased by n.; st->nbits decreased by n.

Note: Not thread-safe.

See also: priv_rar5_decode_num()

Since: Version 0.1.0

Peek and consume n bits from the streaming reader in one step.

Definition at line 145 of file ra8_rar5_tables.c.

References internal_drop(), internal_peek(), and RA8_PRIV.

Referenced by internal_decode_distance(), internal_read_bd_lengths(), internal_read_filter(), internal_read_filter_data(), internal_slot_to_length(), priv_rar5_apply_run(), and priv_rar5_read_block_header().

◆ priv_rar5_read_block_header()

ra8_err_t priv_rar5_read_block_header	(	ra8_rar5_state_t *	st,
		r5_block_t *	b )

Read and validate one RAR5 block header at the current bit position.

Byte-aligns, reads the flags, the variable-width block size, and the checksum byte, rejecting a header whose checksum does not match. The driver's block loop calls it once per compressed block.

Parameters

[in,out]	st	Decoder state (non-NULL).
[out]	b	Receives the decoded block fields (non-NULL).

Returns: ra8_err_t status.

Return values

k_ra8_ok	Header decoded and checksum valid.
k_ra8_err_validation_failed	The checksum byte did not match.

Precondition: st is a bound decoder state.; b is writable.

Postcondition: On k_ra8_ok, b holds the block size, table flag, and last flag.; st->consumed advanced past the whole header.

Note: Not thread-safe.

See also: priv_rar5_read_tables()

Since: Version 0.1.0

Read and validate one RAR5 block header at the current bit position.

Definition at line 287 of file ra8_rar5_tables.c.

References internal_align(), internal_checksum(), k_r5_bf_bcount_mask, k_r5_bf_bcount_shift, k_r5_bf_bitsize_mask, k_r5_bf_last, k_r5_bf_tables, k_r5_byte_bits, k_ra8_err_validation_failed, k_ra8_ok, priv_rar5_get(), and RA8_PRIV.

Referenced by internal_open_block().

◆ priv_rar5_read_tables()

ra8_err_t priv_rar5_read_tables ( ra8_rar5_state_t * st )

Parse a table block: build the BD pre-table then the four LZ tables.

Reads the BD lengths, builds the BD decode table, decodes the combined length table, and splits it into the main / distance / low-distance / repeat-length decode tables. The driver calls it whenever a block header flags fresh tables.

Parameters

[in,out] st Decoder state (non-NULL).

Returns: ra8_err_t status.

Return values

k_ra8_ok	All five tables were built.
k_ra8_err_validation_failed	A malformed length table.

Precondition: st is a bound decoder state at a table block.; st scratch tables are writable.

Postcondition: On k_ra8_ok, st->tables_ready is true.; The four LZ decode tables are usable by priv_rar5_decode_num.

Note: Not thread-safe.

See also: priv_rar5_read_block_header()

Since: Version 0.1.0

Parse a table block: build the BD pre-table then the four LZ tables.

Definition at line 419 of file ra8_rar5_tables.c.

References internal_make_tables(), internal_read_bd_lengths(), internal_read_full_table(), k_ra8_ok, k_ra8_rar5_bc, k_ra8_rar5_dc, k_ra8_rar5_huff_total, k_ra8_rar5_ldc, k_ra8_rar5_nc, k_ra8_rar5_rc, and RA8_PRIV.

Referenced by internal_open_block().

Data Structures

Enumerations

Functions

Detailed Description

Enumeration Type Documentation

◆ r5_armmask_t

◆ r5_bits_t

◆ r5_blockflag_t

◆ r5_distth_t

◆ r5_filter_t

◆ r5_filterop_t

◆ r5_mainsym_t

◆ r5_mask_t

◆ r5_pad_t

◆ r5_tblcode_t

Function Documentation

◆ priv_rar5_apply_run()

◆ priv_rar5_copy_match()

◆ priv_rar5_decode_num()

◆ priv_rar5_fill_zeros()

◆ priv_rar5_filter_delta()

◆ priv_rar5_get()

◆ priv_rar5_read_block_header()

◆ priv_rar5_read_tables()