src/site/php/mb sanitize.php

Functions

cfw_mb_sanitize()

cfw_mb_sanitize(mixed  $s, string  $sub = '?') : mixed

Replaces every ill-formed UTF-8 sequence with "?".

Matches native mbstring byte for byte, and the rule is NOT one "?" per bad byte -- that was measured, not assumed. Native emits ONE "?" per maximal valid PREFIX it consumed, then resumes at the byte that broke the sequence:

$sub exists for one caller: mb_convert_case TITLE, which needs the substituted run to be a case-IGNORABLE character rather than "?" while the word boundaries are found.

"abc\xe4\xbddef" -> abc?def one "?", two bytes consumed "abc\xe4\xbd" -> abc? same at end of string "abc\xed\xa0\x80def" -> abc???def ED is a valid lead but A0 is out of its range, so ED alone is one "?" and A0 and 80 are then lone continuations "abc\xc0\xafdef" -> abc??def C0 is never a valid lead "abc\xf5\x80\x80\x80def" -> abc????def

A first version advanced one byte at a time on failure and got the truncated cases wrong (abc?? for abc?). The oracle is native mb_substr(); the gate test pins every case above against it.

Parameters

mixed $s

The string to clean; anything else is returned as it came.

string $sub

What replaces each ill-formed run.

Returns

mixed —

The cleaned string, or the input when it was not a string.