Control Sequence Aware Version of strsplit

A drop-in replacement for base::strsplit.

Usage

strsplit_ctl(
  x,
  split,
  fixed = FALSE,
  perl = FALSE,
  useBytes = FALSE,
  warn = getOption("fansi.warn", TRUE),
  term.cap = getOption("fansi.term.cap", dflt_term_cap()),
  ctl = "all",
  normalize = getOption("fansi.normalize", FALSE),
  carry = getOption("fansi.carry", FALSE),
  terminate = getOption("fansi.terminate", TRUE)
)

Arguments

x

a character vector, or, unlike base::strsplit an object that can be coerced to character.

split

character vector (or object which can be coerced to such) containing regular expression(s) (unless fixed = TRUE) to use for splitting. If empty matches occur, in particular if split has length 0, x is split into single characters. If split has length greater than 1, it is re-cycled along x.

fixed

logical. If TRUE match split exactly, otherwise use regular expressions. Has priority over perl.

perl

logical. Should Perl-compatible regexps be used?

useBytes

logical. If TRUE the matching is done byte-by-byte rather than character-by-character, and inputs with marked encodings are not converted. This is forced (with a warning) if any input is found which is marked as "bytes" (see Encoding).

warn

TRUE (default) or FALSE, whether to warn when potentially problematic Control Sequences are encountered. These could cause the assumptions fansi makes about how strings are rendered on your display to be incorrect, for example by moving the cursor (see ?fansi). At most one warning will be issued per element in each input vector. Will also warn about some badly encoded UTF-8 strings, but a lack of UTF-8 warnings is not a guarantee of correct encoding (use validUTF8 for that).

term.cap

character a vector of the capabilities of the terminal, can be any combination of "bright" (SGR codes 90-97, 100-107), "256" (SGR codes starting with "38;5" or "48;5"), "truecolor" (SGR codes starting with "38;2" or "48;2"), and "all". "all" behaves as it does for the ctl parameter: "all" combined with any other value means all terminal capabilities except that one. fansi will warn if it encounters SGR codes that exceed the terminal capabilities specified (see term_cap_test for details). In versions prior to 1.0, fansi would also skip exceeding SGRs entirely instead of interpreting them. You may add the string "old" to any otherwise valid term.cap spec to restore the pre 1.0 behavior. "old" will not interact with "all" the way other valid values for this parameter do.

ctl

character, which Control Sequences should be treated specially. Special treatment is context dependent, and may include detecting them and/or computing their display/character width as zero. For the SGR subset of the ANSI CSI sequences, and OSC hyperlinks, fansi will also parse, interpret, and reapply the sequences as needed. You can modify whether a Control Sequence is treated specially with the ctl parameter.

"nl": newlines.
"c0": all other "C0" control characters (i.e. 0x01-0x1f, 0x7F), except for newlines and the actual ESC (0x1B) character.
"sgr": ANSI CSI SGR sequences.
"csi": all non-SGR ANSI CSI sequences.
"url": OSC hyperlinks
"osc": all non-OSC-hyperlink OSC sequences.
"esc": all other escape sequences.
"all": all of the above, except when used in combination with any of the above, in which case it means "all but".

normalize

TRUE or FALSE (default) whether SGR sequence should be normalized out such that there is one distinct sequence for each SGR code. normalized strings will occupy more space (e.g. "\033[31;42m" becomes "\033[31m\033[42m"), but will work better with code that assumes each SGR code will be in its own escape as crayon does.

carry

TRUE, FALSE (default), or a scalar string, controls whether to interpret the character vector as a "single document" (TRUE or string) or as independent elements (FALSE). In "single document" mode, active state at the end of an input element is considered active at the beginning of the next vector element, simulating what happens with a document with active state at the end of a line. If FALSE each vector element is interpreted as if there were no active state when it begins. If character, then the active state at the end of the carry string is carried into the first element of x (see "Replacement Functions" for differences there). The carried state is injected in the interstice between an imaginary zeroeth character and the first character of a vector element. See the "Position Semantics" section of substr_ctl and the "State Interactions" section of ?fansi for details. Except for strwrap_ctl where NA is treated as the string "NA", carry will cause NAs in inputs to propagate through the remaining vector elements.

terminate

TRUE (default) or FALSE whether substrings should have active state closed to avoid it bleeding into other strings they may be prepended onto. This does not stop state from carrying if carry = TRUE. See the "State Interactions" section of ?fansi for details.

Value

Like base::strsplit, with Control Sequences excluded.

Details

This function works by computing the position of the split points after removing Control Sequences, and uses those positions in conjunction with substr_ctl to extract the pieces. This concept is borrowed from crayon::col_strsplit. An important implication of this is that you cannot split by Control Sequences that are being treated as Control Sequences. You can however limit which control sequences are treated specially via the ctl parameters (see examples).

Note

The split positions are computed after both x and split are converted to UTF-8.

Non-ASCII strings are converted to and returned in UTF-8 encoding. Width calculations will not work properly in R < 3.2.2.

Control and Special Sequences

Control Sequences are non-printing characters or sequences of characters. Special Sequences are a subset of the Control Sequences, and include CSI SGR sequences which can be used to change rendered appearance of text, and OSC hyperlinks. See fansi for details.

Output Stability

Several factors could affect the exact output produced by fansi functions across versions of fansi, R, and/or across systems. In general it is best not to rely on exact fansi output, e.g. by embedding it in tests.

Width and grapheme calculations depend on Unicode database version (see fansi_unicode_version, and grapheme processing logic among other things (see "Graphemes"). Individual character width are intended to match R4.5.1 definitions in an English locale, except for differences introduced by Unicode Database Version updates and grapheme processing.

How a particular display format is encoded in Control Sequences is not guaranteed to be stable across fansi versions. Additionally, which Special Sequences are re-encoded vs transcribed untouched may change. In general we will strive to keep the rendered appearance stable.

To maximize the odds of getting stable output set normalize_state to TRUE and type to "chars" in functions that allow it, and set term.cap to a specific set of capabilities.

Bidirectional Text

fansi is unaware of text directionality and operates as if all strings are left to right (LTR). Using fansi function with strings that contain mixed direction scripts (i.e. both LTR and RTL) may produce undesirable results.

Examples

strsplit_ctl("\033[31mhello\033[42m world!", " ")
#> [[1]]
#> [1] "\033[31mhello\033[0m"     "\033[31;42mworld!\033[0m"
#> 

## Splitting by newlines does not work as they are _Control
## Sequences_, but we can use `ctl` to treat them as ordinary
strsplit_ctl("\033[31mhello\033[42m\nworld!", "\n")
#> [[1]]
#> [1] "\033[31mhello\033[42m\nworld!"
#> 
strsplit_ctl("\033[31mhello\033[42m\nworld!", "\n", ctl=c("all", "nl"))
#> [[1]]
#> [1] "\033[31mhello\033[0m"     "\033[31;42mworld!\033[0m"
#>