TextEncoder encodes valid UTF-8 (or WTF-8) byte sequences to non-UTF-8 encodings, in accordance with the WHATWG encoding standard.
This is a low-level interface; you may also be interested in encoder, which provides useful wrapper procedures.
To encode a stream, sequentially call encode on any number of chunks with finish = false, then with finish = true on the last chunk. If you don't know which chunk is the last, use an empty chunk.
(finish only has a significance for the ISO-2022-JP encoder, which is specified to emit a sequence at the end of the queue to reset the decoder state to ASCII.)
The input stream must be valid UTF-8, with the exception that (invalid) surrogate codepoints are automagically replaced with replacement characters. So if std/unicode's validateUtf8 returns -1 on a buffer, it is safe to feed it to TextEncoder.
encode expects an input queue iq, an output queue oq, and an index in the output queue n. Output is placed in the output queue starting from n. encode may either return terDone, terReqOutput, or terError.
terDone means the entire input buffer has been successfully encoded; the output can be found in oq, and the number of bytes output is stored in n.
terReqOutput is a request for more output space to decode the current buffer. Users may handle this by processing the contents of oq until n, resetting n to 0, then calling encode on the same input queue iq again.
terError represents an error with a specific code point, as specified by the standard. Per spec, implementations should handle this either by immediately aborting the encoding process (error mode "fatal"), or outputting a decimal representation of the code point stored in the TextEncoder object's c member as an HTML reference (error mode "html"). For an example, see encoder's source code.
Types
Iso2022JPState = enum i2jsAscii, i2jsRoman, i2jsJis0208
TextEncoder = object i*: int c*: uint32 charset*: Charset state*: Iso2022JPState
TextEncoderResult = enum terDone, terReqOutput, terError
Procs
proc encode(te: var TextEncoder; iq: openArray[uint8]; oq: var openArray[uint8]; n: var int; finish = false): TextEncoderResult {....raises: [], tags: [], forbids: [].}