std::codecvt_utf8

From cppreference.com

Defined in header `<codecvt>`

template< class Elem, unsigned long Maxcode = 0x10ffff, std::codecvt_mode Mode = (std::codecvt_mode)0 > class codecvt_utf8 : public std::codecvt<Elem, char, std::mbstate_t>;

std::codecvt_utf8 is a std::codecvt facet which encapsulates conversion between a UTF-8 encoded byte string and UCS2 or UCS4 character string (depending on the type of Elem). This codecvt facet can be used to read and write UTF-8 files, both text and binary.

Inherited from std::codecvt

Member types


Member type	Definition

`intern_type`	`internT`

`extern_type`	`externT`

`state_type`	`stateT`

Member objects


Member name	Type

`id` (static)	std::locale::id

Member functions


out	invokes `do_out` (public member function of `std::codecvt`)

in	invokes `do_in` (public member function of `std::codecvt`)

unshift	invokes `do_unshift` (public member function of `std::codecvt`)

encoding	invokes `do_encoding` (public member function of `std::codecvt`)

always_noconv	invokes `do_always_noconv` (public member function of `std::codecvt`)

length	invokes `do_length` (public member function of `std::codecvt`)

max_length	invokes `do_max_length` (public member function of `std::codecvt`)

Protected member functions


do_out [virtual]	converts a string from internT to externT, such as when writing to file (virtual protected member function of `std::codecvt`)

do_in [virtual]	converts a string from externT to internT, such as when reading from file (virtual protected member function of `std::codecvt`)

do_unshift [virtual]	generates the termination character sequence of externT characters for incomplete conversion (virtual protected member function of `std::codecvt`)

do_encoding [virtual]	returns the number of externT characters necessary to produce one internT character, if constant (virtual protected member function of `std::codecvt`)

do_always_noconv [virtual]	tests if the facet encodes an identity conversion for all valid argument values (virtual protected member function of `std::codecvt`)

do_length [virtual]	calculates the length of the externT string that would be consumed by conversion into given internT buffer (virtual protected member function of `std::codecvt`)

do_max_length [virtual]	returns the maximum number of externT characters that could be converted into a single internT character (virtual protected member function of `std::codecvt`)

Inherited from std::codecvt_base


Member type	Definition

enum result { ok, partial, error, noconv };	Unscoped enumeration type


Enumeration constant	Definition

`ok`	conversion was completed with no error

`partial`	not all source characters were converted

`error`	encountered an invalid character

`noconv`	no conversion required, input and output types are the same

[edit] Example

The following example demonstrates the difference between UCS2/UTF-8 and UTF-16/UTF-8 conversions: the third character in the string is not a valid UCS2 character.

run this code

#include <iostream>
#include <string>
#include <locale>
#include <codecvt>
 
int main()
{
    // UTF-8 data. The character U+1d10b, musical sign segno, does not fit in UCS2
    std::string utf8 = u8"z\u6c34\U0001d10b";
 
    // the UTF-8 / UTF-16 standard conversion facet
    std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> utf16conv;
    std::u16string utf16 = utf16conv.from_bytes(utf8);
    std::cout << "UTF16 conversion produced " << utf16.size() << " code points:\n";
    for (char16_t c : utf16)
        std::cout << std::hex << std::showbase << c << '\n';
 
    // the UTF-8 / UCS2 standard conversion facet
    std::wstring_convert<std::codecvt_utf8<char16_t>, char16_t> ucs2conv;
    try {
        std::u16string ucs2 = ucs2conv.from_bytes(utf8);
    } catch(const std::range_error& e) {
        std::u16string ucs2 = ucs2conv.from_bytes(utf8.substr(0, ucs2conv.converted()));
        std::cout << "UCS2 failed after producing " << std::dec << ucs2.size()<<" characters:\n";
        for (char16_t c : ucs2)
            std::cout << std::hex << std::showbase << c << '\n';
    }
}

Output:

UTF16 conversion produced 4 code points:
0x7a
0x6c34
0xd834
0xdd0b
UCS2 failed after producing 2 characters:
0x7a
0x6c34

[edit] See also

Character conversions	narrow multibyte (char)	UTF-8 (char)	UTF-16 (char16_t)
UTF-16	`mbrtoc16` / `c16rtomb`	`codecvt`<char16_t, char, mbstate_t> `codecvt_utf8_utf16`<char16_t> `codecvt_utf8_utf16`<char32_t> `codecvt_utf8_utf16`<wchar_t>	N/A
UCS2	No	`codecvt_utf8`<char16_t>	`codecvt_utf16`<char16_t>
UTF-32/UCS4 (char32_t)	`mbrtoc32` / `c32rtomb`	`codecvt`<char32_t, char, mbstate_t> `codecvt_utf8`<char32_t>	`codecvt_utf16`<char32_t>
UCS2/UCS4 (wchar_t)	No	`codecvt_utf8`<wchar_t>	`codecvt_utf16`<wchar_t>
wide (wchar_t)	`codecvt`<wchar_t, char, mbstate_t> `mbsrtowcs` / `wcsrtombs`	No	No


codecvt	converts between character encodings, including UTF-8, UTF-16, UTF-32 (class template)

codecvt_mode (C++11)	tags to alter behavior of the standard codecvt facets (enum)

codecvt_utf16 (C++11)	converts between UTF-16 and UCS2/UCS4 (class template)

codecvt_utf8_utf16 (C++11)	converts between UTF-8 and UTF-16 (class template)

Language
Standard library headers
Concepts
Utilities library
Strings library
Containers library
Algorithms library
Iterators library
Numerics library
Input/output library
Localizations library
Regular expressions library (C++11)
Atomic operations library (C++11)
Thread support library (C++11)