Text - A spicy text library for C++ that has the explicit goal of enabling the entire ecosystem to share in proper forward progress towards a bright Unicode future.

Overview

ztd.text

Because if text works well in two of the most popular systems programming languages, the entire world over can start to benefit properly.

This repository is an implementation of an up and coming proposal percolating through SG16, P1629 - Standard Text Encoding. It will also include implementations of some downstream ideas covered in Previous Work in this area, including Zach Laine's Boost.Text (proposed), rmf's libogonek, and Tom Honermann's text_view.

Current Plans

The current roadmap is to obtain funding to make text encoding viable in time for standardization for C++23. See the roadmap for details.

Please considering sponsoring the work via any of the available means listed near the bottom of this page or from the "Sponsor" button on this website.

Documentation

The documentation can be found in full on https://ztdtext.rtfd.io/!

License & Contributions

By contributing to this repository, you agree that your contributions will be released to the copyright of JeanHeyd "ThePhD" Meneide and Shepherd's Oasis, LLC under the Apache 2.0 License or the relevant Commercial License. See the LICENSE file and the CONTRIBUTING file for details.

Issues
  • Naming discussion: count_{en,de,trans}codable

    Naming discussion: count_{en,de,trans}codable

    First off, thanks a lot for the awesome work you're doing! 🙃

    I have been loosely following @ThePhD's stream (flood?) of papers & blog posts, and was wondering about a naming discussion they had posed on twitter a while ago.

    Naming is hard, though worth it - the right name substantially reduces cognitive overhead, especially when accumulated over a long period of time. Since I was intrigued by the riddle of squaring that circle, I had written to @ThePhD with a suggestion, and they suggested to open an issue here.

    One of the key points about the count_* functions that doesn't exactly jump out from the documentation (at least to me) is that the actual work of encoding/... (especially around memory allocation etc., AFAIU) is not being done. Quoth @ThePhD (in private communication):

    I'm trying to avoid the participle because count_decoded sounds like "count after the work is already done", and that work expressly isn't being done. That's why I took the adjective-y version of decodable (and friends). Or, at least, that was the reasoning....! Not sure if it's good reasoning 😅

    Since I wasn't warming up to count_encodable et. al (it's not intrinsically obvious to from the name whether things are being counted before or after the operation. The "-able" suffix makes it seem like it could be the former - "these code points are encodable [into code units], now count them"), I wanted to come up with something that more intrinsically reflects the absence of real work™.

    My suggestion boils down to changing the verb from count to infer, freeing up the participle to rejoin the party unburdened of assumed labour. IOW: infer_encoded_{size,length,count}

    count could even make a comeback as a noun, but whether the choice is size, length or count is (from my POV) not that important.

    I have no beef with the validate_* functions (whose expressiveness I applaud), so, to sum up my proposed answer to the initial riddle from the tweet (using my preferred noun size, if only due to its lack thereof, typographically speaking):

    encode
    decode
    transcode
    validate_encodable_as
    validate_decodable_as
    validate_transcodable_as
    infer_encoded_size
    infer_decoded_size
    infer_transcoded_size
    

    Thanks for reading!

    documentation enhancement question thank you!! 
    opened by h-vetinari 14
  • Request: Add port for vcpkg

    Request: Add port for vcpkg

    First I'd like to thank you and all the contributors of the library for all your work done so far.

    Going further, I'd like to propose making the library available through vcpkg (and other package managers) to ease up the process of integrating it on different projects, let them be existing or new ones. The library is header-only so should be (relatively) easy to integrate and sol2's "model" could be as well adapted for the library (not sure if there is any problem related to this).

    enhancement good first issue help wanted question 
    opened by TerensTare 8
  • 🔨 Fixes for GCC 11

    🔨 Fixes for GCC 11

    Tests fail under GCC 11 due to wider constexpr support. These are the fixes necessary to make tests pass

    • Fix wide_execution_cwchar copy-paste bug
    • Bump transcode_iterator tags to prevent type ambiguity
    • Make word_iterator::_M_base_range constexpr
    enhancement question 
    opened by nickelpro 6
  • decode_view iterator post-increment operator is broken

    decode_view iterator post-increment operator is broken

    It posts but it does not increment. 3/5 :star:'s, room for improvement

    More things are also probably broken but I only wrote the test for the decode_view, see https://github.com/soasis/text/pull/27

    bug thank you!! 
    opened by nickelpro 5
  • Cannot run tests with gcc 11 (error: invalid use of ‘this’ at top level)

    Cannot run tests with gcc 11 (error: invalid use of ‘this’ at top level)

    Configured with:

    CC=gcc CXX=g++ cmake ..  -DZTD_TEXT_TESTS=ON -DZTD_TEXT_EXAMPLES=ON -DZTD_TEXT_GENERATE_SINGLE=ON -GNinja
    
    g++ (GCC) 11.1.1 20210531 (Red Hat 11.1.1-3)
    Copyright (C) 2021 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
    CMake Output
    -- The C compiler identification is GNU 11.1.1
    -- The CXX compiler identification is GNU 11.1.1
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working C compiler: /usr/bin/gcc - skipped
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /usr/bin/g++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- Performing Test CXX_CHECK_FLAG_DISABLE_PERMISSIVE
    -- Performing Test CXX_CHECK_FLAG_DISABLE_PERMISSIVE - Success
    -- Performing Test C_CHECK_FLAG_DISABLE_PERMISSIVE
    -- Performing Test C_CHECK_FLAG_DISABLE_PERMISSIVE - Success
    -- Performing Test CXX_CHECK_FLAG_WARN_PEDANTIC
    -- Performing Test CXX_CHECK_FLAG_WARN_PEDANTIC - Success
    -- Performing Test C_CHECK_FLAG_WARN_PEDANTIC
    -- Performing Test C_CHECK_FLAG_WARN_PEDANTIC - Success
    -- Performing Test CXX_CHECK_FLAG_WARN_ALL
    -- Performing Test CXX_CHECK_FLAG_WARN_ALL - Success
    -- Performing Test C_CHECK_FLAG_WARN_ALL
    -- Performing Test C_CHECK_FLAG_WARN_ALL - Success
    -- Performing Test CXX_CHECK_FLAG_WARN_ERRORS
    -- Performing Test CXX_CHECK_FLAG_WARN_ERRORS - Success
    -- Performing Test C_CHECK_FLAG_WARN_ERRORS
    -- Performing Test C_CHECK_FLAG_WARN_ERRORS - Success
    -- Performing Test CXX_CHECK_FLAG_UTF8_LITERAL_ENCODING
    -- Performing Test CXX_CHECK_FLAG_UTF8_LITERAL_ENCODING - Success
    -- Performing Test C_CHECK_FLAG_UTF8_LITERAL_ENCODING
    -- Performing Test C_CHECK_FLAG_UTF8_LITERAL_ENCODING - Success
    -- Performing Test CXX_CHECK_FLAG_UTF8_SOURCE_ENCODING
    -- Performing Test CXX_CHECK_FLAG_UTF8_SOURCE_ENCODING - Success
    -- Performing Test C_CHECK_FLAG_UTF8_SOURCE_ENCODING
    -- Performing Test C_CHECK_FLAG_UTF8_SOURCE_ENCODING - Success
    -- Performing Test CXX_CHECK_FLAG_EXTRA_CONSTEXPR_DEPTH
    -- Performing Test CXX_CHECK_FLAG_EXTRA_CONSTEXPR_DEPTH - Success
    -- Performing Test C_CHECK_FLAG_EXTRA_CONSTEXPR_DEPTH
    -- Performing Test C_CHECK_FLAG_EXTRA_CONSTEXPR_DEPTH - Failed
    -- Performing Test CXX_CHECK_FLAG_EXTRA_CONSTEXPR_STEPS
    -- Performing Test CXX_CHECK_FLAG_EXTRA_CONSTEXPR_STEPS - Success
    -- Performing Test C_CHECK_FLAG_EXTRA_CONSTEXPR_STEPS
    -- Performing Test C_CHECK_FLAG_EXTRA_CONSTEXPR_STEPS - Failed
    -- Performing Test CXX_CHECK_FLAG_TEMPLATE_DEBUGGING_MODE
    -- Performing Test CXX_CHECK_FLAG_TEMPLATE_DEBUGGING_MODE - Success
    -- Performing Test C_CHECK_FLAG_TEMPLATE_DEBUGGING_MODE
    -- Performing Test C_CHECK_FLAG_TEMPLATE_DEBUGGING_MODE - Failed
    -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.9.6", minimum required is "3")
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /tmp/text/build
    
    [3/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o -c ../tests/basic_run_time/source/detail/encoding_name.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/is_ignorable_error_handler.hpp:38,
                     from ../include/ztd/text/utf8.hpp:41,
                     from ../include/ztd/text/detail/encoding_name.hpp:38,
                     from ../tests/basic_run_time/source/detail/encoding_name.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    
    
    Full build output
    [0/2] Re-checking globbed directories...
    [2/162] Generating include/ztd/text.hpp, include/ztd/text/forward.hpp
    Current version: v0.0.1 (revision 811ecc2)
    Project path: /tmp/text/single/../include
    
    Creating single header for ztd.text
    processing /tmp/text/include/ztd/text.hpp
    processing /tmp/text/include/ztd/text/version.hpp
    processing /tmp/text/include/ztd/text/forward.hpp
    processing /tmp/text/include/ztd/text/encoding.hpp
    processing /tmp/text/include/ztd/text/execution.hpp
    processing /tmp/text/include/ztd/text/no_encoding.hpp
    processing /tmp/text/include/ztd/text/code_point.hpp
    processing /tmp/text/include/ztd/text/unicode_code_point.hpp
    processing /tmp/text/include/ztd/text/type_traits.hpp
    processing /tmp/text/include/ztd/text/detail/assert.hpp
    processing /tmp/text/include/ztd/text/detail/unicode.hpp
    processing /tmp/text/include/ztd/text/code_unit.hpp
    processing /tmp/text/include/ztd/text/encode_result.hpp
    processing /tmp/text/include/ztd/text/encoding_error.hpp
    processing /tmp/text/include/ztd/text/detail/to_underlying.hpp
    processing /tmp/text/include/ztd/text/detail/encoding_range.hpp
    processing /tmp/text/include/ztd/text/state.hpp
    processing /tmp/text/include/ztd/text/text_tag.hpp
    processing /tmp/text/include/ztd/text/decode_result.hpp
    processing /tmp/text/include/ztd/text/error_handler.hpp
    processing /tmp/text/include/ztd/text/c_string_view.hpp
    processing /tmp/text/include/ztd/text/basic_c_string_view.hpp
    processing /tmp/text/include/ztd/text/is_code_points_replaceable.hpp
    processing /tmp/text/include/ztd/text/is_code_units_replaceable.hpp
    processing /tmp/text/include/ztd/text/is_unicode_code_point.hpp
    processing /tmp/text/include/ztd/text/detail/pass_through_handler.hpp
    processing /tmp/text/include/ztd/text/detail/forwarding_handler.hpp
    processing /tmp/text/include/ztd/text/detail/transcode_one.hpp
    processing /tmp/text/include/ztd/text/transcode_result.hpp
    processing /tmp/text/include/ztd/text/validate_result.hpp
    processing /tmp/text/include/ztd/text/count_result.hpp
    processing /tmp/text/include/ztd/text/detail/is_lossless.hpp
    processing /tmp/text/include/ztd/text/is_full_range_representable.hpp
    processing /tmp/text/include/ztd/text/is_transcoding_compatible.hpp
    processing /tmp/text/include/ztd/text/unicode_scalar_value.hpp
    processing /tmp/text/include/ztd/text/detail/encoding_iterator_storage.hpp
    processing /tmp/text/include/ztd/text/detail/span_or_reconstruct.hpp
    processing /tmp/text/include/ztd/text/is_ignorable_error_handler.hpp
    processing /tmp/text/include/ztd/text/detail/empty_state.hpp
    processing /tmp/text/include/ztd/text/detail/execution_cuchar.hpp
    processing /tmp/text/include/ztd/text/utf8.hpp
    processing /tmp/text/include/ztd/text/detail/cast.hpp
    processing /tmp/text/include/ztd/text/utf16.hpp
    processing /tmp/text/include/ztd/text/detail/progress_handler.hpp
    processing /tmp/text/include/ztd/text/detail/windows.hpp
    processing /tmp/text/include/ztd/text/detail/posix.hpp
    processing /tmp/text/include/ztd/text/detail/encoding_name.hpp
    processing /tmp/text/include/ztd/text/utf32.hpp
    processing /tmp/text/include/ztd/text/encoding_scheme.hpp
    processing /tmp/text/include/ztd/text/is_bidirectional_encoding.hpp
    processing /tmp/text/include/ztd/text/is_unicode_encoding.hpp
    processing /tmp/text/include/ztd/text/ascii.hpp
    processing /tmp/text/include/ztd/text/detail/replacement.hpp
    processing /tmp/text/include/ztd/text/detail/execution_mac_os.hpp
    processing /tmp/text/include/ztd/text/detail/execution_iconv.hpp
    processing /tmp/text/include/ztd/text/wide_execution.hpp
    processing /tmp/text/include/ztd/text/detail/wide_execution_windows.hpp
    processing /tmp/text/include/ztd/text/detail/wide_execution_iso10646.hpp
    processing /tmp/text/include/ztd/text/detail/wide_execution_iconv.hpp
    processing /tmp/text/include/ztd/text/detail/wide_execution_cwchar.hpp
    processing /tmp/text/include/ztd/text/literal.hpp
    processing /tmp/text/include/ztd/text/wide_literal.hpp
    processing /tmp/text/include/ztd/text/any_encoding.hpp
    processing /tmp/text/include/ztd/text/default_encoding.hpp
    processing /tmp/text/include/ztd/text/encode.hpp
    processing /tmp/text/include/ztd/text/decode.hpp
    processing /tmp/text/include/ztd/text/validate_decodable_as.hpp
    processing /tmp/text/include/ztd/text/validate_encodable_as.hpp
    processing /tmp/text/include/ztd/text/count_encodable.hpp
    processing /tmp/text/include/ztd/text/count_decodable.hpp
    processing /tmp/text/include/ztd/text/transcode.hpp
    processing /tmp/text/include/ztd/text/detail/transcode_extension_points.hpp
    processing /tmp/text/include/ztd/text/encode_view.hpp
    processing /tmp/text/include/ztd/text/encode_iterator.hpp
    processing /tmp/text/include/ztd/text/detail/encoding_iterator.hpp
    processing /tmp/text/include/ztd/text/decode_view.hpp
    processing /tmp/text/include/ztd/text/decode_iterator.hpp
    processing /tmp/text/include/ztd/text/transcode_view.hpp
    processing /tmp/text/include/ztd/text/transcode_iterator.hpp
    processing /tmp/text/include/ztd/text/text_view.hpp
    processing /tmp/text/include/ztd/text/basic_text_view.hpp
    processing /tmp/text/include/ztd/text/normalization.hpp
    processing /tmp/text/include/ztd/text/text.hpp
    processing /tmp/text/include/ztd/text/basic_text.hpp
    finished creating single header for ztd.text
    
    Creating single forward declaration header for ztd.text
    processing /tmp/text/include/ztd/text/forward.hpp
    processing /tmp/text/include/ztd/text/version.hpp
    finished creating single forward declaration header for ztd.text
    
    writing /tmp/text/build/single/include/ztd/text.hpp...
    writing /tmp/text/build/single/include/ztd/forward.hpp...
    [3/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/detail/encoding_name.cpp.o -c ../tests/basic_run_time/source/detail/encoding_name.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/is_ignorable_error_handler.hpp:38,
                     from ../include/ztd/text/utf8.hpp:41,
                     from ../include/ztd/text/detail/encoding_name.hpp:38,
                     from ../tests/basic_run_time/source/detail/encoding_name.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [4/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode.cpp.o -c ../tests/basic_run_time/source/decode.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/no_encoding.hpp:42,
                     from ../include/ztd/text/execution.hpp:38,
                     from ../include/ztd/text/default_encoding.hpp:38,
                     from ../include/ztd/text/decode.hpp:39,
                     from ../tests/basic_run_time/source/decode.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [5/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode_view.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode_view.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode_view.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode_view.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/decode_view.cpp.o -c ../tests/basic_run_time/source/decode_view.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/decode_iterator.hpp:38,
                     from ../include/ztd/text/decode_view.hpp:38,
                     from ../tests/basic_run_time/source/decode_view.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [6/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/encode.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/encode.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/encode.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/encode.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/encode.cpp.o -c ../tests/basic_run_time/source/encode.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/encode.hpp:40,
                     from ../tests/basic_run_time/source/encode.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [7/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_encodable.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_encodable.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_encodable.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_encodable.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_encodable.cpp.o -c ../tests/basic_run_time/source/count_encodable.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/no_encoding.hpp:42,
                     from ../include/ztd/text/execution.hpp:38,
                     from ../include/ztd/text/default_encoding.hpp:38,
                     from ../include/ztd/text/count_encodable.hpp:39,
                     from ../tests/basic_run_time/source/count_encodable.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [8/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_decodable.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_decodable.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_decodable.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_decodable.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_decodable.cpp.o -c ../tests/basic_run_time/source/count_decodable.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/no_encoding.hpp:42,
                     from ../include/ztd/text/execution.hpp:38,
                     from ../include/ztd/text/default_encoding.hpp:38,
                     from ../include/ztd/text/count_decodable.hpp:39,
                     from ../tests/basic_run_time/source/count_decodable.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [9/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.encode.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.encode.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.encode.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.encode.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.encode.cpp.o -c ../tests/basic_run_time/source/any_encoding.encode.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/is_ignorable_error_handler.hpp:38,
                     from ../include/ztd/text/encoding_scheme.hpp:43,
                     from ../tests/basic_run_time/source/any_encoding.encode.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [10/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_transcodable.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_transcodable.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_transcodable.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_transcodable.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/count_transcodable.cpp.o -c ../tests/basic_run_time/source/count_transcodable.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/no_encoding.hpp:42,
                     from ../include/ztd/text/execution.hpp:38,
                     from ../include/ztd/text/default_encoding.hpp:38,
                     from ../include/ztd/text/count_transcodable.hpp:39,
                     from ../tests/basic_run_time/source/count_transcodable.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [11/162] Building CXX object tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.decode.cpp.o
    FAILED: tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.decode.cpp.o
    /usr/bin/g++ -DZTD_TEXT_COMPILE_TIME_ENCODING_NAME=\"UTF-8\" -I../tests/basic_run_time/../shared/include -I../include -Isingle/include -I_deps/ztd.idk-src/include -I_deps/catch2-src/single_include -fexec-charset=utf-8 -finput-charset=utf-8 -pedantic -Wall -Werror -std=gnu++17 -MD -MT tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.decode.cpp.o -MF tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.decode.cpp.o.d -o tests/basic_run_time/CMakeFiles/ztd.text.tests.basic_run_time.dir/source/any_encoding.decode.cpp.o -c ../tests/basic_run_time/source/any_encoding.decode.cpp
    In file included from ../include/ztd/text/detail/transcode_one.hpp:56,
                     from ../include/ztd/text/error_handler.hpp:52,
                     from ../include/ztd/text/is_ignorable_error_handler.hpp:38,
                     from ../include/ztd/text/encoding_scheme.hpp:43,
                     from ../tests/basic_run_time/source/any_encoding.decode.cpp:31:
    ../include/ztd/ranges/unbounded.hpp:273:50: error: invalid use of ‘this’ at top level
      273 |                         noexcept(ranges::advance(this->_M_it, __diff))) {
          |                                                  ^~~~
    ../include/ztd/ranges/unbounded.hpp:287:49: error: invalid use of ‘this’ at top level
      287 |                         noexcept(ranges::recede(this->_M_it, __diff))) {
          |                                                 ^~~~
    [12/162] Building CXX object _deps/catch2-build/CMakeFiles/Catch2WithMain.dir/src/catch_with_main.cpp.o
    ninja: build stopped: subcommand failed.
    

    CI is using GCC 9, which is why this hasn't come up. It's probably also worth testing clang there also, although it's Worked On My Machine (TM)

    bug thank you!! 
    opened by aDotInTheVoid 4
  • __iterator aliases assume std::iterator_traits has members

    __iterator aliases assume std::iterator_traits has members

    Most of the __iterator_* aliases depend on std::iterator_traits, but this doesn't necessarily mesh with C++20 iterators, since not all C++20 iterators necessarily have a populated std::iterator_traits (e.g. stdr::iterator_t<stdr::basic_istream_view<char, char, std::char_traits<char>>>).

    Most of these either have a C++20 counterpart traits type (e.g. value_type) or can probably be worked out (e.g. pointer). One exception to this is iterator_concept, which might only exist as a member alias (e.g. basic_istream_view).

    bug 
    opened by cjdb 3
  • Compilation issue with CMAKE_BUILD_TYPE=RelWithDebInfo

    Compilation issue with CMAKE_BUILD_TYPE=RelWithDebInfo

    Building with -DCMAKE_BUILD_TYPE=RelWithDebInfo produces errors.

    /home/sdowney/src/text/text/include/ztd/text/utf32.hpp: In function ‘constexpr auto ztd::text::__v0::validate_code_points(_Input&&, _Encoding&&, _EncodeState&, _DecodeState&) [with _Input = std::span<const char32_t, 18446744073709551615>; _Encoding = const ztd::text::__v0::encoding_scheme<ztd::text::__v0::basic_utf32<char32_t, char32_t>, ztd::text::__v0::endian::little, std::byte>&; _EncodeState = ztd::text::__v0::__detail::__empty_state; _DecodeState = ztd::text::__v0::__detail::__empty_state]’:
    /home/sdowney/src/text/text/include/ztd/text/utf32.hpp:189:38: error: ‘__code_unit_buf’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
      189 |     _
    

    Compile differences seem to be adding -O2 -g -DNDEBUG to the compile. First error is in any_encoding.encode.cpp

    bug 
    opened by steve-downey 3
  • R-values? More like Revenge-values!!

    R-values? More like Revenge-values!!

    The following code breaks:

            std::stringstream message_stream;
            /* blah blah manipulate message stream */
            std::wstring winMessage = ztd::text::transcode(message_stream.str(), ztd::text::compat_utf8, ztd::text::wide_utf16, ztd::text::replacement_handler);
    
            MessageBoxW(NULL, winMessage.c_str(), L"GlueGDLoader Error", MB_OK);
    

    The error happens because passing a temporary into these functions (the message_stream.str()) breaks everything. This is one of the fundamental problems with std::ranges and ranges in general: r-values are not handled by the lower-levels.

    But honestly, there's nothing wrong with this code in particular.

    An owning-container (and not a temporary container) is handed to the end-user by the plain and _to versions of these functions. We should detect r-values in the plain functions and hold onto them normally, rather than forward them, and let the result be returned back to the end user. This would result in a more pleasant user experience.

    We should also add an explicit top-level r-value screamer for the Input on the _into functions.

    (Thanks for catching this one, @s5bug.)

    bug enhancement 
    opened by ThePhD 2
  • Design: do we need state in the replacement functions?

    Design: do we need state in the replacement functions?

    It could be conceivable that, if an error handler is called and replacement is happening, replacement characters may be inserted into the stream which may need additional state information. Does this mean encode_state and decode_state need to be (required) parameters to replacement_code_units, replacement_code_points, maybe_replacement_code_units, and maybe_replacement_code_points?

    Are we missing other places where the state information is necessary?

    I didn't initially think of this because I assumed most people would override the entire error handler to take care of this case, but now I'm not sure. The parameter will be ignored in 99% of cases, but it's probably no issue to just make sure it takes it anyways even if we turn out to be wrong...

    enhancement good first issue question 
    opened by ThePhD 2
  • [BUG] Discrepency when determining result type's types

    [BUG] Discrepency when determining result type's types

    The below sample will fail to compile.

    #include <ztd/text.hpp>
    
    void repro() {
        std::string     str;
        ztd::text::utf8 enc;
        auto            enc_state = ztd::text::make_encode_state(enc);
        auto            dec_state = ztd::text::make_decode_state(enc);
        ztd::text::transcode_into(str,
                                  enc,
                                  str,
                                  enc,
                                  ztd::text::pass_handler{},
                                  ztd::text::pass_handler{},
                                  dec_state,
                                  enc_state);
    }
    

    basic_transcode_into uses using _WorkingOutput = __recontsruct_t<Output> and declares a variable _WorkingOutput __working_output. Later, in a deeper call to __super_basic_transcode_one(), the output range type for the transcode_result is determined as the return type of __basic_encode_one(). __basic_encode_one uses a series of if constexpr checks to determines its return type. The type of transcode_result::output can differ from __working_output. basic_transcode_into then attempts to assign __transcode_result.output into __working_output, which will fail if they are not the same (or at least convertible).

    In this case, the __basic_encode_one() is returning an unbounded_view<std::string::iterator>, whereas the __working_output is of type subrange<std::string::iterator>. I cannot understand the if constexpr checks enough to tell why the branch returning an unbounded_view is selected.

    bug 
    opened by vector-of-bool 2
  • Fix: `throw_handler` should return non-void

    Fix: `throw_handler` should return non-void

    The following code fails to compile:

    string in;
    u8string out;
    utf8::state state;
    utf8::encode_one(in, out, throw_handler{}, state);
    

    utf8::encode_one() (and presumably other functions) rely on return type deduction. Sometimes they return a _Result directly, but sometimes they return the result of invoking __error_handler. If they do both, then __error_handler must return _Result or there will be conflicting return type deduction. Explicitly annotating a return type that matches the expectations of other error handlers, despite not returning normally, will fix this compile-time error.

    enhancement thank you!! 
    opened by vector-of-bool 2
  • fixed/static/inline_vector for decode_one/encode_one/transcode_one

    fixed/static/inline_vector for decode_one/encode_one/transcode_one

    These *_one functions return heap-allocating containers right now. This is not necessary, as the maximum output size is known ahead-of-time.

    We should write a fixed/static/inline_vector<T, max_size>, alongside a fixed/static/inline_string<T, max_size, [char_traits<T> ??]>.

    enhancement 
    opened by ThePhD 0
  • Don't Cursed Open Inside

    Don't Cursed Open Inside

    This is a running list of all the (mildly to extremely) cursed encodings, and whether or not we should implement them. More can be suggested on Twitter here Here goes:

    • [ ] UNIVAC encoding
    • [ ] UTF-EBCDIC
    • [ ] UTF-7
    • [ ] UTF-7-IMAP
    • [ ] UTF-1
    • [ ] SHIFT-JIS (already implemented in example code)
    • [ ] ISO/IEC 2022 Encodings (https://en.wikipedia.org/wiki/ISO/IEC_2022)
    • [ ] ISO/IEC 646 Encodings (https://en.wikipedia.org/wiki/ISO/IEC_646)

    Some that might not be possible within the framework of this library:

    • Early Canjie input method translation: this is moreso a system of input that is then converted to characters, rather than a character set itself. It also seems to have a (potentially?) unbounded set of inputs that can produce an equally wild amount of outputs, making the encode_one/decode_one limitations potentially useless? Needs more research
    opened by ThePhD 0
  • Missing encodings: Tatar

    Missing encodings: Tatar

    Two encodings that are missing from the list of available encodings are the Tatar encodings as specified by Resolution of the Cabinet of Ministers of the Republic of Tatarstan from the 9th of December, 1996 No. 1026 "On the standards of the encoding symbols of Tatar alphabet for computer applications"

    The document specifies an ‘ASCII’ version (for DOS and console applications; compatible with cp866) and ‘ANSI’ version (for Windows applications; compatible with cp1251).

    I don't think there are established codes for these encodings. I would suggest something along the lines of cp1251-tatar and cp866-tatar.


    ‘ANSI’ version (variant of cp1251)

    The scans of the actual resolution are hard to read, but the ASCII version is described in Russian Wikipedia's article on cp1251, under the subheading «Татарский вариант». Here are the rows that are different from cp1251:

    0x80 = Ә (U+04D8), 0x8A = Ө (U+04E8), 0x8C = Ү (U+04AE), 0x8D = Җ (U+0496), 0x8E = Ң (U+04A2), 0x8F = Һ (U+04BA), 0x90 = ә (U+04D9), 0x9A = ө (U+04E9), 0x9C = ү (U+04AF),m 0x9D = җ (U+0497), 0x9e = ң (U+04A3), 0x9F = һ (U+04BB).

    Basically, the letters that differ from cp1251 are:

    0x80 = Ә (U+04D8)
    0x8A = Ө (U+04E8)
    0x8C = Ү (U+04AE)
    0x8D = Җ (U+0496)
    0x8E = Ң (U+04A2)
    0x8F = Һ (U+04BA)
    0x90 = ә (U+04D9)
    0x9A = ө (U+04E9)
    0x9C = ү (U+04AF)
    0x9D = җ (U+0497)
    0x9e = ң (U+04A3)
    0x9F = һ (U+04BB)
    

    This encoding can still be encountered in the internet. For example, search for хђзер (for хәзер 'now'), кирђк (for кирәк 'needed') or мљмкин (for мөмкин 'possible') in Google to find some example usage. Some fonts for this encoding can be found here: https://kashapovnail.ucoz.ru/load/1-1-0-1 (they would usually replace Southern Slavic letters with Tatar letters, so they're not really Unicode-compatible; it's a hard because old Windows version didn't support other way to input this encoding).


    ‘ASCII’ version (variant of cp866)

    I don’t know how widely this encoding is used. But the document provides information about it, so it can be implemented:

    image

    It's basically cp866 with the following changes (0xF0 and 0xF1 might not be the change depending on what you consider the 'basic variant' of cp866):

    0xF0 = Ё (U+0401)
    0xF1 = ё (U+0451)
    0xF2 = Ә (U+04D8)
    0xF3 = Ө (U+04E8)
    0xF4 = Ү (U+04AE)
    0xF5 = Җ (U+0496)
    0xF6 = Ң (U+04A2)
    0xF7 = Һ (U+04BA)
    0xF8 = ә (U+04D9)
    0xF9 = ө (U+04E9)
    0xFA = ү (U+04AF)
    0xFB = җ (U+0497)
    0xFC = ң (U+04A3)
    0xFD = һ (U+04BB)
    
    enhancement good first issue help wanted 
    opened by rohkea 3
  • Improve CMake integration

    Improve CMake integration

    At the moment, the CMake build files are messy.

    • There is no clear separation between consumer and developer code paths.
      This makes it impossible to build/install/package the library without heavily patching build files, which is completely avoidable. It's not nice to force consumers to pass 10 flags just so they can dodge everything but the library itself.
    • The build files make network requests during configure.
      If you wish this library to be reach a wide audience, this is an absolute no go. For dependencies like Catch2, you can use vcpkg, even for those you grab from GitHub, like the other ztd libraries.
    • The library does not declare its standard requirement.
      I had to find out from #13 that this library is targetting C++17. A simple target_compile_features(ztd_text INTERFACE cxx_std_17) would be helpful. Hardcoding CMAKE_CXX_STANDARD is also wrong, because it's purpose is to be used from the CLI to control the build.
    • Globbing sources. This is mainly for scripting. Again, if you wish this library to be usable by a wide audience, this is an absolute no go. Mainly because it doesn't even work reliably for most build systems. Please read the big note from the CMake developers and heed their words.
    • Install rules are wrong.

    Consumer side of lists files should do nothing more than describe the build and usage requirements of a library and the CMake build scripts should be as minimal as possible with no hardcoded details. It is noone's benefit to force people trying to build/package/use your library to patch things.

    I recommend taking a look at cmake-init, which is designed to solve all of these issues. Since you have dependencies from GitHub as well, I recommend taking a look at the vcpkg example repository.
    A better solution would be however, if instead of introducing potentially exponential complexity to building the library with optional dependencies, just make a separate library that depends on both this one and that optional dependency.

    enhancement build 
    opened by friendlyanon 6
  • Library Concepts?

    Library Concepts?

    Is there an intention to add library concepts in place of the current named requirements?

    There's admittedly some challenges in doing so, but I enjoy the benefits of clearer error messages and simpler overload sets.

    e.g.

    template <typename E>
    concept encoding =
        (requires() {
            // One state type
            typename E::state;
        } ||
        requires() {
            // OR distinct state types
            typename E::encode_state;
            typename E::decode_state;
        }) &&
        requires(E encoding,
                 encode_state_t<E> enc_state,
                 decode_state_t<E> dec_state,
                 range_archetype<convertible_to_archetype<code_point_t<E>>> cp_in_range,
                 range_archetype<convertible_to_archetype<code_unit_t<E>>> cu_in_range,
                 range_archetype<assignable_from_archetype<code_point_t<E>>> cp_out_range,
                 range_archetype<assignable_from_archetype<code_unit_t<E>>> cu_out_range,
                 handler_archetype handler) {
            { E::max_code_points } -> convertible_to<size_t>;
            { E::max_code_units } -> convertible_to<size_t>;
            { encoding.encode_one(cp_in_range, cu_out_range, handler, enc_state) };
            { encoding.decode_one(cu_in_range, cp_out_range, handler, dec_state) };
        };
    

    Such a definition won't be perfect because we can't be certain that encode_one() will be valid without knowing the exact range type that will be used until we actually call it, but one can make a very close approximation using concept archetypes.

    enhancement 
    opened by vector-of-bool 2
Releases(v0.1.0)
  • v0.1.0(Aug 13, 2021)

    Sparkly Sandwich 🥪

    This release makes some fundamental and hard changes to the API with respect to error handlers and encoding type names in the library.

    • error_handler types now takes 4 parameters instead of 3. This was the addition of 1 extra parameter, which represents any prepared but unwritten characters to the output stream. More information can be found at this point in the documentation.
    • encode_state/decode_state/state types are attempted to pass to the replacement_code_units/replacement_code_points/maybe_replacement_code_units/maybe_replacement_code_points functions, if at all possible. This allows for a specific state type to be properly.
    • ztd::text::propagate_error is now available. This properly filters a partially-completed transcode operation (that fails at the decode_result step, for example) and pushes it through the encode part of the process with the appropriate error_handler. This function helps get the novelty and issues done right.
    • The names of the counting functions have been changed (as debated in #16). Thank you to everyone who participated and made it happen!
    • All the names of encoding types (utf8, utf16, etc.) are now objects, and their types are now suffixed with _t. This is becase 99.9% of the time, one just wants to use the encoding object, not create a new instance every single time they use it (thanks for the push here @hanickadot !)

    The documentation has been vastly improved in many places, as well, to make it easier to find out specific kinds of information. It still needs more work, but that's why this isn't a v1.0.0 library yet!

    Go forth, and text to your heart's delight!

    💚, ThePhD & Shepherd's Oasis

    Source code(tar.gz)
    Source code(zip)
  • v0.0.0(Feb 28, 2021)

    Zero Standard - Text Library

    🎉 This is the first release for ztd.text !~

    The API and expectations are fully documented over at the documentation page, here: https://ztdtext.rtfd.io.

    We wanted to try something different when we first released this library, so we focused on fully documenting all of the public-facing APIs and writing full design docs the first go-around, rather than leaving a hodgepodge of use cases and buildup for everyone to see. We've found this approach to be helpful in making sure that nothing in the library goes undone, and we hope to write about it and talk about that process soon!

    The examples, benchmarks, and "Getting Started" are not completely filled out yet. There are still some examples in the examples directory, though, and we encourage you to peek at them, even if they are quite sparse!

    We're quite proud of Version 0, and we hope to release even better versions, soon.

    Thanks, Shepherd's Oasis & ThePhD

    Source code(tar.gz)
    Source code(zip)
    forward.hpp(26.16 KB)
    text.hpp(699.06 KB)
Owner
Shepherd's Oasis
Software Services and Consulting.
Shepherd's Oasis
A bright opening, a clear sight, a clean slate.

Skylight A bright opening, a clean window. Etymology According to dictionary.com, a "skylight" is: an opening in a roof or ceiling, fitted with glass,

null 44 Jun 22, 2022
Adds proper Apple Pencil support to Unity's InputSystem.

This project aims to create an Apple Pencil device for Unity's InputSystem package. The problem: Unity does not fully integrate with Apple Pencil. App

Oliver Smith 8 May 9, 2022
A modern port of Turbo Vision 2.0, the classical framework for text-based user interfaces. Now cross-platform and with Unicode support.

Turbo Vision A modern port of Turbo Vision 2.0, the classical framework for text-based user interfaces. Now cross-platform and with Unicode support. I

null 1.3k Jun 24, 2022
Building game development ecosystem for @ziglang!

If you are new to low-level graphics programming or you would like to learn about some zig-gamedev libraries I recommend starting with intro applicati

Michal Ziulek 920 Jun 24, 2022
This is the repo for the microcontroller part of project Cleanurge - a scalable IoT powered waste management system, our solution towards a greener world.

Cleanurge A scalable waste management system powered by IoT. There are 3 repositories for the entire cleanurge system in total cleanurge-mcu: Containi

Developer Student Clubs KGEC 6 Oct 11, 2021
FlexOS: Towards Flexible OS Isolation (ASPLOS'22) Artifact Evaluation Repository

FlexOS ASPLOS'22 Artifact Evaluation This repository contains the artifacts, including experiments and graphs, for the paper: FlexOS: Towards Flexible

null 9 Apr 15, 2022
Fastest tiktok share botter. Over 100k shares a second.

TikTok Share Botter ⚠️ CURRENTLY PATCHED WAITING FOR WORK AROUND ⚠️ Fastest multi-threaded and proxyless TikTok Share Botter written in C++. Click her

HNT8 Organization 3 Jun 22, 2022
Header-only library providing unicode aware string support for C++

CsString Introduction CsString is a standalone library which provides unicode aware string support. The CsBasicString class is a templated class which

CopperSpice 89 Jun 4, 2022
Libnaomi - Toolchain and system library enabling hobby/homebrew software to be developed for the Sega Naomi platform.

Naomi Toolchain & Libraries A minimal Naomi homebrew environment, very loosely based off of KallistiOS toolchain work but bare metal and implemented f

Jennifer Taylor 17 Jun 19, 2022
Path Tracking PID offers a tuneable PID control loop, decouling steering and forward velocity

path_tracking_pid Overview Path Tracking PID offers a tuneable PID control loop decouling steerting and forward velocity. The forward velocity is gene

Nobleo Technology 71 Jun 25, 2022
Swift Framework for Apple Health Integration with Terra Enabling Developers LTD.

TerraSwift This library allows developers to connect to TERRA ENABLING DEVELOPERS LTD. through Swift (implemented with Swift5.0). This library uses He

null 7 Mar 18, 2022
Kexts enabling native support for Intel Bluetooth chipsets in macOS.

IntelBluetoothFamily Kexts enabling native support for Intel Bluetooth chipsets in macOS. Most of the code is complete, and I am now in the testing ph

null 119 Jun 26, 2022
libu8ident - Follow unicode security guidelines for identifiers

libu8ident - Follow unicode security guidelines for identifiers without adding the full Unicode database. This library does the unicode identifier sec

Reini Urban 5 Mar 3, 2022
This is the laplight software for enabling flashlight support on a laptop/netbook. For the specification, see: https://github.com/LapLight/

By: Seanpm2001, Et; Al. Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrikaans Afrikaans |

Sean P. Myrick V19.1.7.2 2 Dec 8, 2021
Simdutf - Unicode routines (UTF8, UTF16): billions of characters per second.

simdutf: Unicode validation and transcoding at billions of characters per second Most modern software relies on the Unicode standard. In memory, Unico

simdutf: Unicode at gigabytes per second 159 Jun 26, 2022
Neo - Simulates the digital rain from "The Matrix" (cmatrix clone with 32-bit color and Unicode support)

neo WARNING: neo may cause discomfort and seizures in people with photosensitive epilepsy. User discretion is advised. neo recreates the digital rain

Stew Reive 390 Jun 18, 2022
CppUTest For QP/C++ implements a CppUTest port of the QP Framework, with supporting utilities, enabling easy host based unit testing of active objects.

CppUTest for the QP/C++ Real-Time Embedded Framework Build and Test status: Copyright Matthew Eshleman If this project inspires your team to select th

Cove Mountain Software 4 May 14, 2022
Pandex is a light but FAST programming language written in C . Pandex goal is that be hard & it's good for eductional goals

The Pandex programming language version 1.0.0.3 Pandex versions release type The Pandex version has four numbers. the first number holds 1 ( or 0 in s

null 8 May 23, 2022
Kernel source for j7y17lte - the goal is to make it as closest to linux-stable sources as possible without breaking OneUI compatibility.

Linux kernel release 3.x <http://kernel.org/> These are the release notes for Linux version 3. Read them carefully, as they tell you what this is al

Exynos7870 1 Oct 28, 2021