Scnlib - scanf for modern C++

Overview

scnlib

Linux macOS Windows Latest Release License C++ Standard

#include <scn/scn.h>
#include <cstdio>

int main() {
    int i;
    // Read an integer from stdin
    // with an accompanying message
    scn::prompt("What's your favorite number? ", "{}", i);
    printf("Oh, cool, %d!", i);
}

// Example result:
// What's your favorite number? 42
// Oh, cool, 42!

What is this?

scnlib is a modern C++ library for replacing scanf and std::istream. This library attempts to move us ever so closer to replacing iostreams and C stdio altogether. It's faster than iostream (see Benchmarks) and type-safe, unlike scanf. Think {fmt} but in the other direction.

This library is the reference implementation of the ISO C++ standards proposal P1729 "Text Parsing".

This library is currently of pre-release quality (version 0.4), but its interface and behavior should remain relatively stable. Unless significant design flaws can be found, the next major release will be 1.0-rc1. 1.0 will be released when an rc-version will prove itself reasonably bug-free.

Documentation

The documentation can be found online, from https://scnlib.readthedocs.io.

To build the docs yourself, build the doc and doc-sphinx targets generated by CMake. The doc target requires Doxygen, and doc-sphinx requires Python 3.8, Sphinx and Breathe.

Examples

Reading a std::string

#include <scn/scn.h>
#include <iostream>
#include <string_view>

int main() {
    std::string word;
    auto result = scn::scan("Hello world", "{}", word);

    std::cout << word << '\n'; // Will output "Hello"
    std::cout << result.string() << '\n';  // Will output " world!"
}

Reading multiple values

#include <scn/scn.h>

int main() {
    int i, j;
    auto result = scn::scan("123 456 foo", "{} {}", i, j);
    // result == true
    // i == 123
    // j == 456

    std::string str;
    ret = scn::scan(result.range(), "{}", str);
    // result == true
    // str == "foo"
}

Using the tuple-return API

#include <scn/scn.h>
#include <scn/tuple_return.h>

int main() {
    auto [r, i] = scn::scan_tuple<int>("42", "{}");
    // r is a result object, contextually convertible to `bool`
    // i == 42
}

Error handling

#include <scn/scn.h>
#include <string_view>
#include <iostream>

int main() {
    int i;
    // "foo" is not a valid integer
    auto result = scn::scan("foo", "{}", i);
    if (!result) {
        // i is not touched (still unconstructed)
        // result.range() == "foo" (range not advanced)
        std::cout << "Integer parsing failed with message: " << result.error().msg() << '\n';
    }
}

Features

  • Blazing-fast parsing of values (see benchmarks)
  • Modern C++ interface, featuring type safety (variadic templates), convenience (ranges) and customizability
    • No << chevron >> hell
    • Requires C++11 or newer
  • "{python}"-like format string syntax
  • Optionally header only
  • Minimal code size increase (see benchmarks)
  • No exceptions (supports building with -fno-exceptions -fno-rtti with minimal loss of functionality)
    • Localization requires exceptions, because of the way std::locale is

Installing

scnlib uses CMake. If your project already uses CMake, integration is easy. First, clone, build, and install the library

# Whereever you cloned scnlib to
$ mkdir build
$ cd build
$ cmake ..
$ make -j
$ make install

Then, in your project:

# Find scnlib package
find_package(scn CONFIG REQUIRED)

# Target which you'd like to use scnlib
# scn::scn-header-only to use the header-only version
add_executable(my_program ...)
target_link_libraries(my_program scn::scn)

Alternatively, if you have scnlib downloaded somewhere, or maybe even bundled inside your project (like a git submodule), you can use add_subdirectory:

add_subdirectory(path/to/scnlib)

# like above
add_executable(my_program ...)
target_link_libraries(my_program scn::scn)

See docs for usage without CMake.

Compiler support

Every commit is tested with

  • gcc 5.5 and newer (until v10)
  • clang 6.0 and newer (until v10)
  • Visual Studio 2017 and 2019
  • gcc 10 on ARM, and clang on macOS

with very extreme warning flags (see cmake/flags.cmake) and with multiple build configurations for each compiler.

Other compilers and compiler versions may work, but it is not guaranteed. If your compiler does not work, it may be a bug in the library. However, support will not be provided for:

  • GCC 4.9 (or earlier): C++11 support is too buggy
  • VS 2015 (or earlier): unable to handle templates

Benchmarks

Run-time performance

Benchmark results

These benchmarks were run on a Ubuntu 20.04 machine running kernel version 5.4.0-52, with an Intel Core i5-6600K processor, and compiled with gcc version 9.3.0, with -O3 -DNDEBUG -march=native. The source code for the benchmarks can be seen in the benchmark directory.

You can run the benchmarks yourself by enabling SCN_BUILD_BENCHMARKS. SCN_BUILD_BENCHMARKS is enabled by default if scn is the root CMake project, and disabled otherwise.

$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release -DSCN_BUILD_BENCHMARKS=ON -DSCN_NATIVE_ARCH=ON -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON ..
$ make -j
# choose benchmark to run in ./benchmark/runtime/*/bench-*
$ ./benchmark/runtime/integer/bench-int

Performance comparison benchmarks with Boost.Spirit.x3 can be found here

Times are in nanoseconds of CPU time. Lower is better.

Integer parsing (int)

Test std::stringstream sscanf scn::scan scn::scan_default
Test 1 274 96.5 43.0 40.3
Test 2 77.7 526 68.1 60.5

Floating-point parsing (double)

Test std::stringstream sscanf scn::scan
Test 1 416 164 167
Test 2 223 570 195

Reading random whitespace-separated strings

Character type scn::scan scn::scan and string_view std::stringstream
char 40.7 38.0 50.2
wchar_t 42.7 38.3 122

Test 1 vs. Test 2

In the above comparisons:

  • "Test 1" refers to parsing a single value from a string which only contains the string representation for that value. The time used for constructing parser state is included. For example, the source string could be "123". In this case, a parser is constructed, and a value (123) is parsed. This test is called "single" in the benchmark sources.
  • "Test 2" refers to the average time of parsing a value from a string containing multiple string representations separated by spaces. The time used for constructing parser state is not included. For example, the source string could be "123 456". In this case, a parser is constructed before the timer is started. Then, a single value is read from the source, and the source is advanced to the start of the next value. The time it took to parse a single value is averaged out. This test is called "repeated" in the benchmark sources.

Code size

Code size benchmarks test code bloat for nontrivial projects. It generates 25 translation units and reads values from stdin five times to simulate a medium sized project. The resulting executable size is shown in the following tables.

The code was compiled on Ubuntu 20.04 with g++ 9.3.0. scnlib is linked dynamically to level out the playing field compared to already dynamically linked libc and libstdc++. See the directory benchmark/bloat for more information, e.g. templates for each TU.

To run these tests yourself:

$ cd build
# For Debug
$ cmake -DCMAKE_BUILD_TYPE=Debug -DSCN_BLOAT=ON -DBUILD_SHARED_LIBS=ON -DSCN_INSTALL=OFF ..
# For Release
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON -DSCN_BLOAT=ON -DBUILD_SHARED_LIBS=ON -DSCN_INSTALL=OFF ..
# For Minimized Release
$ cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON -DSCN_BLOAT=ON -DBUILD_SHARED_LIBS=ON -DSCN_INSTALL=OFF ..

$ make -j
$ ./benchmark/bloat/run-bloat-tests.py ./benchmark/bloat

Sizes are in kibibytes (KiB). Lower is better.

Minimized build (-Os -DNDEBUG)

Method Executable size Stripped size
empty 18 14
scanf 23 18
std::istream / std::cin 25 18
scn::input 35 30
scn::input (header only) 138 98

Release build (-O3 -DNDEBUG)

Method Executable size Stripped size
empty 18 14
scanf 24 18
std::istream / std::cin 30 22
scn::input 41 34
scn::input (header only) 177 146

Debug build (-g)

Method Executable size Stripped size
empty 29 14
scanf 600 18
std::istream / std::cin 662 22
scn::input 1709 51
scn::input (header only) 6858 281

Build time

This test measures the time it takes to compile a binary when using different libraries. Note, that the time it takes to compile the library is not taken into account (unfair measurement against precompiled stdlibs).

These tests were run on an Ubuntu 20.04 machine with an i5-6600K and 16 GB of RAM, using GCC 9.3.0. The compiler flags for a debug build were -g, and -O3 -DNDEBUG for a release build.

To run these tests yourself, enable CMake flag SCN_BUILD_BUILDTIME. In order for these tests to work, c++ must point to a gcc-compatible C++ compiler binary, and a POSIX-compatible /usr/bin/time must be present.

$ cd build
$ cmake -DSCN_BUILD_BUILDTIME=ON ..
$ make -j
$ ./benchmark/buildtime/run-buildtime-tests.sh

Build time

Time is in seconds of CPU time (user time + sys/kernel time). Lower is better.

Method Debug Release
empty 0.03 0.04
scanf 0.24 0.25
std::istream / std::cin 0.29 0.31
scn::input 0.53 0.62
scn::input (header only) 1.38 2.54

Memory consumption

Memory is in mebibytes (MiB). Lower is better.

Method Debug Release
empty 22.3 23.9
scanf 47.0 46.7
std::istream / std::cin 55.2 54.7
scn::input 82.9 83.9
scn::input (header only) 143.1 167.6

Acknowledgements

The contents of this library are heavily influenced by {fmt} and its derivative works.
https://github.com/fmtlib/fmt

The bundled ranges implementation found from this library is based on NanoRange:
https://github.com/tcbrindle/NanoRange

License

scnlib is licensed under the Apache License, version 2.0.
Copyright (c) 2017 Elias Kosunen
See LICENSE for further details

See the directory licenses/ for third-party licensing information.

Issues
  • Scanning with width, or customize separator.

    Scanning with width, or customize separator.

    What if input string is not seperate by space?

        std::string_view v1, v2;
        auto ret = scn::scanf("42,43", "%2s,%2s", v1, v2);
    // or
        auto ret = scn::scan("42,43", "{:2}{:2}", v1, v2);
    

    both failed. but sscanf works fine with following code:

        char buf1[20], buf2[20];
        sscanf("42,43", "%2s,%2s", buf1, buf2);
    
    opened by nanoric 8
  • Unable to use scn::scan with string_view under MSVC.

    Unable to use scn::scan with string_view under MSVC.

    The code from README is unable to compile under MSVC (Visual Studio 2019 16.4.2):

    #pragma once
    #include <string>
    #include <string_view>
    #include <scn/scn.h>
    #include <iostream>
    
    using namespace std;
    using namespace std::string_literals;
    using namespace std::string_view_literals;
    
    int main()
    {
        std::string_view str = "Hello world!"sv;
    
        std::string word;
        scn::scan(str, "{}", word);
    
        std::cout << word << '\n'; // Will output "Hello"
        std::cout << str << '\n';  // Will output " world!"
    }
    
    
    

    Error Output:

    1>Source.cpp
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(77,1): error C2440: 'return': cannot convert from 'initializer list' to 'std::basic_string_view<char,std::char_traits<char>>'
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(77,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(121): message : see reference to function template instantiation 'std::basic_string_view<char,std::char_traits<char>> scn::v0::detail::reconstruct<char,std::char_traits<char>,std::_String_view_iterator<_Traits>,std::_String_view_iterator<_Traits>>(scn::v0::detail::reconstruct_tag<std::basic_string_view<char,_Traits>>,Iterator,Sentinel)' being compiled
    1>        with
    1>        [
    1>            _Traits=std::char_traits<char>,
    1>            Iterator=std::_String_view_iterator<std::char_traits<char>>,
    1>            Sentinel=std::_String_view_iterator<std::char_traits<char>>
    1>        ]
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(119): message : while compiling class template member function 'std::basic_string_view<char,std::char_traits<char>> scn::v0::detail::range_wrapper<std::string_view &>::range(void) const'
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(126): message : see reference to function template instantiation 'std::basic_string_view<char,std::char_traits<char>> scn::v0::detail::range_wrapper<std::string_view &>::range(void) const' being compiled
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(263): message : see reference to class template instantiation 'scn::v0::detail::range_wrapper<std::string_view &>' being compiled
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\range.h(261): message : while compiling class template member function 'scn::v0::detail::range_wrapper<std::string_view &> scn::v0::detail::_wrap::fn::operator ()<std::string_view&>(Range) noexcept const'
    1>        with
    1>        [
    1>            Range=std::string_view &
    1>        ]
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\scan.h(32): message : see reference to class template instantiation 'scn::v0::detail::range_wrapper_for<Range>' being compiled
    1>        with
    1>        [
    1>            Range=std::string_view &
    1>        ]
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\scan.h(32): message : see reference to alias template instantiation 'scn::v0::detail::range_wrapper_for_t<std::basic_string_view<char,std::char_traits<char>>&>' being compiled
    1>C:\Users\Weizehua\source\repos\test_scnlib\test_scnlib\Source.cpp(15): message : see reference to class template instantiation 'scn::v0::detail::scan_result_for_range<std::string_view &,scn::v0::wrapped_error>' being compiled
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\scan.h(73): message : see reference to alias template instantiation 'scn::v0::detail::scan_result_for_range_t<std::string_view&,scn::v0::wrapped_error>' being compiled
    1>C:\projects\vcpkg\installed\x64-windows\include\scn\detail\scan.h(135): message : see reference to function template instantiation 'scan_result_for_range<Range,scn::v0::wrapped_error>::type scn::v0::scan(Range &&,scn::v0::detail::default_t,Args &...)' being compiled
    1>Done building project "test_scnlib.vcxproj" -- FAILED.
    

    It seems std::basic_string_view don't accept an iterator and a length as arguments. std::basic_string_view can only construct from (char *, size_t), So this forum is accetable:

    // range.h #77:
    // distance uses end - begin, which is unsupported for std::string_view with different range in DEBUG mode.
                return {&*begin, static_cast<size_t>(std::to_address(end) - std::to_address(begin))};
    

    BTW, I guess the test case inside string_view.cpp should construct an run time string(std::string, raw string buffer, etc) instead of passing a compile-time string literal as first argument to scn::scan. Because nobody needs a library that scans from compile-time string.

    opened by nanoric 8
  • Locale not respected when parsing floating-points

    Locale not respected when parsing floating-points

    Hi! I discovered a locale-related parsing bug.

    For floating-point values, the docs say:

    First, there's a localization specifier:

    • n: Use decimal and thousands separator from the given locale
    • (default): Use . as decimal point and , as thousands separator

    This would suggest that parsing of floating-points is independent of locale by default. This turns out to be not the case. After receiving bug reports from users with more outlandish environments, I investigated and found that the default floating-point parser uses STL's std::strto{f,d,ld} functions, which do depend on the locale. Furthermore, I found no trace of code that would respond to the localization specifier mentioned above.

    Here's an obligatory minimal failing example with decimal commas:

    #include <string>
    #include <iostream>
    #include <scn/scn.h>
    
    void main()
    {
      double value_point;
      const std::string str_point{"123.456"};
      auto ret_point{scn::scan(str_point, "{}", value_point)};
      std::cout << "parsed decimal point: " << ret_point << ", value: " << value_point << std::endl;
    
      double value_comma;
      const std::string dec_comma{"123,456"};
      auto ret_comma{scn::scan(dec_comma, "{}", value_comma)};
      std::cout << "parsed decimal comma: " << ret_comma << ", value: " << value_comma << std::endl;
    
      // with LC_ALL=cs_CZ.UTF-8:
      //   parsed decimal point: 0, value: 0
      //   parsed decimal comma: 1, value: 123.456
      // ... this is the behavior I would expect from the format string "{:n}", not from "{}"
    }
    
    opened by petrmanek 7
  • Stack Overflow

    Stack Overflow

    Running the readme example with v0.2 and Visual Studio 2019 I get a stack overflow on this line https://github.com/eliaskosunen/scnlib/blob/104405179ccb4c51feca63e979f56abf0b3f11b3/include/scn/detail/range.h#L250

    opened by SuperWig 5
  • scn::scan/prompt from stdin with literal text does not work

    scn::scan/prompt from stdin with literal text does not work

    When trying to scan a format string with a literal at the start, a debug assertions fires:

    File: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\include\xstring
    Line: 1853
    
    Expression: cannot seek string iterator before begin
    

    Code I used, adapted from the examples (using scnlib 0.3):

    #include <scn/scn.h>
    #include <iostream>
    
    int main() {
        int i = 0;
        if (scn::prompt("Hi there! What's your favorite number? ", "uhh, {}", i)) {
            std::cout << "Ooh, " << i << ", interesting!\n";
        }
        else {
            std::cout << "That doesn't look like any number I've ever seen.\n";
        }
    }
    
    opened by melak47 4
  • How to scan from std::string

    How to scan from std::string

    With an older version of this library, I used to do

    std::string str = ...;
    auto stream = scn::make_stream(str);
    auto ret = scn::scan(stream, ...);
    

    After updating to the latest, make_stream is no longer present (although it still appears in some test files).

    What is the canonical way to parse from a std::string? Is there a reason this doesn't work (I get verbose template errors)?

    std::string str = ...;
    auto ret = scn::scan(str, ...);
    

    I had some success with

    std::string str = ...;
    auto ret = scn::scan(std::string_view(str), ...);
    

    but that seems unnecessarily verbose and there might be some issues with AppleClang on macOS.

    Before i dig deeper, to find the error, I wanted to check what is the way it's supposed to work. Maybe scanning from std::string warrants a prominent example (in the Readme; the docs seem to be down).

    opened by NikolausDemmel 4
  • Segfault

    Segfault

    Trying out the library and can't run the examples. The problem seems to be this line https://github.com/eliaskosunen/scnlib/blob/b9adabe4806bae4dd749791c09ed5e4acc83af36/include/scn/detail/small_vector.h#L705

    opened by SuperWig 4
  • Scaning to Char []?

    Scaning to Char []?

    Without support to char[], we must construcat at least one std::string before calling scn::scan, if the input contains string, which is anoying.

    What's more, istream support scanning to char[]. People (such as me) will get confused if scanning into char[] is unsupported when porting code from istream from scnlib.

    opened by nanoric 3
  • Returning scanned values instead of passing by reference

    Returning scanned values instead of passing by reference

    A concern has been raised by multiple people, as to why does scn take arguments by reference, instead of returning a tuple, forcing the user to default construct their arguments.

    // What we have now
    int i;
    scn::input("{}", i);
    // Hypothetical alternative
    auto [err, i] = scn::input<int>("{}");
    
    opened by eliaskosunen 3
  • check value of _HAS_EXCEPTIONS instead of only it's existence

    check value of _HAS_EXCEPTIONS instead of only it's existence

    It's a common practice to define _HAS_EXCEPTIONS=0 to disable exceptions on windows. This can result in a false positive detection of whether exceptions are present.

    Other libraries like {fmt} also check the value of _HAS_EXCEPTIONS. https://github.com/fmtlib/fmt/blob/7e4ad40171aa552d38cb99a5c181a0d7b150facc/include/fmt/core.h#L146

    opened by Agga 2
  • small-vector test failure when building with GCC 11

    small-vector test failure when building with GCC 11

    small-vector test failure when building with GCC 11.

    GCC version 11.0.0.

    Log:

    + /usr/bin/ctest --output-on-failure --force-new-ctest-process -j48
    Test project /builddir/build/BUILD/scnlib-0.4/x86_64-redhat-linux-gnu
          Start  1: test
          Start  2: empty
          Start  3: util
          Start  4: small-vector
          Start  5: string-view
          Start  6: reader
          Start  7: range
          Start  8: locale
          Start  9: std-string-view
          Start 10: pmr-string
          Start 11: result
          Start 12: istream
          Start 13: tuple-return
          Start 14: char
          Start 15: integer
          Start 16: float
          Start 17: string
          Start 18: buffer
          Start 19: bool
          Start 20: usertype
          Start 21: list
          Start 22: each-integer
          Start 23: each-char
          Start 24: file
     1/24 Test  #1: test .............................   Passed    0.02 sec
     2/24 Test  #2: empty ............................   Passed    0.02 sec
     3/24 Test  #3: util .............................   Passed    0.02 sec
     4/24 Test  #5: string-view ......................   Passed    0.02 sec
     5/24 Test  #6: reader ...........................   Passed    0.02 sec
     6/24 Test  #7: range ............................   Passed    0.01 sec
     7/24 Test  #8: locale ...........................   Passed    0.01 sec
     8/24 Test  #9: std-string-view ..................   Passed    0.01 sec
     9/24 Test #10: pmr-string .......................   Passed    0.01 sec
    10/24 Test #11: result ...........................   Passed    0.01 sec
    11/24 Test #12: istream ..........................   Passed    0.01 sec
    12/24 Test #13: tuple-return .....................   Passed    0.01 sec
    13/24 Test #14: char .............................   Passed    0.01 sec
    14/24 Test #15: integer ..........................   Passed    0.01 sec
    15/24 Test #16: float ............................   Passed    0.01 sec
    16/24 Test #17: string ...........................   Passed    0.01 sec
    17/24 Test #18: buffer ...........................   Passed    0.01 sec
    18/24 Test #19: bool .............................   Passed    0.01 sec
    19/24 Test #20: usertype .........................   Passed    0.01 sec
    20/24 Test #21: list .............................   Passed    0.01 sec
    21/24 Test #23: each-char ........................   Passed    0.00 sec
    22/24 Test #24: file .............................   Passed    0.00 sec
    23/24 Test #22: each-integer .....................   Passed    0.07 sec
    24/24 Test  #4: small-vector .....................Subprocess aborted***Exception:   0.10 sec
    free(): invalid size
    [doctest] doctest version is "2.4.4"
    [doctest] run with "--help" for options
    ===============================================================================
    ../test/small_vector.cpp:263:
    TEST CASE:  small_vector<signed char>
      size+value construct stack
    ../test/small_vector.cpp:97: ERROR: CHECK( vec.is_small() ) is NOT correct!
      values: CHECK( false )
    ../test/small_vector.cpp:99: ERROR: CHECK( vec.capacity() == 64 ) is NOT correct!
      values: CHECK( 3038287259199220266 == 64 )
    ===============================================================================
    ../test/small_vector.cpp:263:
    TEST CASE:  small_vector<signed char>
      accessors stack
    ../test/small_vector.cpp:263: FATAL ERROR: test case CRASHED: SIGABRT - Abort (abnormal termination) signal
    ===============================================================================
    ../test/small_vector.cpp:263:
    TEST CASE:  small_vector<signed char>
    DEEPEST SUBCASE STACK REACHED (DIFFERENT FROM THE CURRENT ONE):
      accessors stack
    ===============================================================================
    [doctest] test cases:  1 |  0 passed | 1 failed | 5 skipped
    [doctest] assertions: 41 | 39 passed | 2 failed |
    [doctest] Status: FAILURE!
    96% tests passed, 1 tests failed out of 24
    Total Test time (real) =   0.11 sec
    The following tests FAILED:
    	  4 - small-vector (Subprocess aborted)
    
    opened by xvitaly 2
  • Minor issues in documentation

    Minor issues in documentation

    https://scnlib.readthedocs.io/en/latest/guide.html?highlight=make_value#scn-scan-value

     To use make_result with make_value, this needs to be taken into account
    

    Not sure, but I guess this should probably be scan_value instead of make_value:

     To use make_result with scan_value, this needs to be taken into account
    

    https://scnlib.readthedocs.io/en/latest/rationale/vscan.html?highlight=peace

    peace => piece

    opened by ea1111 1
  • Conditional locale support

    Conditional locale support

    This fixes the remaining issues in #69. It was not as difficult as I imagined, but it took a much deeper understanding of the codebase. Note that I dropped the locale override macros, as I later realized they were ill-conceived.

    I have tried to be as minimally invasive as possible, mainly cutting out call-sites and letting the optimizer take care of the rest. This may not be the best solution, but it required the fewest changes. As far as I can tell, there are no remnants of std::locale.

    My final size reduction is 135K!

    opened by cjvaughter 1
  • Conditional type support

    Conditional type support

    Resolves non-locale problems listed in #69.

    ~~There are probably a few sections which do not require conditional compilation, as eliminating calls further up the chain would suffice. But I have not determined where best to do so, and this is easy to modify later.~~

    After becoming more acquainted with the code, I found the correct location. I also decided that all types should fall into this setup, not just floating-point.

    The one thing I'm not 100% sure of is the static_assert placement. I think the two of them cover everything, but it's a little hard to tell.

    The size reduction for my project is 23K (no fallbacks; schar, uchar, long, ulong, bool, float, buffer, and custom enabled).

    opened by cjvaughter 2
  • Improvements for embedded targets

    Improvements for embedded targets

    Problem

    This is a great library, but there are a few changes that would drastically improve support for embedded targets. These ideas are mostly based on what has been done for fmt.

    • Conditional locale support would remove an enormous amount of code.
    • Conditional float, double, and long double support would provide another substantial reduction. Especially for targets without a floating-point unit.
    • Making the fast_float -> std::from_chars -> strtod fallback optional would help as well.
    • The current version of ARM GCC has partial support for std::from_chars (no floating-point), so SCN_HAS_FLOAT_CHARCONV misbehaves.
    • The bundled fast_float is missing two PRs that fix ARM GCC support (#122 & #123).

    Testing

    Targeting a Cortex-M4 and optimizing for size, any reference to scn increased my code size from ~78K to ~284K (+206K)! After taking an ax to some of the code, removing locale and long double, it reduced to ~151K (+73K). Removing the fallback got it down to ~136K (+58K). A total reduction of 148K.

    I believe that's about as good as it's going to get, and I'm happy to accept that size given it makes my application far simpler. It's about double the size of fmt, but that comes with the territory.

    Proposal

    • Equivalents to FMT_USE_FLOAT, FMT_USE_DOUBLE, and FMT_USE_LONG_DOUBLE
    • SCN_DEFAULT_LOCALE_* instead of hard-coded values for scn::v1::detail::locale_defaults so the user can override
    • SCN_USE_STATIC_LOCALE
      • Skip localization
      • Reject 'L' and 'n' format string flags at compile time, fail at runtime, or ignore
    • SCN_SKIP_FROM_CHARS
      • Skip std::from_chars and fallback directly to strtod
    • SCN_SKIP_STRTOD
      • Fail at runtime when strtod fallback would have occurred
    • I'm not sure how to detect partial std::from_chars support. There doesn't seem to be any indication except the documentation. SCN_SKIP_FROM_CHARS may be sufficient for now.
    • Update fast_float

    I could potentially do a PR if I have time. Let me know your thoughts.

    opened by cjvaughter 0
Releases(v1.1.2)
  • v1.1.2(Mar 19, 2022)

  • v1.1.1(Mar 16, 2022)

    • Fix issue with values being skipped when using files and file.sync() (#56)
      • Every call to file.sync() needs to be accompanied by a call to reset_begin_iterator() to the result object
      • This is a temporary fix, permanent fix coming in v2.0.0
    int i;
    auto ret = scn::scan(scn::cstdin(), "{}", i);
    scn::cstdin().sync();
    ret.range().reset_begin_iterator();
    
    // Not necessary with input and prompt
    ret = scn::input("{}", i);
    
    Source code(tar.gz)
    Source code(zip)
  • v1.1(Mar 12, 2022)

    • Add support for scanning 8-bit integers ((un)signed char, (u)int8_t), and characters (char, wchar_t) as integers
    int8_t i1, i2;
    char c1, c2;
    auto ret = scn::scan("1 2 3 4", "{} {:c} {} {:i}", i1, i2, i3, i4);
    // ret == true
    // i1 == 1
    // i2 == '2'
    // c1 == '3'
    // c2 == 4
    
    • Fix usage of external fast_float in CMake (#53, thanks @xvitaly (Vitaly Zaitsev))
    • Fix tests on big endian architectures (#54)
    • Fix alignment issues with small_vector on 32-bit architectures

    Full Changelog: https://github.com/eliaskosunen/scnlib/compare/v1.0...v1.1

    Source code(tar.gz)
    Source code(zip)
  • v1.0(Feb 28, 2022)

    First stable release!

    The library is now deemed production-ready, and backwards-compatibility will be maintained until the next major release comes out, in accordance to semantic versioning. 1.x-versions will be getting security updates after that, until further notice.

    If you're migrating from v1.0-rc1, no major changes have been made, only bugfixes.

    If you're migrating from v0.4, see the migration guide v0.4 -> v1.0.

    If you're new here, see the documentation and the repository.

    The feature highlights below are the same compared to those of v1.0-rc1

    Feature highlights

    New float parsing

    Now, by default, scnlib uses https://github.com/fastfloat/fast_float for parsing floating-point values, falling back on std::from_chars and std::strtod only if necessary.

    This provides even more performance than before: using scn::scan is now 2x to 8x faster than using std::stringstream, when parsing floats.

    New format strings

    Many things have changed (see the migration guide above), and the same format strings may now do different things.

    // alignment
    int i{};
    auto result = scn::scan("***1***", "{:*^}", i);
    // i == 1
    // result.empty() == true
    
    // localization
    double d{};
    result = scn::scan_localized(std::locale{"fi_FI.UTF-8"}, "3,14", "{:L}", d);
    // d == 3.14
    // result.empty() == true
    
    // width
    std::string str1;
    result = scn::scan("abcde", "{:3}", str1);
    // str1 == "abc"
    // result.range() == "de"
    
    // string set
    std::string str2;
    result = scn::scan("123abc", "{:[0-9]}", str2);
    // str2 == "123"
    // result.range() == "abc"
    

    Unicode support

    // Parse Unicode code points
    scn::code_point cp{};
    auto result = scn::scan("äa", "{}", cp);
    // cp == 0xe4
    // result.range() == "a"
    
    // Parse Unicode strings
    std::string s1, s2;
    // s1: read until whitespace
    // s2: read until non-letter character, according to locale
    result = scn::scan_localized(std::locale{"fi_FI.UTF-8"}, "äa1 äa1", "{} {:L[:alpha:]}", s1, s2);
    // s1 == "äa1"
    // s2 == "äa"
    // result.range() == "1"
    

    And more

    See CHANGELOG.md and the documentation for more details

    Full Changelog (from v1.0-rc1): https://github.com/eliaskosunen/scnlib/compare/v1.0-rc1...v1.0

    Source code(tar.gz)
    Source code(zip)
  • v1.0-rc1(Feb 21, 2022)

    Getting closer to 1.0!

    Feature highlights

    New float parsing

    Now, by default, scnlib uses https://github.com/fastfloat/fast_float for parsing floating-point values, falling back on std::from_chars and std::strtod only if necessary.

    This provides even more performance than before.

    New format strings

    Many things have changed (see the migration guide below), and the same format strings may now do different things.

    // alignment
    int i{};
    auto result = scn::scan("***1***", "{:*^}", i);
    // i == 1
    // result.empty() == true
    
    // localization
    double d{};
    result = scn::scan_localized(std::locale{"fi_FI.UTF-8"}, "3,14", "{:L}", d);
    // d == 3.14
    // result.empty() == true
    
    // width
    std::string str1;
    result = scn::scan("abcde", "{:3}", str1);
    // str1 == "abc"
    // result.range() == "de"
    
    // string set
    std::string str2;
    result = scn::scan("123abc", "{:[0-9]}", str2);
    // str2 == "123"
    // result.range() == "abc"
    

    Unicode support

    // Parse Unicode code points
    scn::code_point cp{};
    auto result = scn::scan("äa", "{}", cp);
    // cp == 0xe4
    // result.range() == "a"
    
    // Parse Unicode strings
    std::string s1, s2;
    // s1: read until whitespace
    // s2: read until non-letter character, according to locale
    result = scn::scan_localized(std::locale{"fi_FI.UTF-8"}, "äa äa", "{} {:L[:alpha:]}", s1, s2);
    // s1 == s2 == "äa"
    // result.empty() == true
    

    And more

    See CHANGELOG.md and the documentation, namely the migration guide for more details

    Full Changelog: https://github.com/eliaskosunen/scnlib/compare/v0.4...v1.0-rc1

    Source code(tar.gz)
    Source code(zip)
  • v0.4(Nov 13, 2020)

    Planned to be the last 0.x release before 1.0-rc1.

    Changes and removals

    • Rework source range handling:
      • Non-views are now accepted as source ranges -- this includes types like std::string and std::vector<char>
      • Non-reconstructible ranges are now also accepted -- scanning functions no longer return a reconstructed source range. The member function .range() can be used to scan the range again, and .reconstruct() reconstructs the range, if possible. Other helper member functions are also available.
      • Source ranges are now either taken by const lvalue reference or rvalue reference, so they are no longer modified by scanning functions. To access the leftover range, use the return value of the scanning function.
    • Rewrite file handling, with hopefully way less bugs this time around
      • Remove file_view and caching ranges
      • Move memory mapped files to the public API
    • Remove default_tag, replace with scan_default function template
    • Remove support for scanf syntax, including scn::scanf and scn::basic_scanf_parse_context.
    • Improve Ranges integration:
      • Move custom Ranges implementation to the public API (out from scn::detail::ranges): scn::custom_ranges
      • Integrate standard library Ranges, if available: scn::std_ranges aliased to std::ranges
      • Use stdlib Ranges, if available, fall back to custom implementation: namespace alias scn::ranges, control behavior with SCN_USE_STD_RANGES

    Additions

    • Add more thorough documentation, tests, benchmarks and examples
    • Add scan_list_until

    Fixes and minor stuff

    Source code(tar.gz)
    Source code(zip)
  • v0.3(Feb 19, 2020)

    Largely a bugfix release

    Changes

    • Remove support for partial successes
      • If the reading of any of the arguments given to scan fail, the whole function fails
      • read-field removed from result
    • Overhaul list scanning
      • Add scan_list

    Fixes

    • Fix issues with std::string_view and MSVC debug iterators (#11, #14, #18, #20)
    • Fix some issues with scanning customized types (#15)
    • Add missing support for custom-allocator std::strings (#16)
    • Fix erroneous git command in README (#13)
    • Fix README example
    • Fix erroneous usage of library feature test macros

    Thanks to @nanoric and @SuperWig for bug reports!

    Removals

    • Remove support for non-std::char_traits std::strings
    • Remove support for clang 3.6
    Source code(tar.gz)
    Source code(zip)
  • v0.2(Oct 18, 2019)

    There are so many changes, that writing a complete changelog isn't really possible. The most substantial change is the removal of streams, which have been replaced with C++20-like ranges. There are also some pretty sweet performance improvements; see benchmarks in README for more details.

    See the documentation for more details, a tutorial, and an API reference.

    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(Jun 25, 2019)

    • Add SCN_RANGES CMake option
    • Add scn::temp helper function
    • Fix -Wpadded warnings on clang
    • Fix -Wfloat-equal and -Wconversion warnings on gcc
    • Fix C4146 error on UWP MSVC
    • Add CONTRIBUTING.md
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Jun 25, 2019)

    Quick bugfix release

    • Add more examples
    • Fix #8: Fix segfault when using scn::cstdin() or scn::wcstdin(), caused by the copy and move constructor of small_vector setting data pointer to nullptr if copying/moving from an empty small_vector. (Thanks @SuperWig for reporting!)
    • Fix compilation error when using scn::ranges::get_value.
    • Fix a badge in README (thanks @p1v0t)
    Source code(tar.gz)
    Source code(zip)
  • v0.1(Jun 24, 2019)

Owner
Elias Kosunen
20 years of age, over a half of which spent with C++, with some Python and webdev stuff sprinkled in.
Elias Kosunen
Gaming meets modern C++ - a fast and reliable entity component system (ECS) and much more

EnTT is a header-only, tiny and easy to use library for game programming and much more written in modern C++. Among others, it's used in Minecraft by

Michele Caini 6.9k Jun 24, 2022
Nameof operator for modern C++, simply obtain the name of a variable, type, function, macro, and enum

_ _ __ _____ | \ | | / _| / ____|_ _ | \| | __ _ _ __ ___ ___ ___ | |_ | | _| |

Daniil Goncharov 1.4k Jun 27, 2022
Random for modern C++ with convenient API

Random for modern C++ with convenient API Design goals Supported compilers Integration Five-minute tutorial Number range Common type number range Char

Ilya Polishchuk 683 Jun 20, 2022
A Minimal, Header only Modern c++ library for terminal goodies 💄✨

rang Colors for your Terminal. Windows Demo Example usage #include "rang.hpp" using namespace std; using namespace rang; int main() { cout << "P

Abhinav Gauniyal 1.2k Jun 27, 2022
A modern C++ tweening library

Tweeny Tweeny is an inbetweening library designed for the creation of complex animations for games and other beautiful interactive software. It levera

Leonardo Guilherme de Freitas 559 Jun 21, 2022
Activity Indicators for Modern C++

Highlights Thread-safe progress bars and spinners Header-only library. Grab a copy of include/indicators. Single-header version in single_include/indi

Pranav 2.1k Jun 28, 2022
Table Maker for Modern C++

Source for the above image can be found here Table of Contents Quick Start Formatting Options Style Inheritance Model Word Wrapping Font Alignment Fon

Pranav 1.3k Jun 28, 2022
🏢 A bold, unapologetic, and honest operating system written in modern C

A bold, unapologetic, and honest operating system written in modern C About Striking modernist shapes and bold use of modern C are the hallmarks of BR

Brutal 817 Jun 25, 2022
🏢 An operating system that combine the desire of UNIX utopia from the 1970s with modern technology and engineering

Striking modernist shapes and bold use of modern C are the hallmarks of BRUTAL. BRUTAL combine the desire of UNIX utopia from the 1970s with modern te

Brutal 818 Jun 28, 2022
"Sigma File Manager" is a free, open-source, quickly evolving, modern file manager (explorer / finder) app for Windows, MacOS, and Linux.

"Sigma File Manager" is a free, open-source, quickly evolving, modern file manager (explorer / finder) app for Windows, MacOS, and Linux.

Aleksey Hoffman 892 Jun 27, 2022
Ceres is designed to be a modern and minimalistic C like language.

Ceres v0.0.1 Ceres is designed to be a modern and minimalistic C like language. For now, it will be interpreted but later on I do want to write a comp

null 9 May 18, 2022
External CS:GO hack for Arduino written using modern C++ and WinAPI

SQ Project CSGO Arduino Edition External CS:GO hack for Arduino written using C++ and WinAPI. Special thanks to hazedumper for hazedumper. Shock Byte

Klim Markevich 27 Jun 2, 2022
Enoki: structured vectorization and differentiation on modern processor architectures

Enoki: structured vectorization and differentiation on modern processor architectures

Mitsuba Physically Based Renderer 1.1k Jun 25, 2022
Modern C++ 20 compile time OpenAPI parser and code generator implementation

OpenApi++ : openapipp This is a proof of concept, currently under active work to become the best OpenAPI implementation for C++. It allows compile tim

tipi.build 5 Apr 8, 2022
A secure authentication system written in modern javascript and C++ 17

A fully fledged authentication system written in modern JavaScript and C++ 17. Written with the MEVN stack (MySQL, Express.js, Vue.js, Node.js) by a pro leet h4x0r.

vmexit 20 Jun 13, 2022
A modern-day Boss Key software tool. Switch instantly from work to play & play to work with Bosky.

Bosky By: Seanpm2001, Bosky-dev Et; Al. Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrika

Sean P. Myrick V19.1.7.2 1 Nov 11, 2021
A modern ESM build of the Potrace library for use in the browser.

ESM Potrace Wasm A modern ESM build of the Potrace library for use in the browser. Installation npm install --save esm-potrace-wasm Usage import potra

Thomas Steiner 31 May 19, 2022