Any plans to make it compatible with clang?

Problematic line: https://github.com/llohse/libnpy/blob/master/npy.hpp#L67

-- Check for working C compiler: /root/workspace/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang
-- Check for working C compiler: /root/workspace/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -- works

-- Check for working CXX compiler: /root/workspace/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++
-- Check for working CXX compiler: /root/workspace/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -- works

/root/workspace/neo-ai-dlr/3rdparty/libnpy/npy.hpp:66:30: error: constexpr variable cannot have non-literal type 'const std::array<char, 3>'
constexpr std::array<char,3> endian_chars = { little_endian_char, big_endian_char, no_endian_char };
                             ^
/root/workspace/neo-ai-dlr/3rdparty/libnpy/npy.hpp:66:30: error: implicit instantiation of undefined template 'std::__ndk1::array<char, 3>'
/root/workspace/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/__tuple:223:64: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TEMPLATE_VIS array;
                                                               ^
In file included from /root/workspace/neo-ai-dlr/demo/cpp/run_resnet.cc:16:
/root/workspace/neo-ai-dlr/3rdparty/libnpy/npy.hpp:67:30: error: constexpr variable cannot have non-literal type 'const std::array<char, 4>'
constexpr std::array<char,4> numtype_chars = { 'f', 'i', 'u', 'c' };
                             ^
/root/workspace/neo-ai-dlr/3rdparty/libnpy/npy.hpp:67:30: error: implicit instantiation of undefined template 'std::__ndk1::array<char, 4>'
/root/workspace/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/__tuple:223:64: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TEMPLATE_VIS array;

More on the issue: https://stackoverflow.com/questions/46576847/clang-vs-gcc-crtp-constexpr-variable-cannot-have-non-literal-type

C++ named requirements: LiteralType: https://en.cppreference.com/w/cpp/named_req/LiteralType

In read_header(), header length is read as an unsigned, little-endian short int, which is currently done by reading two char and concatenating them into an unsigned int.

However the sign-ness of char in C++ is not specified (implementation defined). When char is treated as signed char, the code immediately overflows for byte in range [128, 255], and causing all latter buffer allocating go wrong.

  uint32_t header_length;
  if (version == version_t{1, 0}) {
    char header_len_le16[2];
    istream.read(header_len_le16, 2);
    header_length = (header_len_le16[0] << 0) | (header_len_le16[1] << 8); // header_length incorrect for signed char!
  .......

In my project, we include npy.hpp in different source files, generate .o files, and link them together. However, gcc-10 defaults to -fno-common . This part is raising duplicate symbol error while linking https://github.com/llohse/libnpy/blob/b67016355374fe29d75430842d72865f99e3f697/include/npy.hpp#L131

To reproduce: case.zip Based on libnpy/tests, split test.cpp into 3 cpp files, each containing one function, i.e. load, save, and main. Compile with g++-10 -c, and link. The linker generates:

duplicate symbol 'npy::has_typestring::dtype' in: test_load.o test_save.o duplicate symbol 'npy::has_typestring::dtype' in: test_load.o test_save.o ...

for each has_typestring<T>.

In C++, the signedness of char is platform dependent.

FIY: Fundamental types - cppreference.com

The signedness of char depends on the compiler and the target platform: the defaults for ARM and PowerPC are typically unsigned, the defaults for x86 and x64 are typically signed.

It is better to use signed char instead of char in signedness-aware coding.

• Use custom to_string because it is not available in the standard library in some Android toolchains.
• Fix std::string initialization from char array by explicitly specifying the string length (to avoid including null terminator, which might happen with some compilers).

LoadArrayFromNumpy(const std::string&, std::vector &, std::vector &) was declared after LoadArrayFromNumpy(const std::string&, std::vector &, bool &, std::vector &) and was using it.

Hi,

Using the lib I spotted an ASAN issue du to: l.476

  auto buf_v = std::vector<char>();
  buf_v.reserve(header_length);
  istream.read(buf_v.data(), header_length);

it seems that reserve not changing the size of the vector and using istream read on the buffer (which does not change the size again) lead to a container overflow later on.

The proper way would be to use a resize or simply declare the vector with the correct size:

  auto buf_v = std::vector<char>(header_length);
  istream.read(buf_v.data(), header_length);

This get rid of the issue.

Cheers, Matt

Thanks for making this library! When I'm trying to use it, I found some room for improvement so I changed the codes in the following aspects. I really hope you can help review it. Thanks!

1. Changed sprintf into snprintf because the latter is a safer function.
2. Supported numpy scalars as 0-dimensional arrays. Test cases are expanded to include these cases.
3. Add an interface such that SaveArrayAsNumpy can work with raw pointer buffer, which is very useful if the user is using a contiguous container other than std::vector, such as Eigen::Matrix or std::array.

I've exposed "fortran_order" in LoadArrayFromNumpy because (as shown in added examples created by createnpy.py) transposing the matrix in numpy only changes that attribute by default and the layout of the matrix data after the header stays the same as without trasposition. Having fortran_order is therefore crucial for interpreting the data. Not wanting to break the interface I added overloaded function. Also silenced a casting warning.

Debuger Error;

abort() has been called

490 line; read_header0x00b739c0{Project1.exe!npy::read_header(std::basic_istream<char,std::char_traits> &)}

unable to access read_header function, parameter

Foremost, I should have thank you for an impressive project. However, for a wider approach to this project, it seems like there has no support for saving libtorch datatypes into .npy without catching errors yet. Should there be a feature like this in the future, it would be very helpful for those using torch via C++. Regards.

I find i can't load the numpy file due to "terminate called after throwing an instance of "'std::runtime_error' what(): formatting error: typestrings not matching", and i tried "std::vector<std::complex> data", std::vector<std::complex> data, std::vector data, std::vector data and so on, it didn't work too, the size of my numpy file is (1, 25, 224,224). here is my code: npy::LoadArrayFromNumpy("00000_1_video.npy", video.shape, video.fortran_order, video.data);