Attatch new library for bFloat16 and posit number systems https://github.com/stillwater-sc/universal

Planning to make use of BLAS implementation from this library for CPU implementation. POSIT numbers and bFloat16 will be converted to IEEE754 form for GPU implementation.

As new library assumes support of AVX2 instructions, We are going to assume support of AVX2 as well

There are many classes related to Tensor in this framework. (Tensor/TensorData/TensorInfo/TensorObject/TensorShape) I would like to let you know the role of each class. If you like, you can write in Korean.

We can fix mtl::mat::dense2D as default conversion type from TensorData, but if we are implementing any custom matrix rather than MTL, It might be better to define some custom matrix type and custom functions that can convert custom matrix type into MTL types.

This is required since data alignment of custom implementation can be different from alignment that MTL is using. Also, since hpr-blas implementation of non-posit type matrix multiplication requires operators such as * or + , our custom matrix&vector type should support them.

How do you guys think about it? should we fix MTL types as our default matrix type? or should we define our own and add some custom implementations?

basic graph creation and some example has been implemented. still, we need to consider ways to initialize default weights and biases, and way to stream data into the placeHolder

Trying to build DNN I ran into dependency issues:

Selecting Windows SDK version 10.0.16299.0 to target Windows 10.0.17134. CMake Error at CMakeLists.txt:39 (add_subdirectory): The source directory

C:/Users/tomtz/Documents/dev/clones/CubbyDNN/Libraries/googletest

does not contain a CMakeLists.txt file.

CMake Error at CMakeLists.txt:42 (add_subdirectory): The source directory

C:/Users/tomtz/Documents/dev/clones/CubbyDNN/Libraries/googlebenchmark

does not contain a CMakeLists.txt file.

Configuring incomplete, errors occurred! See also "C:/Users/tomtz/Documents/dev/clones/CubbyDNN/build/CMakeFiles/CMakeOutput.log".

C++ don't support Half-precision floating-point format. Most deep learning frameworks support this type. So, we have to implement Half class to support it.

Universal type wrapper

• To minimize templates, We will put universal wrapper with its type enum
• Classes, methods that implementation doesn't differ by data types will use this general form
• std::reinterpret_cast will be performed to turn them into actual type, and they will be passed to template functions when different implementation is required on different types

Advantages

• Minimize binary size by minimizing templates
• Makes cleaner, readable code with better consistency

• [ ] Get length of string
• [ ] Join given string in string list into one string
• [ ] Split elements based on sep
• [ ] Check that given string matches the regular expression pattern
• [ ] Replace elements that matches regex
• [ ] Return substrings

Role of Graph compile process

• Verify Graph is valid and ready to run (check for tensor shapes, variable,constant configurations)
• Initialize Variables (Several options are possible such as Random distribution, constants, or other user-customized algorithms)

• function Constant: Creates a constant tensor.
• class Variable: A variable maintains state in the graph across calls to run(). You add a variable to the graph by constructing an instance of the class Variable.
• function Placeholder: Inserts a placeholder for a tensor that will be always fed.

• Implement code
  • Convert Tensor-related classes to template (#16)
  • Implement Half class (#17)
  • Implement Graph class (#18)
  • Implement Constant/Variable/Placeholder (#23)
• CI and code coverage
  • Add code quality tool (#19)
  • Add code coverage tool (Codecov) (#20)
  • Apply lcov settings to generate code coverage report (#21)

• Fix computation code to work on Linux and macOS
• Add computation code for supporting GPU parallel using CUDA
• Implement base code for making CNN example
• Create more nodes for improving availability

To put image to a tensor, implementation of image class is needed

Basic Features

• Support some image types(jpg, png, gif, bmp)
• Load and save images
• Resizable and croppable
• Rotate, transform
• Convertable to tensor(or array)

Additional features

• Gaussian blur
• Noise
• Adjust hue, rgb etc...
• ETC...

This revision sets lcov to generate code coverage report.

CAUTION lcov --directory . --capture --output-file coverage.info doesn't work. I added the compiler flag and link flag, and when I built it, the .gcno file was created, but the .gcda file was not created. It seems to be a problem caused by the updated version of gcov. (WARNING: no .gcda files found in .)

The following is a list of commands to print a report using lcov.

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug
make -j 8
lcov -c -i -d Tests/UnitTests -o base.info
bin/UnitTests
lcov -c -d Tests/UnitTests -o test.info
lcov -a base.info -a test.info -o coverage.info
lcov -r coverage.info '/usr/*' -o coverage.info
lcov -r coverage.info '*/Libraries/*' -o coverage.info
lcov -r coverage.info '*/Tests/*' -o coverage.info
lcov -l coverage.info
genhtml coverage.info -o out