(R) Efficient methods and operators for the sparse matrix classes in 'Matrix' (esp. CSR format or "RsparseMatrix")

Overview

MatrixExtra

MatrixExtra is an R package which extends the sparse matrix and sparse vector types in the Matrix package, particularly the CSR or RsparseMatrix formats (row-major), by providing methods that work natively and efficiently on them without converting them to another format along the way, such as slicing (selecting rows/columns) or concatenating by rows/columns, along with replacing some Matrix methods with more efficient versions, such as multi-threaded <sparse, dense> matrix multiplications, much faster slicing for all the sparse types, and faster elementwise addition/subtraction/multiplication, among others.

This package is based on code originally written for the rsparse package by Dmitriy Selivanov.

What's missing from Matrix

The Matrix package provides a rich set of sparse matrix and sparse vector classes with many methods and operators so that they could be used as drop-in replacements of base R's matrices. Unfortunately, the whole package is centered around the CSC format (CsparseMatrix, column-major), and calling methods and operators which in principle should be efficient in CSR or COO formats will imply first converting the whole matrix to CSC format (a slow and inefficient operation which duplicates the data), on which the operation might be less efficient due to the storage order.

(Longer introduction in the vignette)

Examples:

library(Matrix)
X = matrix(c(0,0,2,1, 0,3,0,0, 0,0,0,0), nrow=3, ncol=4, byrow=TRUE)
X = as(X, "RsparseMatrix")
X
3 x 4 sparse Matrix of class "dgRMatrix"
            
[1,] . . 2 1
[2,] . 3 . .
[3,] . . . .
### This will forcibly convert the matrix to triplets
X[1:2, ]
2 x 4 sparse Matrix of class "dgTMatrix"
            
[1,] . . 2 1
[2,] . 3 . .
### This will forcibly convert the matrix to CSC
rbind(X, X)
6 x 4 sparse Matrix of class "dgCMatrix"
            
[1,] . . 2 1
[2,] . 3 . .
[3,] . . . .
[4,] . . 2 1
[5,] . 3 . .
[6,] . . . .
### This will forcibly convert the matrix to CSC
X * X
3 x 4 sparse Matrix of class "dgCMatrix"
            
[1,] . . 4 1
[2,] . 9 . .
[3,] . . . .
### This should be a straighforward operation
as(X, "ngRMatrix")
Error in as(X, "ngRMatrix") : 
  no method or default for coercing “dgRMatrix” to “ngRMatrix”
### This doesn't do what it should
as(X, "nsparseMatrix")
3 x 4 sparse Matrix of class "ngCMatrix"
            
[1,] . . | |
[2,] . | . .
[3,] . . . .

Why is CSR needed?

The CSR sparse format is particularly useful when dealing with machine learning applications - e.g. splitting between a train and test set, tokenizing text features, multiplying a matrix by a vector of coefficients, calculating a gradient observation-by-observation, among others. Many stochastic optimization techniques and libraries (e.g. LibSVM, VowpalWabbit) require the inputs to be in CSR format or alike (see also readsparse), which does not play well with the column-centric methods of Matrix.

In principle, one could stick with just the CSC format from Matrix and keep a mental map of the matrix as being transposed. This however gets complicated rather soon and is very prone to errors. Additionally, one might want to pass sparse matrices to another package whose code is outside of one's control, for which the storage format can make a large difference in performance.

Installation

install.packages("MatrixExtra")

Documentation

Documentation is available on CRAN.

A package vignette is available here (and in CRAN).

Features

  • Multi-threaded matrix multiplications (%*%), crossprod and tcrossprod for many <sparse,dense> and <dense,sparse> types, including those from the float package.
  • Slicing or subsetting (operator [) CSR matrices, along with faster slicing of CSC and COO, and slicing with sparse vectors.
  • Efficient rbinding (concatenating by rows) and cbinding (concatenating by columns) for different sparse matrices and sparse vector types (e.g. rbind(CSR, CSR) and cbind(CSR, CSR)).
  • Overloaded operators for <RsparseMatrix, RsparseMatrix> and some <RsparseMatrix, TsparseMatrix> and <sparse, dense> types, such as +, -, *, &, |.
  • Overloaded mathematical functions and operators which act only on the non-zero entries for CSR and COO matrices, such as sqrt(CSR) or CSR * scalar.
  • Overloaded operators for CSR/COO and vectors, such as CSR * vector, CSR ^ vector, etc., which can mimick all the quirks of base R if needed.
  • Convenience conversion functions between different sparse formats, and registered coercion methods between pairs which are not in Matrix (e.g. matrix -> ngRMatrix or dgRMatrix -> lgCMatrix).
  • Fast transposes which work by outputting in the opposite format (CSR -> CSC and CSC -> CSR).
  • Utility functions for sorting sparse indices and removing zero-valued entries.

TODO

  • Cover more cases in assignment operator ([<-).
  • Matrix multiplications between float32 and sparseVector.
  • Better handling of dimension names of the output matrices.
  • Outer products with sparse vectors.
  • Try to port some parts to Matrix.
  • Add timings against the methods from Matrix.
  • Perhaps add methods specifically for symmetric and triangular matrices.

Examples

library(Matrix)
library(MatrixExtra)
X = matrix(c(0,0,2,1, 0,3,0,0, 0,0,0,0), nrow=3, ncol=4, byrow=TRUE)
X = as(X, "RsparseMatrix")

X[1:2, ]
X + X
X * X
X %*% 1:4
X * 1:3
rbind(X, X)
cbind(X, X)
sqrt(X)
diag(X)

as(as.matrix(X), "dgRMatrix")
as.csc.matrix(X)

### New and optional
set_new_matrix_behavior()
inherits(X[1,,drop=TRUE], "sparseVector")
inherits(t(X), "CsparseMatrix")
restore_old_matrix_behavior()
Issues
  • Problem installing {MatrixExtra} for R 4.2.0 in macOS 12.4 (Intel)

    Problem installing {MatrixExtra} for R 4.2.0 in macOS 12.4 (Intel)

    Hi, after trying for a while to solve this problem I ran out of options. I keep getting this error message. Can you help me?

    install.packages("MatrixExtra")
    Installing package into ‘/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library’
    (as ‘lib’ is unspecified)
    trying URL 'https://cran.rstudio.com/src/contrib/MatrixExtra_0.1.10.tar.gz'
    Content type 'application/x-gzip' length 210398 bytes (205 KB)
    ==================================================
    downloaded 205 KB
    
    * installing *source* package ‘MatrixExtra’ ...
    ** package ‘MatrixExtra’ successfully unpacked and MD5 sums checked
    ** using staged installation
    checking for g++... g++
    checking whether the C++ compiler works... yes
    checking for C++ compiler default output file name... a.out
    checking for suffix of executables... 
    checking whether we are cross compiling... no
    checking for suffix of object files... o
    checking whether the compiler supports GNU C++... yes
    checking whether g++ accepts -g... yes
    checking for g++ option to enable C++11 features... none needed
    configure: creating ./config.status
    config.status: creating src/Makevars
    ** libs
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c assignment.cpp -o assignment.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c cbind.cpp -o cbind.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c matmul.cpp -o matmul.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c misc.cpp -o misc.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c operators.cpp -o operators.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c rbind.cpp -o rbind.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c slice.cpp -o slice.o
    clang++ -std=gnu++11 -I"/usr/local/Cellar/r/4.2.0/lib/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/Rcpp/include' -I'/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/xz/include -I/usr/local/include   -fno-trapping-math -fno-math-errno  -fPIC  -g -O2  -c slice_coo.cpp -o slice_coo.o
    /bin/sh: -c: line 0: syntax error near unexpected token `('
    /bin/sh: -c: line 0: `if test  "zRcppExports.o assignment.o cbind.o matmul.o misc.o operators.o rbind.o slice.o slice_coo.o" != "z"; then   echo clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L"/usr/local/Cellar/r/4.2.0/lib/R/lib" -L/usr/local/opt/gettext/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/xz/lib -L/usr/local/lib -o MatrixExtra.so RcppExports.o assignment.o cbind.o matmul.o misc.o operators.o rbind.o slice.o slice_coo.o  -L/usr/local/opt/openblas/lib -lopenblas -L/usr/local/opt/gcc/lib/gcc/11/gcc/x86_64-apple-darwin21/11 -L/usr/local/opt/gcc/lib/gcc/11/gcc -L/usr/local/opt/gcc/lib/gcc/11 -lgfortran -lquadmath -lm /usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/libs/libfloat.a -llapack -lblas  -L"/usr/local/Cellar/r/4.2.0/lib/R/lib" -lR -lintl -Wl,-framework -Wl,CoreFoundation;   clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L"/usr/local/Cellar/r/4.2.0/lib/R/lib" -L/usr/local/opt/gettext/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/xz/lib -L/usr/local/lib -o MatrixExtra.so RcppExports.o assignment.o cbind.o matmul.o misc.o operators.o rbind.o slice.o slice_coo.o  -L/usr/local/opt/openblas/lib -lopenblas -L/usr/local/opt/gcc/lib/gcc/11/gcc/x86_64-apple-darwin21/11 -L/usr/local/opt/gcc/lib/gcc/11/gcc -L/usr/local/opt/gcc/lib/gcc/11 -lgfortran -lquadmath -lm /usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/float/libs/libfloat.a -llapack -lblas  -L"/usr/local/Cellar/r/4.2.0/lib/R/lib" -lR -lintl -Wl,-framework -Wl,CoreFoundation; fi'
    make: *** [MatrixExtra.so] Error 2
    ERROR: compilation failed for package ‘MatrixExtra’
    * removing ‘/usr/local/lib/R/¡Welcome to R version 4.2.0 (2022-04-22)/site-library/MatrixExtra’
    Warning in install.packages :
      installation of package ‘MatrixExtra’ had non-zero exit status
    
    The downloaded source packages are in
    	‘/private/var/folders/pf/6730cw150k318n5bbz3qz8dw0000gn/T/Rtmp3DdIfO/downloaded_packages’
    

    My sessionInfo() also is:

    R version 4.2.0 (2022-04-22)
    Platform: x86_64-apple-darwin21.3.0 (64-bit)
    Running under: macOS Monterey 12.4
    
    Matrix products: default
    LAPACK: /usr/local/Cellar/r/4.2.0/lib/R/lib/libRlapack.dylib
    
    locale:
    [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    
    attached base packages:
    [1] stats     graphics  grDevices utils     datasets  methods   base     
    
    loaded via a namespace (and not attached):
     [1] rstudioapi_0.13   magrittr_2.0.3    usethis_2.1.6     devtools_2.4.3   
     [5] pkgload_1.2.4     R6_2.5.1          rlang_1.0.2       fastmap_1.1.0    
     [9] tools_4.2.0       pkgbuild_1.3.1    sessioninfo_1.2.2 cli_3.3.0        
    [13] withr_2.5.0       ellipsis_0.3.2    remotes_2.4.2     rprojroot_2.0.3  
    [17] lifecycle_1.0.1   crayon_1.5.1      brio_1.1.3        processx_3.6.1   
    [21] purrr_0.3.4       callr_3.7.0       fs_1.5.2          ps_1.7.1         
    [25] curl_4.3.2        testthat_3.1.4    memoise_2.0.1     glue_1.6.2       
    [29] cachem_1.0.6      compiler_4.2.0    desc_1.4.1        prettyunits_1.1.1
    

    The content of my .zshrc file is:

    export PROMPT='%2/> '
    export CLICOLOR=1
    export LSCOLORS=GxFxCxDxBxegedabagaced
    export GREP_OPTIONS=”–color=always”
    export LANG=es_ES.UTF-8
    export LC_ALL=es_ES.UTF-8
    export LC_CTYPE=es_ES.UTF-8
    export PATH="/usr/local/opt/[email protected]/bin:$PATH"
    export LDFLAGS="-L/usr/local/opt/[email protected]/lib"
    export CPPFLAGS="-I/usr/local/opt/[email protected]/include"
    export PKG_CONFIG_PATH="/usr/local/opt/[email protected]/lib/pkgconfig"
    export PKG_CPPFLAGS="-Xclang -fopenmp"
    export PKG_LIBS=-lomp
    export PATH="/usr/local/opt/openjdk/bin:$PATH"
    export CPPFLAGS="-I/usr/local/opt/openjdk/include"
    export PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}"
    alias python="/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8"
    export PATH="/usr/local/opt/make/libexec/gnubin:$PATH"
    export PKG_CONFIG_PATH="/usr/local/opt/lapack/lib/pkgconfig"
    export LDFLAGS="-L/usr/local/opt/lapack/lib"
    export CPPFLAGS="-I/usr/local/opt/lapack/include"
    

    I used homebrew to install both R and gcc compilers.

    Thanks.

    opened by sicabi 4
  • Correct isinf() scope.

    Correct isinf() scope.

    Add std scope to isinf(), as already done in other places across the repo. This fixed the Windows build in conda-forge (https://github.com/conda-forge/staged-recipes/pull/17116).

    opened by cbrueffer 2
  • MatrixExtra 0.1.9 requires something newer than g++ 8.3.1

    MatrixExtra 0.1.9 requires something newer than g++ 8.3.1

    This is mostly an FYI for thoserunning into installation issues. I just noticed that, contrary to MatrixExtra 0.1.8, MatrixExtra 0.1.9 requires something never than g++ 8.3.1, e.g. g++ 9.3.1. I'm not sure what's needed from that version, because it appears to compile with g++ -std=gnu++11 .... Maybe this can be described in SystemRequirements: ....

    gcc 8.3.1 (fails)

    $ g++ --version
    g++ (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3)
    Copyright (C) 2018 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
    $ R CMD INSTALL MatrixExtra_0.1.9.tar.gz 
    * installing to library ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8’
    * installing *source* package ‘MatrixExtra’ ...
    ** package ‘MatrixExtra’ successfully unpacked and MD5 sums checked
    ** using staged installation
    checking for g++... g++
    checking whether the C++ compiler works... yes
    checking for C++ compiler default output file name... a.out
    checking for suffix of executables... 
    checking whether we are cross compiling... no
    checking for suffix of object files... o
    checking whether the compiler supports GNU C++... yes
    checking whether g++ accepts -g... yes
    checking for g++ option to enable C++11 features... none needed
    configure: creating ./config.status
    config.status: creating src/Makevars
    ** libs
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c assignment.cpp -o assignment.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c cbind.cpp -o cbind.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c matmul.cpp -o matmul.o
    matmul.cpp: In function ‘void gemm_csr_drm_as_dcm(int, int, const int*, const int*, const double*, const real_t*, size_t, real_t*, int, int)’:
    matmul.cpp:137:41: error: ‘ldc’ is predetermined ‘shared’ for ‘shared’
                 private(write_ptr, temp_arr)
                                             ^
    make: *** [/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/etc/Makeconf:177: matmul.o] Error 1
    ERROR: compilation failed for package ‘MatrixExtra’
    * removing ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/MatrixExtra’
    * restoring previous ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/MatrixExtra’
    

    gcc 9.3.1 (works)

    $ g++ --version
    g++ (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2)
    Copyright (C) 2019 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
    $ R CMD INSTALL MatrixExtra_0.1.9.tar.gz 
    * installing to library ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8’
    * installing *source* package ‘MatrixExtra’ ...
    ** package ‘MatrixExtra’ successfully unpacked and MD5 sums checked
    ** using staged installation
    checking for g++... g++
    checking whether the C++ compiler works... yes
    checking for C++ compiler default output file name... a.out
    checking for suffix of executables... 
    checking whether we are cross compiling... no
    checking for suffix of object files... o
    checking whether the compiler supports GNU C++... yes
    checking whether g++ accepts -g... yes
    checking for g++ option to enable C++11 features... none needed
    configure: creating ./config.status
    config.status: creating src/Makevars
    ** libs
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c assignment.cpp -o assignment.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c cbind.cpp -o cbind.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c matmul.cpp -o matmul.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c misc.cpp -o misc.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c operators.cpp -o operators.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c rbind.cpp -o rbind.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c slice.cpp -o slice.o
    g++ -std=gnu++11 -I"/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/Rcpp/include' -I'/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/include' -I/usr/local/include  -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic  -g -O2  -c slice_coo.cpp -o slice_coo.o
    g++ -std=gnu++11 -shared -L/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/lib -L/usr/local/lib64 -o MatrixExtra.so RcppExports.o assignment.o cbind.o matmul.o misc.o operators.o rbind.o slice.o slice_coo.o -fopenmp -L/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/lib -lRblas -lgfortran -lm -lquadmath -L/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/libs -l:float.so -Wl,-rpath=/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/float/libs -L/software/c4/cbi/software/R-4.1.2-gcc8/lib64/R/lib -lR
    installing to /c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/00LOCK-MatrixExtra/00new/MatrixExtra/libs
    ** R
    ** inst
    ** byte-compile and prepare package for lazy loading
    ** help
    *** installing help indices
    ** building package indices
    ** installing vignettes
    ** testing if installed package can be loaded from temporary location
    ** checking absolute paths in shared objects and dynamic libraries
    ** testing if installed package can be loaded from final location
    ** testing if installed package keeps a record of temporary installation path
    * DONE (MatrixExtra)
    

    FWIW, I've noticed this while test installation all 22,000+ packages on CRAN and Bioconductor. All but 5-10 packages work with gcc 8.3.1.

    opened by HenrikBengtsson 2
  • Installation failure: matmul.cpp:137:41: error: ‘ldc’ is predetermined ‘shared’ for ‘shared’  /  private(write_ptr, temp_arr)

    Installation failure: matmul.cpp:137:41: error: ‘ldc’ is predetermined ‘shared’ for ‘shared’ / private(write_ptr, temp_arr)

    First tried to install MatrixExtra from within RSudio on a newly updated R. Started by reinstalling all the Imports packages without problems. Then went to Terminal:

    After $R CMD INSTALL ~/MatrixExtra_0.1.9.tar.gz on an Ubuntu 18.04 box I get:

    [email protected]:/usr/lib/R$ R CMD INSTALL ~/MatrixExtra_0.1.9.tar.gz

    • installing to library ‘/home/david/R/x86_64-pc-linux-gnu-library/3.5.1’
    • installing source package ‘MatrixExtra’ ... ** package ‘MatrixExtra’ successfully unpacked and MD5 sums checked ** using staged installation checking for g++... g++ checking whether the C++ compiler works... yes checking for C++ compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether the compiler supports GNU C++... yes checking whether g++ accepts -g... yes checking for g++ option to enable C++11 features... none needed configure: creating ./config.status config.status: creating src/Makevars ** libs g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/Rcpp/include' -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/float/include' -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-J7pprH/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wno-ignored-attributes -c RcppExports.cpp -o RcppExports.o g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/Rcpp/include' -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/float/include' -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-J7pprH/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wno-ignored-attributes -c assignment.cpp -o assignment.o g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/Rcpp/include' -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/float/include' -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-J7pprH/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wno-ignored-attributes -c cbind.cpp -o cbind.o g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DUSE_ROBINMAP -DHAS_LD=1 -DSUPPORTS_RESTRICT=1 -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/Rcpp/include' -I'/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/float/include' -fopenmp -fno-trapping-math -fno-math-errno -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-J7pprH/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wno-ignored-attributes -c matmul.cpp -o matmul.o matmul.cpp: In function ‘void gemm_csr_drm_as_dcm(int, int, const int*, const int*, const double*, const real_t*, size_t, real_t*, int, int)’: matmul.cpp:137:41: error: ‘ldc’ is predetermined ‘shared’ for ‘shared’ private(write_ptr, temp_arr) ^ /usr/lib/R/etc/Makeconf:177: recipe for target 'matmul.o' failed make: *** [matmul.o] Error 1 ERROR: compilation failed for package ‘MatrixExtra’
    • removing ‘/home/david/R/x86_64-pc-linux-gnu-library/3.5.1/MatrixExtra’

    [email protected]:/usr/lib/R$ R --version R version 4.1.2 (2021-11-01) -- "Bird Hippie" Copyright (C) 2021 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit)

    opened by dwinsemius 1
  • Rcpp?

    Rcpp?

    Is it necessary for you to use Rcpp instead of plain C (which is rather faster than slower)? I'm asking because this will make any back porting to Matrix impossible, AFAICS.

    opened by mmaechler 1
The official SuiteSparse library: a suite of sparse matrix algorithms authored or co-authored by Tim Davis, Texas A&M University

SuiteSparse: A Suite of Sparse matrix packages at http://suitesparse.com May 17, 2021. SuiteSparse VERSION 5.10.1 Now includes GraphBLAS, SLIP_LU, and

Tim Davis 612 Jul 1, 2022
A C++17 library of computationally efficient methods for calculating sample statistics

Vectorized statistics using SIMD primitives Introduction is a C++17 library of computationally efficient methods for calculating sample statistics (me

HEAL 10 Jan 7, 2022
ESP Insights is a remote diagnostics solution that allows users to remotely monitor the health of ESP devices in the field.

ESP Insights is a remote diagnostics solution that allows users to remotely monitor the health of ESP devices in the field.

Espressif Systems 27 Jun 9, 2022
Veml7700-esp-idf - VEML7700 Light Sensor driver for ESP-IDF

VEML7700 Light Sensor driver for ESP-IDF Overview This project aims to provide a very simple interface for configuring and reading data from the VEML7

Kristijan Grozdanovski 3 Apr 4, 2022
Matryoshka loader is a tool that red team operators can leverage to generate shellcode for Microsoft Office document phishing payloads.

Overview Matryoshka loader is a tool that red team operators can leverage to generate shellcode for an egghunter to bypass size-limitations and perfor

Praetorian 24 Jun 26, 2022
Tiny header-only library providing bitwise operators for enums in C++11

bitflags Tiny header-only library providing bitwise operators for enums in C++11. Getting started Import the operators from namespace avakar::bitflags

Martin Vejnár 2 Jan 29, 2022
The OpenEXR project provides the specification and reference implementation of the EXR file format, the professional-grade image storage format of the motion picture industry.

OpenEXR OpenEXR provides the specification and reference implementation of the EXR file format, the professional-grade image storage format of the mot

Academy Software Foundation 1.2k Jun 27, 2022
(Simple String Format) is an syntax of format and a library for parse this.

SSFMT (Simple String Format) is an syntax of format and a library for parse this. SSFMT != {fmt} SSFMT is NOT an API/library for parse {fmt} syntax !

null 2 Jan 30, 2022
matrix-effect This is a dumb matrix effect type thing

matrix-effect This is a dumb matrix effect type thing. It's only like one source file which should compile... Define __POSIX or __WIN though, for posi

null 41 Jun 8, 2022
very simple matrix library, int addition to a recreation of DCM attitude estimation in the form of matrix of C.

matrix very simple matrix library, int addition to a recreation of DCM attitude estimation in the form of matrix of C. the lib include matrix inversio

null 4 Mar 21, 2022
OpenVDB - Sparse volume data structure and tools

OpenVDB AX Nano Houdini Website | Discussion Forum | Documentation OpenVDB is an open source C++ library comprising a novel hierarchical data structur

Academy Software Foundation 1.7k Jun 23, 2022
The QPEP-Enhanced Direct Sparse Odometry (DSO) with Loop Closure

QPEP-DSO: Quadratic Pose Estimation Problems (QPEP) Enhanced Direct Sparse Odometry with Loop Closure Code Arch The codes are motivated by DSO (https:

Jin Wu 8 Jun 23, 2022
Fast, hierarchical, sparse Voxel Grid

Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimens

Davide Faconti 241 Jun 24, 2022
Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

Haoyang Ye 74 Jun 25, 2022
Fast, hierarchical, sparse Voxel Grid

Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimens

Davide Faconti 241 Jun 24, 2022
CPU Sparse Voxel Octree Implementation

Sparse Voxel Octrees This project provides a multithreaded, CPU Sparse Voxel Octree implementation in C++, capable of raytracing large datasets in rea

Benedikt Bitterli 604 Jun 23, 2022
Fast and Light-weight path smoothing methods for vehicles

path_smoother About Fast and Light-weight path smoothing methods for vehicles Denpendencies This project has been tested on Ubuntu 18.04. sudo apt-get

MingwangZhao 4 Dec 1, 2021
A gazebo actor plugin that utilizes the map of the environment and graph search methods to generate random actor trajectories that don't pass through walls, furniture, etc.

Gazebo-Map-Actor-Plugin A gazebo actor plugin that utilizes the map of the environment and graph search methods to generate random actor trajectories

Yasin Sonmez 10 Jun 25, 2022
Mobile platform for analysis of localization methods using the Intel RealSense T265 sensor

OptiBot Mobile platform for analysis of localization methods using the Intel RealSense T265 sensor About | Content | Implementation | License | Author

Kamil Goś 2 Feb 17, 2022