A C library for statistical and scientific computing

Related tags

Math apophenia
Overview
Apophenia is an open statistical library for working with data sets and statistical or simulation models. It provides functions on the same level as those of the typical stats package (such as OLS, probit, or singular value decomposition) but gives the user more flexibility to be creative in model-building. Being in C, it is often an order of magnitude faster when searching for optima or running MCMC chains. The core functions are written in C, but experience has shown them to be easy to bind to Python/Julia/Perl/Ruby/&c.

http://apophenia.info/gentle.html provides an overview of the basics of using the library. If you want to know more about the package, see the web site, http://apophenia.info, or have a look at the textbook from Princeton University Press that coevolved with Apophenia, downloadable from http://modelingwithdata.org .


The quick summary for installation:

∙ The library depends on the GNU Scientific Library and SQLite3. If you are using a system with a package manager of some sort, there is certainly a package for them. Be sure to include both the main package and the lib-, -dev, or -devel package. Sample package manager calls:

    sudo apt-get install make gcc libgsl0-dev libsqlite3-dev 
or 
    sudo yum install make gcc gsl-devel libsqlite3x-devel
or 
    sudo pacman -S make gcc gsl sqlite 

∙ The prebuilt package, that has only basic prerequisites (no Autotools or m4) can be downloaded from another Git branch:

    #Download the zip file, via wget or your preferred downloading method:
    wget https://github.com/b-k/Apophenia/archive/pkg.zip

    #unzip and build
    unzip pkg.zip
    cd Apophenia-pkg
    ./configure
    make
    sudo make install

Or check out the branch via git:

    git clone https://github.com/b-k/Apophenia.git
    cd Apophenia
    git checkout pkg
    ./configure
    make
    sudo make install

∙ This master branch of the git repository requires Autotools, so it can build the
package. Try (apt-get || yum install) autoconf automake libtool. If you have Autotools installed, then from this branch you can run:

    ./configure
    cd apophenia-1.0
    make 
    sudo make install

∙ Find detailed setup instructions and some troubleshooting notes at
http://apophenia.info/setup.html .


Thanks for your interest. I do hope that Apophenia helps you learn more from your data.

--BK

PS: Lawyers, please note that a file named COPYING in the install/ directory describes how this package is licensed under GPLv2.
Comments
  • FAIL: distribution_tests

    FAIL: distribution_tests

    Hi,

    The testsuite is nearly perfect except one below:

    Bernoulli distribution: ... PASS. Bernoulli distribution: ...... PASS. Beta distribution: ... PASS. Beta distribution: gsl: simplex.c:265: ERROR: non-finite function value encountered Default GSL error handler invoked.

    gsl version: 1.16

    opened by cicku 26
  • make check fails with 'sort_example' segfault on OS X 10.11.3

    make check fails with 'sort_example' segfault on OS X 10.11.3

    Hello!

    I'm working on a package for macports but the testing suite fails on 'sort_example' with El Capitan 10.11.3:

    ../build-aux/test-driver: line 107: 78802 Segmentation fault: 11  "$@" > $log_file 2>&1
    FAIL: sort_example
    

    Looks to be line 117 of the 'tests/sort_example' shell script,

    117       exec "$progdir/$program" ${1+"$@"}
    

    Which when run directly confirms the symptom:

    tests$ pwd
    [...]/ports/math/apophenia/work/apophenia-1.0/tests
    tests$ ./sort_example
    Segmentation fault: 11
    

    I know this is a wrapped call to the binary, but without the shell script's env attached the error from the standalone call isn't really much to go on. I'm also not sure how to load this all together into LLDB. Any tips?

    .libs$ pwd
    [...]/ports/math/apophenia/work/apophenia-1.0/tests/.libs
    .libs$ ./sort_example
    dyld: Library not loaded: /opt/local/lib/libapophenia.2.dylib
      Referenced from: /opt/local/var/macports/build/[...]/ports_math_apophenia/apophenia/work/apophenia-1.0/tests/.libs/./sort_example
      Reason: image not found
    Trace/BPT trap: 5
    

    I've currently got the macport pulling the release tarball "v1.0.tar.gz" dated 2015-11-30. Some observations:

    1. I'm unable to reproduce the symptom in Ubuntu 14.04.4 "trusty".
    2. I re-installed Xcode and the command-line tools just for grins but no luck.
    3. I went back to release tag 0.999a dated 2014-08-04 and all tests PASSED. Looks like there was a commit on sort_example.c that day.
    4. I also noticed there are more tests in 0.999a, 35 total as opposed to 30 in v1.0.
    5. Release 0.999b doesn't build and never reaches 'make check' on my system.
    6. Every once in awhile the 'test_apop' test also fails with "Aborted" but it's intermittent, and I've found that if I uninstall gsl and sqlite3, then re-build, that issue goes away. I see Issue #10 from Aug. 2014 was similar.

    So I'm not sure where to go from here to troubleshoot further. I'd be happy to try and debug it but I would need some guidance in that area. Given that some tests have been removed since 2014, how important is this sort_example test?

    Thanks for your help. Rob

    opened by mindonly 20
  • Apophenia not available using homebrew (on Mac OSX)

    Apophenia not available using homebrew (on Mac OSX)

    This has no formula available so Mac OSX users cannot brew install apophenia

    I created a pull request https://github.com/Homebrew/homebrew-core/pull/331 (now closed, see below) against the homebrew-core project to build apophenia from scratch.

    But really, that should be part of this project, not something written and maintained by outsiders.

    I do observe that there's a macports portfile as well, apparently maintained outside this project, also.

    opened by bdobyns 4
  • aclocal version issue

    aclocal version issue

    On a Debian Wheezy desktop, I cloned the git repository for Apophenia, checked out the "pkg" branch, and ran ./configure successfully. However, when I run "make all" I get

    CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/rsmith/warez/Apophenia/missing aclocal-1.13 -I m4 /home/rsmith/warez/Apophenia/missing: line 81: aclocal-1.13: command not found WARNING: 'aclocal-1.13' is missing on your system. You should only need it if you modified 'acinclude.m4' or 'configure.ac' or m4 files included by 'configure.ac'. The 'aclocal' program is part of the GNU Automake package: http://www.gnu.org/software/automake It also requires GNU Autoconf, GNU m4 and Perl in order to run: http://www.gnu.org/software/autoconf http://www.gnu.org/software/m4/ http://www.perl.org/ make: *** [aclocal.m4] Error 127

    I have automake 1.11 installed, and both "aclocal" and "aclocal-1.11" are installed in /usr/bin/. I did not modify any files, so I was hoping automake 1.11 in Wheezy would work. Any suggestions?

    opened by raegis 3
  • Add git tags for the releases

    Add git tags for the releases

    Hi,

    I'm going to package it into Fedora, I need a versioned tarball.

    I can handle the version by gittags, so please tag the releases from now on.

    Thanks.

    opened by cicku 3
  • Installation problem with mysql

    Installation problem with mysql

    Whenever I try to install the latest, I get:

    In file included from apop_db.c:21:0:
    apop_db_mysql.c:6:23: fatal error: my_global.h: No such file or directory
    compilation terminated.
    

    To get around this, I comment out lines 12427-12434 in config, which I assume prevents me from interfacing with mysql.

    #if test "x$ac_cv_lib_mysqlclient_mysql_query" = xyes; then :
    #  cat >>confdefs.h <<_ACEOF
    #define HAVE_LIBMYSQLCLIENT 1
    #_ACEOF
    
    #  LIBS="-lmysqlclient $LIBS"
    
    #fi
    
    opened by dfd 3
  • Functions dont work

    Functions dont work

    Hi,

    Some functions/macros are not working. I noticed there are no prototype declaration in the apop.h file. Can you please resolve this.

    Thank you.

    For example apop_matrix_to_data or apop_vector_to_data.

    /usr/bin/ld: error: undefined symbol: apop_matrix_to_data

    opened by heshriti 2
  • Compilation errors, Mac OS X Yosemite 10.10.3

    Compilation errors, Mac OS X Yosemite 10.10.3

    Hi,

    I am getting the following errors when trying to compile:

    ../../apophenia-pkg/model/apop_histogram.c:116:21: error: expected expression OMP_for_reduce(+:ll, int i=0; i< datasize; i++){ ^ ../../apophenia-pkg/apop_internal.h:65:33: note: expanded from macro 'OMP_for_reduce'

    define OMP_for_reduce(...) for(VA_ARGS)

                                ^
    

    ../../apophenia-pkg/model/apop_histogram.c:116:38: error: use of undeclared identifier 'i' OMP_for_reduce(+:ll, int i=0; i< datasize; i++){ ^ ../../apophenia-pkg/apop_internal.h:65:33: note: expanded from macro 'OMP_for_reduce'

    define OMP_for_reduce(...) for(VA_ARGS)

                                ^
    

    ../../apophenia-pkg/model/apop_histogram.c:116:51: error: use of undeclared identifier 'i' OMP_for_reduce(+:ll, int i=0; i< datasize; i++){

    I am able to compile fine on Linux. Is this a clang issue? I am using clang-602.0.49, Apple LLVM version 6.1.0.

    Thanks, Neil

    opened by ntedeschi 2
  • Update configure.ac

    Update configure.ac

    This line will prevent errors such as the following on Mac OS X: automake: warning: source file '$(top_builddir)/eg/apop_map_row.c' is in a subdirectory, automake: but option 'subdir-objects' is disabled

    opened by mcheema 2
  • Sparse Data Support

    Sparse Data Support

    Hello,

    I'm very interested in Apophenia but I'm working mostly with very sparse data and wasn't able to find any mention about sparse data support on http://apophenia.info/ , in 21st Century C and Modeling With Data.

    The sparsity comes often from one-hot encoding of categorical variables with high cardinality (50-100.000) and I want to apply logistic regression type models to this data. I plan to write a specialised solver myself but I'm wondering if you could recommend a sparse matrix library (such as CSPARSE) that would work well with Apophenia or if I missed already included support for sparse matrices.

    best, Immanuel

    opened by ibayer 2
  • utilities_test fails if there is a sqliterc with

    utilities_test fails if there is a sqliterc with "headers=on"

    This is minor issue I get when using the latest git version. In the tests/utilities_test script, in the function fixed_read(), the line

    if Diff $($SQLITE3 td.db "select col_2 from td where rowid=4") 2.71828

    compares the output of a sqlite3 query to 2.71828. However, this doesn't work with my current $HOME/.sqliterc which includes "headers=on" because then the column name is compared instead of its contents. Using the command line option "-noheader" fixes this, but I don't know if there are side effects from using other nodefaults in a sqliterc file. The version of sqlite3 I'm using does not seem to have an option for ignoring the sqliterc.

    opened by raegis 1
  • tests errors on macOS: apop_arms_settings_init: init failed, error 1004; Dirichlet distribution: ..gsl: psi.c:385: ERROR: domain error

    tests errors on macOS: apop_arms_settings_init: init failed, error 1004; Dirichlet distribution: ..gsl: psi.c:385: ERROR: domain error

    Two object files build with no symbols on macOS 10.6.8:

    /opt/local/bin/ranlib: file: .libs/libapophenia.a(asprintf.o) has no symbols
    /opt/local/bin/ranlib: file: .libs/libapophenia.a(libapopmodel_la-apop_wishart.o) has no symbols
    ranlib: file: .libs/libapophenia.a(asprintf.o) has no symbols
    ranlib: file: .libs/libapophenia.a(libapopmodel_la-apop_wishart.o) has no symbols
    

    Then, three tests fail:

    ../../../build-aux/test-driver: line 112: 25908 Bus error               "$@" >> "$log_file" 2>&1
    FAIL: test_apop
    ../../../build-aux/test-driver: line 112: 26497 Abort trap              "$@" >> "$log_file" 2>&1
    FAIL: distribution_tests
    FAIL: ../eg/test_updating
    

    Complete output:

    make  check-TESTS
    PASS: utilities_test
    PASS: db_tests
    PASS: error_test
    PASS: factors
    PASS: nist_tests
    PASS: sort_example
    ../../../build-aux/test-driver: line 112: 25908 Bus error               "$@" >> "$log_file" 2>&1
    FAIL: test_apop
    PASS: ../eg/apop_map_row
    PASS: ../eg/binning
    PASS: ../eg/boot_clt
    PASS: ../eg/data_fill
    PASS: ../eg/draw_to_db
    PASS: ../eg/db_fns
    PASS: ../eg/dot_products
    PASS: ../eg/entropy_vector
    PASS: ../eg/iv
    PASS: ../eg/ks_tests
    PASS: ../eg/logit
    PASS: ../eg/fake_logit
    PASS: ../eg/normalization_demo
    PASS: ../eg/ols
    PASS: ../eg/ols_oneliner
    PASS: ../eg/parameterization
    PASS: ../eg/simple_subsets
    PASS: ../eg/t_test_by_rows
    PASS: ../eg/test_distances
    PASS: ../eg/test_fisher
    PASS: ../eg/test_harmonic
    PASS: ../eg/test_pruning
    PASS: ../eg/test_regex
    ../../../build-aux/test-driver: line 112: 26497 Abort trap              "$@" >> "$log_file" 2>&1
    FAIL: distribution_tests
    PASS: lognormal_test
    PASS: rake_test
    PASS: test_kernel_ll
    PASS: update_via_rng
    PASS: ../eg/cross_models
    PASS: ../eg/dconstrain
    PASS: ../eg/entropy_model
    PASS: ../eg/faithful
    PASS: ../eg/f_test
    PASS: ../eg/fix_params
    PASS: ../eg/hills2
    PASS: ../eg/jack
    PASS: ../eg/jacobian
    PASS: ../eg/ml_imputation
    PASS: ../eg/pmf_test
    PASS: ../eg/some_cdfs
    PASS: ../eg/test_kl_divergence
    PASS: ../eg/test_ranks
    FAIL: ../eg/test_updating
    PASS: ../eg/transform
    ============================================================================
    Testsuite summary for apophenia 1.0
    ============================================================================
    # TOTAL: 51
    # PASS:  48
    # SKIP:  0
    # XFAIL: 0
    # FAIL:  3
    # XPASS: 0
    # ERROR: 0
    ============================================================================
    See tests/test-suite.log
    Please report to [email protected]
    ============================================================================
    make[4]: *** [test-suite.log] Error 1
    make[3]: *** [check-TESTS] Error 2
    make[2]: *** [check-am] Error 2
    make[1]: *** [check-recursive] Error 1
    make: *** [distcheck] Error 1
    
    opened by barracuda156 1
  • configure script ignores passed args, build fails: ld: library not found for -lmysqlclient

    configure script ignores passed args, build fails: ld: library not found for -lmysqlclient

    I am trying to build apophenia from the latest commit, and configure (autoconf, in effect) ignores arguments passed:

    1. --build= is ignored, and configure forced cpu_arch instead of build_arch.
    2. --with-mysql=no is ignored, and the build eventually fails on:
      CCLD     libapopkernel.la
      CCLD     libapophenia.la
    **ld: library not found for -lmysqlclient**
    collect2: ld returned 1 exit status
    make[3]: *** [libapophenia.la] Error 1
    make[2]: *** [all-recursive] Error 1
    make[1]: *** [all] Error 2
    make: *** [distcheck] Error 1
    
    1. --disable-silent-rules is ignored, no verbose output.
    opened by barracuda156 3
  • Build error on 10.6: Undefined symbols:

    Build error on 10.6: Undefined symbols: "_gomp_thread_attr", referenced from: _gomp_run_sched_chunk in libgomp.a(env.o)

    I am trying to build the latest commit from here on 10.6 for PPC, and get the following error:

    Undefined symbols:
      "_gomp_thread_attr", referenced from:
          _gomp_run_sched_chunk in libgomp.a(env.o)
    ld: symbol(s) not found
    collect2: ld returned 1 exit status
    make[3]: *** [test_apop] Error 1
    make[2]: *** [check-am] Error 2
    make[1]: *** [check-recursive] Error 1
    make: *** [distcheck] Error 1
    
    ---------------------
    
    OK, built. From the apophenia-1.0 directory, you can run: make && sudo make install
    --->  Building apophenia
    Executing:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_math_apophenia/apophenia/work/apophenia-113671c16afa1724f9f225acdc88e838770071ce" && /usr/bin/make -j4 -w all 
    make: Entering directory `/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_math_apophenia/apophenia/work/b-k-apophenia-113671c'
    make: *** No rule to make target `all'.  Stop.
    

    Any idea how to fix this?

    opened by barracuda156 4
  • toy example `census.c` crashing

    toy example `census.c` crashing

    Hello,

    I've just installed apophenia on my system running Ubuntu 20.04 (libgsl23).

    The first example, census.c from the page http://apophenia.info/gentle.html, compiles well:

    $ make census cc -Wall -Wextra -g census.c -lapophenia -lgsl -lgslcblas -lsqlite3 -o census census.c: In function ‘main’: census.c:7:5: warning: missing initializer for field ‘output_pipe’ of ‘variadic_type_apop_model_print’ {aka ‘struct ’} [-Wmissing-field-initializers] 7 | apop_model_print(est); | ^~~~~~~~~~~~~~~~ In file included from census.c:1: /usr/local/include/apop.h:452:2: note: ‘output_pipe’ declared here 452 | apop_varad_declare(void, apop_model_print, apop_model * model; FILE *output_pipe); | ^~~~~~~~~~~~~~~~~~

    But I get an error when running it:

    $ ./census gsl: ../gsl/gsl_vector_double.h:180: ERROR: index out of range Default GSL error handler invoked.

    which I tracked, with gdb, to:

    apop_ols.c:166 ll += logl(gsl_ran_gaussian_pdf(gsl_vector_get(errors, i), sigma)* weight * x_prob);

    opened by chrplr 1
  • Runtime error

    Runtime error

    Hi,

    I recently upgraded all the packages in my OS , and unfortunately, can't get apophenia to run:

    ld-elf.so.1: Shared object "libmysqlclient.so.18" not found, required by "libapophenia.so.2"

    Is there a way to get rid of this dependence. I use sqlite- not mysql.

    Thank you.

    opened by heshriti 1
  • apop_plot

    apop_plot

    Hi,

    Just wanted to clarify.

    The plot function may also have been deprecated- many of the examples from the book don't work. Can you please provide list of errata or something equivalent, and perhaps some alternatives, so that for those us following your (MWD) book can get all the examples/macros to work. Thank you. Appreciate it.

    /usr/bin/ld: error: undefined symbol: apop_plot_histogram

    referenced by cltdemo.c:23 /tmp/cltdemo-a5e244.o:(main)

    ated. /usr/bin/ld: error: undefined symbol: apop_plot_histogram

    referenced by cltdemo.c:23 /tmp/cltdemo-a5e244.o:(main)

    opened by heshriti 1
A C++ header-only library of statistical distribution functions.

StatsLib StatsLib is a templated C++ library of statistical distribution functions, featuring unique compile-time computing capabilities and seamless

Keith O'Hara 423 Jan 3, 2023
C++ tensors with broadcasting and lazy computing

Multi-dimensional arrays with broadcasting and lazy computing. Introduction xtensor is a C++ library meant for numerical analysis with multi-dimension

Xtensor Stack 2.8k Jan 8, 2023
C++ class for creating and computing arbitrary-length integers

BigNumber BigNumber is a C++ class that allows for the creation and computation of arbitrary-length integers. The maximum possible length of a BigNumb

Limeoats 143 Dec 14, 2022
MIRACL Cryptographic SDK: Multiprecision Integer and Rational Arithmetic Cryptographic Library is a C software library that is widely regarded by developers as the gold standard open source SDK for elliptic curve cryptography (ECC).

MIRACL What is MIRACL? Multiprecision Integer and Rational Arithmetic Cryptographic Library – the MIRACL Crypto SDK – is a C software library that is

MIRACL 527 Jan 7, 2023
a lean linear math library, aimed at graphics programming. Supports vec3, vec4, mat4x4 and quaternions

linmath.h -- A small library for linear math as required for computer graphics linmath.h provides the most used types required for programming compute

datenwolf 729 Jan 9, 2023
nml is a simple matrix and linear algebra library written in standard C.

nml is a simple matrix and linear algebra library written in standard C.

Andrei Ciobanu 45 Dec 9, 2022
C++ Mathematical Expression Parsing And Evaluation Library

C++ Mathematical Expression Toolkit Library Documentation Section 00 - Introduction Section 01 - Capabilities Section 02 - Example Expressions

Arash Partow 445 Jan 4, 2023
libmpc++ is a C++ header-only library to solve linear and non-linear MPC

libmpc++ libmpc++ is a C++ library to solve linear and non-linear MPC. The library is written in modern C++17 and it is tested to work on Linux, macOS

Nicola Piccinelli 46 Dec 20, 2022
A lightweight, minimal and customisable maths library for C99

Small Maths Library A lightweight, minimal and customisable maths library for C99, generated by Lua. Generating Requires Lua 5.3. lua sml.lua Generat

null 5 May 6, 2022
C++ Matrix -- High performance and accurate (e.g. edge cases) matrix math library with expression template arithmetic operators

Matrix This is a math and arithmetic matrix library. It has stood many years of performing in mission critical production for financial systems. It ha

Hossein Moein 71 Oct 29, 2022
Header only, single file, simple and efficient C++ library to compute the signed distance function to a triangle mesh

TriangleMeshDistance Header only, single file, simple and efficient C++11 library to compute the signed distance function to a triangle mesh. The dist

Interactive Computer Graphics 100 Dec 28, 2022
Library for nonconvex constrained optimization using the augmented Lagrangian method and the matrix-free PANOC algorithm.

alpaqa Alpaqa is an efficient implementation of the Augmented Lagrangian method for general nonlinear programming problems, which uses the first-order

OPTEC 21 Dec 9, 2022
A simple C++ complex & real matrix library, with matrix inversion, left division and determinant calculation

NaiveMatrixLib 帆帆的简易矩阵计算库 A simple C++ stdlib-based complex & real matrix library, with matrix inversion, left division (A\b) and determinant calculat

FerryYoungFan 50 Dec 28, 2022
A work-in-progress C++20/23 header-only maths library for game development, embedded, kernel and general-purpose that works in constant context.

kMath /kmæθ/ A work-in-progress general-purpose C++20/23 header-only maths library that works in constant context Abstract The kMath Project aims to p

The λ Project 13 Sep 5, 2022
Kraken is an open-source modern math library that comes with a fast-fixed matrix class and math-related functions.

Kraken ?? Table of Contents Introduction Requirement Contents Installation Introduction Kraken is a modern math library written in a way that gives ac

yahya mohammed 24 Nov 30, 2022
P(R*_{3, 0, 1}) specialized SIMD Geometric Algebra Library

Klein ?? ?? Project Site ?? ?? Description Do you need to do any of the following? Quickly? Really quickly even? Projecting points onto lines, lines t

Jeremy Ong 635 Dec 30, 2022
linalg.h is a single header, public domain, short vector math library for C++

linalg.h linalg.h is a single header, public domain, short vector math library for C++. It is inspired by the syntax of popular shading and compute la

Sterling Orsten 758 Jan 7, 2023
LibTomMath is a free open source portable number theoretic multiple-precision integer library written entirely in C.

libtommath This is the git repository for LibTomMath, a free open source portable number theoretic multiple-precision integer (MPI) library written en

libtom 543 Dec 27, 2022
OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.

OpenBLAS Travis CI: AppVeyor: Drone CI: Introduction OpenBLAS is an optimized BLAS (Basic Linear Algebra Subprograms) library based on GotoBLAS2 1.13

Zhang Xianyi 4.9k Jan 5, 2023