Apophenia is an open statistical library for working with data sets and statistical or simulation models. It provides functions on the same level as those of the typical stats package (such as OLS, probit, or singular value decomposition) but gives the user more flexibility to be creative in model-building. Being in C, it is often an order of magnitude faster when searching for optima or running MCMC chains. The core functions are written in C, but experience has shown them to be easy to bind to Python/Julia/Perl/Ruby/&c. http://apophenia.info/gentle.html provides an overview of the basics of using the library. If you want to know more about the package, see the web site, http://apophenia.info, or have a look at the textbook from Princeton University Press that coevolved with Apophenia, downloadable from http://modelingwithdata.org . The quick summary for installation: ∙ The library depends on the GNU Scientific Library and SQLite3. If you are using a system with a package manager of some sort, there is certainly a package for them. Be sure to include both the main package and the lib-, -dev, or -devel package. Sample package manager calls: sudo apt-get install make gcc libgsl0-dev libsqlite3-dev or sudo yum install make gcc gsl-devel libsqlite3x-devel or sudo pacman -S make gcc gsl sqlite ∙ The prebuilt package, that has only basic prerequisites (no Autotools or m4) can be downloaded from another Git branch: #Download the zip file, via wget or your preferred downloading method: wget https://github.com/b-k/Apophenia/archive/pkg.zip #unzip and build unzip pkg.zip cd Apophenia-pkg ./configure make sudo make install Or check out the branch via git: git clone https://github.com/b-k/Apophenia.git cd Apophenia git checkout pkg ./configure make sudo make install ∙ This master branch of the git repository requires Autotools, so it can build the package. Try (apt-get || yum install) autoconf automake libtool. If you have Autotools installed, then from this branch you can run: ./configure cd apophenia-1.0 make sudo make install ∙ Find detailed setup instructions and some troubleshooting notes at http://apophenia.info/setup.html . Thanks for your interest. I do hope that Apophenia helps you learn more from your data. --BK PS: Lawyers, please note that a file named COPYING in the install/ directory describes how this package is licensed under GPLv2.
A C library for statistical and scientific computing
Overview
Comments
-
FAIL: distribution_tests
Hi,
The testsuite is nearly perfect except one below:
Bernoulli distribution: ... PASS. Bernoulli distribution: ...... PASS. Beta distribution: ... PASS. Beta distribution: gsl: simplex.c:265: ERROR: non-finite function value encountered Default GSL error handler invoked.
gsl version: 1.16
-
make check fails with 'sort_example' segfault on OS X 10.11.3
Hello!
I'm working on a package for macports but the testing suite fails on 'sort_example' with El Capitan 10.11.3:
../build-aux/test-driver: line 107: 78802 Segmentation fault: 11 "$@" > $log_file 2>&1 FAIL: sort_example
Looks to be line 117 of the 'tests/sort_example' shell script,
117 exec "$progdir/$program" ${1+"$@"}
Which when run directly confirms the symptom:
tests$ pwd [...]/ports/math/apophenia/work/apophenia-1.0/tests tests$ ./sort_example Segmentation fault: 11
I know this is a wrapped call to the binary, but without the shell script's env attached the error from the standalone call isn't really much to go on. I'm also not sure how to load this all together into LLDB. Any tips?
.libs$ pwd [...]/ports/math/apophenia/work/apophenia-1.0/tests/.libs .libs$ ./sort_example dyld: Library not loaded: /opt/local/lib/libapophenia.2.dylib Referenced from: /opt/local/var/macports/build/[...]/ports_math_apophenia/apophenia/work/apophenia-1.0/tests/.libs/./sort_example Reason: image not found Trace/BPT trap: 5
I've currently got the macport pulling the release tarball "v1.0.tar.gz" dated 2015-11-30. Some observations:
- I'm unable to reproduce the symptom in Ubuntu 14.04.4 "trusty".
- I re-installed Xcode and the command-line tools just for grins but no luck.
- I went back to release tag 0.999a dated 2014-08-04 and all tests PASSED. Looks like there was a commit on sort_example.c that day.
- I also noticed there are more tests in 0.999a, 35 total as opposed to 30 in v1.0.
- Release 0.999b doesn't build and never reaches 'make check' on my system.
- Every once in awhile the 'test_apop' test also fails with "Aborted" but it's intermittent, and I've found that if I uninstall gsl and sqlite3, then re-build, that issue goes away. I see Issue #10 from Aug. 2014 was similar.
So I'm not sure where to go from here to troubleshoot further. I'd be happy to try and debug it but I would need some guidance in that area. Given that some tests have been removed since 2014, how important is this sort_example test?
Thanks for your help. Rob
-
Apophenia not available using homebrew (on Mac OSX)
This has no formula available so Mac OSX users cannot
brew install apophenia
I created a pull request https://github.com/Homebrew/homebrew-core/pull/331 (now closed, see below) against the homebrew-core project to build apophenia from scratch.
But really, that should be part of this project, not something written and maintained by outsiders.
I do observe that there's a macports portfile as well, apparently maintained outside this project, also.
-
aclocal version issue
On a Debian Wheezy desktop, I cloned the git repository for Apophenia, checked out the "pkg" branch, and ran ./configure successfully. However, when I run "make all" I get
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/rsmith/warez/Apophenia/missing aclocal-1.13 -I m4 /home/rsmith/warez/Apophenia/missing: line 81: aclocal-1.13: command not found WARNING: 'aclocal-1.13' is missing on your system. You should only need it if you modified 'acinclude.m4' or 'configure.ac' or m4 files included by 'configure.ac'. The 'aclocal' program is part of the GNU Automake package: http://www.gnu.org/software/automake It also requires GNU Autoconf, GNU m4 and Perl in order to run: http://www.gnu.org/software/autoconf http://www.gnu.org/software/m4/ http://www.perl.org/ make: *** [aclocal.m4] Error 127
I have automake 1.11 installed, and both "aclocal" and "aclocal-1.11" are installed in /usr/bin/. I did not modify any files, so I was hoping automake 1.11 in Wheezy would work. Any suggestions?
-
Add git tags for the releases
Hi,
I'm going to package it into Fedora, I need a versioned tarball.
I can handle the version by gittags, so please tag the releases from now on.
Thanks.
-
Installation problem with mysql
Whenever I try to install the latest, I get:
In file included from apop_db.c:21:0: apop_db_mysql.c:6:23: fatal error: my_global.h: No such file or directory compilation terminated.
To get around this, I comment out lines 12427-12434 in config, which I assume prevents me from interfacing with mysql.
#if test "x$ac_cv_lib_mysqlclient_mysql_query" = xyes; then : # cat >>confdefs.h <<_ACEOF #define HAVE_LIBMYSQLCLIENT 1 #_ACEOF # LIBS="-lmysqlclient $LIBS" #fi
-
Functions dont work
Hi,
Some functions/macros are not working. I noticed there are no prototype declaration in the apop.h file. Can you please resolve this.
Thank you.
For example apop_matrix_to_data or apop_vector_to_data.
/usr/bin/ld: error: undefined symbol: apop_matrix_to_data
-
Compilation errors, Mac OS X Yosemite 10.10.3
Hi,
I am getting the following errors when trying to compile:
../../apophenia-pkg/model/apop_histogram.c:116:21: error: expected expression OMP_for_reduce(+:ll, int i=0; i< datasize; i++){ ^ ../../apophenia-pkg/apop_internal.h:65:33: note: expanded from macro 'OMP_for_reduce'
define OMP_for_reduce(...) for(VA_ARGS)
^
../../apophenia-pkg/model/apop_histogram.c:116:38: error: use of undeclared identifier 'i' OMP_for_reduce(+:ll, int i=0; i< datasize; i++){ ^ ../../apophenia-pkg/apop_internal.h:65:33: note: expanded from macro 'OMP_for_reduce'
define OMP_for_reduce(...) for(VA_ARGS)
^
../../apophenia-pkg/model/apop_histogram.c:116:51: error: use of undeclared identifier 'i' OMP_for_reduce(+:ll, int i=0; i< datasize; i++){
I am able to compile fine on Linux. Is this a clang issue? I am using clang-602.0.49, Apple LLVM version 6.1.0.
Thanks, Neil
-
Update configure.ac
This line will prevent errors such as the following on Mac OS X: automake: warning: source file '$(top_builddir)/eg/apop_map_row.c' is in a subdirectory, automake: but option 'subdir-objects' is disabled
-
Sparse Data Support
Hello,
I'm very interested in Apophenia but I'm working mostly with very sparse data and wasn't able to find any mention about sparse data support on http://apophenia.info/ , in 21st Century C and Modeling With Data.
The sparsity comes often from one-hot encoding of categorical variables with high cardinality (50-100.000) and I want to apply logistic regression type models to this data. I plan to write a specialised solver myself but I'm wondering if you could recommend a sparse matrix library (such as CSPARSE) that would work well with Apophenia or if I missed already included support for sparse matrices.
best, Immanuel
-
utilities_test fails if there is a sqliterc with "headers=on"
This is minor issue I get when using the latest git version. In the tests/utilities_test script, in the function fixed_read(), the line
if Diff $($SQLITE3 td.db "select col_2 from td where rowid=4") 2.71828
compares the output of a sqlite3 query to 2.71828. However, this doesn't work with my current $HOME/.sqliterc which includes "headers=on" because then the column name is compared instead of its contents. Using the command line option "-noheader" fixes this, but I don't know if there are side effects from using other nodefaults in a sqliterc file. The version of sqlite3 I'm using does not seem to have an option for ignoring the sqliterc.
-
tests errors on macOS: apop_arms_settings_init: init failed, error 1004; Dirichlet distribution: ..gsl: psi.c:385: ERROR: domain error
Two object files build with no symbols on macOS 10.6.8:
/opt/local/bin/ranlib: file: .libs/libapophenia.a(asprintf.o) has no symbols /opt/local/bin/ranlib: file: .libs/libapophenia.a(libapopmodel_la-apop_wishart.o) has no symbols ranlib: file: .libs/libapophenia.a(asprintf.o) has no symbols ranlib: file: .libs/libapophenia.a(libapopmodel_la-apop_wishart.o) has no symbols
Then, three tests fail:
../../../build-aux/test-driver: line 112: 25908 Bus error "$@" >> "$log_file" 2>&1 FAIL: test_apop ../../../build-aux/test-driver: line 112: 26497 Abort trap "$@" >> "$log_file" 2>&1 FAIL: distribution_tests FAIL: ../eg/test_updating
Complete output:
make check-TESTS PASS: utilities_test PASS: db_tests PASS: error_test PASS: factors PASS: nist_tests PASS: sort_example ../../../build-aux/test-driver: line 112: 25908 Bus error "$@" >> "$log_file" 2>&1 FAIL: test_apop PASS: ../eg/apop_map_row PASS: ../eg/binning PASS: ../eg/boot_clt PASS: ../eg/data_fill PASS: ../eg/draw_to_db PASS: ../eg/db_fns PASS: ../eg/dot_products PASS: ../eg/entropy_vector PASS: ../eg/iv PASS: ../eg/ks_tests PASS: ../eg/logit PASS: ../eg/fake_logit PASS: ../eg/normalization_demo PASS: ../eg/ols PASS: ../eg/ols_oneliner PASS: ../eg/parameterization PASS: ../eg/simple_subsets PASS: ../eg/t_test_by_rows PASS: ../eg/test_distances PASS: ../eg/test_fisher PASS: ../eg/test_harmonic PASS: ../eg/test_pruning PASS: ../eg/test_regex ../../../build-aux/test-driver: line 112: 26497 Abort trap "$@" >> "$log_file" 2>&1 FAIL: distribution_tests PASS: lognormal_test PASS: rake_test PASS: test_kernel_ll PASS: update_via_rng PASS: ../eg/cross_models PASS: ../eg/dconstrain PASS: ../eg/entropy_model PASS: ../eg/faithful PASS: ../eg/f_test PASS: ../eg/fix_params PASS: ../eg/hills2 PASS: ../eg/jack PASS: ../eg/jacobian PASS: ../eg/ml_imputation PASS: ../eg/pmf_test PASS: ../eg/some_cdfs PASS: ../eg/test_kl_divergence PASS: ../eg/test_ranks FAIL: ../eg/test_updating PASS: ../eg/transform ============================================================================ Testsuite summary for apophenia 1.0 ============================================================================ # TOTAL: 51 # PASS: 48 # SKIP: 0 # XFAIL: 0 # FAIL: 3 # XPASS: 0 # ERROR: 0 ============================================================================ See tests/test-suite.log Please report to [email protected] ============================================================================ make[4]: *** [test-suite.log] Error 1 make[3]: *** [check-TESTS] Error 2 make[2]: *** [check-am] Error 2 make[1]: *** [check-recursive] Error 1 make: *** [distcheck] Error 1
-
configure script ignores passed args, build fails: ld: library not found for -lmysqlclient
I am trying to build
apophenia
from the latest commit, and configure (autoconf, in effect) ignores arguments passed:--build=
is ignored, and configure forced cpu_arch instead of build_arch.--with-mysql=no
is ignored, and the build eventually fails on:
CCLD libapopkernel.la CCLD libapophenia.la **ld: library not found for -lmysqlclient** collect2: ld returned 1 exit status make[3]: *** [libapophenia.la] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all] Error 2 make: *** [distcheck] Error 1
--disable-silent-rules
is ignored, no verbose output.
-
Build error on 10.6: Undefined symbols: "_gomp_thread_attr", referenced from: _gomp_run_sched_chunk in libgomp.a(env.o)
I am trying to build the latest commit from here on 10.6 for PPC, and get the following error:
Undefined symbols: "_gomp_thread_attr", referenced from: _gomp_run_sched_chunk in libgomp.a(env.o) ld: symbol(s) not found collect2: ld returned 1 exit status make[3]: *** [test_apop] Error 1 make[2]: *** [check-am] Error 2 make[1]: *** [check-recursive] Error 1 make: *** [distcheck] Error 1 --------------------- OK, built. From the apophenia-1.0 directory, you can run: make && sudo make install ---> Building apophenia Executing: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_math_apophenia/apophenia/work/apophenia-113671c16afa1724f9f225acdc88e838770071ce" && /usr/bin/make -j4 -w all make: Entering directory `/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_math_apophenia/apophenia/work/b-k-apophenia-113671c' make: *** No rule to make target `all'. Stop.
Any idea how to fix this?
-
toy example `census.c` crashing
Hello,
I've just installed apophenia on my system running Ubuntu 20.04 (libgsl23).
The first example,
census.c
from the page http://apophenia.info/gentle.html, compiles well:$ make census cc -Wall -Wextra -g census.c -lapophenia -lgsl -lgslcblas -lsqlite3 -o census census.c: In function ‘main’: census.c:7:5: warning: missing initializer for field ‘output_pipe’ of ‘variadic_type_apop_model_print’ {aka ‘struct
’} [-Wmissing-field-initializers] 7 | apop_model_print(est); | ^~~~~~~~~~~~~~~~ In file included from census.c:1: /usr/local/include/apop.h:452:2: note: ‘output_pipe’ declared here 452 | apop_varad_declare(void, apop_model_print, apop_model * model; FILE *output_pipe); | ^~~~~~~~~~~~~~~~~~ But I get an error when running it:
$ ./census gsl: ../gsl/gsl_vector_double.h:180: ERROR: index out of range Default GSL error handler invoked.
which I tracked, with gdb, to:
apop_ols.c:166 ll += logl(gsl_ran_gaussian_pdf(gsl_vector_get(errors, i), sigma)* weight * x_prob);
-
Runtime error
Hi,
I recently upgraded all the packages in my OS , and unfortunately, can't get apophenia to run:
ld-elf.so.1: Shared object "libmysqlclient.so.18" not found, required by "libapophenia.so.2"
Is there a way to get rid of this dependence. I use sqlite- not mysql.
Thank you.
-
apop_plot
Hi,
Just wanted to clarify.
The plot function may also have been deprecated- many of the examples from the book don't work. Can you please provide list of errata or something equivalent, and perhaps some alternatives, so that for those us following your (MWD) book can get all the examples/macros to work. Thank you. Appreciate it.
/usr/bin/ld: error: undefined symbol: apop_plot_histogram
referenced by cltdemo.c:23 /tmp/cltdemo-a5e244.o:(main)
ated. /usr/bin/ld: error: undefined symbol: apop_plot_histogram
referenced by cltdemo.c:23 /tmp/cltdemo-a5e244.o:(main)
A C++ header-only library of statistical distribution functions.
StatsLib StatsLib is a templated C++ library of statistical distribution functions, featuring unique compile-time computing capabilities and seamless
C++ tensors with broadcasting and lazy computing
Multi-dimensional arrays with broadcasting and lazy computing. Introduction xtensor is a C++ library meant for numerical analysis with multi-dimension
C++ class for creating and computing arbitrary-length integers
BigNumber BigNumber is a C++ class that allows for the creation and computation of arbitrary-length integers. The maximum possible length of a BigNumb
MIRACL Cryptographic SDK: Multiprecision Integer and Rational Arithmetic Cryptographic Library is a C software library that is widely regarded by developers as the gold standard open source SDK for elliptic curve cryptography (ECC).
MIRACL What is MIRACL? Multiprecision Integer and Rational Arithmetic Cryptographic Library – the MIRACL Crypto SDK – is a C software library that is
a lean linear math library, aimed at graphics programming. Supports vec3, vec4, mat4x4 and quaternions
linmath.h -- A small library for linear math as required for computer graphics linmath.h provides the most used types required for programming compute
nml is a simple matrix and linear algebra library written in standard C.
nml is a simple matrix and linear algebra library written in standard C.
C++ Mathematical Expression Parsing And Evaluation Library
C++ Mathematical Expression Toolkit Library Documentation Section 00 - Introduction Section 01 - Capabilities Section 02 - Example Expressions
libmpc++ is a C++ header-only library to solve linear and non-linear MPC
libmpc++ libmpc++ is a C++ library to solve linear and non-linear MPC. The library is written in modern C++17 and it is tested to work on Linux, macOS
A lightweight, minimal and customisable maths library for C99
Small Maths Library A lightweight, minimal and customisable maths library for C99, generated by Lua. Generating Requires Lua 5.3. lua sml.lua Generat
C++ Matrix -- High performance and accurate (e.g. edge cases) matrix math library with expression template arithmetic operators
Matrix This is a math and arithmetic matrix library. It has stood many years of performing in mission critical production for financial systems. It ha
Header only, single file, simple and efficient C++ library to compute the signed distance function to a triangle mesh
TriangleMeshDistance Header only, single file, simple and efficient C++11 library to compute the signed distance function to a triangle mesh. The dist
Library for nonconvex constrained optimization using the augmented Lagrangian method and the matrix-free PANOC algorithm.
alpaqa Alpaqa is an efficient implementation of the Augmented Lagrangian method for general nonlinear programming problems, which uses the first-order
A simple C++ complex & real matrix library, with matrix inversion, left division and determinant calculation
NaiveMatrixLib 帆帆的简易矩阵计算库 A simple C++ stdlib-based complex & real matrix library, with matrix inversion, left division (A\b) and determinant calculat
A work-in-progress C++20/23 header-only maths library for game development, embedded, kernel and general-purpose that works in constant context.
kMath /kmæθ/ A work-in-progress general-purpose C++20/23 header-only maths library that works in constant context Abstract The kMath Project aims to p
Kraken is an open-source modern math library that comes with a fast-fixed matrix class and math-related functions.
Kraken ?? Table of Contents Introduction Requirement Contents Installation Introduction Kraken is a modern math library written in a way that gives ac
P(R*_{3, 0, 1}) specialized SIMD Geometric Algebra Library
Klein ?? ?? Project Site ?? ?? Description Do you need to do any of the following? Quickly? Really quickly even? Projecting points onto lines, lines t
linalg.h is a single header, public domain, short vector math library for C++
linalg.h linalg.h is a single header, public domain, short vector math library for C++. It is inspired by the syntax of popular shading and compute la
LibTomMath is a free open source portable number theoretic multiple-precision integer library written entirely in C.
libtommath This is the git repository for LibTomMath, a free open source portable number theoretic multiple-precision integer (MPI) library written en
OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.
OpenBLAS Travis CI: AppVeyor: Drone CI: Introduction OpenBLAS is an optimized BLAS (Basic Linear Algebra Subprograms) library based on GotoBLAS2 1.13