OptimLib: a lightweight C++ library of numerical optimization methods for nonlinear functions

Overview

OptimLib   Build Status Coverage Status Codacy Badge License Documentation Status

OptimLib is a lightweight C++ library of numerical optimization methods for nonlinear functions.

Features:

  • A C++11 library of local and global optimization algorithms, as well as root finding techniques.
  • Derivative-free optimization using advanced, parallelized metaheuristic methods.
  • Constrained optimization routines to handle simple box constraints, as well as systems of nonlinear constraints.
  • For fast and efficient matrix-based computation, OptimLib supports the following templated linear algebra libraries:
  • OpenMP-accelerated algorithms for parallel computation.
  • Straightforward linking with parallelized BLAS libraries, such as OpenBLAS.
  • Available as a header-only library, or as a compiled shared library.
  • Released under a permissive, non-GPL license.

Contents:

Algorithms

A list of currently available algorithms includes:

  • Broyden's Method (for root finding)
  • Newton's method, BFGS, and L-BFGS
  • Gradient descent: basic, momentum, Adam, AdaMax, Nadam, NadaMax, and more
  • Nonlinear Conjugate Gradient
  • Nelder-Mead
  • Differential Evolution (DE)
  • Particle Swarm Optimization (PSO)

Documentation

Full documentation is available online:

Documentation Status

A PDF version of the documentation is available here.

API

The OptimLib API follows a relatively simple convention, with most algorithms called in the following manner:

algorithm_id(<initial/final values>, <objective function>, <objective function data>);

The inputs, in order, are:

  • A writable vector of initial values to define the starting point of the algorithm. In the event of successful completion, the initial values will be overwritten by the solution vector.
  • The 'objective function' is the user-defined function to be minimized (or zeroed-out in the case of root finding methods).
  • The final input is optional: it is any object that contains additional parameters necessary to evaluate the objective function.

For example, the BFGS algorithm is called using

bfgs(Vec_t& init_out_vals, std::function<double (const Vec_t& vals_inp, Vec_t* grad_out, void* opt_data)> opt_objfn, void* opt_data);

where Vec_t is used to represent either arma::vec or Eigen::VectorXd types.

Installation Method 1: Shared Library

The library can be installed on Unix-alike systems via the standard ./configure && make method:

# clone optim into the current directory
git clone https://github.com/kthohr/optim ./optim

# build and install
cd ./optim
./configure -i "/usr/local" -l arma -p
make
make install

The final command will install OptimLib into /usr/local.

Configuration options (see ./configure -h):

      Primary

  • -h print help
  • -i installation path; default: the build directory
  • -l specify the choice of linear algebra library; arma or eigen
  • -m specify the BLAS and Lapack libraries to link against; for example, -m "-lopenblas" or -m "-framework Accelerate"
  • -o compiler optimization options; defaults to -O3 -march=native -ffp-contract=fast -flto -DARMA_NO_DEBUG
  • -p enable OpenMP parallelization features (recommended)

      Secondary

  • -c a coverage build (used with Codecov)
  • -d a 'development' build
  • -g a debugging build (optimization flags set to -O0 -g)

      Special

  • --header-only-version generate a header-only version of OptimLib (see below)

Linear Algebra Library

OptimLib requires either the Armadillo or Eigen C++ linear algebra libraries.

Set (one) of the following environment variables before running configure:

export ARMA_INCLUDE_PATH=/path/to/armadillo
export EIGEN_INCLUDE_PATH=/path/to/eigen

Installation Method 2: Header-only Library

OptimLib is also available as a header-only library (i.e., without the need to compile a shared library). Simply run configure with the --header-only-version option:

./configure --header-only-version

This will create a new directory, header_only_version, containing a copy of OptimLib, modified to work on an inline basis. With this header-only version, simply include the header files (#include "optim.hpp) and set the include path to the head_only_version directory (e.g.,-I/path/to/optimlib/header_only_version).

R Compatibility

To use OptimLib with an R package, first generate a header-only version of the library (see above). Then add the compiler definition USE_RCPP_ARMADILLO before including the OptimLib files:

#define USE_RCPP_ARMADILLO
#include "optim.hpp"

Examples

To illustrate OptimLib at work, consider searching for the global minimum of the Ackley function:

Ackley

This is a well-known test function with many local minima. Newton-type methods (such as BFGS) are sensitive to the choice of initial values, and will perform rather poorly here. As such, we will employ a global search method; in this case: Differential Evolution.

Code:

#define OPTIM_ENABLE_ARMA_WRAPPERS
#include "optim.hpp"

//
// Ackley function

double ackley_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data)
{
    const double x = vals_inp(0);
    const double y = vals_inp(1);
    const double pi = arma::datum::pi;

    double obj_val = -20*std::exp( -0.2*std::sqrt(0.5*(x*x + y*y)) ) - std::exp( 0.5*(std::cos(2*pi*x) + std::cos(2*pi*y)) ) + 22.718282L;

    //

    return obj_val;
}

int main()
{
    // initial values:
    arma::vec x = arma::ones(2,1) + 1.0; // (2,2)

    //

    std::chrono::time_point<std::chrono::system_clock> start = std::chrono::system_clock::now();

    bool success = optim::de(x,ackley_fn,nullptr);

    std::chrono::time_point<std::chrono::system_clock> end = std::chrono::system_clock::now();
    std::chrono::duration<double> elapsed_seconds = end-start;

    if (success) {
        std::cout << "de: Ackley test completed successfully.\n"
                  << "elapsed time: " << elapsed_seconds.count() << "s\n";
    } else {
        std::cout << "de: Ackley test completed unsuccessfully." << std::endl;
    }

    arma::cout << "\nde: solution to Ackley test:\n" << x << arma::endl;

    return 0;
}

Compile and run:

g++ -Wall -std=c++11 -O3 -march=native -ffp-contract=fast -I/path/to/armadillo -I/path/to/optim/include optim_de_ex.cpp -o optim_de_ex.out -L/path/to/optim/lib -loptim
./optim_de_ex.out

Output:

de: Ackley test completed successfully.
elapsed time: 0.028167s

de: solution to Ackley test:
  -1.2702e-17
  -3.8432e-16

On a standard laptop, OptimLib will compute a solution to within machine precision in a fraction of a second.

Check the /tests directory for additional examples, and http://www.kthohr.com/optimlib.html for a detailed description of each algorithm.

Logistic regression

For a data-based example, consider maximum likelihood estimation of a logit model, common in statistics and machine learning. In this case we have closed-form expressions for the gradient and hessian. We will employ a popular gradient descent method, Adam (Adaptive Moment Estimation), and compare to a pure Newton-based algorithm.

#define OPTIM_ENABLE_ARMA_WRAPPERS
#include "optim.hpp"

// sigmoid function

inline
arma::mat sigm(const arma::mat& X)
{
    return 1.0 / (1.0 + arma::exp(-X));
}

// log-likelihood function data

struct ll_data_t
{
    arma::vec Y;
    arma::mat X;
};

// log-likelihood function with hessian

double ll_fn_whess(const arma::vec& vals_inp, arma::vec* grad_out, arma::mat* hess_out, void* opt_data)
{
    ll_data_t* objfn_data = reinterpret_cast<ll_data_t*>(opt_data);

    arma::vec Y = objfn_data->Y;
    arma::mat X = objfn_data->X;

    arma::vec mu = sigm(X*vals_inp);

    const double norm_term = static_cast<double>(Y.n_elem);

    const double obj_val = - arma::accu( Y%arma::log(mu) + (1.0-Y)%arma::log(1.0-mu) ) / norm_term;

    //

    if (grad_out)
    {
        *grad_out = X.t() * (mu - Y) / norm_term;
    }

    //

    if (hess_out)
    {
        arma::mat S = arma::diagmat( mu%(1.0-mu) );
        *hess_out = X.t() * S * X / norm_term;
    }

    //

    return obj_val;
}

// log-likelihood function for Adam

double ll_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data)
{
    return ll_fn_whess(vals_inp,grad_out,nullptr,opt_data);
}

//

int main()
{
    int n_dim = 5;     // dimension of parameter vector
    int n_samp = 4000; // sample length

    arma::mat X = arma::randn(n_samp,n_dim);
    arma::vec theta_0 = 1.0 + 3.0*arma::randu(n_dim,1);

    arma::vec mu = sigm(X*theta_0);

    arma::vec Y(n_samp);

    for (int i=0; i < n_samp; i++)
    {
        Y(i) = ( arma::as_scalar(arma::randu(1)) < mu(i) ) ? 1.0 : 0.0;
    }

    // fn data and initial values

    ll_data_t opt_data;
    opt_data.Y = std::move(Y);
    opt_data.X = std::move(X);

    arma::vec x = arma::ones(n_dim,1) + 1.0; // initial values

    // run Adam-based optim

    optim::algo_settings_t settings;

    settings.gd_method = 6;
    settings.gd_settings.step_size = 0.1;

    std::chrono::time_point<std::chrono::system_clock> start = std::chrono::system_clock::now();

    bool success = optim::gd(x,ll_fn,&opt_data,settings);

    std::chrono::time_point<std::chrono::system_clock> end = std::chrono::system_clock::now();
    std::chrono::duration<double> elapsed_seconds = end-start;

    //

    if (success) {
        std::cout << "Adam: logit_reg test completed successfully.\n"
                  << "elapsed time: " << elapsed_seconds.count() << "s\n";
    } else {
        std::cout << "Adam: logit_reg test completed unsuccessfully." << std::endl;
    }

    arma::cout << "\nAdam: true values vs estimates:\n" << arma::join_rows(theta_0,x) << arma::endl;

    //
    // run Newton-based optim

    x = arma::ones(n_dim,1) + 1.0; // initial values

    start = std::chrono::system_clock::now();

    success = optim::newton(x,ll_fn_whess,&opt_data);

    end = std::chrono::system_clock::now();
    elapsed_seconds = end-start;

    //

    if (success) {
        std::cout << "newton: logit_reg test completed successfully.\n"
                  << "elapsed time: " << elapsed_seconds.count() << "s\n";
    } else {
        std::cout << "newton: logit_reg test completed unsuccessfully." << std::endl;
    }

    arma::cout << "\nnewton: true values vs estimates:\n" << arma::join_rows(theta_0,x) << arma::endl;

    return 0;
}

Output:

Adam: logit_reg test completed successfully.
elapsed time: 0.025128s

Adam: true values vs estimates:
   2.7850   2.6993
   3.6561   3.6798
   2.3379   2.3860
   2.3167   2.4313
   2.2465   2.3064

newton: logit_reg test completed successfully.
elapsed time: 0.255909s

newton: true values vs estimates:
   2.7850   2.6993
   3.6561   3.6798
   2.3379   2.3860
   2.3167   2.4313
   2.2465   2.3064

Author

Keith O'Hara

License

Apache Version 2

Comments
  • cannot build it with eigen

    cannot build it with eigen

    ./configure -i "usr/local" -l eigen -p return Summary:

    • OS: linux-gnu

    • C++ compiler: g++

    • Build version: release

    • OPTIM_LINEAR_ALG_LIB set to: eigen

    • OPTIM_MATLIB_INCLUDE_PATH set to:

    • BLAS and Lapack libraries set to: -lblas -llapack

    • OpenMP features: enabled

    • optimization flags: -fPIC -O3 -march=native -ffp-contract=fast -flto -DNDEBUG -fopenmp

    • OptimLib install path: usr/local

    • Additional notes:

    Configuration completed. Creating Makefile... done.

    but when I run make, it fails, return g++ -std=c++11 -Wall -fPIC -O3 -march=native -ffp-contract=fast -flto -DNDEBUG -fopenmp -DOPTIM_ENABLE_EIGEN_WRAPPERS -I -I./include src/line_search/more_thuente.cpp -c -o src/line_search/more_thuente.o src/line_search/more_thuente.cpp:27:10: fatal error: optim.hpp: no such file #include "optim.hpp" ^~~~~~~~~~~ compilation terminated. make: *** [Makefile:54:src/line_search/more_thuente.o]

    opened by Horizon-42 12
  • Last step of GD leads to crash (eigen version)

    Last step of GD leads to crash (eigen version)

    The problem is in "error_reporting.ipp", line 80

    //settings_inp->opt_fn_value     = opt_objfn(x_p,nullptr,opt_data);
    

    https://drive.google.com/file/d/1XyAOyllOcnY3q10nHETtd6gvrWqGU4ov/view?usp=sharing https://drive.google.com/file/d/12qemu3_UEhJGcZ21mwypk5RPIFuXCXgf/view?usp=sharing

    Seems something was "std::moved" out of memory. Probably this is Eigen problem but may be you can work around it.

    Ubuntu 21.10. PS: could provide the full source if needed Btw: many thanks for the library )

    opened by FROL256 9
  • settings parameters not working

    settings parameters not working

    Hi,

    I am running your example of pso with bounded values. I have notices that if I set:

    settings.iter_max = 1; settings.err_tol = 1000; settings.opt_iter = 1;

    Nothing works. The algorithm still runs for iterations more than 1, and it reaches the global minimum (when correct upper and lower bounds are set). I would like to stop the optimization loop at specific number of iterations. Can you help in this please?

    opened by OsamaMazhar 7
  • PSO center_particle

    PSO center_particle

    First, I would like to thank you for making this library available.

    However, I was trying to use the PSO method with upper and lower bounds. For example, the Ackley function with

    X= [0.75 , 0.75]
    Lower = [0.5, 0.5]
    Upper = [1, 1]
    

    After some tests I understood that I should send the pso_settings.initial_ub and pso_settings.initial_lb, because the default broke the bound limits. So that was not a big issue. (I used 0.05 instead of 0.5)

    However, I think the center_particle option is a big issue with the bounds (At least in the initialize loop). In line 147

     P.row(i) = BMO_MATOPS_COLWISE_SUM( BMO_MATOPS_MIDDLE_ROWS(P, 0, n_pop-2) ) / static_cast<fp_t>(n_pop-1); // center vector
    

    It gets the center position of the particles using matrix P. However, in line 162

    if (vals_bound) {
          P.row(i) = transform<RowVec_t>(P.row(i), bounds_type, lower_bounds, upper_bounds);
    }
    

    The lines of P are rewritten with some values that are not inside the bounds. So the center vector is not inside as well.

    With the center_particle activated the results are very odd. After I deactivated it, the result is always good.

    opened by suzanamatos 6
  • How do we disable the transforming of bounds/parameters (LBFGS with val_bounds)

    How do we disable the transforming of bounds/parameters (LBFGS with val_bounds)

    Ive noticed theres a (log) transformation of the bounds which causes my parameters to become negative, whereby my parameters are strictly positive and Ive implemented this transformation external to the optmizer. In lbfgs.hpp

    // Line 200
        if (vals_bound) { // should we transform the parameters?
            x = transform(x, bounds_type, lower_bounds, upper_bounds);
        }
    

    I mean sure, I could comment it out, but is there an external way to disable this? without having to change the source code? Thanks

    opened by izsahara 6
  • Compiler issue using Eigen Wrappers

    Compiler issue using Eigen Wrappers

    Hello,

    I happen to have the following issue when I try to compile the library using Eigen. I have the same issue if I try to compile the header-only or the shared-library.

    It looks like there is a mismatch with Eigen and optim.

    I am using Eigen 3.3.4 version.

    /home/user/project/src/../header_only_version/unconstrained/nm.hpp:202:69 error: no match for call to ‘(optim::Vec_t {aka Eigen::Matrix<double, -1, 1>}) (optim::VecInt_t&)’
             simplex_fn_vals = OPTIM_MATOPS_EVAL(simplex_fn_vals(sort_vec));
    
    /home/user/project/src/../header_only_version/misc/matrix_ops/eval.hpp:30:35: note: in definition of macro ‘OPTIM_MATOPS_EVAL’
         #define OPTIM_MATOPS_EVAL(x) (x).eval()
                                       ^
    In file included from /usr/include/eigen3/Eigen/Core:414:0,
                     from /usr/include/eigen3/Eigen/StdVector:14,
                     from /usr/include/pcl-1.8/pcl/point_cloud.h:46,
                     from /usr/include/pcl-1.8/pcl/io/pcd_io.h:43,
                     from /home/kfavre/git_repository/3d-vision/recalage/PCL/mutual_information/src/mutual_information.cpp:12:
    /usr/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:115:41: note: candidate: Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType Eigen::DenseCoeffsBase<Derived, 0>::operator()(Eigen::Index, Eigen::Index) const [with Derived = Eigen::Matrix<double, -1, 1>; Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType = const double&; Eigen::Index = long int]
         EIGEN_STRONG_INLINE CoeffReturnType operator()(Index row, Index col) const
    
    
    etc....
    
    
    /home/user/project/src/../header_only_version/misc/matrix_ops/access.hpp:35:50: error: ‘all’ is not a member of ‘Eigen’
         #define OPTIM_MATOPS_ROWS(x, v) (x)(v,Eigen::all) // v is a vector
                                                      ^
    /home/user/project/src/../header_only_version/misc/matrix_ops/eval.hpp:30:35: note: in definition of macro ‘OPTIM_MATOPS_EVAL’
         #define OPTIM_MATOPS_EVAL(x) (x).eval()
    

    Here is the code that I want to execute (which is pretty simple):

    #define OPTIM_ENABLE_EIGEN_WRAPPERS
    #include "../header_only_version/optim.hpp"
      double ackley_fn(const Eigen::VectorXd& vals_inp, Eigen::VectorXd* grad_out, void* opt_data)
      {
          const double x = vals_inp(0);
          const double y = vals_inp(1);
    
          double obj_val = -20*std::exp( -0.2*std::sqrt(0.5*(x*x + y*y)) ) - std::exp( 0.5*(std::cos(2*M_PI*x) + std::cos(2*M_PI*y)) ) + 22.718282L;
    
          return obj_val;
      }
    
    int main (int argc, char** argv)
    {
    // initial values:
    		Eigen::VectorXd x = Eigen::VectorXd::Zero(2,1); // (2,2)
    		x[0] = 1.3;
    		x[1] = 1.4;
    
    	    bool success = optim::de(x,ackley_fn,nullptr);
                std::cout << "\nde: solution to Ackley test:\n" << x << std::endl;
    }
    
    

    Do not hesitate to ask if I can give any more precisions. Thank you in advance for your help!

    opened by kfavre 6
  • Use on Windows

    Use on Windows

    Is there a possibility to use the library on Windows ? I have built a header only version on a Linux system with the command "./configure --header-only-version". I copied it to Windows and want to use it now. With the compiler msvc2015. I also have Armadillo only as a header only variant. This is also in the program directory of my software.

    How do I get OptimLib to use Armadillo from the same workdirectory and get it running in general ?

    Since there is no DLL or LIB file available, the library should be included everywhere and compiler independent.

    opened by poggo96 5
  • Using header-only-version with Eigen failed to compile

    Using header-only-version with Eigen failed to compile

    System: macOS Compile cmd: g++ -DOPTIM_ENABLE_EIGEN_WRAPPERS --std=c++11 aa.cpp -I./header_only_version/ -I ./eigen-3.3.8/

    compiler output: In file included from aa.cpp:1: In file included from ./header_only_version/optim.hpp:51: ./header_only_version/unconstrained/nm.hpp:201:45: error: no matching function for call to object of type 'optim::Vec_t' (aka 'Matrix<double, Dynamic, 1>') simplex_fn_vals = OPTIM_MATOPS_EVAL(simplex_fn_vals(sort_vec)); ^~~~~~~~~~~~~~~ ./header_only_version/misc/matrix_ops/eval.hpp:30:35: note: expanded from macro 'OPTIM_MATOPS_EVAL' #define OPTIM_MATOPS_EVAL(x) (x).eval() ^ ./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:423:5: note: candidate function not viable: no known conversion from 'optim::VecInt_t' (aka 'Matrix<int, Dynamic, 1>') to 'Eigen::Index' (aka 'long') for 1st argument operator()(Index index) ^ ./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:178:5: note: candidate function not viable: no known conversion from 'optim::VecInt_t' (aka 'Matrix<int, Dynamic, 1>') to 'Eigen::Index' (aka 'long') for 1st argument operator()(Index index) const ^ ./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:115:41: note: candidate function not viable: requires 2 arguments, but 1 was provided EIGEN_STRONG_INLINE CoeffReturnType operator()(Index row, Index col) const ^ ./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:362:5: note: candidate function not viable: requires 2 arguments, but 1 was provided operator()(Index row, Index col) ^ In file included from aa.cpp:1: In file included from ./header_only_version/optim.hpp:51: ./header_only_version/unconstrained/nm.hpp:202:44: error: no member named 'all' in namespace 'Eigen' simplex_points = OPTIM_MATOPS_EVAL(OPTIM_MATOPS_ROWS(simplex_points, sort_vec)); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./header_only_version/misc/matrix_ops/access.hpp:35:50: note: expanded from macro 'OPTIM_MATOPS_ROWS' #define OPTIM_MATOPS_ROWS(x, v) (x)(v,Eigen::all) // v is a vector ~~~~~~~^ ./header_only_version/misc/matrix_ops/eval.hpp:30:35: note: expanded from macro 'OPTIM_MATOPS_EVAL' #define OPTIM_MATOPS_EVAL(x) (x).eval() ^ aa.cpp:3:24: error: use of undeclared identifier 'arma' double sphere_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data) ^ aa.cpp:3:45: error: use of undeclared identifier 'arma' double sphere_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data) ^ aa.cpp:5:22: error: use of undeclared identifier 'arma' double obj_val = arma::dot(vals_inp,vals_inp); ^ aa.cpp:14:23: error: use of undeclared identifier 'arma' double booth_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data) ^ aa.cpp:14:44: error: use of undeclared identifier 'arma' double booth_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data) ^ aa.cpp:36:5: error: use of undeclared identifier 'arma' arma::vec x = arma::ones(test_dim,1); // initial values (1,1,...,1) ^ aa.cpp:36:19: error: use of undeclared identifier 'arma' arma::vec x = arma::ones(test_dim,1); // initial values (1,1,...,1) ^ aa.cpp:46:5: error: use of undeclared identifier 'arma' arma::cout << "lbfgs: solution to sphere test:\n" << x << arma::endl; ^ aa.cpp:46:63: error: use of undeclared identifier 'arma' arma::cout << "lbfgs: solution to sphere test:\n" << x << arma::endl; ^ aa.cpp:51:5: error: use of undeclared identifier 'arma' arma::vec x_2 = arma::zeros(2,1); // initial values (0,0) ^ aa.cpp:51:21: error: use of undeclared identifier 'arma' arma::vec x_2 = arma::zeros(2,1); // initial values (0,0) ^ aa.cpp:61:5: error: use of undeclared identifier 'arma' arma::cout << "lbfgs: solution to Booth test:\n" << x_2 << arma::endl; ^ aa.cpp:61:64: error: use of undeclared identifier 'arma' arma::cout << "lbfgs: solution to Booth test:\n" << x_2 << arma::endl; ^ 15 errors generated.

    opened by lshok 4
  • Passing

    Passing "optim::algo_settings_t" object makes the optimizer crash

    Hi,

    First I'd like to thank you for creating such a wonderful library. I have created a simple example to learn how to use your library the right way. For example, I tried to estimate the parameters of the quadratic function y = 2 * x ^ 2 - 3 * x + 7 So the weights should be estimated as [2 -3 7]

    When I use the line bool success = optim::bfgs(w, quadratic, &data); everything seems to be working correctly and the weights are estimated as expected. Yet when I try to manipulate the optimization settings by simply using the block below

    optim::algo_settings_t settings;
    settings.iter_max = 500;
    bool success = optim::bfgs(w, quadratic, &data, settings);
    

    the program crashes without even returning a success flag. The following is the code I have written. I would very much appreciate your help.

    Thanks, Davar

    #include "optim.hpp"
    #include <iostream> 
    
    using std::cout;
    using std::endl;
    using std::vector;
    
    struct Data {
        arma::vec x;
        arma::vec y;
    };
    
    double quadratic(const arma::vec& w, arma::vec* grad, void* opt_data) {
    
        Data* data = reinterpret_cast<Data*>(opt_data);
        arma::vec x = data->x;
        arma::vec y = data->y;
    
        double f = 0;
        (*grad)(0) = (*grad)(1) = (*grad)(2) = 0;
    
        for (int i = 0; i < x.n_elem; i++) {
    
            double t = y(i) - (w(0) * pow(x(i), 2) + w(1) * x(i) + w(2));
            f += pow(t, 2);
    
            (*grad)(0) += -2 * pow(x(i), 2) * t;
            (*grad)(1) += -2 * x(i) * t;
            (*grad)(2) += -2 * t;
    
        }
    
        return f;
    }
    
    int main(){
    
        int numOfObservations = 100;
        arma::vec x(numOfObservations, arma::fill::randn);
        arma::vec y(numOfObservations);
        for (int i = 0; i < numOfObservations; i++) {
            y(i) = 2 * pow(x[i], 2) - 3 * x[i] + 7;
        }
        Data data;
        data.x = std::move(x);
        data.y = std::move(y);
    
        int numOfDims = 3;
        arma::vec w(numOfDims, arma::fill::zeros);
    
        bool success = optim::bfgs(w, quadratic, &data);
    
        //optim::algo_settings_t settings;
        //settings.iter_max = 500;
        //bool success = optim::bfgs(w, quadratic, &data, settings);
    
        cout << "success = " << success << " --> w = " << w.t() << endl;
    
        return 0;
    }
    
    
    
    opened by Davar1990 4
  • PSO parameter bounds causes error

    PSO parameter bounds causes error

    I‘m really a newbie to c++/Rcpp. Thus, I‘m not sure what causes my problem. I'm running a simulation in Rcpp using PSO to optimize parameter of an ode system in order to match my measured data. If I use the PSO with an unbounded parameter space everything works fine. However, if the parameter space is restricted part of the parameter passed to the objective function are “nan“. Any suggestion what the problem could be?

    Many thanks in advance for your help.

    opened by Konrad1991 4
  • Building Error

    Building Error

    I build OpenBlas 0.3.3 from sourceforge through cmake on Win7. But when building optim, There are 16 unresolved external symbols.

    logit_reg.obj: error: LNK2001: unresolved external symbol dgesv bfgs.obj: error: LNK2001: unresolved external symbol dposv_ lbfgs.obj: error: LNK2001: unresolved external symbol dgesdd_ D:\Project\Opti\Opti\optim\Bin\optim.exe: error: LNK1120: 16 unresolved external symbol_

    opened by 461353349 3
  • cg: modify search direction

    cg: modify search direction

    CG converges in two steps for Booth function with the modified search direction.

    The pull request includes the followings:

    • use search direction updated by cg_update().
    • use the first step size scaled by gradient norm rather than 1.
    opened by tenomoto 0
  • Cite OptimLib

    Cite OptimLib

    Is the following Citation correct for OptimLib?

    @misc{optimlib, author = {Keith O'Hara}, title = {{OptimLib, a lightweight C++ library of numerical optimization methods for nonlinear functions}}, url = {https://www.kthohr.com/optimlib.html}, howpublished = {\url{https://www.kthohr.com/optimlib.html}}, year = {2017} }

    opened by Mephphisto 0
  • Adding a conda recipe for this library

    Adding a conda recipe for this library

    Hello,

    It would be great if you could add a conda recipe for this library. This would allow to install it and use it easily in a jupyter notebook with xeus-cling.

    More background here, it's a similar request I made. https://github.com/Dobiasd/FunctionalPlus/issues/161

    Thank you

    opened by faysou 0
  • How to build OptimLib into shared library with MSVC compiler under Windows OS?

    How to build OptimLib into shared library with MSVC compiler under Windows OS?

    This is a fantastic C++ optimization library. I have compiled the examples with MSVC compiler and they ran well under Windows OS. I wonder if there is a guide to build OptimLib into windows dynamic link library (dll)?

    opened by ruixunbao 2
Owner
Keith O'Hara
Ph.D. (NYU), Econometrics and Quantitative Economics. Econometrician @aws
Keith O'Hara
Source code for the data dependency part of Jan Kossmann's PhD thesis "Unsupervised Database Optimization: Efficient Index Selection & Data Dependency-driven Query Optimization"

Unsupervised Database Optimization: Data Dependency-Driven Query Optimization Source code for the experiments presented in Chapter 8 of Jan Kossmann's

Jan Koßmann 4 Apr 24, 2022
A nonlinear MPC used to control an autonomous car.

MPC local planner A nonlinear MPC used to control an autonomous car. Description This repository contains an implementation of a nonlinear MPC that is

Tor Børve Rasmussen 11 Dec 8, 2022
Libft is an individual project at 42 that requires us to re-create some standard C library functions including some additional ones that can be used later to build a library of useful functions for the rest of the program.

Libft is an individual project at 42 that requires us to re-create some standard C library functions including some additional ones that can be used later to build a library of useful functions for the rest of the program.

Paulo Rafael Ramalho 0 Jan 1, 2023
SANM: A Symbolic Asymptotic Numerical Solver

SANM: A Symbolic Asymptotic Numerical Solver This repository is the official implementation of the SANM solver described in our paper to appear at SIG

Kai Jia 28 Sep 22, 2022
A C++17 library of computationally efficient methods for calculating sample statistics

Vectorized statistics using SIMD primitives Introduction is a C++17 library of computationally efficient methods for calculating sample statistics (me

HEAL 12 Nov 20, 2022
(R) Efficient methods and operators for the sparse matrix classes in 'Matrix' (esp. CSR format or "RsparseMatrix")

MatrixExtra MatrixExtra is an R package which extends the sparse matrix and sparse vector types in the Matrix package, particularly the CSR or Rsparse

null 15 Aug 29, 2022
Mobile platform for analysis of localization methods using the Intel RealSense T265 sensor

OptiBot Mobile platform for analysis of localization methods using the Intel RealSense T265 sensor About | Content | Implementation | License | Author

Kamil Goś 2 Feb 17, 2022
Fast and Light-weight path smoothing methods for vehicles

path_smoother About Fast and Light-weight path smoothing methods for vehicles Denpendencies This project has been tested on Ubuntu 18.04. sudo apt-get

MingwangZhao 4 Dec 1, 2021
A gazebo actor plugin that utilizes the map of the environment and graph search methods to generate random actor trajectories that don't pass through walls, furniture, etc.

Gazebo-Map-Actor-Plugin A gazebo actor plugin that utilizes the map of the environment and graph search methods to generate random actor trajectories

Yasin Sonmez 11 Dec 23, 2022
Injection - Windows process injection methods

Windows Process Injection Here are some popular methods used for process injection on the windows operating system. Conhost ExtraBytes PROPagate Servi

null 1.4k Dec 28, 2022
A PIC/FLIP fluid simulation based on the methods found in Robert Bridson's "Fluid Simulation for Computer Graphics"

GridFluidSim3d This program is an implementation of a PIC/FLIP liquid fluid simulation written in C++11 based on methods described in Robert Bridson's

Ryan Guy 727 Dec 8, 2022
std::tuple like methods for user defined types without any macro or boilerplate code

Boost.PFR This is a C++14 library for very basic reflection that gives you access to structure elements by index and provides other std::tuple like me

Boost.org 1.1k Dec 23, 2022
DG-Mesh-Optimization - Discontinuous Galerkin (DG) solver coupled with a Quasi-Newton line-search algorithm to optimize the DG mesh.

Date written: December 2020 This project was pursued as my final project for MECH 579 (Multidisciplinary Design Optimization) at McGill University, ta

Julien Brillon 8 Sep 18, 2022
Mod - MASTERS of DATA, a course about videogames data processing and optimization

MASTERS of DATA Welcome to MASTERS of DATA. A course oriented to Technical Designers, Technical Artists and any game developer that wants to understan

Ray 35 Dec 28, 2022
Artificial Intelligence for Kinematics, Dynamics, and Optimization

AIKIDO - AI for KIDO ⚠️ Warning: AIKIDO is under heavy development. These instructions are primarily for reference by the developers. AIKIDO is a C++

Personal Robotics Laboratory 190 Dec 9, 2022
HESS (Hyper Exponential Space Sorting) is a polynomial black-box optimization algorithm, that work very well with any NP-Complete, or NP-Hard problem

The original HESS (Hyper Exponential Space Sorting) is a polynomial black-box optimization algorithm, that work very well with any NP-Complete, or NP-Hard problem, at 2021 thanks to suggestions of Daniel Mattes, work like a complete algorithm.

SAT-X 3 Jan 18, 2022
Optimization-Based GNSS/INS Integrated Navigation System

OB_GINS Optimization-Based GNSS/INS Integrated Navigation System We open-source OB_GINS, an optimization-based GNSS/INS integrated navigation system.

i2Nav-WHU 289 Jan 7, 2023
GLSL optimizer based on Mesa's GLSL compiler. Used to be used in Unity for mobile shader optimization.

GLSL optimizer ⚠️ As of mid-2016, the project is unlikely to have any significant developments. At Unity we are moving to a different shader compilati

Aras Pranckevičius 1.6k Jan 3, 2023