I cloned the repo, created a "build" directory inside and ran "cmake .." from within that directory. Finally, I attempted to run "make". The compilation seems to be going well till this point:

[ 71%] Building CXX object CMakeFiles/bayesopt.dir/utils/fileparser.cpp.o /home/vlad/tools/bayesopt/utils/fileparser.cpp: In member function ‘bool bayesopt::utils::FileParser::fileExists()’: /home/vlad/tools/bayesopt/utils/fileparser.cpp:88:23: error: cannot convert ‘std::ifstream {aka std::basic_ifstream<char>}’ to ‘bool’ in initialization bool result = ifile; ^~~~~ CMakeFiles/bayesopt.dir/build.make:734: recipe for target 'CMakeFiles/bayesopt.dir/utils/fileparser.cpp.o' failed make[2]: *** [CMakeFiles/bayesopt.dir/utils/fileparser.cpp.o] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/bayesopt.dir/all' failed make[1]: *** [CMakeFiles/bayesopt.dir/all] Error 2 Makefile:127: recipe for target 'all' failed make: *** [all] Error 2

There isn't much Python code in this repo at all, just demos really. The interface is written in Cython which emits code that works with Python 2 or 3.

Because of this, it is trivial to support both 2 and 3 from the same code base.

Just some changes to print statements within the demos, and a simple bytes/strings conversion on the bopt parameters.

Is there any way to parallelize the function evaluations for the matlab implementation of bayesoptcont()?

If not for the actual optimization (due to the sequential dependency), then at least for the initial function evaluations (set by n_init_samples)? If I can have 20 of those initial samples running simultaneously using HTcondor, it's pretty wasteful to do them serially.

Hey Ruben, Sorry to disturb you. I have a question about categorical variables. My inputs are 8 binary variables (0/1). Here is the running status and the results.

bayesopt1.txt log1.txt bayesopt2.txt log2.txt

When "mParameters.noise" is small e.g. 1e-10, there is a error in log1.txt. But when "mParameters.noise" is equal to 1.0, there is not any error. Why did this happen ? This question has been bothering me for a long time. Have a favor.

Thanks a lot.

Cui

According to the python code, it seems to be compatible with python 3 but when I tried to install it with python 3.5, and import bayesopt, the error occurred:

ImportError: dynamic module does not define module export function(PyInit_bayesopt)

Any idea to solve this, many thanks.

I am not getting the DiscreteModel to run without a segmentation fault. (continuous model works fine)

It might be connected to the discrete parameter space. The error can be reproduced by changing the number of discrete points (line 80) in examples/bo_disc.cpp:

const size_t nPoints = 1000  // large space - works fine
const size_t nPoints = 10  // small space - segfaults

The first output of valgrind:

==64059== Memcheck, a memory error detector
==64059== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==64059== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==64059== Command: ./bin/bo_disc
==64059==
Running C++ interface
- 12:30:51.665538 INFO: Expected 6 hyperparameters. Replicating parameters and prior.
- 12:30:51.728427 INFO: Using default parameters for criteria.
==64059== Invalid read of size 8
==64059==    at 0x43D25A: bayesopt::DiscreteModel::generateInitialPoints(boost::numeric::ublas::matrix<double, boost::numeric::ublas::basic_row_major<unsigned long, long>, boost::numeric::ublas::unbounded_array<double, std::allocator<double> > >&) (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==    by 0x442E43: bayesopt::BayesOptBase::initializeOptimization() (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==    by 0x44383C: bayesopt::BayesOptBase::optimize(boost::numeric::ublas::vector<double, boost::numeric::ublas::unbounded_array<double, std::allocator<double> > >&) (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==    by 0x43528C: main (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==  Address 0x5a94bd8 is 8 bytes after a block of size 240 alloc'd
==64059==    at 0x4C2A6F0: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==64059==    by 0x43D355: bayesopt::DiscreteModel::generateInitialPoints(boost::numeric::ublas::matrix<double, boost::numeric::ublas::basic_row_major<unsigned long, long>, boost::numeric::ublas::unbounded_array<double, std::allocator<double> > >&) (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==    by 0x442E43: bayesopt::BayesOptBase::initializeOptimization() (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==    by 0x44383C: bayesopt::BayesOptBase::optimize(boost::numeric::ublas::vector<double, boost::numeric::ublas::unbounded_array<double, std::allocator<double> > >&) (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==    by 0x43528C: main (in /home/bach_tm/git_repos_temporary/bayesopt/build/bin/bo_disc)
==64059==

Compiler used: gcc / g++ 4.8.5

When optimising multidimensional functions I find that the center of the upper and lower bounds are often sampled. This is the precise center every time which makes me think that there is a hard coded probability of sampling the center somewhere, but I've not found where.

The reason I'm asking is because I would like to benchmark this package and many of the benchmark functions have their optimum near or at the center of the search space. These are the settings I've used for benchmark purposes

sc_type = SC_MAP
l_type = L_MCMC
init_method=1
n_inner_iterations=5000
n_iter_relearn=1
l_all=1

I have also tried force_jump = 0 however I still have the same problem of sampling the precise center. For example, optimising a 20 dimensional Rosenbrock function in the full space (-5.0,5.0) for every dimension, performs better than reducing the search space to (-5.0,2.0) due to the center being sampled. Is there a way for me to turn this off to make the benchmark fairer?

Also if you have suggestions of settings (other than those I showed) for benchmark purposes I'd be happy to change them. The run time is not important, I'm interested in reducing the function value as much as possible per iteration.

Hi I have been trying to install this on windows platform in MATLAB 2018a. After building successfully using mingw32, matlab_compile.m shows error MEX cannot find library 'bayesopt' specified with the -l option. MEX looks for a file with one of the names: libbayesopt.lib bayesopt.lib Please specify the path to this library with the -L option.

The lib files generated after building the source code are located in build/lib however they have extension of libbayesopt.a and libnlopt.a. The MATLAB are looking for .lib files. How can this issue be resolved.

About noise the instruction says: Too much noise results in slow convergence while not enough noise might result in not converging at all Why is that?

Hi Thanks for viewing the question. I run the bayes optimization for my problem and get the data below. The first column is output y and second column is x 317.402,-0.278159//10 sample, 70 iteration 485.787,-0.325544 489.675,-0.256076 577.022,-0.244675 339.859,-0.310618 289.268,-0.282267 603.213,-0.241337 383.85,-0.314427 428.921,-0.265534 264.013,-0.296477// sample ends 296.115,-0.2817 297.359,-0.2817 296.15,-0.2817 295.788,-0.2817 296.233,-0.2817 296.172,-0.2817 295.955,-0.2817 295.969,-0.2817 295.711,-0.2817 295.864,-0.2817 296.037,-0.2817 296.08,-0.2817 .....

As you can see the value of x converge to -0.2817, however you can see when x = 0.296477, the output is smaller(264). In fact as I run my function by adding 0.001 to x from -0.33 to 0.23 I found when x is about -0.2917 y will output the smallest value. In my problem the output is not deterministic because an algorithm called RANSAC but it will be near a certain value as you can see when x = -0.2817 y has different values but won't change too much. So what might be the problem? Why the bayesopt cannot find the minimum? I use the default parameter.

According to Porting Guide GCC 6 as mentioned in: https://stackoverflow.com/questions/38659115/make-fails-with-error-cannot-convert-stdistream-aka-stdbasic-istreamchar

Hi Ruben,

Thank you for this very nice Bayesian Optimization library! It works very well, and has some well thought out defaults and features! :-)

I managed to get this running on Google Colaboratory in a Python 3.6 environment (after many wasted hours), and I just wanted to share how I did this. I have a collaborator who is stuck on Windows, and this looks like it could be a possible solution for us.

I'm not sure if this would be worth mentioning in your documentation as an option, but maybe another user would find this information helpful.

This is a fairly ugly, so I'd be interested to hear if anyone has any cleaner ways to get this up and running.

The following are the commands I used to install this within Google Colaboratory notebook:

!apt install libboost-dev cmake cmake-curses-gui g++ python3-dev libboost-dev cmake cmake-curses-gui g++ cython3 freeglut3-dev
rm -rf /usr/include/numpy
!ln -s /usr/local/lib/python3.6/dist-packages/numpy/core/include/numpy /usr/include/numpy
!git clone https://github.com/rmcantin/bayesopt
cd bayesopt/
!cmake -DBAYESOPT_PYTHON_INTERFACE=ON -DPYTHON_LIBRARY=/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so -DPYTHON_INCLUDE_DIR=$(python-config --prefix)/include/python3.6 -DPYTHON_NUMPY_INCLUDE_DIR=/usr/lib/python3.6/dist-packages/numpy/core/include . && make && make install
cp /usr/lib/python2.7/dist-packages/bayes* /usr/lib/python3.6/
cd python/
%run demo_distance.py

And here is a sample notebook: https://colab.research.google.com/drive/1ajWJGdrZCdfRML4O6Ltv2NpOE4w_oyFF

Very excited to optimize some functions now!

All the best, CJ

Hi

I like your package and wrote a little julia wrapper.

One thing I couldn't easily figure out was how to access the surrogate model after fitting (my C++ is a bit limited). Is there an easy way (ideally through functions similar to the ones in the current C API) to access the surrogate model (e.g. inspect kernel parameters, sampling from the model or evaluating mean and sigma for some inputs)?

I'm using Python 3.6.3 :: Anaconda custom (64-bit) - I get the following error, when I try running the examples - File "demo_quad.py", line 22, in import bayesopt ImportError: dynamic module does not define module export function (PyInit_bayesopt)

I have tried re-generating the bayesopt.cpp with cython and rebuilding and installing the entire code base with the new cpp, no luck. I know that this issue cropped up(#10 and #11 ), but I still face the same problem with python 3.6.3, thanks. Kumar

Currently the optimization routine returns the x_i with smallest y_i observed. While this makes sense with deterministic functions, it doesn't make that much sense with stochastic functions where the minimum of the mean doesn't always coincide with the sample minimum. For this reason, it would be good to have an option for retrieving the minimum of the mean function instead of the minimum sampled value.

Have there been any thoughts of implementing an optimization routine (with interface to Python) that can optimize multiple types of parameters at once? This is often needed when optimizing machine learning algorithms. I'd really like to use it with deep learning, which has both continuous, discrete and categorical hyperparameters.