TrailDB is an efficient tool for storing and querying series of events

Overview

Gitter Travis CI Coverage Status

TrailDB

TrailDB is an efficient tool for storing and querying series of events. This repository contains the core C library and the tdb command line tool.

Learn more at traildb.io.

Quick start

For detailed installation instructions, see Getting Started guide.

Installing binaries

On OSX, TrailDB is available through homebrew:

$ brew install traildb

Linux binaries are not available yet.

Compiling and installing from source

Install Dependencies

$ apt-get install libarchive-dev libjudy-dev pkg-config

For RPM-based distros:

$ yum install judy-devel libarchive-devel pkg-config

For OSX:

$ brew install traildb/judy/judy libarchive pkg-config

For FreeBSD:

$ sudo pkg install python libarchive Judy pkgconf gcc

Note that your systems package manager may have too old of libjudy. You may also require a patch if you are using gcc 4.9.

Build TrailDB

$ ./waf configure
$ ./waf install

Alternatively you may use autotools

$ ./autogen.sh
$ ./configure
$ make
$ make install

Run Tests

$ ./waf test
Issues
  • Cannot build traildb on mac (El Capitan) using ./waf -configure

    Cannot build traildb on mac (El Capitan) using ./waf -configure

    The steps I used to build traildb on Mac El Capitan:

    1. brew install homebrew/boneyard/judy libarchive pkg-config
    2. Navigated to my users folder and ran: git clone https://github.com/traildb/traildb
    3. Navigated to the newly created traildb folder and ran: ./waf configure I get the following error: Setting top to : /Users/Sami/traildb Setting out to : /Users/Sami/traildb/build Checking for 'clang' (C compiler) : /usr/bin/clang Checking for program 'pkg-config' : /usr/local/bin/pkg-config Checking for 'libarchive' : yes Checking for library Judy : not found; install with 'brew install homebrew/boneyard/judy && brew link judy'
    4. relinking judy did not help.

    I have attached the config.log file. Kind Regards, Sami

    config.log.zip

    opened by samiislam 10
  • Build from master fails in docker

    Build from master fails in docker

    I am trying to build TrailDB from master on Ubuntu Xenial in Docker. First of all, your build dependencies are incorrect, they should be:

    apt-get install -y git libjudy-dev libarchive-dev pkg-config autoconf libtool

    However, going further:

    $ ./autogen.sh
    autoreconf: Entering directory `.'
    autoreconf: configure.ac: not using Gettext
    autoreconf: running: aclocal -I m4
    autoreconf: configure.ac: tracing
    autoreconf: running: libtoolize --copy
    autoreconf: running: /usr/bin/autoconf
    autoreconf: configure.ac: not using Autoheader
    autoreconf: running: automake --add-missing --copy --no-force
    autoreconf: Leaving directory `.'
    $ ./configure
    checking for a BSD-compatible install... /usr/bin/install -c
    checking whether build environment is sane... yes
    checking for a thread-safe mkdir -p... /bin/mkdir -p
    checking for gawk... no
    checking for mawk... mawk
    checking whether make sets $(MAKE)... yes
    checking whether make supports nested variables... yes
    checking for style of include used by make... GNU
    checking for gcc... gcc
    checking whether the C compiler works... yes
    checking for C compiler default output file name... a.out
    checking for suffix of executables...
    checking whether we are cross compiling... no
    checking for suffix of object files... o
    checking whether we are using the GNU C compiler... yes
    checking whether gcc accepts -g... yes
    checking for gcc option to accept ISO C89... none needed
    checking whether gcc understands -c and -o together... yes
    checking dependency style of gcc... gcc3
    checking for ar... ar
    checking the archiver (ar) interface... ar
    checking build system type... x86_64-pc-linux-gnu
    checking host system type... x86_64-pc-linux-gnu
    checking how to print strings... printf
    checking for a sed that does not truncate output... /bin/sed
    checking for grep that handles long lines and -e... /bin/grep
    checking for egrep... /bin/grep -E
    checking for fgrep... /bin/grep -F
    checking for ld used by gcc... /usr/bin/ld
    checking if the linker (/usr/bin/ld) is GNU ld... yes
    checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
    checking the name lister (/usr/bin/nm -B) interface... BSD nm
    checking whether ln -s works... yes
    checking the maximum length of command line arguments... 1572864
    checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
    checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
    checking for /usr/bin/ld option to reload object files... -r
    checking for objdump... objdump
    checking how to recognize dependent libraries... pass_all
    checking for dlltool... no
    checking how to associate runtime and link libraries... printf %s\n
    checking for archiver @FILE support... @
    checking for strip... strip
    checking for ranlib... ranlib
    checking command to parse /usr/bin/nm -B output from gcc object... ok
    checking for sysroot... no
    checking for a working dd... /bin/dd
    checking how to truncate binary pipes... /bin/dd bs=4096 count=1
    checking for mt... no
    checking if : is a manifest tool... no
    checking how to run the C preprocessor... gcc -E
    checking for ANSI C header files... yes
    checking for sys/types.h... yes
    checking for sys/stat.h... yes
    checking for stdlib.h... yes
    checking for string.h... yes
    checking for memory.h... yes
    checking for strings.h... yes
    checking for inttypes.h... yes
    checking for stdint.h... yes
    checking for unistd.h... yes
    checking for dlfcn.h... yes
    checking for objdir... .libs
    checking if gcc supports -fno-rtti -fno-exceptions... no
    checking for gcc option to produce PIC... -fPIC -DPIC
    checking if gcc PIC flag -fPIC -DPIC works... yes
    checking if gcc static flag -static works... yes
    checking if gcc supports -c -o file.o... yes
    checking if gcc supports -c -o file.o... (cached) yes
    checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
    checking whether -lc should be explicitly linked in... no
    checking dynamic linker characteristics... GNU/Linux ld.so
    checking how to hardcode library paths into programs... immediate
    checking whether stripping libraries is possible... yes
    checking if libtool supports shared libraries... yes
    checking whether to build shared libraries... yes
    checking whether to build static libraries... yes
    checking for gcc... (cached) gcc
    checking whether we are using the GNU C compiler... (cached) yes
    checking whether gcc accepts -g... (cached) yes
    checking for gcc option to accept ISO C89... (cached) none needed
    checking whether gcc understands -c and -o together... (cached) yes
    checking dependency style of gcc... (cached) gcc3
    checking for JudyHSIns in -lJudy... yes
    checking Judy.h usability... yes
    checking Judy.h presence... yes
    checking for Judy.h... yes
    checking that Judy is not broken... not broken
    checking archive.h usability... yes
    checking archive.h presence... yes
    checking for archive.h... yes
    checking for archive_write_new in -larchive... yes
    checking for __uint128_t... yes
    checking size of off_t... 8
    checking size of size_t... 8
    checking that generated files are newer than configure... done
    configure: creating ./config.status
    config.status: creating Makefile
    config.status: executing depfiles commands
    config.status: executing libtool commands
    $ make
    make
    /bin/bash ./libtool  --tag=CC   --mode=link gcc -Isrc/ -DJUDYERROR=judyerror_macro_missing_fix_this -O3 -g -Wall    -o tdbcli/tdb tdbcli/tdbcli_tdb-main.o tdbcli/tdbcli_tdb-op_dump.o tdbcli/tdbcli_tdb-op_make.o tdbcli/jsmn/tdbcli_tdb-jsmn.o libtraildb.la -larchive -lJudy
    libtool: link: gcc -Isrc/ -DJUDYERROR=judyerror_macro_missing_fix_this -O3 -g -Wall -o tdbcli/.libs/tdb tdbcli/tdbcli_tdb-main.o tdbcli/tdbcli_tdb-op_dump.o tdbcli/tdbcli_tdb-op_make.o tdbcli/jsmn/tdbcli_tdb-jsmn.o  ./.libs/libtraildb.so -larchive -lJudy
    tdbcli/tdbcli_tdb-op_dump.o: In function `op_dump':
    /tmp/traildb/tdbcli/op_dump.c:215: undefined reference to `parse_filter'
    collect2: error: ld returned 1 exit status
    Makefile:627: recipe for target 'tdbcli/tdb' failed
    make: *** [tdbcli/tdb] Error 1
    

    Missing dependency again? Or source code issue. Thanks for any info.

    opened by radekg 8
  • Installation process failing on OSX

    Installation process failing on OSX

    Following the instructions here fails like so:

    [email protected] ~/D/traildb (master) [1]> brew install judy
    Warning: judy-1.0.5 already installed
    [email protected] ~/D/traildb (master)> brew link judy
    Warning: Already linked: /usr/local/Cellar/judy/1.0.5
    To relink: brew unlink judy && brew link judy
    [email protected] ~/D/traildb (master)> ./waf configure
    Setting top to                           : /Users/liamdyer/DevApps/traildb
    Setting out to                           : /Users/liamdyer/DevApps/traildb/build
    Checking for 'clang' (C compiler)        : /usr/bin/clang
    Checking for program 'pkg-config'        : /usr/local/bin/pkg-config
    Checking for 'libarchive'                : yes
    Checking for library Judy                : not found; install with 'brew install judy && brew link judy'
    The configuration failed
    (complete log in /Users/liamdyer/DevApps/traildb/build/config.log)
    

    build/config.log:

    # project traildb (0.5) configured on Fri May 27 10:42:53 2016 by
    # waf 1.8.20 (abi 98, python 2070af0 on darwin)
    # using ./waf configure
    #
    ----------------------------------------
    Setting top to
    /Users/liamdyer/DevApps/traildb
    ----------------------------------------
    Setting out to
    /Users/liamdyer/DevApps/traildb/build
    ----------------------------------------
    Checking for 'clang' (C compiler)
    find program=['clang'] paths=['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/opt/X11/bin', '/usr/local/go/bin'] var='CC' -> ['/usr/bin/clang']
    ['/usr/bin/clang', '-dM', '-E', '-']
    out: #define OBJC_NEW_PROPERTIES 1
    #define _LP64 1
    #define __APPLE_CC__ 6000
    #define __APPLE__ 1
    #define __ATOMIC_ACQUIRE 2
    #define __ATOMIC_ACQ_REL 4
    #define __ATOMIC_CONSUME 1
    #define __ATOMIC_RELAXED 0
    #define __ATOMIC_RELEASE 3
    #define __ATOMIC_SEQ_CST 5
    #define __BIGGEST_ALIGNMENT__ 16
    #define __BLOCKS__ 1
    #define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__
    #define __CHAR16_TYPE__ unsigned short
    #define __CHAR32_TYPE__ unsigned int
    #define __CHAR_BIT__ 8
    #define __CONSTANT_CFSTRINGS__ 1
    #define __DBL_DENORM_MIN__ 4.9406564584124654e-324
    #define __DBL_DIG__ 15
    #define __DBL_EPSILON__ 2.2204460492503131e-16
    #define __DBL_HAS_DENORM__ 1
    #define __DBL_HAS_INFINITY__ 1
    #define __DBL_HAS_QUIET_NAN__ 1
    #define __DBL_MANT_DIG__ 53
    #define __DBL_MAX_10_EXP__ 308
    #define __DBL_MAX_EXP__ 1024
    #define __DBL_MAX__ 1.7976931348623157e+308
    #define __DBL_MIN_10_EXP__ (-307)
    #define __DBL_MIN_EXP__ (-1021)
    #define __DBL_MIN__ 2.2250738585072014e-308
    #define __DECIMAL_DIG__ 21
    #define __DYNAMIC__ 1
    #define __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ 101100
    #define __FINITE_MATH_ONLY__ 0
    #define __FLT_DENORM_MIN__ 1.40129846e-45F
    #define __FLT_DIG__ 6
    #define __FLT_EPSILON__ 1.19209290e-7F
    #define __FLT_EVAL_METHOD__ 0
    #define __FLT_HAS_DENORM__ 1
    #define __FLT_HAS_INFINITY__ 1
    #define __FLT_HAS_QUIET_NAN__ 1
    #define __FLT_MANT_DIG__ 24
    #define __FLT_MAX_10_EXP__ 38
    #define __FLT_MAX_EXP__ 128
    #define __FLT_MAX__ 3.40282347e+38F
    #define __FLT_MIN_10_EXP__ (-37)
    #define __FLT_MIN_EXP__ (-125)
    #define __FLT_MIN__ 1.17549435e-38F
    #define __FLT_RADIX__ 2
    #define __GCC_ATOMIC_BOOL_LOCK_FREE 2
    #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 2
    #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
    #define __GCC_ATOMIC_CHAR_LOCK_FREE 2
    #define __GCC_ATOMIC_INT_LOCK_FREE 2
    #define __GCC_ATOMIC_LLONG_LOCK_FREE 2
    #define __GCC_ATOMIC_LONG_LOCK_FREE 2
    #define __GCC_ATOMIC_POINTER_LOCK_FREE 2
    #define __GCC_ATOMIC_SHORT_LOCK_FREE 2
    #define __GCC_ATOMIC_TEST_AND_SET_TRUEVAL 1
    #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
    #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 1
    #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 1
    #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 1
    #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 1
    #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 1
    #define __GNUC_MINOR__ 2
    #define __GNUC_PATCHLEVEL__ 1
    #define __GNUC_STDC_INLINE__ 1
    #define __GNUC__ 4
    #define __GXX_ABI_VERSION 1002
    #define __GXX_RTTI 1
    #define __INT16_C_SUFFIX__
    #define __INT16_FMTd__ "hd"
    #define __INT16_FMTi__ "hi"
    #define __INT16_MAX__ 32767
    #define __INT16_TYPE__ short
    #define __INT32_C_SUFFIX__
    #define __INT32_FMTd__ "d"
    #define __INT32_FMTi__ "i"
    #define __INT32_MAX__ 2147483647
    #define __INT32_TYPE__ int
    #define __INT64_C_SUFFIX__ LL
    #define __INT64_FMTd__ "lld"
    #define __INT64_FMTi__ "lli"
    #define __INT64_MAX__ 9223372036854775807LL
    #define __INT64_TYPE__ long long int
    #define __INT8_C_SUFFIX__
    #define __INT8_FMTd__ "hhd"
    #define __INT8_FMTi__ "hhi"
    #define __INT8_MAX__ 127
    #define __INT8_TYPE__ signed char
    #define __INTMAX_C_SUFFIX__ L
    #define __INTMAX_FMTd__ "ld"
    #define __INTMAX_FMTi__ "li"
    #define __INTMAX_MAX__ 9223372036854775807L
    #define __INTMAX_TYPE__ long int
    #define __INTMAX_WIDTH__ 64
    #define __INTPTR_FMTd__ "ld"
    #define __INTPTR_FMTi__ "li"
    #define __INTPTR_MAX__ 9223372036854775807L
    #define __INTPTR_TYPE__ long int
    #define __INTPTR_WIDTH__ 64
    #define __INT_FAST16_FMTd__ "hd"
    #define __INT_FAST16_FMTi__ "hi"
    #define __INT_FAST16_MAX__ 32767
    #define __INT_FAST16_TYPE__ short
    #define __INT_FAST32_FMTd__ "d"
    #define __INT_FAST32_FMTi__ "i"
    #define __INT_FAST32_MAX__ 2147483647
    #define __INT_FAST32_TYPE__ int
    #define __INT_FAST64_FMTd__ "ld"
    #define __INT_FAST64_FMTi__ "li"
    #define __INT_FAST64_MAX__ 9223372036854775807L
    #define __INT_FAST64_TYPE__ long int
    #define __INT_FAST8_FMTd__ "hhd"
    #define __INT_FAST8_FMTi__ "hhi"
    #define __INT_FAST8_MAX__ 127
    #define __INT_FAST8_TYPE__ signed char
    #define __INT_LEAST16_FMTd__ "hd"
    #define __INT_LEAST16_FMTi__ "hi"
    #define __INT_LEAST16_MAX__ 32767
    #define __INT_LEAST16_TYPE__ short
    #define __INT_LEAST32_FMTd__ "d"
    #define __INT_LEAST32_FMTi__ "i"
    #define __INT_LEAST32_MAX__ 2147483647
    #define __INT_LEAST32_TYPE__ int
    #define __INT_LEAST64_FMTd__ "ld"
    #define __INT_LEAST64_FMTi__ "li"
    #define __INT_LEAST64_MAX__ 9223372036854775807L
    #define __INT_LEAST64_TYPE__ long int
    #define __INT_LEAST8_FMTd__ "hhd"
    #define __INT_LEAST8_FMTi__ "hhi"
    #define __INT_LEAST8_MAX__ 127
    #define __INT_LEAST8_TYPE__ signed char
    #define __INT_MAX__ 2147483647
    #define __LDBL_DENORM_MIN__ 3.64519953188247460253e-4951L
    #define __LDBL_DIG__ 18
    #define __LDBL_EPSILON__ 1.08420217248550443401e-19L
    #define __LDBL_HAS_DENORM__ 1
    #define __LDBL_HAS_INFINITY__ 1
    #define __LDBL_HAS_QUIET_NAN__ 1
    #define __LDBL_MANT_DIG__ 64
    #define __LDBL_MAX_10_EXP__ 4932
    #define __LDBL_MAX_EXP__ 16384
    #define __LDBL_MAX__ 1.18973149535723176502e+4932L
    #define __LDBL_MIN_10_EXP__ (-4931)
    #define __LDBL_MIN_EXP__ (-16381)
    #define __LDBL_MIN__ 3.36210314311209350626e-4932L
    #define __LITTLE_ENDIAN__ 1
    #define __LONG_LONG_MAX__ 9223372036854775807LL
    #define __LONG_MAX__ 9223372036854775807L
    #define __LP64__ 1
    #define __MACH__ 1
    #define __MMX__ 1
    #define __NO_INLINE__ 1
    #define __NO_MATH_INLINES 1
    #define __ORDER_BIG_ENDIAN__ 4321
    #define __ORDER_LITTLE_ENDIAN__ 1234
    #define __ORDER_PDP_ENDIAN__ 3412
    #define __PIC__ 2
    #define __POINTER_WIDTH__ 64
    #define __PRAGMA_REDEFINE_EXTNAME 1
    #define __PTRDIFF_FMTd__ "ld"
    #define __PTRDIFF_FMTi__ "li"
    #define __PTRDIFF_MAX__ 9223372036854775807L
    #define __PTRDIFF_TYPE__ long int
    #define __PTRDIFF_WIDTH__ 64
    #define __REGISTER_PREFIX__
    #define __SCHAR_MAX__ 127
    #define __SHRT_MAX__ 32767
    #define __SIG_ATOMIC_MAX__ 2147483647
    #define __SIG_ATOMIC_WIDTH__ 32
    #define __SIZEOF_DOUBLE__ 8
    #define __SIZEOF_FLOAT__ 4
    #define __SIZEOF_INT128__ 16
    #define __SIZEOF_INT__ 4
    #define __SIZEOF_LONG_DOUBLE__ 16
    #define __SIZEOF_LONG_LONG__ 8
    #define __SIZEOF_LONG__ 8
    #define __SIZEOF_POINTER__ 8
    #define __SIZEOF_PTRDIFF_T__ 8
    #define __SIZEOF_SHORT__ 2
    #define __SIZEOF_SIZE_T__ 8
    #define __SIZEOF_WCHAR_T__ 4
    #define __SIZEOF_WINT_T__ 4
    #define __SIZE_FMTX__ "lX"
    #define __SIZE_FMTo__ "lo"
    #define __SIZE_FMTu__ "lu"
    #define __SIZE_FMTx__ "lx"
    #define __SIZE_MAX__ 18446744073709551615UL
    #define __SIZE_TYPE__ long unsigned int
    #define __SIZE_WIDTH__ 64
    #define __SSE2_MATH__ 1
    #define __SSE2__ 1
    #define __SSE3__ 1
    #define __SSE_MATH__ 1
    #define __SSE__ 1
    #define __SSP__ 1
    #define __SSSE3__ 1
    #define __STDC_HOSTED__ 1
    #define __STDC_UTF_16__ 1
    #define __STDC_UTF_32__ 1
    #define __STDC_VERSION__ 201112L
    #define __STDC__ 1
    #define __UINT16_C_SUFFIX__
    #define __UINT16_FMTX__ "hX"
    #define __UINT16_FMTo__ "ho"
    #define __UINT16_FMTu__ "hu"
    #define __UINT16_FMTx__ "hx"
    #define __UINT16_MAX__ 65535
    #define __UINT16_TYPE__ unsigned short
    #define __UINT32_C_SUFFIX__ U
    #define __UINT32_FMTX__ "X"
    #define __UINT32_FMTo__ "o"
    #define __UINT32_FMTu__ "u"
    #define __UINT32_FMTx__ "x"
    #define __UINT32_MAX__ 4294967295U
    #define __UINT32_TYPE__ unsigned int
    #define __UINT64_C_SUFFIX__ ULL
    #define __UINT64_FMTX__ "llX"
    #define __UINT64_FMTo__ "llo"
    #define __UINT64_FMTu__ "llu"
    #define __UINT64_FMTx__ "llx"
    #define __UINT64_MAX__ 18446744073709551615ULL
    #define __UINT64_TYPE__ long long unsigned int
    #define __UINT8_C_SUFFIX__
    #define __UINT8_FMTX__ "hhX"
    #define __UINT8_FMTo__ "hho"
    #define __UINT8_FMTu__ "hhu"
    #define __UINT8_FMTx__ "hhx"
    #define __UINT8_MAX__ 255
    #define __UINT8_TYPE__ unsigned char
    #define __UINTMAX_C_SUFFIX__ UL
    #define __UINTMAX_FMTX__ "lX"
    #define __UINTMAX_FMTo__ "lo"
    #define __UINTMAX_FMTu__ "lu"
    #define __UINTMAX_FMTx__ "lx"
    #define __UINTMAX_MAX__ 18446744073709551615UL
    #define __UINTMAX_TYPE__ long unsigned int
    #define __UINTMAX_WIDTH__ 64
    #define __UINTPTR_FMTX__ "lX"
    #define __UINTPTR_FMTo__ "lo"
    #define __UINTPTR_FMTu__ "lu"
    #define __UINTPTR_FMTx__ "lx"
    #define __UINTPTR_MAX__ 18446744073709551615UL
    #define __UINTPTR_TYPE__ long unsigned int
    #define __UINTPTR_WIDTH__ 64
    #define __UINT_FAST16_FMTX__ "hX"
    #define __UINT_FAST16_FMTo__ "ho"
    #define __UINT_FAST16_FMTu__ "hu"
    #define __UINT_FAST16_FMTx__ "hx"
    #define __UINT_FAST16_MAX__ 65535
    #define __UINT_FAST16_TYPE__ unsigned short
    #define __UINT_FAST32_FMTX__ "X"
    #define __UINT_FAST32_FMTo__ "o"
    #define __UINT_FAST32_FMTu__ "u"
    #define __UINT_FAST32_FMTx__ "x"
    #define __UINT_FAST32_MAX__ 4294967295U
    #define __UINT_FAST32_TYPE__ unsigned int
    #define __UINT_FAST64_FMTX__ "lX"
    #define __UINT_FAST64_FMTo__ "lo"
    #define __UINT_FAST64_FMTu__ "lu"
    #define __UINT_FAST64_FMTx__ "lx"
    #define __UINT_FAST64_MAX__ 18446744073709551615UL
    #define __UINT_FAST64_TYPE__ long unsigned int
    #define __UINT_FAST8_FMTX__ "hhX"
    #define __UINT_FAST8_FMTo__ "hho"
    #define __UINT_FAST8_FMTu__ "hhu"
    #define __UINT_FAST8_FMTx__ "hhx"
    #define __UINT_FAST8_MAX__ 255
    #define __UINT_FAST8_TYPE__ unsigned char
    #define __UINT_LEAST16_FMTX__ "hX"
    #define __UINT_LEAST16_FMTo__ "ho"
    #define __UINT_LEAST16_FMTu__ "hu"
    #define __UINT_LEAST16_FMTx__ "hx"
    #define __UINT_LEAST16_MAX__ 65535
    #define __UINT_LEAST16_TYPE__ unsigned short
    #define __UINT_LEAST32_FMTX__ "X"
    #define __UINT_LEAST32_FMTo__ "o"
    #define __UINT_LEAST32_FMTu__ "u"
    #define __UINT_LEAST32_FMTx__ "x"
    #define __UINT_LEAST32_MAX__ 4294967295U
    #define __UINT_LEAST32_TYPE__ unsigned int
    #define __UINT_LEAST64_FMTX__ "lX"
    #define __UINT_LEAST64_FMTo__ "lo"
    #define __UINT_LEAST64_FMTu__ "lu"
    #define __UINT_LEAST64_FMTx__ "lx"
    #define __UINT_LEAST64_MAX__ 18446744073709551615UL
    #define __UINT_LEAST64_TYPE__ long unsigned int
    #define __UINT_LEAST8_FMTX__ "hhX"
    #define __UINT_LEAST8_FMTo__ "hho"
    #define __UINT_LEAST8_FMTu__ "hhu"
    #define __UINT_LEAST8_FMTx__ "hhx"
    #define __UINT_LEAST8_MAX__ 255
    #define __UINT_LEAST8_TYPE__ unsigned char
    #define __USER_LABEL_PREFIX__ _
    #define __VERSION__ "4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)"
    #define __WCHAR_MAX__ 2147483647
    #define __WCHAR_TYPE__ int
    #define __WCHAR_WIDTH__ 32
    #define __WINT_TYPE__ int
    #define __WINT_WIDTH__ 32
    #define __amd64 1
    #define __amd64__ 1
    #define __apple_build_version__ 7000181
    #define __block __attribute__((__blocks__(byref)))
    #define __clang__ 1
    #define __clang_major__ 7
    #define __clang_minor__ 0
    #define __clang_patchlevel__ 2
    #define __clang_version__ "7.0.2 (clang-700.1.81)"
    #define __core2 1
    #define __core2__ 1
    #define __llvm__ 1
    #define __nonnull _Nonnull
    #define __null_unspecified _Null_unspecified
    #define __nullable _Nullable
    #define __pic__ 2
    #define __strong
    #define __tune_core2__ 1
    #define __unsafe_unretained
    #define __weak __attribute__((objc_gc(weak)))
    #define __x86_64 1
    #define __x86_64__ 1
    
    find program=['llvm-ar', 'ar'] paths=['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/opt/X11/bin', '/usr/local/go/bin'] var='AR' -> ['/usr/bin/ar']
    find program=['ar'] paths=['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/opt/X11/bin', '/usr/local/go/bin'] var='AR' -> ['/usr/bin/ar']
    /usr/bin/clang
    ----------------------------------------
    Checking for program 'pkg-config'
    /usr/local/bin/pkg-config
    find program=['pkg-config'] paths=['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/opt/X11/bin', '/usr/local/go/bin'] var='PKGCONFIG' -> ['/usr/local/bin/pkg-config']
    ----------------------------------------
    Checking for 'libarchive'
    ['/usr/local/bin/pkg-config', 'libarchive >= 3.0.0', '--cflags', '--libs', 'libarchive']
    out: -I/usr/local/Cellar/libarchive/3.1.2/include -L/usr/local/Cellar/libarchive/3.1.2/lib -larchive
    
    yes
    ----------------------------------------
    Checking for library Judy
    ==>
    
    int main(int argc, char **argv) {
        (void)argc; (void)argv;
        return 0;
    }
    
    <==
    [1/2] Compiling build/.conf_check_d705d54cbd73c367ed5df132f0653521/test.c
    
    ['/usr/bin/clang', '-std=c99', '-O3', '-g', '-DDSFMT_MEXP=521', '-DHAVE_ARCHIVE_H=1', '../test.c', '-c', '-o', '/Users/liamdyer/DevApps/traildb/build/.conf_check_d705d54cbd73c367ed5df132f0653521/testbuild/test.c.1.o']
    [2/2] Linking build/.conf_check_d705d54cbd73c367ed5df132f0653521/testbuild/testprog
    
    ['/usr/bin/clang', 'test.c.1.o', '-o', '/Users/liamdyer/DevApps/traildb/build/.conf_check_d705d54cbd73c367ed5df132f0653521/testbuild/testprog', '-lJudy']
    err: ld: library not found for -lJudy
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    
    from /Users/liamdyer/DevApps/traildb: Test does not build: Traceback (most recent call last):
      File "/Users/liamdyer/DevApps/traildb/.waf-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Configure.py", line 335, in run_build
        bld.compile()
      File "/Users/liamdyer/DevApps/traildb/.waf-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Build.py", line 185, in compile
        raise Errors.BuildError(self.producer.error)
    BuildError: Build failed
     -> task in 'testprog' failed (exit status 1):
        {task 4370778576: cprogram test.c.1.o -> testprog}
    ['/usr/bin/clang', 'test.c.1.o', '-o', '/Users/liamdyer/DevApps/traildb/build/.conf_check_d705d54cbd73c367ed5df132f0653521/testbuild/testprog', '-lJudy']
    
    not found; install with 'brew install judy && brew link judy'
    from /Users/liamdyer/DevApps/traildb: The configuration failed
    
    opened by dyerw 8
  • add time-slice filter

    add time-slice filter

    Similar to event / column / row filter, support a filter that excludes all events outside a given time-range. Having this built-in would allow it to be used with tdb_cons_append for efficient materialized views.

    enhancement 
    opened by tuulos 8
  • Build Problems

    Build Problems

    On a fresh install of Debian 8.5 I am unable to get TDB to build properly using either waf or autotools.

    Here is what I am doing:

    pwd
    ~/Development/traildb
    ./waf configure
    Setting top to                           : /home/username/Development/traildb
    Setting out to                           : /home/username/Development/traildb/build
    Checking for 'gcc' (C compiler)          : /usr/bin/gcc
    Checking for program 'pkg-config'        : /usr/bin/pkg-config
    Checking for 'libarchive'                : yes
    Checking for library Judy                : yes
    Checking Judy version                    : yes
    
    ./waf build
    {lots of output, looked good to me}
    
    sudo ./waf install
    Waf: Entering directory `/home/username/Development/traildb/build'
    + install /usr/local/include/traildb.h (from src/traildb.h)
    + install /usr/local/lib/libtraildb.a (from build/libtraildb.a)
    + install /usr/local/include/tdb_error.h (from src/tdb_error.h)
    + install /usr/local/include/tdb_types.h (from src/tdb_types.h)
    + install /usr/local/include/tdb_limits.h (from src/tdb_limits.h)
    + install /usr/local/lib/libtraildb.so.0 (from build/libtraildb.so)
    + symlink /usr/local/lib/libtraildb.so (to libtraildb.so.0)
    + install /usr/local/bin/traildb_bench (from build/traildb_bench)
    + install /usr/local/bin/tdb (from build/tdb)
    + install /usr/local/lib/libtdbindex.so.0 (from build/libtdbindex.so)
    + symlink /usr/local/lib/libtdbindex.so (to libtdbindex.so.0)
    + install /usr/local/share/pkgconfig/traildb.pc (from build/traildb.pc)
    Waf: Leaving directory `/home/username/Development/traildb/build'
    'install' finished successfully (0.141s)
    

    Looks good so far, however running tdb gives the following error:

    tdb: error while loading shared libraries: libtraildb.so.0: cannot open shared object file: No such file or directory
    

    Running ldd on the tdb binary shows the following problem:

    ldd /usr/local/bin/tdb
        linux-vdso.so.1 (0x00007ffde11f7000)
        libtraildb.so.0 => not found
        libJudy.so.1 => /usr/lib/x86_64-linux-gnu/libJudy.so.1 (0x00007f8e97a93000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8e97876000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8e974cb000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8e97ce5000)
    

    I am also unable to use the libtraildb.so.0 file for native bindings (javascript). The libtraildb.so.0 is present in the following directory: /usr/local/lib.

    Any idea what I am doing wrong here? Thanks.

    opened by KalenAnson 7
  • Fix several issues with merge in tdbcli

    Fix several issues with merge in tdbcli

    Issues fixed include:

    1. tdb merge now works with a single traildb (useful for filtering)
    2. Correctly identify when input traildbs have the same fields and use direct cons'ing
    3. When merging traildbs with the same fields, return fields in order found in first traildb to avoid failure when cons'ing
    4. Check number of user fields against counter after checking for duplicates to avoid edge case where last traildb has a different field name but same number of fields.
    opened by rnowling-adroll 4
  • Build error

    Build error

    Hi

    I'm trying to build traildb on centos and get following error:

    make /bin/sh ./libtool --tag=CC --mode=link gcc -Isrc/ -DJUDYERROR=judyerror_macro_missing_fix_this -O3 -g -Wall -o tdbcli/tdb tdbcli/tdbcli_tdb-main.o tdbcli/tdbcli_tdb-op_dump.o tdbcli/tdbcli_tdb-op_make.o tdbcli/jsmn/tdbcli_tdb-jsmn.o libtraildb.la -larchive -lJudy libtool: link: gcc -Isrc/ -DJUDYERROR=judyerror_macro_missing_fix_this -O3 -g -Wall -o tdbcli/.libs/tdb tdbcli/tdbcli_tdb-main.o tdbcli/tdbcli_tdb-op_dump.o tdbcli/tdbcli_tdb-op_make.o tdbcli/jsmn/tdbcli_tdb-jsmn.o ./.libs/libtraildb.so -larchive -lJudy -Wl,-rpath -Wl,/usr/local/lib tdbcli/tdbcli_tdb-op_dump.o: In function tdb_cursor_next': /nfs/science/globalscience/mindis/traildb/src/traildb.h:220: multiple definition oftdb_cursor_next' tdbcli/tdbcli_tdb-main.o:/nfs/science/globalscience/mindis/traildb/src/traildb.h:220: first defined here tdbcli/tdbcli_tdb-op_make.o: In function tdb_cursor_next': /nfs/science/globalscience/mindis/traildb/src/traildb.h:220: multiple definition oftdb_cursor_next' tdbcli/tdbcli_tdb-main.o:/nfs/science/globalscience/mindis/traildb/src/traildb.h:220: first defined here tdbcli/tdbcli_tdb-op_dump.o: In function op_dump': op_dump.c:(.text+0xf1): undefined reference toparse_filter' collect2: error: ld returned 1 exit status make: *** [tdbcli/tdb] Error 1

    opened by mindis 4
  • D binding + compile stuff

    D binding + compile stuff

    @tuulos please review. This follows our work from today. This PR:

    • fixes the python example on mac
    • improves the compile script to run on OSX
    • adds all the D binding stuff into dsrc/ + a compile script
    • starts a high-level API in TrailDB.d to query the database. Example of usage in examples.d, which currently gives this output:
    Size of the DB: 49147613
    Number of dimensions: 9
    Dimensions: ["type", "campaign_eid", "segment_eid", "advertisable_eid", "adgroup_eid", "ad_eid", "geo_country_code", "network", "inventory_source"]
    --- Detail of cookie index 120 trail. ---
    Full trail:
    1407665036&pxl&&AIRHWWJQQJGWZB4BZMNWQ7&IIJ3DMHWNVCHFODZWRIVMO&&&US&&|1407665682&pxl&&AIRHWWJQQJGWZB4BZMNWQ7&IIJ3DMHWNVCHFODZWRIVMO&&&US&&|1407666452&imp&&&&JNJRRE7JX5BRJICOFCRN7H&4FDOAPI7JFEJ3JGNI64LJY&US&g&|1407666472&imp&&&&JNJRRE7JX5BRJICOFCRN7H&65B2I63OSRCEZCM4EHMROT&US&g&|1407666472&imp&&&&JNJRRE7JX5BRJICOFCRN7H&4FDOAPI7JFEJ3JGNI64LJY&US&g&|1407666969&imp&&&&JNJRRE7JX5BRJICOFCRN7H&65B2I63OSRCEZCM4EHMROT&US&g&|1407668236&pxl&&AIRHWWJQQJGWZB4BZMNWQ7&IIJ3DMHWNVCHFODZWRIVMO&&&US&&
    Timestamps of events: [2014-Aug-10 10:03:56, 2014-Aug-10 10:14:42, 2014-Aug-10 10:27:32, 2014-Aug-10 10:27:52, 2014-Aug-10 10:27:52, 2014-Aug-10 10:36:09, 2014-Aug-10 10:57:16]
    Number of events: 7
    Number of pxl events: 3
    Number of imp events: 4
    0M cookies seen.
    1M cookies seen.
    2M cookies seen.
    3M cookies seen.
    4M cookies seen.
    5M cookies seen.
    6M cookies seen.
    7M cookies seen.
    8M cookies seen.
    9M cookies seen.
    10M cookies seen.
    11M cookies seen.
    12M cookies seen.
    13M cookies seen.
    14M cookies seen.
    15M cookies seen.
    16M cookies seen.
    17M cookies seen.
    18M cookies seen.
    19M cookies seen.
    20M cookies seen.
    21M cookies seen.
    22M cookies seen.
    23M cookies seen.
    24M cookies seen.
    25M cookies seen.
    26M cookies seen.
    27M cookies seen.
    28M cookies seen.
    29M cookies seen.
    30M cookies seen.
    31M cookies seen.
    32M cookies seen.
    33M cookies seen.
    34M cookies seen.
    35M cookies seen.
    36M cookies seen.
    37M cookies seen.
    38M cookies seen.
    39M cookies seen.
    40M cookies seen.
    41M cookies seen.
    42M cookies seen.
    43M cookies seen.
    44M cookies seen.
    45M cookies seen.
    46M cookies seen.
    47M cookies seen.
    48M cookies seen.
    49M cookies seen.
    Average number of pxl fired per cookie: 3.47027
    

    I will keep improving the DB to add hundreds of nice ways to query the DB + add doc, but I'd like your first go on this so I can keep iterating on the module TrailDB.d.

    opened by benoitroll 4
  • Can't create field name of more than 7 characters on Mac

    Can't create field name of more than 7 characters on Mac

    When running this command I get a crash with Abort trap: 6:

    tdb make -f uuid,time,username -o bob
    

    However, if the 8 characters field is not the first, it works:

    tdb make -f uuid,time,hello,username -o bob
    

    I tried installing traildb manually or with Homebrew with the same results.

    I found this bug because the first tutorial has a field with 'username' and it made my program crash.

    Machine configuration: Macbook pro retina Processor: 2.9 GHz Intel Core i5 OS: macOS Sierra (10.12.5)

    Note that I tried on a mac which didn't run on Sierra and tdb worked fine.

    opened by ChrisJamesC 3
  • create a Dockerfile for building tdb base docker image

    create a Dockerfile for building tdb base docker image

    Dear @tuulos :

    I just separated the original Dockerfile into two Dockerfiles.

    Here is the base image. I tried to pull it and run it up ... it works. Here is the snapshots:

    image

    image

    Another Dockerfile with python-client and ipython notebook binding is here: https://github.com/traildb/traildb-python/pull/9

    opened by c3h3 3
  • Cannot use autotools build

    Cannot use autotools build

    I use the following command to build traildb with autotools

    ./autogen.sh
    ./configure
    make
    

    But have the following errors.

    tdbcli/tdbcli_tdb-main.o: In function `main':
    /home/czchen/src/github.com/traildb/traildb/tdbcli/main.c:307: undefined reference to `op_index'
    /home/czchen/src/github.com/traildb/traildb/tdbcli/main.c:309: undefined reference to `op_merge'
    tdbcli/tdbcli_tdb-op_dump.o: In function `op_dump':
    /home/czchen/src/github.com/traildb/traildb/tdbcli/op_dump.c:226: undefined reference to `apply_filter'
    /home/czchen/src/github.com/traildb/traildb/tdbcli/op_dump.c:235: undefined reference to `tdb_index_open'
    /home/czchen/src/github.com/traildb/traildb/tdbcli/op_dump.c:236: undefined reference to `tdb_index_match_candidates'
    /home/czchen/src/github.com/traildb/traildb/tdbcli/op_dump.c:247: undefined reference to `tdb_index_close'
    /home/czchen/src/github.com/traildb/traildb/tdbcli/op_dump.c:232: undefined reference to `tdb_index_find'
    

    It looks like autotools build does not update according to the latest code, please help to update it.

    opened by czchen 2
  • add missing tdb_multi_cursor.c into build

    add missing tdb_multi_cursor.c into build

    when using autoconf to build traildb, tdb_multi_cursor.c is not compiled because it is not exist in Makefile.am

    so I change Makefile.am to include tdb_multi_cursor into library.

    but ./waf doesn't have this problem.

    opened by charsyam 2
  • Travis-ci:added support for ppc64le

    Travis-ci:added support for ppc64le

    Hi, I have added support for ppc64le build on travis-ci in the branch on behalf of IBM. The travis-ci build log can be tracked on the link :https://travis-ci.com/github/sanjaymsh/traildb/builds/210305950 . I believe it is ready for the final review and merge. Please have a look on it.

    Thanks !!

    opened by sanjaymsh 2
  • docs: fix simple typo, substract -> subtract

    docs: fix simple typo, substract -> subtract

    There is a small typo in tests/c-tests/bit_operations.c.

    Should read subtract rather than substract.

    Semi-automated pull request generated by https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

    opened by timgates42 2
  • Flag to dump version

    Flag to dump version

    It'd be useful have a flag to dump out the version of traildb installed on the system (esp. useful when testing against multiple local versions)

    e.g

    tdb --version
    
    opened by aimran-adroll 0
Owner
TrailDB
TrailDB is an efficient tool for storing and querying series of events
TrailDB
GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.

Overview GridDB is Database for IoT with both NoSQL interface and SQL Interface. Please refer to GridDB Features Reference for functionality. This rep

GridDB 1.8k Aug 10, 2022
SiriDB is a highly-scalable, robust and super fast time series database

SiriDB is a highly-scalable, robust and super fast time series database. Build from the ground up SiriDB uses a unique mechanism to operate without a global index and allows server resources to be added on the fly. SiriDB's unique query language includes dynamic grouping of time series for easy analysis over large amounts of time series.

SiriDB 464 Jun 13, 2022
TimescaleDB is an open-source database designed to make SQL scalable for time-series data.

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.

Timescale 13.4k Aug 7, 2022
以简单、易用、高性能为目标、开源的时序数据库,支持Linux和Windows, Time Series Database

松果时序数据库(pinusdb) 松果时序数据库是一款针对中小规模(设备数少于10万台,每天产生的数据量少于10亿条)场景设计的时序数据库。以简单、易用、高性能为设计目标。使用SQL语句进行交互,拥有极低的学习、使用成本, 提供了丰富的功能、较高的性能。 我们的目标是成为最简单、易用、健壮的单机时序

null 95 Aug 5, 2022
High-performance time-series aggregation for PostgreSQL

PipelineDB has joined Confluent, read the blog post here. PipelineDB will not have new releases beyond 1.0.0, although critical bugs will still be fix

PipelineDB 2.5k Aug 3, 2022
🧬 Efficient parallelizable petabyte-sized database searching algorithm

Petasearch Installation Petasearch is dependent on block-aligner for fast computation of Smith-Waterman alignments in the computeAlignments module. Th

Steinegger Lab 21 Jun 13, 2022
SpDB is a data integration tool designed to organize scientific data from different sources under the same namespace according to a global schema and to provide access to them in a unified form (views)

SpDB is a data integration tool designed to organize scientific data from different sources under the same namespace according to a global schema and to provide access to them in a unified form (views). Its main purpose is to provide a unified data access interface for complex scientific computations in order to enable the interaction and integration between different programs and databases.

YU Zhi 0 Jun 22, 2022
DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite.

DB Browser for SQLite What it is DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files c

null 16.9k Aug 8, 2022
A MariaDB-based command line tool to connect to OceanBase Database.

什么是 OceanBase Client OceanBase Client(简称 OBClient) 是一个基于 MariaDB 开发的客户端工具。您可以使用 OBClient 访问 OceanBase 数据库的集群。OBClient 采用 GPL 协议。 OBClient 依赖 libobclie

OceanBase 48 Jun 29, 2022
A friendly and lightweight C++ database library for MySQL, PostgreSQL, SQLite and ODBC.

QTL QTL is a C ++ library for accessing SQL databases and currently supports MySQL, SQLite, PostgreSQL and ODBC. QTL is a lightweight library that con

null 157 Aug 4, 2022
Simple-MySQL-API is a free and easy API to manipulate MySQL with C99 and GCC compiler under GNU/Linux OS.

Simple-MySQL-API is a free and easy API to manipulate MySQL with C99 and GCC compiler under GNU/Linux OS.

Neptune 7 Apr 24, 2022
ObjectBox C and C++: super-fast database for objects and structs

ObjectBox Embedded Database for C and C++ ObjectBox is a superfast C and C++ database for embedded devices (mobile and IoT), desktop and server apps.

ObjectBox 137 Aug 7, 2022
Reading, thinking and coding about MySQL,InnoDB and MGR

MySQL 8.0.27 Review by adzfolc Notifications In this doc, I will make the rules of my code review and period summary for later review. This chapter ma

null 5 Jun 25, 2022
dqlite is a C library that implements an embeddable and replicated SQL database engine with high-availability and automatic failover

dqlite dqlite is a C library that implements an embeddable and replicated SQL database engine with high-availability and automatic failover. The acron

Canonical 3.1k Aug 10, 2022
MySQL Server, the world's most popular open source database, and MySQL Cluster, a real-time, open source transactional database.

Copyright (c) 2000, 2021, Oracle and/or its affiliates. This is a release of MySQL, an SQL database server. License information can be found in the

MySQL 8.1k Aug 6, 2022
ESE is an embedded / ISAM-based database engine, that provides rudimentary table and indexed access.

Extensible-Storage-Engine A Non-SQL Database Engine The Extensible Storage Engine (ESE) is one of those rare codebases having proven to have a more th

Microsoft 782 Jul 29, 2022
Modern, asynchronous, and wicked fast C++11 client for Redis

redox Modern, asynchronous, and wicked fast C++11 client for Redis [] (https://travis-ci.org/hmartiro/redox) Redox is a C++ interface to the Redis key

Hayk Martiros 376 Jul 27, 2022
Nebula Graph is a distributed, fast open-source graph database featuring horizontal scalability and high availability

Nebula Graph is an open-source graph database capable of hosting super large scale graphs with dozens of billions of vertices (nodes) and trillions of edges, with milliseconds of latency.

vesoft inc. 817 Aug 4, 2022
HybridSE (Hybrid SQL Engine) is an LLVM-based, hybrid-execution and high-performance SQL engine

HybridSE (Hybrid SQL Engine) is an LLVM-based, hybrid-execution and high-performance SQL engine. It can provide fast and consistent execution on heterogeneous SQL data systems, e.g., OLAD database, HTAP system, SparkSQL, and Flink Stream SQL.

4Paradigm 45 Sep 12, 2021