The Universal Storage Engine

Related tags

Scientific Computing data-science storage-engine s3 sparse-data scientific-computing s3-storage arrays hdfs data-analysis dataframes tiledb dense-data sparse-arrays
Overview

TileDB logo

Azure Pipelines Anaconda download count badge

The Universal Storage Engine

TileDB is a powerful engine for storing and accessing dense and sparse multi-dimensional arrays, which can help you model any complex data efficiently. It is an embeddable C++ library that works on Linux, macOS, and Windows. It is open-sourced under the permissive MIT License, developed and maintained by TileDB, Inc. To distinguish this project from other TileDB offerings, we often refer to it as TileDB Embedded.

TileDB includes the following features:

  • Support for both dense and sparse arrays
  • Support for dataframes and key-value stores (via sparse arrays)
  • Cloud storage (AWS S3, Google Cloud Storage, Azure Blob Storage)
  • Chunked (tiled) arrays
  • Multiple compression, encryption and checksum filters
  • Fully multi-threaded implementation
  • Parallel IO
  • Data versioning (rapid updates, time traveling)
  • Array metadata
  • Array groups
  • Numerous APIs on top of the C++ library
  • Numerous integrations (Spark, Dask, MariaDB, GDAL, etc.)

You can use TileDB to store data in a variety of applications, such as Genomics, Geospatial, Finance and more. The power of TileDB stems from the fact that any data can be modeled efficiently as either a dense or a sparse multi-dimensional array, which is the format used internally by most data science tooling. By storing your data and metadata in TileDB arrays, you abstract all the data storage and management pains, while efficiently accessing the data with your favorite data science tool.

Quickstart

You can install the TileDB library as follows:

# Homebrew (macOS):
$ brew update
$ brew install tiledb-inc/stable/tiledb

# Or Conda (macOS, Linux, Windows):
$ conda install -c conda-forge tiledb

Alterantively, you can use the Docker image we provide:

$ docker pull tiledb/tiledb
$ docker run -it tiledb/tiledb

We include several examples. You can start with the following:

Documentation

You can find the detailed TileDB documentation at https://docs.tiledb.com.

Format Specification

The TileDB data format is open-source and can be found here.

APIs

The TileDB team maintains a variety of APIs built on top of the C++ library:

Integrations

TileDB is also integrated with several popular databases and data science tools:

Get involved

TileDB Embedded is an open-source project and welcomes all forms of contributions. Contributors to the project should read over the contribution docs for more information.

We'd love to hear from you. Drop us a line at [email protected], visit our forum or contact form, or follow us on Twitter to stay informed of updates and news.

Comments
  • Build on Ubuntu 20.04 LTS

    Build on Ubuntu 20.04 LTS

    Hi,

    I just tried building TileDB on a new system with Ubuntu 20.04 LTS and for some reason I get a cmake error:

    RegularExpression::compile(): Nested *?+.
RegularExpression::compile(): Error in compile.
CMake Error at examples/cpp_api/CMakeLists.txt:52 (string):
  string sub-command REGEX, mode REPLACE failed to compile regex
  "^/home/arne/Development/c++/TileDB/examples/cpp_api/".


CMake Error at examples/cpp_api/CMakeLists.txt:56 (string):
  string sub-command REGEX, mode REPLACE needs at least 6 arguments total to
  command.


CMake Error at examples/cpp_api/CMakeLists.txt:62 (build_TileDB_example_cppapi):
  build_TileDB_example_cppapi Function invoked with incorrect arguments for
  function named: build_TileDB_example_cppapi

    I've tried building it with CMake 3.10.2 and CMake 3.18.20.

    opened by aosterthun 28
  • Compiling tiledb examples results in an import error

    Compiling tiledb examples results in an import error

    Running tiledb examples results in an error: What am I missing ?

    wsl

    error:

    (tiledbenv) roy@LP-0209:~/TileDB/examples/c_api$ gcc quickstart_dense.c
quickstart_dense.c:35:10: fatal error: tiledb/tiledb.h: No such file or directory
   35 | #include <tiledb/tiledb.h>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.

    tiledb version:

    (tiledbenv) roy@LP-0209:/mnt/c/Users/roya$ conda list | grep tiledb
# packages in environment at /home/roy/anaconda3/envs/tiledbenv:
tiledb                    2.3.3                h1132f93_2

    os version:

    (tiledbenv) roy@LP-0209:/mnt/c/Users/roya$ uname -a
Linux LP-0209 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

    gcc version:

    (tiledbenv) roy@LP-0209:~/TileDB/examples/c_api$ gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    windows 10

    error:

    (tiledbenv) PS C:\Users\roya\Downloads\TileDB\examples\c_api> gcc .\quickstart_dense.c
.\quickstart_dense.c:35:10: fatal error: tiledb/tiledb.h: No such file or directory
   35 | #include <tiledb/tiledb.h>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.

    tiledb version:

    (tiledbenv) PS C:\Users\roya\Downloads\TileDB\examples\c_api> conda list | grep tiledb

# packages in environment at C:\Users\roya\Anaconda3\envs\tiledbenv:
tiledb                    2.10.3               h5689973_0    conda-forge

    os version:

    (tiledbenv) PS C:\Users\roya\Downloads\TileDB\examples\c_api> (Get-WmiObject Win32_OperatingSystem).Caption
Microsoft Windows 10 Pro

    gcc version:

    (tiledbenv) PS C:\Users\roya\Downloads\TileDB\examples\c_api> gcc --version
gcc.exe (MinGW-W64 x86_64-posix-seh, built by Brecht Sanders) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    opened by royassis 19
  • No documentation on how to actually use the S3 backend

    No documentation on how to actually use the S3 backend

    We have no easy to use guide / docs on how to setup the S3 backend (AWS), connect to a bucket (with correct config settings), write and read back some data.

    enhancement docs 
    opened by jakebolewski 17
  • Deserialization error?

    Deserialization error?

    Hi,

    I'm using release-1.7.6 of TileDB-core, and have been receiving the following:

    Error in libtiledb_array_open(ctx@ptr, uri, query_type) : [TileDB::Filter] Error: Deserialization error; unexpected metadata length

    Creating schemas etc. seem to go ok (no errors are thrown, and the schema looks ok in a debugger), and using the created domain from within C++ seems to go ok (i.e., I can read in / write out data from TileDB into temp arrays). However, in trying to see and work with the same array from Python & R I get the above error. Both Python and R are 'latest'; I've also tried R from within your Docker instance.

    Thanks in advance for any insight. Steve

    opened by iphcsteve 12
  • Support TileDB compilation under Cygwin

    Support TileDB compilation under Cygwin

    If you want TileDB to compile under Cygwin, then all of the #if(n)def _WIN32 code lines need to be converted t #if(n)def _MSC_VER. The authors assumed that the _WIN32 flag was only set by Visual Studio, but in fact it is also set for Cygwin (a linux variant) and maybe Mingw. To ensure that the _WIN32 conditionals are invoked only when using Visual Studio, one needs to use _MSC_VER.

    build Windows cygwin 
    opened by DennisHeimbigner 11
  • support non-continuous subsetting

    support non-continuous subsetting

    This has been a major hurdle preventing me do more extensive benchmarking since this type of query would be the most frequently used in our applications.

    I know I was told the random slicing is not optimized yet. But I wonder if is even currentlysupported by the current C/C++ API? By random slicing, I simply mean [ like indexing in R. That is, instead of providing start and end for each dim, we supply an array of index for each e.g.

     query.set_subarray<int>{{1,3,5,7,9}, {2,4,6,8}};

    I know I could build my query by submitting multiple start,end queries, but I was hoping it was part of native supported API because chances are my version won't be the efficient implementation.

    enhancement C API C++ API API Addition 
    opened by mikejiang 11
  • Close magic resources

    Close magic resources

    The valgrind checker (from the the R package) noticed that some the magic_open() calls did not have corresponding magic_close() calls. This PR correct this, and passes subsequent valgrind checks.

    It also makes one minor efficiency gain as rv = magic_buffer(magic, data, size) was being called twice in libmagic_file_is_compressed. The result from the first call can be reused in the non-error case which is what we do now.

    TYPE: BUG DESC: Close magic resources

    backport release-2.10 
    opened by eddelbuettel 10
  • replace ReadFromOffset with ReadRange

    replace ReadFromOffset with ReadRange

    After switching to tiledb, we (Satelligence) had much higher (as in 50 - 100 times) egress traffic from google storage, even leading to getting a warning from google that we hit the limit/quotum for storage egress bandwidth. We've been able to work around this by limiting the amount of simultaneous read processes, but that is of course not a sustainable solution.

    After investigating, the excess traffic appeared to be caused by cancelled requests, which in turn were issued by tiledb because it is using ReadFromOffset instead of ReadRange. Where the latter translates to a closed http range request, the former translates to an open-ended http range request, effectively (starting to) read the whole blob (starting from offset). Switching to ReadRange in GCS::read fixed the issue.

    Note that the egress increase in this case might even be completely administrative. Tiledb does issue open ended range requests, but then only reads length + read_ahead_length bytes before closing the stream (and thereby cancelling the request), effectively resulting in a closed range read. Also, when looking at the instance/pod ingress instead of the bucket's egress, the excess traffic did not appear. So apparently google measures bucket egress traffic by analyzing requests, instead of probing nic's. Another possibility is that there sits some buffering appliance in between, and bucket's egress is measure before the buffer, while pod's ingress is measured after the buffer. Anyway, even if this would be purely administrative, we need a fix for it, to allow us to scale our processing again without hitting google's bandwith limit.

    The fix appeared simple: use ReadRange instead of ReadFromOffset.

    Because an image is worth more than 1000 words, here it is:

    image

    This graph shows sent bytes from bucket, running 2 times the same job, first without and then with the proposed fix. Note that the blue colored line (in the left part) is bytes from requests with status "CANCELLED", while the greenish line is from requests with the status "OK". In the right part, there is no blue line because there are no cancelled requests.

    NB this was also discussed yesterday in a chat with @stavrospapadopoulos and @normanb.

    Btw, this is my first PR to TileDB, so though I've read the contribution guidelines, I might miss some of your conventions. Like: should I add a line to HISTORY? Or is this change too small for that? Also, I'm not sure what I should put at "TYPE". Improvement? Bug?

    TYPE: IMPROVEMENT DESC: replace ReadFromOffset with ReadRange in GCS::read() to avoid excess gcs egress traffic

    opened by vincentschut 10
  • Undefined references when linking tiledb.so

    Undefined references when linking tiledb.so

    Hi, I followed the step by step on https://docs.tiledb.io/v1.2.0/docs/installation for Ubuntu.

    For installing, I do:

    ../boostrap --prefix=/usr/local 
make -j
sudo make install
sudo ldconfig

    Everything seems fine, but then when I try to compile the example for create an array:

    g++ -o create.o -c -std=c++11 -O3 create.cc
g++ -o create create.o -L/usr/local/lib -ltiledb

    I keep getting undefined references at linking time (the second step above).

    Some of the undefined references I get are:

    create.cc:(.text.startup+0x962): undefined reference to tiledb::Query::set_layout(tiledb_layout_t)' create.cc:(.text.startup+0xa9a): undefined reference totiledb::Query::submit()' create.cc:(.text.startup+0xb07): undefined reference to vtable for tiledb::ArraySchema' create.cc:(.text.startup+0xbfc): undefined reference tovtable for tiledb::ArraySchema' collect2: error: ld returned 1 exit status

    Is this a known issue? I am using the tag:1.2.0 (commit 4d1a205)

    Thanks,

    opened by luisremis 10
  • Issue with subarray writes in sorted order.

    Issue with subarray writes in sorted order.

    Hi, I am facing an issue in updating a dense subarray with ordered writes.

    • I have modified the tiledb_dense_create.cc example to create an array of dimensions 2^15 x 2^15, with one integer attribute, and one float attribute , and space tile extant of 128X128.
    • I have modified the "tiledb_dense_write_ordered_subarray.cc" to create an int buffer and a float buffer to update an subarray of capacity 2^18 elements (512 x 512 subarray).
    • When the file tiledb_dense_write_ordered_subarray is executed it crashes.
    • This crash does not happen with smaller subarray size. It also does not happen with the global order writes. I understand that tileDB performs data reorganization to convert from sorted order to global order. But is there any memory limit on the size of subarray for sorted writes?

    I have attached the modified versions of both the files. tiledb issue.zip

    Regards, Kalyan.

    opened by kavskalyan 10
  • Add cmd line arg '--vfs' to 'tiledb_unit'

    Add cmd line arg '--vfs' to 'tiledb_unit'

    Currently, the unit tests determine which VFS filesystem from the build context of the core library. For example: If the core library was built with "--enable-hdfs", unit tests will use the HDFS VFS implementation. This command line argument "--vfs" allows the user to override the VFS filesystem.

    ./tiledb_unit --vfs native will use either POSIX/Win VFS FS, even if the core was built with S3 and/or HDFS support.

    Note that the above will still run the S3 and/or HDFS specific unit tests. If the user built with HDFS but does not wish to run ANY tests with HDFS, they will need to use ./tiledb_unit ~[hdfs] --vfs native. This is required because the flag does not override the compile-time building of the FS-specific unit tests. This seems OK since this shouldn't be commonly used.

    Note that the command line argument now shows up with the -h flag.

    ./tiledb/test/tiledb_unit -h

Catch v2.2.1
usage:
  tiledb_unit [<test name|pattern|tags> ... ] options

where options are:
  ...
  --vfs <native|s3|hdfs>                    Override the VFS filesystem to
                                            use for generic tests

For more detailed usage please see the project docs
    opened by joe-maley 9
  • For Comment Only: Use lambda template to simplify

    For Comment Only: Use lambda template to simplify "copy pasta" in switch/case

    (This is a second PR for the switch/case fix, branched from dev.)

    There is way too much "copy pasta" in the switch/case statements in query_condition.cc. This PR illustrates how to use C++20 lambda templates to greatly simplify the switch/case statements. There is probably a slightly simpler syntax for invoking the lambda than the one that is there, but the approach used centralizes the function call and parameter passing that is otherwise spread throughout the switch/case. It is probably possible to use something like a variant to get rid of the switch/case altogether, but I am not sure what that would do performance relative to the slight simplification of the code.

    Template argument deduction rather than a lambda template could be used to accomplish the same simplification, so it is another approach to consider if we don't want to rely on (or wait for) C++20.

    There are a few other switch/case that could probably also be cleaned up, but they are also inside some template meta programming so I didn't attempt to shorten those.

    This simplifies some things enough that it might be more obvious how to unify legacy/sparse/dense, by picking the appropriate lambda based on legacy/dense/sparse -- but I haven't dived into that. (I leave it as an exercise to the reader.)

    Several of the sets of "Cmp" functions can also be condensed. That will be slightly more involved, but would not require C++20.

    Follow Up: I measured the object file size before and after to make sure the use of lambda templates didn't increase the object file size. There was no appreciable change.

    dev:

    query_condition.cc.o: 1253096 unit_query_condition.cc.o: 1019552 lums/tmp/query_condition:

    query_condition.cc.o: 1255328 unit_query_condition.cc.o: 1019552

    TYPE: IMPROVEMENT DESC: Simplify switch/case with C++20 lambda templates.

    opened by lums658 0
  • Add UTF-8 attribute support for QueryConditions

    Add UTF-8 attribute support for QueryConditions

    Add basic UTF-8 support for QueryConditions on attributes.

    This adds a basic binary comparison of UTF-8 data. This means that QueryConditions can work with UTF-8 attribute data while it does not allow for Unicode aware comparisons. A specific description of the algorithm would be:

    UTF-8 string data is comparisons are performed on the underlying Unicode code points without regard for any Unicode aware semantics.

    An alternative way of stating that is that we're comparing UTF-8 data with memcmp and completely disregard the Unicode properties of the data. Future work will be required to support proper Unicode aware comparisons.

    TYPE: FEATURE DESC: Add UTF-8 support for QueryConditions on attributes

    opened by davisp 0
  • Refactor filestore API so that it uses internal TileDB library calls.

    Refactor filestore API so that it uses internal TileDB library calls.

    I am currently refactoring tiledb/c_api/tiledb_filestore.cc so that it uses only internal API calls, as opposed to the C++ external API calls.

    TYPE: IMPROVEMENT DESC: Refactor filestore API so that it uses internal TileDB library calls.

    opened by abigalekim 1
  • Remote global order write cache

    Remote global order write cache

    Still a WIP, just sharing where I'm at currently and testing my branch against CI. I've made a lot of progress here but running into the following issues

    • Cached offset buffers don't respect absolute global position. If we aggregate two submissions into one offsets will not be provided in ascending order. I plan to correct offsets in subsequent submissions in a future commit to respect their global order position.
    • Sparse arrays with multiple submissions to REST fail with Coordinate (655361) comes before last written coordinate or similar. I haven't quite sorted this one out yet, I'll have to debug the REST server with gdb some more to see where the problem is.
    • Large submissions (1 GB array) takes a long time to complete. I've tried running my test for a 1 GB array using this branch and dev and it takes ~15 minutes to complete in both cases. I created SC-24334 to track this.

    TYPE: FEATURE DESC: Removes tile-aligned restriction for remote global order writes by trimming and caching submissions.

    opened by shaunrd0 1
  • Remove C API functions `tiledb_array_open_at` etc.

    Remove C API functions `tiledb_array_open_at` etc.

    Remove tiledb_array_open_at, tiledb_array_reopen_at, tiledb_array_open_with_key, tiledb_array_get_timestamp Remove corresponding deprecated C++ API functions Remove implementation of tiledb_fragment_info_load_with_key, whose visibility was removed in #3740

    TYPE: C_API DESC: Remove C API functions tiledb_array_open_at etc.

    opened by eric-hughes-tiledb 0
Releases(2.13.0)

  • 2.13.0(Nov 30, 2022)

    TileDB v2.13.0 Release Notes

    Announcements

    Disk Format

    Documentation

    • Document dictionary encoding format #3566
    • Improve documentation of consolidated commits files and ignore files #3606

    New features

    • Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
    • Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

    API Changes

    C API

    • Added tiledb_status_code #3580
    • Added tiledb_group_get_is_relative_uri_by_name #3550
    • Added tiledb_array_delete_array API #3539

    Documentation

    • Document missing fragment info C APIs. #3577
    • Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

    Improvements

    Defects removed

    • Sparse global order reader: fix tile cleanup when ending an iteration. 3674 https://github.com/TileDB-Inc/TileDB/pull/3674) (https://github.com/TileDB-Inc/TileDB/pull/3678)
    • Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
    • Dense consolidation: set correct non-empty domain #3635
    • Sparse global order reader: fixing incomplete reason for rest queries #3620
    • Fixes dim label reader range order to always take valid ranges #3561
    • Global order writes should send relative uris over serialization #3557
    • Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
    • Fix duplicate logger instantiations in global state #3591
    • Add missing query_type in array_open capnp #3616
    • Fix use-after-free on a capnp::FlatArrayMessageReader #3631
    • Bitsort filter: preallocate filtered buffer for dim tiles. #3632
    • Add Boolean Support For Sparse Query Conditions #3651

    Internal

    • Read tiles: refactor tile creation code. #3492
    • Use snprintf to avoid deprecation warnings as errors #3608
    • Removes non-C.41 range setting (part 1) #3598
    • Add [[nodiscard]] attribute to Status class and fix failures #3559
    • Remove WhiteboxTile. #3612
    • Global order writer: allow splitting fragments. #3605
    • Move unit uuid to unit tests. #3614
    • Consolidation: add fragment max size. #3607
    • Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
    • Adding rtree object library. #3613
    • Global order writes serialization: no tests when serialization disabled. #3603
    • Fragment info serialization support #3530
    • Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
    • Class VFS C.41 compliance, Part 1 #3477
    • Updates: adding strategy. #3513
    • Remove stale declarations from query #3565
    • Improvements in C API implementation #3524
    • Sparse readers: adding relevant cells stats. #3593
    • Fix find_heap_api_violations.py #3634

    Build system changes

    • Update to catch2 ver3 #3504
    • Linux release artifacts now target manylinux2014 #3656

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.13.0-db00e70.tar.gz(9.05 MB)
    tiledb-linux-x86_64-noavx2-2.13.0-db00e70.tar.gz(8.89 MB)
    tiledb-macos-arm64-2.13.0-db00e70.tar.gz(6.13 MB)
    tiledb-macos-x86_64-2.13.0-db00e70.tar.gz(7.58 MB)
    tiledb-windows-x86_64-2.13.0-db00e70.zip(32.44 MB)

  • 2.13-rc3(Nov 29, 2022)

    TileDB v2.13.0 Release Notes

    Announcements

    Disk Format

    Documentation

    • Document dictionary encoding format #3566
    • Improve documentation of consolidated commits files and ignore files #3606

    New features

    • Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
    • Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

    API Changes

    C API

    • Added tiledb_status_code #3580
    • Added tiledb_group_get_is_relative_uri_by_name #3550
    • Added tiledb_array_delete_array API #3539

    Documentation

    • Document missing fragment info C APIs. #3577
    • Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

    Improvements

    Defects removed

    • Sparse global order reader: fix tile cleanup when ending an iteration. 3674 https://github.com/TileDB-Inc/TileDB/pull/3674) (https://github.com/TileDB-Inc/TileDB/pull/3678)
    • Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
    • Dense consolidation: set correct non-empty domain #3635
    • Sparse global order reader: fixing incomplete reason for rest queries #3620
    • Fixes dim label reader range order to always take valid ranges #3561
    • Global order writes should send relative uris over serialization #3557
    • Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
    • Fix duplicate logger instantiations in global state #3591
    • Add missing query_type in array_open capnp #3616
    • Fix use-after-free on a capnp::FlatArrayMessageReader #3631
    • Bitsort filter: preallocate filtered buffer for dim tiles. #3632
    • Add Boolean Support For Sparse Query Conditions #3651

    Internal

    • Read tiles: refactor tile creation code. #3492
    • Use snprintf to avoid deprecation warnings as errors #3608
    • Removes non-C.41 range setting (part 1) #3598
    • Add [[nodiscard]] attribute to Status class and fix failures #3559
    • Remove WhiteboxTile. #3612
    • Global order writer: allow splitting fragments. #3605
    • Move unit uuid to unit tests. #3614
    • Consolidation: add fragment max size. #3607
    • Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
    • Adding rtree object library. #3613
    • Global order writes serialization: no tests when serialization disabled. #3603
    • Fragment info serialization support #3530
    • Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
    • Class VFS C.41 compliance, Part 1 #3477
    • Updates: adding strategy. #3513
    • Remove stale declarations from query #3565
    • Improvements in C API implementation #3524
    • Sparse readers: adding relevant cells stats. #3593
    • Fix find_heap_api_violations.py #3634

    Build system changes

    • Update to catch2 ver3 #3504
    • Linux release artifacts now target manylinux2014 #3656

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.13-rc3-db00e70.tar.gz(9.05 MB)
    tiledb-linux-x86_64-noavx2-2.13-rc3-db00e70.tar.gz(8.89 MB)
    tiledb-macos-arm64-2.13-rc3-db00e70.tar.gz(6.13 MB)
    tiledb-macos-x86_64-2.13-rc3-db00e70.tar.gz(7.58 MB)
    tiledb-windows-x86_64-2.13-rc3-db00e70.zip(32.43 MB)

  • 2.12.3(Nov 22, 2022)

  • 2.13.0-rc2(Nov 11, 2022)

    TileDB v2.13.0 Release Notes

    Disk Format

    Documentation

    • Document dictionary encoding format #3566
    • Improve documentation of consolidated commits files and ignore files #3606

    New features

    • Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
    • Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

    API Changes

    C API

    • Added tiledb_status_code #3580
    • Added tiledb_group_get_is_relative_uri_by_name #3550
    • Added tiledb_array_delete_array API #3539

    Documentation

    • Document missing fragment info C APIs. #3577
    • Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

    Improvements

    Defects removed

    • Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
    • Dense consolidation: set correct non-empty domain #3635
    • Sparse global order reader: fixing incomplete reason for rest queries #3620
    • Fixes dim label reader range order to always take valid ranges #3561
    • Global order writes should send relative uris over serialization #3557
    • Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
    • Fix duplicate logger instantiations in global state #3591
    • Add missing query_type in array_open capnp #3616
    • Fix use-after-free on a capnp::FlatArrayMessageReader #3631
    • Bitsort filter: preallocate filtered buffer for dim tiles. #3632
    • Add Boolean Support For Sparse Query Conditions #3651

    Internal

    • Read tiles: refactor tile creation code. #3492
    • Use snprintf to avoid deprecation warnings as errors #3608
    • Removes non-C.41 range setting (part 1) #3598
    • Add [[nodiscard]] attribute to Status class and fix failures #3559
    • Remove WhiteboxTile. #3612
    • Global order writer: allow splitting fragments. #3605
    • Move unit uuid to unit tests. #3614
    • Consolidation: add fragment max size. #3607
    • Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
    • Adding rtree object library. #3613
    • Global order writes serialization: no tests when serialization disabled. #3603
    • Fragment info serialization support #3530
    • Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
    • Class VFS C.41 compliance, Part 1 #3477
    • Updates: adding strategy. #3513
    • Remove stale declarations from query #3565
    • Improvements in C API implementation #3524
    • Sparse readers: adding relevant cells stats. #3593
    • Fix find_heap_api_violations.py #3634

    Build system changes

    • Update to catch2 ver3 #3504
    • Linux release artifacts now target manylinux2014 #3656

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.13.0-rc2-0b2d27a.tar.gz(9.05 MB)
    tiledb-linux-x86_64-noavx2-2.13.0-rc2-0b2d27a.tar.gz(8.89 MB)
    tiledb-macos-arm64-2.13.0-rc2-0b2d27a.tar.gz(6.13 MB)
    tiledb-macos-x86_64-2.13.0-rc2-0b2d27a.tar.gz(7.58 MB)
    tiledb-windows-x86_64-2.13.0-rc2-0b2d27a.zip(32.43 MB)

  • 2.13.0-rc1(Nov 10, 2022)

    TileDB v2.13.0 Release Notes

    Disk Format

    Documentation

    • Document dictionary encoding format #3566
    • Improve documentation of consolidated commits files and ignore files #3606

    New features

    • Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
    • Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

    API Changes

    C API

    • Added tiledb_status_code #3580
    • Added tiledb_group_get_is_relative_uri_by_name #3550
    • Added tiledb_array_delete_array API #3539

    Documentation

    • Document missing fragment info C APIs. #3577
    • Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

    Improvements

    Defects removed

    • Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
    • Dense consolidation: set correct non-empty domain #3635
    • Sparse global order reader: fixing incomplete reason for rest queries #3620
    • Fixes dim label reader range order to always take valid ranges #3561
    • Global order writes should send relative uris over serialization #3557
    • Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
    • Fix duplicate logger instantiations in global state #3591
    • Add missing query_type in array_open capnp #3616
    • Fix use-after-free on a capnp::FlatArrayMessageReader #3631
    • Bitsort filter: preallocate filtered buffer for dim tiles. #3632

    Internal

    • Read tiles: refactor tile creation code. #3492
    • Use snprintf to avoid deprecation warnings as errors #3608
    • Removes non-C.41 range setting (part 1) #3598
    • Add [[nodiscard]] attribute to Status class and fix failures #3559
    • Remove WhiteboxTile. #3612
    • Global order writer: allow splitting fragments. #3605
    • Move unit uuid to unit tests. #3614
    • Consolidation: add fragment max size. #3607
    • Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
    • Adding rtree object library. #3613
    • Global order writes serialization: no tests when serialization disabled. #3603
    • Fragment info serialization support #3530
    • Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
    • Class VFS C.41 compliance, Part 1 #3477
    • Updates: adding strategy. #3513
    • Remove stale declarations from query #3565
    • Improvements in C API implementation #3524
    • Sparse readers: adding relevant cells stats. #3593
    • Fix find_heap_api_violations.py #3634

    Build system changes

    • Update to catch2 ver3 #3504

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-macos-arm64-2.13.0-rc1-390b6e2.tar.gz(6.13 MB)
    tiledb-macos-x86_64-2.13.0-rc1-390b6e2.tar.gz(7.58 MB)
    tiledb-windows-x86_64-2.13.0-rc1-390b6e2.zip(32.43 MB)

  • 2.12.2(Nov 4, 2022)

  • 2.12.1(Nov 2, 2022)

  • 2.12.0(Oct 21, 2022)

    TileDB v2.12.0 Release Notes

    Disk Format

    New features

    • Support for DELETE query type, providing the capability to non-destructively (until consolidation with purge) delete data from array from query timestamp forward

      DELETE feature pull-requests
      • Deletes: legacy reader process deletes. #3387
      • Deletes: implement delete strategy. #3337
      • Deletes: refactored readers process deletes. #3374
      • Deletes: adding support for commits consolidation. #3378
      • Dense reader: adding num tiles to stats. #3434
      • Opt-in core-to-REST-server instrumentation #3432
      • Deletes: implement consolidation. #3402
      • Deletes: adding examples. #3437
      • Deletes: implement serialization. #3450
      • Deletes consolidation: switch from marker hashes to condition indexes. #3451
      • Deletes: adding purge option for consolidation. #3458
      • Deletes: disallow in middle of consolidated fragment with no timestamps. #3470

    • Implement delete_fragments API for removing all fragments within a specified time range #3400

    • Implement XOR Filter #3383

    • Fragment info serialization support for tiledb:// URIs #3530

    • Add support for global order writes to tiledb:// URIs #3393

    API Changes

    Config parameters

    • Add new config rest.curl.buffersize for setting CURLOPT_BUFFERSIZE. #3440

    C API

    • Add tiledb_group_get_is_relative_uri_by_name #3550
    • Adding experimental API for getting relevant fragments, tiledb_query_get_relevant_fragment_num. #3413
    • Add tiledb_query_get_relevant_fragment_num for experimental API to get relevant fragments. #3413
    • Add include policy for non-TileDB headers in the C API #3414

    Improvements

    Performance

    • Sparse global order reader: merge algorithm optimization. #3331
    • Azure: parallelize remove_dir. #3357
    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394
    • Reduce the number of requests in dir_size #3382
    • VFS: Adding option to disable batching for read_tiles. #3421
    • Avoid duplicate string_view creation in compute_results_count_sparse_string_range #3491
    • Memory tracker: using the correct type for setting default budget. #3509
    • Sparse global order reader: compute hilbert vals before filtering tiles. #3497
    • Avoid string copy in Dictionary Encoding decompression. #3490

    Defects removed

    • Add check that Dict/RLE for strings is the first filter in the pipeline; allow use w/ other filters #3510
    • Remove stale declarations from query #3565
    • Fix SC-19287: segfault due to deref nonexistent filestore key #3359
    • Demonstrating mingw handle leakage (tiledb_unit extract) #3362
    • Fix and regression test for SC-19240 #3360
    • Don't try to consolidate empty array; fixes SC-19516 #3389
    • Fixes check for experimental schema features to be current version #3404
    • Prevent possible compiler dependent errors serializing groups #3399
    • use stoul() to correctly parse (32bit unsigned values) experimental version numbers cross-platform #3410
    • Fix deserialize to set array_schema_all_ into array object #3363
    • #3430 #3431
    • Removes unneeded fabs causing warning on clang #3484
    • Dictionary encoding should handle zero length strings #3493
    • Fix empty metadata after array open/query submit #3495
    • Handle filter_from_capnp FilterType::NONE case #3516
    • Sparse global order reader: incomplete reads when hitting memory limits. #3518
    • Fixes silent failure for mismatched layout and bad layout/array combos on query #3521
    • Correct defect in source of keying material #3529
    • Rework delete_fragments API #3505
    • Fix segfault after schema evolution when reading using TILEDB_UNORDERED #3528
    • Fix SC-21741, array evolve via REST #3532
    • Don't fetch Array data in Controller until the Array is fully open #3538
    • Do not allow creation of sparse array with zero capacity. #3546
    • Tile metadata: fixing for ordered writes. #3527

    Internal

    • Array consistency controller #3130
    • Experimental build format versioning #3364
    • Implementation of DataBlock, DataBlock allocator, join view, and updates to Source and Sink. #3366
    • Implemented basic platform library #3420
    • Adds Edge and simple Node classes to the TileDB task graph library. #3453
    • Adds attribute ranges to Subarray for internal usage #3520
    • Add list of point ranges to Subarray #3502
    • Add support for new array open REST call #3339
    • Added Config::must_find marker for use with new Config::get signature. Throws Status_ConfigError if value cannot be found. #3482

    Build system changes

    • Add abseil/absl to build via ExternalProject_Add #3454
    • Add Crc32c to tiledb build via ExternalProject_Add #3455
    • Enable superbuild libcurl to support zstd #3469
    • Adjust example dockerfile so layers can be cached better #3488

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.11.0...2.12.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.12.0-ac8a0df.tar.gz(8.05 MB)
    tiledb-linux-x86_64-noavx2-2.12.0-ac8a0df.tar.gz(7.99 MB)
    tiledb-macos-arm64-2.12.0-ac8a0df.tar.gz(5.81 MB)
    tiledb-macos-x86_64-2.12.0-ac8a0df.tar.gz(7.21 MB)
    tiledb-windows-x86_64-2.12.0-ac8a0df.zip(30.76 MB)

  • 2.12.0-rc2(Oct 20, 2022)

    TileDB v2.12.0 Release Notes

    Disk Format

    New features

    • Support for DELETE query type, providing the capability to non-destructively (until consolidation with purge) delete data from array from query timestamp forward

      DELETE feature pull-requests
      • Deletes: legacy reader process deletes. #3387
      • Deletes: implement delete strategy. #3337
      • Deletes: refactored readers process deletes. #3374
      • Deletes: adding support for commits consolidation. #3378
      • Dense reader: adding num tiles to stats. #3434
      • Opt-in core-to-REST-server instrumentation #3432
      • Deletes: implement consolidation. #3402
      • Deletes: adding examples. #3437
      • Deletes: implement serialization. #3450
      • Deletes consolidation: switch from marker hashes to condition indexes. #3451
      • Deletes: adding purge option for consolidation. #3458
      • Deletes: disallow in middle of consolidated fragment with no timestamps. #3470

    • Implement delete_fragments API for removing all fragments within a specified time range #3400

    • Implement XOR Filter #3383

    • Fragment info serialization support for tiledb:// URIs #3530

    • Add support for global order writes to tiledb:// URIs #3393

    API Changes

    Config parameters

    • Add new config rest.curl.buffersize for setting CURLOPT_BUFFERSIZE. #3440

    C API

    • Add tiledb_group_get_is_relative_uri_by_name #3550
    • Adding experimental API for getting relevant fragments, tiledb_query_get_relevant_fragment_num. #3413
    • Add tiledb_query_get_relevant_fragment_num for experimental API to get relevant fragments. #3413
    • Add include policy for non-TileDB headers in the C API #3414

    Improvements

    Performance

    • Sparse global order reader: merge algorithm optimization. #3331
    • Azure: parallelize remove_dir. #3357
    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394
    • Reduce the number of requests in dir_size #3382
    • VFS: Adding option to disable batching for read_tiles. #3421
    • Avoid duplicate string_view creation in compute_results_count_sparse_string_range #3491
    • Memory tracker: using the correct type for setting default budget. #3509
    • Sparse global order reader: compute hilbert vals before filtering tiles. #3497
    • Avoid string copy in Dictionary Encoding decompression. #3490

    Defects removed

    • Add check that Dict/RLE for strings is the first filter in the pipeline; allow use w/ other filters #3510
    • Remove stale declarations from query #3565
    • Fix SC-19287: segfault due to deref nonexistent filestore key #3359
    • Demonstrating mingw handle leakage (tiledb_unit extract) #3362
    • Fix and regression test for SC-19240 #3360
    • Don't try to consolidate empty array; fixes SC-19516 #3389
    • Fixes check for experimental schema features to be current version #3404
    • Prevent possible compiler dependent errors serializing groups #3399
    • use stoul() to correctly parse (32bit unsigned values) experimental version numbers cross-platform #3410
    • Fix deserialize to set array_schema_all_ into array object #3363
    • #3430 #3431
    • Removes unneeded fabs causing warning on clang #3484
    • Dictionary encoding should handle zero length strings #3493
    • Fix empty metadata after array open/query submit #3495
    • Handle filter_from_capnp FilterType::NONE case #3516
    • Sparse global order reader: incomplete reads when hitting memory limits. #3518
    • Fixes silent failure for mismatched layout and bad layout/array combos on query #3521
    • Correct defect in source of keying material #3529
    • Rework delete_fragments API #3505
    • Fix segfault after schema evolution when reading using TILEDB_UNORDERED #3528
    • Fix SC-21741, array evolve via REST #3532
    • Don't fetch Array data in Controller until the Array is fully open #3538
    • Do not allow creation of sparse array with zero capacity. #3546
    • Tile metadata: fixing for ordered writes. #3527

    Internal

    • Array consistency controller #3130
    • Experimental build format versioning #3364
    • Implementation of DataBlock, DataBlock allocator, join view, and updates to Source and Sink. #3366
    • Implemented basic platform library #3420
    • Adds Edge and simple Node classes to the TileDB task graph library. #3453
    • Adds attribute ranges to Subarray for internal usage #3520
    • Add list of point ranges to Subarray #3502
    • Add support for new array open REST call #3339
    • Added Config::must_find marker for use with new Config::get signature. Throws Status_ConfigError if value cannot be found. #3482

    Build system changes

    • Add abseil/absl to build via ExternalProject_Add #3454
    • Add Crc32c to tiledb build via ExternalProject_Add #3455
    • Enable superbuild libcurl to support zstd #3469
    • Adjust example dockerfile so layers can be cached better #3488

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.11.0...2.12.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.12.0-rc2-ac8a0df.tar.gz(8.05 MB)
    tiledb-linux-x86_64-noavx2-2.12.0-rc2-ac8a0df.tar.gz(7.99 MB)
    tiledb-macos-arm64-2.12.0-rc2-ac8a0df.tar.gz(5.81 MB)
    tiledb-macos-x86_64-2.12.0-rc2-ac8a0df.tar.gz(7.21 MB)
    tiledb-windows-x86_64-2.12.0-rc2-ac8a0df.zip(30.77 MB)

  • 2.12.0-rc1(Oct 16, 2022)

    TileDB v2.12.0-rc1 Release Notes

    Disk Format

    New features

    • Support for DELETE query type, providing the capability to non-destructively (until consolidation with purge) delete data from array from query timestamp forward

      DELETE feature pull-requests
      • Deletes: legacy reader process deletes. #3387
      • Deletes: implement delete strategy. #3337
      • Deletes: refactored readers process deletes. #3374
      • Deletes: adding support for commits consolidation. #3378
      • Dense reader: adding num tiles to stats. #3434
      • Opt-in core-to-REST-server instrumentation #3432
      • Deletes: implement consolidation. #3402
      • Deletes: adding examples. #3437
      • Deletes: implement serialization. #3450
      • Deletes consolidation: switch from marker hashes to condition indexes. #3451
      • Deletes: adding purge option for consolidation. #3458
      • Deletes: disallow in middle of consolidated fragment with no timestamps. #3470

    • Implement delete_fragments API for removing all fragments within a specified time range #3400

    • Implement XOR Filter #3383

    • Fragment info serialization support for tiledb:// URIs #3530

    • Add support for global order writes to tiledb:// URIs #3393

    API Changes

    Config parameters

    • Add new config rest.curl.buffersize for setting CURLOPT_BUFFERSIZE. #3440

    C API

    • Add tiledb_group_get_is_relative_uri_by_name #3550
    • Adding experimental API for getting relevant fragments, tiledb_query_get_relevant_fragment_num. #3413
    • Add tiledb_query_get_relevant_fragment_num for experimental API to get relevant fragments. #3413
    • Add include policy for non-TileDB headers in the C API #3414

    Improvements

    Performance

    • Sparse global order reader: merge algorithm optimization. #3331
    • Azure: parallelize remove_dir. #3357
    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394
    • Reduce the number of requests in dir_size #3382
    • VFS: Adding option to disable batching for read_tiles. #3421
    • Avoid duplicate string_view creation in compute_results_count_sparse_string_range #3491
    • Memory tracker: using the correct type for setting default budget. #3509
    • Sparse global order reader: compute hilbert vals before filtering tiles. #3497
    • Avoid string copy in Dictionary Encoding decompression. #3490

    Defects removed

    • Add check that Dict/RLE for strings is the first filter in the pipeline; allow use w/ other filters #3510
    • Remove stale declarations from query #3565
    • Fix SC-19287: segfault due to deref nonexistent filestore key #3359
    • Demonstrating mingw handle leakage (tiledb_unit extract) #3362
    • Fix and regression test for SC-19240 #3360
    • Don't try to consolidate empty array; fixes SC-19516 #3389
    • Fixes check for experimental schema features to be current version #3404
    • Prevent possible compiler dependent errors serializing groups #3399
    • use stoul() to correctly parse (32bit unsigned values) experimental version numbers cross-platform #3410
    • Fix deserialize to set array_schema_all_ into array object #3363
    • #3430 #3431
    • Removes unneeded fabs causing warning on clang #3484
    • Dictionary encoding should handle zero length strings #3493
    • Fix empty metadata after array open/query submit #3495
    • Handle filter_from_capnp FilterType::NONE case #3516
    • Sparse global order reader: incomplete reads when hitting memory limits. #3518
    • Fixes silent failure for mismatched layout and bad layout/array combos on query #3521
    • Correct defect in source of keying material #3529
    • Rework delete_fragments API #3505
    • Fix segfault after schema evolution when reading using TILEDB_UNORDERED #3528
    • Fix SC-21741, array evolve via REST #3532
    • Don't fetch Array data in Controller until the Array is fully open #3538
    • Do not allow creation of sparse array with zero capacity. #3546
    • Tile metadata: fixing for ordered writes. #3527

    Internal

    • Array consistency controller #3130
    • Experimental build format versioning #3364
    • Implementation of DataBlock, DataBlock allocator, join view, and updates to Source and Sink. #3366
    • Implemented basic platform library #3420
    • Adds Edge and simple Node classes to the TileDB task graph library. #3453
    • Adds attribute ranges to Subarray for internal usage #3520
    • Add list of point ranges to Subarray #3502
    • Add support for new array open REST call #3339
    • Added Config::must_find marker for use with new Config::get signature. Throws Status_ConfigError if value cannot be found. #3482

    Build system changes

    • Add abseil/absl to build via ExternalProject_Add #3454
    • Add Crc32c to tiledb build via ExternalProject_Add #3455
    • Enable superbuild libcurl to support zstd #3469
    • Adjust example dockerfile so layers can be cached better #3488

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.11.0...2.12.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.12.0-rc1-c8b41a7.tar.gz(8.05 MB)
    tiledb-linux-x86_64-noavx2-2.12.0-rc1-c8b41a7.tar.gz(7.99 MB)
    tiledb-macos-arm64-2.12.0-rc1-c8b41a7.tar.gz(5.81 MB)
    tiledb-macos-x86_64-2.12.0-rc1-c8b41a7.tar.gz(7.21 MB)
    tiledb-windows-x86_64-2.12.0-rc1-c8b41a7.zip(30.75 MB)

  • 2.12.0-rc0(Oct 14, 2022)

    TileDB v2.12.0 Release Notes

    Disk Format

    New features

    • Support for DELETE query type, providing the capability to non-destructively (until consolidation with purge) delete data from array from query timestamp forward

      DELETE feature pull-requests
      • Deletes: legacy reader process deletes. #3387
      • Deletes: implement delete strategy. #3337
      • Deletes: refactored readers process deletes. #3374
      • Deletes: adding support for commits consolidation. #3378
      • Dense reader: adding num tiles to stats. #3434
      • Opt-in core-to-REST-server instrumentation #3432
      • Deletes: implement consolidation. #3402
      • Deletes: adding examples. #3437
      • Deletes: implement serialization. #3450
      • Deletes consolidation: switch from marker hashes to condition indexes. #3451
      • Deletes: adding purge option for consolidation. #3458
      • Deletes: disallow in middle of consolidated fragment with no timestamps. #3470

    • Implement delete_fragments API for removing all fragments within a specified time range #3400

    • Implement XOR Filter #3383

    • Fragment info serialization support for tiledb:// URIs #3530

    • Add support for global order writes to tiledb:// URIs #3393

    API Changes

    Config parameters

    • Add new config rest.curl.buffersize for setting CURLOPT_BUFFERSIZE. #3440

    C API

    • Add tiledb_group_get_is_relative_uri_by_name #3550
    • Adding experimental API for getting relevant fragments, tiledb_query_get_relevant_fragment_num. #3413
    • Add tiledb_query_get_relevant_fragment_num for experimental API to get relevant fragments. #3413
    • Add include policy for non-TileDB headers in the C API #3414

    Improvements

    Performance

    • Sparse global order reader: merge algorithm optimization. #3331
    • Azure: parallelize remove_dir. #3357
    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394
    • Reduce the number of requests in dir_size #3382
    • VFS: Adding option to disable batching for read_tiles. #3421
    • Avoid duplicate string_view creation in compute_results_count_sparse_string_range #3491
    • Memory tracker: using the correct type for setting default budget. #3509
    • Sparse global order reader: compute hilbert vals before filtering tiles. #3497
    • Avoid string copy in Dictionary Encoding decompression. #3490

    Defects removed

    • Add check that Dict/RLE for strings is the first filter in the pipeline; allow use w/ other filters #3510
    • Remove stale declarations from query #3565
    • Fix SC-19287: segfault due to deref nonexistent filestore key #3359
    • Demonstrating mingw handle leakage (tiledb_unit extract) #3362
    • Fix and regression test for SC-19240 #3360
    • Don't try to consolidate empty array; fixes SC-19516 #3389
    • Fixes check for experimental schema features to be current version #3404
    • Prevent possible compiler dependent errors serializing groups #3399
    • use stoul() to correctly parse (32bit unsigned values) experimental version numbers cross-platform #3410
    • Fix deserialize to set array_schema_all_ into array object #3363
    • #3430 #3431
    • Removes unneeded fabs causing warning on clang #3484
    • Dictionary encoding should handle zero length strings #3493
    • Fix empty metadata after array open/query submit #3495
    • Handle filter_from_capnp FilterType::NONE case #3516
    • Sparse global order reader: incomplete reads when hitting memory limits. #3518
    • Fixes silent failure for mismatched layout and bad layout/array combos on query #3521
    • Correct defect in source of keying material #3529
    • Rework delete_fragments API #3505
    • Fix segfault after schema evolution when reading using TILEDB_UNORDERED #3528
    • Fix SC-21741, array evolve via REST #3532
    • Don't fetch Array data in Controller until the Array is fully open #3538
    • Do not allow creation of sparse array with zero capacity. #3546
    • Tile metadata: fixing for ordered writes. #3527

    Internal

    • Array consistency controller #3130
    • Experimental build format versioning #3364
    • Implementation of DataBlock, DataBlock allocator, join view, and updates to Source and Sink. #3366
    • Implemented basic platform library #3420
    • Adds Edge and simple Node classes to the TileDB task graph library. #3453
    • Adds attribute ranges to Subarray for internal usage #3520
    • Add list of point ranges to Subarray #3502
    • Add support for new array open REST call #3339
    • Added Config::must_find marker for use with new Config::get signature. Throws Status_ConfigError if value cannot be found. #3482

    Build system changes

    • Add abseil/absl to build via ExternalProject_Add #3454
    • Add Crc32c to tiledb build via ExternalProject_Add #3455
    • Enable superbuild libcurl to support zstd #3469
    • Adjust example dockerfile so layers can be cached better #3488

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.11.0...2.12.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.12.0-rc0-97b7d23.tar.gz(8.05 MB)
    tiledb-linux-x86_64-noavx2-2.12.0-rc0-97b7d23.tar.gz(7.99 MB)
    tiledb-macos-arm64-2.12.0-rc0-97b7d23.tar.gz(5.81 MB)
    tiledb-macos-x86_64-2.12.0-rc0-97b7d23.tar.gz(7.21 MB)
    tiledb-windows-x86_64-2.12.0-rc0-97b7d23.zip(30.75 MB)

  • 2.11.3(Sep 23, 2022)

    TileDB v2.11.3 Release Notes

    Improvements

    • Backport changes from #3326 to add experimental docs to stable #3526

    Defects removed

    • Memory tracker: using the correct type for setting default budget. #3509
    • Sparse global order reader: incomplete reads when hitting memory limits. #3518
    • Fix segfault after schema evolution when reading using TILEDB_UNORDERED #3528
    • Sparse GO reader: issue when no tile progress because user buffers full. #3531
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.11.3-a55a910.tar.gz(7.81 MB)
    tiledb-linux-x86_64-noavx2-2.11.3-a55a910.tar.gz(7.76 MB)
    tiledb-macos-arm64-2.11.3-a55a910.tar.gz(5.58 MB)
    tiledb-macos-x86_64-2.11.3-a55a910.tar.gz(7.00 MB)
    tiledb-windows-x86_64-2.11.3-a55a910.zip(29.89 MB)

  • 2.11.2(Sep 12, 2022)

    TileDB v2.11.2 Release Notes

    Improvements

    Build

    • Adjust example dockerfile so layers can be cached better #3488

    Defects removed

    • Dictionary encoding should handle zero length strings #3493
    • Fix empty metadata after array open/query submit #3495
    • Sparse global order reader: compute hilbert vals before filtering tiles. #3497
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.11.2-6ad6f76.tar.gz(7.81 MB)
    tiledb-linux-x86_64-noavx2-2.11.2-6ad6f76.tar.gz(7.76 MB)
    tiledb-macos-arm64-2.11.2-6ad6f76.tar.gz(5.57 MB)
    tiledb-macos-x86_64-2.11.2-6ad6f76.tar.gz(6.99 MB)
    tiledb-windows-x86_64-2.11.2-6ad6f76.zip(29.89 MB)

  • 2.11.1(Aug 25, 2022)

    TileDB v2.11.1 Release Notes

    Improvements

    Performance

    • Add support for new array open REST call #3339
    • Fix reader serialization for old clients. #3446
    • Sparse global order reader: merge algorithm optimization. #3331

    Internal

    • Dense reader: adding num tiles to stats. #3434
    • Opt-in core-to-REST-server instrumentation #3432
    • Serialization: improvements around query serialization/deserialization. #3379

    Defects removed

    • #3430 #3431
    • Fix deserialize to set array_schema_all_ into array object #3363
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.11.1-15a1161.tar.gz(7.81 MB)
    tiledb-linux-x86_64-noavx2-2.11.1-15a1161.tar.gz(7.76 MB)
    tiledb-macos-arm64-2.11.1-15a1161.tar.gz(5.58 MB)
    tiledb-macos-x86_64-2.11.1-15a1161.tar.gz(6.99 MB)
    tiledb-windows-x86_64-2.11.1-15a1161.zip(29.89 MB)

  • 2.11.0(Aug 5, 2022)

    TileDB v2.11.0 Release Notes

    Disk Format

    • Bump format version and remove config option for consolidation with timestamps #3267

    Breaking C API changes

    • tiledb_filter_alloc no longer returns TILEDB_OK when passed a nullptr as the third argument. #3222

    Breaking behavior

    • Enforce version upper bound for reads, #3248. TileDB will no longer attempt to read an array written with a newer library version. This has never been officially supported, and in practice it usually leads to confusing errors due to format changes. Backward read-compatibility is unaffected, and is tested for each release.

    Potential change

    • If you have a use-case with tiles larger than 64 MB (in-memory), please get in touch. In order to facilitate improved memory usage and i/o processing performance, we are considering a default upper-bound on tile size in a future version.

    New features

    • Floating point scaling filter (#3243, #3083)
      • API Note: input parameters and compatibility with input data are currently unvalidated. This filter requires care in the selection of scale and offset factors compatible with the input data range.
    • Support for overlapping ranges in QueryCondition. #3264
    • Enable query condition on dimensions for sparse arrays. #3302

    Improvements

    Performance

    • AWS/GCS: parallelize remove_dir. #3338
    • Optimize compute_results_count_sparse_string. #3263
    • Use sparse global ordered reader for unordered queries with no dups. #3207
    • compute_results_count_sparse_string: using cached ranges properly. #3314
    • GCS/AWS: remove unnecessary classA operations. #3323
    • Reduce the number of requests in dir_size #3382

    Internal

    • Add DataBlocks, port finite state machine, and other DAG infrastructure #3328
    • Storage manager: exposing methods to load/store generic tile. #3325
    • Replace unnecessary uses of std::unique_lock and std::scope_lock with std::lock_guard. #3340
    • Deletes: implement negate for query condition. #3299
    • Deletes: adding configuration parameter for purging deleted cells. #3334

    Defects removed

    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394
    • Fix printing of TILEDB_BOOL attributes in Attribute::Dump. #3251
    • Store compression filter's version as uint32. #3341
    • Sparse global order reader: consider qc results after deduplication. #3350
    • Serialization: using same functions to choose strategy as in query. #3352
    • Fix error reporting in tiledb_subarray_alloc. #3220
    • Fix printing of TILEDB_BLOB attributes in Attribute::Dump. #3250
    • Add missing filters to switch case for Filter serialization. #3256
    • Fix timestamp_now_ms() on 32-bit Windows. #3292
    • Avoid incorrect use of deflateEnd on init errors. #3007
    • Datatype for domain must be serialized for backwards client compatibility. #3343
    • Fix issue with sparse unordered without duplicates query deserialization to use Indexed Reader. #3347
    • Bug Fix: Wrong results when using OR condition with nullable attributes. #3308
    • Fix SC-19287: segfault due to deref nonexistent filestore key. #3359

    API additions

    C API

    • New (experimental): tiledb_fragment_info_get_total_cell_num #3234
    • New (experimental): tiledb_query_get_relevant_fragment_num #3413
    • Remove incorrect noexcept annotations from C API implementations in filestore API. #3273

    Test only changes

    • Adding tests to count VFS calls on array open. #3358
    • Print seed in unit_thread_pool on failure. #3355

    Build system changes

    • Produce a TileDBConfigVersion.cmake file. #3240
    • Add nlohmann/json.hpp to superbuild. #3279
    • Update pkg-config private requirements on Windows only. #3330

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.10.0...2.11.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.11.0-34e5dbc.tar.gz(7.80 MB)
    tiledb-linux-x86_64-noavx2-2.11.0-34e5dbc.tar.gz(7.75 MB)
    tiledb-macos-arm64-2.11.0-34e5dbc.tar.gz(5.57 MB)
    tiledb-macos-x86_64-2.11.0-34e5dbc.tar.gz(6.98 MB)
    tiledb-windows-x86_64-2.11.0-34e5dbc.zip(29.86 MB)

  • 2.11.0-rc1(Aug 5, 2022)

    TileDB v2.11.0 Release Notes

    Disk Format

    • Bump format version and remove config option for consolidation with timestamps #3267

    Breaking C API changes

    • tiledb_filter_alloc no longer returns TILEDB_OK when passed a nullptr as the third argument. #3222

    Breaking behavior

    • Enforce version upper bound for reads, #3248. TileDB will no longer attempt to read an array written with a newer library version. This has never been officially supported, and in practice it usually leads to confusing errors due to format changes. Backward read-compatibility is unaffected, and is tested for each release.

    Potential change

    • If you have a use-case with tiles larger than 64 MB (in-memory), please get in touch. In order to facilitate improved memory usage and i/o processing performance, we are considering a default upper-bound on tile size in a future version.

    New features

    • Floating point scaling filter (#3243, #3083)
      • API Note: input parameters and compatibility with input data are currently unvalidated. This filter requires care in the selection of scale and offset factors compatible with the input data range.
    • Support for overlapping ranges in QueryCondition. #3264
    • Enable query condition on dimensions for sparse arrays. #3302

    Improvements

    Performance

    • AWS/GCS: parallelize remove_dir. #3338
    • Optimize compute_results_count_sparse_string. #3263
    • Use sparse global ordered reader for unordered queries with no dups. #3207
    • compute_results_count_sparse_string: using cached ranges properly. #3314
    • GCS/AWS: remove unnecessary classA operations. #3323
    • Reduce the number of requests in dir_size #3382

    Internal

    • Add DataBlocks, port finite state machine, and other DAG infrastructure #3328
    • Storage manager: exposing methods to load/store generic tile. #3325
    • Replace unnecessary uses of std::unique_lock and std::scope_lock with std::lock_guard. #3340
    • Deletes: implement negate for query condition. #3299
    • Deletes: adding configuration parameter for purging deleted cells. #3334

    Defects removed

    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394
    • Fix printing of TILEDB_BOOL attributes in Attribute::Dump. #3251
    • Store compression filter's version as uint32. #3341
    • Sparse global order reader: consider qc results after deduplication. #3350
    • Serialization: using same functions to choose strategy as in query. #3352
    • Fix error reporting in tiledb_subarray_alloc. #3220
    • Fix printing of TILEDB_BLOB attributes in Attribute::Dump. #3250
    • Add missing filters to switch case for Filter serialization. #3256
    • Fix timestamp_now_ms() on 32-bit Windows. #3292
    • Avoid incorrect use of deflateEnd on init errors. #3007
    • Datatype for domain must be serialized for backwards client compatibility. #3343
    • Fix issue with sparse unordered without duplicates query deserialization to use Indexed Reader. #3347
    • Bug Fix: Wrong results when using OR condition with nullable attributes. #3308
    • Fix SC-19287: segfault due to deref nonexistent filestore key. #3359

    API additions

    C API

    • New (experimental): tiledb_fragment_info_get_total_cell_num #3234
    • New (experimental): tiledb_query_get_relevant_fragment_num #3413
    • Remove incorrect noexcept annotations from C API implementations in filestore API. #3273

    Test only changes

    • Adding tests to count VFS calls on array open. #3358
    • Print seed in unit_thread_pool on failure. #3355

    Build system changes

    • Produce a TileDBConfigVersion.cmake file. #3240
    • Add nlohmann/json.hpp to superbuild. #3279
    • Update pkg-config private requirements on Windows only. #3330

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.10.0...2.11.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.11.0-rc1-34e5dbc.tar.gz(7.80 MB)
    tiledb-linux-x86_64-noavx2-2.11.0-rc1-34e5dbc.tar.gz(7.75 MB)
    tiledb-macos-arm64-2.11.0-rc1-34e5dbc.tar.gz(5.57 MB)
    tiledb-macos-x86_64-2.11.0-rc1-34e5dbc.tar.gz(6.98 MB)
    tiledb-windows-x86_64-2.11.0-rc1-34e5dbc.zip(29.86 MB)

  • 2.10.4(Aug 4, 2022)

    TileDB v2.10.4 Release Notes

    Improvements

    • Adding experimental API for getting relevant fragments. #3413

    API additions

    C API

    • Add tiledb_query_get_relevant_fragment_num for experimental API to get relevant fragments. #3413
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.4-f2b5d11.tar.gz(7.72 MB)
    tiledb-linux-x86_64-noavx2-2.10.4-f2b5d11.tar.gz(7.67 MB)
    tiledb-macos-arm64-2.10.4-f2b5d11.tar.gz(5.50 MB)
    tiledb-macos-x86_64-2.10.4-f2b5d11.tar.gz(6.95 MB)
    tiledb-windows-x86_64-2.10.4-f2b5d11.zip(29.44 MB)

  • 2.10.3(Aug 2, 2022)

    TileDB v2.10.3 Release Notes

    Improvements

    • Sparse refactored readers, mark empty fragments as fully loaded early. #3394

    Bug fixes

    • Fix SC-19287: segfault due to deref nonexistent filestore key #3359
    • Bug Fix: Wrong results when using OR condition with nullable attributes #3308
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.3-7a5d1cd.tar.gz(7.72 MB)
    tiledb-linux-x86_64-noavx2-2.10.3-7a5d1cd.tar.gz(7.67 MB)
    tiledb-macos-arm64-2.10.3-7a5d1cd.tar.gz(5.50 MB)
    tiledb-macos-x86_64-2.10.3-7a5d1cd.tar.gz(6.95 MB)
    tiledb-windows-x86_64-2.10.3-7a5d1cd.zip(29.43 MB)

  • 2.11.0-rc0(Jul 28, 2022)

    TileDB v2.11.0 Release Notes

    Disk Format

    • Bump format version and remove config option for consolidation with timestamps #3267

    Breaking C API changes

    • tiledb_filter_alloc no longer returns TILEDB_OK when passed a nullptr as the third argument. #3222

    Breaking behavior

    • Enforce version upper bound for reads, #3248. TileDB will no longer attempt to read an array written with a newer library version. This has never been officially supported, and in practice it usually leads to confusing errors due to format changes. Backward read-compatibility is unaffected, and is tested for each release.

    New features

    • Floating point scaling filter (#3243, #3083)
      • API Note: input parameters and compatibility with input data are currently unvalidated. This filter requires care in the selection of scale and offset factors compatible with the input data range.
    • Support for overlapping ranges in QueryCondition. #3264
    • Enable query condition on dimensions for sparse arrays. #3302

    Improvements

    Performance

    • AWS/GCS: parallelize remove_dir. #3338
    • Optimize compute_results_count_sparse_string. #3263
    • Use sparse global ordered reader for unordered queries with no dups. #3207
    • compute_results_count_sparse_string: using cached ranges properly. #3314
    • GCS/AWS: remove unnecessary classA operations. #3323

    Internal

    • Add DataBlocks, port finite state machine, and other DAG infrastructure #3328
    • Storage manager: exposing methods to load/store generic tile. #3325
    • Replace unnecessary uses of std::unique_lock and std::scope_lock with std::lock_guard. #3340
    • Deletes: implement negate for query condition. #3299
    • Deletes: adding configuration parameter for purging deleted cells. #3334

    Defects removed

    • Fix printing of TILEDB_BOOL attributes in Attribute::Dump. #3251
    • Store compression filter's version as uint32. #3341
    • Sparse global order reader: consider qc results after deduplication. #3350
    • Serialization: using same functions to choose strategy as in query. #3352
    • Fix error reporting in tiledb_subarray_alloc. #3220
    • Fix printing of TILEDB_BLOB attributes in Attribute::Dump. #3250
    • Add missing filters to switch case for Filter serialization. #3256
    • Fix timestamp_now_ms() on 32-bit Windows. #3292
    • Avoid incorrect use of deflateEnd on init errors. #3007
    • Datatype for domain must be serialized for backwards client compatibility. #3343
    • Fix issue with sparse unordered without duplicates query deserialization to use Indexed Reader. #3347
    • Bug Fix: Wrong results when using OR condition with nullable attributes. #3308
    • Fix SC-19287: segfault due to deref nonexistent filestore key. #3359

    API additions

    C API

    • New: tiledb_fragment_info_get_total_cell_num #3234
    • Remove incorrect noexcept annotations from C API implementations in filestore API. #3273

    Test only changes

    • Adding tests to count VFS calls on array open. #3358
    • Print seed in unit_thread_pool on failure. #3355

    Build system changes

    • Produce a TileDBConfigVersion.cmake file. #3240
    • Add nlohmann/json.hpp to superbuild. #3279
    • Update pkg-config private requirements on Windows only. #3330

    Full Changelog:

    • https://github.com/TileDB-Inc/TileDB/compare/2.10.0...2.11.0
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.11.0-rc0-a1452f8.tar.gz(7.80 MB)
    tiledb-linux-x86_64-noavx2-2.11.0-rc0-a1452f8.tar.gz(7.75 MB)
    tiledb-macos-arm64-2.11.0-rc0-a1452f8.tar.gz(5.57 MB)
    tiledb-macos-x86_64-2.11.0-rc0-a1452f8.tar.gz(6.98 MB)
    tiledb-windows-x86_64-2.11.0-rc0-a1452f8.zip(29.85 MB)

  • 2.10.2(Jul 7, 2022)

    TileDB v2.10.2 Release Notes

    Bug fixes

    • Close magic resources #3319
    • Correct simple typo in assignment #3335
    • Datatype for domain must be serialized for backwards client compatibility #3343
    • Sparse global order reader: disable reader for reads with qc. #3342
    • Fix issue with sparse unordered without duplicates query deserialization to use Indexed Reader #3347
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.2-9ab84f9.tar.gz(7.70 MB)
    tiledb-linux-x86_64-noavx2-2.10.2-9ab84f9.tar.gz(7.63 MB)
    tiledb-macos-arm64-2.10.2-9ab84f9.tar.gz(5.48 MB)
    tiledb-macos-x86_64-2.10.2-9ab84f9.tar.gz(6.93 MB)
    tiledb-windows-x86_64-2.10.2-9ab84f9.zip(29.36 MB)

  • 2.10.2-rc0(Jul 7, 2022)

    TileDB v2.10.2-rc0 Release Notes

    Bug fixes

    • Close magic resources #3319
    • Correct simple typo in assignment #3335
    • Datatype for domain must be serialized for backwards client compatibility #3343
    • Sparse global order reader: disable reader for reads with qc. #3342
    • Fix issue with sparse unordered without duplicates query deserialization to use Indexed Reader #3347
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.2-rc0-9ab84f9.tar.gz(7.70 MB)
    tiledb-linux-x86_64-noavx2-2.10.2-rc0-9ab84f9.tar.gz(7.63 MB)
    tiledb-macos-arm64-2.10.2-rc0-9ab84f9.tar.gz(5.48 MB)
    tiledb-macos-x86_64-2.10.2-rc0-9ab84f9.tar.gz(6.93 MB)
    tiledb-windows-x86_64-2.10.2-rc0-9ab84f9.zip(29.35 MB)

  • 2.10.1(Jun 24, 2022)

  • 2.10.0(Jun 23, 2022)

    TileDB v2.10.0 Release Notes

    Full Changelog: https://github.com/TileDB-Inc/TileDB/compare/2.9.0...2.10.0

    Disk Format

    • Consolidation with timestamps: add includes timestamps to fragment footer. #3138

    New features

    • OR clause support in Query Conditions (#3041 #3083 #3112 #3264)
    • New examples for QueryCondition usage with the C++ API (#3225) and C API (#3242)
    • TILEDB_BOOL Datatype #3164
    • Support for group metadata consolidation and vacuuming #3175

    Breaking behavior

    • Remove timestamp-range vacuuming (experimental) #3214

    Improvements

    • Sparse global order reader: refactor merge algorithm. #3173
    • Dense reader: add better stats for attribute copy. #3199
    • Dense reader: adding ability to fully disable tile cache. #3227
    • Optimize compute_results_count_sparse_string. #3263

    Deprecations

    • Deprecate sm.num_tbb_threads config option #3177

    Bug fixes

    • Dense reader: fixing query conditions with overlapping domains. #3244
    • Consolidation w timestamps: cell slab length computations fix. #3230
    • Unordered writer: fixing segfault for empty writes. #3161
    • Sparse global order reader: Check the right incomplete reason is returned in case of too small user buffer #3170
    • Global writes: check global order on write continuation. #3109
    • Fixing Dimension::splitting_value to prevent overflows. #3116
    • Parse minor and patch version with more than one digit #3098
    • Sparse unordered w/ dups reader: fix incomplete reason for cloud reads. #3104
    • Rearrange context member initialization so logger is initialized prior to getting thread counts (which may log) #3128
    • adjust assert for var Range usage in legacy global order reader #3122
    • Fix SC-17415: segfault due to underflow in for loop #3143
    • Fix fragment_consolidation.cc example #3145
    • Change test_assert path used to locate try_assert #3158
    • example writing_sparse_global_<c,cpp>, change illegal write to be legal #3159
    • avoid unit_range warning as error build failures #3171
    • change to avoid warning causing build error with msvc #3162
    • Sparse unordered w/ dups reader: fixing overflow on int value. #3181
    • Sparse index readers: fixing queries with overlapping ranges. #3208
    • Fix bad_optional_access exceptions when running consolidation with timestamps tests #3213
    • Fix SC-18250: segfault due to empty default-constructed FilterPipeline #3233
    • Fixed regression test for SC12024, Incorrect selected type in Dimension::oob #3219
    • Fixed bug in documentation for cpp_api query condition examples. #3239
    • Fix File API failure when importing into TileDB Cloud array #3246
    • Fix undefined behavior in filestore whilst detecting compression #3291
    • Fix printing of TILEDB_BLOB attributes in Attribute::Dump #3250
    • Add missing filters to switch case for Filter serialization #3256
    • Fix a typo in the byteshuffle constructor for capnp serialization #3284
    • Remove incorrect noexcept annotations from C API implementations in filestore API #3273
    • Update ensure_datatype_is_valid to fix deserialization issues #3303

    API Changes

    C++ API

    • Apply TILEDB_NO_API_DEPRECATION_WARNINGS to C++ API #3236

    Build System

    • Add tiledb_regression test target #3143
    • Produce a TileDBConfigVersion.cmake file #3240
    • Integrate build of webp into tiledb superbuild (note: build-only) #3113

    New Contributors

    • @-OgreTransporter made their first contribution in https://github.com/TileDB-Inc/TileDB/pull/3118
    • @-Biswa96 made their first contribution in https://github.com/TileDB-Inc/TileDB/pull/3124
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.0-0b54e51.tar.gz(7.69 MB)
    tiledb-linux-x86_64-noavx2-2.10.0-0b54e51.tar.gz(7.63 MB)
    tiledb-macos-arm64-2.10.0-0b54e51.tar.gz(5.48 MB)
    tiledb-macos-x86_64-2.10.0-0b54e51.tar.gz(6.93 MB)
    tiledb-windows-x86_64-2.10.0-0b54e51.zip(29.35 MB)

  • 2.10.0-rc1(Jun 21, 2022)

    TileDB v2.10.0 Release Notes

    Full Changelog: https://github.com/TileDB-Inc/TileDB/compare/2.9.0...2.10.0

    Disk Format

    • Consolidation with timestamps: add includes timestamps to fragment footer. #3138

    New features

    • OR clause support in Query Conditions (#3041 #3083 #3112 #3264)
    • New examples for QueryCondition usage with the C++ API (#3225) and C API (#3242)
    • TILEDB_BOOL Datatype #3164
    • Support for group metadata consolidation and vacuuming #3175

    Breaking behavior

    • Remove timestamp-range vacuuming (experimental) #3214

    Improvements

    • Sparse global order reader: refactor merge algorithm. #3173
    • Dense reader: add better stats for attribute copy. #3199
    • Dense reader: adding ability to fully disable tile cache. #3227
    • Optimize compute_results_count_sparse_string. #3263

    Deprecations

    • Deprecate sm.num_tbb_threads config option #3177

    Bug fixes

    • Dense reader: fixing query conditions with overlapping domains. #3244
    • Consolidation w timestamps: cell slab length computations fix. #3230
    • Unordered writer: fixing segfault for empty writes. #3161
    • Sparse global order reader: Check the right incomplete reason is returned in case of too small user buffer #3170
    • Global writes: check global order on write continuation. #3109
    • Fixing Dimension::splitting_value to prevent overflows. #3116
    • Parse minor and patch version with more than one digit #3098
    • Sparse unordered w/ dups reader: fix incomplete reason for cloud reads. #3104
    • Rearrange context member initialization so logger is initialized prior to getting thread counts (which may log) #3128
    • adjust assert for var Range usage in legacy global order reader #3122
    • Fix SC-17415: segfault due to underflow in for loop #3143
    • Fix fragment_consolidation.cc example #3145
    • Change test_assert path used to locate try_assert #3158
    • example writing_sparse_global_<c,cpp>, change illegal write to be legal #3159
    • avoid unit_range warning as error build failures #3171
    • change to avoid warning causing build error with msvc #3162
    • Sparse unordered w/ dups reader: fixing overflow on int value. #3181
    • Sparse index readers: fixing queries with overlapping ranges. #3208
    • Fix bad_optional_access exceptions when running consolidation with timestamps tests #3213
    • Fix SC-18250: segfault due to empty default-constructed FilterPipeline #3233
    • Fixed regression test for SC12024, Incorrect selected type in Dimension::oob #3219
    • Fixed bug in documentation for cpp_api query condition examples. #3239
    • Fix File API failure when importing into TileDB Cloud array #3246
    • Fix undefined behavior in filestore whilst detecting compression #3291
    • Fix printing of TILEDB_BLOB attributes in Attribute::Dump #3250
    • Add missing filters to switch case for Filter serialization #3256
    • Fix a typo in the byteshuffle constructor for capnp serialization #3284
    • Remove incorrect noexcept annotations from C API implementations in filestore API #3273
    • Update ensure_datatype_is_valid to fix deserialization issues #3303

    API Changes

    C++ API

    • Apply TILEDB_NO_API_DEPRECATION_WARNINGS to C++ API #3236

    Build System

    • Add tiledb_regression test target #3143
    • Produce a TileDBConfigVersion.cmake file #3240
    • Integrate build of webp into tiledb superbuild (note: build-only) #3113

    New Contributors

    • @-OgreTransporter made their first contribution in https://github.com/TileDB-Inc/TileDB/pull/3118
    • @-Biswa96 made their first contribution in https://github.com/TileDB-Inc/TileDB/pull/3124
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.0-rc1-8ef6882.tar.gz(7.69 MB)
    tiledb-linux-x86_64-noavx2-2.10.0-rc1-8ef6882.tar.gz(7.63 MB)
    tiledb-macos-arm64-2.10.0-rc1-8ef6882.tar.gz(5.48 MB)
    tiledb-macos-x86_64-2.10.0-rc1-8ef6882.tar.gz(6.93 MB)
    tiledb-windows-x86_64-2.10.0-rc1-8ef6882.zip(29.36 MB)

  • 2.9.5(Jun 17, 2022)

  • 2.10.0-rc0(Jun 9, 2022)

    What's Changed

    • Update group post from rest to use group_metadata_serialize over config_serialize by @Shelnutt2 in https://github.com/TileDB-Inc/TileDB/pull/3095
    • Bump dev version to 2.10 by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3094
    • Parse minor and patch version with more than one digit by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3098
    • Add range helper functions and unit tests by @jp-dark in https://github.com/TileDB-Inc/TileDB/pull/3066
    • ArraySchema::deserialize factory conversion by @bekadavis9 in https://github.com/TileDB-Inc/TileDB/pull/3012
    • add libmagic to build process embedding magic.mgc data by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3088
    • Fix build error after merge of #3012 by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3103
    • Sparse unordered w/ dups reader: fix incomplete reason for cloud reads. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3104
    • Add doc config and tiledb_version.h to CI ignore by @Shelnutt2 in https://github.com/TileDB-Inc/TileDB/pull/3107
    • 3088 follow-ups by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3111
    • Sparse global order reader: remove possible empty cell slab. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3115
    • Fixing Dimension::splitting_value to prevent overflows. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3116
    • Lums/sc 11594/replace two phase construction for threadpool by @lums658 in https://github.com/TileDB-Inc/TileDB/pull/3084
    • Bugfix for #3088 by @OgreTransporter in https://github.com/TileDB-Inc/TileDB/pull/3118
    • Remove msys2 from workflows, reduce rtools40 matrix to ucrt by @eddelbuettel in https://github.com/TileDB-Inc/TileDB/pull/3120
    • Fix manylinux release build linkage error after 3088 by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3123
    • cmake: Fix undefined MSVC_MP_FLAG variable in mingw by @Biswa96 in https://github.com/TileDB-Inc/TileDB/pull/3124
    • varying_size_datum_at: fixing comparison error. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3127
    • Rearrange context member initialization by @lums658 in https://github.com/TileDB-Inc/TileDB/pull/3128
    • Dimension::dimension_ptr, part 1 by @eric-hughes-tiledb in https://github.com/TileDB-Inc/TileDB/pull/3135
    • adjust assert for problematic var_size_ Range usage after earlier changes by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3122
    • Global writes: check global order on write continuation. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3109
    • Consolidation with timestamps: adding configuration option. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3136
    • use pre-generated magic_mgc_gzipped.bin by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3132
    • Add regression test target; Test and fix for SC-17415 by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3143
    • dimension_ptr functions, part 2 by @eric-hughes-tiledb in https://github.com/TileDB-Inc/TileDB/pull/3142
    • File API support by @robertbindar in https://github.com/TileDB-Inc/TileDB/pull/3121
    • Correct fragment_consolidation.cc example adjusting for functionality removed from core by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3145
    • Including array schema name to fragment footer format. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3139
    • Fixes implicit conversion causing clang warning by @jp-dark in https://github.com/TileDB-Inc/TileDB/pull/3131
    • Consolidation with timestamps: add includes timestamps to frag footer. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3138
    • Filter pipeline: fixing empty pipeline, multi chunk, refactored queries. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3149
    • Consolidation with timestamps: enabling in fragment consolidator. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3144
    • Sparse global order reader: prevent dims from being unfiltered twice. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3150
    • Dimension: fixing tile_idx for int8_t and int16_t types. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3151
    • Move real unit tests to proper directory. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3156
    • Use range helper functions to add range in Subarray by @jp-dark in https://github.com/TileDB-Inc/TileDB/pull/3110
    • Consolidation w/ timestamps: writing timestamps tiles. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3155
    • Anticipatory version of C++23 std::unreachable() by @eric-hughes-tiledb in https://github.com/TileDB-Inc/TileDB/pull/3099
    • Changes test_assert path used to locate try_assert by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3158
    • Add regression test for SC12024, Incorrect selected type in Dimension::oob by @gsakkis in https://github.com/TileDB-Inc/TileDB/pull/2621
    • example writing_sparse_global_<c,cpp>, change illegal write to be legal by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3159
    • Unordered writer: fixing segfault for empty writes. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3161
    • Add TEST as PR type by @ypatia in https://github.com/TileDB-Inc/TileDB/pull/3174
    • Prepare release script does not check pr merge status by @gsakkis in https://github.com/TileDB-Inc/TileDB/pull/3176
    • avoid vs2019 unit_range warning as error by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3171
    • change to avoid msvc warning causing build error by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3162
    • Sync HISTORY for 2.8 and 2.9 branches to dev by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3178
    • Sparse global order reader: refactor merge algorithm. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3173
    • Deprecate sm.num_tbb_threads config option by @bekadavis9 in https://github.com/TileDB-Inc/TileDB/pull/3177
    • Sparse unordered w/ dups reader: fixing overflow on int value. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3181
    • relocate magic_mgc_gzipped.bin for build by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3179
    • Sparse global order reader: Check correct incomplete reason in case of too small user buffer by @ypatia in https://github.com/TileDB-Inc/TileDB/pull/3170
    • Adding QueryCondition testing support within the query interface by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/3112
    • Add Datatype::TILEDB_BOOL by @bekadavis9 in https://github.com/TileDB-Inc/TileDB/pull/3164
    • Query condition: cache cell_num when processing nullable attributes. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3188
    • Refactor tiledb::sm::serialization::dimension_from_capnp (C41) by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/2949
    • Size types by @eric-hughes-tiledb in https://github.com/TileDB-Inc/TileDB/pull/3194
    • Activate rtools builds to support R 4.1.* by @eddelbuettel in https://github.com/TileDB-Inc/TileDB/pull/3197
    • Update Zlib Download URL by @nguyenv in https://github.com/TileDB-Inc/TileDB/pull/3200
    • Dense reader: add better stats for attribute copy. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3199
    • Add partial timestamp coverage for consolidation to array directory by @ypatia in https://github.com/TileDB-Inc/TileDB/pull/3193
    • Consolidation tests: using smaller buffers. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3203
    • Array directory: write queries should open in schema only mode. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3204
    • Sparse global order reader: read fragments with timestamps. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3196
    • Integrate local build of webp into tiledb build by @dhoke4tdb in https://github.com/TileDB-Inc/TileDB/pull/3113
    • Sparse index readers: fixing queries with overlapping ranges. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3208
    • Add support for Group metadata consolidation and vacuuming by @robertbindar in https://github.com/TileDB-Inc/TileDB/pull/3175
    • Domain_from_capnp C.41 conversion by @bekadavis9 in https://github.com/TileDB-Inc/TileDB/pull/3198
    • Remove timestamp-based vacuuming by @robertbindar in https://github.com/TileDB-Inc/TileDB/pull/3214
    • Fix bad_optional_access exceptions when running consolidation with timestamps tests by @ypatia in https://github.com/TileDB-Inc/TileDB/pull/3213
    • Consolidation with timestamps: consider timestamps when computing memory bugdet by @ypatia in https://github.com/TileDB-Inc/TileDB/pull/3206
    • Convert Array* to shared_ptr by @bekadavis9 in https://github.com/TileDB-Inc/TileDB/pull/3195
    • Make filestore api get configurable buffer sizes by @robertbindar in https://github.com/TileDB-Inc/TileDB/pull/3223
    • Dense reader: adding ability to fully disable tile cache. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3227
    • Posix: cache cwd. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3226
    • Consolidation with timestamps: filter sparse reader results based on read time. by @ypatia in https://github.com/TileDB-Inc/TileDB/pull/3221
    • Consolidation w timestamps: cell slab length computations fix. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3230
    • Special case tiledb cloud uris to use row_major writes by @robertbindar in https://github.com/TileDB-Inc/TileDB/pull/3216
    • Consolidation with timestamps: original reader reads fragments. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3231
    • Fix filterpipeline segfault by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3233
    • Add examples for query conditions: cpp API by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/3225
    • Apply TILEDB_NO_API_DEPRECATION_WARNINGS to C++ API by @ihnorton in https://github.com/TileDB-Inc/TileDB/pull/3236
    • Fixed regression test for SC12024, Incorrect selected type in Dimension::oob by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/3219
    • Fixed bug in documentation for cpp_api query condition examples. by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/3239
    • Adding basic unit test for ArrayDirectory. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3245
    • QueryCondition OR support: Implement query condition evaluation via AST by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/3041
    • Fix File API failure when importing into TileDB Cloud array by @robertbindar in https://github.com/TileDB-Inc/TileDB/pull/3246
    • Dense reader: fixing query conditions with overlapping domains. by @KiterLuc in https://github.com/TileDB-Inc/TileDB/pull/3244
    • Add examples for query conditions: C API by @abigalekim in https://github.com/TileDB-Inc/TileDB/pull/3242
    • [Backport release-2.10] Support TILEDB_BLOB in tiledb::sm::utils::parse::to_str by @github-actions in https://github.com/TileDB-Inc/TileDB/pull/3252
    • [Backport release-2.10] Support TILEDB_BOOL in tiledb::sm::utils::parse::to_str by @github-actions in https://github.com/TileDB-Inc/TileDB/pull/3255
    • [Backport release-2.10] Add missing filters to switch case for Filter serialization by @github-actions in https://github.com/TileDB-Inc/TileDB/pull/3257

    New Contributors

    • @OgreTransporter made their first contribution in https://github.com/TileDB-Inc/TileDB/pull/3118
    • @Biswa96 made their first contribution in https://github.com/TileDB-Inc/TileDB/pull/3124

    Full Changelog: https://github.com/TileDB-Inc/TileDB/compare/2.9.3...2.10.0-rc0

    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.10.0-rc0-b3dd41e.tar.gz(7.69 MB)
    tiledb-linux-x86_64-noavx2-2.10.0-rc0-b3dd41e.tar.gz(7.63 MB)
    tiledb-macos-arm64-2.10.0-rc0-b3dd41e.tar.gz(5.47 MB)
    tiledb-macos-x86_64-2.10.0-rc0-b3dd41e.tar.gz(6.92 MB)
    tiledb-windows-x86_64-2.10.0-rc0-b3dd41e.zip(29.34 MB)

  • 2.9.4(Jun 10, 2022)

    TileDB v2.9.4 Release Notes

    Bug fixes

    • Fix File API failure when importing into TileDB Cloud array #3246
    • Fix printing of TILEDB_BLOB attributes in Attribute::Dump #3250
    • Add missing filters to switch case for Filter serialization #3256
    • Fix filterpipeline segfault on release-2.9 #3261
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.9.4-4e14c01.tar.gz(7.59 MB)
    tiledb-linux-x86_64-noavx2-2.9.4-4e14c01.tar.gz(7.54 MB)
    tiledb-macos-arm64-2.9.4-4e14c01.tar.gz(5.42 MB)
    tiledb-macos-x86_64-2.9.4-4e14c01.tar.gz(6.82 MB)
    tiledb-windows-x86_64-2.9.4-4e14c01.zip(29.08 MB)

  • 2.9.3-rc0(Jun 2, 2022)

    TileDB v2.9.3 Release Notes

    Improvements

    • Make filestore api get configurable buffer sizes #3223
    • Special case tiledb cloud uris to use row_major writes 3232]([https://github.com/TileDB-Inc/TileDB/pull/3232)

    C++ API

    • Apply TILEDB_NO_API_DEPRECATION_WARNINGS to C++ API #3236
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.9.3-rc0-9f4129b.tar.gz(7.59 MB)
    tiledb-linux-x86_64-noavx2-2.9.3-rc0-9f4129b.tar.gz(7.53 MB)
    tiledb-macos-arm64-2.9.3-rc0-9f4129b.tar.gz(5.42 MB)
    tiledb-macos-x86_64-2.9.3-rc0-9f4129b.tar.gz(6.82 MB)
    tiledb-windows-x86_64-2.9.3-rc0-9f4129b.zip(29.07 MB)

  • 2.9.3(Jun 3, 2022)

    TileDB v2.9.3 Release Notes

    Improvements

    • Make filestore api get configurable buffer sizes #3223
    • Special case tiledb cloud uris to use row_major writes #3232

    C++ API

    • Apply TILEDB_NO_API_DEPRECATION_WARNINGS to C++ API #3236
    Source code(tar.gz)
    Source code(zip)
    tiledb-linux-x86_64-2.9.3-9f4129b.tar.gz(7.59 MB)
    tiledb-linux-x86_64-noavx2-2.9.3-9f4129b.tar.gz(7.53 MB)
    tiledb-macos-arm64-2.9.3-9f4129b.tar.gz(5.42 MB)
    tiledb-macos-x86_64-2.9.3-9f4129b.tar.gz(6.82 MB)
    tiledb-windows-x86_64-2.9.3-9f4129b.zip(29.07 MB)

  • 2.9.2-rc0(May 27, 2022)

Owner
TileDB, Inc.
TileDB, Inc.
GitHub https://tiledb.com
Spacex Storage is an offchain storage work inspector of Mannheim Network running inside TEE enclave.

Spacex Storage Spacex Storage is an offchain storage work inspector of Mannheim Network running inside TEE enclave. Contribution Thank you for conside

Mannheim Network 10 Oct 21, 2022
Beringei is a high performance, in-memory storage engine for time series data.

** THIS REPO HAS BEEN ARCHIVED AND IS NO LONGER BEING ACTIVELY MAINTAINED ** Beringei A high performance, in memory time series storage engine In the

Meta Archive 3.1k Dec 24, 2022
OrioleDB – building a modern cloud-native storage engine

OrioleDB is a new storage engine for PostgreSQL, bringing a modern approach to database capacity, capabilities and performance to the world's most-loved database platform.

OrioleDB 1.3k Dec 31, 2022
universal serialization engine

A Universal Serialization Engine Based on compile-time Reflection iguana is a modern, universal and easy-to-use serialization engine developed in c++1

qicosmos 711 Jan 7, 2023
Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.

Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

Cossack Labs 1.6k Jan 6, 2023
Simple constant key/value storage library, for read-heavy systems with infrequent large bulk inserts.

Sparkey is a simple constant key/value storage library. It is mostly suited for read heavy systems with infrequent large bulk inserts. It includes bot

Spotify 989 Dec 14, 2022
Firebase Arduino Client Library for ESP8266 and ESP32. The unified version of Firebase-ESP8266 and Firebase-ESP32 Realtime database libraries with Cloud Firestore, Firebase and Google Cloud Storage, Cloud messaging and Cloud Functions supports.

Firebase Arduino Client Library for ESP8266 and ESP32 Google's Firebase Arduino Client Library for ESP8266 and ESP32 v 2.0.3 This library supports ESP

null 313 Dec 31, 2022
DNS and Target HTTP History Local Storage and Search

dooked DNS and Target HTTP History Local Storage and Search Installation Download Boost Library from the official website Extract the library into any

Michael Skelton 60 Oct 31, 2022
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB

Facebook 24.3k Jan 5, 2023
Simplified distributed block storage with strong consistency, like in Ceph (repository mirror)

Vitastor Читать на русском The Idea Make Software-Defined Block Storage Great Again. Vitastor is a small, simple and fast clustered block storage (sto

Vitaliy Filippov 63 Jan 7, 2023
Kreon is a key-value store library optimized for flash-based storage

Kreon is a key-value store library optimized for flash-based storage, where CPU overhead and I/O amplification are more significant bottlenecks compared to I/O randomness.

Computer Architecture and VLSI Systems (CARV) Laboratory 24 Jul 14, 2022
Poseidon OS (POS) is a light-weight storage OS

Poseidon OS Poseidon OS (POS) is a light-weight storage OS that offers the best performance and valuable features over storage network. POS exploits t

null 59 Jan 5, 2023
C++ DataFrame for statistical, Financial, and ML analysis -- in modern C++ using native types, continuous memory storage, and no pointers are involved

C++ DataFrame for statistical, Financial, and ML analysis -- in modern C++ using native types, continuous memory storage, and no pointers are involved

Hossein Moein 1.7k Jan 9, 2023
MinIO C++ Client SDK for Amazon S3 Compatible Cloud Storage

The MinIO C++ Client SDK provides simple APIs to access any Amazon S3 compatible object storage.

Multi-Cloud Object Storage 45 Dec 26, 2022
LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. Authors: Sanjay Ghem

Google 31.6k Jan 7, 2023
XTAO Unified Distributed Storage

Anna - A branch project from CEPH Anna is a XTAO project branched from CEPH distributed storage. CEPH is a nice opensource project for unified distrib

XTAO Technolgy 3 Nov 12, 2021
A simple console client for pCloud cloud storage.

pCloud Console Client A simple console client for pCloud cloud storage. Project Information pCloud Console Client was forked from the project initiall

Serghei Iakovlev 24 Oct 28, 2022
The database built for IoT streaming data storage and real-time stream processing.

The database built for IoT streaming data storage and real-time stream processing.

HStreamDB 575 Dec 26, 2022
Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform.

serverless-sqlite Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform. NOTES: This repository is still in t

老雷 7 May 12, 2022
An efficient, small mobile key-value storage framework developed by WeChat. Works on Android, iOS, macOS, Windows, and POSIX.

中文版本请参看这里 MMKV is an efficient, small, easy-to-use mobile key-value storage framework used in the WeChat application. It's currently available on Andr

Tencent 15.4k Jan 8, 2023