Kvrocks is a distributed key value NoSQL database based on RocksDB and compatible with Redis protocol.

Overview

kvrocks_logo

image image GitHub license GitHub stars

Kvrocks is an open-source key-value database which is based on rocksdb and compatible with Redis protocol. Intention to decrease the cost of memory and increase the capability while compared to Redis. The design of replication and storage was inspired by rocksplicator and blackwidow.

Kvrocks has the following key features:

  • Redis protocol, user can use redis client to visit the kvrocks
  • Namespace, similar to redis db but use token per namespace
  • Replication, async replication using binlog like MySQL
  • High Available, supports redis sentinel to failover when master or slave was failed

Thanks for designers @田凌宇 and @范世丽 contribute the kvrocks logo for us.

Who uses kvrocks

Meitu Ctrip
Baidu BaishanCloud

Tickets a pull reqeust to let us known that you're using kvrocks and add your logo to README

Building kvrocks

requirements

  • g++ (required by c++11, version >= 4.8)
  • autoconf automake libtool snappy

Build

NOTE: You should install the snappy first:

# Centos/Redhat
sudo yum install -y epel-release && sudo yum install -y git gcc gcc-c++ make snappy snappy-devel autoconf automake libtool which gtest gtest-devel

# Ubuntu
sudo apt-get install gcc g++ make libsnappy-dev autoconf automake libtool which libgtest-dev

# MACOSX
brew install snappy googletest

It is as simple as:

$ git clone --recursive https://github.com/kvrockslabs/kvrocks.git
$ cd kvrocks
$ make -j4

Running kvrocks

$ ./src/kvrocks -c kvrocks.conf

Running test cases

NOTE: You should install the googletest first

make test

Supported platforms

  • centos 6/7
  • ubuntu
  • macosx

Try kvrocks using Docker

$ docker run -it -p 6666:6666 kvrocks/kvrocks
$ redis-cli -p 6666

127.0.0.1:6666> get a
(nil)

Namespace

namespace was used to isolate data between users. unlike all the redis databases can be visited by requirepass, we use one token per namespace. requirepass was regraded as admin token, only admin token allows to access the namespace command, as well as some commands like config, slaveof, bgsave, etc…

namespace del ns1 OK ">
# add token
127.0.0.1:6666>  namespace add ns1 mytoken
OK

# update token
127.0.0.1:6666> namespace set ns1 new_token
OK

# list namespace
127.0.0.1:6666> namespace get *
1) "ns1"
2) "new_token"
3) "__namespace"
4) "foobared"

# delete namespace
127.0.0.1:6666> namespace del ns1
OK

DOCs

Migrate tools

  • migrate from redis to kvrocks, use redis-migrate-tool which was developed by vipshop
  • migrate from kvrocks to redis. use kvrocks2redis in build dir

Performance

Hardware

  • CPU: 48 cores Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  • Memory: 32 GiB
  • NET: Intel Corporation I350 Gigabit Network Connection
  • DISK: 2TB NVMe Intel SSD DC P460

Benchmark Client: multi-thread redis-benchmark(unstable branch)

1. Commands QPS

kvrocks: workers = 16, benchmark: 8 threads/ 512 conns / 128 payload

latency: 99.9% < 10ms

image

2. QPS on different payloads

kvrocks: workers = 16, benchmark: 8 threads/ 512 conns

latency: 99.9% < 10ms

image

3. QPS on different workers

kvrocks: workers = 16, benchmark: 8 threads/ 512 conns / 128 payload

latency: 99.9% < 10ms

image

License

Kvrocks is under the BSD-3-Clause license. See the LICENSE file for details.

Issues
  • [QUESTION] Change the license to Apache License Version 2.0

    [QUESTION] Change the license to Apache License Version 2.0

    Hi, all Kvrocks contributors, we want to donate Kvrocks to Apache Software Foundation, the first step for us is changing the license to Apache License Version 2.0. As we know, Kvrocks is an open source software under the BSD-3-Clause license, we truly thank every contributor and user, you guys make Kvrocks great and have an active community, before we could submit PR to change license, we must to get your approval.

    @git-hulk @karelrooted @ShooterIT @Alfejik @caipengbo @ChrisZMF @shangxiaoxiong @ColinChamber @guoxiangCN @calvinxiao @patpatbear @WyattJia @smartlee @popunit @clyang @byronhe @QQxiaoyuyu @nioshield @singgel @liangjingyang @gaoyuanzong @rongzeng54 @unext-ops @fishery86 @xujianhai666 @LykxSassinator

    Please reply "Approved" if you agree.

    opened by ShooterIT 27
  • 同步基本不可用

    同步基本不可用

    1. masterA 有数据 slaveA slaveof过来,数据可以同步,flushall masterA的数据, slaveA 没有同步清空掉。都执行过dbsize scan 2)masterB 没数据 slaveB 同步过来up后 ,masterB上执行写数据,数据没有同步到slaveB 3)slaveof后 slave节点上经常会报以下错误, 但这个目录是清空后再启动slave的
    E0216 13:35:53.319345 26935 storage.cc:703] [storage] Failed to delete dir: IO error: file rmdir: ./kvrocksdatamaster/backup/meta: Directory not empty
    E0216 13:35:53.415446 26935 replication.cc:584] [replication] Failed to restore backup while IO error: No such file or directoryWhile opening a file for sequentially reading: ./kvrocksdatamaster/backup/meta/5: No such file or directory
    Segmentation fault (core dumped)
    
    need more information 
    opened by smartlee 23
  • Revert

    Revert "Fix the misuse of multi-get API"

    Reverts KvrocksLabs/kvrocks#442 l find that new multi-get api can get keys which have been deleted before. the case could be reproduced with my PR KvrocksLabs/kvrocks/pull/444 . It seems a bug of RocksDB and I would submit a relevant PR to RocksDB in the near future.

    opened by shangxiaoxiong 21
  • Tracking issue for build system enhancements

    Tracking issue for build system enhancements

    Search before asking

    • [X] I had searched in the issues and found no similar issues.

    Motivation

    Modern CMake provides many amazing features to organize the build structure of a project in a more convenient way, and we can make full use of them to improve the development experience! @tisonkun had a good suggestion in https://github.com/apache/incubator-kvrocks/issues/562#issuecomment-1123771508 of creating a tracking issue to help keep the discussion more focused, so this issue is created for that purpose.

    Solution

    The list below tracks some of the improvements that have been proposed so far, and I'd appreciate it if anyone came up with new ideas for improvements, PRs for current improvements, or participated in this discussion!

    • [x] #561
    • [x] #562
    • [x] #574
    • [x] #569
    • [x] #563
    • [x] #580
    • [x] #589
    • [x] #590

    Are you willing to submit a PR?

    • [X] I'm willing to submit a PR!
    enhancement 
    opened by PragmaTwice 16
  • Memory leaks in Lua::redisGenericCommand

    Memory leaks in Lua::redisGenericCommand

    Search before asking

    • [X] I had searched in the issues and found no similar issues.

    Version

    redisGenericCommand use raiseError to process errors, but raiseError will perform a longjmp in the lua_error api call, so raiseError never returns.

    longjmp does not compatible with RAII in C++ (C++ exceptions may use longjmp in implementation but the compiler will insert some dtor call autometically and carefully, i.e. stack unwinding), so every object in the stack of redisGenericCommand will never be destroyed, i.e. no destructor will be call, hence all resource allocated in ctor like std::vector, std::unique_ptr etc. cannot be released.

    @git-hulk

    Minimal reproduce step

    ./runtest --dont-clean on a kvrocks binary with Address Sanitizer enabled. (try to build with #599) You can only run tests in unit/scripting to save time : )

    What did you expect to see?

    No memory leaks

    What did you see instead?

    Memory leaks are reported by Address Sanitizer

    Anything Else?

    No response

    Are you willing to submit a PR?

    • [X] I'm willing to submit a PR!
    bug 
    opened by PragmaTwice 15
  • [BUG]  Failed to open DB with missing CURRENT file

    [BUG] Failed to open DB with missing CURRENT file

    Describe the bug A clear and concise description of what the bug is.

    • OS: MAC OS 10.15.7
    • Version 2.0.5 @6a4ed16a

    get following error message when try to start up kvrocks ( first time run )

    E1229 15:41:02.217978 276704704 main.cc:314] Failed to open: IO error: No such file or directory: While opening a file for sequentially reading: /tmp/kvrocks/db/CURRENT: No such file or directory

    /tmp/kvrocks directory did exist but no sub-directory ./db under it.

    opened by jasonhe88 15
  • [NEW] Consider using `FetchContent` instead of `ExternalProject` and git submodules?

    [NEW] Consider using `FetchContent` instead of `ExternalProject` and git submodules?

    FetchContent has been adopted by many projects as a dependency fetching method introduced in newer versions of cmake, for example the official documentation for gtest already recommends using FetchContent to import itself into other projects (replacing the previous recommended ExternalProject).

    The advantage of FetchContent is that it is executed during the cmake configuration phase, which allows the configuration information of dependent projects to be introduced into the main project, unlike ExternalProject, which is executed during the build phase.

    This is quite useful especially for dependency projects built with cmake, because the configured cmake target of a dependency project can be used directly in the main project via add_subdirectory (e.g. gtest::gtest), instead of having to manually configure the dependency's header directories, static or shared libraries, compilation options, install directories, etc. and keeping maintaining these redundant information in the main project.

    This also makes git submodules unnecessary: by FetchContent, dependencies can be downloaded and configured at the cmake configuration stage, and URLs and commit hash/tag can be flexibly adjusted based on cmake options.

    There are NOT a lot of dependencies in kvrocks, and I think using FetchContent to manage dependencies as opposed to ExternalProject will improve the development experience. Of course, introducing a dependency manager such as vcpkg or conan may be a good option, but it introduces additional complexity in development and build management, so there is a trade-off.

    I'm happy to provide PRs if that sounds like a good idea.

    @tisonkun

    enhancement 
    opened by PragmaTwice 13
  • [BUG] Kvrocks could not swich databases

    [BUG] Kvrocks could not swich databases

    Describe the bug Kvrocks could not swich databases, any database key-value changes effect to all-other databases;

    • OS: Ununtu 20.04 /docker
    • Version 2.06/2.05

    To Reproduce Steps to reproduce the behavior:

    1. select 1 -> set t 1 -> get t -> "1"
    2. select 2 -> get t -> "1"

    Expected behavior database-1 key-value should not exist in database-2

    127.0.0.1:6666> info
    # Server
    version:2.0.6
    git_sha1:7c23f2fe
    os:Linux 5.11.0-36-generic x86_64
    gcc_version:7.5.0
    arch_bits:64
    process_id:1
    tcp_port:6666
    uptime_in_seconds:1277
    uptime_in_days:0
    # Clients
    maxclients:10000
    connected_clients:2
    monitor_clients:0
    # Memory
    used_memory_rss:26529792
    used_memory_human:25.30M
    used_memory_lua:36864
    used_memory_lua_human:36.00K
    # Persistence
    loading:0
    bgsave_in_progress:0
    last_bgsave_time:-1
    last_bgsave_status:ok
    last_bgsave_time_sec:-1
    # Stats
    total_connections_received:514
    total_commands_processed:3090
    instantaneous_ops_per_sec:1
    total_net_input_bytes:76505
    total_net_output_bytes:3490146
    instantaneous_input_kbps:0
    instantaneous_output_kbps:5
    sync_full:0
    sync_partial_ok:0
    sync_partial_err:0
    pubsub_channels:0
    pubsub_patterns:0
    # Replication
    role:master
    connected_slaves:0
    master_repl_offset:1175757
    # CPU
    used_cpu_sys:0
    used_cpu_user:1
    # Commandstats
    cmdstat_auth:calls=514,usec=1788,usec_per_call=3
    cmdstat_client:calls=511,usec=747,usec_per_call=1
    cmdstat_command:calls=3,usec=1770,usec_per_call=590
    cmdstat_config:calls=510,usec=82148,usec_per_call=161
    cmdstat_get:calls=4,usec=148,usec_per_call=37
    cmdstat_info:calls=515,usec=373014,usec_per_call=724
    cmdstat_keys:calls=5,usec=1709,usec_per_call=341
    cmdstat_select:calls=6,usec=16,usec_per_call=2
    cmdstat_set:calls=2,usec=268,usec_per_call=134
    cmdstat_slowlog:calls=1020,usec=3864,usec_per_call=3
    # Keyspace
    # Last scan db time: Thu Jan  1 00:00:00 1970
    db0:keys=0,expires=0,avg_ttl=0,expired=0
    sequence:1175757
    used_db_size:51464
    max_db_size:0
    used_percent: 0%
    disk_capacity:490651459584
    used_disk_size:111368089600
    used_disk_percent: 22%
    # RocksDB
    estimate_keys[default]:0
    block_cache_usage[default]:4256
    block_cache_pinned_usage[default]:480
    index_and_filter_cache_usage[default]:0
    level0_file_limit_slowdown[default]:0
    level0_file_limit_stop[default]:0
    pending_compaction_bytes_slowdown[default]:0
    pending_compaction_bytes_stop[default]:0
    memtable_count_limit_slowdown[default]:0
    memtable_count_limit_stop[default]:0
    estimate_keys[metadata]:4
    block_cache_usage[metadata]:4256
    block_cache_pinned_usage[metadata]:480
    index_and_filter_cache_usage[metadata]:224
    level0_file_limit_slowdown[metadata]:0
    level0_file_limit_stop[metadata]:0
    pending_compaction_bytes_slowdown[metadata]:0
    pending_compaction_bytes_stop[metadata]:0
    memtable_count_limit_slowdown[metadata]:0
    memtable_count_limit_stop[metadata]:0
    estimate_keys[zset_score]:0
    block_cache_usage[zset_score]:4256
    block_cache_pinned_usage[zset_score]:480
    index_and_filter_cache_usage[zset_score]:0
    level0_file_limit_slowdown[zset_score]:0
    level0_file_limit_stop[zset_score]:0
    pending_compaction_bytes_slowdown[zset_score]:0
    pending_compaction_bytes_stop[zset_score]:0
    memtable_count_limit_slowdown[zset_score]:0
    memtable_count_limit_stop[zset_score]:0
    estimate_keys[pubsub]:0
    block_cache_usage[pubsub]:432
    block_cache_pinned_usage[pubsub]:0
    index_and_filter_cache_usage[pubsub]:0
    level0_file_limit_slowdown[pubsub]:0
    level0_file_limit_stop[pubsub]:0
    pending_compaction_bytes_slowdown[pubsub]:0
    pending_compaction_bytes_stop[pubsub]:0
    memtable_count_limit_slowdown[pubsub]:0
    memtable_count_limit_stop[pubsub]:0
    estimate_keys[propagate]:0
    block_cache_usage[propagate]:0
    block_cache_pinned_usage[propagate]:0
    index_and_filter_cache_usage[propagate]:0
    level0_file_limit_slowdown[propagate]:0
    level0_file_limit_stop[propagate]:0
    pending_compaction_bytes_slowdown[propagate]:0
    pending_compaction_bytes_stop[propagate]:0
    memtable_count_limit_slowdown[propagate]:0
    memtable_count_limit_stop[propagate]:0
    all_mem_tables:7343776
    cur_mem_tables:7343776
    snapshots:0
    num_immutable_tables:0
    num_running_flushes:0
    memtable_flush_pending:0
    compaction_pending:1
    num_running_compactions:0
    num_live_versions:5
    num_superversion:7
    num_background_errors:0
    flush_count:0
    compaction_count:2
    put_per_sec:0
    get_per_sec:0
    seek_per_sec:0
    next_per_sec:0
    prev_per_sec:0
    is_bgsaving:no
    is_compacting:no
    
    127.0.0.1:6666> set ttt 1
    OK
    127.0.0.1:6666> get ttt
    "1"
    127.0.0.1:6666> select 2
    OK
    127.0.0.1:6666[2]> get ttt
    "1"
    
    opened by yorkane 13
  • [BUG] Incorrect data after SLAVEOF

    [BUG] Incorrect data after SLAVEOF

    Describe the bug After SLAVEOF, 2 instances can contain different values for a given key.

    To Reproduce Start 2 kvrocks instances and execute the following commands in the respective instances.

    • SERVER2: SLAVEOF SERVER1
    • SERVER2: SLAVEOF NO ONE
    • SERVER2: SET test 222
    • SERVER1: SET test 111
    • SERVER2: SLAVEOF SERVER1
    • SERVER2: GET test
      • Returns "222". However, it should return "111".

    If a new SET command is used on SERVER1, SERVER2 immediately picks up the latest value. This issue doesn't occur if, on the "test" key, SERVER1 executed multiple SET commands while SERVER2 was not a slave.

    Expected behavior After SLAVEOF command, SERVER2 should always contain identical data to SERVER1.

    bug 
    opened by DenizPiri 11
  • [BUG] It is wrong to rewrite config `rename-command` and lead kvrocks to fail to restart

    [BUG] It is wrong to rewrite config `rename-command` and lead kvrocks to fail to restart

    Describe

    If your kvrocks config file has multiple rename-command, kvrocks would generate duplicate configuration items after executing config rewrite, whatever automatically or manually, and leads kvrocks to fail to restart.

    It is introduced in #272

    one example

    rename-command debug                               290033_dont_run_debug_rudely
    rename-command bgsave                              290033_dont_run_bgsave_rudely
    rename-command shutdown                            290033_dont_run_shutdown_rudely
    rename-command keys                                290033_dont_run_keys_rudely
    rename-command debug                               290033_dont_run_debug_rudely
    

    there are two debug rename-commands, so there will be the following error logs.

    * Failed to load config, err: at line: #L19, err: No such command in rename-command
    

    How to fix

    • you can find the duplicate configuration items, and remove duplicate items, only keep one remame-command for one command.
    • of course, you can cherry-pick this commit https://github.com/KvrocksLabs/kvrocks/commit/6c8534a17f056f06faa09bfeada3f92df686975a (#429 )
    • use 2.0.6 version (coming soon)
    opened by iushas 11
  • CMake: support ninja as a build backend

    CMake: support ninja as a build backend

    Close #621.

    Benchmark between make and ninja build for kvrocks can be found in #625. TL;DR: The benchmark shows that ninja is slightly slower than make build in clean build, but has huge advantages in incremental build, which is very friendly to kvrocks developers who need to do frequent incremental builds.

    So in this PR, kvrocks still use make by default, but the problem that blocks ninja backend to successfully build in cmake is fixed and now build.sh has a --ninja option to enable ninja.

    opened by PragmaTwice 10
  • BUG: Use after free in CommandBPop reported by ASan

    BUG: Use after free in CommandBPop reported by ASan

    Search before asking

    • [X] I had searched in the issues and found no similar issues.

    Version

    unstable

    Minimal reproduce step

    ./build.sh build -DENABLE_ASAN=ON
    cd tests/tcl
    ./runtest --dont-clean
    grep Sanitizer tests/tmp -r
    

    This bug is not 100% reproducible and may require multiple test executions before it appears.

    What did you expect to see?

    No ASan report.

    What did you see instead?

    =================================================================
    ==1872112==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000014070 at pc 0x55d1a126595a bp 0x7fa436fee820 sp 0x7fa436fee810
    READ of size 8 at 0x608000014070 thread T98
        #0 0x55d1a1265959 in Redis::CommandBPop::TryPopFromList() /home/twice/incubator-kvrocks/src/redis_cmd.cc:1568
        #1 0x55d1a1266851 in Redis::CommandBPop::WriteCB(bufferevent*, void*) /home/twice/incubator-kvrocks/src/redis_cmd.cc:1590
        #2 0x55d1a190f519 in bufferevent_run_deferred_callbacks_unlocked /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/bufferevent.c:215
        #3 0x55d1a19169ac in event_process_active_single_queue /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/event.c:1726
        #4 0x55d1a1917406 in event_process_active /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/event.c:1789
        #5 0x55d1a1917406 in event_base_loop /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/event.c:2012
        #6 0x55d1a14ffe9f in Worker::Run(std::thread::id) /home/twice/incubator-kvrocks/src/worker.cc:234
        #7 0x55d1a150017d in operator() /home/twice/incubator-kvrocks/src/worker.cc:435
        #8 0x55d1a150017d in __invoke_impl<void, WorkerThread::Start()::<lambda()> > /usr/include/c++/11.2.0/bits/invoke.h:61
        #9 0x55d1a150017d in __invoke<WorkerThread::Start()::<lambda()> > /usr/include/c++/11.2.0/bits/invoke.h:96
        #10 0x55d1a150017d in _M_invoke<0> /usr/include/c++/11.2.0/bits/std_thread.h:253
        #11 0x55d1a150017d in operator() /usr/include/c++/11.2.0/bits/std_thread.h:260
        #12 0x55d1a150017d in _M_run /usr/include/c++/11.2.0/bits/std_thread.h:211
        #13 0x55d1a19fa4a3 in execute_native_thread_routine (/home/twice/incubator-kvrocks/tests/tcl/redis-server+0xacd4a3)
        #14 0x7fa440e6b5c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1)
        #15 0x7fa440ef0583 in __clone (/usr/lib/libc.so.6+0x112583)
    
    0x608000014070 is located 80 bytes inside of 96-byte region [0x608000014020,0x608000014080)
    freed by thread T98 here:
        #0 0x7fa4411a48d9 in operator delete(void*, unsigned long) /usr/src/debug/gcc/libsanitizer/asan/asan_new_delete.cpp:172
        #1 0x55d1a1424770 in std::default_delete<Redis::Commander>::operator()(Redis::Commander*) const /usr/include/c++/11.2.0/bits/unique_ptr.h:85
        #2 0x55d1a1424770 in std::__uniq_ptr_impl<Redis::Commander, std::default_delete<Redis::Commander> >::reset(Redis::Commander*) /usr/include/c++/11.2.0/bits/unique_ptr.h:182
        #3 0x55d1a1424770 in std::__uniq_ptr_impl<Redis::Commander, std::default_delete<Redis::Commander> >::operator=(std::__uniq_ptr_impl<Redis::Commander, std::default_delete<Redis::Commander> >&&) /usr/include/c++/11.2.0/bits/unique_ptr.h:167
        #4 0x55d1a1424770 in std::__uniq_ptr_data<Redis::Commander, std::default_delete<Redis::Commander>, true, true>::operator=(std::__uniq_ptr_data<Redis::Commander, std::default_delete<Redis::Commander>, true, true>&&) /usr/include/c++/11.2.0/bits/unique_ptr.h:212
        #5 0x55d1a1424770 in std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >::operator=(std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >&&) /usr/include/c++/11.2.0/bits/unique_ptr.h:371
        #6 0x55d1a1424770 in Server::LookupAndCreateCommand(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >*) /home/twice/incubator-kvrocks/src/server.cc:1340
        #7 0x55d1a12c65ad in Redis::Connection::ExecuteCommands(std::vector<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::allocator<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > const&) /home/twice/incubator-kvrocks/src/redis_connection.cc:330
        #8 0x55d1a12cef3f in Redis::Connection::OnRead(bufferevent*, void*) /home/twice/incubator-kvrocks/src/redis_connection.cc:89
        #9 0x55d1a190f4ad in bufferevent_run_deferred_callbacks_unlocked /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/bufferevent.c:208
    
    previously allocated by thread T98 here:
        #0 0x7fa4411a3811 in operator new(unsigned long) /usr/src/debug/gcc/libsanitizer/asan/asan_new_delete.cpp:99
        #1 0x55d1a11c3c56 in Redis::redisCommandTable::{lambda()#81}::operator()() const /home/twice/incubator-kvrocks/src/redis_cmd.cc:4807
        #2 0x55d1a11c3c56 in std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> > std::__invoke_impl<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >, Redis::redisCommandTable::{lambda()#81}&>(std::__invoke_other, Redis::redisCommandTable::{lambda()#81}&) /usr/include/c++/11.2.0/bits/invoke.h:61
        #3 0x55d1a11c3c56 in std::enable_if<is_invocable_r_v<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >, Redis::redisCommandTable::{lambda()#81}&>, std::enable_if>::type std::__invoke_r<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >, Redis::redisCommandTable::{lambda()#81}&>(std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >&&, (Redis::redisCommandTable::{lambda()#81}&)...) /usr/include/c++/11.2.0/bits/invoke.h:116
        #4 0x55d1a11c3c56 in std::_Function_handler<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> > (), Redis::redisCommandTable::{lambda()#81}>::_M_invoke(std::_Any_data const&) /usr/include/c++/11.2.0/bits/std_function.h:292
        #5 0x55d1a142439e in std::function<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> > ()>::operator()() const /usr/include/c++/11.2.0/bits/std_function.h:560
        #6 0x55d1a142439e in Server::LookupAndCreateCommand(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >*) /home/twice/incubator-kvrocks/src/server.cc:1340
        #7 0x55d1a12c65ad in Redis::Connection::ExecuteCommands(std::vector<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::allocator<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > const&) /home/twice/incubator-kvrocks/src/redis_connection.cc:330
        #8 0x55d1a12cef3f in Redis::Connection::OnRead(bufferevent*, void*) /home/twice/incubator-kvrocks/src/redis_connection.cc:89
        #9 0x55d1a190f4ad in bufferevent_run_deferred_callbacks_unlocked /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/bufferevent.c:208
    
    Thread T98 created by T0 here:
        #0 0x7fa441143eb7 in __interceptor_pthread_create /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:216
        #1 0x55d1a19fa78a in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/home/twice/incubator-kvrocks/tests/tcl/redis-server+0xacd78a)
        #2 0x55d1a1443c6b in Server::Start() /home/twice/incubator-kvrocks/src/server.cc:125
        #3 0x55d1a1086cf5 in main /home/twice/incubator-kvrocks/src/main.cc:354
        #4 0x7fa440e0b30f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f)
    
    SUMMARY: AddressSanitizer: heap-use-after-free /home/twice/incubator-kvrocks/src/redis_cmd.cc:1568 in Redis::CommandBPop::TryPopFromList()
    Shadow bytes around the buggy address:
      0x0c107fffa7b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c107fffa7c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c107fffa7d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c107fffa7e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c107fffa7f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    =>0x0c107fffa800: fa fa fa fa fd fd fd fd fd fd fd fd fd fd[fd]fd
      0x0c107fffa810: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c107fffa820: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
      0x0c107fffa830: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c107fffa840: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
      0x0c107fffa850: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07 
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==1872112==ABORTING
    

    Anything Else?

    No response

    Are you willing to submit a PR?

    • [ ] I'm willing to submit a PR!
    bug 
    opened by PragmaTwice 3
  • ✨ feat: luajit replace lua

    ✨ feat: luajit replace lua

    1. Luajit replace lua
    2. rm test case "cmsgpack can pack and unpack circular references". Becase luajit's core code diffrent from lua. Also, different compilers produce different code execution effects (GCC&clang on Ubuntu).
    opened by xiaobiaozhao 2
  • Automatically publish the nightly version of kvrocks to docker hub in daily CI

    Automatically publish the nightly version of kvrocks to docker hub in daily CI

    Search before asking

    • [X] I had searched in the issues and found no similar issues.

    Motivation

    Refer to #689 and #691, since we build the nightly version of the docker image we can automatically publish it to some registries like the docker hub, e.g. some tags like nightly, nightly-20220629.

    Solution

    No response

    Are you willing to submit a PR?

    • [ ] I'm willing to submit a PR!
    enhancement 
    opened by PragmaTwice 2
  • Exclude unstable branch while canceling previous workflows

    Exclude unstable branch while canceling previous workflows

    Search before asking

    • [X] I had searched in the issues and found no similar issues.

    Motivation

    image

    As shown in the figure above, the retrying workflow was canceled due to a new commit. We may retry the kvrocks ci action of some commits in the unstable branch. At this time, if the unstable branch can only run one workflow at the same time, it will cause some trouble.

    Solution

    No response

    Are you willing to submit a PR?

    • [ ] I'm willing to submit a PR!
    enhancement 
    opened by PragmaTwice 5
  • Use `unique_ptr` to eliminate some trivial manual deallocation

    Use `unique_ptr` to eliminate some trivial manual deallocation

    Search before asking

    • [X] I had searched in the issues and found no similar issues.

    Motivation

    There are many manual memory (or other system resources like file) allocation and deallocation such as new/delete in kvrocks.

    Memory management is a complex topic (after RC/tracing GC is involved quickly) and there are some smart pointers like unique_ptr, shared_ptr in C++ to abstract a specific way to manage resources.

    For some trivial scenario which do not require to share objects, we can do it via unique_ptr, which is a zero-overhead abstraction.

    We can switch some manual deallocations to unique_ptr to make the code more maintainable and less prone to memory leaks.

    NOTE: We try to use unique_ptr instead of creating our own RAII class types because unique_ptr has good semantics: it prohibits copy construction and only allows move construction, which keeps us from accidentally sharing objects and causing Use After Free or Double Free. (And we should carefully to retrieve raw pointers via get() by checking that the callee do not require ownership)

    Solution

    No response

    Are you willing to submit a PR?

    • [ ] I'm willing to submit a PR!
    enhancement help wanted 
    opened by PragmaTwice 2
Releases(v2.0.6)
  • v2.0.6(Jan 28, 2022)

    Thanks ChrisZMF who implements slot-based data migration for Kvrocks cluster mode, this authentically makes kvrocks cluster scalable. Even though it already is used in production environments, you still should be cautious of it.

    Here is a comprehensive list of changes in this release compared to 2.0.5

    New features

    • Support slot-based data migration (#430) Biggest feature of this version, authentically makes kvrocks scalable, user can use CLUSTERX MIGRATE command to migrate slot and use CLUSTERX SETSLOT to change cluster slot distribution after finishing migration.
    • Make level0_file_num_compaction_trigger of RocksDB configurable (#455)
    • Support CLUSTERX SETSLOT command (#463)

    Improvements

    • Support executing INFO command during restoring database (#375)
    • Auto Rewrite the config after the namespace was updated (#423)
    • Support to convert sortedint type to zset type in kvrocks2redis (#431)
    • Optimize the seek of RocksDB iterators (#438)
    • Make the bitmap range the same with that of Redis (#465)
    • Support GET command on bitmap type (#447)
    • Support to use row cache of RocksDB (#407)

    Bug fixes

    • Fix the parameter blob_garbage_collection_age_cutoff would be always zero (#425)
    • Fix config rewrite overwrite the first rename-command with the last one (#429)
    • Fix zrevrangebyscore for multi members with the same max score (#434)
    • Fix data race in SetCurrentConnection (#440)
    • Fix failing full replication when enabling slave-empty-db-before-fullsync (#446)
    • Fix memory leak when full replication (#461) Maybe leak dozen kilobytes per full replication
    • Fix the bitmap range is smaller than that of Redis (#465)
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.6-release.x86_64.rpm(129.69 MB)
    kvrocks_2.0.6-release_amd64.deb(118.73 MB)
  • v2.0.5(Dec 7, 2021)

    Here is a comprehensive list of changes in this release compared to 2.0.4

    New features

    • Implement zrevrangebylex command (#381)

    • Use some rocksdb new feature (#395)

      • Support blob db (key-value separation) which requires RocksDB version greater than 6.18
      • Enable partitioned index/filter for SST and hash index for data block
      • Enable whole key filter for memtable
      • Remove dynamic resize block and SST
      • Only apply compression to SST which level is more than 2

      Please note that, if your rocksdb version is lower than 6.6.0, it can't reload the data of new version, you should update cautiously.

    • Implement CAS and CAD commands for string type (#415)

    • Support to use row cache of RocksDB (#407)

    Improvements

    • Add rocksdb ops statistics into the info command (#387) These statistics can more truly reflect the load of rocksdb.
    • Add the maxclients to the info command output (#391)
    • Add write stall/stop stats to info command output (#396)
    • Make kvrocks2redis work (#404)
    • Adjust the max open file limit according to the max clients and rocksdb max open files (#418)

    Related projects

    • Kvrocks exporter, a fork of oliver006/redis_exporter to export the kvrocks metrics, https://github.com/KvrocksLabs/kvrocks_exporter
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.5-release.x86_64.rpm(127.80 MB)
    kvrocks_2.0.5-release_amd64.deb(116.75 MB)
  • v2.0.4(Nov 2, 2021)

    Here is a comprehensive list of changes in this release compared to 2.0.3

    New features

    • Support LUA script commands (#363) (#369) Currently, support EVAL, EVALSHA, and SCRIPT commands, but SCRIPT KILL and DEBUG subcommands are not supported.

    Improvements

    • Allow metadata and subkey CF to share a single block cache (#365) (#373)
    • Add master_repl_offset in INFO command output (#374)

    Bugfixes

    • Fixed dynamic resize target_file_size_base bug (#368) Before we only apply this strategy to the default CF(i.e. subkey CF)
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.4-release.x86_64.rpm(127.47 MB)
    kvrocks_2.0.4-release_amd64.deb(116.46 MB)
  • v2.0.3(Sep 13, 2021)

    If your kvrocks instances are 2.0.2 and use redis-sentinel to implement HA, you should upgrade urgently, please see #352 for details.

    Here is a comprehensive list of changes in this release compared to 2.0.2

    Improvements

    • Use the rocksdb MultiGet interface in the MGET command (#331)
    • Avoid useless space allocation in SetBit (#338)
    • Auto set master/replica replication by cluster topology (#356)
    • Support client kill type subcommands to adapt redis-sentinel (#352) (#361) From 2.0.2, we support MULTI-EXEC, but don't support client kill type $type, so MULTI-EXEC will be rollbacked when failing to parse unknown kill type in client command, and cause redis-sentinel fails to failover.

    Bugfixes

    • Fix SETNX and MSETNX commands can't guarantee atomicity (#337)
    • Fix the replicas won't reconnect when failing to connect with master (#344)
    • Fix master pauses synchronizing with replicas (#347)
    • Fix wrong mater port if enabling master-use-repl-port (#360)
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.3-release.x86_64.rpm(126.69 MB)
    kvrocks_2.0.3-release_amd64.deb(115.68 MB)
  • v2.0.2(Jul 19, 2021)

    We design new cluster solution for kvrocks, but it is experimental, don't recommend to deploy in production environments. And because in cluster mode, to be efficient to migrate keys based on slot when scale, we encode key with slot id, but in order to be compatible with the old version, we don't do that in standalone mode, so data is not compatible between standalone mode with cluster mode, you must migrate data if you want to change mode, otherwise, kvrocks will make data corrupt. In order to better support the cluster mode, we are developing a meta server to manage kvrocks cluster, coming soon!

    Here is a comprehensive list of changes in this release compared to 2.0.1

    New features

    • Support rename command in config file (#272)
    • Support transaction (#285) Support MULTI, EXEC, DISCARD command, but don't support WATCH and UNWATCH commands.
    • Support the auto-resize-block-and-sst config directive (#289)
    • Support to resize rocksdb.block_size automatically (#294)
    • Support cluster mode (#302) Similar with redis cluster mode, but use CLUSTERX command to set cluster topology (#324).

    Data encoding changes

    • Encode key with slot id (#291) (#330) In cluster mode, to be efficient to migrate keys by slot when scale, we encode key with slot id, but in standalone mode, we don't. That is to say, data is not compatible between standalone mode with cluster mode, you must migrate data if you want to change mode, otherwise, kvrocks will make data corrupt.

    Bug fixes

    • Set TCP_KEEPALIVE for replication connection (#284) Replicas can handle network failure with master, before this commit, network failure may result in unrecoverable replication interrupt.
    • Fix ZSET can't add the same member with different scores (#298)
    • Make CONFIG SET command thread safe (#310)
    • Fix LTRIM only delete first item, LREM can't correctly remove repeated items (#314)

    Improvements

    • HSET command supports to set multiple fields (#274)
    • Allow to use host in the master-slave replication (#301)
    • Adapt redis sentinel ping check (#316)
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.2-release.x86_64.rpm(126.28 MB)
    kvrocks_2.0.2-release_amd64.deb(115.27 MB)
  • v2.0.1(May 18, 2021)

    Here is a comprehensive list of changes in this release compared to 2.0.0

    New features

    • Support COMMAND command (#251)
    • Support to purge backups on fullsync to reduce disk space usage (#245)

    Bug fixes

    • Fix uninitialized Rocksdb.level0_stop_writes_trigger (#236)
    • Fix condition race for ReclaimOldDBPtr (#246)
    • Fix data race for accessing database in some commands (#253) (#261) In spop, zremrangebylex, zremrangebyscore commands, the lock guard for accessing the database may not work actually.
    • Fix returning wrong string range in GETRANGE command (#254)
    • Fix skipping wrong CURRENT file when full synchronization (#260) Resuming broken transfer based files may be not available if replicas skip transferred wrong CURRENT file of RocksDB.

    Improvements

    • Replicas can execute publish command (#238)
    • Optimizations for avoiding unnecessary data copy (#239) (#247)
    • Allow to run the kvrocks without assigning the config file (#264)

    Dependencies

    • Upgrade the jemalloc version to the special commit (#265) To be able to compile on macOS, upgrade jemalloc to the production test commit which mentioned in jemalloc/jemalloc#2060.
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.1-release.x86_64.rpm(125.30 MB)
    kvrocks_2.0.1-release_amd64.deb(114.31 MB)
  • v1.3.3(May 18, 2021)

    Here is a comprehensive list of changes in this release compared to 1.3.2

    Bug fixes

    • Fix rocksdb can't auto resume after no space error (#229)
    • Fix uninitialized Rocksdb.level0_stop_writes_trigger (#236)
    • Fix condition race for ReclaimOldDBPtr (#246)
    • Fix returning wrong string range in GETRANGE command (#254)
    • Fix data race for accessing database in some commands (#253) In spop, zremrangebylex, zremrangebyscore commands, the lock guard for accessing the database may not work actually.

    Improvements

    • Replicas can execute publish command (#238)
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0(Apr 23, 2021)

    Kvrocks 2.0 brings some new useful features that solve actual problems in the production environment. Please note that this version instance still can replicate from old version master but old version replicas can't replicate new version master. Otherwise, we start to use new logo, thanks @tianlingyu1997

    Here is a comprehensive list of changes in this release compared to 1.3.2

    New features

    • Don't use extra port for replication (#200)
    • Use checkpoint to implement data snapshot when full replication (#208) Reduce disk space and bandwidth usage and decrease the time of replicas recovery during full synchronization.
    • Use checkpoint to implement kvrocks backup (#232)
    • Replicas can empty db before fullsync (#233) Replicas use less disks space if enabled, but have a risk to lose data.

    Command changes

    • Add QPS and input/output kbps for INFO command (#213)
    • Add pubsub_patterns for INFO command (#234)
    • Slowlog omits args or argument string if too many or big (#215)

    Cluster

    • Remove codis support (#217) From now, we doesn't support codis, you can adopt pre-sharding to implement cluster, and we are developing new cluster solution that is similar with redis cluster mode, please see #219.

    Bugfixes

    • Fix kvrocks can't auto resume after no space error (#229)

    Dependencies

    • Upgrade rocksdb to latest tag v6.19.3 (#226)
    Source code(tar.gz)
    Source code(zip)
  • v1.3.2(Apr 20, 2021)

    Here is a comprehensive list of changes in this release compared to 1.3.1

    Bug fixes

    • Fix incorrect used_db_size in INFO command (#204)
    • Fix the SST file creation time maybe 0 in some unknown conditions (#211)
    • Fix get corruption writebatch data after psync restart (#221)

    Improvements

    • Optimize TCP keepalive detection time (#201)
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Feb 8, 2021)

    Here is a comprehensive list of changes in this release compared to 1.2.0

    License

    • Change the license from MIT to BSD-3-Clause

    New features

    • Support to set separate log directory to store server logs (#155)
    • Implement PSETEX command (#156)
    • Implement MSETNX command (#165)

    Improvements

    • Improve the precision of the INCRBYFLOAT/HINCRBYFLOAT command to double (#169)
    • Support +INF/-INF for the score of zset type members (#172)
    • The UNSUBSCRIBE and PUNSUBSCRIBE commands support multiple channels or patterns (#174)

    Behavior changes

    • The UNSUBSCRIBE and PUNSUBSCRIBE commands have the same reply as Redis (#174)
    • The same message format as Redis for pattern subscribers when publishing message (#175) Message format is from 'message channel message' to 'pmessage pattern channel message'

    Bug fixes

    • Fix skipping some SST files when picking compaction files (#154)
    • Handle negative expire time in set command to avoid deleting keys (#156)
    • List type command doesn't rewrite other types of data (#157)
    • Only set positive expire time when migrating key (#160)
    • Don't store an empty key for setrange command (#161)
    • Fix SETBIT/GETBIT bit offset out of range (#164)
    • Fix INCRBY/INCRBYFLOAT command operates on wrong value and precision check (#167)
    • Fix HINCRBY/HINCRBYFLOAT command operates on wrong value and precision check (#168)
    • RPOPLPUSH command doesn't operate non list type (#170)
    • LINSERT command returns -1 if don't find index (#170)
    • Fix LTRIM command range calculation when start is a very large negative number (#170)
    • SMOVE command doesn't operate non set type key (#171)
    • Fix ZREMRANGEBYRANK command corrupts the size of zset (#172)
    • SUBSCRIBE and PSUBSCRIBE commands return the sum of subscription and psubscription (#174)
    • Fix PUNSUBSCRIBE command doesn't take effort (#174)
    • Allow setting slowlog-log-slower-than to -1 to disable the slowlog (#178)
    Source code(tar.gz)
    Source code(zip)
  • v1.1.27(Aug 26, 2020)

  • v1.1.6(Feb 28, 2020)

    • Upgrade the libevent to stable 2.1.11
    • FIX: create the backup at the same time may cause a crash while the CreateNewBackupWithMetadata wasn't thread-safe call
    • FIX: some incorrect implements in redis command
    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Aug 23, 2019)

Owner
Kvrocks Labs
Home of Kvrocks
Kvrocks Labs
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB

Facebook 23k Jul 1, 2022
KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language

KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language. It is designed for persistent memory and provides unified APIs for both volatile and persistent scenarios. It also demonstrates several optimization methods for high performance with persistent memory. Besides providing the basic APIs of key-value store, it offers several advanced features, like transaction, snapshot as well.

Persistent Memory Programming 167 Jun 30, 2022
RediSearch is a Redis module that provides querying, secondary indexing, and full-text search for Redis.

A query and indexing engine for Redis, providing secondary indexing, full-text search, and aggregations.

null 3.8k Jun 27, 2022
Nebula Graph is a distributed, fast open-source graph database featuring horizontal scalability and high availability

Nebula Graph is an open-source graph database capable of hosting super large-scale graphs with billions of vertices (nodes) and trillions of edges, with milliseconds of latency. It delivers enterprise-grade high performance to simplify the most complex data sets imaginable into meaningful and useful information.

vesoft inc. 7.6k Jun 24, 2022
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

?? ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

ArangoDB 12.4k Jun 24, 2022
Scylla is the real-time big data database that is API-compatible with Apache Cassandra and Amazon DynamoDB

Scylla is the real-time big data database that is API-compatible with Apache Cassandra and Amazon DynamoDB. Scylla embraces a shared-nothing approach that increases throughput and storage capacity to realize order-of-magnitude performance improvements and reduce hardware costs.

ScyllaDB 8k Jun 24, 2022
FEDB is a NewSQL database optimised for realtime inference and decisioning application

FEDB is a NewSQL database optimised for realtime inference and decisioning applications. These applications put real-time features extracted from multiple time windows through a pre-trained model to evaluate new data to support decision making. Existing in-memory databases cost hundreds or even thousands of milliseconds so they cannot meet the requirements of inference and decisioning applications.

4Paradigm 1.5k Jun 27, 2022
The MongoDB Database

The MongoDB Database

mongodb 22k Jun 28, 2022
Kvrocks is a key-value NoSQL database based on RocksDB and compatible with Redis protocol.

Kvrocks is a key-value NoSQL database based on RocksDB and compatible with Redis protocol.

Bit Leak 1.6k Jun 23, 2022
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB

Facebook 23k Jul 1, 2022
KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language

KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language. It is designed for persistent memory and provides unified APIs for both volatile and persistent scenarios. It also demonstrates several optimization methods for high performance with persistent memory. Besides providing the basic APIs of key-value store, it offers several advanced features, like transaction, snapshot as well.

Persistent Memory Programming 167 Jun 30, 2022
FoundationDB - the open source, distributed, transactional key-value store

FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as

Apple 11.5k Jun 29, 2022
RediSearch is a Redis module that provides querying, secondary indexing, and full-text search for Redis.

A query and indexing engine for Redis, providing secondary indexing, full-text search, and aggregations.

null 3.8k Jun 27, 2022
redis-cpp is a header-only library in C++17 for Redis (and C++11 backport)

redis-cpp - lightweight C++ client library for Redis redis-cpp is a C++17 library for executing Redis commands with support for pipelines and the publ

null 68 Jun 7, 2022
libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with permissive license

One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.

Леонид Юрьев (Leonid Yuriev) 1k Apr 13, 2022
A redis module, similar to redis zset, but you can set multiple scores for each member to support multi-dimensional sorting

TairZset: Support multi-score sorting zset Introduction Chinese TairZset is a data structure developed based on the redis module. Compared with the na

Alibaba 43 Jun 15, 2022
An Embedded NoSQL, Transactional Database Engine

UnQLite - Transactional Embedded Database Engine

PixLab | Symisc Systems 1.7k Jun 23, 2022
Kunlun distributed DBMS is a NewSQL OLTP relational distributed database management system

Kunlun distributed DBMS is a NewSQL OLTP relational distributed database management system. Application developers can use Kunlun to build IT systems that handles terabytes of data, without any effort on their part to implement data sharding, distributed transaction processing, distributed query processing, crash safety, high availability, strong consistency, horizontal scalability. All these powerful features are provided by Kunlun.

zettadb 99 Jun 24, 2022
John Walker 15 Jun 16, 2022
Kreon is a key-value store library optimized for flash-based storage

Kreon is a key-value store library optimized for flash-based storage, where CPU overhead and I/O amplification are more significant bottlenecks compared to I/O randomness.

Computer Architecture and VLSI Systems (CARV) Laboratory 20 Jun 10, 2022
An efficient, small mobile key-value storage framework developed by WeChat. Works on Android, iOS, macOS, Windows, and POSIX.

中文版本请参看这里 MMKV is an efficient, small, easy-to-use mobile key-value storage framework used in the WeChat application. It's currently available on Andr

Tencent 14.7k Jun 27, 2022
Simple constant key/value storage library, for read-heavy systems with infrequent large bulk inserts.

Sparkey is a simple constant key/value storage library. It is mostly suited for read heavy systems with infrequent large bulk inserts. It includes bot

Spotify 965 Jun 19, 2022
BerylDB is a data structure data manager that can be used to store data as key-value entries.

BerylDB is a data structure data manager that can be used to store data as key-value entries. The server allows channel subscription and is optimized to be used as a cache repository. Supported structures include lists, sets, and keys.

BerylDB 195 Jun 24, 2022
Header-only compile time key-value map written in C++20.

C++ Static Map Header-only compile time key-value map written in C++20. Getting Started Simply add the files in your source and #include "@dir/Static_

null 1 Oct 19, 2021
LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. Authors: Sanjay Ghem

Google 29.5k Jun 22, 2022
A high performance, shared memory, lock free, cross platform, single file, no dependencies, C++11 key-value store

SimDB A high performance, shared memory, lock free, cross platform, single file, no dependencies, C++11 key-value store. SimDB is part of LAVA (Live A

null 442 Jun 25, 2022
Modern transactional key-value/row storage library.

Sophia is advanced transactional MVCC key-value/row storage library. How does it differ from other storages? Sophia is RAM-Disk hybrid storage. It is

Dmitry Simonenko 1.8k Jun 25, 2022
Allows to swap the Fn key and left Control key and other tweaks on Macbook Pro and Apple keyboards in GNU/Linux

A patched hid-apple kernel module UPDATE August 2020: swap_fn_leftctrl is now built-in in Linux 5.8 ?? UPDATE Jun 2020: New feature added (swap_fn_f13

Zakhar Semenov 295 Jun 23, 2022
Distributed, Encrypted, Fractured File System - A custom distributed file system written in C with FUSE

A custom FUSE-based filesystem that distributes encrypted shards of data across machines on a local network, allowing those files to be accessible from any machine.

Charles Averill 14 May 18, 2022