Kvrocks is a distributed key value NoSQL database based on RocksDB and compatible with Redis protocol.

Overview

kvrocks_logo

image image GitHub license GitHub stars

Kvrocks is an open-source key-value database which is based on rocksdb and compatible with Redis protocol. Intention to decrease the cost of memory and increase the capability while compared to Redis. The design of replication and storage was inspired by rocksplicator and blackwidow.

Kvrocks has the following key features:

  • Redis protocol, user can use redis client to visit the kvrocks
  • Namespace, similar to redis db but use token per namespace
  • Replication, async replication using binlog like MySQL
  • High Available, supports redis sentinel to failover when master or slave was failed

Thanks for designers @田凌宇 and @范世丽 contribute the kvrocks logo for us.

Who uses kvrocks

Meitu Ctrip
Baidu BaishanCloud

Tickets a pull reqeust to let us known that you're using kvrocks and add your logo to README

Building kvrocks

requirements

  • g++ (required by c++11, version >= 4.8)
  • autoconf automake libtool snappy

Build

NOTE: You should install the snappy first:

# Centos/Redhat
sudo yum install -y epel-release && sudo yum install -y git gcc gcc-c++ make snappy snappy-devel autoconf automake libtool which gtest gtest-devel

# Ubuntu
sudo apt-get install gcc g++ make libsnappy-dev autoconf automake libtool which libgtest-dev

# MACOSX
brew install snappy googletest

It is as simple as:

$ git clone --recursive https://github.com/kvrockslabs/kvrocks.git
$ cd kvrocks
$ make -j4

Running kvrocks

$ ./src/kvrocks -c kvrocks.conf

Running test cases

NOTE: You should install the googletest first

make test

Supported platforms

  • centos 6/7
  • ubuntu
  • macosx

Try kvrocks using Docker

$ docker run -it -p 6666:6666 kvrocks/kvrocks
$ redis-cli -p 6666

127.0.0.1:6666> get a
(nil)

Namespace

namespace was used to isolate data between users. unlike all the redis databases can be visited by requirepass, we use one token per namespace. requirepass was regraded as admin token, only admin token allows to access the namespace command, as well as some commands like config, slaveof, bgsave, etc…

namespace del ns1 OK ">
# add token
127.0.0.1:6666>  namespace add ns1 mytoken
OK

# update token
127.0.0.1:6666> namespace set ns1 new_token
OK

# list namespace
127.0.0.1:6666> namespace get *
1) "ns1"
2) "new_token"
3) "__namespace"
4) "foobared"

# delete namespace
127.0.0.1:6666> namespace del ns1
OK

DOCs

Migrate tools

  • migrate from redis to kvrocks, use redis-migrate-tool which was developed by vipshop
  • migrate from kvrocks to redis. use kvrocks2redis in build dir

Performance

Hardware

  • CPU: 48 cores Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  • Memory: 32 GiB
  • NET: Intel Corporation I350 Gigabit Network Connection
  • DISK: 2TB NVMe Intel SSD DC P460

Benchmark Client: multi-thread redis-benchmark(unstable branch)

1. Commands QPS

kvrocks: workers = 16, benchmark: 8 threads/ 512 conns / 128 payload

latency: 99.9% < 10ms

image

2. QPS on different payloads

kvrocks: workers = 16, benchmark: 8 threads/ 512 conns

latency: 99.9% < 10ms

image

3. QPS on different workers

kvrocks: workers = 16, benchmark: 8 threads/ 512 conns / 128 payload

latency: 99.9% < 10ms

image

License

Kvrocks is under the BSD-3-Clause license. See the LICENSE file for details.

Issues
  • 同步基本不可用

    同步基本不可用

    1. masterA 有数据 slaveA slaveof过来,数据可以同步,flushall masterA的数据, slaveA 没有同步清空掉。都执行过dbsize scan 2)masterB 没数据 slaveB 同步过来up后 ,masterB上执行写数据,数据没有同步到slaveB 3)slaveof后 slave节点上经常会报以下错误, 但这个目录是清空后再启动slave的
    E0216 13:35:53.319345 26935 storage.cc:703] [storage] Failed to delete dir: IO error: file rmdir: ./kvrocksdatamaster/backup/meta: Directory not empty
    E0216 13:35:53.415446 26935 replication.cc:584] [replication] Failed to restore backup while IO error: No such file or directoryWhile opening a file for sequentially reading: ./kvrocksdatamaster/backup/meta/5: No such file or directory
    Segmentation fault (core dumped)
    
    need more information 
    opened by smartlee 23
  • Enhancement/only generate version when needed

    Enhancement/only generate version when needed

    resolved #66

    opened by karelrooted 11
  • deadlock

    deadlock

    目前发现一个deadlock,原始代码没有复现,我改写的代码会出现,大概几天会出现一次,看原始代码觉得可能也有这个隐患,以下是deadlock时候的堆栈,如有不正确,请指正谢谢!

    Thread 7 (Thread 0x7fe5e47f8700 (LWP 97)):
    #0  0x00007fe5f8f4ef4d in __lll_lock_wait () from /lib64/libpthread.so.0
    #1  0x00007fe5f8f4ad02 in _L_lock_791 () from /lib64/libpthread.so.0
    #2  0x00007fe5f8f4ac08 in pthread_mutex_lock () from /lib64/libpthread.so.0
    #3  0x00000000004437d8 in __gthread_mutex_lock(pthread_mutex_t*) ()
    #4  0x0000000000443b10 in std::mutex::lock() ()
    #5  0x00000000004db93a in std::lock_guard<std::mutex>::lock_guard(std::mutex&) ()
    #6  0x00000000004f7731 in Server::UnSubscribeChannel(std::string const&, Redis::Connection*) ()
    #7  0x00000000004a7eee in Redis::Connection::UnSubscribeAll() ()
    #8  0x00000000004a723d in Redis::Connection::~Connection() ()
    #9  0x00000000005223bd in Worker::FreeConnectionByID(int, unsigned long) ()
    #10 0x00000000005235b2 in Worker::KickoutIdleClients(int) ()
    #11 0x000000000052158e in Worker::TimerCB(int, short, void*) ()
    
    Thread 2 (Thread 0x7fe5e7fff700 (LWP 129)):
    #0  0x00007fe5f8f4ef4d in __lll_lock_wait () from /lib64/libpthread.so.0
    #1  0x00007fe5f8f4ad02 in _L_lock_791 () from /lib64/libpthread.so.0
    #2  0x00007fe5f8f4ac08 in pthread_mutex_lock () from /lib64/libpthread.so.0
    #3  0x00000000004437d8 in __gthread_mutex_lock(pthread_mutex_t*) ()
    #4  0x0000000000443b10 in std::mutex::lock() ()
    #5  0x00000000004efc87 in std::unique_lock<std::mutex>::lock() ()
    #6  0x00000000004ee65d in std::unique_lock<std::mutex>::unique_lock(std::mutex&) ()
    #7  0x0000000000522665 in Worker::Reply(int, std::string const&) ()
    #8  0x00000000004f7288 in Server::PublishMessage(std::string const&, std::string const&) ()
    #9  0x000000000048d06f in Redis::CommandPublish::Execute(Server*, Redis::Connection*, std::string*) ()
    
    

    1)一个worker获得了server的锁 pubsub_channels_mu_ 等待worker的锁conns_mu_ 线程2 另外一个定时器触发,到时间清理timeout的链接 线程7 获得了worker的锁conns_mu_在connection析构时会等待server的锁 pubsub_channels_mu_,就卡死在这里

    2) KickoutIdleClients 这个函数的行为不是很理解,按理是清理conns_和monitor_conns_的连接数,但这里面并没有遍历monitor_conns_的map,却在FreeConnectionByID里面去找monitor_conns_,此外这个地方难道不是直接map erase掉conns_和monitor_conns_,并将符合条件的加入到to_be_killed_conns里面更好?这样FreeConnectionByID无需加锁。

    bug 
    opened by smartlee 10
  • 提个bug

    提个bug

    https://github.com/meitu/kvrocks/blob/2ff46ef91d47f37945354d3a9e77ad49f33c4ce0/src/redis_request.cc#L49 这地方不能return ok,否则可能会导致命令断流,解析出错,我这边可以稳定复现,只能break。

    bug need more information 
    opened by smartlee 10
  • Fix data race for accessing database in some commands

    Fix data race for accessing database in some commands

    Describe this PR

    Found several data races with -fsanitize=thread flags

    1. Data race when starting up the server

    Output:

    ==================
    WARNING: ThreadSanitizer: data race (pid=11804)
      Read of size 8 at 0x7b0400004900 by thread T16:
        #0 operator() /home/eagle/github/my/kvrocks/src/worker.cc:362 (kvrocks+0x222b68)
        #1 __invoke_impl<void, WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x222b68)
        #2 __invoke<WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x222b68)
        #3 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x222b68)
        #4 operator() /usr/include/c++/9/thread:251 (kvrocks+0x222b68)
        #5 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x222b68)
        #6 <null> <null> (libstdc++.so.6+0xd6d83)
    
      Previous write of size 8 at 0x7b0400004900 by main thread:
        #0 std::enable_if<std::__and_<std::__not_<std::__is_tuple_like<std::thread::id> >, std::is_move_constructible<std::thread::id>, std::is_move_assignable<std::thread::id> >::value, void>::type std::swap<std::thread::id>(std::thread::id&, std::thread::id&) /usr/include/c++/9/bits/move.h:194 (kvrocks+0x223742)
        #1 std::thread::swap(std::thread&) /usr/include/c++/9/thread:159 (kvrocks+0x223742)
        #2 std::thread::operator=(std::thread&&) /usr/include/c++/9/thread:153 (kvrocks+0x223742)
        #3 WorkerThread::Start() /home/eagle/github/my/kvrocks/src/worker.cc:363 (kvrocks+0x223742)
        #4 Server::Start() /home/eagle/github/my/kvrocks/src/server.cc:60 (kvrocks+0x1e9427)
        #5 __libc_start_main <null> (libc.so.6+0x270b2)
    
      Location is heap block of size 16 at 0x7b0400004900 allocated by main thread:
        #0 operator new(unsigned long) <null> (libtsan.so.0+0x8c012)
        #1 Server::Server(Engine::Storage*, Config*) /home/eagle/github/my/kvrocks/src/server.cc:35 (kvrocks+0x1ed2dd)
        #2 main /home/eagle/github/my/kvrocks/src/main.cc:310 (kvrocks+0x9471b)
    
      Thread T16 'worker' (tid=11882, running) created by main thread at:
        #0 pthread_create <null> (libtsan.so.0+0x5ea99)
        #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
        #2 Server::Start() /home/eagle/github/my/kvrocks/src/server.cc:60 (kvrocks+0x1e9427)
        #3 main /home/eagle/github/my/kvrocks/src/main.cc:317 (kvrocks+0x94839)
    
    SUMMARY: ThreadSanitizer: data race /home/eagle/github/my/kvrocks/src/worker.cc:362 in operator()
    ==================
    

    2. Thread Sanitizer warning in make test

    ==================
    WARNING: ThreadSanitizer: data race (pid=12120)
      Read of size 1 at 0x7b7c00009031 by thread T1:
        #0 memcmp <null> (libtsan.so.0+0x651af)
        #1 rocksdb::Slice::compare(rocksdb::Slice const&) const include/rocksdb/slice.h:250 (unittest+0x56987d)
        #2 Compare util/comparator.cc:27 (unittest+0x56987d)
        #3 Engine::SubKeyFilter::Filter(int, rocksdb::Slice const&, rocksdb::Slice const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const /home/eagle/github/my/kvrocks/src/compact_filter.cc:94 (unittest+0xdb539)
        #4 rocksdb::CompactionFilter::FilterV2(int, rocksdb::Slice const&, rocksdb::CompactionFilter::ValueType, rocksdb::Slice const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const include/rocksdb/compaction_filter.h:167 (unittest+0x5c4d32)
        #5 rocksdb::CompactionIterator::InvokeFilterIfNeeded(bool*, rocksdb::Slice*) db/compaction/compaction_iterator.cc:273 (unittest+0x5c4d32)
    
      Previous write of size 1 at 0x7b7c00009031 by main thread (mutexes: write M668075411208911328):
        #0 memcpy <null> (libtsan.so.0+0x42333)
        #1 memcpy /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34 (unittest+0x3c9539)
        #2 rocksdb::MemTable::Add(unsigned long, rocksdb::ValueType, rocksdb::Slice const&, rocksdb::Slice const&, rocksdb::ProtectionInfoKVOTS<unsigned long> const*, bool, rocksdb::MemTablePostProcessInfo*, void**) db/memtable.cc:555 (unittest+0x3c9539)
        #3 Redis::Set::Overwrite(rocksdb::Slice, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) /home/eagle/github/my/kvrocks/src/redis_set.cc:31 (unittest+0x1c0a0b)
        #4 RedisSetTest_Overwrite_Test::TestBody() ../tests/t_set_test.cc:156 (unittest+0x2931b6)
        #5 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) googletest/src/gtest.cc:2433 (unittest+0x6b99a0)
        #6 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) googletest/src/gtest.cc:2469 (unittest+0x6b99a0)
        #7 __libc_start_main <null> (libc.so.6+0x270b2)
    
      Location is heap block of size 3216 at 0x7b7c00008c00 allocated by main thread:
        #0 operator new(unsigned long) <null> (libtsan.so.0+0x8c012)
        #1 rocksdb::ColumnFamilyData::ConstructNewMemtable(rocksdb::MutableCFOptions const&, unsigned long) db/column_family.cc:1057 (unittest+0x5b30b1)
        #2 Engine::Storage::Open() /home/eagle/github/my/kvrocks/src/storage.cc:227 (unittest+0x222707)
        #3 TestBase::TestBase() ../tests/test_base.h:15 (unittest+0x2958a8)
        #4 RedisSetTest::RedisSetTest() ../tests/t_set_test.cc:7 (unittest+0x2958a8)
        #5 RedisSetTest_Overwrite_Test::RedisSetTest_Overwrite_Test() ../tests/t_set_test.cc:152 (unittest+0x2961a9)
        #6 testing::internal::TestFactoryImpl<RedisSetTest_Overwrite_Test>::CreateTest() /usr/include/gtest/internal/gtest-internal.h:460 (unittest+0x2961a9)
        #7 testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) googletest/src/gtest.cc:2433 (unittest+0x6b9b70)
        #8 testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) googletest/src/gtest.cc:2469 (unittest+0x6b9b70)
        #9 __libc_start_main <null> (libc.so.6+0x270b2)
    
      Mutex M668075411208911328 is already destroyed.
    
      Thread T1 'rocksdb:low0' (tid=12122, running) created by main thread at:
        #0 pthread_create <null> (libtsan.so.0+0x5ea99)
        #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
        #2 Engine::Storage::Open(bool) /home/eagle/github/my/kvrocks/src/storage.cc:165 (unittest+0x2207af)
        #3 Engine::Storage::Open() /home/eagle/github/my/kvrocks/src/storage.cc:227 (unittest+0x222707)
        #4 TestBase::TestBase() ../tests/test_base.h:15 (unittest+0x25bbbd)
        #5 RedisTypeTest::RedisTypeTest() ../tests/t_metadata_test.cc:48 (unittest+0x25bbbd)
        #6 RedisTypeTest_GetMetadata_Test::RedisTypeTest_GetMetadata_Test() ../tests/t_metadata_test.cc:64 (unittest+0x25c249)
        #7 testing::internal::TestFactoryImpl<RedisTypeTest_GetMetadata_Test>::CreateTest() /usr/include/gtest/internal/gtest-internal.h:460 (unittest+0x25c249)
        #8 testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) googletest/src/gtest.cc:2433 (unittest+0x6b9b70)
        #9 testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) googletest/src/gtest.cc:2469 (unittest+0x6b9b70)
        #10 __libc_start_main <null> (libc.so.6+0x270b2)
    
    SUMMARY: ThreadSanitizer: data race (/lib/x86_64-linux-gnu/libtsan.so.0+0x651af) in memcmp
    ==================
    

    3. Thread Sanitizer warning when benchmarking

    Benchmark command redis-benchmark -p 6666 -n 100000 -q

    Full output

    Part of the output:

    ==================
    WARNING: ThreadSanitizer: data race (pid=20423)
      Write of size 8 at 0x7f1282ce71c0 by thread T19:
        #0 localtime <null> (libtsan.so.0+0x4701c)
        #1 operator() /home/eagle/github/kvrocks/src/server.cc:78 (kvrocks+0x15d669)
        #2 __invoke_impl<void, Server::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x15d669)
        #3 __invoke<Server::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x15d669)
        #4 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x15d669)
        #5 operator() /usr/include/c++/9/thread:251 (kvrocks+0x15d669)
        #6 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x15d669)
        #7 <null> <null> (libstdc++.so.6+0xd6d83)
    
      Previous write of size 8 at 0x7f1282ce71c0 by thread T18:
        #0 localtime <null> (libtsan.so.0+0x4701c)
        #1 Server::cron() /home/eagle/github/kvrocks/src/server.cc:463 (kvrocks+0x15ea8a)
        #2 operator() /home/eagle/github/kvrocks/src/server.cc:66 (kvrocks+0x15f3b4)
        #3 __invoke_impl<void, Server::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x15f3b4)
        #4 __invoke<Server::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x15f3b4)
        #5 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x15f3b4)
        #6 operator() /usr/include/c++/9/thread:251 (kvrocks+0x15f3b4)
        #7 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x15f3b4)
        #8 <null> <null> (libstdc++.so.6+0xd6d83)
    
      As if synchronized via sleep:
        #0 usleep <null> (libtsan.so.0+0x62f5b)
        #1 operator() /home/eagle/github/kvrocks/src/server.cc:94 (kvrocks+0x15d719)
        #2 __invoke_impl<void, Server::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x15d719)
        #3 __invoke<Server::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x15d719)
        #4 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x15d719)
        #5 operator() /usr/include/c++/9/thread:251 (kvrocks+0x15d719)
        #6 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x15d719)
        #7 <null> <null> (libstdc++.so.6+0xd6d83)
    
      Location is global '<null>' at 0x000000000000 (libc.so.6+0x0000001f11c0)
    
      Thread T19 'compaction-checker' (tid=20504, running) created by main thread at:
        #0 pthread_create <null> (libtsan.so.0+0x5ea99)
        #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
        #2 main /home/eagle/github/kvrocks/src/main.cc:317 (kvrocks+0x8b0d9)
    
      Thread T18 'server-cron' (tid=20503, running) created by main thread at:
        #0 pthread_create <null> (libtsan.so.0+0x5ea99)
        #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
        #2 main /home/eagle/github/kvrocks/src/main.cc:317 (kvrocks+0x8b0d9)
    
    SUMMARY: ThreadSanitizer: data race (/lib/x86_64-linux-gnu/libtsan.so.0+0x4701c) in localtime
    ==================
    ==================
    WARNING: ThreadSanitizer: data race (pid=14056)
      Read of size 1 at 0x7b2400014ebc by thread T9:
        #0 memcpy <null> (libtsan.so.0+0x42333)
        #1 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) <null> (libstdc++.so.6+0x142ec3)
        #2 Redis::Set::Take(rocksdb::Slice const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*, int, bool) /home/eagle/github/my/kvrocks/src/redis_set.cc:201 (kvrocks+0x19d8ea)
        #3 Redis::CommandSPop::Execute(Server*, Redis::Connection*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/eagle/github/my/kvrocks/src/redis_cmd.cc:1822 (kvrocks+0x14ffb9)
        #4 Redis::Request::ExecuteCommands(Redis::Connection*) /home/eagle/github/my/kvrocks/src/redis_request.cc:211 (kvrocks+0x195d1d)
        #5 Redis::Connection::OnRead(bufferevent*, void*) /home/eagle/github/my/kvrocks/src/redis_connection.cc:64 (kvrocks+0x16a3f9)
        #6 bufferevent_run_deferred_callbacks_unlocked /home/eagle/github/my/kvrocks/external/libevent/bufferevent.c:208 (kvrocks+0x250401)
        #7 operator() /home/eagle/github/my/kvrocks/src/worker.cc:362 (kvrocks+0x222b73)
        #8 __invoke_impl<void, WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x222b73)
        #9 __invoke<WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x222b73)
        #10 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x222b73)
        #11 operator() /usr/include/c++/9/thread:251 (kvrocks+0x222b73)
        #12 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x222b73)
        #13 <null> <null> (libstdc++.so.6+0xd6d83)
    
      Previous write of size 8 at 0x7b2400014eb8 by thread T13:
        #0 operator new(unsigned long) <null> (libtsan.so.0+0x8c012)
        #1 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) <null> (libstdc++.so.6+0x142e7d)
        #2 Redis::CommandSPop::Execute(Server*, Redis::Connection*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/eagle/github/my/kvrocks/src/redis_cmd.cc:1822 (kvrocks+0x14ffb9)
        #3 Redis::Request::ExecuteCommands(Redis::Connection*) /home/eagle/github/my/kvrocks/src/redis_request.cc:211 (kvrocks+0x195d1d)
        #4 Redis::Connection::OnRead(bufferevent*, void*) /home/eagle/github/my/kvrocks/src/redis_connection.cc:64 (kvrocks+0x16a3f9)
        #5 bufferevent_run_deferred_callbacks_unlocked /home/eagle/github/my/kvrocks/external/libevent/bufferevent.c:208 (kvrocks+0x250401)
        #6 operator() /home/eagle/github/my/kvrocks/src/worker.cc:362 (kvrocks+0x222b73)
        #7 __invoke_impl<void, WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x222b73)
        #8 __invoke<WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x222b73)
        #9 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x222b73)
        #10 operator() /usr/include/c++/9/thread:251 (kvrocks+0x222b73)
        #11 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x222b73)
        #12 <null> <null> (libstdc++.so.6+0xd6d83)
    
      Location is heap block of size 133 at 0x7b2400014eb0 allocated by thread T13:
        #0 operator new(unsigned long) <null> (libtsan.so.0+0x8c012)
        #1 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) <null> (libstdc++.so.6+0x142e7d)
        #2 Redis::CommandSPop::Execute(Server*, Redis::Connection*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/eagle/github/my/kvrocks/src/redis_cmd.cc:1822 (kvrocks+0x14ffb9)
        #3 Redis::Request::ExecuteCommands(Redis::Connection*) /home/eagle/github/my/kvrocks/src/redis_request.cc:211 (kvrocks+0x195d1d)
        #4 Redis::Connection::OnRead(bufferevent*, void*) /home/eagle/github/my/kvrocks/src/redis_connection.cc:64 (kvrocks+0x16a3f9)
        #5 bufferevent_run_deferred_callbacks_unlocked /home/eagle/github/my/kvrocks/external/libevent/bufferevent.c:208 (kvrocks+0x250401)
        #6 operator() /home/eagle/github/my/kvrocks/src/worker.cc:362 (kvrocks+0x222b73)
        #7 __invoke_impl<void, WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (kvrocks+0x222b73)
        #8 __invoke<WorkerThread::Start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (kvrocks+0x222b73)
        #9 _M_invoke<0> /usr/include/c++/9/thread:244 (kvrocks+0x222b73)
        #10 operator() /usr/include/c++/9/thread:251 (kvrocks+0x222b73)
        #11 _M_run /usr/include/c++/9/thread:195 (kvrocks+0x222b73)
        #12 <null> <null> (libstdc++.so.6+0xd6d83)
    
      Thread T9 'worker' (tid=14127, running) created by main thread at:
        #0 pthread_create <null> (libtsan.so.0+0x5ea99)
        #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
        #2 Server::Start() /home/eagle/github/my/kvrocks/src/server.cc:60 (kvrocks+0x1e9427)
        #3 main /home/eagle/github/my/kvrocks/src/main.cc:317 (kvrocks+0x94839)
    
      Thread T13 'worker' (tid=14131, running) created by main thread at:
        #0 pthread_create <null> (libtsan.so.0+0x5ea99)
        #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
        #2 Server::Start() /home/eagle/github/my/kvrocks/src/server.cc:60 (kvrocks+0x1e9427)
        #3 main /home/eagle/github/my/kvrocks/src/main.cc:317 (kvrocks+0x94839)
    
    SUMMARY: ThreadSanitizer: data race (/lib/x86_64-linux-gnu/libtsan.so.0+0x42333) in memcpy
    ==================
    

    Steps to reproduce

    1. add -fsanitize=thread to compiler flags
    2. g++ -v
    Using built-in specs.
    COLLECT_GCC=g++
    COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
    OFFLOAD_TARGET_NAMES=nvptx-none:hsa
    OFFLOAD_TARGET_DEFAULT=1
    Target: x86_64-linux-gnu
    Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
    Thread model: posix
    gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
    
    1. startup command ./src/kvrocks -c kvrocks.conf
    2. test command make test
    3. benchmark command redis-benchmark -p 6666 -n 100000 -q
    4. platform: Windows 10 + WSL2

    Questions:

    1. I am not C++ expert, can we use localtime_r?
    2. The destructor of LockGuard is called at the end of if block, is it standard behavior across all C++ versions?
    bug enhancement release notes need back patch 
    opened by calvinxiao 10
  • Fix LTRIM only delete first item, LREM cannot correctly remove repeated items

    Fix LTRIM only delete first item, LREM cannot correctly remove repeated items

    Fix LTRIM command only can delete the first index item, the metadata is changed, but items are not delete from RocksDB. Cases: rpush listkey 0 1 2 3 4 3 6 7 3 9 -> listkey: 0 1 2 3 4 3 6 7 3 9 ltrim listkey 2 -3 -> listkey: 2 3 4 3 6 7 lrem listkey 3 3 -> listkey should be: 2 4 6 7, but it is: 2 4 6

    Fix LREM cannot remove item correctly from sequence with repeated items, the reason is that the number of deleted elements is not judged, which will cause the repeated items that need to be retained are not copied. Case: rpush listkey a b c d e f a f -> listkey: a b c d e f a f lrem listkey 1 f -> listkey should be: a b c d e a f, but it is: a b c d e a a

    Other fixes: Fix wrong read/write attributes of some commands, such as lrem, linsert, flushdb

    bug release notes need back patch 
    opened by ChrisZMF 9
  • Encode key with slot id

    Encode key with slot id

    Encode key with slot id, so it will be efficient to migrate keys by slot when scale

    Also support CLUSTER KEYSLOT command

    release notes major decision 
    opened by shangxiaoxiong 9
  • build error with master

    build error with master

    Hi,

    I'm getting the following build error when trying to build from git.

    [ 87%] Building CXX object CMakeFiles/kvrocks2redis.dir/tools/kvrocks2redis/parser.cc.o [ 88%] Linking CXX executable kvrocks2redis /usr/bin/ld: cannot find -lstdc++ collect2: error: ld returned 1 exit status make[2]: *** [kvrocks2redis] Error 1 make[1]: *** [CMakeFiles/kvrocks2redis.dir/all] Error 2 make: *** [all] Error 2

    • OS: Red Hat Enterprise Linux Server release 7.2 (Maipo)

    To Reproduce Steps to reproduce the behavior:

    1. sh build.sh build
    opened by tkram01 9
  • Fix:SUBSCRIBE can't update last interaction time

    Fix:SUBSCRIBE can't update last interaction time

    Act like redis,SUBSCRIBE must update last interaction time while a publish message received, otherwise this channel may be closed for timeout reason by default.

    bug release notes need back patch 
    opened by smartlee 9
  • Fix slave can't execute publish command

    Fix slave can't execute publish command

    bug release notes need back patch 
    opened by karelrooted 8
  • Support executing INFO command during restoring database

    Support executing INFO command during restoring database

    TODO:

    • [ ] Must add test cases, since it may cause crash.
    need tests 
    opened by ShooterIT 0
  • Does kvrocks support ROCKSDB's native KEY/VALUE separation, BLOB STORE?

    Does kvrocks support ROCKSDB's native KEY/VALUE separation, BLOB STORE?

    The KVROCKS I use has 100 billion records , VALUE IS 4KB BYTES , Reading speed ( GET ) is very slow

    opened by hangzhouminghe88 6
  • [QUESTION] Clarification on Redis version support

    [QUESTION] Clarification on Redis version support

    I love kvrocks but it would be useful if there was a version vs Redis version support. Whilst supported commands are listed, their Redis version isnt.

    E.g. LPOP in Redis 6.2 supports a [count] parameter, allowing for multiple records to be returned whilst kvrocks returns just a single record.

    I love would LPOP (etc) and all commands to be in line with Redis release but realise that this isn't going to be the case but it would be great if kvrocks published which Redis version each command is compatible with

    opened by stopforumspam 1
  • [QUESTION] What's the recommended filesystem for kvrocks?

    [QUESTION] What's the recommended filesystem for kvrocks?

    We are using kvrocks in production using ext4 filesystem and having around 4000 SST files. We were wondering if XFS would be better for performance. Did you do some specific tests with the different filesystems? Thank you very much.

    opened by adulau 3
  • Upgrade the version of cpplint from 1.5.0 to 1.5.5.

    Upgrade the version of cpplint from 1.5.0 to 1.5.5.

    Here, i've found a misleading cpplint errors generated by cpplint.sh in the progress of CI. And i've made sure that i've format the code by the Google code-style standard, with the necessary double-check by cpplint tool. However, the CI cpplint checking showed that there still exists some style errors in the code. So, i tried to check the difference of cpplint-1.5.0, defaultly used in the CI of kvrocks, between the version 1.5.5. And the difference showed as bellow: image

    Finally, i've checked the releasing logs of cpplint, and found the following results: image

    So, could we upgrade the version of cpplint in the CI of kvrocks to 1.5.5 ?

    I‘ve verified the upgrade was valid in my local repository CI checking progress. image image

    opened by LykxSassinator 0
  • possible to implement redisearch?

    possible to implement redisearch?

    do you have the implementation guideline on how to implement redisearch? written like this? https://kvrocks.medium.com/design-complex-structure-on-rocksdb-993a00fd4f6d?source=post_internal_links---------0----------------------------

    opened by hiqsociety 1
  • any benchmark against redis?

    any benchmark against redis?

    any benchmark against redis?

    opened by hiqsociety 12
  • [NEW]  Allow forcing use the full sync when the lag was too large

    [NEW] Allow forcing use the full sync when the lag was too large

    Background

    Currently, we're using the WAL to do the partial sync like the Redis backlog, as well as the write buffer between worker threads and feed slave thread. Users should tune the WAL size with the key-value size and it's hard in most cases, since the user key-value size may be varied over time. We prefer to give it a generic larger value to the WAL to fit into most cases, but every coin has two sides, it may slow down the partial sync when the key-value size was small and failed to trigger the full sync.

    Proposal

    The core problem was the generic WAL size may cause the partial sync too slowly when the key-value size was small, and it's hard to adjust the WAL size instance by instance since the size may be changed over time. So we need to find another way to work around this case, and the lag threshold should be a good choice since increment logs were synced by sequences.

    TODO

    • [ ] Add the force-full-sync-lag-threshold configuration to specify and enable the full sync lag threshold, default is 0 which means disabled
    • [ ] Check the next seq, if it's a new slave and overrun the full sync lag threshold, let's use the full sync mechanism instead of the partial sync
    • [ ] Add test cases

    cool idea from @Alfejik

    enhancement new feature 
    opened by git-hulk 2
  • Support Watch/Unwatch

    Support Watch/Unwatch

    I would like to use Watch and Unwatch for full transaction support.

    yinqiwen/ardb has a working watch/unwatch impl in case that is useful as a reference.

    https://github.com/yinqiwen/ardb

    new feature 
    opened by snowbldr 1
Releases(v2.0.3)
  • v2.0.3(Sep 13, 2021)

    If your kvrocks instances are 2.0.2 and use redis-sentinel to implement HA, you should upgrade urgently, please see #352 for details.

    Here is a comprehensive list of changes in this release compared to 2.0.2

    Improvements

    • Use the rocksdb MultiGet interface in the MGET command (#331)
    • Avoid useless space allocation in SetBit (#338)
    • Auto set master/replica replication by cluster topology (#356)
    • Support client kill type subcommands to adapt redis-sentinel (#352) (#361) From 2.0.2, we support MULTI-EXEC, but don't support client kill type $type, so MULTI-EXEC will be rollbacked when failing to parse unknown kill type in client command, and cause redis-sentinel fails to failover.

    Bugfixes

    • Fix SETNX and MSETNX commands can't guarantee atomicity (#337)
    • Fix the replicas won't reconnect when failing to connect with master (#344)
    • Fix master pauses synchronizing with replicas (#347)
    • Fix wrong mater port if enabling master-use-repl-port (#360)
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.3-release.x86_64.rpm(126.69 MB)
    kvrocks_2.0.3-release_amd64.deb(115.68 MB)
  • v2.0.2(Jul 19, 2021)

    We design new cluster solution for kvrocks, but it is experimental, don't recommend to deploy in production environments. And because in cluster mode, to be efficient to migrate keys based on slot when scale, we encode key with slot id, but in order to be compatible with the old version, we don't do that in standalone mode, so data is not compatible between standalone mode with cluster mode, you must migrate data if you want to change mode, otherwise, kvrocks will make data corrupt. In order to better support the cluster mode, we are developing a meta server to manage kvrocks cluster, coming soon!

    Here is a comprehensive list of changes in this release compared to 2.0.1

    New features

    • Support rename command in config file (#272)
    • Support transaction (#285) Support MULTI, EXEC, DISCARD command, but don't support WATCH and UNWATCH commands.
    • Support the auto-resize-block-and-sst config directive (#289)
    • Support to resize rocksdb.block_size automatically (#294)
    • Support cluster mode (#302) Similar with redis cluster mode, but use CLUSTERX command to set cluster topology (#324).

    Data encoding changes

    • Encode key with slot id (#291) (#330) In cluster mode, to be efficient to migrate keys by slot when scale, we encode key with slot id, but in standalone mode, we don't. That is to say, data is not compatible between standalone mode with cluster mode, you must migrate data if you want to change mode, otherwise, kvrocks will make data corrupt.

    Bug fixes

    • Set TCP_KEEPALIVE for replication connection (#284) Replicas can handle network failure with master, before this commit, network failure may result in unrecoverable replication interrupt.
    • Fix ZSET can't add the same member with different scores (#298)
    • Make CONFIG SET command thread safe (#310)
    • Fix LTRIM only delete first item, LREM can't correctly remove repeated items (#314)

    Improvements

    • HSET command supports to set multiple fields (#274)
    • Allow to use host in the master-slave replication (#301)
    • Adapt redis sentinel ping check (#316)
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.2-release.x86_64.rpm(126.28 MB)
    kvrocks_2.0.2-release_amd64.deb(115.27 MB)
  • v2.0.1(May 18, 2021)

    Here is a comprehensive list of changes in this release compared to 2.0.0

    New features

    • Support COMMAND command (#251)
    • Support to purge backups on fullsync to reduce disk space usage (#245)

    Bug fixes

    • Fix uninitialized Rocksdb.level0_stop_writes_trigger (#236)
    • Fix condition race for ReclaimOldDBPtr (#246)
    • Fix data race for accessing database in some commands (#253) (#261) In spop, zremrangebylex, zremrangebyscore commands, the lock guard for accessing the database may not work actually.
    • Fix returning wrong string range in GETRANGE command (#254)
    • Fix skipping wrong CURRENT file when full synchronization (#260) Resuming broken transfer based files may be not available if replicas skip transferred wrong CURRENT file of RocksDB.

    Improvements

    • Replicas can execute publish command (#238)
    • Optimizations for avoiding unnecessary data copy (#239) (#247)
    • Allow to run the kvrocks without assigning the config file (#264)

    Dependencies

    • Upgrade the jemalloc version to the special commit (#265) To be able to compile on macOS, upgrade jemalloc to the production test commit which mentioned in jemalloc/jemalloc#2060.
    Source code(tar.gz)
    Source code(zip)
    kvrocks-2.0.1-release.x86_64.rpm(125.30 MB)
    kvrocks_2.0.1-release_amd64.deb(114.31 MB)
  • v1.3.3(May 18, 2021)

    Here is a comprehensive list of changes in this release compared to 1.3.2

    Bug fixes

    • Fix rocksdb can't auto resume after no space error (#229)
    • Fix uninitialized Rocksdb.level0_stop_writes_trigger (#236)
    • Fix condition race for ReclaimOldDBPtr (#246)
    • Fix returning wrong string range in GETRANGE command (#254)
    • Fix data race for accessing database in some commands (#253) In spop, zremrangebylex, zremrangebyscore commands, the lock guard for accessing the database may not work actually.

    Improvements

    • Replicas can execute publish command (#238)
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0(Apr 23, 2021)

    Kvrocks 2.0 brings some new useful features that solve actual problems in the production environment. Please note that this version instance still can replicate from old version master but old version replicas can't replicate new version master. Otherwise, we start to use new logo, thanks @tianlingyu1997

    Here is a comprehensive list of changes in this release compared to 1.3.2

    New features

    • Don't use extra port for replication (#200)
    • Use checkpoint to implement data snapshot when full replication (#208) Reduce disk space and bandwidth usage and decrease the time of replicas recovery during full synchronization.
    • Use checkpoint to implement kvrocks backup (#232)
    • Replicas can empty db before fullsync (#233) Replicas use less disks space if enabled, but have a risk to lose data.

    Command changes

    • Add QPS and input/output kbps for INFO command (#213)
    • Add pubsub_patterns for INFO command (#234)
    • Slowlog omits args or argument string if too many or big (#215)

    Cluster

    • Remove codis support (#217) From now, we doesn't support codis, you can adopt pre-sharding to implement cluster, and we are developing new cluster solution that is similar with redis cluster mode, please see #219.

    Bugfixes

    • Fix kvrocks can't auto resume after no space error (#229)

    Dependencies

    • Upgrade rocksdb to latest tag v6.19.3 (#226)
    Source code(tar.gz)
    Source code(zip)
  • v1.3.2(Apr 20, 2021)

    Here is a comprehensive list of changes in this release compared to 1.3.1

    Bug fixes

    • Fix incorrect used_db_size in INFO command (#204)
    • Fix the SST file creation time maybe 0 in some unknown conditions (#211)
    • Fix get corruption writebatch data after psync restart (#221)

    Improvements

    • Optimize TCP keepalive detection time (#201)
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Feb 8, 2021)

    Here is a comprehensive list of changes in this release compared to 1.2.0

    License

    • Change the license from MIT to BSD-3-Clause

    New features

    • Support to set separate log directory to store server logs (#155)
    • Implement PSETEX command (#156)
    • Implement MSETNX command (#165)

    Improvements

    • Improve the precision of the INCRBYFLOAT/HINCRBYFLOAT command to double (#169)
    • Support +INF/-INF for the score of zset type members (#172)
    • The UNSUBSCRIBE and PUNSUBSCRIBE commands support multiple channels or patterns (#174)

    Behavior changes

    • The UNSUBSCRIBE and PUNSUBSCRIBE commands have the same reply as Redis (#174)
    • The same message format as Redis for pattern subscribers when publishing message (#175) Message format is from 'message channel message' to 'pmessage pattern channel message'

    Bug fixes

    • Fix skipping some SST files when picking compaction files (#154)
    • Handle negative expire time in set command to avoid deleting keys (#156)
    • List type command doesn't rewrite other types of data (#157)
    • Only set positive expire time when migrating key (#160)
    • Don't store an empty key for setrange command (#161)
    • Fix SETBIT/GETBIT bit offset out of range (#164)
    • Fix INCRBY/INCRBYFLOAT command operates on wrong value and precision check (#167)
    • Fix HINCRBY/HINCRBYFLOAT command operates on wrong value and precision check (#168)
    • RPOPLPUSH command doesn't operate non list type (#170)
    • LINSERT command returns -1 if don't find index (#170)
    • Fix LTRIM command range calculation when start is a very large negative number (#170)
    • SMOVE command doesn't operate non set type key (#171)
    • Fix ZREMRANGEBYRANK command corrupts the size of zset (#172)
    • SUBSCRIBE and PSUBSCRIBE commands return the sum of subscription and psubscription (#174)
    • Fix PUNSUBSCRIBE command doesn't take effort (#174)
    • Allow setting slowlog-log-slower-than to -1 to disable the slowlog (#178)
    Source code(tar.gz)
    Source code(zip)
  • v1.1.27(Aug 26, 2020)

  • v1.1.6(Feb 28, 2020)

    • Upgrade the libevent to stable 2.1.11
    • FIX: create the backup at the same time may cause a crash while the CreateNewBackupWithMetadata wasn't thread-safe call
    • FIX: some incorrect implements in redis command
    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Aug 23, 2019)

Owner
Kvrocks Labs
Home of Kvrocks
Kvrocks Labs
RediSearch is a Redis module that provides querying, secondary indexing, and full-text search for Redis.

A query and indexing engine for Redis, providing secondary indexing, full-text search, and aggregations.

null 3.2k Oct 12, 2021
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB

Facebook 21k Oct 15, 2021
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

?? ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

ArangoDB 11.7k Oct 21, 2021
Scylla is the real-time big data database that is API-compatible with Apache Cassandra and Amazon DynamoDB

Scylla is the real-time big data database that is API-compatible with Apache Cassandra and Amazon DynamoDB. Scylla embraces a shared-nothing approach that increases throughput and storage capacity to realize order-of-magnitude performance improvements and reduce hardware costs.

ScyllaDB 7.2k Oct 18, 2021
Nebula Graph is a distributed, fast open-source graph database featuring horizontal scalability and high availability

Nebula Graph is an open-source graph database capable of hosting super large-scale graphs with billions of vertices (nodes) and trillions of edges, with milliseconds of latency. It delivers enterprise-grade high performance to simplify the most complex data sets imaginable into meaningful and useful information.

vesoft inc. 6.6k Oct 16, 2021
The MongoDB Database

The MongoDB Database

mongodb 20.5k Oct 13, 2021
ClickHouse® is a free analytics DBMS for big data

ClickHouse® is an open-source column-oriented database management system that allows generating analytical data reports in real time.

ClickHouse 20k Oct 22, 2021
FEDB is a NewSQL database optimised for realtime inference and decisioning application

FEDB is a NewSQL database optimised for realtime inference and decisioning applications. These applications put real-time features extracted from multiple time windows through a pre-trained model to evaluate new data to support decision making. Existing in-memory databases cost hundreds or even thousands of milliseconds so they cannot meet the requirements of inference and decisioning applications.

4Paradigm 2.4k Oct 16, 2021
KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language

KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language. It is designed for persistent memory and provides unified APIs for both volatile and persistent scenarios. It also demonstrates several optimization methods for high performance with persistent memory. Besides providing the basic APIs of key-value store, it offers several advanced features, like transaction, snapshot as well.

Persistent Memory Programming 56 Oct 14, 2021