A redis module, similar to redis zset, but you can set multiple scores for each member to support multi-dimensional sorting

Related tags

Database TairZset
Overview

TairZset: Support multi-score sorting zset

Introduction Chinese

     TairZset is a data structure developed based on the redis module. Compared with the native zset data structure of redis, TairZset not only has the same rich data interface and high performance as the native zset, but also provides (arbitrary) multi-score sorting capabilities.

Features:

  • Support multi-score(Limited to 255)sorting, and the accuracy of any dimension is not lost
  • Incrby semantics is still supported under multi-score sorting
  • The syntax is similar to redis zset

Sorting rules:

For multi-dimensional scores, the priority of the score on the left is greater than the score on the right. Take a three-dimensional score as an example: score1#score2#score3. When comparing, tairzset will compare score1 first, and only compare score2 when score1 is equal, otherwise it will Take the comparison result of score1 as the comparison result of the entire score. In the same way, score3 will be compared only when score2 is equal.

Application scenario:

  • Sorting among gamers
  • Anchor popularity ranking in live room

Quick start

127.0.0.1:6379> exzadd tairzsetkey 1.1 x 2.2 y
(integer) 2
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "x"
2) "1.1000000000000001"
3) "y"
4) "2.2000000000000002"
127.0.0.1:6379> exzincrby tairzsetkey 2 x 
"3.1000000000000001"
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "y"
2) "2.2000000000000002"
3) "x"
4) "3.1000000000000001"
127.0.0.1:6379> exzadd tairzsetkey 3.3#3.3 z
(error) ERR score is not a valid format
127.0.0.1:6379> del tairzsetkey
(integer) 1
127.0.0.1:6379> exzadd tairzsetkey 1.1#3.3 x 2.2#2.2 y 3.3#1.1 z
(integer) 3
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "x"
2) "1.1000000000000001#3.2999999999999998"
3) "y"
4) "2.2000000000000002#2.2000000000000002"
5) "z"
6) "3.2999999999999998#1.1000000000000001"
127.0.0.1:6379> exzincrby tairzsetkey 2 y 
(error) ERR score is not a valid format
127.0.0.1:6379> exzincrby tairzsetkey 2#0 y 
"4.2000000000000002#2.2000000000000002"
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "x"
2) "1.1000000000000001#3.2999999999999998"
3) "z"
4) "3.2999999999999998#1.1000000000000001"
5) "y"
6) "4.2000000000000002#2.2000000000000002"

Build

mkdir build  
cd build  
cmake ../ && make -j

then the tairzset_module.so library file will be generated in the lib directory

./redis-server --loadmodule /path/to/tairzset_module.so

Test

  1. Modify the path in the tairzset.tcl file in the tests directory to set testmodule [file your_path/tairzset_module.so]
  2. Add the path of the tairzset.tcl file in the tests directory to the all_tests of redis test_helper.tcl
  3. run ./runtest --single tairzset

Client

Java : https://github.com/aliyun/alibabacloud-tairjedis-sdk

API

Ref

Roadmap

  • support multi scores
  • support EXZUNIONSTORE、EXZUNION、EXZSCAN、EXZDIFF、EXZDIFFSTORE、EXZINTER、EXZINTERSTORE、EXZRANDMEMBER、EXZMSCORE

Applicable redis version

redis 5.x 、redis 6.x

Our modules

TairHash: A redis module, similar to redis hash, but you can set expire and version for the field
TairZset: A redis module, similar to redis zset, but you can set multiple scores for each member to support multi-dimensional sorting
TairString: A redis module, similar to redis string, but you can set expire and version for the value. It also provides many very useful commands, such as cas/cad, etc.

Issues
  • Fix an compiler error

    Fix an compiler error

    1. Fixed an compiler error caused by declaring non-static variables in header file and included by external .c files more than once~
    2. Fix a typo in assert()
    /usr/bin/ld: CMakeFiles/tairzset_module.dir/__/dep/skiplist.c.o:/data/alibaba/TairZset/dep/skiplist.h:59: multiple definition of `shared_maxstring'; CMakeFiles/tairzset_module.dir/tairzset.c.o:/data/alibaba/TairZset/dep/skiplist.h:59: first defined here
    /usr/bin/ld: CMakeFiles/tairzset_module.dir/__/dep/skiplist.c.o:/data/alibaba/TairZset/dep/skiplist.h:58: multiple definition of `shared_minstring'; CMakeFiles/tairzset_module.dir/tairzset.c.o:/data/alibaba/TairZset/dep/skiplist.h:58: first defined here
    collect2: error: ld returned 1 exit status
    
    good first issue to-be-merged 
    opened by guoxiangCN 3
  • Feat: implement new command EXZRANDMEMBER.

    Feat: implement new command EXZRANDMEMBER.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: #4

    Implementation reference: Redis 6.2 https://redis.io/commands/zrandmember/

    Command Description

    EXZRANDMEMBER

    EXZRANDMEMBER key [ count [WITHSCORES]] time complexity:O(N) where N is the number of elements returned.

    Command Description:

    When called with just the key argument, return a random element from the sorted set value stored at key.

    If the provided count argument is positive, return an array of distinct elements. The array's length is either count or the sorted set's cardinality (EXZCARD), whichever is lower.

    If called with a negative count, the behavior changes and the command is allowed to return the same element multiple times. In this case, the number of returned elements is the absolute value of the specified count.

    The optional WITHSCORES modifier changes the reply so it includes the respective scores of the randomly selected elements from the sorted set.

    Return value

    Bulk string reply: without the additional count argument, the command returns a Bulk Reply with the randomly selected element, or nil when key does not exist.

    Array reply: when the additional count argument is passed, the command returns an array of elements, or an empty array when key does not exist. If the WITHSCORES modifier is used, the reply is a list elements and their scores from the sorted set.

    TODO

    • [x] Fixing the compatibility problems with Redis 5.0 and 6.0.
    to-be-merged 
    opened by RinChanNOWWW 1
  • Feat: implement new command EXZMSCORE

    Feat: implement new command EXZMSCORE

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: #4

    EXZMSCORE

    EXZMSCORE key member [member ...] time complexity:O(N) where N is the number of members being requested.

    Command Description:

    Returns the scores associated with the specified members in the sorted set stored at key. For every member that does not exist in the sorted set, a nil value is returned.

    Return value

    Array reply: list of scores or nil associated with the specified member values (a double precision floating point number), represented as strings.

    to-be-merged 
    opened by RinChanNOWWW 1
  • Feat: implement new command EXZINTER and EXZINTERSTORE.

    Feat: implement new command EXZINTER and EXZINTERSTORE.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 6.2

    • https://redis.io/commands/zinterstore/
    • https://redis.io/commands/zinter/
    to-be-merged 
    opened by RinChanNOWWW 0
  • Feat: implement new command EXUNIONSTORE and EXUNION.

    Feat: implement new command EXUNIONSTORE and EXUNION.

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 6.2

    • https://redis.io/commands/zunionstore/
    • https://redis.io/commands/zunion/

    TBD

    The behavior when a field in the multi score become NaN.

    to-be-merged 
    opened by RinChanNOWWW 0
  • Feat: implement new command EXZSCAN and fix compile warnings.

    Feat: implement new command EXZSCAN and fix compile warnings.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 5.0 https://redis.io/commands/scan/

    Command Description

    EXZSCAN

    EXZSCAN key cursor [MATCH pattern] [COUNT count] time complexity: O(1) for every call. O(N) for a complete iteration, including enough command calls for the cursor to return back to 0. N is the number of elements inside the collection.

    Command Description:

    Iterates elements of TairZset types and their associated scores.

    EXSCAN is a cursor based iterator. This means that at every call of the command, the server returns an updated cursor that the user needs to use as the cursor argument in the next call. An iteration starts when the cursor is set to 0, and terminates when the cursor returned by the server is 0.

    While EXSCAN does not provide guarantees about the number of elements returned at every iteration, it is possible to empirically adjust the behavior of EXZSCAN using the COUNT option. Basically with COUNT the user specified the amount of work that should be done at every call in order to retrieve elements from the collection. This is just a hint for the implementation, however generally speaking this is what you could expect most of the times from the implementation.

    It is possible to only iterate elements matching a given glob-style pattern, similarly to the behavior of the KEYS command that takes a pattern as only argument. To do so, just append the MATCH arguments at the end of the EXZSCAN command. It is important to note that the MATCH filter is applied after elements are retrieved from the collection, just before returning data to the client. This means that if the pattern matches very little elements inside the collection, EXZSCAN will likely return no elements in most iterations.

    More detail information: https://redis.io/commands/scan/

    Return value

    A two elements multi-bulk reply, where the first element is a string representing an unsigned 64 bit number (the cursor), and the second element is a multi-bulk with an array of elements.

    Others

    • Fix the compile warnings.
    • Add a new data structure list (double linked list) from Redis source codes.
    to-be-merged 
    opened by RinChanNOWWW 0
  • Feat: implement new command EXZDIFF and EXZDIFFSTORE.

    Feat: implement new command EXZDIFF and EXZDIFFSTORE.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 6.2

    • https://redis.io/commands/zdiffstore/
    • https://redis.io/commands/zdiff/
    opened by RinChanNOWWW 0
  • [ASoC]新命令开发支持

    [ASoC]新命令开发支持

    活动官网

    https://asoc2022.opensource.alibaba.com/

    题目描述

    TairZset作为redis原生zset的一个多增增强结构,前期只实现了一些基本的命令, 还有部分命令(使用频率相对较低)没有支持, 为了更好的对齐redis原生zset语义,需要补全这些命令的实现和对应的测试。

    任务目标

    1、开发支持EXZUNIONSTORE、EXZUNION、EXZSCAN、EXZDIFF、EXZDIFFSTORE、EXZINTER、EXZINTERSTORE、EXZRANDMEMBER、EXZMSCORE等确实命令(对齐到redis 5.0版本即可) 2、添加对应的tcl测试

    项目技术要求

    熟悉c、tcl,了解redis zset实现原理

    题目难度

    导师信息

    Qu Dong ,@chenyang8094,ASoC Mentor, redis contributor. [email protected](联系导师请附上个人简历)

    welcome-pr 
    opened by chenyang8094 1
Releases(release-v0.0.1-20211027)
  • release-v0.0.1-20211027(Oct 27, 2021)

Owner
Alibaba
Alibaba Open Source
Alibaba
StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks 2.7k Jun 26, 2022
C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

Simon Ninon 967 Jun 22, 2022
cpp_redis is a C++11 Asynchronous Multi-Platform Lightweight Redis Client

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

CPP Redis 501 Jun 22, 2022
Minimalistic C client for Redis >= 1.2

This Readme reflects the latest changed in the master branch. See v1.0.0 for the Readme and documentation for the latest release (API/ABI history). HI

Redis 5.3k Jun 29, 2022
Modern, asynchronous, and wicked fast C++11 client for Redis

redox Modern, asynchronous, and wicked fast C++11 client for Redis [] (https://travis-ci.org/hmartiro/redox) Redox is a C++ interface to the Redis key

Hayk Martiros 374 Jun 14, 2022
Redis client written in C++

redis-plus-plus Overview Features Branches Installation Install hiredis Install redis-plus-plus Run Tests (Optional) Use redis-plus-plus In Your Proje

null 871 Jul 4, 2022
Aredis - a clean redis C++ client

aredis a clean redis C++ client redis_conn rc; if (rc.connect("127.0.0.1"/*host*/, 6379/*port*/, nullptr/*password*/, 1/*db*/)) { redis_command cmd;

Evan 26 Dec 10, 2021
A C++ Redis client

redis3m A C++ Redis client, born to bring my experience using Redis and C++ on a opensource library. Main goals Provide a simple and efficient wrapper

Luca Marturana 184 Apr 21, 2022
μSQLite library module for MicroPython

μSQLite library module for MicroPython WARNING: This project is in the beta development stage and may be subject to change. usqlite is a SQL database

Elvin Slavik 28 Jun 8, 2022
upstream module that allows nginx to communicate directly with PostgreSQL database.

About ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database. Configuration directives postgres_server

RekGRpth 1 Apr 29, 2022
BerylDB is a data structure data manager that can be used to store data as key-value entries.

BerylDB is a data structure data manager that can be used to store data as key-value entries. The server allows channel subscription and is optimized to be used as a cache repository. Supported structures include lists, sets, and keys.

BerylDB 195 Jun 24, 2022
YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features

YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features. It is best to fit for cloud-native OLTP (i.e. real-time, business-critical) applications that need absolute data correctness and require at least one of the following: scalability, high tolerance to failures, or globally-distributed deployments.

yugabyte 6.6k Jul 1, 2022
RediSearch is a Redis module that provides querying, secondary indexing, and full-text search for Redis.

A query and indexing engine for Redis, providing secondary indexing, full-text search, and aggregations.

null 3.8k Jun 27, 2022
Multi-dimensional dynamically distorted staggered multi-bandpass LV2 plugin

B.Angr A multi-dimensional dynamicly distorted staggered multi-bandpass LV2 plugin, for extreme soundmangling. Based on Airwindows XRegion. Key featur

null 18 Apr 17, 2022
Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict outcomes.

Linear-Regression Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict o

vincent laizer 1 Nov 3, 2021
redis-cpp is a header-only library in C++17 for Redis (and C++11 backport)

redis-cpp - lightweight C++ client library for Redis redis-cpp is a C++17 library for executing Redis commands with support for pipelines and the publ

null 69 Jun 30, 2022
Distributed (Deep) Machine Learning Community 677 Apr 14, 2022
StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks 2.7k Jun 26, 2022
Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimensional grid

Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimens

Davide Faconti 241 Jun 24, 2022
Simple and portable (but not inflexible) GUI library in C that uses the native GUI technologies of each platform it supports.

libui: a portable GUI library for C This README is being written. Status It has come to my attention that I have not been particularly clear about how

Pietro Gagliardi 10.3k Jun 30, 2022
This repo contains BOTH c++ and BP examples to acheive the same logic, but in each frameworks specific ways

ApparatusCppMoveRandomly Hey there! This repo contains BOTH c++ and BP examples to acheive the same logic, but in each frameworks specific ways. I int

null 2 Jan 24, 2022
Sorting data on a stack with a limited set of instructions

push_swap Sorting data on a stack with a limited set of instructions Final Score: 125/100 Average 100 set speed: 575 Actions (Minimum 490 / Maximum 61

null 13 Jun 28, 2022
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

memory The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to b

Jonathan Müller 1.1k Jun 22, 2022
Process Ghosting - a PE injection technique, similar to Process Doppelgänging, but using a delete-pending file instead of a transacted file

Process Ghosting This is my implementation of the technique presented by Gabriel Landau: https://www.elastic.co/blog/process-ghosting-a-new-executable

hasherezade 457 Jun 21, 2022
Similar to C++ streams, but the stream elements are structured JSON data rather than characters.

JIOS : JSON Input Output Streams Similar to C++ streams, but the stream elements are structured JSON data rather than characters. Contents Features [P

Castedo Ellerman 3 Aug 16, 2019
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

Jonathan Müller 1k Dec 2, 2021
A docker image where you can run a judge program and a converter for multiple sequence alignment

genocon2021-docker 本リポジトリでは、ジャッジプログラム(eval.c)と Multiple Sequence Alignment (MSA) 変換プログラム(decode_cigar.py)を同梱した Docker イメージを提供しています。 また、サンプル解答プログラム(sam

Sakamoto, Kazunori 4 Sep 20, 2021