A redis module, similar to redis zset, but you can set multiple scores for each member to support multi-dimensional sorting

Related tags

Database TairZset
Overview

TairZset: Support multi-score sorting zset

Introduction Chinese

     TairZset is a data structure developed based on the redis module. Compared with the native zset data structure of redis, TairZset not only has the same rich data interface and high performance as the native zset, but also provides (arbitrary) multi-score sorting capabilities.

Features:

  • Support multi-score(Limited to 255)sorting, and the accuracy of any dimension is not lost
  • Incrby semantics is still supported under multi-score sorting
  • The syntax is similar to redis zset

Sorting rules:

For multi-dimensional scores, the priority of the score on the left is greater than the score on the right. Take a three-dimensional score as an example: score1#score2#score3. When comparing, tairzset will compare score1 first, and only compare score2 when score1 is equal, otherwise it will Take the comparison result of score1 as the comparison result of the entire score. In the same way, score3 will be compared only when score2 is equal.

Application scenario:

  • Sorting among gamers
  • Anchor popularity ranking in live room

Quick start

127.0.0.1:6379> exzadd tairzsetkey 1.1 x 2.2 y
(integer) 2
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "x"
2) "1.1000000000000001"
3) "y"
4) "2.2000000000000002"
127.0.0.1:6379> exzincrby tairzsetkey 2 x 
"3.1000000000000001"
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "y"
2) "2.2000000000000002"
3) "x"
4) "3.1000000000000001"
127.0.0.1:6379> exzadd tairzsetkey 3.3#3.3 z
(error) ERR score is not a valid format
127.0.0.1:6379> del tairzsetkey
(integer) 1
127.0.0.1:6379> exzadd tairzsetkey 1.1#3.3 x 2.2#2.2 y 3.3#1.1 z
(integer) 3
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "x"
2) "1.1000000000000001#3.2999999999999998"
3) "y"
4) "2.2000000000000002#2.2000000000000002"
5) "z"
6) "3.2999999999999998#1.1000000000000001"
127.0.0.1:6379> exzincrby tairzsetkey 2 y 
(error) ERR score is not a valid format
127.0.0.1:6379> exzincrby tairzsetkey 2#0 y 
"4.2000000000000002#2.2000000000000002"
127.0.0.1:6379> exzrange tairzsetkey 0 -1 withscores
1) "x"
2) "1.1000000000000001#3.2999999999999998"
3) "z"
4) "3.2999999999999998#1.1000000000000001"
5) "y"
6) "4.2000000000000002#2.2000000000000002"

Build

mkdir build  
cd build  
cmake ../ && make -j

then the tairzset_module.so library file will be generated in the lib directory

./redis-server --loadmodule /path/to/tairzset_module.so

Test

  1. Modify the path in the tairzset.tcl file in the tests directory to set testmodule [file your_path/tairzset_module.so]
  2. Add the path of the tairzset.tcl file in the tests directory to the all_tests of redis test_helper.tcl
  3. run ./runtest --single tairzset

Client

Java : https://github.com/aliyun/alibabacloud-tairjedis-sdk

API

Ref

Roadmap

  • support multi scores
  • support EXZUNIONSTORE、EXZUNION、EXZSCAN、EXZDIFF、EXZDIFFSTORE、EXZINTER、EXZINTERSTORE、EXZRANDMEMBER、EXZMSCORE

Applicable redis version

redis 5.x 、redis 6.x

Our modules

TairHash: A redis module, similar to redis hash, but you can set expire and version for the field
TairZset: A redis module, similar to redis zset, but you can set multiple scores for each member to support multi-dimensional sorting
TairString: A redis module, similar to redis string, but you can set expire and version for the value. It also provides many very useful commands, such as cas/cad, etc.

Comments
  • Fix an compiler error

    Fix an compiler error

    1. Fixed an compiler error caused by declaring non-static variables in header file and included by external .c files more than once~
    2. Fix a typo in assert()
    /usr/bin/ld: CMakeFiles/tairzset_module.dir/__/dep/skiplist.c.o:/data/alibaba/TairZset/dep/skiplist.h:59: multiple definition of `shared_maxstring'; CMakeFiles/tairzset_module.dir/tairzset.c.o:/data/alibaba/TairZset/dep/skiplist.h:59: first defined here
    /usr/bin/ld: CMakeFiles/tairzset_module.dir/__/dep/skiplist.c.o:/data/alibaba/TairZset/dep/skiplist.h:58: multiple definition of `shared_minstring'; CMakeFiles/tairzset_module.dir/tairzset.c.o:/data/alibaba/TairZset/dep/skiplist.h:58: first defined here
    collect2: error: ld returned 1 exit status
    
    good first issue to-be-merged 
    opened by guoxiangCN 3
  • Feat: implement new command EXZINTERCARD.

    Feat: implement new command EXZINTERCARD.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/12

    Implementation reference: Redis 7.0

    • https://redis.io/commands/zintercard/
    to-be-merged 
    opened by RinChanNOWWW 2
  • [NEW] Implement the new command `EXZINTERCARD`

    [NEW] Implement the new command `EXZINTERCARD`

    Description of the feature

    The new command ZINTERCARD was introduced in Redis 7.0. We can implement this command, too.

    Additional information

    https://redis.io/commands/zintercard/

    opened by RinChanNOWWW 1
  • Feat: implement new command EXZRANDMEMBER.

    Feat: implement new command EXZRANDMEMBER.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: #4

    Implementation reference: Redis 6.2 https://redis.io/commands/zrandmember/

    Command Description

    EXZRANDMEMBER

    EXZRANDMEMBER key [ count [WITHSCORES]] time complexity:O(N) where N is the number of elements returned.

    Command Description:

    When called with just the key argument, return a random element from the sorted set value stored at key.

    If the provided count argument is positive, return an array of distinct elements. The array's length is either count or the sorted set's cardinality (EXZCARD), whichever is lower.

    If called with a negative count, the behavior changes and the command is allowed to return the same element multiple times. In this case, the number of returned elements is the absolute value of the specified count.

    The optional WITHSCORES modifier changes the reply so it includes the respective scores of the randomly selected elements from the sorted set.

    Return value

    Bulk string reply: without the additional count argument, the command returns a Bulk Reply with the randomly selected element, or nil when key does not exist.

    Array reply: when the additional count argument is passed, the command returns an array of elements, or an empty array when key does not exist. If the WITHSCORES modifier is used, the reply is a list elements and their scores from the sorted set.

    TODO

    • [x] Fixing the compatibility problems with Redis 5.0 and 6.0.
    to-be-merged 
    opened by RinChanNOWWW 1
  • Feat: implement new command EXZMSCORE

    Feat: implement new command EXZMSCORE

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: #4

    EXZMSCORE

    EXZMSCORE key member [member ...] time complexity:O(N) where N is the number of members being requested.

    Command Description:

    Returns the scores associated with the specified members in the sorted set stored at key. For every member that does not exist in the sorted set, a nil value is returned.

    Return value

    Array reply: list of scores or nil associated with the specified member values (a double precision floating point number), represented as strings.

    to-be-merged 
    opened by RinChanNOWWW 1
  • [ASoC]新命令开发支持

    [ASoC]新命令开发支持

    活动官网

    https://asoc2022.opensource.alibaba.com/

    题目描述

    TairZset作为redis原生zset的一个多增增强结构,前期只实现了一些基本的命令, 还有部分命令(使用频率相对较低)没有支持, 为了更好的对齐redis原生zset语义,需要补全这些命令的实现和对应的测试。

    任务目标

    1、开发支持EXZUNIONSTORE、EXZUNION、EXZSCAN、EXZDIFF、EXZDIFFSTORE、EXZINTER、EXZINTERSTORE、EXZRANDMEMBER、EXZMSCORE等确实命令(对齐到redis 5.0版本即可) 2、添加对应的tcl测试

    项目技术要求

    熟悉c、tcl,了解redis zset实现原理

    题目难度

    导师信息

    Qu Dong ,@chenyang8094,ASoC Mentor, redis contributor. [email protected](联系导师请附上个人简历)

    welcome-pr to-be-closed 
    opened by chenyang8094 1
  • Feat: implement new command EXZDIFF and EXZDIFFSTORE.

    Feat: implement new command EXZDIFF and EXZDIFFSTORE.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 6.2

    • https://redis.io/commands/zdiffstore/
    • https://redis.io/commands/zdiff/
    to-be-merged 
    opened by RinChanNOWWW 0
  • Feat: implement new command EXZINTER and EXZINTERSTORE.

    Feat: implement new command EXZINTER and EXZINTERSTORE.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 6.2

    • https://redis.io/commands/zinterstore/
    • https://redis.io/commands/zinter/
    to-be-merged 
    opened by RinChanNOWWW 0
  • Feat: implement new command EXUNIONSTORE and EXUNION.

    Feat: implement new command EXUNIONSTORE and EXUNION.

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 6.2

    • https://redis.io/commands/zunionstore/
    • https://redis.io/commands/zunion/

    TBD

    The behavior when a field in the multi score become NaN.

    to-be-merged 
    opened by RinChanNOWWW 0
  • Feat: implement new command EXZSCAN and fix compile warnings.

    Feat: implement new command EXZSCAN and fix compile warnings.

    Signed-off-by: RinChanNOWWW [email protected]

    Summary

    related issues: https://github.com/alibaba/TairZset/issues/4

    Implementation reference: Redis 5.0 https://redis.io/commands/scan/

    Command Description

    EXZSCAN

    EXZSCAN key cursor [MATCH pattern] [COUNT count] time complexity: O(1) for every call. O(N) for a complete iteration, including enough command calls for the cursor to return back to 0. N is the number of elements inside the collection.

    Command Description:

    Iterates elements of TairZset types and their associated scores.

    EXSCAN is a cursor based iterator. This means that at every call of the command, the server returns an updated cursor that the user needs to use as the cursor argument in the next call. An iteration starts when the cursor is set to 0, and terminates when the cursor returned by the server is 0.

    While EXSCAN does not provide guarantees about the number of elements returned at every iteration, it is possible to empirically adjust the behavior of EXZSCAN using the COUNT option. Basically with COUNT the user specified the amount of work that should be done at every call in order to retrieve elements from the collection. This is just a hint for the implementation, however generally speaking this is what you could expect most of the times from the implementation.

    It is possible to only iterate elements matching a given glob-style pattern, similarly to the behavior of the KEYS command that takes a pattern as only argument. To do so, just append the MATCH arguments at the end of the EXZSCAN command. It is important to note that the MATCH filter is applied after elements are retrieved from the collection, just before returning data to the client. This means that if the pattern matches very little elements inside the collection, EXZSCAN will likely return no elements in most iterations.

    More detail information: https://redis.io/commands/scan/

    Return value

    A two elements multi-bulk reply, where the first element is a string representing an unsigned 64 bit number (the cursor), and the second element is a multi-bulk with an array of elements.

    Others

    • Fix the compile warnings.
    • Add a new data structure list (double linked list) from Redis source codes.
    to-be-merged 
    opened by RinChanNOWWW 0
  • Feature Request: Different Sort Options for Scores

    Feature Request: Different Sort Options for Scores

    Hi, Thanks for your nice work!

    I think there might be a use case for having different scores being compared in independent orders. Let's say I want to create a leaderboard for my game and I have a highscore with an ascending score (meaning the higher the score, the higher the rank) and for the same scores I would compare the time of the score (which the lower the time the higher the rank). From what I see with your module, we have to accomplish this with saving either the time or the highscore with some math function to reverse it's meaning and then use the module. But because I think this feature would be used alot, It would be worth implementation. Can this be implemented?

    Thanks

    good first issue 
    opened by H-Shafiei 6
Releases(release-v0.0.1-20211027)
  • release-v0.0.1-20211027(Oct 27, 2021)

Owner
Alibaba
Alibaba Open Source
Alibaba
StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks 3.7k Dec 30, 2022
C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

Simon Ninon 1k Jan 8, 2023
cpp_redis is a C++11 Asynchronous Multi-Platform Lightweight Redis Client

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform

CPP Redis 561 Jan 3, 2023
Minimalistic C client for Redis >= 1.2

This Readme reflects the latest changed in the master branch. See v1.0.0 for the Readme and documentation for the latest release (API/ABI history). HI

Redis 5.5k Jan 5, 2023
Modern, asynchronous, and wicked fast C++11 client for Redis

redox Modern, asynchronous, and wicked fast C++11 client for Redis [] (https://travis-ci.org/hmartiro/redox) Redox is a C++ interface to the Redis key

Hayk Martiros 380 Jan 7, 2023
Redis client written in C++

redis-plus-plus Overview Features Branches Installation Install hiredis Install redis-plus-plus Run Tests (Optional) Use redis-plus-plus In Your Proje

null 1k Dec 27, 2022
Aredis - a clean redis C++ client

aredis a clean redis C++ client redis_conn rc; if (rc.connect("127.0.0.1"/*host*/, 6379/*port*/, nullptr/*password*/, 1/*db*/)) { redis_command cmd;

Evan 26 Dec 10, 2021
A C++ Redis client

redis3m A C++ Redis client, born to bring my experience using Redis and C++ on a opensource library. Main goals Provide a simple and efficient wrapper

Luca Marturana 183 Dec 16, 2022
μSQLite library module for MicroPython

μSQLite library module for MicroPython WARNING: This project is in the beta development stage and may be subject to change. usqlite is a SQL database

Elvin Slavik 44 Jan 6, 2023
upstream module that allows nginx to communicate directly with PostgreSQL database.

About ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database. Configuration directives postgres_server

RekGRpth 1 Apr 29, 2022
BerylDB is a data structure data manager that can be used to store data as key-value entries.

BerylDB is a data structure data manager that can be used to store data as key-value entries. The server allows channel subscription and is optimized to be used as a cache repository. Supported structures include lists, sets, and keys.

BerylDB 203 Dec 16, 2022
YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features

YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features. It is best to fit for cloud-native OLTP (i.e. real-time, business-critical) applications that need absolute data correctness and require at least one of the following: scalability, high tolerance to failures, or globally-distributed deployments.

yugabyte 7.4k Jan 7, 2023
RediSearch is a Redis module that provides querying, secondary indexing, and full-text search for Redis.

A query and indexing engine for Redis, providing secondary indexing, full-text search, and aggregations.

null 4k Jan 5, 2023
Multi-dimensional dynamically distorted staggered multi-bandpass LV2 plugin

B.Angr A multi-dimensional dynamicly distorted staggered multi-bandpass LV2 plugin, for extreme soundmangling. Based on Airwindows XRegion. Key featur

null 21 Nov 7, 2022
Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict outcomes.

Linear-Regression Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict o

vincent laizer 1 Nov 3, 2021
redis-cpp is a header-only library in C++17 for Redis (and C++11 backport)

redis-cpp - lightweight C++ client library for Redis redis-cpp is a C++17 library for executing Redis commands with support for pipelines and the publ

null 77 Dec 11, 2022
Distributed (Deep) Machine Learning Community 682 Dec 28, 2022
StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks is a next-gen sub-second MPP database for full analysis senarios, including multi-dimensional analytics, real-time analytics and ad-hoc query, formerly known as DorisDB.

StarRocks 3.7k Dec 30, 2022
Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimensional grid

Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimens

Davide Faconti 324 Jan 4, 2023