YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features

Overview

YugabyteDB


License Documentation Status Ask in forum Slack chat Analytics

What is YugabyteDB?

YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features. It is best to fit for cloud-native OLTP (i.e. real-time, business-critical) applications that need absolute data correctness and require at least one of the following: scalability, high tolerance to failures, or globally-distributed deployments.

The core features of YugabyteDB include:

  • Powerful RDBMS capabilities Yugabyte SQL (YSQL for short) reuses the query layer of PostgreSQL (similar to Amazon Aurora PostgreSQL), thereby supporting most of its features (datatypes, queries, expressions, operators and functions, stored procedures, triggers, extensions, etc). Here is a detailed list of features currently supported by YSQL.

  • Distributed transactions The transaction design is based on the Google Spanner architecture. Strong consistency of writes is achieved by using Raft consensus for replication and cluster-wide distributed ACID transactions using hybrid logical clocks. Snapshot and serializable isolation levels are supported. Reads (queries) have strong consistency by default, but can be tuned dynamically to read from followers and read-replicas.

  • Continuous availability YugabyteDB is extremely resilient to common outages with native failover and repair. YugabyteDB can be configured to tolerate disk, node, zone, region, and cloud failures automatically. For a typical deployment where a YugabyteDB cluster is deployed in one region across multiple zones on a public cloud, the RPO is 0 (meaning no data is lost on failure) and the RTO is 3 seconds (meaning the data being served by the failed node is available in 3 seconds).

  • Horizontal scalability Scaling a YugabyteDB cluster to achieve more IOPS or data storage is as simple as adding nodes to the cluster.

  • Geo-distributed, multi-cloud YugabyteDB can be deployed in public clouds and natively inside Kubernetes. It supports deployments that span three or more fault domains, such as multi-zone, multi-region, and multi-cloud deployments. It also supports xCluster asynchronous replication with unidirectional master-slave and bidirectional multi-master configurations that can be leveraged in two-region deployments. To serve (stale) data with low latencies, read replicas are also a supported feature.

  • Multi API design The query layer of YugabyteDB is built to be extensible. Currently, YugabyteDB supports two distributed SQL APIs: Yugabyte SQL (YSQL), a fully relational API that re-uses query layer of PostgreSQL, and Yugabyte Cloud QL (YCQL), a semi-relational SQL-like API with documents/indexing support with Apache Cassandra QL roots.

  • 100% open source YugabyteDB is fully open-source under the Apache 2.0 license. The open-source version has powerful enterprise features such as distributed backups, encryption of data-at-rest, in-flight TLS encryption, change data capture, read replicas, and more.

Read more about YugabyteDB in our Docs.

Get Started

Cannot find what you are looking for? Have a question? Please post your questions or comments on our Community Slack or Forum.

Build Apps

YugabyteDB supports several languages and client drivers. Below is a brief list.

Language ORM YSQL Drivers YCQL Drivers
Java Spring/Hibernate PostgreSQL JDBC cassandra-driver-core-yb
Go Gorm pq gocql
NodeJS Sequelize pg cassandra-driver
Python SQLAlchemy psycopg2 yb-cassandra-driver
Ruby ActiveRecord pg yugabyte-ycql-driver
C# EntityFramework npgsql CassandraCSharpDriver
C++ Not tested libpqxx cassandra-cpp-driver
C Not tested libpq Not tested

What's being worked on?

This section was last updated in Mar, 2021.

Current roadmap

Here is a list of some of the key features being worked on for the upcoming releases (the YugabyteDB v2.7 latest release is expected to be released in Apr, 2021, and the v2.4 stable release was released in Jan 2021).

Feature Status Release Target Progress Comments
Automatic tablet splitting enabled by default PROGRESS v2.6, v2.7 Track
Point in time restores PROGRESS v2.6, v2.7 Track
YSQL: table statistics and CBO PROGRESS v2.7 Track
Support GIN indexes PROGRESS v2.7 Track
[YSQL] Feature support - ALTER TABLE PROGRESS v2.6, v2.7 Track
Online schema migration PROGRESS v2.7 Track
Support Liquibase, Flyway, ORM schema migrations PROGRESS v2.7
Support Spark 3 on YCQL DONE v2.5, v2.6 Track
Incorporate PostgreSQL 12 features PLANNING v2.7 Track
Improving day 2 operations of Yugabyte Platform PROGRESS v2.5 Track
Row-level geo-partitioning PROGRESS v2.7 Track Enhance YSQL language support
Improve TPC-C benchmarking PROGRESS v2.7 Track
Transparently restart transactions PROGRESS v2.5 Track Decrease the incidence of transaction restart errors seen in various scenarios
Pessimistic locking PROGRESS v2.7 Track

Planned additions to the roadmap

The following items are being planned as additions to the roadmap.

Feature Status Release Target Progress Comments
Support pgloader to migrate from MySQL PLANNING Track
Make COLOCATED tables default for YSQL PLANNING Track
Support Kafka as source and sink PLANNING Support source and sink for both YSQL and YCQL
Support for transactions in async xCluster replication PLANNING Track Apply transactions atomically on consumer cluster.

Recently released features

Feature Status Release Target Docs / Enhancements Comments
Support ALTER TABLE add primary key PROGRESS v2.6, v2.7 Track
Identity and access management in YSQL DONE v2.5 Track LDAP and Active Directory support
Follower reads in YSQL BETA v2.5 Issue Ability to perform follower reads for YSQL and transactional tables in YCQL.
YSQL cluster administration features - Node-Level statistics DONE v2.5 Issue Per-node view of currently active queries, find which queries are slow, what active connections are doing, etc.
Support loading large data sets into YSQL using COPY DONE v2.5 Issue Improving transactions which have a very large number of operations, as well as provide various options to batch load data more efficiently
Database runtime activity monitoring DONE v2.5 Issue Activity monitoring, audit logging, inactivity monitoring
Online rebuild of indexes DONE v2.2 Docs coming soon. See pending enhancements
DEFERRED constraints in YSQL DONE v2.2 Docs coming soon. See pending enhancements.
COLOCATED tables GA DONE v2.2 Docs coming soon
Online schema migration framework DONE v2.2 Note that this is just the framework implementation. See planned enhancements in this area.
Distributed backups for transactional tables DONE v2.2 Docs coming soon. See pending enhancements.
IPV6 support for YugabyteDB DONE v2.2 Docs coming soon
Automatic tablet splitting BETA v2.2 Docs See further enhancements
Change data capture BETA This feature is currently available but in beta.
xCluster replication (async cross-cluster replication) DONE v2.1 Docs
Encryption of data at rest DONE v2.1 Docs

Architecture

YugabyteDB Architecture

Review detailed architecture in our Docs.

Need Help?

Contribute

As an an open-source project with a strong focus on the user community, we welcome contributions as GitHub pull requests. See our Contributor Guides to get going. Discussions and RFCs for features happen on the design discussions section of our Forum.

License

Source code in this repository is variously licensed under the Apache License 2.0 and the Polyform Free Trial License 1.0.0. A copy of each license can be found in the licenses directory.

The build produces two sets of binaries:

  • The entire database with all its features (including the enterprise ones) are licensed under the Apache License 2.0
  • The binaries that contain -managed in the artifact and help run a managed service are licensed under the Polyform Free Trial License 1.0.0.

By default, the build options generate only the Apache License 2.0 binaries.

Read More

Issues
  • [#7349] Platform: Support for multiple backup configs.

    [#7349] Platform: Support for multiple backup configs.

    Description: Currently, the platform does not support multiple backup configs for any of the backup config types (viz S3, NFS, GCS, Azure). This feature will help users create multiple backup configs and edit/delete/show all the linked universes functionality.

    We are adding one column to store the name of the config that is config_name.

    incase of any migration failure,

    alter table customer_config drop column config_name ;

    to go back to a stable previous state.

    Test Plan:

    1. Go to configs and hit the backup tab.
    2. Hit the create backup button and fill in all the required details.
    3. Hit the save button, redirecting you to the backup config page view.
    4. You'll see the action button for the respective backup config along with the Edit/Delete/Show universe option.
    5. You can use these option to do perform the respective task.
    6. Took some backups for S3, nfs.

    Here's the screenshot:

    NFS

    Create_NFS

    [info] Test com.yugabyte.yw.commissioner.TaskGarbageCollectorTest.testPurge_invalidData started
    [info] Test run finished: 0 failed, 0 ignored, 7 total, 0.138s
    [error] Failed: Total 1359, Failed 1, Errors 0, Passed 1354, Skipped 4
    [error] Failed tests:
    [error] 	com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest
    [error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
    [error] Total time: 2080 s (34:40), completed 2 Jun, 2021 5:44:57 PM
    [email protected] managed % sbt "testOnly com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest"
    [info] welcome to sbt 1.3.13 (AdoptOpenJDK Java 1.8.0_292)
    [info] loading settings for project managed-build from plugins.sbt ...
    [info] loading project definition from /Users/mahebhat/Projects/yugabyte/yugabyte-db/managed/project
    [info] loading settings for project root from ui-build.sbt,build.sbt ...
    [info] set current project to yugaware (in build file:/Users/mahebhat/Projects/yugabyte/yugabyte-db/managed/)
    [Yugabyte sbt log] [Resolver] Maven cache server (such as Nexus or Artifactory), specified by YB_MVN_CACHE_URL: List()
    [Yugabyte sbt log] [Resolver] Local resolver (enabled by USE_MAVEN_LOCAL, path can be customized with YB_MVN_LOCAL_REPO): List(cache:Maven2 Local: /Users/mahebhat/.m2/repository)
    [Yugabyte sbt log] [Resolver] Default resolver: Vector(FileRepository(local, Patterns(ivyPatterns=Vector(${ivy.home}/local/[organisation]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)([branch]/)[revision]/[type]s/[artifact](-[classifier]).[ext]), artifactPatterns=Vector(${ivy.home}/local/[organisation]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)([branch]/)[revision]/[type]s/[artifact](-[classifier]).[ext]), isMavenCompatible=false, descriptorOptional=false, skipConsistencyCheck=false), FileConfiguration(true, None)), public: https://repo1.maven.org/maven2/)
    [Yugabyte sbt log] [Resolver] Snapshot resolver for yb-client jar (used when USE_MAVEN_LOCAL is not set, mostly during local development, configured with YB_MVN_SNAPSHOT_URL): List()
    [info] Instrumenting 737 classes to /Users/mahebhat/Projects/yugabyte/yugabyte-db/managed/target/scala-2.12/jacoco/instrumented-classes
    [info] Test run started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveNodeSuccess started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveNodeWithMaster started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveNonExistentNode started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveNodeRF5 started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveNodeWithNoDataMoveRF5 started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveNodeWithNoDataMove started
    [info] Test com.yugabyte.yw.commissioner.tasks.RemoveNodeFromUniverseTest.testRemoveUnknownNode started
    [info] Test run finished: 0 failed, 0 ignored, 7 total, 26.982s
    [info] Passed: Total 7, Failed 0, Errors 0, Passed 7
    [success] Total time: 31 s, completed 2 Jun, 2021 5:45:51 PM
    [email protected] managed % ```
    opened by nishantSharma459 35
  • Very poor performance in comparison to Postgresql

    Very poor performance in comparison to Postgresql

    The test with YugabyteDB 2.9.0 performs about 300 times slower than Postgres :( both with rf=1 and 1 master/tablet and rf=3 and 3 masters/tablets and with rf=1, one master and 3 tablet servers.

    I have a simple test schema: basically a user with UUID primary can subscribe any number of channels also with UUID primary key.

    image

    The script create_data.py creates Python processes, each with own connection, and creates a user, a channels and then proceeds to subscribe the user to a random number of previously created channels (randing from 1 to a few thousands), channel_ids are kept in memory for performance.

    (I attached all the test python files in this gist The schema is created here. )

    I run either python create_data.py YUGABYTE_URL 100 (run on Yugabyte, with 100 parallel connections using 100 Python processes) or python create_data.py POSTGRES_URL 100 (same, on Postgres). (Connection URLs are kept in .env file and loaded with dotenv)

    PostgreSQL achieves about 40,000 subscriptions per second, while YugaByte can't get any higher than ~150 subscriptions/second.

    Disabling indexes does not help. Adding tablets does not help.

    Restarting is done with python create_data.py YUGABYTE_URL --drop, then re-init with python create_data.py YUGABYTE_URL --init.

    The worst part is that tablet servers barely show any load:

    image

    No other processes are running. I run YugaByte 2.9.0 as shown in either rf1-master.sh, rf1-tablet.sh (for replication factor 1) or using rf3-master{1,2,3}.sh rf3-tablet{1,2,3}.sh (obviously each one on their own server).

    The servers have 16 vCPUs and 30G memory (about 1G used). All servers have SSDs. Ubuntu 20.04.

    The load (create_data.py) is created from the 4th machine.

    All are connected using private IPs and the UI correctly shows 3 masters, 3 tablets (or 1 master, 1 tablet/3 tablets in rf1 case). .WARNING logs don't show anything interesting.

    With Python when I go from 1 connection to 100 connections, I get a proportional increase up until about 100 processes, where I get a sustained rate of ~40,000 subscriptions per second...

    Postgres:

    image

    YugaByte 2.9.0:

    image

    With YugaByte I get about 110 subscriptions/sec with 1 connection, about 150/sec with 3 connections and then that's it, doesn't matter if I have 100 connections, and even if I connect to random nodes. It does not matter if the connection stays open or opened for each "batch" of ten users.

    No transactions are performed.

    The same situation is with rf=1 tablet_servers=3, there's really no benefit in running 2 more servers. The worst part is that they all are basically idle:

    image

    The weird part is that inserting 10 random users takes about the same time on Postgres and YugaByteDB: 1-5 seconds on YugaByteDB, 0.4-2.5 seconds on Postgres... But the concurrency on Postgres is much higher.

    image

    Is this expected? Am I missing something here?

    Thank you!

    area/ysql 
    opened by slava-vishnyakov 32
  • [YSQL] invalid locale name

    [YSQL] invalid locale name "en_US.UTF-8"

    [EDIT] See @mbautin 's update here: https://github.com/YugaByte/yugabyte-db/issues/3732#issuecomment-653261887

    Root Cause: << When the installation directory is a symlink, post_install.sh currently is skipping a major part of what it is supposed to be doing (replacing substrings in files to point to the actual installation location). >>


    Manually setting up YB 2.0.11 tservers on Ubuntu 16.04 on AWS and getting the following errors:

    In YugaByte DB, setting LC_COLLATE to C and all other locale settings to en_US.UTF-8 by default. Locale support will be enhanced as part of addressing https://github.com/YugaByte/yugabyte-db/issues/1557initdb: invalid locale name "en_US.UTF-8"
    

    I've followed the advice and troubleshooting from the following links and github issues:

    • https://github.com/yugabyte/yugabyte-db/issues/2000#issuecomment-520945181
    • https://github.com/yugabyte/yugabyte-db/issues/1557
    • https://askubuntu.com/questions/76013/how-do-i-add-locale-to-ubuntu-server
    • https://www.digitalocean.com/community/questions/language-problem-on-ubuntu-14-04
    • https://stackoverflow.com/questions/17712700/postgres-locale-error
    • https://www.postgresql.org/docs/9.4/locale.html
    • etc.

    I've also tried performing variations on the following to ensure the locale of en_US.UTF-8 is installed on the machines:

    locale-gen en_US.UTF-8
    dpkg-reconfigure locales --frontend=noninteractive
    update-locale LC_ALL="en_US.UTF-8" LANG="en_US.UTF-8"
    

    and unfortunately still getting the invalid locale name error.

    Here are the results of locale -a:

    C
    C.UTF-8
    en_US.utf8
    POSIX
    

    and of locale:

    LANG=en_US.UTF-8
    LANGUAGE=
    LC_CTYPE="en_US.UTF-8"
    LC_NUMERIC="en_US.UTF-8"
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_US.UTF-8"
    LC_MONETARY="en_US.UTF-8"
    LC_MESSAGES="en_US.UTF-8"
    LC_PAPER="en_US.UTF-8"
    LC_NAME="en_US.UTF-8"
    LC_ADDRESS="en_US.UTF-8"
    LC_TELEPHONE="en_US.UTF-8"
    LC_MEASUREMENT="en_US.UTF-8"
    LC_IDENTIFICATION="en_US.UTF-8"
    LC_ALL=
    

    if we cat out /etc/locale.gen we'll see en_US.UTF-8 UTF-8 is uncommented and should be available for when we run locale-gen en_US.UTF-8:

    cat /etc/locale.gen
    ...
    # en_SG.UTF-8 UTF-8
    # en_US ISO-8859-1
    # en_US.ISO-8859-15 ISO-8859-15
    en_US.UTF-8 UTF-8
    # en_ZA ISO-8859-1
    # en_ZA.UTF-8 UTF-8
    ...
    
    locale-gen en_US.UTF-8
    Generating locales (this might take a while)...
      en_US.UTF-8... done
    Generation complete.
    

    and when checking etc/default/locale, I see the following:

    cat /etc/default/locale 
    LANG=en_US.UTF-8
    

    Finally, when cating out the /i18n/supported locales, we see en_US.UTF-8

    cat /usr/share/i18n/SUPPORTED | grep "en_"
    ...
    en_SG.UTF-8 UTF-8
    en_SG ISO-8859-1
    en_US.UTF-8 UTF-8
    en_US ISO-8859-1
    ...
    

    I've tested running the initdb process similarly to how the pg_wrapper calling initdb would run the initdb script and the only success I've had has been if I run with --locale C, --locale POSIX, or --no-locale (same as --locale C):

    reminder, results of `locale -a`:
    C
    C.UTF-8
    en_US.utf8
    POSIX
    
    # these options work fine
    ./yugabyte/postgres/bin/initdb --locale C -D /tmp/ysqltest
    ./yugabyte/postgres/bin/initdb --locale POSIX -D /tmp/ysqltest
    ./yugabyte/postgres/bin/initdb --no-locale -D /tmp/ysqltest
    
    ## all result in...
    
    The database cluster will be initialized with locale "C".
    The default database encoding has accordingly been set to "SQL_ASCII".
    The default text search configuration will be set to "english".
    
    # fails
    ./yugabyte/postgres/bin/initdb --locale en_US.UTF-8 -D /tmp/ysqltest
    ./yugabyte/postgres/bin/initdb --locale en_US.utf8 -D /tmp/ysqltest
    ./yugabyte/postgres/bin/initdb --locale C.UTF-8.utf8 -D /tmp/ysqltest
    
    ## all result in...
    
    initdb: invalid locale name "C.UTF-8" # or some variant
    

    I'm assuming that this section of the initdb operations configuring locale is going to force locale to be en_US.UTF-8 by default, however I could really use some advice to figure out why en_US.UTF-8 is not being configured and made available on Ubuntu 16.04?

    Thanks in advice 👍

    opened by aegershman 32
  • [Jepsen] Recovery after network partitions seems to take a while

    [Jepsen] Recovery after network partitions seems to take a while

    This may not be a real issue in YB; I'm still exploring behavior and collecting more evidence!

    On Yugabyte CE, version 1.1.10.0, recovery from network partitions seems to take a little longer than one might expect. In particular, when performing a series of single-row CQL inserts during a mixture of randomized 3/2 split network partitions, followed by the end of all partitions, 30 seconds for the cluster to heal, and a final read of all rows, that read consistently times out.

    This cluster is running on 5 nodes, with replication factor 3. n1, n2, and n3 are masters; n1, n2, n3, n4, and n5 are tservers.

    For instance, in this test, the second partition (the shaded region in this plot) appears to prevent all clients from performing writes, and even with 30 seconds of recovery before the final read, that read times out:

    latency-raw 64

    This may be expected behavior, or might be some sort of client issue, so I'm going to keep exploring. I'll update this issue as I find out more. :)

    community/request 
    opened by aphyr 31
  • [YSQL] Data lost when use range partition index

    [YSQL] Data lost when use range partition index

    I'm migrating an application from mysql to yugabyte. After import data dumped from mysql, the application run into an error:

    SELECT COUNT(id) from order_items where PurchaseDate > '2020-11-20';
    
    Query error: Given ybctid is not associated with any row in table.
    

    order_items table has about 7000000 records, and has an index on PurchaseDate:

    "idx_33539_idx_oi_PurchaseDate" lsm ("PurchaseDate" DESC)
    
    area/ysql area/docdb 
    opened by iswarezwp 30
  • add SimpleQueryTest CQL testsuite

    add SimpleQueryTest CQL testsuite

    This pull request is for adding SimpleQueryTest suite to Yugabyte-DB's CQL testsuite. I have successfully run the first 7 tests of Cassandra's CQL test suite SimpleQueryTest.java (under Apache license) with Yugabyte-DB. The changes I had to make are as follows:

    SimpleQueryTest.java

    This is a testsuite with many individual tests. The first 7 tests currently run fine, the rest are commented out for now. Goal was to keep the changes to this file to the minimum so that new individual tests can be run easily. The changes to this file are:

    • replaced the inclusion of cassandra package with org.yb.cql
    • calls to flush() has been commented out, as I do not know the equivalent in Yugabyte-DB. The flush() in Cassandra's testsuite is a public method of CQLTester base class that calls the ColumnFamilyStore method forceBlockingFlush()

    CQLTester.java The class SimpleQueryTest inherits from CQLTester. I modified CQLTester.java (also under Apache license) much more to only include the minimum necessary to successfully compile and run SimpleQueryTest. These modifications were important for keeping the changes in SimpleQueryTest to be minimum. The CQLTester.java changes are:

    • import yugabyte-db relevant classes/package
    • made CQLTester a sub-class of BaseCQLTest to pull in yugabyte-db boilerplate
    • modified createTable and execute methods to provide the behavior expected by the individual tests in SimpleQueryTest. For example, the individual tests do not hardcode the table name instead it gets generated. During execution of statements (e.g. select, insert) the CQL statement string gets built with the looked up table name
    • modified assertRows() methods which is an easy way to verify the rows inserted/deleted match the rows selected. This method required more modifications
    • a comparison with Cassandra's CQLTester.java can be used to understand the modifications

    Thanks!

    kind/enhancement 
    opened by ananyaks 25
  • [YCQL] EXPLAIN SELECT causes tserver to crash

    [YCQL] EXPLAIN SELECT causes tserver to crash

    I'm running release version 2.0.1. When running EXPLAIN SELECT * FROM accounts_api.logins WHERE ... via cqlsh the local tserver crashes with:

    Oct 15 17:37:25 watson bash[16013]: *** Aborted at 1571161045 (unix time) try "date -d @1571161045" if you are using GNU date ***
    Oct 15 17:37:25 watson bash[16013]: PC: @     0x7f51308454e6 std::_Maybe_get_result_type<>
    Oct 15 17:37:25 watson bash[16013]: *** SIGILL (@0x7f51308454e6) received by PID 16013 (TID 0x7f50fa4b8700) from PID 813978854; stack trace: ***
    Oct 15 17:37:25 watson bash[16013]: @     0x7f51231d5ba0 (unknown)
    Oct 15 17:37:25 watson bash[16013]: @     0x7f51308454e5  std::_Maybe_get_result_type<>
    Oct 15 17:37:25 watson bash[16013]: @     0x7f513135663e  yb::ql::QLProcessor::ExecuteAsync()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f513135680d  yb::ql::QLProcessor::RunAsync()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f5131febf97  yb::cqlserver::CQLProcessor::ProcessRequest()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f5131fefb64  yb::cqlserver::CQLProcessor::ProcessRequest()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f5131fefed4  yb::cqlserver::CQLProcessor::ProcessCall()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f51320097ad  yb::cqlserver::CQLServiceImpl::Handle()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f512a0dd0d0  yb::rpc::ServicePoolImpl::Handle()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f512a089763  yb::rpc::InboundCall::InboundCallTask::Run()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f512a0e8c57  yb::rpc::(anonymous namespace)::Worker::Execute()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f51286cea18  yb::Thread::SuperviseThread()
    Oct 15 17:37:25 watson bash[16013]: @     0x7f51231cd694 start_thread
    Oct 15 17:37:25 watson bash[16013]: @     0x7f512290a41d __clone
    Oct 15 17:37:25 watson bash[16013]: @                0x0 (unknown)
    Oct 15 17:37:25 watson systemd[1]: yugabyte-tserver.service: main process exited, code=killed, status=4/ILL
    Oct 15 17:37:25 watson systemd[1]: Unit yugabyte-tserver.service entered failed state.
    

    I haven't tried other tables or other queries to see if they cause it to crash.

    Here's the table, if that helps:

    > DESCRIBE TABLE accounts_api.logins;
    CREATE TABLE accounts_api.logins (
        auth_id text,
        auth_type smallint,
        scope text,
        auth_desc text,
        auth_secret text,
        auth_email text,
        auth_updated timestamp,
        scope_string_id text,
        scope_int_id bigint,
        created timestamp,
        updated timestamp,
        last_used timestamp,
        account_ids map<uuid, smallint>,
        flags int,
        alias authidtype,
        PRIMARY KEY (auth_id, auth_type, scope)
    ) WITH CLUSTERING ORDER BY (auth_type ASC, scope ASC)
        AND default_time_to_live = 0
        AND transactions = {'enabled': 'false'};
    
    community/request 
    opened by jameshartig 24
  • Too many Network Connection - High Cpu load after update.

    Too many Network Connection - High Cpu load after update.

    Problem : CPU load on Each T-SERVER, Reason : After Update to Last Version from Previous. NO - WARN message, NO - ERROR NO - FATAL continuesly.

    TCP Package content : Yugabyte consensus - Update .

    400 KiB Bandwith each seconds between servers make cpu load.

    conse_2

    community/request 
    opened by kutayzorlu 23
  • [Platform] Old backups are not deleted by a schedule

    [Platform] Old backups are not deleted by a schedule

    Backups records in DB (table backup) have empty/null values for schedule_uuid. But old backups are selected for further deletion using the next finder:

      public static List<Backup> getExpiredBackups(UUID scheduleUUID) {
        // Get current timestamp.
        Date now = new Date();
        return Backup.find.query().where()
          .eq("schedule_uuid", scheduleUUID)
          .lt("expiry", now)
          .eq("state", BackupState.Completed)
          .findList();
      }
    
    

    So no backups could be selected if they don't have the schedule_uuid specified. Also this explains why the customer is able to delete old backups manually. Possible reason is that the schedule in task_params field has value: "scheduleUUID":null

    If a task, which creates backups, takes schedule_uuid from these parameters then this could be the root cause. Additional information is inside the ticket 661 (application.log and YW db dump).

    priority/high area/platform 2.2 Backport Required 2.4 Backport Completed 
    opened by SergeyPotachev 22
  • No support for time with micro-second precision

    No support for time with micro-second precision

    Example:

    cqlsh> SELECT totimestamp(now()) FROM myapp.stock_market;
    
     totimestamp(now())
    ---------------------------------
     2018-07-02 16:28:44.433000+0000
     2018-07-02 16:28:44.433000+0000
     2018-07-02 16:28:44.434000+0000
     2018-07-02 16:28:44.434000+0000
     2018-07-02 16:28:44.434000+0000
     2018-07-02 16:28:44.434000+0000
    

    As one can see, even though there are 6 significant digits after second, there are only 3 non-zero digits. Therefore, the minimal time precision is only millisecond. Is it possible to get it to the level of micro-second?

    I find this useful in my application. Suppose I have a high rate of requests from many users from multiple servers so that each millisecond sometimes could easily contain multiple requests. Then how do I now the order of the requests? Does TIMEUUID guarantee the correct order? Additionally, if I want the users to know the order of the requests of all users (say I run a blockchain and want the users to be able to verify the results), how could that be easily done without adding much overhead (they do not have YugaByte installed). Is it possible to add a field of id corresponding to the order of insertion with ACID guarantee? Thanks!

    kind/question wontfix 
    opened by yjiangnan 22
  • [YSQL] Foreign Data Wrapper support

    [YSQL] Foreign Data Wrapper support

    This PR adds support for postgres_fdw. This PR targets the following github issue: https://github.com/yugabyte/yugabyte-db/issues/5714. At the current stage, it is possible to do the following:

    bash-4.2$ ysqlsh
    ysqlsh (11.2-YB-2.7.3.0-b0)
    Type "help" for help.
    
    yugabyte=# create role tenant1 with login password 'tenant1' connection limit 10;
    CREATE ROLE
    yugabyte=# create database tenant1db owner = tenant1;
    
    CREATE DATABASE
    yugabyte=# create role tenant2 with login password 'tenant2' connection limit 10;
    CREATE ROLE
    yugabyte=# create database tenant2db owner = tenant2;
    CREATE DATABASE
    yugabyte=# \connect tenant2db
    You are now connected to database "tenant2db" as user "yugabyte".
    tenant2db=# set role tenant2;
    SET
    tenant2db=> create table table1tenant2 (rowid int, rowval text);
    CREATE TABLE
    tenant2db=> set role yugabyte;
    SET
    tenant2db=# grant select, insert on table table1tenant2 to yugabyte;
    GRANT
    tenant2db=# \connect tenant1db
    You are now connected to database "tenant1db" as user "yugabyte".
    tenant1db=# create extension postgres_fdw;
    CREATE EXTENSION
    tenant1db=# CREATE SERVER foreigndb_fdw FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '127.0.0.1', port '5433', dbname 'tenant2db');
    CREATE SERVER
    tenant1db=# GRANT USAGE ON FOREIGN SERVER foreigndb_fdw TO yugabyte;
    GRANT
    tenant1db=# CREATE USER MAPPING FOR yugabyte SERVER foreigndb_fdw OPTIONS (user 'tenant2', password 'tenant2');
    CREATE USER MAPPING
    tenant1db=# CREATE FOREIGN TABLE table1tenant2 (
    tenant1db(#   rowid integer OPTIONS (column_name 'rowid'),
    tenant1db(#   rowval text OPTIONS (column_name 'rowval')
    tenant1db(# ) SERVER foreigndb_fdw
    tenant1db-# OPTIONS (schema_name 'public', table_name 'table1tenant2');
    CREATE FOREIGN TABLE
    tenant1db=# select * from table1tenant2;
     rowid | rowval
    -------+--------
    (0 rows)
    
    tenant1db=# insert into table1tenant2 (rowid, rowval) values (1, 'aaa');
    INSERT 0 1
    tenant1db=# select * from table1tenant2;
     rowid | rowval
    -------+--------
         1 | aaa
    (1 row)
    
    tenant1db=# \connect tenant2db
    You are now connected to database "tenant2db" as user "yugabyte".
    tenant2db=# set role tenant2;
    SET
    tenant2db=> select * from table1tenant2;
     rowid | rowval
    -------+--------
         1 | aaa
    (1 row)
    
    tenant2db=>
    

    There are some immediate issues:

    1. I am not yet able to import foreign schema:
    tenant1db=# IMPORT FOREIGN SCHEMA public LIMIT TO (table1tenant2) FROM SERVER foreigndb_fdw INTO public;
    ERROR:  COLLATE not supported yet
    LINE 3:   rowval text OPTIONS (column_name 'rowval') COLLATE pg_cata...
                                                         ^
    HINT:  See https://github.com/YugaByte/yugabyte-db/issues/1127. Click '+' on the description to raise its priority
    QUERY:  CREATE FOREIGN TABLE table1tenant2 (
      rowid integer OPTIONS (column_name 'rowid'),
      rowval text OPTIONS (column_name 'rowval') COLLATE pg_catalog."default"
    ) SERVER foreigndb_fdw
    OPTIONS (schema_name 'public', table_name 'table1tenant2');
    

    ~~It seems that I was not able to locate the correct unsupported COLLATE case (https://github.com/YugaByte/yugabyte-db/issues/1127).~~ This one is due to this: https://github.com/yugabyte/yugabyte-db/blob/v2.7.2/src/postgres/src/backend/parser/gram.y#L3722, as an interim solution I can configure and compile YugabyteDB with YB_POSTGRES_WITH_ICU=[true | 1 | t | yes | y] environment variable, then it all works:

    tenant1db=# IMPORT FOREIGN SCHEMA public LIMIT TO (table1tenant2) FROM SERVER foreigndb_fdw INTO public;
    IMPORT FOREIGN SCHEMA
    
    1. Not sure if this is a real issue:
    tenant1db=> IMPORT FOREIGN SCHEMA public LIMIT TO (table1tenant2) FROM SERVER foreigndb_fdw INTO public;
    ERROR:  password is required
    DETAIL:  Non-superuser cannot connect if the server does not request a password.
    HINT:  Target server's authentication method must be changed.
    
    1. Currently, the postgres_fdw extension has been hard coded in the shared_preload_libraries configuration directive. Preferably, this should be supported as a user configurable switch. However, for this to be the case, the following PR would have to go in first: https://github.com/yugabyte/yugabyte-db/pull/9576.
    2. Regression tests have to be adapted.
    3. I would assume some a test suite more comprehensive than a regression test suite might be needed.
    4. Obviously, there might be some limitations on the DocDB side leading to this not working in a distributed environment, which I am not aware of.
    5. Needs to be tested in a distributed cluster. Currently tested using the single node launched via yugabyted.

    Early feedback welcomed.

    opened by radekg 21
  • [YSQL] Enhance pg_stat_statements with latest timestamp for query execution

    [YSQL] Enhance pg_stat_statements with latest timestamp for query execution

    Jira Link: DB-2830

    Description

    Currently pg_stat_statements doesn't record when the query is executed.

    In some scenarios, we need to know when the query is last executed. See 13124 as an example.

    kind/bug area/ysql priority/medium status/awaiting-triage 
    opened by tedyu 0
  • [YSQL] Allow COPY FROM command to resume

    [YSQL] Allow COPY FROM command to resume

    Jira Link: DB-2828

    Description

    Currently when COPY FROM command fails midway, the user needs to query pg_stat_progress_copy view and find out how many rows have been processed. The user then needs to resubmit COPY FROM command with SKIP directive.

    This process can be automated. The user can specify RESUME directive in which case the lookup of processed row count is done automatically. The previous COPY COMMAND can be found via pg_stat_statements. The RESUME directive would supersede SKIP directive.

    kind/bug area/ysql priority/medium status/awaiting-triage 
    opened by tedyu 3
  • [YSQL] Push down nextval() validation for sequences

    [YSQL] Push down nextval() validation for sequences

    Jira Link: DB-2827

    Description

    Context:

    • Sequence metadata is stored pg_sequences table: https://www.postgresql.org/docs/11/catalog-pg-sequence.html
    • Additionally, for each sequence its state is stored (in YSQL) in a sharded table with one row per sequence and the schema (db_oid bigint, seq_oid bigint, last_value bigint, is_called bool, PRIMARY KEY (db_oid, seq_oid)
    • Sequence functions (nextval in particular): https://www.postgresql.org/docs/11/functions-sequence.html. Typically these function calls read and/or modify a sequence's state, such as retrieving or incrementing last_value.

    In particular nextval() advances sequences last_val and returns the new value.

    Currently, in YSQL, the nextval() implementation has the following steps:

    1. read last_val and is_called columns from the sequences_data table (from remote node)
    2. validate next sequence value (w.r.t to the user-defined sequence bounds) (on local/pg-connection node)
    3. If validate succeeded, try to update last_val and is_called (write to the remote node). if last_val changed since then (e.g. due to a concurrent nextval() call from another backend), then retry by starting from the beginning.

    So the implementation needs to send two RPCs to a possibly remote node (for step 1 and step 3 above). Moreover, if anything changed on the remote between step 1 and 3 we need to restart from the beginning. The original intent might have been that increasing the sequence cache value can mitigate perf impact of these two-step process.

    However, as @suranjan noted on an internal thread, this design has a number of issues:

    1. Whatever be the cache value, the current design suffers with the issue of a lot of retries in case of highly concurrent applications.
    2. In case of geo distributed cluster, the problem could be worse. The region farther away from the "leader tablet of sequence table" will starve out. As it will have to retry a lot compared to others. Worse if cache value is lower.

    There are multiple workarounds for this including using UUID type instead.

    But a straightforward improvement here should be to simply push down step 2 (the validation) to the remote node and therefore execute all 3 steps using 1 RPC and without every needing to retry. The 1 RPC will need to do both the write and return the (new) last_value and is_called so it will be similar to the pushdown for an UPDATE .. RETURNING. Additionally the bounds validation needs to happen in DocDB -- so we need to send extra info to DocDB in order to do it.

    If this works the changes would be along the lines of:

    In sequence.c the retry, read and validation part can be removed (for YSQL). And instead the update, can get the metadata needed for validation. For instance something like:

    HandleYBStatus(YBCUpdateSequenceTupleConditionally(MyDatabaseId,
                                                       relid,
                                                       yb_catalog_cache_version,
                                                       inc /* seqincrement */,
                                                       cycle /* seqcycle */,
                                                       bound_val /* seqmax is inc > 0, else seqmin */
                                                       &seq->last_value /* read from DocDB after update*/,
                                                       &seq->is_called /* read from DocDB after update */,
    	));
    

    Then across the stack some functions/PBs need to be updated accordingly: PgUpdateSequenceTupleRequestPB/PgUpdateSequenceTupleResponsePB, UpdateSequenceTuple PgsqlWriteRequestPB might need a few new fields as well.

    Finally the DocDB side needs to PgsqlWriteOperation::ApplyUpdate code for validation and returning last_value/is_called needs to be added. Additionally this might be an opportunity to remove the special case for sequences but that's TBD.

    kind/bug area/ysql priority/high 
    opened by m-iancu 0
  • [Docs] [YSQL] The

    [Docs] [YSQL] The "SQL statements" section does not have a section for DISCARD

    Description

    The Version 11 PG doc describes the DISCARD statement HERE. Simple ad hoc tests in YB-2.15.0.0 show that it seems to work.

    @m-iancu, please assign a YSQL developer to check the status—and in particular to check that the feature is automatically properly tested when new YB releases are created.

    area/documentation 
    opened by bllewell 0
  • [YSQL] Check relkind before setting REINDEX_REL_YB_DROP_AND_CREATE in finish_heap_swap()

    [YSQL] Check relkind before setting REINDEX_REL_YB_DROP_AND_CREATE in finish_heap_swap()

    Jira Link: DB-2826

    Description

    I was playing with ALTER TABLE and I found that there is no checking of relkind when setting REINDEX_REL_YB_DROP_AND_CREATE in finish_heap_swap():

    	 * For YB materialized views, we need to drop and create the index instead
    	 * of reindexing the same index.
    

    This issue is to add the check of relkind.

    kind/bug area/ysql priority/medium status/awaiting-triage 
    opened by tedyu 0
Releases(v2.15.0.0)
  • v2.15.0.0(Jun 30, 2022)

  • v2.12.7.0(Jun 25, 2022)

  • v2.12.4.2(Jun 18, 2022)

  • v2.8.7.0(Jun 30, 2022)

  • v1.0.0-beta-yugabyted-ui(Jun 14, 2022)

    This release is for the YugabyteDB UI project. This release provides the Yugabyted UI Console for clusters deployed using yugabyted CLI for monitoring the YugabyteDB clusters. This release provides an overview of the YugabyteDB console which is going to be bundled in subsequent YugabyteDB releases.

    Yugabyted UI console has the following features

    • Overview tab for monitoring the cluster metrics
    • Nodes tab for listing the YugabyteDB cluster nodes
    • Performance tab for monitoring the cluster including Slow Queries and Live Queries.
    Source code(tar.gz)
    Source code(zip)
    yugabyted-ui-1.0.0-darwin-x86_64.tar.gz(9.69 MB)
    yugabyted-ui-1.0.0-linux-x86_64.tar.gz(9.52 MB)
  • v2.12.6.0(Jun 10, 2022)

  • v2.13.2.0(May 25, 2022)

  • v2.12.5.0(May 19, 2022)

  • v2.8.6.0(Jun 18, 2022)

  • v2.8.5.0(May 3, 2022)

  • v2.6.18(Apr 27, 2022)

  • v2.13.1.0(Apr 25, 2022)

  • v2.6.17.0-b10(Apr 20, 2022)

  • v2.12.3.0-b19(Apr 20, 2022)

  • v2.8.4.0-b30(Mar 31, 2022)

  • v2.12.2.0-b58(Mar 31, 2022)

  • v2.13.0.1(Mar 22, 2022)

  • v2.13.0(Mar 9, 2022)

  • v2.12.1(Mar 9, 2022)

  • v2.8.3(Mar 9, 2022)

  • v2.8.2(Mar 9, 2022)

  • v2.11.2(Feb 15, 2022)

    v2.11.2.0 - January 26, 2022

    Build: 2.11.2.0-b89

    Downloads

               

    Docker

    docker pull yugabytedb/yugabyte:2.11.2.0-b89
    

    New Features

    Yugabyte Platform

    • [PLAT-1885] Hashicorp Vault Integration
    • [PLAT-2341] UI support for Multiple on-prem provider instances
    • [PLAT-2362] Maintenance windows API and backend
    • [PLAT-2382] New backups API
    • [PLAT-2401] [PLAT-2404] New GFlag form component and validation for user entries
    • [PLAT-2402] UI form for user to add/edit gflags in more accurate way
    • [PLAT-2403] Add Multiple G-flags from free text(json)
    • [PLAT-2479] LDAP Integration with Platform
    • [PLAT-2535] Add API endpoint for aborting tasks
    • [PLAT-2553] Node Comparison Feature in Metrics Page (Backend)
    • [PLAT-2562] UI support for abort task capability
    • [PLAT-2658] Integrate t3 instance type support
    • [PLAT-2750] YSQL_DUMP based backup-create and backup-restore
    • [PLAT-2766] Can now delete paused GCP instances

    Database

    • [8032] [YSQL] Add support for ALTER TYPE .. RENAME TO
    • [10451] [ysql] Add support for ADD CONSTRAINT .. UNIQUE .. USING INDEX
    • [10509] [10510] [YSQL] Add support for CREATE, DROP, and REFRESH MATERIALIZED VIEW

    Improvements

    Yugabyte Platform

    • [PLAT-501] [PLAT-1850] Add Custom CA cert start and expiry in backend and add extra validations for custom certs
    • [PLAT-580] Add API to sync platform XClusterConfig state with universe
    • [PLAT-2058] Add Custom Port UI Validation
    • [PLAT-2066] Read write alert
    • [PLAT-2131] Addition of Default GFlags while GFlags updating
    • [PLAT-2152] Task Improvement - changes in task framework to wait for previous subtask, ability to send abort signal to a task
    • [PLAT-2236] Add leader blacklisting to kubernetesUpgradeTaskBase
    • [PLAT-2296] Timezone Selector on user profile (UI)
    • [PLAT-2324] Universe with RF=3 and 3 nodes could be updated to have 2 nodes only
    • [PLAT-2362] Active alert notification period
    • [PLAT-2435] Task Improvement - Make CreateUniverse retryable.
    • [PLAT-2460] Task Improvement - Make EditUniverse retryable
    • [PLAT-2461] [UI] Enhance KMS Config listing UI
    • [PLAT-2467] [UI] XCluster UI Changes
    • [PLAT-2480] Change in request payload and response format for G-Flag validation API
    • [PLAT-2500] [UI] Add support to edit KMS configs
    • [PLAT-2501] [Alerts-UI] Rename the "Edit" button to "Save" in Edit Notification Channels
    • [PLAT-2572] Task Improvement - Move/Include retryable and abortable in CustomerTask API response payload
    • [PLAT-2577] UI - Edit KMS, handling values that user want to empty and masked values.
    • [PLAT-2613] Adding a runtime config flag to retain shellprocess cmdOutputLogs.
    • [PLAT-2641] Task Improvement - CreateUniverse retry for onprem nodes does not work due to precheck failure
    • [PLAT-2690] Run upgrade_ysql during universe upgrade
    • [PLAT-2719] Allow path style access for non-standard endpoints in S3 backup configs
    • [PLAT-2784] Add support for dev db-versions while listing gflags
    • [PLAT-2821] Change default instance type to c5.4xlarge in universe creation
    • [Platform] GFlags validation Improvements

    Database

    • [5755] [DocDB] Transaction cleanup improvements
    • [7977] [DocDB] Load balance leaders across disks
    • [8730] [dst] Enable compaction and flush of sys catalog tablet from admin CLI.
    • [8978] [docdb] Generate system.partitions vtable proactively instead of via a background task
    • [9468] [YSQL] Perform indefinite retries on kReadRestart for each statement in READ COMMITTED isolation (Part-2)
    • [9956] [ysql] support ybgin index in hstore extension
    • [10076] [docdb] [DST] Enhance rocksdb UI page with info about OPTIONS
    • [10159] [ysql] Tablegroup placement
    • [10168] [ysql] support tablegroup dumps
    • [10217] [10791] [docdb] Bootstrap from tablet with leader in same AZ, remove skip leader code.
    • [10456] [ysql] Import Fix detaching partitions with cloned row triggers
    • [10479] [docdb] Multi-RAFT V0 - Batch Heartbeat Requests (Bypass Heartbeats)
    • [10510] [YSQL] Improve performance of concurrent refreshes for matviews
    • [10527] [ycql] Upgrade Cassandra driver to 4.6.0-yb-10
    • [10548] [Spark Connector] Upgrade to Spark Connector 3.0-yb-10
    • [10571] A separate flag to check for large clock skew on hybrid clock reads
    • [10584] [DocDB] Add support for precompiled headers
    • [10584] [DocDB] Generate protobuf forward headers
    • [10584] [DocDB] Split complex proto files into smaller parts
    • [10584] [DocDB] Split master service into multiple services
    • [10643] [10642] Create tool to modify hybrid time in SST and WAL files
    • [10668] [ysql] Import Adjust pg_dump's priority ordering for casts.
    • [10717] [YSQL] Reduce execution time of OpenTable by table_id
    • [10748] [ysql] Import Warning on SET of nonexisting setting with a prefix reserved by an extension
    • [10775] [ysql] Import Avoid leaking memory during large-scale REASSIGN OWNED BY operations.
    • [10857] [DocDB] Lightweight protobuf generation
    • [10862] [ysql] Tab Completion for Tablegroups
    • [10883] [YSQL] Rename the GUC for disabling transactional writes
    • [10920] Support build with Clang 12 and Linuxbrew

    Bug Fixes

    Yugabyte Platform

    • [PLAT-32] provision_instance.py should not have any version specific dir paths (#5538)
    • [PLAT-580] Fix sync api bug with previously-existing xcluster configs
    • [PLAT-1955] Preflight checks also take into account free space
    • [PLAT-2007] Do not enable leader blacklisting for RF1 clusters
    • [PLAT-2055] [UI] Fix UI on clicking Retry Task button
    • [PLAT-2080] Set Universe keys before starting tserver on encryption at rest enabled universes.
    • [PLAT-2099] Fix abort Multi-namespace backup task
    • [PLAT-2241] Replace non-unicode symbols in a tool output decode
    • [PLAT-2252] fixing Oracle8 ntpd health check
    • [PLAT-2316] Fix local privilege escalation issue with collect_metrics script
    • [PLAT-2331] Install s3cmd/gsutil on graviton
    • [PLAT-2342] [UI] Make sure "Desired Home Directory" and Mount path are different for on-prem providers
    • [PLAT-2420] ".local" as a TLD in AWS host base
    • [PLAT-2424] [UI] Encryption At Rest setting is not persistent while configuring Read replica
    • [PLAT-2434] Disabling logging config customization
    • [PLAT-2434] Fix omitted log configurations after application start
    • [PLAT-2457] Fix overwriting of username and password for slow queries
    • [PLAT-2478] Add synchronization to release APIs
    • [PLAT-2481] Fix log filtering error
    • [PLAT-2484] Set incrementVersion to false in unlockUniverseUpdate
    • [PLAT-2497] Disable cert checking when using s3client for backups
    • [PLAT-2498] Fix for Wrong timestamps in Health Check and Users Tab
    • [PLAT-2514] Fix leaderless and underreplicated tablet alerts in case master quorum is missing
    • [PLAT-2517] Fix typo (missing backslash) in the priority-regex for Prometheus in replicated.yaml
    • [PLAT-2521] Health checks failing for universe with read replica after full move
    • [PLAT-2528] adding architecture suffix to node_exporter role
    • [PLAT-2530] Use DB transaction to lock Universe and any DB changes in the callback during the locking
    • [PLAT-2533] Fix build version health check for invalid build names
    • [PLAT-2537] Fix alert count for read-only user Summary: Alert count endpoint is using POST request, which is not available by default for read-only user. Need to add it manually to the exclusion list.
    • [PLAT-2541] CreateUniverse retry - ServerSetup state match missing
    • [PLAT-2568] universe_configure doesn't return a good/detailed message in case of errors
    • [PLAT-2569] ignoring squashfs in disk utilization health check
    • [PLAT-2570] Add run_node_action.py and node_client_utils.py scripts to platform release
    • [PLAT-2579] Disable strict host checking for run_node_action.py
    • [PLAT-2580] When decrypt universe key is called, also attempt to renew token and update KMSConfig.
    • [PLAT-2586] Fix logs download
    • [PLAT-2587] Error during create universe for yugabundle
    • [PLAT-2590] Some tasks create multiple SubTaskGroupQueue in the same run() method with the expectation to run only the subtasks added to the queue.
    • [PLAT-2602] Fix various write read table creation issues
    • [PLAT-2612] Health check not stopped for the Universe being edited
    • [PLAT-2626] Fix backup deletion issue during destroy universe
    • [PLAT-2634] Fix xcluster APIs for HA-enabled platform deployments
    • [PLAT-2634] Log exception on cluster config version increment failure
    • [PLAT-2658] [Hc-Vault] Validation for vault address field not working
    • [PLAT-2672] Skip system_platform DB in 'All namespaces' YSQL backup
    • [PLAT-2681] Hashicorp config - attempting to add a config fails when transit mount path specified does not have any keys
    • [PLAT-2700] Fix OOM kills alert for cron universe + fix QueryAlerts execution
    • [PLAT-2701] Instances getting added to incorrect provider in onprem multi-provider config
    • [PLAT-2752] Increase running task beating time interval from 1sec to 10secs.
    • [PLAT-2754] Add run_node_action.py to the bin section of yb_release_manifest.json
    • [PLAT-2765] Redact GCP creds in provider creation audit log
    • [PLAT-2791] RunYsqlUpgrade subtask should not be run on upgrades of universes with YSQL disabled
    • [Platform] [10950] [Backups] Allow TS to have no tablets for the table
    • [Platform] Reset pg_stat_statements after creating/updating db credentials
    • [Platorm] [CLOUDGA-2711] wait for boot script completion + save vm console output if provisioning fails
    • [Platorm] Wait for boot script completion + save vm console output if provisioning fails

    Database

    • [2266] [YSQL] Fix YSQL PG crypto function crash issue
    • [6149] [10587] [YSQL] Partitioned table primary key is not correctly inherited by partitions
    • [10096] [ycql] Error out when ignore_null_jsonb_attributes is misspelled
    • [10482] [DST] Rate-limit Create/Delete Snapshot requests
    • [10547] [YBASE] [DocDB] Avoid unnecessary GetTransactionStatus calls during reads #10547
    • [10589] [YSQL] ysql_dump duplicates primary key for partition table
    • [10602] [DocDB] Initialize new fs root dirs
    • [10742] [Spark Connector] Accommodate extra backticks in Spark 3.2.0 Plan
    • [10773] [YSQL] [Backups] Change YSQLDump during the backup restoring
    • [10792] [ysql] Add YB operations to validate replication info
    • [10845] [xCluster] Fix rename universe replication with multiple renames
    • [10858] [YSQL] Disable sending DocDB requests with trivially false ANY filters
    • [10868] [YSQL] Make ALTER ADD PK persist reloptions
    • [10881] [DocDB] Fix "loop variable is always a copy because the range of type"
    • [10907] [Backup] [platform] Prevent double config file uploading.
    • [10943] [YSQL] Masking ldapbindpasswd in logs after authentication fails
    • [10962] Use random_generator_mt19937 which is seeded via /dev/urandom
    • [10963] Avoid float division by zero error
    • [11008] [YSQL] Fix failures in concurrent refreshes for matviews
    • [11054] [YCQL] Aggregate updates to JSONB column before inserting Subdoc
    • [adhoc] [DST] Downgrade noisy log when there are no memtables to flush

    Known Issues

    Yugabyte Platform

    N/A

    Database

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.11.1(Jan 7, 2022)

    v2.11.1.0 - December 15, 2021

    Build: 2.11.1.0-b305

    Downloads

               

    Docker

    docker pull yugabytedb/yugabyte:2.11.1.0-b305
    

    New Features

    Yugabyte Platform

    • [PLAT-526] [UI] UI support for xCluster replication management phase-1
    • [PLAT-529] GFlags validation APIs

    Database

    • [6639] [YSQL] Add ALTER TABLE SET TABLESPACE
    • [8242] [DST] Enable automatic tablet splitting by default
    • [9468] [YSQL] [Part-1] READ COMMITTED isolation level
    • [9984] Support arm64 build on macOS
    • [10563] [xCluster] Create Rename Replication Command
    • [10655] [YSQL] Add --postmaster_cgroup gflag to tserver
    • [10671] [yugabyted] use env var YB_DISABLE_CALLHOME

    Improvements

    Yugabyte Platform

    • [8517] [6302] [5355] Auto populate provider dropdown, remove parenthesis around range partition and adjust DB version margin (#10229)
    • [8517] Add handling for provider input in edit universe scenario
    • [PLAT-506] Allow custom port in AWS host base
    • [PLAT-1773] Auto refresh of metrics page with intervals dropdown
    • [PLAT-1783] Add loading icon for universe list page
    • [PLAT-1923] [PLAT-1924] Ability to create universes with AZs count > RF + definition of default region
    • [PLAT-1931] Add Prometheus links to metrics panels
    • [PLAT-1941] Add status quick filter and table actions
    • [PLAT-1944] Allow customers to modify platform log rollover and retention settings
    • [PLAT-1950] Collect additional metrics
    • [PLAT-1957] Check ports availability in preflight check
    • [PLAT-1993] [UI] Show backup config name in list of backups
    • [PLAT-2004] Add leader blacklisting to UpgradeTaskBase
    • [PLAT-2061] Add PagerDuty and WebHook channels
    • [PLAT-2063] YSQL connections count, Leaderless tablets, Under-replicated tablets, Missing master leader alerts
    • [PLAT-2064] DB Overload alerts
    • [PLAT-2108] NFS backup with Encryption at rest enabled
    • [PLAT-2143] [UI] Add an optional field AWS KMS Endpoint while creating KMS config
    • [PLAT-2154] Add UI support for new log search API endpoint
    • [PLAT-2173] Task Improvement - implement ways to save and read states changed by subtasks
    • [PLAT-2195] Add support for multi-universe filter alert count endpoint
    • [PLAT-2227] Added a runtime config setting for disabling all the cert validations
    • [PLAT-2229] Retrieve YB version via ssh/kubectl during health check
    • [PLAT-2237] Add leader blacklisting to stop node in universe
    • [PLAT-2244] Update instance creation and listing to accept and return node UUID and Universe UUID.
    • [PLAT-2261] Enhancements to /logs UI endpoint
    • [PLAT-2303] Change Prometheus link icon in Metrics Page
    • [PLAT-2306] Add UI support for GCP user tags
    • [PLAT-2345] Show metrics for paused universes
    • [PLAT-2348] Increase the configuration name max size to 100
    • [PLAT-2361] Filter and sort alert configurations on various fields + return alert count in page API
    • [PLAT-2467] [UI] XCluster UI Changes
    • Update certs to use certLabel as CN

    Database

    • [1127] [YSQL] Improve collation upgrade performance.
    • [1127] [YSQL] Use attribute collation for SPLIT AT clause
    • [1979] Support PG11 text search configuration
    • [5395] [docdb] Add metrics for master async tasks
    • [7809] [YSQL] Add session flag to enable non-transactional writes on COPY
    • [8597] [YSQL] Changing Catalog Version Mismatch in Postgres layer as retryable error
    • [10158] [Geo] Selecting different txn status table based on local vs global requirements
    • [10190] [YSQL] create and use the pg_yb_tablegroup table
    • [10159] [Geo] Added global_transaction session variable
    • [10352] [YSQL] Import Invalidate partitions of table being attached/detached
    • [10358] [xCluster] [Tablet-splitting] Handle consumer side splits
    • [10428] [DocDB] Improve logging for SST file expiration
    • [10388] [YSQL] Import Fix pg_dump for disabled triggers on partitioned tables
    • [10431] [YSQL] Import pg_dump: fix mis-dumping of non-global default privileges.
    • [10433] [YSQL] Catalog cache refresh improvements
    • [10435] [DocDB] Display split depth in the master Tables UI
    • [10503] [DocDB] Add GFlag to trust value-level TTL metadata during file expiration.
    • [10543] [YCQL] Allow NULL in filtering clauses WHERE/IF inside IN/NOT IN operators in SELECT statement.
    • [10576] [YSQL] Import Reject extraneous data after SSL or GSS encryption handshake.
    • [10579] [YSQL] Import libpq: reject extraneous data after SSL or GSS encryption handshake.
    • [10586] [YSQL] Reduce number of reads from the pg_yb_catalog_version table on staring new connection
    • [10604] [YCQL] Handle DeleteIndex while backfilling
    • Update cassandra driver version to 3.10.3-yb-2 towards vulnerabilities fixes.

    Bug Fixes

    Yugabyte Platform

    • [PLAT-1643] On-prem provider configuration, "Desired Home Directory" can't be equal to the first mount paths item
    • [PLAT-1678] Multitable backup fails with NPE
    • [PLAT-1702] [UI] Azure YW provider does not allow port customization
    • [PLAT-1795] /home/yugabyte not getting cleared out with a custom home dir for onprem universe
    • [PLAT-1891] [Backup] [IAM-Platform] Backup is hanging for universe with read replicas on IAM-enabled platform
    • [PLAT-1948] [UI] Delete release should call delete API, not update release API
    • [PLAT-2088] Some primary cluster fields not inherited properly into add read replica page
    • [PLAT-2098] fix: Certificate details page shows 'invalid date' for certificate start and expiration fields on Safari Browser only
    • [PLAT-2099] Remove taskUUID from audit entry while aborting backup
    • [PLAT-2101] fix: Save button doesn't throw field errors when all the fields are empty
    • [PLAT-2102] Stop creation of KMS configs with duplicate name
    • [PLAT-2103] [UI] [Encr at rest] AWS credentials enabled when IAM is checked in
    • [PLAT-2109] Skip hostname validation in certificate
    • [PLAT-2121] [UI] [KMS] UI displaying Successfully added even when it fails to add KMS config
    • [PLAT-2133] Need to allow platform to repair a server that's stuck in a OOM state
    • [PLAT-2148] Unable to change availability zone while configuring new universe
    • [PLAT-2158] Getting rid of useHostnames
    • [PLAT-2177] Avoid adding duplicate delete backup tasks to tasks queue
    • [PLAT-2184] Fix a bug where new leader thinks of itself as remote instance
    • [PLAT-2189] Fix universe creation on airgap install
    • [PLAT-2214] fix: Cloud provider section overlapping with universe details
    • [PLAT-2218] Removing and adding a single AZ to univ placement triggers a full move
    • [PLAT-2241] Replace non-unicode symbols on traceback output decode
    • [PLAT-2243] Fix chrony for airgap install (aws, gcp)
    • [PLAT-2246] Fix Timezone UI Bugs
    • [PLAT-2250] [Alerts] [UI] Alert Creation page is in loading state in brand new portal
    • [PLAT-2251] Make health check run and store results to DB without health config
    • [PLAT-2253] Skip delete release from local filesystem step in case of non-local release
    • [PLAT-2257] Provision in AnsibleSetupServer cannot be retried if it fails in the middle because the default SSH port is no longer reachable.
    • [PLAT-2259] [xcluster] [UI] Master Node Address unavailable on replication tab
    • [PLAT-2265] Resolve missing list key and clean up YBMultiSelect styling.
    • [PLAT-2269] Hide quick filter
    • [PLAT-2270] Non-transactional backup with list of tables in YCQL fails
    • [PLAT-2274] [UI] No way to remove region from provider
    • [PLAT-2294] [UI] Disable toggle TLS / cert rotation dialogs in UI for k8s universes
    • [PLAT-2298] Allow outOfOrder migrations and ignoreMissingMigrations
    • [PLAT-2319] Remove sensitive data from platform log while during universe creation using s3 package
    • [PLAT-2320] Fix Prometheus links to have correct timestamp
    • [PLAT-2321] [PLAT-2314] Fix backup util behavior on error for onprem providers
    • [PLAT-2323] Preparation subtask with error message should be added only if no other subtasks exist
    • [PLAT-2325] Fix broken migration in older releases
    • [PLAT-2326] Incorrect masters selection for GP universe
    • [PLAT-2328] Reuse instance in cloud provider if it matches the universe ID and node ID tags.
    • [PLAT-2344] Fix preflight checks
    • [PLAT-2349] [UI] Blank screen when trying to add read replica to an universe
    • [PLAT-2351] Disable upgrade to systemd for manually provisioned universes
    • [PLAT-2373] Get rid of repeated migrations
    • [PLAT-2376] Fixing resize node to utilize ssh_user param
    • [PLAT-2400] Universe scaling out by adding nodes keeps new nodes in blacklisted state.
    • [PLAT-2434] Fix omitted log configurations after application start
    • [PLAT-2514] Fix leaderless and underreplicated tablet alerts in case master quorum is missing
    • [PLAT-2348] Fix UT failure during change in max size of config name
    • [T9574] [T8510] Force delete backup Configuration after deleting backups and schedule associated with it

    Database

    • [8660] [DocDB] Tablet splitting thresholds should be determined based on number of nodes in a table's placement policy
    • [9665] [YSQL] ysql_dump should not need master addresses to be passed explicitly.
    • [9929] [2809] [2742] [YSQL] Fix explicit row locking semantics + support SKIP LOCKED for REPEATABLE READ isolation level
    • [10163] [DST] Disable large transaction batch apply until it does not affect tablet splitting
    • [10193] [YSQL] Disallow deleting tablespace used by tablegroups
    • [10236] [YSQL] Properly assign junk_filter_needed value based on type of Relation
    • [10256] [DST] [PITR] Fix Incorrect Restore logic for System Catalog tables
    • [10262] [DocDB] Tablespace task in the YB-Master must first check whether ysql is enabled
    • [10303] [YSQL] fix unexpected transaction state error
    • [10472] Use curl instead of wget for client download script
    • [10519] Refresh master leader after election
    • [10519] Reset master leader on meta cache timeouts
    • [10521] [YSQL] Always republish migrations
    • [10591] [YSQL] COPY for partitioned table writes to parent instead of child nodes
    • [10622] [DocDB] Fix ASAN build
    • [10629] [YSQL] Ensure follower reads go to the follower for reads from the index
    • [10631] [YSQL] Hide index backfill auth key in logs
    • [10646] [YSQL] fix client backfill timeout
    • [10737] [backup] [YSQL] yb_backup must use '--masters' argument for ysql_dump.
    • [10741] [YSQL] change ybgin substitution msg to LOG
    • [10769] [Geo] Fixed data race when updating transaction status hash
    • [10185] Check for schema column id mismatch in SetupUniverseReplication

    Known Issues

    Yugabyte Platform

    • [PLAT-2487] When you add read replicas, the instance type and volume may not show any values. Re-select the provider and the values should populate correctly as primary cluster.

    • [PLAT-2490] You may see a "Warning: cronjobs are not active on some nodes" message on the Universe health check page, even when cron jobs are active on the database nodes. To remove the warning:

      • For Replicated- and Yugabundle-based Platform installs, launch the devops/bin/edit_universe_details.py script and globally replace the value of cronsActive field from true to false. The number of cronsActive entries corresponds to the number of database nodes in your universe — you need to replace all of them.

      • For Kubernetes-based Platform installs, please contact Yugabyte support for help as this requires modifying the values in the database directly.

    Database

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.8.1(Jan 7, 2022)

    v2.8.1.0 - January 4, 2022

    Build: 2.8.1.0-b37

    Downloads

               

    Docker

    docker pull yugabytedb/yugabyte:2.8.1.0-b37
    

    New features

    Yugabyte Platform

    • [PLAT-2295] [PLAT-2296] Timezone Selector on user profile
    • [PLAT-2306] Add UI support for GCP user tags

    Database

    • [7809] [YSQL] Add session flag to enable non-transactional writes on COPY
    • [10094] [DocDB] Implemented advanced delta encoding/decoding optimized for DocDB-specific rocksdb keys
    • [10204] [YSQL] Add functionality for the yb_fdw role
    • [10473] Implement YSQL Follower reads.
    • [10563] [xCluster] Create Rename Replication Command

    Improvements

    Yugabyte Platform

    • [PLAT-506] Allow custom port in AWS host base
    • [PLAT-2103] [UI] [Encr at rest] AWS credentials enabled when IAM is checked in
    • [PLAT-2298] Allow outOfOrder migrations and ignoreMissingMigrations
    • [PLAT-2420] ".local" as a TLD in AWS host base
    • [PLAT-2522] Backport Migrations for LDAP support
    • Add leader blacklisting to UpgradeTaskBase, Stop Processes, upgradeKubernetes, and handling rf1 universes
    • Update certs to use certLabel as CN

    Database

    • [10428] [DocDB] Improve logging for SST file expiration
    • [10482] [DST] Rate-limit Create/Delete Snapshot requests
    • [10503] [DocDB] Add GFlag to trust value-level TTL metadata during file expiration.
    • [10543] [YCQL] Allow NULL in filtering clauses WHERE/IF inside IN/NOT IN operators in SELECT statement.
    • [10579] [YSQL] Import libpq: reject extraneous data after SSL or GSS encryption handshake.
    • [10655] [YSQL] Add --postmaster_cgroup gflag to tserver
    • [10883] [YSQL] Rename the GUC for disabling transactional writes

    Bug fixes

    Yugabyte Platform

    • [10907] [Backup] Prevent double config file uploading.
    • [PLAT-1990] Ensure universe size doesn't change dramatically during a full move
    • [PLAT-2080] Set Universe keys before starting tserver on encrption at rest enabled universes.
    • [PLAT-2101] Save button doesn't throw field errors when all the fields are empty
    • [PLAT-2102] Stop creation of KMS configs with duplicate name
    • [PLAT-2298] Disable third-party flyway plugin in yugabundle
    • [PLAT-2121] [UI] [KMS UI] displaying Successfully added even when it fails to add KMS config
    • [PLAT-2177] Avoid adding duplicate delete backup tasks to tasks queue
    • [PLAT-2246] Fix Timezone UI Bugs
    • [PLAT-2311] API Should return region active status
    • [PLAT-2314] [PLAT-2321] Fix backup util behavior on error for onprem providers
    • [PLAT-2339] [UI] Disable toggle TLS / cert rotation dialogs in UI for k8s universes
    • [PLAT-2351] Disable upgrade to systemd for manually provisioned universes
    • [PLAT-2373] Get rid of repeated migrations
    • [PLAT-2400] Universe scaling out by adding nodes keeps new nodes in blacklisted state.
    • [PLAT-2427] [UI] No way to remove region from provider
    • [PLAT-2498] Fix for Wrong timestamps in Health Check and Users Tab
    • [PLAT-2517] Fix typo (missing backslash) in the priority-regex for prometheus in replicated.yaml

    Database

    • [10256] [PITR] Fix Incorrect Restore logic for System Catalog tables
    • [10591] [YSQL] COPY for partitioned table writes to parent instead of child nodes
    • [10646] [YSQL] fix client backfill timeout

    Known issues

    Yugabyte Platform

    • N/A

    Database

    • N/A
    Source code(tar.gz)
    Source code(zip)
  • v2.11.0(Nov 23, 2021)

    v2.11.0.0 - November 22, 2021

    Build: 2.11.0.0-b7

    Downloads

               

    Docker

    docker pull yugabytedb/yugabyte:2.11.0.0-b7
    

    New Features

    For all 2.11 new features, please refer to this release announcement blog post

    Yugabyte Platform

    • [CLOUDGA-2875] Add additional permissions to yb_superuser
    • [CLOUDGA-3033] Grant additional roles to yb_superuser
    • [PLAT-26] [9612] Add logs purge threshold option to zip_purge_yb_logs.sh
    • [PLAT-580] Add new xCluster create/get/edit/delete APIs and data model
    • [PLAT-1669] initial OEL 8 support
    • [PLAT-1748] Create POST and GET endpoint to generate support bundle and download support bundle v1
    • [PLAT-1817] Add support for new certificate creation when rotating certs
    • [PLAT-1856] Support pause/resume for GCP
    • [PLAT-1870] Add a table view to universe page
    • [PLAT-1941] Add status quick filter and table actions
    • [PLAT-2071] Implement read-only user functionality for Alert UIs
    • [PLAT-2104] Enable/disable Auth in k8s

    Database

    • [1127] [YSQL] Collation support
    • [2272] [YSQL] Migration framework for YSQL cluster upgrade
    • [7850] [YSQL] Implement GIN (YBGIN) indexes
    • [8242] Enable automatic tablet splitting by default
    • [8422] [YSQL] pg_stat_monitor extension
    • [9370] Set enable_stream_compression flag to true by default
    • [9595] [YSQL] Support YBGIN index in pg_trgm extension
    • [10019] [DocDB] Add support for ZLib compression
    • [10094] [DocDB] added data_block_key_value_encoding_format option
    • [10094] [DocDB] Implemented advanced delta encoding/decoding optimized for DocDB-specific RocksDB keys
    • [10141] [DocDB] Remove feature gate on savepoints
    • [10150] [YSQL] Add functionality for the yb_extension role
    • [10157] [Geo] Added command to create new transaction status tables
    • [10204] [YSQL] Add functionality for the yb_fdw role
    • [10473] Implement YSQL Follower reads

    Improvements

    Yugabyte Platform

    • [PLAT-1506] Enhancement: Support to create/mark Alert Definition Group in Active or Inactive state from UI
    • [PLAT-1513] Enhance metrics that use sum without to also exclude namespace_name (#9759)
    • [PLAT-1580] Implement OOM killer alert
    • [PLAT-1585] k8s example for create universe
    • [PLAT-1704] Make Platform health checks more scalable
    • [PLAT-1731] Add more logging for Platform HA feature
    • [PLAT-1740] Make backup utility python3-compatible for different OS.
    • [PLAT-1753] Enable taking backups using custom ports
    • [PLAT-1760] Add readable type names
    • [Plat-1777] Add basic filtering and sorting
    • [Plat-1797] Create a pagination component
    • [PLAT-1808] [Alert UI] cleanup tasks
    • [PLAT-1867] AWS Provider and Universe examples
    • [PLAT-1916] Moved default access token key to configuration
    • [PLAT-1934] Adding UI to set KUBE_DOMAIN
    • [PLAT-1953] Improve performance of /logs endpoint
    • [PLAT-1956] Expose preflight check as a standalone action
    • [PLAT-1962] Add optional AWS KMS Endpoint field while creating KMS configuration.
    • [PLAT-1967] API Add support for k8s provider creation
    • [PLAT-1989] Show alert configuration target in page view
    • [PLAT-2032] Append number to self-signed certificate labels when rotating certs
    • [PLAT-2033] [Alert] [UI] Move seconds in Duration under conditions similar to Threshold in Alert Definition Page
    • [PLAT-2034] Specific task type name for TLS toggle
    • [PLAT-2093] Replace pause icon with resume icon for resume universe

    Database

    • [1127] [YSQL] Improve collation upgrade performance.
    • [3745] [DocDB] Added flag for making log cache memory percent-based.
    • [5310] [YSQL] Cherry-pick upstream PostgreSQL commit that performs refactor of ExecUpdate() function.
    • [5310] [YSQL] Support row-level partition UPDATE across partitions
    • [7293] [YSQL] Import Fix tablespace handling for partitioned tables
    • [8242] [DocDB] Update defaults for automatic tablet splitting
    • [8862] [DocDB] Enable CHANGE_CONFIG_OP to be added to raft log for split tablet
    • [9178] [YSQL] Support a way to read from local partitions first
    • [9467] [YSQL] Increase scope of cases where transparent retries are performed
    • [9468] [YSQL] [Part-1] READ COMMITTED isolation level
    • [9512] Add optional bootstrap IDs parameter to AlterUniverseReplication add_tables
    • [9606] [DocDB] Add flag --force for command delete_tablet to set state TABLET_DATA_DELETED for tool yb-ts-cli
    • [9969] [DocDB] Add a gflag for RocksDB block_restart_interval
    • [10038] [YQL] Support for displaying the bind values for a prepared statement(s).
    • [10110] [DocDB] Enables compaction file filter during manual compactions
    • [10136] [YSQL] Import Reject SELECT ... GROUP BY GROUPING SETS (()) FOR UPDATE.
    • [10151] [YSQL] Import Avoid fetching from an already-terminated plan.
    • [10199] [YSQL] Import Reset memory context once per tuple in validateForeignKeyConstraint.
    • [10208] [YSQL] Adding negative caching for types and metrics collection for catalog cache misses
    • [10211] [xCluster] Allow for overriding the default CDCConsumerHandler thread pool size
    • [10266] [YSQL] Import In security-restricted operations, block enqueue of at-commit user code.
    • [10317] [YSQL] Import Allow users with BYPASSRLS to alter their own passwords.
    • [10335] [YSQL] Import Avoid lockup of a parallel worker when reporting a long error message.
    • [10343] [DocDB] Adds a GFlag to ignore value-level TTL during SST file expiration
    • [10359] [YCQL] [YSQL] Update dependencies for Java subprojects
    • [10377] [DocDB] Add multi drives to TS servers on ExternalMiniCluster
    • [10381] [DocDB] enhance debug logs for RWCLock
    • [10382] [YSQL] Import Make pg_dump acquire lock on partitioned tables that are to be dumped.
    • [10385] [YSQL] Import Add strict_multi_assignment and too_many_rows plpgsql checks
    • [10386] [YSQL] Import Fix some errhint and errdetail strings missing a period
    • [10406] [YSQL] Create callback for index write
    • [10427] [DocDB] Added transaction_table_num_tablets_per_tserver flag
    • [10487] [10489] [YSQL] Import Prevent drop of tablespaces used by partitioned relations
    • [YSQL] Import Fix cloning of row triggers to sub-partitions

    Bug Fixes

    Yugabyte Platform

    • [CLOUDGA-2800] Fix possible customer removal issue related to persisted metrics
    • [PLAT-364] [9391] Incorrect masters selection leads to universe creation failures
    • [PLAT-490] Display timezone with timestamp
    • [PLAT-1504] Delete release from yugaware_property table and local filesystem.
    • [PLAT-1511] Fix legend overflowing in metrics tab
    • [PLAT-1607] Upgrade systemd API fix
    • [PLAT-1618] Make TLS Enable/Disable UI to display as default instead of under feature flag
    • [PLAT-1634] Backup page is not loading because of empty configuration column
    • [PLAT-1716] Import releases modal on releases page doesn't work
    • [PLAT-1751] [UI] DB Version field setting getting reset to first item in the dropdown on toggling between the Read Replica and Primary cluster tabs
    • [PLAT-1752] Potential resource leak in thread pool for subtasks
    • [PLAT-1789] [PLAT-1727] Addition/Removal of default platform gflags does not retain the universe's initial preferences.
    • [PLAT-1803] Not able to change cert for client to node in TLS enable feature
    • [PLAT-1806] Resolve issue in TlsToggle where certs_for_client_dir is set as empty
    • [PLAT-1819] [PLAT-1828] Release backup lock when Platform restarts, and update Backup state
    • [PLAT-1824] Improve backup retention in case of backup failure
    • [PLAT-1829] [ycql/ysql auth password] Wrong error message
    • [PLAT-1831] Fix DB version dropdown from being reset when switching between primary and async cluster forms
    • [PLAT-1831] Fix when navigating from home page to Create Universe
    • [PLAT-1836] Clean up needless releases data from YB nodes
    • [PLAT-1837] Change Replication factor field to be editable for async universe form.
    • [PLAT-1840] Fix 30 sec customer_controller list API
    • [PLAT-1842] Fix long universe list query
    • [PLAT-1845] GET /api/v1/customers/cUUID/universes/uniUUID/leader takes 9 seconds on dev portal
    • [PLAT-1853] Frequent error log related to health checks on portal.k8s
    • [PLAT-1855] Edit Universe example and missing implicit parameters
    • [PLAT-1862] Backup Frequency cannot be negative number
    • [PLAT-1886] Set locale on GCP Ubuntu universes.
    • [PLAT-1887] Fix creation of read-only on-prem universe + code cleanup
    • [PLAT-1892] Remove default template for error log + remove error logs from health check report
    • [PLAT-1895] Fix backup failure alert in case restore fails
    • [PLAT-1897] Make client_max_body_size configurable in replicated
    • [PLAT-1897] Take-2. Make client_max_body_size configurable in replicated
    • [PLAT-1907] Mismatching address should not cause standby to overwrite local instance
    • [PLAT-1907] Missed a rename of assertYWSE to assertPlatformException
    • [PLAT-1921] [Backup] [UI] Disappearance of Encrypt backup toggle in 2.9.1 and 2.9.2 portals
    • [PLAT-1923] [PLAT-1924] Ability to create universes with AZs count > RF + definition of default region
    • [PLAT-1942] Backup/restore failing on KMS enabled universes
    • [PLAT-1952] Correctly mark status of root and client root during add operation
    • [PLAT-1953] Add /logs sh script to release package
    • [PLAT-1969] [UI] Universe creation - Create button is disabled when YSQL/YCQL auth is disabled
    • [PLAT-1972] Check certificates existence
    • [PLAT-1976] Fix EditUniverse for on-prem
    • [PLAT-1987] Only adding gcp_internal flag for GCP provider
    • [PLAT-1990] Ensure universe size doesn't change dramatically during a full move
    • [PLAT-1998] Fix NPE in SystemdUpgrade task for TLS enabled universes
    • [PLAT-1999] Update cert directories gflags during cert rotation
    • [PLAT-2002] Fixing zip_purgs_yb_logs to not error without threshold flag
    • [PLAT-2015] Remove Sort functionality from "Target universe" in alert listing
    • [PLAT-2018] Fix instance restart alert during universe operation
    • [PLAT-2019] Fix permission denied issues during find command
    • [PLAT-2030] [UI] [Platform]UI should display the name of the newly created cert instead of "Create new cert" option
    • [PLAT-2034] Fix migration version
    • [PLAT-2045] Minor fix to release name regex
    • [PLAT-2046] Yb-client: Possible race condition while getting Master-Leader
    • [PLAT-2049] Fix metric storage + fix health check for development and ARM builds
    • [PLAT-2053] Fix the wrong error message in TLS configuration modal
    • [PLAT-2068] [UI] Screen going blank when removed regions in Edit Universe
    • [PLAT-2069] Hiding systemd upgrade option for read-only users
    • [PLAT-2073] [UI] Enable Systemd Services toggle shows wrong status
    • [PLAT-2074] Alert configuration active status in page view + activate/deactivate from actions + sorting fixes
    • [PLAT-2081] Show Error message when trying to create existing user
    • [PLAT-2092] Fix Task list default sorting by create time
    • [PLAT-2094] Fix k8s universe certificate expiry checks
    • [PLAT-2096] [UI] Restore backup UI refresh issue
    • [PLAT-2097] Fix repeated migration V68 : approach 2
    • [PLAT-2107] Resolve multiple UI fixes in Encryption-at-Rest modal
    • [PLAT-2109] Skip hostname validation in certificate
    • [PLAT-2110] Fix wrong default destination migration for multi-tenant platforms.
    • [PLAT-2111] Systemd Upgrade failing with read replica
    • [PLAT-2113] Fix HA failing with entity_too_large
    • [PLAT-2124] [Alert] [UI] Select Alert Metrics doesn't load the template if the metrics is created twice
    • [PLAT-2126] Fix stopping periodical tasks in case of failure
    • [PLAT-2128] Fix alert message field to print the whole message + alert channel error message fix
    • [PLAT-2129] [Alert] Full Alert message is not displayed in Alert listing page on selecting the alert
    • [PLAT-2134] Fix beforeValidate migration for the case of empty database
    • [PLAT-2138] Fix OOM Kill alert query in DB migration
    • [PLAT-2157] Flyway plugin patch for ignoreMissingMigration and default java package issue
    • [PLAT-2167] Fix 3000 seconds timeout for IAM profile retrieval operation
    • [PLAT-2180] Missing error response logging when demoteInstance fails
    • [PLAT-2189] Fix universe creation on airgap install
    • [PLAT-2200] [UI] Fix regression with HA "standby" overlay

    Database

    • [2272] [YSQL] Fix OID generation for initdb migration
    • [2397] [YSQL] Fix wrong results after modification statement failure in procedure block
    • [2866] Add deadline to CdcProducer::GetChanges call
    • [5536] [YSQL] fast-path not used when type conversion from timestamp with & without timezone happens
    • [7092] [10046] [10222] [10224] [10230] [10251] [10295] Enable Clang 12 ASAN build on AlmaLinux 8 and fix relevant bugs
    • [8148] [DocDB] Potential issue on crash after creating post-split tablets
    • [8229] [Backup] repartition table if needed on YSQL restore
    • [8718] [YSQL] Free Bitmapset used in match_index_to_operand()
    • [9541] [YSQL] Restart metrics webserver when postmaster recovers
    • [9645] [YSQL] Cleanup DDL transaction state in case of query failure
    • [9898] [DocDB] Fix queries on system.partitions when unable to resolve some addresses
    • [9936] [YBase] Don't parse RequestHeader protobuf
    • [9936] Fix ysql_dump in encrypted k8s environment
    • [9957] [YSQL] Fix memory usage when translating decimal data into Postgres's datum format.
    • [10044] [DST] [PITR] Fix race in snapshot/schedule cleanup
    • [10071] Fix Locking Issues with DeleteTableInMemory
    • [10077] [DocDB] Compaction file filter factory uses HistoryRetention instead of Schema
    • [10082] Clean up environment on SetupUniverseReplication failure
    • [10116] [YSQL] Starting a new cluster with an old YSQL snapshot fails in debug build
    • [10164] [DocDB] Max file size for compactions should only affect TTL tables
    • [10166] Acquire lock in GetUniverseParamsWithVersion
    • [10167] [DocDB] Save source tablet mutations to sys catalog when splitting
    • [10207] [DocDB] Make read_buffer_memory_limit a percentage of process memory instead of total memory.
    • [10218] CheckLocalHostInMasterAddresses should check all specified RPC addresses
    • [10220] [DocDB] splitting: deprecate TabletForSplitPB.tablets_for_split
    • [10225] [DST] Adhere to the definitions of partitions_ and tablets_ during DeleteTable
    • [10240] Add IPv6 address filters to default value of net_address_filter
    • [10254] [YSQL] Fix 100% CPU usage regression bug in SELECT with FOR KEY SHARE/IN/missing keys
    • [10259] [YSQL] Allow setting tablegroups on tables using WITH
    • [10304] [DocDB] fix deadlock in ProcessTabletReportBatch
    • [10308] [YSQL] Prevent setting tablespaces for temp tables
    • [10314] [YSQL] remove mention of HaveYouForgottenAboutMigration
    • [10323] [YBase] Fix outbound call timeout handling
    • [10364] [YCQL] Fix issue when dropping col that is not in an existing non-partial secondary index
    • [10374] [YSQL] Cannot start a cluster with --ysql_pg_conf_csv='statement_timeout=1000'
    • [10415] [Backup] Backup-restore failures for old backups.
    • [10419] [YSQL] Shorten string to get rid of output truncation warning
    • [10430] [YSQL] Limit to IPv4 for sys catalog initialization
    • [10433] [YSQL] Load pg_depend and pg_shdepend on demand
    • [10496] [YSQL] Adjust cost when enable_seqscan=off
    • [adhoc] [DST] Reword loud log line in raft_consensus.cc to remove the word Failure

    Known Issues

    Yugabyte Platform

    N/A

    Database

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.8.0(Nov 16, 2021)

    v2.8.0.0 - November 8, 2021

    Build: 2.8.0.0-b37

    Downloads

         

    Docker

    docker pull yugabytedb/yugabyte:2.8.0.0-b37
    

    New features

    Please refer to our blog post for new features in this release - https://blog.yugabyte.com/announcing-yugabytedb-2-8/

    Improvements

    Yugabyte Platform

    [#9131] [Platform] Enable/disable YCQL endpoint while universe creation and force password requirement [#9580] [Platform] Add restore_time field for all universes. [#9613] [Platform] Update UI to accommodate Authentication changes [#9733] [Platform] [Alerts] Implement alert listing [#9978] [Platform] [UI] Change stop backup icon and label to abort icon and label. [PLAT-26] [#9612] Add logs purge threshold option to zip_purge_yb_logs.sh [PLAT-59] [#5236] Allow log levels to be changed through POST /logging_config endpoint [PLAT-386] [#9407] Implement base YSQL/YCQL alerts [PLAT-417] Add support for Ubuntu 18.04 distributions [PLAT-490] Display timezone with timestamp [PLAT-523] [#7645] Show error summary at the top of the health check email [PLAT-541] Allow configuring no destination for alert config + UI improvements [PLAT-1530] Made assignStaticPublicIP optional parameter for create universe. Ran swaggerGen. [PLAT-1556] List Storage Configs Create Scheduled backup examples [PLAT-1573] Adding 'create new cert' in enable TLS new feature [PLAT-1582] [Alert] Limit Severity to maximum 2(Severe/warn), now we can add multiple severity's but after edit we are displaying only 2 (1 Severe/1 Warn) [PLAT-1585] k8s example for create universe [PLAT-1620] Added secondary subnet for allowing two network interfaces [PLAT-1647] Provide more details for default channel on UI [PLAT-1664] Enable new alert UIs and remove deprecated alert UI + configs from Health tab + config from replication tab [PLAT-1669] initial OEL 8 support [PLAT-1691] Task, API and thread pool metrics [PLAT-1704] Make Platform health checks more scalable [PLAT-1704] WIP: Make Platform health checks more scalable [PLAT-1705] Add auditing and transaction for /register API action [PLAT-1731] Add more logging for Platform HA feature [PLAT-1747] supporting n2 instance types for GCP internally [PLAT-1774] Add a customer ID field in Customer Profile page [Plat-1777] Add basic filtering and sorting [PLAT-1793] DB Error logs alert [Plat-1797] Create a pagination component [PLAT-1808] [Alert UI] cleanup tasks [PLAT-1817] Add support for new certificate creation when rotating certs [PLAT-1818] Add pagination to Tables tab and add classNames [PLAT-1824] Improve backup retention in case of backup failure [PLAT-1855] Edit Universe example and missing implicit params [PLAT-1867] AWS Provider and Universe examples [PLAT-1934] Adding UI to set KUBE_DOMAIN [PLAT-1943] Remove feature flagging for enable/disable TLS [PLAT-1956] Expose on-prem node pre-flight checks as a standalone action [PLAT-1962] Add optional AWS KMS Endpoint field while creating KMS config. [PLAT-1989] Show alert configuration target in page view [PLAT-2033] [Alert] [UI] Move seconds in Duration under conditions similar to Threshold in Alert Definition Page [PLAT-2071] Implement read-only user functionality for Alert UIs [PLAT-2104] Enable/disable Auth in k8s [PLAT-2143] [UI] Add an optional field AWS KMS Endpoint while creating KMS config [PLAT-2229] Retrieve YB version via ssh/kubectl during health check [Platform] Return direct URL to Prometheus for metrics from metrics API call

    Database

    [#2220] [YSQL] Enabling relation size estimation for temporary tables in optimizer [#2272] [YSQL] Migration framework for YSQL cluster upgrade [#5492] yb-admin: Added error message when attempting to create snapshot of YCQL system tables [#6541] [YSQL] Enable row-locking feature in CURSOR [#7612] [DocDB] Allow TTL-expired SST files that are too large for compaction to be directly expired [#7612] [DocDB] Modified compaction file filter to filter files out of order [#7889] Reduce timeout for ysql backfill. [#8162] YSQL Support single-request optimization for UPDATE with RETURNING clause [#8229] [Backup] repartition table if needed on YSQL restore [#8242] [DocDB] Update defaults for automatic tablet splitting [#8402] [YSQL] change gin to ybgin for YB indexes [#8452] Speed up restoring YSQL system catalog [#8501] [DocDB] Add metric to monitor server uptime [#8807] [YBase] Add HTTP URL param for limiting the number of tables whose metrics are displayed [#8979] [DocDB] Improve master load balancer state presentation [#9279] [YSQL] Enable -Wextra on pgwrapper [#9279] [YSQL] Enable -Wextra on yql folder [#9370] Set enable_stream_compression flag to true by default [#9439] [YBase] Allow sst-dump to decode docdb keys and dump data in human readable format [#9467] [YSQL] Increase scope of cases where transparent retries are performed [#9512] Add optional bootstrap IDs parameter to AlterUniverseReplication add_tables [#9606] [docdb] Add flag --force for command delete_tablet to set state TABLET_DATA_DELETED for tool yb-ts-cli [#9969] [DocDB] Add a gflag for rocksdb block_restart_interval [#10019] [DocDB] Add support for zlib compression [#10064] [xCluster] Lag Metric Improvements [#10094] [DocDB] added data_block_key_value_encoding_format option [#10141] [DocDB] Remove feature gate on savepoints [#10150] [YSQL] Add functionality for the yb_extension role [#10240] Add IPv6 address filters to default value of net_address_filter [#10430] [YSQL] Limit to IPv4 for sys catalog initialization [YSQL] Foreign Data Wrapper Support Added more information in logs for understanding concurrency control + downgraded two log lines to VLOG(4) Added new AWS regions to metadata files

    Bug fixes

    Yugabyte Platform

    [#1525] [Platform] New Universe creation gets public IP assigned even with flag = false [#1598] [Platform] [UI] Suggested Default File Path for CA Signed Certificate and Private Key is Incorrect [#7396] [Platform] Splitting up create/provision tasks to delete orphaned resources [#7738] [PLAT-611] Health checks can overlap with universe update operations started after them [#8510] [Platform] Allow the deletion of Failed Backups [#9571] [Platform] Backup and Restore failing in k8s auth enabled environment [#9743] [Platform] Fix universe reset config option (#9863) [#9850] [YW] Correct the node path (#9864) [CLOUDGA-1893] [Platform] fix client-to-node cert path in health checks [PLAT-253] Fix the backupTable params while creating Table backups using APIs. [PLAT-253] Fix universe's backupInprogress flag to avoid multiple backup at a time due to low frequency scheduler. [PLAT-289] Stopped node should not allow Release action [PLAT-368] [#9366] Disable Delete Configuration button for backups when in use. [PLAT-482] [#7573] Health Checks should run when Backup/Restore Tasks are in progress [PLAT-509] [#9014] Refresh Pricing data for Azure provider seems to be stuck [PLAT-521] [#9315] BackupsController: small fixes required [PLAT-525] Add IP address to SAN of node certificates [PLAT-599] Fix error messages in alert destination and configuration services [PLAT-1511] Fix legend overflowing in metrics tab [PLAT-1520] Stop displaying external script schedule among Backup Schedules. [PLAT-1522] Fix s3 release breakage [PLAT-1523] Make Alert APIs to be consistent with UI terminology [PLAT-1528] Change YWError handler to default to json response on client error. [PLAT-1530] [#9794] Creates static IP during cluster creation for cloud free tier clusters. Releases IPs on deletion. [PLAT-1549] [PLAT-1697] Fix Stop backup race condition. Add non-schedlued backup examples [PLAT-1559] Stop the external script scheduler if the universe is not present. [PLAT-1563] Fix instance down alerts + make sure instance restart alert is not fired on universe operations [PLAT-1578] Do not specify storage class (use default if provided) [PLAT-580] Fix DB migration ordering; use repeatable for backport [PLAT-1586] [Alert] Able to add multiple alert configuration with same name. Add duplicate check for alert configuration name [PLAT-1599] [UI] Root Certificate and node-node and client-node TLS missing on Edit Universe [PLAT-1600] add conf entries for various ansible settings [PLAT-1603] YBFormInput's OnBlur throws error on AddCertificateForm [PLAT-1605] Fix duplicate alert definitions handling + all locks to avoid duplicates creation [PLAT-1606] Disk name too long for Google Cloud clone disk [PLAT-1607] Upgrade systemd API fix [PLAT-1611] Add python depedencies required for executing external scripts [PLAT-1613] [Alerts] Logs filled with NPE related to "Error while sending notification for alert " [PLAT-1617] Added GCP region metadata for missing regions. [PLAT-1617] Fix issue with GCP Subnet CIDR [PLAT-1619] Check for FAILED status in wait_for_snapshot method. [PLAT-1621] Health check failed in K8s portal [PLAT-1625] Fix task details NPE [PLAT-1626] Skip preprovision for systemd upgrade. [PLAT-1631] [Alert] Universe filter is not working in Alert Listing [PLAT-1634] Backup page is not loading because of empty config column [PLAT-1638] Fix naming convention for external script endpoints as per our standards [PLAT-1639] [PLAT-1681] Make proxy requests async to keep them from blocking other requests. Reduce log spew from akka-http-core for proxy requests. [PLAT-1644] Fix k8s universe creation failure for Platform configured with HA [PLAT-1646] Remove Unsupported Instance types from pull down menu for Azure [PLAT-1650] Added yum lock_timeout to prevent yum lockfile errors for use_custom_ssh_port.yml [PLAT-1653] Fix region get/list. [PLAT-1656] [UI] [Alert] Group Type filter is not working in Alert Listing [PLAT-1661] Fix alert messages for notification failures [PLAT-1664] Clean unused code [PLAT-1667] Platform should not scrape all per-table metrics from db hosts (part 2) [PLAT-1668] Yugabundle failing because can't find YWErrorHandler [PLAT-1682] Fix node comparison function from accessing undefined cluster [PLAT-1687] [Alert] Not able to create destination channel using "default recipients + default smtp settings + empty email field" [PLAT-1691] Set oshi LinuxFileSystem log level to ERROR [PLAT-1694] Fix Intermittent failure to back up k8s universe [PLAT-1707] Fix performance issue [PLAT-1715] Check for YB version only for 2.6+ release DB [PLAT-1717] Full move fails midway if system tablet takes more than 2 mins to bootstrap [PLAT-1721] Stop storage type from automatically changing when instance type is changed [PLAT-1723] Allow disabling prometheus management + write alerts and metrics effectively [PLAT-1726] Allow user to completely remove all gFlags after addtion of several gFlags. [PLAT-1730] Fix resize node logic for RF1 clusters [PLAT-1736] Create default alert configs and destination on DB seed [PLAT-1737] "This field is required" error message is shown on alert configuration creation with default threshold == 0 [PLAT-1740] [PLAT-1886] Make backup util python3 compatible for different OS. [PLAT-1746] Delete prometheus_snapshot directory once Platform backup package is created [PLAT-1751] [UI] DB Version field setting getting reset to first item in the dropdown on toggling between the Read Replica and Primary cluster tabs [PLAT-1753] Enable taking backups using custom ports [PLAT-1757] Health Check failure message has Actual and expected values interchanged [PLAT-1760] Add readable type names [PLAT-1761] Fix alert message in case of unprovisioned nodes [PLAT-1768] Universe tasks take lot more time because thread pool executors do not reach max_threads [PLAT-1780] Redact YSQL/YCQL passwords from task_info table. [PLAT-1791] Use hibernate validator for all alert related entities [PLAT-1796] Edit Universe page has password fields editable [PLAT-1802] Replication graphs stopped showing on replication tab (replicated.yml change) [PLAT-1803] Not able to change cert for client to node in tls enable feature [PLAT-1804] Fix 'Querying for {} metric keys - may affect performance' log [PLAT-1806] Resolve issue in TlsToggle where certs_for_client_dir is set as empty [PLAT-1816] Forward port restricted user creation to master [PLAT-1819] [PLAT-1828] Release backup lock when Platform restarts, and update Backup state [PLAT-1831] Fix DB version dropdown from being reset when switching between primary and async cluster forms [PLAT-1831] Fix when navigating from home page to Create Universe [PLAT-1833] Fix missing create time on alert configuration creation issue [PLAT-1837] Change Replication factor field to be editable for async universe form. [PLAT-1840] Fix 30 sec customer_controller list API [PLAT-1842] Fix long universe list query [PLAT-1853] Frequent error log related to health checks on portal.k8s [PLAT-1862] Backup Frequency cannot be negative number [PLAT-1887] fix creation readonly onprem universe + code cleanup [PLAT-1891] [Backup] [IAM-Platform] Backup is hanging for universe with read replicas on IAM-enabled platform [PLAT-1892] Remove default template for error log + remove error logs from health check report [PLAT-1895] Fix backup failure alert in case restore fails [PLAT-1897] [PLAT-1995] Make client_max_body_size configurable in replicated [PLAT-1897] Make client_max_body_size configurable in replicated [PLAT-1897] Take-2. Make client_max_body_size configurable in replicated [PLAT-1921] [Backup] [UI] Disappearance of Encrypt backup toggle [PLAT-1942] Backup/restore failing on KMS enabled universes [PLAT-1969] [UI] Universe creation - Create button is disabled when YSQL/YCQL auth is disabled [PLAT-1976] Fix EditUniverse for on-prem [PLAT-1998] Fix NPE in SystemdUpgrade task for TLS enabled universes [PLAT-2002] Fixing zip_purgs_yb_logs to not error without threshold flag [PLAT-2012] Update cert directories gflags during cert rotation [PLAT-2015] Remove Sort functionality from "Target universe" in alert listing. [PLAT-2019] Fix permission denied issues during find command [PLAT-2030] [UI] UI should display the name of the newly created cert instead of "Create new cert" option [PLAT-2032] Append number to self-signed certificate labels when rotating certs [PLAT-2034] Specific task type name for tls toggle [PLAT-2053] Fix the wrong error message in TLS configuration modal [PLAT-2068] [UI] Screen going blank when removed regions in Edit Universe [PLAT-2069] Hiding systemd upgrade option for ReadOnly users [PLAT-2073] [UI] Enable Systemd Services toggle shows wrong status [PLAT-2081] Show Error message when trying to create existing user [PLAT-2092] Fix Task list default sorting by create time [PLAT-2094] Fix k8s universe certificate expiry checks [PLAT-2096] [UI] Restore backup UI refresh issue [PLAT-2097] Fix repeated migration V68 : approach 2 [PLAT-2098] Certificate details page shows 'invalid date' for certificate start and expiration fields on Safari Browser only. [PLAT-2107] Resolve multiple UI fixes in Encryption-at-Rest modal [PLAT-2109] Skip hostname validation in certificate [PLAT-2110] Fix wrong default destination migration for multitenant Platforms. [PLAT-2111] Systemd upgrade failing with read replica [PLAT-2113] [PLAT-2117] Fix HA failing with entity_too_large [PLAT-2124] [Alert] [UI] Select Alert Metrics doesn't load the template if the metrics is created twice [PLAT-2126] Fix stopping periodical tasks in case of failure [PLAT-2128] Fix alert message field to print the whole message + alert channel error message fix [PLAT-2129] [Alert] Full Alert message is not displayed in Alert listing page on selecting the alert [PLAT-2134] Fix beforeValidate migration for the case of empty database [PLAT-2157] Flyway plugin patch for ignoreMissingMigration and default java package issue [PLAT-2167] Fix 3000 seconds timeout for IAM profile retrieval operation [PLAT-2180] [PLAT-2182] Missing error response logging when demoteInstance fails [PLAT-2189] Fix universe creation on airgap install [PLAT-2200] [UI] Fix regression with HA "standby" overlay [PLAT-2263] application.log not updating [Platform] Fix NPE in VM image upgrade for TLS enabled universes [Platform] Hooking GCP Create Method into Create Root Volumes method

    Database

    [#2272] [YSQL] Fix OID generation for initdb migration [#4421] [YCQL] Disallow Unauthenticated LDAP binding + add handling for ycql_ldap_search_filter [#5920] Fix bootstrapping with preallocated log segment [#7528] [YSQL] Error out when Tablespaces are set for colocated tables [#8043] [YBase] Remove information about LB skipping deleted tables from the admin UI [#8580] [#9489] [YSQL] Inherit default PGSQL proxy bind address from rpc bind address [#8675] [DocDB] Prevent tablet splitting when there is post split data [#8772] Fix fatal that occurs when running alter_universe_replication and producer master has [#8804] [YSQL] [backup] Support in backups the same table name across different schemas. [#8807] [YBase] Rename the flag controlling maxmimum number of tables to retrieve metrics for [#9061] [docdb] Master task tracking should point to the table it is operating on [#9436] [YSQL] Statement reads rows it has inserted [#9475] Fetch Universe Key From Masters on TS Init [#9541] [YSQL] Restart metrics webserver when postmaster recovers [#9616] Fix master crash when restoring snapshot schedule with deleted namespace [#9655] [xCluster] Label cdc streams with relevant metadata [#9668] Alert configurations implement missing parts and few small changes [#9685] [xCluster] Make delete_universe_replication fault tolerant [#9746] Set WAL footer close_timestamp_micros on Bootstrap [#9749] [DocDB] Log::CopyTo - fixed handling kLogInitialized state [#9762] [Part-1] Populate partial index predicate in "options" column of system_schema.indexes [#9781] Mark snapshot as deleted if tablet was removed [#9782] docdb Tablet Splitting - Wait for all peers to finish compacting during throttling [#9786] Universe Actions-> Add Read Replica is failing on 2.6.1.0-b23 [#9789] [docdb] Load Balancer should use tablet count while looking tablets to move [#9802] [xCluster] Set proper deadline for YBSession in CDCServiceImpl [#9803] [YSQL] Import Avoid trying to lock OLD/NEW in a rule with FOR UPDATE. [#9806] [DocDB] fixed Batcher::FlushBuffersIsReady [#9812] [YSQL] Check database is colocated before adding colocated option for Alter Table [#9822] [DocDB] Check table pointer is not nullptr before dereferencing [#9831] [YSQL] Import Fix broken snapshot handling in parallel workers. [#9855] [DocDB] Set aborted subtransaction data on local apply [#9860] [YSQL] fix limit vars to uint64 [#9862] Allow PITR in conjunction with tablet split [#9862] PITR: Allow consecutive restore [#9865] Fix internal retry of kReadRestart for SELECT func() with a DML in the func [#9867] [YSQL] Fix double type overflow in case of SET yb_transaction_priority_lower_bound/yb_transaction_priority_upperr_bound command [#9878] [YBase] Reduce regex expression evaluation in nested loop [#9892] Mask sensitive gflag info [#9898] [DocDB] Fix queries on system.partitions when unable to resolve some addresses [#9899] [YSQL] Import Fix corner-case uninitialized-variable issues in plpgsql. [#9906] [YSQL] Fix not being able to add a range primary key [#9909] [YSQL] further fix backup restore for NULL col attr [#9911] [YSQL] Import In pg_dump, avoid doing per-table queries for RLS policies. [#9922] [YSQL] Import Fix float4/float8 hash functions to produce uniform results for NaNs. [#9924] [YSQL] always check schema name on backup import [#9926] [YSQL] Import Disallow creating an ICU collation if the DB encoding won't support it. [#9927] YCQL - Handle unset correctly [#9932] [YSQL] Initialize t_ybctid field in acquire_sample_rows() [#9933] [Part-0] Update logic for using num_tablets from internal or user requests. [#9933] [YCQL] [Part-1] DESC TABLE does not directly match the "CREATE TABLE" command for number of tablets. [#9934] [docdb] Don't update rocksdb_dir on Remote Bootstrap [#9935] [YSQL] Import Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join. [#9936] Alter and Create table via PgClient [#9936] Fix ysql_dump in encrypted k8s environment [#9936] Fix ysql_dump in TLS encrypted environment [#9936] Generate session ID in tserver [#9936] Remove all direct YBClient usage from PgSession [#9940] [DocDB] use correct kv_store_id for post-split tablets [#9947] [YSQL] remove runtime tag for ysql_disable_index_backfill [#9957] [YSQL] Fix memory usage when translating decimal data into Postgres's datum format. [#9963] [Backup] fix to reallow YEDIS on restore [#9965] [YSQL] Fix copy/paste error causing incorrect conversion [#9966] [YSQL] Import Rearrange pgstat_bestart() to avoid failures within its critical section. [#9969] [DocDB] Couple of minor fixes [#9981] Fix transaction coordinator returning wrong status hybrid time [#9994] [YSQL] copy t_ybctid field in modify tuple functions [#9995] [YSQL] Import Fix EXIT out of outermost block in plpgsql. [#10025] [YSQL] Import jit: Do not try to shut down LLVM state in case of LLVM triggered errors. [#10034] [YSQL] Preserve operation buffering state in case of transparent retries [#10038] [YQL] Support for displaying the bind values for a prepared statement(s). [#10042] [Backup] allow system table for YEDIS restore [#10044] [DST] PITR - Fix race in snapshot/schedule cleanup [#10051] [DocDB] use RETURN_NOT_OK on an unchecked status [#10071] Fix Locking Issues with DeleteTableInMemory [#10072] [YSQL] Check the return status of certain YB functions [#10077] [DocDB] Compaction file filter factory uses HistoryRetention instead of Schema [#10082] Clean up environment on SetupUniverseReplication failure [#10085] YSQL fix FATAL caused by wrong sum pushdown [#10098] [YSQL] Fix index creation on temp table via ALTER TABLE [#10110] [DocDB] Enables compaction file filter during manual compactions [#10111] [YSQL] Import Force NO SCROLL for plpgsql's implicit cursors. [#10120] [DocDB] added safe version of FastDecodeSignedVarInt [#10121] [YSQL] Import Avoid misbehavior when persisting a non-stable cursor. [#10139] [YBase] Avoid unnecessary table locking in CatalogManager::DeleteYsqlDBTables [#10164] [DocDB] Max file size for compactions should only affect TTL tables [#10166] Acquire lock in GetUniverseParamsWithVersion [#10167] [DocDB] Save source tablet mutations to sys catalog when splitting [#10199] [YSQL] Import Reset memory context once per tuple in validateForeignKeyConstraint. [#10211] [xCluster] Allow for overriding the default CDCConsumerHandler threadpool size [#10218] CheckLocalHostInMasterAddresses should check all specified RPC addresses [#10254] [YSQL] Fix 100% CPU usage regression bug in SELECT with FOR KEY SHARE/IN/missing keys [#10304] [DocDB] fix deadlock in ProcessTabletReportBatch [#10317] [YSQL] Import Allow users with BYPASSRLS to alter their own passwords. [#10364] [YCQL] Fix issue when dropping col that is not in an existing non-partial secondary index [#10374] [YSQL] Cannot start a cluster with --ysql_pg_conf_csv='statement_timeout=1000' [#10415] [backup] Backup-restore failures for old backups. [#10519] Reset master leader on meta cache timeouts, refresh master leader after election. [adhoc] [DocDB] Remove GetTabletPeers method with return argument [adhoc] [DST] Reword loud log line in raft_consensus.cc to remove the word Failure [xCluster] [#9418] Add cdc_state Schema Caching to Producer Cluster [YBase] Properly pass number of tables via MetricPrometheusOptions [YSQL] [#9572] Correctly determine is_yb_relation for row-marked relations when preparing target list [YSQL] Change file and function names to match Yugabyte convention in catalog and access directories [YSQL] Import Fix performance bug in regexp's citerdissect/creviterdissect. Add S3 Bucket Host Base as endpoint in case of S3 compatible storage Fixed bug in yb-ctl for stopping processes, when os.kill raises an exception Increase column length for availability zone subnets

    Known issues

    Yugabyte Platform

    N/A

    Database

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.9.1(Nov 16, 2021)

    Note: To upgrade a pre-version 2.9.0.0 Yugabyte Platform or universe instance deployed on Kubernetes that did not specify a storage class override, you need to override the storage class Helm chart value (which is now "", the empty string) and set it to the previous value, "standard".

    For Yugabyte Platform, the class is yugaware.storageClass. For YugabyteDB, the classes are storage.master.storageClass and storage.tserver.storageClass.

    v2.9.1.0 - Oct 29, 2021

    Build: 2.9.1.0-b140

    Downloads

         

    Docker

    docker pull yugabytedb/yugabyte:2.9.1.0-b140
    

    New Features

    Yugabyte Platform

    • [9124] Create REST endpoints to manage async replication relationships
    • [9733] [Alerts] Implement alert listing
    • [PLAT-417] Add support for Ubuntu 18.04 distributions
    • [PLAT-1573] Adding 'create new cert' in enable TLS new feature
    • [PLAT-1620] Added secondary subnet for allowing two network interfaces
    • [PLAT-1669] Initial OEL 8 support
    • [PLAT-1695] Create new API endpoint to be able to query logs by Universe
    • [PLAT-1753] Enable taking backups using custom ports

    Database

    • [1127] [YSQL] Collation Support (part 2)
    • [1127] [YSQL] Collation Support (part 3)
    • [6541] [YSQL] Enable row-locking feature in CURSOR
    • [7850] [YSQL] create new access method ybgin
    • [8402] [YSQL] change gin to ybgin for YB indexes
    • [YSQL] Foreign Data Wrapper Support

    Improvements

    Yugabyte Platform

    • [5236] [PLAT-59] Allow log levels to be changed through POST /logging_config endpoint
    • [7396] Splitting up create/provision tasks to delete orphaned resources
    • [7645] [PLAT-523] Show error summary at the top of the health check email
    • [9131] Enable/disable YCQL endpoint while universe creation and force password requirement
    • [9407] [PLAT-386] Implement base YSQL/YCQL alerts
    • [9580] Add restore_time field for all universes.
    • [9613] Update UI to accommodate Authentication changes
    • [9668] Alert configurations implement missing parts and few small changes
    • [9794] [PLAT-1530] Creates static IP during cluster creation for cloud free tier clusters. Releases IPs on deletion.
    • [9892] Mask sensitive gflag info
    • [9978] Platform UI: Change stop backup icon and label to abort icon and label.
    • [CLOUDGA-2345] Implement MDC propagation and add request/universe ID to MDC
    • [PLAT-525] Add IP address to SAN of node certificates
    • [PLAT-541] Allow configuring no destination for alert config + UI improvements
    • [PLAT-1523] Update Alert APIs to be consistent with UI terminology
    • [PLAT-1528] Change YWError handler to default to JSON response on client error.
    • [PLAT-1546] [PLAT-1547] [PLAT-1571] [PLAT-1572] Add API docs for UniverseClustersController, and other misc fixes
    • [PLAT-1549] Add (non-generated client, "simple") Python API examples
    • [PLAT-1549] Cleanup list/create provider API usage examples
    • [PLAT-1555] Add Python API client example for create Universe
    • [PLAT-1555] Add Python API client example for list Universe
    • [PLAT-1556] List Storage Configs Create Scheduled backup examples
    • [PLAT-1582] [Alert] Limit Severity to maximum 2(Severe/warn), now we can add multiple severity's but after edit we are displaying only 2 (1 Severe/1 Warn)
    • [PLAT-1611] Add python depedencies required for executing external scripts
    • [PLAT-1647] Provide more details for default channel on UI
    • [PLAT-1664] Enable new alert UIs and remove deprecated alert UI + configs from Health tab + config from replication tab
    • [PLAT-1691] Set oshi LinuxFileSystem log level to ERROR
    • [PLAT-1691] Task, API and thread pool metrics
    • [PLAT-1705] Add auditing and transaction for /register API action
    • [PLAT-1723] Allow disabling prometheus management + write alerts and metrics effectively
    • [PLAT-1766] [Alerts] UI: Cleanup
    • [PLAT-1774] Add a customer ID field in Customer Profile page
    • [PLAT-1791] Use hibernate validator for all alert related entities
    • [PLAT-1818] Add pagination to Tables tab and add classNames
    • Added new AWS regions to metadata files.
    • Hooking GCP Create Method into Create Root Volumes method

    Database

    • [2220] [YSQL] Enabling relation size estimation for temporary tables in optimizer
    • [5492] yb-admin: Added error message when attempting to create snapshot of YCQL system tables
    • [7612] [DocDB] Allow TTL-expired SST files that are too large for compaction to be directly expired
    • [7612] [DocDB] Modified compaction file filter to filter files out of order
    • [7889] Reduce timeout for ysql backfill.
    • [8043] [YBase] Remove information about LB skipping deleted tables from the admin UI
    • [8162] [YSQL] Support single-request optimization for UPDATE with RETURNING clause
    • [8229] [backup] repartition table if needed on YSQL restore
    • [8452] Speed up restoring YSQL system catalog
    • [8501] [DocDB] Add metric to monitor server uptime
    • [8508] [DocDB] moved GetSplitKey from TabletServerAdminService into TabletServerService
    • [8580] [9489] [YSQL] Inherit default PGSQL proxy bind address from rpc bind address
    • [8804] [YSQL] [backup] Support in backups the same table name across different schemas.
    • [8807] [YBase] Add a limit on number of metrics for the prometheus metrics endpoint
    • [8979] [DocDB] Improve master load balancer state presentation
    • [9279] [YSQL] Enable -Wextra on pgwrapper
    • [9279] [YSQL] Enable -Wextra on yql folder
    • [9418] [xCluster] Add cdc_state Schema Caching to Producer Cluster
    • [9439] [YBase] Allow sst-dump to decode DocDB keys and dump data in human readable format
    • [9467] [YSQL] Increase scope of cases where transparent retries are performed
    • [9685] [xCluster] Make delete_universe_replication fault tolerant
    • [9739] Added placement info to /api/v1/tablet-servers
    • [9746] Set WAL footer close_timestamp_micros on Bootstrap
    • [9762] [Part-1] Populate partial index predicate in "options" column of system_schema.indexes
    • [9803] [YSQL] Import Avoid trying to lock OLD/NEW in a rule with FOR UPDATE.
    • [9831] [YSQL] Import Fix broken snapshot handling in parallel workers.
    • [9862] [PITR] Allow consecutive restore
    • [9862] Allow PITR in conjunction with tablet split
    • [9899] [YSQL] Import Fix corner-case uninitialized-variable issues in plpgsql.
    • [9911] [YSQL] Import In pg_dump, avoid doing per-table queries for RLS policies.
    • [9922] [YSQL] Import Fix float4/float8 hash functions to produce uniform results for NaNs.
    • [9926] [YSQL] Import Disallow creating an ICU collation if the DB encoding won't support it.
    • [9935] [YSQL] Import Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
    • [9936] Remove YBClient from Postgres: Introduce PgClient and implement ReserveOids using it; Open table via PgClient; Remove all direct YBClient usage from PgSession
    • [9966] [YSQL] Import Rearrange pgstat_bestart() to avoid failures within its critical section.
    • [9995] [YSQL] Import Fix EXIT out of outermost block in plpgsql.
    • [10025] [YSQL] Import jit: Do not try to shut down LLVM state in case of LLVM triggered errors.
    • [10034] [YSQL] Preserve operation buffering state in case of transparent retries
    • [10064] [xCluster] Lag Metric Improvements
    • [10111] [YSQL] Import Force NO SCROLL for plpgsql's implicit cursors.
    • [10121] [YSQL] Import Avoid misbehavior when persisting a non-stable cursor.
    • [YSQL] Import Fix performance bug in regexp's citerdissect/creviterdissect.

    Bug Fixes

    Yugabyte Platform

    • [1525] New Universe creation gets public IP assigned even with flag = false
    • [1598] [UI] Suggested Default File Path for CA Signed Certificate and Private Key is Incorrect
    • [7573] [PLAT-482] Health Checks should run when Backup/Restore Tasks are in progress
    • [7738] [PLAT-611] Health checks can overlap with universe update operations started after them
    • [8510] Allow the deletion of Failed Backups
    • [9014] [PLAT-509] Refresh Pricing data for Azure provider seems to be stuck
    • [9315] [PLAT-521] BackupsController: small fixes required
    • [9366] [PLAT-368] Disable Delete Configuration button for backups when in use.
    • [9850] [YW] Correct the node path (#9864)
    • [CLOUDGA-1893] fix client-to-node cert path in health checks
    • [PLAT-253] Fix the backupTable params while creating Table backups using Apis.
    • [PLAT-253] Fix universe's backupInprogress flag to avoid multiple backup at a time due to low frequency scheduler.
    • [PLAT-289] Stopped node should not allow Release action
    • [PLAT-580] Fix create xCluster config API call
    • [PLAT-599] Fix error messages in alert destination and configuration services
    • [PLAT-1520] Stop displaying external script schedule among Backup Schedules.
    • [PLAT-1522] Fix s3 release breakage
    • [PLAT-1549] [PLAT-1697] Fix Stop backup race condition. Add non-schedlued backup examples
    • [PLAT-1559] Stop the external script scheduler if the universe is not present.
    • [PLAT-1563] Fix instance down alerts + make sure instance restart alert is not fired on universe operations
    • [PLAT-1578] Do not specify storage class (use default if provided)
    • [PLAT-1586] [Alert] Able to add multiple alert configuration with same name. Add duplicate check for alert configuration name
    • [PLAT-1599] [UI] Root Certificate and node-node and client-node TLS missing on Edit Universe
    • [PLAT-1603] [Platform]YBFormInput's OnBlur throws error on AddCertificateForm
    • [PLAT-1605] Fix duplicate alert definitions handling + all locks to avoid duplicates creation
    • [PLAT-1606] Disk name too long for Google Cloud clone disk
    • [PLAT-1613] Alerts: Logs filled with NPE related to "Error while sending notification for alert "
    • [PLAT-1617] Added GCP region metadata for missing regions.
    • [PLAT-1617] Fix issue with GCP Subnet CIDR
    • [PLAT-1619] Check for FAILED status in wait_for_snapshot method.
    • [PLAT-1621] Health check failed in K8s portal
    • [PLAT-1625] Fix task details NPE
    • [PLAT-1626] Skip preprovision for systemd upgrade.
    • [PLAT-1631] [Alert] Universe filter is not working in Alert Listing
    • [PLAT-1638] Fix naming convention for external script endpoints as per our standards
    • [PLAT-1639] [PLAT-1681] Make proxy requests async to keep them from blocking other requests
    • [PLAT-1639] [PLAT-1681] Reduce log spew from akka-http-core for proxy requests.
    • [PLAT-1644] Fix k8s universe creation failure for platform configured with HA
    • [PLAT-1646] Remove Unsupported Instance types from pull down menu for Azure
    • [PLAT-1650] Added yum lock_timeout to prevent yum lockfile errors for use_custom_ssh_port.yml
    • [PLAT-1653] Fix region get/list.
    • [PLAT-1656] [UI] [Alert] Group Type filter is not working in Alert Listing
    • [PLAT-1661] Fix alert messages for notification failures
    • [PLAT-1667] Platform should not scrape all per-table metrics from db hosts (part 2)
    • [PLAT-1668] Yugabundle failing because can't find YWErrorHandler
    • [PLAT-1682] Fix node comparison function from accessing undefined cluster
    • [PLAT-1687] ALERT: Not able to create destination channel using "default recipients + default smtp settings + empty email field"
    • [PLAT-1694] Fix Intermittent failure to back up k8s universe
    • [PLAT-1707] Fix performance issue
    • [PLAT-1715] Check for YB version only for 2.6+ release DB
    • [PLAT-1717] Full move fails midway if system tablet takes more than 2 mins to bootstrap
    • [PLAT-1721] Stop storage type from automatically changing when instance type is changed
    • [PLAT-1726] Allow user to completely remove all gFlags after addtion of several gFlags.
    • [PLAT-1730] Fix resize node logic for RF1 clusters
    • [PLAT-1736] Create default alert configs and destination on DB seed
    • [PLAT-1737] "This field is required" error message is shown on alert configuration creation with default threshold == 0
    • [PLAT-1746] Delete prometheus_snapshot directory once platform backup package is created
    • [PLAT-1757] Health Check failure message has Actual and expected values interchanged
    • [PLAT-1761] Fix alert message in case of unprovisioned nodes
    • [PLAT-1768] Universe tasks take lot more time because thread pool executors do not reach max_threads
    • [PLAT-1780] Redact YSQL/YCQL passwords from task_info table.
    • [PLAT-1793] DB Error logs alert
    • [PLAT-1796] Edit Universe page has password fields editable
    • [PLAT-1802] Replication graphs stopped showing on replication tab (replicated.yml change)
    • [PLAT-1804] Fix 'Querying for {} metric keys - may affect performance' log
    • [PLAT-1816] Forward port restricted user creation to master
    • [PLAT-1819] [PLAT-1828] Release backup lock when Platform restarts, and update Backup state
    • [PLAT-1829] [ycql/ysql] auth password: wrong error message
    • [PLAT-1833] Fix missing create time on alert configuration creation issue
    • [PLAT-1839] Fix typo in DB migration
    • [PLAT-1892] Make error alert be disabled by default
    • [PLAT-1969] [UI] Universe creation - Create button is disabled when YSQL/YCQL auth is disabled
    • Backup and Restore failing in k8s auth enabled environment
    • Fix NPE in VM image upgrade for TLS enabled universes
    • Use TaskInfo instead of CustomerTask in shouldIncrementVersion check

    Database

    • [1252] Do not link with system libpq
    • [5920] Fix bootstrapping with preallocated log segment
    • [7528] [YSQL] Error out when Tablespaces are set for colocated tables
    • [8675] [DocDB] Prevent tablet splitting when there is post split data
    • [8772] Fix fatal that occurs when running alter_universe_replication and producer master has
    • [9061] [DocDB] Master task tracking should point to the table it is operating on
    • [9216] [YSQL] Fix NULL pointer access in case of failed test
    • [9436] [YSQL] Statement reads rows it has inserted
    • [9475] Fetch Universe Key From Masters on TS Init
    • [9616] Fix master crash when restoring snapshot schedule with deleted namespace
    • [9655] [xCluster] Label cdc streams with relevant metadata
    • [9743] Fix universe reset config option (#9863)
    • [9781] Mark snapshot as deleted if tablet was removed
    • [9782] [DocDB] Tablet Splitting - Wait for all peers to finish compacting during throttling
    • [9786] Universe Actions -> Add Read Replica is failing
    • [9789] [DocDB] Load Balancer should use tablet count while looking tablets to move
    • [9802] [xCluster] Set proper deadline for YBSession in CDCServiceImpl
    • [9806] DocDB: fixed Batcher::FlushBuffersIsReady
    • [9812] [YSQL] Check database is colocated before adding colocated option for Alter Table
    • [9822] DocDB: Check table pointer is not nullptr before dereferencing
    • [9855] [DocDB] Set aborted subtransaction data on local apply
    • [9860] [YSQL] fix limit vars to uint64
    • [9865] Fix internal retry of kReadRestart for SELECT func() with a DML in the func
    • [9867] [YSQL] Fix double type overflow in case of SET yb_transaction_priority_lower_bound/yb_transaction_priority_upperr_bound command
    • [9906] [YSQL] Fix not being able to add a range primary key
    • [9909] [YSQL] further fix backup restore for NULL col attr
    • [9924] [YSQL] always check schema name on backup import
    • [9927] YCQL - Handle unset correctly
    • [9932] [YSQL] Initialize t_ybctid field in acquire_sample_rows()
    • [9933] [Part-0] Update logic for using num_tablets from internal or user requests.
    • [9933] [YCQL] [Part-1] DESC TABLE does not directly match the "CREATE TABLE" command for number of tablets.
    • [9934] [DocDB] Don't update rocksdb_dir on Remote Bootstrap
    • [9936] Fix ysql_dump in encrypted k8s environment
    • [9936] Fix ysql_dump in TLS encrypted environment
    • [9940] DocDB: use correct kv_store_id for post-split tablets
    • [9947] [YSQL] remove runtime tag for ysql_disable_index_backfill
    • [9963] backup: fix to reallow YEDIS on restore
    • [9965] [YSQL] Fix copy/paste error causing incorrect conversion
    • [9981] Fix transaction coordinator returning wrong status hybrid time
    • [10042] [backup] allow system table for YEDIS restore
    • [10051] DocDB: use RETURN_NOT_OK on an unchecked status
    • [10085] YSQL fix FATAL caused by wrong sum pushdown
    • [10098] [YSQL] Fix index creation on temp table via ALTER TABLE
    • [10139] ybase: Avoid unnecessary table locking in CatalogManager::DeleteYsqlDBTables
    • [ybase] Properly pass number of tables via MetricPrometheusOptions
    • [YSQL] [9572] Correctly determine is_yb_relation for row-marked relations when preparing target list
    • Fix for resource leaks
    • Fixed bug in yb-ctl for stopping processes, when os.kill raises an exception
    • Make SSH wait behavior consistent across operations

    Known Issues

    Yugabyte Platform

    N/A

    Database

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.6.5(Nov 16, 2021)

    v2.6.5.0 - Oct 29, 2021

    Build: 2.6.5.0-b4

    Downloads

         

    Docker

    docker pull yugabytedb/yugabyte:2.6.5.0-b4
    

    Improvements

    Database

    9606 [DocDB] Add flag --force for command delete_tablet to set state TABLET_DATA_DELETED for tool yb-ts-cli

    Yugabyte Platform

    N/A

    Bug fixes

    Database

    • 9924 [YSQL] Always check schema name on backup import
    • 10042 [Backup] Allow system table for YEDIS restore
    • 10415 [backup] Backup-restore failures for old backups.

    Yugabyte Platform

    • [PLAT-1634] Backup page is not loading because of empty config column
    • [PLAT-1934] Adding UI to set KUBE_DOMAIN
    • [PLAT-2113] [PLAT-2116] Fix HA failing with entity_too_large
    • [PLAT-2020] [PLAT-2041] Fix HA on IPv6-enabled K8s setup
    • [YB] Replaced the Busybox image with a YB image, as some environments are unable to access Busybox

    Known issues

    Database

    N/A

    Yugabyte Platform

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.6.4(Nov 16, 2021)

    v2.6.4.0 - Oct 26, 2021

    Build: 2.6.4.0-b3

    Downloads

         

    Docker

    docker pull yugabytedb/yugabyte:2.6.4.0-b3
    

    Improvements

    Database

    • [10150] [YSQL] Add functionality for the yb_extension role
    • [10317] [YSQL] Import "Allow users with BYPASSRLS to alter their own passwords."

    Yugabyte Platform

    • [9612] [PLAT-26] [PLAT-1674] [Platform] Add logs purge threshold option to zip_purge_yb_logs.sh
    • [PLAT-1897] Make client_max_body_size configurable in replicated

    Bug fixes

    Database

    • [9909] [YSQL] Further fix backup restore for NULL col attr
    • [9957] [YSQL] Fix memory usage when translating decimal data into Postgres's datum format.
    • [9963] [Backup] Fix to reallow YEDIS on restore
    • [10240] Add IPv6 address filters to default value of net_address_filter
    • [10364] [YCQL] Fix issue when dropping col that is not in an existing non-partial secondary index

    Yugabyte Platform

    N/A

    Known issues

    Database

    N/A

    Yugabyte Platform

    N/A

    Source code(tar.gz)
    Source code(zip)
  • v2.6.3(Nov 16, 2021)

    v2.6.3.0 - Oct 22, 2021

    Build: 2.6.3.0-b12

    Downloads

         

    Docker

    docker pull yugabytedb/yugabyte:2.6.3.0-b12
    

    Improvements

    Database

    • [3785] [DocDB] Add support for LZ4 compression
    • [7612] [DocDB] Allow TTL-expired SST files that are too large for compaction to be directly expired
    • [7612] [DocDB] Improves TTL handling by removing a file completely if all data is expired
    • [7612] [DocDB] Modified compaction file filter to filter files out of order
    • [10019] [DocDB] Add support for zlib compression
    • [10110] [DocDB] Enables compaction file filter during manual compactions

    Yugabyte Platform

    • [9113] [9114] [Platform] Populate the task id field in the backup table
    • [PLAT-1753] Enable taking backups using custom ports

    Bug fixes

    Database

    • [9436] [YSQL] Statement reads rows it has inserted
    • [10077] [DocDB] Compaction file filter factory uses HistoryRetention instead of Schema
    • [10164] [DocDB] Max file size for compactions should only affect TTL tables
    • [YSQL] Restart metrics webserver when postmaster recovers backend

    Yugabyte Platform

    • [PLAT-1819] [PLAT-1828] Release backup lock when Platform restarts, and update Backup state

    Known issues

    Database

    N/A

    Yugabyte Platform

    N/A

    Source code(tar.gz)
    Source code(zip)
Owner
yugabyte
The high-performance distributed SQL database for global, internet-scale apps.
yugabyte
HybridSE (Hybrid SQL Engine) is an LLVM-based, hybrid-execution and high-performance SQL engine

HybridSE (Hybrid SQL Engine) is an LLVM-based, hybrid-execution and high-performance SQL engine. It can provide fast and consistent execution on heterogeneous SQL data systems, e.g., OLAD database, HTAP system, SparkSQL, and Flink Stream SQL.

4Paradigm 45 Sep 12, 2021
PGSpider: High-Performance SQL Cluster Engine for distributed big data.

PGSpider: High-Performance SQL Cluster Engine for distributed big data.

PGSpider 127 Jun 24, 2022
PolarDB for PostgreSQL (PolarDB for short) is an open source database system based on PostgreSQL.

PolarDB for PostgreSQL (PolarDB for short) is an open source database system based on PostgreSQL. It extends PostgreSQL to become a share-nothing distributed database, which supports global data consistency and ACID across database nodes, distributed SQL processing, and data redundancy and high availability through Paxos based replication. PolarDB is designed to add values and new features to PostgreSQL in dimensions of high performance, scalability, high availability, and elasticity. At the same time, PolarDB remains SQL compatibility to single-node PostgreSQL with best effort.

Alibaba 2.3k Jun 30, 2022
pgagroal is a high-performance protocol-native connection pool for PostgreSQL.

pgagroal is a high-performance protocol-native connection pool for PostgreSQL.

Agroal 524 Jun 15, 2022
Kunlun distributed DBMS is a NewSQL OLTP relational distributed database management system

Kunlun distributed DBMS is a NewSQL OLTP relational distributed database management system. Application developers can use Kunlun to build IT systems that handles terabytes of data, without any effort on their part to implement data sharding, distributed transaction processing, distributed query processing, crash safety, high availability, strong consistency, horizontal scalability. All these powerful features are provided by Kunlun.

zettadb 99 Jun 24, 2022
dqlite is a C library that implements an embeddable and replicated SQL database engine with high-availability and automatic failover

dqlite dqlite is a C library that implements an embeddable and replicated SQL database engine with high-availability and automatic failover. The acron

Canonical 3k Jun 27, 2022
A PostgreSQL extension providing an async networking interface accessible via SQL using a background worker and curl.

pg_net is a PostgreSQL extension exposing a SQL interface for async networking with a focus on scalability and UX.

Supabase 41 Jun 19, 2022
High-performance time-series aggregation for PostgreSQL

PipelineDB has joined Confluent, read the blog post here. PipelineDB will not have new releases beyond 1.0.0, although critical bugs will still be fix

PipelineDB 2.5k Jun 24, 2022
Nebula Graph is a distributed, fast open-source graph database featuring horizontal scalability and high availability

Nebula Graph is an open-source graph database capable of hosting super large scale graphs with dozens of billions of vertices (nodes) and trillions of edges, with milliseconds of latency.

vesoft inc. 807 Jun 30, 2022
Distributed PostgreSQL as an extension

What is Citus? Citus is a PostgreSQL extension that transforms Postgres into a distributed database—so you can achieve high performance at any scale.

Citus Data 6.8k Jun 28, 2022
DuckDB is an in-process SQL OLAP Database Management System

DuckDB is an in-process SQL OLAP Database Management System

DuckDB 5.4k Jun 27, 2022
TimescaleDB is an open-source database designed to make SQL scalable for time-series data.

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.

Timescale 13.3k Jun 27, 2022
A friendly and lightweight C++ database library for MySQL, PostgreSQL, SQLite and ODBC.

QTL QTL is a C ++ library for accessing SQL databases and currently supports MySQL, SQLite, PostgreSQL and ODBC. QTL is a lightweight library that con

null 155 Jun 26, 2022
upstream module that allows nginx to communicate directly with PostgreSQL database.

About ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database. Configuration directives postgres_server

RekGRpth 1 Apr 29, 2022
A framework to monitor and improve the performance of PostgreSQL using Machine Learning methods.

pg_plan_inspector pg_plan_inspector is being developed as a framework to monitor and improve the performance of PostgreSQL using Machine Learning meth

suzuki hironobu 168 Jul 1, 2022
BaikalDB, A Distributed HTAP Database.

BaikalDB supports sequential and randomised realtime read/write of structural data in petabytes-scale. BaikalDB is compatible with MySQL protocol and it supports MySQL style SQL dialect, by which users can migrate their data storage from MySQL to BaikalDB seamlessly.

Baidu 955 Jun 30, 2022
GalaxyEngine is a MySQL branch originated from Alibaba Group, especially supports large-scale distributed database system.

GalaxyEngine is a MySQL branch originated from Alibaba Group, especially supports large-scale distributed database system.

null 229 Jun 30, 2022
Test any type of cloud database on Android apps. No need of a dedicated backend.

DB Kong - Database connections simplified DB Kong is an Android library that allows you to connect, interact and test any type of cloud database on An

Arjun 9 May 9, 2022
OrioleDB – building a modern cloud-native storage engine

OrioleDB is a new storage engine for PostgreSQL, bringing a modern approach to database capacity, capabilities and performance to the world's most-loved database platform.

OrioleDB 1.1k Jul 3, 2022