Basis Universal GPU Texture Codec

Overview

basis_universal

Basis Universal Supercompressed GPU Texture Codec

Basis Universal is a "supercompressed" GPU texture data interchange system that supports two highly compressed intermediate file formats (.basis or the .KTX2 open standard from the Khronos Group) that can be quickly transcoded to a very wide variety of GPU compressed and uncompressed pixel formats: ASTC 4x4 L/LA/RGB/RGBA, PVRTC1 4bpp RGB/RGBA, PVRTC2 RGB/RGBA, BC7 mode 6 RGB, BC7 mode 5 RGB/RGBA, BC1-5 RGB/RGBA/X/XY, ETC1 RGB, ETC2 RGBA, ATC RGB/RGBA, ETC2 EAC R11 and RG11, FXT1 RGB, and uncompressed raster image formats 8888/565/4444.

The system now supports two modes: a high quality mode which is internally based off the UASTC compressed texture format, and the original lower quality mode which is based off a subset of ETC1 called "ETC1S". UASTC is for extremely high quality (similar to BC7 quality) textures, and ETC1S is for very small files. The ETC1S system includes built-in data compression, while the UASTC system includes an optional Rate Distortion Optimization (RDO) post-process stage that conditions the encoded UASTC texture data in the .basis file so it can be more effectively LZ compressed by the end user. More technical details about UASTC integration are here.

Basis files support non-uniform texture arrays, so cubemaps, volume textures, texture arrays, mipmap levels, video sequences, or arbitrary texture "tiles" can be stored in a single file. The compressor is able to exploit color and pattern correlations across the entire file, so multiple images with mipmaps can be stored very efficiently in a single file.

The system's bitrate depends on the quality setting and image content, but common usable ETC1S bitrates are .3-1.25 bits/texel. ETC1S .basis files are typically 10-25% smaller than using RDO texture compression of the internal texture data stored in the .basis file followed by LZMA. For UASTC files, the bitrate is fixed at 8bpp, but with RDO post-processing and user-provided LZ compression on the .basis file the effective bitrate can be as low as 2bpp for video or for individual textures approximately 4-6bpp.

The .basis and .KTX2 transcoders have been fuzz tested using zzuf.

So far, we've compiled the code using MSVC 2019, under Ubuntu 18.04 and 20 x64 using cmake with either clang 3.8 or gcc 5.4, and emscripten 1.35 to asm.js. (Be sure to use this version or later of emcc, as earlier versions fail with internal errors/exceptions during compilation.)

Basis Universal supports "skip blocks" in ETC1S compressed texture arrays, which makes it useful for basic compressed texture video applications. Note that Basis Universal is still at heart a GPU texture compression system, not a dedicated video codec, so bitrates will be larger than even MPEG1. 1/10/21 release notes:

For v1.13, we've added numerous ETC1S encoder optimizations designed to greatly speed up single threaded encoding time, as well as greatly reducing overall CPU utilization when multithreading is enabled. For benchmarking, we're using "-q 128 -no_multithreading -mip_fast". The encoder now uses approximately 1/3rd as much total CPU time for the same PSNR. The encoder can now optionally utilize SSE 4.1 - see the "-no_sse" command line option.

Release Notes

The first texture (and possibly image/video) compression codec developed without "Lena"

We retired Lena years ago. No testing is done with this image:

https://www.losinglena.com/

Quick Introduction

Probably the most important concept to understand about Basis Universal before using it: The system supports two very different universal texture modes: The original "ETC1S" mode is low/medium quality, but the resulting file sizes are very small because the system has built-in compression for ETC1S texture format files. This is the command line encoding tool's default mode. ETC1S textures work best on images, photos, map data, or albedo/specular/etc. textures, but don't work as well on normal maps.

There's the second "UASTC" mode, which is significantly higher quality (comparable to BC7 and highest quality LDR ASTC 4x4), and is usable on all texture types including complex normal maps. UASTC mode purposely does not have built-in file compression like ETC1S mode does, so the resulting files are quite large (8-bits/texel - same as BC7) compared to ETC1S mode. The UASTC encoder has an optional Rate Distortion Optimization (RDO) encoding mode (implemented as a post-process over the encoded UASTC texture data), which conditions the output texture data in a way that results in better lossless compression when UASTC .basis files are compressed with Deflate/Zstd, etc. In UASTC mode, you must losslessly compress .basis files yourself. .KTX2 files have built-in lossless compression support using Zstandard, which is used by default on UASTC textures.

Basis Universal is not an image compression codec, but a GPU texture compression codec. It can be used just like an image compression codec, but that's not the only use case. Here's a good intro to GPU texture compression. If you're looking to primarily use the system as an image compression codec on sRGB photographic content, use the default ETC1S mode, because it has built-in compression.

The "-q X" option controls the output quality in ETC1S mode. The default is quality level 128. "-q 255" will increase quality quite a bit. If you want even higher quality, try "-max_selectors 16128 -max_endpoints 16128" instead of -q. -q internally tries to set the codebook sizes (or the # of quantization intervals for endpoints/selectors) for you. You need to experiment with the quality level on your content.

For tangent space normal maps, you should separate X into RGB and Y into Alpha, and provide the compressor with 32-bit/pixel input images. Or use the "-separate_rg_to_color_alpha" command line option which does this for you. The internal texture format that Basis Universal uses (ETC1S) doesn't handle tangent space normal maps encoded into RGB well. You need to separate the channels and recover Z in the pixel shader using z=sqrt(1-x^2-y^2).

License and 3rd party code dependencies

Detailed legal, license, and IP information is here. Basis Universal itself uses the Apache 2.0 licenses, but it utilizes some zlib and optional BSD code (Zstandard). The supported texture formats are open Khronos Group standards.

All C/C++ code dependencies are present inside the Basis Universal repo itself to simplify building.

The stand-alone transcoder (in the "transcoder" directory) is a single .cpp source file library which has no 3rd party code dependencies apart from zstd/zstddeclib.c, which is optional. (It's only used for decompressing UASTC KTX2 files that use Zstandard.)

The encoder uses lodepng for loading and saving PNG images, which is Copyright (c) 2005-2019 Lode Vandevenne. It uses the zlib license. It also uses apg_bmp for loading BMP images, which is Copyright 2019 Anton Gerdelan. It uses the Apache 2.0 license.

The encoder uses tcuAstcUtil.cpp, from the Android drawElements Quality Program (deqp) Testing Suite, for unpacking the transcoder's ASTC output for testing/validation purposes. This code is Copyright 2016 The Android Open Source Project, and uses the Apache 2.0 license. We have modified the code so it has no external dependencies, and disabled HDR support.

The encoder optionally uses Zstandard's single source file compressor (in zstd/zstd.c) to support compressing supercompressed KTX2 files.

Command Line Compression Tool

The command line tool used to create, validate, and transcode/unpack .basis/.KTX2 files is named "basisu". Run basisu without any parameters for help.

The library and command line tool have no other 3rd party dependencies (that are not already in the repo), so it's pretty easy to build.

To build basisu (without SSE 4.1 support - the default):

cmake CMakeLists.txt
make

To build with SSE 4.1 support on x86/x64 systems (encoding is roughly 15-30% faster):

cmake -D SSE=TRUE CMakeLists.txt
make

For Visual Studio 2019, you can now either use the CMakeLists.txt file or the included basisu.sln file. Earlier versions of Visual Studio (particularly 2017) should work but aren't actively tested. We develop with the most up to date version of 2019.

To compress a sRGB PNG/BMP/TGA/JPEG image to an ETC1S .KTX2 file:

basisu -ktx2 x.png

To compress a sRGB PNG/BMP/TGA/JPEG image to an UASTC .KTX2 file:

basisu -ktx2 -uastc x.png

To compress a sRGB PNG/BMP/TGA/JPEG image to an RDO UASTC .KTX2 file with mipmaps:

basisu -ktx2 -uastc -uastc_rdo_l 1.0 -mipmap x.png

To compress a sRGB PNG/BMP/TGA/JPEG image to an ETC1S .basis file:

basisu x.png

To compress a image to a higher quality UASTC .basis file:

basisu -uastc -uastc_level 2 x.png

To compress a image to a higher quality UASTC .basis file with RDO post processing, so the .basis file is more compressible:

basisu -uastc -uastc_level 2 -uastc_rdo_l .75 x.png

-uastc_level X ranges from 0-4 and controls the UASTC encoder's performance vs. quality tradeoff. Level 0 is very fast, but low quality, level 2 is the default quality, while level 3 is the highest practical quality. Level 4 is impractically slow, but highest quality.

-uastc_rdo_l X controls the rate distortion stage's quality setting. The lower this value, the higher the quality, but the larger the compressed file size. Good values to try are between .2-3.0. The default is 1.0. RDO post-processing is currently pretty slow, but we'll be optimizing it over time.

UASTC texture video is supported and has been tested. In RDO mode with 7zip LZMA, we've seen average bitrates between 1-2 bpp. ETC1S mode is recommended for texture video, which gets bitrates around .25-.3 bpp.

Note that basisu defaults to sRGB colorspace metrics. If the input is a normal map, or some other type of non-sRGB (non-photographic) texture content, be sure to use -linear to avoid extra unnecessary artifacts. (Note: Currently, UASTC mode always uses linear colorspace metrics. sRGB and angulate metrics are comming soon.)

To add automatically generated mipmaps to the .basis file, at a higher than default quality level (which ranges from [1,255]):

basisu -mipmap -q 190 x.png

There are several mipmap options that allow you to change the filter kernel, the filter colorspace for the RGB channels (linear vs. sRGB), the smallest mipmap dimension, etc. The tool also supports generating cubemap files, 2D/cubemap texture arrays, etc.

To create a slightly higher quality ETC1S .basis file (one with better codebooks) at the default quality level (128) - note this is much slower to encode:

basisu -comp_level 2 x.png

On some rare images (ones with blue sky gradients come to bind), you may need to increase the ETC1S -comp_level setting. This controls the amount of overall effort the encoder uses to optimize the ETC1S codebooks (palettes) and compressed data stream. Higher comp_level's are significantly slower, and shouldn't be used unless necessary:

basisu -ktx2 x.png -comp_level 5 -q 255

Or try: basisu -ktx2 x.png -comp_level 5 -max_endpoints 16128 -max_selectors 16128

Note -comp_level's 3-4 are almost as good as 5 and are a lot faster.

The compressor is multithreaded by default, but this can be disabled using the -no_multithreading command line option. The transcoder is currently single threaded although it supports multithreading decompression of multiple texture slices in parallel.

Unpacking .basis/.KTX2 files to .PNG/.KTX files

You can either use the command line tool or call the transcoder directly from JavaScript or C/C++ code to decompress .basis/.KTX2 files to GPU texture data or uncompressed images.

To use the command line tool to unpack a .basis or .KTX2 file to multiple .png/.ktx files:

basisu x.basis

Use the -no_ktx and -etc1_only options to unpack to less files. -info and -validate will just display file information and not output any files. The output .KTX1 files are currently in the KTX1 file format, not KTX2.

The mipmapped or cubemap .KTX files will be in a wide variety of compressed GPU texture formats (PVRTC1 4bpp, ETC1-2, BC1-5, BC7, etc.), and to my knowledge there is no single .KTX viewer tool that correctly and reliably supports every GPU texture format that we support. BC1-5 and BC7 files are viewable using AMD's Compressonator, ETC1/2 using Mali's Texture Compression Tool, and PVRTC1 using Imagination Tech's PVRTexTool. Links:

Mali Texture Compression Tool

Compressonator

PVRTexTool

After compression, the compressor transcodes all slices in the output .basis file to validate that the file decompresses correctly. It also validates all header, compressed data, and slice data CRC16's.

For best quality, you must supply basisu with original uncompressed source images. Any other type of lossy compression applied before basisu (including ETC1/BC1-5, BC7, JPEG, etc.) will cause multi-generational artifacts to appear in the final output textures.

For the maximum possible achievable ETC1S mode quality with the current format and encoder (completely ignoring encoding speed!), use:

basisu x.png -comp_level 5 -max_endpoints 16128 -max_selectors 16128 -no_selector_rdo -no_endpoint_rdo

Level 5 is extremely slow, so unless you have a very powerful machine, levels 1-4 are recommended.

Note that "-no_selector_rdo -no_endpoint_rdo" are optional. Using them hurts rate distortion performance, but increases quality. An alternative is to use -selector_rdo_thresh X and -endpoint_rdo_thresh, with X ranging from [1,2] (higher=lower quality/better compression - see the tool's help text).

To compress small video sequences, say using tools like ffmpeg and VirtualDub:

basisu -comp_level 2 -tex_type video -stats -debug -multifile_printf "pic%04u.png" -multifile_num 200 -multifile_first 1 -max_selectors 16128 -max_endpoints 16128 -endpoint_rdo_thresh 1.05 -selector_rdo_thresh 1.05

For video, the more cores your machine has, the better. Basis is intended for smaller videos of a few dozen seconds or so. If you are very patient and have a Threadripper or Xeon workstation, you should be able to encode up to a few thousand 720P frames. The "webgl_videotest" directory contains a very simple video viewer. For texture video, use -comp_level 2 or 3. The default is 1, which isn't quite good enough for texture video. Higher comp_level's result in reduced ETC1S artifacts.

The .basis file will contain multiple images (all using the same global codebooks), which you can retrieve using the transcoder's image API. The system now supports conditional replenisment (CR, or "skip blocks"). CR can reduce the bitrate of some videos (highly dependent on how dynamic the content is) by over 50%. For videos using CR, the images must be requested from the transcoder in sequence from first to last, and random access is only allowed to I-Frames.

If you are doing rate distortion comparisons vs. other similar systems, be sure to experiment with increasing the endpoint RDO threshold (-endpoint_rdo_thresh X). This setting controls how aggressively the compressor's backend will combine together nearby blocks so they use the same block endpoint codebook vectors, for better coding efficiency. X defaults to a modest 1.5, which means the backend is allowed to increase the overall color distance by 1.5x while searching for merge candidates. The higher this setting, the better the compression, with the tradeoff of more block artifacts. Settings up to ~2.25 can work well, and make the codec more competitive. "-endpoint_rdo_thresh 1.75" is a good setting on many textures.

For video, level 1 should result in decent results on most clips. For less banding, level 2 can make a big difference. This is still an active area of development, and quality/encoding perf. will improve over time.

To control the ETC1S encoder's quality vs. encoding speed tradeoff, see ETC1S Compression Effort Levels.

More Examples

basisu x.png
Compress sRGB image x.png to a ETC1S format x.basis file using default settings (multiple filenames OK). ETC1S format files are typically very small on disk (around .5-1.5 bits/texel).

basisu -uastc x.png
Compress image x.png to a UASTC format x.basis file using default settings (multiple filenames OK). UASTC files are the same size as BC7 on disk (8-bpp). Be sure to compress UASTC .basis files yourself using Deflate, zstd, etc. To increase .basis file compressibility (trading off quality for smaller compressed files) use the "-uastc_rdo_q X" command line parameter.

basisu -q 255 x.png
Compress sRGB image x.png to x.basis at max quality level achievable without manually setting the codebook sizes (multiple filenames OK)

basisu x.basis
Unpack x.basis to PNG/KTX files (multiple filenames OK)

basisu -validate -file x.basis
Validate x.basis (check header, check file CRC's, attempt to transcode all slices)

basisu -unpack -file x.basis
Validates, transcodes and unpacks x.basis to mipmapped .KTX and RGB/A .PNG files (transcodes to all supported GPU texture formats)

basisu -q 255 -file x.png -mipmap -debug -stats
Compress sRGB x.png to x.basis at quality level 255 with compressor debug output/statistics

basisu -linear -max_endpoints 16128 -max_selectors 16128 -file x.png
Compress non-sRGB x.png to x.basis using the largest supported manually specified codebook sizes

basisu -linear -global_sel_pal -no_hybrid_sel_cb -file x.png
Compress a non-sRGB image, use virtual selector codebooks for improved compression (but slower encoding)

basisu -linear -global_sel_pal -file x.png
Compress a non-sRGB image, use hybrid selector codebooks for slightly improved compression (but slower encoding)

basisu -tex_type video -comp_level 2 -framerate 20 -multifile_printf "x%02u.png" -multifile_first 1 -multifile_count 20 -selector_rdo_thresh 1.05 -endpoint_rdo_thresh 1.05
Compress a 20 sRGB source image video sequence (x01.png, x02.png, x03.png, etc.) to x01.basis

basisu -comp_level 2 -q 255 -file x.png -mipmap -y_flip
Compress a mipmapped x.basis file from an sRGB image named x.png, Y flip each source image, set encoder to level 2 for slightly higher quality (but slower encoding).

WebGL test

The "WebGL" directory contains three simple WebGL demos that use the transcoder and compressor compiled to wasm with emscripten. See more details here.

Screenshot of 'texture' example running in a browser. Screenshot of 'gltf' example running in a browser. Screenshot of 'encode_test' example running in a browser.

Installation using the vcpkg dependency manager

You can download and install Basis Universal using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
vcpkg install basisu

The Basis Universal port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

WebAssembly Support Using Emscripten

Both the transcoder and now the compressor (as of 12/17/2020) may be compiled using emscripten to WebAssembly and used on the web. Currently, multithreading is not supported by the compressor when compiled with emscripten. A simple Web compression demo is in webgl/encode_test. All compressor features, including texture video, are supported and fully exposed.

To enable compression support compile the JavaScript wrappers in webgl/transcoding/basis_wrappers.cpp with BASISU_SUPPORT_ENCODING set to 1. See the webgl/encoding directory.

Low-level C++ encoder API

You can call the encoder directly, instead of using the command line tool. We'll be adding documentation and some examples by the end of the year. For now, some important notes:

First, ALWAYS call basisu::basisu_encoder_init() to initialize the library. Otherwise, you'll get undefined behavior or black textures.

Create a job pool, fill in the basis_compress_params struct, then call basisu::basis_compressor::init(), then basisu::basis_compressor::process(). Like this for UASTC:

bool test()
{
	basisu_encoder_init();

	image img;
	if (!load_image("test.png", img))
	{
		printf("Can't load image\n");
		return false;
	}

	basis_compressor_params basisCompressorParams;

	basisCompressorParams.m_source_images.push_back(img);
	basisCompressorParams.m_perceptual = false;
	basisCompressorParams.m_mip_srgb = false;

	basisCompressorParams.m_write_output_basis_files = true;
	basisCompressorParams.m_out_filename = "test.basis";

	basisCompressorParams.m_uastc = true;
	basisCompressorParams.m_rdo_uastc_multithreading = false;
	basisCompressorParams.m_multithreading = false;
	basisCompressorParams.m_debug = true;
	basisCompressorParams.m_status_output = true;
	basisCompressorParams.m_compute_stats = true;
	
	basisu::job_pool jpool(1);
	basisCompressorParams.m_pJob_pool = &jpool;

	basisu::basis_compressor basisCompressor;
	basisu::enable_debug_printf(true);

	bool ok = basisCompressor.init(basisCompressorParams);
	if (ok)
	{
		basisu::basis_compressor::error_code result = basisCompressor.process();

		if (result == basisu::basis_compressor::cECSuccess)
			printf("Success\n");
		else
		{
			printf("Failure\n");
			ok = false;
		}
	}
	else
		printf("Failure\n");
	return ok;
}

The command line tool uses this API too, so you can always look at that to see what it does given a set of command line options.

Repository Licensing with REUSE

The repository has been updated to be compliant with the REUSE licenese checking tool (https://reuse.software/). This was done by adding the complete text of all licenses used under the LICENSES/ directory and adding the .reuse/dep5 file which specifies licenses for files which don't contain them in a form which can be automatically parse by the reuse tool. REUSE does not alter copyrights or licenses, simply captures information about licensing to ensure the entire repository has explicit licensing information.

To ensure continued REUSE compliance, run reuse lint at the root of a clean, checked-out repository periodically, or run it during CI tests before any build artifacts have been created.

Special Thanks

A huge thanks to Google for partnering with us and enabling this system to be open sourced.

Thank you to Esri - Environmental Systems Research Institute for sponsoring the encoder optimization work in the v1.13 release, and the KTX2 work in the v1.15 release.

Thanks to the Khronos Group for building an open ecosystem that supports ETC1S/UASTC.

Thanks to a number of companies or groups who have supported or helped out Binomial over the years: Intel, SpaceX, Netflix, Forgotten Empires, Microsoft, Polystream, Hothead Games, BioDigital, Magic Leap, Blizzard Entertainment, Insomniac Games, Rockstar Games, Facebook, Activision, the Khronos Group, and the organizers at CppCon.

Thanks to Dave Wilkinson (AMD/Khronos) for supporting us and giving us very valuable feedback while we developed Basis Universal.

Thanks to Chris Wein (Netflix), who showed us the path to Texture Video.

Thanks to Mike Dussault (SpaceX) and Elon Musk for supporting Binomial in the early days.

Thanks to Graeme Devine at Magic Leap.

Thanks to Matt Pritchard, formerly of Valve Software and Microsoft, for helping me with the computer hardware I used while building this system and its predecessor.

Thanks to John Brooks at Blue Shift, Inc. for inspiring this work by showing me his Dreamcast texture compression system around 2002, and for releasing etc2comp. I first saw the subblock flip estimation approach (used in basisu_etc.cpp) in etc2comp.

Thanks to Colt McAnlis, for advertising one of my earlier open source texture compression libraries at GDC, and Won Chun, who originally suggested making a universal system.

Thanks to Chas Boyd (Microsoft), for inspiring us to work on texture compression full-time. Chas also gave us great feedback about UASTC before it was released.

Thanks to Mark Callow at Edgewise Consulting for his work on glTF and the KTX2 file format.

I first saw using precomputed tables for quickly computing optimal encodings of solid color blocks in ryg_dxt. The method that limits the canonical Huffman codelengths to a maximum codesize was used in Yoshizaki's lharc. The canonical Huffman codelength compression system is similar to Katz's Deflate method.

Possible improvements

The codebook generation process is basically a high quality, but slow and brute force reference. It's possible to massively speed up codebook gen in several ways. One way is to not throw away the tree structures constructed during the creation of the initial codebooks.

The way the -q (quality) option is converted to codebook sizes is very simple (fixed formulas), and could be improved. It has a tendency to plateau on some files.

The various Huffman codes could be divided up into groups (like Zstd), for much faster Huffman decoding in the transcoder. Also, larger slices could be divided up into multiple segments, and each segment transcoded using a different thread. Both of these changes would modify the format.

PVRTC1 modulation values could be determined using multiple threads and/or SIMD code.

PVRTC1 2bpp support wouldn't be hard to add.

The transcoder's BC7 tables are a bit large, and can be reduced, which would allow the transcoder to be downloaded more quickly.

3-bit selectors for alpha would greatly improve the quality of the alpha, but would break the file format and require extensive additions to the compressor/transcoder.

Fast 6x6 ASTC support may be possible.

Issues
  • basis textures with alpha appearing fully black and white on some iPads and older mobile devices

    basis textures with alpha appearing fully black and white on some iPads and older mobile devices

    Basis textures have been lifesaving when it comes to the texture space leverage it provides in my three.js projects! They work great but I've just encountered an issue I've never seen before and since it's on mobile/tablet devices it's tough to debug.

    Most of the textures used in this scene are basis and are working perfectly but the two textures (trees and 3D buttons) that have an alpha component for their shadows are showing up black and white in some cases.

    Image from iOS

    I've witnessed it on my older iPad air running iOS 12, a brand new iPad pro I believe was running iPadOS 14 (not sure, tested on it at a best buy and had to be quick), and another old school Android Galaxy phone. I tried it on a regular iPad that was a 2019 model I believe and the textures showed up correctly. This has occurred in safari, chrome, and firefox so I don't believe it's a browser issue. I converted the textures from png to basis using the basisu command line program. Applying them programmatically to the glb's map in three.js v111 via BasisTextureLoader and GLTFLoader.

    If anyone has any insights on why this would be occurring or potential solutions to the problem, I'd appreciate it a lot. I've just started looking into ktx2 textures - maybe that's a solution?

    opened by hayden2114 23
  • Possible wrapping leading to wrong colours in PVRTC decoding

    Possible wrapping leading to wrong colours in PVRTC decoding

    On iOS only (not on Android or e.g. Chrome on Linux, where everything looks good), the emscripten-compiled Basis decoder apparently generates wrong colours at the texture edges along the X-axis.

    I reported this issue here, with a full repro:

    https://github.com/mrdoob/three.js/issues/17597

    Repro link: https://nh2.me/webgl-basis-problem/ -- it looks different on iOS than on other platforms.

    Linux:

    image

    iOS:

    image

    But it is very possible that it's not a three.js, but a Basis problem.

    Looking at the pixel colours carefully, it seems colours "fading" into the edge are exactly the colours at the opposing end of the image.

    Are you aware of anything that could cause this? Perhaps some modulo operation that takes "surrounding pixels" account wrapping around the image boundaries?

    opened by nh2 16
  • Textures not divided by four are crashing in Chromium

    Textures not divided by four are crashing in Chromium

    General question

    I need to set compression format to S3TC with basisu CLI but I don't know which flag I should pass in order to enable only s3tc compression.

    Detailed quetion

    Hello, I want to use basis on my site for loading textures (using BasisTextureLoader from THREE.js). And in order to use it I need the textures to work on all popular webgl engines using s3tc format. I don't know how I should compile my assets with CLI using only s3tc compression.

    Reproduction

    Firefox 70 + NVIDIA GT740M

    They load successfully on Firefox 70 with NVIDIA GT740M card (and OpenGL ES 1.0):

    image

    And in console there are these warnings (but the texture still loads):

    THREE.WebGLRenderer: WEBGL_compressed_texture_astc extension not supported.
    THREE.WebGLRenderer: WEBGL_compressed_texture_etc1 extension not supported.
    THREE.WebGLRenderer: WEBGL_compressed_texture_pvrtc extension not supported.
    THREE.WebGLRenderer: WEBKIT_WEBGL_compressed_texture_pvrtc extension not supported.
    

    MAX Parameters test reports that Firefox rendererer support these things:

    EXT_texture_compression_bptc
    EXT_texture_compression_rgtc
    WEBGL_compressed_texture_etc
    WEBGL_compressed_texture_s3tc
    WEBGL_compressed_texture_s3tc_srgb
    

    But fails to load in other browsers with other different webgl engines.

    WebKit + NVIDIA GT740M

    for instance, WebKit (same card) (OpenGL ES 2.0) outputs this:

    image

    with these warnings:

    THREE.WebGLRenderer: WEBGL_compressed_texture_astc extension not supported.
    THREE.WebGLRenderer: WEBGL_compressed_texture_etc1 extension not supported.
    THREE.WebGLRenderer: WEBGL_compressed_texture_pvrtc extension not supported.
    THREE.WebGLRenderer: WEBKIT_WEBGL_compressed_texture_pvrtc extension not supported.
    

    And from MAX Parameters test it supports these:

    EXT_texture_compression_bptc
    EXT_texture_compression_rgtc
    WEBGL_compressed_texture_s3tc
    WEBKIT_WEBGL_compressed_texture_s3tc
    WEBGL_compressed_texture_s3tc_srgb
    

    Chromium 81 + Intel 4000

    Same black screen with these warnings:

    THREE.WebGLRenderer: WEBGL_compressed_texture_astc extension not supported.
    THREE.WebGLRenderer: EXT_texture_compression_bptc extension not supported.
    THREE.WebGLRenderer: WEBGL_compressed_texture_etc1 extension not supported.
    THREE.WebGLRenderer: WEBGL_compressed_texture_pvrtc extension not supported.
    THREE.WebGLRenderer: WEBKIT_WEBGL_compressed_texture_pvrtc extension not supported.
    

    While these extensions are supported:

    EXT_texture_compression_rgtc
    WEBGL_compressed_texture_s3tc
    WEBKIT_WEBGL_compressed_texture_s3tc
    WEBGL_compressed_texture_s3tc_srgb
    

    I noticed however, that WEBGL_compressed_texture_s3tc is the most supported webgl extension in popular browsers.

    What I want to do is to compile a png image to basis only with s3tc compression. I couldn't find how to do it in readme (sorry if it is written how but I didn't find it).

    I would be thankful if someone showed me a way to do it.

    Additional Context

    live example: https://v1rtl.site/ source code: https://github.com/talentlessguy/v1rtl_site Three.js BasisTextureLoader docs: https://threejs.org/docs/#examples/en/loaders/BasisTextureLoader

    opened by talentlessguy 14
  • UASTC textures appear corrupt on transcoding

    UASTC textures appear corrupt on transcoding

    I've been using Basis successfully with standard encoding for many months, but the quality is not sufficient for some textures. I tried enabling uastc, but the uastc-encoded textures appear to be corrupted. Can anyone point me in the right direction? Assets#Scenes#lightmapTest#Lightmap-0_comp_light.zip This is the original: lightmap-original-256x256 This is the corrupted UASTC texture after recomposition: lightmap-transcoded-256x253 This is the texture compressed and transcoded with ETC1: Assets#Scenes#lightmapTest#Lightmap-0_comp_light_unpacked_rgba_PVRTC2_4_RGBA_0000-256x256-128x128

    opened by rvkennedy 13
  • API documentation?

    API documentation?

    Hello. Where can I find API documentation or examples on how to use the library for importing and transcoding images? I need to do specifically 2 operations:

    • feed the raw bitmap (it's in memory, no need to open files, etc) and obtain the basis texture (also in memory)
    • feed the basis texture and obtain a gpu-specific texture (also in memory, to be fed to the gles api)

    Thanks!

    opened by punto- 13
  • ASTC and mipmaps fail with THREE.js under WebGL2

    ASTC and mipmaps fail with THREE.js under WebGL2

    WebGL1 works fine, and so does disabling mipmaps in generation of the basis file.

    Same results across Chrome, Firefox on MacOS with Intel GPU, and Windows 10 with nVidia RTX Quadro.

    demo: http://ahyoomee.miru.hk/basis/demo/threejs.html

    .WebGL-0x7f9769868000]GL ERROR :GL_INVALID_OPERATION : glCompressedTexImage2D: width or height invalid for level
    253[.WebGL-0x7f9769868000]RENDER WARNING: texture bound to texture unit 0 is not renderable. It might be non-power-of-2 or have incompatible texture filtering (maybe)?
    threejs.html:1 WebGL: too many errors, no more errors will be reported to the console for this context.
    
    opened by steve-o 12
  • Fixes to enable libktx use of Basis

    Fixes to enable libktx use of Basis

    This PR does the following:

    Enables access to the low-level decoder:

    • Makes a new signature for transcode_slice that does not require faking up bits of a .basis file. A function with the old signature remains as an inline function that calls the new one.
    • Makes write_opaque_blocks a public static function of lowlevel_decoder.

    Enables embedding of the basis code within a library to be linked to applications:

    • Changes the setting of _ITERATOR_DEBUG_LEVEL to the default VS value. There appears to be absolutely no need to set this value or _SECURE_SCL within these files.

    Fixes all the unused variable warnings which totalled about 45.

    It does NOT fix other -pedantic warnings which include one about misuse of a comma operator and 9 about anonymous structs and unions being extensions. There is also 1 unused variable warning that appears when compiled in release mode that this PR does not fix.

    Please turn on -pedantic and fix them.

    opened by MarkCallow 12
  • Modularize emscripten build.

    Modularize emscripten build.

    By default, emscripten's generated javascript modules lives in the global object "Module'.

    This makes it impossible to load multiple emscripten modules on the same page if they are all built using this default.

    In attempt to avoid namespace collision, this change modularizes the emscripten build so the Module is instead called 'BASIS'. See the emscripten FAQ for more info: https://emscripten.org/docs/getting_started/FAQ.html

    Many of the target applications for Basis are possibly using other emscripten modules, so this change will reduce friction of adding Basis(assuming any of their existing modules are not modularized)

    This will require existing consumers of this package to change how they load Basis when they next upgrade their Basis build.

    Testing:

    The existing webgl/(texture|gltf) pages have been updated to use the new name and seem to operate as before

    (Built/tested on OS X with emscripten sdk-1.38.29-64bit)

    opened by prestomation 11
  • Simplify solid-color BC7 transcoding

    Simplify solid-color BC7 transcoding

    The spec and the source code suggest trying to use BC7 mode 6 first. However, the g_bc7_mode_5_optimal_endpoints array has zero errors for all 256 values, thus always providing lossless results.

    @richgel999 could you please verify?

    opened by lexaknyazev 9
  • block artifacts remains for certain image, even set -endpoint_rdo_thresh 1

    block artifacts remains for certain image, even set -endpoint_rdo_thresh 1

    Hi, we are trying to use '.basis' as our basic texture file format. But we found that there are some block artifacts for certain images: original: ldr-rgb-00-s unpacked (ETC1_RGB): ldr-rgb-00-s-m-e-100_unpacked_rgb_ETC1_RGB_0000

    original: ldr-rgb-03 unpacked (ETC1_RGB): ldr-rgb-03_unpacked_rgb_ETC1_RGB_0000

    command used:

    ./basisu ldr-rgb-03.png -comp_level 4 -max_selectors 16128 -max_endpoints 16128 -endpoint_rdo_thresh 1
    

    could anyone help to find out why?

    opened by goldenyz 9
  • Extremely slow, even on lowest quality

    Extremely slow, even on lowest quality

    A setting where compression speed is priority over file size and quality would be very desired, so at least the files can be imported first like this and then get properly compressed on background.

    Right now compression times for 128x128 files are about 5 seconds on lowest quality (0), vs 2/3 milliseconds for a S3TC or ETC compressor.

    opened by reduz 9
  • [Question] Use Encoder from Native Android

    [Question] Use Encoder from Native Android

    Hello,

    This is probably a stupid question, but is it possible to use / compile this project to encode files to .basis files directly from Android ? For example building a shared library .so, that can be called from Android. (Use case is to be able to pick any image on the fly on Android and to encode it to .basis format)

    Sorry if I'm totally off the point here. Have a good day

    opened by tsoen 0
  • Suspicious shift

    Suspicious shift

    In basisu.h at line 310 is

    		inline operator uint32_t() const
    		{
    			switch (NumBytes)
    			{
                                     ...
    				case 8:  
    				{
    					uint32_t l = read_le_dword(m_bytes);
    					uint32_t h = read_le_dword(m_bytes + 4);
    					return static_cast<uint64_t>(l) | (static_cast<uint64_t>(h) << 32U);  <=========
    				}
                                    ...
    			}
    		}
    

    When compiling for iOS arm64, the latest Xcode/clang (13.5 / 13.1.6) throws an implicit conversion loses integer precision warning which drew my attention to the highlighted line. This line looks extremely suspicious because the shift of h by 32-bits is throwing away the value of h in so far as the 32-bit return value will never see it. Many other cases also return a value that is statically cast to 64-bits which seems pointless as the return value is uint32_t. There is no warning on those cases.

    If the highlighted line is truly correct, please add additional casts to fix the warnings, otherwise fix the code.

    opened by MarkCallow 1
  • Merging multiple basis compressed images

    Merging multiple basis compressed images

    This is a continuation of a thread started on three.js forums here (https://discourse.threejs.org/t/texture-atlas-composed-of-basis-compressed-images/38313)

    This is a bit of a longshot: I was wondering if it is possible to merge several basisu compressed images into a new, single basisu texture (i.e. like a texture atlas) without having to uncompress, merge, and recompress the entire thing.

    For context: I have a bunch of images that are all already individually basis compressed. I also am currently constructing atlases on my server using uncompressed images, and then basis compressing the entire atlas. I’d like to move the atlas creation to the client. I know that I can use WASM basisu implementations to do client side compression of uncompressed textures, but that would likely be prohibitively slow. I'm wondering if there is a way to take advantage of the fact that the individual images are already compressed, even if we don't get the full amount of memory improvement that we might if we compressed the full atlas.

    Thanks for this amazing project!

    opened by theahura 2
  • [Fix 300] Make sure either quality level or endpoint and selector clusters are set

    [Fix 300] Make sure either quality level or endpoint and selector clusters are set

    Fix 300 (https://github.com/BinomialLLC/basis_universal/issues/300). Make sure either quality level or endpoint and selector clusters are set. Without this the front-end will fail when trying compress in ETC1S mode and neither quality or the endpoint and selector clusters where changed from their defaults.

    The new behavior is similar to the command line tool, which has a similar check.

    opened by roy-t 1
Releases(1.16.3)
A BOF to parse the imports of a provided PE-file, optionally extracting symbols on a per-dll basis.

PE Import Enumerator BOF What is this? This is a BOF to enumerate DLL files to-be-loaded by a given PE file. Depending on the number of arguments, thi

null 74 May 15, 2022
Python bindings of silk codec.

Python silk module. --- pysilk --- APIs See test\test.py. import pysilk as m m.silkEncode(buf , 24000) m.silkDecode(buf , 24000) #the first param is b

DCZ_Yewen 14 May 27, 2022
Nuvoton codec/amp driver with different ASoC version

Nuvoton-ASoC Nuvoton codec and amp drivers with different ASoC versions. The Nuvoton-ASoC repository stores the Linux driver for codec and amplifier m

Nuvoton Technology Corp. 2 Dec 7, 2021
Stack-based texture generation tool written in C99!

Stack-based texture generation tool written in C99! Brought to you by @zaklaus and contributors Introduction zpl.texed is a cross-platform stack-based

zpl | pushing the boundaries of simplicity. 17 May 1, 2022
Fast single source file BC7/BPTC texture encoder with perceptual metric support

Note: Since this repo was created, we've released two new codecs with better BC7 encoders: https://github.com/richgel999/bc7enc_rdo https://github.com

Rich Geldreich 142 Apr 11, 2022
Texture Packer for Game Development Using MaxRects Algorithm

Overview Texture Packer for Game Development Using MaxRects Algorithm. Note: The game assets used in this example were download from Grassland Tileset

Jeremy HU 61 May 2, 2022
Builds atlas texture from a bunch of input images.

Atlasc @septag atlasc is a command-line program that builds atlas texture from a bunch of input images. Main Features Cross-platform. Runs on linux/ma

Sepehr Taghdisian 74 Jun 10, 2022
Single source file ASTC texture decompression in C++ (derived from Google's open source Android project)

astc_dec astc_dec is a single source file ASTC texture decompressor with the Apache 2.0 license, derived from Google's open source Android sources. Th

Rich Geldreich 28 Apr 11, 2022
Simple font renderer library written in Opengl 3.3 using stb_truetype.h to load a packed bitmap into texture of a .ttf font.

mv_easy_font Simple font renderer library written in Opengl 3.3 using stb_truetype.h to load a packed bitmap into texture of a .ttf font. Uses instanc

null 27 May 13, 2022
An efficient texture-free GLSL procedural noise library

Wombat An efficient texture-free GLSL procedural noise library Source: https://github.com/BrianSharpe/Wombat Derived from: https://github.com/BrianSha

Brian Sharpe 192 Jun 21, 2022
A simulation of Newton's law of universal gravitation

Newton's law of universal gravitation This simulation uses the famous equation of Isaac Newton

Long Nguyen 54 Feb 24, 2022
A universal way to create a noclip mod in Unity games (Mono/IIL2CPP)

Universal-Unity-NoClip This projects aim to show how a noclip mod can be created in any unity game, regardless if its using an il2cpp or mono backend.

Jonah 20 Jun 26, 2022
Universal State Monitor software for home automation input devices

Universal State Monitor Copyright 2019-2021 SuperHouse Automation Pty Ltd www.superhouse.tv A binary state monitor for DIY home automation projects. T

SuperHouse Automation 3 Aug 24, 2021
Universal binaries for Linux.

FatELF The latest information about FatELF can be found at https://icculus.org/fatelf/ What is this? FatELF is a simple file format that allows you to

Ryan C. Gordon 32 May 22, 2022
Miryoku is an ergonomic, minimal, orthogonal, and universal keyboard layout.

Miryoku is an ergonomic, minimal, orthogonal, and universal keyboard layout. This is the Miryoku implementation for QMK.

Manna Harbour 57 Jun 21, 2022
A Script to thin Universal Apps on macOS quickly

UBThinner A Script to thin Universal Apps on macOS quickly. It traverses through the given folder recursively, identifies any universal binaries and t

Arm 2 Dec 26, 2021
Somewhat Universal Widescreen Fix

SUWSF Somewhat Universal Widescreen Fix is intended to enable widescreen aspect ratios (e.g. 21:9, 32:9, 48:9) in games where it is unsupported. WARNI

Chris Yeninas 27 Jun 19, 2022
Miryoku is an ergonomic, minimal, orthogonal, and universal keyboard layout

Miryoku is an ergonomic, minimal, orthogonal, and universal keyboard layout. Miryoku KMonad is the Miryoku implementation for KMonad.

Manna Harbour 16 May 30, 2022