SPIRV-Cross is a tool designed for parsing and converting SPIR-V to other shader languages.

Overview

SPIRV-Cross

SPIRV-Cross is a tool designed for parsing and converting SPIR-V to other shader languages.

CI Build Status

Features

  • Convert SPIR-V to readable, usable and efficient GLSL
  • Convert SPIR-V to readable, usable and efficient Metal Shading Language (MSL)
  • Convert SPIR-V to readable, usable and efficient HLSL
  • Convert SPIR-V to a JSON reflection format
  • Convert SPIR-V to debuggable C++ [DEPRECATED]
  • Reflection API to simplify the creation of Vulkan pipeline layouts
  • Reflection API to modify and tweak OpDecorations
  • Supports "all" of vertex, fragment, tessellation, geometry and compute shaders.

SPIRV-Cross tries hard to emit readable and clean output from the SPIR-V. The goal is to emit GLSL or MSL that looks like it was written by a human and not awkward IR/assembly-like code.

NOTE: Individual features are expected to be mostly complete, but it is possible that certain obscure GLSL features are not yet supported. However, most missing features are expected to be "trivial" improvements at this stage.

Building

SPIRV-Cross has been tested on Linux, iOS/OSX, Windows and Android. CMake is the main build system.

Linux and macOS

Building with CMake is recommended, as it is the only build system which is tested in continuous integration. It is also the only build system which has install commands and other useful build system features.

However, you can just run make on the command line as a fallback if you only care about the CLI tool.

A non-ancient GCC (4.8+) or Clang (3.x+) compiler is required as SPIRV-Cross uses C++11 extensively.

Windows

Building with CMake is recommended, which is the only way to target MSVC. MinGW-w64 based compilation works with make as a fallback.

Android

SPIRV-Cross is only useful as a library here. Use the CMake build to link SPIRV-Cross to your project.

C++ exceptions

The make and CMake build flavors offer the option to treat exceptions as assertions. To disable exceptions for make just append SPIRV_CROSS_EXCEPTIONS_TO_ASSERTIONS=1 to the command line. For CMake append -DSPIRV_CROSS_EXCEPTIONS_TO_ASSERTIONS=ON. By default exceptions are enabled.

Static, shared and CLI

You can use -DSPIRV_CROSS_STATIC=ON/OFF -DSPIRV_CROSS_SHARED=ON/OFF -DSPIRV_CROSS_CLI=ON/OFF to control which modules are built (and installed).

Usage

Using the C++ API

The C++ API is the main API for SPIRV-Cross. For more in-depth documentation than what's provided in this README, please have a look at the Wiki. NOTE: This API is not guaranteed to be ABI-stable, and it is highly recommended to link against this API statically. The API is generally quite stable, but it can change over time, see the C API for more stability.

To perform reflection and convert to other shader languages you can use the SPIRV-Cross API. For example:

#include extern std::vector load_spirv_file(); int main() { // Read SPIR-V from disk or similar. std::vector spirv_binary = load_spirv_file(); spirv_cross::CompilerGLSL glsl(std::move(spirv_binary)); // The SPIR-V is now parsed, and we can perform reflection on it. spirv_cross::ShaderResources resources = glsl.get_shader_resources(); // Get all sampled images in the shader. for (auto &resource : resources.sampled_images) { unsigned set = glsl.get_decoration(resource.id, spv::DecorationDescriptorSet); unsigned binding = glsl.get_decoration(resource.id, spv::DecorationBinding); printf("Image %s at set = %u, binding = %u\n", resource.name.c_str(), set, binding); // Modify the decoration to prepare it for GLSL. glsl.unset_decoration(resource.id, spv::DecorationDescriptorSet); // Some arbitrary remapping if we want. glsl.set_decoration(resource.id, spv::DecorationBinding, set * 16 + binding); } // Set some options. spirv_cross::CompilerGLSL::Options options; options.version = 310; options.es = true; glsl.set_options(options); // Compile to GLSL, ready to give to GL driver. std::string source = glsl.compile(); }">
#include "spirv_glsl.hpp"
#include <vector>
#include <utility>

extern std::vector<uint32_t> load_spirv_file();

int main()
{
	// Read SPIR-V from disk or similar.
	std::vector<uint32_t> spirv_binary = load_spirv_file();

	spirv_cross::CompilerGLSL glsl(std::move(spirv_binary));

	// The SPIR-V is now parsed, and we can perform reflection on it.
	spirv_cross::ShaderResources resources = glsl.get_shader_resources();

	// Get all sampled images in the shader.
	for (auto &resource : resources.sampled_images)
	{
		unsigned set = glsl.get_decoration(resource.id, spv::DecorationDescriptorSet);
		unsigned binding = glsl.get_decoration(resource.id, spv::DecorationBinding);
		printf("Image %s at set = %u, binding = %u\n", resource.name.c_str(), set, binding);

		// Modify the decoration to prepare it for GLSL.
		glsl.unset_decoration(resource.id, spv::DecorationDescriptorSet);

		// Some arbitrary remapping if we want.
		glsl.set_decoration(resource.id, spv::DecorationBinding, set * 16 + binding);
	}

	// Set some options.
	spirv_cross::CompilerGLSL::Options options;
	options.version = 310;
	options.es = true;
	glsl.set_options(options);

	// Compile to GLSL, ready to give to GL driver.
	std::string source = glsl.compile();
}

Using the C API wrapper

To facilitate C compatibility and compatibility with foreign programming languages, a C89-compatible API wrapper is provided. Unlike the C++ API, the goal of this wrapper is to be fully stable, both API and ABI-wise. This is the only interface which is supported when building SPIRV-Cross as a shared library.

An important point of the wrapper is that all memory allocations are contained in the spvc_context. This simplifies the use of the API greatly. However, you should destroy the context as soon as reasonable, or use spvc_context_release_allocations() if you intend to reuse the spvc_context object again soon.

Most functions return a spvc_result, where SPVC_SUCCESS is the only success code. For brevity, the code below does not do any error checking.

#include <spirv_cross_c.h>

const SpvId *spirv = get_spirv_data();
size_t word_count = get_spirv_word_count();

spvc_context context = NULL;
spvc_parsed_ir ir = NULL;
spvc_compiler compiler_glsl = NULL;
spvc_compiler_options options = NULL;
spvc_resources resources = NULL;
const spvc_reflected_resource *list = NULL;
const char *result = NULL;
size_t count;
size_t i;

// Create context.
spvc_context_create(&context);

// Set debug callback.
spvc_context_set_error_callback(context, error_callback, userdata);

// Parse the SPIR-V.
spvc_context_parse_spirv(context, spirv, word_count, &ir);

// Hand it off to a compiler instance and give it ownership of the IR.
spvc_context_create_compiler(context, SPVC_BACKEND_GLSL, ir, SPVC_CAPTURE_MODE_TAKE_OWNERSHIP, &compiler_glsl);

// Do some basic reflection.
spvc_compiler_create_shader_resources(compiler_glsl, &resources);
spvc_resources_get_resource_list_for_type(resources, SPVC_RESOURCE_TYPE_UNIFORM_BUFFER, &list, &count);

for (i = 0; i < count; i++)
{
    printf("ID: %u, BaseTypeID: %u, TypeID: %u, Name: %s\n", list[i].id, list[i].base_type_id, list[i].type_id,
           list[i].name);
    printf("  Set: %u, Binding: %u\n",
           spvc_compiler_get_decoration(compiler_glsl, list[i].id, SpvDecorationDescriptorSet),
           spvc_compiler_get_decoration(compiler_glsl, list[i].id, SpvDecorationBinding));
}

// Modify options.
spvc_compiler_create_compiler_options(compiler_glsl, &options);
spvc_compiler_options_set_uint(options, SPVC_COMPILER_OPTION_GLSL_VERSION, 330);
spvc_compiler_options_set_bool(options, SPVC_COMPILER_OPTION_GLSL_ES, SPVC_FALSE);
spvc_compiler_install_compiler_options(compiler_glsl, options);

spvc_compiler_compile(compiler_glsl, &result);
printf("Cross-compiled source: %s\n", result);

// Frees all memory we allocated so far.
spvc_context_destroy(context);

Linking

CMake add_subdirectory()

This is the recommended way if you are using CMake and want to link against SPIRV-Cross statically.

Integrating SPIRV-Cross in a custom build system

To add SPIRV-Cross to your own codebase, just copy the source and header files from root directory and build the relevant .cpp files you need. Make sure to build with C++11 support, e.g. -std=c++11 in GCC and Clang. Alternatively, the Makefile generates a libspirv-cross.a static library during build that can be linked in.

Linking against SPIRV-Cross as a system library

It is possible to link against SPIRV-Cross when it is installed as a system library, which would be mostly relevant for Unix-like platforms.

pkg-config

For Unix-based systems, a pkg-config is installed for the C API, e.g.:

$ pkg-config spirv-cross-c-shared --libs --cflags
-I/usr/local/include/spirv_cross -L/usr/local/lib -lspirv-cross-c-shared
CMake

If the project is installed, it can be found with find_package(), e.g.:

cmake_minimum_required(VERSION 3.5)
set(CMAKE_C_STANDARD 99)
project(Test LANGUAGES C)

find_package(spirv_cross_c_shared)
if (spirv_cross_c_shared_FOUND)
        message(STATUS "Found SPIRV-Cross C API! :)")
else()
        message(STATUS "Could not find SPIRV-Cross C API! :(")
endif()

add_executable(test test.c)
target_link_libraries(test spirv-cross-c-shared)

test.c:

#include <spirv_cross_c.h>

int main(void)
{
        spvc_context context;
        spvc_context_create(&context);
        spvc_context_destroy(context);
}

CLI

The CLI is suitable for basic cross-compilation tasks, but it cannot support the full flexibility that the API can. Some examples below.

Creating a SPIR-V file from GLSL with glslang

glslangValidator -H -V -o test.spv test.frag

Converting a SPIR-V file to GLSL ES

glslangValidator -H -V -o test.spv shaders/comp/basic.comp
./spirv-cross --version 310 --es test.spv

Converting to desktop GLSL

glslangValidator -H -V -o test.spv shaders/comp/basic.comp
./spirv-cross --version 330 --no-es test.spv --output test.comp

Disable prettifying optimizations

glslangValidator -H -V -o test.spv shaders/comp/basic.comp
./spirv-cross --version 310 --es test.spv --output test.comp --force-temporary

Using shaders generated from C++ backend

Please see samples/cpp where some GLSL shaders are compiled to SPIR-V, decompiled to C++ and run with test data. Reading through the samples should explain how to use the C++ interface. A simple Makefile is included to build all shaders in the directory.

Implementation notes

When using SPIR-V and SPIRV-Cross as an intermediate step for cross-compiling between high level languages there are some considerations to take into account, as not all features used by one high-level language are necessarily supported natively by the target shader language. SPIRV-Cross aims to provide the tools needed to handle these scenarios in a clean and robust way, but some manual action is required to maintain compatibility.

HLSL source to GLSL

HLSL entry points

When using SPIR-V shaders compiled from HLSL, there are some extra things you need to take care of. First make sure that the entry point is used correctly. If you forget to set the entry point correctly in glslangValidator (-e MyFancyEntryPoint), you will likely encounter this error message:

Cannot end a function before ending the current block.
Likely cause: If this SPIR-V was created from glslang HLSL, make sure the entry point is valid.
Vertex/Fragment interface linking

HLSL relies on semantics in order to effectively link together shader stages. In the SPIR-V generated by glslang, the transformation from HLSL to GLSL ends up looking like

struct VSOutput {
   // SV_Position is rerouted to gl_Position
   float4 position : SV_Position;
   float4 coord : TEXCOORD0;
};

VSOutput main(...) {}
struct VSOutput {
   float4 coord;
}
layout(location = 0) out VSOutput _magicNameGeneratedByGlslang;

While this works, be aware of the type of the struct which is used in the vertex stage and the fragment stage. There may be issues if the structure type name differs in vertex stage and fragment stage.

You can make use of the reflection interface to force the name of the struct type.

// Something like this for both vertex outputs and fragment inputs.
compiler.set_name(varying_resource.base_type_id, "VertexFragmentLinkage");

Some platform may require identical variable name for both vertex outputs and fragment inputs. (for example MacOSX) to rename variable base on location, please add

--rename-interface-variable 
    
     
     

     
    
   

HLSL source to legacy GLSL/ESSL

HLSL tends to emit varying struct types to pass data between vertex and fragment. This is not supported in legacy GL/GLES targets, so to support this, varying structs are flattened. This is done automatically, but the API user might need to be aware that this is happening in order to support all cases.

Modern GLES code like this:

struct Output {
   vec4 a;
   vec2 b;
};
out Output vout;

Is transformed into:

struct Output {
   vec4 a;
   vec2 b;
};
varying vec4 Output_a;
varying vec2 Output_b;

Note that now, both the struct name and the member names will participate in the linking interface between vertex and fragment, so API users might want to ensure that both the struct names and member names match so that vertex outputs and fragment inputs can link properly.

Separate image samplers (HLSL/Vulkan) for backends which do not support it (GLSL)

Another thing you need to remember is when using samplers and textures in HLSL these are separable, and not directly compatible with GLSL. If you need to use this with desktop GL/GLES, you need to call Compiler::build_combined_image_samplers first before calling Compiler::compile, or you will get an exception.

get_name(remap.image_id), compiler->get_name(remap.sampler_id))); }">
// From main.cpp
// Builds a mapping for all combinations of images and samplers.
compiler->build_combined_image_samplers();

// Give the remapped combined samplers new names.
// Here you can also set up decorations if you want (binding = #N).
for (auto &remap : compiler->get_combined_image_samplers())
{
   compiler->set_name(remap.combined_id, join("SPIRV_Cross_Combined", compiler->get_name(remap.image_id),
            compiler->get_name(remap.sampler_id)));
}

If your target is Vulkan GLSL, --vulkan-semantics will emit separate image samplers as you'd expect. The command line client calls Compiler::build_combined_image_samplers automatically, but if you're calling the library, you'll need to do this yourself.

Descriptor sets (Vulkan GLSL) for backends which do not support them (pre HLSL 5.1 / GLSL)

Descriptor sets are unique to Vulkan, so make sure that descriptor set + binding is remapped to a flat binding scheme (set always 0), so that other APIs can make sense of the bindings. This can be done with Compiler::set_decoration(id, spv::DecorationDescriptorSet). For other backends like MSL and HLSL, descriptor sets can be used, with some minor caveats, see below.

MSL 2.0+

Metal supports indirect argument buffers (--msl-argument-buffers). In this case, descriptor sets become argument buffers, and bindings are mapped to [[id(N)]] within the argument buffer. One quirk is that arrays of resources consume multiple ids, where Vulkan does not. This can be worked around either from shader authoring stage or remapping bindings as needed to avoid the overlap. There is also a rich API to declare remapping schemes which is intended to work like the pipeline layout in Vulkan. See CompilerMSL::add_msl_resource_binding. Remapping combined image samplers for example must be split into two bindings in MSL, so it's possible to declare an id for the texture and sampler binding separately.

HLSL - SM 5.1+

In SM 5.1+, descriptor set bindings are interpreted as register spaces directly. In HLSL however, arrays of resources consume multiple binding slots where Vulkan does not, so there might be overlap if the SPIR-V was not authored with this in mind. This can be worked around either from shader authoring stage (don't assign overlapping bindings) or remap bindings in SPIRV-Cross as needed to avoid the overlap.

Linking by name for targets which do not support explicit locations (legacy GLSL/ESSL)

Modern GLSL and HLSL sources (and SPIR-V) relies on explicit layout(location) qualifiers to guide the linking process between shader stages, but older GLSL relies on symbol names to perform the linking. When emitting shaders with older versions, these layout statements will be removed, so it is important that the API user ensures that the names of I/O variables are sanitized so that linking will work properly. The reflection API can rename variables, struct types and struct members to deal with these scenarios using Compiler::set_name and friends.

Clip-space conventions

SPIRV-Cross can perform some common clip space conversions on gl_Position/SV_Position by enabling CompilerGLSL::Options.vertex.fixup_clipspace. While this can be convenient, it is recommended to modify the projection matrices instead as that can achieve the same result.

For GLSL targets, enabling this will convert a shader which assumes [0, w] depth range (Vulkan / D3D / Metal) into [-w, w] range. For MSL and HLSL targets, enabling this will convert a shader in [-w, w] depth range (OpenGL) to [0, w] depth range.

By default, the CLI will not enable fixup_clipspace, but in the API you might want to set an explicit value using CompilerGLSL::set_options().

Y-flipping of gl_Position and similar is also supported. The use of this is discouraged, because relying on vertex shader Y-flipping tends to get quite messy. To enable this, set CompilerGLSL::Options.vertex.flip_vert_y or --flip-vert-y in CLI.

Reserved identifiers

When cross-compiling, certain identifiers are considered to be reserved by the implementation. Code generated by SPIRV-Cross cannot emit these identifiers as they are reserved and used for various internal purposes, and such variables will typically show up as _RESERVED_IDENTIFIER_FIXUP_ or some similar name to make it more obvious that an identifier has been renamed.

Reflection output will follow the exact name specified in the SPIR-V module. It might not be a valid identifier in the C sense, as it may contain non-alphanumeric/non-underscore characters.

Reserved identifiers currently assumed by the implementation are (in pseudo-regex):

  • _$digit+, e.g. _100, _2
  • $digit+.+, e.g. _100_tmp, _2_foobar. _2Bar is not reserved.
  • gl_- prefix
  • spv- prefix
  • SPIRV_Cross prefix. This prefix is generally used for interface variables where app needs to provide data for workaround purposes. This identifier will not be rewritten, but be aware of potential collisions.
  • Double underscores (reserved by all target languages).

Members of structs also have a reserved identifier:

  • _m$digit+$END, e.g. _m20 and _m40 are reserved, but not _m40Foobar.

Contributing

Contributions to SPIRV-Cross are welcome. See Testing and Licensing sections for details.

Testing

SPIRV-Cross maintains a test suite of shaders with reference output of how the output looks after going through a roundtrip through glslangValidator/spirv-as then back through SPIRV-Cross again. The reference files are stored inside the repository in order to be able to track regressions.

All pull requests should ensure that test output does not change unexpectedly. This can be tested with:

./checkout_glslang_spirv_tools.sh # Checks out glslang and SPIRV-Tools at a fixed revision which matches the reference output.
./build_glslang_spirv_tools.sh    # Builds glslang and SPIRV-Tools.
./test_shaders.sh                 # Runs over all changes and makes sure that there are no deltas compared to reference files.

./test_shaders.sh currently requires a Makefile setup with GCC/Clang to be set up. However, on Windows, this can be rather inconvenient if a MinGW environment is not set up. To use a spirv-cross binary you built with CMake (or otherwise), you can pass in an environment variable as such:

SPIRV_CROSS_PATH=path/to/custom/spirv-cross ./test_shaders.sh

However, when improving SPIRV-Cross there are of course legitimate cases where reference output should change. In these cases, run:

./update_test_shaders.sh          # SPIRV_CROSS_PATH also works here.

to update the reference files and include these changes as part of the pull request. Always make sure you are running the correct version of glslangValidator as well as SPIRV-Tools when updating reference files. See checkout_glslang_spirv_tools.sh which revisions are currently expected. The revisions change regularly.

In short, the master branch should always be able to run ./test_shaders.py shaders and friends without failure. SPIRV-Cross uses Travis CI to test all pull requests, so it is not strictly needed to perform testing yourself if you have problems running it locally. A pull request which does not pass testing on Travis will not be accepted however.

When adding support for new features to SPIRV-Cross, a new shader and reference file should be added which covers usage of the new shader features in question. Travis CI runs the test suite with the CMake, by running ctest. This is a more straight-forward alternative to ./test_shaders.sh.

Licensing

Contributors of new files should add a copyright header at the top of every new source code file with their copyright along with the Apache 2.0 licensing stub.

Formatting

SPIRV-Cross uses clang-format to automatically format code. Please use clang-format with the style sheet found in .clang-format to automatically format code before submitting a pull request.

To make things easy, the format_all.sh script can be used to format all source files in the library. In this directory, run the following from the command line:

./format_all.sh

Regression testing

In shaders/ a collection of shaders are maintained for purposes of regression testing. The current reference output is contained in reference/. ./test_shaders.py shaders can be run to perform regression testing.

See ./test_shaders.py --help for more.

Metal backend

To test the roundtrip path GLSL -> SPIR-V -> MSL, --msl can be added, e.g. ./test_shaders.py --msl shaders-msl.

HLSL backend

To test the roundtrip path GLSL -> SPIR-V -> HLSL, --hlsl can be added, e.g. ./test_shaders.py --hlsl shaders-hlsl.

Updating regression tests

When legitimate changes are found, use --update flag to update regression files. Otherwise, ./test_shaders.py will fail with error code.

Mali Offline Compiler cycle counts

To obtain a CSV of static shader cycle counts before and after going through spirv-cross, add --malisc flag to ./test_shaders. This requires the Mali Offline Compiler to be installed in PATH.

Issues
  • MSL: gl_Position is undetectable in tessellation stage IO using the SPIRV-Cross reflection API

    MSL: gl_Position is undetectable in tessellation stage IO using the SPIRV-Cross reflection API

    At Unity we're working on adding Metal tessellation support using SPIRV-Cross but have hit a limitation, hopefully there's already a way of doing this that we're unaware of.

    The issue is that there does not seem to be a way to tell when and at what offset the generated MSL will contain a gl_Position member in IO interface for the Vertex or Hull kernels or the Domain vertex function as it's not reported along with the other SPVC_RESOURCE_TYPE_STAGE_INPUT or SPVC_RESOURCE_TYPE_STAGE_OUTPUT resource list members even though it's in the generated MSL.

    This results in the following problems when trying to validate and link up the different Metal stages:

    1. To be able to set up control-point attribute data (MTLVertexAttributeDescriptor data in MTLVertexDescriptor ) for the post-tess vertex (domain) stage it is necessary to know the layout of the Hull kernel's control point output buffer. As the data transfer is done via buffer it does not seem possible to work around this and get the data layout from the Metal reflection for the Domain MTLFunction.
    2. If we don't know that gl_Position is in the vertex kernel output or the hull kernel output, we cannot know the control point size/stride to be able to allocate minimally sized buffers for communication between the tessellation stages (this can be worked around by always assuming gl_Position is there and padding each point with a float4 though is suboptimal)
    3. As we use --msl-multi-patch-workgroup the communication between the vertex and hull kernels happens directly through a buffer. We'd like to validate that the format is identical but since we cannot know if only one of them contains gl_Position this can't be done for all cases. Note that if --msl-multi-patch-workgroup is not used, stage IO validation is still not possible since Vertex stage output may or may not contain gl_Position even though it could be detected in the Hull stage input using the Metal runtime reflection (but not using SPIRV-Cross)
    4. Similar as point 3, we'd like to be able to verify that the gl_Position post-tess vertex (domain) function input attribute was written to by the previous stage after compiling the two stages with SPIRV-Cross

    Currently the only solution seems to be to manually parse out the OpEntryPoint instruction and to go over the list of Interface <id>s and only use SPIRV-Cross to check if any of them have a BuiltIn decoration of Position. This also requires us to assume that the gl_Position member will always be places at the end of the IO structs, as there's no way to get its offset in the IO buffers as well. As the placement looks like an arbitrary SPIRV-Cross implementation detail it does not seem like something we should rely on.

    • The following is the source for the vert/hull/domain stages: tess_source.txt

    • This is the associated SPIR-V after compiling with DXC (binaries downloaded from shader playground, .o files renamed to .spv.txt to upload to github) Vert.spv.txt Hull.spv.txt Domain.spv.txt

    • The following is the MSL generated with SPIRV-Cross Vert.msl.txt ./spirv-cross --msl --msl-vertex-for-tessellation Vert.spv.txt --output Vert.msl.txt Hull.msl.txt ./spirv-cross --msl --msl-multi-patch-workgroup Hull.spv.txt --output Hull.msl.txt Domain.msl.txt ./spirv-cross --msl Domain.spv.txt --output Domain.msl.txt

    question 
    opened by Dredhog 51
  • MSL : Translate C style array to array<T> causes regression on Dawn end2end tests

    MSL : Translate C style array to array causes regression on Dawn end2end tests

    After applying this patch, VertexFormatTests) from end2end tests of Dawn is broken on Mac of Intel Iris Pro platform. The root cause seems to be Intel platform doesn't support variable as indexing of array<array> type on Iris Pro. Here are detail infos. This is the original glsl shader :

    #version 450
            layout(location = 0) in ivec2 test;
            layout(location = 0) out vec4 color;
            
            void main() {
                int expected[3][2];
                expected[0][0] = int(127);
                expected[0][1] = int(0);
                expected[1][0] = int(-128);
                expected[1][1] = int(-2);
                expected[2][0] = int(120);
                expected[2][1] = int(-121);
                
                bool success = true;
                int testVal0;
                int expectedVal0;
                int testVal1;
                int expectedVal1;
                testVal0 = test[0];
                testVal1 = test[1];
                expectedVal0 = expected[gl_VertexIndex][0];
                expectedVal1 = expected[gl_VertexIndex][1];
                /* Workaround     
                    if (gl_VertexIndex == 0) {
                        expectedVal0 = expected[0][0];
                        expectedVal1 = expected[0][1];
                    } else if (gl_VertexIndex == 1) {
                        expectedVal0 = expected[1][0];
                        expectedVal1 = expected[1][1];
                    } else {
                        expectedVal0 = expected[2][0];
                        expectedVal1 = expected[2][1];
                    }
                }*/
                success = success && (testVal0 == expectedVal0);
                success = success && (testVal1 == expectedVal1);
                if (success) {
                    color = vec4(0.0f, 1.0f, 0.0f, 1.0f);
                } else {
                    color = vec4(1.0f, 0.0f, 0.0f, 1.0f);
                }
               
                const vec2 pos[3] = vec2[3](vec2(0.0f, 0.5f), vec2(-0.5f, -0.5f), vec2(0.5f, -0.5f));
                gl_Position = vec4(pos[gl_VertexIndex], 0.0, 1.0);
    

    And the translated metal shader before applying array patch is :

    #include <simd/simd.h>
    
    using namespace metal;
    
    constant float2 _88[3] = { float2(0.0, 0.5), float2(-0.5), float2(0.5, -0.5) };
    
    struct main0_out
    {
        float4 color [[user(locn0)]];
        float4 gl_Position [[position]];
    };
    
    struct main0_in
    {
        int2 test [[attribute(0)]];
    };
    
    vertex main0_out main0(main0_in in [[stage_in]], uint gl_VertexIndex [[vertex_id]])
    {
        main0_out out = {};
        int expected[3][2];
        expected[0][0] = 127;
        expected[0][1] = 0;
        expected[1][0] = -128;
        expected[1][1] = -2;
        expected[2][0] = 120;
        expected[2][1] = -121;
        bool success = true;
        bool useVariable = true;
        int testVal0 = in.test.x;
        int testVal1 = in.test.y;
        int expectedVal0 = expected[int(gl_VertexIndex)][0];
        int expectedVal1 = expected[int(gl_VertexIndex)][1];
        success = success && (testVal0 == expectedVal0);
        success = success && (testVal1 == expectedVal1);
        if (success)
        {
            out.color = float4(0.0, 1.0, 0.0, 1.0);
        }
        else
        {
            out.color = float4(1.0, 0.0, 0.0, 1.0);
        }
        out.gl_Position = float4(_88[int(gl_VertexIndex)], 0.0, 1.0);
        return out;
    }
    

    The translated metal shader after applying array patch is :

    #pragma clang diagnostic ignored "-Wmissing-braces"
    #pragma clang diagnostic ignored "-Wunused-variable"
    
    #include <metal_stdlib>
    #include <simd/simd.h>
    	
    template <typename T, size_t Num>
    struct unsafe_array
    {
    	T __Elements[Num ? Num : 1];
    	
    	constexpr size_t size() const thread { return Num; }
    	constexpr size_t max_size() const thread { return Num; }
    	constexpr bool empty() const thread { return Num == 0; }
    	
    	constexpr size_t size() const device { return Num; }
    	constexpr size_t max_size() const device { return Num; }
    	constexpr bool empty() const device { return Num == 0; }
    	
    	constexpr size_t size() const constant { return Num; }
    	constexpr size_t max_size() const constant { return Num; }
    	constexpr bool empty() const constant { return Num == 0; }
    	
    	constexpr size_t size() const threadgroup { return Num; }
    	constexpr size_t max_size() const threadgroup { return Num; }
    	constexpr bool empty() const threadgroup { return Num == 0; }
    	
    	thread T &operator[](size_t pos) thread
    	{
    		return __Elements[pos];
    	}
    	constexpr const thread T &operator[](size_t pos) const thread
    	{
    		return __Elements[pos];
    	}
    	
    	device T &operator[](size_t pos) device
    	{
    		return __Elements[pos];
    	}
    	constexpr const device T &operator[](size_t pos) const device
    	{
    		return __Elements[pos];
    	}
    	
    	constexpr const constant T &operator[](size_t pos) const constant
    	{
    		return __Elements[pos];
    	}
    	
    	threadgroup T &operator[](size_t pos) threadgroup
    	{
    		return __Elements[pos];
    	}
    	constexpr const threadgroup T &operator[](size_t pos) const threadgroup
    	{
    		return __Elements[pos];
    	}
    };
    
    using namespace metal;
    
    constant unsafe_array<float2,3> _88 = unsafe_array<float2,3>({ float2(0.0, 0.5), float2(-0.5), float2(0.5, -0.5) });
    
    struct main0_out
    {
        float4 color [[user(locn0)]];
        float4 gl_Position [[position]];
    };
    
    struct main0_in
    {
        int2 test [[attribute(0)]];
    };
    
    vertex main0_out main0(main0_in in [[stage_in]], uint gl_VertexIndex [[vertex_id]])
    {
        main0_out out = {};
        unsafe_array<unsafe_array<int,3>,2> expected;
        expected[0][0] = 127;
        expected[0][1] = 0;
        expected[1][0] = -128;
        expected[1][1] = -2;
        expected[2][0] = 120;
        expected[2][1] = -121;
        bool success = true;
        bool useVariable = true;
        int testVal0 = in.test.x;
        int testVal1 = in.test.y;
        int expectedVal0 = expected[int(gl_VertexIndex)][0];
        int expectedVal1 = expected[int(gl_VertexIndex)][1];
        success = success && (testVal0 == expectedVal0);
        success = success && (testVal1 == expectedVal1);
        if (success)
        {
            out.color = float4(0.0, 1.0, 0.0, 1.0);
        }
        else
        {
            out.color = float4(1.0, 0.0, 0.0, 1.0);
        }
        out.gl_Position = float4(_88[int(gl_VertexIndex)], 0.0, 1.0);
        return out;
    }
    

    The only difference between these two shaders are using of unsafe_array to replace C style array.

    I'm doing some initial experiment and find a simple workaround in glsl is to replace

                expectedVal0 = expected[gl_VertexIndex][0];
                expectedVal1 = expected[gl_VertexIndex][1];
    

    with a non variable indexing form like this:

     if (gl_VertexIndex == 0) {
                        expectedVal0 = expected[0][0];
                        expectedVal1 = expected[0][1];
                    } else if (gl_VertexIndex == 1) {
                        expectedVal0 = expected[1][0];
                        expectedVal1 = expected[1][1];
                    } else {
                        expectedVal0 = expected[2][0];
                        expectedVal1 = expected[2][1];
                    }
    

    can make the tests work again. This proves that Intel platform doesn't support variable as indexing of array<array> type on Iris Pro.

    I also did some experiment on MacOS of UHD630 Intel graphics platform, it turns out that all the cases using variable as indexing of a 1D array failed.

    I think this maybe a driver issue for Intel GPU on metal(Need to double check). And for spirv-cross, can we workaround Intel Platform by not translating C style array to array form?

    question needs triage workaround 
    opened by shaoboyan 46
  • Segmentation fault when initializing CompilerGLSL object.

    Segmentation fault when initializing CompilerGLSL object.

    I am trying to parse some spirv code generated from glsl and it crashes on construction. I am sure the spirv array is correct because I am creating vulkan shader modules using shaderc that compile and run fine.

    This is the smallest program I could make that reproduces the error:

    
       Compiler compiler;
       CompileOptions options;
       shaderc::SpvCompilationResult compilation_result = compiler.CompileGlslToSpv(
           shader_source, shaderc_shader_kind::shaderc_glsl_vertex_shader, shader_path.c_str(), options);
    
       std::vector<uint32_t> spirv_shader;
       spirv_shader.assign(compilation_result.cbegin(), compilation_result.cend());
    
       std::vector<uint32_t> source_copy = spirv_shader;
       spirv_cross::CompilerGLSL glsl(source_copy);
    

    where shader_source is the std string containing the glsl string of the shader.

    With input:

    #version 450
    
    layout(location = 0) in vec2 inPosition;
    layout(location = 1) in vec3 inColor;
    
    layout(location = 0) out vec3 fragColor;
    
    layout(binding = 0) uniform MatUBO
    {
       mat2 transform;
    };
    
    
    void main() {
       gl_Position = vec4(inPosition, 0.5, 1.0);
       fragColor = inColor;
    }
    

    OS Ubuntu 21.10, compiler clang. The error is a segmentation fault thrown at line: spirv_common.hpp:86

    opened by Makogan 29
  • Lots of changes and fixes with new test cases to make SPIRV-Cross work in UE4 for Metal

    Lots of changes and fixes with new test cases to make SPIRV-Cross work in UE4 for Metal

    This pull request is the result of a bunch of work from my former colleague and some adjustments by myself in order to make SPIRV-Cross work in conjunction with ShaderConductor in Unreal Engine 4.

    Several new test cases were added to justify the changes that were made. I also renamed the respective test shaders to match the guideline of this project. Most of the changed files in this pull request are the updated reference shaders.

    One of the most notable changes is the introduction of the spvUnsafeArray<> template struct to make arrays a value type in Metal. It's called "unsafe" because there are no boundary checks and it's really just a wrapper with respective operator [] functions around a native array to make them compatible with how structured and byte-address buffers work. The spvArrayCopy* functions can now be avoided most of the time to let the Metal compiler do a better optimization. However, spvArrayCopy* wrappers are still required when arrays are declared in threadgroup address space.

    Another change worth mentioning is that the translation of HLSL's SV_InstanceID and SV_VertexID works correct now, since these system values always start at 0. Metal's [[instance_id]]/[[vertex_id]] and GLSL's gl_VertexIndex/gl_InstanceIndex, however, always start at the base index specified in the draw command. Both Metal and GLSL support system values for these base indices which are now exposed to make the translation of SV_InstanceID and SV_VertexID behave in the same way.

    We also had to re-enable some features that were removed recently, like the user defined decoration binding, which can now be enabled again with the respective boolean in CompilerMSL::Options but it's disabled by default, to keep the behavior as it was without these changes.

    We hope to get these changes into Khronos' repository to simplify our update iterations. Any feedback is highly welcome.

    opened by LukasBanana 27
  • Treat array of writable textures as individual textures when cross-compiling to MSL

    Treat array of writable textures as individual textures when cross-compiling to MSL

    Hello! We recently faced a problem with array of writable textures in MSL: the following shader fails to correctly compile after conversion to MSL (https://github.com/KhronosGroup/MoltenVK/issues/517, https://github.com/DiligentGraphics/DiligentCore/issues/66):

    layout(rgba8) uniform writeonly image2DArray OutMip[4];
    ...
    imageStore(OutMip[3], ivec3(GlobalInd.xy / 8u, ArraySlice), PackColor(Src1));
    

    The problem is that it does not seem that array of textures can be writable in MSL. @billhollings suggests that this can potentially be remedied by by forcing an array of writable textures to be treated as individual textures and assigning consecutive Metal texture indices to the declarations.

    Is that something that you may consider to implement?

    wontfix 
    opened by DiligentGraphics 27
  • Crash in spirv_cross::Meta::~Meta() on iOS

    Crash in spirv_cross::Meta::~Meta() on iOS

    Hello!

    I am seeing the following crash when releasing spirv-cross compiler:

    #0	0x000000010445a308 in std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, unsigned int> > >::__deallocate_node(std::__1::__hash_node_base<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, unsigned int>, void*>*>*) at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1547
    #1	0x000000010445a1bc in std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, unsigned int> > >::~__hash_table() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1503
    #2	0x000000010445a16c in std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, unsigned int>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, unsigned int> > >::~__hash_table() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1495
    #3	0x000000010445a138 in std::__1::unordered_map<unsigned int, unsigned int, std::__1::hash<unsigned int>, std::__1::equal_to<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, unsigned int> > >::~unordered_map() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1370
    #4	0x0000000104459968 in std::__1::unordered_map<unsigned int, unsigned int, std::__1::hash<unsigned int>, std::__1::equal_to<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, unsigned int> > >::~unordered_map() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1370
    #5	0x000000010445a900 in spirv_cross::Meta::~Meta() at /git/DiligentEngine/DiligentCore/External/SPIRV-Cross/spirv_common.hpp:1381
    #6	0x000000010445a8c4 in spirv_cross::Meta::~Meta() at /git/DiligentEngine/DiligentCore/External/SPIRV-Cross/spirv_common.hpp:1381
    #7	0x000000010445a890 in std::__1::pair<unsigned int const, spirv_cross::Meta>::~pair() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/utility:312
    #8	0x000000010445a854 in std::__1::pair<unsigned int const, spirv_cross::Meta>::~pair() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/utility:312
    #9	0x000000010445a7d8 in void std::__1::allocator_traits<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, void*> > >::__destroy<std::__1::pair<unsigned int const, spirv_cross::Meta> >(std::__1::integral_constant<bool, false>, std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, void*> >&, std::__1::pair<unsigned int const, spirv_cross::Meta>*) [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:1732
    #10	0x000000010445a7d0 in void std::__1::allocator_traits<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, void*> > >::destroy<std::__1::pair<unsigned int const, spirv_cross::Meta> >(std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, void*> >&, std::__1::pair<unsigned int const, spirv_cross::Meta>*) [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:1595
    #11	0x000000010445a7c0 in std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta> > >::__deallocate_node(std::__1::__hash_node_base<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, void*>*>*) at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1564
    #12	0x000000010445a614 in std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta> > >::~__hash_table() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1503
    #13	0x000000010445a5c4 in std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, spirv_cross::Meta>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, spirv_cross::Meta> > >::~__hash_table() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1495
    #14	0x000000010445a590 in std::__1::unordered_map<unsigned int, spirv_cross::Meta, std::__1::hash<unsigned int>, std::__1::equal_to<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, spirv_cross::Meta> > >::~unordered_map() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1370
    #15	0x00000001044599d0 in std::__1::unordered_map<unsigned int, spirv_cross::Meta, std::__1::hash<unsigned int>, std::__1::equal_to<unsigned int>, std::__1::allocator<std::__1::pair<unsigned int const, spirv_cross::Meta> > >::~unordered_map() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__hash_table:1370
    #16	0x00000001044598a8 in spirv_cross::ParsedIR::~ParsedIR() at /git/DiligentEngine/DiligentCore/External/SPIRV-Cross/spirv_cross_parsed_ir.hpp:33
    #17	0x0000000104da9a70 in spirv_cross::Compiler::~Compiler() ()
    #18	0x0000000104ddc920 in spirv_cross::Compiler::~Compiler() ()
    

    There is nothing special about how compiler is used as well as about the byte code it parses:

    std::unique_ptr<spirv_cross::Parser> pParser(new spirv_cross::Parser(std::move(spirv_binary)));
    pParser->parse();
    auto ParsedIRSource = pParser->get_parsed_ir().source;
    std::unique_ptr<spirv_cross::Compiler> pCompiler(new spirv_cross::Compiler(std::move(pParser->get_parsed_ir())));
    

    This code works fine on Windows, Linux and Mac, but on iOS it crashes. If both parser and compiler are not destroyed, the app continues successfully and runs as expected.

    ready for work cross-project 
    opened by DiligentGraphics 23
  • GLSL: RelaxedPrecision decoration applied to operation result IDs is not correctly translated to essl

    GLSL: RelaxedPrecision decoration applied to operation result IDs is not correctly translated to essl

    Hi, It appears that SPIRV-Cross might not be translating RelaxedPrecision decorations correctly into essl.

    This is similar to case https://github.com/KhronosGroup/SPIRV-Cross/issues/1855 but more general.

    Here's an example HLSL fragment shader:

    min16float RelaxedCalc(min16float3 input)
    {
        min16float temp0 = input.x;
        min16float temp1 = input.y;
        min16float temp2 = input.z;
        return temp0 * temp1 * temp2;
    }
    
    float FloatCalc(min16float3 input)
    {
        float temp0 = input.x;
        float temp1 = input.y;
        float temp2 = input.z;
        return temp0 * temp1 * temp2;
    }
    
    float4 PSMain(min16float4 color : COLOR) : SV_TARGET
    {
        return float4(RelaxedCalc(color), FloatCalc(color), 0, 0);
    }
    

    Here's the SPIR-V that DXC generates for it:

    ; SPIR-V
    ; Version: 1.0
    ; Generator: Google spiregg; 0
    ; Bound: 21
    ; Schema: 0
                   OpCapability Shader
                   OpMemoryModel Logical GLSL450
                   OpEntryPoint Fragment %PSMain "PSMain" %in_var_COLOR %out_var_SV_TARGET
                   OpExecutionMode %PSMain OriginUpperLeft
                   OpSource HLSL 600
                   OpName %in_var_COLOR "in.var.COLOR"
                   OpName %out_var_SV_TARGET "out.var.SV_TARGET"
                   OpName %PSMain "PSMain"
                   OpDecorate %in_var_COLOR Location 0
                   OpDecorate %out_var_SV_TARGET Location 0
                   OpDecorate %in_var_COLOR RelaxedPrecision
                   OpDecorate %4 RelaxedPrecision
                   OpDecorate %5 RelaxedPrecision
                   OpDecorate %6 RelaxedPrecision
                   OpDecorate %7 RelaxedPrecision
                   OpDecorate %8 RelaxedPrecision
                   OpDecorate %9 RelaxedPrecision
          %float = OpTypeFloat 32
        %float_0 = OpConstant %float 0
        %v4float = OpTypeVector %float 4
    %_ptr_Input_v4float = OpTypePointer Input %v4float
    %_ptr_Output_v4float = OpTypePointer Output %v4float
           %void = OpTypeVoid
             %16 = OpTypeFunction %void
    %in_var_COLOR = OpVariable %_ptr_Input_v4float Input
    %out_var_SV_TARGET = OpVariable %_ptr_Output_v4float Output
         %PSMain = OpFunction %void None %16
             %17 = OpLabel
              %4 = OpLoad %v4float %in_var_COLOR
              %5 = OpCompositeExtract %float %4 0
              %6 = OpCompositeExtract %float %4 1
              %7 = OpCompositeExtract %float %4 2
              %8 = OpFMul %float %5 %6
              %9 = OpFMul %float %8 %7
             %18 = OpFMul %float %5 %6
             %19 = OpFMul %float %18 %7
             %20 = OpCompositeConstruct %v4float %9 %19 %float_0 %float_0
                   OpStore %out_var_SV_TARGET %20
                   OpReturn
                   OpFunctionEnd
    

    SPIRV-Cross generates the following essl:

    #version 320 es
    precision mediump float;
    precision highp int;
    
    layout(location = 0) in vec4 in_var_COLOR;
    layout(location = 0) out highp vec4 out_var_SV_TARGET;
    
    void main()
    {
        out_var_SV_TARGET = vec4((in_var_COLOR.x * in_var_COLOR.y) * in_var_COLOR.z, (in_var_COLOR.x * in_var_COLOR.y) * in_var_COLOR.z, 0.0, 0.0);
    }
    

    Basically, in the essl produced by SPIRV-Cross, all of the multiplications are allowed to run at mediump, even though only the result IDs of the multiplications in RelaxedCalc i.e. %8, %9 are decorated with RelaxedPrecision, and the result IDs corresponding to FloatCalc i.e. %19, %20 don't have RelaxedPrecision decorations, which to my understanding is not correct.

    What I would expect is that whenever an operation doesn't have a RelaxedPrecision decoration the mediump inputs for it in the essl should first be moved into highp temporaries (only necessary when there isn't at least one highp input already). The opposite should also be true i.e. whenever an operation is decorated with RelaxedPrecision all of the operands which aren't already mediump should be first moved into mediump temporaries.

    Here's my understanding of the situation:

    • In SPIR-V decorating the result of e.g. an OpFMul with RelaxedPrecision means it should execute at relaxed precision.
    • Not decorating OpFMul with RelaxedPrecision means it should execute at full precision, this is independent of if the operands have RelaxedPrecision decorations.
    • For RelaxedPrecision variables the only operations which implicitly inherit the decoration seem to be loads and stores, not arithmetic ops, this is how I understand the SPIR-V spec.
    • On the other hand, in essl, an operation's precision is independent of the destination's precision but depends on the precision of the operands (I believe there are some rare exceptions when none of the operands have a precision specified, but I don't think this applies to SPIRV-Cross as it always provides a default precision).
    enhancement in progress 
    opened by Dredhog 22
  • Metal indirect argument buffers

    Metal indirect argument buffers

    Feature

    SPIRV-Cross needs to be able to generate argument buffer descriptions from given bindings. For example, given the SPIRV generated from the following GLSL:

    layout(set = 0, binding = 0) uniform texture2D u_Texture;
    layout(set = 0, binding = 1) uniform sampler u_Sampler;
    

    The Metal-1 version of the PS entry point (using those bindings) would look like:

    fragment float4 ps_main(VsOutput in [[stage_in]], 
        texture2d<float> u_Texture [[ texture(0) ]],
        sampler u_Sampler [[ sampler(1) ]])
    {...}
    

    With Metal-2 and the indirect argument buffers, we'd need the following code generated:

    struct IndirectArgSet0 { // contains all the resources with `set = 0`
      texture2d<float> u_Texture [[ id(0) ]]; // binding indices are the same
      sampler u_Sampler [[ id(1) ]];
    };
    fragment float4 ps_main(VsOutput in [[stage_in]],
      device IndirectArgSet0* pixelArg [[ buffer(0) ]]) // note the buffer index
    {...}
    

    Using indirect argument buffers provides a fast path of binding resources on Metal2, similar to how descriptor sets are bound in Vulkan/DX12.

    Issues

    One implementation detail that is not clear is what buffer index to assign to. A vertex program may still have vertex attributes provided in buffers (while everything else will go into the indirect arguments), so a collision is possible. I propose an option to specify the starting indirect buffer binding:

    spirv-cross --msl-indirect-argument-buffers <start_index>
    

    Background

    gfx-rs has the indirect argument code path implemented for the descriptor set binding implementation on Metal2. Here is the switch in the core/quad example. We currently provide MSL code manually, which is obviously different between regular and indirect argument paths. We'd like both versions to be generated by SPIRV-Cross.

    enhancement in progress 
    opened by kvark 20
  • MSL: RWBuffer<uint> is translated into a texture2d and buffer of atomic_int even though SPIRV-Cross can tell the original type

    MSL: RWBuffer is translated into a texture2d and buffer of atomic_int even though SPIRV-Cross can tell the original type

    We're using DXC to turn the following HLSL:

    RWBuffer<uint> g_DispatchIndirectBuffer : register( u0 );
    
    [numthreads(1, 1, 1)]
    void CSMain()
    {
        uint prevGroupCnt;
        InterlockedAdd(g_DispatchIndirectBuffer[0], 4, prevGroupCnt);
    }
    

    Into the following SPIR-V

    ; SPIR-V
    ; Version: 1.0
    ; Generator: Google spiregg; 0
    ; Bound: 15
    ; Schema: 0
                   OpCapability Shader
                   OpCapability SampledBuffer
                   OpCapability ImageBuffer
                   OpMemoryModel Logical GLSL450
                   OpEntryPoint GLCompute %CSMain "CSMain"
                   OpExecutionMode %CSMain LocalSize 1 1 1
                   OpSource HLSL 630
                   OpName %type_buffer_image "type.buffer.image"
                   OpName %g_DispatchIndirectBuffer "g_DispatchIndirectBuffer"
                   OpName %CSMain "CSMain"
                   OpDecorate %g_DispatchIndirectBuffer DescriptorSet 0
                   OpDecorate %g_DispatchIndirectBuffer Binding 0
           %uint = OpTypeInt 32 0
         %uint_0 = OpConstant %uint 0
         %uint_4 = OpConstant %uint 4
    %type_buffer_image = OpTypeImage %uint Buffer 2 0 0 2 R32ui
    %_ptr_UniformConstant_type_buffer_image = OpTypePointer UniformConstant %type_buffer_image
           %void = OpTypeVoid
              %9 = OpTypeFunction %void
    %_ptr_Image_uint = OpTypePointer Image %uint
         %uint_1 = OpConstant %uint 1
    %g_DispatchIndirectBuffer = OpVariable %_ptr_UniformConstant_type_buffer_image UniformConstant
         %CSMain = OpFunction %void None %9
             %12 = OpLabel
             %13 = OpImageTexelPointer %_ptr_Image_uint %g_DispatchIndirectBuffer %uint_0 %uint_0
             %14 = OpAtomicIAdd %uint %13 %uint_1 %uint_0 %uint_4
                   OpReturn
                   OpFunctionEnd
    

    and then we use SPIRV-Cross to generate MSL:

    #pragma clang diagnostic ignored "-Wunused-variable"
    
    #include <metal_stdlib>
    #include <simd/simd.h>
    #include <metal_atomic>
    
    using namespace metal;
    
    kernel void CSMain(texture2d<uint> g_DispatchIndirectBuffer [[texture(0)]], device atomic_uint* g_DispatchIndirectBuffer_atomic [[buffer(0)]])
    {
        uint _14 = atomic_fetch_add_explicit((device atomic_uint*)&g_DispatchIndirectBuffer_atomic[0u], 4u, memory_order_relaxed);
    }
    

    However, it creates a redundant texture2d and also changes the name of the buffer resource by appending _atomic at the end. Both of these could be avoided by doing something similar as done for RWStructuredBuffer, which would produce a regular MSL buffer of uint (not atomic_int) and would only cast to atomic where necessary (seen here). If I use SPIRV-Cross to generate HLSL instead of MSL it will produce the RWBuffer as in the original source:

    RWBuffer<uint> g_DispatchIndirectBuffer : register(u0, space0);
    
    void comp_main()
    {
        uint _14;
        InterlockedAdd(g_DispatchIndirectBuffer[0u], 4u, _14);
    }
    
    [numthreads(1, 1, 1)]
    void main()
    {
        comp_main();
    }
    

    so it seems to have enough context to know that the texture won't be needed. This issue does not seem to be specific to DXC as I get the same output with glslang as seen here. Also to the point about context - using SPIRV-Cross reflection on the compiled MSL reports this resource as a SPVC_RESOURCE_TYPE_STORAGE_IMAGE and a dimensionality of buffer

    spvc_type imageType = spvc_compiler_get_type_handle(spvCompiler, textureResource.base_type_id);        
    bool isBuffer = textureDimensionality == SpvDimBuffer;  //This is true for RWBuffers and not for RWTextures
    

    The issue seems vaguely related to https://github.com/KhronosGroup/SPIRV-Cross/issues/950.

    However I don't agree with the recommendation that instead of a RWBuffer we should use a RWStructuredBuffer or RWByteAddressBuffer since they produce the desired MSL buffer. This is not an option for us at Unity as our users can write their own shaders and thus the issue can't be fixed at the HLSL level.

    question rejected workaround 
    opened by Dredhog 18
  • Use constant address space for SPIR-V parameters when generating tessellation control shader.

    Use constant address space for SPIR-V parameters when generating tessellation control shader.

    In MoltenVK, MVKCommandEncoder::setComputeBytes can call either setBytes or setBuffer, but Metal only allows setBytes to be used with the constant address space. As a small read-only buffer, this should be a safe change for both cases.

    opened by mbarriault 18
  • DecorationNonWritable and DecorationNonReadable are always set on Storage Images

    DecorationNonWritable and DecorationNonReadable are always set on Storage Images

    Hi SPIRV-Cross developers:

    I am a Chromium/Dawn developer and now I am meeting several issues in SPIR-V Cross when I am implementing storage textures in Dawn.

    Now readonly-storage-texture and writeonly-storage-texture have been supported as WebGPU binding types, on D3D12. readonly-storage-texture is intended to match SRVs, and writeonly-storage-texture is intended to match UAVs.

    But in SPIRV-Cross, I find whether I declare DecorationNonReadable or DecorationNonWritable on an image variable:

    • I can always detect both the decoration DecorationNonWritable and DecorationNonReadable be set. It seems here is the code in SPIRV-Cross where both these two decorations are set on all image variables.
    • The image variable is always translated into RWTexture2D when it is translated into HLSL.

    Here is an example. Considering the following GLSL shader:

    // 1.comp
    #version 450
    layout(set = 0, binding = 0, rgba8) uniform highp readonly image2D image0;
    layout(set = 0, binding = 1, rgba8) uniform highp writeonly image2D image1;
    void main() {
        vec4 pixel = imageLoad(image0, ivec2(gl_LocalInvocationID));
        imageStore(image1, ivec2(gl_LocalInvocationID), pixel);
    }
    

    Here is the translation result of 1.comp with glslang. You can see NonWritable and NonReadable has been correctly translated in the result.

    // 1.spv
    // glslang_validator.exe -H -V -o 1.spv 1.comp
    // Module Version 10000
    // Generated by (magic number): 80008
    // Id's are bound by 35
    
                                  Capability Shader
                   1:             ExtInstImport  "GLSL.std.450"
                                  MemoryModel Logical GLSL450
                                  EntryPoint GLCompute 4  "main" 17
                                  ExecutionMode 4 LocalSize 1 1 1
                                  Source GLSL 450
                                  Name 4  "main"
                                  Name 9  "pixel"
                                  Name 12  "image0"
                                  Name 17  "gl_LocalInvocationID"
                                  Name 27  "image1"
                                  Decorate 12(image0) DescriptorSet 0
                                  Decorate 12(image0) Binding 0
                                  Decorate 12(image0) NonWritable
                                  Decorate 17(gl_LocalInvocationID) BuiltIn LocalInvocationId
                                  Decorate 27(image1) DescriptorSet 0
                                  Decorate 27(image1) Binding 1
                                  Decorate 27(image1) NonReadable
                   2:             TypeVoid
                   3:             TypeFunction 2
                   6:             TypeFloat 32
                   7:             TypeVector 6(float) 4
                   8:             TypePointer Function 7(fvec4)
                  10:             TypeImage 6(float) 2D nonsampled format:Rgba8
                  11:             TypePointer UniformConstant 10
          12(image0):     11(ptr) Variable UniformConstant
                  14:             TypeInt 32 0
                  15:             TypeVector 14(int) 3
                  16:             TypePointer Input 15(ivec3)
    17(gl_LocalInvocationID):     16(ptr) Variable Input
                  19:             TypeInt 32 1
                  20:             TypeVector 19(int) 3
                  22:             TypeVector 19(int) 2
          27(image1):     11(ptr) Variable UniformConstant
             4(main):           2 Function None 3
                   5:             Label
            9(pixel):      8(ptr) Variable Function
                  13:          10 Load 12(image0)
                  18:   15(ivec3) Load 17(gl_LocalInvocationID)
                  21:   20(ivec3) Bitcast 18
                  23:     19(int) CompositeExtract 21 0
                  24:     19(int) CompositeExtract 21 1
                  25:   22(ivec2) CompositeConstruct 23 24
                  26:    7(fvec4) ImageRead 13 25
                                  Store 9(pixel) 26
                  28:          10 Load 27(image1)
                  29:   15(ivec3) Load 17(gl_LocalInvocationID)
                  30:   20(ivec3) Bitcast 29
                  31:     19(int) CompositeExtract 30 0
                  32:     19(int) CompositeExtract 30 1
                  33:   22(ivec2) CompositeConstruct 31 32
                  34:    7(fvec4) Load 9(pixel)
                                  ImageWrite 28 33 34
                                  Return
                                  FunctionEnd
    

    Here is the translation result of translating 1.spv into HLSL shaders. With "--dump-resources" we can see both readonly and writeonly are set on both image0 and image1, and they are all translated into RWTexture2D.

    // spirv-cross.exe 1.spv --dump-resources --hlsl --shader-model 50
    ...
    images
    =============
     ID 012 : image0 (Set : 0) (Binding : 0) writeonly readonly
     ID 027 : image1 (Set : 0) (Binding : 1) writeonly readonly
    =============
    ...
    =============
    RWTexture2D<unorm float4> image0 : register(u0);
    RWTexture2D<unorm float4> image1 : register(u1);
    
    static uint3 gl_LocalInvocationID;
    struct SPIRV_Cross_Input
    {
        uint3 gl_LocalInvocationID : SV_GroupThreadID;
    };
    
    void comp_main()
    {
        float4 pixel = image0[int2(int3(gl_LocalInvocationID).xy)];
        image1[int2(int3(gl_LocalInvocationID).xy)] = pixel;
    }
    
    [numthreads(1, 1, 1)]
    void main(SPIRV_Cross_Input stage_input)
    {
        gl_LocalInvocationID = stage_input.gl_LocalInvocationID;
        comp_main();
    }
    
    question 
    opened by Jiawei-Shao 17
  • HLSL->SPIRV->GLSL gives Messy/unoptimized code

    HLSL->SPIRV->GLSL gives Messy/unoptimized code

    See the original issues: https://github.com/microsoft/DirectXShaderCompiler/issues/2401 https://github.com/microsoft/DirectXShaderCompiler/issues/2402 https://github.com/KhronosGroup/SPIRV-Tools/issues/2816 https://github.com/KhronosGroup/SPIRV-Tools/issues/2817

    in short:

    the code

    VecH4 v;
       v.rgb=Img.Sample(SamplerDefault, tex).rgb*col.rgb;
       v.w=VtxHeightmap;
       return v;
    

    gets unoptimized into:

    vec4 _37;
    
    void main()
    {
        vec3 _17 = texture(Img, IO1).xyz * IO0.xyz;
        vec4 _20 = vec4(_17.x, _17.y, _17.z, _37.w);
        _20.w = VtxHeightmap;
        RT0 = _20;
    }
    

    it reads an invalid value _37.w which could be optimized into: vec4 _20 = vec4(_17.x, _17.y, _17.z, VtxHeightmap); or:

    {
        vec4 _20 = vec4(texture(Img, IO1).xyz * IO0.xyz, VtxHeightmap);
        RT0 = _20;
    }
    

    or RT0 = vec4(texture(Img, IO1).xyz * IO0.xyz, VtxHeightmap);


    AND SIMILAR PROBLEM

    Vec4 Test_VS(VtxInput vtx, out MatrixH3 mtrx:MTRX):SV_Position
    {
       mtrx[0]=vtx.nrm();
       mtrx[1]=vtx.tan();
       mtrx[2]=vtx.bin(0, 0);
       return 0;
    }
    

    -> SPIRV -> GLSL

    mat3 _43;
    
    void main()
    {
        mat3 _45 = _43;
        _45[0] = ATTR2;
        mat3 _46 = _45;
        _46[1] = ATTR3.xyz;
        mediump vec3 _23 = cross(vec3(0.0), vec3(0.0)) * ATTR3.w;
        mat3 _47 = _46;
        _47[2] = vec3(_23.x, _23.y, _23.z);
        gl_Position = vec4(0.0);
        IO0 = _47;
    }
    

    Problem: in HLSL I setup only 1 matrix, on GLSL modifying it results in a new matrix object being created every time. Also it's initialized at the start from an invalid value mat3 _45 = _43;

    opened by GregSlazinski 0
  • MSL: Add support for SPV_KHR_physical_storage_buffer extension.

    MSL: Add support for SPV_KHR_physical_storage_buffer extension.

    • Determine sizing and alignments of pointers to types as distinct from the size and alignment of the types themselves.
    • Declare all buffer pointers in the MSL device address space.
    • Support struct pointer recursion, where structs can contain pointers to themselves or to a parent struct.
    • Add SPIRType::was_forward_referenced to track if a type was forward referenced, to help emit MSL structs in the correct dependency order.
    • Handle pointers to pointers that are not just arrays of arrays.
    • Add unit test shaders.
    opened by billhollings 3
  • [MSL] Translate `SPV_NV_mesh_shader` Mesh shaders to Metal 3 mesh shaders

    [MSL] Translate `SPV_NV_mesh_shader` Mesh shaders to Metal 3 mesh shaders

    SPV_NV_mesh_shader seems to map quite well to Metal 3 mesh shaders and we'd need to be able to translate them for https://github.com/KhronosGroup/MoltenVK/issues/1618.

    I looked into implementing this for naga and got fairly far, but ran into a potential driver issue without much ability to debug it. I'm dumping some thoughts on how mesh shaders could be translated for SPIRV-Cross here.

    Here's a sample mesh shader, adapted from http://zone.dog/braindump/mesh_shaders/:

    #version 450
    #extension GL_NV_mesh_shader : require
    layout(local_size_x=1) in;
    layout(max_vertices=4, max_primitives=2) out;
    layout(triangles) out;
    
    out gl_MeshPerVertexNV {
        vec4 gl_Position;
    } gl_MeshVerticesNV[];
    
    out uint gl_PrimitiveIndicesNV[];
    layout(location = 0) out float vertexColor[];
    
    void main()
    {
        gl_MeshVerticesNV[0].gl_Position = vec4(-1.0, -1.0, 0.0, 1.0); // Upper Left
        gl_MeshVerticesNV[1].gl_Position = vec4( 1.0, -1.0, 0.0, 1.0); // Upper Right
        gl_MeshVerticesNV[2].gl_Position = vec4(-1.0,  1.0, 0.0, 1.0); // Bottom Left
        gl_MeshVerticesNV[3].gl_Position = vec4( 1.0,  1.0, 0.0, 1.0); // Bottom Right
        
        vertexColor[0] = 0.0;
        vertexColor[1] = 0.25;
        vertexColor[2] = 0.5;
        vertexColor[3] = 0.75;
        
        gl_PrimitiveIndicesNV[0] = 0;
        gl_PrimitiveIndicesNV[1] = 1;
        gl_PrimitiveIndicesNV[2] = 2;
        gl_PrimitiveIndicesNV[3] = 2;
        gl_PrimitiveIndicesNV[4] = 1;
        gl_PrimitiveIndicesNV[5] = 3;
        gl_PrimitiveCountNV = 2;
    }
    

    The SPIR-V disassembly for this shader compiled with glslc looks like:

    ; SPIR-V
    ; Version: 1.0
    ; Generator: Google Shaderc over Glslang; 10
    ; Bound: 62
    ; Schema: 0
                   OpCapability MeshShadingNV
                   OpExtension "SPV_NV_mesh_shader"
              %1 = OpExtInstImport "GLSL.std.450"
                   OpMemoryModel Logical GLSL450
                   OpEntryPoint MeshNV %main "main" %gl_MeshVerticesNV %vertexColor %gl_PrimitiveIndicesNV %gl_PrimitiveCountNV
                   OpExecutionMode %main LocalSize 1 1 1
                   OpExecutionMode %main OutputVertices 4
                   OpExecutionMode %main OutputPrimitivesNV 2
                   OpExecutionMode %main OutputTrianglesNV
                   OpSource GLSL 450
                   OpSourceExtension "GL_GOOGLE_cpp_style_line_directive"
                   OpSourceExtension "GL_GOOGLE_include_directive"
                   OpSourceExtension "GL_NV_mesh_shader"
                   OpName %main "main"
                   OpName %gl_MeshPerVertexNV "gl_MeshPerVertexNV"
                   OpMemberName %gl_MeshPerVertexNV 0 "gl_Position"
                   OpName %gl_MeshVerticesNV "gl_MeshVerticesNV"
                   OpName %vertexColor "vertexColor"
                   OpName %gl_PrimitiveIndicesNV "gl_PrimitiveIndicesNV"
                   OpName %gl_PrimitiveCountNV "gl_PrimitiveCountNV"
                   OpMemberDecorate %gl_MeshPerVertexNV 0 BuiltIn Position
                   OpDecorate %gl_MeshPerVertexNV Block
                   OpDecorate %vertexColor Location 0
                   OpDecorate %gl_PrimitiveIndicesNV BuiltIn PrimitiveIndicesNV
                   OpDecorate %gl_PrimitiveCountNV BuiltIn PrimitiveCountNV
                   OpDecorate %gl_WorkGroupSize BuiltIn WorkgroupSize
    ...
    %gl_PrimitiveCountNV = OpVariable %_ptr_Output_uint Output
         %v3uint = OpTypeVector %uint 3
    %gl_WorkGroupSize = OpConstantComposite %v3uint %uint_1 %uint_1 %uint_1
           %main = OpFunction %void None %3
              %5 = OpLabel
    ...
                   OpStore %gl_PrimitiveCountNV %uint_2
                   OpReturn
                   OpFunctionEnd
    

    Metal mesh shaders are marked with [[mesh]] and take a mesh<V, P, NV, NP, T> as input where

    • V is the per-vertex data, in this case
      struct Vertex {
        metal::float4 gl_Position [[position]];
        float vertexColor;
      };
      
    • P is the per-primitive data, which should be void as there is none
    • NV is the number of vertices, which is the OutputVertices OpExecutionMode in SPIR-V (4)
    • NP is the number of primitives, which is the OutputPrimitivesNV OpExecutionMode in SPIR-V (2)
    • T is the topology which is metal::topology::triangle here as marked by the OutputTrianglesNV OpExecutionMode.

    All together you have an input of metal::mesh<Vertex, void, 4, 2, metal::topology::triangle> mesh

    The current output of SPIRV-Cross looks like this: (truncated for bevity)

    #pragma clang diagnostic ignored "-Wmissing-prototypes"
    #pragma clang diagnostic ignored "-Wmissing-braces"
    
    #include <metal_stdlib>
    #include <simd/simd.h>
    
    using namespace metal;
    
    template<typename T, size_t Num>
    struct spvUnsafeArray
    {
    ,,,
    };
    
    constant uint3 gl_WorkGroupSize [[maybe_unused]] = uint3(1u);
    
    struct main0_out
    {
        float vertexColor_0;
        float vertexColor_1;
        float vertexColor_2;
        float vertexColor_3;
        float4 gl_Position;
        float4 gl_Position_1;
        float4 gl_Position_2;
        float4 gl_Position_3;
    };
    
    unknown main0_out main0()
    {
        main0_out out = {};
        spvUnsafeArray<float, 4> vertexColor = {};
        _RESERVED_IDENTIFIER_FIXUP_gl_MeshVerticesNV[0].out.gl_Position = float4(-1.0, -1.0, 0.0, 1.0);
        _RESERVED_IDENTIFIER_FIXUP_gl_MeshVerticesNV[1].out.gl_Position = float4(1.0, -1.0, 0.0, 1.0);
        _RESERVED_IDENTIFIER_FIXUP_gl_MeshVerticesNV[2].out.gl_Position = float4(-1.0, 1.0, 0.0, 1.0);
        _RESERVED_IDENTIFIER_FIXUP_gl_MeshVerticesNV[3].out.gl_Position = float4(1.0, 1.0, 0.0, 1.0);
        vertexColor[0] = 0.0;
        vertexColor[1] = 0.25;
        vertexColor[2] = 0.5;
        vertexColor[3] = 0.75;
        gl_BuiltIn_5276[0] = 0u;
        gl_BuiltIn_5276[1] = 1u;
        gl_BuiltIn_5276[2] = 2u;
        gl_BuiltIn_5276[3] = 2u;
        gl_BuiltIn_5276[4] = 1u;
        gl_BuiltIn_5276[5] = 3u;
        gl_BuiltIn_5275 = 2u;
        out.vertexColor_0 = vertexColor[0];
        out.vertexColor_1 = vertexColor[1];
        out.vertexColor_2 = vertexColor[2];
        out.vertexColor_3 = vertexColor[3];
        return out;
    }
    
    • gl_BuiltIn_5276 maps pretty easily to mesh.set_index.
    • gl_BuiltIn_5275 maps pretty easily to mesh.set_primitive_count (might want to only do this per-workgroup).
    • mesh.set_vertex takes an index and a Vertex struct, which makes mapping the SPIR-V where you set the vertex data values individually a bit tricker as you need to zip them together.

    You probably want to end up with an output that looks like:

    [[mesh]] void main(
      metal::mesh<main_Output, void, 4, 2, metal::topology::triangle> mesh
    ) {
        spvUnsafeArray<float, 4> vertexColor = {};
        spvUnsafeArray<float4, 4> gl_MeshPerVertexNV = {};
        gl_MeshPerVertexNV[0] = float4(-1.0, -1.0, 0.0, 1.0);
        gl_MeshPerVertexNV[1] = float4(1.0, -1.0, 0.0, 1.0);
        gl_MeshPerVertexNV[2] = float4(-1.0, 1.0, 0.0, 1.0);
        gl_MeshPerVertexNV[3] = float4(1.0, 1.0, 0.0, 1.0);
        vertexColor[0] = 0.0;
        vertexColor[1] = 0.25;
        vertexColor[2] = 0.5;
        vertexColor[3] = 0.75;
        mesh.set_index(0, 0u);
        mesh.set_index(1, 1u);
        mesh.set_index(2, 2u);
        mesh.set_index(3, 2u);
        mesh.set_index(4, 1u);
        mesh.set_index(5, 3u);
        mesh.set_primitive_count(2u);
        
        for (uint i = 0; i < 4u; i++) {
            mesh.set_vertex(i, { gl_MeshPerVertexNV[i], vertexColor[i] });
        }
    
        return out;
    }
    
    opened by expenses 0
  • [MSL] SPIRV-Cross emits undefined behaviour for OpShiftLeftLogical

    [MSL] SPIRV-Cross emits undefined behaviour for OpShiftLeftLogical

    Consider the following shader:

    ; SPIR-V
    ; Version: 1.3
    ; Generator: Khronos SPIR-V Tools Assembler; 0
    ; Bound: 15
    ; Schema: 0
                   OpCapability Shader
                   OpMemoryModel Logical GLSL450
                   OpEntryPoint GLCompute %1 "main"
                   OpExecutionMode %1 LocalSize 1 1 1
                   OpDecorate %_struct_2 Block
                   OpDecorate %3 DescriptorSet 0
                   OpDecorate %3 Binding 0
                   OpMemberDecorate %_struct_2 0 Offset 0
           %void = OpTypeVoid
              %5 = OpTypeFunction %void
            %int = OpTypeInt 32 1
      %_struct_2 = OpTypeStruct %int
    %_ptr_StorageBuffer_int = OpTypePointer StorageBuffer %int
    %_ptr_StorageBuffer__struct_2 = OpTypePointer StorageBuffer %_struct_2
          %int_0 = OpConstant %int 0
         %int_n1 = OpConstant %int -1
          %int_2 = OpConstant %int 2
              %3 = OpVariable %_ptr_StorageBuffer__struct_2 StorageBuffer
              %1 = OpFunction %void DontInline %5
             %12 = OpLabel
             %13 = OpAccessChain %_ptr_StorageBuffer_int %3 %int_0
             %14 = OpShiftLeftLogical %int %int_n1 %int_2
                   OpStore %13 %14
                   OpReturn
                   OpFunctionEnd
    

    It is seemingly valid in SPIR-V to apply an OpShiftLeftLogical to a negative base:

    image

    spirv-cross emits the following MSL:

    #include <metal_stdlib>
    #include <simd/simd.h>
    
    using namespace metal;
    
    struct _2
    {
        int _m0;
    };
    
    kernel void main0(device _2& _3 [[buffer(0)]])
    {
        _3._m0 = (-1) << 2;
    }
    

    This is an undefined operation as demonstrated by running the Metal validation layers:

    interesting_shaders/right_shift_UB_msl.metal:13:19: warning: The result of the left shift is undefined because the left operand is
          negative
        _3._m0 = (-1) << 2;
                 ~~~~ ^
    1 warning generated.
    

    This is a fuzzer-found bug using SPIRVSmith

    opened by rayanht 2
  • Vulkan GLSL to MSL, sharing sampler state for split-up combined samplers

    Vulkan GLSL to MSL, sharing sampler state for split-up combined samplers

    Hey, In my workflow the graphics API developer is entirely separate from the GLSL shader writer (who is usually an artist with limited GPU hardware knowledge). They don't want to have to think about hardware limits or sampler states, and so all their shaders just use combined sampler declarations (sampler2D, sampler3D etc). Indeed, the shaders they write are often agnostic to the sampler states, and different usage cases for the same shaders will want different sampler states for the inputs. On Windows this is fine since modern GPUs have large limits for combined samplers. However on MSL where the sampler limit is often much smaller than the texture limit, we'd be able to use more textures if we were able to re-use the samplers between multiple textures. Is this possible right now with SPIRV-cross, or is it always just forced splitting a sampler2D into 1 sampler state and 1 texture, with no ability to share that sampler between other textures? Thanks

    question 
    opened by mbechard 11
  • [MSL] SPIRV-Cross generates undefined behaviour for OpSMod

    [MSL] SPIRV-Cross generates undefined behaviour for OpSMod

    Given the following shader that computes the mod by 32 of a number provided as an input:

    ; Magic:     0x07230203 (SPIR-V)
    ; Version:   0x00010300 (Version: 1.3.0)
    ; Generator: 0x00220001 (SPIRVSmith)
    ; Schema:    0
    OpCapability Shader
    OpMemoryModel Logical GLSL450
    OpEntryPoint GLCompute %main "main"
    OpExecutionMode %main LocalSize 1 1 1
    OpDecorate %struct_t Block
    OpDecorate %struct_variable DescriptorSet 0
    OpDecorate %struct_variable Binding 0
    OpMemberDecorate %struct_t 0 Offset 0
    %void_t = OpTypeVoid
    %main_t = OpTypeFunction %void_t
    %int_t = OpTypeInt 32 1
    %struct_t = OpTypeStruct %int_t
    %int_ptr_t = OpTypePointer StorageBuffer %int_t
    %struct_ptr_t = OpTypePointer StorageBuffer %struct_t
    %const_mod_operand = OpConstant %int_t 32
    %const_zero = OpConstant %int_t 0
    %struct_variable = OpVariable %struct_ptr_t StorageBuffer
    %main = OpFunction %void_t DontInline %main_t
    %main_label = OpLabel
    %struct_element_ptr = OpAccessChain %int_ptr_t %struct_variable %const_zero
    %struct_element = OpLoad %int_t %struct_element_ptr
    %op_mod = OpSMod %int_t %struct_element %const_mod_operand
    OpStore %struct_element_ptr %op_mod
    OpReturn
    OpFunctionEnd
    

    SPIRV-Cross generates the following MSL equivalent:

    #include <metal_stdlib>
    #include <simd/simd.h>
    
    using namespace metal;
    
    struct _2
    {
        int _m0;
    };
    
    kernel void main0(device _2& _3 [[buffer(0)]])
    {
        _3._m0 %= 32;
    }
    

    This is undefined if the input is negative, as specified in MSL Spec Section 3.1.2. Could we have a safe wrapper for OpSMod, similar to the one generated for OpFMod?

    Note that this is an issue even in the constant case where we don't perform the mod on an operand from the input buffer:

    ; Magic:     0x07230203 (SPIR-V)
    ; Version:   0x00010300 (Version: 1.3.0)
    ; Generator: 0x00220001 (SPIRVSmith)
    ; Schema:    0
    OpCapability Shader
    OpMemoryModel Logical GLSL450
    OpEntryPoint GLCompute %main "main"
    OpExecutionMode %main LocalSize 1 1 1
    OpDecorate %struct_t Block
    OpDecorate %struct_variable DescriptorSet 0
    OpDecorate %struct_variable Binding 0
    OpMemberDecorate %struct_t 0 Offset 0
    %void_t = OpTypeVoid
    %main_t = OpTypeFunction %void_t
    %int_t = OpTypeInt 32 1
    %struct_t = OpTypeStruct %int_t
    %int_ptr_t = OpTypePointer StorageBuffer %int_t
    %struct_ptr_t = OpTypePointer StorageBuffer %struct_t
    %const_mod_operand1 = OpConstant %int_t 32
    %const_mod_operand2 = OpConstant %int_t -22
    %const_zero = OpConstant %int_t 0
    %struct_variable = OpVariable %struct_ptr_t StorageBuffer
    %main = OpFunction %void_t DontInline %main_t
    %main_label = OpLabel
    %struct_element_ptr = OpAccessChain %int_ptr_t %struct_variable %const_zero
    %op_mod = OpSMod %int_t %const_mod_operand1 %const_mod_operand2
    OpStore %struct_element_ptr %op_mod
    OpReturn
    OpFunctionEnd
    

    Translates to:

    #include <metal_stdlib>
    #include <simd/simd.h>
    
    using namespace metal;
    
    struct _2
    {
        int _m0;
    };
    
    kernel void main0(device _2& _3 [[buffer(0)]])
    {
        _3._m0 = (-22) % 32;
    }
    

    As far as my understanding of the SPIR-V spec goes, taking the mod of a negative number is legal and doesn't produce UB but I'd be happy if anyone can point to me something that says otherwise.

    needs triage 
    opened by rayanht 0
Releases(MoltenVK-1.1.5)
Owner
The Khronos Group
Connecting Software to Silicon
The Khronos Group
The SPIR-V Tools project provides an API and commands for processing SPIR-V modules.

SPIR-V Tools Overview The SPIR-V Tools project provides an API and commands for processing SPIR-V modules. The project includes an assembler, binary m

The Khronos Group 756 Jun 28, 2022
Shader cross compiler to translate HLSL (Shader Model 4 and 5) to GLSL

XShaderCompiler ("Cross Shader Compiler") Features Cross compiles HLSL shader code (Shader Model 4 and 5) into GLSL Simple to integrate into other pro

Lukas Hermanns 342 Jun 20, 2022
A cross platform shader language with multi-threaded offline compilation or platform shader source code generation

A cross platform shader language with multi-threaded offline compilation or platform shader source code generation. Output json reflection info and c++ header with your shaders structs, fx-like techniques and compile time branch evaluation via (uber-shader) "permutations".

Alex Dixon 273 Jun 10, 2022
Notepad++ is a free source code editor and Notepad replacement that supports several programming languages and natural languages

Npp / Notepad++ is my customized text editor highly enhanced for coding such as insta-run, much more file extensions made self-recognizable, logically colored syntax highlighting for nearly every programming language and designed for very easy customizability -- from the toolbar, context menu, syntax coloring, plug-ins for optional increased capabilities and much more

SkyN9ne 1 Jan 23, 2022
Shader languages support for VS Code

vscode-shader Description Shader languages support for VS Code HLSL - High-Level Shading Language GLSL - OpenGL Shading Language Cg - C for Graphics M

null 139 Jun 1, 2022
Shader Playground is a website for exploring shader compilers.

Shader Playground is a website for exploring shader compilers. Visit website Supported backends Compilers ANGLE Clspv DXC FXC Glslan

Tim Jones 406 Jun 20, 2022
PoC tool to coerce Windows hosts to authenticate to other machines via MS-EFSRPC EfsRpcOpenFileRaw or other functions.

PetitPotam PoC tool to coerce Windows hosts to authenticate to other machines via MS-EFSRPC EfsRpcOpenFileRaw or other functions :) The tools use the

Topotam 1.2k Jun 25, 2022
Jaws is an invisible programming language! Inject invisible code into other languages and files! Created for security research -- see blog post

Jaws is an invisible interpreted programming language that was created for antivirus research. Since Jaws code is composed entirely of whitespace char

C.J. May 205 Jun 13, 2022
⚔️ A tool for cross compiling shaders. Convert between GLSL, HLSL, Metal Shader Language, or older versions of GLSL.

A cross compiler for shader languages. Convert between SPIR-V, GLSL / GLSL ES, HLSL, Metal Shader Language, or older versions of a given language. Cross Shader wraps glslang and SPIRV-Cross, exposing a simpler interface to transpile shaders.

Alain Galvan 157 Jun 23, 2022
Mix C with other programming languages

extern "C" This project demonstrates how to mix C with other programming languages. It aims to create a Rosetta Stone of the procedure shown in the fo

Ralph 12 Mar 25, 2022
🎨 HiColor is a program for converting images to 15- and 16-bit RGB color

HiColor is a program for converting images to 15- and 16-bit RGB color, the color depth of old display modes known as “high color”. In 15-bit mode images have 5 bits for each of red, green, and blue, and the last bit is reserved. In 16-bit mode green, the color the human eye is generally most sensitive to, gets 6 bits.

D. Bohdan 175 Jun 11, 2022
This is a library that can fix the crash on android 5.0 and 5.1 caused by modified utf8 converting.

FixModifiedUtf8ConvertError This is a library that can fix the crash on android 5.0 and 5.1 caused by modified utf8 converting. What's this On Android

Windy 1 Nov 23, 2021
A program for converting instructions from Ben Eater's 8 Bit computer design to binary.

8Bit-Assembler A program for converting instructions from Ben Eater's 8 Bit computer design to binary. We used C because is used for it's mid-level-ne

Marek Borik 2 Nov 21, 2021
LLVM IR and optimizer for shaders, including front-end adapters for GLSL and SPIR-V and back-end adapter for GLSL

Licensing LunarGLASS is available via a three clause BSD-style open source license. Goals The primary goals of the LunarGLASS project are: Reduce the

LunarG, Inc. 151 Jun 18, 2022
yariv.h is a single C/C++ header to encode and decode SPIR-V shaders into a more compressed form I've called YARI-V.

YARI-V yariv.h is a single C/C++ header to encode and decode SPIR-V shaders into a more compressed form I've called YARI-V. YARI-V is an alternative e

Neil Henning 32 May 3, 2022
Khronos-reference front end for GLSL/ESSL, partial front end for HLSL, and a SPIR-V generator.

News Visual Studio 2013 is no longer supported As scheduled, Microsoft Visual Studio 2013 is no longer officially supported. Please upgrade to at leas

The Khronos Group 2.2k Jun 25, 2022
Unix pager (with very rich functionality) designed for work with tables. Designed for PostgreSQL, but MySQL is supported too. Works well with pgcli too. Can be used as CSV or TSV viewer too. It supports searching, selecting rows, columns, or block and export selected area to clipboard.

Unix pager (with very rich functionality) designed for work with tables. Designed for PostgreSQL, but MySQL is supported too. Works well with pgcli too. Can be used as CSV or TSV viewer too. It supports searching, selecting rows, columns, or block and export selected area to clipboard.

Pavel Stehule 1.8k Jun 21, 2022
glslcc: Cross-compiler for GLSL shader language (GLSL->HLSL,METAL,GLES,GLSLv3)

glslcc: Cross-compiler for GLSL shader language (GLSL->HLSL,METAL,GLES,GLSLv3) @septag glslcc is a command line tool that converts GLSL code to HLSL,

Sepehr Taghdisian 405 Jun 23, 2022
Lightweight, cross-platform & full-featured shader IDE

SHADERed is a lightweight tool for writing and debugging shaders. It is easy to use, open source, cross-platform (runs on Windows, Linux & Web).

dfranx 3.5k Jun 24, 2022