GraphicsFuzz provides tools for automatically finding and simplifying bugs in graphics drivers, specifically graphics shader compilers.

Overview

GraphicsFuzz

License Build Status

GraphicsFuzz is a set of tools for testing shader compilers

GraphicsFuzz provides tools for automatically finding and simplifying bugs in graphics drivers, specifically graphics shader compilers. The glsl-fuzz and glsl-reduce tools manipulate GLSL shaders, targeting SPIR-V compilers via translation. The spirv-fuzz and spirv-reduce tools directly manipulate SPIR-V shaders.

Download and run

Follow the gfauto README. The gfauto command line tool is the recommended way of automatically downloading and running our fuzzers to test Vulkan drivers in a "push-button" fashion with minimal interaction. See below if you want to read about individual tools and/or use them as standalone command line tools.

Tool documentation

  • gfauto: the recommended way of automatically downloading and running our fuzzers to test Vulkan drivers in a "push-button" fashion with minimal interaction
  • glsl-fuzz: a family of tools for testing GLSL shader compilers using randomized metamorphic testing
  • glsl-reduce: a stand-alone GLSL shader reducer
  • spirv-fuzz: a stand-alone SPIR-V shader fuzzer and shrinker that uses randomized metamorphic testing
  • spirv-reduce: a stand-alone SPIR-V shader reducer

glsl-fuzz

glsl-reduce

spirv-fuzz

spirv-reduce

Contribute

Further reading

GraphicsFuzz blog posts:

Academic research project blog posts:

Academic publications:

This is not an officially supported Google product.

Issues
  • array index is out of range - Crashed with Family_100_bubblesort_flag

    array index is out of range - Crashed with Family_100_bubblesort_flag

    While we test GraphicsFuzz (v1.3) & some specific test (e.g. Family_100_bubblesort_flag), we saw random crashes (2/10 times).

    Right before Crash, we see array index is out of range (//DEBUG TEST: i=4294947814)

    opened by malayaku 14
  • Investigate stability of mergesort_mosaic shader

    Investigate stability of mergesort_mosaic shader

    Similar to #739, the mergesort_mosaic shader gives pretty different results across platforms. In particular, Swiftshader and Intel Mesa give rather different results.

    @jarikomppa would you be able to take a look and see whether it is possible to fix the trig functions to get something more stable?

    component:samples 
    opened by afd 10
  • How to confirm the golden image for reference shader?

    How to confirm the golden image for reference shader?

    When we test same shader on ARM and Adreno, sometimes reference image of shader is different. Is there any source of shader which gives us golden image too?

    opened by dubeyabhijeet 8
  • Investigate whether binarysearch_tree shader is stable

    Investigate whether binarysearch_tree shader is stable

    binarysearch_tree.zip

    I have been finding that the 'binarysearch_tree' shader that ships with the GraphicsFuzz samples gives significantly different reference images across various devices.

    In particular, I get noticeably different images when running the shader using SwiftShader, vs. using Intel Mesa. (Those would be suitable to investigate this issue, but so would any two Vulkan implementations that give significantly different results.)

    This means that either:

    (1) one of SwiftShader or Intel Mesa has a shader compiler bug or (2) the shader is not numerically stable

    The goal of this issue is to investigate which is the case.

    If it is (1), an issue should be filed against the appropriate open source tool.

    If it is (2), a proposal for how to fix the shader to make it stable should be made.

    I suggest approaching this by manually simplifying the shader so that it renders a successively simpler image, to gain confidence that that image is predictable, but at the same time checking that different images are rendered by SwiftShader and Intel Mesa.

    This will either end up leading to a very small, easy-to-understand shader that exposes a bug in one of the drivers, or will lead to the source of floating-point instability if that is indeed the problem.

    Do:

    runspv host reference.json outdir

    to run the shader, after which outdir/image_0.png will contain the rendered image.

    The attached zip has the reference shader and .json file, plus the two images I am seeing.

    component:samples 
    opened by afd 7
  • Add support to gfauto for generating wrong image CTS patches

    Add support to gfauto for generating wrong image CTS patches

    The gfauto tool, which is currently in this branch (due to be merged soon) supports continuous fuzzing using GraphicsFuzz to find crash bugs, de-duplicate them, and transform unique crash bugs into patches that are nearly ready to be added to CTS (requiring a small amount of manual triage).

    We would like an analogous flow for wrong image bugs, where GraphicsFuzz identifies a pair of equivalent shaders that lead to radically different images being rendered (due to a shader compiler bug). We would like to be able to de-duplicate such issues and, for a unique wrong image bug generate a CTS patch.

    The focus of this issue is on the second thing: generating a CTS patch.

    Specifically, having found a pair of shaders, shader1 and shader2, that are equivalent but are found to expose a wrong image bug, we want to be able to obtain an Amber file that:

    • Creates to graphics pipelines, pipeline1 and pipeline2 that use shader1 and shader 2, respectively
    • Runs both pipelines
    • Asserts that the resulting framebuffers are similar according to a fuzzy diff metric

    Amber provides support for running multiple pipelines, and has a root mean square error diff facility thanks to this PR.

    opened by afd 7
  • ExpressionGenerator: using FactManager to help finding numbers for the addition value

    ExpressionGenerator: using FactManager to help finding numbers for the addition value

    When finding two numbers whose sum is equal to Value, we can involve FactManager to bias choosing at least one number for which we have a known fact.

    E.g., if we have facts about variables or functions that compute 12, 16, 3, 6, 2, and we're trying to generate 100, then it would be better if we can choose 12 + 88, or 94 + 6, or whatever.

    opened by jiradeto 6
  • Linux amber does not get properly killed

    Linux amber does not get properly killed

    We observed several amber commands still running (and eating all memory), although only one should be running at a time. I suspect the python subprocess timeout does not properly kill the amber process in case of timeout.

    We should double check that a real SIGKILL is sent to amber when it times out.

    component:vulkan worker 
    opened by hevrard 6
  • Add experiments directory for coverage experiments

    Add experiments directory for coverage experiments

    This PR adds the experiments directory for conducting coverage experiments. It will be used to keep a track of the coverage experiments done under GSoC 2020

    cla: yes 
    opened by nipunG314 5
  • Update Fuzzer API

    Update Fuzzer API

    Fixes #922. Note that I experienced compilation issues with protobuf library. It looks like protobuf adds debug postfix to its target so that we are getting libprotobufd.a instead of libprotobuf.a. This causes build script to crash saying that the latter target is undefined. To fix use:

    $ mkdir build && cd build
    $ cmake -Dprotobuf_DEBUG_POSTFIX="" ..
    
    cla: yes 
    opened by Vasniktel 5
  • GLFW throws error when start gles-worker desktop on Mac

    GLFW throws error when start gles-worker desktop on Mac

    Environment

    • os.name: Mac OS X
    • os.version: 10.13.4
    • GL_VERSION : 2.1 INTEL-10.32.48
    • GL_SHADING_LANGUAGE_VERSION: 1.20
    • GL_VENDOR: Intel Inc
    • GL_RENDERER: Intel Iris OpenGL Engine
    • Java version: 1.8.0_191
    • Python version: 3.7.2

    Description

    I built the gles-worker and tried to execute with command java -ea -jar gles-worker-desktop.jar. However, the GLFW library failed to create window and throw an error.

    The error log:

    java.lang.ExceptionInInitializerError
    	at org.lwjgl.glfw.GLFW.glfwCreateWindow(GLFW.java:1647)
    	at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.createGlfwWindow(Lwjgl3Application.java:442)
    	at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.createWindow(Lwjgl3Application.java:391)
    	at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.createWindow(Lwjgl3Application.java:379)
    	at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.<init>(Lwjgl3Application.java:108)
    	at com.graphicsfuzz.glesworker.desktop.DesktopLauncher.start(DesktopLauncher.java:263)
    	at com.graphicsfuzz.glesworker.desktop.DesktopLauncher.main(DesktopLauncher.java:164)
    Caused by: java.lang.IllegalStateException: GLFW windows may only be created on the main thread and that thread must be the first thread in the process. Please run the JVM with -XstartOnFirstThread. For offscreen rendering, make sure another window toolkit (e.g. AWT or JavaFX) is initialized before GLFW.
    	at org.lwjgl.glfw.EventLoop$OffScreen.<clinit>(EventLoop.java:37)
    	... 7 more
    
    

    Current Workaround

    • For running via command line, use java -XstartOnFirstThread -ea -jar gles-worker-desktop.jar
    • For running on IntelliJ IDEA, set -XstartOnFirstThread as the VM options for the configuration.
    opened by jiradeto 5
  • Add initial version of libFuzzer custom mutator server

    Add initial version of libFuzzer custom mutator server

    This adds a working server that can accept shaders over TCP from a libFuzzer custom mutator and returns a mutated shader.

    I've confirmed this works by modifying ANGLE's translator fuzzer to use this server instead of using libFuzzer's standard mutations. I still need to figure out how those changes will be upstreamed to ANGLE (I want to do it in a way that doesn't mess things up on ClusterFuzz).

    This implementation may be replaced with something faster (e.g. shared memory).

    Other details of the implementation will probably change, these include:

    • Currently the server listens on port 8666 and can only accept one connection. This doesn't allow parallel use of libFuzzer and other libFuzzer features.
    • Whether we even try to support libFuzzer's custom crossover
    • How we handle sizes on 32 bit platforms. Currently we treat sizes as 64 bits (long) because we use size_t (64 bits on x86_64 but 32 on x86) on the libFuzzer side.
    cla: yes 
    opened by jonathanmetzman 5
  • build(deps): bump gson from 2.8.6 to 2.8.9 in /parent-all

    build(deps): bump gson from 2.8.6 to 2.8.9 in /parent-all

    Bumps gson from 2.8.6 to 2.8.9.

    Release notes

    Sourced from gson's releases.

    Gson 2.8.9

    • Make OSGi bundle's dependency on sun.misc optional (#1993).
    • Deprecate Gson.excluder() exposing internal Excluder class (#1986).
    • Prevent Java deserialization of internal classes (#1991).
    • Improve number strategy implementation (#1987).
    • Fix LongSerializationPolicy null handling being inconsistent with Gson (#1990).
    • Support arbitrary Number implementation for Object and Number deserialization (#1290).
    • Bump proguard-maven-plugin from 2.4.0 to 2.5.1 (#1980).
    • Don't exclude static local classes (#1969).
    • Fix RuntimeTypeAdapterFactory depending on internal Streams class (#1959).
    • Improve Maven build (#1964).
    • Make dependency on java.sql optional (#1707).

    Gson 2.8.8

    • Fixed issue with recursive types (#1390).
    • Better behaviour with Java 9+ and Unsafe if there is a security manager (#1712).
    • EnumTypeAdapter now works better when ProGuard has obfuscated enum fields (#1495).
    Changelog

    Sourced from gson's changelog.

    Version 2.8.9

    • Make OSGi bundle's dependency on sun.misc optional (#1993).
    • Deprecate Gson.excluder() exposing internal Excluder class (#1986).
    • Prevent Java deserialization of internal classes (#1991).
    • Improve number strategy implementation (#1987).
    • Fix LongSerializationPolicy null handling being inconsistent with Gson (#1990).
    • Support arbitrary Number implementation for Object and Number deserialization (#1290).
    • Bump proguard-maven-plugin from 2.4.0 to 2.5.1 (#1980).
    • Don't exclude static local classes (#1969).
    • Fix RuntimeTypeAdapterFactory depending on internal Streams class (#1959).
    • Improve Maven build (#1964).
    • Make dependency on java.sql optional (#1707).

    Version 2.8.8

    • Fixed issue with recursive types (#1390).
    • Better behaviour with Java 9+ and Unsafe if there is a security manager (#1712).
    • EnumTypeAdapter now works better when ProGuard has obfuscated enum fields (#1495).

    Version 2.8.7

    • Fixed ISO8601UtilsTest failing on systems with UTC+X.
    • Improved javadoc for JsonStreamParser.
    • Updated proguard.cfg (#1693).
    • Fixed IllegalStateException in JsonTreeWriter (#1592).
    • Added JsonArray.isEmpty() (#1640).
    • Added new test cases (#1638).
    • Fixed OSGi metadata generation to work on JavaSE < 9 (#1603).
    Commits
    • 6a368d8 [maven-release-plugin] prepare release gson-parent-2.8.9
    • ba96d53 Fix missing bounds checks for JsonTreeReader.getPath() (#2001)
    • ca1df7f #1981: Optional OSGi bundle's dependency on sun.misc package (#1993)
    • c54caf3 Deprecate Gson.excluder() exposing internal Excluder class (#1986)
    • e6fae59 Prevent Java deserialization of internal classes (#1991)
    • bda2e3d Improve number strategy implementation (#1987)
    • cd748df Fix LongSerializationPolicy null handling being inconsistent with Gson (#1990)
    • fe30b85 Support arbitrary Number implementation for Object and Number deserialization...
    • 1cc1627 Fix incorrect feature request template label (#1982)
    • 7b9a283 Bump bnd-maven-plugin from 5.3.0 to 6.0.0 (#1985)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 0
  • There is no problem with testing the Qualcomm chip, whether the mtk chip is compatible

    There is no problem with testing the Qualcomm chip, whether the mtk chip is compatible

    I test Qualcomm chip here, can pass, However, when testing the mtk chip mobile phone, an error occurs, and it keeps prompting CompositionEngine: Fence::merge return error: Inappropriate ioctl for device (-25) Is it compatible to test mtk chip mobile phones?

    opened by yufeilong92 0
  • build(deps-dev): bump ipython from 7.15.0 to 7.16.3 in /gfauto

    build(deps-dev): bump ipython from 7.15.0 to 7.16.3 in /gfauto

    Bumps ipython from 7.15.0 to 7.16.3.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 0
  • glsl-reduce enhancement: based on perses

    glsl-reduce enhancement: based on perses

    I have been experimenting with both glsl-reduce and perses as reducers. It seems that Perses performs more reduction opportunities than glsl-reduce, especially in reducing more aggressively array indexes. Please find below some examples of structures reduced first with glsl-reduce and then with perses.

    • [ ] Reducing ternary operators in array indexes

    glsl-reduce

    void main()
    {
     ext_2[(all(bvec2(true)) ? (+ abs(ext_1[1])) : (ext_0 & -- ext_2[2]))] |= 1;
    }
    

    perses

    void main()
    {
     ext_2[abs(ext_1[1])] |= 1;
    }
    
    • [ ] Removing the outer array in an array index expression

    glsl-reduce

    void main()
    {
     ext_3[(+ + (ext_1 /= (ext_5[2086006349])))] = 1u;
     ++ ext_4[int(var_0[ext_4[(ext_4[ext_4[(ext_4[ext_4[ext_4[1893896488]]])]])]].ttt)];
    }
    

    perses

    void main()
    {
     ext_1/=  ext_5[2086006349];
     ++ ext_4[int(var_0[ext_4[1893896488]])];
    }
    
    • [ ] Reducing complicated swizzles (some can be removed altogether)

    glsl-reduce

    void main()
    {
      ivec4 var_1 = ivec4(- 2147483634);
     (ext_3[ext_7]) = bitfieldExtract(114u, 1, (~ ext_1) / var_1.ttp.yx.x);
    }
    

    perses

    void main()
    {
      ivec4 var_1 = ivec4(- 2147483634);
      ext_3[ext_7]  = bitfieldExtract(114u, 1,  ~ ext_1  / var_1.x);
    }
    
    • [ ] Reducing the number of parameters in type constructors

    glsl-reduce

    void main()
    {
     while(1u + ext_3 <= uint(uvec4(bvec2(false), 1u, true)))
      {
       ext_10 = 1u;
      }
    }
    

    perses

     void main()
    {
     while(1u + ext_3 <= uint(false))
     {
       ext_10 = 1u;
     }
    }
    
    opened by AaronGhost 2
  • DonateCodeTransformation might move injectionSwitch up above the precision declaration, causing shader to fail validation

    DonateCodeTransformation might move injectionSwitch up above the precision declaration, causing shader to fail validation

    https://github.com/google/graphicsfuzz/blob/de47649bced75c1d4c0980d1033742a0f9d1de88/generator/src/main/java/com/graphicsfuzz/generator/transformation/DonateCodeTransformation.java#L447

    opened by paulthomson 0
Releases(v1.3)
Owner
Google
Google ❤️ Open Source
Google
An interpreter for finding subtle bugs in programs written in standard C

tis-interpreter This is tis-interpreter, an interpreter of C for detecting undefined behavior. tis-interpreter detects subtle bugs in C programs that

TrustInSoft 527 Jun 22, 2022
Shader cross compiler to translate HLSL (Shader Model 4 and 5) to GLSL

XShaderCompiler ("Cross Shader Compiler") Features Cross compiles HLSL shader code (Shader Model 4 and 5) into GLSL Simple to integrate into other pro

Lukas Hermanns 342 Jun 20, 2022
A cross platform shader language with multi-threaded offline compilation or platform shader source code generation

A cross platform shader language with multi-threaded offline compilation or platform shader source code generation. Output json reflection info and c++ header with your shaders structs, fx-like techniques and compile time branch evaluation via (uber-shader) "permutations".

Alex Dixon 273 Jun 10, 2022
collection of C/C++ programs that try to get compilers to exploit undefined behavior

------------------------------------------------------------------------------- UB Canaries: A collection of C/C++ programs that detect undefined beh

John Regehr 160 Jun 27, 2022
Comprehensive Hashing Library for C++ Compilers.

HashLib4CPP HashLib4CPP is a C++11 library that provides an easy to use interface for computing hashes and checksums of strings, files, streams, bytea

Telepati 24 Jan 2, 2022
Set of examples how to use CLion with various unsupported compilers using Custom Defined Compiler feature

About This repository contains: Set of examples how to use CLion with various unsupported compilers using Custom Defined Compiler feature Public set o

JetBrains 30 Jun 15, 2022
Not related to software bugs and exploits; this repo contains snippets of code that demonstrate some interesting functionality or a handy trick.

Proof-of-Concept Not related to software bugs and exploits; this repo contains snippets of code that demonstrate some interesting functionality or a h

Alisa Esage 31 May 29, 2022
OpenDCDiag is an open-source project designed to identify defects and bugs in CPUs.

OpenDCDiag is an open-source project designed to identify defects and bugs in CPUs. It consists of a set of tests built around a sophisticated CPU testing framework. OpenDCDiag is primarily intended for, but not limited to, Data Center CPUs.

OpenDCDiag 21 Jun 20, 2022
Vulkan and other GPU API bugs I found.

GPU-my-list-of-bugs what is it - list of bugs I found writing shaders, mostly shader bugs. Maybe this is my code bug or/and shader bugs, but this code

Danil 7 Mar 31, 2022
BokutachiHook - Hook for Lunatic Rave 2 to parse score data and send it to an HTTP server, made specifically for Bokutachi IR.

BokutachiHook Hook for Lunatic Rave 2 to parse score data and send it to an HTTP server, made specifically for Bokutachi IR (https://bokutachi.xyz). T

null 5 May 16, 2022
PoC that fixes two GTA Online bugs and drastically improves load times for CPU-bound systems

Project status Officially fixed by R* 2021-03-16 :) PoC that fixes two GTA Online bugs and drastically improves load times for CPU-bound systems All a

null 2.8k Jun 23, 2022
Arduino code to interface with quadrature-encoder mice, specifically the Depraz mouse

Depraz Mice on USB via Arduino This code lets you connect a Depraz mouse to a modern computer via USB. The Depraz mouse has a male DE-9 connector but

John Floren 4 Jan 7, 2022
A subset of WidgetsFlutterBinding specifically for initializing the ServicesBinding.

flutter_services_binding A subset of WidgetsFlutterBinding specifically for initializing the ServicesBinding. When executing runApp within a custom Zo

Felix Angelov 28 Apr 4, 2022
Type safe - Zero overhead utilities for preventing bugs at compile time

type_safe type_safe provides zero overhead abstractions that use the C++ type system to prevent bugs. Zero overhead abstractions here and in following

Jonathan Müller 1.1k Jun 24, 2022
A collection of tools, libraries, and tests for Vulkan shader compilation.

Shaderc A collection of tools, libraries and tests for shader compilation. At the moment it includes: glslc, a command line compiler for GLSL/HLSL to

Google 1.3k Jun 22, 2022
Lee Thomason 290 Jun 9, 2022
A biome finder adapted from cubiomes intended for structure finding in Minecraft Bedrock 1.7

A biome finder adapted from cubiomes intended for structure finding in Minecraft Bedrock 1.7. This will work from 1.7 all the way up to 1.14.

Luke7720 2 Jun 11, 2022
The SPIR-V Tools project provides an API and commands for processing SPIR-V modules.

SPIR-V Tools Overview The SPIR-V Tools project provides an API and commands for processing SPIR-V modules. The project includes an assembler, binary m

The Khronos Group 756 Jun 28, 2022
Suckless-tools - My fork of suckless tools.

suckless-tools Here is my fork of suckless tools. I didn't include tabbed, i was using but not actively. I am using xfce4-terminal instead of st. Beca

null 2 Jan 7, 2022