WebGL Texture plugin for Flutter

Overview

FlutterWebGL [Pub Version][Pub]

So far there is no way to render 3D objects efficiently in Flutter. Also directly accessing and programming the GPU from Dart isn't supported yet. This plugin shall close this gap.

Originally @kentcb and I were working on a wrapper for OpenGL for a project for the company Aaronia. Although we got pretty far with that one looking at the fact the Apple deprecated OpenGL I wondered if this still is the right approach.

Luckily Simon made me aware that there is the Angle project from Google that implements an OpenGL ES API for all platforms that don't support them

This project will solve two goals:

  • Offer a low-level Dart FFI layer to render into a Flutter Texture Widget with the OpenGL ES 3.0 API
  • Realise a Dart Version of the WebGL interface which covers most of the functionality of OpenGL ES, but without the hassle to deal with FFI NativeTypes and proper error handling.

On top of that other packages can build easier to use 3D APIs, e.g. there is a Dart port of the JS 3D framework three.js

How does this work:

Flutter has a Texture widget which is a placeholder for graphic content that is rendered onto a native Texture object. This works by creating a Texture object in the native part of this plugin and registering this texture with the Flutter Engine. The Engine returns an id which can then be used in Texture widgets in Flutter. Flutter will then display the current content of that texture. If the content of this texture was changed by native code we have to inform the Flutter Engine, so that it can update the linked Texture Widgets.

This plugin uses OpenGL ES to render to that native textures. Because of the way OpenGL works we can access the OpenGL "render buffer object" (RBO) from the native part of the plugin as well over the Dart-FFI bindings to the OpenGL ES API so that we can write all rendering code (with the exception of Shaders) in Dart.

To use this you have to do:

  1. Call FlutterWebGL.initOpenGL() which will setup all the needed infrastructure on the Dart and native side of the plugin
  2. Create a Texture with FlutterWebGL.createTexture which will return you the textureID to use it in the TextureWidget
  3. Activate the Texture to access it with OpenGL commands (this part isn't done yet, so far only one texture is supported).
  4. Render to this texture with OpenGL ES low level or WebGL commands
  5. Inform the Flutter Engine that new content is available on that texture by calling FlutterWebGL.updateTexture.

Check out the example to see how it works.

So far the handling of rendering is intentionally not connected to any special widget, but kept separate. This will make it possible to update the texture content even from a separate Isolate which opens a lot of possibilities.

Roadmap

Luckily Aaronia supports this project by that I can work part of the time I do for them on this project.

As there is a lot to do, and it needs knowledge of the tool chains for the different platforms it would be amazing if you could join this project.

Currently, I see the following tasks that need to be solved.

  • Build the Angle Framework for

    • Windows
    • Linux,(not necessary until the Flutter Engine supports native Textures on Linux, see this stale PR )
    • MacOS
    • iOS (it seems that the main Angle project hasn't caught up with its fork metalangle so we probably should use this one)
  • Create a Dart-FFI layer for OpenGL ES 3.0 API and EGL 1.5

  • Implement native plugin parts that register and update the textures

    • Windows
    • Android
    • iOS
    • macOS
    • Linux (see above)
    • Web??? I have no idea how, but maybe someone else does :-)
  • Implement WebGL in Dart

  • add a way to render text. My current idea would be to render text using Flutter onto a canvas and pass that down to OpenGL so that it can be used as a texture on Polygons.

  • design widgets that let you use WebGL without needing to care about Texture allocation and notifying

  • write examples and documentation

resources

https://chromium.googlesource.com/angle/angle https://github.com/kakashidinho/metalangle https://www.khronos.org/registry/OpenGL/specs/es/3.2/es_spec_3.2.pdf https://www.khronos.org/registry/EGL/specs/eglspec.1.5.pdf

https://learnopengl.com/Getting-started/Hello-Triangle

Issues
  • android support

    android support

    Hi now this plugin create EGLContext use dart ffi then pass (int pointer ?) to native convert back to Native EGLContext Object

    but on Android i can't find method to convert EGLContext int back to Android Native EGLContext use java or kotlin

    public class EGLContext extends EGLObjectHandle {
        EGLContext(long handle) {
            super(0);
            throw new RuntimeException("Stub!");
        }
    
        public boolean equals(Object o) {
            throw new RuntimeException("Stub!");
        }
    }
    

    maybe we need create EGLContext on Native side then pass to dart

    opened by wasabia 2
  • Linux support

    Linux support

    External texture support finally landed. https://github.com/flutter/engine/pull/24916

    Likely will be available in master / dev channel in the next few days.

    opened by TimWhiting 1
  • I get errors in console when I try to run the app on android emulator

    I get errors in console when I try to run the app on android emulator

    Hi when I try to run my app with the package I get this error

    PS G:\dev\george35mk\Flutter\game_engine2> flutter run
    Using hardware rendering with device sdk gphone x86 arm. If you notice graphics artifacts, consider enabling software rendering with
    "--enable-software-rendering".
    Launching lib\main.dart on sdk gphone x86 arm in debug mode...
    /G:/Programs/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.2.0-nullsafety.1/lib/src/utf8.dart:68:33: Error: The getter 'addressOf' isn't defined for the class 'Utf8'.
     - 'Utf8' is from 'package:ffi/src/utf8.dart' ('/G:/Programs/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.2.0-nullsafety.1/lib/src/utf8.dart').
    Try correcting the name to the name of an existing getter, or defining a getter or field named 'addressOf'.
      String toString() => fromUtf8(addressOf);
                                    ^^^^^^^^^
    Running Gradle task 'assembleDebug'...   
    

    OS: Windows 10

    flutter:

    Flutter 2.3.0-24.0.pre • channel dev • https://github.com/flutter/flutter.git
    Framework • revision 2b9537c783 (3 months ago) • 2021-06-11 22:11:53 -0700
    Engine • revision ddbac024d1
    Tools • Dart 2.14.0 (build 2.14.0-188.0.dev)
    

    I run the exact code you have provided in the pub.dev https://pub.dev/packages/flutter_web_gl/example

    opened by George35mk 1
  • let ios and Mac implementations return the EGLConfigId

    let ios and Mac implementations return the EGLConfigId

    Windows and Android return the EGL-Config that they have chosen via an config-id that can be retrieved like:

          EGLint configId;
          eglGetConfigAttrib(display,config,EGL_CONFIG_ID,&configId);
    

    and is returned as result in

          auto response = flutter::EncodableValue(flutter::EncodableMap{
              {flutter::EncodableValue("context"),
               flutter::EncodableValue((int64_t) context)},
              {flutter::EncodableValue("dummySurface"),
               flutter::EncodableValue((int64_t) dummySurfaceForDartSide)},
              {flutter::EncodableValue("eglConfigId"),
               flutter::EncodableValue((int64_t) configId)}
              });
          result->Success(response);
    

    This has to be added to the iOS and MacOs implemtations. When this is done we can remove the default config from the dart side.

    iOS MacOs todo 
    opened by escamoteur 0
  • Consider making all gl calls plain proxies w/o additional functionality

    Consider making all gl calls plain proxies w/o additional functionality

    example:

    void compileShader(Shader shader, [bool checkForErrors = true]) {
        gl.glCompileShader(shader.shaderId);
    
        if (checkForErrors) {
          final compiled = tempInt32s[0];
          gl.glGetShaderiv(shader.shaderId, GL_COMPILE_STATUS, compiled);
          if (compiled.value == 0) {
            final infoLen = tempInt32s[1];
    
            gl.glGetShaderiv(shader.shaderId, GL_INFO_LOG_LENGTH, infoLen);
    
            String message = '';
            if (infoLen.value > 1) {
              final infoLog = calloc<Int8>(infoLen.value);
    
              gl.glGetShaderInfoLog(shader.shaderId, infoLen.value, nullptr, infoLog);
              message = "\nError compiling shader:\n${infoLog.cast<Utf8>().toDartString()}";
    
              calloc.free(infoLog);
            }
            throw OpenGLException(message, 0);
          }
        }
      }
    

    gl.compileShader on the web doesn't have anything related to errors, so this would be a misalignment with browser's WebGL. My suggestions is to make sure gl api is exactly the same as it is on the web

    opened by lesnitsky 1
GPU Texture Generator

Imogen GPU/CPU Texture Generator GPU Texture generator using dear imgui for UI. Not production ready and a bit messy but really fun to code. This is a

Cedric Guillemet 703 Jun 17, 2022
DirectXTex texture processing library

DirectXTex texture processing library http://go.microsoft.com/fwlink/?LinkId=248926 Copyright (c) Microsoft Corporation. November 8, 2021 This package

Microsoft 1.3k Jun 23, 2022
Basic framework for D3D11 init, model/texture loading, shader compilation and camera movement.

reed-framework Basic framework for D3D11 init, model/texture loading, camera movement, etc. Instructions: #include <framework.h> Link with framework.l

Nathan Reed 34 May 18, 2022
Advanced DXTc texture compression and transcoding library

crunch/crnlib v1.04 - Advanced DXTn texture compression library Copyright (C) 2010-2017 Richard Geldreich, Jr. and Binomial LLC http://binomial.info F

Unity Technologies 204 Jun 23, 2022
A simple animation challenge made with Flutter

Slide Show - Flutter Getting Started This project is a starting point for a Flutter application. A few resources to get you started if this is your fi

Pedro Massango 24 Dec 2, 2021
[WIP] A media playback library for Dart & Flutter apps on Windows & Linux. Based on libVLC & libVLC++.

dart_vlc Bringing power of VLC to Flutter & Dart apps on Windows & Linux Installation dependencies: ... dart_vlc: ^0.0.1 Documentation Create a ne

Hitesh Kumar Saini 335 Jun 29, 2022
Flutter package that lets you simply animate a widget into a visible state.

Entry This Flutter package introduces a new widget : Entry. It lets you simply animate a widget into a visible state. Don't just display widgets : mak

Mickaël Hernandez 22 Apr 13, 2022
Utility on top of the Flutter Driver API that facilitates measuring the performance of your app in an automated way created by Very Good Ventures 🦄

Very Good Performance Developed with ?? by Very Good Ventures ?? Utility on top of the Flutter Driver API that facilitates measuring the performance o

Very Good Open Source 72 Jun 24, 2022
Plugin implementing Kirchhoff rods to simulate any 1D flexible structure

BeamAdapter plugin This SOFA plugin implements a 1-dimensional Finite Element Method (FEM) in the context of coil embolization in neurology. The metho

SOFA Framework 6 Jan 30, 2022
A flutter plugin that runs a wayland compositor to stream textures onto flutter, allows creation of wayland based shells using flutter

flcompositor A new flutter plugin project. Getting Started This project is a starting point for a Flutter plug-in package, a specialized package that

Davide Bianco 3 Jan 10, 2022
Minimalistic C++/Python GUI library for OpenGL, GLES2/3, Metal, and WebAssembly/WebGL

NanoGUI NanoGUI is a minimalistic cross-platform widget library for OpenGL 3+, GLES 2/3, and Metal. It supports automatic layout generation, stateful

Mitsuba Physically Based Renderer 1k Jun 27, 2022
Twitter thread explorer made in C++/Wasm/WebGL

Visa Viz Twitter thread explorer made in C++/Wasm/WebGL Once upon a time, I stumbled upon @visakanv's network of threads, but it was too cumbersome to

Ilia Demianenko 10 Feb 13, 2022
Deno gl - WIP Low-level OpenGL (GLFW) bindings and WebGL API implementation for Deno.

deno_gl WIP Low-level OpenGL (GLFW) bindings and WebGL API implementation for Deno. Building Make dist directory if it doesn't exist. Build gl helper

DjDeveloper 14 Jun 11, 2022
Minimal C# WebAssembly WebGL Example

Minimal C# WebAssembly WebGL Example a minimal example of C# wasm WebGL app based on dotnet/runtimelab/NativeAOT-LLVM WebAssembly This example is port

Zou Wei 8 Jun 13, 2022
A Flutter package that makes it easy to customize and work with your Flutter desktop app's system tray.

system_tray A Flutter package that that enables support for system tray menu for desktop flutter apps. on Windows, macOS and Linux. Features: - Modify

AnTler 103 Jun 24, 2022
A flutter package for making game levels map like candy crush or similar games using flutter with ability to be horizontal or vertical

Game Levels Scrolling Map A package for making game levels map like candy crush or similar games using flutter with ability to be horizontal or vertic

Mohamed Sayed 9 Jun 7, 2022
Flutter-Clock-and-Reminder-App - a highly functional clock and reminder app developed on flutter framework.

clock_app A new Flutter project. Getting Started This project is a starting point for a Flutter application. A few resources to get you started if thi

Umar Baloch 5 Dec 27, 2021
A cross-platform (Android/iOS/Windows/macOS) cronet plugin for Flutter via `dart:ffi`

cronet_flutter A cross-platform (Android/iOS/Windows/macOS) cronet plugin for Flutter via dart:ffi

null 20 Jun 15, 2022
A Flutter Web Plugin to display Text Widget as Html for SEO purpose

SEO Renderer A flutter plugin (under development) to render text widgets as html elements for SEO purpose. Created specifically for issue https://gith

Sahdeep Singh 88 Jun 17, 2022
A new JS script plugin for flutter

js_script Run JS script. Usage // Create a JS context. JsScript script = JsScript(); // Define a class. var classInfo = ClassInfo<TestClass>( newI

null 5 Mar 8, 2022
This plugin allows Flutter desktop apps to defines system tray.

tray_manager This plugin allows Flutter desktop apps to defines system tray. tray_manager Platform Support Quick Start Installation ⚠️ Linux requireme

LeanFlutter 79 Jun 25, 2022
A cross-platform flutter plugin for C/C++/ObjC crash report via Google Breakpad

quick_breakpad A cross-platform flutter plugin for C/C++/ObjC crash report via Google Breakpad Use breakpad for quick_breakpad_example $CLI_BREAKPAD i

Woodemi Co., Ltd 15 Apr 26, 2022
Flutter plugin serving utilities related to Windows taskbar. 💙

windows_taskbar Flutter plugin serving utilities related to Windows taskbar ?? Install dependencies: windows_taskbar: ^0.0.1 Demo Checkout the exam

Hitesh Kumar Saini 83 Jun 25, 2022
This plugin allows Flutter desktop apps to Auto launch on startup / login.

This plugin allows Flutter desktop apps to Auto launch on startup / login.

LeanFlutter 28 Jun 18, 2022
This plugin allows Flutter desktop apps to capture screenshots.

screen_capturer This plugin allows Flutter desktop apps to capture screenshots. screen_capturer Platform Support Quick Start Installation Usage Licens

LeanFlutter 35 Jun 21, 2022
This plugin allows Flutter desktop apps to defines system/inapp wide hotkey (i.e. shortcut).

hotkey_manager This plugin allows Flutter desktop apps to defines system/inapp wide hotkey (i.e. shortcut). hotkey_manager Platform Support Quick Star

LeanFlutter 61 Jun 16, 2022
RDO BC1-7 GPU texture encoders

bc7enc - Fast BC1-7 GPU texture encoders with Rate Distortion Optimization (RDO) This repo contains fast texture encoders for BC1-7. All formats suppo

Rich Geldreich 88 Jun 24, 2022
Stack-based texture generation tool written in C99!

Stack-based texture generation tool written in C99! Brought to you by @zaklaus and contributors Introduction zpl.texed is a cross-platform stack-based

zpl | pushing the boundaries of simplicity. 17 May 1, 2022
NVIDIA Texture Tools samples for compression, image processing, and decompression.

NVTT 3 Samples This repository contains a number of samples showing how to use NVTT 3, a GPU-accelerated texture compression and image processing libr

NVIDIA DesignWorks Samples 31 Jun 13, 2022