StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR!

Overview

StereoKit Logo

StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR! Inspired by libraries like XNA and Processing, StereoKit is meant to be fun to use and easy to develop with, yet still quite capable of creating professional and business ready software.

The getting started guide can be found here!

Interested in news and updates about StereoKit? Maybe just looking for some extra help?

Screenshot

StereoKit Features:

  • Platforms: HoloLens 2, Oculus Quest, Windows Mixed Reality, Oculus Desktop, SteamVR, Monado Linux, and eventually everywhere OpenXR is!
  • Flat screen mode with input emulation for easy development
  • Builds your application to device in seconds, not minutes
  • Mixed Reality inputs like hands and eyes are trivial to access
  • Easy and powerful UI and interactions
  • Model formats: .gltf, .glb, .fbx(partial), .obj, .stl, procedural
  • Texture formats: .jpg, .png, .tga, .bmp, .psd, .gif, .hdr, .pic, equirectangular cubemap, procedural
  • Runtime asset loading
  • Physics
  • Performance-by-default instanced render pipeline
  • Flexible shader/material system with built-in PBR
  • Documentation is generated directly from the source code, including screenshots

Getting started

Follow this guide for a detailed introduction! This repository is the raw source for those who wish to build StereoKit themselves, the Visul Studio templates and the NuGet packages referenced in the guide are how most people should build their applications!

StereoKit focuses on getting you productive with the least amount of code possible. You can actually do most tasks with a single line of code, including UI! Here's hello world with StereoKit, this is all you need to get up and running!

{ helmet.Draw(Matrix.TS(Vec3.Zero, 0.1f)); })); SK.Shutdown(); } } ">
using StereoKit;

class Program
{
	static void Main(string[] args)
	{
		SK.Initialize(new SKSettings{ appName = "Project" });

		Model helmet = Model.FromFile("Assets/DamagedHelmet.gltf");

		while (SK.Step(() => {
			helmet.Draw(Matrix.TS(Vec3.Zero, 0.1f));
		}));

		SK.Shutdown();
	}
}

Hello World

Roadmap

Where is StereoKit going next? That depends on you! What do you need? Is StereoKit missing something that it really should have? File an issue and let us know!

In the short term, StereoKit will focus on the Core API, improving performance, and ensuring all basic features are present and work well! Some basic tooling, things like a visual shader editor might get built here. The Core of StereoKit is an Immediate Mode system that does not provide any framework or application state management.

Long term, we'd love to add StereoKit Framework! Framework is a higher-level layer that manages application state, so more complicated functionality can be taken care of. Features such as automatic multi-user capabilities, component systems, WYSIWYG UI design tools, visual scene editors and code-free design environments will become possible.

Dependencies

Just like all software, StereoKit is built on the shoulders of incredible people! Here's a list of the libraries StereoKit uses to get things done.

And some of my own libraries that I maintain separately from this repository.

Comments
  • Cannot build out of the box latest 0.3.6.

    Cannot build out of the box latest 0.3.6.

    Description

    I like to test an "out of the box" build before adding any changes to my projects. This one does not work, and I could not find the reasons why. Things done in the past when it would not build OOB did not work this time. I spent several hours trying to work out the issues with the build.

    Trying to build out of the box following the instructions. There is a problem with the OpenXR version 1.0.22. The first error is a pathspec xxxxxxxx is not a known path in git. It also complains that the CMakelists.txt file is missing from the tools folder. (copying the file there does not fix the issue) After that error I get a collection of errors. Interestingly I had several problems with the shaderxxxxxx.h files having syntax errors. I tried downloading new clones and starting over from the beginning and the syntax errors seemed to move around, like the decompression was corrupt. There would be sequences like this in the files: ... 11, 116, , 23, ... and ... 11, 116, }; , 45, 123, ...

    In any case I was not able to build successfully.

    Platform / Environment

    Windows 11 with Visual Studio 2022 both latest version. I am currently using SK 0.3.5 with no issues.

    Logs or exception details

    Attached the error log and the build raw output.
    
    _If you have an exception, details about that would also be essential._
    [SK0-3-6_build errors list.zip](https://github.com/StereoKit/StereoKit/files/8883970/SK0-3-6_build.errors.list.zip)
    
    
    opened by laultman 18
  • Remote rendering on PC with StereoKit

    Remote rendering on PC with StereoKit

    Hi Nick,

    Dose StereoKit support remote rendering on PC? I am wondering if I can use a PC to render the frames and send them back to HoloLens. It's kind of like Holographic remoting.

    Thanks a lot.

    opened by xiaolongrenlzj 17
  • BVH for ray-triangle intersection tests

    BVH for ray-triangle intersection tests

    Here's an initial implementation (C++ parts only, and 3 lines of C#) of using a bounding-volume hierarchy for ray-triangle intersection testing. This is based on existing BVH code I had laying around, plus some inspiration from Jacco Bikker's blog series on BVH construction (see bvh.cpp comments). It contains a fair amount of tracing printf()s, as a code such as this is challenging to debug otherwise, so for now those are best to leave in (I'm sure we'll find bugs ;-)).

    I also added a small standalone C++ test for checking with different models and rays, but it's mostly for debugging right now. The next step would be to add the C# layer for easier testing, i.e. interactive ray casting in 3D, possibly as part of StereoKitTest.

    This has only been tested with a few 3D models, but ranging from small ones to complex ones (up to 500k triangles, and even a 24M triangle model). Construction seems fairly quick, given that it's not very optimized (models of up to 50k triangles in less than 50ms). ~~Construction has had more testing than intersection testing, as the latter can be better done interactively (as mentioned above).~~

    ~~One thing I haven't figured out is why the added model_ray_intersect_bvh() does not return the same results as using mesh_ray_intersect() on each of the submeshes and taking the closest hit. In most models I tested the subparts do not have a non-identity transform, so that should not influence the outcome, yet it does.~~

    opened by paulmelis 16
  • Use correct LUID when creating OpenXR session

    Use correct LUID when creating OpenXR session

    I tried StereoKit with the Oculus runtime and ran into the issue at openxr.cpp#L165 because the laptop I was using has Optimus and creates an integrated rather than discrete graphics device.

    bug 
    opened by gigadude 16
  • App on HoloLens 2 crashes with

    App on HoloLens 2 crashes with "[SK error] sk_gpu: CreateBuffer failed!"

    Description

    I start my UWP app on HoloLens 2. The app has one windows with text and one with an image looking like this: image I picked the Windows with gestures to reposition. However, sometimes the error also happens without gestures. After some time (30 - 90sec) the error appears

    Platform / Environment

    UWP on HoloLens 2

    Logs or exception details

    2022-02-10 01:03:23.345 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  PelvisSiLiveApplication(): created.
    2022-02-10 01:03:23.755 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Using audio backend: ISAC
    2022-02-10 01:03:23.836 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Initialization successful
    2022-02-10 01:03:23.885 | FluoroImageAcquisitionServer | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionServer(): created.
    2022-02-10 01:03:23.885 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionManager(): created.
    2022-02-10 01:03:23.925 | RestServer[0.0.0.0:9005][/ImageAcquisition] | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Start(): started listening Listener[http://+:9005/ImageAcquisition/]
    2022-02-10 01:03:23.925 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
    2022-02-10 01:03:23.925 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
    2022-02-10 01:03:23.925 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
    2022-02-10 01:06:31.925 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  PelvisSiLiveMrSystem(): created.
    2022-02-10 01:06:32.388 | AppSettings          | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  AppSettings.ReadString(): PelvisSiLiveMrConfigurationPath[PelvisSiLiveMr.xml]
    2022-02-10 01:06:32.405 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  LoadConfiguration(): Data loaded successfully
    Scene[LogWindow[Enabled[True] Pose[-1, 0.5, -1, 1, 1, 5] Size[100, 0] FontSize[] FontSize[0.012]] XrayImage[Enabled[True] Pose[1, 0.5, -1, -1, 1, 5] Size[100, 0]]]
    ProNavigation[NavigationServerUri[http://0.0.0.0:9005/ProNavigation]]
    FluoroImageAcquisition[ImageAcquisitionServerUri[http://0.0.0.0:9005/ImageAcquisition]]
    
    2022-02-10 01:06:32.405 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  PelvisSiLiveApplication(): created.
    2022-02-10 01:06:32.845 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Using audio backend: ISAC
    2022-02-10 01:06:32.889 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Initialization successful
    2022-02-10 01:06:32.917 | FluoroImageAcquisitionServer | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionServer(): created.
    2022-02-10 01:06:32.917 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionManager(): created.
    2022-02-10 01:06:32.949 | RestServer[0.0.0.0:9005][/ImageAcquisition] | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Start(): started listening Listener[http://+:9005/ImageAcquisition/]
    2022-02-10 01:06:32.949 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
    2022-02-10 01:06:32.949 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
    2022-02-10 01:06:32.949 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
    2022-02-10 01:07:54.697 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:54.702 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    

    The error keeps looping with some additions:

    2022-02-10 01:07:58.906 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.906 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.977 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] xrWaitFrame [XR_ERROR_OUT_OF_MEMORY]
    2022-02-10 01:07:58.977 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.977 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.002 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.002 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.005 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.005 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.017 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] xrEndFrame [XR_ERROR_RUNTIME_FAILURE]
    2022-02-10 01:07:59.017 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.017 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] xrBeginFrame [XR_ERROR_RUNTIME_FAILURE]
    2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
    

    The app finally "crashes". However, HoloLens 2 does not create a crashdump even if the app is marked to create one.

    opened by llfab 12
  • [Docs-Sprite.FromFile] Is it true, Atlased sprites are not implemented?

    [Docs-Sprite.FromFile] Is it true, Atlased sprites are not implemented?

    Getting this when trying to use an Atlas file. [SK diagnostic] sprite_create: Atlased sprites not implemented yet! Switching to single. [SK warning] A GPU asset is blocking its thread until the main thread is available, has async code accidentally shifted execution to a different thread since SK.Initialize?

    I've spent days building all the supporting structures for atlased handling and get down to this line straight out of the documentation, Note: fi is a FileInfo object val.Key is a string with the value "00001" where this points to the atlas image 00001.png.

    var sprite = Sprite.FromFile(fi.FullName, SpriteType.Atlased, val.Key);

    It is in fact hanging the thread. The code never returns after the call to the method.

    There are some other issues around this call. If the file name is invalid a hard exception is thrown somewhere down low in the C++ that is not trappable by the C#.

    opened by laultman 11
  • Unable to run example application inside an OpenXR overlay using XR_EXTX_overlay API layer

    Unable to run example application inside an OpenXR overlay using XR_EXTX_overlay API layer

    Description

    What's happening? What are you expecting? More details are better.

    I'm trying to get the sample StereoKit project to work inside an OpenXR overlay with the XR_EXTX_overlay API layer. When I try to run the project with SKSettings.overlayApp set to true, I'm getting an error that says:

    [SK info] Couldn't create an OpenXR session, no MR device attached/ready? [XR_ERROR_INITIALIZATION_FAILED]
    

    And the application then runs in flatscreen mode.

    SystemInfo.overlayApp is returning True.

    Platform / Environment

    • Windows/Linux/Android?
      • Windows.
    • Which OpenXR runtime, or is this the flatscreen simulator?
      • SteamVR, although logs suggest that Vive OpenXR: Vive SRanipal runtime is used even when SteamVR runtime is explicitly specified with the XR_RUNTIME_JSON environment variable.
    • What XR device is involved?
      • HTC Vive Pro 2.
    • C# or C++?
      • C#.
    • Are you using StereoKit templates, or are you building SK from scratch?
      • I'm using the StereoKit .NET Core template from the Visual Studio marketplace.
    • Are your graphics drivers up-to-date? Etc.
      • Yes, Nvidia driver version 472.12.

    XR_EXTX_overlay compiled using the OpenXR SDK version 1.0.15 with the instructions on the repository

    Logs or exception details

    Running the example application with SKSettings.overlayApp set to true and the following environment variables:

    XR_ENABLE_API_LAYERS=xr_extx_overlay
    XR_API_LAYER_PATH=C:\Overlay_OpenXR
    XR_LOADER_DEBUG=all
    XR_RUNTIME_JSON=C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    

    Contents of C:\Overlay_OpenXR folder (referred to as $WORKING_DIRECTORY in the LunarG XR_EXTX_overlay repo):

    avatar1.png
    avatar2.png
    FreeImage.dll
    FreeImagePlus.dll
    highlighted1.png
    openxr_loader.dll
    openxr_loaderd.dll
    openxr_loaderd.lib
    OverlaySample.exe
    selected1.png
    xr_extx_overlay.dll
    xr_extx_overlay.json
    xr_extx_overlay.lib
    

    Logs:

    [SK diagnostic] Initializing StereoKit v0.3.6 Win32 x64...
    [SK diagnostic] Initializing Assets
    [SK diagnostic] Initializing Platform
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer xr_extx_overlay using interface version 1 and OpenXR API version 1.0
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 4 layers enabled and runtime interface - created instance = 0x00000241646e6420
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader terminator
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader trampoline
    Info [GENERAL | xrDestroyInstance | OpenXR-Loader] : RuntimeInterface::UnloadRuntime - Unloading RuntimeInterface
    Info [GENERAL |  | OpenXR-Loader] : RuntimeInterface being destroyed.
    [SK diagnostic] sk_gpu: Using Direct3D 11: NVIDIA GeForce RTX 3080
    [SK diagnostic] Starting mixed reality mode
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
    [SK diagnostic] available: XR_HTC_facial_tracking
    [SK diagnostic] available: XR_HTC_vive_srworks_pass_through
    [SK diagnostic] available: XR_KHR_vulkan_enable
    [SK diagnostic] available: XR_KHR_vulkan_enable2
    [SK diagnostic] available: XR_KHR_D3D12_enable
    [SK diagnostic] available: XR_KHR_opengl_enable
    [SK diagnostic] available: XR_EXT_win32_appcontainer_compatible
    [SK diagnostic] available: XR_KHR_visibility_mask
    [SK diagnostic] available: XR_KHR_binding_modification
    [SK diagnostic] available: XR_EXT_frame_composition_report
    [SK diagnostic] available: XR_EXT_hand_joints_motion_range
    [SK diagnostic] available: XR_HTC_vive_cosmos_controller_interaction
    [SK diagnostic] available: XR_HTCX_vive_tracker_interaction
    [SK diagnostic] available: XR_UNITY_hand_model_pose
    [SK diagnostic] available: XR_VALVE_analog_threshold
    [SK diagnostic] available: XR_EXT_dpad_binding
    [SK diagnostic] available: XR_FB_display_refresh_rate
    [SK diagnostic] available: XR_EXT_debug_utils
    [SK diagnostic] REQUESTED: XR_EXT_hand_tracking
    [SK diagnostic] REQUESTED: XR_EXT_eye_gaze_interaction
    [SK diagnostic] REQUESTED: XR_MSFT_scene_understanding
    [SK diagnostic] REQUESTED: XR_EXTX_overlay
    [SK diagnostic] REQUESTED: XR_KHR_D3D11_enable
    [SK diagnostic] REQUESTED: XR_KHR_win32_convert_performance_counter_time
    [SK diagnostic] REQUESTED: XR_KHR_composition_layer_depth
    [SK diagnostic] REQUESTED: XR_EXT_hp_mixed_reality_controller
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    [SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_hand_tracking
    [SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_facial_tracking
    [SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_srworks
    [SK diagnostic] OpenXR layer found: xr_extx_overlay
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer xr_extx_overlay using interface version 1 and OpenXR API version 1.0
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 4 layers enabled and runtime interface - created instance = 0x0000024165441ec0
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
    [SK diagnostic] Using system: Vive OpenXR: Vive SRanipal
    [SK diagnostic] Platform supports single-pass rendering
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    [SK diagnostic] Rejecting OpenXR's provided hand tracking extension due to the suspicion that it is inadequate for StereoKit.
    [SK diagnostic] OpenXR depth LSR ext enabled!
    [SK diagnostic] OpenXR world occlusion enabled! (Scene Understanding)
    [SK info] Couldn't create an OpenXR session, no MR device attached/ready? [XR_ERROR_INITIALIZATION_FAILED]
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader terminator
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader trampoline
    Info [GENERAL | xrDestroyInstance | OpenXR-Loader] : RuntimeInterface::UnloadRuntime - Unloading RuntimeInterface
    Info [GENERAL |  | OpenXR-Loader] : RuntimeInterface being destroyed.
    [SK info] MixedReality display mode failed, falling back to Flatscreen
    [SK diagnostic] Starting flatscreen mode
    [SK diagnostic] Created swapchain: 1280x720 color:rgba32_linear depth:depth32
    [SK diagnostic] Initializing Audio
    [SK diagnostic] ISAC audio backend not available, falling back to miniaudio! It's likely the device doesn't have Windows Sonic enabled, which can be found under Settings->Sound->Device Properties->Spatial Sound.
    [SK info] Using audio backend: WASAPI
    [SK diagnostic] Initializing Defaults
    [SK diagnostic] Initializing World
    [SK diagnostic] Initializing Sprites
    [SK diagnostic] Initializing Lines
    [SK diagnostic] Initializing UI
    [SK diagnostic] Initializing Renderer
    [SK diagnostic] Initializing Physics
    [SK diagnostic] Initializing Input
    [SK info] Initialization successful
    

    When running the application with exact same setup, but with SKSettings.overlayApp set to false, the following exception occurs in openxr.cpp: image

    Running the application without enabling the XR_EXTX_overlay API layer just with the following environment variables works as expected:

    XR_LOADER_DEBUG=all
    XR_RUNTIME_JSON=C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    

    Here are the logs for when the application is running as expected and not in an OpenXR overlay:

    [SK diagnostic] Initializing StereoKit v0.3.6 Win32 x64...
    [SK diagnostic] Initializing Assets
    [SK diagnostic] Initializing Platform
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 3 layers enabled and runtime interface - created instance = 0x000001f5b83d6240
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader terminator
    Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader trampoline
    Info [GENERAL | xrDestroyInstance | OpenXR-Loader] : RuntimeInterface::UnloadRuntime - Unloading RuntimeInterface
    Info [GENERAL |  | OpenXR-Loader] : RuntimeInterface being destroyed.
    [SK diagnostic] sk_gpu: Using Direct3D 11: NVIDIA GeForce RTX 3080
    [SK diagnostic] Starting mixed reality mode
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
    Info [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
    Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
    [SK diagnostic] available: XR_HTC_facial_tracking
    [SK diagnostic] available: XR_HTC_vive_srworks_pass_through
    [SK diagnostic] available: XR_KHR_vulkan_enable
    [SK diagnostic] available: XR_KHR_vulkan_enable2
    [SK diagnostic] available: XR_KHR_D3D12_enable
    [SK diagnostic] available: XR_KHR_opengl_enable
    [SK diagnostic] available: XR_EXT_win32_appcontainer_compatible
    [SK diagnostic] available: XR_KHR_visibility_mask
    [SK diagnostic] available: XR_KHR_binding_modification
    [SK diagnostic] available: XR_EXT_frame_composition_report
    [SK diagnostic] available: XR_EXT_hand_joints_motion_range
    [SK diagnostic] available: XR_HTC_vive_cosmos_controller_interaction
    [SK diagnostic] available: XR_HTCX_vive_tracker_interaction
    [SK diagnostic] available: XR_UNITY_hand_model_pose
    [SK diagnostic] available: XR_VALVE_analog_threshold
    [SK diagnostic] available: XR_EXT_dpad_binding
    [SK diagnostic] available: XR_FB_display_refresh_rate
    [SK diagnostic] available: XR_EXT_debug_utils
    [SK diagnostic] REQUESTED: XR_EXT_hand_tracking
    [SK diagnostic] REQUESTED: XR_EXT_eye_gaze_interaction
    [SK diagnostic] REQUESTED: XR_MSFT_scene_understanding
    [SK diagnostic] REQUESTED: XR_KHR_D3D11_enable
    [SK diagnostic] REQUESTED: XR_KHR_win32_convert_performance_counter_time
    [SK diagnostic] REQUESTED: XR_KHR_composition_layer_depth
    [SK diagnostic] REQUESTED: XR_EXT_hp_mixed_reality_controller
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    [SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_hand_tracking
    [SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_facial_tracking
    [SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_srworks
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
    Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 3 layers enabled and runtime interface - created instance = 0x000001f5b9207e90
    Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
    [SK diagnostic] Using system: Vive OpenXR: Vive SRanipal
    [SK diagnostic] Platform supports single-pass rendering
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
    Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
    [SK diagnostic] Rejecting OpenXR's provided hand tracking extension due to the suspicion that it is inadequate for StereoKit.
    [SK diagnostic] OpenXR depth LSR ext enabled!
    [SK diagnostic] OpenXR world occlusion enabled! (Scene Understanding)
    [SK diagnostic] Creating view: PrimaryStereo color:rgba32_sRGB depth:depth32 blend:Opaque
    [SK diagnostic] Setting view: PrimaryStereo to 3740x3740
    [SK diagnostic] Initializing Audio
    [SK diagnostic] ISAC audio backend not available, falling back to miniaudio! It's likely the device doesn't have Windows Sonic enabled, which can be found under Settings->Sound->Device Properties->Spatial Sound.
    [SK info] Using audio backend: WASAPI
    [SK diagnostic] Initializing Defaults
    [SK diagnostic] Initializing World
    [SK diagnostic] Initializing Sprites
    [SK diagnostic] Initializing Lines
    [SK diagnostic] Initializing UI
    [SK diagnostic] Initializing Renderer
    [SK diagnostic] Initializing Physics
    [SK diagnostic] Initializing Input
    [SK info] Initialization successful
    [SK diagnostic] OpenXR session begin.
    [SK diagnostic] Switched left controller profile to htc/vive_controller
    [SK diagnostic] Switched right controller profile to htc/vive_controller
    
    opened by mpoc 9
  • Providing Eye Gaze Sample Time

    Providing Eye Gaze Sample Time

    The OpenXR spec indicates that it is possible to query the actual sample time for a given eye gaze pose, which is useful for reasoning about high accuracy eye tracking, computing precise velocities over time, etc. It would be great if StereoKit could also provide that information.

    I've implemented and locally tested one possible approach, which I was wondering if you might be willing to provide feedback on in a PR? I'm open to alternative approaches to surfacing this information, of course.

    Basically, we just need a slightly modified version of openxr_get_space() to get the gaze space along with this extra timing information (it's specific to just gaze, and doesn't seem to be anything analogous for the head or hands).

    bool32_t openxr_get_gaze_space(pose_t* out_pose, XrTime &out_sample_time, XrTime time) {
    	if (time == 0) time = xr_time;
    
    	XrEyeGazeSampleTimeEXT gaze_sample_time = { XR_TYPE_EYE_GAZE_SAMPLE_TIME_EXT };
    	XrSpaceLocation space_location = { XR_TYPE_SPACE_LOCATION, &gaze_sample_time };
    	XrResult        res = xrLocateSpace(xr_gaze_space, xr_app_space, time, &space_location);
    	if (XR_UNQUALIFIED_SUCCESS(res) && openxr_loc_valid(space_location)) {
    		memcpy(&out_pose->position, &space_location.pose.position, sizeof(vec3));
    		memcpy(&out_pose->orientation, &space_location.pose.orientation, sizeof(quat));
    		out_sample_time = gaze_sample_time.time;
    		return true;
    	}
    	return false;
    }
    

    Then this sample time can be accessible in Input.cs as something like Input.EyesSampleTime.

    Thoughts?

    opened by sandrist 9
  • Add a `NotifyMode` to `UI.HSlider`

    Add a `NotifyMode` to `UI.HSlider`

    Describe the feature

    This might be a bit too specific of a use case, but I'm looking for a way to determine if a slider is still being grabbed or not at the moment that e.g. HSlider() returns true.

    An example use-case

    The use case is for a slider value that controls a longer-running (remote) computation of the order of seconds, on-the-fly. I'd like to be able to have a user change the slider value with some precision, but only kick off the computation at the moment the user releases the slider grab, so as not to produce too many unnecessary recomputations while manipulating the slider, but still have reasonably fast updates. I can do it in the current API by starting a timer since the last value change and then force a recomputation after X milliseconds of no change, but it's a bit hacky and error-prone. However, I think adding this feature would not be easy to fit into the current API, as it would require an extra pass-by-ref value signaling the grab state.

    ui 
    opened by paulmelis 9
  • Locatable camera, mediaframe

    Locatable camera, mediaframe

    When I use MediaFrameSourceGroupto grab camera frames (together with their SpatialCoordinateSystem), how would I be able to use these frames in a Stereokit application? I looked at the QR code example where you get a pose for a SpatialNodeGuid, I am not clear though how this would apply for getting the pose of eg. the locatable camera with respect to the the world space (like the Unity CameraToWorldMatrix).

    Eg. I would like to

    • Grab mediaframe
    • extract mediaframe spatialcoordinatesystem
    • get world coordinate system
    • calculate the CameraToWorldMatrix

    I saw on your twitter that you planned to show an example, is that already possible with the current version of Stereokit together with the MediaFrame reference class?

    opened by cwule 9
  • Missing/mismatch in shared libs with VS Code on Linux

    Missing/mismatch in shared libs with VS Code on Linux

    Description

    I'm following the VSCode guide at https://stereokit.net/Pages/Guides/Getting-Started-VS-Code.html on Linux. It appears some native shared libs are missing, either because the StereoKit package expects them to be installed globally, or perhaps due to some project dependency that wasn't added.

    When running dotnet run in a terminal window within VS Code I get:

    melis@juggle 15:50:~/concepts/skmap$ dotnet run
    [SK error] Failed to load StereoKitC!
    Unhandled exception. System.DllNotFoundException: Unable to load shared library 'StereoKitC' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libStereoKitC: cannot open shared object file: No such file or directory
       at StereoKit.NativeAPI.sk_run(Action app_update, Action app_shutdown)
       at StereoKit.SK.Run(Action onStep, Action onShutdown) in C:\Data\Repositories\Stereo Kit\StereoKit\SK.cs:line 199
       at Program.Main(String[] args) in /home/melis/concepts/skmap/Program.cs:line 9
    

    Looking a bit closer with LD_DEBUG=all dotnet run it seems /home/melis/concepts/skmap/bin/Debug/net6.0/runtimes/linux-x64/native/libStereoKitC.so is not finding two libs:

    melis@juggle 15:53:~$ ldd /home/melis/concepts/skmap/bin/Debug/net6.0/runtimes/linux-x64/native/libStereoKitC.so |grep not
    	libGLEW.so.2.1 => not found
    	libjsoncpp.so.1 => not found
    

    I have both GLEW and jsoncpp installed as system packages, but the shared lib versions don't match. And I don't see them in the native code dir within the project:

    melis@juggle 15:55:~/concepts/skmap/bin/Debug/net6.0$ find . -name libGLEW\*
    melis@juggle 15:55:~/concepts/skmap/bin/Debug/net6.0$ find . -name \*jsoncpp\*
    melis@juggle 15:55:~/concepts/skmap/bin/Debug/net6.0$ 
    

    Should these be installed separately through dotnet add package?

    Platform / Environment

    Arch Linux, VS Code 1.66.0, dotnet 6.0.110

    build 
    opened by paulmelis 8
  • SK Light Baking code crashes in `mesh_ray_intersect`

    SK Light Baking code crashes in `mesh_ray_intersect`

    Description

    mesh_ray_intersect crashes randomly when running SK Light Bake. Could be a multithreaded mesh update issue?

    Yes, multithreading sounds like a plausible culprit.. since it's a 50/50 it crashes.. I also noted it's harder to make it crash on my slower notebook than it is on my main workstation. I noticed it's increasingly easy to get it to crash the heavier the mesh is.

    Platform / Environment

    SK v0.3.7-preview.7?

    Logs or exception details

    image

    image

    bug 
    opened by maluoi 0
  • UI Handle overload with scaling

    UI Handle overload with scaling

    Hey Nick, I wrote a handle extension for scaling but noticed that you had previous and active pinch points in the source code, so I thought it could fit nicely here instead. I added the scaling to the clipboard for the DemoUI script for testing it out. Let me know what you think!

    opened by austinbhale 4
  • BVH Raycast in public API

    BVH Raycast in public API

    BVH raycast was added, but still needs hooked properly into the public API! This may depend on some async mesh feature work #411, since the BVH build can take some non-trivial time on larger objects.

    feature 
    opened by maluoi 0
  • Android exit from app

    Android exit from app

    Currently, when using SK.Quit on Android, the app exits just fine! But after two or three shutdowns, the OS will eventually pop up a warning about the app not closing properly.

    Currently using Android.OS.Process.KillProcess(Android.OS.Process.MyPid()), but have also tried FinishAffinity and Finish, all seem to behave the same.

    bug 
    opened by maluoi 0
  • Calculating a `Pose` in front of the user

    Calculating a `Pose` in front of the user

    In HoloLens 2, at startup in SK Remote Rendering there is no way to begin a scene from a known position relative to the world coordinates. Problem: User is standing at a point. We want to start/go "home" the application from that point and we want to place all elements in the scene based on a line directly in front of the user's head position. The head position is completely arbitrary only have meaning to the user in the mixed reality. The user wants all the visible assets in front of him. The position of the elements in the world coordinates has not yet been decided by the user. Concrete example. User is at a conference demonstrating the HoloLens 2 with SK as the platform. The world location is unimportant. The person wants to stand at a point of beginning. Whatever direction his head is facing at the moment the app HoloLens 2 Remote Rendering Player connects the SK rendering should be the start position. And there is a need to repeat this where the user has a menu button that he pushes to "Start" and the scene begins at the current head position. How could I accomplish this?

    opened by laultman 6
  • Drawing with Text.Add does not show text in Window

    Drawing with Text.Add does not show text in Window

    Description

    Using this simple window does not draw text:

    		UI.WindowBegin(MenuHeaderText, ref infoPose, infoPoseSize * U.cm);
    		StereoKit.Text.Add(DisplayText, Matrix.TR(new Vec3(0f, 0, -0.001f), Quat.LookDir(0, 0, -1)), 
    			infoWindowSize, TextFit.Wrap, style,
    			TextAlign.XLeft, TextAlign.CenterLeft, infoWindowCenter.x);
    		UI.WindowEnd();
    

    Where the infoPoseSize is a Vec2 30,16 and infoWindowsSize is a Vec2 = infoPoseSize * .999f. DisplayText is multiple lines of text with new line characters. The goal is to draw text inside the Window left aligned inside a margin box.

    		UI.WindowBegin(MenuHeaderText, ref infoPose, infoPoseSize * U.cm);
    		StereoKit.Text.Add(DisplayText, Matrix.TR(new Vec3(0f, 0, -0.001f), Quat.LookDir(0, 0, -1)), 
    			infoWindowSize, TextFit.Overflow, style,
    			TextAlign.Center, TextAlign.Center, infoWindowCenter.x);
    		UI.WindowEnd();
    

    Text center is drawn centered on the infoWindowSize top left corner.

    Any setting of TextAlign other than Center, no text is visible.

    Platform / Environment

    Windows 11, VS 2022, SK 0.3.7 preview 7

    Logs or exception details

    Logs are super helpful, please include them if they're at all relevant!
    

    If you have an exception, details about that would also be essential.

    opened by laultman 1
Releases(v0.3.6)
  • v0.3.6(Jun 6, 2022)

    Another chonky update, v0.3.6 weighs in at over 200 commits! The primary focus for this update was adding asynchronous asset loading. This applies only to textures for now, but this helps significantly when trying to decode large and expensive .png or .jpeg files! A staged loading process will display temporary textures, and smartly load textures in order of importance. All of this is automatic in the background, with no breaking API changes. But that's definitely not all that we've been up to, check out the Feature Highlights below for a list of notable items!

    A big thanks to all the contributors that have features is this release, as well as the Discord community's valuable feedback and testing! StereoKit is much better thanks to all of your contributions :)

    Quick plugs

    Just to highlight some related items here, there will be 3 StereoKit presentations at Mixed Reality Dev Days on June 8th! These won't be streamed, but will be made available online sometime after the event. If you'll be present at the event, do stop by and say hello! There will probably still be a few StereoKit stickers left :)

    Also, Microsoft is hosting another MR hackathon that includes StereoKit! Plenty of prize money there too, so be sure to check it out if you have the time. We're very excited to see what you'll make!

    Feature Highlights

    Additions

    Misc. Improvements

    • Updated OpenXR Loader to v1.0.22.
    • Updated ReactPhysics3D to v0.9.
    • Added support for Unicode in window titles, Win32 and Linux, #346, #347. Thanks @slitcch!
    • UI.HSlider pinch has better hit volumes now, should mean less window grabs. #335
    • Disabled UI elements now dim.
    • Improved UI shader finger ring visuals.
    • Increased vert count on default UI meshes for smoother corners.
    • System initialization log now happens before initialization, so crashes obviously show what system was initializing.
    • ISteppers can now be added before SK.Initialize, their Initialize call is deferred until after SK's initialization.
    • A number of docs updates from @paulmelis, @Zee2, @TopperDEL, and @Strepto!

    Fixes

    • Tex.Rough is now (1,0,1,1) instead of (0,0,1,1).
    • Fixed Material.Copy crashing when provided null.
    • Fixed a bug in converting C# filenames to UTF-8.
    • Fixed the MR fallback FilePicker failing to parse Unicode text. #374
    • Fixed an issue where FilePicker would add a trailing \0 to C# strings.
    • Fixed a bug where the FilePicker would crash on cancel.
    • Fix for NRE when GLTF texture failed to load.
    • GLTF loader now supports interlaced data. #320
    • Fixed incorrect sprite alignment mixing up left and right.
    • Added a warning for when SK.Initialize and SK.Run happen on separate threads, or execution is transferred to a blocking asset. #361
    • Fixed an issue with UVs on non-unit sized rounded cubes.
    • Fixed incorrect alpha values getting written to the screen buffer, causing compositing artifacts.
    • Fixed an issue with Tex.GetColors returning black on separate threads.
    • Fixed a bug in Sound stream's ring buffer.
    • Fixed a bug in ui_quadrant_size_mesh.
    • Fix for fallback keyboard not accounting for Renderer.CameraRoot.
    • Fixed SK using unavailable stage bounds. Thanks @slitcch! #331
    • Fix for stage (World.BoundsPose) not locating when stage bounds weren't available.
    • Fix for crash when loading unavailable OpenXR functions via Backend.OpenXR.GetFunction.
    • Fix for a bug with UI.Push/PopPreserveKeyboard not working with elements downstream a UI.Input element.
    • Fix for simulated hand's palm pose being in the incorrect location.
    • Fix for ModelNode.Mesh improperly returning empty Mesh objects. #302, Thanks @Strepto !
    • Fix for Mesh references not releasing properly. #303, Thanks @Strepto!
    • Fix for UI.HSeparator sometimes adding an extra empty layout line.
    • Fix for empty layout line adding unwanted padding at the end of UI Windows.
    • File picker handles different filter formats better (*.x, .x, or just x). #309
    • SK handles audio playback failure more gracefully now.
    • Fix for non-msvc compilers complaining about SK_CONST's weird attributes, now it's just static const for those compilers.
    • Fix for improper capitalization in Linux shader compiler paths, thanks @slitcch!
    • Fix a number of issues related to loading libdl on .NET/Linux.
    • Fixed a GLX issue in the Linux graphics binding. #359, Thanks @ChristophHaag!
    • No longer request Monado's EGL extension when using GLX.
    • Fix for a surface resize failure on UWP flatscreen, #298.
    • Fixed an issue with loading StereoKitC.dll in C# when starting in a working directory different from the exe.
    • Fix for Win32 Windows not correctly restoring the mouse cursor sprite.
    • Fixed SK crashing when the flatscreen app starts with a window too large for the screen's resolution.

    SDK and Sample Updates

    • Related StereoKit GitHub repositories were moved to the StereoKit organization!
    • Added a Windows desktop mirroring demo to C++.
    • Added a Windows window mirroring demo to C++.
    • Added environment map demo to C++.
    • Added point cloud rendering demo to C#.
    • Updated skshaderc to provide better errors to VS, generate files in folders that don't exist yet, and no longer treat warnings as errors. May fix #329
    • Added a "Dream Dev Template" C# project that includes Windows desktop duplication and Facebook's Passthrough extension.
    • Moved the Cmake and XPlat templates to their own repositories with some light revisions.
    • Updated StereoKit to VS 2022.
    • Cleaned up directory structure, less items in the root.
    • Added cmake option for removing physics, #354. Thanks @technobaboo and @slitcch!
    • Improved StereoKit's Visual Studio dependency build process.
    • Added StereoKit logo in .glb format.
    • Added StereoKit logo with background for GitHub dark mode.
    • Documenter support for operator overloads, #349.
    • Contributor's guide.
    • Updated samples for V and Zig.
    • Added oculus-openxr option to xmake for using non-oculus OpenXR loader.
    • Documented StereoKit's Row-Major matrix status, #352. Thanks @paulmelis!

    Install or Update

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.5(Jan 15, 2022)

    v0.3.5 is a big, general improvement update composed of more than two months, ~185 commits, and code from 4 different people! This was also during the StereoKit hackathon, and many improvements here come from suggestions and reports from the participants. If you participated in the hackathon, thank you so much for the feedback! Everyone else should check out the awesome projects the hackers put together!

    You won't find any huge features in this update, but there's a lot of smaller ones, and a ton of excellent fixes! I think everyone should find something in here to enjoy :) In particular, I'm pleased to have added SourceLink support! With a few easy tweaks, you can debug and step through StereoKit's source code, just from the NuGet package!

    I'd also like to specifically thank @Faolan for their work on the fallback soft keyboard, among other things, @opl- for a couple of really great fixes, and @slitcch for catching a sneaky OpenXR input issue!

    New Features

    • Added template support for VS 2022.
    • Added .qoi image format support.
    • Added MSAA to flatscreen UWP.
    • Added SourceLink, allows for stepping through SK's source in Debug!
    • Added fallback soft keyboard, thanks Faolan! Check out the demo code here.
    • Added Ctrl+Shift in the Simulator to swap between right and left hands.
    • Added a small debugging guide to the docs.
    • Added limited direct access to the OpenXR backend. #231

    API additions

    C/C++ API additions

    • Added vert_create.
    • Added matrix_t, matrix_r, matrix_s, and matrix_ts shorthands.
    • Added vec3_project, vec4_magnitude, vec4_normalize, and a few vector operators.
    • Added render_global_texture.

    Fixes and improvements

    • Improved cubemap Spherical Harmonics calculation speed.
    • Improved .hdr load speed.
    • UI interaction glow is brighter.
    • Far interact auto-disable behavior was tweaked.
    • GLTF loader now supports emissionFactor.
    • Fixed support for Unicode in asset file paths on Windows. #254
    • Fixed an issue where the C# debugging data wasn't getting copied into projects.
    • Fixed a GLTF issue with multiple UV channels stomping on each-other.
    • Fixed a mouse issue on UWP related to DPI and UI scaling.
    • Fixes for header C compatibility.
    • Lines.AddAxis now draws lines on unit vectors, with a white line for Z Forward. #231
    • World.Bounds are now available immediately after SK.Initialize.
    • Fixed a bug in SKMath.AngleDist. #232
    • Fixed a bug with UI.Space. #248
    • Fixed issues with Renderer.CameraRoot related to latency and drifting World.Bounds.
    • Fixed rare phantom UI presses for purely axis locked motion.
    • Fixed an issue with mouse/keyboard input lag on Win32.
    • Fixed an issue with 'tracked' state with OpenXR controllers. Thanks Moses! #241
    • Fixed an issue with GLTF animations using cubic interpolation.
    • Fixed an issue with GLTF node based animations on root nodes being incorrectly rotated. #240
    • Fix for skinned meshes not updating when no animation is playing.
    • Reduced GLTF warning count on models with warnings.
    • Fixed a simulator position offset being applied when simulator is off.
    • Fixed Linux defaulting mouse to available at first load.
    • Fixed Linux not obeying window size settings.
    • Fixed refcount issue with ModelNodes.
    • Fixed an HSlider bug on Android.
    • Fixed a UWP file reference issue.
    • Fix for a Potential bug with auto-sizing labels
    • Fix for Linux/C#, where SK would prefer system SK libraries instead of its own
    • Matrix's internal System.Vector type is now public.
    • Miniaudio update that fixes an issue with mics on Linux, thanks opl-!
    • Fixed a Mixed Reality Capture compositing issue on HoloLens. #216
    • Fixed a bug in GL screenshots.

    Install or Update

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.4(Oct 30, 2021)

    v0.3.4 brings skinned animation to StereoKit! I hope you'll enjoy using it as much as I've enjoyed making it :) Along with that comes a solid collection of bug-fixes, and some polish to the UI layout system. Plus a whole bunch more! A big thanks to @slitcch, @maxgolov, @sjando, and @aprilspeight, who all made contributions to this release!

    In other big news, Microsoft is hosting a StereoKit hackathon for the month of November! It's a great moment to try StereoKit out, and maybe win some cash prizes along the way. Along with that, we'll be hosting some online social events and AMAs, so keep your eyes peeled for those, even if you aren't participating in the hack!

    For those getting started with StereoKit, @aprilspeight has created a list of getting started videos! These are also embedded in the docs site, and should help ease new StereoKit devs into their first StereoKit application. Many thanks April!

    And check out the new documentation guides for Drawing and Working with Materials! Keep your eyes peeled in November for more docs like these :)

    New Additions

    GLTF Additions

    • GLTF loader now supports skinning and animation data. Doesn't include morph targets yet.
    • Added support for GLTF mask transparency.
    • .vrm files are now recognized as GLTF files. This does not include VRM specific extensions.
    • Added a fallback for Specular/Glossy GLTF materials, not a full implementation, but an improvement.
    • Additional logging surrounding some cases that aren't supported yet.

    Bug Fixes

    • Fixed Unicode entry on Win32 while in VR.
    • Fixed right arrow key and del key on Linux. By @slitcch #192
    • Fixed some UI elements not sizing properly with custom text styles.
    • Fixed crash with mip-maps on unsupported formats. #195
    • Fixed a render sort issue when compiling for ARM. By @maxgolov #197
    • Fixed a crash on Windows devices lacking the Win10 SDK. #205
    • Fixed crash when Linux tried to read a folder as if it were a file. #183
    • Fixed an Android issue with Radio and Toggle buttons.
    • Fixed an issue with WMR and SteamVR where the system would provide insufficiently simulated articulated hands.

    Development Tool Updates

    • VS Template was updated to use .Net Core 3.1 by default.
    • Added a new shader compiler backend that works on both Windows and now Linux.

    Install or Update

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.3(Sep 22, 2021)

    Another two months and over 100 commits! Some really awesome features for this update including Unicode support, revamped Model inspection API, UI updates, anti-aliased flatscreen, and a whole bunch more! For a high level look at this update with some flashy gifs, check out the accompanying blog post!

    I also really wanted to thank @ClonedPuppy for their input during this update, a good number of features and fixes came from their feedback!

    New Additions

    • Added ModelNode API to the Model assets for detailed exploration of a Model's contents. Check here for some examples.
    • Added Model.Copy.
    • Added Model.Intsersect and a Ray.Intersect(Model) overload.
    • Added static and instance Vec2.InRadius and Vec3.InRadius methods.
    • Added Hand.pinchPt, a stable, non-sliding pinch location indicator.
    • Added UI.Model.
    • Added UI.PushSurface and UI.PopSurface for more control over placing UI layouts within the scene.
    • Added UI.HSlider confirmation mode VariablePinch, for more fine tuned manipulation of slider values.
    • Added UI.SetElementVisual to override UI Meshes and Materials, along with UI.QuadrantSizeVerts and UI.QuadrantSizeMesh to aid in working with 'quadrant sizing' meshes.
    • New UI panel look
    • Material now works with Vec2 and Vec3 material parameter types.
    • Material will now warn the developer if they try to set a parameter with the wrong type.
    • Unicode support in text rendering and entry.
    • Fonts are now composed of a series of font files, which will be searched in-order for any glyph.
    • Antialiasing in Flatscreen mode.
    • Removed partial FBX support, which was a little too partial.
    • Improved pointer ray based on shoulder estimate.
    • UWP ARM binaries added to the NuGet package.
    • Text shader now combines tint color alpha into the final alpha value.
    • Added texture sampler support for GLTF files.
    • Added a DemoWorldAnchor to the main collection of demos.

    Bug Fixes

    • Fixed bug where Win32 file picker would change working directory.
    • Handles no longer slide when releasing the pinch gesture when pinch is driven by controllers/mouse, see Hand.pinchPt.
    • Poke interactions now account for finger radius.
    • Fixed issue with id overlap from equirect textures and regular textures created from the same file .
    • Renderer.Blit renders upside-right now.
    • GLTF material tint colors now use the correct color space.
    • Fixed Stepper initialization causing issues with late load libraries. #182
    • Fixed an issue with hand meshes and occlusion meshes not obeying the camera root transform. #184
    • OpenXR time is now available on the first frame, this affects a number of calls, such as World.Bounds. #188
    • Fixed a number of reference counting issues.
    • Fixed issue with xrCreateInstance failure not reporting the correct issue.
    • Fixed Android asset loading prefix issue.
    • Fixed a crash when using UWP with the Oculus runtime.

    Native Specific Additions

    C# methods and properties all have an equivalent C++ function, but here are some additional changes that are of interest specifically to the native developer.

    • Partial cmake support (Windows and Linux).
    • Native cmake based C++ template.
    • Asset types now have _addref functions.
    • Added color_hex, color32_hex, and color32_to_128.
    • Added material_set_vector2, material_set_vector3, material_set_vector4.
    • Unicode support for UTF-8 and UTF-16, see the text_* and text_*_16 functions, as well as ui_* and ui_*_16.
    • Added a complex lighting demo to StereoKitCTest.
    • Default text_style_t is now 0 instead of -1.

    Install or Update

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Jul 11, 2021)

    This update brings in support for a couple new extensions, as well as a steady march of general improvements! In particular, the recently added XR_MSFT_scene_understanding extension makes world occlusion and raycasting just a line of code away for HoloLens 2 devices. Also, the file picker is now integrated into StereoKit's core, and works properly with native pickers and permissions when available!

    Many thanks to @slitcch and @technobaboo for their contributions to this update!

    Additions

    Changes/Improvements

    • Updates to the docs site, added additional information and improved some formatting.
    • Added new confirmation options to UI.HSlider, should allow for fewer accidental activations. #171
    • Diagnostic logging is now on by default.
    • Improved line rendering visuals. #156
    • Linux now loads the system's default font.
    • STL now loads as flat-shaded instead of smooth.
    • If a GLTF comes in without normals, they'll be automatically calculated.

    Fixes

    • Fixed a bug where the Alt key was going unrecognized in Win32
    • Fixed a bug where the Perception Bridge extension was incorrectly disabled. #151
    • Fixed a bug where out-of-focus UWP apps were still receiving inputs.
    • Fixed a UI.Text layout issue.
    • StereoKit now recognizes absolute paths on Linux.
    • Cursor now properly reports unavailable when leaving window on Linux.

    Install or Update

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(May 14, 2021)

    A new API to interact with controller input was added. Hands are still the primary input device, whether simulated or articulated, but you can now get detailed information about the controller directly! A lot of work was also done to audio: a new Microphone API, streaming audio support, and and API for sound play instances.

    Another big visual change was a new default lighting! The previous one was a little dim, and had a lot of extra color in it. The new one is much brighter and a lot less colorful, making it much easier to make white objects look white! If you want the old lighting back, see the bottom of this page for a code snippet.

    UI also got a bunch of polish, a lot of small details were fixed, improved, or made prettier! An initial 2-handed grab was added, and a number of useful new UI functions were added. The HSeparator and Push/PopTextStyle methods allow for some extra styling of your UI, and UI.Text enables you to display large blocks of text! UI.Input also got an overhaul that makes it look better, and behave far nicer!

    During this update, Varjo also shipped a conformant OpenXR runtime! StereoKit worked with it out of the box, and the SKSettings.blendPreference option was added to hook into their passthrough AR capabilities. A pass was also done on Quest to make the experience there nicer, it's now more stable, supports hand tracking, and brings back a few features that had been temporarily disabled for the v0.3 release.

    Enjoy :)

    Interactive Demo

    To see a number of these new features in action, check out the interactive release notes demo!
    https://github.com/maluoi/StereoKitReleaseNotes/tree/main/v0.3.1

    Additions

    Changes/Improvements

    • Improved the default lighting: brighter, less saturated, and white objects now feel white.
    • UI.Input text entry improvements, now spawns the OS virtual keyboard on UWP. #110
    • UIMove.FaceUser elements now account for hand's vertical position on the UI.
    • Button interaction is now a capsule->bounds intersection test instead of a line segment->bounds intersect, making hit detection more reliable.
    • Far Interact ray visual and behavior improvements. #125
    • Extra consistency and improvements in UI activation animations.
    • Monado now uses a rgb10a2 display surface.
    • Rounder fingertips with a new fingernail shape on the hand mesh.
    • Improved switching between hand tracking and controllers, this makes Quest much nicer to work with.
    • GLTF models now support multiple materials per-mesh, null materials, sparse vertex data, and additional data formats.
    • PBR shader now also uses the AO map.
    • DX11 now picks best graphics card on load.
    • Android systems use the OS default font.
    • Improved spatial audio fallback on Android/Linux, better falloff and stereo panning.
    • Improved UVs and geometry layout of procedural 3d shape types.
    • Default.MeshQuad was changed to be a normal quad, Default.MeshScreenQuad was added to contain the replaced mesh.
    • Improved performance reporting: now excludes vsync time, and includes app init time.

    Fixes

    • Fix for Oculus color banding bug. #138
    • Fix for SphericalHarmonics missing when loading an already loaded cubemap.
    • Fixed a number of UI layout inconsistencies.
    • UI.AreaRemaining is accurate now.
    • Fixed a number of issues with UI.HSlider.
    • Fixed a crash on Android when loading sounds from file. #126
    • PBR shader now works on Quest. #127
    • Exit button on Quest now shuts down properly.
    • Fixed some loss of mouse movement on flatscreen UWP while look-rotating.
    • Fixed an issue preventing .Net Core 5 projects from working on Windows. #133
    • Fixed equirect cubemap conversion on Linux.
    • Fixed Renderer.CameraRoot not working on Linux.
    • Fix for equirect cubemap conversion failing before first frame on HoloLens.
    • Fixed some visual bleed on the second mip of text.
    • Fixed a near clipping plane issue on DX systems.

    Install or Update

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.

    Old Lighting

    If you need the old lighting/skybox back, stick this after SK.Initialize:

    Renderer.SkyLight = new SphericalHarmonics(new Vec3[] {
        new Vec3( 0.27f,  0.26f,  0.25f),
        new Vec3( 0.07f,  0.09f,  0.11f),
        new Vec3(-0.06f, -0.06f, -0.04f),
        new Vec3(-0.06f, -0.04f, -0.01f),
        new Vec3(-0.04f, -0.05f, -0.06f),
        new Vec3( 0.15f,  0.16f,  0.16f),
        new Vec3(-0.04f, -0.05f, -0.05f),
        new Vec3( 0.05f,  0.05f,  0.04f),
        new Vec3(-0.11f, -0.13f, -0.13f),
    });
    Renderer.SkyTex = Tex.GenCubemap(Renderer.SkyLight);
    
    Source code(tar.gz)
    Source code(zip)
  • v0.3(Feb 26, 2021)

    I would describe v0.3 as the platform compatability update! This big update has been a lot of work, but brings in initial support for Oculus Quest 1 & 2 on Android, as well as Linux! This meant a complete overhaul of the graphics pipeline, and even resulted in the development of a new single-header cross platform graphics library, sk_gpu.h!

    Additions

    Changes

    • The StereoKitApp class is now SK, and the initialization flow has been re-worked to better accomodate Android.
    • C# math types (Vec/Matrix/Quat etc.) are now based on System.Numerics, allowing for easier interop with other libraries.
    • Shader meta syntax has been changed, additional details about this will be in an upcoming guide, see the builtin shaders for now.
    • Instanced rendering pattern changed, and now renders more than 2x the previous number of instances at once.
    • Removed direct Leap support in favor of Leap's OpenXR layer.

    There's a lot more, but I'll say these are most of the big items! This update has been in the works for 8 months, and you can see all the changes over in this commit :)

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Jul 12, 2020)

    Exciting update! StereoKit now works on SteamVR's OpenXR runtime! Also, a number of features finally land as OpenXR 1.0.9 brings a couple cool extensions out of preview. Which means StereoKit also now supports OpenXR's hand tracking instead of UWP's, 3rd eye render for more accurate screenshots/video, as well as QR code support!

    Additions / Changes

    • SteamVR support!
    • Updated to OpenXR 1.0.9.
    • QR Code support through the OpenXR Spatial Bridge extension. See Pose.FromSpatialNode.
    • 3rd eye render on HoloLens 2 though the OpenXR Secondary View extension! Now your holograms will line up with the real world :)
    • Articulated hand tracking through OpenXR. Previously was platform specific through UWP functions.
    • Flatscreen options to set FOV and background clear color
    • stereokit.h is now fully C compatible, allowing for easier interop with other languages. See here for basic V and Zig examples.
    • Default asset ids are now available as constants in C.

    Fixes

    • Fixed OpenXR implementation ignoring near/far clip planes.

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.2(May 7, 2020)

    This is a big update! This changelog is not an exhaustive list. Also see this brand new sample about loading and drawing elevation maps from Bing Maps!

    Additions / Changes

    -Oculus Desktop support. -Updated to OpenXR 1.0.8. -Depth buffer sharing for depth based reprojection functionality, stabilizes AR holograms pretty solidly! -New UI skin has a round edge with white borders, rendered using fewer instances, all in a single draw call! -UI windows also have backs now, and can auto-calculate their size. -UI has a far grab interaction using the hand pointers. -Flatscreen mode now supports moving around with Shift or CapsLock + WASDEQ, -Precompiled shaders, included shaders are now precompiled, saving loads of time on startup! -Mesh now tracks geometry data in RAM for intersection information. You can access this data with Mesh.GetVerts and Mesh.GetInds. -Mesh now has an Intersect method, and Ray has an Intersect method for Mesh! -Defaults now contains sphere and cube meshes. -Some in-progress support for FBX file format, works moderately well with models straight from Blender. -Added FromMemory methods to Tex, Model, and Shader. -Added Material.Wireframe property. -Added R8, R16, and R32 texture formats, plus accompanying overloads for Tex.SetColors. -Added shortcuts for Unit conversions, see the U class. -Visible messagebox for some errors that result in complete failure.

    Breaking Changes

    -Renamed UI.AffordanceBegin and UI.AffordanceEnd to UI.HandleBegin and UI.HandleEnd -Renderer.SetView is now Renderer.CamRoot, and behaves like a root transform instead. -DefaultIds class is now private, use Defaults instead.

    Fixes

    -Lots of various fixes! You'll have to check the commit log to see exactly what though, sorry!

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.9(Mar 6, 2020)

    Additions / Changes

    • Real spatial audio on Windows and HoloLens 2! Implemented by @MikeChemi
    • Hands glow when pinched.
    • Grip and pinch now require a little extra motion to de-activate, making them more reliable.
    • Demo code for colorizing the hand, and individual fingers.

    Fixes

    • Fixed a bug where Leap Motion hands would reference the wrong joint for orientation.

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.8(Feb 28, 2020)

    Additions / Changes

    • Added a more robust API for working with the contents of Model assets.
    • Made the finger glow/ring effect more bold.

    Fixes

    • Fixed a bug that would prevent hand tracking from working on non-retail builds of the OS

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.7(Feb 23, 2020)

    Additions / Changes

    • Updated to OpenXR 1.0.6, new update script should make future updates even easier.
    • Added Mesh.SetVerts and Mesh.SetInds, check out the docs for a quick demo!
    • Added Tex.SetColors, and a similar demo of that.
    • Large hand system refactor! Better more efficient code, more resilient fallbacks.
    • Added a hand override system that allows you to set or animate interactive hand movements. Great for testing!
    • Hand now has pinch and grip activation values.
    • Added Time.SetTime for explicit control over the application's current time. Primarily for testing.

    Fixes

    • Fixed some threading issues that would cause Leap Motion hands to flicker.
    • Fixed a crash when attempting to build mip-maps on non-power-of-two textures.
    • Resolved an issue where sampler states would occasionally get assigned out of order.

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.6(Feb 9, 2020)

    Additions / Changes

    • Added .stl model format support.
    • Added a C# sound API.
    • Added a radial hand menu.
    • Added multiple movement modes for moving affordances, including a default 'facing user' mode for windows.
    • Added explicit size variants of UI elements.
    • Text layout was re-written to support clipping, word wrap, and some resize modes for text constrained to a box! This also includes some improvements to sizing and centering behavior.
    • Shaders now compile to a temp folder instead of a cache folder. This was to enable shader caching in UWP, which can't write to its own install folder. This does mean that initial load after install could take extra time, but this is a temporary fix.

    Fixes

    • Fixed a flatscreen window resizing bug.
    • Fixed line rendering size issues.
    • Fixed a bug in StereoKitApp.VersionName.
    • .obj loader should work on more file format variants.
    • Fixed a reference counting bug, #45
    • Text input now behaves more appropriately.

    Install

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.3(Jan 15, 2020)

    Additions / Changes

    • Leap motion support now works for VR, and no longer works in Flatscreen
    • Line lists were exposed to C#
    • Setting texture data from C#
    • Added UI.AreaRemaining to get the remaining UI layout space
    • Added a UI.VolumeAt control
    • Added Vec3.Lerp
    • A number of additions to docs and examples!
    • Build pipeline scripts for easier NuGet builds and publishing

    Fixes

    • Fixed a shutdown issue where the swapchain would get destroyed more times than it existed.
    • Text.MakeStyle binding fix, thanks to @sjando
    • Distance formula fix
    • Graphics adapter picking for OpenXR

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(Jan 8, 2020)

    Changelog:

    • Fixed a bug that would cause UWP flatscreen applications to show black.
    • Adjusted grip event so it'll trigger during flatscreen's grip pose.

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Jan 3, 2020)

    Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html

    Source code(tar.gz)
    Source code(zip)
Owner
Nick Klingensmith
Creative coder: procedural content, game development tools and VR/AR/MR!
Nick Klingensmith
A set of open c++ game development tools that are lightweight, easy-to-integrate and free to use. Currently hosting a magicavoxel .vox full scene loader.

open game tools Open game tools is a set of unencumbered, free, lightweight, easy-to-integrate tools for use in game development. So far it contains:

null 291 Dec 29, 2022
A minimalist library with basic facilities for developing interactive real-time 3D applications, with a strong emphasis on simplicity and ease of use.

SlimEngine A minimalist and platform-agnostic base project for interactive graphical applications (2D/3D) with a strong emphasis on simplicity, ease o

Arnon Marcus 67 Oct 29, 2022
OpenCorr is an open source C++ library for development of 2D, 3D/stereo, and volumetric digital image correlation

OpenCorr OpenCorr is an open source C++ library for development of 2D, 3D/stereo, and volumetric digital image correlation. It aims to provide a devel

Zhenyu Jiang 66 Jan 6, 2023
Open source Altium Database Library with over 147,000 high quality components and full 3d models.

Open source Altium Database Library with over 147,000 high quality components and full 3d models.

Mark 1.4k Dec 29, 2022
Vizzu is a free, open-source Javascript/C++ library for animated data visualizations and data stories.

Vizzu is a free, open-source Javascript/C++ library utilizing a generic dataviz engine that generates many types of charts and seamlessly animates between them

Vizzu 1.6k Jan 3, 2023
An Open-Source subdivision surface library.

OpenSubdiv OpenSubdiv is a set of open source libraries that implement high performance subdivision surface (subdiv) evaluation on massively parallel

Pixar Animation Studios 2.7k Jan 2, 2023
A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

null 2.5k Jan 4, 2023
Modern C++14 library for the development of real-time graphical applications

CI Community Support bs::framework is a C++ library that aims to provide a unified foundation for the development of real-time graphical applications,

null 1.7k Jan 2, 2023
SPIRV-Reflect is a lightweight library that provides a C/C++ reflection API for SPIR-V shader bytecode in Vulkan applications.

SPIRV-Reflect SPIRV-Reflect is a lightweight library that provides a C/C++ reflection API for SPIR-V shader bytecode in Vulkan applications. SPIRV-Ref

The Khronos Group 457 Dec 26, 2022
Open 3D Engine (O3DE) is an Apache 2.0-licensed multi-platform AAA Open 3D Engine

Open 3D Engine (O3DE) is an Apache 2.0-licensed multi-platform 3D engine that enables developers and content creators to build AAA games, cinema-quality 3D worlds, and high-fidelity simulations without any fees or commercial obligations.

O3DE 5.8k Jan 7, 2023
appleseed is an open source, physically-based global illumination rendering engine primarily designed for animation and visual effects.

appleseed is an open source, physically-based global illumination rendering engine primarily designed for animation and visual effects.

appleseedhq 2k Jan 8, 2023
An open-source implementation of Autodesk's FBX

SmallFBX An open-source implementation of Autodesk's FBX that is capable of import & export mesh, blend shape, skin, and animations. Mainly intended t

Seiya Ishibashi 43 Dec 21, 2022
The open-source tool for creating of 3D models

The open-source tool for creating of 3D models

3D geoinformation research group at TU Delft 428 Dec 21, 2022
Cocos2d-x is a suite of open-source, cross-platform, game-development tools used by millions of developers all over the world.

Cocos2d-x is a suite of open-source, cross-platform, game-development tools used by millions of developers all over the world.

cocos2d 16.7k Dec 26, 2022
Tesseract Open Source OCR Engine (main repository)

Tesseract OCR Table of Contents Tesseract OCR About Brief history Installing Tesseract Running Tesseract For developers Support License Dependencies L

null 48.2k Jan 2, 2023
A completely free, open-source, 2D game engine built on proven torque technology.

Torque2D 4.0 Early Access 1 MIT Licensed Open Source version of Torque2D from GarageGames. Maintained by the Torque Game Engines team and contribution

Torque Game Engines 692 Jan 9, 2023
Open-Source Vulkan C++ API

Vulkan-Hpp: C++ Bindings for Vulkan The goal of the Vulkan-Hpp is to provide header only C++ bindings for the Vulkan C API to improve the developers V

The Khronos Group 2.5k Jan 8, 2023
ZBar Bar Code Reader is an open source software suite for reading bar codes from various sources

ZBar Bar Code Reader is an open source software suite for reading bar codes from various sources

null 2.4k Dec 26, 2022
Open-source, cross-platform, C++ game engine for creating 2D/3D games.

GamePlay v3.0.0 GamePlay is an open-source, cross-platform, C++ game framework/engine for creating 2D/3D mobile and desktop games. Website Wiki API De

gameplay3d 3.9k Jan 2, 2023