A simple parser for the PBRT file format

Related tags

CLI pbrt-parser
Overview

PBRT-Parser (V1.1)

The goal of this project is to provide a free (apache-lincensed) open source tool to easily (and quickly) load PBRT files (such as PBRT's own "pbrt-v3-scenes" test scenes, or Disney's Moana island model).

In particular it

  • contains both a purely syntatical as well as a more advanced semantical parser (see below)

  • can parse the original (ie, ascii-) '.pbrt' files from pretty much any pbrt file I could find.

  • comes with support for reading and writing the resulting pbrt scene graph to a binary ".pbf" format (.pbf = binary pbrt format) that is significantly faster to read. I.e., you can use the included pbrt2pbf to convert to convert a ascii .pbrt file to a binary .pbf file, which when loaded by an application will yield exactly the same scene graph as parsing the original .pbrt file, just much faster (for moana, that drops parsing time from 30ish minutes to seconds!).

A few screenshots:

ecosys.pbrt landscape.pbrt moana.pbrt

Contributors (in "order of appearance")

  • Ingo Wald
  • Fabio Pellacini (University of Rome)
  • Stefan Zellman (University of Cologne)
  • Will Usher
  • Nate Morrical
  • lots of other people, through suggestions, bug reports, etc ...

Release Notes

V 2.4:

<<<<<<< HEAD

  • added point and spot light sources - can now parse villa-lights-on =======
  • major bugfixes; changed attribute handling, can not properly parse area lights again

devel

V 2.3:

  • added 'hair' shape (can now parse pbrt v3 hair files)
  • removed all DLL build for windows; now using only static libraries on windows (this avoids windows issues with passing std::string etc through dll boundaries)

V 2.2:

  • have first area light sources (distant and infinite)
  • added reverseorientation
  • added InfiniteLight::{nsamples,L,scale}

V 2.1.4: Various bugfixes:

  • textures now have 'name' field, which now gets read and written to/from binary
  • fixed core dump when reading/writing spectrum values in pbfs.
  • bumped binary file format to v1.0 (due to changes in format)

V 2.1:

  • added semantic parsing of 'AreaLights', which are now attached to shapes, and stored to/loaded from PBF files.

  • significant cleanup of semantic parser: SemanticParser is now a separate class in a sepa rate header file, with all texture-, mateiral-, geometry-, etc based parsing in separate implementation files. Also, the single giant "parseMaterial" etc have been split into type-specific parse functions (eg, parseMaterial_disney vs parsematieral_mix, etc), further increasing readability.

V 2.0:

  • amalgamated all public API into a single header file (include/pbrtParser/Scene.h) that now only contains the fully semantically parsed Scene. All intermediary syntax-only parsing is now hidden in the impl/ directory, is no longer installed, and is no longer exported as a separate library. From now on, only a single library with a single header files is required after install.

Status

The semantical parser currently supports:

  • Shapes: trianglemesh, disk, sphere, and 'curve' are supported.
  • Materials: Disney, Uber, Mix, Metal, Mirror, Matte, Translucent, Plastic, Substrate, Fourier, and Glass should all work. In particular, all indirect references (e.g., a "mix" material referencing two other materials by name) are now properly resolved.
  • Textures: Image, PtexFile (storing only the filename, not the ptx data), Fbm, Windy, Marble, Scale, Wrinkled, Mix, and Constant. As with materials, all indirect references should be fully recognized.
  • File formats
    • .pbrt : the original pbrt file format - slow to parse, but will work
    • .pbf : our own binary file format (use pbrt2pbf to convert ascii pbrt to binary pbf)

Disclaimer(s): I did do a significant amonut of testing to make sure that the parser can load all .pbrt files without complaining, and that the above classes will parse everything that's in those files .... BUT:

  • I may have missed some things - please let me know...

  • The parser should parse all .pbrt files I could find, but there will likely be other files that are valid PBRT files that the parser would choke. If you find any, please let me know.

  • I do not currently have a fully PBRT compliant renderer, so cannot test whether all the parsed data is actualy parsed correctly. Generally triangle meshes, objects, instances, transforms, material and texture types, mapping of materials/textures etc to shapes, etc, should all work, but I can't entirely vouch for all material properties or texture properties of all material types.

  • I will parse auxiliary .ply files as part of the parsing process, but will not parse texture formats, ptex, spectrum specs, etc. For all such auxiliary files I'll include the file name, but leave it to the app to load such files (else I'd require tons of external dependencies)

Known Limitations

  • loopsubdiv shapes are still ignored.

  • curve is currently a single shape type; would make senes to split into FlatCurve, CylinderCurve, etc.

  • some models use camera space for object definitions - this isn't supported yet.

A Brief History of this Project

This project started out as being mostly a toy project for my own use, originally with the sole goal of being able to load PBRT's heavily instanced models (ecosys, landcape, and sanmiguel) for some ray tracing/instancing research.

Since then, it's come a long way, and has pretty much become my method of choice for getting content into my various ray tracing projects: First, with more powerful hardware around it's now more practical to have good material data around for one's ray tracing research, and other than PBRT's models there's previous few freely available models with anything other than OBJ Wavefront's material model. Second, last year Disney released the Moana island in two file formats, one of which is PBRT - and since my original "toy project" already did most of what was required to load Moana I ended up spending some more time on this, and bringing it to a state where I can use it for pretty much any PBRT model (including Moana, of course).

For those that have looked at this library in its early stages: it has changed a lot! In its original form, it was a purely syntactical parser that did parse the transforms and object hierarchy, as well as external ply files for triangle meshes, but wouldn't go much beyond that. Anytying else - materials, textures, lights, and even most Shape-related stuff - wasn't parsed beyond pure "name:value" pairs that you'd then have to parse interpret yourself to figure out, for example, what exact kind of material it was, what it's name-value pairs meant in actual material parameters, etc.

Since then, after having had to realize myself that that wasn't enough I eventually went ahead and significantly extended this library to have both a purely syntactical and a more advanced semantical parser that would also parse materials, textures, shapes, etc (see below), and eventually also added a binary file format to deal with, in particular, the egregious load times for the 40+GB Moana model (in binary format this now takes only seconds rather than half an hour...).

At its current stage, the library should be able to parse pretty much anything I could find in pbrt file format. If you find something it doesn't parse at all, let me know. That said, since I don't have a fully PBRT compliant renderer yet there will, by necessity, be several things that I haven't tested at all. If you do find something that's obviously broken, let me know (and I'll gladly take pull requests, too!).

Semantical vs Syntactical Parser

When looking at parsers, they typically consist of two intermingled stages: pure syntax (e.g., regnozizing the word "Shape" as the beginning of a shape definition) to full semantic (e.g., what a shape actually is, and how it behaves). For many formats, the boundary between those two extremes is "a bit" wishy-washy, with much of the semantics requiring way more effort to understand than the pure syntax. Arguably the most extreme format on that spectrum is XML, where the syntax itself only defines "nodes", "attributes", and "content", with all semantical meaning of what specific nodes or attributes actually mean left for the app to figure out.

PBRT specifies some more semantics in the format (e.g., the file format itself specifies that a Shape is different from a Material), but still leaves a lot to be figured out afterwards. For example, in terms of pure syntax a triangle mesh and a sphere are both "shapes" that just differ in what parameters got attached to that shape. That makes the format easy to extend (you could add a new geometry type of material type to PBRT without changing the file format at all!), but leaves more work for the app to figure our what certain name:value pairs actually meant.

Initially, this project only focussed on the syntax, and left all interpretation of name:value pairs to the application. This however can be rather tricky, and after I wrote that "name:value interpretation code" multiple times in multiple project I eventually decided to put that into this library as well, resulting in both a purely syntactical, as well as a more ready-to-use semantical parser.

To explain the difference: In the syntactical parser, for a given triangle mesh you'd end with a C++ class of type "Shape" that would have a parameter "name" with type "string", a size of 1, and a value of [ "trianglemesh" ], as well as a pamameter "P" with type "float", a size of 3*N, and a value of [ x0 y0 z0 x1 .... zN ] .... but it'd have to be the application that has to figure out that this is a triangle mesh with a vertex array. In the semantical parser (which of course builds on top of the syntactical parser as an intermediary parsing step) you will instead end up with a class "TriangleMesh" that has a C++ class member of "std::vector position", etc. Similarly for materials: The syntactical parser only tells you that there is a material with a string specifying, for example, the type "disney", and a list of parameters; while the semantical parser would parse this down to a C++ "DisneyMaterial" class, with C++ members for the material parameters, etc.

LICENSE

This project comes with Apache 2.0 license - pretty much "use as you see fit".

USAGE

For examples usages, see the simply tools (e.g., pbrtInfo or pbfInfo) in the apps directory.

Suggested use is with CMake, in which form you can use it in either one of two ways:

Option 1: Make Install

Build the project with cmake, and "make install" to a install directory. Once installed (say, to /usr/local)

  • In your source code, include "pbrtParser/semantic/Scene.h"
  • In your project, link to the pbrtParser_semantic library.

Of course, this way you can use whatever build system you want to use.

Option 2: Cmake with source-access

If you don't like the "make install" option, you can also include the full source three, either as a complete copy of this project (ugh), or (preferred!) as a git submodule.

Assuming you're building your main project with cmake, and assuming this parser is included in /external/pbrtParser (hint: git submodule add https://github.com/ingowald/pbrtParser.git external/pbrtParser ... just saying....):

  • in your CMakeLists.txt:

    add_subdirectory(external/pbrtParser EXCLUDE_FROM_ALL) include_directories(${CMAKE_PROJECT_SOURCE_DIR}/external/pbrtParser) ... target_link_libraries( pbrtParser_semantic)

  • in your source file:

    #include "pbrtParser/semantic/Scene.h" ... pbrt::semantic::Scene::SP scene = pbrt::semantic::Scene::loadFrom(fileName); ...

Optional: Use your own vector library

One of the most common issues with using other peoples' C++ libraries in graphics is that - of course! - everybody wants to use their own favorite libraries for math and vector operations.

This project internally uses a somewhat older copy of the "ospcommon" library that comes with the ospray project; however, to avoid some of the otherwise common naming clashes all of this is hidden within the actual parser and lexer implementation, and the publicly visible interface in semantic/Scene.h uses simple, plain-C structs that only specify the data layout of the vec3f, vec4i, etc classes.

As such, if you want to use your own vector classes, there are two ways of doing this:

  • Option 1: Add a default constructor to your own vector classes that will be able to construct your own types from these plain-C classes.

  • Option 2: Assuming your own vector calsses do have the same data layout as the ones that the parser expects, you can also do a

    #define PBRT_PARSER_PARSER_VECTYPE_NAMESPACE myown::math #include <pbrtParser/semantic/Scene.h>

    ... which will make your own code "think" that the parser was actually using your own vector library. This is, of course, a bit hacky, so use at your own risk.

Issues
  • base path + fix for named materials

    base path + fix for named materials

    I'm finding that I need to set a base path for relative PBRT imports to work (eg the relative triangle meshes imported in the white room scene.), so I am now exposing that basepath through the public API, and defaulting it to an empty string.

    Also am now passing names to material constructors when going from semantic -> syntactic

    opened by natevm 26
  • Fix path separator handling in basepath detection

    Fix path separator handling in basepath detection

    Previously, the assumption was that all windows paths would use \ and all others use / This broke loading of resources when supplying a pbrt loading path with / or mix of / and \

    opened by Wumpf 8
  • Instance Transforms

    Instance Transforms

    Hi again,

    I am slowly coming to terms with pbrt and your parser.

    I wish to render using GPU instances, which means I want to duplicate the PBRT structure as lean as possible.

    I have noticed that the duplication of Objects, is done via:

    AttributeBegin Transform [16] ObjectInstance "..." AttributeEnd

    Your parser/ scene proviodes objects(I have only world, yet) and object instances together with their affine transforms "xfm". Do you also provide a translation vector, or rather the entire transform matrix? or am I missing something essential?

    opened by Chrjstoph 7
  • Build error on Mac OS

    Build error on Mac OS

    Hi,

    I was trying to build pbrt-parser on Mac OS and I got the following errors:

    /Users/yangyangzhao/codes/pbrt-parser/pbrtParser/include/pbrtParser/math.h:81:37: error: expected member name or ';' after declaration specifiers explicit vec2f(float v) : x{v}, y{v} { } ^ /Users/yangyangzhao/codes/pbrt-parser/pbrtParser/include/pbrtParser/math.h:81:34: error: expected '(' explicit vec2f(float v) : x{v}, y{v} { } ^ /Users/yangyangzhao/codes/pbrt-parser/pbrtParser/include/pbrtParser/math.h:81:36: error: expected ';' after expression explicit vec2f(float v) : x{v}, y{v} { }

    I feel like I missed one vec lib dependency..

    Cheers Yangyang

    opened by zhaoyangyang316 4
  • Traversing materials, help needed

    Traversing materials, help needed

    Hi,

    I want to traverse the Materials and extract a texture.

    auto kd = pbrtMat->findParam < ??? >("Kd");

    I am struggling to identify the proper type and vs2013 is silent. kd is always empty. Stepping through the source shows that the data is available prior to the dynamic cast.

    opened by Chrjstoph 4
  • Semantic parser crashes if material refers to non-existent texture

    Semantic parser crashes if material refers to non-existent texture

    Sorry, lot's of noise :-)

    If the pbrt file erroneously refers to a non-existent texture in a material, the semantic parser simply crashes here due to Material:: getParamTexture() returning a nullptr here. You can repro with pbrt-v3-scenes/measure-one/frame120.pbrt. A possible fix would be to test if !p->texture right here and in that case throw an exception (that can be caught and maybe handled at application level).

    opened by szellmann 2
  • Default values for material properties

    Default values for material properties

    Currently the default values for the material properties (kd,ks,roughness,eta etc.) differ from the specs. Especially with the glass material this is problematic because the current default values violate the energy conservation property of the BSDF, and it seems like every pbrt-v3-scenes model just uses the default glass material. Would you accept a PR with the default values adjusted to the specs? Or would this interfere with some personal setup that you're using and don't want to be changed?

    opened by szellmann 2
  • Transform normals with inverse transpose

    Transform normals with inverse transpose

    Should this here: https://github.com/ingowald/pbrt-parser/blob/master/pbrtParser/impl/semantic/Geometry.cpp#L97 say xfmNormal() instead of xfmVector()? (I'm not a 100% sure, I just stumbled upon it while inspecting the code and haven't verified it.)

    opened by szellmann 2
  • Resolving Materialname of Shape

    Resolving Materialname of Shape

    Hi again,

    I might be too hasty, but I have problems mapping Materials to shapes.

    It appears that the Parser returns the material itself, omitting the material name, thus making nonredundant material storage ... difficult.

    I dont know which would be better, for the material to sport its own name, or the shape carying the name along side the data.

    Or am I missing something?

    opened by Chrjstoph 2
  • VS specific errors

    VS specific errors

    Hi again,

    I have begun loading of Materials and included #include "pbrt/Parser.h" in my Material header.

    Compiling fails:

    Its easily solved but maybe you want to know.

    Error 304 error C4996: 'getenv': This function or variable may be unsafe. Consider using _dupenv_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. f:\vr\pbrt-parser\ospcommon\common.h 147

    opened by Chrjstoph 2
  • Build fails on Apple MacBook Pro M1

    Build fails on Apple MacBook Pro M1

    Hello, I tried to build this on a MacBook Pro (13-inch, M1, 2020) running macOS Big Sur Version 11.2.1 but the build fails.

    git clone [email protected]:ingowald/pbrt-parser.git
    cd pbrt-parser
    mkdir build
    cd build
    
    cmake ..
    
    -- The C compiler identification is AppleClang 12.0.0.12000032
    -- The CXX compiler identification is AppleClang 12.0.0.12000032
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /Users/user/src/pbrt-parser/build
    
    make
    
    Scanning dependencies of target pbrtParser
    [  3%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/syntactic/FileMapping.cpp.o
    [  7%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/syntactic/Scene.cpp.o
    [ 10%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/Geometry.cpp.o
    [ 14%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/Camera.cpp.o
    [ 17%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/Textures.cpp.o
    [ 21%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/Materials.cpp.o
    [ 25%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/Lights.cpp.o
    [ 28%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/Scene.cpp.o
    [ 32%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/BinaryFileFormat.cpp.o
    [ 35%] Building CXX object pbrtParser/CMakeFiles/pbrtParser.dir/impl/semantic/importPBRT.cpp.o
    [ 39%] Linking CXX static library ../libpbrtParser.a
    [ 39%] Built target pbrtParser
    Scanning dependencies of target pbrtParser_shared
    [ 42%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/syntactic/FileMapping.cpp.o
    [ 46%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/syntactic/Scene.cpp.o
    [ 50%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/Geometry.cpp.o
    [ 53%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/Camera.cpp.o
    [ 57%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/Textures.cpp.o
    [ 60%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/Materials.cpp.o
    [ 64%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/Lights.cpp.o
    [ 67%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/Scene.cpp.o
    [ 71%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/BinaryFileFormat.cpp.o
    [ 75%] Building CXX object pbrtParser/CMakeFiles/pbrtParser_shared.dir/impl/semantic/importPBRT.cpp.o
    [ 78%] Linking CXX shared library ../libpbrtParser_shared.dylib
    Undefined symbols for architecture arm64:
      "pbrt::createSampler(std::__1::shared_ptr<pbrt::Scene>, std::__1::shared_ptr<pbrt::syntactic::Scene>)", referenced from:
          pbrt::importPBRT(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in importPBRT.cpp.o
      "pbrt::createIntegrator(std::__1::shared_ptr<pbrt::Scene>, std::__1::shared_ptr<pbrt::syntactic::Scene>)", referenced from:
          pbrt::importPBRT(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in importPBRT.cpp.o
      "pbrt::createPixelFilter(std::__1::shared_ptr<pbrt::Scene>, std::__1::shared_ptr<pbrt::syntactic::Scene>)", referenced from:
          pbrt::importPBRT(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in importPBRT.cpp.o
      "_ply_close", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_get_argument_element", referenced from:
          rply_vertex_callback_vec3(t_ply_argument_*) in Geometry.cpp.o
          rply_vertex_callback_vec2(t_ply_argument_*) in Geometry.cpp.o
          rply_face_callback(t_ply_argument_*) in Geometry.cpp.o
      "_ply_get_argument_property", referenced from:
          rply_face_callback(t_ply_argument_*) in Geometry.cpp.o
      "_ply_get_argument_user_data", referenced from:
          rply_vertex_callback_vec3(t_ply_argument_*) in Geometry.cpp.o
          rply_vertex_callback_vec2(t_ply_argument_*) in Geometry.cpp.o
          rply_face_callback(t_ply_argument_*) in Geometry.cpp.o
      "_ply_get_argument_value", referenced from:
          rply_vertex_callback_vec3(t_ply_argument_*) in Geometry.cpp.o
          rply_vertex_callback_vec2(t_ply_argument_*) in Geometry.cpp.o
          rply_face_callback(t_ply_argument_*) in Geometry.cpp.o
      "_ply_get_element_info", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_get_next_element", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_get_next_property", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_get_property_info", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_open", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_read", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_read_header", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
      "_ply_set_read_cb", referenced from:
          pbrt::ply::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec3f, std::__1::allocator<pbrt::math::vec3f> >&, std::__1::vector<pbrt::math::vec2f, std::__1::allocator<pbrt::math::vec2f> >&, std::__1::vector<pbrt::math::vec3i, std::__1::allocator<pbrt::math::vec3i> >&) in Geometry.cpp.o
    ld: symbol(s) not found for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make[2]: *** [libpbrtParser_shared.2.4.1.dylib] Error 1
    make[1]: *** [pbrtParser/CMakeFiles/pbrtParser_shared.dir/all] Error 2
    make: *** [all] Error 2
    
    opened by ctsrc 1
  • Support PBRTv4

    Support PBRTv4

    Currently supported format is v3, however the latest is https://github.com/mmp/pbrt-v4 It would be great if this parser was updated to handle the new format! A bit unfortunate that PBRT files themselves don't contain information about the format version.

    opened by kvark 0
  • Crashes during parsing of malformed PBRT files

    Crashes during parsing of malformed PBRT files

    Hi there,

    During fuzz testing of the pbrt parsing there were a couple crashes discovered. Although these files only crash the apps, they could potentially be crafted further into security issues where a malformed pbrt file would be able compromise the process's memory through memory corruption, so hardening the code to prevent these kinds of bugs would be great.

    You can download the crashing files in a zip from Ufile to debug and understand where the code is crashing.

    Here's a snip of one of the crash logs.

    Program received signal SIGSEGV, Segmentation fault.
    __memmove_avx_unaligned_erms ()
        at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:345
    345	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
    
    #0  __memmove_avx_unaligned_erms ()
        at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:345
    #1  0x00007ffff7f11285 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
       from /lib/x86_64-linux-gnu/libstdc++.so.6
    #2  0x000055555559ccdc in pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseParam(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) ()
    #3  0x000055555559e1b2 in pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseParams(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<pbrt::syntactic::Param>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<pbrt::syntactic::Param> > > >&) ()
    #4  0x00005555555a4e90 in pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseWorld() ()
    #5  0x00005555555a6dc7 in pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseScene() ()
    #6  0x00005555555a7914 in pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parse(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
    #7  0x000055555558f8c6 in pbrt::syntactic::Scene::parse(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
    #8  0x0000555555587ebd in pbrt::importPBRT(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
    #9  0x0000555555567af4 in pbrt::semantic::pbrtInfo(int, char**) ()
    #10 0x000055555556710d in main ()
    
    rax            0x555555600030      93824992935984
    rbx            0x7fffffffd600      140737488344576
    rcx            0x6ac04             437252
    rdx            0xe1                225
    rsi            0x7005555555ffd5f   504496983258430815
    rdi            0x555555600030      93824992935984
    rbp            0x5555555ffd30      0x5555555ffd30
    rsp            0x7fffffffd1d8      0x7fffffffd1d8
    r8             0x555555600030      93824992935984
    r9             0xff                255
    r10            0x0                 0
    r11            0x7ffff7c5dbe0      140737350327264
    r12            0xe1                225
    r13            0x555555600030      93824992935984
    r14            0x7fffffffd610      140737488344592
    r15            0x7fffffffd530      140737488344368
    rip            0x7ffff7c00749      0x7ffff7c00749 <__memmove_avx_unaligned_erms+217>
    eflags         0x10202             [ IF RF ]
    cs             0x33                51
    ss             0x2b                43
    ds             0x0                 0
    es             0x0                 0
    fs             0x0                 0
    gs             0x0                 0
    
    => 0x7ffff7c00749 <__memmove_avx_unaligned_erms+217>:	vmovdqu (%rsi),%ymm0
       0x7ffff7c0074d <__memmove_avx_unaligned_erms+221>:	
        vmovdqu 0x20(%rsi),%ymm1
       0x7ffff7c00752 <__memmove_avx_unaligned_erms+226>:	
        vmovdqu 0x40(%rsi),%ymm2
       0x7ffff7c00757 <__memmove_avx_unaligned_erms+231>:	
        vmovdqu 0x60(%rsi),%ymm3
    
    'exploitable' version 1.32
    Linux ubuntu 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 x86_64
    Signal si_signo: 11 Signal si_addr: 0
    Nearby code:
       0x00007ffff7c0072d <+189>:	ja     0x7ffff7c0069f <__memmove_avx_unaligned_erms+47>
       0x00007ffff7c00733 <+195>:	cmp    rdx,0x100
       0x00007ffff7c0073a <+202>:	ja     0x7ffff7c007d1 <__memmove_avx_unaligned_erms+353>
       0x00007ffff7c00740 <+208>:	cmp    rdx,0x80
       0x00007ffff7c00747 <+215>:	jb     0x7ffff7c007a3 <__memmove_avx_unaligned_erms+307>
    => 0x00007ffff7c00749 <+217>:	vmovdqu ymm0,YMMWORD PTR [rsi]
       0x00007ffff7c0074d <+221>:	vmovdqu ymm1,YMMWORD PTR [rsi+0x20]
       0x00007ffff7c00752 <+226>:	vmovdqu ymm2,YMMWORD PTR [rsi+0x40]
       0x00007ffff7c00757 <+231>:	vmovdqu ymm3,YMMWORD PTR [rsi+0x60]
       0x00007ffff7c0075c <+236>:	vmovdqu ymm4,YMMWORD PTR [rsi+rdx*1-0x20]
    
    Stack trace:
    #  0 __memmove_avx_unaligned_erms at 0x7ffff7c00749 in /usr/lib/x86_64-linux-gnu/libc-2.31.so (BL)
    #  1 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at 0x7ffff7f11285 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
    #  2 pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseParam(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) at 0x55555559ccdc in pbrt-parser/build/pbrtInfo
    #  3 pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseParams(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<pbrt::syntactic::Param>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<pbrt::syntactic::Param> > > >&) at 0x55555559e1b2 in pbrt-parser/build/pbrtInfo
    #  4 pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseWorld() at 0x5555555a4e90 in pbrt-parser/build/pbrtInfo
    #  5 pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parseScene() at 0x5555555a6dc7 in pbrt-parser/build/pbrtInfo
    #  6 pbrt::syntactic::BasicParser<pbrt::syntactic::MappedFile>::parse(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at 0x5555555a7914 in pbrt-parser/build/pbrtInfo
    #  7 pbrt::syntactic::Scene::parse(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at 0x55555558f8c6 in pbrt-parser/build/pbrtInfo
    #  8 pbrt::importPBRT(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at 0x555555587ebd in pbrt-parser/build/pbrtInfo
    #  9 pbrt::semantic::pbrtInfo(int, char**) at 0x555555567af4 in pbrt-parser/build/pbrtInfo
    # 10 main at 0x55555556710d in pbrt-parser/build/pbrtInfo
    Faulting frame: #  1 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at 0x7ffff7f11285 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
    
    Description: Access violation
    Short description: AccessViolation (21/22)
    Hash: 41700d7a55320c3aca7f6b05969902fe.625993d40956e3876a160cd881311244
    Exploitability Classification: UNKNOWN
    Explanation: The target crashed due to an access violation but there is not enough additional information available to determine exploitability.
    

    Thanks!

    opened by retpoline 0
  • Semantic parser seg faults if the input file contains an unsupported shape type

    Semantic parser seg faults if the input file contains an unsupported shape type

    Hi,

    There is a seg fault in SemanticParser::findOrCreateShape when the input file contains an unsupported shape type.

    SemanticParser::emitShape returns a null pointer for any shape type that isn't supported yet. SemanticParser::findOrCreateShape calls emitShape then tries to set the reverseOrientation property on the returned pointer without checking whether it was null. That's where we get a seg fault if the shape type was not supported.

    This means the semantic parser can't load some of the example scenes from https://pbrt.org/scenes-v3.html. For example the Contemporary Bathroom scene: this contains some loopsubdiv shapes, which make the parser crash.

    Hope that helps.

    opened by vilya 0
  • pbrt::syntactic::Object and others does not have virtual destructor

    pbrt::syntactic::Object and others does not have virtual destructor

    For example, Object has virtual methods but no virtual destructor.

    https://github.com/ingowald/pbrt-parser/blob/master/pbrtParser/include/pbrtParser/Scene.h#L1003

    Clang(9.0) with increased warnings(e.g. -Wall) reports an warning about it:

    Here is minimal reproducible code based on pbrtParser:

    // clang++ -Wall -std=c++11 
    #include <string>
    #include <memory>
    #include <cstdio>
    #include <cstdlib>
    #include <iostream>
    #include <map>
    
    struct Object
    {
      Object(const std::string &name) : name_(name) {}
    
      // virtual ~Object() { std::cout << "dtor" << "\n"; }
    
      std::string name_;
    
      virtual std::string toString(const int depth = 0) const;
    
    };
    
    struct Scene
    {
      Scene() : o(std::make_shared<Object>("<root>")) {
      }
    
      virtual ~Scene();
    
      std::shared_ptr<Object> o;
    };
    
    Scene::~Scene() {
      o = nullptr;
    }
    
    int main(int argc, char **argv)
    {
      (void)argc;
      (void)argv;
    
      std::map<std::string, std::shared_ptr<Scene>> pmap;
    
      pmap["root"] = std::make_shared<Scene>();
    
      std::cout << "name = " << pmap["root"]->o->name_ << "\n";
    
      return 0;
    }
    
    
    In file included from main.cc:1:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/string:41:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/allocator.h:46:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9/bits/c++allocator.h:33:
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:153:4: warning: destructor called on non-final 'Object' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
            { __p->~_Up(); }
              ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/alloc_traits.h:497:8: note: in instantiation of function template specialization '__gnu_cxx::new_allocator<Object>::destroy<Object>' requested here
            { __a.destroy(__p); }
                  ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:557:28: note: in instantiation of function template specialization 'std::allocator_traits<std::allocator<Object> >::destroy<Object>' requested here
            allocator_traits<_Alloc>::destroy(_M_impl._M_alloc(), _M_ptr());
                                      ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:543:2: note: in instantiation of member function 'std::_Sp_counted_ptr_inplace<Object, std::allocator<Object>, __gnu_cxx::_S_atomic>::_M_dispose' requested here
            _Sp_counted_ptr_inplace(_Alloc __a, _Args&&... __args)
            ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:680:6: note: in instantiation of function template specialization 'std::_Sp_counted_ptr_inplace<Object, std::allocator<Object>, __gnu_cxx::_S_atomic>::_Sp_counted_ptr_inplace<char const (&)[7]>' requested here
                _Sp_cp_type(__a._M_a, std::forward<_Args>(__args)...);
                ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:1344:14: note: in instantiation of function template specialization 'std::__shared_count<__gnu_cxx::_S_atomic>::__shared_count<Object, std::allocator<Object>, char const (&)[7]>' requested here
            : _M_ptr(), _M_refcount(_M_ptr, __tag, std::forward<_Args>(__args)...)
                        ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr.h:359:4: note: in instantiation of function template specialization 'std::__shared_ptr<Object, __gnu_cxx::_S_atomic>::__shared_ptr<std::allocator<Object>, char const (&)[7]>' requested here
            : __shared_ptr<_Tp>(__tag, std::forward<_Args>(__args)...)
              ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr.h:701:14: note: in instantiation of function template specialization 'std::shared_ptr<Object>::shared_ptr<std::allocator<Object>, char const (&)[7]>' requested here
          return shared_ptr<_Tp>(_Sp_alloc_shared_tag<_Alloc>{__a},
                 ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr.h:717:19: note: in instantiation of function template specialization 'std::allocate_shared<Object, std::allocator<Object>, char const (&)[7]>' requested here
          return std::allocate_shared<_Tp>(std::allocator<_Tp_nc>(),
                      ^
    main.cc:22:20: note: in instantiation of function template specialization 'std::make_shared<Object, char const (&)[7]>' requested here
      Scene() : o(std::make_shared<Object>("<root>")) {
                       ^
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:153:10: note: qualify call to silence this warning
            { __p->~_Up(); }
    

    https://stackoverflow.com/questions/10024796/c-virtual-functions-but-no-virtual-destructors

    No explicit virtual destructor may give unexpected behavior, so it would be better to define virtual destructor explicitly as done in Shape:

    https://github.com/ingowald/pbrt-parser/blob/75c7b21decb507f7173755e6931bf368c9ec8593/pbrtParser/include/pbrtParser/Scene.h#L741

    ( Same modification may be required to other classes(e.g. Camera)) )

    opened by syoyo 2
  • Missing support for most light types, like spot & point lights

    Missing support for most light types, like spot & point lights

    .. as used in for example in villa-lights-on.pbrt which is part of git://git.pbrt.org/pbrt-v3-scenes Looking at the code actually it seems only infinite and distant are supported, but there are many more: https://www.pbrt.org/fileformat-v3.html#lights

    opened by Wumpf 0
Releases(v1.1)
Owner
Ingo Wald
Ingo Wald
A Windows Shell Extension for the Pixar USD file format.

Activision USD Shell Extension A Windows Shell Extension for the Pixar USD file format. Windows Explorer Features Hydra Realtime Preview Thumbnails Co

null 288 Jun 24, 2022
Elf and PE file parser

PelfParser PelfParser is a very simple C++ library for parsing Windows portable executable files and Executable and Linkable Format files, it only sup

Rebraws 1 Oct 29, 2021
A parser for InnoDB file formats

Introduction Inno_space is a parser for InnoDB file formats. It parse the .ibd file to human readable format. The origin idea come from Jeremy Cole's

Zongzhi Chen 76 Jun 20, 2022
A simple header-only C++ argument parser library. Supposed to be flexible and powerful, and attempts to be compatible with the functionality of the Python standard argparse library (though not necessarily the API).

args Note that this library is essentially in maintenance mode. I haven't had the time to work on it or give it the love that it deserves. I'm not add

Taylor C. Richberger 981 Jun 23, 2022
A simple to use, composable, command line parser for C++ 11 and beyond

Clara v1.1.5 !! This repository is unmaintained. Go here for a fork that is somewhat maintained. !! A simple to use, composable, command line parser f

Catch Org 651 Jun 15, 2022
CLI11 is a command line parser for C++11 and beyond that provides a rich feature set with a simple and intuitive interface.

CLI11: Command line parser for C++11 What's new • Documentation • API Reference CLI11 is a command line parser for C++11 and beyond that provides a ri

null 2.1k Jun 24, 2022
A simple to use, composable, command line parser for C++ 11 and beyond

Lyra A simple to use, composing, header only, command line arguments parser for C++ 11 and beyond. Obtain License Standards Stats Tests License Distri

Build Frameworks Group 342 Jun 16, 2022
A simple header-only C++ argument parser library. Supposed to be flexible and powerful, and attempts to be compatible with the functionality of the Python standard argparse library (though not necessarily the API).

args Note that this library is essentially in maintenance mode. I haven't had the time to work on it or give it the love that it deserves. I'm not add

Taylor C. Richberger 896 Aug 31, 2021
⛳ Simple, extensible, header-only C++17 argument parser released into the public domain.

⛳ flags Simple, extensible, header-only C++17 argument parser released into the public domain. why requirements api get get (with default value) posit

sailormoon 198 Jun 18, 2022
JSONes - c++ json parser & writer. Simple api. Easy to use.

JSONes Just another small json parser and writer. It has no reflection or fancy specs. It is tested with examples at json.org Only standart library. N

Enes Kaya ÖCAL 2 Dec 28, 2021
Lightweight C++ command line option parser

Release versions Note that master is generally a work in progress, and you probably want to use a tagged release version. Version 3 breaking changes I

null 3.1k Jun 25, 2022
Argument Parser for Modern C++

Highlights Single header file Requires C++17 MIT License Quick Start Simply include argparse.hpp and you're good to go. #include <argparse/argparse.hp

Pranav 1.2k Jun 28, 2022
Tiny command-line parser for C / C++

tinyargs Another commandline argument parser for C / C++. This one is tiny, source only, and builds cleanly with -Wall -pedantic under C99 and C++11 o

Erik Agsjö 5 Nov 11, 2021
A math parser made in 1 hour using copilot.

An entire math parser made with Copilot Copilot wrote 91% of the code in this, amazing isn't it? It supports all normal mathematical expressions excep

Duckie 4 Dec 7, 2021
An extremely fast FEC filing parser written in C

FastFEC A C program to stream and parse FEC filings, writing output to CSV. This project is in early stages but works on a wide variety of filings and

The Washington Post 51 May 26, 2022
SimPle SHell - minimalist Unix interactive shell written in a single C file

SimPle SHell - minimalist Unix interactive shell written in a single C file. The shell does not support scripting yet and is in an early stage of development. If you notice any bug, please open an issue on github.

sewe2000 2 Oct 24, 2021
n³ The unorthodox terminal file manager

n³ The unorthodox terminal file manager

Mischievous Meerkat 14.3k Jun 28, 2022
The KISS file manager: CLI-based, ultra-lightweight, lightning fast, and written in C

CliFM is a CLI-based, shell-like (non-curses) and KISS terminal file manager written in C: simple, fast, and lightweight as hell

leo-arch 511 Jun 24, 2022
Yori is a CMD replacement shell that supports backquotes, job control, and improves tab completion, file matching, aliases, command history, and more.

Yori is a CMD replacement shell that supports backquotes, job control, and improves tab completion, file matching, aliases, command history, and more.

Malcolm Smith 1.1k Jun 21, 2022