Pharos Static Binary Analysis Framework

Overview

Pharos Static Binary Analysis Framework

The Pharos static binary analysis framework is a project of the Software Engineering Institute at Carnegie Mellon University. The framework is designed to facilitate the automated analysis of binary programs. It uses the ROSE compiler infrastructure developed by Lawrence Livermore National Laboratory for disassembly, control flow analysis, instruction semantics, and more. This software is released under a BSD license.

The current distribution is a substantial update to the previous version, and adds a variety of features including improvements to the OOAnalyzer tool, experimental path analysis code, partitioner improvements, multi-threading, and many other smaller features.

The Pharos framework is a research project, and the code is undergoing active development. No warranties of fitness for any purpose are provided. While this release provides build instructions, unit tests, and some documentation, much work remains to be done. We've tested a few select build configurations, but we have not actively tested the portability of the source code. See the installation instructions for more details.

Since the primary objective for releasing this code is to provide transparency into our research and stimulate conversation with other binary static analysis researchers, please feel free to contact Cory Cohen [email protected] with questions you may have about this work. I may be unable to respond in a timely manner, but I will do my best.

Pharos Static Binary Analysis Tools

APIAnalyzer

ApiAnalyzer is a tool for finding sequences of API calls with the specified data and control relationships. This capability is intended to be used to detect common operating system interaction paradigms like opening a file, writing to it, and the closing it.

OOAnalyzer

OOAnalyzer is a tool for the analysis and recovery of object oriented constructs. This tool was the subject of a paper titled "Using Logic Programming to Recover C++ Classes and Methods from Compiled Executables" which was published at the ACM Conference on Computer and Communications Security in 2018. The tool identifies object members and methods by tracking object pointers between functions in the program. A previous implementation of this tool was named "Objdigger", but it was renamed to reflect a substantial redesign using Prolog rules to recover the object attributes. For more detailed instructons on how to run OOAnalyzer on very large executables, see these notes.

The Pharos distribution used to include a plugin that imported OO information exported by OOAnalayzer into the Ghidra reverse engineering tool set. To get that functionality now and in the future, install the Kaiju Ghidra plugin, which includes the functionality that was provided by the OOAnalayzer plugin.

CallAnalyzer

CallAnalyzer is a tool for reporting the static parameters to API calls in a binary program. It is largely a demonstration of our current calling convention, parameter analysis, and type detection capabilities, although it also provides useful analysis of the code in a program.

FN2Yara

FN2Yara is a tool to generate YARA signatures for matching functions in an executable program. Programs that share significant numbers of functions are are likely to have behavior in common.

FN2Hash

FN2Hash is tool for generating a variety of hashes and other descriptive properties for functions in an executable program. Like FN2Yara it can be used to support binary similarity analysis, or provide features for machine learning algorithms.

DumpMASM

DumpMASM is a tool for dumping disassembly listings from an executable using the Pharos framework in the same style as the other tools. It has not been actively maintained, and you should consider using ROSE's standard recursiveDisassemble instead http://rosecompiler.org/ROSE_HTML_Reference/rosetools.html.

Issues
  • Initial sanity check failed : Contradictory information about constructor

    Initial sanity check failed : Contradictory information about constructor

    This could be related / a duplicate of #139 and #136 but I prefer to report.
    While writting this report, I noticed there is a more up to date docker image than tag 9477bc817743 from last week that I used, so I pulled it and launch the command again ( and is running while you read this message).

    image

    The command line I used did no use --no-semantics or --partitioner=rose so this is what makes me doubt about the duplicate thing, moreover my thoughts were that it was fixed for more than 1 week.
    ooanalyzer --serialize=serialize.ser --timing --verbose 5 --per-function-maximum-memory 6 --per-function-timeout 2000 --maximum-memory 360000 --new-method 0x8D98A0 --delete-method 0x8DBD90 --threads=8 --timeout 2000000 --json my_program.json my_program.exe

    I prefer not to attach the binary here, but I can send it by mail if you provide one.

    For the record > There IS a constructor at 0x402d40.
    I can run with -debug as said in the message if you ask, but I don't know where to put it, please elaborate if needed.

    Here are the facts, split in two and renamed to .txt for github's sake, comming from a previous run with the exact same parameters, but --prolog-facts, ofc.
    my_program.pl.1.txt my_program.pl.2.txt

    bug prolog 
    opened by Phylante 95
  • Error in Ghidra plugin, after importing the classes

    Error in Ghidra plugin, after importing the classes

    I have successfully analyzed a very large project with about 9000 classes with the OOAnalyzer. Also I was able to create the JSON file for the import to Ghidra without any problems and import at least the first part of it.

    The first steps work without problems, but when the classes are imported and then assigned in the second step, the first thing you get is the success message that the 9000 classes have been imported.

    In the background there are still some tasks running which seem to finish something. In any case, I can't save the project, because I always get the following message (can sometimes be more than one task, as shown in this example)

    image

    And then suddenly, out of nowhere, the following hard error appears.

    An unexpected error occurred while processing the command: OOAnalyzer Import java.lang.IllegalArgumentException: Data type TestClassT can't contain itself. at ghidra.program.database.data.CompositeDB.checkAncestry(CompositeDB.java:265) at ghidra.program.database.data.StructureDB.doReplaceWithUnaligned(StructureDB.java:1194) at ghidra.program.database.data.StructureDB.doReplaceWith(StructureDB.java:1133) at ghidra.program.database.data.DataTypeManagerDB.createStructure(DataTypeManagerDB.java:2160) at ghidra.program.database.data.DataTypeManagerDB.createDataType(DataTypeManagerDB.java:2099) at ghidra.program.database.data.DataTypeManagerDB.resolveNoSourceDataType(DataTypeManagerDB.java:847) at ghidra.program.database.data.DataTypeManagerDB.resolve(DataTypeManagerDB.java:735) at ghidra.program.database.data.DataTypeManagerDB.addDataType(DataTypeManagerDB.java:1005) at ooanalyzer.OOAnalyzer.updateTypeManager(OOAnalyzer.java:1419) at ooanalyzer.OOAnalyzer.analyzeClasses(OOAnalyzer.java:434) at ooanalyzer.OOAnalyzerGhidraPlugin$ImportCommand.cmdConfigureAndExecute(OOAnalyzerGhidraPlugin.java:166) at ooanalyzer.OOAnalyzerGhidraPlugin$ImportCommand.applyTo(OOAnalyzerGhidraPlugin.java:99) at ghidra.framework.plugintool.mgr.BackgroundCommandTask.run(BackgroundCommandTask.java:101) at ghidra.framework.plugintool.mgr.ToolTaskManager.run(ToolTaskManager.java:315) at java.base/java.lang.Thread.run(Thread.java:835)


    Build Date: 2020-Feb-12 1149 EST Ghidra Version: 9.1.2 Java Home: C:\Tools\openjdk-12.0.2_windows-x64_bin\jdk-12.0.2 JVM Version: Oracle Corporation 12.0.2 OS: Windows 10 10.0 amd64 Workstation: Demo

    But the tasks are still executed in the background, and everything imported, at least the function assignments or class assignments of the second step are removed. Seems to be some kind of rollback, and only affects the second step, but according to the message "xyz classes have been successfully imported" it should already be completed.

    But I can neither save the project, see the picture above, nor somehow find out which tasks are responsible for it in the background.

    To my regret I can't provide the original file or the generated JSON file here. Nevertheless I hope that the bug can be fixed somehow.

    For the import to Ghidra I used the following version of the plugin: ghidra_9.1.2_PUBLIC_20200718_OOAnalyzerPlugin

    bug plugins 
    opened by 0xBEEEF 61
  • Problems with simple multiple inheritance, without special cases

    Problems with simple multiple inheritance, without special cases

    I have taken a small example from the Internet, and slightly adapted it. There are two basic classes and one class that inherits from both. There are no special cases. Thus an example from everyday life. It must occur a thousand times.

    What are the problems with this?

    Well, let's say the main problems in this case are at the VFTables. Since the derived class has both VFTables, they are of course initialized and set via the structure. This is what I thought at first. Until I took a closer look and realized that unfortunately this is not the case. The first VFTable is recognized completely and without problems.

    image

    The second one, which Ghidra can also correctly assign, is just a normal member according to the analysis.

    image

    No distinction is made here.

    image

    Furthermore, none of the functions were resolved anywhere in the code for the second VFTable, such as here in the Main. In Ghidra I see clearly that a function is called via a memory address.

    image

    In my eyes there seems to be a general problem with multiple inheritance.

    bug 
    opened by 0xBEEEF 29
  • SQLite concurrency problem in OOAnalyzer

    SQLite concurrency problem in OOAnalyzer

    Short version: Step two now completed successfully, step three as well:

    Guessed 1,324 methods of 4,492, guessed contrary conclusions: 0 of 0
    Guessed 372 constructors of 674, guessed contrary conclusions: 1,653 of 3,813
    Guessed 14 destructors of 250, guessed contrary conclusions: 0 of 3,420
    Guessed 8 deleting destructors of 199, guessed contrary conclusions: 0 of 5,185
    Guessed 12 virtual function calls of 381, guessed contrary conclusions: 4 of 4
    Guessed 126 virtual function tables of 539, guessed contrary conclusions: 11 of 6,186
    Guessed 8 virtual base tables of 8, guessed contrary conclusions: 0 of 0
    Guessed 413 virtual function table entries of 5,026, guessed contrary conclusions: 1 of 490
    Guessed 173 derived classes of 483, guessed contrary conclusions: 0 of 132
    Guessed 28 embedded objects of 132, guessed contrary conclusions: 0 of 2,323
    Guessed 40 has a base class of 461, guessed contrary conclusions: 2,511 of 2,535
    Object oriented analysis and reporting complete, exiting.
    

    Long story: Just to be sure, I wanted to start from scratch; step one and two don't take that long anyways.

    Initially, I could not get past step two:

    ooanalyzer --serialize=ooprog.ser --maximum-memory 250000 --prolog-facts=ooprog-facts.pl --threads=128 --per-function-timeout=900000 TS2.exe &> 02-analysis.txt ended in

    [INFO ]: Pharos main error: (pharos::SQLLiteApiDictionary::SQLError) bad parameter or other API misuse
    OOAN[FATAL]: Backtrace:
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(+0x942b67) [0x7ff77c688b67]
    OOAN[FATAL]: | /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7ff7752e128c]
    OOAN[FATAL]: | /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7ff7752e12f7]
    OOAN[FATAL]: | /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae558) [0x7ff7752e1558]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::SQLLiteApiDictionary::Data::throw_error(int)+0x50) [0x7ff77c6687ba]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(+0x9208de) [0x7ff77c6668de]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::SQLLiteApiDictionary::Data::get_api_definition(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x2ab) [0x7ff77c88501b]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::SQLLiteApiDictionary::get_api_definition(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x26) [0x7ff77c8850f6]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::DLLStateCacheDict::get_api_definition(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x9a) [0x7ff77c8819ca]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(std::vector<std::shared_ptr<pharos::APIDefinition const>, std::allocator<std::shared_ptr<pharos::APIDefinition const> > > pharos::ReportDictionary::get_definition<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x75) [0x7ff77c898e25]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::ReportDictionary::get_api_definition(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x2f) [0x7ff77c8875df]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::MultiApiDictionary::get_api_definition(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x51) [0x7ff77c8815d1]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(std::vector<std::shared_ptr<pharos::APIDefinition const>, std::allocator<std::shared_ptr<pharos::APIDefinition const> > > pharos::ReportDictionary::get_definition<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x75) [0x7ff77c898e25]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::ReportDictionary::get_api_definition(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x2f) [0x7ff77c8875df]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(pharos::OOAnalyzer::visit(pharos::FunctionDescriptor*)+0x8fc) [0x7ff77c9d0f8c]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(+0xbab42d) [0x7ff77c8f142d]
    OOAN[FATAL]: | /usr/local/bin/../lib/libpharos.so(+0xbab720) [0x7ff77c8f1720]
    OOAN[FATAL]: | /lib/x86_64-linux-gnu/libboost_thread.so.1.74.0(+0x150cb) [0x7ff774c2d0cb]
    OOAN[FATAL]: | /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7ff77507db43]
    ooanalyzer: /usr/include/boost/thread/pthread/mutex.hpp:70: void boost::mutex::unlock(): Assertion `!posix::pthread_mutex_unlock(&m)' failed.
    

    How does a change in the prolog rules make SQLite fall on its face? To be extra sure, I just rebuilt the docker container with the Dockerfile from your branch, as I just swapped rules.pl in the official container in my first attempt; just on the off chance that some information from the .pl file ends up compiled into the main program.... But I got the same error.

    Then I tried adding --verbose 5 in the hopes of getting anything useful, but to my surprise, the run completed successfully now. Other workarounds are to get rid of my excessive per-function timeout (tried 90, resulting in 2397 timeouts in log), or lowering the thread count to 64. Looks like a completely unrelated concurrency issue somewhere that got triggered just now. Is it worth opening another issue for this?

    Originally posted by @srett in https://github.com/cmu-sei/pharos/issues/227#issuecomment-1179720435

    bug 
    opened by sei-eschwartz 28
  • reasonClassSizeGTE_D is buggy

    reasonClassSizeGTE_D is buggy

    ooprolog --facts DK2.facts.pl --results DK2.results.pl <
    
    Consistency checks failed.
    insanityClassSizeInvalid failed: Class=0x66eca4 LTESize=0x84 GTESize=0xb0
    Initial sanity check failed, indicating the OO rules are incorrect.
    Please report this failure to the Pharos developers!
      [29] prolog_stack:get_prolog_backtrace(100,[frame(29,clause(<clause>(0x55fc298618b0),6),_24981518)|_24981506],[goal_term_depth(100)]) at /usr/local/lib/swipl/library/prolog_stack.pl:134
      [28] throw_with_backtrace(error(system_error(initialSanityChecks))) at /usr/local/share/pharos/prolog/oorules/util.pl:185
      [26] solve_internal at /usr/local/share/pharos/prolog/oorules/setup.pl:659
    

    Facts

    Even when using the purecall(0x634d60). hack from #139. Still the same testbed.

    bug prolog fix pending merge 
    opened by Trass3r 27
  • How can I extend the API database?

    How can I extend the API database?

    Hi, I got a warning: APID[WARN ]: API database has no data for DLL: MFC42

    So my question is how I would add data for that? Is it as simple as using ooanalyzer on that dll and grabbing the json and putting it somewhere? Some other way of integrating perhaps publicly available information on MFC?

    Thanks in advance,

    Jan

    enhancement 
    opened by janbbeck 23
  • Enhancement: Ghidra plugin cannot match methods on rebased project

    Enhancement: Ghidra plugin cannot match methods on rebased project

    I wanted to suggest an enhancemet for OOA Ghidra plugin:

    • My ghidra project is rebased, meaning I have base address set to 0x0. After I import .json, member functions (methods) are missing.
    • If I change base back, import works better, but there's another issue. None of the functions I renamed (dozens and dozens of them) are matched by the plugin to a member function. I will have to re-start my project, which is worth it, but it feels like something that can be fixed.

    Thanks for the consideration and amazing tools.

    enhancement plugins 
    opened by TheIronWolfModding 22
  • Sanity check failure during Prolog Analysis : Contradictory information about merging classes

    Sanity check failure during Prolog Analysis : Contradictory information about merging classes

    Tail of log is:

    reasonForwardAsManyTimesAsPossible complete. Entering stage 'Initial reasoning complete'. failed. Consistency checks failed. Contradictory information about merging classes: Method1=0x102f2528 Method2=0x102f24e4 Initial sanity check failed, indicating the OO rules are incorrect. Please report this failures to the Pharos developers!

    If you would like the facts.pl to reproduce the failure, please contact me.

    bug prolog 
    opened by gwgill 19
  • ooJson finds duplicate keys

    ooJson finds duplicate keys

    I have some large programs i am using ooanalyzer on, using the multi-step instructions.

    They successfully complete inferencing, but when i extract with ^final and run oojson on the results, it finds duplicate keys:

    ERROR: -g ['/root/pharos/share/prolog/oorules/oojson.P'],exportJSON('foo-output.P'),halt.: Duplicate key: '[email protected][email protected]@[email protected]@@[email protected]@'
    ERROR: In:
    ERROR:   [12] dict_create(_138,structures,[cls_0x4103b0: ...,...|...])
    ERROR:   [10] exportJSON('foo-output.P') at /root/pharos/share/prolog/oorules/oojson.P:383
    ERROR:    [9] '<meta-call>'('<garbage_collected>') <foreign>
    ERROR:    [8] catch(user:(...,...),error(duplicate_key('[email protected][email protected]@[email protected]@@[email protected]@'),context(...,_286)),'$toplevel':true) at /usr/local/lib/swipl/boot/init.pl:482
    ERROR:    [7] catch_with_backtrace('<garbage_collected>',error(duplicate_key('[email protected][email protected]@[email protected]@@[email protected]@'),context(...,_342)),'<garbage_collected>') at /usr/local/lib/swipl/boot/init.pl:532
    ERROR:
    ERROR: Note: some frames are missing due to last-call optimization.
    ERROR: Re-run your program in debug mode (:- debug.) to get more detail.
    

    Running in debug mode, it is this function:

    makeAllStructuresJson(Json):-
         % Get a list of Class IDs and sort them by the class ID.
         findall(ClsId, finalClass(ClsId, _, _, _, _, _), ClassIDList),
         sort(0, @=<, ClassIDList, SortedClassIDList),
         bagof(NameKey:ClsJson, makeOneClassInOrder(SortedClassIDList, NameKey, ClsJson), KVPairs),
         dict_create(Json, structures, KVPairs).
    

    From what i can tell, what happens is that we get multiple finalClass entries with the same first argument in the output:

    finalClass(0xa5d638, 0xa5d638, 0x48, 0x48, 0, [0x4532e0, 0x453350, 0x453490, 0x4534b0, 0x4534d0, 0x4538f0, 0x453bf0, 0x453d60, 0x453d70, 0x453d80, 0x454d10, 0x455e90, 0x455f10]).
    ...
    
    finalClass(0xa5d638, 0xa5d638, 0x40, 0x40, 0, [0x4564e0, 0x456600, 0x456690, 0x4569e0, 0x456ad0, 0x456f10, 0x457280]).
    

    This in turn makes it very unhappy because when it pulls the names out, it ends up with the same names (generating duplicate keys)

    bug prolog 
    opened by dberlin 19
  • Member class throwing when calling mbr.getOffset() from OOAnalyzer.java.

    Member class throwing when calling mbr.getOffset() from OOAnalyzer.java.

    EDIT: I got it working by deleting an entry/class in the json file. See my comment below.

    I have my .json file that I got by running OOanalyzer. When I import it into the tool in Ghidra it throws an exception: java.lang.NumberFormatException: Sign character in wrong position

    Interestingly it throwns on a different member each time I run the tool. For example here it throws on <lambda_1c7d96af0f0b63f6b7070c698dcc1b90> at offset 8

    the previous time I've tried to run it's thrown at: 0x706bd0 0x7a02c0 0xa037ec 0x9d2548 0x99db20 0x8785b0

    the .json file is approximately 150000 lines long on a 8MB program with ~ 18000 functions.

    Interestingly enough I got it to run correctly in Ghidra after deleting ~half of the lines in the .json file. But I would like all classes imported if possible.

    The relevant log is:

    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Adding member [name=mbr_0x828, type=, struc=, parent=false, offset=0x828, size=4] to class cls_0x4268bc at offset 2088
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 0 0 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 1 1 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 2 2 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 3 3 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Adding member [name=mbr_0x0, type=, struc=, parent=false, offset=0x0, size=4] to class <lambda_1c7d96af0f0b63f6b7070c698dcc1b90> at offset 0
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 1 4 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 2 5 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 3 6 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 4 7 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Adding member [name=mbr_0x4, type=, struc=, parent=false, offset=0x4, size=4] to class <lambda_1c7d96af0f0b63f6b7070c698dcc1b90> at offset 4
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 2 8 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 3 9 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 4 10 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Clearing component 5 11 undefined 1 null null
    2022-01-03 16:27:56 DEBUG (OOAnalyzer) Adding member [name=mbr_0x8, type=, struc=, parent=false, offset=0x8, size=4] to class <lambda_1c7d96af0f0b63f6b7070c698dcc1b90> at offset 8
    2022-01-03 16:27:56 ERROR (BackgroundCommandTask) Command Failure: An unexpected error occurred while processing the command: OOAnalyzer Import java.lang.NumberFormatException: Sign character in wrong position at java.base/java.lang.Integer.decode(Integer.java:1425) at kaiju.tools.ooanalyzer.jsontypes.Member.getOffset(Member.java:99) at kaiju.tools.ooanalyzer.OOAnalyzer.analyzeMembers(OOAnalyzer.java:1310) at kaiju.tools.ooanalyzer.OOAnalyzer.lambda$analyzeClasses$7(OOAnalyzer.java:453) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.WhileOps$1$1.accept(WhileOps.java:99) at java.base/java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1785) at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127) at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at kaiju.tools.ooanalyzer.OOAnalyzer.analyzeClasses(OOAnalyzer.java:447) at kaiju.tools.ooanalyzer.OOAnalyzerGhidraPlugin$ImportCommand.cmdConfigureAndExecute(OOAnalyzerGhidraPlugin.java:201) at kaiju.tools.ooanalyzer.OOAnalyzerGhidraPlugin$ImportCommand.applyTo(OOAnalyzerGhidraPlugin.java:134) at ghidra.framework.plugintool.mgr.BackgroundCommandTask.run(BackgroundCommandTask.java:102) at ghidra.framework.plugintool.mgr.ToolTaskManager.run(ToolTaskManager.java:319) at java.base/java.lang.Thread.run(Thread.java:829)

    2022-01-03 16:27:56 ERROR (ToolTaskManager) OOAnalyzer Import Failed: Unspecified error occurred.
    2022-01-03 16:27:56 DEBUG (ToolTaskManager) Mon Jan 03 16:27:56 CST 2022 Background processing complete (255.141 secs) -=============================================================-

    The relevant code is:

    OOAnalyzer.java on line 1310 1293 private void analyzeMembers(OOAnalyzerClassType ooaType, Structure ghidraType) { 1294
    1295 if (!classSymbolMap.containsKey(ghidraType)) { 1296 Msg.warn(this, "Skipping " + ooaType.getName() + " because there is no associated symbol."); 1297 Msg.warn(this, "The Ghidra type was " + ghidraType.getName()); 1298 return; 1299 } 1300 1301 // If there are members for this type, process them 1302 Collection members = ooaType.getMembers(); 1303 for (Member mbr : members) { 1304 1305 if (mbr.getBase()) { 1306 Msg.trace (this, "Skipping member " + mbr.getName() + " because it was defined on the base class."); 1307 continue; 1308 } 1309 1310 if (mbr.getOffset() == Member.INVALID_OFFSET) { 1311 Msg.error(this, "Cannot add member " + mbr.getName() + " due to invalid offset"); 1312 continue; 1313 }

    Member.java 98 public Integer getOffset() { 99 return Integer.decode (offset); 100 }

    Perhaps some exception handling could be added to the getOffset() method so that it returns INVALID_OFFSET instead of throwing.

    bug prolog 
    opened by Jonathan-Greve 18
  • Memory limits

    Memory limits

    I am having a hard time getting ooanalyzer to finish.

    This is what i get:

    [28839601.841345] Out of memory: Kill process 30340 (ooanalyzer) score 736 or sacrifice child [28839601.842291] Killed process 30340 (ooanalyzer) total-vm:18192656kB, anon-rss:13268556kB, file-rss:4kB

    This is how i execute it:

    ooanalyzer --json result.json -f Sample.exe --serialize Sample.exe.rose --maxmem 4096 --relmaxmem 500

    How am i supposed to use these memory limits? They do not seem to work.

    enhancement 
    opened by rokups 18
  • DKII consistency failures

    DKII consistency failures

    Not sure if this is still related to the original problem but it still doesn't run through:

    Contradictory information about constructor: factConstructor(0x5de050) but reasonNOTConstructor(0x5de050)
    Constraint checks failed, retracting guess!
    failed.
    Consistency checks failed.
    Contradictory information about constructor: factConstructor(0x600400) but reasonNOTConstructor(0x600400)
    Constraint checks failed, retracting guess!
    tryBinarySearch completely failed on [0x5de050] and will now backtrack to fix an upstream problem.
    Refusing to backtrack into reasoningLoop to fix an upstream problem because backtrackForUpstream/0 is not set.
    This likely indicates that there is a problem with the OO rules.
    Please report this failure to the Pharos developers!
     [150] prolog_stack:get_prolog_backtrace(100,[frame(150,clause(<clause>(0x5ce281eb4d00),6),_5626440)|_5626428],[goal_term_depth(100)]) at /usr/local/lib/swipl/library/prolog_stack.pl:137
     [149] throw_with_backtrace(error(system_error(upstreamProblem))) at /usr/local/share/pharos/prolog/oorules/util.pl:185
      [26] solve_internal at /usr/local/share/pharos/prolog/oorules/setup.pl:681
      [25] catch(user:solve_internal,_5626664,user:((_5626732=error(resource_error(private_table_space),_5626746)->complain_table_space(ooscript);_5626796=error(resource_error(stack),_5626810)->complain_stack_size(ooscript);true),throw(_5626842))) at /usr/local/lib/swipl/boot/init.pl:562
      [24] solve(ooscript) at /usr/local/share/pharos/prolog/oorules/setup.pl:617
    

    Originally posted by @Trass3r in https://github.com/cmu-sei/pharos/issues/209#issuecomment-1179610775

    bug prolog fix pending merge 
    opened by sei-eschwartz 4
  • Calling conventions and being smarter about statically linked binaries

    Calling conventions and being smarter about statically linked binaries

    I have recently been reviewing the reasoning rules for deciding what functions are methods. Many of these methods are in the form of "Well, we passed this pointer as a thisptr to a known method, and we passed it to this candidate method, so the candidate method is probably a method". Why do we have these?

    If we think about it, on 32-bit MSVC, excluding WPO, if we see an argument in ecx the function is either fastcall or thiscall. And the user probably isn't writing fastcall functions. Here's a short list of fastcall functions found in our test suite:

         15 _EH4_CallFilterFunc(x,
         15 _EH4_GlobalUnwind2(x,
         15 _EH4_LocalUnwind(x,
         15 _EH4_TransferToHandler(x,
         48 _RTC_AllocaHelper(x,
         48 _RTC_CheckStackVars2(x,
         48 _RTC_CheckStackVars(x,
         87 __security_check_cookie(x)
    

    These are short and probably easily detectable. So excluding those, ecx argument usage means a method, right?

    Nope. It turns out that even if you have WPO turned off, statically compiling an executable will include WPO functions in the binary, because libc was compiled with WPO turned on. So unfortunately you get weird stuff like cdecl functions that pass their arguments in the eax and ecx registers.

    In short, we should be smarter about dealing with statically linked binaries. If we have a dynamically linked binary, we should be more liberal with how we detect methods. If we have a statically linked binary, we can use the current conservative behaviors, or maybe just fingerprint all the standard functions and exclude them.

    The first step of doing this is to detect whether a binary is static or dynamic. To answer this, we can look at the executable's imports and see if the c++ runtime is there or not. And then export that as part of the facts file.

    enhancement prolog 
    opened by sei-eschwartz 0
  • guessMethodD is broken?

    guessMethodD is broken?

    guessMethodD is currently broken and will never work. For example, validMethodCallAtOffset(_Insn1, Caller1, Method, _Size1), can only be true if factMethod(Method), which is what the rule is trying to guess.

    opened by sei-eschwartz 0
  • OOAnalyzer: Initial sanity check failed, indicating the OO rules are incorrect.

    OOAnalyzer: Initial sanity check failed, indicating the OO rules are incorrect.

    Trying OOAnalyzer for the first time, no idea what I'm doing really, but a simple run of ooanalyzer -F=facts.txt -R=results.txt --prolog-loglevel=3 TS2.exe resulted in an error telling me to report a bug, so here I am.

    Log file attached, I shortened it quite a bit in the middle because the "Missing return value from new() call" errors were quite excessive (12k+ lines) oo_log.txt

    =facts.txt

    The binary was compiled with VS2015 and CFG enabled as far as I know.

    bug prolog fix pending merge 
    opened by srett 19
  • Nonreturn methods are problematic

    Nonreturn methods are problematic

    Example: 0x412780 in ooex_vs2010/Debug/ooex7

    void __thiscall std::string::_Xran(std::string *this)

    This method is basically a utility function that calls std::Xout_of_range, which is noreturn.

    The compiler does weird stuff with respect to stack cleanup: https://godbolt.org/z/x5Wh4M3rT

    This causes us to get confused about the calling convention :-(

    bug 
    opened by sei-eschwartz 0
Owner
Software Engineering Institute
At the SEI, we research software engineering, cybersecurity, and AI engineering problems; create innovative technologies; and put solutions into practice.
Software Engineering Institute
CITL's static analysis engine for native code artifacts

citl-static-analyzer Fast binary hardening analysis tooling. Building on Linux The build process varies by Linux distribution, owing to differences be

Cyber Independent Testing Lab 17 Jul 11, 2022
A static analysis tool that helps security researchers scan a list of Windows kernel drivers for common vulnerability patterns in drivers (CVE makers!)

Driver Analyzer A static analysis tool that helps security researchers scan a list of Windows kernel drivers for common vulnerability patterns in driv

BehroozAbbassi 43 Jul 31, 2022
Static analysis of C/C++ code

Cppcheck GitHub Actions Linux Build Status Windows Build Status OSS-Fuzz Coverity Scan Build Status License About the name The original name of this p

Daniel Marjamäki 4.3k Aug 11, 2022
Qt-oriented static code analyzer based on the Clang framework

WARNING: master is the development branch. Please use the v1.10 branch. clazy v1.11 clazy is a compiler plugin which allows clang to understand Qt sem

KDE GitHub Mirror 529 Aug 13, 2022
Static code checker for C++

cpplint - static code checker for C++ Cpplint is a command-line tool to check C/C++ files for style issues following Google's C++ style guide. Cpplint

null 1.1k Aug 10, 2022
A static analyzer for Java, C, C++, and Objective-C

Infer Infer is a static analysis tool for Java, C++, Objective-C, and C. Infer is written in OCaml. Installation Read our Getting Started page for det

Facebook 13.4k Aug 9, 2022
Static analyzer for C/C++ based on the theory of Abstract Interpretation.

IKOS IKOS (Inference Kernel for Open Static Analyzers) is a static analyzer for C/C++ based on the theory of Abstract Interpretation. Introduction IKO

NASA - Software V&V 1.7k Aug 6, 2022
Static analyzer for C/C++ based on the theory of Abstract Interpretation.

IKOS IKOS (Inference Kernel for Open Static Analyzers) is a static analyzer for C/C++ based on the theory of Abstract Interpretation. Introduction IKO

NASA - Software V&V 1.7k Aug 8, 2022
Analysis of Argon and Xenon Collision data to test Bardeen-Cooper-Schrieffer Approximation

Univeristy of North Carolina Wilmington Analysis of Argon and Xenon Collision data to test Bardeen-Cooper-Schrieffer Approximation. Argon Data Tree Fi

William Jarratt 1 Nov 15, 2021
Clang build analysis tool using -ftime-trace

Clang Build Analyzer Clang C/C++ build analysis tool when using Clang 9+ -ftime-trace. The -ftime-trace compiler flag (see blog post or Clang 9 releas

Aras Pranckevičius 670 Aug 4, 2022
Probabilistic Risk Analysis Tool (fault tree analysis, event tree analysis, etc.)

SCRAM SCRAM is a Command-line Risk Analysis Multi-tool. This project aims to build a command line tool for probabilistic risk analysis. SCRAM is capab

Olzhas Rakhimov 113 Jul 1, 2022
ELF static analysis and injection framework that parse, manipulate and camouflage ELF files.

elfspirit elfspirit is a useful program that parse, manipulate and camouflage ELF files. It provides a variety of functions, including adding or delet

null 12 Jul 27, 2022
Maat is an open-source Dynamic Symbolic Execution and Binary Analysis framework

About Maat is an open-source Dynamic Symbolic Execution and Binary Analysis framework. It provides various functionalities such as symbolic execution,

Trail of Bits 463 Aug 9, 2022
CITL's static analysis engine for native code artifacts

citl-static-analyzer Fast binary hardening analysis tooling. Building on Linux The build process varies by Linux distribution, owing to differences be

Cyber Independent Testing Lab 17 Jul 11, 2022
A static analysis tool that helps security researchers scan a list of Windows kernel drivers for common vulnerability patterns in drivers (CVE makers!)

Driver Analyzer A static analysis tool that helps security researchers scan a list of Windows kernel drivers for common vulnerability patterns in driv

BehroozAbbassi 43 Jul 31, 2022
Cmake check - Static analysis for CMake files.

cmake_check Cmake_check is a linter for the CMake language. It takes a set of user-defined rules and reports violations for CMakeLists.txt files and C

Dael 20 Mar 22, 2020
Static analysis of C/C++ code

Cppcheck GitHub Actions Linux Build Status Windows Build Status OSS-Fuzz Coverity Scan Build Status License About the name The original name of this p

Daniel Marjamäki 4.3k Aug 11, 2022
Static analysis of structures is a fundamental step for determining the stability of structures

StAnD: A Dataset of Linear Static Analysis Problems [Abstract] [Paper] Static analysis of structures is a fundamental step for determining the stabili

Zuru Tech 3 Jan 20, 2022
This is like Inverting Binary Tree, but instead of a Binary Tree it's a File Tree.

Invert File Tree in C++ This is like Inverting Binary Tree, but instead of the Binary Tree it's a File Tree. This is intended as a simple exercise to

Tsoding 11 Jun 18, 2021
Binary Analysis Craft!

BinCraft - Binary Analysis Craft BinCraft is a future binary analysis toolkit. Features: Layered Architecture: composed by multiple libraries that can

PortalLab 231 Jul 10, 2022
Binary Analysis Craft!

BinCraft - Binary Analysis Craft BinCraft is a future binary analysis toolkit. Features: Layered Architecture: composed by multiple libraries that can

PortalLab 61 May 25, 2022
Binary data analysis and visualization tool

Veles - A new age tool for binary analysis It is a very difficult task for a human to notice subtle patterns in large amounts of binary data, however,

CodiLime Sp. z o.o. 855 Aug 8, 2022
Terrain Analysis Using Digital Elevation Models (TauDEM) software for hydrologic terrain analysis and channel network extraction.

TauDEM (Terrain Analysis Using Digital Elevation Models) is a suite of Digital Elevation Model (DEM) tools for the extraction and analysis of hydrolog

David Tarboton 185 Jul 12, 2022
config-loader is a static reflection framework written in C++17 from parse configuration file to native data structure.

config-loader is a static reflection framework written in C++17 from parse configuration file to native data structure.

Netcan 97 Jul 20, 2022
Qt-oriented static code analyzer based on the Clang framework

WARNING: master is the development branch. Please use the v1.10 branch. clazy v1.11 clazy is a compiler plugin which allows clang to understand Qt sem

KDE GitHub Mirror 529 Aug 13, 2022
Qt based simple SCADA framework, with dashboard, static and dynamic components

QSimpleScada Qt/C++ based simple SCADA library for your IoT projects. We created QSimpleScada to speed up and simplify visualising any data, so we (an

Indeema Software Inc. 167 Aug 4, 2022
A compiling time static reflection framework for C++

static_reflect This is a fully compiling time static reflection lightweight framework for C++. It provides a very rich compile-time reflection functio

null 10 Jul 19, 2022
SuanPan - 🧮 An Open Source, Parallel and Heterogeneous Finite Element Analysis Framework

suanPan Introduction ?? suanPan is a finite element method (FEM) simulation platform for applications in fields such as solid mechanics and civil/stru

Theodore 17 May 14, 2022
Header-only, event based, tiny and easy to use libuv wrapper in modern C++ - now available as also shared/static library!

Do you have a question that doesn't require you to open an issue? Join the gitter channel. If you use uvw and you want to say thanks or support the pr

Michele Caini 1.5k Aug 10, 2022