MemoryLeakDetector is a native memory leak monitoring tool developed by Xigua video android team

Overview

MemoryLeakDetector

简体中文版说明 >>>

GitHub license Platform API

MemoryLeakDetector is a native memory leak monitoring tool developed by Xigua video android team. It has simple access, wide monitoring range, excellent performance and good stability. It is widely used in native-memory-leak-governance of ByteDance's major apps, and the benefits are significant!

Apps using MemoryLeakDetector

Get started

Step 1: Add the JitPack repository to your build file

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

Step 2: Add the dependency

dependencies {
    implementation 'com.github.bytedance:memory-leak-detector:0.1.8'
}

Step 3: Add code for simple usage (This step is not necessary for using broadcast control)

// Using MemoryLeakDetector to monitor specified so
Raphael.start(
    Raphael.MAP64_MODE|Raphael.ALLOC_MODE|0x0F0000|1024,
    "/storage/emulated/0/raphael", // need sdcard permission
    ".*libxxx\\.so$"
);
// Using MemoryLeakDetector to monitor current process
Raphael.start(
    Raphael.MAP64_MODE|Raphael.ALLOC_MODE|0x0F0000|1024,
    "/storage/emulated/0/raphael", // need sdcard permission
    null
);
## broadcast command for specified so
adb shell am broadcast -a com.bytedance.raphael.ACTION_START -f 0x01000000 --es configs 0xCF0400 --es regex ".*libXXX\\.so$"
## broadcast command (RaphaelReceiver component process)
adb shell am broadcast -a com.bytedance.raphael.ACTION_START -f 0x01000000 --es configs 0xCF0400

Step 4: Print result

// code control
Raphael.print();
## broadcast command
adb shell am broadcast -a com.bytedance.raphael.ACTION_PRINT -f 0x01000000

Step 5: Analysis

## analysis report
##   -r: report path
##   -o: output file name
##   -s: symbol file dir
python3 library/src/main/python/raphael.py -r report -o leak-doubts.txt -s ./symbol/
## analysis maps
##   -m: maps file path
python3 library/src/main/python/mmap.py -m maps

Step 6: Stop monitoring

// code control
Raphael.stop();
## broadcast command
adb shell am broadcast -a com.bytedance.raphael.ACTION_STOP -f 0x01000000

Extra

  1. Android Camera内存问题剖析
  2. 西瓜视频稳定性治理体系建设一:Tailor 原理及实践
  3. 西瓜视频稳定性治理体系建设二:Raphael 原理及实践

Support

  1. Communicate on GitHub issues
  2. Mail: [email protected]
  3. WeChat: 429013449
  4. QQ Group

QQ Group

License

Copyright (c) 2021 ByteDance Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Issues
  • 修复realloc_proxy中存在的Bug。

    修复realloc_proxy中存在的Bug。

    当address != NULL, ptr == NULL, size == 0时,realloc不做任何事情,仅返回base address。当前的方案会bad remove, bad insert 当address != NULL, ptr != NULL, size != 0时,realloc成功,需要先删除原有信息,然后再更新。当前的方案仅会insert。 原有代码size >= limit也有问题,如:size == 0,limit == 1024,可能走到realloc equal to free逻辑,需要remove,代码会被跳过。 另外,我的新代码也存在Bug,如果realloc size < limit,则会被跳过,此逻辑问题好像无法解决。 https://man7.org/linux/man-pages/man3/free.3.html

    opened by huchao 4
  • 运行分析脚本raphael.py 报错

    运行分析脚本raphael.py 报错

    Raphael script version: 1.0.0 Traceback (most recent call last): File "library/src/main/python/raphael.py", line 215, in report = parse_report(string) File "library/src/main/python/raphael.py", line 183, in parse_report report.append(Trace(match[0][0], match[0][1], match[0][2], stack)) IndexError: list index out of range

    opened by winchances 3
  • 配置项的各个参数含义具体是什么

    配置项的各个参数含义具体是什么

    Raphael.MAP64_MODE | Raphael.ALLOC_MODE | 0x0F0000 | 1024 前两个mode我大概可以猜到是什么,一个是监听 MMAP,一个是监听 ALLOC,查看源码我看到还有一个 DIFF_CACHE,这是什么含义? 后面的 0x0F0000 和 1024 又是配置的什么?

    opened by Cr321 2
  • 使用std::make_unique创建的对象,无法检测到泄漏

    使用std::make_unique创建的对象,无法检测到泄漏

    extern "C" JNIEXPORT void JNICALL
    Java_com_xxxx_filament_ObjAssimpLoader_nLoad(JNIEnv *env, jclass, jstring path, jlong nativeEngine) {
        g_meshSet = std::make_unique<ObjMeshAssimp>(*((Engine*)nativeEngine));
        utils::Path filename = env->GetStringUTFChars(path, 0);
        g_meshSet->addFromFile(filename, g_materialInstances, true);
        //人为制造一个内存泄漏对象
        std::unique_ptr<ObjMeshAssimp> temp = std::make_unique<ObjMeshAssimp>(*((Engine*)nativeEngine));
    }
    

    大佬,问下,我集成这个工具后,生成report,并没有找到我泄漏的so代码行号,唯一相关的so信息如下:

    0x00000076e2647000, 4096, 1
    0x0000000000214cc0 /data/app/com.xxxx.filament.textured-OlqQ08eyhNALYGjMKOUN6w==/lib/arm64/libfilament_jni.so (std::__ndk1::deque<unsigned int, std::__ndk1::allocator<unsigned int> >::__add_back_capacity())
    
    0x00000076e2ba9000, 4096, 1
    0x00000000003127c8 /system/lib64/libart.so (art::jit::JitCodeCache::AddProfilingInfoInternal(art::Thread*, art::ArtMethod*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > const&) + 276)
    0x0000000000312680 /system/lib64/libart.so (art::jit::JitCodeCache::AddProfilingInfo(art::Thread*, art::ArtMethod*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > const&, bool) + 132)
    0x0000000000322c14 /system/lib64/libart.so (art::ProfilingInfo::Create(art::Thread*, art::ArtMethod*, bool) + 372)
    0x0000000000309040 /system/lib64/libart.so (art::jit::Jit::AddSamples(art::Thread*, art::ArtMethod*, unsigned short, bool) + 504)
    0x0000000000280394 /system/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*) + 180)
    0x000000000027a444 /system/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*) + 972)
    0x0000000000526f0c /system/lib64/libart.so (MterpInvokeVirtual + 592)
    0x000000000054a598 /system/lib64/libart.so (ExecuteMterpImpl + 14232)
    0x0000000000254148 /system/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool) (.llvm.3404100416) + 492)
    0x0000000000259c3c /system/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*) + 220)
    0x000000000027a428 /system/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*) + 944)
    0x0000000000526f0c /system/lib64/libart.so (MterpInvokeVirtual + 592)
    0x000000000054a598 /system/lib64/libart.so (ExecuteMterpImpl + 14232)
    0x0000000000254148 /system/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool) (.llvm.3404100416) + 492)
    
    

    但是这看起来像是系统编译的内存信息

    opened by fancychendong 2
  • 部分场景raphael.py聚合的不准

    部分场景raphael.py聚合的不准

    部分场景raphael.py聚合的不准
    比如

    0x0000007010381000, 16785408, 1
    0x00000000005932e0 /data/app/xxx/lib/arm64/libmya.so (unknown)
    0x000000000045b8e4 /system/lib64/libsystem.so (unknown)
    0x00000000000291c0 /data/app/xxx/lib/arm64/libmyb.so (unknown)
    

    根据 group_record聚合函数,从栈底到栈顶遍历,只要是先找到了/data/下的so就返回,这种场景返回的是libmyb.so,实际应该返回libmya.so

    def group_record(record):
        default = None
        for i in range(len(record.stack) - 1, -1, -1):
            match = re.match(r'.+\/(.+\.(so|apk|oat))', record.stack[i].path, re.M | re.I)
            if not match or match.group(1) == 'libraphael.so':
                continue
            elif record.stack[i].path.startswith('/data/'):
                return match.group(1)
            elif match.group(1) in system_group and not default:
                default = match.group(1)
        return default if default else 'extras'
    
    opened by jinshiyi11 2
  • raphael.py执行报错

    raphael.py执行报错

    使用python2.7,执行raphael.py分析report文件,报错如下,是否有python版本要求?

    File "D:\open\memory-leak-detector\library\src\main\python\raphael.py", line 149, in print_report
        report.sort(lambda x, y: x.size - y.size, reverse=True)
    TypeError: comparison function must return int, not long
    
    opened by jinshiyi11 2
  • 存在内存泄露

    存在内存泄露

    void Raphael::start(JNIEnv *env, jobject obj, jint configs, jstring space, jstring regex) { mSpace = (char *) env->GetStringUTFChars(space, 0); 其他省略 }

    mSpace 并没有释放,导致内存泄露

    opened by hyh-qz 2
  • no more stack trace in operator_new(_Znwj) on armeabi-v7a

    no more stack trace in operator_new(_Znwj) on armeabi-v7a

    use libudf_unwind_backtrace(xframes, 2, STACKTRACEDEPTH) on hook _Znwj callback function the reuslt:

    xHook ### allocated [12,679]K 12983343 allocCnt:2522 freeCnt:2003 by: ###00 0x00300135 _Znwj

    only 1 frame! no more???

    opened by shenghanzhouyou 2
  • 请教一下案例中的代码是如何调用的?

    请教一下案例中的代码是如何调用的?

    我按照案例中的代码实现了一遍,放到我的代码里是这样子的:

    protected final CameraCaptureSession.CaptureCallback  captureCallback = new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
            CameraActivity.recycle(result);
        }
    };
    
    static Field sTargetField;
    static Method sTargetMethod;
    
    @SuppressLint("SoonBlockedPrivateApi")
    static void recycle(TotalCaptureResult tcr) {
        try {
            if (sTargetMethod == null || sTargetField == null) {
                Method forName = Class.class.getDeclaredMethod("forName", String.class);
                Method getDeclaredMethod = Class.class.getDeclaredMethod("getDeclaredMethod", String.class, Class[].class);
                Class<?> vmRuntimeClass = (Class<?>) forName.invoke(null, "dalvik.system.VMRuntime");
                Method getRuntime = (Method) getDeclaredMethod.invoke(vmRuntimeClass, "getRuntime", null);
                Method setHiddenApiExemptions = (Method) getDeclaredMethod.invoke(vmRuntimeClass, "setHiddenApiExemptions", new Class[]{String[].class});
                Object sVmRuntime = getRuntime.invoke(null);
                setHiddenApiExemptions.invoke(sVmRuntime, new Object[]{new String[]{"L"}});
            }
    
            if (sTargetField == null) {
                sTargetField = tcr.getClass().getSuperclass().getDeclaredField("mResults");
                sTargetField.setAccessible(true);
            }
    
            if (sTargetMethod == null) {
                sTargetMethod = Class.forName("android.hardware.camera2.impl.CameraMetadataNative").getDeclaredMethod("close");
                sTargetMethod.setAccessible(true);
            }
    
            sTargetMethod.invoke(sTargetField.get(tcr));
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (InvocationTargetException e) {
            e.printStackTrace();
        }
    }
    

    请问是这样调用的吗?

    opened by BangwenHe 1
  • commands.getstatusoutput在windows平台不支持

    commands.getstatusoutput在windows平台不支持

    raphael.py中commands模块不支持windows平台,且在只Python 3中已被移除,建议改为subprocess
    比如改为

    output = subprocess.check_output('arm-linux-androideabi-addr2line -C -e %s -f %s' % (symbol_path, address))
    

    另外addr2line 的-C参数支持demangle name,是否可以替换c++filt

    opened by jinshiyi11 1
  • raphael脚本增加-s参数报错

    raphael脚本增加-s参数报错

    user$ python python/raphael.py -r raphael/report -o leak-doubts.txt -s symbol/ Raphael script version: 1.0.0 {'libmapv2.so': '/Users/user/Documents/driver/libmapv2.so', 'libhawiinav.so': '/Users/user/Documents/driver/libhawiinav.so'} Traceback (most recent call last): File "python/raphael.py", line 219, in print_report(writer, report) File "python/raphael.py", line 151, in print_report retry_symbol(record) File "python/raphael.py", line 110, in retry_symbol symbol = addr_to_line(frame.pc, symbol_table.get(match.group(1))) File "python/raphael.py", line 88, in addr_to_line raise Exception('execute [/Users/user/Library/Android/sdk/ndk/21.1.6352462/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-addr2line -e %s -f %s] failed' % (symbol_path, address)) Exception: execute [/Users/user/Library/Android/sdk/ndk/21.1.6352462/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-addr2line -e /Users/user/Documents/driver/libmapv2.so -f 0x0017c91b] failed

    opened by wustor 1
  • Android 12 exported

    Android 12 exported

    Android12适配

    INSTALL_PARSE_FAILED_MANIFEST_MALFORMED: Failed parse during installPackageLI: /data/app/vmdl1017191547.tmp/base.apk (at Binary XML file line #223): com.bytedance.raphael.RaphaelReceiver: Targeting S+ (version 31 and above) requires that an explicit value for android:exported be defined when intent filters are present

    opened by caoshen 0
  • 【必现】线程卡住问题

    【必现】线程卡住问题

    MemoryCache.cpp 中 pthread_mutex_lock(&alloc_mutex); for (auto p : alloc_table) { for (; p != nullptr; p = p->next) { write_trace(report, p, nullptr, &dl_cache); } } pthread_mutex_lock(&alloc_mutex);

    pthread_mutex_lock(&alloc_mutex); 在调试中发现这句代码卡住线程

    opened by hansweidong 1
  • 【必现】PLT Hook方案不兼容Android Gradle Plugin 3.6.0+

    【必现】PLT Hook方案不兼容Android Gradle Plugin 3.6.0+

    重现步骤:

    1. 将demo工程目录下的build.gradle的版本号改为3.6.0或以上,如:
    dependencies {
            classpath 'com.android.tools.build:gradle:4.1.2'
            classpath 'com.github.dcendents:android-maven-gradle-plugin:2.1'
    }
    
    1. 将gradle-wrapper.properties的Url改为6.8.3,如:
    distributionUrl=https\://services.gradle.org/distributions/gradle-6.8.3-all.zip
    
    1. gradle-wrapper.jar无需修改
    2. 将demo/build.gradle中的minSdkVersion改为23
    defaultConfig {
        minSdkVersion 23
        targetSdkVersion 28
    }
    
    1. 在Raphael.start中将so列表改为libtester.so(仅为了方便问题定位)
    Raphael.start(Raphael.MAP64_MODE|Raphael.ALLOC_MODE|0x0F0000|1024, space, "libtester.so");
    
    1. 添加测试工程,用于内存申请
    2. 重新编译并运行demo工程 如上修改均已完成,地址:https://github.com/huchao/memory-leak-detector 【案例1】直接编译运行后,点击界面上的Malloc按钮,然后点击打印按钮,最终并没有Hook到malloc调用。 【案例2】在App的AndroidManifest.xml中的Application中添加android:extractNativeLibs="true"配置禁用uncompress属性后,重新编译运行案例1,report文件中能够找到malloc调用。
    0x72e516e000, 1024, 1
    0x000006A4 /data/app/~~KhDIb1J0OYri3Rjwjas_nA==/com.bytedance.raphael.demo-mSCPf4jfHg_fgGGIxGcAdw==/lib/arm64/libtester.so (Java_com_huchao_jni_Heap_malloc + ?)
    0x00012ED8 /apex/com.android.art/lib64/libart.so (unknown)
    0x00012ED8 <unknown> (unknown)
    

    【问题原因】Android系统6.0+版本,Android Gradle Plugin 3.6.0+版本引入了so uncompress特性,并且默认开启,开启后,so以非压缩方式被打包到apk中,load so时,将从apk中直接load入内存,无需解压。这也就导致xh_core_refresh中,查找/proc/[pid]/maps时正则匹配不到对应的so,最终整套PLT Hook逻辑无法运行。 image 图中摘抄了maps的libtester.so部分,可见原有的/lib/arm64/libtester.so被替换为base.apk了,并且offset也不为0了,无法通过这个来判断ELF Header了。

    opened by huchao 7
Releases(0.1.8)
Owner
Bytedance Inc.
Bytedance Inc.
A single file drop-in memory leak tracking solution for C++ on Windows

MemLeakTracker A single file drop-in memory leak tracking solution for C++ on Windows This small piece of code allows for global memory leak tracking

null 22 Apr 23, 2022
Memory instrumentation tool for android app&game developers.

Overview LoliProfiler is a C/C++ memory profiling tool for Android games and applications. LoliProfiler supports profiling debuggable applications out

Tencent 416 May 13, 2022
Memory-dumper - A tool for dumping files from processes memory

What is memory-dumper memory-dumper is a tool for dumping files from process's memory. The main purpose is to find patterns inside the process's memor

Alexander Nestorov 29 Feb 5, 2022
The Hoard Memory Allocator: A Fast, Scalable, and Memory-efficient Malloc for Linux, Windows, and Mac.

The Hoard Memory Allocator Copyright (C) 1998-2020 by Emery Berger The Hoard memory allocator is a fast, scalable, and memory-efficient memory allocat

Emery Berger 882 May 13, 2022
Custom memory allocators in C++ to improve the performance of dynamic memory allocation

Table of Contents Introduction Build instructions What's wrong with Malloc? Custom allocators Linear Allocator Stack Allocator Pool Allocator Free lis

Mariano Trebino 1.2k May 9, 2022
MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

A.P. Jo. 4 Oct 2, 2021
Mesh - A memory allocator that automatically reduces the memory footprint of C/C++ applications.

Mesh: Compacting Memory Management for C/C++ Mesh is a drop in replacement for malloc(3) that can transparently recover from memory fragmentation with

PLASMA @ UMass 1.4k May 7, 2022
A tool for tracking memory allocation based ld-preload

libmallocTrace A tool for tracking memory allocation based ld-preload how to build make cd example && make how to use a simple way is to execute some

赵政 1 Mar 12, 2022
Tool for profiling heap usage and memory management

vizzy > ./build/vizzytrace /tmp/heapinfo.trace /bin/find /home/zznop -name vizzy _ _ ____ ____ ____ _ _ ( \/ )(_ _)(_ )(_ )( \/ ) \ /

Brandon Miller 29 Mar 19, 2022
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

memory The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to b

Jonathan Müller 1.1k May 9, 2022
Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

rpmalloc - General Purpose Memory Allocator This library provides a public domain cross platform lock free thread caching 16-byte aligned memory alloc

Mattias Jansson 1.5k May 10, 2022
OpenXenium JTAG and Flash Memory programmer

OpenXenium JTAG and Flash Memory programmer * Read: "Home Brew" on ORIGINAL XBOX - a detailed article on why and how * The tools in this repo will all

Koos du Preez 25 Feb 14, 2022
manually map driver for a signed driver memory space

smap manually map driver for a signed driver memory space credits https://github.com/btbd/umap tested system Windows 10 Education 20H2 UEFI installati

ekknod 71 Apr 9, 2022
Dump the memory of a PPL with a userland exploit

PPLdump This tool implements a userland exploit that was initially discussed by James Forshaw (a.k.a. @tiraniddo) - in this blog post - for dumping th

Clément Labro 546 May 14, 2022
Implementation of System V shared memory (a type of inter process communication) in xv6 operating system.

NOTE: we have stopped maintaining the x86 version of xv6, and switched our efforts to the RISC-V version (https://github.com/mit-pdos/xv6-riscv.git)

Viraj Jadhav 5 Feb 21, 2022
An In-memory Embedding of CPython

An In-memory Embedding of CPython This repository contains all the build artifacts necessary to build an embedding of CPython 3.8.2 that can be run en

null 100 Apr 26, 2022
Execute MachO binaries in memory using CGo

Execute Thin Mach-O Binaries in Memory This is a CGo implementation of the initial technique put forward by Stephanie Archibald in her blog, Running E

Dwight Hohnstein 55 Apr 15, 2022
Initialize the 8-bit computer memory with a program to be executed automatically on powering.

Initialize the 8-bit computer memory with a program to be executed automatically on powering. This project is small extension of Ben Eater's computer

Dmytro Striletskyi 62 Dec 13, 2021
Artifacts of that Memory Management Tsoding Session

Artifacts of those Memory Management Tsoding Sessions Quick Start $ make $ ./heap Limitations The pointers to the heap can only be located in the heap

Tsoding 51 May 15, 2022