MemoryLeakDetector is a native memory leak monitoring tool developed by Xigua video android team

Overview

MemoryLeakDetector

简体中文版说明 >>>

GitHub license Platform API

MemoryLeakDetector is a native memory leak monitoring tool developed by Xigua video android team. It has simple access, wide monitoring range, excellent performance and good stability. It is widely used in native-memory-leak-governance of ByteDance's major apps, and the benefits are significant!

Apps using MemoryLeakDetector

Get started

Step 1: Add the JitPack repository to your build file

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

Step 2: Add the dependency

dependencies {
    implementation 'com.github.bytedance:memory-leak-detector:0.1.8'
}

Step 3: Add code for simple usage (This step is not necessary for using broadcast control)

// Using MemoryLeakDetector to monitor specified so
Raphael.start(
    Raphael.MAP64_MODE|Raphael.ALLOC_MODE|0x0F0000|1024,
    "/storage/emulated/0/raphael", // need sdcard permission
    ".*libxxx\\.so$"
);
// Using MemoryLeakDetector to monitor current process
Raphael.start(
    Raphael.MAP64_MODE|Raphael.ALLOC_MODE|0x0F0000|1024,
    "/storage/emulated/0/raphael", // need sdcard permission
    null
);
## broadcast command for specified so
adb shell am broadcast -a com.bytedance.raphael.ACTION_START -f 0x01000000 --es configs 0xCF0400 --es regex ".*libXXX\\.so$"
## broadcast command (RaphaelReceiver component process)
adb shell am broadcast -a com.bytedance.raphael.ACTION_START -f 0x01000000 --es configs 0xCF0400

Step 4: Print result

// code control
Raphael.print();
## broadcast command
adb shell am broadcast -a com.bytedance.raphael.ACTION_PRINT -f 0x01000000

Step 5: Analysis

## analysis report
##   -r: report path
##   -o: output file name
##   -s: symbol file dir
python3 library/src/main/python/raphael.py -r report -o leak-doubts.txt -s ./symbol/
## analysis maps
##   -m: maps file path
python3 library/src/main/python/mmap.py -m maps

Step 6: Stop monitoring

// code control
Raphael.stop();
## broadcast command
adb shell am broadcast -a com.bytedance.raphael.ACTION_STOP -f 0x01000000

Extra

  1. Android Camera内存问题剖析
  2. 西瓜视频稳定性治理体系建设一:Tailor 原理及实践
  3. 西瓜视频稳定性治理体系建设二:Raphael 原理及实践

Support

  1. Communicate on GitHub issues
  2. Mail: [email protected]
  3. WeChat: 429013449
  4. QQ Group

QQ Group

License

Copyright (c) 2021 ByteDance Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Comments
  • 【必现】PLT Hook方案不兼容Android Gradle Plugin 3.6.0+

    【必现】PLT Hook方案不兼容Android Gradle Plugin 3.6.0+

    重现步骤:

    1. 将demo工程目录下的build.gradle的版本号改为3.6.0或以上,如:
    dependencies {
            classpath 'com.android.tools.build:gradle:4.1.2'
            classpath 'com.github.dcendents:android-maven-gradle-plugin:2.1'
    }
    
    1. 将gradle-wrapper.properties的Url改为6.8.3,如:
    distributionUrl=https\://services.gradle.org/distributions/gradle-6.8.3-all.zip
    
    1. gradle-wrapper.jar无需修改
    2. 将demo/build.gradle中的minSdkVersion改为23
    defaultConfig {
        minSdkVersion 23
        targetSdkVersion 28
    }
    
    1. 在Raphael.start中将so列表改为libtester.so(仅为了方便问题定位)
    Raphael.start(Raphael.MAP64_MODE|Raphael.ALLOC_MODE|0x0F0000|1024, space, "libtester.so");
    
    1. 添加测试工程,用于内存申请
    2. 重新编译并运行demo工程 如上修改均已完成,地址:https://github.com/huchao/memory-leak-detector 【案例1】直接编译运行后,点击界面上的Malloc按钮,然后点击打印按钮,最终并没有Hook到malloc调用。 【案例2】在App的AndroidManifest.xml中的Application中添加android:extractNativeLibs="true"配置禁用uncompress属性后,重新编译运行案例1,report文件中能够找到malloc调用。
    0x72e516e000, 1024, 1
    0x000006A4 /data/app/~~KhDIb1J0OYri3Rjwjas_nA==/com.bytedance.raphael.demo-mSCPf4jfHg_fgGGIxGcAdw==/lib/arm64/libtester.so (Java_com_huchao_jni_Heap_malloc + ?)
    0x00012ED8 /apex/com.android.art/lib64/libart.so (unknown)
    0x00012ED8 <unknown> (unknown)
    

    【问题原因】Android系统6.0+版本,Android Gradle Plugin 3.6.0+版本引入了so uncompress特性,并且默认开启,开启后,so以非压缩方式被打包到apk中,load so时,将从apk中直接load入内存,无需解压。这也就导致xh_core_refresh中,查找/proc/[pid]/maps时正则匹配不到对应的so,最终整套PLT Hook逻辑无法运行。 image 图中摘抄了maps的libtester.so部分,可见原有的/lib/arm64/libtester.so被替换为base.apk了,并且offset也不为0了,无法通过这个来判断ELF Header了。

    opened by huchao 7
  • 修复realloc_proxy中存在的Bug。

    修复realloc_proxy中存在的Bug。

    当address != NULL, ptr == NULL, size == 0时,realloc不做任何事情,仅返回base address。当前的方案会bad remove, bad insert 当address != NULL, ptr != NULL, size != 0时,realloc成功,需要先删除原有信息,然后再更新。当前的方案仅会insert。 原有代码size >= limit也有问题,如:size == 0,limit == 1024,可能走到realloc equal to free逻辑,需要remove,代码会被跳过。 另外,我的新代码也存在Bug,如果realloc size < limit,则会被跳过,此逻辑问题好像无法解决。 https://man7.org/linux/man-pages/man3/free.3.html

    opened by huchao 4
  • 【必现】线程卡住问题

    【必现】线程卡住问题

    MemoryCache.cpp 中 pthread_mutex_lock(&alloc_mutex); for (auto p : alloc_table) { for (; p != nullptr; p = p->next) { write_trace(report, p, nullptr, &dl_cache); } } pthread_mutex_lock(&alloc_mutex);

    pthread_mutex_lock(&alloc_mutex); 在调试中发现这句代码卡住线程

    opened by hansweidong 3
  • 运行分析脚本raphael.py 报错

    运行分析脚本raphael.py 报错

    Raphael script version: 1.0.0 Traceback (most recent call last): File "library/src/main/python/raphael.py", line 215, in report = parse_report(string) File "library/src/main/python/raphael.py", line 183, in parse_report report.append(Trace(match[0][0], match[0][1], match[0][2], stack)) IndexError: list index out of range

    opened by winchances 3
  • 配置项的各个参数含义具体是什么

    配置项的各个参数含义具体是什么

    Raphael.MAP64_MODE | Raphael.ALLOC_MODE | 0x0F0000 | 1024 前两个mode我大概可以猜到是什么,一个是监听 MMAP,一个是监听 ALLOC,查看源码我看到还有一个 DIFF_CACHE,这是什么含义? 后面的 0x0F0000 和 1024 又是配置的什么?

    opened by Cr321 2
  • 使用std::make_unique创建的对象,无法检测到泄漏

    使用std::make_unique创建的对象,无法检测到泄漏

    extern "C" JNIEXPORT void JNICALL
    Java_com_xxxx_filament_ObjAssimpLoader_nLoad(JNIEnv *env, jclass, jstring path, jlong nativeEngine) {
        g_meshSet = std::make_unique<ObjMeshAssimp>(*((Engine*)nativeEngine));
        utils::Path filename = env->GetStringUTFChars(path, 0);
        g_meshSet->addFromFile(filename, g_materialInstances, true);
        //人为制造一个内存泄漏对象
        std::unique_ptr<ObjMeshAssimp> temp = std::make_unique<ObjMeshAssimp>(*((Engine*)nativeEngine));
    }
    

    大佬,问下,我集成这个工具后,生成report,并没有找到我泄漏的so代码行号,唯一相关的so信息如下:

    0x00000076e2647000, 4096, 1
    0x0000000000214cc0 /data/app/com.xxxx.filament.textured-OlqQ08eyhNALYGjMKOUN6w==/lib/arm64/libfilament_jni.so (std::__ndk1::deque<unsigned int, std::__ndk1::allocator<unsigned int> >::__add_back_capacity())
    
    0x00000076e2ba9000, 4096, 1
    0x00000000003127c8 /system/lib64/libart.so (art::jit::JitCodeCache::AddProfilingInfoInternal(art::Thread*, art::ArtMethod*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > const&) + 276)
    0x0000000000312680 /system/lib64/libart.so (art::jit::JitCodeCache::AddProfilingInfo(art::Thread*, art::ArtMethod*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > const&, bool) + 132)
    0x0000000000322c14 /system/lib64/libart.so (art::ProfilingInfo::Create(art::Thread*, art::ArtMethod*, bool) + 372)
    0x0000000000309040 /system/lib64/libart.so (art::jit::Jit::AddSamples(art::Thread*, art::ArtMethod*, unsigned short, bool) + 504)
    0x0000000000280394 /system/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*) + 180)
    0x000000000027a444 /system/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*) + 972)
    0x0000000000526f0c /system/lib64/libart.so (MterpInvokeVirtual + 592)
    0x000000000054a598 /system/lib64/libart.so (ExecuteMterpImpl + 14232)
    0x0000000000254148 /system/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool) (.llvm.3404100416) + 492)
    0x0000000000259c3c /system/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*) + 220)
    0x000000000027a428 /system/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*) + 944)
    0x0000000000526f0c /system/lib64/libart.so (MterpInvokeVirtual + 592)
    0x000000000054a598 /system/lib64/libart.so (ExecuteMterpImpl + 14232)
    0x0000000000254148 /system/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool) (.llvm.3404100416) + 492)
    
    

    但是这看起来像是系统编译的内存信息

    opened by fancychendong 2
  • 部分场景raphael.py聚合的不准

    部分场景raphael.py聚合的不准

    部分场景raphael.py聚合的不准
    比如

    0x0000007010381000, 16785408, 1
    0x00000000005932e0 /data/app/xxx/lib/arm64/libmya.so (unknown)
    0x000000000045b8e4 /system/lib64/libsystem.so (unknown)
    0x00000000000291c0 /data/app/xxx/lib/arm64/libmyb.so (unknown)
    

    根据 group_record聚合函数,从栈底到栈顶遍历,只要是先找到了/data/下的so就返回,这种场景返回的是libmyb.so,实际应该返回libmya.so

    def group_record(record):
        default = None
        for i in range(len(record.stack) - 1, -1, -1):
            match = re.match(r'.+\/(.+\.(so|apk|oat))', record.stack[i].path, re.M | re.I)
            if not match or match.group(1) == 'libraphael.so':
                continue
            elif record.stack[i].path.startswith('/data/'):
                return match.group(1)
            elif match.group(1) in system_group and not default:
                default = match.group(1)
        return default if default else 'extras'
    
    opened by jinshiyi11 2
  • raphael.py执行报错

    raphael.py执行报错

    使用python2.7,执行raphael.py分析report文件,报错如下,是否有python版本要求?

    File "D:\open\memory-leak-detector\library\src\main\python\raphael.py", line 149, in print_report
        report.sort(lambda x, y: x.size - y.size, reverse=True)
    TypeError: comparison function must return int, not long
    
    opened by jinshiyi11 2
  • 存在内存泄露

    存在内存泄露

    void Raphael::start(JNIEnv *env, jobject obj, jint configs, jstring space, jstring regex) { mSpace = (char *) env->GetStringUTFChars(space, 0); 其他省略 }

    mSpace 并没有释放,导致内存泄露

    opened by hyh-qz 2
  • no more stack trace in operator_new(_Znwj) on armeabi-v7a

    no more stack trace in operator_new(_Znwj) on armeabi-v7a

    use libudf_unwind_backtrace(xframes, 2, STACKTRACEDEPTH) on hook _Znwj callback function the reuslt:

    xHook ### allocated [12,679]K 12983343 allocCnt:2522 freeCnt:2003 by: ###00 0x00300135 _Znwj

    only 1 frame! no more???

    opened by shenghanzhouyou 2
  • 请教一下案例中的代码是如何调用的?

    请教一下案例中的代码是如何调用的?

    我按照案例中的代码实现了一遍,放到我的代码里是这样子的:

    protected final CameraCaptureSession.CaptureCallback  captureCallback = new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
            CameraActivity.recycle(result);
        }
    };
    
    static Field sTargetField;
    static Method sTargetMethod;
    
    @SuppressLint("SoonBlockedPrivateApi")
    static void recycle(TotalCaptureResult tcr) {
        try {
            if (sTargetMethod == null || sTargetField == null) {
                Method forName = Class.class.getDeclaredMethod("forName", String.class);
                Method getDeclaredMethod = Class.class.getDeclaredMethod("getDeclaredMethod", String.class, Class[].class);
                Class<?> vmRuntimeClass = (Class<?>) forName.invoke(null, "dalvik.system.VMRuntime");
                Method getRuntime = (Method) getDeclaredMethod.invoke(vmRuntimeClass, "getRuntime", null);
                Method setHiddenApiExemptions = (Method) getDeclaredMethod.invoke(vmRuntimeClass, "setHiddenApiExemptions", new Class[]{String[].class});
                Object sVmRuntime = getRuntime.invoke(null);
                setHiddenApiExemptions.invoke(sVmRuntime, new Object[]{new String[]{"L"}});
            }
    
            if (sTargetField == null) {
                sTargetField = tcr.getClass().getSuperclass().getDeclaredField("mResults");
                sTargetField.setAccessible(true);
            }
    
            if (sTargetMethod == null) {
                sTargetMethod = Class.forName("android.hardware.camera2.impl.CameraMetadataNative").getDeclaredMethod("close");
                sTargetMethod.setAccessible(true);
            }
    
            sTargetMethod.invoke(sTargetField.get(tcr));
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (InvocationTargetException e) {
            e.printStackTrace();
        }
    }
    

    请问是这样调用的吗?

    opened by BangwenHe 1
  • Galaxy A51 crash when startup demo

    Galaxy A51 crash when startup demo

    环境

    手机:Galaxy A51 操作系统:Android 10

    现象

    直接运行demo最新版,启动会直接crash,下面是日志 ` 2022-09-22 17:10:13.487 1215-1215/? E/Zygote: isWhitelistProcess - Process is Whitelisted 2022-09-22 17:10:13.495 1215-1215/? E/Zygote: accessInfo : 1 2022-09-22 17:10:13.503 1215-1215/? I/ce.raphael.dem: Late-enabling -Xcheck:jni 2022-09-22 17:10:13.533 1215-1215/? E/ce.raphael.dem: Unknown bits set in runtime_flags: 0x8000 2022-09-22 17:10:13.554 1215-1215/? D/ActivityThread: setConscryptValidator 2022-09-22 17:10:13.554 1215-1215/? D/ActivityThread: setConscryptValidator - put 2022-09-22 17:10:13.734 1215-1215/com.bytedance.raphael.demo I/A64_HOOK: insns pool initialized. 2022-09-22 17:10:13.735 1215-1215/com.bytedance.raphael.demo E/RAPHAEL: invoke failed at xdl_sym 2022-09-22 17:10:13.735 1215-1215/com.bytedance.raphael.demo W/ce.raphael.dem: 0xebadde09 skipped times: 0 2022-09-22 17:10:13.736 1215-1215/com.bytedance.raphael.demo A/libc: Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x753ff6532c in tid 1215 (ce.raphael.demo), pid 1215 (ce.raphael.demo)

    `

    opened by wustor 1
Releases(0.2.1)
Owner
Bytedance Inc.
Bytedance Inc.
A single file drop-in memory leak tracking solution for C++ on Windows

MemLeakTracker A single file drop-in memory leak tracking solution for C++ on Windows This small piece of code allows for global memory leak tracking

null 22 Jul 18, 2022
Memory instrumentation tool for android app&game developers.

Overview LoliProfiler is a C/C++ memory profiling tool for Android games and applications. LoliProfiler supports profiling debuggable applications out

Tencent 482 Nov 25, 2022
Memory-dumper - A tool for dumping files from processes memory

What is memory-dumper memory-dumper is a tool for dumping files from process's memory. The main purpose is to find patterns inside the process's memor

Alexander Nestorov 31 Nov 9, 2022
The Hoard Memory Allocator: A Fast, Scalable, and Memory-efficient Malloc for Linux, Windows, and Mac.

The Hoard Memory Allocator Copyright (C) 1998-2020 by Emery Berger The Hoard memory allocator is a fast, scalable, and memory-efficient memory allocat

Emery Berger 920 Nov 20, 2022
Custom memory allocators in C++ to improve the performance of dynamic memory allocation

Table of Contents Introduction Build instructions What's wrong with Malloc? Custom allocators Linear Allocator Stack Allocator Pool Allocator Free lis

Mariano Trebino 1.4k Nov 21, 2022
MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

A.P. Jo. 4 Oct 2, 2021
Mesh - A memory allocator that automatically reduces the memory footprint of C/C++ applications.

Mesh: Compacting Memory Management for C/C++ Mesh is a drop in replacement for malloc(3) that can transparently recover from memory fragmentation with

PLASMA @ UMass 1.5k Nov 29, 2022
A tool for tracking memory allocation based ld-preload

libmallocTrace A tool for tracking memory allocation based ld-preload how to build make cd example && make how to use a simple way is to execute some

赵政 1 Mar 12, 2022
Tool for profiling heap usage and memory management

vizzy > ./build/vizzytrace /tmp/heapinfo.trace /bin/find /home/zznop -name vizzy _ _ ____ ____ ____ _ _ ( \/ )(_ _)(_ )(_ )( \/ ) \ /

Brandon Miller 28 Jul 22, 2022
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

memory The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to b

Jonathan Müller 1.2k Nov 22, 2022
Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

rpmalloc - General Purpose Memory Allocator This library provides a public domain cross platform lock free thread caching 16-byte aligned memory alloc

Mattias Jansson 1.7k Nov 22, 2022
OpenXenium JTAG and Flash Memory programmer

OpenXenium JTAG and Flash Memory programmer * Read: "Home Brew" on ORIGINAL XBOX - a detailed article on why and how * The tools in this repo will all

Koos du Preez 29 Oct 23, 2022
manually map driver for a signed driver memory space

smap manually map driver for a signed driver memory space credits https://github.com/btbd/umap tested system Windows 10 Education 20H2 UEFI installati

ekknod 88 Nov 19, 2022
Dump the memory of a PPL with a userland exploit

PPLdump This tool implements a userland exploit that was initially discussed by James Forshaw (a.k.a. @tiraniddo) - in this blog post - for dumping th

Clément Labro 680 Nov 23, 2022
Implementation of System V shared memory (a type of inter process communication) in xv6 operating system.

NOTE: we have stopped maintaining the x86 version of xv6, and switched our efforts to the RISC-V version (https://github.com/mit-pdos/xv6-riscv.git)

Viraj Jadhav 5 Feb 21, 2022
An In-memory Embedding of CPython

An In-memory Embedding of CPython This repository contains all the build artifacts necessary to build an embedding of CPython 3.8.2 that can be run en

null 104 Nov 8, 2022
Execute MachO binaries in memory using CGo

Execute Thin Mach-O Binaries in Memory This is a CGo implementation of the initial technique put forward by Stephanie Archibald in her blog, Running E

Dwight Hohnstein 65 Oct 28, 2022
Initialize the 8-bit computer memory with a program to be executed automatically on powering.

Initialize the 8-bit computer memory with a program to be executed automatically on powering. This project is small extension of Ben Eater's computer

Dmytro Striletskyi 64 Aug 27, 2022
Artifacts of that Memory Management Tsoding Session

Artifacts of those Memory Management Tsoding Sessions Quick Start $ make $ ./heap Limitations The pointers to the heap can only be located in the heap

Tsoding 69 Nov 14, 2022