Alloc-test - Cross-platform benchmarking for memory allocators, aiming to be as close to real world as it is practical

Overview
NOTE: 32-BIT PLATFORMS HAVE NOT YET BEEN TESTED!

To test tcmalloc / jemalloc / hoard / ptmalloc:

1. Make sure new_delete_allocator is selected at selector.h

2. See useful build notes at section "Testing" at http://ithare.com/testing-memory-allocators-ptmalloc2-tcmalloc-hoard-jemalloc-while-trying-to-simulate-real-world-loads/

3. Edit build_gcc.sh to enable one of the above allocators (add/edit to have one of -ljemalloc, -ltcmalloc, -lhoard, or -L path/to/libmalloc.a) (
   Note: -lhoard: see https://github.com/emeryberger/Hoard
         -L path/to/libmalloc.a assumes that libmalloc.a is already built for a target platform


To test any other allocator:

1. Create "src/my_allocator.h" file with a class representing an allocator to be tested.
   As a sample "src/new_delete_allocator.h" or "src/void_allocator.h" could be taken.
   class MyAllocator must have all member functions found in the above-mentioned samples
   as this class is used as a template parameter in the testing routine
   
2. Edit "src/selector.h" to switch to using a new allocator

3. Rebuild and run
Issues
  • Unseeded random generator is used, it returns always 0

    Unseeded random generator is used, it returns always 0

    Random number generation algorith actually used (the one not commented out) cannot be used with seed of 0, because it will never change the seed value to anything else and will always generate 0. But default constructor initializes the seed to zero causing the instance is unusable until PRNG::seed() method is called with nonzero value.

    The default constructed PRNG instance without calling PRNG::seed() is used in the actual test to generate slot index and size of the block. So it always picks slot 0, allocates 5 bytes block in it and deallocates it in the next loop. Which makes the test pretty useless.

    opened by mezkov 0
  • compiler support?

    compiler support?

    Curious what compilers you're using. I'm trying to build on CentOS 7 with gcc 5.3.0 and clang 8.0.0 and both give (different) compile errors.

    gcc

    In file included from ../src/allocator_tester.h:54:0,
                     from ../src/allocator_tester.cpp:35:
    ../src/void_allocator.h: In member function \u2018void* VoidAllocatorForTest<ActualAllocator>::allocateSlots(size_t)\u2019:
    ../src/void_allocator.h:60:60: error: expected \u2018,\u2019 before \u2018)\u2019 token
      void* allocateSlots( size_t sz ) { static_assert( isFake()); assert( sz <= fakeBufferSize ); return alloc.allocate( sz ); }
                                                                ^
    ../src/void_allocator.h:60:60: error: expected string-literal before \u2018)\u2019 token
    In file included from ../src/allocator_tester.cpp:35:0:
    ../src/allocator_tester.h: In function \u2018void randomPos_RandomSize(AllocatorUnderTest&, size_t, size_t, size_t, size_t, size_t)\u2019:
    ../src/allocator_tester.h:285:5: error: expected \u2018(\u2019 before \u2018constexpr\u2019
      if constexpr ( !allocatorUnderTest.isFake() )
         ^
    ../src/allocator_tester.h:287:2: error: \u2018else\u2019 without a previous \u2018if\u2019
      else
      ^
    ../src/allocator_tester.h:306:8: error: expected \u2018(\u2019 before \u2018constexpr\u2019
         if constexpr ( doMemAccess )
            ^
    ../src/allocator_tester.cpp:185:1: error: expected \u2018}\u2019 at end of input
     }
     ^
    ../src/allocator_tester.cpp:185:1: error: expected \u2018}\u2019 at end of input
    ../src/allocator_tester.cpp:185:1: error: expected \u2018}\u2019 at end of input
    

    clang

    [email protected] build]$ ./build_gcc.sh
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:46:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<NewDeleteAllocatorForTest, none>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::none>( allocator, testParams->startupParams.iterCount, test...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
          requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
          requested here
                    runTest<MyAllocatorT>( &params );
                    ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:49:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<NewDeleteAllocatorForTest, full>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::full>( allocator, testParams->startupParams.iterCount, test...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
          requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
          requested here
                    runTest<MyAllocatorT>( &params );
                    ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:52:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<NewDeleteAllocatorForTest, single>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::single>( allocator, testParams->startupParams.iterCount, te...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
          requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
          requested here
                    runTest<MyAllocatorT>( &params );
                    ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:55:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<NewDeleteAllocatorForTest, check>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::check>( allocator, testParams->startupParams.iterCount, tes...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
          requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
          requested here
                    runTest<MyAllocatorT>( &params );
                    ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:46:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, none>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::none>( allocator, testParams->startupParams.iterCount, test...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
          'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
          'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                            runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                            ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:49:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, full>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::full>( allocator, testParams->startupParams.iterCount, test...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
          'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
          'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                            runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                            ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:52:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, single>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::single>( allocator, testParams->startupParams.iterCount, te...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
          'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
          'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                            runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                            ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
          (aka 'unsigned long') [-Wformat]
      ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                     ~~~~                                                                                                      ^~~~~~~~
                     %zu
    ../src/allocator_tester.cpp:55:4: note: in instantiation of function template specialization
          'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, check>' requested here
                            randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::check>( allocator, testParams->startupParams.iterCount, tes...
                            ^
    ../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
          'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                    std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                    ^
    ../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
          'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                            runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                            ^
    In file included from ../src/allocator_tester.cpp:35:
    ../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    ../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
            if constexpr ( !allocatorUnderTest.isFake() )
                            ^
    8 warnings and 16 errors generated.
    
    opened by WallStProg 2
Allocator bench - bench of various memory allocators

To run benchmarks Install lockless from https://locklessinc.com/downloads/ in lockless_allocator path make Install Hoard from https://github.com/emery

Sam 44 Dec 3, 2021
Mimalloc-bench - Suite for benchmarking malloc implementations.

Mimalloc-bench Suite for benchmarking malloc implementations, originally developed for benchmarking mimalloc. Collection of various benchmarks from th

Daan 141 May 11, 2022
Test cpu and memory speed at linux-vps

Тест скорости процессора и памяти на linux-vps. Занимается бессмысленным перемножением массивов случайных чисел, для определения скорости процессора и

Anton 3 Nov 30, 2021
Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

rpmalloc - General Purpose Memory Allocator This library provides a public domain cross platform lock free thread caching 16-byte aligned memory alloc

Mattias Jansson 1.5k May 10, 2022
Cross-platform shared memory stream/buffer, header-only library for IPC in C/C++.

libsharedmemory libsharedmemory is a small C++11 header-only library for using shared memory on Windows, Linux and macOS. libsharedmemory makes it eas

Aron Homberg 7 Apr 3, 2022
The Hoard Memory Allocator: A Fast, Scalable, and Memory-efficient Malloc for Linux, Windows, and Mac.

The Hoard Memory Allocator Copyright (C) 1998-2020 by Emery Berger The Hoard memory allocator is a fast, scalable, and memory-efficient memory allocat

Emery Berger 882 May 13, 2022
MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

A.P. Jo. 4 Oct 2, 2021
Memory-dumper - A tool for dumping files from processes memory

What is memory-dumper memory-dumper is a tool for dumping files from process's memory. The main purpose is to find patterns inside the process's memor

Alexander Nestorov 29 Feb 5, 2022
Mesh - A memory allocator that automatically reduces the memory footprint of C/C++ applications.

Mesh: Compacting Memory Management for C/C++ Mesh is a drop in replacement for malloc(3) that can transparently recover from memory fragmentation with

PLASMA @ UMass 1.4k May 7, 2022
Test your malloc protection

Test your allocs protections and leaks ! Report Bug · Request Feature Table of Contents About The Tool Getting Started Prerequisites Quickstart Usage

tmatis 32 Apr 3, 2022
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

memory The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to b

Jonathan Müller 1.1k May 9, 2022
OpenXenium JTAG and Flash Memory programmer

OpenXenium JTAG and Flash Memory programmer * Read: "Home Brew" on ORIGINAL XBOX - a detailed article on why and how * The tools in this repo will all

Koos du Preez 25 Feb 14, 2022
manually map driver for a signed driver memory space

smap manually map driver for a signed driver memory space credits https://github.com/btbd/umap tested system Windows 10 Education 20H2 UEFI installati

ekknod 71 Apr 9, 2022
Memory instrumentation tool for android app&game developers.

Overview LoliProfiler is a C/C++ memory profiling tool for Android games and applications. LoliProfiler supports profiling debuggable applications out

Tencent 416 May 13, 2022
A single file drop-in memory leak tracking solution for C++ on Windows

MemLeakTracker A single file drop-in memory leak tracking solution for C++ on Windows This small piece of code allows for global memory leak tracking

null 22 Apr 23, 2022
Dump the memory of a PPL with a userland exploit

PPLdump This tool implements a userland exploit that was initially discussed by James Forshaw (a.k.a. @tiraniddo) - in this blog post - for dumping th

Clément Labro 546 May 14, 2022
Implementation of System V shared memory (a type of inter process communication) in xv6 operating system.

NOTE: we have stopped maintaining the x86 version of xv6, and switched our efforts to the RISC-V version (https://github.com/mit-pdos/xv6-riscv.git)

Viraj Jadhav 5 Feb 21, 2022
An In-memory Embedding of CPython

An In-memory Embedding of CPython This repository contains all the build artifacts necessary to build an embedding of CPython 3.8.2 that can be run en

null 100 Apr 26, 2022
Execute MachO binaries in memory using CGo

Execute Thin Mach-O Binaries in Memory This is a CGo implementation of the initial technique put forward by Stephanie Archibald in her blog, Running E

Dwight Hohnstein 55 Apr 15, 2022