Alloc-test - Cross-platform benchmarking for memory allocators, aiming to be as close to real world as it is practical

Related tags

Memory Allocation alloc-test
Overview
NOTE: 32-BIT PLATFORMS HAVE NOT YET BEEN TESTED!

To test tcmalloc / jemalloc / hoard / ptmalloc:

1. Make sure new_delete_allocator is selected at selector.h

2. See useful build notes at section "Testing" at http://ithare.com/testing-memory-allocators-ptmalloc2-tcmalloc-hoard-jemalloc-while-trying-to-simulate-real-world-loads/

3. Edit build_gcc.sh to enable one of the above allocators (add/edit to have one of -ljemalloc, -ltcmalloc, -lhoard, or -L path/to/libmalloc.a) (
   Note: -lhoard: see https://github.com/emeryberger/Hoard
         -L path/to/libmalloc.a assumes that libmalloc.a is already built for a target platform


To test any other allocator:

1. Create "src/my_allocator.h" file with a class representing an allocator to be tested.
   As a sample "src/new_delete_allocator.h" or "src/void_allocator.h" could be taken.
   class MyAllocator must have all member functions found in the above-mentioned samples
   as this class is used as a template parameter in the testing routine
   
2. Edit "src/selector.h" to switch to using a new allocator

3. Rebuild and run
You might also like...
STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.

memory The C++ STL allocator model has various flaws. For example, they are fixed to a certain type, because they are almost necessarily required to b

Jonathan Müller 1.2k Dec 26, 2022
OpenXenium JTAG and Flash Memory programmer
OpenXenium JTAG and Flash Memory programmer

OpenXenium JTAG and Flash Memory programmer * Read: "Home Brew" on ORIGINAL XBOX - a detailed article on why and how * The tools in this repo will all

Koos du Preez 29 Oct 23, 2022
manually map driver for a signed driver memory space

smap manually map driver for a signed driver memory space credits https://github.com/btbd/umap tested system Windows 10 Education 20H2 UEFI installati

ekknod 89 Dec 17, 2022
Memory instrumentation tool for android app&game developers.
Memory instrumentation tool for android app&game developers.

Overview LoliProfiler is a C/C++ memory profiling tool for Android games and applications. LoliProfiler supports profiling debuggable applications out

Tencent 491 Jan 6, 2023
A single file drop-in memory leak tracking solution for C++ on Windows

MemLeakTracker A single file drop-in memory leak tracking solution for C++ on Windows This small piece of code allows for global memory leak tracking

null 22 Jul 18, 2022
Dump the memory of a PPL with a userland exploit
Dump the memory of a PPL with a userland exploit

PPLdump This tool implements a userland exploit that was initially discussed by James Forshaw (a.k.a. @tiraniddo) - in this blog post - for dumping th

Clément Labro 707 Dec 29, 2022
Implementation of System V shared memory (a type of inter process communication) in xv6 operating system.

NOTE: we have stopped maintaining the x86 version of xv6, and switched our efforts to the RISC-V version (https://github.com/mit-pdos/xv6-riscv.git)

Viraj Jadhav 5 Feb 21, 2022
An In-memory Embedding of CPython

An In-memory Embedding of CPython This repository contains all the build artifacts necessary to build an embedding of CPython 3.8.2 that can be run en

null 105 Dec 5, 2022
Execute MachO binaries in memory using CGo

Execute Thin Mach-O Binaries in Memory This is a CGo implementation of the initial technique put forward by Stephanie Archibald in her blog, Running E

Dwight Hohnstein 68 Dec 2, 2022
Comments
  • Unseeded random generator is used, it returns always 0

    Unseeded random generator is used, it returns always 0

    Random number generation algorith actually used (the one not commented out) cannot be used with seed of 0, because it will never change the seed value to anything else and will always generate 0. But default constructor initializes the seed to zero causing the instance is unusable until PRNG::seed() method is called with nonzero value.

    The default constructed PRNG instance without calling PRNG::seed() is used in the actual test to generate slot index and size of the block. So it always picks slot 0, allocates 5 bytes block in it and deallocates it in the next loop. Which makes the test pretty useless.

    opened by mezkov 0
  • compiler support?

    compiler support?

    Curious what compilers you're using. I'm trying to build on CentOS 7 with gcc 5.3.0 and clang 8.0.0 and both give (different) compile errors.

    gcc

    In file included from ../src/allocator_tester.h:54:0,
                 from ../src/allocator_tester.cpp:35:
../src/void_allocator.h: In member function \u2018void* VoidAllocatorForTest<ActualAllocator>::allocateSlots(size_t)\u2019:
../src/void_allocator.h:60:60: error: expected \u2018,\u2019 before \u2018)\u2019 token
  void* allocateSlots( size_t sz ) { static_assert( isFake()); assert( sz <= fakeBufferSize ); return alloc.allocate( sz ); }
                                                            ^
../src/void_allocator.h:60:60: error: expected string-literal before \u2018)\u2019 token
In file included from ../src/allocator_tester.cpp:35:0:
../src/allocator_tester.h: In function \u2018void randomPos_RandomSize(AllocatorUnderTest&, size_t, size_t, size_t, size_t, size_t)\u2019:
../src/allocator_tester.h:285:5: error: expected \u2018(\u2019 before \u2018constexpr\u2019
  if constexpr ( !allocatorUnderTest.isFake() )
     ^
../src/allocator_tester.h:287:2: error: \u2018else\u2019 without a previous \u2018if\u2019
  else
  ^
../src/allocator_tester.h:306:8: error: expected \u2018(\u2019 before \u2018constexpr\u2019
     if constexpr ( doMemAccess )
        ^
../src/allocator_tester.cpp:185:1: error: expected \u2018}\u2019 at end of input
 }
 ^
../src/allocator_tester.cpp:185:1: error: expected \u2018}\u2019 at end of input
../src/allocator_tester.cpp:185:1: error: expected \u2018}\u2019 at end of input

    clang

    btorpey@bt-brix build]$ ./build_gcc.sh
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:46:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<NewDeleteAllocatorForTest, none>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::none>( allocator, testParams->startupParams.iterCount, test...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
      requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
      requested here
                runTest<MyAllocatorT>( &params );
                ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:49:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<NewDeleteAllocatorForTest, full>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::full>( allocator, testParams->startupParams.iterCount, test...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
      requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
      requested here
                runTest<MyAllocatorT>( &params );
                ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:52:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<NewDeleteAllocatorForTest, single>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::single>( allocator, testParams->startupParams.iterCount, te...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
      requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
      requested here
                runTest<MyAllocatorT>( &params );
                ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:55:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<NewDeleteAllocatorForTest, check>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::check>( allocator, testParams->startupParams.iterCount, tes...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization 'runRandomTest<NewDeleteAllocatorForTest>'
      requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:135:3: note: in instantiation of function template specialization 'runTest<NewDeleteAllocatorForTest>'
      requested here
                runTest<MyAllocatorT>( &params );
                ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:46:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, none>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::none>( allocator, testParams->startupParams.iterCount, test...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
      'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
      'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                        runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                        ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:49:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, full>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::full>( allocator, testParams->startupParams.iterCount, test...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
      'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
      'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                        runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                        ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:52:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, single>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::single>( allocator, testParams->startupParams.iterCount, te...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
      'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
      'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                        runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                        ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:257:258: warning: format specifies type 'unsigned long long' but the argument has type 'size_t'
      (aka 'unsigned long') [-Wformat]
  ...[rnd_seed = %llu] ...\n", threadID, allocatorUnderTest.name(), maxItemSizeExp, maxItems, iterCount, memAccessTypeStr, rnd_seed );
                 ~~~~                                                                                                      ^~~~~~~~
                 %zu
../src/allocator_tester.cpp:55:4: note: in instantiation of function template specialization
      'randomPos_RandomSize<VoidAllocatorForTest<NewDeleteAllocatorForTest>, check>' requested here
                        randomPos_RandomSize<Allocator,MEM_ACCESS_TYPE::check>( allocator, testParams->startupParams.iterCount, tes...
                        ^
../src/allocator_tester.cpp:83:19: note: in instantiation of function template specialization
      'runRandomTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                std::thread t1( runRandomTest<Allocator>, (void*)(testParams + i) );
                                ^
../src/allocator_tester.cpp:141:4: note: in instantiation of function template specialization
      'runTest<VoidAllocatorForTest<NewDeleteAllocatorForTest> >' requested here
                        runTest<VoidAllocatorForTest<MyAllocatorT>>( &params );
                        ^
In file included from ../src/allocator_tester.cpp:35:
../src/allocator_tester.h:285:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
../src/allocator_tester.h:434:18: error: constexpr if condition is not a constant expression
        if constexpr ( !allocatorUnderTest.isFake() )
                        ^
8 warnings and 16 errors generated.
    opened by WallStProg 2
Owner
null
GitHub
Allocator bench - bench of various memory allocators

To run benchmarks Install lockless from https://locklessinc.com/downloads/ in lockless_allocator path make Install Hoard from https://github.com/emery

Sam 47 Dec 4, 2022
Mimalloc-bench - Suite for benchmarking malloc implementations.

Mimalloc-bench Suite for benchmarking malloc implementations, originally developed for benchmarking mimalloc. Collection of various benchmarks from th

Daan 186 Dec 24, 2022
Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

rpmalloc - General Purpose Memory Allocator This library provides a public domain cross platform lock free thread caching 16-byte aligned memory alloc

Mattias Jansson 1.7k Dec 28, 2022
Cross-platform shared memory stream/buffer, header-only library for IPC in C/C++.

libsharedmemory libsharedmemory is a small C++11 header-only library for using shared memory on Windows, Linux and macOS. libsharedmemory makes it eas

Aron Homberg 10 Dec 4, 2022
Test cpu and memory speed at linux-vps

Тест скорости процессора и памяти на linux-vps. Занимается бессмысленным перемножением массивов случайных чисел, для определения скорости процессора и

Anton 3 Nov 30, 2021
The Hoard Memory Allocator: A Fast, Scalable, and Memory-efficient Malloc for Linux, Windows, and Mac.

The Hoard Memory Allocator Copyright (C) 1998-2020 by Emery Berger The Hoard memory allocator is a fast, scalable, and memory-efficient memory allocat

Emery Berger 927 Jan 2, 2023
MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

MMCTX (Memory Management ConTeXualizer), is a tiny (< 300 lines), single header C99 library that allows for easier memory management by implementing contexts that remember allocations for you and provide freeall()-like functionality.

A.P. Jo. 4 Oct 2, 2021
Memory-dumper - A tool for dumping files from processes memory

What is memory-dumper memory-dumper is a tool for dumping files from process's memory. The main purpose is to find patterns inside the process's memor

Alexander Nestorov 31 Nov 9, 2022
Mesh - A memory allocator that automatically reduces the memory footprint of C/C++ applications.

Mesh: Compacting Memory Management for C/C++ Mesh is a drop in replacement for malloc(3) that can transparently recover from memory fragmentation with

PLASMA @ UMass 1.5k Dec 30, 2022
Test your malloc protection

Test your allocs protections and leaks ! Report Bug · Request Feature Table of Contents About The Tool Getting Started Prerequisites Quickstart Usage

tmatis 49 Dec 7, 2022