Main gperftools repository

Originally reported on Google Code with ID 200

What steps will reproduce the problem?

I have a multithreaded applicaion that calls LoadLibrary from multiple 
threads. Sometimes the library fails to load with the following error on 
the console:

Check failed: sidestep::SIDESTEP_SUCCESS == 
PreamblePatcher::Patch(windows_fn_[i], perftools_fn_[i], &origstub_fn_[i])

What is the expected output? What do you see instead?

The expected behavior is consistent loading of the libraries, with the same 
result as if without tcmalloc.

What version of the product are you using? On what operating system?

Windows XP, MSVC 7.1, STLPort 5.1.4, perftools 1.4 + patch from issue 199 
and minor modifications for STLPort.

Please provide any additional information below.

The problem is related to issue 199: origstub_fn_[i] is dirty when calling 
PreamblePatcher::RawPatch from LibcInfoWithPatchFunctions<T>::Patch. 
However, the proposed patch from that issue does not help in my case 
because after cleaning origstub_fn_[i] and patching the functions, the 
resulting functions become double-patched. This leads to an infinite loop 
when calling, e.g. operator delete - it calls 
LibcInfoWithPatchFunctions<0>::Perftools_delete, which calls 
LibcInfoWithPatchFunctions<1>::Perftools_delete, which calls 
LibcInfoWithPatchFunctions<0>::Perftools_delete and so on.

The root of the problem is in the PatchAllModules function, which is called 
from LoadLibrary. It appears that CreateToolhelp32Snapshot can fail 
occasionally, which leads to assuming the process has no modules at all and 
marking all module_libcs as invalid. The next call to LoadLibrary, for 
which CreateToolhelp32Snapshot succeeds, will double-patch the modules and 
lead to the described behavior.

I'm not sure what causes CreateToolhelp32Snapshot to fail (in my case 
GetLastError returns 8, Not enough storage is available to process this 
command, but I clearly have enough memory). MSDN mentions that it may 
spuriously return ERROR_BAD_LENGTH (24). Probably, when multiple threads 
call to LoadLibrary, CreateToolhelp32Snapshot fails to synchronize with the 
library manager properly, even though the call is for the current process.

One other thing I noticed. The patch_all_modules_lock in PatchAllModules is 
locked after traversing the modules snapshot. But module_libcs is used 
during the traverse, which may lead to a race condition. However, moving 
patch_all_modules_lock before it does not help my situation.

Reported by andrey.semashev on 2009-12-24 07:16:39

Originally reported on Google Code with ID 532

We run tcmalloc on our very high throughput MySQL master. Join buffers > kMaxPages are
allocated by a large percentage of client threads. Over time, as fragmentation increases,
the large_.returned list grows to contain thousands of Spans, which makes PageHeap::AllocLarge
extremely slow and the pageheap_lock becomes a major source of contention, eventually
slowing down MySQL considerably.

I have written a patch which maintains a doubly linked skiplist to calculate address-ordered
best-fit in amortized O(log(n)) instead of O(n) when the combined size of the large_
normal and returned lists exceed a configurable threshold. We're running it in production
now and according to perf top, the percentage of time spent in PageHeap::AllocLarge
has gone from ~8% overall to ~0.4%.

However, while the time complexity of the skiplist is favorable, its space complexity
(each node is 168 bytes) is obviously substantially less favorable than something like
a red/black tree. I haven't had a chance to do my own measurements yet, but it seems
from everything I've read that allocator metadata size has been shown to have a substantial
impact on CPU cache efficiency, so while this patch may lessen a scalability bottleneck,
it may have negative performance effects. I'm up for implementing a different data
structure for this if that has a better chance of being accepted.

I've also written a small test program that attempts to create a sizeable large_ list,
and then times 100000 allocations and frees from the large list. It's far from perfect,
but it's something.

Patch: https://github.com/jamesgolick/gperftools/compare/master...skiplist-trunk
Test program: https://gist.github.com/jamesgolick/cffa78146af264c191fb

Reported by jamesgolick on 2013-05-21 23:38:00

Originally reported on Google Code with ID 83

This is a patch file that takes steps towards finishing the 
windows port of the heap profiler.  It doesn't address CPU profiling 
or heap checking.  It has been tested against some fairly large profilees 
on windows.  Some of the changes in the patch are fixes to issues that 
would affect tcmalloc even in the absence of profiling.

It's possible I may have lost some changes when creating the patch (we 
have some other modifications I am not sharing at this time). 

Please ask questions.

Changes: 
- Use the HANDLE API to dump the profile instead of file descriptors. 

- Fix a couple bugs where there were dependences on the global 
constructor order, which is undefined 

- Make sure the pad member of the Central_Freelist is not of length 0 

- Do not unpatch the windows functions, ever.  This is dangerous since 
you would be depending on global destructor order. 

- Do not report large allocations using printf (or at all for now) 
since printf isn't so safe to call in some processes 

- If we are asked to free something allocated by the CRT heap, then do 
a nop.  This is necessary because a bunch of stuff gets allocated 
before our hooks are in place (global c++ constructors, parts of 
environ, other things?) 

- Hook _msize and _expand.  Otherwise, very subtle bugs can occur. 
Even in programs that don't use these, since the linker/compiler 
introduces calls to _msize for global c++ constructors occurring in 
dlls. 

- Make realloc work if it is given something that was originally 
allocated by the CRT heap. 

- Make pprof forgiving of carriage returns.  Editing pprof is the sum 
total of my life experience using perl, so I've probably done 
something stupid. 

- Added stack walker using RtlCaptureStackBackTrace, which doesn't 
work so well with FPO (I turned this walker on by default) 

- Wrote ad-hoc stack walker, which sometimes works with FPO, but 
sometimes gets a snapshot from an earlier time due to left over stack junk.  
It is possible that this one is faster than the previous one if you turn 
off the heuristics -- I didn't test that. 

- Leak all spinlocks since you cannot count on the global destructor 
order. 

- Hook all instances of the CRT in the process, not just an arbitrary 
one 

- Hook LoadLibraryExW in order to detect dynamic loads of the CRT 

- WART: Using FreeLibrary to free the CRT is dangerous if you later 
LoadLibrary it again.  So we aren't perfect yet. 

- Fix some compile errors with VC8 

- Wrote nm and addr2line programs that use PDB symbols.  Did not add 
these to any build scripts.  You must download Debugging Tools For 
Windows and copy dbghelp.dll and symsrv.dll into the directory of 
nm.exe and addr2line.exe for them to function.  There exist old 
versions of these dlls that do not function. 

- Modified pprof to fall back on these pdb versions of nm and 
addr2line if they are siblings of pprof.  Processes linking in a 
mixture of gcc compiled and cl compiled dlls seem to get the union of 
the symbols. 

Reported by dvitek on 2008-10-16 23:27:13

Originally reported on Google Code with ID 243

What steps will reproduce the problem?
1. configure
2. make
3. make check

What is the expected output? What do you see instead?
0 tests should fail. Instead, 3 out of 38 tests fail.

What version of the product are you using? On what operating system?
Tried both 1.5 and trunk on Mac OS X 10.6.3.

Please provide any additional information below.

Running the 'heap-profiler_unittest' binary manually gives this output:

Starting tracking the heap
Had other new/delete MallocHook-s set. Are you using the heap leak checker? Use --
heap_check="" to avoid this conflict.
Abort trap

Reported by neunon on 2010-05-20 23:15:05

Originally reported on Google Code with ID 311

What steps will reproduce the problem?

1. build the tools with --enable-heap-check

2. write a small program. Compile and link it with libtcmalloc.

#include <stdio.h>
#include <stdlib.h>


int main(int argc, char** argv)
{
    printf("********** start **********\n");
    int* i1 = new int(1000);
    int* i2 = new int(1000);
    int* i3 = new int(1000);
    int* i4 = new int(1000);
    int* i5 = new int(1000);
    int* i6 = new int(1000);
    printf("*********** end ***********\n");

    return 0;
}

$ g++ -L/usr/local/lib main.cpp -o main -ltcmalloc


3. Run the heap checker as follows: 

$ export PERFTOOLS_VERBOSE=10
$ HEAPCHECK=normal ./main
Unable to open /proc/self/environ, falling back on getenv("HEAPCHECK"), which may not
work
MemoryRegionMap Init
MemoryRegionMap Init done
Starting tracking the heap
Found hooked allocator at 3: 0x680a2a69 <- 0x682a5b8f
Found hooked allocator at 2: 0x680a2a69 <- 0x682a3114
Found hooked allocator at 2: 0x680a2a69 <- 0x6808713a
Found hooked allocator at 2: 0x680a2a69 <- 0x682a1b51
Found hooked allocator at 2: 0x680a2a69 <- 0x68087154
Found hooked allocator at 2: 0x680a0e03 <- 0x68087172
Found hooked allocator at 2: 0x680a2a69 <- 0x682a3114
Found hooked allocator at 2: 0x680a2a69 <- 0x682a7a04
Found hooked allocator at 2: 0x680a0e03 <- 0x68097b4a
Going to ignore live object at 0x80cd000 of 4 bytes
Found hooked allocator at 2: 0x680a2a69 <- 0x682a698a
Found hooked allocator at 2: 0x680a0e03 <- 0x681330ff
Found hooked allocator at 2: 0x680a0e03 <- 0x68095fae
Found hooked allocator at 2: 0x680a0e03 <- 0x681330ff
Found hooked allocator at 2: 0x680a0e03 <- 0x681330ff
Found hooked allocator at 2: 0x680a0e03 <- 0x681330ff
No shared libs detected. Will likely report false leak positives for statically linked
executables.
Turning perftools heap leak checking off
MemoryRegionMap Shutdown
MemoryRegionMap Shutdown done
********** start **********
*********** end ***********
$ 


What is the expected output? What do you see instead?

As you can see, we hit a condition in the heap check startup/init that causes heap
checking to be disabled as indicated by the message:

No shared libs detected. Will likely report false leak positives for statically linked
executables.
Turning perftools heap leak checking off

What version of the product are you using? On what operating system?

google-perftools-1.7/FreeBSD6

Please provide any additional information below.

Here is the code snippet where the heap check is bailing out:

src/heap-checker.cc

HeapLeakChecker::ProcMapsResult HeapLeakChecker::UseProcMapsLocked(
                                  ProcMapsTask proc_maps_task) {
  RAW_DCHECK(heap_checker_lock.IsHeld(), "");
  // Need to provide own scratch memory to ProcMapsIterator:
  ProcMapsIterator::Buffer buffer; 
  ProcMapsIterator it(0, &buffer);
  if (!it.Valid()) {
    int errsv = errno;
    RAW_LOG(ERROR, "Could not open /proc/self/maps: errno=%d. "
                   "Libraries will not be handled correctly.", errsv); 
    return CANT_OPEN_PROC_MAPS;
  }
  uint64 start_address, end_address, file_offset;
  int64 inode;
  char *permissions, *filename;
  bool saw_shared_lib = false;
  while (it.Next(&start_address, &end_address, &permissions,
                 &file_offset, &inode, &filename)) {
    if (start_address >= end_address) {
      // Warn if a line we can be interested in is ill-formed:
      if (inode != 0) { 
        RAW_LOG(ERROR, "Errors reading /proc/self/maps. "
                       "Some global memory regions will not "
                       "be handled correctly.");
      }
      // Silently skip other ill-formed lines: some are possible
      // probably due to the interplay of how /proc/self/maps is updated
      // while we read it in chunks in ProcMapsIterator and
      // do things in this loop.
      continue;
    }
    // Determine if any shared libraries are present.
    if (inode != 0 && strstr(filename, "lib") && strstr(filename, ".so")) {
      saw_shared_lib = true; 
    }
    switch (proc_maps_task) {
      case DISABLE_LIBRARY_ALLOCS:
        // All lines starting like
        // "401dc000-4030f000 r??p 00132000 03:01 13991972  lib/bin"
        // identify a data and code sections of a shared library or our binary
        if (inode != 0 && strncmp(permissions, "r-xp", 4) == 0) { 
          DisableLibraryAllocsLocked(filename, start_address, end_address);
        }       
        break;  
      case RECORD_GLOBAL_DATA:
        RecordGlobalDataLocked(start_address, end_address,
                               permissions, filename);
        break;  
      default:
        RAW_CHECK(0, "");
    }
  }
******************************* HERE *********************************  
  if (!saw_shared_lib) {
    RAW_LOG(ERROR, "No shared libs detected. Will likely report false leak "
                   "positives for statically linked executables.");
    return NO_SHARED_LIBS_IN_PROC_MAPS;
  }
  return PROC_MAPS_USED;
}


The documentation seems to suggest that the heap check is only currently supported
on Linux. I suspect that whatever linux specific code could be made platform independent
or easily ported to work with FreeBSD. I would like to add support for FreeBSD but
need a bit of a background overview on the Linux specific bits that would need to change.
Is someone willing to get me up to speed here to get this work underway? 

Regards,

Dave

Reported by chappedm on 2011-02-11 22:01:19

Originally reported on Google Code with ID 246

What steps will reproduce the problem?

The problem is not stable, I have no concrete guide to reproduce it. It 
does seem, however, happen when an application attempts to start a thread 
via pthread_create. I've seen other crashes with no threading involved (at 
least, on the application level).

What is the expected output? What do you see instead?

The application should not crash

What version of the product are you using? On what operating system?

PerfTools 1.4, patched according to ticket #201. RHEL4.

uname -a
Linux ~~~~~~ 2.6.9-78.ELsmp #1 SMP Wed Jul 9 15:39:47 EDT 2008 i686 i686 
i386 GNU/Linux

rpm -qa | grep glibc
glibc-common-2.3.4-2.41
glibc-devel-2.3.4-2.41
glibc-2.3.4-2.41
glibc-headers-2.3.4-2.41
glibc-kernheaders-2.4-9.1.103.EL

/lib/libc.so.6
GNU C Library stable release version 2.3.4, by Roland McGrath et al.
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.4.6 20060404 (Red Hat 3.4.6-9).
Compiled on a Linux 2.4.20 system on 2008-04-15.
Available extensions:
        GNU libio by Per Bothner
        crypt add-on version 2.1 by Michael Glad and others
        linuxthreads-0.10 by Xavier Leroy
        The C stubs add-on version 2.1.2.
        BIND-8.2.3-T5B
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
        Glibc-2.0 compatibility add-on by Cristian Gafton 
        GNU Libidn by Simon Josefsson
        libthread_db work sponsored by Alpha Processor Inc
Thread-local storage support included.
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.

Please provide any additional information below.

Our tests on RHEL4 are randomly crashing. The tests involve different code, 
some are multithreaded, some are not. All modules are linked with 
libtcmalloc_minimal.so.

I managed to recover a stack of one crash (which is not very easy, since 
it's on a remote server):

27/05/10 05:28:45 Using host libthread_db library 
"/lib/tls/libthread_db.so.1".
27/05/10 05:28:45 Core was generated by `/home/nb/KERN-
4.1/4.1/ASAPkernel/dev/unix/dll/i686-pc-linux-gnu-gcc/Release/CBO'.
27/05/10 05:28:45 Program terminated with signal 11, Segmentation fault.
27/05/10 05:28:47 #0  0x002c199e in do_lookup_x () from /lib/ld-linux.so.2
27/05/10 05:28:47 
27/05/10 05:28:47 Thread 1 (process 23039):
27/05/10 05:28:47 #0  0x002c199e in do_lookup_x () from /lib/ld-linux.so.2
27/05/10 05:28:47 #1  0x002c1e22 in _dl_lookup_symbol_x () from /lib/ld-
linux.so.2
27/05/10 05:28:47 #2  0x002c51d6 in fixup () from /lib/ld-linux.so.2
27/05/10 05:28:47 #3  0x002c5110 in _dl_runtime_resolve () from /lib/ld-
linux.so.2
27/05/10 05:28:47 #4  0x08066419 in CrashHandler (sig=11) at 
./src/SignalHandlerPosix.cpp:230
27/05/10 05:28:47 #5  <signal handler called>
27/05/10 05:28:47 #6  0x002c199e in do_lookup_x () from /lib/ld-linux.so.2
27/05/10 05:28:47 #7  0x002c1e22 in _dl_lookup_symbol_x () from /lib/ld-
linux.so.2
27/05/10 05:28:47 #8  0x002c51d6 in fixup () from /lib/ld-linux.so.2
27/05/10 05:28:47 #9  0x002c5110 in _dl_runtime_resolve () from /lib/ld-
linux.so.2
27/05/10 05:28:47 #10 0x003d4fb9 in tcmalloc::CentralFreeList::InsertRange 
()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #11 0x003d8683 in 
tcmalloc::ThreadCache::ReleaseToCentralCache ()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #12 0x003d879a in tcmalloc::ThreadCache::Scavenge ()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #13 0x003cfb67 in (anonymous 
namespace)::do_free_with_callback ()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #14 0x003d1f6f in MallocBlock::ProcessFreeQueue ()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #15 0x003ce407 in DebugDeallocate ()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #16 0x003de860 in realloc ()
27/05/10 05:28:47    from /home/nb/KERN-
4.1/4.1/ASAPkernel/SubstituteBin/i686-pc-linux-gnu-
gcc/Release/libtcmalloc_minimal.so.0
27/05/10 05:28:47 #17 0x00525c44 in pthread_create@@GLIBC_2.1 () from 
/lib/tls/libpthread.so.0
27/05/10 05:28:47 #18 0x0806895f in Impl (this=0x942f990) at 
./src/SignalHandlerPosix.cpp:328
27/05/10 05:28:47 #19 0x0806751e in SignalHandler (this=0x942f9b0)
27/05/10 05:28:47     at ./src/SignalHandlerPosix.cpp:344
27/05/10 05:28:47 #20 0x080598ca in StartUp (lg=@0xbffaff30, 
bUseConsoleHandler=true, 
27/05/10 05:28:47     fileName=0x946b3e0 "localsettings.xml") at 
./src/StartUp.cpp:30
27/05/10 05:28:47 #21 0x080559a6 in (anonymous namespace)::COMMain 
(params=@0xbffaff7c)
27/05/10 05:28:47     at ./src/CBOSSinMain.cpp:81
27/05/10 05:28:47 #22 0x08055c83 in main (argc=1, argv=0xbffb0074) at 
./src/CBOSSinMain.cpp:130

Note that libtcmalloc_minimal.so.0 is actually a renamed 
libtcmalloc_minimal_debug.so.0. I did so in attempt to debug our 
applications and this particular problem.

Here the application is at its very startup, it attempts to create a thread 
to wait for signals. Apparently, the crash appears when the dynamic linker 
attempts to resolve a symbol in run time. The crash handler installed by 
the application fails to do anything useful since it also triggers symbol 
resolution. In other crashes the application was also in its very early 
stage, but I don't have stacks from these.

The problem is very hard to reproduce, out of ~1800 tests we run each 
night, about 5-10 of them fail with these sympthoms, almost every time 
different ones.

I'm suspecting that this is a bug in glibc, but I'm not sure. I'm not very 
knowledgeable in the interworkings of the dynamic linker to go digging into 
it.

I have a suggestion of a possible workaround in perftools, though. I tried 
to compile it with an additional linker flag "-Wl,-z -Wl,now", which would 
force the linker to resolve all symbols in the tcmalloc library immediately 
at its load time, instead of lazily, which is the default. I did one tests 
turnaround and it did not show any tests with the described crash. I'll 
keep monitoring, though. However, I suggest adding the mentioned flag to 
the library makefiles.

Reported by andrey.semashev on 2010-05-28 05:22:47

Originally reported on Google Code with ID 361

What steps will reproduce the problem?
1. cd google-perftools-1.8.2/ && patch -p1 < ../google-perftools-1.8.2-mips.patch
2. -- configure --  *ensure that you read the README_mips.txt
3. make
4. Copy the shared/static libraries and link

What is the expected output? What do you see instead?

Existing MIPS support is not available. No atomicops are present, and some of the system
call infrastructure needs tweaking. This patch addresses these issues. This patch DOES
NOT add full featured perftools support.

What version of the product are you using? On what operating system?

This patch is generated against 1.8.2 of the google perf tools (which is the latest
at the time of this writing). 

Please provide any additional information below.

This patch has only been thoroughly tested on the octeon hardware platform, with mips32
application running under linux. Mips64 has been tested only insofar as the unit tests
seem to pass. Your mileage may vary.

Reported by [email protected] on 2011-08-23 16:19:37

attempting to use malloc_usable_size() with tcmalloc_debug simply hangs here: #0 pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:94 #1 0x00007f10b280bb27 in MallocExtension::instance () at src/malloc_extension.cc:207 #2 0x00007f10b28159e9 in tc_malloc_size (ptr=0x1010020) at src/debugallocation.cc:1419

(gdb) f 1 #1 0x00007f10b280bb27 in MallocExtension::instance () at src/malloc_extension.cc:207

207 perftools_pthread_once(&module_init, InitModule); (gdb) p module_init $1 = 1 (gdb) p InitModule $2 = {void (void)} 0x7f10b280b580 <InitModule()> (gdb) f 0 #0 pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:94

94 jmp 6b (gdb) l 89 orl %fs:PRIVATE_FUTEX, %esi 90 # endif 91 #endif 92 movl $SYS_futex, %eax 93 syscall 94 jmp 6b 95 96 /* Preserve the pointer to the control variable. */ 97 3: pushq %rdi 98 cfi_adjust_cfa_offset(8)

any ideas on what could be going wrong here..

Hi

Thanks for great tool.

I cross-compile it for powerPC with GNU compilers 4.1.1 and it works fine.

Problem is that I also need it for ARM. I configure it with options --host=arm-926ejs-linux-gnueabi --with-pic=no --enable-static=no --disable-cpu-profiler --disable-heap-checker --disable-debugalloc Once I use it on ARM machine with LD_PRELOAD, linux reports Bus error.

Do you have any idea what can be wrong?

best regards Stevo

The error message : In file included from /var/tmp/portage/dev-util/google-perftools-2.5/work/gperftools-2.5/src/malloc_hook.cc:678:0: /var/tmp/portage/dev-util/google-perftools-2.5/work/gperftools-2.5/src/malloc_hook_mmap_linux.h: In function 'void* do_mmap64(void*, size_t, int, int, int, __off64_t)' :

/var/tmp/portage/dev-util/google-perftools-2.5/work/gperftools-2.5/src/malloc_hook_mmap_linux.h:86:30: error: 'SYS_mmap2' was not declared in this scope
 result = (void *)syscall(SYS_mmap2,
                          ^

Makefile:4734: recipe for target 'src/libtcmalloc_minimal_internal_la-malloc_hook.lo' failed make: *** [src/libtcmalloc_minimal_internal_la-malloc_hook.lo] Error 1

• ERROR: dev-util/google-perftools-2.5::gentoo failed (compile phase):
• emake failed
• If you need support, post the output of emerge --info '=dev-util/google-perftools-2.5::gentoo',
• the complete build log and the output of emerge -pqv '=dev-util/google-perftools-2.5::gentoo'.
• The complete build log is located at '/var/tmp/portage/dev-util/google-perftools-2.5/temp/build.log'.
• The ebuild environment file is located at '/var/tmp/portage/dev-util/google-perftools-2.5/temp/environment'.
• Working directory: '/var/tmp/portage/dev-util/google-perftools-2.5/work/gperftools-2.5-abi_mips_n32.n32'
• S: '/var/tmp/portage/dev-util/google-perftools-2.5/work/gperftools-2.5'

Originally reported on Google Code with ID 168

What steps will reproduce the problem?
1. Compiling on Snow Leopard

What is the expected output? What do you see instead?
A working version

What version of the product are you using? On what operating system?
Snow Leopard, Perftools 1.4

Please provide any additional information below.
Compiling fails with the following error:

creating libtcmalloc_debug.la
(cd .libs && rm -f libtcmalloc_debug.la && ln -s ../libtcmalloc_debug.la
libtcmalloc_debug.la)
if /bin/sh ./libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.
-I./src  -I./src   -Wall -Wwrite-strings -Woverloaded-virtual
-Wno-sign-compare  -DNO_FRAME_POINTER -g -O2 -MT profiler.lo -MD -MP -MF
".deps/profiler.Tpo" -c -o profiler.lo `test -f 'src/profiler.cc' || echo
'./'`src/profiler.cc; \
    then mv -f ".deps/profiler.Tpo" ".deps/profiler.Plo"; else rm -f
".deps/profiler.Tpo"; exit 1; fi
 g++ -DHAVE_CONFIG_H -I. -I. -I./src -I./src -Wall -Wwrite-strings
-Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g -O2 -MT
profiler.lo -MD -MP -MF .deps/profiler.Tpo -c src/profiler.cc  -fno-common
-DPIC -o .libs/profiler.o
In file included from src/profiler.cc:37:
src/getpc.h:178: error: expected ‘,’ or ‘...’ before ‘&’ token
src/getpc.h:178: error: ISO C++ forbids declaration of ‘ucontext_t’ with no
type
src/getpc.h: In function ‘void* GetPC(int)’:
src/getpc.h:179: error: ‘signal_ucontext’ was not declared in this scope
src/profiler.cc: At global scope:
src/profiler.cc:49: error: conflicting declaration ‘typedef int ucontext_t’
/usr/include/sys/_structs.h:227: error: ‘ucontext_t’ has a previous
declaration as ‘typedef struct __darwin_ucontext ucontext_t’
In file included from src/base/commandlineflags.h:55,
                 from src/profiler.cc:55:
./src/base/basictypes.h: In constructor
‘AssignAttributeStartEnd::AssignAttributeStartEnd(const char*, char**,
char**)’:
./src/base/basictypes.h:251: warning: ‘_dyld_present’ is deprecated
(declared at /usr/include/mach-o/dyld.h:237)
./src/base/basictypes.h:251: warning: ‘_dyld_present’ is deprecated
(declared at /usr/include/mach-o/dyld.h:237)
src/profiler.cc: In static member function ‘static void
CpuProfiler::prof_handler(int, siginfo_t*, void*, void*)’:
src/profiler.cc:266: error: cannot convert ‘__darwin_ucontext’ to ‘int’ for
argument ‘1’ to ‘void* GetPC(int)’
make: *** [profiler.lo] Error 1

Reported by sebastian.hillig on 2009-09-16 09:07:56

build error when using -std=c++17 or -std=c++20: benchmark/malloc_bench.cc:236:8: error: no member named 'random_shuffle' in namespace 'std'

reason: std::random_shuffle is removed in the new C++ standard.

solution: as the example here: https://en.cppreference.com/w/cpp/algorithm/random_shuffle

So, in pprof there's a --base argument to exclude allocations that were not made until after a certain point.

I'd like to request a similar argument that allows us to exclude allocations that were not live after a given point.

For example, if I have 3 heap files: 0001.heap, 0002.heap and 0003.heap, I only want to show the memory that was allocated when 0002.heap was dumped, and specifically exclude those allocations that were freed before 0003.heap was allocated.

Invoking ReleaseFreeMemory seems to only release the "page heap freelist". How can I release the other freelists?

------------------------------------------------
MALLOC:         784216 (    0.7 MiB) Bytes in use by application
MALLOC: +        49152 (    0.0 MiB) Bytes in page heap freelist
MALLOC: +       465456 (    0.4 MiB) Bytes in central cache freelist
MALLOC: +       144640 (    0.1 MiB) Bytes in transfer cache freelist
MALLOC: +      1169784 (    1.1 MiB) Bytes in thread cache freelists
MALLOC: +       409600 (    0.4 MiB) Bytes in malloc metadata
MALLOC:   ------------
MALLOC: =      3022848 (    2.9 MiB) Actual memory used (physical + swap)
MALLOC: +       532480 (    0.5 MiB) Bytes released to OS (aka unmapped)
MALLOC:   ------------
MALLOC: =      3555328 (    3.4 MiB) Virtual address space used
MALLOC:
MALLOC:            391              Spans in use
MALLOC:              6              Thread heaps in use
MALLOC:           4096              Tcmalloc page size

How do I set a page size of 4 kB? I can see there are at least two different build switches that I believe refers to the same thing?

There is a parameter called TCMALLOC_PAGE_SIZE_SHIFT that seems to allow me to change to 4 kB (shift of 12) https://github.com/gperftools/gperftools/blob/master/cmake/config.h.in#L284

The top cmake recipe has a parameter called gperftools_tcmalloc_pagesize but that only allows setting a page size down to 8 kB. https://github.com/gperftools/gperftools/blob/master/CMakeLists.txt#L122

What is the correct way to set 4 kB page size?

Anyone used this software for php? If so how? I compile release version 2.10 using:

./autogen.sh
./configure
make -j9
make install

I put this in php.ini:

extension="memprof.so"

And then I run php and get:

Unable to load dynamic library 'memprof.so'

And all around google it seems that no one ever discussed this error on any forums! What does the compile script do? What does it install? How to use this software?