Hi,
I have one broker process running from long time and with ps
I can see the process, but it is not accepting connections.
vinod_kumar@mqtt-broker-2:~$ ps aux|grep -i mosquitto
vinod_k+ 3865 0.0 0.0 8212 2164 pts/2 S+ 05:11 0:00 grep --color=auto -i mosquitto
mosquit+ 25389 0.1 0.1 39688 5652 ? R 2016 594:16 mosquitto -c mosquitto.conf -d
vinod_kumar@mqtt-broker-2:~$ mosquitto_sub -t hello -d
Error: Connection refused
The process is using far more than normal cpu usage
Output of top
-
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25389 mosquit+ 20 0 39688 5652 4188 R 99.9 0.1 584:37.48 mosquitto
I am using old version of mosquitto, so not sure if this issue is fixed in newer releases, though I checked and could not find any fix directly stating the problem I am facing.
Mosquitto version
mosquitto version 1.4.5 (build date Mon, 09 Nov 2015 15:00:46 +0000)
mosquitto is an MQTT v3.1 broker.
OS version
vinod_kumar@mqtt-broker-2:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 15.04
Release: 15.04
Codename: vivid
Mosquitto conf
vinod_kumar@mqtt-broker-2:~$ cat mosquitto.conf
autosave_interval 100
persistence true
persistence_file m2.db
persistence_location ./
connection_messages true
log_timestamp true
listener 1883
listener 1884 127.0.0.1
protocol websockets
allow_duplicate_messages false
max_connections -1
max_inflight_messages 50
max_queued_messages 1000
persistent_client_expiration 3m
sys_interval 1
Output of memleax
vinod_kumar@mqtt-broker-2:~$ sudo memleax 25389
Warning: no debug-line found for /usr/sbin/mosquitto
== Begin monitoring process 25389...
CallStack[1]: memory expires with 80 bytes, backtrace:
0x00007fb3dcc5d4a0 libc-2.21.so __libc_malloc()+0 /build/buildd/glibc-2.21/malloc/hooks.c:77
0x000000000040e666 mosquitto
0x000000000040c291 mosquitto
0x000000000040c566 mosquitto
0x000000000041a783 mosquitto
0x000000000040daac mosquitto
0x0000000000404a58 mosquitto
0x00007fb3dcbf9a40 libc-2.21.so __libc_start_main()+240 /build/buildd/glibc-2.21/stdio-common/_i18n_number.h:117
0x0000000000404c87 mosquitto
CallStack[2]: memory expires with 1 bytes, backtrace:
0x00007fb3dcc5d4a0 libc-2.21.so __libc_malloc()+0 /build/buildd/glibc-2.21/malloc/hooks.c:77
0x00007fb3dcc6442a libc-2.21.so strdup()+26 /build/buildd/glibc-2.21/string/strdup.c:42
0x000000000040e736 mosquitto
0x000000000040c2ba mosquitto
0x000000000040c566 mosquitto
0x000000000041a783 mosquitto
0x000000000040daac mosquitto
0x0000000000404a58 mosquitto
0x00007fb3dcbf9a40 libc-2.21.so __libc_start_main()+240 /build/buildd/glibc-2.21/stdio-common/_i18n_number.h:117
0x0000000000404c87 mosquitto
CallStack[3]: memory expires with 19 bytes, backtrace:
0x00007fb3dcc5d4a0 libc-2.21.so __libc_malloc()+0 /build/buildd/glibc-2.21/malloc/hooks.c:77
0x00007fb3dcc6442a libc-2.21.so strdup()+26 /build/buildd/glibc-2.21/string/strdup.c:42
0x000000000040e736 mosquitto
0x000000000040c2f8 mosquitto
0x000000000040c566 mosquitto
0x000000000041a783 mosquitto
0x000000000040daac mosquitto
0x0000000000404a58 mosquitto
0x00007fb3dcbf9a40 libc-2.21.so __libc_start_main()+240 /build/buildd/glibc-2.21/stdio-common/_i18n_number.h:117
0x0000000000404c87 mosquitto
CallStack[4]: memory expires with 16 bytes, backtrace:
0x00007fb3dcc5d4a0 libc-2.21.so __libc_malloc()+0 /build/buildd/glibc-2.21/malloc/hooks.c:77
0x000000000040e666 mosquitto
0x000000000040c3c6 mosquitto
0x000000000040c566 mosquitto
0x000000000041a783 mosquitto
0x000000000040daac mosquitto
0x0000000000404a58 mosquitto
0x00007fb3dcbf9a40 libc-2.21.so __libc_start_main()+240 /build/buildd/glibc-2.21/stdio-common/_i18n_number.h:117
0x0000000000404c87 mosquitto
CallStack[2]: expired-memory frees after 101 seconds
CallStack[3]: expired-memory frees after 101 seconds
CallStack[4]: expired-memory frees after 101 seconds
CallStack[1]: expired-memory frees after 101 seconds
CallStack[1]: memory expires with 80 bytes, 2 times again
CallStack[2]: memory expires with 1 bytes, 2 times again
CallStack[3]: memory expires with 19 bytes, 2 times again
CallStack[4]: memory expires with 16 bytes, 2 times again
CallStack[2]: expired-memory frees after 101 seconds
CallStack[3]: expired-memory frees after 101 seconds
CallStack[4]: expired-memory frees after 101 seconds
CallStack[1]: expired-memory frees after 101 seconds
CallStack[1]: memory expires with 80 bytes, 3 times again
CallStack[2]: memory expires with 1 bytes, 3 times again
CallStack[3]: memory expires with 19 bytes, 3 times again
CallStack[4]: memory expires with 16 bytes, 3 times again
CallStack[2]: expired-memory frees after 101 seconds
Warning: too many expired-free at CallStack[2]. will not show this CallStack later
CallStack[3]: expired-memory frees after 101 seconds
Warning: too many expired-free at CallStack[3]. will not show this CallStack later
CallStack[4]: expired-memory frees after 101 seconds
Warning: too many expired-free at CallStack[4]. will not show this CallStack later
CallStack[1]: expired-memory frees after 101 seconds
Warning: too many expired-free at CallStack[1]. will not show this CallStack later
Output of strace -
Process 25389 attached
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473126553}) = 0
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {34451458, 473166932}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473183767}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473200917}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473217392}) = 0
rt_sigprocmask(SIG_SETMASK, [INT], [], 8) = 0
poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=POLLIN}], 4, 100) = 1 ([{fd=3, revents=POLLNVAL}])
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473295297}) = 0
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {34451458, 473352136}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473379862}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473397232}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473413209}) = 0
rt_sigprocmask(SIG_SETMASK, [INT], [], 8) = 0
poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=POLLIN}], 4, 100) = 1 ([{fd=3, revents=POLLNVAL}])
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473486496}) = 0
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {34451458, 473518710}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473534360}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473550677}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473566255}) = 0
rt_sigprocmask(SIG_SETMASK, [INT], [], 8) = 0
poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=POLLIN}], 4, 100) = 1 ([{fd=3, revents=POLLNVAL}])
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473637297}) = 0
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {34451458, 473677707}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473693527}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473709630}) = 0
clock_gettime(CLOCK_MONOTONIC, {34451458, 473725872}) = 0
rt_sigprocmask(SIG_SETMASK, [INT], [], 8) = 0
poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=POLLIN}], 4, 100) = 1 ([{fd=3, revents=POLLNVAL}])
Please let me know if I can provide anything else to help debug the issue.
Type: Bug Component: mosquitto-broker Status: Blocked