I don't find any documentation on how to load an application into ArangoDB. A better explanation of the arango.conf format would also be great, what options are required for setting up an application properly. For example I would like the default url http://127.0.0.1:8529 point to my app which is mapped to an arbitrary url/path.

How is this done ?

I'm currently on version 1.0.alpha2

I'm suddenly experiencing failures when creating cursors using the nodejs arango-client ArangoDB gets unresponsive and CTRL-C doesn't terminate the process. This message is shown several times after issuing ctrl-c.

2012-09-04T10:17:33Z [9149] WARNING forcefully shutting down thread 'dispatcher'

To replicate, install the arango-client with devRequirements and run the tests.

git clone git://github.com/kaerus/arango-client.git
cd arango-client
npm install -d
npm test

By the way, Arangodb-1.0.0 install procedure (sudo make install) didn't create a log directory so the process failed to launch.

I took the liberty to create a feature request out of this discussion: https://groups.google.com/forum/?fromgroups=#!topic/arangodb/3Wt4c44mc0I

Excerpt: 2) ArangoDB Server provides date handling functionality.

a) As date is a special type, I believe it could be handled that way. This can be provided by defining attributes in a collection as of special type 'date'. This could look like this:

db.myCollection.defineAttributeType('my.attribute', 'date');  
// This would for example set the nested attribute 'attribute' as a date field.

... for a nested document attribute inside a list, it could be:

db.myCollection.defineAttributeType('my[*].attribute', 'date');
  // This would for example set the nested attribute 'attribute' as a date field. ```

... or for a list that would contain dates:

db.myCollection.defineAttributeType('my[*]', 'date');
  // This would for example set the contents of a list as a date field

... mixing both could also be possible. So one could have a list of direct values and nested ones:

 {"my":[{"attribute":"2012-11-30T22:24:32.43Z"}, "2012-11-30T22:24:32.43Z"]}

b) The user would pass datetime strings like { "my":{"attribute":"2012-11-30T22:24:32.43Z"}} or maybe other formats like { "my":{"attribute":"2012-11-30"}}... etc

  Internally, ArangoDB would convert and store those as timestamps.
  Knowing that it's a date field, it would automatically convert the datetime strings to timestamps for storing.

  That way, no special JSON handling is required by either the server or the client.
  Error handling: if the datetime string would have invalid data, the operation should be rejected.

c) With those would of course come some datetime functions to query (AQL) Simple queries would probably not need anything special, as they are limited anyway. AQL queries:

DATE_MATH ("{mathcommand like add || subtract}", {date field to operate on, or a value}, {the modifying value , or a string like INTERVAL 1 WEEK'} );
 // Function to do math on a field or value

DATE_CURRENT (); Gets current date on server.

...maybe more, that I haven't thought of just yet. But you get the idea :)

What do you think?

Besides my main Linux installations of ArangoDB, I also occasionally use a local installation on my Windows 7 (x64). Mainly for running ArangoDB-PHP tests also on Windows.

So, I upgraded the Windows installation a few days ago, from 1.3.1(I think it was) to 1.4.0.

Today I was testing the ArangoDB-PHP Driver and the tests were suddenly failing (only locally on the windows machine). I closed the Batch window, and reopened it, and the tests were running successfully. I'd like to note here, that I hadn't turned off my machine since yesterday, so that batch window was open for at least 12 hours if I remember correctly.

I also noticed that the database files are not deleted from the data directory (The ones that are created and deleted through the ArangoDB-PHP test suite.)

I couldn't attach a zip file here, so I am sending the zipped data directory via mail to @jsteemann (Please also check Spam folder :smile: ) for analysis.

Hi I am having an issue starting up the coordinator using version 3.1.8. I am not sure if this is a bug or user error.

This is the command i am using to launch the coordinator

sudo /usr/sbin/arangod --server.authentication=false --server.endpoint tcp://$HOSTNAME:28530 --cluster.my-address tcp://$HOSTNAME:28530 --cluster.my-local-info osdi-03-coordinator --cluster.my-role COORDINATOR --cluster.agency-endpoint tcp://$HOSTNAME:25001 /data/data/arango/cordinator

And here's the error message i am getting. Please let me know how i can resolve this

2017-01-22T20:22:06Z [15360] INFO file-descriptors (nofiles) hard limit is 999999, soft limit is 999999 2017-01-22T20:22:06Z [15360] INFO created database directory '/data/data/arango/cordinator'. 2017-01-22T20:22:06Z [15360] INFO WAL directory '/data/data/arango/cordinator/journals' does not exist. creating it... 2017-01-22T20:22:06Z [15360] INFO JavaScript using startup '//usr/share/arangodb3/js', application '/var/lib/arangodb3-apps' 2017-01-22T20:22:09Z [15360] INFO Cluster feature is turned on. Agency version: {"server":"arango","version":"3.1.8","license":"community"}, Agency endpoints: http+tcp://osdi-03:25001, server id: 'Coordinator001', internal address: tcp://osdi-03:28530, role: COORDINATOR 2017-01-22T20:22:09Z [15360] INFO using heartbeat interval value '1000 ms' from agency 2017-01-22T20:22:09Z [15360] INFO In database '_system': No version information file found in database directory. 2017-01-22T20:22:09Z [15360] INFO In database '_system': Database is up-to-date (30108/cluster-local/init) 2017-01-22T20:22:09Z [15360] INFO using endpoint 'http+tcp://osdi-03:28530' for non-encrypted requests 2017-01-22T20:22:09Z [15360] INFO In database '_system': Found 14 defined task(s), 12 task(s) to run 2017-01-22T20:22:09Z [15360] INFO In database '_system': state coordinator-global/init, tasks setupGraphs, setupUsers, createUsersIndex, addDefaultUserSystem, createModules, createRouting, insertRedirectionsAll, setupAqlFunctions, createStatistics, createFrontend, setupQueues, setupJobs 2017-01-22T20:22:09Z [15360] WARNING {cluster} createCollectionCoordinator: replicationFactor is too large for the number of DBservers 2017-01-22T20:22:15Z [15360] WARNING {cluster} createCollectionCoordinator: replicationFactor is too large for the number of DBservers 2017-01-22T20:22:25Z [15360] ERROR In database '_system': Executing task #4 (addDefaultUserSystem: add default root user for system database) failed with exception: ArangoError 502: could not determine number of documents in collection (while optimizing plan) ArangoError: could not determine number of documents in collection (while optimizing plan) 2017-01-22T20:22:25Z [15360] ERROR at Error (native) 2017-01-22T20:22:25Z [15360] ERROR at ArangoStatement.execute (/usr/share/arangodb3/js/server/modules/@arangodb/arango-statement.js:81:16) 2017-01-22T20:22:25Z [15360] ERROR at ArangoDatabase._query (/usr/share/arangodb3/js/server/modules/@arangodb/arango-database.js:79:45) 2017-01-22T20:22:25Z [15360] ERROR at SimpleQueryByExample.execute (/usr/share/arangodb3/js/server/modules/@arangodb/simple-query.js:137:42) 2017-01-22T20:22:25Z [15360] ERROR at SimpleQueryByExample.SimpleQuery.toArray (/usr/share/arangodb3/js/common/modules/@arangodb/simple-query-common.js:340:8) 2017-01-22T20:22:25Z [15360] ERROR at ArangoCollection.firstExample (/usr/share/arangodb3/js/server/modules/@arangodb/arango-collection.js:287:71) 2017-01-22T20:22:25Z [15360] ERROR at Object.exports.save (/usr/share/arangodb3/js/server/modules/@arangodb/users.js:135:22) 2017-01-22T20:22:25Z [15360] ERROR at Object.addTask.task (/usr/share/arangodb3/js/server/upgrade-database.js:510:21) 2017-01-22T20:22:25Z [15360] ERROR at runTasks (/usr/share/arangodb3/js/server/upgrade-database.js:274:27) 2017-01-22T20:22:25Z [15360] ERROR at upgradeDatabase (/usr/share/arangodb3/js/server/upgrade-database.js:346:16) 2017-01-22T20:22:25Z [15360] ERROR In database '_system': Executing task #4 (addDefaultUserSystem: add default root user for system database) failed. Aborting init procedure. 2017-01-22T20:22:25Z [15360] ERROR In database '_system': Please fix the problem and try starting the server again. 2017-01-22T20:22:25Z [15360] ERROR upgrade-database.js for cluster script failed! 2017-01-22T20:22:25Z [15360] WARNING {cluster} createCollectionCoordinator: replicationFactor is too large for the number of DBservers 2017-01-22T20:22:35Z [15360] ERROR ArangoError: could not determine number of documents in collection (while optimizing plan) 2017-01-22T20:22:35Z [15360] ERROR at Error (native) 2017-01-22T20:22:35Z [15360] ERROR at ArangoStatement.execute (/usr/share/arangodb3/js/server/modules/@arangodb/arango-statement.js:81:16) 2017-01-22T20:22:35Z [15360] ERROR at ArangoDatabase._query (/usr/share/arangodb3/js/server/modules/@arangodb/arango-database.js:79:45) 2017-01-22T20:22:35Z [15360] ERROR at SimpleQueryAll.execute (/usr/share/arangodb3/js/server/modules/@arangodb/simple-query.js:96:42) 2017-01-22T20:22:35Z [15360] ERROR at SimpleQueryAll.SimpleQuery.hasNext (/usr/share/arangodb3/js/common/modules/@arangodb/simple-query-common.js:388:8) 2017-01-22T20:22:35Z [15360] ERROR at refillCaches (/usr/share/arangodb3/js/server/modules/@arangodb/foxx/manager.js:265:17) 2017-01-22T20:22:35Z [15360] ERROR at Object.initializeFoxx (/usr/share/arangodb3/js/server/modules/@arangodb/foxx/manager.js:1483:3) 2017-01-22T20:22:35Z [15360] ERROR at Object.foxxes (/usr/share/arangodb3/js/server/bootstrap/foxxes.js:64:47) 2017-01-22T20:22:35Z [15360] ERROR at server/bootstrap/cluster-bootstrap.js:51:54 2017-01-22T20:22:35Z [15360] ERROR at server/bootstrap/cluster-bootstrap.js:55:2 2017-01-22T20:22:35Z [15360] ERROR JavaScript exception in file '/usr/share/arangodb3/js/server/modules/@arangodb/foxx/queues/index.js' at 108,7: TypeError: Cannot read property 'save' of undefined 2017-01-22T20:22:35Z [15360] ERROR ! throw err; 2017-01-22T20:22:35Z [15360] ERROR ! ^ 2017-01-22T20:22:35Z [15360] FATAL error during execution of JavaScript file 'server/bootstrap/coordinator.js'

Hi there!

We have regular "arangod.exe stopped working" crashes on Windows. Happens with ArangoDB 2.5.2, 2.5.3, 2.6. We could not isolate what causes it, but it happens quite often on developer machines. Not necessarily when the DB is used heavily, often happens just randomly, sometimes more than once a day.

Attached you can find several instances of the crash happening, screenshots from windows Event Log and the details of the crashes, also a log file from ArangoDB itself.

The common log entries for the crashes are :

• In the ArangoDB log: ERROR Unhandled exception: -1073741819 - will crash now.
• In the windows event log: Unhandled exception: -1073741819 - will crash now. ....\arangod\RestServer\WindowsServiceUtils.cpp unhandledExceptionHandler 527

Full logs can be found here: https://drive.google.com/file/d/0B-Rhoo8GyarIUDFVb0hMUnZNVTA/view and here https://drive.google.com/file/d/0B-Rhoo8GyarId29Rc3lTVDQtZW8/view

my environment running ArangoDB

I'm using ArangoDB version:

• [x] 3.3.8 (subsequently upgraded to 3.3.10)

Mode:

• [x] Cluster

Storage-Engine:

• [x] rocksdb

On this operating system:

• [x] Linux
  • [x] other: self-built Docker container using the latest Ubuntu package

this is an installation-related issue:

Hello,

Since upgrading to 3.3 we've noticed a rather drastic increase in memory usage of server nodes over time. I've had a look at some of the existing open tickets and this looks quite similar to #5414 - I've decided to open a separate issue and let you decide though.

We're seeing this problem only in server instances - both the agency and coordinator nodes have very stable memory usage. Our standard deployment model for Arango is 3x agency, 3x coordinator and 3x server. We're using the following config options (I've skipped ones that seem irrelevant to me like directory locations, ip addresses etc - let me know if you'd like the full list):

--server.authentication=false
--cluster.my-role PRIMARY
--log.level info
--javascript.v8-contexts 16
--javascript.v8-max-heap 3072
--server.storage-engine rocksdb

Here's a list of sysctls we're setting:

- { name: "vm.max_map_count", value: "262144" }
- { name: "vm.overcommit_memory", value: "0" } # arangodb recommend setting this to 2 but this causes a lot of issues bringing other containers up
- { name: "vm.zone_reclaim_mode", value: "0" }
- { name: "vm.swappiness", value: "1" }
- { name: "net.core.somaxconn", value: "65535" }

net.core.somaxconn is also set (same value) on the Docker container.

We're setting the transparent_hugepage defrag and enabled properties to never.

We've upgraded from 3.2.9 to 3.3.0 and have since used 3.3.3, 3.3.4 and on 3.3.8 now. Here's a memory usage (RSS) for the server nodes in one of our environments (which got upgraded to 3.3.x around 25th April - note that this environment gets shutdown in the evening every day hence the large gaps in the graph):

This is the above graph zoomed in to the last 5 working day period:

This is another environment, the upgrade to 3.3.x happened on 3rd of April. The change in the memory usage pattern on the 27th April has been caused by applying a docker memory limit of 2g.

The above environments have extremely light usage of Arango.

Here's one of which gets used a bit more:

As a reference point of sorts, here's what it looks like when we run a load test against our application: The server nodes eventually tail off at 6.4GB and memory usage remains perfectly stable afterwards.

All of the above graphs were taken using the same settings I've mentioned earlier.

Let me know what other information would be useful to provide - I guess disabling statistics and or Foxx queues would be something you might want us to try? If so - shall we try disabling both off or try them one by one (if so - what order would you prefer)?

Thanks, Simon

edit: as part of the investigation I have upgraded from 3.3.8 to 3.3.10

my environment running ArangoDB

I'm using the latest ArangoDB of the respective release series:

• [ ] 2.8
• [ ] 3.0
• [ ] 3.1
• [x] 3.2
• [ ] self-compiled devel branch

Mode:

• [ ] Cluster
• [x] Single-Server

On this operating system:

• [ ] DCOS on
  • [ ] AWS
  • [ ] Azure
  • [ ] own infrastructure
• [ ] Linux
  • [ ] Debian .deb
  • [x] Ubuntu .deb

this is an installation-related issue:

I got the following error while trying to upgrade Arangodb 3.2.4 to 3.2.6 with apt on my ubuntu server:

Paketaktualisierung (Upgrade) wird berechnet... Fertig
Die folgenden Pakete wurden automatisch installiert und werden nicht mehr benötigt:
  linux-headers-4.4.0-96 linux-headers-4.4.0-96-generic linux-image-4.4.0-96-generic linux-image-extra-4.4.0-96-generic
Verwenden Sie »sudo apt autoremove«, um sie zu entfernen.
Die folgenden Pakete werden aktualisiert (Upgrade):
  arangodb3 distro-info-data grub-common grub-pc grub-pc-bin grub2-common initramfs-tools initramfs-tools-bin initramfs-tools-core iproute2 libgnutls-openssl27 libgnutls30 libpam-systemd libsystemd0 libudev1 lshw mdadm
  python3-distupgrade python3-update-manager resolvconf snap-confine snapd systemd systemd-sysv ubuntu-core-launcher ubuntu-release-upgrader-core udev update-manager-core
28 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen 56,5 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 8.393 kB Plattenplatz zusätzlich benutzt.
Holen:1 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 grub-pc amd64 2.02~beta2-36ubuntu3.14 [197 kB]
Holen:2 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 grub2-common amd64 2.02~beta2-36ubuntu3.14 [511 kB]
Holen:3 https://www.arangodb.com/repositories/arangodb32/xUbuntu_16.04  arangodb3 3.2.6 [33,6 MB]
Holen:4 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 grub-pc-bin amd64 2.02~beta2-36ubuntu3.14 [888 kB]
Holen:5 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 grub-common amd64 2.02~beta2-36ubuntu3.14 [1.705 kB]
Holen:6 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libsystemd0 amd64 229-4ubuntu21 [205 kB]
Holen:7 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpam-systemd amd64 229-4ubuntu21 [115 kB]
Holen:8 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 systemd amd64 229-4ubuntu21 [3.628 kB]
Holen:9 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 udev amd64 229-4ubuntu21 [992 kB]
Holen:10 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 mdadm amd64 3.3-2ubuntu7.5 [393 kB]
Holen:11 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libudev1 amd64 229-4ubuntu21 [54,6 kB]
Holen:12 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 initramfs-tools all 0.122ubuntu8.9 [8.616 B]
Holen:13 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 initramfs-tools-core all 0.122ubuntu8.9 [42,7 kB]
Holen:14 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 initramfs-tools-bin amd64 0.122ubuntu8.9 [9.632 B]
Holen:15 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 systemd-sysv amd64 229-4ubuntu21 [12,1 kB]
Holen:16 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 ubuntu-core-launcher amd64 2.28.5 [1.568 B]
Holen:17 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 snapd amd64 2.28.5 [12,6 MB]
Holen:18 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 snap-confine amd64 2.28.5 [1.726 B]
Holen:19 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 distro-info-data all 0.28ubuntu0.5 [4.224 B]
Holen:20 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 iproute2 amd64 4.3.0-1ubuntu3.16.04.2 [522 kB]
Holen:21 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgnutls-openssl27 amd64 3.4.10-4ubuntu1.4 [22,0 kB]
Holen:22 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgnutls30 amd64 3.4.10-4ubuntu1.4 [548 kB]
Holen:23 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 resolvconf all 1.78ubuntu5 [52,0 kB]
Holen:24 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 lshw amd64 02.17-1.1ubuntu3.3 [215 kB]
Holen:25 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 ubuntu-release-upgrader-core all 1:16.04.23 [29,4 kB]
Holen:26 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-distupgrade all 1:16.04.23 [104 kB]
Holen:27 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-update-manager all 1:16.04.10 [31,9 kB]
Holen:28 http://de.archive.ubuntu.com/ubuntu xenial-updates/main amd64 update-manager-core all 1:16.04.10 [5.320 B]
Es wurden 56,5 MB in 4 min 33 s geholt (207 kB/s).
Vorkonfiguration der Pakete ...
(Lese Datenbank ... 132347 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../grub-pc_2.02~beta2-36ubuntu3.14_amd64.deb ...
Entpacken von grub-pc (2.02~beta2-36ubuntu3.14) über (2.02~beta2-36ubuntu3.12) ...
Vorbereitung zum Entpacken von .../grub2-common_2.02~beta2-36ubuntu3.14_amd64.deb ...
Entpacken von grub2-common (2.02~beta2-36ubuntu3.14) über (2.02~beta2-36ubuntu3.12) ...
Vorbereitung zum Entpacken von .../grub-pc-bin_2.02~beta2-36ubuntu3.14_amd64.deb ...
Entpacken von grub-pc-bin (2.02~beta2-36ubuntu3.14) über (2.02~beta2-36ubuntu3.12) ...
Vorbereitung zum Entpacken von .../grub-common_2.02~beta2-36ubuntu3.14_amd64.deb ...
Entpacken von grub-common (2.02~beta2-36ubuntu3.14) über (2.02~beta2-36ubuntu3.12) ...
Vorbereitung zum Entpacken von .../libsystemd0_229-4ubuntu21_amd64.deb ...
Entpacken von libsystemd0:amd64 (229-4ubuntu21) über (229-4ubuntu19) ...
Trigger für man-db (2.7.5-1) werden verarbeitet ...
Trigger für install-info (6.1.0.dfsg.1-5) werden verarbeitet ...
Trigger für ureadahead (0.100.0-19) werden verarbeitet ...
Trigger für libc-bin (2.23-0ubuntu9) werden verarbeitet ...
libsystemd0:amd64 (229-4ubuntu21) wird eingerichtet ...
Trigger für libc-bin (2.23-0ubuntu9) werden verarbeitet ...
(Lese Datenbank ... 132347 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../libpam-systemd_229-4ubuntu21_amd64.deb ...
Entpacken von libpam-systemd:amd64 (229-4ubuntu21) über (229-4ubuntu19) ...
Vorbereitung zum Entpacken von .../systemd_229-4ubuntu21_amd64.deb ...
Entpacken von systemd (229-4ubuntu21) über (229-4ubuntu19) ...
Trigger für man-db (2.7.5-1) werden verarbeitet ...
Trigger für dbus (1.10.6-1ubuntu3.3) werden verarbeitet ...
Trigger für ureadahead (0.100.0-19) werden verarbeitet ...
systemd (229-4ubuntu21) wird eingerichtet ...
addgroup: Die Gruppe »systemd-journal« existiert bereits als Systemgruppe. Programmende.
(Lese Datenbank ... 132347 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../udev_229-4ubuntu21_amd64.deb ...
Entpacken von udev (229-4ubuntu21) über (229-4ubuntu19) ...
Vorbereitung zum Entpacken von .../mdadm_3.3-2ubuntu7.5_amd64.deb ...
Entpacken von mdadm (3.3-2ubuntu7.5) über (3.3-2ubuntu7.4) ...
Vorbereitung zum Entpacken von .../libudev1_229-4ubuntu21_amd64.deb ...
Entpacken von libudev1:amd64 (229-4ubuntu21) über (229-4ubuntu19) ...
Trigger für man-db (2.7.5-1) werden verarbeitet ...
Trigger für ureadahead (0.100.0-19) werden verarbeitet ...
Trigger für systemd (229-4ubuntu21) werden verarbeitet ...
Trigger für libc-bin (2.23-0ubuntu9) werden verarbeitet ...
libudev1:amd64 (229-4ubuntu21) wird eingerichtet ...
Trigger für libc-bin (2.23-0ubuntu9) werden verarbeitet ...
(Lese Datenbank ... 132349 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../initramfs-tools_0.122ubuntu8.9_all.deb ...
Entpacken von initramfs-tools (0.122ubuntu8.9) über (0.122ubuntu8.8) ...
Vorbereitung zum Entpacken von .../initramfs-tools-core_0.122ubuntu8.9_all.deb ...
Entpacken von initramfs-tools-core (0.122ubuntu8.9) über (0.122ubuntu8.8) ...
Vorbereitung zum Entpacken von .../initramfs-tools-bin_0.122ubuntu8.9_amd64.deb ...
Entpacken von initramfs-tools-bin (0.122ubuntu8.9) über (0.122ubuntu8.8) ...
Vorbereitung zum Entpacken von .../systemd-sysv_229-4ubuntu21_amd64.deb ...
Entpacken von systemd-sysv (229-4ubuntu21) über (229-4ubuntu19) ...
Trigger für man-db (2.7.5-1) werden verarbeitet ...
systemd-sysv (229-4ubuntu21) wird eingerichtet ...
(Lese Datenbank ... 132349 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../ubuntu-core-launcher_2.28.5_amd64.deb ...
Entpacken von ubuntu-core-launcher (2.28.5) über (2.27.5) ...
Vorbereitung zum Entpacken von .../snapd_2.28.5_amd64.deb ...
Warning: Stopping snapd.service, but it can still be activated by:
  snapd.socket
Entpacken von snapd (2.28.5) über (2.27.5) ...
Vorbereitung zum Entpacken von .../snap-confine_2.28.5_amd64.deb ...
Entpacken von snap-confine (2.28.5) über (2.27.5) ...
Vorbereitung zum Entpacken von .../distro-info-data_0.28ubuntu0.5_all.deb ...
Entpacken von distro-info-data (0.28ubuntu0.5) über (0.28ubuntu0.3) ...
Vorbereitung zum Entpacken von .../iproute2_4.3.0-1ubuntu3.16.04.2_amd64.deb ...
Entpacken von iproute2 (4.3.0-1ubuntu3.16.04.2) über (4.3.0-1ubuntu3.16.04.1) ...
Vorbereitung zum Entpacken von .../libgnutls-openssl27_3.4.10-4ubuntu1.4_amd64.deb ...
Entpacken von libgnutls-openssl27:amd64 (3.4.10-4ubuntu1.4) über (3.4.10-4ubuntu1.3) ...
Vorbereitung zum Entpacken von .../libgnutls30_3.4.10-4ubuntu1.4_amd64.deb ...
Entpacken von libgnutls30:amd64 (3.4.10-4ubuntu1.4) über (3.4.10-4ubuntu1.3) ...
Vorbereitung zum Entpacken von .../resolvconf_1.78ubuntu5_all.deb ...
Entpacken von resolvconf (1.78ubuntu5) über (1.78ubuntu4) ...
Vorbereitung zum Entpacken von .../lshw_02.17-1.1ubuntu3.3_amd64.deb ...
Entpacken von lshw (02.17-1.1ubuntu3.3) über (02.17-1.1ubuntu3.2) ...
Vorbereitung zum Entpacken von .../ubuntu-release-upgrader-core_1%3a16.04.23_all.deb ...
Entpacken von ubuntu-release-upgrader-core (1:16.04.23) über (1:16.04.22) ...
Vorbereitung zum Entpacken von .../python3-distupgrade_1%3a16.04.23_all.deb ...
Entpacken von python3-distupgrade (1:16.04.23) über (1:16.04.22) ...
Vorbereitung zum Entpacken von .../python3-update-manager_1%3a16.04.10_all.deb ...
Entpacken von python3-update-manager (1:16.04.10) über (1:16.04.9) ...
Vorbereitung zum Entpacken von .../update-manager-core_1%3a16.04.10_all.deb ...
Entpacken von update-manager-core (1:16.04.10) über (1:16.04.9) ...
Vorbereitung zum Entpacken von .../arangodb3_3.2.6_amd64.deb ...
Entpacken von arangodb3 (3.2.6) über (3.2.4) ...
Trigger für man-db (2.7.5-1) werden verarbeitet ...
Trigger für libc-bin (2.23-0ubuntu9) werden verarbeitet ...
Trigger für ureadahead (0.100.0-19) werden verarbeitet ...
Trigger für systemd (229-4ubuntu21) werden verarbeitet ...
grub-common (2.02~beta2-36ubuntu3.14) wird eingerichtet ...
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
grub2-common (2.02~beta2-36ubuntu3.14) wird eingerichtet ...
grub-pc-bin (2.02~beta2-36ubuntu3.14) wird eingerichtet ...
grub-pc (2.02~beta2-36ubuntu3.14) wird eingerichtet ...
Installing for i386-pc platform.
installation beendet. Keine Fehler aufgetreten.
Grub-Konfigurationsdatei wird generiert …
Linux-Abbild gefunden: /boot/vmlinuz-4.4.0-98-generic
initrd-Abbild gefunden: /boot/initrd.img-4.4.0-98-generic
Linux-Abbild gefunden: /boot/vmlinuz-4.4.0-97-generic
initrd-Abbild gefunden: /boot/initrd.img-4.4.0-97-generic
Linux-Abbild gefunden: /boot/vmlinuz-4.4.0-96-generic
initrd-Abbild gefunden: /boot/initrd.img-4.4.0-96-generic
erledigt
libpam-systemd:amd64 (229-4ubuntu21) wird eingerichtet ...
udev (229-4ubuntu21) wird eingerichtet ...
addgroup: Die Gruppe »input« existiert bereits als Systemgruppe. Programmende.
update-initramfs: deferring update (trigger activated)
mdadm (3.3-2ubuntu7.5) wird eingerichtet ...
Neue Version der Konfigurationsdatei /etc/init.d/mdadm wird installiert ...
update-initramfs: deferring update (trigger activated)
Grub-Konfigurationsdatei wird generiert …
Linux-Abbild gefunden: /boot/vmlinuz-4.4.0-98-generic
initrd-Abbild gefunden: /boot/initrd.img-4.4.0-98-generic
Linux-Abbild gefunden: /boot/vmlinuz-4.4.0-97-generic
initrd-Abbild gefunden: /boot/initrd.img-4.4.0-97-generic
Linux-Abbild gefunden: /boot/vmlinuz-4.4.0-96-generic
initrd-Abbild gefunden: /boot/initrd.img-4.4.0-96-generic
File descriptor 3 (pipe:[14045643]) leaked on lvs invocation. Parent PID 2676: /bin/sh
erledigt
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
initramfs-tools-bin (0.122ubuntu8.9) wird eingerichtet ...
initramfs-tools-core (0.122ubuntu8.9) wird eingerichtet ...
initramfs-tools (0.122ubuntu8.9) wird eingerichtet ...
update-initramfs: deferring update (trigger activated)
snapd (2.28.5) wird eingerichtet ...
Neue Version der Konfigurationsdatei /etc/apparmor.d/usr.lib.snapd.snap-confine.real wird installiert ...
Neue Version der Konfigurationsdatei /etc/profile.d/apps-bin-path.sh wird installiert ...
ubuntu-core-launcher (2.28.5) wird eingerichtet ...
snap-confine (2.28.5) wird eingerichtet ...
distro-info-data (0.28ubuntu0.5) wird eingerichtet ...
iproute2 (4.3.0-1ubuntu3.16.04.2) wird eingerichtet ...
libgnutls30:amd64 (3.4.10-4ubuntu1.4) wird eingerichtet ...
libgnutls-openssl27:amd64 (3.4.10-4ubuntu1.4) wird eingerichtet ...
resolvconf (1.78ubuntu5) wird eingerichtet ...
lshw (02.17-1.1ubuntu3.3) wird eingerichtet ...
arangodb3 (3.2.6) wird eingerichtet ...
FATAL ERROR: EXIT_FAILED - "exit with error"
dpkg: Fehler beim Bearbeiten des Paketes arangodb3 (--configure):
 Unterprozess installiertes post-installation-Skript gab den Fehlerwert 1 zurück
python3-distupgrade (1:16.04.23) wird eingerichtet ...
python3-update-manager (1:16.04.10) wird eingerichtet ...
ubuntu-release-upgrader-core (1:16.04.23) wird eingerichtet ...
update-manager-core (1:16.04.10) wird eingerichtet ...
Trigger für systemd (229-4ubuntu21) werden verarbeitet ...
Trigger für ureadahead (0.100.0-19) werden verarbeitet ...
Trigger für initramfs-tools (0.122ubuntu8.9) werden verarbeitet ...
update-initramfs: Generating /boot/initrd.img-4.4.0-98-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
Trigger für libc-bin (2.23-0ubuntu9) werden verarbeitet ...
Trigger für resolvconf (1.78ubuntu5) werden verarbeitet ...
Fehler traten auf beim Bearbeiten von:
 arangodb3
E: Sub-process /usr/bin/dpkg returned an error code (1)

I tried uninstalling -> reinstalling arangodb completly with no success:

[email protected]:~# apt-get install arangodb3=3.2.6
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Die folgenden NEUEN Pakete werden installiert:
  arangodb3
0 aktualisiert, 1 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen noch 0 B von 33,6 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 277 MB Plattenplatz zusätzlich benutzt.
Vorkonfiguration der Pakete ...
Vormals nicht ausgewähltes Paket arangodb3 wird gewählt.
(Lese Datenbank ... 94794 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../arangodb3_3.2.6_amd64.deb ...
Entpacken von arangodb3 (3.2.6) ...
Trigger für systemd (229-4ubuntu21) werden verarbeitet ...
Trigger für ureadahead (0.100.0-19) werden verarbeitet ...
Trigger für man-db (2.7.5-1) werden verarbeitet ...
arangodb3 (3.2.6) wird eingerichtet ...
FATAL ERROR: EXIT_FAILED - "exit with error"
dpkg: Fehler beim Bearbeiten des Paketes arangodb3 (--configure):
 Unterprozess installiertes post-installation-Skript gab den Fehlerwert 1 zurück
Fehler traten auf beim Bearbeiten von:
 arangodb3
E: Sub-process /usr/bin/dpkg returned an error code (1)

Also trying to re-install 3.2.4 failed with the same error. Before each install command I forced apt to clean its package cache and update sources:

apt-get clean
apt-get autoclean
apt-get update

Thanks for any help.

• Michl

As the title says, for being able to write and run ci-tests for the APIs, it would be nice to add ArangoDB to travis-ci like other databases have been added, too. --> http ://about.travis-ci.org/docs/user/database-setup/

Also, Is it possible to have different versions of AramgoDB to test against? Say 1.0, 1.1, 1.2, 2.0, etc...

My Environment

• ArangoDB Version: e.g. 3.3.19 (all 3 instances)
• Storage Engine: RocksDB
• Deployment Mode: Master/Slave
• Infrastructure: own
• Operating System: Debian 9.5 (master and slave 1) Ubuntu 16.04 (slave 2)
• Used Package: Debian or Ubuntu .deb

Component, Query & Data

Affected feature: Continuous replication.

Dataset: 14 collections, the 3 biggest having 50M, 10M, 3M docs. Those 3 big collections are continuously undergoing many writes, with peaks of 10 writes per second.

Notable operations: Every night some CRON job performs a huge lot of writes on the master, among 4 collections, over a range of 3 hours (between 12 AM and 3 AM roughly). The total number of writes each night is around 9M ! Yes, that's huge.

Those writes are documents modifications, sometimes (very rarely) documents removals, never documents creations.

Steps to reproduce

1. On each slave, setup a continuous replication from the same master using arangosh

2. Wait for initial sync to finish and continuous replication to start. Everything works fine

3. Wait for a nightly CRON job to set off

4. The next morning, count the documents, look for the date of most recently synced documents

5. Wait for a few hours, new documents getting created in realtime on the master get replicated again, but documents created in the meantime are missing on the slaves

Problem: Continuous replications works very well until the CRON job sets off, then stalls (too many writes I guess). Continuous replication does not crash, and eventually resumes syncing new docs again. I found no relevant log entries, either on master or slaves.

But, on slaves the log usually shows a huge lot of lines like INFO {replication} fetching master log from tick… and on both slaves those entries get less and less frequent since around 2 AM (almost the end of the CRON job). Then they get very numerous again at around 5 PM. The master tick mentioned in those entries is always different and increasing, as expected.

Note 1: a collection that constantly undergoes many writes, but is not affected by nightly CRON job, still gets stuck during the night. The collections affected by the CRON seem synced because the number of documents is the same on master and slaves, but it's hard to tell because only modifications happened on those collections (no creation / removal)

Note 2: on the slaves, the dates of latest synced docs in the different collections are close to each other, but not the same (usually in a range of 30 minutes)

Note 3: for a given collection, the dates of latest synced docs on the 2 different slaves is always exactly the same (the latest synced doc is the same)

Note 4: the date/time of the first docs that get synced again each day varies, roughly from 9 AM to 11 AM. This is always way sooner than the time we see the new replication log entries appear on the slaves (around 5 PM).

Expected result: The documentation says that the replication is single-threaded and that it might not keep up with huge number of writes. Although I understand this limitation, I would expect replication to be delayed but not docs to be lost. Or maybe an option could allow to dedicate multiple threads to replication (I guess it's not easy to implement).

Thank you, Mathias

I get this error when trying to deploy files to the key-value index.

2012-10-11T06:51:07Z [736] TRACE [[email protected]/VocBase/datafile.c:825] cannot write marker, not enough space

It is preceded by this strange output

2012-10-11T06:51:06Z [736] TRACE [[email protected]/HttpServer/HttpCommTask.cpp:98] HTTP READ FOR 0x100d41220:\nonger existing\n);\nPUT /_api/key/test/js/pages.js?create=1 HTTP/1.1\r\ncontent-type: application/json\r\ncontent-length: 8619\r\nx-voc-expires: 2012-10-20T00:00:00.000Z\r\nx-voc-extended: {"contentType":"application/javascript"}\r\nHost: 127.0.0.1:8529\r\nConnection: keep-alive\r\n\r\n
2012-10-11T06:51:06Z [736] TRACE [[email protected]/HttpServer/HttpCommTask.cpp:112] server port = 8529, client port = 44741
2012-10-11T06:51:06Z [736] WARNING got corrupted HTTP request 'onger '

any ideas what's going on?

The file that trigger the error contains this string at the end... no illegal characters (gremlins) was found.

js/element.details.js:  'open' in document.createElement('details')// Chrome 10 will fail this detection, but Chrome 10 is no longer existing

Will investigate the arango-client next... if that is the reason for the corrupted request.

Scope & Purpose

(Please describe the changes in this PR for reviewers, motivation, rationale - mandatory)

• [ ] :hankey: Bugfix
• [ ] :pizza: New feature
• [ ] :fire: Performance improvement
• [ ] :hammer: Refactoring/simplification

Checklist

• [ ] Tests
  • [ ] Regression tests
  • [ ] C++ Unit tests
  • [ ] integration tests
  • [ ] resilience tests
• [ ] :book: CHANGELOG entry made
• [ ] :books: documentation written (release notes, API changes, ...)
• [ ] Backports
  • [ ] Backport for 3.10: (Please link PR)
  • [ ] Backport for 3.9: (Please link PR)
  • [ ] Backport for 3.8: (Please link PR)

Related Information

(Please reference tickets / specification / other PRs etc)

• [ ] Docs PR:
• [ ] Enterprise PR:
• [ ] GitHub issue / Jira ticket:
• [ ] Design document:

Scope & Purpose

The performance improvements include:

• resolving replications waitFor futures multithreaded. Previously they were resolved sequentially. This adds more parallelism.
• Enable blod-db for the replicated log column family. This reduced the compaction debt by a lot and since replicated logs are written to be deleted instead of read, this is a fair trade off.
• We also added more metrics to precisely pin down where the performance is lost. Those new metrics point to writing into rocksdb. As expected, RocksDB is not the best (TM) storage engine for writing replicated logs.

Scope & Purpose

(Please describe the changes in this PR for reviewers, motivation, rationale - mandatory)

• [ ] :hankey: Bugfix
• [ ] :pizza: New feature
• [ ] :fire: Performance improvement
• [ ] :hammer: Refactoring/simplification

Checklist

• [ ] Tests
  • [ ] Regression tests
  • [ ] C++ Unit tests
  • [ ] integration tests
  • [ ] resilience tests
• [ ] :book: CHANGELOG entry made
• [ ] :books: documentation written (release notes, API changes, ...)
• [ ] Backports
  • [ ] Backport for 3.10: (Please link PR)
  • [ ] Backport for 3.9: (Please link PR)
  • [ ] Backport for 3.8: (Please link PR)

Related Information

(Please reference tickets / specification / other PRs etc)

• [ ] Docs PR:
• [ ] Enterprise PR:
• [ ] GitHub issue / Jira ticket:
• [ ] Design document:

Scope & Purpose

Backport of https://github.com/arangodb/arangodb/pull/17891

Added startup option to configure the number of bloom bits per key. The default value is 10, which is downwards-compatible to the previously hard-coded value.

• [ ] :hankey: Bugfix
• [ ] :pizza: New feature
• [ ] :fire: Performance improvement
• [ ] :hammer: Refactoring/simplification

Checklist

• [ ] Tests
  • [ ] Regression tests
  • [ ] C++ Unit tests
  • [ ] integration tests
  • [ ] resilience tests
• [ ] :book: CHANGELOG entry made
• [ ] :books: documentation written (release notes, API changes, ...)
• [ ] Backports
  • [x] Backport for 3.10: this PR
  • [ ] Backport for 3.9: -
  • [ ] Backport for 3.8: -

Related Information

• [ ] Docs PR:
• [ ] Enterprise PR:
• [ ] GitHub issue / Jira ticket:
• [ ] Design document:

Scope & Purpose

These guards checked whether OpenSSL was version < OpenSSL 1.1.1, which are not supported anymore and should hence always evaluate to false anyway.

Scope & Purpose

Some gratuitous cleanup that I asked for on #17864 ;

Requires enterprise buddy https://github.com/arangodb/enterprise/pull/1188 and is based on #17864