We’ve been using Callgrind with its powerful frontend KCachegrind for quiet some time to analyse where the hot spots can be found inside of ArangoDB. One thing always accounting for a huge chunk of the resource usage was string comparison. Yes, string comparison isn’t as cheap as one may think, but its been even a bit more than one would expect. And since much of the business of a database is string comparison, its used a lot.

ArangoDB and V8 use the ICU Library for these purposes (with no alternatives on the market) – so basically we heavily rely on the performance of the ICU library...

While in search for further AQL query optimizations last week, we found that intermediate AQL query results were copied one time too often in some cases.

Precisely, the data that a query’s ReturnNode will return to the caller was copied into the ReturnNode’s own register. With ReturnNode’s never modifying their input data, this demanded for something that is called return-value optimization in compilers.

ArangoDB 2.6 will feature an alternative hash implementation of the AQL COLLECT operation. The new implementation can speed up some AQL queries that can not exploit indexes on the COLLECT group criteria.

This blog post provides a preview of the feature and shows some nice performance improvements. It also explains the COLLECT-related optimizer parts and how the optimizer will decide whether to use the new or the traditional implementation.

Introduction to COLLECT

A quick recap: in AQL, the COLLECT operation can be used for grouping and optionally counting values.

Inspired by the feature request in Github issue #1298, we added update and replace support for ArangoDB’s import facilities.

This extends ArangoDB’s HTTP REST API for importing documents plus the arangoimp binary so they can not only insert new documents but also update existing ones.

ArangoDB 2.6 provides a specialized export API for exporting all documents from a collection and shipping them to a client application. It is rather limited but faster than the general-purpose AQL cursor API and can store its snapshots using less memory.

This week we pushed some modifications for ArangoDB’s cursor API into the devel branch. The change will result in less copying of AQL query results between the AQL and the HTTP layers. As a positive side effect, this will reduce the amount of garbage collection the built-in V8 has to do.

These modifications should improve the cursor API performance significantly for many cases, while at the same time keeping its REST API stable. Client programs do not need to be adjusted to reap the benefits. In a blog post, Jan shows some first unscientific performance tests comparing the old cursor API with..

Great to know your database scales and database vendors like ArangoDB add some statistics on node / cluster health directly in their products.

But running a bunch of different servers and applications you need a central hub to collect monitoring data from all services running. In a series of recipes Willi shows how you could easily add monitoring capabilities to NoSQL databases like ArangoDB using collectd with the cURL JSON plugin.

Contained in 2.5 are some small but useful AQL language improvements plus several AQL optimizer improvements.

We are working on further AQL improvements for 2.5, but work is still ongoing. This post summarizes the improvements that are already completed and will be shipped with the initial ArangoDB 2.5 release.

This version is deprecated. Download the new version of ArangoDB

In ArangoDB 2.5, hash and skiplist indexes can optionally be made sparse.

Such sparse indexes will exclude documents in which at least one of the index attributes is either not set or has a value of null. Declaring an index as sparse can provide great savings in memory and index creation CPU time for some cases.

For example, the following document only has attributes _key and name:

{ "_key" : "test", "name" : "foobar" }

In a sparse index on (non-existing) attribute value, the above document will simply be ignored. And in a..

In the upcoming ArangoDB 2.5 (current devel branch) a new optimizer rule move-calculations-down was added. Jan showcases in his latest blog post how queries with calculations could benefit from this new optimiser rule.