From 13bc69d79a3f4ee134d5c03a3f9086d82b0e8355 Mon Sep 17 00:00:00 2001 From: sleto-it <31849787+sleto-it@users.noreply.github.com> Date: Tue, 27 Mar 2018 17:32:51 +0100 Subject: [PATCH] Doc - WAL: MMFile / RocksDB distinction (#4939) --- .../Manual/Architecture/WriteAheadLog.md | 27 ++++++++++++++++--- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/Documentation/Books/Manual/Architecture/WriteAheadLog.md b/Documentation/Books/Manual/Architecture/WriteAheadLog.md index aeb0a61aa9..fb7436ec75 100644 --- a/Documentation/Books/Manual/Architecture/WriteAheadLog.md +++ b/Documentation/Books/Manual/Architecture/WriteAheadLog.md @@ -1,9 +1,7 @@ Write-ahead log =============== -The Write-ahead log is part of the MMFiles storage engine; This doesn't apply to your -ArangoDB if you are running with the [RocksDB](../Administration/Configuration/RocksDB.md) -storage engine. +Both storage engines use a form of write ahead logging (WAL). Starting with version 2.2 ArangoDB stores all data-modification operation in its write-ahead log. The write-ahead log is sequence of append-only files containing all the write operations that were executed on the server. @@ -12,9 +10,11 @@ It is used to run data recovery after a server crash, and can also be used in a replication setup when slaves need to replay the same sequence of operations as on the master. +MMFiles WAL Details +------------------- + By default, each write-ahead logfile is 32 MiB in size. This size is configurable via the option *--wal.logfile-size*. - When a write-ahead logfile is full, it is set to read-only, and following operations will be written into the next write-ahead logfile. By default, ArangoDB will reserve some spare logfiles in the background so switching logfiles should be fast. How many reserve @@ -38,3 +38,22 @@ them if required. How many collected logfiles will be kept before they get delet configurable via the option *--wal.historic-logfiles*. For all write-ahead log configuration options, please refer to the page [Write-ahead log options](../Administration/Configuration/Wal.md). + + +RocksDB WAL Details +------------------- + +The options mentioned above only apply for MMFiles. The WAL in the rocksdb storage engine +works slightly differently. +_Note:_ In rocksdb the WAL options are all prefixed with +`--rocksdb.*`. The `--wal.*` options do have no effect. + +The individual RocksDB WAL files are per default about 64 MiB big. The size will always be proportionally +sized to the value specified via `--rocksdb.write-buffer-size`. The value specifies the amount of +data to build up in memory (backed by the unsorted WAL on disk) before converting it to a sorted on-disk file. +Larger values can increase performance, especially during bulk loads. Up to `--rocksdb.max-write-buffer-number` +write buffers may be held in memory at the same time, so you may wish to adjust this parameter to control memory usage. A larger write buffer will result in a longer recovery time the next time the database is opened. + +The RocksDB WAL only contains committed transactions. This means you will never see partial transactions +in the replication log, but it also means transactions are tracked completely in-memory. In practice +this causes RocksDB transaction sizes to be limited, for more information see the [RocksDB Configuration](../Administration/Configuration/RocksDB.md)