!CHAPTER Upgrading to ArangoDB 2.2 Please read the following sections if you upgrade from a previous version to ArangoDB 2.2. Please note first that a database directory used with ArangoDB 2.2 cannot be used with earlier versions (e.g. ArangoDB 2.1) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade. !SECTION Database Directory Version Check and Upgrade ArangoDB will perform a database version check at startup. When ArangoDB 2.2 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional. The output will then look like this: ``` 2014-07-07T22:04:53Z [18675] ERROR In database '_system': Database directory version (2.1) is lower than server version (2.2). 2014-07-07T22:04:53Z [18675] ERROR In database '_system': ---------------------------------------------------------------------- 2014-07-07T22:04:53Z [18675] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary. 2014-07-07T22:04:53Z [18675] ERROR In database '_system': If this is what you wanted to do, please restart with the 2014-07-07T22:04:53Z [18675] ERROR In database '_system': --upgrade 2014-07-07T22:04:53Z [18675] ERROR In database '_system': option to upgrade the data in the database directory. 2014-07-07T22:04:53Z [18675] ERROR In database '_system': Normally you can use the control script to upgrade your database 2014-07-07T22:04:53Z [18675] ERROR In database '_system': /etc/init.d/arangodb stop 2014-07-07T22:04:53Z [18675] ERROR In database '_system': /etc/init.d/arangodb upgrade 2014-07-07T22:04:53Z [18675] ERROR In database '_system': /etc/init.d/arangodb start 2014-07-07T22:04:53Z [18675] ERROR In database '_system': ---------------------------------------------------------------------- 2014-07-07T22:04:53Z [18675] FATAL Database version check failed for '_system'. Please start the server with the --upgrade option ``` To make ArangoDB 2.2 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the `--upgrade` option: unix> arangod data --upgrade where `data` is ArangoDB's main data directory. Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the `--upgrade` command on each individual database subfolder (named `database-`). For example, if you regularly start your ArangoDB server with unix> arangod mydatabasefolder then running unix> arangod mydatabasefolder --upgrade will perform the upgrade for the whole ArangoDB instance, including all of its databases. Starting with `--upgrade` will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade. The output should look like this: ``` 2014-07-07T22:11:30Z [18867] INFO In database '_system': starting upgrade from version 2.1 to 2.2.0 2014-07-07T22:11:30Z [18867] INFO In database '_system': Found 19 defined task(s), 2 task(s) to run 2014-07-07T22:11:30Z [18867] INFO In database '_system': upgrade successfully finished 2014-07-07T22:11:30Z [18867] INFO database upgrade passed ``` Please check the output the `--upgrade` run. It may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved, you can start ArangoDB 2.2 regularly. !SECTION Upgrading a cluster planned in the web interface A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end. We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows: 1. First shut down your cluster using the graphical front end as usual. 2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them. 3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster". 4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems. There is an alternative way using the `ArangoDB` shell. Instead of steps 3. and 4. above you can launch `arangosh`, point it to the dispatcher that you have used to plan and launch the cluster using the option ``--server.endpoint``, and execute arangosh> require("org/arangodb/cluster").Upgrade("root",""); This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace `"root"` with a user name and `""` with a password that is valid for authentication with the cluster. !SECTION Changed behavior !SUBSECTION Replication The *_replication* system collection is not used anymore in ArangoDB 2.2 because all write operations will be logged in the write-ahead log. There is no need to additionally log operations in the *_replication* system collection. Usage of the *_replication* system collection in user scripts is discouraged. !SUBSECTION Replication logger The replication methods `logger.start`, `logger.stop` and `logger.properties` are no-ops in ArangoDB 2.2 as there is no separate replication logger anymore. Data changes are logged into the write-ahead log in ArangoDB 2.2, and need not be separately written to the *_replication* system collection by the replication logger. The replication logger object is still there in ArangoDB 2.2 to ensure API backwards-compatibility, however, starting, stopping or configuring the logger are no-ops in ArangoDB 2.2. This change also affects the following HTTP API methods: - `PUT /_api/replication/logger-start` - `PUT /_api/replication/logger-stop` - `GET /_api/replication/logger-config` - `PUT /_api/replication/logger-config` The start and stop commands will do nothing, and retrieving the logger configuration will return a dummy configuration. Setting the logger configuration does nothing and will return the dummy configuration again. Any user scripts that invoke the replication logger should be checked and adjusted before performing the upgrade to 2.2. !SUBSECTION Replication of transactions Replication of transactions has changed in ArangoDB 2.2. Previously, transactions were logged on the master in one big block and were shipped to a slave in one block, too. Now transaction operations will be logged and replicated as separate entries, allowing transactions to be bigger and also ensure replication progress. This also means the replication format is not fully compatible between ArangoDB 2.2 and previous versions. When upgrading a master-slave pair from ArangoDB 2.1 to 2.2, please stop operations on the master first and make sure everything has been replicated to the slave server. Then upgrade and restart both servers. !SUBSECTION Replication applier This change also affects the behavior of the *stop* method of the replication applier. If the replication applier is now stopped manually using the *stop* method and later restarted using the *start* method, any transactions that were unfinished at the point of stopping will be aborted on a slave, even if they later commit on the master. In ArangoDB 2.2, stopping the replication applier manually should be avoided unless the goal is to stop replication permanently or to do a full resync with the master anyway. If the replication applier still must be stopped, it should be made sure that the slave has fetched and applied all pending operations from a master, and that no extra transactions are started on the master before the `stop` command on the slave is executed. Replication of transactions in ArangoDB 2.2 might also lock the involved collections on the slave while a transaction is either committed or aborted on the master and the change has been replicated to the slave. This change in behavior may be important for slave servers that are used for read-scaling. In order to avoid long lasting collection locks on the slave, transactions should be kept small. Any user scripts that invoke the replication applier should be checked and adjusted before performing the upgrade to 2.2. !SUBSECTION Collection figures The figures reported by the *collection.figures* method only reflect documents and data contained in the journals and datafiles of collections. Documents or deletions contained only in the write-ahead log will not influence collection figures until the write-ahead log garbage collection kicks in and copies data over into the collections. The figures of a collection might therefore underreport the total resource usage of a collection. Additionally, the attributes *lastTick* and *uncollectedLogfileEntries* have been added to the figures. This also affects the HTTP API method *PUT /_api/collection/figures*. Any user scripts that process collection figures should be checked and adjusted before performing the upgrade to 2.2. !SUBSECTION Storage of non-JSON attribute values Previous versions of ArangoDB allowed storing JavaScript native objects of type `Date`, `Function`, `RegExp` or `External`, e.g. db.test.save({ foo: /bar/ }); db.test.save({ foo: new Date() }); Objects of these types were silently converted into an empty object (`{ }`) when being saved, an no warning was issued. This led to a silent data loss. ArangoDB 2.2 changes this, and disallows storing JavaScript native objects of the mentioned types. When this is attempted, the operation will now fail with the following error: Error: cannot be converted into JSON shape: could not shape document To store such data in a collection, explicitly convert them into strings like so: db.test.save({ foo: String(/bar/) }); db.test.save({ foo: String(new Date()) }); Please review your server-side data storage operation code (if any) before performing the upgrade to 2.2. !SUBSECTION AQL keywords The following keywords have been added to AQL in ArangoDB 2.2 to support data modification queries: - *INSERT* - *UPDATE* - *REPLACE* - *REMOVE* - *WITH* Unquoted usage of these keywords for attribute names in AQL queries will likely fail in ArangoDB 2.2. If any such attribute name needs to be used in a query, it should be enclosed in backticks to indicate the usage of a literal attribute name. For example, the following query will fail in ArangoDB 2.2 with a parse error: FOR i IN foo RETURN i.remove The attribute name *remove* needs to be quoted with backticks to indicate that the literal *remove* is meant: FOR i IN foo RETURN i.`remove` Before upgrading to 2.2, please check if any of your collections or queries use of the new keywords. !SECTION Removed features !SUBSECTION MRuby integration for arangod ArangoDB had an experimental MRuby integration in some of the publish builds. This wasn't continuously developed, and so it has been removed in ArangoDB 2.2. This change has led to the following startup options being superfluous: - `--ruby.gc-interval` - `--ruby.action-directory` - `--ruby.modules-path` - `--ruby.startup-directory` Specifying these startup options will do nothing in ArangoDB 2.2, so using these options should be avoided from now on as they might be removed in a future version of ArangoDB. !SUBSECTION Removed startup options The following startup options have been removed in ArangoDB 2.2. Specifying them in the server's configuration file will not produce an error to make migration easier. Still, usage of these options should be avoided as they will not have any effect and might fully be removed in a future version of ArangoDB: - `--database.remove-on-drop` - `--database.force-sync-properties` - `--random.no-seed` - `--ruby.gc-interval` - `--ruby.action-directory` - `--ruby.modules-path` - `--ruby.startup-directory` - `--server.disable-replication-logger` Before upgrading to 2.2, please check your configuration files and adjust them so no superfluous options are used.