!CHAPTER Upgrading to ArangoDB 2.3 Please read the following sections if you upgrade from a previous version to ArangoDB 2.3. Please note first that a database directory used with ArangoDB 2.3 cannot be used with earlier versions (e.g. ArangoDB 2.2) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade. !SECTION Database Directory Version Check and Upgrade ArangoDB will perform a database version check at startup. When ArangoDB 2.3 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional. The output will then look like this: ``` 2014-11-03T15:48:06Z [2694] ERROR In database '_system': Database directory version (2.2) is lower than current version (20300). 2014-11-03T15:48:06Z [2694] ERROR In database '_system': ---------------------------------------------------------------------- 2014-11-03T15:48:06Z [2694] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary. 2014-11-03T15:48:06Z [2694] ERROR In database '_system': If this is what you wanted to do, please restart with the 2014-11-03T15:48:06Z [2694] ERROR In database '_system': --upgrade 2014-11-03T15:48:06Z [2694] ERROR In database '_system': option to upgrade the data in the database directory. 2014-11-03T15:48:06Z [2694] ERROR In database '_system': Normally you can use the control script to upgrade your database 2014-11-03T15:48:06Z [2694] ERROR In database '_system': /etc/init.d/arangodb stop 2014-11-03T15:48:06Z [2694] ERROR In database '_system': /etc/init.d/arangodb upgrade 2014-11-03T15:48:06Z [2694] ERROR In database '_system': /etc/init.d/arangodb start 2014-11-03T15:48:06Z [2694] ERROR In database '_system': ---------------------------------------------------------------------- 2014-11-03T15:48:06Z [2694] FATAL Database '_system' needs upgrade. Please start the server with the --upgrade option ``` To make ArangoDB 2.3 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the `--upgrade` option: unix> arangod data --upgrade where `data` is ArangoDB's main data directory. Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the `--upgrade` command on each individual database subfolder (named `database-`). For example, if you regularly start your ArangoDB server with unix> arangod mydatabasefolder then running unix> arangod mydatabasefolder --upgrade will perform the upgrade for the whole ArangoDB instance, including all of its databases. Starting with `--upgrade` will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade. The output should look like this: ``` 2014-11-03T15:48:47Z [2708] INFO In database '_system': Found 24 defined task(s), 5 task(s) to run 2014-11-03T15:48:47Z [2708] INFO In database '_system': state prod/standalone/upgrade, tasks updateUserModel, createStatistics, upgradeClusterPlan, setupQueues, setupJobs 2014-11-03T15:48:48Z [2708] INFO In database '_system': upgrade successfully finished 2014-11-03T15:48:48Z [2708] INFO database upgrade passed ``` Please check the output the `--upgrade` run. It may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved, you can start ArangoDB 2.3 regularly. !SECTION Upgrading a cluster planned in the web interface A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end. We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows: 1. First shut down your cluster using the graphical front end as usual. 2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them. 3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster". 4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems. There is an alternative way using the `ArangoDB` shell. Instead of steps 3. and 4. above you can launch `arangosh`, point it to the dispatcher that you have used to plan and launch the cluster using the option ``--server.endpoint``, and execute arangosh> require("org/arangodb/cluster").Upgrade("root",""); This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace `"root"` with a user name and `""` with a password that is valid for authentication with the cluster. !SECTION AQL: Changed behavior !SUBSECTION AQL queries throw less exceptions ArangoDB 2.3 contains a completely rewritten AQL query optimizer and execution engine. This means that AQL queries will be executed with a different engine than in ArangoDB 2.2 and earlier. Parts of AQL queries might be executed in different order than before because the AQL optimizer has more freedom to move things around in a query. In previous versions of ArangoDB, AQL queries aborted with an exception in many situations and threw a runtime exception. Exceptions were thrown when trying to find a value using the `IN` operator in a non-list element, when trying to use non-boolean values with the logical operands `&&` or `||` or `!`, when using non-numeric values in arithmetic operations, when passing wrong parameters into functions etc. In ArangoDB 2.3 this has been changed in many cases to make AQL more user-friendly and to allow the optimization to perform much more query optimizations. Here is a summary of changes: - when a non-list value is used on the right-hand side of the `IN` operator, the result will be `false` in ArangoDB 2.3, and no exception will be thrown. - the boolean operators `&&` and `||` do not throw in ArangoDB 2.3 if any of the operands is not a boolean value. Instead, they will perform an implicit cast of the values to booleans. Their result will be as follows: - `lhs && rhs` will return `lhs` if it is `false` or would be `false` when converted into a boolean. If `lhs` is `true` or would be `true` when converted to a boolean, `rhs` will be returned. - `lhs || rhs` will return 'lhs` if it is `true` or would be `true` when converted into a boolean. If `lhs` is `false` or would be `false` when converted to a boolean, `rhs` will be returned. - `! value` will return the negated value of `value` converted into a boolean - the arithmetic operators (`+`, `-`, `*`, `/`, `%`) can be applied to any value and will not throw exceptions when applied to non-numeric values. Instead, any value used in these operators will be casted to a numeric value implicitly. If no numeric result can be produced by an arithmetic operator, it will return `null` in ArangoDB 2.3. This is also true for division by zero. - passing arguments of invalid types into AQL functions does not throw a runtime exception in most cases, but may produce runtime warnings. Built-in AQL functions that receive invalid arguments will then return `null`. !SUBSECTION Changed return values in ArangoQueryCursor.getExtra() The return value of `ArangoQueryCursor.getExtra()` has been changed in ArangoDB 2.3. It now contains a `stats` attribute with statistics about the query previously executed. It also contains a `warnings` attribute with warnings that happened during query execution. The new return value format looks like this: ``` arangosh> stmt = db._createStatement("FOR i IN mycollection RETURN i"); stmt.execute().getExtra() { "stats" : { "writesExecuted" : 0, "writesIgnored" : 0, "scannedFull" : 2600, "scannedIndex" : 0 }, "warnings" : [ ] } arangosh> stmt = db._createStatement("FOR i IN xx REMOVE i IN xx"); stmt.execute().getExtra() { "stats" : { "writesExecuted" : 2600, "writesIgnored" : 0, "scannedFull" : 2600, "scannedIndex" : 0 }, "warnings" : [ ] } ``` In ArangoDB 2.2, the return value of `ArangoQueryCursor.getExtra()` was empty for read-only queries and contained two queries for data-modification queries: ``` arangosh> stmt = db._createStatement("FOR i IN mycollection RETURN i"); stmt.execute().getExtra() { } arangosh> stmt = db._createStatement("FOR i IN mycollection REMOVE i IN mycollection"); stmt.execute().getExtra() { "operations" : { "executed" : 2600, "ignored" : 0 } } ``` !SUBSECTION Changed return values in ArangoStatement.explain() The return value of `ArangoStatement.explain()` has been changed in ArangoDB 2.3. In ArangoDB 2.3, the full execution plan for an AQL query is returned alongside all applied optimizer rules, optimization warnings etc. It is also possible to have the optimizer return all execution plans. This required a new data structure. Client programs that use `ArangoStatement.explain()` or the HTTP REST API method `POST /_api/explain` may need to be adjusted to use the new return format. The return value of `ArangoStatement.parse()` has been extended in ArangoDB 2.3. In addition to the existing attributes, ArangoDB 2.3 will also return an `ast` attribute containing the abstract syntax tree of the statement. This extra attribute can safely be ignored by client programs. !SUBSECTION New AQL keywords The following keywords have been added to AQL in ArangoDB 2.3: - *NOT* - *AND* - *OR* Unquoted usage of these keywords for attribute names in AQL queries will likely fail in ArangoDB 2.3. If any such attribute name needs to be used in a query, it should be enclosed in backticks to indicate the usage of a literal attribute name. !SECTION Removed features !SUBSECTION Bitarray indexes Bitarray indexes were only half-way documented and integrated in previous versions of ArangoDB so their benefit was limited. The support for bitarray indexes has thus been removed in ArangoDB 2.3. It is not possible to create indexes of type "bitarray" with ArangoDB 2.3. When a collection is openend that contains a bitarray index definition created with a previous version of ArangoDB, ArangoDB will ignore it and log the following warning: index type 'bitarray' is not supported in this version of ArangoDB and is ignored Future versions of ArangoDB may automatically remove such index definitions so the warnings will eventually disappear. !SUBSECTION Other removed features The HTTP API method at `POST /_admin/modules/flush` has been removed.