diff --git a/Documentation/Books/AQL/Extending/README.md b/Documentation/Books/AQL/Extending/README.md index 78b86ad48e..6d825d9209 100644 --- a/Documentation/Books/AQL/Extending/README.md +++ b/Documentation/Books/AQL/Extending/README.md @@ -45,11 +45,10 @@ in terms of used V8 contexts and server threads. If you run out of these resources, your query may abort with a [**cluster backend unavailable**](../../Manual/Appendix/ErrorCodes.html) error. -To overcome these mentioned limitations, you may want to -[increase the number of available V8 contexts](../../Manual/Administration/Configuration/GeneralArangod.html#v8-contexts) -(at the expense of increased memory usage), -and -[the number of available server threads](../../Manual/Administration/Configuration/GeneralArangod.html#server-threads). +To overcome these mentioned limitations, you may want to increase the +[number of available V8 contexts](../../Manual/Programs/Arangod/Javascript.html#v8-contexts) +(at the expense of increased memory usage), and the +[number of available server threads](../../Manual/Programs/Arangod/Server.html#server-threads). ### Deployment Details diff --git a/Documentation/Books/AQL/Fundamentals/TypeValueOrder.md b/Documentation/Books/AQL/Fundamentals/TypeValueOrder.md index 8f38bb298e..5af7a7791e 100644 --- a/Documentation/Books/AQL/Fundamentals/TypeValueOrder.md +++ b/Documentation/Books/AQL/Fundamentals/TypeValueOrder.md @@ -71,7 +71,7 @@ result is defined as follows: - boolean: *false* is less than *true* - number: numeric values are ordered by their cardinal value - string: string values are ordered using a localized comparison, using the configured - [server language](../../Manual/Administration/Configuration/GeneralArangod.html#default-language) + [server language](../../Manual/Programs/Arangod/Global.html#default-language) for sorting according to the alphabetical order rules of that language Note: unlike in SQL, *null* can be compared to any value, including *null* diff --git a/Documentation/Books/Cookbook/Administration/XCopyInstallWindows.md b/Documentation/Books/Cookbook/Administration/XCopyInstallWindows.md index e93e81b218..829ecf4a34 100644 --- a/Documentation/Books/Cookbook/Administration/XCopyInstallWindows.md +++ b/Documentation/Books/Cookbook/Administration/XCopyInstallWindows.md @@ -25,9 +25,9 @@ Open an explorer, choose a place where you want ArangoDB to be and unzip the fil Edit `etc\arangodb3\arangod.conf` if the default values don't suit your needs like: -- [the location of the database files](../..//Manual/Administration/Configuration/Wal.html) -- [ports to bind](../../Manual/Administration/Configuration/Endpoint.html) -- [storage engine](../../Manual/Administration/Configuration/GeneralArangod.html#storage-engine) +- [the location of the database files](../../Manual/Programs/Arangod/Wal.html) +- [ports to bind](../../Manual/Programs/Arangod/Server.html#managing-endpoints) +- [storage engine](../../Manual/Programs/Arangod/Server.html#storage-engine) and so on. diff --git a/Documentation/Books/HTTP/General/README.md b/Documentation/Books/HTTP/General/README.md index 9194c534a0..03619040a5 100644 --- a/Documentation/Books/HTTP/General/README.md +++ b/Documentation/Books/HTTP/General/README.md @@ -113,7 +113,8 @@ Client authentication can be achieved by using the *Authorization* HTTP header i client requests. ArangoDB supports authentication via HTTP Basic or JWT. Authentication is turned on by default for all internal database APIs but turned off for custom Foxx apps. -To toggle authentication for incoming requests to the internal database APIs, use the option [--server.authentication](../../Manual/Administration/Configuration/GeneralArangod.html). +To toggle authentication for incoming requests to the internal database APIs, use the option +[--server.authentication](../../Manual/Programs/Arangod/Server.html#enabledisable-authentication). This option is turned on by default so authentication is required for the database APIs. Please note that requests using the HTTP OPTIONS method will be answered by @@ -123,7 +124,7 @@ requests (see [Cross Origin Resource Sharing requests](#cross-origin-resource-sh The response to an HTTP OPTIONS request will be generic and not expose any private data. There is an additional option to control authentication for custom Foxx apps. The option -[--server.authentication-system-only](../../Manual/Administration/Configuration/GeneralArangod.html) +[--server.authentication-system-only](../../Manual/Programs/Arangod/Server.html#enabledisable-authentication-for-system-api-requests-only) controls whether authentication is required only for requests to the internal database APIs and the admin interface. It is turned on by default, meaning that other APIs (this includes custom Foxx apps) do not require authentication. diff --git a/Documentation/Books/Manual/Administration/Configuration/README.md b/Documentation/Books/Manual/Administration/Configuration.md similarity index 86% rename from Documentation/Books/Manual/Administration/Configuration/README.md rename to Documentation/Books/Manual/Administration/Configuration.md index 9237db3835..7cc7b71e64 100644 --- a/Documentation/Books/Manual/Administration/Configuration/README.md +++ b/Documentation/Books/Manual/Administration/Configuration.md @@ -1,55 +1,11 @@ -Command-line options -==================== +# Configuration -General Options ---------------- +The [programs and tools](../Programs/README.md) shipped in an ArangoDB package, +except for the *ArangoDB Starter*, can be configured with a `.conf` file and/or +started with options on a command line. The options are called the same, but +the syntax is slightly different as explained below. -### General help - -`--help` - -`-h` - -Prints a list of the most common options available and then exits. In order to -see all options use *--help-all*. - -To receive the startup options in JSON format, pass the `--dump-options` flag. This will -print out all options and exit. - -### Version - -`--version` - -`-v` - -Prints the version of the server and exits. - - -### Configuration Files - -Options can be specified on the command line or in configuration files. If a -string *Variable* occurs in the value, it is replaced by the corresponding -environment variable. - -`--configuration filename` - -`-c filename` - -Specifies the name of the configuration file to use. - -If this command is not passed to the server, then by default, the server will -attempt to first locate a file named *~/.arango/arangod.conf* in the user's home -directory. - -If no such file is found, the server will proceed to look for a file -*arangod.conf* in the system configuration directory. The system configuration -directory is platform-specific, and may be changed when compiling ArangoDB -yourself. It may default to */etc/arangodb* or */usr/local/etc/arangodb*. This -file is installed when using a package manager like rpm or dpkg. If you modify -this file and later upgrade to a new version of ArangoDB, then the package -manager normally warns you about the conflict. In order to avoid these warning -for small adjustments, you can put local overrides into a file -*arangod.conf.local*. +## Startup Options Only command line options with a value should be set within the configuration file. Command line options which act as flags should be entered on the command @@ -123,3 +79,30 @@ or ``` when starting up the server. Note that, the word *none* is case-insensitive. + + +## Configuration Files + +Options can be specified on the command line or in configuration files. If a +string *Variable* occurs in the value, it is replaced by the corresponding +environment variable. + +`--configuration filename` + +`-c filename` + +Specifies the name of the configuration file to use. + +If this command is not passed to the server, then by default, the server will +attempt to first locate a file named *~/.arango/arangod.conf* in the user's home +directory. + +If no such file is found, the server will proceed to look for a file +*arangod.conf* in the system configuration directory. The system configuration +directory is platform-specific, and may be changed when compiling ArangoDB +yourself. It may default to */etc/arangodb* or */usr/local/etc/arangodb*. This +file is installed when using a package manager like rpm or dpkg. If you modify +this file and later upgrade to a new version of ArangoDB, then the package +manager normally warns you about the conflict. In order to avoid these warning +for small adjustments, you can put local overrides into a file +*arangod.conf.local*. diff --git a/Documentation/Books/Manual/Administration/Configuration/Asynchronous.md b/Documentation/Books/Manual/Administration/Configuration/Asynchronous.md deleted file mode 100644 index e84cfb1388..0000000000 --- a/Documentation/Books/Manual/Administration/Configuration/Asynchronous.md +++ /dev/null @@ -1,14 +0,0 @@ -Asynchronous Tasks -================== - -Maximal queue size ------------------- - -Maximum size of the queue for requests: `--server.maximal-queue-size -size` - -Specifies the maximum *size* of the queue for asynchronous task -execution. If the queue already contains *size* tasks, new tasks will -be rejected until other tasks are popped from the queue. Setting this -value may help preventing from running out of memory if the queue is -filled up faster than the server can process requests. diff --git a/Documentation/Books/Manual/Administration/Configuration/Endpoint.md b/Documentation/Books/Manual/Administration/Configuration/Endpoint.md deleted file mode 100644 index 155585e9b5..0000000000 --- a/Documentation/Books/Manual/Administration/Configuration/Endpoint.md +++ /dev/null @@ -1,136 +0,0 @@ -Managing Endpoints -================== - -The ArangoDB server can listen for incoming requests on multiple *endpoints*. - -The endpoints are normally specified either in ArangoDB's configuration file or -on the command-line, using the `--server.endpoint`. ArangoDB supports different -types of endpoints: - -- tcp://ipv4-address:port - TCP/IP endpoint, using IPv4 -- tcp://[ipv6-address]:port - TCP/IP endpoint, using IPv6 -- ssl://ipv4-address:port - TCP/IP endpoint, using IPv4, SSL encryption -- ssl://[ipv6-address]:port - TCP/IP endpoint, using IPv6, SSL encryption -- unix:///path/to/socket - Unix domain socket endpoint - -If a TCP/IP endpoint is specified without a port number, then the default port -(8529) will be used. If multiple endpoints need to be used, the option can be -repeated multiple times. - -The default endpoint for ArangoDB is *tcp://127.0.0.1:8529* or -*tcp://localhost:8529*. - -**EXAMPLES** - -``` -unix> ./arangod --server.endpoint tcp://127.0.0.1:8529 - --server.endpoint ssl://127.0.0.1:8530 - --ssl.keyfile server.pem /tmp/vocbase -2012-07-26T07:07:47Z [8161] INFO using SSL protocol version 'TLSv1' -2012-07-26T07:07:48Z [8161] INFO using endpoint 'ssl://127.0.0.1:8530' for http ssl requests -2012-07-26T07:07:48Z [8161] INFO using endpoint 'tcp://127.0.0.1:8529' for http tcp requests -2012-07-26T07:07:49Z [8161] INFO ArangoDB (version 1.1.alpha) is ready for business -2012-07-26T07:07:49Z [8161] INFO Have Fun! -``` - -TCP Endpoints -------------- - -Given a hostname: - -`--server.endpoint tcp://hostname:port` - -Given an IPv4 address: - -`--server.endpoint tcp://ipv4-address:port` - -Given an IPv6 address: - -`--server.endpoint tcp://[ipv6-address]:port` - -On one specific ethernet interface each port can only be bound **exactly -once**. You can look up your available interfaces using the *ifconfig* command -on Linux / MacOSX - the Windows equivalent is *ipconfig* ([See Wikipedia for -more details](http://en.wikipedia.org/wiki/Ifconfig)). The general names of the -interfaces differ on OS's and hardwares they run on. However, typically every -host has a so called -[loopback interface](http://en.wikipedia.org/wiki/Loop_device), -which is a virtual interface. By convention it always has the address -*127.0.0.1* or *::1* (ipv6), and can only be reached from exactly the very same -host. Ethernet interfaces usually have names like *eth0*, *wlan0*, *eth1:17*, -*le0* or a plain text name in Windows. - -To find out which services already use ports (so ArangoDB can't bind them -anymore), you can use the -[netstat command](http://en.wikipedia.org/wiki/Netstat) -(it behaves a little different on each platform, run it with *-lnpt* on Linux, -*-p tcp* on MacOSX or with *-an* on windows for valuable information). - -ArangoDB can also do a so called *broadcast bind* using -*tcp://0.0.0.0:8529*. This way it will be reachable on all interfaces of the -host. This may be useful on development systems that frequently change their -network setup like laptops. - -### Special note on IPv6 link-local addresses - -ArangoDB can also listen to IPv6 link-local addresses via adding the zone ID -to the IPv6 address in the form `[ipv6-link-local-address%zone-id]`. However, -what you probably instead want is to bind to a local IPv6 address. Local IPv6 -addresses start with `fd`. If you only see a `fe80:` IPv6 address in your -interface configuration but no IPv6 address starting with `fd` your interface -has no local IPv6 address assigned. You can read more about IPv6 link-local -addresses [here](https://en.wikipedia.org/wiki/Link-local_address#IPv6). - -** Example ** - -Bind to a link-local and local IPv6 address. - - unix> ifconfig - -This command lists all interfaces and assigned ip addresses. The link-local -address may be `fe80::6257:18ff:fe82:3ec6%eth0` (IPv6 address plus interface name). -A local IPv6 address may be `fd12:3456::789a`. To bind ArangoDB to it start -*arangod* with `--server.endpoint tcp://[fe80::6257:18ff:fe82:3ec6%eth0]:8529`. -Use telnet to test the connection. - - unix> telnet fe80::6257:18ff:fe82:3ec6%eth0 8529 - Trying fe80::6257:18ff:fe82:3ec6... - Connected to my-machine. - Escape character is '^]'. - GET / HTTP/1.1 - - HTTP/1.1 301 Moved Permanently - Location: /_db/_system/_admin/aardvark/index.html - Content-Type: text/html - Server: ArangoDB - Connection: Keep-Alive - Content-Length: 197 - - Moved

Moved

This page has moved to /_db/_system/_admin/aardvark/index.html.

- - -### Reuse address - -`--tcp.reuse-address` - -If this boolean option is set to *true* then the socket option SO_REUSEADDR is -set on all server endpoints, which is the default. If this option is set to -*false* it is possible that it takes up to a minute after a server has -terminated until it is possible for a new server to use the same endpoint -again. This is why this is activated by default. - -Please note however that under some operating systems this can be a security -risk because it might be possible for another process to bind to the same -address and port, possibly hijacking network traffic. Under Windows, ArangoDB -additionally sets the flag SO_EXCLUSIVEADDRUSE as a measure to alleviate this -problem. - -### Backlog size - -`--tcp.backlog-size` - -Allows to specify the size of the backlog for the *listen* system call The -default value is 10. The maximum value is platform-dependent. Specifying a -higher value than defined in the system header's SOMAXCONN may result in a -warning on server start. The actual value used by *listen* may also be silently -truncated on some platforms (this happens inside the *listen* system call). diff --git a/Documentation/Books/Manual/Administration/Configuration/GeneralArangod.md b/Documentation/Books/Manual/Administration/Configuration/GeneralArangod.md deleted file mode 100644 index 0169623590..0000000000 --- a/Documentation/Books/Manual/Administration/Configuration/GeneralArangod.md +++ /dev/null @@ -1,657 +0,0 @@ -General Options -=============== - -### Database Upgrade - -`--database.auto-upgrade` - -Specifying this option will make the server perform a database upgrade instead -of starting the server normally. A database upgrade will first compare the -version number stored in the file VERSION in the database directory with the -current server version. - -If the version number found in the database directory is higher than the version -number the server is running, the server expects this is an unintentional -downgrade and will warn about this. Using the server in these conditions is -neither recommended nor supported. - -If the version number found in the database directory is lower than the version -number the server is running, the server will check whether there are any -upgrade tasks to perform. It will then execute all required upgrade tasks and -print their statuses. If one of the upgrade tasks fails, the server will exit -with an error. Re-starting the server with the upgrade option will then again -trigger the upgrade check and execution until the problem is fixed. - -Whether or not this option is specified, the server will always perform a -version check on startup. Running the server with a non-matching version number -in the VERSION file will make the server refuse to start. - -### Storage Engine -ArangoDB's "traditional" storage engine is called `MMFiles`, which also was the -default storage engine up to including ArangoDB 3.3. - -Since ArangoDB 3.2, an alternative engine based on [RocksDB](http://rocksdb.org) -is also provided and could be turned on manually. Since ArangoDB 3.4, the RocksDB -storage engine is the default storage engine for new installations. - -One storage engine type is supported per server per installation. -Live switching of storage engines on already installed systems isn't supported. -Configuring the wrong engine (not matching the previously used one) will result -in the server refusing to start. You may however use `auto` to let ArangoDB choose -the previously used one. - -`--server.storage-engine [auto|mmfiles|rocksdb]` - -Note that `auto` will default to `rocksdb` starting with ArangoDB 3.4, but in -previous versions it defaulted to `mmfiles`. - - -### Daemon - -`--daemon` - -Runs the server as a daemon (as a background process). This parameter can only -be set if the pid (process id) file is specified. That is, unless a value to the -parameter pid-file is given, then the server will report an error and exit. - - -### Default Language - -`--default-language default-language` - -The default language ist used for sorting and comparing strings. The language -value is a two-letter language code (ISO-639) or it is composed by a two-letter -language code with and a two letter country code (ISO-3166). Valid languages are -"de", "en", "en_US" or "en_UK". - -The default default-language is set to be the system locale on that platform. - - -### Supervisor - -`--supervisor` - -Executes the server in supervisor mode. In the event that the server -unexpectedly terminates due to an internal error, the supervisor will -automatically restart the server. Setting this flag automatically implies that -the server will run as a daemon. Note that, as with the daemon flag, this flag -requires that the pid-file parameter will set. - -```js -unix> ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ -2012-06-27T15:58:28Z [10133] INFO starting up in supervisor mode -``` - -As can be seen (e.g. by executing the ps command), this will start a supervisor -process and the actual database process: - -```js -unix> ps fax | grep arangod -10137 ? Ssl 0:00 ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ -10142 ? Sl 0:00 \_ ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ -``` - -When the database process terminates unexpectedly, the supervisor process will -start up a new database process: - -``` -> kill -SIGSEGV 10142 - -> ps fax | grep arangod -10137 ? Ssl 0:00 ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ -10168 ? Sl 0:00 \_ ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ -``` - - -### User identity - -`--uid uid` - -The name (identity) of the user the server will run as. If this parameter is not -specified, the server will not attempt to change its UID, so that the UID used -by the server will be the same as the UID of the user who started the server. If -this parameter is specified, then the server will change its UID after opening -ports and reading configuration files, but before accepting connections or -opening other files (such as recovery files). This is useful when the server -must be started with raised privileges (in certain environments) but security -considerations require that these privileges be dropped once the server has -started work. - -Observe that this parameter cannot be used to bypass operating system -security. In general, this parameter (and its corresponding relative gid) can -lower privileges but not raise them. - - -### Group identity - -`--gid gid` - -The name (identity) of the group the server will run as. If this parameter is -not specified, then the server will not attempt to change its GID, so that the -GID the server runs as will be the primary group of the user who started the -server. If this parameter is specified, then the server will change its GID -after opening ports and reading configuration files, but before accepting -connections or opening other files (such as recovery files). - -This parameter is related to the parameter uid. - - -### Process identity - -`--pid-file filename` - -The name of the process ID file to use when running the server as a -daemon. This parameter must be specified if either the flag *daemon* or -*supervisor* is set. - -### Check max memory mappings - -`--server.check-max-memory-mappings` can be used on Linux to make arangod -check the number of memory mappings currently used by the process (as reported in -*/proc//maps*) and compare it with the maximum number of allowed mappings as -determined by */proc/sys/vm/max_map_count*. If the current number of memory -mappings gets near the maximum allowed value, arangod will log a warning -and disallow the creation of further V8 contexts temporarily until the current -number of mappings goes down again. - -If the option is set to false, no such checks will be performed. All non-Linux -operating systems do not provide this option and will ignore it. - - -### Console - -`--console` - -Runs the server in an exclusive emergency console mode. When -starting the server with this option, the server is started with -an interactive JavaScript emergency console, with all networking -and HTTP interfaces of the server disabled. - -No requests can be made to the server in this mode, and the only -way to work with the server in this mode is by using the emergency -console. -Note that the server cannot be started in this mode if it is -already running in this or another mode. - - -### Random Generator - -`--random.generator arg` - -The argument is an integer (1,2,3 or 4) which sets the manner in which -random numbers are generated. The default method (3) is to use the a -non-blocking random (or pseudorandom) number generator supplied by the -operating system. - -Specifying an argument of 2, uses a blocking random (or -pseudorandom) number generator. Specifying an argument 1 sets a -pseudorandom -number generator using an implication of the Mersenne Twister MT19937 -algorithm. Algorithm 4 is a combination of the blocking random number -generator and the Mersenne Twister. - - -### Enable/disable authentication - -@startDocuBlock server_authentication - -### JWT Secret - -`--server.jwt-secret secret` - -ArangoDB will use JWTs to authenticate requests. Using this option lets -you specify a JWT. - -In single server setups and when not specifying this secret ArangoDB will -generate a secret. - -In cluster deployments which have authentication enabled a secret must -be set consistently across all cluster tasks so they can talk to each other. - -### Enable/disable authentication for UNIX domain sockets - -`--server.authentication-unix-sockets value` - -Setting *value* to true will turn off authentication on the server side -for requests coming in via UNIX domain sockets. With this flag enabled, -clients located on the same host as the ArangoDB server can use UNIX domain -sockets to connect to the server without authentication. -Requests coming in by other means (e.g. TCP/IP) are not affected by this option. - -The default value is *false*. - -**Note**: this option is only available on platforms that support UNIX -domain sockets. - -### Enable/disable authentication for system API requests only - -@startDocuBlock serverAuthenticateSystemOnly - - -### Enable authentication cache timeout - -`--server.authentication-timeout value` - -Sets the cache timeout to *value* (in seconds). This is only necessary -if you use an external authentication system like LDAP. - -### Enable local authentication - -`--server.local-authentication value` - -If set to *false* only use the external authentication system. If -*true* also use the local *_users* collections. - -The default value is *true*. - -### Enable/disable replication applier - -`--database.replication-applier flag` - -If *false* the server will start with replication appliers turned off, -even if the replication appliers are configured with the *autoStart* option. -Using the command-line option will not change the value of the *autoStart* -option in the applier configuration, but will suppress auto-starting the -replication applier just once. - -If the option is not used, ArangoDB will read the applier configuration -from the file *REPLICATION-APPLIER-CONFIG* on startup, and use the value of the -*autoStart* attribute from this file. - -The default is *true*. - -### Keep-alive timeout - -`--http.keep-alive-timeout` - -Allows to specify the timeout for HTTP keep-alive connections. The timeout -value must be specified in seconds. -Idle keep-alive connections will be closed by the server automatically -when the timeout is reached. A keep-alive-timeout value 0 will disable the keep -alive feature entirely. - -### Hide Product header - -`--http.hide-product-header` - -If *true*, the server will exclude the HTTP header "Server: ArangoDB" in -HTTP responses. If set to *false*, the server will send the header in -responses. - -The default is *false*. - - -### Allow method override - -`--http.allow-method-override` - -When this option is set to *true*, the HTTP request method will optionally -be fetched from one of the following HTTP request headers if present in -the request: - -- *x-http-method* -- *x-http-method-override* -- *x-method-override* - -If the option is set to *true* and any of these headers is set, the -request method will be overridden by the value of the header. For example, -this allows issuing an HTTP DELETE request which to the outside world will -look like an HTTP GET request. This allows bypassing proxies and tools that -will only let certain request types pass. - -Setting this option to *true* may impose a security risk so it should only -be used in controlled environments. - -The default value for this option is *false*. - -### Server threads - -`--server.minimal-threads number` - -`--server.maximal-threads number` - -Specifies the *number* of threads that are spawned to handle requests. - -The actual number of request processing threads is adjusted dynamically at runtime -and will float between `--server.minimal-threads` and `--server.maximal-threads`. - -`--server.minimal-threads` determines the minimum number of request processing -threads the server will start and that will always be kept around. The default -value is *2*. - -`--server.maximal-threads` determines the maximum number of request processing -threads the server is allowed to start for request handling. If that number of -threads is already running, arangod will not start further threads for request -handling. The default value is - -### Toggling server statistics - -`--server.statistics value` - -If this option is *value* is *false*, then ArangoDB's statistics gathering -is turned off. Statistics gathering causes regular CPU activity so using this -option to turn it off might relieve heavy-loaded instances a bit. - -### Foxx queues -@startDocuBlock foxxQueues - -### Foxx queues poll interval -@startDocuBlock foxxQueuesPollInterval - -### Directory - -`--database.directory directory` - -The directory containing the collections and datafiles. Defaults -to */var/lib/arango*. When specifying the database directory, please -make sure the directory is actually writable by the arangod process. - -You should further not use a database directory which is provided by a -network filesystem such as NFS. The reason is that networked filesystems -might cause inconsistencies when there are multiple parallel readers or -writers or they lack features required by arangod (e.g. flock()). - -`directory` - -When using the command line version, you can simply supply the database -directory as argument. - - -**Examples** - - -``` -> ./arangod --server.endpoint tcp://127.0.0.1:8529 --database.directory -/tmp/vocbase -``` - -### Database directory state precondition - -`--database.require-directory-state state` - -Using this option it is possible to require the database directory to be -in a specific state on startup. the options for this value are: - -- non-existing: database directory must not exist -- existing: database directory must exist -- empty: database directory must exist but be empty -- populated: database directory must exist and contain specific files already -- any: any directory state allowed - - -### Journal size (MMFiles only) -@startDocuBlock databaseMaximalJournalSize - - -### Wait for sync -@startDocuBlock databaseWaitForSync - - -### Force syncing of properties -@startDocuBlock databaseForceSyncProperties - - -### Data source flush synchronization - -`--server.flush-interval` - -ArangoDB will periodically ensure that all data sources (databases, views, etc.) -have flushed all committed data to disk and write some checkpoint data to aid in -future recovery. Increasing this value will result in fewer, larger write -batches, while decreasing it will result in more, smaller writes. Setting the -value too low can easily overwhelm the server, while setting the value too high -may result in high memory usage and periodic slowdowns. Value is given in -microseconds, with a typical range of 100000 (100ms) to 10000000 (10s) and a -default of 1000000 (1s). Use caution when changing from the default. - -### Limiting memory for AQL queries - -`--query.memory-limit value` - -The default maximum amount of memory (in bytes) that a single AQL query can use. -When a single AQL query reaches the specified limit value, the query will be -aborted with a *resource limit exceeded* exception. In a cluster, the memory -accounting is done per shard, so the limit value is effectively a memory limit per -query per shard. - -The global limit value can be overriden per query by setting the *memoryLimit* -option value for individual queries when running an AQL query. - -The default value is *0*, meaning that there is no memory limit. - - -### Turning AQL warnings into errors - -`--query.fail-on-warning value` - -When set to *true*, AQL queries that produce warnings will instantly abort and -throw an exception. This option can be set to catch obvious issues with AQL -queries early. When set to *false*, AQL queries that produce warnings will not -abort and return the warnings along with the query results. -The option can also be overridden for each individual AQL query. - - -### Enable/disable AQL query tracking - -`--query.tracking flag` - -If *true*, the server's AQL slow query tracking feature will be enabled by -default. Tracking of queries can be disabled by setting the option to *false*. - -The default is *true*. - - -### Enable/disable tracking of bind variables in AQL queries - -`--query.tracking-with-bindvars flag` - -If *true*, then the bind variables will be tracked for all running and slow -AQL queries. This option only has an effect if `--query.tracking` was set to -*true*. Tracking of bind variables can be disabled by setting the option to *false*. - -The default is *true*. - - -### Threshold for slow AQL queries - -`--query.slow-threshold value` - -By setting *value* it can be controlled after what execution time an AQL query -is considered "slow". Any slow queries that exceed the execution time specified -in *value* will be logged when they are finished. The threshold value is -specified in seconds. Tracking of slow queries can be turned off entirely by -setting the option `--query.tracking` to *false*. - -The default value is *10.0*. - - -### Limiting the number of query execution plans created by the AQL optimizer - -`--query.optimizer-max-plans value` - -By setting *value* it can be controlled how many different query execution plans -the AQL query optimizer will generate at most for any given AQL query. Normally -the AQL query optimizer will generate a single execution plan per AQL query, but -there are some cases in which it creates multiple competing plans. More plans -can lead to better optimized queries, however, plan creation has its costs. The -more plans are created and shipped through the optimization pipeline, the more -time will be spent in the optimizer. -Lowering *value* will make the optimizer stop creating additional plans when it -has already created enough plans. -Note that this setting controls the default maximum number of plans to create. The -value can still be adjusted on a per-query basis by setting the *maxNumberOfPlans* -attribute when running a query. - -The default value is *128*. - - -### Throw collection not loaded error - -`--database.throw-collection-not-loaded-error flag` - -Accessing a not-yet loaded collection will automatically load a collection on -first access. This flag controls what happens in case an operation would need to -wait for another thread to finalize loading a collection. If set to *true*, then -the first operation that accesses an unloaded collection will load it. Further -threads that try to access the same collection while it is still loading will -get an error (1238, *collection not loaded*). When the initial operation has -completed loading the collection, all operations on the collection can be -carried out normally, and error 1238 will not be thrown. - -If set to *false*, the first thread that accesses a not-yet loaded collection -will still load it. Other threads that try to access the collection while -loading will not fail with error 1238 but instead block until the collection is -fully loaded. This configuration might lead to all server threads being blocked -because they are all waiting for the same collection to complete -loading. Setting the option to *true* will prevent this from happening, but -requires clients to catch error 1238 and react on it (maybe by scheduling a -retry for later). - -The default value is *false*. - - -### AQL Query caching mode - -`--query.cache-mode` - -Toggles the AQL query cache behavior. Possible values are: - -* *off*: do not use query cache -* *on*: always use query cache, except for queries that have their *cache* - attribute set to *false* -* *demand*: use query cache only for queries that have their *cache* - attribute set to *true* - - -### AQL Query cache size - -`--query.cache-entries` - -Maximum number of query results that can be stored per database-specific query -cache. If a query is eligible for caching and the number of items in the -database's query cache is equal to this threshold value, another cached query -result will be removed from the cache. - -This option only has an effect if the query cache mode is set to either *on* or -*demand*. - - -### JavaScript code execution - -`--javascript.allow-admin-execute` - -This option can be used to control whether user-defined JavaScript code -is allowed to be executed on server by sending via HTTP to the API endpoint -`/_admin/execute` with an authenticated user account. -The default value is *false*, which disables the execution of user-defined -code. This is also the recommended setting for production. In test environments, -it may be convenient to turn the option on in order to send arbitrary setup -or teardown commands for execution on the server. - - -### V8 contexts - -`--javascript.v8-contexts number` - -Specifies the maximum *number* of V8 contexts that are created for executing -JavaScript code. More contexts allow executing more JavaScript actions in parallel, -provided that there are also enough threads available. Please note that each V8 context -will use a substantial amount of memory and requires periodic CPU processing -time for garbage collection. - -Note that this value configures the maximum number of V8 contexts that can be -used in parallel. Upon server start only as many V8 contexts will be created as are -configured in option `--javascript.v8-contexts-minimum`. The actual number of -available V8 contexts may float at runtime between `--javascript.v8-contexts-minimum` -and `--javascript.v8-contexts`. When there are unused V8 contexts that linger around, -the server's garbage collector thread will automatically delete them. - - -`--javascript.v8-contexts-minimum number` - -Specifies the minimum *number* of V8 contexts that will be present at any time -the server is running. The actual number of V8 contexts will never drop below this -value, but it may go up as high as specified via the option `--javascript.v8-contexts`. - -When there are unused V8 contexts that linger around and the number of V8 contexts -is greater than `--javascript.v8-contexts-minimum` the server's garbage collector -thread will automatically delete them. - - -`--javascript.v8-contexts-max-invocations` - -Specifies the maximum number of invocations after which a used V8 context is -disposed. The default value of `--javascript.v8-contexts-max-invocations` is 0, -meaning that the maximum number of invocations per context is unlimited. - -`--javascript.v8-contexts-max-age` - -Specifies the time duration (in seconds) after which time a V8 context is disposed -automatically after its creation. If the time is elapsed, the context will be disposed. -The default value for `--javascript.v8-contexts-max-age` is 60 seconds. - -If both `--javascript.v8-contexts-max-invocations` and `--javascript.v8-contexts-max-age` -are set, then the context will be destroyed when either of the specified threshold -values is reached. - - -### Garbage collection frequency (time-based) - -`--javascript.gc-frequency frequency` - -Specifies the frequency (in seconds) for the automatic garbage collection of -JavaScript objects. This setting is useful to have the garbage collection still -work in periods with no or little numbers of requests. - - -### Garbage collection interval (request-based) - -`--javascript.gc-interval interval` - -Specifies the interval (approximately in number of requests) that the garbage -collection for JavaScript objects will be run in each thread. - - -### V8 options - -`--javascript.v8-options options` - -Optional arguments to pass to the V8 Javascript engine. The V8 engine will run -with default settings unless explicit options are specified using this -option. The options passed will be forwarded to the V8 engine which will parse -them on its own. Passing invalid options may result in an error being printed on -stderr and the option being ignored. - -Options need to be passed in one string, with V8 option names being prefixed -with double dashes. Multiple options need to be separated by whitespace. To get -a list of all available V8 options, you can use the value *"--help"* as follows: - -``` ---javascript.v8-options="--help" -``` - -Another example of specific V8 options being set at startup: - -``` ---javascript.v8-options="--log" -``` - -Names and features or usable options depend on the version of V8 being used, and -might change in the future if a different version of V8 is being used in -ArangoDB. Not all options offered by V8 might be sensible to use in the context -of ArangoDB. Use the specific options only if you are sure that they are not -harmful for the regular database operation. - - -#### Enable or Disable V8 JavaScript Engine entirely - -``` ---javascript.enabled bool -``` - -In certain types of ArangoDB instances you can now completely disable the V8 JavaScript engine. Be aware that this is -an **highly experimental** feature and it is to be expected that certain functionality (e.g. some API endpoints, the WebUI, -some AQL functions etc) will be missing or severly broken. Nevertheless you may whish to reduce the footprint of ArangoDB by disabling V8. - -This option is expected to **only** work reliably on a _single server_, _agency_ or in an _active failover_ setup. Do not try to use -this feature on a _coordinator_, or _cluster database server_. diff --git a/Documentation/Books/Manual/Administration/ManagingUsers/README.md b/Documentation/Books/Manual/Administration/ManagingUsers/README.md index 98765508ec..cc29b61842 100644 --- a/Documentation/Books/Manual/Administration/ManagingUsers/README.md +++ b/Documentation/Books/Manual/Administration/ManagingUsers/README.md @@ -293,8 +293,9 @@ This feature is only available in the [**Enterprise Edition**](https://www.arangodb.com/why-arangodb/arangodb-enterprise/) {% endhint %} -ArangoDB supports LDAP as an external authentication system. For detailed information please have look into -the [LDAP configuration guide](../Configuration/Ldap.md). +ArangoDB supports LDAP as an external authentication system. For detailed +information please have look into the +[LDAP configuration guide](../../Programs/Arangod/Ldap.md). There are a few differences to *normal* ArangoDB users: - ArangoDB does not "*know*" LDAP users before they first authenticate, calls to various API's using endpoints in `_api/users/*` will **fail** until the user first logs-in diff --git a/Documentation/Books/Manual/Appendix/JavaScriptModules/Request.md b/Documentation/Books/Manual/Appendix/JavaScriptModules/Request.md index b2e9d90260..7d97aa9b04 100644 --- a/Documentation/Books/Manual/Appendix/JavaScriptModules/Request.md +++ b/Documentation/Books/Manual/Appendix/JavaScriptModules/Request.md @@ -62,7 +62,7 @@ The request function takes the following options: * *json*: if `true`, *body* will be serialized to a JSON string and the *Content-Type* header will be set to `"application/json"`. Additionally the response body will also be parsed as JSON (unless *encoding* is set to `null`). Default: `false`. * *form* (optional): when set to a string or object and no *body* has been set, *body* will be set to a querystring representation of that value and the *Content-Type* header will be set to `"application/x-www-form-urlencoded"`. Also see *useQuerystring*. * *auth* (optional): an object with the properties *username* and *password* for HTTP Basic authentication or the property *bearer* for HTTP Bearer token authentication. -* *sslProtocol* (optional): which tls version should be used to connect to the url. The default is `4` which is TLS 1.0. See [ssl protocol](../../Administration/Configuration/SSL.md#ssl-protocol) for more opitions. +* *sslProtocol* (optional): which tls version should be used to connect to the url. The default is `4` which is TLS 1.0. See [SSL protocol](../../Programs/Arangod/Ssl.md#ssl-protocol) for more opitions. * *followRedirect*: whether HTTP 3xx redirects should be followed. Default: `true`. * *maxRedirects*: the maximum number of redirects to follow. Default: `10`. * *encoding*: encoding to be used for the response body. If set to `null`, the response body will be returned as a `Buffer`. Default: `"utf-8"`. diff --git a/Documentation/Books/Manual/Appendix/JavaScriptModules/WAL.md b/Documentation/Books/Manual/Appendix/JavaScriptModules/WAL.md index eb04ddf46c..d7aad8ee7e 100644 --- a/Documentation/Books/Manual/Appendix/JavaScriptModules/WAL.md +++ b/Documentation/Books/Manual/Appendix/JavaScriptModules/WAL.md @@ -5,7 +5,7 @@ Write-ahead log This module provides functionality for administering the write-ahead logs. Most of these functions only return sensible values when invoked with the -[mmfiles engine being active](../../Administration/Configuration/GeneralArangod.md#storage-engine). +[MMFiles engine being active](../../Programs/Arangod/Server.md#storage-engine). Configuration ------------- diff --git a/Documentation/Books/Manual/Architecture/StorageEngines.md b/Documentation/Books/Manual/Architecture/StorageEngines.md index 11fa256b17..2dd6bb4ab1 100644 --- a/Documentation/Books/Manual/Architecture/StorageEngines.md +++ b/Documentation/Books/Manual/Architecture/StorageEngines.md @@ -76,11 +76,11 @@ The entire user transaction will not necessarily have ACID properties in this ca The threshold values for transaction sizes can be configured globally using the startup options -* [`--rocksdb.intermediate-commit-size`](../Administration/Configuration/RocksDB.md#non-pass-through-options) +* [`--rocksdb.intermediate-commit-size`](../Programs/Arangod/Rocksdb.md#non-pass-through-options) -* [`--rocksdb.intermediate-commit-count`](../Administration/Configuration/RocksDB.md#non-pass-through-options) +* [`--rocksdb.intermediate-commit-count`](../Programs/Arangod/Rocksdb.md#non-pass-through-options) -* [`--rocksdb.max-transaction-size`](../Administration/Configuration/RocksDB.md#non-pass-through-options) +* [`--rocksdb.max-transaction-size`](../Programs/Arangod/Rocksdb.md#non-pass-through-options) It is also possible to override these thresholds per transaction. diff --git a/Documentation/Books/Manual/Architecture/WriteAheadLog.md b/Documentation/Books/Manual/Architecture/WriteAheadLog.md index 35eb6b8115..6b45741b35 100644 --- a/Documentation/Books/Manual/Architecture/WriteAheadLog.md +++ b/Documentation/Books/Manual/Architecture/WriteAheadLog.md @@ -38,23 +38,33 @@ already collected write-ahead logfiles so replication slaves still can fetch dat them if required. How many collected logfiles will be kept before they get deleted is configurable via the option *--wal.historic-logfiles*. -For all write-ahead log configuration options, please refer to the page [Write-ahead log options](../Administration/Configuration/Wal.md). +For all write-ahead log configuration options, please refer to the page +[Write-ahead log options](../Programs/Arangod/Wal.md). RocksDB WAL Details ------------------- -The options mentioned above only apply for MMFiles. The WAL in the RocksDB storage engine -works slightly differently. -_Note:_ In rocksdb the WAL options are all prefixed with -`--rocksdb.*`. The `--wal.*` options do have no effect. +The options mentioned above only apply for MMFiles. The WAL in the RocksDB +storage engine works slightly differently. -The individual RocksDB WAL files are per default about 64 MiB big. The size will always be proportionally -sized to the value specified via `--rocksdb.write-buffer-size`. The value specifies the amount of -data to build up in memory (backed by the unsorted WAL on disk) before converting it to a sorted on-disk file. -Larger values can increase performance, especially during bulk loads. Up to `--rocksdb.max-write-buffer-number` -write buffers may be held in memory at the same time, so you may wish to adjust this parameter to control memory usage. A larger write buffer will result in a longer recovery time the next time the database is opened. +_Note:_ In rocksdb the WAL options are all prefixed with `--rocksdb.*`. +The `--wal.*` options do have no effect. -The RocksDB WAL only contains committed transactions. This means you will never see partial transactions -in the replication log, but it also means transactions are tracked completely in-memory. In practice -this causes RocksDB transaction sizes to be limited, for more information see the [RocksDB Configuration](../Administration/Configuration/RocksDB.md) +The individual RocksDB WAL files are per default about 64 MiB big. +The size will always be proportionally sized to the value specified via +`--rocksdb.write-buffer-size`. The value specifies the amount of data to build +up in memory (backed by the unsorted WAL on disk) before converting it to a +sorted on-disk file. + +Larger values can increase performance, especially during bulk loads. +Up to `--rocksdb.max-write-buffer-number` write buffers may be held in memory +at the same time, so you may wish to adjust this parameter to control memory +usage. A larger write buffer will result in a longer recovery time the next +time the database is opened. + +The RocksDB WAL only contains committed transactions. This means you will never +see partial transactions in the replication log, but it also means transactions +are tracked completely in-memory. In practice this causes RocksDB transaction +sizes to be limited, for more information see the +[RocksDB Configuration](../Programs/Arangod/Rocksdb.md) diff --git a/Documentation/Books/Manual/Foxx/Reference/Modules/Queues.md b/Documentation/Books/Manual/Foxx/Reference/Modules/Queues.md index dc0db7d07c..20c80f5191 100644 --- a/Documentation/Books/Manual/Foxx/Reference/Modules/Queues.md +++ b/Documentation/Books/Manual/Foxx/Reference/Modules/Queues.md @@ -15,7 +15,7 @@ Please note that Foxx queues are database-specific. Queues and jobs are always relative to the database in which they are created or accessed. For disabling the Foxx queues feature or adjusting the polling interval see the -[`foxx.queues` and `foxx.queues-poll-interval` options](../../../Administration/Configuration/GeneralArangod.md#foxx-queues). +[`foxx.queues` and `foxx.queues-poll-interval` options](../../../Programs/Arangod/Foxx.md). For the low-level functionality see the chapter on the [task management module](../../../Appendix/JavaScriptModules/Tasks.md). diff --git a/Documentation/Books/Manual/Highlights.md b/Documentation/Books/Manual/Highlights.md index 3215a55630..a61e4393b1 100644 --- a/Documentation/Books/Manual/Highlights.md +++ b/Documentation/Books/Manual/Highlights.md @@ -54,7 +54,7 @@ Version 3.2 - **Enterprise**: Working with some of our largest customers, we have added further security and scalability features to ArangoDB Enterprise like - [LDAP integration](Administration/Configuration/Ldap.md), + [LDAP integration](Programs/Arangod/Ldap.md), [Encryption at Rest](Security/Encryption/README.md), and the brand new [Satellite Collections](Satellites.md). @@ -68,7 +68,7 @@ Version 3.1 of your application layer to shard your graph efficiently to your machines and let traversals run locally. -- **Encryption Control**: Choose your level of [SSL encryption](Administration/Configuration/SSL.md) +- **Encryption Control**: Choose your level of [SSL encryption](Programs/Arangod/Ssl.md) - [**Auditing**](Security/Auditing/README.md): Keep a detailed log of all the important things that happened in ArangoDB. diff --git a/Documentation/Books/Manual/Indexing/Persistent.md b/Documentation/Books/Manual/Indexing/Persistent.md index ca3ffe5bde..8bb5b7887e 100644 --- a/Documentation/Books/Manual/Indexing/Persistent.md +++ b/Documentation/Books/Manual/Indexing/Persistent.md @@ -161,7 +161,7 @@ Persistent Indexes and Server Language -------------------------------------- The order of index entries in persistent indexes adheres to the configured -[server language](../Administration/Configuration/GeneralArangod.md#default-language). +[server language](../Programs/Arangod/Global.md#default-language). If, however, the server is restarted with a different language setting as when the persistent index was created, not all documents may be returned anymore and the sort order of those which are returned can be wrong (whenever the persistent diff --git a/Documentation/Books/Manual/Administration/Configuration/OperatingSystem.md b/Documentation/Books/Manual/Installation/LinuxOSConfiguration.md similarity index 96% rename from Documentation/Books/Manual/Administration/Configuration/OperatingSystem.md rename to Documentation/Books/Manual/Installation/LinuxOSConfiguration.md index 07cb0c9abb..61d7acd085 100644 --- a/Documentation/Books/Manual/Administration/Configuration/OperatingSystem.md +++ b/Documentation/Books/Manual/Installation/LinuxOSConfiguration.md @@ -1,13 +1,11 @@ -Operating System Configuration -============================== +Linux Operating System Configuration +==================================== File Systems ------------ -_Linux only_ - We recommend to **not** use BTRFS on linux, it's known to not work -well in conjunction with ArangoDB. We experienced that arangodb +well in conjunction with ArangoDB. We experienced that ArangoDB facing latency issues on accessing its database files on BTRFS partitions. In conjunction with BTRFS and AUFS we also saw data loss on restart. @@ -16,8 +14,6 @@ on restart. Virtual Memory Page Sizes -------------------------- -_Linux only_ - By default, ArangoDB uses Jemalloc as the memory allocator. Jemalloc does a good job of reducing virtual memory fragmentation, especially for long-running processes. Unfortunately, some OS configurations can interfere with Jemalloc's @@ -38,8 +34,6 @@ before executing `arangod`. Swap Space ---------- -_Linux only_ - It is recommended to assign swap space for a server that is running arangod. Configuring swap space can prevent the operating system's OOM killer from killing ArangoDB too eagerly on Linux. @@ -107,8 +101,6 @@ should be started with interleave on such system. This can be achieved using Max Memory Mappings ------------------- -_Linux only_ - Linux kernels by default restrict the maximum number of memory mappings of a single process to about 64K mappings. While this value is sufficient for most workloads, it may be too low for a process that has lots of parallel threads @@ -137,8 +129,6 @@ looking at. Environment Variables --------------------- -_Linux only_ - It is recommended to set the environment variable `GLIBCXX_FORCE_NEW` to 1 on systems that use glibc++ in order to disable the memory pooling built into glibc++. That memory pooling is unnecessary because Jemalloc will already do @@ -157,4 +147,3 @@ before starting `arangod`. While it is possible to compile ArangoDB on 32bit system, this is not a recommended environment. 64bit systems can address a significantly bigger memory region. - diff --git a/Documentation/Books/Manual/Programs/Arangod/Agency.md b/Documentation/Books/Manual/Programs/Arangod/Agency.md new file mode 100644 index 0000000000..4a84c9525d --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Agency.md @@ -0,0 +1,61 @@ +# ArangoDB Server Agency Options + +## Activate + +`agency.activate` + +Activate agency. + +## Compaction + +`agency.compaction-keep-size` + +Keep as many indices before compaction point. + +## Election + +`agency.election-timeout-max` + +Maximum timeout before an agent calls for new election in seconds. + +`agency.election-timeout-min` + +Minimum timeout before an agent calls for new election in seconds. + +## Endpoint + +`agency.endpoint` + +Agency endpoints. + +## My address + +`agency.my-address` + +Which address to advertise to the outside. + +## Pool size + +`agency.pool-size` + +Number of agent pool. + +## Size + +`agency.size` + +Number of agents. + +## Supervision + +`agency.supervision` + +Perform ArangoDB cluster supervision. + +`agency.supervision-frequency` + +ArangoDB cluster supervision frequency in seconds. + +`agency.supervision-grace-period` + +Supervision time, after which a server is considered to have failed in seconds. diff --git a/Documentation/Books/Manual/Programs/Arangod/Arangosearch.md b/Documentation/Books/Manual/Programs/Arangod/Arangosearch.md new file mode 100644 index 0000000000..9e21e1014d --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Arangosearch.md @@ -0,0 +1,12 @@ +# ArangoDB Server ArangoSearch Options + +## Threads + +`arangosearch.threads` + +The exact number of threads to use for asynchronous tasks (0 == autodetect). + +`arangosearch.threads-limit` + +Upper limit to the auto-detected number of threads to use for asynchronous +tasks (0 == use default). diff --git a/Documentation/Books/Manual/Programs/Arangod/Audit.md b/Documentation/Books/Manual/Programs/Arangod/Audit.md new file mode 100644 index 0000000000..591155e40e --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Audit.md @@ -0,0 +1,13 @@ +# ArangoDB Server Audit Options + +## Hostname + +`audit.hostname` + +Hostname to use. + +## Output + +`audit.output` + +Audit destination(s). diff --git a/Documentation/Books/Manual/Administration/Configuration/Cache.md b/Documentation/Books/Manual/Programs/Arangod/Cache.md similarity index 61% rename from Documentation/Books/Manual/Administration/Configuration/Cache.md rename to Documentation/Books/Manual/Programs/Arangod/Cache.md index 0bb7824189..076bb65c78 100644 --- a/Documentation/Books/Manual/Administration/Configuration/Cache.md +++ b/Documentation/Books/Manual/Programs/Arangod/Cache.md @@ -1,14 +1,19 @@ -Hash cache options -================== +# ArangoDB Server Cache Options -Since ArangoDB 3.2, the several core components of the server use a cache system -which pools memory across many different cache tables. In order to provide -intelligent internal memory management, the system periodically reclaims memory -from caches which are used less often and reallocates it to caches which get -more activity. +Since ArangoDB 3.2, the several core components of the server use a cache +system which pools memory across many different cache tables. In order to +provide intelligent internal memory management, the system periodically +reclaims memory from caches which are used less often and reallocates it to +caches which get more activity. -Cache size ----------- +## Rebalancing interval + +Time between cache rebalancing attempts: `--cache.rebalancing-interval` + +The value is specified in microseconds with a default of 2 seconds and a +minimum of 500 milliseconds. + +## Cache size Global size limit for all hash caches: `--cache.size` @@ -16,11 +21,3 @@ The global caching system, all caches, and all the data contained therein will fit inside this limit. The size is specified in bytes. If there is less than 4GiB of RAM on the system, the default value is 256MiB. If there is more, the default is `(system RAM size - 2GiB) * 0.3`. - -Rebalancing interval --------------------- - -Time between cache rebalancing attempts: `--cache.rebalancing-interval` - -The value is specified in microseconds with a default of 2 seconds and a -minimum of 500 milliseconds. diff --git a/Documentation/Books/Manual/Administration/Configuration/Cluster.md b/Documentation/Books/Manual/Programs/Arangod/Cluster.md similarity index 85% rename from Documentation/Books/Manual/Administration/Configuration/Cluster.md rename to Documentation/Books/Manual/Programs/Arangod/Cluster.md index a00ff6aeff..a2c8da863a 100644 --- a/Documentation/Books/Manual/Administration/Configuration/Cluster.md +++ b/Documentation/Books/Manual/Programs/Arangod/Cluster.md @@ -1,12 +1,9 @@ -Clusters Options -================ +# ArangoDB Server Clusters Options -Agency endpoint ---------------- +## Agency endpoint - List of agency endpoints: `--cluster.agency-endpoint endpoint` @@ -29,12 +26,10 @@ alternative endpoint if one of them becomes unavailable. --cluster.agency-endpoint tcp://192.168.1.1:4001 --cluster.agency-endpoint tcp://192.168.1.2:4002 ... ``` -My address ----------- +## My address - This server's address / endpoint: `--cluster.my-address endpoint` @@ -51,31 +46,30 @@ for the server's id, ArangoDB will refuse to start. **Examples** -Listen only on interface with address `192.168.1.1` +Listen only on interface with address `192.168.1.1`: + ``` --cluster.my-address tcp://192.168.1.1:8530 ``` -Listen on all ipv4 and ipv6 addresses, which are configured on port `8530` +Listen on all ipv4 and ipv6 addresses, which are configured on port `8530`: + ``` --cluster.my-address ssl://[::]:8530 ``` -My role -------- +## My role - This server's role: `--cluster.my-role [dbserver|coordinator]` The server's role. Is this instance a db server (backend data server) or a coordinator (frontend server for external and application access) -Require existing ID -------------------- - +## Require existing ID + Require an existing server id: `--cluster.require-persisted-id bool` If set to true, then the instance will only start if a UUID file is found @@ -85,8 +79,7 @@ start, and not a new one. For the first start, the UUID file must either be created manually in the database directory, or the option must be set to false for the initial startup and only turned on for restarts. -More advanced options ---------------------- +## More advanced options {% hint 'warning' %} These options should generally remain untouched. @@ -94,13 +87,13 @@ These options should generally remain untouched. +Synchronous replication timing: +`--cluster.synchronous-replication-timeout-factor double` -Synchronous replication timing: `--cluster.synchronous-replication-timeout-factor double` - -Strech or clinch timeouts for internal synchroneous replication +Stretch or clinch timeouts for internal synchronous replication mechanism between db servers. All such timeouts are affected by this change. Please change only with intent and great care. Default at `1.0`. -System replication factor: `--cluster.system-replication-factorinteger` +System replication factor: `--cluster.system-replication-factor integer` Change default replication factor for system collections. Default at `2`. diff --git a/Documentation/Books/Manual/Administration/Configuration/Compaction.md b/Documentation/Books/Manual/Programs/Arangod/Compaction.md similarity index 53% rename from Documentation/Books/Manual/Administration/Configuration/Compaction.md rename to Documentation/Books/Manual/Programs/Arangod/Compaction.md index b7796714de..75501bc429 100644 --- a/Documentation/Books/Manual/Administration/Configuration/Compaction.md +++ b/Documentation/Books/Manual/Programs/Arangod/Compaction.md @@ -1,87 +1,100 @@ -MMFiles Compaction options -========================== +# ArangoDB Server Compaction Options (MMFiles) The ArangoDB MMFiles storage engine will run a compaction over data files. -ArangoDB writes Documents in the WAL file. Once they have been sealed in the wal file, -the collector may copy them into a per collection journal file. +ArangoDB writes Documents in the WAL file. Once they have been sealed in the +WAL file, the collector may copy them into a per collection journal file. Once journal files fill up, they're sealed to become data files. -=> one collection may have documents in the WAL logs, its journal file, and an arbitrary -number of data files. +One collection may have documents in the WAL logs, its journal file, and an +arbitrary number of data files. -If a collection is loaded, each of these files are opened (thus use a file handle) and -are mmap'ed. Since file handles and memory mapped files are also a sparse resource, -that number should be kept low. +If a collection is loaded, each of these files are opened (thus use a file +handle) and are mmap'ed. Since file handles and memory mapped files are also +a sparse resource, that number should be kept low. -Once you update or remove documents from data files (or already did while it was the journal file) -these documents are marked as 'dead' with a deletion marker. +Once you update or remove documents from data files (or already did while it was +the journal file) these documents are marked as 'dead' with a deletion marker. -Over time the number of dead documents may rise, and we don't want to use the previously mentioned -resources, plus the disk space should be given back to the system. -Thus several journal files can be combined to one, omitting the dead documents. +Over time the number of dead documents may rise, and we don't want to use the +previously mentioned resources, plus the disk space should be given back to +the system. Thus several journal files can be combined to one, omitting the +dead documents. -Combining several of these data files into one is called compaction. The compaction process reads -the alive documents from the original data files, and writes them into new data file. +Combining several of these data files into one is called compaction. +The compaction process reads the alive documents from the original data files, +and writes them into new data file. -Once that is done, the memory mappings to the old data files is released, and the files are erased. +Once that is done, the memory mappings to the old data files is released, and +the files are erased. -Since the compaction locks the collection, and also uses I/O resources, its carefully configurable -under which conditions the system should perform which amount of these compaction jobs: +Since the compaction locks the collection, and also uses I/O resources, its +carefully configurable under which conditions the system should perform which +amount of these compaction jobs: -ArangoDB spawns one compactor thread per database. The settings below vary in scope. +ArangoDB spawns one compactor thread per database. The settings below vary +in scope. +## Activity control -Activity control ----------------- - -The activity control parameters alter the behaviour in terms of scan / execution frequency -of the compaction. +The activity control parameters alter the behavior in terms of scan / execution +frequency of the compaction. Sleep interval between two compaction runs (in seconds): `--compaction.db-sleep-time` -The number of seconds the collector thread will wait between two attempts to search for -compactable data files of collections in one Database. If the compactor has actually executed work, -a subsequent lookup is done. Scope: Database. + +The number of seconds the collector thread will wait between two attempts to +search for compactable data files of collections in one Database. +If the compactor has actually executed work, a subsequent lookup is done. + +Scope: Database. Minimum sleep time between two compaction runs (in seconds): `--compaction.min-interval` -When an actual compaction was executed for one collection, we wait for this time -before we execute the compaction on this collection again. -This is here to let eventually piled up user load be worked out. Scope: collection. +When an actual compaction was executed for one collection, we wait for this +time before we execute the compaction on this collection again. +This is here to let eventually piled up user load be worked out. -Source data files ------------------ +Scope: collection. -These parameters control which data files are taken into account for a compaction run. -You can specify several criteria which each off may be sufficcient alone. +## Source data files -The scan over the data files belonging to one collection is executed from oldest -data file to newest; if files qualify for a compaction they may be merged with newer files -(containing younger documents) +These parameters control which data files are taken into account for a +compaction run. You can specify several criteria which each off may be +sufficient alone. + +The scan over the data files belonging to one collection is executed from +oldest data file to newest; if files qualify for a compaction they may be +merged with newer files (containing younger documents). Scope: Collection level, some are influenced by collection settings. -minimal filesize threshold original data files have to be below for a compaction: +Minimal file size threshold original data files have to be below for +a compaction: `--compaction.min-small-data-file-size` + This is the threshold which controls below which minimum total size a data file will always be taken into account for the compaction. Minimum unused count of documents in a datafile: `--compaction.dead-documents-threshold` + Data files will often contain dead documents. This parameter specifies their top most accetpeable count until the data file qualifies for compaction. How many bytes of the source data file are allowed to be unused at most: `--compaction.dead-size-threshold` + The dead data size varies along with the size of your documents. -If you have many big documents, this threshold may hit before the document count threshold. +If you have many big documents, this threshold may hit before the document +count threshold. How many percent of the source data file should be unused at least: `--compaction.dead-size-percent-threshold` -since the size of the documents may vary this threshold works on the + +Since the size of the documents may vary this threshold works on the percentage of the dead documents size. Thus, if you have many huge dead documents, this threshold kicks in earlier. @@ -90,32 +103,36 @@ and 400 kbytes of dead documents, the share of the dead documents is: `400 / (400 + 800) = 33 %`. -If this value if higher than the specified threshold, the data file will be compacted. +If this value if higher than the specified threshold, the data file will +be compacted. +## Compacted target files -Compacted target files ----------------------- - -Once data files of a collection are qualified for a compaction run, these parameters control -how many data files are merged into one, (or even one source data file may be compacted into -one smaller target data file) +Once data files of a collection are qualified for a compaction run, these +parameters control how many data files are merged into one, (or even one source +data file may be compacted into one smaller target data file) Scope: Collection level, some are influenced by collection settings. Maximum number of files to merge to one file: `--compaction.dest-max-files` -How many data files (at most) we may merge into one resulting data file during one compaction run. + +How many data files (at most) we may merge into one resulting data file during +one compaction run. How large the resulting file may be in comparison to the collections `database.maximal-journal-size` setting: `--compaction.dest-max-file-size-factor` -In ArangoDB you can configure a default `journal filesize` globally and override it on a per collection level. -This value controls the size of collected data files relative to the configured journal file size of the collection in question. -A factor of 3 means that the maximum filesize of the compacted file is +In ArangoDB you can configure a default *journal file size* globally and +override it on a per collection level. This value controls the size of +collected data files relative to the configured journal file size of the +collection in question. + +A factor of 3 means that the maximum file size of the compacted file is 3 times the size of the maximum collection journal file size. -how large may the compaction result file become: +How large may the compaction result file become: `--compaction.dest-max-result-file-size` -next to the factor above, a totally maximum allowed filesize in bytes may be specified. -This will overrule all previous parameters. +Next to the factor above, a totally maximum allowed file size in bytes may +be specified. This will overrule all previous parameters. diff --git a/Documentation/Books/Manual/Programs/Arangod/Database.md b/Documentation/Books/Manual/Programs/Arangod/Database.md new file mode 100644 index 0000000000..50388578fb --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Database.md @@ -0,0 +1,116 @@ +# ArangoDB Server Database Options + +## Auto upgrade + +`--database.auto-upgrade` + +Specifying this option will make the server perform a database upgrade instead +of starting the server normally. A database upgrade will first compare the +version number stored in the file VERSION in the database directory with the +current server version. + +If the version number found in the database directory is higher than the version +number the server is running, the server expects this is an unintentional +downgrade and will warn about this. Using the server in these conditions is +neither recommended nor supported. + +If the version number found in the database directory is lower than the version +number the server is running, the server will check whether there are any +upgrade tasks to perform. It will then execute all required upgrade tasks and +print their statuses. If one of the upgrade tasks fails, the server will exit +with an error. Re-starting the server with the upgrade option will then again +trigger the upgrade check and execution until the problem is fixed. + +Whether or not this option is specified, the server will always perform a +version check on startup. Running the server with a non-matching version number +in the VERSION file will make the server refuse to start. + +## Directory + +`--database.directory directory` + +The directory containing the collections and datafiles. Defaults +to */var/lib/arango*. When specifying the database directory, please +make sure the directory is actually writable by the arangod process. + +You should further not use a database directory which is provided by a +network filesystem such as NFS. The reason is that networked filesystems +might cause inconsistencies when there are multiple parallel readers or +writers or they lack features required by arangod (e.g. flock()). + +`directory` + +When using the command line version, you can simply supply the database +directory as argument. + +**Examples** + +``` +> ./arangod --server.endpoint tcp://127.0.0.1:8529 --database.directory +/tmp/vocbase +``` + +## Database directory state precondition + +`--database.require-directory-state state` + +Using this option it is possible to require the database directory to be +in a specific state on startup. the options for this value are: + +- non-existing: database directory must not exist +- existing: database directory must exist +- empty: database directory must exist but be empty +- populated: database directory must exist and contain specific files already +- any: any directory state allowed + +## Force syncing of properties + +@startDocuBlock databaseForceSyncProperties + +## Maximal Journal size (MMFiles only) + +@startDocuBlock databaseMaximalJournalSize + +## Wait for sync + +@startDocuBlock databaseWaitForSync + +## More advanced options + +`--database.throw-collection-not-loaded-error flag` + +Accessing a not-yet loaded collection will automatically load a collection on +first access. This flag controls what happens in case an operation would need to +wait for another thread to finalize loading a collection. If set to *true*, then +the first operation that accesses an unloaded collection will load it. Further +threads that try to access the same collection while it is still loading will +get an error (1238, *collection not loaded*). When the initial operation has +completed loading the collection, all operations on the collection can be +carried out normally, and error 1238 will not be thrown. + +If set to *false*, the first thread that accesses a not-yet loaded collection +will still load it. Other threads that try to access the collection while +loading will not fail with error 1238 but instead block until the collection is +fully loaded. This configuration might lead to all server threads being blocked +because they are all waiting for the same collection to complete +loading. Setting the option to *true* will prevent this from happening, but +requires clients to catch error 1238 and react on it (maybe by scheduling a +retry for later). + +The default value is *false*. + +`--database.replication-applier flag` + +Enable/disable replication applier. + +If *false* the server will start with replication appliers turned off, +even if the replication appliers are configured with the *autoStart* option. +Using the command-line option will not change the value of the *autoStart* +option in the applier configuration, but will suppress auto-starting the +replication applier just once. + +If the option is not used, ArangoDB will read the applier configuration +from the file *REPLICATION-APPLIER-CONFIG* on startup, and use the value of +the *autoStart* attribute from this file. + +The default is *true*. diff --git a/Documentation/Books/Manual/Programs/Arangod/Foxx.md b/Documentation/Books/Manual/Programs/Arangod/Foxx.md new file mode 100644 index 0000000000..87d27a4b86 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Foxx.md @@ -0,0 +1,9 @@ +# ArangoDB Server Foxx Options + +## Foxx queues + +@startDocuBlock foxxQueues + +## Foxx queues poll interval + +@startDocuBlock foxxQueuesPollInterval diff --git a/Documentation/Books/Manual/Programs/Arangod/Frontend.md b/Documentation/Books/Manual/Programs/Arangod/Frontend.md new file mode 100644 index 0000000000..36501d83cd --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Frontend.md @@ -0,0 +1,12 @@ +# ArangoDB Server Frontend Options + +## Proxy settings + +`frontend.proxy-request-check` + +Enable proxy request checking. + +`frontend.trusted-proxy` + +List of proxies to trust (may be IP or network). +Make sure `frontend.proxy-request-check` is enabled. diff --git a/Documentation/Books/Manual/Programs/Arangod/Global.md b/Documentation/Books/Manual/Programs/Arangod/Global.md new file mode 100644 index 0000000000..0240d6500d --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Global.md @@ -0,0 +1,131 @@ +# ArangoDB Server Global Options + +## General help + +`--help` + +`-h` + +Prints a list of the most common options available and then exits. In order to +see all options use *--help-all*. + +To receive the startup options in JSON format, pass the `--dump-options` flag. This will +print out all options and exit. + +## Version + +`--version` + +`-v` + +Prints the version of the server and exits. + +## Daemon + +`--daemon` + +Runs the server as a daemon (as a background process). This parameter can only +be set if the pid (process id) file is specified. That is, unless a value to the +parameter pid-file is given, then the server will report an error and exit. + +## Default Language + +`--default-language default-language` + +The default language ist used for sorting and comparing strings. The language +value is a two-letter language code (ISO-639) or it is composed by a two-letter +language code with and a two letter country code (ISO-3166). Valid languages are +"de", "en", "en_US" or "en_UK". + +The default default-language is set to be the system locale on that platform. + +## Supervisor + +`--supervisor` + +Executes the server in supervisor mode. In the event that the server +unexpectedly terminates due to an internal error, the supervisor will +automatically restart the server. Setting this flag automatically implies that +the server will run as a daemon. Note that, as with the daemon flag, this flag +requires that the pid-file parameter will set. + +```js +unix> ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ +2012-06-27T15:58:28Z [10133] INFO starting up in supervisor mode +``` + +As can be seen (e.g. by executing the ps command), this will start a supervisor +process and the actual database process: + +```js +unix> ps fax | grep arangod +10137 ? Ssl 0:00 ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ +10142 ? Sl 0:00 \_ ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ +``` + +When the database process terminates unexpectedly, the supervisor process will +start up a new database process: + +``` +> kill -SIGSEGV 10142 + +> ps fax | grep arangod +10137 ? Ssl 0:00 ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ +10168 ? Sl 0:00 \_ ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/ +``` + +## User identity + +`--uid uid` + +The name (identity) of the user the server will run as. If this parameter is not +specified, the server will not attempt to change its UID, so that the UID used +by the server will be the same as the UID of the user who started the server. If +this parameter is specified, then the server will change its UID after opening +ports and reading configuration files, but before accepting connections or +opening other files (such as recovery files). This is useful when the server +must be started with raised privileges (in certain environments) but security +considerations require that these privileges be dropped once the server has +started work. + +Observe that this parameter cannot be used to bypass operating system +security. In general, this parameter (and its corresponding relative gid) can +lower privileges but not raise them. + + +## Group identity + +`--gid gid` + +The name (identity) of the group the server will run as. If this parameter is +not specified, then the server will not attempt to change its GID, so that the +GID the server runs as will be the primary group of the user who started the +server. If this parameter is specified, then the server will change its GID +after opening ports and reading configuration files, but before accepting +connections or opening other files (such as recovery files). + +This parameter is related to the parameter uid. + + +## Process identity + +`--pid-file filename` + +The name of the process ID file to use when running the server as a +daemon. This parameter must be specified if either the flag *daemon* or +*supervisor* is set. + +## Console + +`--console` + +Runs the server in an exclusive emergency console mode. When +starting the server with this option, the server is started with +an interactive JavaScript emergency console, with all networking +and HTTP interfaces of the server disabled. + +No requests can be made to the server in this mode, and the only +way to work with the server in this mode is by using the emergency +console. +Note that the server cannot be started in this mode if it is +already running in this or another mode. diff --git a/Documentation/Books/Manual/Programs/Arangod/Http.md b/Documentation/Books/Manual/Programs/Arangod/Http.md new file mode 100644 index 0000000000..083c5666a4 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Http.md @@ -0,0 +1,45 @@ +# ArangoDB Server HTTP Options + +## Keep-alive timeout + +`--http.keep-alive-timeout` + +Allows to specify the timeout for HTTP keep-alive connections. The timeout +value must be specified in seconds. +Idle keep-alive connections will be closed by the server automatically +when the timeout is reached. A keep-alive-timeout value 0 will disable the keep +alive feature entirely. + +## Hide Product header + +`--http.hide-product-header` + +If *true*, the server will exclude the HTTP header "Server: ArangoDB" in +HTTP responses. If set to *false*, the server will send the header in +responses. + +The default is *false*. + + +## Allow method override + +`--http.allow-method-override` + +When this option is set to *true*, the HTTP request method will optionally +be fetched from one of the following HTTP request headers if present in +the request: + +- *x-http-method* +- *x-http-method-override* +- *x-method-override* + +If the option is set to *true* and any of these headers is set, the +request method will be overridden by the value of the header. For example, +this allows issuing an HTTP DELETE request which to the outside world will +look like an HTTP GET request. This allows bypassing proxies and tools that +will only let certain request types pass. + +Setting this option to *true* may impose a security risk so it should only +be used in controlled environments. + +The default value for this option is *false*. diff --git a/Documentation/Books/Manual/Programs/Arangod/Javascript.md b/Documentation/Books/Manual/Programs/Arangod/Javascript.md new file mode 100644 index 0000000000..888dbb50c6 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Javascript.md @@ -0,0 +1,117 @@ +# ArangoDB Server JavaScript Options + +## JavaScript code execution + +`--javascript.allow-admin-execute` + +This option can be used to control whether user-defined JavaScript code +is allowed to be executed on server by sending via HTTP to the API endpoint +`/_admin/execute` with an authenticated user account. +The default value is *false*, which disables the execution of user-defined +code. This is also the recommended setting for production. In test environments, +it may be convenient to turn the option on in order to send arbitrary setup +or teardown commands for execution on the server. + +## V8 contexts + +`--javascript.v8-contexts number` + +Specifies the maximum *number* of V8 contexts that are created for executing +JavaScript code. More contexts allow executing more JavaScript actions in +parallel, provided that there are also enough threads available. Please note +that each V8 context will use a substantial amount of memory and requires +periodic CPU processing time for garbage collection. + +Note that this value configures the maximum number of V8 contexts that can be +used in parallel. Upon server start only as many V8 contexts will be created as +are configured in option `--javascript.v8-contexts-minimum`. The actual number of +available V8 contexts may float at runtime between `--javascript.v8-contexts-minimum` +and `--javascript.v8-contexts`. When there are unused V8 contexts that linger around, +the server's garbage collector thread will automatically delete them. + +`--javascript.v8-contexts-minimum number` + +Specifies the minimum *number* of V8 contexts that will be present at any time +the server is running. The actual number of V8 contexts will never drop below this +value, but it may go up as high as specified via the option `--javascript.v8-contexts`. + +When there are unused V8 contexts that linger around and the number of V8 contexts +is greater than `--javascript.v8-contexts-minimum` the server's garbage collector +thread will automatically delete them. + +`--javascript.v8-contexts-max-invocations` + +Specifies the maximum number of invocations after which a used V8 context is +disposed. The default value of `--javascript.v8-contexts-max-invocations` is 0, +meaning that the maximum number of invocations per context is unlimited. + +`--javascript.v8-contexts-max-age` + +Specifies the time duration (in seconds) after which time a V8 context is disposed +automatically after its creation. If the time is elapsed, the context will be disposed. +The default value for `--javascript.v8-contexts-max-age` is 60 seconds. + +If both `--javascript.v8-contexts-max-invocations` and `--javascript.v8-contexts-max-age` +are set, then the context will be destroyed when either of the specified threshold +values is reached. + +## Garbage collection frequency (time-based) + +`--javascript.gc-frequency frequency` + +Specifies the frequency (in seconds) for the automatic garbage collection of +JavaScript objects. This setting is useful to have the garbage collection still +work in periods with no or little numbers of requests. + +## Garbage collection interval (request-based) + +`--javascript.gc-interval interval` + +Specifies the interval (approximately in number of requests) that the garbage +collection for JavaScript objects will be run in each thread. + +## V8 options + +`--javascript.v8-options options` + +Optional arguments to pass to the V8 Javascript engine. The V8 engine will run +with default settings unless explicit options are specified using this +option. The options passed will be forwarded to the V8 engine which will parse +them on its own. Passing invalid options may result in an error being printed on +stderr and the option being ignored. + +Options need to be passed in one string, with V8 option names being prefixed +with double dashes. Multiple options need to be separated by whitespace. To get +a list of all available V8 options, you can use the value *"--help"* as follows: + +``` +--javascript.v8-options="--help" +``` + +Another example of specific V8 options being set at startup: + +``` +--javascript.v8-options="--log" +``` + +Names and features or usable options depend on the version of V8 being used, and +might change in the future if a different version of V8 is being used in +ArangoDB. Not all options offered by V8 might be sensible to use in the context +of ArangoDB. Use the specific options only if you are sure that they are not +harmful for the regular database operation. + +### Enable or Disable V8 JavaScript Engine entirely + +``` +--javascript.enabled bool +``` + +In certain types of ArangoDB instances you can now completely disable the V8 +JavaScript engine. Be aware that this is an **highly experimental** feature and +it is to be expected that certain functionality (e.g. some API endpoints, the +WebUI, some AQL functions etc) will be missing or severely broken. Nevertheless +you may whish to reduce the footprint of ArangoDB by disabling V8. + +This option is expected to **only** work reliably on a _single server_, +_agency_ or in an _active failover_ setup. Do not try to use this feature on a +_coordinator_, or _cluster database server_. diff --git a/Documentation/Books/Manual/Administration/Configuration/Ldap.md b/Documentation/Books/Manual/Programs/Arangod/Ldap.md similarity index 98% rename from Documentation/Books/Manual/Administration/Configuration/Ldap.md rename to Documentation/Books/Manual/Programs/Arangod/Ldap.md index 877442a8b4..18ab66e46b 100644 --- a/Documentation/Books/Manual/Administration/Configuration/Ldap.md +++ b/Documentation/Books/Manual/Programs/Arangod/Ldap.md @@ -1,13 +1,11 @@ -LDAP -==== +# ArangoDB Server LDAP Options {% hint 'info' %} This feature is only available in the [**Enterprise Edition**](https://www.arangodb.com/why-arangodb/arangodb-enterprise/) {% endhint %} -Basics Concepts ---------------- +## Basics Concepts The basic idea is that one can keep the user authentication setup for an ArangoDB instance (single or cluster) outside of ArangoDB in an LDAP @@ -189,8 +187,7 @@ Assuming you have the TLS CAcert file that is given to the server at You can use TLS with any of the following authentication mechanisms. -Authentication methods ----------------------- +## Authentication methods In order to authenticate users in LDAP we have two options available. We need to pick exactly one them. @@ -268,9 +265,7 @@ all `person` objects where the `uid` is equal to the given username. From these the `dn` will be extracted and used as `{{USER}}` in the roles later on. - -Fetching roles for a user -------------------------- +## Fetching roles for a user After authentication, the next step is to derive authorization information from the authenticated LDAP user. @@ -303,7 +298,7 @@ user is associated with from an attribute in the LDAP user object. If the configuration option --ldap.roles-attribute-name - + configuration option is set, then the value of that option is the name of the attribute being used. @@ -433,8 +428,7 @@ In combination with the `superuser-role` we could make all --ldap.superuser-role=project-a -Complete configuration examples -------------------------------- +## Complete configuration examples In this section we would like to present complete examples for a successful LDAP configuration of ArangoDB. diff --git a/Documentation/Books/Manual/Administration/Configuration/Logging.md b/Documentation/Books/Manual/Programs/Arangod/Log.md similarity index 98% rename from Documentation/Books/Manual/Administration/Configuration/Logging.md rename to Documentation/Books/Manual/Programs/Arangod/Log.md index abd61d1d3f..5f915f573e 100644 --- a/Documentation/Books/Manual/Administration/Configuration/Logging.md +++ b/Documentation/Books/Manual/Programs/Arangod/Log.md @@ -1,8 +1,6 @@ -Command-Line Options for Logging -================================ +# ArangoDB Server Log Options -Log levels and topics ---------------------- +## Log levels and topics ArangoDB's log output is grouped into topics. `--log.level` can be specified multiple times at startup, for as many topics as needed. The log verbosity and @@ -111,7 +109,6 @@ will be printed in UTC / Zulu time. The date and time format used in logs is always `YYYY-MM-DD HH:MM:SS`, regardless of this setting. If UTC time is used, a `Z` will be appended to indicate Zulu time. - ### Escaping `--log.escape value` @@ -135,7 +132,6 @@ is set to a very verbose level (e.g. debug or trace). The default value for this option is `true`. - ### Color logging `--log.color value` @@ -143,7 +139,6 @@ The default value for this option is `true`. Logging to terminal output is by default colored. Colorful logging can be turned off by setting the value to false. - ### Source file and Line number Log line number: `--log.line-number` diff --git a/Documentation/Books/Manual/Programs/Arangod/Nonce.md b/Documentation/Books/Manual/Programs/Arangod/Nonce.md new file mode 100644 index 0000000000..374f8ab0f4 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Nonce.md @@ -0,0 +1,7 @@ +# ArangoDB Server Nonce Options + +## Size + +`nonce.size` + +The size of the hash array for nonces. diff --git a/Documentation/Books/Manual/Programs/Arangod/Options.md b/Documentation/Books/Manual/Programs/Arangod/Options.md index 598a895028..ea230a4203 100644 --- a/Documentation/Books/Manual/Programs/Arangod/Options.md +++ b/Documentation/Books/Manual/Programs/Arangod/Options.md @@ -18,7 +18,7 @@ if the value contains whitespace. Extra whitespace around `=` is allowed: arangod --database.directory = "/path with spaces/to/datadir" -See [Configuration](../../Administration/Configuration/README.md#configuration-files) +See [Configuration](../../Administration/Configuration.md) if you want to translate startup parameters to configuration files. @startDocuBlock program_options_arangod diff --git a/Documentation/Books/Manual/Programs/Arangod/Query.md b/Documentation/Books/Manual/Programs/Arangod/Query.md new file mode 100644 index 0000000000..f5d4b09ee5 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Query.md @@ -0,0 +1,100 @@ +# ArangoDB Server Query Options + +## Limiting memory for AQL queries + +`--query.memory-limit value` + +The default maximum amount of memory (in bytes) that a single AQL query can use. +When a single AQL query reaches the specified limit value, the query will be +aborted with a *resource limit exceeded* exception. In a cluster, the memory +accounting is done per shard, so the limit value is effectively a memory limit per +query per shard. + +The global limit value can be overridden per query by setting the *memoryLimit* +option value for individual queries when running an AQL query. + +The default value is *0*, meaning that there is no memory limit. + +## Turning AQL warnings into errors + +`--query.fail-on-warning value` + +When set to *true*, AQL queries that produce warnings will instantly abort and +throw an exception. This option can be set to catch obvious issues with AQL +queries early. When set to *false*, AQL queries that produce warnings will not +abort and return the warnings along with the query results. +The option can also be overridden for each individual AQL query. + +## Enable/disable AQL query tracking + +`--query.tracking flag` + +If *true*, the server's AQL slow query tracking feature will be enabled by +default. Tracking of queries can be disabled by setting the option to *false*. + +The default is *true*. + +## Enable/disable tracking of bind variables in AQL queries + +`--query.tracking-with-bindvars flag` + +If *true*, then the bind variables will be tracked for all running and slow +AQL queries. This option only has an effect if `--query.tracking` was set to +*true*. Tracking of bind variables can be disabled by setting the option to *false*. + +The default is *true*. + +## Threshold for slow AQL queries + +`--query.slow-threshold value` + +By setting *value* it can be controlled after what execution time an AQL query +is considered "slow". Any slow queries that exceed the execution time specified +in *value* will be logged when they are finished. The threshold value is +specified in seconds. Tracking of slow queries can be turned off entirely by +setting the option `--query.tracking` to *false*. + +The default value is *10.0*. + +## Limiting the number of query execution plans created by the AQL optimizer + +`--query.optimizer-max-plans value` + +By setting *value* it can be controlled how many different query execution plans +the AQL query optimizer will generate at most for any given AQL query. Normally +the AQL query optimizer will generate a single execution plan per AQL query, but +there are some cases in which it creates multiple competing plans. More plans +can lead to better optimized queries, however, plan creation has its costs. The +more plans are created and shipped through the optimization pipeline, the more +time will be spent in the optimizer. +Lowering *value* will make the optimizer stop creating additional plans when it +has already created enough plans. +Note that this setting controls the default maximum number of plans to create. The +value can still be adjusted on a per-query basis by setting the *maxNumberOfPlans* +attribute when running a query. + +The default value is *128*. + +## AQL Query caching mode + +`--query.cache-mode` + +Toggles the AQL query cache behavior. Possible values are: + +* *off*: do not use query cache +* *on*: always use query cache, except for queries that have their *cache* + attribute set to *false* +* *demand*: use query cache only for queries that have their *cache* + attribute set to *true* + +## AQL Query cache size + +`--query.cache-entries` + +Maximum number of query results that can be stored per database-specific query +cache. If a query is eligible for caching and the number of items in the +database's query cache is equal to this threshold value, another cached query +result will be removed from the cache. + +This option only has an effect if the query cache mode is set to either *on* or +*demand*. diff --git a/Documentation/Books/Manual/Programs/Arangod/Replication.md b/Documentation/Books/Manual/Programs/Arangod/Replication.md new file mode 100644 index 0000000000..4ea870d292 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Replication.md @@ -0,0 +1,7 @@ +# ArangoDB Server Replication Options + +## Active failover + +`replication.active-failover` + +Enable active-failover during asynchronous replication. diff --git a/Documentation/Books/Manual/Administration/Configuration/RocksDB.md b/Documentation/Books/Manual/Programs/Arangod/Rocksdb.md similarity index 97% rename from Documentation/Books/Manual/Administration/Configuration/RocksDB.md rename to Documentation/Books/Manual/Programs/Arangod/Rocksdb.md index a0c8fb0c89..bf53272e39 100644 --- a/Documentation/Books/Manual/Administration/Configuration/RocksDB.md +++ b/Documentation/Books/Manual/Programs/Arangod/Rocksdb.md @@ -1,22 +1,22 @@ -RocksDB engine options -====================== +# ArangoDB Server RocksDB Options RocksDB is a highly configurable key-value store used to power our RocksDB storage engine. Most of the options on this page are pass-through options to the underlying RocksDB instance, and we change very few of their default settings. -Depending [on the storage engine you have chosen](GeneralArangod.md#storage-engine) the availability -and the scope of these options changes. +Depending on the [storage engine you have chosen](Server.md#storage-engine) +the availability and the scope of these options changes. -In case you have chosen `mmfiles` some of the following options apply to persistent indexes. +In case you have chosen `mmfiles` some of the following options apply to +persistent indexes. In case of `rocksdb` it will apply to all data stored as well as indexes. ## Pass-through options `--rocksdb.wal-directory` -Absolute path for the RocksDB WAL files. If left empty, this will use a subdirectory -`journals` inside the data directory. +Absolute path for the RocksDB WAL files. If left empty, this will use a +subdirectory `journals` inside the data directory. ### Write buffers @@ -204,11 +204,10 @@ By decreasing this option's value, the server will start the removal of obsolete WAL files earlier after server start. This is useful in testing environments that are space-restricted and do not require keeping much WAL file data at all. - `--rocksdb.max-transaction-size` Transaction size limit (in bytes). Transactions store all keys and values in -RAM, so large transactions run the risk of causing out-of-memory sitations. +RAM, so large transactions run the risk of causing out-of-memory situations. This setting allows you to ensure that does not happen by limiting the size of any individual transaction. Transactions whose operations would consume more RAM than this threshold value will abort automatically with error 32 ("resource diff --git a/Documentation/Books/Manual/Programs/Arangod/Server.md b/Documentation/Books/Manual/Programs/Arangod/Server.md new file mode 100644 index 0000000000..0292304297 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Server.md @@ -0,0 +1,270 @@ +# ArangoDB Server _Server_ Options + +## Managing Endpoints + +The ArangoDB server can listen for incoming requests on multiple *endpoints*. + +The endpoints are normally specified either in ArangoDB's configuration file or +on the command-line, using the `--server.endpoint`. ArangoDB supports different +types of endpoints: + +- tcp://ipv4-address:port - TCP/IP endpoint, using IPv4 +- tcp://[ipv6-address]:port - TCP/IP endpoint, using IPv6 +- ssl://ipv4-address:port - TCP/IP endpoint, using IPv4, SSL encryption +- ssl://[ipv6-address]:port - TCP/IP endpoint, using IPv6, SSL encryption +- unix:///path/to/socket - Unix domain socket endpoint + +If a TCP/IP endpoint is specified without a port number, then the default port +(8529) will be used. If multiple endpoints need to be used, the option can be +repeated multiple times. + +The default endpoint for ArangoDB is *tcp://127.0.0.1:8529* or +*tcp://localhost:8529*. + +**Examples** + +``` +unix> ./arangod --server.endpoint tcp://127.0.0.1:8529 + --server.endpoint ssl://127.0.0.1:8530 + --ssl.keyfile server.pem /tmp/vocbase +2012-07-26T07:07:47Z [8161] INFO using SSL protocol version 'TLSv1' +2012-07-26T07:07:48Z [8161] INFO using endpoint 'ssl://127.0.0.1:8530' for http ssl requests +2012-07-26T07:07:48Z [8161] INFO using endpoint 'tcp://127.0.0.1:8529' for http tcp requests +2012-07-26T07:07:49Z [8161] INFO ArangoDB (version 1.1.alpha) is ready for business +2012-07-26T07:07:49Z [8161] INFO Have Fun! +``` + +Given a hostname: + +`--server.endpoint tcp://hostname:port` + +Given an IPv4 address: + +`--server.endpoint tcp://ipv4-address:port` + +Given an IPv6 address: + +`--server.endpoint tcp://[ipv6-address]:port` + +On one specific ethernet interface each port can only be bound **exactly +once**. You can look up your available interfaces using the *ifconfig* command +on Linux / MacOSX - the Windows equivalent is *ipconfig* ([See Wikipedia for +more details](http://en.wikipedia.org/wiki/Ifconfig)). The general names of the +interfaces differ on OS's and hardwares they run on. However, typically every +host has a so called +[loopback interface](http://en.wikipedia.org/wiki/Loop_device), +which is a virtual interface. By convention it always has the address +*127.0.0.1* or *::1* (ipv6), and can only be reached from exactly the very same +host. Ethernet interfaces usually have names like *eth0*, *wlan0*, *eth1:17*, +*le0* or a plain text name in Windows. + +To find out which services already use ports (so ArangoDB can't bind them +anymore), you can use the +[netstat command](http://en.wikipedia.org/wiki/Netstat) +(it behaves a little different on each platform, run it with *-lnpt* on Linux, +*-p tcp* on MacOSX or with *-an* on windows for valuable information). + +ArangoDB can also do a so called *broadcast bind* using +*tcp://0.0.0.0:8529*. This way it will be reachable on all interfaces of the +host. This may be useful on development systems that frequently change their +network setup like laptops. + +### Special note on IPv6 link-local addresses + +ArangoDB can also listen to IPv6 link-local addresses via adding the zone ID +to the IPv6 address in the form `[ipv6-link-local-address%zone-id]`. However, +what you probably instead want is to bind to a local IPv6 address. Local IPv6 +addresses start with `fd`. If you only see a `fe80:` IPv6 address in your +interface configuration but no IPv6 address starting with `fd` your interface +has no local IPv6 address assigned. You can read more about IPv6 link-local +addresses [here](https://en.wikipedia.org/wiki/Link-local_address#IPv6). + +**Example** + +Bind to a link-local and local IPv6 address. + + unix> ifconfig + +This command lists all interfaces and assigned ip addresses. The link-local +address may be `fe80::6257:18ff:fe82:3ec6%eth0` (IPv6 address plus interface name). +A local IPv6 address may be `fd12:3456::789a`. To bind ArangoDB to it start +*arangod* with `--server.endpoint tcp://[fe80::6257:18ff:fe82:3ec6%eth0]:8529`. +Use telnet to test the connection. + + unix> telnet fe80::6257:18ff:fe82:3ec6%eth0 8529 + Trying fe80::6257:18ff:fe82:3ec6... + Connected to my-machine. + Escape character is '^]'. + GET / HTTP/1.1 + + HTTP/1.1 301 Moved Permanently + Location: /_db/_system/_admin/aardvark/index.html + Content-Type: text/html + Server: ArangoDB + Connection: Keep-Alive + Content-Length: 197 + + Moved

Moved

This page has moved to /_db/_system/_admin/aardvark/index.html.

+ +### Reuse address + +`--tcp.reuse-address` + +If this boolean option is set to *true* then the socket option SO_REUSEADDR is +set on all server endpoints, which is the default. If this option is set to +*false* it is possible that it takes up to a minute after a server has +terminated until it is possible for a new server to use the same endpoint +again. This is why this is activated by default. + +Please note however that under some operating systems this can be a security +risk because it might be possible for another process to bind to the same +address and port, possibly hijacking network traffic. Under Windows, ArangoDB +additionally sets the flag SO_EXCLUSIVEADDRUSE as a measure to alleviate this +problem. + +### Backlog size + +`--tcp.backlog-size` + +Allows to specify the size of the backlog for the *listen* system call The +default value is 10. The maximum value is platform-dependent. Specifying a +higher value than defined in the system header's SOMAXCONN may result in a +warning on server start. The actual value used by *listen* may also be silently +truncated on some platforms (this happens inside the *listen* system call). + +## Maximal queue size + +Maximum size of the queue for requests: `--server.maximal-queue-size +size` + +Specifies the maximum *size* of the queue for asynchronous task +execution. If the queue already contains *size* tasks, new tasks will +be rejected until other tasks are popped from the queue. Setting this +value may help preventing from running out of memory if the queue is +filled up faster than the server can process requests. + +## Storage engine + +ArangoDB's "traditional" storage engine is called `MMFiles`, which also was the +default storage engine up to including ArangoDB 3.3. + +Since ArangoDB 3.2, an alternative engine based on [RocksDB](http://rocksdb.org) +is also provided and could be turned on manually. Since ArangoDB 3.4, the RocksDB +storage engine is the default storage engine for new installations. + +One storage engine type is supported per server per installation. +Live switching of storage engines on already installed systems isn't supported. +Configuring the wrong engine (not matching the previously used one) will result +in the server refusing to start. You may however use `auto` to let ArangoDB choose +the previously used one. + +`--server.storage-engine [auto|mmfiles|rocksdb]` + +Note that `auto` will default to `rocksdb` starting with ArangoDB 3.4, but in +previous versions it defaulted to `mmfiles`. + +## Check max memory mappings + +`--server.check-max-memory-mappings` can be used on Linux to make arangod +check the number of memory mappings currently used by the process (as reported in +`/proc//maps`) and compare it with the maximum number of allowed mappings as +determined by */proc/sys/vm/max_map_count*. If the current number of memory +mappings gets near the maximum allowed value, arangod will log a warning +and disallow the creation of further V8 contexts temporarily until the current +number of mappings goes down again. + +If the option is set to false, no such checks will be performed. All non-Linux +operating systems do not provide this option and will ignore it. + +## Enable/disable authentication + +@startDocuBlock server_authentication + +## JWT Secret + +`--server.jwt-secret secret` + +ArangoDB will use JWTs to authenticate requests. Using this option lets +you specify a JWT. + +In single server setups and when not specifying this secret ArangoDB will +generate a secret. + +In cluster deployments which have authentication enabled a secret must +be set consistently across all cluster tasks so they can talk to each other. + +## Enable/disable authentication for UNIX domain sockets + +`--server.authentication-unix-sockets value` + +Setting *value* to true will turn off authentication on the server side +for requests coming in via UNIX domain sockets. With this flag enabled, +clients located on the same host as the ArangoDB server can use UNIX domain +sockets to connect to the server without authentication. +Requests coming in by other means (e.g. TCP/IP) are not affected by this option. + +The default value is *false*. + +**Note**: this option is only available on platforms that support UNIX +domain sockets. + +## Enable/disable authentication for system API requests only + +@startDocuBlock serverAuthenticateSystemOnly + +## Enable authentication cache timeout + +`--server.authentication-timeout value` + +Sets the cache timeout to *value* (in seconds). This is only necessary +if you use an external authentication system like LDAP. + +## Enable local authentication + +`--server.local-authentication value` + +If set to *false* only use the external authentication system. If +*true* also use the local *_users* collections. + +The default value is *true*. + +## Server threads + +`--server.minimal-threads number` + +`--server.maximal-threads number` + +Specifies the *number* of threads that are spawned to handle requests. + +The actual number of request processing threads is adjusted dynamically at runtime +and will float between `--server.minimal-threads` and `--server.maximal-threads`. + +`--server.minimal-threads` determines the minimum number of request processing +threads the server will start and that will always be kept around. The default +value is *2*. + +`--server.maximal-threads` determines the maximum number of request processing +threads the server is allowed to start for request handling. If that number of +threads is already running, arangod will not start further threads for request +handling. The default value is + +## Toggling server statistics + +`--server.statistics value` + +If this option is *value* is *false*, then ArangoDB's statistics gathering +is turned off. Statistics gathering causes regular CPU activity so using this +option to turn it off might relieve heavy-loaded instances a bit. + +## Data source flush synchronization + +`--server.flush-interval` + +ArangoDB will periodically ensure that all data sources (databases, views, etc.) +have flushed all committed data to disk and write some checkpoint data to aid in +future recovery. Increasing this value will result in fewer, larger write +batches, while decreasing it will result in more, smaller writes. Setting the +value too low can easily overwhelm the server, while setting the value too high +may result in high memory usage and periodic slowdowns. Value is given in +microseconds, with a typical range of 100000 (100ms) to 10000000 (10s) and a +default of 1000000 (1s). Use caution when changing from the default. diff --git a/Documentation/Books/Manual/Administration/Configuration/SSL.md b/Documentation/Books/Manual/Programs/Arangod/Ssl.md similarity index 97% rename from Documentation/Books/Manual/Administration/Configuration/SSL.md rename to Documentation/Books/Manual/Programs/Arangod/Ssl.md index 1c731d96d1..9570cf5f96 100644 --- a/Documentation/Books/Manual/Administration/Configuration/SSL.md +++ b/Documentation/Books/Manual/Programs/Arangod/Ssl.md @@ -1,8 +1,6 @@ -SSL Configuration -================= +# ArangoDB Server SSL Options -SSL Endpoints -------------- +## SSL Endpoints Given a hostname: @@ -27,7 +25,7 @@ If SSL encryption is used, this option must be used to specify the filename of the server private key. The file must be PEM formatted and contain both the certificate and the server's private key. -The file specified by *filename* can be generated using openssl: +The file specified by *filename* can be generated using OpenSSL: ``` # create private key in file "server.key" @@ -109,7 +107,6 @@ Set to true if SSL session caching should be used. *value* has a default value of *false* (i.e. no caching). - ### SSL peer certificate {% hint 'info' %} diff --git a/Documentation/Books/Manual/Programs/Arangod/Temp.md b/Documentation/Books/Manual/Programs/Arangod/Temp.md new file mode 100644 index 0000000000..4d0c832ba1 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Temp.md @@ -0,0 +1,7 @@ +# ArangoDB Server Temp Options + +## Path + +`temp.path` + +Path for temporary files. diff --git a/Documentation/Books/Manual/Programs/Arangod/Vst.md b/Documentation/Books/Manual/Programs/Arangod/Vst.md new file mode 100644 index 0000000000..5e60814730 --- /dev/null +++ b/Documentation/Books/Manual/Programs/Arangod/Vst.md @@ -0,0 +1,9 @@ +# ArangoDB Server VST Options + +VST is an acronym for [VelocyStream](https://github.com/arangodb/velocystream). + +## Maxsize + +`vst.maxsize` + +Maximal size (in bytes) for a VelocyPack chunk. diff --git a/Documentation/Books/Manual/Administration/Configuration/Wal.md b/Documentation/Books/Manual/Programs/Arangod/Wal.md similarity index 90% rename from Documentation/Books/Manual/Administration/Configuration/Wal.md rename to Documentation/Books/Manual/Programs/Arangod/Wal.md index cc09aa3208..c3a08a0e30 100644 --- a/Documentation/Books/Manual/Administration/Configuration/Wal.md +++ b/Documentation/Books/Manual/Programs/Arangod/Wal.md @@ -1,11 +1,6 @@ -MMFiles Write-ahead log options -=============================== +# ArangoDB Server WAL Options (MMFiles) -Since ArangoDB 2.2, the MMFiles storage engine will write all data-modification -operations into its write-ahead log. - -With ArangoDB 3.2 another storage engine option becomes available - [RocksDB](RocksDB.md). -In case of using RocksDB the subsequent options don't have a useful meaning. +WAL is an acronym for write-ahead log. The write-ahead log is a sequence of logfiles that are written in an append-only fashion. Full logfiles will eventually be garbage-collected, and the relevant data @@ -13,8 +8,14 @@ might be transferred into collection journals and datafiles. Unneeded and alread garbage-collected logfiles will either be deleted or kept for the purpose of keeping a replication backlog. -Directory ---------- +Since ArangoDB 2.2, the MMFiles storage engine will write all data-modification +operations into its write-ahead log. + +With ArangoDB 3.2 another storage engine option becomes available: +[RocksDB](../../Architecture/StorageEngines.md#rocksdb). +In case of using RocksDB the subsequent options don't have a useful meaning. + +## Directory @@ -25,44 +26,43 @@ stored. If this option is not specified, it defaults to the subdirectory *journals* in the server's global database directory. If the directory is not present, it will be created. -Logfile size ------------- +## Logfile size + @startDocuBlock WalLogfileSize -Allow oversize entries ----------------------- +## Allow oversize entries + @startDocuBlock WalLogfileAllowOversizeEntries -Number of reserve logfiles --------------------------- +## Number of reserve logfiles + @startDocuBlock WalLogfileReserveLogfiles -Number of historic logfiles ---------------------------- +## Number of historic logfiles + @startDocuBlock WalLogfileHistoricLogfiles -Sync interval -------------- +## Sync interval + @startDocuBlock WalLogfileSyncInterval -Flush timeout -------------- +## Flush timeout + @startDocuBlock WalLogfileFlushTimeout -Throttling ----------- +## Throttling @@ -96,8 +96,7 @@ over, the operation will be aborted with an error. This option only has an effect if `--wal.throttle-when-pending` has a non-zero value, which is not the default. -Number of slots ---------------- +## Number of slots @@ -109,12 +108,11 @@ operations in parallel. Any write operation will lease a slot and return it to the write-ahead log when it is finished writing the data. A slot will remain blocked until the data in it was synchronized to disk. After that, a slot becomes reusable by following operations. The required number of -slots is thus determined by the parallelity of write operations and the +slots is thus determined by the parallelism of write operations and the disk synchronization speed. Slow disks probably need higher values, and fast disks may only require a value lower than the default. -Ignore logfile errors ---------------------- +## Ignore logfile errors @@ -134,8 +132,7 @@ procedure even in case it detects corrupt logfile entries. In this case it will stop at the first corrupted logfile entry and ignore all others, which might cause data loss. -Ignore recovery errors ----------------------- +## Ignore recovery errors @@ -146,8 +143,7 @@ Ignores any recovery errors not caused by corrupted logfiles but by logical errors. Logical errors can occur if logfiles or any other server datafiles have been manually edited or the server is somehow misconfigured. -Ignore (non-WAL) datafile errors --------------------------------- +## Ignore (non-WAL) datafile errors @@ -177,7 +173,7 @@ of the original data and potential follow up errors. The WAL recovery will still abort when encountering a collection with a corrupted datafile, at least if `--wal.ignore-recovery-errors` is not set to `true`. -Setting the option to `true` will also automaticall repair potentially +Setting the option to `true` will also automatically repair potentially corrupted VERSION files of databases on startup, so that the startup can proceed. diff --git a/Documentation/Books/Manual/ReleaseNotes/NewFeatures22.md b/Documentation/Books/Manual/ReleaseNotes/NewFeatures22.md index c2a51b08c9..5a114bcc23 100644 --- a/Documentation/Books/Manual/ReleaseNotes/NewFeatures22.md +++ b/Documentation/Books/Manual/ReleaseNotes/NewFeatures22.md @@ -97,7 +97,8 @@ from the master's write-ahead log to get informed about most recent data changes This removes the need to store data-modification operations in the *_replication* collection altogether. -For the configuration of the write-ahead log, please refer to [Write-ahead log options](../Administration/Configuration/Wal.md). +For the configuration of the write-ahead log, please refer to +[Write-ahead log options](../Programs/Arangod/Wal.md). The introduction of the write-ahead log also removes the need to configure and start the replication logger on a master. Though the replication logger object diff --git a/Documentation/Books/Manual/ReleaseNotes/NewFeatures32.md b/Documentation/Books/Manual/ReleaseNotes/NewFeatures32.md index 9b23392675..99a9f4096c 100644 --- a/Documentation/Books/Manual/ReleaseNotes/NewFeatures32.md +++ b/Documentation/Books/Manual/ReleaseNotes/NewFeatures32.md @@ -349,7 +349,7 @@ Client tools Authentication -------------- -* added [LDAP](../Administration/Configuration/Ldap.md) authentication (Enterprise only) +* added [LDAP](../Programs/Arangod/Ldap.md) authentication (Enterprise only) Authorization diff --git a/Documentation/Books/Manual/ReleaseNotes/NewFeatures33.md b/Documentation/Books/Manual/ReleaseNotes/NewFeatures33.md index 15316a4265..7022e15830 100644 --- a/Documentation/Books/Manual/ReleaseNotes/NewFeatures33.md +++ b/Documentation/Books/Manual/ReleaseNotes/NewFeatures33.md @@ -227,7 +227,7 @@ The following options have been added to it: - `--ldap.roles-exclude` - `--ldap.superuser-role` - Please refer to [LDAP](../Administration/Configuration/Ldap.md) for a detailed + Please refer to [LDAP](../Programs/Arangod/Ldap.md) for a detailed explanation. diff --git a/Documentation/Books/Manual/SUMMARY.md b/Documentation/Books/Manual/SUMMARY.md index 94c0b8cd6c..7ab418ccb5 100644 --- a/Documentation/Books/Manual/SUMMARY.md +++ b/Documentation/Books/Manual/SUMMARY.md @@ -25,6 +25,29 @@ * [Programs & Tools](Programs/README.md) * [ArangoDB Server](Programs/Arangod/README.md) * [Options](Programs/Arangod/Options.md) + * [Global](Programs/Arangod/Global.md) + * [Agency](Programs/Arangod/Agency.md) + * [ArangoSearch](Programs/Arangod/Arangosearch.md) + * [Audit](Programs/Arangod/Audit.md) + * [Cache](Programs/Arangod/Cache.md) + * [Cluster](Programs/Arangod/Cluster.md) + * [Compaction](Programs/Arangod/Compaction.md) + * [Database](Programs/Arangod/Database.md) + * [Foxx](Programs/Arangod/Foxx.md) + * [Frontend](Programs/Arangod/Frontend.md) + * [HTTP](Programs/Arangod/Http.md) + * [JavaScript](Programs/Arangod/Javascript.md) + * [LDAP](Programs/Arangod/Ldap.md) + * [Log](Programs/Arangod/Log.md) + * [Nonce](Programs/Arangod/Nonce.md) + * [Query](Programs/Arangod/Query.md) + * [Replication](Programs/Arangod/Replication.md) + * [RocksDB](Programs/Arangod/Rocksdb.md) + * [Server](Programs/Arangod/Server.md) + * [SSL](Programs/Arangod/Ssl.md) + * [Temp](Programs/Arangod/Temp.md) + * [VST](Programs/Arangod/Vst.md) + * [WAL](Programs/Arangod/Wal.md) * [Web Interface](Programs/WebInterface/README.md) * [Dashboard](Programs/WebInterface/Dashboard.md) * [Cluster](Programs/WebInterface/Cluster.md) @@ -214,6 +237,7 @@ * [Installation](Installation/README.md) * [Linux](Installation/Linux.md) + * [Operating System Configuration](Installation/LinuxOSConfiguration.md) * [Mac OS X](Installation/MacOSX.md) * [Windows](Installation/Windows.md) * [Compiling](Installation/Compiling.md) @@ -290,24 +314,12 @@ * [Downgrading](Downgrading/README.md) * [Scaling](Scaling/README.md) * [Administration](Administration/README.md) + * [Configuration](Administration/Configuration.md) * [Backup & Restore](Administration/BackupRestore.md) * [Import & Export](Administration/ImportExport.md) * [User Management](Administration/ManagingUsers/README.md) * [In Arangosh](Administration/ManagingUsers/InArangosh.md) - * [Server Configuration](Administration/Configuration/README.md) - * [Operating System Configuration](Administration/Configuration/OperatingSystem.md) - * [Managing Endpoints](Administration/Configuration/Endpoint.md) - * [SSL Configuration](Administration/Configuration/SSL.md) - * [LDAP Options](Administration/Configuration/Ldap.md) - * [Logging Options](Administration/Configuration/Logging.md) - * [General Options](Administration/Configuration/GeneralArangod.md) - * [Write-Ahead Log Options](Administration/Configuration/Wal.md) - * [Compaction Options](Administration/Configuration/Compaction.md) - * [Cluster Options](Administration/Configuration/Cluster.md) - * [RocksDB Engine Options](Administration/Configuration/RocksDB.md) - * [Hash Cache Options](Administration/Configuration/Cache.md) - * [Asynchronous Tasks](Administration/Configuration/Asynchronous.md) - * [Durability](Administration/Configuration/Durability.md) +# * [Durability](Administration/Configuration/Durability.md) * [Master/Slave](Administration/MasterSlave/README.md) * [Setting up](Administration/MasterSlave/SettingUp.md) * [Replication Applier](Administration/MasterSlave/ReplicationApplier.md) diff --git a/Documentation/Books/Manual/Upgrading/Manually/Cluster.md b/Documentation/Books/Manual/Upgrading/Manually/Cluster.md index f17192e8a9..7829a9f55e 100644 --- a/Documentation/Books/Manual/Upgrading/Manually/Cluster.md +++ b/Documentation/Books/Manual/Upgrading/Manually/Cluster.md @@ -15,7 +15,7 @@ Preparations The ArangoDB installation packages (e.g. for Debian or Ubuntu) set up a convenient standalone instance of `arangod`. During installation, this instance's database will be upgraded (see [`--database.auto-upgrade` in General -Options](../../Administration/Configuration/GeneralArangod.md#database-upgrade)) +Options](../../Programs/Arangod/Database.md#auto-upgrade)) and the service will be (re)started. You have to make sure that your _cluster_ deployment is independent of this diff --git a/Documentation/Books/Manual/Administration/Configuration/Durability.md b/Documentation/Books/stash/Durability.md similarity index 100% rename from Documentation/Books/Manual/Administration/Configuration/Durability.md rename to Documentation/Books/stash/Durability.md diff --git a/Documentation/Scripts/generateMdFiles.py b/Documentation/Scripts/generateMdFiles.py index 3fc5e72884..d7a612c3e6 100644 --- a/Documentation/Scripts/generateMdFiles.py +++ b/Documentation/Scripts/generateMdFiles.py @@ -733,7 +733,9 @@ def loadProgramOptionBlocks(): # Output table header with column labels (one table per section) output.append('\n

{} Options

'.format(groupName.title())) - output.append('') + if program in ['arangod']: + output.append('\nAlso see {0} details.'.format(groupName.title())) + output.append('\n
') output.append(''.format('Name', 'Type', 'Description')) output.append('')
{}{}{}