Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'devel' of https://github.com/arangodb/arangodb into devel

This commit is contained in:
Kaveh Vahedipour 2016-06-09 12:01:50 +02:00
parent e6ec1864c5 f5e7187d8b
commit a88a93b450
11 changed files with 204 additions and 277 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

77
Documentation/Books/Manual/Administration/Configuration/Arangod.mdpp
View File

@ -8,7 +8,7 @@
try to reuse address
`--server.reuse-address`
`--tcp.reuse-address`
If this boolean option is set to *true* then the socket option
SO_REUSEADDR is set on all server endpoints, which is the default.
@ -25,15 +25,15 @@ SO_EXCLUSIVEADDRUSE as a measure to alleviate this problem.
!SUBSECTION Disable authentication
!SUBSECTION Enable/disable authentication
@startDocuBlock server_authentication
!SUBSECTION Disable authentication-unix-sockets
!SUBSECTION Enable/disable authentication for UNIX domain sockets
disable authentication for requests via UNIX domain sockets
`--server.disable-authentication-unix-sockets value`
`--server.authentication-unix-sockets value`
Setting *value* to true will turn off authentication on the server side
for requests coming in via UNIX domain sockets. With this flag enabled,
@ -44,33 +44,31 @@ Requests coming in by other means (e.g. TCP/IP) are not affected by this option.
The default value is *false*.
**Note**: this option is only available on platforms that support UNIX
domain
sockets.
domain sockets.
!SUBSECTION Authenticate system only
!SUBSECTION Enable/disable authentication for system API requests only
@startDocuBlock serverAuthenticateSystemOnly
!SUBSECTION Disable replication-applier
!SUBSECTION Enable/disable replication applier
disable the replication applier on server startup
`--server.disable-replication-applier flag`
Enable/disable the replication applier on server startup
`--database.replication-applier flag`
If *true* the server will start with the replication applier turned off,
even if the replication applier is configured with the *autoStart* option.
If *false* the server will start with replication appliers turned off,
even if the replication appliers are configured with the *autoStart* option.
Using the command-line option will not change the value of the *autoStart*
option in the applier configuration, but will suppress auto-starting the
replication applier just once.
If the option is not used, ArangoDB will read the applier configuration
from
the file *REPLICATION-APPLIER-CONFIG* on startup, and use the value of the
from the file *REPLICATION-APPLIER-CONFIG* on startup, and use the value of the
*autoStart* attribute from this file.
The default is *false*.
The default is *true*.
@ -82,7 +80,7 @@ The default is *false*.
hide the "Server: ArangoDB" header in HTTP responses
`--server.hide-product-header`
`--http.hide-product-header`
If *true*, the server will exclude the HTTP header "Server: ArangoDB" in
HTTP responses. If set to *false*, the server will send the header in
@ -96,7 +94,7 @@ The default is *false*.
allow HTTP method override via custom headers?
`--server.allow-method-override`
`--http.allow-method-override`
When this option is set to *true*, the HTTP request method will optionally
be fetched from one of the following HTTP request headers if present in
@ -193,7 +191,7 @@ started with at least one SSL endpoint.
CA file
`--server.cafile filename`
`--ssl.cafile filename`
This option can be used to specify a file with CA certificates that are
sent
@ -235,7 +233,7 @@ to POODLE attack variants.
whether or not to use SSL session caching
`--server.ssl-cache value`
`--ssl.session-cache value`
Set to true if SSL session caching should be used.
@ -250,7 +248,7 @@ only if the client supports sending the session id.
ssl options to use
`--server.ssl-options value`
`--ssl.options value`
This option can be used to set various SSL-related options. Individual
option values must be combined using bitwise OR.
@ -284,7 +282,7 @@ used.
ssl cipher list to use
`--server.ssl-cipher-list cipher-list`
`--ssl.cipher-list cipher-list`
This option can be used to restrict the server to certain SSL ciphers
only,
@ -317,7 +315,7 @@ The default value for *cipher-list* is "ALL".
listen backlog size
`--server.backlog-size`
`--tcp.backlog-size`
Allows to specify the size of the backlog for the *listen* system call
The default value is 10. The maximum value is platform-dependent.
@ -402,16 +400,31 @@ directory as argument.
@startDocuBlock databaseForceSyncProperties
!SUBSECTION Disable AQL query tracking
!SUBSECTION Enable/disable AQL query tracking
disable the query tracking feature
`--database.disable-query-tracking flag`
`--query.tracking flag`
If *true*, the server's query tracking feature will be disabled by
default.
If *true*, the server's AQL slow query tracking feature will be enabled by
default. Tracking of queries can be disabled by setting the option to *false*.
The default is *false*.
The default is *true*.
!SUBSECTION Threshold for slow AQL queries
adjust the threshold for slow AQL queries
`--query.slow-threshold value`
By setting *value* it can be controlled after what execution time an AQL query
is considered "slow". Any slow queries that exceed the execution time specified
in *value* will be logged when they are finished. The threshold value is
specified in seconds. Tracking of slow queries can be turned off entirely by
setting the option `--query.tracking` to *false*.
The default value is *10.0*.
@ -447,8 +460,8 @@ The default value is *false*.
!SUBSECTION AQL Query caching mode
whether or not to enable the AQL query cache
`--database.query-cache-mode`
whether or not to enable the AQL query result cache
`--query.cache-mode`
Toggles the AQL query cache behavior. Possible values are:
@ -465,7 +478,7 @@ Toggles the AQL query cache behavior. Possible values are:
maximum number of elements in the query cache per database
`--database.query-cache-max-results`
`--query.cache-entries`
Maximum number of query results that can be stored per database-specific
query cache. If a query is eligible for caching and the number of items in
@ -499,7 +512,7 @@ edge index in the collection to be split into multiple buckets).
number of V8 contexts for executing JavaScript actions
`--server.v8-contexts number`
`--javascript.v8-contexts number`
Specifies the *number* of V8 contexts that are created for executing
JavaScript code. More contexts allow execute more JavaScript actions in
@ -555,7 +568,7 @@ the value *"--help"* as follows:
Another example of specific V8 options being set at startup:
```
--javascript.v8-options "--harmony --log"
--javascript.v8-options "--log"
```
Names and features or usable options depend on the version of V8 being used,

15
Documentation/Books/Manual/Administration/Configuration/Cluster.mdpp
View File

@ -123,17 +123,14 @@ username used for cluster-internal communication
The username used for authorization of cluster-internal requests.
This username will be used to authenticate all requests and responses in
cluster-internal communication, i.e. requests exchanged between
coordinators
and individual database servers.
coordinators and individual database servers.
This option is used for cluster-internal requests only. Regular requests
to
coordinators are authenticated normally using the data in the *_users*
to coordinators are authenticated normally using the data in the *_users*
collection.
If coordinators and database servers are run with authentication turned
off,
(e.g. by setting the *--server.disable-authentication* option to *true*),
off, (e.g. by setting the *--server.authentication* option to *false*),
the cluster-internal communication will also be unauthenticated.
@ -147,8 +144,7 @@ password used for cluster-internal communication
The password used for authorization of cluster-internal requests.
This password will be used to authenticate all requests and responses in
cluster-internal communication, i.e. requests exchanged between
coordinators
and individual database servers.
coordinators and individual database servers.
This option is used for cluster-internal requests only. Regular requests
to
@ -156,7 +152,6 @@ coordinators are authenticated normally using the data in the `_users`
collection.
If coordinators and database servers are run with authentication turned
off,
(e.g. by setting the *--server.disable-authentication* option to *true*),
off, (e.g. by setting the *--server.authentication* option to *false*),
the cluster-internal communication will also be unauthenticated.

7
Documentation/Books/Manual/Administration/Configuration/Communication.mdpp
View File

@ -15,14 +15,13 @@ scheduler. The default is 1.
maximum size of the dispatcher queue for asynchronous requests
`--scheduler.maximal-queue-size size`
`--server.maximal-queue-size size`
Specifies the maximum *size* of the dispatcher queue for asynchronous
task execution. If the queue already contains *size* tasks, new tasks
will be rejected until other tasks are popped from the queue. Setting this
value may help preventing from running out of memory if the queue is
filled
up faster than the server can process requests.
filled up faster than the server can process requests.
@ -40,7 +39,7 @@ and epoll.
!SUBSECTION Io backends
`--show-io-backends`
`--scheduler.show-backends`
If this option is specified, then the server will list available backends and
exit. This option is useful only when used in conjunction with the option

292
Documentation/Books/Manual/Administration/Configuration/Logging.mdpp
View File

@ -1,162 +1,95 @@
!CHAPTER Command-Line Options for Logging
There are two different kinds of logs. Human-readable logs and machine-readable
logs. The human-readable logs are used to provide an administration with
information about the server. The machine-readable logs are used to provide
statistics about executed requests and timings about computation steps.
!SUBSECTION Log levels and topics
!SECTION General Logging Options
!SUBSECTION Logfile
`--log.file filename`
This option allows the user to specify the name of a file to which information
is logged. By default, if no log file is specified, the standard output is
used. Note that if the file named by *filename* does not exist, it will be
created. If the file cannot be created (e.g. due to missing file privileges),
the server will refuse to start. If the specified file already exists, output is
appended to that file.
Use *+* to log to standard error. Use *-* to log to standard output.
Use *""* to disable logging to file.
`--log.tty filename`
Be default, if started on a tty, the log output will also go to the ttyp.
Use *""* to disable.
!SUBSECTION Request
<!-- lib/ApplicationServer/ApplicationServer.h -->
log file for requests
`--log.requests-file filename`
This option allows the user to specify the name of a file to which
requests are logged. By default, no log file is used and requests are
not logged. Note that if the file named by *filename* does not
exist, it will be created. If the file cannot be created (e.g. due to
missing file privileges), the server will refuse to start. If the
specified
file already exists, output is appended to that file.
Use *+* to log to standard error. Use *-* to log to standard output.
Use *""* to disable request logging altogether.
The log format is
- `"http-request"`: static string indicating that an HTTP request was
logged
- client address: IP address of client
- HTTP method type, e.g. `GET`, `POST`
- HTTP version, e.g. `HTTP/1.1`
- HTTP response code, e.g. 200
- request body length in bytes
- response body length in bytes
- server request processing time, containing the time span between
fetching
the first byte of the HTTP request and the start of the HTTP response
!SECTION Human Readable Logging
!SUBSECTION Logfiles
<!-- lib/ApplicationServer/ApplicationServer.h -->
log file
!SUBSECTION Level
`--log.level level`
`--log level`
Allows the user to choose the level of information which is logged by
the server. The argument *level* is specified as a string and can be
one of the values listed below. Note that, fatal errors, that is,
errors which cause the server to terminate, are always logged
irrespective of the log level assigned by the user. The first variant
log.level can be used in configuration files, the second variant log
for command line options.
**fatal**:
Logs errors which cause the server to terminate.
Fatal errors generally indicate some inconsistency with the manner in
which the server has been coded. Fatal errors may also indicate a
problem with the platform on which the server is running. Fatal errors
always cause the server to terminate. For example,
ArangoDB's log output is grouped into topics. The log verbosity and output files can
be adjusted per log topic. For example
```
2010-09-20T07:32:12Z [4742] FATAL a http server has already been created
--log.level startup=trace --log.level queries=trace --log.level info
```
**error**:
Logs errors which the server has encountered.
will log messages concerning startup at trace level, AQL queries at trace level and
everything else at info level. `--log.level` can be specified multiple times at startup,
for as many topics as needed.
These errors may not necessarily result in the termination of the
server. For example,
The available log levels are:
- *fatal*: only logs fatal errors
- *error*: only logs errors
- *warning*: only logs warnings and errors
- *info*: logs information messages, warnings and errors
- *debug*: logs debug and information messages, warnings and errors
- *trace*: logs trace, debug and information messages, warnings and errors
Note that levels *debug* and *trace* will be very verbose.
Some relevant log topics available in 3.0 are:
- *collector*: information about the WAL collector's state
- *compactor*: information about the collection datafile compactor
- *datafiles*: datafile-related operations
- *mmap*: information about memory-mapping operations (including msync)
- *performance*: performance-releated messages
- *queries*: executed AQL queries, slow queries
- *replication*: replication-related info
- *requests*: HTTP requests
- *startup*: information about server startup and shutdown
- *threads*: information about threads
The old option `--log.performance` is still available in 3.0. It is now a shortcut
for the more general option `--log.level performance=trace`.
!SUBSECTION Log outputs
The log option `--log.output <definition>` allows directing the global
or per-topic log output to different outputs. The output definition "<definition>"
can be one of
- "-" for stdin
- "+" for stderr
- "syslog://<syslog-facility>"
- "syslog://<syslog-facility>/<application-name>"
- "file://<relative-path>"
The option can be specified multiple times in order to configure the output
for different log topics. To set up a per-topic output configuration, use
`--log.output <topic>=<definition>`, e.g.
queries=file://queries.txt
logs all queries to the file "queries.txt".
The old option `--log.file` is still available in 3.0 for convenience reasons. In
3.0 it is a shortcut for the more general option `--log.output file://filename`.
The old option `--log.requests-file` is still available in 3.0. It is now a shortcut
for the more general option `--log.output requests=file://...`.
Using `--log.output` also allows directing log output to different files based on
topics. For example, to log all AQL queries to a file "queries.log" one can use the
options:
```
2010-09-17T13:10:22Z [13967] ERROR strange log level 'errors'\, going to
'warning'
--log.level queries=trace --log.output queries=file:///path/to/queries.log
```
**warning**:
Provides information on errors encountered by the server, which are
not necessarily detrimental to it's continued operation.
For example,
To additionally log HTTP request to a file named "requests.log" add the options:
```
2010-09-20T08:15:26Z [5533] WARNING got corrupted HTTP request 'POS?'
--log.level requests=info --log.output requests=file:///path/to/requests.log
```
**Note**: The setting the log level to warning will also result in all
errors to be logged as well.
!SUBSECTION
**info**:
Logs information about the status of the server.
For example,
```
2010-09-20T07:40:38Z [4998] INFO SimpleVOC ready for business
```
**Note**: The setting the log level to info will also result in all
errors and warnings to be logged as well.
**debug**:
Logs all errors, all warnings and debug information.
Debug log information is generally useful to find out the state of the
server in the case of an error. For example,
```
2010-09-17T13:02:53Z [13783] DEBUG opened port 7000 for any
```
**Note**: The setting the log level to debug will also result in all
errors, warnings and server status information to be logged as well.
**trace**:
As the name suggests, logs information which may be useful to trace
problems encountered with using the server.
For example,
```
2010-09-20T08:23:12Z [5687] TRACE trying to open port 8000
```
**Note**: The setting the log level to trace will also result in all
errors, warnings, status information, and debug information to be
logged as well.
The option `--log.force-direct` can be used to disable logging in an extra
logging thread. If set to `true`, any log messages are immediately printed in the
thread that triggered the log message. This is non-optimal for performance but
can aid debugging. If set to `false`, log messages are handed off to an extra
logging thread, which asynchronously writes the log messages.
!SUBSECTION Local Time
<!-- lib/ApplicationServer/ApplicationServer.h -->
!SUBSECTION Local time
log dates and times in local time zone
`--log.use-local-time`
@ -169,7 +102,6 @@ is used, a `Z` will be appended to indicate Zulu time.
!SUBSECTION Line number
<!-- lib/ApplicationServer/ApplicationServer.h -->
log line number
@ -177,14 +109,11 @@ log line number
Normally, if an human readable fatal, error, warning or info message is
logged, no information about the file and line number is provided. The
file
and line number is only logged for debug and trace message. This option
can
be use to always log these pieces of information.
file and line number is only logged for debug and trace message. This option
can be use to always log these pieces of information.
!SUBSECTION Prefix
<!-- lib/ApplicationServer/ApplicationServer.h -->
log prefix
@ -194,7 +123,6 @@ This option is used specify an prefix to logged text.
!SUBSECTION Thread
<!-- lib/ApplicationServer/ApplicationServer.h -->
log thread identifier
@ -215,73 +143,3 @@ when no thread is logged and
```
when this command line option is set.
!SUBSECTION Content Filter
<!-- lib/ApplicationServer/ApplicationServer.h -->
log content filter
`--log.content-filter arg`
Only log message containing the specified string *arg*.
!SUBSECTION Performance
<!-- lib/ApplicationServer/ApplicationServer.h -->
performance logging
`--log.performance`
If this option is set, performance-related info messages will be logged
via
the regular logging mechanisms. These will consist of mostly timing and
debugging information for performance-critical operations.
Currently performance-related operations are logged as INFO messages.
Messages starting with prefix `[action]` indicate that an instrumented
operation was started (note that its end won't be logged). Messages with
prefix `[timer]` will contain timing information for operations. Note that
no timing information will be logged for operations taking less time than
1 second. This is to ensure that sub-second operations do not pollute
logs.
The contents of performance-related log messages enabled by this option
are subject to change in future versions of ArangoDB.
!SECTION Machine Readable Logging
!SUBSECTION Application
<!-- lib/ApplicationServer/ApplicationServer.h -->
log application name
`--log.application name`
Specifies the *name* of the application which should be logged if this
item of
information is to be logged.
!SUBSECTION Facility
<!-- lib/ApplicationServer/ApplicationServer.h -->
log facility
`--log.facility name`
If this option is set, then in addition to output being directed to the
standard output (or to a specified file, in the case that the command line
log.file option was set), log output is also sent to the system logging
facility. The *arg* is the system log facility to use. See syslog for
further details.
The value of *arg* depends on your syslog configuration. In general it
will be *user*. Fatal messages are mapped to *crit*, so if *arg*
is *user*, these messages will be logged as *user.crit*. Error
messages are mapped to *err*. Warnings are mapped to *warn*. Info
messages are mapped to *notice*. Debug messages are mapped to
*info*. Trace messages are mapped to *debug*.

3
arangod/Aql/Query.cpp
View File

@ -129,6 +129,9 @@ std::shared_ptr<VPackBuilder> Profile::toVelocyPack() {
return result;
}
/// @brief global threshold value for slow queries
double Query::SlowQueryThresholdValue = 10.0;
/// @brief whether or not query tracking is disabled globally
bool Query::DoDisableQueryTracking = false;

11
arangod/Aql/Query.h
View File

@ -282,6 +282,14 @@ class Query {
DoDisableQueryTracking = value;
}
/// @brief fetch the global slow query threshold value
static double SlowQueryThreshold() { return SlowQueryThresholdValue; }
/// @brief set global slow query threshold value
static void SlowQueryThreshold(double value) {
SlowQueryThresholdValue = value;
}
/// @brief get a description of the query's current state
std::string getStateString() const;
@ -414,6 +422,9 @@ class Query {
/// @brief whether or not the query is a data modification query
bool _isModificationQuery;
/// @brief global threshold value for slow queries
static double SlowQueryThresholdValue;
/// @brief whether or not query tracking is disabled globally
static bool DoDisableQueryTracking;
};

4
arangod/Aql/QueryList.cpp
View File

@ -51,8 +51,8 @@ QueryList::QueryList(TRI_vocbase_t*)
_slow(),
_slowCount(0),
_enabled(!Query::DisableQueryTracking()),
_trackSlowQueries(true),
_slowQueryThreshold(QueryList::DefaultSlowQueryThreshold),
_trackSlowQueries(!Query::DisableQueryTracking()),
_slowQueryThreshold(Query::SlowQueryThreshold()),
_maxSlowQueries(QueryList::DefaultMaxSlowQueries),
_maxQueryStringLength(QueryList::DefaultMaxQueryStringLength) {
_current.reserve(64);

18
arangod/RestServer/QueryRegistryFeature.cpp
View File

@ -37,7 +37,11 @@ using namespace arangodb::options;
aql::QueryRegistry* QueryRegistryFeature::QUERY_REGISTRY = nullptr;
QueryRegistryFeature::QueryRegistryFeature(ApplicationServer* server)
: ApplicationFeature(server, "QueryRegistry") {
: ApplicationFeature(server, "QueryRegistry"),
_queryTracking(true),
_slowThreshold(10.0),
_queryCacheMode("off"),
_queryCacheEntries(128) {
setOptional(false);
requiresElevatedPrivileges(false);
startsAfter("DatabaseServer");
@ -51,15 +55,18 @@ void QueryRegistryFeature::collectOptions(
options->addOldOption("database.query-cache-max-results", "query.cache-entries");
options->addOldOption("database.disable-query-tracking", "query.tracking");
options->addOption("--query.tracking", "whether to track queries",
options->addOption("--query.tracking", "whether to track slow AQL queries",
new BooleanParameter(&_queryTracking));
options->addOption("--query.slow-threshold", "threshold for slow AQL queries (in seconds)",
new DoubleParameter(&_slowThreshold));
options->addOption("--query.cache-mode",
"mode for the AQL query cache (on, off, demand)",
"mode for the AQL query result cache (on, off, demand)",
new StringParameter(&_queryCacheMode));
options->addOption("--query.cache-entries",
"maximum number of results in query cache per database",
"maximum number of results in query result cache per database",
new UInt64Parameter(&_queryCacheEntries));
}
@ -71,6 +78,9 @@ void QueryRegistryFeature::prepare() {
// set global query tracking flag
arangodb::aql::Query::DisableQueryTracking(!_queryTracking);
// set global threshold value for slow queries
arangodb::aql::Query::SlowQueryThreshold(_slowThreshold);
// configure the query cache
std::pair<std::string, size_t> cacheProperties{_queryCacheMode,
_queryCacheEntries};

7
arangod/RestServer/QueryRegistryFeature.h
View File

@ -45,9 +45,10 @@ class QueryRegistryFeature final : public application_features::ApplicationFeatu
void unprepare() override final;
private:
bool _queryTracking = true;
std::string _queryCacheMode = "off";
uint64_t _queryCacheEntries = 128;
bool _queryTracking;
double _slowThreshold;
std::string _queryCacheMode;
uint64_t _queryCacheEntries;
public:
aql::QueryRegistry* queryRegistry() const { return _queryRegistry.get(); }

41
js/client/tests/agency/agency-test.js
View File

@ -80,10 +80,8 @@ function agencyTestSuite () {
var res = writeAgency(list);
assertEqual(res.statusCode, 200);
}
return {
////////////////////////////////////////////////////////////////////////////////
/// @brief set up
////////////////////////////////////////////////////////////////////////////////
@ -336,6 +334,45 @@ function agencyTestSuite () {
assertEqual(readAndCheck([["/op/a/b/d"]]), [{op:{a:{b:{d:{ttl:15}}}}}]);
writeAndCheck([[{"/op/a/b/d/ttl":{"op":"decrement"}}]]);
assertEqual(readAndCheck([["/op/a/b/d"]]), [{op:{a:{b:{d:{ttl:14}}}}}]);
},
testInit: function() {
// mop: wait until leader has been determined
let leaderEndpoint;
let res;
if (agencyServers.length > 1) {
let leaderTests = 0;
while (true) {
res = request.get(agencyServers[whoseTurn] + "/_api/agency/config");
var config = JSON.parse(res.body);
// mop: leader election still in progress
if (config.leaderId > 127) {
if (leaderTests++ > 10) {
throw new Error("Agency doesn't report a valid leader after 10 checks. Bailing out!");
}
} else {
leaderEndpoint = config.configuration.endpoints[config.leaderId].replace("tcp", "http");
break;
}
wait(0.2);
}
} else {
leaderEndpoint = agencyServers[0].replace("tcp", "http");
}
var requests = [
["/_api/agency/write", [[{"/arango/Plan/DBServers/DBServer1":{"new":"none","op":"set"}}]]],
["/_api/agency/read", [["/arango/Plan/DBServers"]]],
];
requests.forEach(requestStruct => {
res = request.post(leaderEndpoint + requestStruct[0], {body: JSON.stringify(requestStruct[1]), headers: {"Content-Type": "application/json"}});
assertEqual(res.statusCode, 200);
});
assertEqual(res.body, JSON.stringify([{"arango":{"Plan":{"DBServers":{"DBServer1":"none"}}}}]));
}
};
}

2
lib/Logger/Logger.cpp
View File

@ -84,7 +84,7 @@ void Logger::setLogLevel(std::string const& levelName) {
level = LogLevel::FATAL;
} else if (l == "error") {
level = LogLevel::ERR;
} else if (l == "warning") {
} else if (l == "warning" || l == "warn") {
level = LogLevel::WARN;
} else if (l == "info") {
level = LogLevel::INFO;