Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Foxx Security (#8845)

This commit is contained in:
Jan Christoph Uhde 2019-04-25 09:56:29 +02:00 committed by Frank Celler
parent efe451e8fc
commit 677a79026c
153 changed files with 6078 additions and 1362 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

66
CHANGELOG
View File

@ -1,6 +1,30 @@
devel
-----
* added options to make server more secure:
- `--server.harden`: denies access to certain REST APIs that return server internals
- `--foxx.api`: set to false disables Foxx management API
- `--foxx.store`: set to false disables Foxx UI
- `--javascript.allow-port-testing`: enables internal.testPort()
- `--javascript.allow-external-process-control`: enables external process control
- `--javascript.harden`: disables getPid(), processStatistics() and logLevel()
- `--javascript.startup-options-whitelist`: control startup options visible in JavaScript
- `--javascript.environment-variables-whitelist`: control environment variables visible in JavaScript
- `--javascript.endpoints-whitelist`: control accessible endpoints in JavaScript
- `--javascript.files-whitelist`: control file access in JavaScript
Note: There is a [detailed description of all options](https://github.com/arangodb-helper/arangodb/blob/master/Documentation/Books/Manual/Security/SecurityOptions.md).
* prevent arangod from making a call to www.arangodb.com at startup
This call was done to check for available updates, but it could have contributed
to small startup delays in case outgoing connections were blocked.
* removed support for undocumented HTTP header `x-arango-v8-context`, which
allowed controlling in which particular V8 context number a JavaScript-based
action was executed. This header was only used internally for testing.
* db._query now handles additional arguments correctly when passing an AQL
query object instead of a query string and separate bindVars
@ -25,7 +49,7 @@ devel
* updated bundled version of jemalloc memory allocator to 5.2.0.
* don't create per-database system collection `_frontend` automatically.
This collection is only needed by the web UI, and it can be created lazily
This collection is only needed by the web UI, and it can be created lazily
when needed.
* added logging option `--log.time-format` to configure the time format used
@ -37,22 +61,22 @@ devel
- uptime: seconds since server start
- uptime-millis: seconds since server start, with millisecond precision
- uptime-micros: seconds since server start, with microsecond precision
- utc-datestring: UTC-based date and time in format YYYY-MM-DDTHH:MM:SSZ
- utc-datestring-millis: UTC-based date and time in format YYYY-MM-DDTHH:MM:SS.FFFZ
- utc-datestring: UTC-based date and time in format YYYY-MM-DDTHH:MM:SSZ
- utc-datestring-millis: UTC-based date and time in format YYYY-MM-DDTHH:MM:SS.FFFZ
- local-datestring: local date and time in format YYYY-MM-DDTHH:MM:SS
This change deprecates the existing options `--log.use-microtime` and
This change deprecates the existing options `--log.use-microtime` and
`--log.use-localtime`, because the functionality provided by these options
is covered by `--log.time-format` too.
* added "smart joins" to the ArangoDB Enterprise Edition that allows running cluster
joins between two certain sharded collections with performance close to that of a
local join operation.
joins between two certain sharded collections with performance close to that of a
local join operation.
* fixed internal issue #3815: fixed the removal of connected edges when
removing a vertex graph node in a smart graph environment.
* show startup warning in case kernel setting `vm.overcommit_memory` is set
* show startup warning in case kernel setting `vm.overcommit_memory` is set
to a value of 2 and the jemalloc memory allocator is in use. This combination
does not play well together.
@ -68,7 +92,7 @@ devel
executables with the `--version` command. It also changes the attribute
name in the detailed response of the `/_api/version` REST API.
* internal issue #2276: fixed the sorting of the databases in the database
* internal issue #2276: fixed the sorting of the databases in the database
selection dropdown in the web ui. The sort order differed based on whether
authentication was enabled or disabled.
@ -77,7 +101,7 @@ devel
* fixed internal issue #3789: restricted the allowed query names for user
defined custom queries within the web ui.
* upgraded bundled RocksDB version to 6.0
* added "--log.ids" option to arangod
@ -94,7 +118,7 @@ devel
* fixed internal issue #528: ArangoSearch range query sometimes doesn't work
correctly with numeric values
* changed type of the startup option `--rocksdb.recycle-log-file-num` from
* changed type of the startup option `--rocksdb.recycle-log-file-num` from
numeric to boolean, as this is also the type the options has in the RocksDB
library.
@ -102,22 +126,22 @@ devel
consistent `--rocksdb.delayed-write-rate`. When the old option name is
used, the arangod startup will be aborted with a descriptive error message.
* if not explicitly configured, make agency nodes start removing their unused
* if not explicitly configured, make agency nodes start removing their unused
WAL files a few seconds after the completed server startup already. This is
because on agency nodes, unused WAL files do not need to be retained for
potential replication clients to read them.
potential replication clients to read them.
* added option `--all-databases` to arangodump and arangorestore
When set to true, this makes arangodump dump all available databases
the current user has access to. The option `--all-databases` cannot be
used in combination with the option `--server.database`.
used in combination with the option `--server.database`.
When `--all-databases` is used, arangodump will create a subdirectory
with the data of each dumped database. Databases will be dumped one
with the data of each dumped database. Databases will be dumped one
after the other. However, inside each database, the collections of the
database can be dumped in parallel using multiple threads.
For arangorestore, this makes it restore all databases from inside the
subdirectories of the specified dump directory. Using the option for
arangorestore only makes sense for dumps created with arangodump and
@ -125,12 +149,12 @@ devel
be invoked with the options `--all-databases` and `--server.database`
at the same time. Additionally, the option `--force-same-database` cannot
be used together with `--all-databases`.
If the to-be-restored databases do not exist on the target server, then
restoring data into them will fail unless the option `--create-database`
is also specified. Please note that in this case a database user must
be used that has access to the `_system` database, in order to create
the databases on restore.
the databases on restore.
* added index hints feature to AQL
@ -138,12 +162,12 @@ devel
If a name is not specified on index creation, one will be auto-generated.
* Under normal circumstances there should be no need to connect to a
database server in a cluster with one of the client tools, and it is
* Under normal circumstances there should be no need to connect to a
database server in a cluster with one of the client tools, and it is
likely that any user operations carried out there with one of the client
tools may cause trouble.
tools may cause trouble.
The client tools arangosh, arangodump and arangorestore will now emit
The client tools arangosh, arangodump and arangorestore will now emit
a warning when connecting with them to a database server node in a cluster.
* fix compation behavior of followers

12
Documentation/Books/Manual/Appendix/JavaScriptModules/FileSystem.md
View File

@ -113,7 +113,10 @@ absolute path to the same location is returned.
sets file permissions of specified files (non windows only)
`fs.exists(path)`
`fs.chmod(path, mode)`
where `mode` is a string with a leading zero matching the `OCTAL-MODE` as explained
in *nix `man chmod`.
Returns true on success.
@ -288,6 +291,13 @@ the
specified filename.
### linkFile
creates a symbolic link from a target in the place of linkpath.
`fs.linkFile(target, linkpath)`
In `linkpath` a symbolic link to `target` will be created.
### move

52
Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md
View File

@ -303,6 +303,41 @@ Mon Apr 01 2019 02:00:00 GMT+0200 (Central European Summer Time)
> new Date("2019-04-01T00:00:00Z");
Mon Apr 01 2019 02:00:00 GMT+0200 (Central European Summer Time)
```
### JavaScript security options
ArangoDB 3.5 provides several new options for restricting the functionality of
JavaScript application code running in the server, with the intent to make a setup
more secure.
There now exist startup options for restricting which environment variables and
values of which configuration options JavaScript code is allowed to read. These
options can be set to prevent leaking of confidential information from the
environment or the setup into the JavaScript application code.
Additionally there are options to restrict outbound HTTP connections from JavaScript
applications to certain endpoints and to restrict filesystem access from JavaScript
applications to certain directories only.
Finally there are startup options to turn off the REST APIs for managing Foxx
services, which can be used to prevent installation and uninstallation of Foxx
applications on a server. A separate option is provided to turn off access and
connections to the central Foxx app store via the web interface.
A complete overview of the security options can be found in [Security Options](../Security/SecurityOptions.md).
### Foxx
Request credentials are now exposed via the `auth` property:
```js
const tokens = context.collection("tokens");
router.get("/authorized", (req, res) => {
if (!req.auth || !req.auth.bearer || !tokens.exists(req.auth.bearer)) {
res.throw(403, "Not authenticated");
}
// ...
});
```
### API improvements
@ -381,6 +416,7 @@ Client configurations that use this configuration variable should adjust their
configuration and set this variable to a boolean value instead of to a numeric
value.
Miscellaneous
-------------
@ -455,19 +491,3 @@ The bundled JEMalloc memory allocator used in ArangoDB release packages has been
upgraded from version 5.0.1 to version 5.2.0.
The bundled version of the RocksDB library has been upgraded from 5.16 to 6.0.
Foxx
----
Request credentials are now exposed via the `auth` property:
```js
const tokens = context.collection("tokens");
router.get("/authorized", (req, res) => {
if (!req.auth || !req.auth.bearer || !tokens.exists(req.auth.bearer)) {
res.throw(403, "Not authenticated");
}
// ...
});
```

1
Documentation/Books/Manual/SUMMARY.md
View File

@ -359,6 +359,7 @@
* [Removal Procedure](Administration/Starter/Removal.md)
* [Recovery Procedure](Administration/Starter/Recovery.md)
* [Security](Security/README.md)
* [Security Options](Security/SecurityOptions.md)
* [Change Root Password](Security/ChangeRootPassword.md)
* [Encryption at Rest](Security/Encryption/README.md)
* [Auditing](Security/Auditing/README.md)

1
Documentation/Books/Manual/Security/README.md
View File

@ -1,5 +1,6 @@
# Security
- [Security Options](SecurityOptions.md)
- [Change Root Password](ChangeRootPassword.md)
- [Encryption at Rest](Encryption/README.md)
- [Auditing](Auditing/README.md)

160
Documentation/Books/Manual/Security/SecurityOptions.md Normal file
View File

@ -0,0 +1,160 @@
# Server security options
_arangod_ provides a variety of options to make a setup more secure.
Administrators can use these options to limit access to certain ArangoDB
server functionality as well as providing the leakage of information about
the environment that a server is running in.
## General security options
The following security options are available:
- `--server.harden`
If this option is set to `true` and authentication is enabled, non-admin users
will be denied access to the following REST APIs:
* `/_admin/log`
* `/_admin/log/level`
* `/_admin/status`
* `/_admin/statistics`
* `/_admin/statistics-description`
* `/_api/engine/stats`
Additionally, no version details will be revealed by the version REST API at
`/_api/version`.
`
The default value for this option is `false`.
## JavaScript security options
`arangod` has several options that allow you to make your installation more
secure when it comes to running application code in it. Below you will find
an overview of the relevant options.
### Blacklist and whitelists
Several options exists to restrict JavaScript application code functionality
to just certain allowed subsets. Which subset of functionality is available
can be controlled via blacklisting and whitelisting access to individual
components. Blacklists can be used to disallow access to dedicated functionality,
whereas whitelists can be used to explicitly allow access to certain functionality.
If an item is covered by both a blacklist and a whitelist, the whitelist will
overrule and access to the functionality will be allowed.
Values for blacklist and whitelist options need to be specified as ECMAScript
regular expressions. Each option can be used multiple times. In this case,
the individual values for each option will be combined with a _logical or_.
For example, the following combination of startup options
--javascript.startup-options-whitelist "^server\."
--javascript.startup-options-whitelist "^log\."
--javascript.startup-options-blacklist "^javascript\."
--javascript.startup-options-blacklist "endpoint"
will resolve internally to the following regular expressions:
```
--javascript.startup-options-whitelist = "^server\.|^log\."
--javascript.startup-options-blacklist = "^javascript\.|endpoint"
```
Access to directories and files from JavaScript operations is only
controlled via a whitelist, which can be specified via the startup
option `--javascript.files-whitelist`.
For example, when using the following startup options
--javascript.startup-options-whitelist "^/etc/required/"
--javascript.startup-options-whitelist "^/etc/mtab/"
all files in the directories `/etc/required` and `/etc/mtab` plus their
subdirectories will be accessible, while access to files in any other directories
will be disallowed from JavaScript operations, with the following exceptions:
- ArangoDB's temporary directory: JavaScript code is given access to this
directory for storing temporary files. The temporary directory location
can be specified explicitly via the `--temp.path` option at startup.
If the option is not specified, ArangoDB will automatically use a subdirectory
of the system's temporary directory).
- ArangoDB's own JavaScript code, shipped with the ArangoDB release packages.
Files in this directory and its subdirectories will be readable for JavaScript
code running in ArangoDB. The exact path can be specified by the startup option
`--javascript.startup-directory`.
### Options for blacklisting and whitelisting
The following options are available for blacklisting and whitelisting access
to dedicated functionality for application code:
- `--javascript.startup-options-whitelist` and `--javascript.startup-options-blacklist`:
These options control which startup options will be exposed to JavaScript code,
following above rules for blacklists and whitelists.
- `--javascript.environment-variables-whitelist` and `--javascript.environment-variables-blacklist`:
These options control which environment variables will be exposed to JavaScript
code, following above rules for blacklists and whitelists.
- `--javascript.endpoints-whitelist` and `--javascript.endpoints-blacklist`:
These options control which endpoints can be used from within the `@arangodb/request`
JavaScript module.
Endpoint values are passed into the filter in a normalized format starting
with either of the prefixes `tcp://`, `ssl://`, `unix://` or `srv://`.
Note that for HTTP/SSL-based endpoints the port number will be included too,
and that the endpoint can be specified either as an IP address or host name
from application code.
- `--javascript.files-whitelist`:
This option controls which filesystem paths can be accessed from JavaScript code.
### Additional JavaScript security options
In addition to the blacklisting and whitelisting security options, the following
extra options are available for locking down JavaScript access to server functionality:
- `--javascript.allow-port-testing`:
If set to `true`, this option enables the `testPort` JavaScript function in the
`internal` module. The default value is `false`.
- `--javascript.allow-external-process-control`:
If set to `true`, this option allows the execution and control of external processes
from JavaScript code via the functions from the `internal` module:
- executeExternal
- executeExternalAndWait
- getExternalSpawned
- killExternal
- suspendExternal
- continueExternal
- statusExternal
- `--javascript.harden`:
If set to `true`, this setting will deactivate the following JavaScript functions
which may leak information about the environment:
- `internal.clientStatistics()`
- `internal.httpStatistics()`
- `internal.processStatistics()`
- `internal.getPid()`
- `internal.logLevel()`.
The default value is `false`.
## Security options for managing Foxx applications
The following options are available for controlling the installation of Foxx applications
in an ArangoDB server:
- `--foxx.api`:
If set to `false`, this option disables the Foxx management API, which will make it
impossible to install and uninstall Foxx applications. Setting the option to `false`
will also deactivate the "Services" section in the web interface.
The default value is `true`, meaning that Foxx apps can be installed and uninstalled.
- `--foxx.store`:
If set to `false`, this option disables the Foxx app store in ArangoDB's web interface,
which will also prevent ArangoDB and its web interface from making calls to the main Foxx
application Github repository at https://github.com/arangodb/foxx-apps.
The default value is `true`.

21
README_maintainers.md
View File

@ -798,3 +798,24 @@ for `mocha`) match the versions required by the updated module and delete any
duplicated nested dependencies if necessary (e.g. `mocha/node_modules/glob`)
to make sure the global (mocked) version is used instead.
Changing the FrontEnd
=====================
Change to `js/apps/system/_admin/aardvark/APP/` and open
`manifest.json`. Then apply the following change:
```
"/app.js": {
- "path": "frontend/build/app.min.js",
- "gzip": true
+ "path": "frontend/build/app.js",
+ "gzip": false
},
```
Then run `grunt`, `grunt deploy` and `grunt watch`. This
should make every change in the code available after a
reload for the browser.
Note: You might need to do the same for other files. Usually
the change for `app` should suffice.

2
UnitTests/OskarTestSuitesBlackList
View File

@ -1,5 +1,3 @@
upgrade_data_3.2.*
upgrade_data_3.3.*
permissions
permissions_server
audit

4
arangod/Actions/ActionFeature.cpp
View File

@ -58,8 +58,8 @@ void ActionFeature::start() {
V8DealerFeature* dealer =
ApplicationServer::getFeature<V8DealerFeature>("V8Dealer");
dealer->defineContextUpdate([](v8::Isolate* isolate, v8::Handle<v8::Context> context,
size_t) { TRI_InitV8Actions(isolate, context); },
dealer->defineContextUpdate([](v8::Isolate* isolate, v8::Handle<v8::Context> /*context*/,
size_t) { TRI_InitV8Actions(isolate); },
nullptr);
}

7
arangod/Actions/actions.h
View File

@ -46,7 +46,11 @@ class TRI_action_result_t {
/// @brief action descriptor
class TRI_action_t {
public:
TRI_action_t() : _urlParts(0), _isPrefix(false), _allowUseDatabase(false) {}
TRI_action_t() :
_urlParts(0),
_isPrefix(false),
_allowUseDatabase(false),
_isSystem(false) {}
virtual ~TRI_action_t() {}
@ -69,6 +73,7 @@ class TRI_action_t {
bool _isPrefix;
bool _allowUseDatabase;
bool _isSystem;
};
/// @brief fake action class used only inside /_admin/execute RestHandler

6
arangod/Aql/Expression.cpp
View File

@ -882,13 +882,9 @@ AqlValue Expression::executeSimpleExpressionFCallJS(AstNode const* node,
{
ISOLATE;
TRI_ASSERT(isolate != nullptr);
TRI_V8_CURRENT_GLOBALS_AND_SCOPE;
v8::HandleScope scope(isolate); \
_ast->query()->prepareV8Context();
auto old = v8g->_query;
v8g->_query = static_cast<void*>(_ast->query());
TRI_DEFER(v8g->_query = old);
std::string jsName;
size_t const n = static_cast<int>(member->numMembers());
size_t callArgs = (node->type == NODE_TYPE_FCALL_USER ? 2 : n);

6
arangod/Aql/Functions.cpp
View File

@ -1086,16 +1086,12 @@ AqlValue callApplyBackend(ExpressionContext* expressionContext, transaction::Met
// JavaScript function (this includes user-defined functions)
{
ISOLATE;
TRI_V8_CURRENT_GLOBALS_AND_SCOPE;
v8::HandleScope scope(isolate); \
Query* query = expressionContext->query();
TRI_ASSERT(query != nullptr);
query->prepareV8Context();
auto old = v8g->_query;
v8g->_query = query;
TRI_DEFER(v8g->_query = old);
std::string jsName;
int const n = static_cast<int>(invokeParams.size());
int const callArgs = (func == nullptr ? 3 : n);

5
arangod/Aql/ModificationExecutor.cpp
View File

@ -46,14 +46,13 @@ std::string toString(SingleBlockFetcher<false>&) {
template <typename FetcherType>
ModificationExecutorBase<FetcherType>::ModificationExecutorBase(Fetcher& fetcher, Infos& infos)
: _infos(infos), _fetcher(fetcher), _prepared(false){};
: _infos(infos), _fetcher(fetcher), _prepared(false) {}
template <typename Modifier, typename FetcherType>
ModificationExecutor<Modifier, FetcherType>::ModificationExecutor(Fetcher& fetcher, Infos& infos)
: ModificationExecutorBase<FetcherType>(fetcher, infos), _modifier() {
this->_infos._trx->pinData(this->_infos._aqlCollection->id()); // important for mmfiles
// LOG_DEVEL << toString(_modifier) << " " << toString(this->_fetcher); // <-- enable this first when debugging modification problems
};
}
template <typename Modifier, typename FetcherType>
ModificationExecutor<Modifier, FetcherType>::~ModificationExecutor() = default;

4
arangod/Aql/Query.cpp
View File

@ -51,6 +51,7 @@
#include "Transaction/V8Context.h"
#include "Utils/CollectionNameResolver.h"
#include "Utils/ExecContext.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-conv.h"
#include "V8/v8-vpack.h"
#include "V8Server/V8DealerFeature.h"
@ -1176,7 +1177,8 @@ void Query::enterContext() {
"V8 engine is disabled");
}
TRI_ASSERT(V8DealerFeature::DEALER != nullptr);
_context = V8DealerFeature::DEALER->enterContext(&_vocbase, false);
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createQueryContext();
_context = V8DealerFeature::DEALER->enterContext(&_vocbase, securityContext);
if (_context == nullptr) {
THROW_ARANGO_EXCEPTION_MESSAGE(

1
arangod/CMakeLists.txt
View File

@ -365,6 +365,7 @@ SET(ARANGOD_SOURCES
GeneralServer/ListenTask.cpp
GeneralServer/RestHandler.cpp
GeneralServer/RestHandlerFactory.cpp
GeneralServer/ServerSecurityFeature.cpp
GeneralServer/SocketSslTcp.cpp
GeneralServer/SocketTask.cpp
GeneralServer/Task.cpp

84
arangod/GeneralServer/ServerSecurityFeature.cpp Normal file
View File

@ -0,0 +1,84 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2016 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Jan Steemann
////////////////////////////////////////////////////////////////////////////////
#include "GeneralServer/ServerSecurityFeature.h"
#include "Logger/Logger.h"
#include "ProgramOptions/ProgramOptions.h"
#include "ProgramOptions/Section.h"
#include "Utils/ExecContext.h"
using namespace arangodb;
using namespace arangodb::basics;
using namespace arangodb::options;
ServerSecurityFeature::ServerSecurityFeature(application_features::ApplicationServer& server)
: ApplicationFeature(server, "ServerSecurity"),
_enableFoxxApi(true),
_enableFoxxStore(true),
_hardenedRestApi(false) {
setOptional(false);
startsAfter("ServerPlatform");
}
void ServerSecurityFeature::collectOptions(std::shared_ptr<ProgramOptions> options) {
options->addSection("server", "Server features");
options->addOption("--server.harden",
"lock down REST APIs that reveal version information or server "
"internals for non-admin users",
new BooleanParameter(&_hardenedRestApi))
.setIntroducedIn(30500);
options->addSection("foxx", "Configure Foxx");
options->addOption("--foxx.api", "enables Foxx management REST APIs",
new BooleanParameter(&_enableFoxxApi))
.setIntroducedIn(30500);
options->addOption("--foxx.store", "enables Foxx store in web interface",
new BooleanParameter(&_enableFoxxStore))
.setIntroducedIn(30500);
}
bool ServerSecurityFeature::isFoxxApiDisabled() const {
return !_enableFoxxApi;
}
bool ServerSecurityFeature::isFoxxStoreDisabled() const {
return !_enableFoxxStore || !_enableFoxxApi;
}
bool ServerSecurityFeature::isRestApiHardened() const {
return _hardenedRestApi;
}
bool ServerSecurityFeature::canAccessHardenedApi() const {
bool allowAccess = !isRestApiHardened();
if (!allowAccess) {
ExecContext const* exec = ExecContext::CURRENT;
if (exec == nullptr || exec->isAdminUser()) {
// also allow access if there is not authentication
// enabled or when the user is an administrator
allowAccess = true;
}
}
return allowAccess;
}

49
arangod/GeneralServer/ServerSecurityFeature.h Normal file
View File

@ -0,0 +1,49 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2016 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Jan Steemann
////////////////////////////////////////////////////////////////////////////////
#ifndef ARANGODB_APPLICATION_FEATURES_SERVER_SECURITY_FEATURE_H
#define ARANGODB_APPLICATION_FEATURES_SERVER_SECURITY_FEATURE_H 1
#include "ApplicationFeatures/ApplicationFeature.h"
namespace arangodb {
class ServerSecurityFeature final : public application_features::ApplicationFeature {
public:
explicit ServerSecurityFeature(application_features::ApplicationServer& server);
void collectOptions(std::shared_ptr<options::ProgramOptions>) override final;
bool isRestApiHardened() const;
bool isFoxxApiDisabled() const;
bool isFoxxStoreDisabled() const;
bool canAccessHardenedApi() const;
private:
bool _enableFoxxApi;
bool _enableFoxxStore;
bool _hardenedRestApi;
};
} // namespace arangodb
#endif

62
arangod/RestHandler/RestAdminExecuteHandler.cpp
View File

@ -29,6 +29,7 @@
#include "Basics/StaticStrings.h"
#include "Basics/StringUtils.h"
#include "Logger/Logger.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-globals.h"
#include "V8/v8-vpack.h"
#include "V8Server/v8-actions.h"
@ -53,11 +54,11 @@ RestStatus RestAdminExecuteHandler::execute() {
}
TRI_ASSERT(V8DealerFeature::DEALER->allowAdminExecute());
arangodb::velocypack::StringRef bodyStr = _request->rawPayload();
char const* body = bodyStr.data();
size_t bodySize = bodyStr.size();
if (bodySize == 0) {
// nothing to execute. return an empty response
VPackBuilder result;
@ -72,29 +73,14 @@ RestStatus RestAdminExecuteHandler::execute() {
try {
LOG_TOPIC("c838e", WARN, Logger::FIXME) << "about to execute: '" << Logger::CHARS(body, bodySize) << "'";
ssize_t forceContext = -1;
bool found;
std::string const& c = _request->header("x-arango-v8-context", found);
if (found && !c.empty()) {
forceContext = basics::StringUtils::int32(c);
}
// get a V8 context
bool const allowUseDatabase = ActionFeature::ACTION->allowUseDatabase();
V8Context* context = V8DealerFeature::DEALER->enterContext(&_vocbase, allowUseDatabase, forceContext);
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createRestAdminScriptActionContext(allowUseDatabase);
V8ContextGuard guard(&_vocbase, securityContext);
// note: the context might be nullptr in case of shut-down
if (context == nullptr) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_RESOURCE_LIMIT,
"unable to acquire V8 context in time");
}
TRI_DEFER(V8DealerFeature::DEALER->exitContext(context));
{
v8::Isolate* isolate = context->_isolate;
{
v8::Isolate* isolate = guard.isolate();
v8::HandleScope scope(isolate);
v8::Handle<v8::Object> current = isolate->GetCurrentContext()->Global();
@ -106,9 +92,9 @@ RestStatus RestAdminExecuteHandler::execute() {
v8::Handle<v8::Value> args[1] = {TRI_V8_PAIR_STRING(isolate, body, bodySize)};
v8::Local<v8::Object> function = ctor->NewInstance(TRI_IGETC, 1, args).FromMaybe(v8::Local<v8::Object>());
v8::Handle<v8::Function> action = v8::Local<v8::Function>::Cast(function);
v8::Handle<v8::Value> rv;
if (!action.IsEmpty()) {
action->SetName(TRI_V8_ASCII_STRING(isolate, "source"));
@ -116,18 +102,18 @@ RestStatus RestAdminExecuteHandler::execute() {
TRI_fake_action_t adminExecuteAction("_admin/execute", 2);
v8g->_currentRequest = TRI_RequestCppToV8(isolate, v8g, _request.get(), &adminExecuteAction);
v8g->_currentRequest = TRI_RequestCppToV8(isolate, v8g, _request.get(), &adminExecuteAction);
v8g->_currentResponse = v8::Object::New(isolate);
auto guard = scopeGuard([&v8g, &isolate]() {
v8g->_currentRequest = v8::Undefined(isolate);
v8g->_currentResponse = v8::Undefined(isolate);
});
v8::Handle<v8::Value> args[] = {v8::Null(isolate)};
rv = action->Call(current, 0, args);
}
if (tryCatch.HasCaught()) {
// got an error
std::string errorMessage;
@ -142,7 +128,7 @@ RestStatus RestAdminExecuteHandler::execute() {
errorMessage = *tryCatchMessage;
}
}
_response->setResponseCode(rest::ResponseCode::SERVER_ERROR);
switch (_response->transportType()) {
case Endpoint::TransportType::HTTP: {
@ -153,7 +139,7 @@ RestStatus RestAdminExecuteHandler::execute() {
_response->setContentType(rest::ContentType::TEXT);
httpResponse->body().appendText(errorMessage.data(), errorMessage.size());
break;
}
}
case Endpoint::TransportType::VST: {
VPackBuffer<uint8_t> buffer;
VPackBuilder builder(buffer);
@ -168,11 +154,11 @@ RestStatus RestAdminExecuteHandler::execute() {
bool returnAsJSON = _request->parsedValue("returnAsJSON", false);
if (returnAsJSON) {
// if the result is one of the following type, we return it as is
returnAsJSON &= (rv->IsString() || rv->IsStringObject() ||
rv->IsNumber() || rv->IsNumberObject() ||
returnAsJSON &= (rv->IsString() || rv->IsStringObject() ||
rv->IsNumber() || rv->IsNumberObject() ||
rv->IsBoolean());
}
VPackBuilder result;
bool handled = false;
int res = TRI_ERROR_FAILED;
@ -182,7 +168,7 @@ RestStatus RestAdminExecuteHandler::execute() {
result.add(StaticStrings::Error, VPackValue(false));
result.add(StaticStrings::Code, VPackValue(static_cast<int>(rest::ResponseCode::OK)));
if (rv->IsObject()) {
res = TRI_V8ToVPack(isolate, result, rv, false);
res = TRI_V8ToVPack(isolate, result, rv, false);
handled = true;
}
result.close();
@ -190,16 +176,16 @@ RestStatus RestAdminExecuteHandler::execute() {
if (!handled) {
result.clear();
VPackBuilder temp;
res = TRI_V8ToVPack(isolate, temp, rv, false);
res = TRI_V8ToVPack(isolate, temp, rv, false);
result.add(temp.slice());
}
}
if (res != TRI_ERROR_NO_ERROR) {
THROW_ARANGO_EXCEPTION(res);
}
generateResult(rest::ResponseCode::OK, result.slice());
}
}

13
arangod/RestHandler/RestAdminLogHandler.cpp
View File

@ -28,6 +28,7 @@
#include <velocypack/velocypack-aliases.h>
#include "Basics/StringUtils.h"
#include "GeneralServer/ServerSecurityFeature.h"
#include "Logger/LogBuffer.h"
#include "Logger/Logger.h"
#include "Rest/HttpRequest.h"
@ -40,13 +41,23 @@ RestAdminLogHandler::RestAdminLogHandler(GeneralRequest* request, GeneralRespons
: RestBaseHandler(request, response) {}
RestStatus RestAdminLogHandler::execute() {
size_t const len = _request->suffixes().size();
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
if (!security->canAccessHardenedApi()) {
generateError(rest::ResponseCode::FORBIDDEN, TRI_ERROR_FORBIDDEN);
return RestStatus::DONE;
}
size_t const len = _request->suffixes().size();
if (len == 0) {
reportLogs();
} else {
setLogLevel();
}
return RestStatus::DONE;
}

12
arangod/RestHandler/RestAdminStatisticsHandler.cpp
View File

@ -21,6 +21,7 @@
////////////////////////////////////////////////////////////////////////////////
#include "RestAdminStatisticsHandler.h"
#include "GeneralServer/ServerSecurityFeature.h"
#include "Statistics/Descriptions.h"
#include "Statistics/StatisticsFeature.h"
@ -37,6 +38,17 @@ RestStatus RestAdminStatisticsHandler::execute() {
generateError(rest::ResponseCode::METHOD_NOT_ALLOWED, TRI_ERROR_HTTP_METHOD_NOT_ALLOWED);
return RestStatus::DONE;
}
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
if (!security->canAccessHardenedApi()) {
// dont leak information about server internals here
generateError(rest::ResponseCode::FORBIDDEN, TRI_ERROR_FORBIDDEN);
return RestStatus::DONE;
}
if (_request->requestPath() == "/_admin/statistics") {
getStatistics();

2
arangod/RestHandler/RestCollectionHandler.cpp
View File

@ -297,7 +297,7 @@ void RestCollectionHandler::handleCommandPost() {
type = TRI_col_type_e::TRI_COL_TYPE_EDGE;
}
// for some "security" a white-list of allowed parameters
// for some "security" a whitelist of allowed parameters
VPackBuilder filtered = VPackCollection::keep(
body, std::unordered_set<std::string>{
"doCompact", StaticStrings::DataSourceSystem,

30
arangod/RestHandler/RestEngineHandler.cpp
View File

@ -21,6 +21,7 @@
/// @author Jan Steemann
////////////////////////////////////////////////////////////////////////////////
#include "GeneralServer/ServerSecurityFeature.h"
#include "RestEngineHandler.h"
#include "Rest/HttpRequest.h"
#include "StorageEngine/EngineSelectorFeature.h"
@ -39,13 +40,13 @@ RestEngineHandler::RestEngineHandler(GeneralRequest* request, GeneralResponse* r
RestStatus RestEngineHandler::execute() {
// extract the sub-request type
auto const type = _request->requestType();
if (type == rest::RequestType::GET) {
handleGet();
if (type != rest::RequestType::GET) {
generateError(rest::ResponseCode::METHOD_NOT_ALLOWED, TRI_ERROR_HTTP_METHOD_NOT_ALLOWED);
return RestStatus::DONE;
}
generateError(rest::ResponseCode::METHOD_NOT_ALLOWED, TRI_ERROR_HTTP_METHOD_NOT_ALLOWED);
handleGet();
return RestStatus::DONE;
}
@ -58,11 +59,24 @@ void RestEngineHandler::handleGet() {
return;
}
if (suffixes.size() == 0) {
if (suffixes.empty()) {
getCapabilities();
} else {
getStats();
return;
}
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
if (!security->canAccessHardenedApi()) {
// dont leak information about server internals here
generateError(rest::ResponseCode::FORBIDDEN, TRI_ERROR_FORBIDDEN);
return;
}
// access to engine stats is disallowed in hardened mode
getStats();
}
void RestEngineHandler::getCapabilities() {

12
arangod/RestHandler/RestStatusHandler.cpp
View File

@ -28,6 +28,7 @@
#include "Agency/Agent.h"
#include "ApplicationFeatures/ApplicationServer.h"
#include "Cluster/ServerState.h"
#include "GeneralServer/ServerSecurityFeature.h"
#include "Rest/HttpRequest.h"
#include "Rest/Version.h"
#include "RestServer/ServerFeature.h"
@ -51,6 +52,17 @@ RestStatusHandler::RestStatusHandler(GeneralRequest* request, GeneralResponse* r
: RestBaseHandler(request, response) {}
RestStatus RestStatusHandler::execute() {
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
if (!security->canAccessHardenedApi()) {
// dont leak information about server internals here
generateError(rest::ResponseCode::FORBIDDEN, TRI_ERROR_FORBIDDEN);
return RestStatus::DONE;
}
VPackBuilder result;
result.add(VPackValue(VPackValueType::Object));
result.add("server", VPackValue("arango"));

64
arangod/RestHandler/RestTasksHandler.cpp
View File

@ -27,8 +27,7 @@
#include "Basics/VelocyPackHelper.h"
#include "Cluster/ServerState.h"
#include "Rest/HttpRequest.h"
#include "Scheduler/Scheduler.h"
#include "Scheduler/SchedulerFeature.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-globals.h"
#include "V8/v8-vpack.h"
#include "V8Server/V8DealerFeature.h"
@ -140,12 +139,6 @@ void RestTasksHandler::registerTask(bool byId) {
}
}
if (SchedulerFeature::SCHEDULER == nullptr) {
generateError(rest::ResponseCode::SERVER_ERROR, TRI_ERROR_INTERNAL,
"no scheduler found");
return;
}
ExecContext const* exec = ExecContext::CURRENT;
if (exec != nullptr) {
if (exec->databaseAuthLevel() != auth::Level::RW) {
@ -206,40 +199,35 @@ void RestTasksHandler::registerTask(bool byId) {
return;
}
{
Result res;
v8::Isolate* isolate = nullptr;
try {
V8ContextDealerGuard dealerGuard(res, isolate, &_vocbase, true);
if (res.fail()) {
generateError(res);
return;
}
v8::HandleScope scope(isolate);
v8::Handle<v8::Object> bv8 = TRI_VPackToV8(isolate, body).As<v8::Object>();
try {
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createRestrictedContext();
V8ContextGuard guard(&_vocbase, securityContext);
v8::Isolate* isolate = guard.isolate();
v8::HandleScope scope(isolate);
v8::Handle<v8::Object> bv8 = TRI_VPackToV8(isolate, body).As<v8::Object>();
if (bv8->Get(TRI_V8_ASCII_STRING(isolate, "command"))->IsFunction()) {
// need to add ( and ) around function because call will otherwise break
command = "(" + cmdSlice.copyString() + ")(params)";
} else {
command = cmdSlice.copyString();
}
if (bv8->Get(TRI_V8_ASCII_STRING(isolate, "command"))->IsFunction()) {
// need to add ( and ) around function because call will otherwise break
command = "(" + cmdSlice.copyString() + ")(params)";
} else {
command = cmdSlice.copyString();
}
if (!Task::tryCompile(isolate, command)) {
generateError(rest::ResponseCode::BAD, TRI_ERROR_BAD_PARAMETER,
"cannot compile command");
return;
}
} catch (arangodb::basics::Exception const& ex) {
generateError(Result{ex.code(), ex.what()});
return;
} catch (std::exception const& ex) {
generateError(Result{TRI_ERROR_INTERNAL, ex.what()});
return;
} catch (...) {
generateError(TRI_ERROR_INTERNAL);
if (!Task::tryCompile(isolate, command)) {
generateError(rest::ResponseCode::BAD, TRI_ERROR_BAD_PARAMETER,
"cannot compile command");
return;
}
} catch (arangodb::basics::Exception const& ex) {
generateError(Result{ex.code(), ex.what()});
return;
} catch (std::exception const& ex) {
generateError(Result{TRI_ERROR_INTERNAL, ex.what()});
return;
} catch (...) {
generateError(TRI_ERROR_INTERNAL);
return;
}
// extract the parameters

6
arangod/RestHandler/RestTransactionHandler.cpp
View File

@ -23,6 +23,7 @@
#include "RestTransactionHandler.h"
#include "Actions/ActionFeature.h"
#include "ApplicationFeatures/ApplicationServer.h"
#include "Basics/ReadLocker.h"
#include "Basics/WriteLocker.h"
@ -33,6 +34,7 @@
#include "Transaction/ManagerFeature.h"
#include "Transaction/Helpers.h"
#include "Transaction/Status.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8Server/V8Context.h"
#include "V8Server/V8DealerFeature.h"
#include "VocBase/Methods/Transactions.h"
@ -239,7 +241,9 @@ void RestTransactionHandler::executeJSTransaction() {
std::string portType = _request->connectionInfo().portType();
_v8Context = V8DealerFeature::DEALER->enterContext(&_vocbase, true /*allow use database*/);
bool allowUseDatabase = ActionFeature::ACTION->allowUseDatabase();
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createRestActionContext(allowUseDatabase);
_v8Context = V8DealerFeature::DEALER->enterContext(&_vocbase, securityContext);
if (!_v8Context) {
generateError(Result(TRI_ERROR_INTERNAL, "could not acquire v8 context"));

61
arangod/RestHandler/RestVersionHandler.cpp
View File

@ -21,12 +21,13 @@
/// @author Achim Brandt
////////////////////////////////////////////////////////////////////////////////
#include "RestVersionHandler.h"
#include "ApplicationFeatures/ApplicationServer.h"
#include "Cluster/ServerState.h"
#include "GeneralServer/ServerSecurityFeature.h"
#include "Rest/HttpRequest.h"
#include "Rest/Version.h"
#include "RestServer/ServerFeature.h"
#include "RestVersionHandler.h"
#include <velocypack/Builder.h>
#include <velocypack/velocypack-aliases.h>
@ -44,42 +45,48 @@ RestVersionHandler::RestVersionHandler(GeneralRequest* request, GeneralResponse*
RestStatus RestVersionHandler::execute() {
VPackBuilder result;
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
bool const allowInfo = security->canAccessHardenedApi();
result.add(VPackValue(VPackValueType::Object));
result.add("server", VPackValue("arango"));
result.add("version", VPackValue(ARANGODB_VERSION));
#ifdef USE_ENTERPRISE
result.add("license", VPackValue("enterprise"));
result.add("license", VPackValue("enterprise"));
#else
result.add("license", VPackValue("community"));
result.add("license", VPackValue("community"));
#endif
bool found;
std::string const& detailsStr = _request->value("details", found);
if (allowInfo) {
result.add("version", VPackValue(ARANGODB_VERSION));
if (found && StringUtils::boolean(detailsStr)) {
result.add("details", VPackValue(VPackValueType::Object));
bool found;
std::string const& detailsStr = _request->value("details", found);
if (found && StringUtils::boolean(detailsStr)) {
result.add("details", VPackValue(VPackValueType::Object));
Version::getVPack(result);
Version::getVPack(result);
if (application_features::ApplicationServer::server != nullptr) {
auto server = application_features::ApplicationServer::server->getFeature<ServerFeature>(
"Server");
result.add("mode", VPackValue(server->operationModeString()));
auto serverState = ServerState::instance();
if (serverState != nullptr) {
result.add("role", VPackValue(ServerState::roleToString(serverState->getRole())));
if (application_features::ApplicationServer::server != nullptr) {
auto server = application_features::ApplicationServer::server->getFeature<ServerFeature>(
"Server");
result.add("mode", VPackValue(server->operationModeString()));
auto serverState = ServerState::instance();
if (serverState != nullptr) {
result.add("role", VPackValue(ServerState::roleToString(serverState->getRole())));
}
}
}
std::string host = ServerState::instance()->getHost();
if (!host.empty()) {
result.add("host", VPackValue(host));
}
result.close();
}
std::string host = ServerState::instance()->getHost();
if (!host.empty()) {
result.add("host", VPackValue(host));
}
result.close();
} // found
} // allowInfo
result.close();
generateResult(rest::ResponseCode::OK, result.slice());
return RestStatus::DONE;

4
arangod/RestServer/CheckVersionFeature.cpp
View File

@ -138,7 +138,9 @@ void CheckVersionFeature::checkVersion() {
bool ignoreDatafileErrors = false;
{
VPackBuilder options = server()->options(std::unordered_set<std::string>());
VPackBuilder options = server()->options([](std::string const& name) {
return (name.find("database.ignore-datafile-errors") != std::string::npos);
});
VPackSlice s = options.slice();
if (s.get("database.ignore-datafile-errors").isBoolean()) {
ignoreDatafileErrors = s.get("database.ignore-datafile-errors").getBool();

20
arangod/RestServer/ConsoleThread.cpp
View File

@ -31,6 +31,7 @@
#include "Basics/tri-strings.h"
#include "Logger/Logger.h"
#include "Rest/Version.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/V8LineEditor.h"
#include "V8/v8-conv.h"
#include "V8/v8-utils.h"
@ -48,7 +49,6 @@ Mutex ConsoleThread::serverConsoleMutex;
ConsoleThread::ConsoleThread(ApplicationServer* applicationServer, TRI_vocbase_t* vocbase)
: Thread("Console"),
_applicationServer(applicationServer),
_context(nullptr),
_vocbase(vocbase),
_userAborted(false) {}
@ -66,18 +66,12 @@ void ConsoleThread::run() {
}
// enter V8 context
_context = V8DealerFeature::DEALER->enterContext(_vocbase, true);
if (_context == nullptr) {
LOG_TOPIC("921b8", FATAL, arangodb::Logger::FIXME) << "cannot acquire V8 context";
FATAL_ERROR_EXIT();
}
TRI_DEFER(V8DealerFeature::DEALER->exitContext(_context));
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createAdminScriptContext();
V8ContextGuard guard(_vocbase, securityContext);
// work
try {
inner();
inner(guard);
} catch (char const* error) {
if (strcmp(error, USER_ABORTED) != 0) {
LOG_TOPIC("6e7fd", ERR, arangodb::Logger::FIXME) << error;
@ -91,11 +85,11 @@ void ConsoleThread::run() {
_applicationServer->beginShutdown();
}
void ConsoleThread::inner() {
void ConsoleThread::inner(V8ContextGuard const& guard) {
// flush all log output before we print the console prompt
Logger::flush();
v8::Isolate* isolate = _context->_isolate;
v8::Isolate* isolate = guard.isolate();
v8::HandleScope globalScope(isolate);
// run the shell
@ -105,7 +99,7 @@ void ConsoleThread::inner() {
v8::Local<v8::String> name(TRI_V8_ASCII_STRING(isolate, TRI_V8_SHELL_COMMAND_NAME));
auto localContext = v8::Local<v8::Context>::New(isolate, _context->_context);
auto localContext = v8::Local<v8::Context>::New(isolate, guard.context()->_context);
localContext->Enter();
{
v8::Context::Scope contextScope(localContext);

4
arangod/RestServer/ConsoleThread.h
View File

@ -30,6 +30,7 @@
struct TRI_vocbase_t;
namespace arangodb {
class V8ContextGuard;
class V8LineEditor;
}
@ -59,11 +60,10 @@ class ConsoleThread final : public Thread {
void userAbort() { _userAborted.store(true); }
private:
void inner();
void inner(V8ContextGuard const&);
private:
application_features::ApplicationServer* _applicationServer;
V8Context* _context;
TRI_vocbase_t* _vocbase;
std::atomic<bool> _userAborted;
};

15
arangod/RestServer/ScriptFeature.cpp
View File

@ -27,6 +27,7 @@
#include "ProgramOptions/Section.h"
#include "RestServer/ServerFeature.h"
#include "RestServer/SystemDatabaseFeature.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-conv.h"
#include "V8/v8-globals.h"
#include "V8/v8-utils.h"
@ -68,21 +69,15 @@ int ScriptFeature::runScript(std::vector<std::string> const& scripts) {
auto* sysDbFeature =
arangodb::application_features::ApplicationServer::lookupFeature<arangodb::SystemDatabaseFeature>();
auto database = sysDbFeature->use();
V8Context* context = V8DealerFeature::DEALER->enterContext(database.get(), true);
if (context == nullptr) {
LOG_TOPIC("d01d3", FATAL, arangodb::Logger::FIXME) << "cannot acquire V8 context";
FATAL_ERROR_EXIT();
}
TRI_DEFER(V8DealerFeature::DEALER->exitContext(context));
auto isolate = context->_isolate;
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createAdminScriptContext();
V8ContextGuard guard(database.get(), securityContext);
auto isolate = guard.isolate();
{
v8::HandleScope globalScope(isolate);
auto localContext = v8::Local<v8::Context>::New(isolate, context->_context);
auto localContext = v8::Local<v8::Context>::New(isolate, guard.context()->_context);
localContext->Enter();
{
v8::Context::Scope contextScope(localContext);

4
arangod/RestServer/UpgradeFeature.cpp
View File

@ -180,7 +180,9 @@ void UpgradeFeature::upgradeDatabase() {
bool ignoreDatafileErrors = false;
{
VPackBuilder options = server()->options(std::unordered_set<std::string>());
VPackBuilder options = server()->options([](std::string const& name) {
return (name.find("database.ignore-datafile-errors") != std::string::npos);
});
VPackSlice s = options.slice();
if (s.get("database.ignore-datafile-errors").isBoolean()) {
ignoreDatafileErrors = s.get("database.ignore-datafile-errors").getBool();

6
arangod/RestServer/arangod.cpp
View File

@ -54,6 +54,7 @@
#include "ApplicationFeatures/TempFeature.h"
#include "ApplicationFeatures/V8Phase.h"
#include "ApplicationFeatures/V8PlatformFeature.h"
#include "ApplicationFeatures/V8SecurityFeature.h"
#include "ApplicationFeatures/VersionFeature.h"
#include "Aql/AqlFunctionFeature.h"
#include "Aql/OptimizerRulesFeature.h"
@ -64,6 +65,7 @@
#include "Cluster/ReplicationTimeoutFeature.h"
#include "GeneralServer/AuthenticationFeature.h"
#include "GeneralServer/GeneralServerFeature.h"
#include "GeneralServer/ServerSecurityFeature.h"
#include "Logger/LoggerBufferFeature.h"
#include "Logger/LoggerFeature.h"
#include "Pregel/PregelFeature.h"
@ -203,6 +205,7 @@ static int runServer(int argc, char** argv, ArangoGlobalContext& context) {
server.addFeature(new ScriptFeature(server, &ret));
server.addFeature(new ServerFeature(server, &ret));
server.addFeature(new ServerIdFeature(server));
server.addFeature(new ServerSecurityFeature(server));
server.addFeature(new ShardingFeature(server));
server.addFeature(new ShellColorsFeature(server));
server.addFeature(new ShutdownFeature(server, {"Script"}));
@ -211,12 +214,13 @@ static int runServer(int argc, char** argv, ArangoGlobalContext& context) {
server.addFeature(new StorageEngineFeature(server));
server.addFeature(new SystemDatabaseFeature(server));
server.addFeature(new TempFeature(server, name));
server.addFeature(new transaction::ManagerFeature(server));
server.addFeature(new TraverserEngineRegistryFeature(server));
server.addFeature(new TtlFeature(server));
server.addFeature(new UpgradeFeature(server, &ret, nonServerFeatures));
server.addFeature(new V8DealerFeature(server));
server.addFeature(new V8PlatformFeature(server));
server.addFeature(new V8SecurityFeature(server));
server.addFeature(new transaction::ManagerFeature(server));
server.addFeature(new VersionFeature(server));
server.addFeature(new ViewTypesFeature(server));
server.addFeature(new aql::AqlFunctionFeature(server));

64
arangod/RocksDBEngine/RocksDBCollection.cpp
View File

@ -315,7 +315,7 @@ std::shared_ptr<Index> RocksDBCollection::createIndex(VPackSlice const& info,
TRI_vocbase_t& vocbase = _logicalCollection.vocbase();
TRI_vocbase_col_status_e status;
Result res = vocbase.useCollection(&_logicalCollection, status);
if (res.fail()) {
THROW_ARANGO_EXCEPTION(res);
}
@ -381,9 +381,9 @@ std::shared_ptr<Index> RocksDBCollection::createIndex(VPackSlice const& info,
}
}
}
do {
// Step 3. add index to collection entry (for removal after a crash)
auto buildIdx =
std::make_shared<RocksDBBuilderIndex>(std::static_pointer_cast<RocksDBIndex>(idx));
@ -400,7 +400,7 @@ std::shared_ptr<Index> RocksDBCollection::createIndex(VPackSlice const& info,
res.reset(rocksutils::convertStatus(s));
break;
}
VPackBuilder builder;
builder.openObject();
for (auto const& pair : VPackObjectIterator(VPackSlice(ps.data()))) {
@ -421,7 +421,7 @@ std::shared_ptr<Index> RocksDBCollection::createIndex(VPackSlice const& info,
break;
}
}
// Step 4. fill index
const bool inBackground =
basics::VelocyPackHelper::getBooleanValue(info, StaticStrings::IndexInBackground, false);
@ -435,7 +435,7 @@ std::shared_ptr<Index> RocksDBCollection::createIndex(VPackSlice const& info,
break;
}
locker.lock(); // always lock to avoid inconsistencies
// Step 5. register in index list
WRITE_LOCKER(guard, _indexesLock);
if (inBackground) { // swap in actual index
@ -452,7 +452,7 @@ std::shared_ptr<Index> RocksDBCollection::createIndex(VPackSlice const& info,
#if USE_PLAN_CACHE
arangodb::aql::PlanCache::instance()->invalidate(_logicalCollection.vocbase());
#endif
// inBackground index might not recover selectivity estimate w/o sync
if (inBackground && !idx->unique() && idx->hasSelectivityEstimate()) {
engine->settingsManager()->sync(false);
@ -703,7 +703,7 @@ Result RocksDBCollection::truncate(transaction::Methods& trx, OperationOptions&
TRI_ASSERT(_objectId == RocksDBKey::objectId(iter->key()));
VPackSlice document(iter->value().data());
TRI_ASSERT(document.isObject());
// tmp may contain a pointer into rocksdb::WriteBuffer::_rep. This is
// a 'std::string' which might be realloc'ed on any Put/Delete operation
docBuffer.clear();
@ -720,7 +720,7 @@ Result RocksDBCollection::truncate(transaction::Methods& trx, OperationOptions&
state->prepareOperation(_logicalCollection.id(),
rid, // actual revision ID!!
TRI_VOC_DOCUMENT_OPERATION_REMOVE);
LocalDocumentId const docId = RocksDBKey::documentId(iter->key());
auto res = removeDocument(&trx, docId, docBuffer.slice(), options);
@ -738,7 +738,7 @@ Result RocksDBCollection::truncate(transaction::Methods& trx, OperationOptions&
guard.finish(hasPerformedIntermediateCommit);
trackWaitForSync(&trx, options);
}
// reset to previous value after truncate is finished
@ -762,7 +762,7 @@ Result RocksDBCollection::truncate(transaction::Methods& trx, OperationOptions&
}
return Result{};
}
LocalDocumentId RocksDBCollection::lookupKey(transaction::Methods* trx,
VPackSlice const& key) const {
TRI_ASSERT(key.isString());
@ -800,7 +800,7 @@ Result RocksDBCollection::read(transaction::Methods* trx,
if (!documentId.isSet()) {
return TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND;
} // found
std::string* buffer = result.setManaged();
rocksdb::PinnableSlice ps(buffer);
Result res = lookupDocumentVPack(trx, documentId, ps, /*readCache*/true, /*fillCache*/true);
@ -810,7 +810,7 @@ Result RocksDBCollection::read(transaction::Methods* trx,
} // else value is already assigned
result.setRevisionId(); // extracts id from buffer
}
return res;
}
@ -848,7 +848,7 @@ Result RocksDBCollection::insert(arangodb::transaction::Methods* trx,
OperationOptions& options,
bool /*lock*/, KeyLockInfo* /*keyLockInfo*/,
std::function<void()> const& cbDuringLock) {
bool const isEdgeCollection = (TRI_COL_TYPE_EDGE == _logicalCollection.type());
transaction::BuilderLeaser builder(trx);
@ -888,7 +888,7 @@ Result RocksDBCollection::insert(arangodb::transaction::Methods* trx,
}
}
}
LocalDocumentId const documentId = LocalDocumentId::create();
RocksDBSavePoint guard(trx, TRI_VOC_DOCUMENT_OPERATION_INSERT);
@ -900,7 +900,7 @@ Result RocksDBCollection::insert(arangodb::transaction::Methods* trx,
if (res.ok()) {
trackWaitForSync(trx, options);
if (options.returnNew) {
resultMdr.setManaged(newSlice.begin());
TRI_ASSERT(resultMdr.revisionId() == revisionId);
@ -913,7 +913,7 @@ Result RocksDBCollection::insert(arangodb::transaction::Methods* trx,
keyBuilder->size());
resultMdr.setRevisionId(revisionId);
}
bool hasPerformedIntermediateCommit = false;
res = state->addOperation(_logicalCollection.id(), revisionId,
TRI_VOC_DOCUMENT_OPERATION_INSERT,
@ -933,14 +933,14 @@ Result RocksDBCollection::update(arangodb::transaction::Methods* trx,
arangodb::velocypack::Slice const newSlice,
ManagedDocumentResult& resultMdr, OperationOptions& options,
bool /*lock*/, ManagedDocumentResult& previousMdr) {
VPackSlice keySlice = newSlice.get(StaticStrings::KeyString);
if (keySlice.isNone()) {
return TRI_ERROR_ARANGO_DOCUMENT_HANDLE_BAD;
} else if (!keySlice.isString()) {
return TRI_ERROR_ARANGO_DOCUMENT_KEY_BAD;
}
auto const oldDocumentId = primaryIndex()->lookupKey(trx, VPackStringRef(keySlice));
if (!oldDocumentId.isSet()) {
return TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND;
@ -980,7 +980,7 @@ Result RocksDBCollection::update(arangodb::transaction::Methods* trx,
TRI_voc_rid_t revisionId;
LocalDocumentId const newDocumentId = LocalDocumentId::create();
auto isEdgeCollection = (TRI_COL_TYPE_EDGE == _logicalCollection.type());
transaction::BuilderLeaser builder(trx);
res = mergeObjectsForUpdate(trx, oldDoc, newSlice, isEdgeCollection,
options.mergeObjects, options.keepNull,
@ -1043,14 +1043,14 @@ Result RocksDBCollection::replace(transaction::Methods* trx,
arangodb::velocypack::Slice const newSlice,
ManagedDocumentResult& resultMdr, OperationOptions& options,
bool /*lock*/, ManagedDocumentResult& previousMdr) {
VPackSlice keySlice = newSlice.get(StaticStrings::KeyString);
if (keySlice.isNone()) {
return TRI_ERROR_ARANGO_DOCUMENT_HANDLE_BAD;
} else if (!keySlice.isString()) {
return TRI_ERROR_ARANGO_DOCUMENT_KEY_BAD;
}
auto const oldDocumentId = primaryIndex()->lookupKey(trx, VPackStringRef(keySlice));
if (!oldDocumentId.isSet()) {
return TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND;
@ -1063,7 +1063,7 @@ Result RocksDBCollection::replace(transaction::Methods* trx,
if (res.fail()) {
return res;
}
TRI_ASSERT(previousPS.size() > 0);
VPackSlice const oldDoc(previousPS.data());
previousMdr.setRevisionId(transaction::helpers::extractRevFromDocument(oldDoc));
@ -1076,7 +1076,7 @@ Result RocksDBCollection::replace(transaction::Methods* trx,
return res;
}
}
// merge old and new values
TRI_voc_rid_t revisionId;
LocalDocumentId const newDocumentId = LocalDocumentId::create();
@ -1155,7 +1155,7 @@ Result RocksDBCollection::remove(transaction::Methods& trx, velocypack::Slice sl
if (!keySlice.isString()) {
return TRI_ERROR_ARANGO_DOCUMENT_KEY_BAD;
}
auto const documentId = primaryIndex()->lookupKey(&trx, VPackStringRef(keySlice));
if (!documentId.isSet()) {
return TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND;
@ -1168,7 +1168,7 @@ Result RocksDBCollection::remove(transaction::Methods& trx, velocypack::Slice sl
if (res.fail()) {
return res;
}
TRI_ASSERT(previousPS.size() > 0);
VPackSlice const oldDoc(previousPS.data());
previousMdr.setRevisionId(transaction::helpers::extractRevFromDocument(oldDoc));
@ -1203,7 +1203,7 @@ Result RocksDBCollection::remove(transaction::Methods& trx, velocypack::Slice sl
} else {
previousMdr.clearData();
}
bool hasPerformedIntermediateCommit = false;
res = state->addOperation(_logicalCollection.id(), newRevisionId(), TRI_VOC_DOCUMENT_OPERATION_REMOVE,
hasPerformedIntermediateCommit);
@ -1429,17 +1429,17 @@ arangodb::Result RocksDBCollection::lookupDocumentVPack(transaction::Methods* tr
lockTimeout = true; // we skip the insert in this case
}
}
RocksDBMethods* mthd = RocksDBTransactionState::toMethods(trx);
rocksdb::Status s = mthd->Get(RocksDBColumnFamily::documents(), key->string(), &ps);
if (!s.ok()) {
LOG_TOPIC("f63dd", DEBUG, Logger::ENGINES)
<< "NOT FOUND rev: " << documentId.id() << " trx: " << trx->state()->id()
<< " objectID " << _objectId << " name: " << _logicalCollection.name();
return res.reset(rocksutils::convertStatus(s, rocksutils::document));
}
if (fillCache && useCache() && !lockTimeout) {
TRI_ASSERT(_cache != nullptr);
// write entry back to cache
@ -1459,7 +1459,7 @@ arangodb::Result RocksDBCollection::lookupDocumentVPack(transaction::Methods* tr
}
}
}
return res;
}
@ -1480,7 +1480,7 @@ bool RocksDBCollection::lookupDocumentVPack(transaction::Methods* trx,
return true;
}
}
transaction::StringLeaser buffer(trx);
rocksdb::PinnableSlice ps(buffer.get());
Result res = lookupDocumentVPack(trx, documentId, ps, /*readCache*/false, withCache);

122
arangod/Transaction/Manager.cpp
View File

@ -40,7 +40,7 @@
namespace arangodb {
namespace transaction {
namespace {
struct MGMethods final : arangodb::transaction::Methods {
MGMethods(std::shared_ptr<arangodb::transaction::Context> const& ctx,
@ -50,7 +50,7 @@ namespace {
}
};
}
// register a list of failed transactions
void Manager::registerFailedTransactions(std::unordered_set<TRI_voc_tid_t> const& failedTransactions) {
TRI_ASSERT(_keepTransactionData);
@ -83,7 +83,7 @@ void Manager::registerTransaction(TRI_voc_tid_t transactionId,
const size_t bucket = getBucket(transactionId);
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
try {
// insert into currently running list of transactions
_transactions[bucket]._activeTransactions.emplace(transactionId, std::move(data));
@ -178,7 +178,7 @@ Manager::ManagedTrx::~ManagedTrx() {
// but we are in a destructor and must never throw from here
}
}
using namespace arangodb;
/// @brief register a transaction shard
@ -189,25 +189,25 @@ void Manager::registerAQLTrx(TransactionState* state) {
const size_t bucket = getBucket(state->id());
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
auto& buck = _transactions[bucket];
auto it = buck._managed.find(state->id());
if (it != buck._managed.end()) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_TRANSACTION_INTERNAL,
"transaction ID already used");
}
buck._managed.emplace(std::piecewise_construct,
std::forward_as_tuple(state->id()),
std::forward_as_tuple(MetaType::StandaloneAQL, state,
(defaultTTL + TRI_microtime())));
}
void Manager::unregisterAQLTrx(TRI_voc_tid_t tid) noexcept {
const size_t bucket = getBucket(tid);
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
auto& buck = _transactions[bucket];
auto it = buck._managed.find(tid);
if (it == buck._managed.end()) {
@ -216,7 +216,7 @@ void Manager::unregisterAQLTrx(TRI_voc_tid_t tid) noexcept {
return;
}
TRI_ASSERT(it->second.type == MetaType::StandaloneAQL);
/// we need to make sure no-one else is still using the TransactionState
if (!it->second.rwlock.writeLock(/*maxAttempts*/256)) {
LOG_TOPIC("9f7d7", ERR, Logger::TRANSACTIONS) << "a transaction is still in use";
@ -225,16 +225,16 @@ void Manager::unregisterAQLTrx(TRI_voc_tid_t tid) noexcept {
}
buck._managed.erase(it); // unlocking not necessary
}
Result Manager::createManagedTrx(TRI_vocbase_t& vocbase,
TRI_voc_tid_t tid, VPackSlice const trxOpts) {
Result res;
// parse the collections to register
if (!trxOpts.isObject() || !trxOpts.get("collections").isObject()) {
return res.reset(TRI_ERROR_BAD_PARAMETER, "missing 'collections'");
}
// extract the properties from the object
transaction::Options options;
options.fromVelocyPack(trxOpts);
@ -242,7 +242,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase,
return res.reset(TRI_ERROR_BAD_PARAMETER,
"<lockTimeout> needs to be positive");
}
auto fillColls = [](VPackSlice const& slice, std::vector<std::string>& cols) {
if (slice.isNone()) { // ignore nonexistant keys
return true;
@ -250,7 +250,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase,
cols.emplace_back(slice.copyString());
return true;
}
if (slice.isArray()) {
for (VPackSlice val : VPackArrayIterator(slice)) {
if (!val.isString() || val.getStringLength() == 0) {
@ -270,10 +270,10 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase,
if (!isValid) {
return res.reset(TRI_ERROR_BAD_PARAMETER, "invalid 'collections' attribute");
}
return createManagedTrx(vocbase, tid, reads, writes, exclusives, options);
}
/// @brief create managed transaction
Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
std::vector<std::string> const& readCollections,
@ -283,7 +283,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
Result res;
const size_t bucket = getBucket(tid);
{ // quick check whether ID exists
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
@ -293,7 +293,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
return res.reset(TRI_ERROR_TRANSACTION_INTERNAL, "transaction ID already used");
}
}
std::unique_ptr<TransactionState> state;
try {
// now start our own transaction
@ -304,7 +304,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
}
TRI_ASSERT(state != nullptr);
TRI_ASSERT(state->id() == tid);
// lock collections
CollectionNameResolver resolver(vocbase);
auto lockCols = [&](std::vector<std::string> cols, AccessMode::Type mode) {
@ -315,10 +315,10 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
} else { // only support local collections / shards
cid = resolver.getCollectionIdLocal(cname);
}
if (cid == 0) {
// not found
res.reset(TRI_ERROR_ARANGO_DATA_SOURCE_NOT_FOUND,
res.reset(TRI_ERROR_ARANGO_DATA_SOURCE_NOT_FOUND,
std::string(TRI_errno_string(TRI_ERROR_ARANGO_DATA_SOURCE_NOT_FOUND)) + ":" + cname);
} else {
res.reset(state->addCollection(cid, cname, mode, /*nestingLevel*/0, false));
@ -340,7 +340,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
// no error set. so it must be "data source not found"
return res.reset(TRI_ERROR_ARANGO_DATA_SOURCE_NOT_FOUND);
}
// start the transaction
transaction::Hints hints;
hints.set(transaction::Hints::Hint::LOCK_ENTIRELY);
@ -349,7 +349,7 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
if (res.fail()) {
return res;
}
{ // add transaction to bucket
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
@ -365,9 +365,9 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
std::forward_as_tuple(MetaType::Managed, state.release(),
expires));
}
LOG_TOPIC("d6806", DEBUG, Logger::TRANSACTIONS) << "created managed trx '" << tid << "'";
return res;
}
@ -375,23 +375,23 @@ Result Manager::createManagedTrx(TRI_vocbase_t& vocbase, TRI_voc_tid_t tid,
std::shared_ptr<transaction::Context> Manager::leaseManagedTrx(TRI_voc_tid_t tid,
AccessMode::Type mode) {
const size_t bucket = getBucket(tid);
int i = 0;
TransactionState* state = nullptr;
do {
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
auto it = _transactions[bucket]._managed.find(tid);
if (it == _transactions[bucket]._managed.end()) {
return nullptr;
}
ManagedTrx& mtrx = it->second;
if (mtrx.type == MetaType::Tombstone) {
return nullptr; // already committet this trx
}
if (AccessMode::isWriteOrExclusive(mode)) {
if (mtrx.type == MetaType::StandaloneAQL) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_TRANSACTION_DISALLOWED_OPERATION,
@ -411,11 +411,11 @@ std::shared_ptr<transaction::Context> Manager::leaseManagedTrx(TRI_voc_tid_t tid
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_TRANSACTION_DISALLOWED_OPERATION,
"transaction is already in use");
}
writeLocker.unlock(); // failure;
allTransactionsLocker.unlock();
std::this_thread::yield();
if (i++ > 32) {
LOG_TOPIC("9e972", DEBUG, Logger::TRANSACTIONS) << "waiting on trx lock " << tid;
i = 0;
@ -424,7 +424,7 @@ std::shared_ptr<transaction::Context> Manager::leaseManagedTrx(TRI_voc_tid_t tid
}
}
} while (true);
if (state) {
state->increaseNesting();
return std::make_shared<ManagedContext>(tid, state, mode);
@ -432,7 +432,7 @@ std::shared_ptr<transaction::Context> Manager::leaseManagedTrx(TRI_voc_tid_t tid
TRI_ASSERT(false); // should be unreachable
return nullptr;
}
void Manager::returnManagedTrx(TRI_voc_tid_t tid, AccessMode::Type mode) noexcept {
const size_t bucket = getBucket(tid);
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
@ -444,11 +444,11 @@ void Manager::returnManagedTrx(TRI_voc_tid_t tid, AccessMode::Type mode) noexcep
TRI_ASSERT(false);
return;
}
TRI_ASSERT(it->second.state != nullptr);
TRI_ASSERT(it->second.state->isEmbeddedTransaction());
it->second.state->decreaseNesting();
// garbageCollection might soft abort used transactions
const bool isSoftAborted = it->second.expires == 0;
if (!isSoftAborted) {
@ -461,7 +461,7 @@ void Manager::returnManagedTrx(TRI_voc_tid_t tid, AccessMode::Type mode) noexcep
} else {
TRI_ASSERT(false);
}
if (isSoftAborted) {
abortManagedTrx(tid);
}
@ -472,14 +472,14 @@ transaction::Status Manager::getManagedTrxStatus(TRI_voc_tid_t tid) const {
size_t bucket = getBucket(tid);
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
READ_LOCKER(writeLocker, _transactions[bucket]._lock);
auto it = _transactions[bucket]._managed.find(tid);
if (it == _transactions[bucket]._managed.end()) {
return transaction::Status::UNDEFINED;
}
auto const& mtrx = it->second;
auto const& mtrx = it->second;
if (mtrx.type == MetaType::Tombstone) {
return mtrx.finalStatus;
} else if (mtrx.expires > TRI_microtime() && mtrx.state != nullptr) {
@ -492,7 +492,7 @@ transaction::Status Manager::getManagedTrxStatus(TRI_voc_tid_t tid) const {
return transaction::Status::ABORTED;
}
}
Result Manager::commitManagedTrx(TRI_voc_tid_t tid) {
return updateTransaction(tid, transaction::Status::COMMITTED, false);
}
@ -506,32 +506,32 @@ Result Manager::updateTransaction(TRI_voc_tid_t tid,
bool clearServers) {
TRI_ASSERT(status == transaction::Status::COMMITTED ||
status == transaction::Status::ABORTED);
LOG_TOPIC("7ad2f", DEBUG, Logger::TRANSACTIONS) << "managed trx '" << tid
LOG_TOPIC("7bd2f", DEBUG, Logger::TRANSACTIONS) << "managed trx '" << tid
<< " updating to '" << status << "'";
Result res;
const size_t bucket = getBucket(tid);
bool wasExpired = false;
std::unique_ptr<TransactionState> state;
{
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
auto& buck = _transactions[bucket];
auto it = buck._managed.find(tid);
if (it == buck._managed.end()) {
return res.reset(TRI_ERROR_TRANSACTION_NOT_FOUND);
}
ManagedTrx& mtrx = it->second;
TRY_WRITE_LOCKER(tryGuard, mtrx.rwlock);
if (!tryGuard.isLocked()) {
return res.reset(TRI_ERROR_TRANSACTION_DISALLOWED_OPERATION,
"transaction is in use");
}
if (mtrx.type == MetaType::StandaloneAQL) {
return res.reset(TRI_ERROR_TRANSACTION_DISALLOWED_OPERATION,
"not allowed to change an AQL transaction");
@ -547,13 +547,13 @@ Result Manager::updateTransaction(TRI_voc_tid_t tid,
return res.reset(TRI_ERROR_TRANSACTION_DISALLOWED_OPERATION, msg);
}
}
double now = TRI_microtime();
if (mtrx.expires < now) {
status = transaction::Status::ABORTED;
wasExpired = true;
}
state.reset(mtrx.state);
mtrx.state = nullptr;
mtrx.type = MetaType::Tombstone;
@ -561,12 +561,12 @@ Result Manager::updateTransaction(TRI_voc_tid_t tid,
mtrx.finalStatus = status;
// it is sufficient to pretend that the operation already succeeded
}
TRI_ASSERT(state);
if (!state) { // this should never happen
return res.reset(TRI_ERROR_INTERNAL, "managed trx in an invalid state");
}
auto abortTombstone = [&] { // set tombstone entry to aborted
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
WRITE_LOCKER(writeLocker, _transactions[bucket]._lock);
@ -580,10 +580,10 @@ Result Manager::updateTransaction(TRI_voc_tid_t tid,
abortTombstone();
return res.reset(TRI_ERROR_TRANSACTION_ABORTED, "transaction was not running");
}
auto ctx = std::make_shared<ManagedContext>(tid, state.get(), AccessMode::Type::NONE);
state.release(); // now owned by ctx
transaction::Options trxOpts;
MGMethods trx(ctx, trxOpts);
TRI_ASSERT(trx.state()->isRunning());
@ -605,7 +605,7 @@ Result Manager::updateTransaction(TRI_voc_tid_t tid,
}
TRI_ASSERT(!trx.state()->isRunning());
}
return res;
}
@ -677,17 +677,17 @@ bool Manager::garbageCollect(bool abortAll) {
/// @brief abort all transactions matching
bool Manager::abortManagedTrx(std::function<bool(TransactionState const&)> cb) {
SmallVector<TRI_voc_tid_t, 64>::allocator_type::arena_type arena;
SmallVector<TRI_voc_tid_t, 64> toAbort{arena};
READ_LOCKER(allTransactionsLocker, _allTransactionsLock);
for (size_t bucket = 0; bucket < numBuckets; ++bucket) {
READ_LOCKER(locker, _transactions[bucket]._lock);
auto it = _transactions[bucket]._managed.begin();
while (it != _transactions[bucket]._managed.end()) {
ManagedTrx& mtrx = it->second;
if (mtrx.type == MetaType::Managed) {
TRI_ASSERT(mtrx.state != nullptr);
@ -696,11 +696,11 @@ bool Manager::abortManagedTrx(std::function<bool(TransactionState const&)> cb) {
toAbort.emplace_back(it->first);
}
}
++it; // next
}
}
for (TRI_voc_tid_t tid : toAbort) {
Result res = updateTransaction(tid, Status::ABORTED, /*clearSrvs*/true);
if (res.fail()) {

11
arangod/V8Server/FoxxQueuesFeature.h
View File

@ -34,6 +34,15 @@ class FoxxQueuesFeature final : public application_features::ApplicationFeature
void collectOptions(std::shared_ptr<options::ProgramOptions>) override final;
void validateOptions(std::shared_ptr<options::ProgramOptions>) override final;
/// @brief return poll interval for foxx queues. returns a negative number if
/// foxx queues are turned off
double pollInterval() const {
if (!_enabled) {
return -1.0;
}
return _pollInterval;
}
private:
double _pollInterval;
bool _enabled;
@ -41,4 +50,4 @@ class FoxxQueuesFeature final : public application_features::ApplicationFeature
} // namespace arangodb
#endif
#endif

20
arangod/V8Server/V8Context.cpp
View File

@ -37,16 +37,6 @@ std::string const GlobalContextMethods::CodeReloadRouting =
std::string const GlobalContextMethods::CodeReloadAql =
"try { require(\"@arangodb/aql\").reload(); } catch (err) { }";
std::string const GlobalContextMethods::CodeCollectGarbage =
"require(\"internal\").wait(0.01, true);";
std::string const GlobalContextMethods::CodeBootstrapCoordinator =
"require('internal').loadStartup('server/bootstrap/autoload.js').startup();"
"require('internal').loadStartup('server/bootstrap/routing.js').startup();";
std::string const GlobalContextMethods::CodeWarmupExports =
"require(\"@arangodb/actions\").warmupExports()";
V8Context::V8Context(size_t id, v8::Isolate* isolate)
: _id(id),
_isolate(isolate),
@ -158,8 +148,11 @@ void V8Context::handleGlobalContextMethods() {
<< "executing global context method '" << func << "' for context " << _id;
TRI_GET_GLOBALS2(_isolate);
bool allowUseDatabase = v8g->_allowUseDatabase;
v8g->_allowUseDatabase = true;
// save old security context settings
JavaScriptSecurityContext old(v8g->_securityContext);
v8g->_securityContext = JavaScriptSecurityContext::createInternalContext();
try {
v8::TryCatch tryCatch(_isolate);
@ -180,7 +173,8 @@ void V8Context::handleGlobalContextMethods() {
<< "caught exception during global context method '" << func << "'";
}
v8g->_allowUseDatabase = allowUseDatabase;
// restore old security settings
v8g->_securityContext = old;
}
}

27
arangod/V8Server/V8Context.h
View File

@ -38,9 +38,6 @@ class GlobalContextMethods {
UNKNOWN = 0,
RELOAD_ROUTING,
RELOAD_AQL,
COLLECT_GARBAGE,
BOOTSTRAP_COORDINATOR,
WARMUP_EXPORTS
};
public:
@ -51,15 +48,6 @@ class GlobalContextMethods {
if (type == "reloadAql") {
return MethodType::RELOAD_AQL;
}
if (type == "collectGarbage") {
return MethodType::COLLECT_GARBAGE;
}
if (type == "bootstrapCoordinator") {
return MethodType::BOOTSTRAP_COORDINATOR;
}
if (type == "warmupExports") {
return MethodType::WARMUP_EXPORTS;
}
return MethodType::UNKNOWN;
}
@ -70,12 +58,6 @@ class GlobalContextMethods {
return "reloadRouting";
case MethodType::RELOAD_AQL:
return "reloadAql";
case MethodType::COLLECT_GARBAGE:
return "collectGarbage";
case MethodType::BOOTSTRAP_COORDINATOR:
return "bootstrapCoordinator";
case MethodType::WARMUP_EXPORTS:
return "warmupExports";
case MethodType::UNKNOWN:
default:
return "unknown";
@ -88,12 +70,6 @@ class GlobalContextMethods {
return CodeReloadRouting;
case MethodType::RELOAD_AQL:
return CodeReloadAql;
case MethodType::COLLECT_GARBAGE:
return CodeCollectGarbage;
case MethodType::BOOTSTRAP_COORDINATOR:
return CodeBootstrapCoordinator;
case MethodType::WARMUP_EXPORTS:
return CodeWarmupExports;
case MethodType::UNKNOWN:
default:
return StaticStrings::Empty;
@ -103,9 +79,6 @@ class GlobalContextMethods {
public:
static std::string const CodeReloadRouting;
static std::string const CodeReloadAql;
static std::string const CodeCollectGarbage;
static std::string const CodeBootstrapCoordinator;
static std::string const CodeWarmupExports;
};
class V8Context {

154
arangod/V8Server/V8DealerFeature.cpp
View File

@ -27,6 +27,7 @@
#include "Actions/actions.h"
#include "ApplicationFeatures/V8PlatformFeature.h"
#include "ApplicationFeatures/V8SecurityFeature.h"
#include "Basics/ArangoGlobalContext.h"
#include "Basics/ConditionLocker.h"
#include "Basics/FileUtils.h"
@ -43,6 +44,7 @@
#include "RestServer/SystemDatabaseFeature.h"
#include "Scheduler/SchedulerFeature.h"
#include "Transaction/V8Context.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-buffer.h"
#include "V8/v8-conv.h"
#include "V8/v8-globals.h"
@ -110,6 +112,7 @@ V8DealerFeature::V8DealerFeature(application_features::ApplicationServer& server
startsAfter("Action");
startsAfter("V8Platform");
startsAfter("V8Security");
}
void V8DealerFeature::collectOptions(std::shared_ptr<ProgramOptions> options) {
@ -181,6 +184,11 @@ void V8DealerFeature::collectOptions(std::shared_ptr<ProgramOptions> options) {
void V8DealerFeature::validateOptions(std::shared_ptr<ProgramOptions> options) {
ProgramOptions::ProcessingResult const& result = options->processingResult();
V8SecurityFeature* v8security =
application_features::ApplicationServer::getFeature<V8SecurityFeature>(
"V8Security");
TRI_ASSERT(v8security != nullptr);
// DBServer and Agent don't need JS. Agent role handled in AgencyFeature
if (ServerState::instance()->getRole() == ServerState::RoleEnum::ROLE_DBSERVER &&
(!result.touched("console") ||
@ -213,6 +221,8 @@ void V8DealerFeature::validateOptions(std::shared_ptr<ProgramOptions> options) {
}
ctx->normalizePath(_startupDirectory, "javascript.startup-directory", true);
v8security->addToInternalWhitelist(_startupDirectory, FSAccessType::READ);
ctx->normalizePath(_moduleDirectories, "javascript.module-directory", false);
// try to append the current version name to the startup directory,
@ -241,6 +251,7 @@ void V8DealerFeature::validateOptions(std::shared_ptr<ProgramOptions> options) {
// version-specific js path exists!
it = versionedPath;
}
v8security->addToInternalWhitelist(it, FSAccessType::READ);
}
// check whether app-path was specified
@ -253,6 +264,8 @@ void V8DealerFeature::validateOptions(std::shared_ptr<ProgramOptions> options) {
// Tests if this path is either a directory (ok) or does not exist (we create
// it in ::start) If it is something else this will throw an error.
ctx->normalizePath(_appPath, "javascript.app-path", false);
v8security->addToInternalWhitelist(_appPath, FSAccessType::READ);
v8security->addToInternalWhitelist(_appPath, FSAccessType::WRITE);
// use a minimum of 1 second for GC
if (_gcFrequency < 1) {
@ -759,7 +772,7 @@ void V8DealerFeature::loadJavaScriptFileInAllContexts(TRI_vocbase_t* vocbase,
TRI_DEFER(unblockDynamicContextCreation());
LOG_TOPIC("1364d", TRACE, Logger::V8) << "loading JavaScript file '" << file << "' in all ("
<< contexts.size() << ") V8 context";
<< contexts.size() << ") V8 contexts";
// now safely scan the local copy of the contexts
for (auto& context : contexts) {
@ -825,7 +838,7 @@ void V8DealerFeature::startGarbageCollection() {
}
void V8DealerFeature::prepareLockedContext(TRI_vocbase_t* vocbase, V8Context* context,
bool allowUseDatabase) {
JavaScriptSecurityContext const& securityContext) {
TRI_ASSERT(vocbase != nullptr);
// when we get here, we should have a context and an isolate
@ -845,9 +858,8 @@ void V8DealerFeature::prepareLockedContext(TRI_vocbase_t* vocbase, V8Context* co
TRI_GET_GLOBALS();
// initialize the context data
v8g->_query = nullptr;
v8g->_vocbase = vocbase;
v8g->_allowUseDatabase = allowUseDatabase;
v8g->_securityContext = securityContext;
try {
LOG_TOPIC("94226", TRACE, arangodb::Logger::V8)
@ -860,16 +872,9 @@ void V8DealerFeature::prepareLockedContext(TRI_vocbase_t* vocbase, V8Context* co
}
}
/// @brief forceContext == -1 means that any free context may be
/// picked, or a new one will be created if we have not exceeded
/// the maximum number of contexts
/// forceContext == -2 means that any free context may be picked,
/// or a new one will be created if we have not exceeded or exactly
/// reached the maximum number of contexts. this can be used to
/// force the creation of another context for high priority tasks
/// forceContext >= 0 means picking the context with that exact id
V8Context* V8DealerFeature::enterContext(TRI_vocbase_t* vocbase, bool allowUseDatabase,
ssize_t forceContext) {
/// @brief enter a V8 context
/// currently returns a nullptr if no context can be acquired in time
V8Context* V8DealerFeature::enterContext(TRI_vocbase_t* vocbase, JavaScriptSecurityContext const& securityContext) {
TRI_ASSERT(vocbase != nullptr);
if (_stopping) {
@ -890,75 +895,8 @@ V8Context* V8DealerFeature::enterContext(TRI_vocbase_t* vocbase, bool allowUseDa
V8Context* context = nullptr;
// this is for TESTING / DEBUGGING / INIT only
if (forceContext >= 0) {
size_t id = static_cast<size_t>(forceContext);
while (!_stopping) {
{
CONDITION_LOCKER(guard, _contextCondition);
if (_stopping) {
break;
}
for (auto it = _idleContexts.begin(); it != _idleContexts.end(); ++it) {
if ((*it)->id() == id) {
context = (*it);
_idleContexts.erase(it);
_busyContexts.emplace(context);
break;
}
}
if (context == nullptr) {
// still not found
for (auto it = _dirtyContexts.begin(); it != _dirtyContexts.end(); ++it) {
if ((*it)->id() == id) {
context = (*it);
_dirtyContexts.erase(it);
_busyContexts.emplace(context);
break;
}
}
}
if (context != nullptr) {
// found the context
TRI_ASSERT(guard.isLocked());
break;
}
// check if such context exists at all
bool found = false;
for (auto& it : _contexts) {
if (it->id() == id) {
found = true;
break;
}
}
if (!found) {
vocbase->release();
LOG_TOPIC("ba767", WARN, arangodb::Logger::V8)
<< "specified V8 context #" << id << " not found";
return nullptr;
}
}
LOG_TOPIC("603d8", DEBUG, arangodb::Logger::V8)
<< "waiting for V8 context #" << id << " to become available";
std::this_thread::sleep_for(std::chrono::microseconds(50 * 1000));
}
if (context == nullptr) {
vocbase->release();
return nullptr;
}
}
// look for a free context
else {
{
CONDITION_LOCKER(guard, _contextCondition);
while (_idleContexts.empty() && !_stopping) {
@ -1059,7 +997,7 @@ V8Context* V8DealerFeature::enterContext(TRI_vocbase_t* vocbase, bool allowUseDa
context->lockAndEnter();
context->assertLocked();
prepareLockedContext(vocbase, context, allowUseDatabase);
prepareLockedContext(vocbase, context, securityContext);
return context;
}
@ -1139,9 +1077,8 @@ void V8DealerFeature::cleanupLockedContext(V8Context* context) {
TRI_GET_GLOBALS();
// reset the context data; gc should be able to run without it
v8g->_query = nullptr;
v8g->_vocbase = nullptr;
v8g->_allowUseDatabase = false;
v8g->_securityContext.reset();
// now really exit
auto localContext = v8::Local<v8::Context>::New(isolate, context->_context);
@ -1246,8 +1183,10 @@ void V8DealerFeature::applyContextUpdate(V8Context* context) {
continue;
}
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createInternalContext();
context->lockAndEnter();
prepareLockedContext(vocbase, context, true);
prepareLockedContext(vocbase, context, securityContext);
TRI_DEFER(exitContextInternal(context));
{
@ -1474,7 +1413,7 @@ V8Context* V8DealerFeature::buildContext(size_t id) {
TRI_InitV8UserStructures(isolate, localContext);
TRI_InitV8Buffer(isolate);
TRI_InitV8Utils(isolate, localContext, _startupDirectory, modules);
TRI_InitV8DebugUtils(isolate, localContext);
TRI_InitV8ServerUtils(isolate);
TRI_InitV8Shell(isolate);
TRI_InitV8Ttl(isolate);
@ -1553,8 +1492,10 @@ bool V8DealerFeature::loadJavaScriptFileInContext(TRI_vocbase_t* vocbase,
return false;
}
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createInternalContext();
context->lockAndEnter();
prepareLockedContext(vocbase, context, true);
prepareLockedContext(vocbase, context, securityContext);
TRI_DEFER(exitContextInternal(context));
try {
@ -1650,26 +1591,49 @@ void V8DealerFeature::shutdownContext(V8Context* context) {
delete context;
}
V8ContextDealerGuard::V8ContextDealerGuard(Result& res, v8::Isolate*& isolate,
TRI_vocbase_t* vocbase, bool allowModification)
: _isolate(isolate), _context(nullptr), _active(isolate ? false : true) {
V8ContextGuard::V8ContextGuard(TRI_vocbase_t* vocbase,
JavaScriptSecurityContext const& securityContext)
: _isolate(nullptr), _context(nullptr) {
_context = V8DealerFeature::DEALER->enterContext(vocbase, securityContext);
if (_context == nullptr) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_RESOURCE_LIMIT,
"unable to acquire V8 context in time");
}
_isolate = _context->_isolate;
}
V8ContextGuard::~V8ContextGuard() {
if (_context) {
try {
V8DealerFeature::DEALER->exitContext(_context);
} catch (...) {
}
}
}
V8ConditionalContextGuard::V8ConditionalContextGuard(Result& res, v8::Isolate*& isolate,
TRI_vocbase_t* vocbase,
JavaScriptSecurityContext const& securityContext)
: _isolate(isolate),
_context(nullptr),
_active(isolate ? false : true) {
if (_active) {
if (!vocbase) {
res.reset(TRI_ERROR_INTERNAL,
"V8ContextDealerGuard - no vocbase provided");
"V8ConditionalContextGuard - no vocbase provided");
return;
}
_context = V8DealerFeature::DEALER->enterContext(vocbase, allowModification);
_context = V8DealerFeature::DEALER->enterContext(vocbase, securityContext);
if (!_context) {
res.reset(TRI_ERROR_INTERNAL,
"V8ContextDealerGuard - could not acquire context");
"V8ConditionalContextGuard - could not acquire context");
return;
}
isolate = _context->_isolate;
}
}
V8ContextDealerGuard::~V8ContextDealerGuard() {
V8ConditionalContextGuard::~V8ConditionalContextGuard() {
if (_active && _context) {
try {
V8DealerFeature::DEALER->exitContext(_context);

43
arangod/V8Server/V8DealerFeature.h
View File

@ -35,7 +35,7 @@
struct TRI_vocbase_t;
namespace arangodb {
class JavaScriptSecurityContext;
class Thread;
class V8Context;
@ -87,15 +87,9 @@ class V8DealerFeature final : public application_features::ApplicationFeature {
void loadJavaScriptFileInAllContexts(TRI_vocbase_t*, std::string const& file,
VPackBuilder* builder);
/// @brief forceContext == -1 means that any free context may be
/// picked, or a new one will be created if we have not exceeded
/// the maximum number of contexts
/// forceContext == -2 means that any free context may be picked,
/// or a new one will be created if we have not exceeded or exactly
/// reached the maximum number of contexts. this can be used to
/// force the creation of another context for high priority tasks
/// forceContext >= 0 means picking the context with that exact id
V8Context* enterContext(TRI_vocbase_t*, bool allowUseDatabase, ssize_t forceContext = -1);
/// @brief enter a V8 context
/// currently returns a nullptr if no context can be acquired in time
V8Context* enterContext(TRI_vocbase_t*, JavaScriptSecurityContext const& securityContext);
void exitContext(V8Context*);
void defineContextUpdate(std::function<void(v8::Isolate*, v8::Handle<v8::Context>, size_t)>,
@ -136,7 +130,7 @@ class V8DealerFeature final : public application_features::ApplicationFeature {
VPackBuilder* builder);
bool loadJavaScriptFileInContext(TRI_vocbase_t*, std::string const& file,
V8Context* context, VPackBuilder* builder);
void prepareLockedContext(TRI_vocbase_t*, V8Context*, bool allowUseDatabase);
void prepareLockedContext(TRI_vocbase_t*, V8Context*, JavaScriptSecurityContext const&);
void exitContextInternal(V8Context*);
void cleanupLockedContext(V8Context*);
void applyContextUpdate(V8Context* context);
@ -164,14 +158,31 @@ class V8DealerFeature final : public application_features::ApplicationFeature {
std::vector<std::pair<std::function<void(v8::Isolate*, v8::Handle<v8::Context>, size_t)>, TRI_vocbase_t*>> _contextUpdates;
};
/// @brief enters and exits a context and provides an isolate
/// throws an exception when no context can be provided
class V8ContextGuard {
public:
explicit V8ContextGuard(TRI_vocbase_t*, JavaScriptSecurityContext const&);
V8ContextGuard(V8ContextGuard const&) = delete;
V8ContextGuard& operator=(V8ContextGuard const&) = delete;
~V8ContextGuard();
v8::Isolate* isolate() const { return _isolate; }
V8Context* context() const { return _context; }
private:
v8::Isolate* _isolate;
V8Context* _context;
};
// enters and exits a context and provides an isolate
// in case the passed in isolate is a nullptr
class V8ContextDealerGuard {
class V8ConditionalContextGuard {
public:
explicit V8ContextDealerGuard(Result&, v8::Isolate*&, TRI_vocbase_t*, bool allowModification);
V8ContextDealerGuard(V8ContextDealerGuard const&) = delete;
V8ContextDealerGuard& operator=(V8ContextDealerGuard const&) = delete;
~V8ContextDealerGuard();
explicit V8ConditionalContextGuard(Result&, v8::Isolate*&, TRI_vocbase_t*, JavaScriptSecurityContext const&);
V8ConditionalContextGuard(V8ConditionalContextGuard const&) = delete;
V8ConditionalContextGuard& operator=(V8ConditionalContextGuard const&) = delete;
~V8ConditionalContextGuard();
private:
v8::Isolate*& _isolate;

159
arangod/V8Server/v8-actions.cpp
View File

@ -24,6 +24,8 @@
#include "v8-actions.h"
#include "Actions/ActionFeature.h"
#include "Actions/actions.h"
#include "ApplicationFeatures/ApplicationServer.h"
#include "ApplicationFeatures/V8SecurityFeature.h"
#include "Basics/MutexLocker.h"
#include "Basics/ReadLocker.h"
#include "Basics/StringUtils.h"
@ -34,15 +36,18 @@
#include "Cluster/ClusterComm.h"
#include "Cluster/ServerState.h"
#include "GeneralServer/GeneralServer.h"
#include "GeneralServer/ServerSecurityFeature.h"
#include "Logger/Logger.h"
#include "Rest/GeneralRequest.h"
#include "Rest/HttpRequest.h"
#include "Rest/HttpResponse.h"
#include "Utils/ExecContext.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-buffer.h"
#include "V8/v8-conv.h"
#include "V8/v8-utils.h"
#include "V8/v8-vpack.h"
#include "V8Server/FoxxQueuesFeature.h"
#include "V8Server/V8Context.h"
#include "V8Server/V8DealerFeature.h"
#include "V8Server/v8-vocbase.h"
@ -105,40 +110,19 @@ class v8_action_t final : public TRI_action_t {
void** data) override {
TRI_action_result_t result;
// allow use database execution in rest calls
bool allowUseDatabaseInRestActions = ActionFeature::ACTION->allowUseDatabase();
if (_allowUseDatabase) {
allowUseDatabaseInRestActions = true;
}
// this is for TESTING / DEBUGGING only - do not document this feature
ssize_t forceContext = -1;
bool found;
std::string const& c = request->header("x-arango-v8-context", found);
if (found && !c.empty()) {
forceContext = StringUtils::int32(c);
}
// allow use database execution in rest calls?
bool allowUseDatabase = _allowUseDatabase || ActionFeature::ACTION->allowUseDatabase();
// get a V8 context
V8Context* context = V8DealerFeature::DEALER->enterContext(vocbase, allowUseDatabaseInRestActions,
forceContext);
// note: the context might be nullptr in case of shut-down
if (context == nullptr) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_RESOURCE_LIMIT,
"unable to acquire V8 context in time");
}
TRI_DEFER(V8DealerFeature::DEALER->exitContext(context));
V8ContextGuard guard(vocbase, _isSystem ?
JavaScriptSecurityContext::createInternalContext() :
JavaScriptSecurityContext::createRestActionContext(allowUseDatabase));
// locate the callback
READ_LOCKER(readLocker, _callbacksLock);
{
auto it = _callbacks.find(context->_isolate);
auto it = _callbacks.find(guard.isolate());
if (it == _callbacks.end()) {
LOG_TOPIC("94556", WARN, arangodb::Logger::FIXME)
@ -159,17 +143,17 @@ class v8_action_t final : public TRI_action_t {
return result;
}
*data = (void*)context->_isolate;
*data = (void*)guard.isolate();
}
v8::HandleScope scope(context->_isolate);
auto localFunction = v8::Local<v8::Function>::New(context->_isolate, it->second);
v8::HandleScope scope(guard.isolate());
auto localFunction = v8::Local<v8::Function>::New(guard.isolate(), it->second);
// we can release the lock here already as no other threads will
// work in our isolate at this time
readLocker.unlock();
try {
result = ExecuteActionVocbase(vocbase, context->_isolate, this,
result = ExecuteActionVocbase(vocbase, guard.isolate(), this,
localFunction, request, response);
} catch (...) {
result.isValid = false;
@ -241,6 +225,14 @@ static void ParseActionOptions(v8::Isolate* isolate, TRI_v8_global_t* v8g,
} else {
action->_allowUseDatabase = false;
}
TRI_GET_GLOBAL_STRING(IsSystemKey);
if (TRI_HasProperty(context, isolate, options, IsSystemKey)) {
action->_isSystem =
TRI_ObjectToBoolean(isolate, options->Get(IsSystemKey));
} else {
action->_isSystem = false;
}
}
////////////////////////////////////////////////////////////////////////////////
@ -506,7 +498,7 @@ v8::Handle<v8::Object> TRI_RequestCppToV8(v8::Isolate* isolate,
req->Set(RequestTypeKey, HeadConstant);
break;
}
case rest::RequestType::GET:
case rest::RequestType::GET:
default: {
TRI_GET_GLOBAL_STRING(GetConstant);
req->Set(RequestTypeKey, GetConstant);
@ -562,7 +554,7 @@ v8::Handle<v8::Object> TRI_RequestCppToV8(v8::Isolate* isolate,
TRI_GET_GLOBAL_STRING(CookiesKey);
req->Set(CookiesKey, cookiesObject);
}
// copy suffix, which comes from the action:
std::vector<std::string> const& suffixes = request->decodedSuffixes();
std::vector<std::string> const& rawSuffixes = request->suffixes();
@ -823,7 +815,7 @@ static void ResponseV8ToCpp(v8::Isolate* isolate, TRI_v8_global_t const* v8g,
HttpResponse* httpResponse = dynamic_cast<HttpResponse*>(response);
httpResponse->body().appendText(content, length);
TRI_FreeString(content);
}
}
break;
case Endpoint::TransportType::VST: {
@ -835,7 +827,7 @@ static void ResponseV8ToCpp(v8::Isolate* isolate, TRI_v8_global_t const* v8g,
// create vpack from file
response->setContentType(rest::ContentType::TEXT);
response->setPayload(std::move(buffer), true);
}
}
break;
default:
@ -1020,6 +1012,15 @@ static void JS_DefineAction(v8::FunctionCallbackInfo<v8::Value> const& args) {
"defineAction(<name>, <callback>, <parameter>)");
}
V8SecurityFeature* v8security =
application_features::ApplicationServer::getFeature<V8SecurityFeature>(
"V8Security");
TRI_ASSERT(v8security != nullptr);
if (!v8security->isAllowedToDefineHttpAction(isolate)) {
TRI_V8_THROW_EXCEPTION_MESSAGE(TRI_ERROR_FORBIDDEN, "operation only allowed for internal scripts");
}
// extract the action name
TRI_Utf8ValueNFC utf8name(isolate, args[0]);
@ -1376,7 +1377,7 @@ static void JS_GetCurrentResponse(v8::FunctionCallbackInfo<v8::Value> const& arg
/// @brief stores the V8 actions function inside the global variable
////////////////////////////////////////////////////////////////////////////////
void TRI_InitV8Actions(v8::Isolate* isolate, v8::Handle<v8::Context> context) {
void TRI_InitV8Actions(v8::Isolate* isolate) {
v8::HandleScope scope(isolate);
// .............................................................................
@ -1388,7 +1389,7 @@ void TRI_InitV8Actions(v8::Isolate* isolate, v8::Handle<v8::Context> context) {
TRI_AddGlobalFunctionVocbase(
isolate,
TRI_V8_ASCII_STRING(isolate, "SYS_EXECUTE_GLOBAL_CONTEXT_FUNCTION"),
JS_ExecuteGlobalContextFunction);
JS_ExecuteGlobalContextFunction, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate,
"SYS_GET_CURRENT_REQUEST"),
@ -1644,12 +1645,83 @@ static void JS_DebugClearFailAt(v8::FunctionCallbackInfo<v8::Value> const& args)
TRI_V8_TRY_CATCH_END
}
void TRI_InitV8DebugUtils(v8::Isolate* isolate, v8::Handle<v8::Context> context) {
static void JS_IsFoxxApiDisabled(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate)
v8::HandleScope scope(isolate);
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
TRI_V8_RETURN_BOOL(security->isFoxxApiDisabled());
TRI_V8_TRY_CATCH_END
}
static void JS_IsFoxxStoreDisabled(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate)
v8::HandleScope scope(isolate);
ServerSecurityFeature* security =
application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
"ServerSecurity");
TRI_ASSERT(security != nullptr);
TRI_V8_RETURN_BOOL(security->isFoxxStoreDisabled());
TRI_V8_TRY_CATCH_END
}
static void JS_RunInRestrictedContext(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate)
v8::HandleScope scope(isolate);
if (args.Length() != 1 || !args[0]->IsFunction()) {
TRI_V8_THROW_EXCEPTION_USAGE(
"runInRestrictedContext(<function>)");
}
v8::Handle<v8::Function> action = v8::Local<v8::Function>::Cast(args[0]);
if (action.IsEmpty()) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_INTERNAL, "cannot cannot function instance for runInRestrictedContext");
}
TRI_GET_GLOBALS();
{
// take a copy of the previous security context
auto oldContext = v8g->_securityContext;
// patch security context
v8g->_securityContext = JavaScriptSecurityContext::createRestrictedContext();
// make sure the old context will be restored
auto guard = scopeGuard([&oldContext, &v8g]() {
v8g->_securityContext = oldContext;
});
v8::Handle<v8::Object> current = isolate->GetCurrentContext()->Global();
v8::Handle<v8::Value> callArgs[] = {v8::Null(isolate)};
v8::Handle<v8::Value> rv = action->Call(current, 0, callArgs);
TRI_V8_RETURN(rv);
}
TRI_V8_TRY_CATCH_END
}
void TRI_InitV8ServerUtils(v8::Isolate* isolate) {
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "SYS_IS_FOXX_API_DISABLED"), JS_IsFoxxApiDisabled, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "SYS_IS_FOXX_STORE_DISABLED"), JS_IsFoxxStoreDisabled, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "SYS_RUN_IN_RESTRICTED_CONTEXT"), JS_RunInRestrictedContext, true);
// debugging functions
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate,
"SYS_DEBUG_CLEAR_FAILAT"),
JS_DebugClearFailAt);
#ifdef ARANGODB_ENABLE_FAILURE_TESTS
TRI_AddGlobalFunctionVocbase(
isolate, TRI_V8_ASCII_STRING(isolate, "SYS_DEBUG_SEGFAULT"), JS_DebugSegfault);
@ -1666,4 +1738,15 @@ void TRI_InitV8DebugUtils(v8::Isolate* isolate, v8::Handle<v8::Context> context)
"SYS_DEBUG_SHOULD_FAILAT"),
JS_DebugShouldFailAt);
#endif
// poll interval for Foxx queues
FoxxQueuesFeature* foxxQueuesFeature =
application_features::ApplicationServer::getFeature<FoxxQueuesFeature>(
"FoxxQueues");
isolate->GetCurrentContext()->Global()
->DefineOwnProperty(TRI_IGETC,
TRI_V8_ASCII_STRING(isolate, "FOXX_QUEUES_POLL_INTERVAL"),
v8::Number::New(isolate, foxxQueuesFeature->pollInterval()), v8::ReadOnly)
.FromMaybe(false); // ignore result
}

4
arangod/V8Server/v8-actions.h
View File

@ -40,8 +40,8 @@ v8::Handle<v8::Object> TRI_RequestCppToV8(v8::Isolate* isolate,
arangodb::GeneralRequest* request,
TRI_action_t const* action);
void TRI_InitV8Actions(v8::Isolate* isolate, v8::Handle<v8::Context> context);
void TRI_InitV8Actions(v8::Isolate* isolate);
void TRI_InitV8DebugUtils(v8::Isolate* isolate, v8::Handle<v8::Context> context);
void TRI_InitV8ServerUtils(v8::Isolate* isolate);
#endif

5
arangod/V8Server/v8-dispatcher.cpp
View File

@ -140,6 +140,11 @@ static void JS_RegisterTask(v8::FunctionCallbackInfo<v8::Value> const& args) {
.FromMaybe(v8::Local<v8::Value>()));
}
if(isSystem && !v8g->_securityContext.isInternal()) {
TRI_V8_THROW_EXCEPTION_MESSAGE(TRI_ERROR_FORBIDDEN, "Only Internal Context may create System Tasks");
}
// offset in seconds into period or from now on if no period
double offset = 0.0;

8
arangod/V8Server/v8-query.cpp
View File

@ -27,6 +27,7 @@
#include "Basics/StaticStrings.h"
#include "Basics/VelocyPackHelper.h"
#include "Indexes/Index.h"
#include "RestServer/QueryRegistryFeature.h"
#include "StorageEngine/PhysicalCollection.h"
#include "Transaction/Helpers.h"
#include "Transaction/V8Context.h"
@ -58,9 +59,10 @@ using namespace arangodb::basics;
aql::QueryResultV8 AqlQuery(v8::Isolate* isolate, arangodb::LogicalCollection const* col,
std::string const& aql, std::shared_ptr<VPackBuilder> bindVars) {
TRI_ASSERT(col != nullptr);
auto queryRegistry = QueryRegistryFeature::registry();
TRI_ASSERT(queryRegistry != nullptr);
TRI_GET_GLOBALS();
// If we execute an AQL query from V8 we need to unset the nolock headers
arangodb::aql::Query query(true, col->vocbase(), arangodb::aql::QueryString(aql),
bindVars, nullptr, arangodb::aql::PART_MAIN);
@ -70,7 +72,7 @@ aql::QueryResultV8 AqlQuery(v8::Isolate* isolate, arangodb::LogicalCollection co
aql::QueryResultV8 queryResult;
while (true) {
auto state =
query.executeV8(isolate, static_cast<arangodb::aql::QueryRegistry*>(v8g->_queryRegistry),
query.executeV8(isolate, static_cast<arangodb::aql::QueryRegistry*>(queryRegistry),
queryResult);
if (state != aql::ExecutionState::WAITING) {
break;

31
arangod/V8Server/v8-statistics.cpp
View File

@ -24,6 +24,7 @@
#include "v8-statistics.h"
#include "ApplicationFeatures/ApplicationServer.h"
#include "ApplicationFeatures/V8SecurityFeature.h"
#include "Basics/Exceptions.h"
#include "Basics/StringUtils.h"
#include "Basics/process-utils.h"
@ -97,6 +98,16 @@ static void FillDistribution(v8::Isolate* isolate, v8::Handle<v8::Object> list,
static void JS_ServerStatistics(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate)
v8::HandleScope scope(isolate);
V8SecurityFeature* v8security =
application_features::ApplicationServer::getFeature<V8SecurityFeature>(
"V8Security");
TRI_ASSERT(v8security != nullptr);
if (v8security->isInternalModuleHardened(isolate)) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_FORBIDDEN,
"not allowed to provide this information");
}
ServerStatistics info = ServerStatistics::statistics();
@ -164,6 +175,16 @@ static void JS_EnabledStatistics(v8::FunctionCallbackInfo<v8::Value> const& args
static void JS_ClientStatistics(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate)
v8::HandleScope scope(isolate);
V8SecurityFeature* v8security =
application_features::ApplicationServer::getFeature<V8SecurityFeature>(
"V8Security");
TRI_ASSERT(v8security != nullptr);
if (v8security->isInternalModuleHardened(isolate)) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_FORBIDDEN,
"not allowed to provide this information");
}
v8::Handle<v8::Object> result = v8::Object::New(isolate);
@ -209,6 +230,16 @@ static void JS_ClientStatistics(v8::FunctionCallbackInfo<v8::Value> const& args)
static void JS_HttpStatistics(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate);
v8::HandleScope scope(isolate);
V8SecurityFeature* v8security =
application_features::ApplicationServer::getFeature<V8SecurityFeature>(
"V8Security");
TRI_ASSERT(v8security != nullptr);
if (v8security->isInternalModuleHardened(isolate)) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_FORBIDDEN,
"not allowed to provide this information");
}
v8::Handle<v8::Object> result = v8::Object::New(isolate);

209
arangod/V8Server/v8-vocbase.cpp
View File

@ -57,7 +57,7 @@
#include "Rest/Version.h"
#include "RestServer/ConsoleThread.h"
#include "RestServer/DatabaseFeature.h"
#include "RocksDBEngine/RocksDBEngine.h"
#include "RestServer/QueryRegistryFeature.h"
#include "Statistics/StatisticsFeature.h"
#include "StorageEngine/EngineSelectorFeature.h"
#include "StorageEngine/StorageEngine.h"
@ -65,7 +65,6 @@
#include "Utils/Events.h"
#include "Utils/ExecContext.h"
#include "V8/JSLoader.h"
#include "V8/V8LineEditor.h"
#include "V8/v8-conv.h"
#include "V8/v8-helper.h"
#include "V8/v8-utils.h"
@ -193,60 +192,6 @@ static void JS_EnableNativeBacktraces(v8::FunctionCallbackInfo<v8::Value> const&
TRI_V8_TRY_CATCH_END
}
extern V8LineEditor* theConsole;
////////////////////////////////////////////////////////////////////////////////
/// @brief starts a debugging console
////////////////////////////////////////////////////////////////////////////////
static void JS_Debug(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate);
v8::Local<v8::String> name(TRI_V8_ASCII_STRING(isolate, "debug loop"));
v8::Local<v8::String> debug(TRI_V8_ASCII_STRING(isolate, "debug"));
v8::Local<v8::Object> callerScope;
if (args.Length() >= 1) {
TRI_AddGlobalVariableVocbase(isolate, debug, args[0]);
}
MUTEX_LOCKER(mutexLocker, ConsoleThread::serverConsoleMutex);
V8LineEditor* console = ConsoleThread::serverConsole;
if (console != nullptr) {
while (true) {
ShellBase::EofType eof;
std::string input = console->prompt("debug> ", "debug>", eof);
if (eof == ShellBase::EOF_FORCE_ABORT) {
break;
}
if (input.empty()) {
continue;
}
console->addHistory(input);
{
v8::HandleScope scope(isolate);
v8::TryCatch tryCatch(isolate);
;
TRI_ExecuteJavaScriptString(isolate, isolate->GetCurrentContext(),
TRI_V8_STD_STRING(isolate, input), name, true);
if (tryCatch.HasCaught()) {
std::cout << TRI_StringifyV8Exception(isolate, &tryCatch);
}
}
}
}
TRI_V8_RETURN_UNDEFINED();
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief compare two UTF 16 strings
////////////////////////////////////////////////////////////////////////////////
@ -529,41 +474,6 @@ static void JS_ParseAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief registers a warning for the currently running AQL query
/// this function is called from aql.js
////////////////////////////////////////////////////////////////////////////////
static void JS_WarningAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate);
v8::HandleScope scope(isolate);
if (args.Length() != 2) {
TRI_V8_THROW_EXCEPTION_USAGE("AQL_WARNING(<code>, <message>)");
}
// get the query string
if (!args[1]->IsString()) {
TRI_V8_THROW_TYPE_ERROR("expecting string for <message>");
}
TRI_GET_GLOBALS();
if (v8g->_query != nullptr) {
// only register the error if we have a query...
// note: we may not have a query if the AQL functions are called without
// a query, e.g. during tests
int code = static_cast<int>(TRI_ObjectToInt64(isolate, args[0]));
std::string const message = TRI_ObjectToString(isolate, args[1]);
auto query = static_cast<arangodb::aql::Query*>(v8g->_query);
query->registerWarning(code, message.c_str());
}
TRI_V8_RETURN_UNDEFINED();
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief explains an AQL query
////////////////////////////////////////////////////////////////////////////////
@ -706,10 +616,12 @@ static void JS_ExecuteAqlJson(v8::FunctionCallbackInfo<v8::Value> const& args) {
}
}
TRI_GET_GLOBALS();
auto queryRegistry = QueryRegistryFeature::registry();
TRI_ASSERT(queryRegistry != nullptr);
arangodb::aql::Query query(true, vocbase, queryBuilder, options, arangodb::aql::PART_MAIN);
aql::QueryResult queryResult =
query.executeSync(static_cast<arangodb::aql::QueryRegistry*>(v8g->_queryRegistry));
query.executeSync(static_cast<arangodb::aql::QueryRegistry*>(queryRegistry));
if (queryResult.result.fail()) {
events::QueryDocument(vocbase.name(), queryBuilder->slice(), queryResult.result.errorNumber());
@ -812,8 +724,10 @@ static void JS_ExecuteAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
}
}
auto queryRegistry = QueryRegistryFeature::registry();
TRI_ASSERT(queryRegistry != nullptr);
// bind parameters will be freed by the query later
TRI_GET_GLOBALS();
arangodb::aql::Query query(true, vocbase, aql::QueryString(queryString),
bindVars, options, arangodb::aql::PART_MAIN);
@ -823,7 +737,7 @@ static void JS_ExecuteAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
aql::QueryResultV8 queryResult;
while (true) {
auto state =
query.executeV8(isolate, static_cast<arangodb::aql::QueryRegistry*>(v8g->_queryRegistry),
query.executeV8(isolate, static_cast<arangodb::aql::QueryRegistry*>(queryRegistry),
queryResult);
if (state != aql::ExecutionState::WAITING) {
break;
@ -1100,23 +1014,6 @@ static void JS_QueriesKillAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief whether or not a query is killed
////////////////////////////////////////////////////////////////////////////////
static void JS_QueryIsKilledAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate);
v8::HandleScope scope(isolate);
TRI_GET_GLOBALS();
if (v8g->_query != nullptr && static_cast<arangodb::aql::Query*>(v8g->_query)->killed()) {
TRI_V8_RETURN_TRUE();
}
TRI_V8_RETURN_FALSE();
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief configures the AQL query cache
////////////////////////////////////////////////////////////////////////////////
@ -1209,71 +1106,6 @@ static void JS_ThrowCollectionNotLoaded(v8::FunctionCallbackInfo<v8::Value> cons
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief sleeps and checks for query abortion in between
////////////////////////////////////////////////////////////////////////////////
static void JS_QuerySleepAql(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate);
v8::HandleScope scope(isolate);
// extract arguments
if (args.Length() != 1) {
TRI_V8_THROW_EXCEPTION_USAGE("sleep(<seconds>)");
}
TRI_GET_GLOBALS();
arangodb::aql::Query* query = static_cast<arangodb::aql::Query*>(v8g->_query);
if (query == nullptr) {
TRI_V8_THROW_EXCEPTION(TRI_ERROR_QUERY_NOT_FOUND);
}
double n = TRI_ObjectToDouble(isolate, args[0]);
double const until = TRI_microtime() + n;
while (TRI_microtime() < until) {
std::this_thread::sleep_for(std::chrono::microseconds(10000));
if (query != nullptr) {
if (query->killed()) {
TRI_V8_THROW_EXCEPTION(TRI_ERROR_QUERY_KILLED);
}
}
}
TRI_V8_RETURN_UNDEFINED();
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief hashes a V8 object
////////////////////////////////////////////////////////////////////////////////
static void JS_ObjectHash(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_V8_TRY_CATCH_BEGIN(isolate);
v8::HandleScope scope(isolate);
// extract arguments
if (args.Length() != 1) {
TRI_V8_THROW_EXCEPTION_USAGE("hash(<object>)");
}
VPackBuilder builder;
int res = TRI_V8ToVPack(isolate, builder, args[0], false);
if (res != TRI_ERROR_NO_ERROR) {
TRI_V8_THROW_EXCEPTION(res);
}
// throw away the top bytes so the hash value can safely be used
// without precision loss when storing in JavaScript etc.
uint64_t hash = builder.slice().normalizedHash() & 0x0007ffffffffffffULL;
TRI_V8_RETURN(v8::Number::New(isolate, static_cast<double>(hash)));
TRI_V8_TRY_CATCH_END
}
////////////////////////////////////////////////////////////////////////////////
/// @brief wraps a TRI_vocbase_t
////////////////////////////////////////////////////////////////////////////////
@ -1600,7 +1432,7 @@ static void JS_UseDatabase(v8::FunctionCallbackInfo<v8::Value> const& args) {
TRI_GET_GLOBALS();
if (!v8g->_allowUseDatabase) {
if (!v8g->_securityContext.canUseDatabase()) {
TRI_V8_THROW_EXCEPTION(TRI_ERROR_FORBIDDEN);
}
@ -1995,10 +1827,6 @@ void TRI_InitV8VocBridge(v8::Isolate* isolate, v8::Handle<v8::Context> context,
v8g->_transactionContext = new transaction::V8Context(vocbase, true);
static_cast<transaction::V8Context*>(v8g->_transactionContext)->makeGlobal();
// register the query registry
TRI_ASSERT(queryRegistry != nullptr);
v8g->_queryRegistry = queryRegistry;
// register the database
v8g->_vocbase = &vocbase;
@ -2082,9 +1910,6 @@ void TRI_InitV8VocBridge(v8::Isolate* isolate, v8::Handle<v8::Context> context,
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "AQL_PARSE"),
JS_ParseAql, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "AQL_WARNING"),
JS_WarningAql, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate,
"AQL_QUERIES_PROPERTIES"),
@ -2099,13 +1924,6 @@ void TRI_InitV8VocBridge(v8::Isolate* isolate, v8::Handle<v8::Context> context,
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "AQL_QUERIES_KILL"),
JS_QueriesKillAql, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "AQL_QUERY_SLEEP"),
JS_QuerySleepAql, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate,
"AQL_QUERY_IS_KILLED"),
JS_QueryIsKilledAql, true);
TRI_AddGlobalFunctionVocbase(
isolate, TRI_V8_ASCII_STRING(isolate, "AQL_QUERY_CACHE_PROPERTIES"),
JS_QueryCachePropertiesAql, true);
@ -2117,10 +1935,6 @@ void TRI_InitV8VocBridge(v8::Isolate* isolate, v8::Handle<v8::Context> context,
isolate, TRI_V8_ASCII_STRING(isolate, "AQL_QUERY_CACHE_INVALIDATE"),
JS_QueryCacheInvalidateAql, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate, "OBJECT_HASH"),
JS_ObjectHash, true);
TRI_AddGlobalFunctionVocbase(
isolate, TRI_V8_ASCII_STRING(isolate, "THROW_COLLECTION_NOT_LOADED"),
JS_ThrowCollectionNotLoaded, true);
@ -2155,9 +1969,6 @@ void TRI_InitV8VocBridge(v8::Isolate* isolate, v8::Handle<v8::Context> context,
"ENABLE_NATIVE_BACKTRACES"),
JS_EnableNativeBacktraces, true);
TRI_AddGlobalFunctionVocbase(isolate, TRI_V8_ASCII_STRING(isolate, "Debug"),
JS_Debug, true);
TRI_AddGlobalFunctionVocbase(isolate,
TRI_V8_ASCII_STRING(isolate,
"AUTHENTICATION_ENABLED"),

6
arangod/VocBase/Methods/AqlUserFunctions.cpp
View File

@ -34,6 +34,7 @@
#include "Transaction/V8Context.h"
#include "Utils/OperationOptions.h"
#include "Utils/SingleCollectionTransaction.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-globals.h"
#include "V8/v8-utils.h"
#include "V8Server/V8DealerFeature.h"
@ -232,8 +233,9 @@ Result arangodb::registerUserFunction(TRI_vocbase_t& vocbase, velocypack::Slice
{
ISOLATE;
bool throwV8Exception = (isolate != nullptr);
V8ContextDealerGuard dealerGuard(res, isolate, &vocbase, true /*allowModification*/
);
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createRestrictedContext();
V8ConditionalContextGuard contextGuard(res, isolate, &vocbase, securityContext);
if (res.fail()) {
return res;

11
arangod/VocBase/Methods/Collections.cpp
View File

@ -46,6 +46,7 @@
#include "Utils/ExecContext.h"
#include "Utils/OperationCursor.h"
#include "Utils/SingleCollectionTransaction.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-conv.h"
#include "V8/v8-utils.h"
#include "V8Server/V8Context.h"
@ -514,14 +515,10 @@ static int RenameGraphCollections(TRI_vocbase_t* vocbase, std::string const& old
buffer.appendJsonEncoded(newName.c_str(), newName.size());
buffer.appendText(");");
V8Context* context = dealer->enterContext(vocbase, false);
if (context == nullptr) {
LOG_TOPIC("8f0aa", WARN, Logger::FIXME) << "RenameGraphCollections: no V8 context";
return TRI_ERROR_OUT_OF_MEMORY;
}
TRI_DEFER(dealer->exitContext(context));
JavaScriptSecurityContext securityContext = JavaScriptSecurityContext::createInternalContext();
V8ContextGuard guard(vocbase, securityContext);
auto isolate = context->_isolate;
auto isolate = guard.isolate();
v8::HandleScope scope(isolate);
TRI_ExecuteJavaScriptString(isolate, isolate->GetCurrentContext(),
TRI_V8_ASCII_PAIR_STRING(isolate, buffer.c_str(),

70
arangod/VocBase/Methods/Databases.cpp
View File

@ -20,8 +20,8 @@
/// @author Simon Grätzer
////////////////////////////////////////////////////////////////////////////////
#include "Basics/Common.h"
#include "Databases.h"
#include "Basics/Common.h"
#include "Agency/AgencyComm.h"
#include "Basics/StringUtils.h"
@ -34,6 +34,7 @@
#include "RestServer/SystemDatabaseFeature.h"
#include "Utils/Events.h"
#include "Utils/ExecContext.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-utils.h"
#include "V8/v8-vpack.h"
#include "V8Server/V8Context.h"
@ -356,6 +357,8 @@ int dropDBCoordinator(std::string const& dbName) {
}
return TRI_ERROR_NO_ERROR;
}
const std::string dropError = "Error when dropping Datbase";
} // namespace
arangodb::Result Databases::drop(TRI_vocbase_t* systemVocbase, std::string const& dbName) {
@ -371,40 +374,44 @@ arangodb::Result Databases::drop(TRI_vocbase_t* systemVocbase, std::string const
int res;
V8DealerFeature* dealer = V8DealerFeature::DEALER;
if (dealer != nullptr && dealer->isEnabled()) {
V8Context* v8ctx = V8DealerFeature::DEALER->enterContext(systemVocbase, true);
if (v8ctx == nullptr) {
events::DropDatabase(dbName, TRI_ERROR_INTERNAL);
return Result(TRI_ERROR_INTERNAL, "Could not get v8 context");
}
TRI_DEFER(V8DealerFeature::DEALER->exitContext(v8ctx));
v8::Isolate* isolate = v8ctx->_isolate;
v8::HandleScope scope(isolate);
try {
JavaScriptSecurityContext securityContext =
JavaScriptSecurityContext::createInternalContext();
// clear collections in cache object
TRI_ClearObjectCacheV8(isolate);
V8ContextGuard guard(systemVocbase, securityContext);
v8::Isolate* isolate = guard.isolate();
if (ServerState::instance()->isCoordinator()) {
// If we are a coordinator in a cluster, we have to behave differently:
res = ::dropDBCoordinator(dbName);
} else {
res = DatabaseFeature::DATABASE->dropDatabase(dbName, false, true);
v8::HandleScope scope(isolate);
if (res != TRI_ERROR_NO_ERROR) {
events::DropDatabase(dbName, res);
return Result(res);
// clear collections in cache object
TRI_ClearObjectCacheV8(isolate);
if (ServerState::instance()->isCoordinator()) {
// If we are a coordinator in a cluster, we have to behave differently:
res = ::dropDBCoordinator(dbName);
} else {
res = DatabaseFeature::DATABASE->dropDatabase(dbName, false, true);
if (res != TRI_ERROR_NO_ERROR) {
events::DropDatabase(dbName, res);
return Result(res);
}
TRI_RemoveDatabaseTasksV8Dispatcher(dbName);
// run the garbage collection in case the database held some objects
// which can now be freed
TRI_RunGarbageCollectionV8(isolate, 0.25);
V8DealerFeature::DEALER->addGlobalContextMethod("reloadRouting");
}
TRI_RemoveDatabaseTasksV8Dispatcher(dbName);
// run the garbage collection in case the database held some objects which
// can now be freed
TRI_RunGarbageCollectionV8(isolate, 0.25);
TRI_ExecuteJavaScriptString(
isolate, isolate->GetCurrentContext(),
TRI_V8_ASCII_STRING(
isolate,
"require('internal').executeGlobalContextFunction('"
"reloadRouting')"),
TRI_V8_ASCII_STRING(isolate, "reload routing"), false);
} catch (arangodb::basics::Exception const& ex) {
events::DropDatabase(dbName, TRI_ERROR_INTERNAL);
return Result(ex.code(), dropError + ex.message());
} catch (std::exception const& ex) {
events::DropDatabase(dbName, TRI_ERROR_INTERNAL);
return Result(TRI_ERROR_INTERNAL, dropError + ex.what());
} catch (...) {
events::DropDatabase(dbName, TRI_ERROR_INTERNAL);
return Result(TRI_ERROR_INTERNAL, dropError);
}
} else {
if (ServerState::instance()->isCoordinator()) {
@ -412,7 +419,6 @@ arangodb::Result Databases::drop(TRI_vocbase_t* systemVocbase, std::string const
res = ::dropDBCoordinator(dbName);
} else {
res = DatabaseFeature::DATABASE->dropDatabase(dbName, false, true);
;
}
}

16
arangod/VocBase/Methods/Tasks.cpp
View File

@ -41,6 +41,7 @@
#include "Utils/ExecContext.h"
#include "Utils/OperationOptions.h"
#include "Utils/SingleCollectionTransaction.h"
#include "V8/JavaScriptSecurityContext.h"
#include "V8/v8-conv.h"
#include "V8/v8-utils.h"
#include "V8/v8-vpack.h"
@ -376,18 +377,15 @@ void Task::toVelocyPack(VPackBuilder& builder) const {
}
void Task::work(ExecContext const* exec) {
auto context = V8DealerFeature::DEALER->enterContext(&(_dbGuard->database()), _allowUseDatabase);
// note: the context might be 0 in case of shut-down
if (context == nullptr) {
return;
}
TRI_DEFER(V8DealerFeature::DEALER->exitContext(context));
JavaScriptSecurityContext securityContext =
_allowUseDatabase
? JavaScriptSecurityContext::createInternalContext() // internal context that may access internal data
: JavaScriptSecurityContext::createTaskContext(false /*_allowUseDatabase*/); // task context that has no access to dbs
V8ContextGuard guard(&(_dbGuard->database()), securityContext);
// now execute the function within this context
{
auto isolate = context->_isolate;
auto isolate = guard.isolate();
v8::HandleScope scope(isolate);
// get built-in Function constructor (see ECMA-262 5th edition 15.3.2)

5
arangosh/Export/ExportFeature.cpp
View File

@ -237,10 +237,7 @@ void ExportFeature::start() {
}
// successfully connected
std::cout << "Connected to ArangoDB '" << httpClient->getEndpointSpecification()
<< "', version " << httpClient->getServerVersion()
<< ", database: '" << client->databaseName() << "', username: '"
<< client->username() << "'" << std::endl;
std::cout << ClientFeature::buildConnectedMessage(httpClient->getEndpointSpecification(), httpClient->getServerVersion(), /*role*/ "", /*mode*/ "", client->databaseName(), client->username()) << std::endl;
uint64_t exportedSize = 0;

5
arangosh/Import/ImportFeature.cpp
View File

@ -292,10 +292,7 @@ void ImportFeature::start() {
bool createdDatabase = false;
auto successfulConnection = [&]() {
std::cout << "Connected to ArangoDB '"
<< _httpClient->getEndpointSpecification() << "', version "
<< versionString << ", database: '" << client->databaseName()
<< "', username: '" << client->username() << "'" << std::endl;
std::cout << ClientFeature::buildConnectedMessage(_httpClient->getEndpointSpecification(), versionString, /*role*/ "", /*mode*/ "", client->databaseName(), client->username()) << std::endl;
std::cout << "----------------------------------------" << std::endl;
std::cout << "database: " << client->databaseName() << std::endl;

14
arangosh/Shell/ClientFeature.cpp
View File

@ -329,6 +329,20 @@ void ClientFeature::stop() {
#endif
}
std::string ClientFeature::buildConnectedMessage(
std::string const& endpointSpecification,
std::string const& version,
std::string const& role,
std::string const& mode,
std::string const& databaseName,
std::string const& user
) {
return std::string("Connected to ArangoDB '") + endpointSpecification +
((version.empty() || version == "arango") ? "" : ", version: " + version) +
(role.empty() ? "" : " [" + role + ", " + mode + "]") +
", database: '" + databaseName + "', username: '" + user + "'";
}
int ClientFeature::runMain(int argc, char* argv[],
std::function<int(int argc, char* argv[])> const& mainFunc) {
try {

9
arangosh/Shell/ClientFeature.h
View File

@ -93,6 +93,15 @@ class ClientFeature final : public application_features::ApplicationFeature,
bool getWarnConnect() { return _warnConnect; }
static std::string buildConnectedMessage(
std::string const& endpointSpecification,
std::string const& version,
std::string const& role,
std::string const& mode,
std::string const& databaseName,
std::string const& user
);
static int runMain(int argc, char* argv[],
std::function<int(int argc, char* argv[])> const& mainFunc);

24
arangosh/Shell/V8ClientConnection.cpp
View File

@ -48,7 +48,7 @@
#include "V8/v8-vpack.h"
#include <iostream>
using namespace arangodb;
using namespace arangodb::application_features;
using namespace arangodb::basics;
@ -133,6 +133,11 @@ void V8ClientConnection::createConnection() {
_role = role.copyString();
}
}
if (!body.hasKey("version")) {
// if we don't get a version number in return, the server is
// probably running in hardened mode
return;
}
std::string const versionString =
VelocyPackHelper::getStringValue(body, "version", "");
std::pair<int, int> version = rest::Version::parseVersionString(versionString);
@ -233,11 +238,7 @@ void V8ClientConnection::reconnect(ClientFeature* client) {
if (isConnected() && _lastHttpReturnCode == static_cast<int>(rest::ResponseCode::OK)) {
LOG_TOPIC("2d416", INFO, arangodb::Logger::FIXME)
<< "Connected to ArangoDB "
<< "'" << endpointSpecification() << "', "
<< "version " << _version << " [" << _role << ", " << _mode << "], "
<< "database '" << _databaseName << "', "
<< "username: '" << client->username() << "'";
<< ClientFeature::buildConnectedMessage(endpointSpecification(), _version, _role, _mode, _databaseName, client->username());
} else {
if (client->getWarnConnect()) {
LOG_TOPIC("9d7ea", ERR, arangodb::Logger::FIXME)
@ -359,18 +360,15 @@ static void ClientConnection_ConstructorCallback(v8::FunctionCallbackInfo<v8::Va
if (v8connection->isConnected() &&
v8connection->lastHttpReturnCode() == (int)rest::ResponseCode::OK) {
LOG_TOPIC("9c8b4", INFO, arangodb::Logger::FIXME)
<< "Connected to ArangoDB "
<< "'" << v8connection->endpointSpecification() << "', "
<< "version " << v8connection->version() << " [" << v8connection->role() << ", " << v8connection->mode() << "], "
<< "database '" << v8connection->databaseName() << "', "
<< "username: '" << v8connection->username() << "'";
<< ClientFeature::buildConnectedMessage(v8connection->endpointSpecification(), v8connection->version(),
v8connection->role(), v8connection->mode(), v8connection->databaseName(),
v8connection->username());
} else {
std::string errorMessage =
"Could not connect. Error message: " + v8connection->lastErrorMessage();
TRI_V8_THROW_EXCEPTION_MESSAGE(TRI_SIMPLE_CLIENT_COULD_NOT_CONNECT,
errorMessage.c_str());
errorMessage);
}
TRI_V8_RETURN(WrapV8ClientConnection(isolate, v8connection.release()));

30
arangosh/Shell/V8ShellFeature.cpp
View File

@ -24,6 +24,7 @@
#include "ApplicationFeatures/ShellColorsFeature.h"
#include "ApplicationFeatures/V8PlatformFeature.h"
#include "ApplicationFeatures/V8SecurityFeature.h"
#include "Basics/ArangoGlobalContext.h"
#include "Basics/FileUtils.h"
#include "Basics/StringUtils.h"
@ -42,6 +43,7 @@
#include "V8/V8LineEditor.h"
#include "V8/v8-buffer.h"
#include "V8/v8-conv.h"
#include "V8/v8-globals.h"
#include "V8/v8-shell.h"
#include "V8/v8-utils.h"
#include "V8/v8-vpack.h"
@ -78,6 +80,7 @@ V8ShellFeature::V8ShellFeature(application_features::ApplicationServer& server,
startsAfter("BasicsPhase");
startsAfter("Console");
startsAfter("V8Platform");
startsAfter("V8Security");
startsAfter("Random");
}
@ -95,7 +98,7 @@ void V8ShellFeature::collectOptions(std::shared_ptr<ProgramOptions> options) {
options->addOption(
"--javascript.copy-directory",
"target directory to copy files from 'javascript.startup-directory' into"
"target directory to copy files from 'javascript.startup-directory' into "
"(only used when `--javascript.copy-installation` is enabled)",
new StringParameter(&_copyDirectory));
@ -151,6 +154,11 @@ void V8ShellFeature::start() {
v8::Isolate::Scope isolate_scope(_isolate);
v8::HandleScope handle_scope(_isolate);
auto* isolate = _isolate;
TRI_GET_GLOBALS();
v8g = TRI_CreateV8Globals(isolate);
v8g->_securityContext = arangodb::JavaScriptSecurityContext::createAdminScriptContext();
// create the global template
v8::Local<v8::ObjectTemplate> global = v8::ObjectTemplate::New(_isolate);
@ -340,14 +348,11 @@ bool V8ShellFeature::printHello(V8ClientConnection* v8connection) {
if (v8connection != nullptr) {
if (v8connection->isConnected() &&
v8connection->lastHttpReturnCode() == (int)rest::ResponseCode::OK) {
std::ostringstream is;
is << "Connected to ArangoDB '" << v8connection->endpointSpecification()
<< "' version: " << v8connection->version() << " [" << v8connection->role() << ", "
<< v8connection->mode() << "], database: '" << v8connection->databaseName()
<< "', username: '" << v8connection->username() << "'";
_console->printLine(is.str());
std::string msg = ClientFeature::buildConnectedMessage(
v8connection->endpointSpecification(), v8connection->version(),
v8connection->role(), v8connection->mode(), v8connection->databaseName(),
v8connection->username());
_console->printLine(msg);
if (v8connection->role() == "PRIMARY" || v8connection->role() == "DBSERVER") {
std::string msg("WARNING: You connected to a DBServer node, but operations in a cluster should be carried out via a Coordinator");
@ -998,6 +1003,10 @@ void V8ShellFeature::initGlobals() {
ctx->normalizePath(_startupDirectory, "javascript.startup-directory", true);
ctx->normalizePath(_moduleDirectories, "javascript.module-directory", false);
V8SecurityFeature* v8security =
application_features::ApplicationServer::getFeature<V8SecurityFeature>(
"V8Security");
// try to append the current version name to the startup directory,
// so instead of "/path/to/js" we will get "/path/to/js/3.4.0"
std::string const versionAppendix =
@ -1013,6 +1022,7 @@ void V8ShellFeature::initGlobals() {
// version-specific js path exists!
_startupDirectory = versionedPath;
}
v8security->addToInternalWhitelist(_startupDirectory, FSAccessType::READ);
for (auto& it : _moduleDirectories) {
versionedPath = basics::FileUtils::buildFilename(it, versionAppendix);
@ -1024,6 +1034,7 @@ void V8ShellFeature::initGlobals() {
// version-specific js path exists!
it = versionedPath;
}
v8security->addToInternalWhitelist(it, FSAccessType::READ); //expand
}
LOG_TOPIC("930d9", DEBUG, Logger::V8)
@ -1053,6 +1064,7 @@ void V8ShellFeature::initGlobals() {
if (_currentModuleDirectory) {
modules += sep + FileUtils::currentDirectory().result();
v8security->addToInternalWhitelist(FileUtils::currentDirectory().result(), FSAccessType::READ);
}
// we take the last entry in _startupDirectory as global path;

2
arangosh/Shell/arangosh.cpp
View File

@ -33,6 +33,7 @@
#include "ApplicationFeatures/ShutdownFeature.h"
#include "ApplicationFeatures/TempFeature.h"
#include "ApplicationFeatures/V8PlatformFeature.h"
#include "ApplicationFeatures/V8SecurityFeature.h"
#include "ApplicationFeatures/V8ShellPhase.h"
#include "ApplicationFeatures/VersionFeature.h"
#include "Basics/ArangoGlobalContext.h"
@ -80,6 +81,7 @@ int main(int argc, char* argv[]) {
// server.addFeature(new SslFeature(server));
server.addFeature(new TempFeature(server, name));
server.addFeature(new V8PlatformFeature(server));
server.addFeature(new V8SecurityFeature(server));
server.addFeature(new V8ShellFeature(server, name));
server.addFeature(new VersionFeature(server));

5
arangosh/Utils/ClientManager.cpp
View File

@ -110,6 +110,11 @@ Result ClientManager::getConnectedClient(std::unique_ptr<httpclient::SimpleHttpC
return {errorCode};
}
if (versionString.empty() || versionString == "arango") {
// server running in hardened mode?
return {TRI_ERROR_NO_ERROR};
}
if (logServerVersion) {
// successfully connected
LOG_TOPIC("06792", INFO, _topic) << "Server version: " << versionString;

2
etc/jenkins/arangosh.conf
View File

@ -3,6 +3,8 @@ authentication = false
[javascript]
startup-directory = ./js
allow-external-process-control = true
allow-port-testing = true
[javascript:enterprise]
module-directory = ./enterprise/js

2
etc/relative/arangosh.conf
View File

@ -11,6 +11,8 @@ file = -
[javascript]
startup-directory = ./js
allow-external-process-control = true
allow-port-testing = true
[javascript:enterprise]
module-directory = ./enterprise/js

4
etc/testing/arangosh.conf
View File

@ -1,2 +1,6 @@
[log]
file = -
[javascript]
allow-external-process-control = true
allow-port-testing = true

1
js/actions/_admin/app.js
View File

@ -38,6 +38,7 @@ const actions = require('@arangodb/actions');
actions.defineHttp({
url: '_admin/echo',
prefix: true,
isSystem: false,
callback: function (req, res) {
res.responseCode = actions.HTTP_OK;

18
js/actions/_admin/foxx/app.js
View File

@ -118,6 +118,9 @@ function resolveAppInfo (appInfo, refresh) {
return {source: appInfo};
}
// disable foxx manager routes / foxx-cli when foxx api is disabled
if (!require("internal").isFoxxApiDisabled()) {
// //////////////////////////////////////////////////////////////////////////////
// / @brief sets up a Foxx service
// //////////////////////////////////////////////////////////////////////////////
@ -125,6 +128,7 @@ function resolveAppInfo (appInfo, refresh) {
actions.defineHttp({
url: '_admin/foxx/setup',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -143,6 +147,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/teardown',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -161,6 +166,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/install',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -182,6 +188,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/uninstall',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -202,6 +209,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/replace',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -223,6 +231,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/upgrade',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -244,6 +253,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/configure',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -266,6 +276,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/configuration',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -283,6 +294,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/set-dependencies',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -305,6 +317,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/dependencies',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -334,6 +347,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/development',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -356,6 +370,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/tests',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -374,6 +389,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/foxx/script',
prefix: false,
isSystem: true,
callback: easyPostCallback({
body: true,
@ -385,3 +401,5 @@ actions.defineHttp({
}
})
});
} // end - if not foxx api disabled

15
js/actions/api-cluster.js
View File

@ -39,6 +39,7 @@ actions.defineHttp({
url: '_admin/cluster/removeServer',
allowUseDatabase: true,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (req.requestType !== actions.POST || !cluster.isCoordinator()) {
@ -149,6 +150,7 @@ actions.defineHttp({
url: '_admin/cluster/maintenance',
allowUseDatabase: true,
prefix: false,
isSystem: true,
callback: function (req, res) {
let role = global.ArangoServerState.role();
@ -238,6 +240,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/clusterNodeVersion',
prefix: false,
isSystem: true,
callback: function (req, res) {
if (req.requestType !== actions.GET ||
@ -297,6 +300,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/clusterNodeStats',
prefix: false,
isSystem: true,
callback: function (req, res) {
if (req.requestType !== actions.GET ||
@ -356,6 +360,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/clusterNodeEngine',
prefix: false,
isSystem: true,
callback: function (req, res) {
if (req.requestType !== actions.GET ||
@ -415,6 +420,7 @@ actions.defineHttp({
actions.defineHttp({
url: '_admin/clusterStatistics',
prefix: false,
isSystem: true,
callback: function (req, res) {
if (req.requestType !== actions.GET) {
@ -470,6 +476,7 @@ actions.defineHttp({
url: '_admin/cluster/health',
allowUseDatabase: true,
prefix: false,
isSystem: true,
callback: function (req, res) {
let role = global.ArangoServerState.role();
@ -718,6 +725,7 @@ actions.defineHttp({
url: '_admin/cluster/numberOfServers',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -843,6 +851,7 @@ actions.defineHttp({
url: '_admin/cluster/cleanOutServer',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -944,6 +953,7 @@ actions.defineHttp({
url: '_admin/cluster/queryAgencyJob',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -1032,6 +1042,7 @@ actions.defineHttp({
url: '_admin/cluster/moveShard',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -1122,6 +1133,7 @@ actions.defineHttp({
url: '_admin/cluster/collectionShardDistribution',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -1190,6 +1202,7 @@ actions.defineHttp({
url: '_admin/cluster/shardDistribution',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -1235,6 +1248,7 @@ actions.defineHttp({
url: '_admin/cluster/rebalanceShards',
allowUseDatabase: true,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {
@ -1301,6 +1315,7 @@ actions.defineHttp({
url: '_admin/cluster/supervisionState',
allowUseDatabase: false,
prefix: false,
isSystem: true,
callback: function (req, res) {
if (!cluster.isCoordinator()) {

10
js/actions/api-simple.js
View File

@ -73,6 +73,7 @@ function createCursorResponse (req, res, cursor) {
actions.defineHttp({
url: API + 'any',
isSystem: false,
callback: function (req, res) {
try {
@ -108,6 +109,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'near',
isSystem: false,
callback: function (req, res) {
try {
@ -172,6 +174,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'within',
isSystem: false,
callback: function (req, res) {
try {
@ -236,6 +239,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'within-rectangle',
isSystem: false,
callback: function (req, res) {
try {
@ -300,6 +304,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'fulltext',
isSystem: false,
callback: function (req, res) {
try {
@ -352,6 +357,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'first-example',
isSystem: false,
callback: function (req, res) {
try {
@ -394,6 +400,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'range',
isSystem: false,
callback: function (req, res) {
try {
@ -449,6 +456,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'remove-by-example',
isSystem: false,
callback: function (req, res) {
try {
@ -497,6 +505,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'replace-by-example',
isSystem: false,
callback: function (req, res) {
try {
@ -548,6 +557,7 @@ actions.defineHttp({
actions.defineHttp({
url: API + 'update-by-example',
isSystem: false,
callback: function (req, res) {
try {

1
js/actions/api-system.js
View File

@ -37,6 +37,7 @@ var actions = require('@arangodb/actions');
actions.defineHttp({
url: '',
prefix: true,
isSystem: false,
callback: function (req, res) {
req.absoluteUrl = function (url) {

1
js/actions/api-traversal.js
View File

@ -251,6 +251,7 @@ function post_api_traversal (req, res) {
actions.defineHttp({
url: '_api/traversal',
isSystem: false,
callback: function (req, res) {
try {

3
js/apps/system/_admin/aardvark/APP/aardvark.js
View File

@ -88,7 +88,8 @@ router.get('/config.js', function (req, res) {
ldapEnabled: ldapEnabled,
isCluster: cluster.isCluster(),
engine: db._engine().name,
statisticsEnabled: internal.enabledStatistics()
statisticsEnabled: internal.enabledStatistics(),
disableFoxxStore: internal.isFoxxStoreDisabled(),
})}`
);
})

22
js/apps/system/_admin/aardvark/APP/foxxes.js
View File

@ -24,10 +24,12 @@
// / @author Alan Plum
// //////////////////////////////////////////////////////////////////////////////
const internal = require('internal');
if (!internal.isFoxxApiDisabled()) {
const fs = require('fs');
const joi = require('joi');
const dd = require('dedent');
const internal = require('internal');
const crypto = require('@arangodb/crypto');
const errors = require('@arangodb').errors;
const FoxxManager = require('@arangodb/foxx/manager');
@ -359,12 +361,16 @@ foxxRouter.patch('/devel', function (req, res) {
`);
router.get('/fishbowl', function (req, res) {
try {
store.update();
} catch (e) {
console.warn('Failed to update Foxx store from GitHub.');
}
res.json(store.availableJson());
if (internal.isFoxxStoreDisabled()) {
res.json([]);
} else {
try {
store.update();
} catch (e) {
console.warn('Failed to update Foxx store from GitHub.');
}
res.json(store.availableJson());
}
})
.summary('List of all Foxx services submitted to the Foxx store.')
.description(dd`
@ -419,3 +425,5 @@ anonymousRouter.use('/docs', module.context.createDocumentationRouter((req, res)
indexFile: 'index.html'
};
}));
}

2
js/apps/system/_admin/aardvark/APP/frontend/build/app.min.js vendored
View File

File diff suppressed because one or more lines are too long

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/app.min.js.gz
View File

Binary file not shown.

8
js/apps/system/_admin/aardvark/APP/frontend/build/index-min.html
View File

@ -3135,12 +3135,6 @@ Upload a Foxx service bundle. The Foxx service bundle should be a zip archive co
</div> <% counter++; %> <% }); %> </div>
</div> <% } %> <% }); %> <div style="width: 100%; height: 70px;"> <% if (frontendConfig.db === '_system') { %> <button id="rebalanceShards" style="margin-top: 20px; margin-bottom: 20px; margin-right: 10px;" class="button-success pull-right">Rebalance Shards</button> <% } %> </div>
</div></script><script id="shellView.ejs" type="text/template"><div class="headerBar">
<a class="arangoHeader">JS Shell</a>
</div>
<div id="shell_workspace_header"/>
<div id="shell_workspace" class="shell_workspace">
<div id="replShell" class="replShell"/>
</div></script><script id="spotlightView.ejs" type="text/template"><div class="spotlightWrapper">
<div id="spotlight">
<input class="typeahead" type="text" placeholder="Search... ">
@ -3725,4 +3719,4 @@ var cutByResolution = function (str) {
</div>
</div></script><script id="warningList.ejs" type="text/template"> <% if (warnings.length > 0) { %> <div>
<ul> <% console.log(warnings); _.each(warnings, function(w) { console.log(w);%> <li><b><%=w.code%></b>: <%=w.message%></li> <% }); %> </ul>
</div> <% } %> </script></head><body><nav class="navbar" style="display: none"><div class="primary"><div class="navlogo"><a class="logo big" href="#"><img id="ArangoDBLogo" class="arangodbLogo" src="img/arangodb-edition-optimized.svg"></a><a class="logo small" href="#"><img class="arangodbLogo" src="img/arangodb_logo_small.png"></a><a class="version"><span id="currentVersion"></span></a></div><div class="statmenu" id="statisticBar"></div><div class="navmenu" id="navigationBar"></div></div></nav><div id="modalPlaceholder"></div><div class="bodyWrapper" style="display: none"><div class="centralRow"><div id="navbar2" class="navbarWrapper secondary"><div class="subnavmenu" id="subNavigationBar"></div></div><div class="resizecontainer contentWrapper"><div id="loadingScreen" class="loadingScreen" style="display: none"><i class="fa fa-circle-o-notch fa-spin fa-3x fa-fw margin-bottom"></i> <span class="sr-only">Loading...</span></div><div id="content" class="centralContent"></div><footer class="footer"><div id="footerBar"></div></footer></div></div></div><div id="progressPlaceholder" style="display:none"></div><div id="spotlightPlaceholder" style="display:none"></div><div id="graphSettingsContent" style="display: none"></div><div id="filterSelectDiv" style="display:none"></div><div id="offlinePlaceholder" style="display:none"><div class="offline-div"><div class="pure-u"><div class="pure-u-1-4"></div><div class="pure-u-1-2 offline-window"><div class="offline-header"><h3>You have been disconnected from the server</h3></div><div class="offline-body"><p>The connection to the server has been lost. The server may be under heavy load.</p><p>Trying to reconnect in <span id="offlineSeconds">10</span> seconds.</p><p class="animation_state"><span><button class="button-success">Reconnect now</button></span></p></div></div><div class="pure-u-1-4"></div></div></div></div><div class="arangoFrame" style=""><div class="outerDiv"><div class="innerDiv"></div></div></div><script src="libs.js?version=1554822895711"></script><script src="app.js?version=1554822895711"></script></body></html>
</div> <% } %> </script></head><body><nav class="navbar" style="display: none"><div class="primary"><div class="navlogo"><a class="logo big" href="#"><img id="ArangoDBLogo" class="arangodbLogo" src="img/arangodb-edition-optimized.svg"></a><a class="logo small" href="#"><img class="arangodbLogo" src="img/arangodb_logo_small.png"></a><a class="version"><span id="currentVersion"></span></a></div><div class="statmenu" id="statisticBar"></div><div class="navmenu" id="navigationBar"></div></div></nav><div id="modalPlaceholder"></div><div class="bodyWrapper" style="display: none"><div class="centralRow"><div id="navbar2" class="navbarWrapper secondary"><div class="subnavmenu" id="subNavigationBar"></div></div><div class="resizecontainer contentWrapper"><div id="loadingScreen" class="loadingScreen" style="display: none"><i class="fa fa-circle-o-notch fa-spin fa-3x fa-fw margin-bottom"></i> <span class="sr-only">Loading...</span></div><div id="content" class="centralContent"></div><footer class="footer"><div id="footerBar"></div></footer></div></div></div><div id="progressPlaceholder" style="display:none"></div><div id="spotlightPlaceholder" style="display:none"></div><div id="graphSettingsContent" style="display: none"></div><div id="filterSelectDiv" style="display:none"></div><div id="offlinePlaceholder" style="display:none"><div class="offline-div"><div class="pure-u"><div class="pure-u-1-4"></div><div class="pure-u-1-2 offline-window"><div class="offline-header"><h3>You have been disconnected from the server</h3></div><div class="offline-body"><p>The connection to the server has been lost. The server may be under heavy load.</p><p>Trying to reconnect in <span id="offlineSeconds">10</span> seconds.</p><p class="animation_state"><span><button class="button-success">Reconnect now</button></span></p></div></div><div class="pure-u-1-4"></div></div></div></div><div class="arangoFrame" style=""><div class="outerDiv"><div class="innerDiv"></div></div></div><script src="libs.js?version=1555403099256"></script><script src="app.js?version=1555403099256"></script></body></html>

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/index-min.html.gz
View File

Binary file not shown.

5
js/apps/system/_admin/aardvark/APP/frontend/build/libs.min.js vendored
View File

File diff suppressed because one or more lines are too long

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/libs.min.js.gz
View File

Binary file not shown.

2
js/apps/system/_admin/aardvark/APP/frontend/build/style-minified.css
View File

File diff suppressed because one or more lines are too long

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/style-minified.css.gz
View File

Binary file not shown.

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/style.css.gz
View File

Binary file not shown.

6
js/apps/system/_admin/aardvark/APP/frontend/js/collections/arangoCollections.js
View File

@ -164,13 +164,17 @@
data.type = parseInt(object.collType, 10);
if (object.shards) {
data.numberOfShards = object.shards;
data.shardKeys = object.shardBy;
data.shardKeys = object.shardKeys;
}
if (object.smartJoinAttribute &&
object.smartJoinAttribute !== '') {
data.smartJoinAttribute = object.smartJoinAttribute;
}
if (object.distributeShardsLike &&
object.distributeShardsLike !== '') {
data.distributeShardsLike = object.distributeShardsLike;
}
if (object.replicationFactor) {
data.replicationFactor = object.replicationFactor;

8
js/apps/system/_admin/aardvark/APP/frontend/js/templates/indicesView.ejs
View File

@ -61,7 +61,13 @@
<% } %>
</select>
</th>
<th class="" style="width: 18px"/>
<th class="tooltipInfoTh">
<div class="tooltipDiv">
<a class="index-tooltip" data-toggle="tooltip" data-placement="left" title="Type of index to create. <% if (supported.indexOf('persistent') <= -1) { %>Please note that for the RocksDB engine the index types hash, skiplist and persistent are identical, so that they are not offered seperately here.<% } %>">
<span rel="tooltip" class="arangoicon icon_arangodb_info"></span>
</a>
</div>
</th>
</tr>
</table>
<div id="newIndexTypeGeo" class="newIndexClass" style="display: none">

11
js/apps/system/_admin/aardvark/APP/frontend/js/templates/modalCollectionInfo.ejs
View File

@ -84,6 +84,17 @@
</th>
</tr>
<% } %>
<% if (figuresData.distributeShardsLike) { %>
<tr>
<th class="collectionInfoTh2">Distribute shards like:</th>
<th class="collectionInfoTh">
<div class="modal-text"><%=figuresData.distributeShardsLike%></div>
</th>
<th class="collectionInfoTh">
</th>
</tr>
<% } %>
<% if (figuresData.shardKeys) { %>
<tr>

40
js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js
View File

@ -354,8 +354,9 @@
var collType = $('#new-collection-type').val();
var collSync = $('#new-collection-sync').val();
var shards = 1;
var shardBy = [];
var shardKeys = [];
var smartJoinAttribute = '';
var distributeShardsLike = '';
if (replicationFactor === '') {
replicationFactor = 1;
@ -368,6 +369,9 @@
if (frontendConfig.isEnterprise && $('#smart-join-attribute').val() !== '') {
smartJoinAttribute = $('#smart-join-attribute').val().trim();
}
if (frontendConfig.isEnterprise && $('#distribute-shards-like').val() !== '') {
distributeShardsLike = $('#distribute-shards-like').val().trim();
}
shards = $('#new-collection-shards').val();
@ -381,11 +385,11 @@
);
return 0;
}
shardBy = _.pluck($('#new-collection-shardBy').select2('data'), 'text');
if (shardBy.length === 0) {
shardBy.push('_key');
shardKeys = _.pluck($('#new-collection-shardKeys').select2('data'), 'text');
if (shardKeys.length === 0) {
shardKeys.push('_key');
} else {
_.each(shardBy, function (element, index) { shardBy[index] = arangoHelper.escapeHtml(element); });
_.each(shardKeys, function (element, index) { shardKeys[index] = arangoHelper.escapeHtml(element); });
}
}
if (collName.substr(0, 1) === '_') {
@ -429,7 +433,7 @@
replicationFactor: replicationFactor,
collType: collType,
shards: shards,
shardBy: shardBy
shardKeys: shardKeys
};
if (self.engine.name !== 'rocksdb') {
tmpObj.journalSize = collSize;
@ -437,6 +441,9 @@
if (smartJoinAttribute !== '') {
tmpObj.smartJoinAttribute = smartJoinAttribute;
}
if (distributeShardsLike !== '') {
tmpObj.distributeShardsLike = distributeShardsLike;
}
this.collection.newCollection(tmpObj, callback);
window.modalView.hide();
arangoHelper.arangoNotification('Collection', 'Collection "' + collName + '" will be created.');
@ -497,16 +504,15 @@
'new-collection-shards',
'Shards',
'',
'The number of shards to create. You cannot change this afterwards. ' +
'Recommended: DBServers squared',
'The number of shards to create. You cannot change this afterwards. ',
'',
true
)
);
tableContent.push(
window.modalView.createSelect2Entry(
'new-collection-shardBy',
'shardBy',
'new-collection-shardKeys',
'Shard keys',
'',
'The keys used to distribute documents on shards. ' +
'Type the key and press return to add it.',
@ -524,6 +530,18 @@
);
if (window.App.isCluster) {
if (frontendConfig.isEnterprise) {
advancedTableContent.push(
window.modalView.createTextEntry(
'distribute-shards-like',
'Distribute shards like',
'',
'Name of another collection that should be used as a prototype for sharding this collection.',
'',
false,
[
]
)
);
advancedTableContent.push(
window.modalView.createSelectEntry(
'is-satellite-collection',
@ -601,7 +619,7 @@
);
// select2 workaround
$('#s2id_new-collection-shardBy .select2-search-field input').on('focusout', function (e) {
$('#s2id_new-collection-shardKeys .select2-search-field input').on('focusout', function (e) {
if ($('.select2-drop').is(':visible')) {
if (!$('#select2-search-field input').is(':focus')) {
window.setTimeout(function () {

2
js/apps/system/_admin/aardvark/APP/frontend/js/views/dashboardView.js
View File

@ -672,7 +672,7 @@
self.renderStatisticBox('Uptime', moment.duration(data.server.uptime, 'seconds').humanize());
},
error: function () {
self.renderStatisticBox('Uptime', 'Error');
self.renderStatisticBox('Uptime', 'N/A');
}
});
}

3
js/apps/system/_admin/aardvark/APP/frontend/js/views/footerView.js
View File

@ -183,6 +183,9 @@
async: true,
success: function (data) {
frontendConfig.version = data;
if (!frontendConfig.version.hasOwnProperty('version')) {
frontendConfig.version.version = 'N/A';
}
self.showServerStatus(true);
if (self.isOffline === true) {
self.isOffline = false;

15
js/apps/system/_admin/aardvark/APP/frontend/js/views/installServiceView.js
View File

@ -1,5 +1,5 @@
/* jshint browser: true */
/* global Backbone, $, window, arangoHelper, templateEngine, _ */
/* global frontendConfig, Backbone, $, window, arangoHelper, templateEngine, _ */
(function () {
'use strict';
@ -89,10 +89,15 @@
},
render: function () {
// if repo not fetched yet, wait
$(this.el).html(this.template.render({
services: this.collection
}));
if (frontendConfig.disableFoxxStore) {
$(this.el).html('<div>Foxx store is disabled via server configuration.</div>');
return this;
} else {
// if repo not fetched yet, wait
$(this.el).html(this.template.render({
services: this.collection
}));
}
arangoHelper.buildServicesSubNav('Store');
this.breadcrumb();

47
js/apps/system/_admin/aardvark/APP/package.json Normal file
View File

@ -0,0 +1,47 @@
{
"name": "aardvark",
"version": "2.0.0",
"description": "The ArangoDB web interface",
"author": "Heiko Kernbach, Michael Hackstein, moonglum",
"license": "Apache-2.0",
"scripts": {
"grunt": "grunt"
},
"devDependencies": {
"@babel/core": "^7.1.2",
"@babel/preset-env": "^7.1.0",
"babel-core": "^6.26.3",
"eslint": "^5.2.0",
"eslint-config-semistandard": "^12.0.1",
"eslint-config-standard": "^12.0.0",
"eslint-plugin-import": "^2.13.0",
"eslint-plugin-node": "^7.0.1",
"eslint-plugin-promise": "^4.0.1",
"eslint-plugin-standard": "^4.0.0",
"grunt": "^1.0.4",
"grunt-babel": "^8.0.0",
"grunt-concat-in-order": "^0.3.0",
"grunt-concurrent": "^2.3.1",
"grunt-contrib-compress": "^1.4.3",
"grunt-contrib-concat": "^1.0.1",
"grunt-contrib-cssmin": "^3.0.0",
"grunt-contrib-htmlmin": "^3.0.0",
"grunt-contrib-imagemin": "^2.0.1",
"grunt-contrib-jshint": "^2.0.0",
"grunt-contrib-sass": "^1.0.0",
"grunt-contrib-uglify": "^4.0.0",
"grunt-contrib-watch": "^1.1.0",
"grunt-eslint": "^21.0.0",
"grunt-processhtml": "^0.4.1",
"grunt-sass": "^3.0.2",
"grunt-text-replace": "^0.4.0",
"load-grunt-tasks": "^4.0.0",
"matchdep": "^2.0.0",
"node-sass": "^4.9.3",
"sass": "^1.10.1"
},
"repository": {
"type": "git",
"url": "git://github.com/arangodb/arangodb.git"
}
}

3
js/apps/system/_api/foxx/APP/index.js
View File

@ -82,6 +82,7 @@ function prepareServiceRequestBody (req, res, next) {
next();
}
if (!require("internal").isFoxxApiDisabled()) {
router.use((req, res, next) => {
try {
next();
@ -443,3 +444,5 @@ router.use('/_local', localRouter);
localRouter.post('/heal', (req, res) => {
FoxxManager.heal();
});
} // if not foxx lock-down

296
js/client/modules/@arangodb/process-utils.js
View File

@ -1289,6 +1289,67 @@ function detectCurrentLeader(instanceInfo) {
});
return leaderInstance;
}
function checkClusterAlive(options, instanceInfo, addArgs) {
// disabled because not in use (jslint)
// let coordinatorUrl = instanceInfo.url
// let response
let httpOptions = makeAuthorizationHeaders(options);
httpOptions.method = 'POST';
httpOptions.returnBodyOnError = true;
// scrape the jwt token
instanceInfo.authOpts = _.clone(options);
if (addArgs['server.jwt-secret'] && !instanceInfo.authOpts['server.jwt-secret']) {
instanceInfo.authOpts['server.jwt-secret'] = addArgs['server.jwt-secret'];
}
let count = 0;
while (true) {
++count;
instanceInfo.arangods.forEach(arangod => {
const reply = download(arangod.url + '/_api/version', '', makeAuthorizationHeaders(instanceInfo.authOpts));
if (!reply.error && reply.code === 200) {
arangod.upAndRunning = true;
return true;
}
if (!checkArangoAlive(arangod, options)) {
instanceInfo.arangods.forEach(arangod => {
if (!arangod.hasOwnProperty('exitStatus') ||
(arangod.exitStatus.status === 'RUNNING')) {
arangod.exitStatus = killExternal(arangod.pid, abortSignal);
}
analyzeServerCrash(arangod, options, 'startup timeout; forcefully terminating ' + arangod.role + ' with pid: ' + arangod.pid);
});
throw new Error(`cluster startup: pid ${arangod.pid} no longer alive! bailing out!`);
}
wait(0.5, false);
return true;
});
let upAndRunning = 0;
instanceInfo.arangods.forEach(arangod => {
if (arangod.upAndRunning) {
upAndRunning += 1;
}
});
if (upAndRunning === instanceInfo.arangods.length) {
break;
}
// Didn't startup in 10 minutes? kill it, give up.
if (count > 1200) {
instanceInfo.arangods.forEach(arangod => {
arangod.exitStatus = killExternal(arangod.pid, abortSignal);
analyzeServerCrash(arangod, options, 'startup timeout; forcefully terminating ' + arangod.role + ' with pid: ' + arangod.pid);
});
throw new Error('cluster startup timed out after 10 minutes!');
}
}
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief starts an instance
// /
@ -1370,65 +1431,7 @@ function startInstanceCluster (instanceInfo, protocol, options,
sleep(1.0);
}
}
// disabled because not in use (jslint)
// let coordinatorUrl = instanceInfo.url
// let response
let httpOptions = makeAuthorizationHeaders(options);
httpOptions.method = 'POST';
httpOptions.returnBodyOnError = true;
// scrape the jwt token
instanceInfo.authOpts = _.clone(options);
if (addArgs['server.jwt-secret'] && !instanceInfo.authOpts['server.jwt-secret']) {
instanceInfo.authOpts['server.jwt-secret'] = addArgs['server.jwt-secret'];
}
let count = 0;
while (true) {
++count;
instanceInfo.arangods.forEach(arangod => {
const reply = download(arangod.url + '/_api/version', '', makeAuthorizationHeaders(instanceInfo.authOpts));
if (!reply.error && reply.code === 200) {
arangod.upAndRunning = true;
return true;
}
if (!checkArangoAlive(arangod, options)) {
instanceInfo.arangods.forEach(arangod => {
if (!arangod.hasOwnProperty('exitStatus') ||
(arangod.exitStatus.status === 'RUNNING')) {
arangod.exitStatus = killExternal(arangod.pid, abortSignal);
}
analyzeServerCrash(arangod, options, 'startup timeout; forcefully terminating ' + arangod.role + ' with pid: ' + arangod.pid);
});
throw new Error(`cluster startup: pid ${arangod.pid} no longer alive! bailing out!`);
}
wait(0.5, false);
return true;
});
let upAndRunning = 0;
instanceInfo.arangods.forEach(arangod => {
if (arangod.upAndRunning) {
upAndRunning += 1;
}
});
if (upAndRunning === instanceInfo.arangods.length) {
break;
}
// Didn't startup in 10 minutes? kill it, give up.
if (count > 1200) {
instanceInfo.arangods.forEach(arangod => {
arangod.exitStatus = killExternal(arangod.pid, abortSignal);
analyzeServerCrash(arangod, options, 'startup timeout; forcefully terminating ' + arangod.role + ' with pid: ' + arangod.pid);
});
throw new Error('cluster startup timed out after 10 minutes!');
}
}
checkClusterAlive(options, instanceInfo, addArgs);
// we need to find the leading server
if (options.activefailover) {
@ -1445,6 +1448,58 @@ function startInstanceCluster (instanceInfo, protocol, options,
return true;
}
function launchFinalize(options, instanceInfo, startTime) {
if (!options.cluster) {
let count = 0;
instanceInfo.arangods.forEach(arangod => {
while (true) {
wait(0.5, false);
if (options.useReconnect) {
try {
arango.reconnect(instanceInfo.endpoint,
'_system',
options.username,
options.password,
count > 50
);
break;
} catch (e) {
}
} else {
const reply = download(arangod.url + '/_api/version', '', makeAuthorizationHeaders(options));
if (!reply.error && reply.code === 200) {
break;
}
}
++count;
if (count % 60 === 0) {
if (!checkArangoAlive(arangod, options)) {
throw new Error('startup failed! bailing out!');
}
}
}
});
}
print(CYAN + Date() + ' up and running in ' + (time() - startTime) + ' seconds' + RESET);
var matchPort = /.*:.*:([0-9]*)/;
var ports = [];
var processInfo = [];
instanceInfo.arangods.forEach(arangod => {
let res = matchPort.exec(arangod.endpoint);
if (!res) {
return;
}
var port = res[1];
ports.push('port ' + port);
processInfo.push(' [' + arangod.role + '] up with pid ' + arangod.pid + ' on port ' + port);
});
print(Date() + ' sniffing template:\n tcpdump -ni lo -s0 -w /tmp/out.pcap ' + ports.join(' or ') + '\n');
print(processInfo.join('\n') + '\n');
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief starts an instance
// /
@ -1503,6 +1558,7 @@ function startArango (protocol, options, addArgs, rootDir, role) {
}
instanceInfo.url = endpointToURL(instanceInfo.endpoint);
instanceInfo.args = args;
try {
instanceInfo.pid = executeArangod(ARANGOD_BIN, toArgv(args), options).pid;
} catch (x) {
@ -1628,55 +1684,7 @@ function startInstance (protocol, options, addArgs, testname, tmpDir) {
addArgs, rootDir, 'single');
}
if (!options.cluster) {
let count = 0;
instanceInfo.arangods.forEach(arangod => {
while (true) {
wait(0.5, false);
if (options.useReconnect) {
try {
arango.reconnect(instanceInfo.endpoint,
'_system',
options.username,
options.password,
count > 50
);
break;
} catch (e) {
}
} else {
const reply = download(arangod.url + '/_api/version', '', makeAuthorizationHeaders(options));
if (!reply.error && reply.code === 200) {
break;
}
}
++count;
if (count % 60 === 0) {
if (!checkArangoAlive(arangod, options)) {
throw new Error('startup failed! bailing out!');
}
}
}
});
}
print(CYAN + Date() + ' up and running in ' + (time() - startTime) + ' seconds' + RESET);
var matchPort = /.*:.*:([0-9]*)/;
var ports = [];
var processInfo = [];
instanceInfo.arangods.forEach(arangod => {
let res = matchPort.exec(arangod.endpoint);
if (!res) {
return;
}
var port = res[1];
ports.push('port ' + port);
processInfo.push(' [' + arangod.role + '] up with pid ' + arangod.pid + ' on port ' + port);
});
print(Date() + ' sniffing template:\n tcpdump -ni lo -s0 -w /tmp/out.pcap ' + ports.join(' or ') + '\n');
print(processInfo.join('\n') + '\n');
launchFinalize(options, instanceInfo, startTime);
} catch (e) {
print(e, e.stack);
return false;
@ -1684,6 +1692,77 @@ function startInstance (protocol, options, addArgs, testname, tmpDir) {
return instanceInfo;
}
function reStartInstance(options, instanceInfo, moreArgs) {
let launchInstance = function(options, oneInstanceInfo) {
try {
Object.assign(oneInstanceInfo.args, moreArgs);
oneInstanceInfo.pid = executeArangod(ARANGOD_BIN, toArgv(oneInstanceInfo.args), options).pid;
} catch (x) {
print(Date() + ' failed to run arangod - ' + JSON.stringify(x));
throw x;
}
if (platform.substr(0, 3) === 'win' && !options.disableMonitor) {
runProcdump(options, oneInstanceInfo, oneInstanceInfo.rootDir, oneInstanceInfo.pid);
}
};
const startTime = time();
instanceInfo.arangods.forEach(function (oneInstance, i) {
delete(oneInstance.exitStatus);
delete(oneInstance.pid);
oneInstance.upAndRunning = false;
});
if (options.cluster) {
let agencyInstance = {arangods: []};
instanceInfo.arangods.forEach(function (oneInstance, i) {
if (oneInstance.role === 'agent') {
print("relaunching: " + JSON.stringify(oneInstance));
launchInstance(options, oneInstance);
agencyInstance.arangods.push(_.clone(oneInstance));
}
});
let agencyEndpoint = instanceInfo.endpoint;
if (!checkInstanceAlive(agencyInstance, options)) {
throw new Error('startup of agency failed! bailing out!');
}
}
instanceInfo.arangods.forEach(function (oneInstance, i) {
if ((oneInstance.role === 'PRIMARY') ||
(oneInstance.role === 'primary') ||
(oneInstance.role === 'dbserver')) {
print("relaunching: " + JSON.stringify(oneInstance));
launchInstance(options, oneInstance);
}
});
instanceInfo.arangods.forEach(function (oneInstance, i) {
if ((oneInstance.role === 'COORDINATOR') || (oneInstance.role === 'coordinator')) {
print("relaunching: " + JSON.stringify(oneInstance));
launchInstance(options, oneInstance);
}
});
instanceInfo.arangods.forEach(function (oneInstance, i) {
if (oneInstance.role === 'single') {
launchInstance(options, oneInstance);
}
});
if (options.cluster) {
checkClusterAlive(options, instanceInfo, {}); // todo addArgs
arango.reconnect(instanceInfo.endpoint,
'_system',
options.username,
options.password,
false
);
}
launchFinalize(options, instanceInfo, startTime);
}
// exports.analyzeServerCrash = analyzeServerCrash;
exports.makeArgs = {
arangod: makeArgsArangod,
@ -1716,6 +1795,7 @@ exports.run = {
exports.shutdownInstance = shutdownInstance;
exports.startArango = startArango;
exports.startInstance = startInstance;
exports.reStartInstance = reStartInstance;
exports.setupBinaries = setupBinaries;
exports.executableExt = executableExt;
exports.serverCrashed = serverCrashedLocal;

1
js/client/modules/@arangodb/test-utils.js
View File

@ -739,6 +739,7 @@ function runInLocalArangosh (options, instanceInfo, file, addArgs) {
}
let testCode;
// \n's in testCode are required because of content could contain '//' at the very EOF
if (file.indexOf('-spec') === -1) {
let testCase = JSON.stringify(options.testCase);
if (options.testCase === undefined) {

44
js/client/modules/@arangodb/testsuites/arangosh.js
View File

@ -25,13 +25,16 @@
// / @author Max Neunhoeffer
// //////////////////////////////////////////////////////////////////////////////
const _ = require('lodash');
const time = require('internal').time;
const fs = require('fs');
const yaml = require('js-yaml');
const pu = require('@arangodb/process-utils');
const tu = require('@arangodb/test-utils');
const toArgv = require('internal').toArgv;
const executeScript = require('internal').executeScript;
const executeExternalAndWait = require('internal').executeExternalAndWait;
const platform = require('internal').platform;
@ -44,14 +47,16 @@ const RESET = require('internal').COLORS.COLOR_RESET;
// const YELLOW = require('internal').COLORS.COLOR_YELLOW;
const functionsDocumentation = {
'arangosh': 'arangosh exit codes tests'
'arangosh': 'arangosh exit codes tests',
'permissions': 'arangosh javascript access permissions'
};
const optionsDocumentation = [
' - `skipShebang`: if set, the shebang tests are skipped.'
];
const testPaths = {
'arangosh': []
'arangosh': [],
'permissions': [tu.pathForTesting('client/permissions')]
};
// //////////////////////////////////////////////////////////////////////////////
@ -245,11 +250,46 @@ function arangosh (options) {
return ret;
}
function permissions(options) {
let res = {};
let filtered = {};
let rootDir = fs.join(fs.getTempPath(), 'permissions');
const tests = tu.scanTestPaths(testPaths.permissions);
fs.makeDirectoryRecursive(rootDir);
tests.forEach(function (f, i) {
if (tu.filterTestcaseByOptions(f, options, filtered)) {
let content = fs.read(f);
content = `(function(){ const getOptions = true; ${content}
}())`; // DO NOT JOIN WITH THE LINE ABOVE -- because of content could contain '//' at the very EOF
let testOptions = executeScript(content, true, f);
res[f] = tu.runInArangosh(options,
{
endpoint: 'tcp://127.0.0.1:8888',
rootDir: rootDir
},
f,
testOptions
);
} else {
if (options.extremeVerbosity) {
print('Skipped ' + f + ' because of ' + filtered.filter);
}
}
});
return res;
}
exports.setup = function (testFns, defaultFns, opts, fnDocs, optionsDoc, allTestPaths) {
Object.assign(allTestPaths, testPaths);
testFns['arangosh'] = arangosh;
testFns['permissions'] = permissions;
defaultFns.push('arangosh');
defaultFns.push('permissions');
opts['skipShebang'] = false;

122
js/client/modules/@arangodb/testsuites/permissions.js Normal file
View File

@ -0,0 +1,122 @@
/* jshint strict: false, sub: true */
/* global print, params */
'use strict';
// //////////////////////////////////////////////////////////////////////////////
// / DISCLAIMER
// /
// / Copyright 2016 ArangoDB GmbH, Cologne, Germany
// / Copyright 2014 triagens GmbH, Cologne, Germany
// /
// / Licensed under the Apache License, Version 2.0 (the "License")
// / you may not use this file except in compliance with the License.
// / You may obtain a copy of the License at
// /
// / http://www.apache.org/licenses/LICENSE-2.0
// /
// / Unless required by applicable law or agreed to in writing, software
// / distributed under the License is distributed on an "AS IS" BASIS,
// / WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// / See the License for the specific language governing permissions and
// / limitations under the License.
// /
// / Copyright holder is ArangoDB GmbH, Cologne, Germany
// /
// / @author Wilfried Goesgens
// //////////////////////////////////////////////////////////////////////////////
const _ = require('lodash');
const time = require('internal').time;
const fs = require('fs');
const yaml = require('js-yaml');
const pu = require('@arangodb/process-utils');
const tu = require('@arangodb/test-utils');
const toArgv = require('internal').toArgv;
const executeScript = require('internal').executeScript;
const executeExternalAndWait = require('internal').executeExternalAndWait;
const platform = require('internal').platform;
const BLUE = require('internal').COLORS.COLOR_BLUE;
const CYAN = require('internal').COLORS.COLOR_CYAN;
const GREEN = require('internal').COLORS.COLOR_GREEN;
const RED = require('internal').COLORS.COLOR_RED;
const RESET = require('internal').COLORS.COLOR_RESET;
// const YELLOW = require('internal').COLORS.COLOR_YELLOW;
const functionsDocumentation = {
'permissions_server': 'permissions test for the server'
};
const testPaths = {
'permissions_server': [tu.pathForTesting('server/permissions')]
};
function permissions_server(options) {
let count = 0;
let results = {};
let filtered = {};
const tests = tu.scanTestPaths(testPaths.permissions_server);
tests.forEach(function (testFile, i) {
count += 1;
if (tu.filterTestcaseByOptions(testFile, options, filtered)) {
// pass on JWT secret
let clonedOpts = _.clone(options);
let serverOptions = {};
if (serverOptions['server.jwt-secret'] && !clonedOpts['server.jwt-secret']) {
clonedOpts['server.jwt-secret'] = serverOptions['server.jwt-secret'];
}
let paramsFistRun = {};
let paramsSecondRun;
let rootDir = fs.join(fs.getTempPath(), count.toString());
let instanceInfo = pu.startInstance(options.protocol, options, paramsFistRun, "permissions_server", rootDir); // fist start
pu.cleanupDBDirectoriesAppend(instanceInfo.rootDir);
try {
print(BLUE + '================================================================================' + RESET);
print(CYAN + 'Running Setup of: ' + testFile + RESET);
let content = fs.read(testFile);
content = `(function(){ const getOptions = true; ${content}
}())`; // DO NOT JOIN WITH THE LINE ABOVE -- because of content could contain '//' at the very EOF
paramsSecondRun = executeScript(content, true, testFile);
} catch (ex) {
results[testFile] = {
status: false,
messages: 'Warmup of system failed: ' + ex
};
pu.shutdownInstance(instanceInfo, clonedOpts, false); // stop
return;
}
if (paramsSecondRun.hasOwnProperty('server.jwt-secret')) {
clonedOpts['server.jwt-secret'] = paramsSecondRun['server.jwt-secret'];
}
pu.shutdownInstance(instanceInfo, clonedOpts, false); // stop
pu.reStartInstance(clonedOpts, instanceInfo, paramsSecondRun); // restart with restricted permissions
results[testFile] = tu.runInLocalArangosh(options, instanceInfo, testFile, {});
pu.shutdownInstance(instanceInfo, clonedOpts, false);
if (!results[testFile].status) {
print("Not cleaning up " + instanceInfo.rootDir);
results.status = false;
}
else {
pu.cleanupLastDirectory(options);
}
} else {
if (options.extremeVerbosity) {
print('Skipped ' + testFile + ' because of ' + filtered.filter);
}
}
});
return results;
}
exports.setup = function (testFns, defaultFns, opts, fnDocs, optionsDoc, allTestPaths) {
Object.assign(allTestPaths, testPaths);
testFns['permissions_server'] = permissions_server;
for (var attrname in functionsDocumentation) { fnDocs[attrname] = functionsDocumentation[attrname]; }
};

14
js/client/modules/@arangodb/testsuites/resilience.js
View File

@ -73,7 +73,10 @@ var _resilience = function(path) {
if (options.dbServers < 5) {
options.dbServers = 5;
}
return tu.performTests(options, testCases, suiteName, tu.runThere);
return tu.performTests(options, testCases, suiteName, tu.runThere, {
'javascript.allow-external-process-control': 'true',
'javascript.allow-port-testing': 'true',
});
};
};
@ -97,7 +100,10 @@ function clientResilience (options) {
options.coordinators = 2;
}
return tu.performTests(options, testCases, 'client_resilience', tu.runInArangosh);
return tu.performTests(options, testCases, 'client_resilience', tu.runInArangosh, {
'javascript.allow-external-process-control': 'true',
'javascript.allow-port-testing': 'true',
});
}
// //////////////////////////////////////////////////////////////////////////////
@ -121,7 +127,9 @@ function activeFailover (options) {
options.disableMonitor = true;
return tu.performTests(options, testCases, 'client_resilience', tu.runInArangosh, {
'server.authentication': 'true',
'server.jwt-secret': 'haxxmann'
'server.jwt-secret': 'haxxmann',
'javascript.allow-external-process-control': 'true',
'javascript.allow-port-testing': 'true',
});
}

38
js/common/bootstrap/errors.js
View File

@ -366,5 +366,43 @@
// For compatibility with <= 3.3
internal.errors.ERROR_ARANGO_COLLECTION_NOT_FOUND = internal.errors.ERROR_ARANGO_DATA_SOURCE_NOT_FOUND;
//arg1 can be a code or error object
internal.throwArangoError = function (arg1, message, httpCode) {
let errorNum;
if (typeof arg1 === "object" && typeof arg1.code === "number") {
errorNum = arg1.code;
if(message === undefined && arg1.message) {
message = arg1.message;
}
} else if ( typeof arg1 === "number" ) {
errorNum = arg1;
} else {
errorNum = internal.errors.ERROR_INTERNAL.code;
}
if (message === undefined) {
message = "could not resolve errorMessage";
for(var key in internal.errors) {
let attribute = internal.errors[key];
if(attribute.code === errorNum){
message = attribute.message;
break;
}
}
}
if (httpCode === undefined) {
httpCode = internal.errorNumberToHttpCode(errorNum);
}
throw new internal.ArangoError({
errorNum: errorNum,
errorMessage: message,
code: httpCode
});
};
}());

9
js/common/bootstrap/modules/fs.js
View File

@ -376,6 +376,15 @@ global.DEFINE_MODULE('fs', (function () {
// / @brief copy one file
// //////////////////////////////////////////////////////////////////////////////
if (global.FS_LINK_FILE) {
exports.linkFile = global.FS_LINK_FILE;
delete global.FS_LINK_FILE;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief copy one file
// //////////////////////////////////////////////////////////////////////////////
if (global.FS_COPY_FILE) {
exports.copyFile = global.FS_COPY_FILE;
delete global.FS_COPY_FILE;

64
js/common/bootstrap/modules/internal.js
View File

@ -46,10 +46,10 @@ global.DEFINE_MODULE('internal', (function () {
} else {
exports.ArangoError = function (error) {
if (error !== undefined) {
this.error = error.error;
this.code = error.code;
this.errorNum = error.errorNum;
this.errorMessage = error.errorMessage;
this.error = error.error; // bool -- is error or not
this.code = error.code; // int - http status code
this.errorNum = error.errorNum; // int - internal arangodb error code
this.errorMessage = error.errorMessage; // string - error message
}
};
@ -196,6 +196,15 @@ global.DEFINE_MODULE('internal', (function () {
exports.startupPath = '.';
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief convert error number to http code
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_ERROR_NUMBER_TO_HTTP_CODE) {
exports.errorNumberToHttpCode = global.SYS_ERROR_NUMBER_TO_HTTP_CODE;
delete global.SYS_ERROR_NUMBER_TO_HTTP_CODE;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief configureEndpoint
// //////////////////////////////////////////////////////////////////////////////
@ -578,24 +587,6 @@ global.DEFINE_MODULE('internal', (function () {
delete global.SYS_PROCESS_JSON_FILE;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief clientStatistics
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_CLIENT_STATISTICS) {
exports.clientStatistics = global.SYS_CLIENT_STATISTICS;
delete global.SYS_CLIENT_STATISTICS;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief httpStatistics
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_HTTP_STATISTICS) {
exports.httpStatistics = global.SYS_HTTP_STATISTICS;
delete global.SYS_HTTP_STATISTICS;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief executeExternal
// //////////////////////////////////////////////////////////////////////////////
@ -741,20 +732,27 @@ global.DEFINE_MODULE('internal', (function () {
if (longOptsEqual) {
vec.push('--' + key + '=' + structure[key]);
} else {
vec.push('--' + key);
if (structure[key] !== false) {
if (structure[key] !== true) {
if (structure[key] !== null) {
// The null case is for the case one wants to add an option
// with an equals sign all in the key, which is necessary if
// one wants to specify an option multiple times.
vec.push(structure[key]);
if (Array.isArray(structure[key])) {
structure[key].forEach(function (i, j) {
vec.push('--' + key);
vec.push(i);
});
} else {
vec.push('--' + key);
if (structure[key] !== false) {
if (structure[key] !== true) {
if (structure[key] !== null) {
// The null case is for the case one wants to add an option
// with an equals sign all in the key, which is necessary if
// one wants to specify an option multiple times.
vec.push(structure[key]);
}
} else {
vec.push('true');
}
} else {
vec.push('true');
vec.push('false');
}
} else {
vec.push('false');
}
}
}

9
js/common/modules/@arangodb/common.js
View File

@ -555,12 +555,13 @@ exports.checkAvailableVersions = function(version) {
);
return;
}
if (internal.isEnterprise()) {
// don't check for version updates in the enterprise version
if (require('@arangodb').isServer || internal.isEnterprise()) {
// don't check for version updates in the server
// nor in the enterprise version
return;
}
try {
var u =
'https://www.arangodb.com/repositories/versions.php?version=' +

9
js/common/modules/@arangodb/foxx/store.js
View File

@ -321,8 +321,13 @@ function availableJson (matchEngine) {
// //////////////////////////////////////////////////////////////////////////////
var update = function () {
var url = utils.buildGithubUrl(getFishbowlUrl());
var filename = fs.getTempFile('bundles', false);
let url = utils.buildGithubUrl(getFishbowlUrl());
let filename = fs.getTempFile('bundles', false);
let internal = require('internal');
if (internal.isFoxxStoreDisabled && internal.isFoxxStoreDisabled()) {
throw new Error('Foxx store is disabled in configuration');
}
try {
var result = download(url, '', {

31
js/common/modules/@arangodb/graph/traversal.js
View File

@ -1,5 +1,5 @@
/* jshint strict: false, unused: true */
/* global ArangoClusterComm, AQL_QUERY_IS_KILLED */
/* global ArangoClusterComm */
// //////////////////////////////////////////////////////////////////////////////
// / @brief Traversal "classes"
@ -38,27 +38,6 @@ var db = arangodb.db;
var ArangoTraverser;
// //////////////////////////////////////////////////////////////////////////////
// / @brief whether or not the query was aborted
// / use the AQL_QUERY_IS_KILLED function on the server side, and a dummy
// / function otherwise (ArangoShell etc.)
// //////////////////////////////////////////////////////////////////////////////
var throwIfAborted = function () {};
try {
if (typeof AQL_QUERY_IS_KILLED === 'function') {
throwIfAborted = function () {
if (AQL_QUERY_IS_KILLED()) {
var err = new ArangoError();
err.errorNum = arangodb.errors.ERROR_QUERY_KILLED.code;
err.errorMessage = arangodb.errors.ERROR_QUERY_KILLED.message;
throw err;
}
};
}
} catch (err) {}
// //////////////////////////////////////////////////////////////////////////////
// / @brief clone any object
// //////////////////////////////////////////////////////////////////////////////
@ -834,8 +813,6 @@ function breadthFirstSearch () {
throw err;
}
throwIfAborted();
if (current.visit === null || current.visit === undefined) {
current.visit = false;
@ -941,8 +918,6 @@ function depthFirstSearch () {
throw err;
}
throwIfAborted();
// peek at the top of the stack
var current = toVisit[toVisit.length - 1];
var vertex = current.vertex;
@ -1084,8 +1059,6 @@ function dijkstraSearch () {
throw err;
}
throwIfAborted();
var currentNode = heap.pop();
var i, n;
@ -1346,8 +1319,6 @@ function astarSearch () {
throw err;
}
throwIfAborted();
var currentNode = heap.pop();
var i, n;

2
js/common/modules/jsunity.js
View File

@ -264,7 +264,7 @@ function RunTest (path, outputReply, filter) {
content = fs.read(path);
content = `(function(){ require('jsunity').jsUnity.attachAssertions(); return (function() { require('jsunity').setTestFilter(${JSON.stringify(filter)}); ${content} }());
content = `(function(){ require('jsunity').jsUnity.attachAssertions(); return (function() { require('jsunity').setTestFilter(${JSON.stringify(filter)}); const getOptions = false; ${content} }());
});`;
f = internal.executeScript(content, undefined, path);

6
js/server/bootstrap/autoload.js
View File

@ -32,9 +32,9 @@
// //////////////////////////////////////////////////////////////////////////////
(function () {
var internal = require('internal');
var console = require('console');
var db = internal.db;
let internal = require('internal');
let console = require('console');
let db = internal.db;
return {
startup: function () {

57
js/server/bootstrap/modules/internal.js
View File

@ -156,13 +156,13 @@
return global.WAL_WAITCOLLECTOR.apply(null, arguments);
}
};
// / @brief ttlStatistics
if (global.SYS_TTL_STATISTICS) {
exports.ttlStatistics = global.SYS_TTL_STATISTICS;
delete global.SYS_TTL_STATISTICS;
}
// / @brief ttlProperties
if (global.SYS_TTL_PROPERTIES) {
exports.ttlProperties = global.SYS_TTL_PROPERTIES;
@ -178,6 +178,22 @@
delete global.SYS_DEFINE_ACTION;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief expose configuration
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_IS_FOXX_API_DISABLED) {
exports.isFoxxApiDisabled = global.SYS_IS_FOXX_API_DISABLED;
delete global.SYS_IS_FOXX_API_DISABLED;
}
if (global.SYS_IS_FOXX_STORE_DISABLED) {
exports.isFoxxStoreDisabled = global.SYS_IS_FOXX_STORE_DISABLED;
delete global.SYS_IS_FOXX_STORE_DISABLED;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief throw-collection-not-loaded
// //////////////////////////////////////////////////////////////////////////////
@ -206,7 +222,7 @@
}
modules = modules.byExample({ autoload: true }).toArray();
modules.forEach(function (module) {
// this module is only meant to be executed in one thread
if (exports.threadNumber !== 0 && !module.perThread) {
@ -238,19 +254,25 @@
console.debug('autoloading actions finished');
};
// //////////////////////////////////////////////////////////////////////////////
// / @brief executes a string in all V8 contexts
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_EXECUTE_GLOBAL_CONTEXT_FUNCTION) {
exports.executeGlobalContextFunction = global.SYS_EXECUTE_GLOBAL_CONTEXT_FUNCTION;
} else {
exports.executeGlobalContextFunction = function () {
// nothing to do. we're probably in --no-server mode
};
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief clientStatistics
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_CLIENT_STATISTICS) {
exports.clientStatistics = global.SYS_CLIENT_STATISTICS;
delete global.SYS_CLIENT_STATISTICS;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief httpStatistics
// //////////////////////////////////////////////////////////////////////////////
if (global.SYS_HTTP_STATISTICS) {
exports.httpStatistics = global.SYS_HTTP_STATISTICS;
delete global.SYS_HTTP_STATISTICS;
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief getCurrentRequest
// //////////////////////////////////////////////////////////////////////////////
@ -275,11 +297,10 @@
if (global.SYS_EXECUTE_GLOBAL_CONTEXT_FUNCTION) {
exports.reloadAqlFunctions = function () {
exports.executeGlobalContextFunction('reloadAql');
global.SYS_EXECUTE_GLOBAL_CONTEXT_FUNCTION('reloadAql');
require('@arangodb/aql').reload();
};
delete global.SYS_EXECUTE_GLOBAL_CONTEXT_FUNCTION;
}else {
} else {
exports.reloadAqlFunctions = function () {
require('@arangodb/aql').reload();
};

Some files were not shown because too many files have changed in this diff Show More