Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'devel' of github.com:arangodb/arangodb into devel

This commit is contained in:
Michael Hackstein 2016-07-12 14:46:35 +02:00
parent d9e1f41871 f7696c6aef
commit daeef23761
42 changed files with 846 additions and 209 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
CHANGELOG
View File

@ -24,7 +24,13 @@ devel
* When disabling Foxx development mode the setup script is now re-run
v3.0.2 (XXXX-XX-XX)
v3.0.3 (XXXX-XX-XX)
-------------------
* fixed issue #1936
v3.0.2 (2016-07-09)
-------------------
* fixed assertion failure in case multiple remove operations were used in the same query
@ -62,6 +68,17 @@ v3.0.2 (XXXX-XX-XX)
* fix Foxx configuration not being saved
* fix Foxx app access from within the frontend on DC/OS
* add option --default-replication-factor to arangorestore and simplify
the control over the number of shards when restoring
* fix a bug in the VPack -> V8 conversion if special attributes _key,
_id, _rev, _from and _to had non-string values, which is allowed
below the top level
* fix malloc_usable_size for darwin
v3.0.1 (XXXX-XX-XX)
-------------------

89
Documentation/Books/AQL/Extending/Functions.mdpp
View File

@ -9,10 +9,14 @@ var aqlfunctions = require("@arangodb/aql/functions");
To register a function, the fully qualified function name plus the
function code must be specified. This can easily be done in
[arangosh](../../Manual/GettingStarted/Arangosh.html). For testing, it may be
sufficient to directly type the function code in the shell. To manage more
complex code, you may write it in the code editor of your choice and save it
as file. For example:
[arangosh](../../Manual/GettingStarted/Arangosh.html). The [HTTP Interface
](../../HTTP/AqlUserFunctions/index.html) also offers User Functions management.
!SUBSECTION Registering an AQL user function
For testing, it may be sufficient to directly type the function code in the shell.
To manage more complex code, you may write it in the code editor of your choice
and save it as file. For example:
```js
/* path/to/file.js */
@ -39,10 +43,6 @@ Note that a return value of *false* means that the function `HUMAN::GREETING`
was newly created, and not that it failed to register. *true* is returned
if a function of that name existed before and was just updated.
The [HTTP Interface](../../HTTP/AqlUserFunctions/index.html) also offers User Functions management.
!SUBSECTION Registering an AQL user function
`aqlfunctions.register(name, code, isDeterministic)`
Registers an AQL user function, identified by a fully qualified function
@ -73,10 +73,10 @@ when it detects syntactially invalid function code.
```js
require("@arangodb/aql/functions").register("myfunctions::temperature::celsiustofahrenheit",
function (celsius) {
return celsius * 1.8 + 32;
});
require("@arangodb/aql/functions").register("MYFUNCTIONS::TEMPERATURE::CELSIUSTOFAHRENHEIT",
function (celsius) {
return celsius * 1.8 + 32;
});
```
The function code will not be executed in *strict mode* or *strong mode* by
@ -84,13 +84,51 @@ default. In order to make a user function being run in strict mode, use
`use strict` explicitly, e.g.:
```js
require("@arangodb/aql/functions").register("myfunctions::temperature::celsiustofahrenheit",
function (celsius) {
"use strict";
return celsius * 1.8 + 32;
});
require("@arangodb/aql/functions").register("MYFUNCTIONS::TEMPERATURE::CELSIUSTOFAHRENHEIT",
function (celsius) {
"use strict";
return celsius * 1.8 + 32;
});
```
You can access the name under which the AQL function is registered by accessing
the `name` property of `this` inside the JavaScript code:
```js
require("@arangodb/aql/functions").register("MYFUNCTIONS::TEMPERATURE::CELSIUSTOFAHRENHEIT",
function (celsius) {
"use strict";
if (typeof celsius === "undefined") {
const error = require("@arangodb").errors.ERROR_QUERY_FUNCTION_ARGUMENT_NUMBER_MISMATCH;
AQL_WARNING(error.code, require("util").format(error.message, this.name, 1, 1));
}
return celsius * 1.8 + 32;
});
```
`AQL_WARNING()` is automatically available to the code of user-defined
functions. The error code and message is retrieved via `@arangodb` module.
The *argument number mismatch* message has placeholders, which we can substitute
using [format()](http://nodejs.org/api/util.html):
```
invalid number of arguments for function '%s()', expected number of arguments: minimum: %d, maximum: %d
```
In the example above, `%s` is replaced by `this.name` (the AQL function name),
and both `%d` placeholders by `1` (number of expected arguments). If you call
the function without an argument, you will see this:
```
arangosh> db._query("RETURN MYFUNCTIONS::TEMPERATURE::CELSIUSTOFAHRENHEIT()")
[object ArangoQueryCursor, count: 1, hasMore: false, warning: 1541 - invalid
number of arguments for function 'MYFUNCTIONS::TEMPERATURE::CELSIUSTOFAHRENHEIT()',
expected number of arguments: minimum: 1, maximum: 1]
[
null
]
```
!SUBSECTION Deleting an existing AQL user function
@ -107,7 +145,7 @@ exception.
```js
require("@arangodb/aql/functions").unregister("myfunctions::temperature::celsiustofahrenheit");
require("@arangodb/aql/functions").unregister("MYFUNCTIONS::TEMPERATURE::CELSIUSTOFAHRENHEIT");
```
@ -128,9 +166,9 @@ This will return the number of functions unregistered.
```js
require("@arangodb/aql/functions").unregisterGroup("myfunctions::temperature");
require("@arangodb/aql/functions").unregisterGroup("MYFUNCTIONS::TEMPERATURE");
require("@arangodb/aql/functions").unregisterGroup("myfunctions");
require("@arangodb/aql/functions").unregisterGroup("MYFUNCTIONS");
```
@ -152,18 +190,17 @@ by specifying a group prefix:
To list all available user functions:
```js
require("@arangodb/aql/functions").toArray();
require("@arangodb/aql/functions").toArray();
```
To list all available user functions in the *myfunctions* namespace:
To list all available user functions in the *MYFUNCTIONS* namespace:
```js
require("@arangodb/aql/functions").toArray("myfunctions");
require("@arangodb/aql/functions").toArray("MYFUNCTIONS");
```
To list all available user functions in the *myfunctions::temperature* namespace:
To list all available user functions in the *MYFUNCTIONS::TEMPERATURE* namespace:
```js
require("@arangodb/aql/functions").toArray("myfunctions::temperature");
require("@arangodb/aql/functions").toArray("MYFUNCTIONS::TEMPERATURE");
```

36
Documentation/Books/AQL/Fundamentals/BindParameters.mdpp
View File

@ -14,9 +14,11 @@ The syntax for bind parameters is *@name* where *name* is the
actual parameter name. The bind parameter values need to be passed along with
the query when it is executed, but not as part of the query text itself.
FOR u IN users
FILTER u.id == @id && u.name == @name
RETURN u
```js
FOR u IN users
FILTER u.id == @id && u.name == @name
RETURN u
```
Bind parameter names must start with any of the letters *a* to *z* (both in
lower and upper case) or a digit (*0* to *9*), and can be followed by any
@ -24,24 +26,26 @@ letter, digit or the underscore symbol.
Bind variables represent a value like a string, and must not be put in quotes.
If you need to do string processing (concatenation, etc.) in the AQL query, you need
[to use string functions to do so](../Functions/String.md):
FOR u IN users
FILTER u.id == CONCAT('prefix', @id, 'suffix') && u.name == @name
RETURN u
to use [string functions](../Functions/String.md) to do so:
```js
FOR u IN users
FILTER u.id == CONCAT('prefix', @id, 'suffix') && u.name == @name
RETURN u
```
A special type of bind parameter exists for injecting collection names. This
type of bind parameter has a name prefixed with an additional *@* symbol (thus
when using the bind parameter in a query, two *@* symbols must be used).
FOR u IN @@collection
FILTER u.active == true
RETURN u
```js
FOR u IN @@collection
FILTER u.active == true
RETURN u
```
Specific information about parameters binding can also be found in:
Specific information about parameters binding can also be found in
[Aql With Web Interface](../Invocation/WithWebInterface.md) and
[Aql With Arangosh](../Invocation/WithArangosh.md), and
[HTTP Interface for AQL Queries](../../HTTP/AqlQuery/index.html)
- [AQL with Web Interface](../Invocation/WithWebInterface.md)
- [AQL with Arangosh](../Invocation/WithArangosh.md)
- [HTTP Interface for AQL Queries](../../HTTP/AqlQuery/index.html)

2
Documentation/Books/AQL/Invocation/WithWebInterface.mdpp
View File

@ -39,7 +39,7 @@ Bind parameters (JSON view mode):
}
```
How bind parameters work can be found in [Aql Fundamentals](../Fundamentals/BindParameters.md).
How bind parameters work can be found in [AQL Fundamentals](../Fundamentals/BindParameters.md).
Queries can also be saved in the AQL editor along with their bind parameter values
for later reuse. This data is stored in the user profile in the current database

3
Documentation/Books/AQL/Operators.mdpp
View File

@ -70,7 +70,8 @@ The pattern matching performed by the *LIKE* operator is case-sensitive.
The regular expression operators *=~* and *!~* expect their left-hand operands to
be strings, and their right-hand operands to be strings containing valid regular
expressions as specified in the documentation for the AQL function REGEX_TEST.
expressions as specified in the documentation for the AQL function
[REGEX_TEST()](Functions/String.md#regextest).
!SUBSUBSECTION Array comparison operators

2
Documentation/Books/HTTP/AqlUserFunctions/README.mdpp
View File

@ -7,7 +7,7 @@ user functions. AQL user functions are a means to extend the functionality
of ArangoDB's query language (AQL) with user-defined JavaScript code.
For an overview of how AQL user functions work, please refer to
[Extending Aql](../../AQL/Extending/index.html).
[Extending AQL](../../AQL/Extending/index.html).
The HTTP interface provides an API for adding, deleting, and listing
previously registered AQL user functions.

3
Documentation/Books/HTTP/book.json
View File

@ -23,9 +23,6 @@
"js": ["styles/header.js"],
"css": ["styles/header.css"]
},
"add-header": {
"BASE_PATH": "https://docs.arangodb.com/devel"
},
"piwik": {
"URL": "www.arangodb.com/piwik/",
"siteId": 12

6
Documentation/Books/Manual/Administration/Replication/Synchronous/Implementation.mdpp
View File

@ -4,7 +4,9 @@
Synchronous replication can be configured per collection via the property *replicationFactor*. Synchronous replication requires a cluster to operate.
Whenever you specify a *replicationFactor* greater than 1 when creating a collection, synchronous replication will be activated for this collection. The cluster will determine suitable *leaders* and *followers* for every requested shard (*numberOfShards*) within the cluster. When requesting data of a shard only the current leader will be asked whereas followers will only keep their copy in sync. Using *synchronous replication* alone will guarantee consistency and high availabilty at the cost of reduced performance (due to every write-request having to be executed on the followers).
Whenever you specify a *replicationFactor* greater than 1 when creating a collection, synchronous replication will be activated for this collection. The cluster will determine suitable *leaders* and *followers* for every requested shard (*numberOfShards*) within the cluster. When requesting data of a shard only the current leader will be asked whereas followers will only keep their copy in sync. This is due to the current implementation of transactions.
Using *synchronous replication* alone will guarantee consistency and high availabilty at the cost of reduced performance: Write requests will have a higher latency (due to every write-request having to be executed on the followers) and read requests won't scale out as only the leader is being asked.
In a cluster synchronous replication will be managed by the *coordinators* for the client. The data will always be stored on *primaries*.
@ -36,6 +38,8 @@ replicate it to the follower.
synchronous replication is resumed. This happens all transparently
to the client.
The current implementation of ArangoDB does not allow changing the replicationFactor later. This is subject to change. In the meantime the only way is to create a new collection having the desired `replicationFactor`, copy all data and swap names with the original one.
!SUBSECTION Automatic failover
Whenever the leader of a shard is failing and there is a query trying to access data of that shard the coordinator will continue trying to contact the leader until it timeouts. The internal cluster supervision will check cluster health every few seconds and will take action if there is no heartbeat from a server for 15 seconds. If the leader doesn't come back in time the supervision will reorganize the cluster by promoting for each shard a follower that is in sync with its leader to be the new leader. From then on the coordinators will contact the new leader.

2
Documentation/Books/Manual/DataModeling/Collections/CollectionMethods.mdpp
View File

@ -191,7 +191,7 @@ Loads a collection into memory.
!SUBSECTION Reserve
`collection.reserve( number)`
`collection.reserve(number)`
Sends a resize hint to the indexes in the collection. The resize hint allows indexes to reserve space for additional documents (specified by number) in one go.

31
Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.mdpp
View File

@ -60,49 +60,50 @@ to the [naming conventions](../NamingConventions/README.md).
*properties* must be an object with the following attributes:
* *waitForSync* (optional, default *false*): If *true* creating
- *waitForSync* (optional, default *false*): If *true* creating
a document will only return after the data was synced to disk.
* *journalSize* (optional, default is a
- *journalSize* (optional, default is a
configuration parameter: The maximal
size of a journal or datafile. Note that this also limits the maximal
size of a single object. Must be at least 1MB.
* *isSystem* (optional, default is *false*): If *true*, create a
- *isSystem* (optional, default is *false*): If *true*, create a
system collection. In this case *collection-name* should start with
an underscore. End users should normally create non-system collections
only. API implementors may be required to create system collections in
very special occasions, but normally a regular collection will do.
* *isVolatile* (optional, default is *false*): If *true* then the
- *isVolatile* (optional, default is *false*): If *true* then the
collection data is kept in-memory only and not made persistent. Unloading
the collection will cause the collection data to be discarded. Stopping
or re-starting the server will also cause full loss of data in the
collection. Setting this option will make the resulting collection be
collection. The collection itself will remain however (only the data is
volatile). Setting this option will make the resulting collection be
slightly faster than regular collections because ArangoDB does not
enforce any synchronization to disk and does not calculate any CRC
checksums for datafiles (as there are no datafiles).
* *keyOptions* (optional): additional options for key generation. If
- *keyOptions* (optional): additional options for key generation. If
specified, then *keyOptions* should be a JSON array containing the
following attributes (**note**: some of them are optional):
* *type*: specifies the type of the key generator. The currently
- *type*: specifies the type of the key generator. The currently
available generators are *traditional* and *autoincrement*.
* *allowUserKeys*: if set to *true*, then it is allowed to supply
- *allowUserKeys*: if set to *true*, then it is allowed to supply
own key values in the *_key* attribute of a document. If set to
*false*, then the key generator will solely be responsible for
generating keys and supplying own key values in the *_key* attribute
of documents is considered an error.
* *increment*: increment value for *autoincrement* key generator.
- *increment*: increment value for *autoincrement* key generator.
Not used for other key generator types.
* *offset*: initial offset value for *autoincrement* key generator.
- *offset*: initial offset value for *autoincrement* key generator.
Not used for other key generator types.
* *numberOfShards* (optional, default is *1*): in a cluster, this value
- *numberOfShards* (optional, default is *1*): in a cluster, this value
determines the number of shards to create for the collection. In a single
server setup, this option is meaningless.
* *shardKeys* (optional, default is *[ "_key" ]*): in a cluster, this
- *shardKeys* (optional, default is `[ "_key" ]`): in a cluster, this
attribute determines which document attributes are used to determine the
target shard for documents. Documents are sent to shards based on the
values they have in their shard key attributes. The values of all shard
@ -125,7 +126,7 @@ to the [naming conventions](../NamingConventions/README.md).
attribute and this can only be done efficiently if this is the
only shard key by delegating to the individual shards.
* *replicationFactor* (optional, default is 1): in a cluster, this
- *replicationFactor* (optional, default is 1): in a cluster, this
attribute determines how many copies of each shard are kept on
different DBServers. The value 1 means that only one copy (no
synchronous replication) is kept. A value of k means that
@ -209,9 +210,9 @@ for *waitForSync* is *false*.
*properties* must be an object with the following attributes:
* *waitForSync* (optional, default *false*): If *true* creating
- *waitForSync* (optional, default *false*): If *true* creating
a document will only return after the data was synced to disk.
* *journalSize* (optional, default is
- *journalSize* (optional, default is
"configuration parameter"): The maximal size of
a journal or datafile. Note that this also limits the maximal
size of a single object and must be at least 1MB.

16
Documentation/Books/Manual/DataModeling/Documents/README.mdpp
View File

@ -4,16 +4,6 @@ This is an introduction to ArangoDB's interface for working with
documents from the JavaScript shell *arangosh* or in JavaScript code in
the server. For other languages see the corresponding language API.
We begin with a
- [section on the basic approach](DocumentAddress.md)
before we proceed with
- [the detailed API description for collection objects](DocumentMethods.md)
and then
- [the detailed API description for database objects](DatabaseMethods.md)
- [Basics and Terminology](DocumentAddress.md): section on the basic approach
- [Collection Methods](DocumentMethods.md): detailed API description for collection objects
- [Database Methods](DatabaseMethods.md): detailed API description for database objects

9
Documentation/Books/Manual/DataModeling/GraphsVerticesEdges.mdpp
View File

@ -1,3 +1,10 @@
!CHAPTER Graphs, Vertices & Edges
Graphs, vertices & edges are defined in the [Graphs](../Graphs/README.md) chapter in details.
Graphs, vertices & edges are defined in the [Graphs](../Graphs/README.md) chapter in details.
Related blog posts:
- [Graphs in data modeling - is the emperor naked?](
https://medium.com/@neunhoef/graphs-in-data-modeling-is-the-emperor-naked-2e65e2744413#.x0a5z66ji
- [Index Free Adjacency or Hybrid Indexes for Graph Databases](
https://www.arangodb.com/2016/04/index-free-adjacency-hybrid-indexes-graph-databases/)

3
Documentation/Books/Manual/book.json
View File

@ -23,9 +23,6 @@
"js": ["styles/header.js"],
"css": ["styles/header.css"]
},
"add-header": {
"BASE_PATH": "https://docs.arangodb.com/devel"
},
"piwik": {
"URL": "www.arangodb.com/piwik/",
"siteId": 12

7
Documentation/Books/Manual/styles/header.js
View File

@ -9,13 +9,6 @@ document.addEventListener("DOMContentLoaded", function(event) {
}
});
$(document).ready(function() {
var bookVersion = gitbook.state.root.match(/\/(\d\.\d|devel)\//);
if (bookVersion) {
$(".arangodb-version-switcher").val(bookVersion[1]);
}
});
window.onload = function(){
window.localStorage.removeItem(":keyword");

4
Documentation/Books/stash/CollectionsAndDocuments.mdpp
View File

@ -140,7 +140,7 @@ Searching for all persons with an age above 30:
@endDocuBlock 06_workWithColl_AOQL_INT
John was put in there by mistake – so let's delete him again; We fetch
John was put in there by mistake – so let's delete him again. We fetch
the `_id` using *byExample*:
@startDocuBlockInline 07_workWithColl_remove
@ -153,7 +153,7 @@ the `_id` using *byExample*:
@endDocuBlock 07_workWithColl_remove
You can learn all about the query language [Aql](../../AQL/index.html). Note that
You can learn all about the query language [AQL](../../AQL/index.html). Note that
*_query* is a short-cut for *_createStatement* and *execute*. We will
come back to these functions when we talk about cursors.

11
README_maintainers.md
View File

@ -50,6 +50,14 @@ Debugging the build process
---------------------------
If the compile goes wrong for no particular reason, appending 'verbose=' adds more output. For some reason V8 has VERBOSE=1 for the same effect.
Temporary files and temp directories
------------------------------------
Depending on the native way ArangoDB tries to locate the temporary directory.
* Linux/Mac: the environment variable `TMPDIR` is evaluated.
* Windows: the [W32 API function GetTempPath()](https://msdn.microsoft.com/en-us/library/windows/desktop/aa364992%28v=vs.85%29.aspx) is called
* all platforms: `--temp.path` overrules the above system provided settings.
Runtime
-------
* start arangod with `--console` to get a debug console
@ -113,7 +121,7 @@ ArangoDB Unittesting Framework
Dependencies
------------
* *Ruby*, *rspec*, *httparty* to install the required dependencies run:
cd UnitTests/HttpInterface; bundler
`cd UnitTests/HttpInterface; bundler`
* boost_test (compile time)
@ -252,6 +260,7 @@ syntax --option value --sub:option value. Using Valgrind could look like this:
--extraargs:scheduler.threads 1 \
--extraargs:javascript.gc-frequency 1000000 \
--extraargs:javascript.gc-interval 65536 \
--javascript.v8-contexts 2 \
--valgrind /usr/bin/valgrind \
--valgrindargs:log-file /tmp/valgrindlog.%p

2
arangod/Aql/Functions.cpp
View File

@ -621,7 +621,7 @@ static AqlValue MergeParameters(arangodb::aql::Query* query,
// use the first argument as the preliminary result
AqlValue initial = ExtractFunctionParameterValue(trx, parameters, 0);
AqlValueMaterializer materializer(trx);
VPackSlice initialSlice = materializer.slice(initial, false);
VPackSlice initialSlice = materializer.slice(initial, true);
VPackBuilder builder;

4
arangod/RestHandler/RestDocumentHandler.cpp
View File

@ -392,7 +392,7 @@ bool RestDocumentHandler::modifyDocument(bool isPatch) {
}
VPackSlice keyInBody = body.get(StaticStrings::KeyString);
if ((revision != 0 && TRI_ExtractRevisionId(body) != revision) ||
keyInBody.isNone() ||
keyInBody.isNone() || keyInBody.isNull() ||
(keyInBody.isString() && keyInBody.copyString() != key)) {
// We need to rewrite the document with the given revision and key:
builder = std::make_shared<VPackBuilder>();
@ -430,7 +430,7 @@ bool RestDocumentHandler::modifyDocument(bool isPatch) {
generateTransactionError(collectionName, res, "");
return false;
}
OperationResult result(TRI_ERROR_NO_ERROR);
if (isPatch) {
// patching an existing document

213
arangod/RestHandler/RestReplicationHandler.cpp
View File

@ -1947,6 +1947,12 @@ int RestReplicationHandler::processRestoreIndexesCoordinator(
int res = TRI_ERROR_NO_ERROR;
for (VPackSlice const& idxDef : VPackArrayIterator(indexes)) {
VPackSlice type = idxDef.get("type");
if (type.isString() && (type.copyString() == "primary" || type.copyString() == "edge")) {
// must ignore these types of indexes during restore
continue;
}
VPackBuilder tmp;
res = ci->ensureIndexCoordinator(dbName, col->id_as_string(), idxDef, true,
arangodb::Index::Compare, tmp, errorMsg,
@ -2129,48 +2135,197 @@ int RestReplicationHandler::processRestoreDataBatch(
collectionName;
VPackBuilder builder;
VPackBuilder oldBuilder;
std::string const& bodyStr = _request->body();
char const* ptr = bodyStr.c_str();
char const* end = ptr + bodyStr.size();
while (ptr < end) {
char const* pos = strchr(ptr, '\n');
VPackBuilder allMarkers;
VPackValueLength currentPos = 0;
std::unordered_map<std::string, VPackValueLength> latest;
if (pos == nullptr) {
pos = end;
} else {
*((char*)pos) = '\0';
// First parse and collect all markers, we assemble everything in one
// large builder holding an array. We keep for each key the latest
// entry.
{
VPackArrayBuilder guard(&allMarkers);
while (ptr < end) {
char const* pos = strchr(ptr, '\n');
if (pos == nullptr) {
pos = end;
} else {
*((char*)pos) = '\0';
}
if (pos - ptr > 1) {
// found something
std::string key;
std::string rev;
VPackSlice doc;
TRI_replication_operation_e type = REPLICATION_INVALID;
int res = restoreDataParser(ptr, pos, invalidMsg, useRevision, errorMsg,
key, rev, builder, doc, type);
if (res != TRI_ERROR_NO_ERROR) {
return res;
}
// Put into array of all parsed markers:
allMarkers.add(builder.slice());
auto it = latest.find(key);
if (it != latest.end()) {
// Already found, overwrite:
it->second = currentPos;
} else {
latest.emplace(std::make_pair(key, currentPos));
}
++currentPos;
}
ptr = pos + 1;
}
}
if (pos - ptr > 1) {
// found something
std::string key;
std::string rev;
VPackSlice doc;
// First remove all keys of which the last marker we saw was a deletion
// marker:
VPackSlice allMarkersSlice = allMarkers.slice();
VPackBuilder oldBuilder;
{
VPackArrayBuilder guard(&oldBuilder);
for (auto const& p : latest) {
VPackSlice const marker = allMarkersSlice.at(p.second);
VPackSlice const typeSlice = marker.get("type");
TRI_replication_operation_e type = REPLICATION_INVALID;
int res = restoreDataParser(ptr, pos, invalidMsg, useRevision, errorMsg,
key, rev, builder, doc, type);
if (res != TRI_ERROR_NO_ERROR) {
return res;
if (typeSlice.isNumber()) {
int typeInt = typeSlice.getNumericValue<int>();
if (typeInt == 2301) { // pre-3.0 type for edges
type = REPLICATION_MARKER_DOCUMENT;
} else {
type = static_cast<TRI_replication_operation_e>(typeInt);
}
}
oldBuilder.clear();
oldBuilder.openObject();
oldBuilder.add(StaticStrings::KeyString, VPackValue(key));
oldBuilder.close();
res = applyCollectionDumpMarker(trx, resolver, collectionName, type,
oldBuilder.slice(), doc, errorMsg);
if (res != TRI_ERROR_NO_ERROR && !force) {
return res;
if (type == REPLICATION_MARKER_REMOVE) {
oldBuilder.add(VPackValue(p.first)); // Add _key
} else if (type != REPLICATION_MARKER_DOCUMENT) {
errorMsg = "unexpected marker type " + StringUtils::itoa(type);
return TRI_ERROR_REPLICATION_UNEXPECTED_MARKER;
}
}
}
ptr = pos + 1;
// Note that we ignore individual errors here, as long as the main
// operation did not fail. In particular, we intentionally ignore
// individual "DOCUMENT NOT FOUND" errors, because they can happen!
try {
OperationOptions options;
options.silent = true;
options.ignoreRevs = true;
options.isRestore = true;
options.waitForSync = false;
OperationResult opRes = trx.remove(collectionName, oldBuilder.slice(),
options);
if (!opRes.successful()) {
return opRes.code;
}
} catch (arangodb::basics::Exception const& ex) {
return ex.code();
} catch (...) {
return TRI_ERROR_INTERNAL;
}
// Now try to insert all keys for which the last marker was a document
// marker, note that these could still be replace markers!
std::vector<VPackValueLength> insertTried;
builder.clear();
{
VPackArrayBuilder guard(&builder);
for (auto const& p : latest) {
VPackSlice const marker = allMarkersSlice.at(p.second);
VPackSlice const typeSlice = marker.get("type");
TRI_replication_operation_e type = REPLICATION_INVALID;
if (typeSlice.isNumber()) {
int typeInt = typeSlice.getNumericValue<int>();
if (typeInt == 2301) { // pre-3.0 type for edges
type = REPLICATION_MARKER_DOCUMENT;
} else {
type = static_cast<TRI_replication_operation_e>(typeInt);
}
}
if (type == REPLICATION_MARKER_DOCUMENT) {
VPackSlice const doc = marker.get("data");
TRI_ASSERT(doc.isObject());
builder.add(doc);
}
}
}
VPackSlice requestSlice = builder.slice();
OperationResult opRes;
try {
OperationOptions options;
options.silent = false;
options.ignoreRevs = true;
options.isRestore = true;
options.waitForSync = false;
opRes = trx.insert(collectionName, requestSlice, options);
if (!opRes.successful()) {
return opRes.code;
}
} catch (arangodb::basics::Exception const& ex) {
return ex.code();
} catch (...) {
return TRI_ERROR_INTERNAL;
}
// Now go through the individual results and check each error, if it was
// TRI_ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATED, then we have to call
// replace on the document:
VPackSlice resultSlice = opRes.slice();
VPackBuilder replBuilder; // documents for replace operation
{
VPackArrayBuilder guard(&oldBuilder);
VPackArrayBuilder guard2(&replBuilder);
VPackArrayIterator itRequest(requestSlice);
VPackArrayIterator itResult(resultSlice);
while (itRequest.valid()) {
VPackSlice result = *itResult;
VPackSlice error = result.get("error");
if (error.isTrue()) {
error = result.get("errorNum");
if (error.isNumber()) {
int code = error.getNumericValue<int>();
if (code == TRI_ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATED) {
replBuilder.add(*itRequest);
} else {
return code;
}
} else {
return TRI_ERROR_INTERNAL;
}
}
itRequest.next();
itResult.next();
}
}
try {
OperationOptions options;
options.silent = true;
options.ignoreRevs = true;
options.isRestore = true;
options.waitForSync = false;
opRes = trx.replace(collectionName, replBuilder.slice(), options);
if (!opRes.successful()) {
return opRes.code;
}
} catch (arangodb::basics::Exception const& ex) {
return ex.code();
} catch (...) {
return TRI_ERROR_INTERNAL;
}
return TRI_ERROR_NO_ERROR;

7
arangod/V8Server/V8PeriodicTask.cpp
View File

@ -41,6 +41,7 @@ Mutex V8PeriodicTask::RUNNING_LOCK;
void V8PeriodicTask::jobDone(Task* task) {
try {
MUTEX_LOCKER(guard, V8PeriodicTask::RUNNING_LOCK);
RUNNING.erase(task);
} catch (...) {
// ignore any memory error
@ -48,7 +49,7 @@ void V8PeriodicTask::jobDone(Task* task) {
}
V8PeriodicTask::V8PeriodicTask(std::string const& id, std::string const& name,
TRI_vocbase_t* vocbase,
TRI_vocbase_t* vocbase,
double offset, double period,
std::string const& command,
std::shared_ptr<VPackBuilder> parameters,
@ -87,7 +88,7 @@ bool V8PeriodicTask::handlePeriod() {
LOG(WARN) << "could not add task " << _command << ", no dispatcher known";
return false;
}
{
MUTEX_LOCKER(guard, V8PeriodicTask::RUNNING_LOCK);
@ -102,7 +103,7 @@ bool V8PeriodicTask::handlePeriod() {
std::unique_ptr<Job> job(new V8Job(
_vocbase, "(function (params) { " + _command + " } )(params);",
_parameters, _allowUseDatabase, this));
DispatcherFeature::DISPATCHER->addJob(job, false);
return true;

4
arangod/VocBase/document-collection.cpp
View File

@ -3365,7 +3365,7 @@ int TRI_document_collection_t::update(Transaction* trx,
if (!newSlice.isObject()) {
return TRI_ERROR_ARANGO_DOCUMENT_TYPE_INVALID;
}
// initialize the result
TRI_ASSERT(mptr != nullptr);
mptr->setVPack(nullptr);
@ -3832,7 +3832,7 @@ int TRI_document_collection_t::lookupDocument(
TRI_doc_mptr_t*& header) {
if (!key.isString()) {
return TRI_ERROR_INTERNAL;
return TRI_ERROR_ARANGO_DOCUMENT_KEY_BAD;
}
header = primaryIndex()->lookupKey(trx, key);

5
arangod/VocBase/vocbase.cpp
View File

@ -1189,12 +1189,9 @@ void TRI_vocbase_col_t::toVelocyPackIndexes(VPackBuilder& builder,
for (auto const& file : files) {
if (StringUtils::isPrefix(file, "index-") &&
StringUtils::isSuffix(file, ".json")) {
// TODO: fix memleak
char* fqn = TRI_Concatenate2File(_path.c_str(), file.c_str());
std::string path = std::string(fqn, strlen(fqn));
std::string path = basics::FileUtils::buildFilename(_path, file);
std::shared_ptr<VPackBuilder> indexVPack =
arangodb::basics::VelocyPackHelper::velocyPackFromFile(path);
TRI_FreeString(TRI_CORE_MEM_ZONE, fqn);
VPackSlice const indexSlice = indexVPack->slice();
VPackSlice const id = indexSlice.get("id");

11
arangod/Wal/CollectorThread.cpp
View File

@ -409,6 +409,7 @@ int CollectorThread::collectLogfiles(bool& worked) {
#ifdef ARANGODB_ENABLE_ROCKSDB
RocksDBFeature::syncWal();
#endif
_logfileManager->setCollectionDone(logfile);
} else {
// return the logfile to the logfile manager in case of errors
@ -803,6 +804,11 @@ int CollectorThread::collect(Logfile* logfile) {
try {
res = transferMarkers(logfile, cid, state.collections[cid],
state.operationsCount[cid], sortedOperations);
TRI_IF_FAILURE("failDuringCollect") {
THROW_ARANGO_EXCEPTION(TRI_ERROR_DEBUG);
}
} catch (arangodb::basics::Exception const& ex) {
res = ex.code();
} catch (...) {
@ -872,6 +878,11 @@ int CollectorThread::transferMarkers(Logfile* logfile,
try {
res = executeTransferMarkers(document, cache, operations);
TRI_IF_FAILURE("transferMarkersCrash") {
// intentionally kill the server
TRI_SegfaultDebugging("CollectorThreadTransfer");
}
if (res == TRI_ERROR_NO_ERROR && !cache->operations->empty()) {
// now sync the datafile

4
arangod/Wal/LogfileManager.cpp
View File

@ -1500,6 +1500,10 @@ void LogfileManager::setCollectionRequested(Logfile* logfile) {
// mark a file as being done with collection
void LogfileManager::setCollectionDone(Logfile* logfile) {
TRI_IF_FAILURE("setCollectionDone") {
return;
}
TRI_ASSERT(logfile != nullptr);
Logfile::IdType id = logfile->id();

28
arangod/Wal/RecoverState.cpp
View File

@ -493,8 +493,8 @@ bool RecoverState::ReplayMarker(TRI_df_marker_t const* marker, void* data,
int res = opRes.code;
if (res == TRI_ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATED) {
// document/edge already exists, now make it an update
opRes = trx->update(collectionName, VPackSlice(ptr), options);
// document/edge already exists, now make it a replace
opRes = trx->replace(collectionName, VPackSlice(ptr), options);
res = opRes.code;
}
@ -547,16 +547,30 @@ bool RecoverState::ReplayMarker(TRI_df_marker_t const* marker, void* data,
options.silent = true;
options.recoveryMarker = envelope;
options.waitForSync = false;
options.ignoreRevs = true;
OperationResult opRes = trx->remove(collectionName, VPackSlice(ptr), options);
int res = opRes.code;
return res;
try {
OperationResult opRes = trx->remove(collectionName, VPackSlice(ptr), options);
if (opRes.code == TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND) {
// document to delete is not present. this error can be ignored
return TRI_ERROR_NO_ERROR;
}
return opRes.code;
} catch (arangodb::basics::Exception const& ex) {
if (ex.code() == TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND) {
// document to delete is not present. this error can be ignored
return TRI_ERROR_NO_ERROR;
}
return ex.code();
}
// should not get here...
return TRI_ERROR_INTERNAL;
});
if (res != TRI_ERROR_NO_ERROR && res != TRI_ERROR_ARANGO_CONFLICT &&
res != TRI_ERROR_ARANGO_DATABASE_NOT_FOUND &&
res != TRI_ERROR_ARANGO_COLLECTION_NOT_FOUND) {
res != TRI_ERROR_ARANGO_COLLECTION_NOT_FOUND &&
res != TRI_ERROR_ARANGO_DOCUMENT_NOT_FOUND) {
LOG(WARN) << "unable to remove document in collection " << collectionId << " of database " << databaseId << ": " << TRI_errno_string(res);
++state->errorCount;
return state->canContinue();

20
js/apps/system/_admin/aardvark/APP/frontend/build/app.min.js vendored
View File

File diff suppressed because one or more lines are too long

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/app.min.js.gz
View File

Binary file not shown.

10
js/apps/system/_admin/aardvark/APP/frontend/build/index-min.html
View File

@ -1065,14 +1065,14 @@ if (list.length > 0) {
</div>
</div>
<div class="pure-g pure-table pure-table-body"> <% _.each(general, function(val, key) { %> <div class="<%= genClass %> left"><%=key%></div> <% if (val.type === 'select') { %> <div class="<%= genClass %> left">
<select> <% _.each(val, function(option, optKey) { %> <% if (option.name) { %> <option> <%=option.name%> </option> <% } %> <% }); %> </select>
</div> <% } %> <% if (val.type === 'numeric') { %> <input type="text" id="<%=val %>" value="<%=val.value %>" placeholder=""></input> <% } %> <% }); %> </div>
<div class="pure-g pure-table pure-table-body"> <% _.each(general, function(val, key) { %> <div class="<%= genClass %> left"><%=val.name%></div> <% if (val.type === 'select') { %> <div class="<%= genClass %> left">
<select id="g_<%=key%>"> <% _.each(val, function(option, optKey) { %> <% if (option.name) { %> <option value="<%=option.val%>"> <%=option.name%> </option> <% } %> <% }); %> </select>
</div> <% } %> <% if (val.type === 'numeric') { %> <input id="g_<%=key%>" type="text" id="<%=val %>" value="<%=val.value %>" placeholder=""></input> <% } %> <% }); %> </div>
</div>
<button id="saveGraphSettings" style="margin-top: 20px;" class="button-success pull-right">Save</button>
<button id="saveGraphSettings" style="margin-top: 20px;" class="button-success pull-right">Restore defaults</button>
<button id="restoreGraphSettings" style="margin-top: 20px;" class="button-success pull-right">Restore defaults</button>
</div></script><script id="graphViewer2.ejs" type="text/template"><div class="graphContent" id="graphContainer">
<div class="headerBar">
@ -2723,4 +2723,4 @@ var cutByResolution = function (str) {
</div>
<div id="workMonitorContent" class="innerContent">
</div></script></head><body><nav class="navbar" style="display: none"><div class="primary"><div class="navlogo"><a class="logo big" href="#"><img class="arangodbLogo" src="img/arangodb_logo_big.png"></a><a class="logo small" href="#"><img class="arangodbLogo" src="img/arangodb_logo_small.png"></a><a class="version"><span>VERSION: </span><span id="currentVersion"></span></a></div><div class="statmenu" id="statisticBar"></div><div class="navmenu" id="navigationBar"></div></div></nav><div id="modalPlaceholder"></div><div class="bodyWrapper" style="display: none"><div class="centralRow"><div id="navbar2" class="navbarWrapper secondary"><div class="subnavmenu" id="subNavigationBar"></div></div><div class="resizecontainer contentWrapper"><div id="loadingScreen" class="loadingScreen" style="display: none"><i class="fa fa-circle-o-notch fa-spin fa-3x fa-fw margin-bottom"></i> <span class="sr-only">Loading...</span></div><div id="content" class="centralContent"></div><footer class="footer"><div id="footerBar"></div></footer></div></div></div><div id="progressPlaceholder" style="display:none"></div><div id="spotlightPlaceholder" style="display:none"></div><div id="offlinePlaceholder" style="display:none"><div class="offline-div"><div class="pure-u"><div class="pure-u-1-4"></div><div class="pure-u-1-2 offline-window"><div class="offline-header"><h3>You have been disconnected from the server</h3></div><div class="offline-body"><p>The connection to the server has been lost. The server may be under heavy load.</p><p>Trying to reconnect in <span id="offlineSeconds">10</span> seconds.</p><p class="animation_state"><span><button class="button-success">Reconnect now</button></span></p></div></div><div class="pure-u-1-4"></div></div></div></div><div class="arangoFrame" style=""><div class="outerDiv"><div class="innerDiv"></div></div></div><script src="libs.js?version=1467901818158"></script><script src="app.js?version=1467901818158"></script></body></html>
</div></script></head><body><nav class="navbar" style="display: none"><div class="primary"><div class="navlogo"><a class="logo big" href="#"><img class="arangodbLogo" src="img/arangodb_logo_big.png"></a><a class="logo small" href="#"><img class="arangodbLogo" src="img/arangodb_logo_small.png"></a><a class="version"><span>VERSION: </span><span id="currentVersion"></span></a></div><div class="statmenu" id="statisticBar"></div><div class="navmenu" id="navigationBar"></div></div></nav><div id="modalPlaceholder"></div><div class="bodyWrapper" style="display: none"><div class="centralRow"><div id="navbar2" class="navbarWrapper secondary"><div class="subnavmenu" id="subNavigationBar"></div></div><div class="resizecontainer contentWrapper"><div id="loadingScreen" class="loadingScreen" style="display: none"><i class="fa fa-circle-o-notch fa-spin fa-3x fa-fw margin-bottom"></i> <span class="sr-only">Loading...</span></div><div id="content" class="centralContent"></div><footer class="footer"><div id="footerBar"></div></footer></div></div></div><div id="progressPlaceholder" style="display:none"></div><div id="spotlightPlaceholder" style="display:none"></div><div id="offlinePlaceholder" style="display:none"><div class="offline-div"><div class="pure-u"><div class="pure-u-1-4"></div><div class="pure-u-1-2 offline-window"><div class="offline-header"><h3>You have been disconnected from the server</h3></div><div class="offline-body"><p>The connection to the server has been lost. The server may be under heavy load.</p><p>Trying to reconnect in <span id="offlineSeconds">10</span> seconds.</p><p class="animation_state"><span><button class="button-success">Reconnect now</button></span></p></div></div><div class="pure-u-1-4"></div></div></div></div><div class="arangoFrame" style=""><div class="outerDiv"><div class="innerDiv"></div></div></div><script src="libs.js?version=1468309999760"></script><script src="app.js?version=1468309999760"></script></body></html>

BIN
js/apps/system/_admin/aardvark/APP/frontend/build/index-min.html.gz
View File

Binary file not shown.

69
js/apps/system/_admin/aardvark/APP/frontend/js/models/userConfig.js Normal file
View File

@ -0,0 +1,69 @@
/* jshint strict: false */
/* global Backbone, window, arangoHelper, $ */
window.UserConfig = Backbone.Model.extend({
defaults: {
graphs: '',
queries: []
},
model: window.UserConfigModel,
parse: function (response) {
return response.result;
},
url: function () {
if (window.App.currentUser) {
this.username = window.App.currentUser;
} else {
this.username = 'root';
}
return arangoHelper.databaseUrl('/_api/user/' + encodeURIComponent(this.username) + '/config');
},
setItem: function (keyName, keyValue, callback) {
// url PUT /_api/user/<username>/config/<key>
var self = this;
$.ajax({
type: 'PUT',
cache: false,
url: arangoHelper.databaseUrl('/_api/user/' + encodeURIComponent(this.username) + '/config/' + encodeURIComponent(keyName)),
contentType: 'application/json',
processData: false,
data: JSON.stringify({value: keyValue}),
async: true,
success: function () {
self.set(keyName, keyValue);
if (callback) {
callback();
}
},
error: function () {
arangoHelper.arangoNotification('User configuration', 'Could not update user configuration for key: ' + keyName);
}
});
},
getItem: function (keyName, callback) {
// url GET /_api/user/<username>/config/<key>
$.ajax({
type: 'GET',
cache: false,
url: arangoHelper.databaseUrl('/_api/user/' + encodeURIComponent(this.username) + '/config/' + encodeURIComponent(keyName)),
contentType: 'application/json',
processData: false,
async: true,
success: function (keyValue) {
callback(keyValue);
},
error: function () {
arangoHelper.arangoNotification('User configuration', 'Could not fetch user configuration for key: ' + keyName);
}
});
}
});

9
js/apps/system/_admin/aardvark/APP/frontend/js/routers/router.js
View File

@ -210,6 +210,9 @@
window.checkVersion();
this.userConfig = new window.UserConfig();
this.userConfig.fetch();
this.documentsView = new window.DocumentsView({
collection: new window.ArangoDocuments(),
documentStore: this.arangoDocumentStore,
@ -669,7 +672,8 @@
return;
}
this.graphViewer2 = new window.GraphViewer2({
name: name
name: name,
userConfig: this.userConfig
});
this.graphViewer2.render();
},
@ -681,7 +685,8 @@
return;
}
this.graphSettingsView = new window.GraphSettingsView({
name: name
name: name,
userConfig: this.userConfig
});
this.graphSettingsView.render();
},

10
js/apps/system/_admin/aardvark/APP/frontend/js/templates/graphSettingsView.ejs
View File

@ -87,14 +87,14 @@
<div class="pure-g pure-table pure-table-body">
<% _.each(general, function(val, key) { %>
<div class="<%= genClass %> left"><%=key%></div>
<div class="<%= genClass %> left"><%=val.name%></div>
<% if (val.type === 'select') { %>
<div class="<%= genClass %> left">
<select>
<select id="g_<%=key%>">
<% _.each(val, function(option, optKey) { %>
<% if (option.name) { %>
<option> <%=option.name%> </option>
<option value="<%=option.val%>"> <%=option.name%> </option>
<% } %>
<% }); %>
</select>
@ -102,7 +102,7 @@
<% } %>
<% if (val.type === 'numeric') { %>
<input type="text" id="<%=val %>" value="<%=val.value %>" placeholder=""></input>
<input id="g_<%=key%>" type="text" id="<%=val %>" value="<%=val.value %>" placeholder=""></input>
<% } %>
<% }); %>
@ -111,7 +111,7 @@
</div>
<button id="saveGraphSettings" style="margin-top: 20px;" class="button-success pull-right">Save</button>
<button id="saveGraphSettings" style="margin-top: 20px;" class="button-success pull-right">Restore defaults</button>
<button id="restoreGraphSettings" style="margin-top: 20px;" class="button-success pull-right">Restore defaults</button>
</div>
</script>

84
js/apps/system/_admin/aardvark/APP/frontend/js/views/graphSettingsView.js
View File

@ -1,6 +1,6 @@
/* jshint browser: true */
/* jshint unused: false */
/* global arangoHelper, Backbone, templateEngine, $, window*/
/* global arangoHelper, Backbone, templateEngine, $, window, _ */
(function () {
'use strict';
@ -8,29 +8,37 @@
el: '#content',
general: {
'Layout': {
'layout': {
type: 'select',
name: 'Layout algorithm',
noverlap: {
name: 'No overlap (fast)'
name: 'No overlap (fast)',
val: 'noverlap'
},
force: {
name: 'Force (slow)'
name: 'Force (slow)',
val: 'force'
},
fruchtermann: {
name: 'Fruchtermann (very slow)'
name: 'Fruchtermann (very slow)',
val: 'fruchtermann'
}
},
'Renderer': {
'renderer': {
type: 'select',
name: 'Renderer',
canvas: {
name: 'Canvas (editable)'
name: 'Canvas (editable)',
val: 'canvas'
},
webgl: {
name: 'WebGL (only display)'
name: 'WebGL (only display)',
val: 'webgl'
}
},
'depth': {
type: 'numeric',
name: 'Search depth',
value: 2
}
},
@ -89,24 +97,68 @@
initialize: function (options) {
this.name = options.name;
},
loadGraphSettings: function () {
},
saveGraphSettings: function () {
this.userConfig = options.userConfig;
},
events: {
'click #saveGraphSettings': 'saveGraphSettings',
'click #restoreGraphSettings': 'restoreGraphSettings'
},
getGraphSettings: function (render) {
var self = this;
var combinedName = window.App.currentDB.toJSON().name + '_' + this.name;
this.userConfig.fetch({
success: function (data) {
self.graphConfig = data.toJSON().graphs[combinedName];
if (render) {
self.continueRender();
}
}
});
},
saveGraphSettings: function () {
var combinedName = window.App.currentDB.toJSON().name + '_' + this.name;
var config = {};
config[combinedName] = {
layout: $('#g_layout').val(),
renderer: $('#g_renderer').val(),
depth: $('#g_depth').val()
};
var callback = function () {
window.arangoHelper.arangoNotification('Graph ' + this.name, 'Configuration saved.');
}.bind(this);
this.userConfig.setItem('graphs', config, callback);
},
setDefaults: function () {
},
render: function () {
this.getGraphSettings(true);
},
continueRender: function () {
$(this.el).html(this.template.render({
general: this.general,
specific: this.specific
}));
if (this.graphConfig) {
_.each(this.graphConfig, function (val, key) {
$('#g_' + key).val(val);
});
} else {
this.setDefaults();
}
arangoHelper.buildGraphSubNav(this.name, 'Settings');
// load graph settings from local storage

68
js/apps/system/_admin/aardvark/APP/frontend/js/views/graphViewer2.js
View File

@ -11,6 +11,7 @@
initialize: function (options) {
this.name = options.name;
this.userConfig = options.userConfig;
this.initSigma();
},
@ -52,7 +53,6 @@
render: function () {
this.$el.html(this.template.render({}));
arangoHelper.buildGraphSubNav(this.name, 'Content');
this.resize();
this.fetchGraph();
@ -60,26 +60,30 @@
fetchGraph: function () {
var self = this;
arangoHelper.buildGraphSubNav(self.name, 'Content');
$('#content').append(
'<div id="calculatingGraph" style="position: absolute; left: 25px; top: 130px;">' +
'<i class="fa fa-circle-o-notch fa-spin" style="margin-right: 10px;"></i>' +
'Calculating layout. Please wait ... </div>'
);
// TODO LOAD GRAPH SETTINGS
// var settings = this.loadGraphSettings();
var fetchGraph = function () {
$.ajax({
type: 'GET',
url: arangoHelper.databaseUrl('/_admin/aardvark/graph/' + encodeURIComponent(this.name)),
contentType: 'application/json',
success: function (data) {
arangoHelper.buildGraphSubNav(self.name, 'Content');
self.renderGraph(data);
},
error: function () {
$('#calculatingGraph').html('Failed to fetch graph information.');
}
});
}.bind(this);
$.ajax({
type: 'GET',
url: arangoHelper.databaseUrl('/_admin/aardvark/graph/' + encodeURIComponent(this.name)),
contentType: 'application/json',
success: function (data) {
self.renderGraph(data);
},
error: function () {
$('#calculatingGraph').html('Failed to fetch graph information.');
}
});
// TODO LOAD GRAPH SETTINGS
this.getGraphSettings(fetchGraph);
},
clearOldContextMenu: function () {
@ -145,10 +149,19 @@
generateMenu(e, nodeId);
},
loadGraphSettings: function () {
var settings;
getGraphSettings: function (callback) {
var self = this;
var combinedName = window.App.currentDB.toJSON().name + '_' + this.name;
return settings;
this.userConfig.fetch({
success: function (data) {
self.graphConfig = data.toJSON().graphs[combinedName];
if (callback) {
callback();
}
}
});
},
editNode: function (id) {
@ -166,8 +179,25 @@
this.Sigma = sigma;
var algorithm = 'force';
var renderer = 'webgl';
// defaults
var algorithm = 'noverlap';
var renderer = 'canvas';
if (this.graphConfig) {
console.log(this.graphConfig);
if (this.graphConfig.layout) {
algorithm = this.graphConfig.layout;
}
if (this.graphConfig.renderer) {
renderer = this.graphConfig.renderer;
if (renderer === 'canvas') {
self.isEditable = true;
}
}
}
var settings = {
doubleClickEnabled: false,

2
js/apps/system/_admin/aardvark/APP/frontend/scss/_graphViewer2.scss
View File

@ -31,7 +31,7 @@
}
.fa {
&.fa-television {
&.fa-desktop {
margin-top: 6px;
position: absolute;
right: 20px;

23
js/client/modules/@arangodb/testing.js
View File

@ -148,6 +148,7 @@ const optionsDefaults = {
'loopEternal': false,
'loopSleepSec': 1,
'loopSleepWhen': 1,
'maxPort': 32768,
'onlyNightly': false,
'password': '',
'replication': false,
@ -593,9 +594,15 @@ function cleanupDBDirectories (options) {
// / @brief finds a free port
// //////////////////////////////////////////////////////////////////////////////
function findFreePort () {
function findFreePort (maxPort) {
if (typeof maxPort !== 'number') {
maxPort = 32768;
}
if (maxPort < 2048) {
maxPort = 2048;
}
while (true) {
const port = Math.floor(Math.random() * (65536 - 1024)) + 1024;
const port = Math.floor(Math.random() * (maxPort - 1024)) + 1024;
const free = testPort('tcp://0.0.0.0:' + port);
if (free) {
@ -1276,7 +1283,7 @@ function startInstanceCluster (instanceInfo, protocol, options,
let agencyEndpoint = instanceInfo.endpoint;
let i;
for (i = 0; i < options.clusterNodes; i++) {
let endpoint = protocol + '://127.0.0.1:' + findFreePort();
let endpoint = protocol + '://127.0.0.1:' + findFreePort(options.maxPort);
let primaryArgs = _.clone(options.extraArgs);
primaryArgs['server.endpoint'] = endpoint;
primaryArgs['cluster.my-address'] = endpoint;
@ -1287,7 +1294,7 @@ function startInstanceCluster (instanceInfo, protocol, options,
startInstanceSingleServer(instanceInfo, protocol, options, ...makeArgs('dbserver' + i, primaryArgs));
}
let endpoint = protocol + '://127.0.0.1:' + findFreePort();
let endpoint = protocol + '://127.0.0.1:' + findFreePort(options.maxPort);
let coordinatorArgs = _.clone(options.extraArgs);
coordinatorArgs['server.endpoint'] = endpoint;
coordinatorArgs['cluster.my-address'] = endpoint;
@ -1341,7 +1348,7 @@ function startArango (protocol, options, addArgs, name, rootDir, isAgency) {
let port;
if (!addArgs['server.endpoint']) {
port = findFreePort();
port = findFreePort(options.maxPort);
endpoint = protocol + '://127.0.0.1:' + port;
} else {
endpoint = addArgs['server.endpoint'];
@ -1416,7 +1423,7 @@ function startInstanceAgency (instanceInfo, protocol, options,
instanceArgs['database.directory'] = dataDir + String(i);
if (i === N - 1) {
const port = findFreePort();
const port = findFreePort(options.maxPort);
instanceArgs['server.endpoint'] = 'tcp://127.0.0.1:' + port;
let l = [];
instanceInfo.arangods.forEach(arangod => {
@ -3062,6 +3069,8 @@ function runArangodRecovery (instanceInfo, options, script, setup) {
}
const recoveryTests = [
'insert-update-replace',
'die-during-collector',
'disk-full-logfile',
'disk-full-logfile-data',
'disk-full-datafile',
@ -3653,7 +3662,7 @@ testFuncs.upgrade = function (options) {
fs.makeDirectoryRecursive(tmpDataDir);
const appDir = fs.join(tmpDataDir, 'app');
const port = findFreePort();
const port = findFreePort(options.maxPort);
let args = makeArgsArangod(options, appDir);
args['server.endpoint'] = 'tcp://127.0.0.1:' + port;

2
js/server/modules/@arangodb/replication.js
View File

@ -327,7 +327,7 @@ function syncCollectionFinalize (database, collname, from, config) {
response: chunk, exception: err};
}
console.log('Applying chunk:');
console.trace('Applying chunk:', l);
try {
for (var i = 0; i < l.length; i++) {
apply(l[i]);

12
js/server/tests/aql/aql-vpack-externals.js
View File

@ -1,5 +1,5 @@
/*jshint globalstrict:false, strict:false, sub: true, maxlen: 500 */
/*global assertEqual, assertTrue */
/*global assertEqual, assertFalse, assertTrue */
////////////////////////////////////////////////////////////////////////////////
/// @brief tests for query language, graph functions
@ -188,6 +188,16 @@ function aqlVPackExternalsTestSuite () {
const query = `LET us = (FOR u1 IN ${collName} FILTER u1.username == "test1" FOR u2 IN ${collName} FILTER u2.username == "test2" RETURN { u1, u2 }) FOR u IN us FOR msg IN ${edgeColl} FILTER msg._from == u.u1._id && msg._to == u.u2._id RETURN msg._id`;
const result = db._query(query).toArray();
assertEqual(edgeColl + "/test1", result[0]);
},
testExternalInTraversalMerge: function () {
const query = `LET s = (FOR n IN OUTBOUND "${collName}/test1000" ${edgeColl} RETURN n) RETURN MERGE(s)`;
const cursor = db._query(query);
const doc = cursor.next();
assertTrue(doc.hasOwnProperty('_key'));
assertTrue(doc.hasOwnProperty('_rev'));
assertTrue(doc.hasOwnProperty('_id'));
assertFalse(cursor.hasNext());
}
};

116
js/server/tests/recovery/die-during-collector.js Normal file
View File

@ -0,0 +1,116 @@
/* jshint globalstrict:false, strict:false, unused : false */
/* global assertEqual, assertFalse */
// //////////////////////////////////////////////////////////////////////////////
// / @brief tests for transactions
// /
// / @file
// /
// / DISCLAIMER
// /
// / Copyright 2010-2012 triagens GmbH, Cologne, Germany
// /
// / Licensed under the Apache License, Version 2.0 (the "License")
// / you may not use this file except in compliance with the License.
// / You may obtain a copy of the License at
// /
// / http://www.apache.org/licenses/LICENSE-2.0
// /
// / Unless required by applicable law or agreed to in writing, software
// / distributed under the License is distributed on an "AS IS" BASIS,
// / WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// / See the License for the specific language governing permissions and
// / limitations under the License.
// /
// / Copyright holder is triAGENS GmbH, Cologne, Germany
// /
// / @author Jan Steemann
// / @author Copyright 2013, triAGENS GmbH, Cologne, Germany
// //////////////////////////////////////////////////////////////////////////////
var db = require('@arangodb').db;
var internal = require('internal');
var jsunity = require('jsunity');
function runSetup () {
'use strict';
internal.debugClearFailAt();
db._drop('UnitTestsRecovery');
var c = db._create('UnitTestsRecovery'), i;
for (i = 0; i < 1000; ++i) {
c.save({ _key: 'test' + i });
}
internal.wal.flush(true, true, true);
for (i = 0; i < 1000; ++i) {
c.remove('test' + i);
}
internal.debugSetFailAt('setCollectionDone');
internal.wal.flush(true, false, false);
// wait until datafile appears
while (true) {
if (c.figures().dead.deletion === 1000) {
break;
}
internal.wait(0.5, false);
}
c.rotate();
// wait until compactor has finished
while (true) {
if (c.figures().datafiles.count === 0) {
break;
}
internal.wait(0.5, false);
}
internal.debugSegfault('crashing server');
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief test suite
// //////////////////////////////////////////////////////////////////////////////
function recoverySuite () {
'use strict';
jsunity.jsUnity.attachAssertions();
return {
setUp: function () {},
tearDown: function () {},
// //////////////////////////////////////////////////////////////////////////////
// / @brief test whether we can restore the trx data
// //////////////////////////////////////////////////////////////////////////////
testDieDuringCollector: function () {
var c = db._collection('UnitTestsRecovery');
assertEqual(0, c.count());
for (var i = 0; i < 1000; ++i) {
assertFalse(c.exists('test' + i));
}
}
};
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief executes the test suite
// //////////////////////////////////////////////////////////////////////////////
function main (argv) {
'use strict';
if (argv[1] === 'setup') {
runSetup();
return 0;
} else {
jsunity.run(recoverySuite);
return jsunity.done().status ? 0 : 1;
}
}

99
js/server/tests/recovery/insert-update-replace.js Normal file
View File

@ -0,0 +1,99 @@
/* jshint globalstrict:false, strict:false, unused : false */
/* global assertEqual, assertTrue, assertFalse */
// //////////////////////////////////////////////////////////////////////////////
// / @brief tests for dump/reload
// /
// / @file
// /
// / DISCLAIMER
// /
// / Copyright 2010-2012 triagens GmbH, Cologne, Germany
// /
// / Licensed under the Apache License, Version 2.0 (the "License")
// / you may not use this file except in compliance with the License.
// / You may obtain a copy of the License at
// /
// / http://www.apache.org/licenses/LICENSE-2.0
// /
// / Unless required by applicable law or agreed to in writing, software
// / distributed under the License is distributed on an "AS IS" BASIS,
// / WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// / See the License for the specific language governing permissions and
// / limitations under the License.
// /
// / Copyright holder is triAGENS GmbH, Cologne, Germany
// /
// / @author Jan Steemann
// / @author Copyright 2012, triAGENS GmbH, Cologne, Germany
// //////////////////////////////////////////////////////////////////////////////
var db = require('@arangodb').db;
var internal = require('internal');
var jsunity = require('jsunity');
function runSetup () {
'use strict';
internal.debugClearFailAt();
db._drop('UnitTestsRecovery');
var c = db._create('UnitTestsRecovery'), i;
for (i = 0; i < 1000; ++i) {
c.insert({ _key: 'foo' + i, value1: i, value2: "test" + i });
c.replace('foo' + i, { value23: i, value42: "test" + i });
}
c.save({ _key: 'crashme' }, true); // wait for sync
internal.debugSegfault('crashing server');
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief test suite
// //////////////////////////////////////////////////////////////////////////////
function recoverySuite () {
'use strict';
jsunity.jsUnity.attachAssertions();
return {
setUp: function () {},
tearDown: function () {},
// //////////////////////////////////////////////////////////////////////////////
// / @brief test whether we can restore the trx data
// //////////////////////////////////////////////////////////////////////////////
testInsertUpdateReplace: function () {
var i, c = db._collection('UnitTestsRecovery');
assertEqual(1001, c.count());
for (i = 0; i < 1000; ++i) {
var doc = c.document('foo' + i);
assertTrue(doc.hasOwnProperty('value23'));
assertEqual(i, doc.value23);
assertTrue(doc.hasOwnProperty('value42'));
assertEqual('test' + i, doc.value42);
assertFalse(doc.hasOwnProperty('value1'));
assertFalse(doc.hasOwnProperty('value2'));
}
}
};
}
// //////////////////////////////////////////////////////////////////////////////
// / @brief executes the test suite
// //////////////////////////////////////////////////////////////////////////////
function main (argv) {
'use strict';
if (argv[1] === 'setup') {
runSetup();
return 0;
} else {
jsunity.run(recoverySuite);
return jsunity.done().status ? 0 : 1;
}
}

3
lib/Basics/ArangoGlobalContext.cpp
View File

@ -28,6 +28,9 @@
#include "Basics/debugging.h"
#include "Basics/files.h"
#ifdef __arm__
#include "Basics/FileUtils.h"
#endif
#include "Logger/LogAppender.h"
#include "Logger/Logger.h"
#include "Rest/InitializeRest.h"

7
lib/Basics/tri-strings.cpp
View File

@ -932,11 +932,16 @@ char* TRI_EscapeControlsCString(char const* in, size_t inLength,
*qtr++ = '\\';
*qtr = 'r';
break;
case '\t':
*qtr++ = '\\';
*qtr = 't';
break;
default:
n = (uint8_t)(*ptr);
if (n < 32 || n > 127) {
if (n < 32) {
uint8_t n1 = n >> 4;
uint8_t n2 = n & 0x0F;