Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'devel' of github.com:triAGENS/ArangoDB into devel

This commit is contained in:
Thomas Schmidts 2014-06-24 14:28:50 +02:00
parent 0d969e6814 3cc95a23fd
commit 0bf957ddd7
48 changed files with 1626 additions and 524 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
CHANGELOG
View File

@ -1,6 +1,18 @@
v2.2.0 (XXXX-XX-XX)
-------------------
* honor startup option `--server.disable-statistics` when deciding whether or not
to start periodic statistics collection jobs
Previously, the statistics collection jobs were started even if the server was
started with the `--server.disable-statistics` flag being set to `true`
* removed startup option `--random.no-seed`
This option had no effect in previous versions of ArangoDB and was thus removed.
* removed startup option `--database.remove-on-drop`
* introduced write-ahead log
All write operations in an ArangoDB server instance are automatically logged

9
Documentation/Books/Users/Aql/GraphOperations.mdpp
View File

@ -10,7 +10,6 @@ A lot of the following functions accept a vertex (or edge) example as parameter.
* *idString* : Returns the vertex/edge with the id *idString*.
* {*key1* : *value1*, *key2* : *value2*} : Returns the vertices/edges that match this example, which means that both have *key1* and *key2* with the corresponding attributes.
* [{*key1* : *value1*}, {*key2* : *value2*}] : Returns the vertices/edges that match one of the examples, which means that either *key1* or *key2* are set with the corresponding value.
the examples.
!SUBSECTION Edges and Vertices related functions
@ -24,6 +23,10 @@ This section describes various AQL functions which can be used to receive inform
@startDocuBlock JSF_ahuacatl_general_graph_vertices
!SUBSUBSECTION GRAPH_NEIGHBORS
@startDocuBlock JSF_ahuacatl_general_graph_neighbors
!SUBSUBSECTION GRAPH_COMMON_NEIGHBORS
@startDocuBlock JSF_ahuacatl_general_graph_common_neighbors
@ -32,10 +35,6 @@ This section describes various AQL functions which can be used to receive inform
@startDocuBlock JSF_ahuacatl_general_graph_common_properties
!SUBSUBSECTION GRAPH_NEIGHBORS
@startDocuBlock JSF_ahuacatl_general_graph_neighbors
!SUBSECTION Shortest Paths, distances and traversals.
This section describes AQL functions, that calculate pathes from a subset of vertices in a graph to another subset of vertices.

7
Documentation/Books/Users/ConfigureArango/Arangod.mdpp
View File

@ -346,13 +346,6 @@ If turned off, syncing will still happen for collection that have a waitForSync
The default is true.
`--database.remove-on-drop flag`
If true and you drop a collection, then they directory and all associated datafiles will be removed from disk. If false, then they collection directory will be renamed to deleted-..., but remains on hard disk. To restore such a dropped collection, you can rename the directory back to collection-..., but you must also edit the file parameter.json inside the directory.
The default is true.
`--javascript.gc-frequency frequency`
Specifies the frequency (in seconds) for the automatic garbage collection of JavaScript objects. This setting is useful to have the garbage collection still work in periods with no or little numbers of requests.

6
Documentation/Books/Users/ConfigureArango/RandomNumbers.mdpp
View File

@ -6,11 +6,7 @@ The argument is an integer (1,2,3 or 4) which sets the manner in which random nu
Specifying an argument of 2, uses a blocking random (or pseudorandom) number generator. Specifying an argument 1 sets a pseudorandom number generator using an implication of the Mersenne Twister MT19937 algorithm. Algorithm 4 is a combination of the blocking random number generator and the Mersenne Twister.
`--random.no-seed`
By default, the random generator is seeded. Setting this option causes the random number generator not to be seeded. (Seeding the random number generator only occurs if the generator is set to Mersenne, refer to random.generator for details.)
<!--
@anchor CommandLineRandomGenerator
@copydetails triagens::rest::ApplicationServer::_randomGenerator
-->
-->

11
Documentation/Books/Users/General-Graphs/Functions.mdpp
View File

@ -37,6 +37,17 @@ A lot of these accept a vertex (or edge) example as parameter as defined in the
@startDocuBlock JSF_general_graph_count_common_properties
!SECTION _paths
@startDocuBlock JSF_general_graph_paths
!SECTION _shortestPath
@startDocuBlock JSF_general_graph_shortest_path
!SECTION _distanceTo
@startDocuBlock JSF_general_graph_distance_to
!SECTION _absoluteEccentricity

4
Documentation/Books/Users/General-Graphs/README.mdpp
View File

@ -11,3 +11,7 @@ A Graph consists of *vertices* and *edges*. Edges are stored as documents in *ed
collections*. In general a vertex is stored in a document collection. Which collections are used within
a graph is defined via *edge definitions*. A graph can have an arbitrary number of edge
definitions.
!SUBSUBSECTION Three Steps to create a graph
@startDocuBlock JSF_general_graph_how_to_create

134
Documentation/Books/Users/HttpGharial/Edges.mdpp
View File

@ -1,115 +1,151 @@
!CHAPTER Handling Edges
!SECTION Create an edge
`POST /system/gharial/graph-name/edge/collection-name`*(create an edge)*
!SUBSECTION URL parameters
`graph-name (string, required)`
The name of the graph.
`collection-name (string, required)`
The name of the edge collection the edge belongs to.
!SUBSECTION Description
@RESTHEADER{POST /system/gharial/graph-name/edge/collection-name, Create an edge}
@RESTDESCRIPTION
Creates a new edge in the collection.
Within the body the has to contain a *\_from* and *\_to* value referencing to valid vertices in the graph.
Furthermore the edge has to be valid in the definition of this edge collection.
@RESTURLPARAMS
@RESTPARAM{graph-name, string, required}
The name of the graph.
@RESTPARAM{collection-name, string, required}
The name of the edge collection the edge belongs to.
@RESTBODYPARAMS
@RESTPARAM{_from, string, required}
@RESTPARAM{_to, string, required}
The body has to be the JSON object to be stored.
@RESTRETURNCODES
@RESTRETURNCODE{201}
Returned if the edge could be created.
@RESTRETURNCODE{404}
Returned if no graph with this name, no edge collection or no edge with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_edge_create_http_examples
!SECTION Get an edge
@RESTHEADER{GET /system/gharial/graph-name/edge/collection-name/edge-key, Get an edge}
`GET /system/gharial/graph-name/edge/collection-name/edge-key`*(get an edge)*
@RESTDESCRIPTION
Gets an edge from the given collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the edge collection the edge belongs to.
`edge-key (string, required)`
The `_key` attribute of the edge.
@RESTPARAM{edge-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTRETURNCODES
@RESTRETURNCODE{200}
Returned if the edge could be found.
Gets an edge from the given collection.
@RESTRETURNCODE{404}
Returned if no graph with this name, no edge collection or no edge with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_edge_get_http_examples
!SECTION Modify an edge
@RESTHEADER{PATCH /system/gharial/graph-name/edge/collection-name/edge-key, Modify an edge}
`PATCH /system/gharial/graph-name/edge/collection-name/edge-key`*(modify an edge)*
@RESTDESCRIPTION
Updates the data of the specific edge in the collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the edge collection the edge belongs to.
`edge-key (string, required)`
The `_key` attribute of the edge.
@RESTPARAM{edge-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTBODYPARAMS
The body has to be a JSON object containing the attributes to be updated.
Updates the data of the specific edge in the collection.
@RESTRETURNCODES
@RESTRETURNCODE{200}
Returned if the edge could be updated.
@RESTRETURNCODE{404}
Returned if no graph with this name, no edge collection or no edge with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_edge_modify_http_examples
!SECTION Replace an edge
@RESTHEADER{PUT /system/gharial/graph-name/edge/collection-name/edge-key, Replace an edge}
`PATCH /system/gharial/graph-name/edge/collection-name/edge-key`*(replace an edge)*
@RESTDESCRIPTION
Replaces the data of an edge in the collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the edge collection the edge belongs to.
`edge-key (string, required)`
The `_key` attribute of the edge.
@RESTPARAM{edge-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTBODYPARAMS
The body has to be the JSON object to be stored.
Replaces the data of an edge in the collection.
@RESTRETURNCODES
@RESTRETURNCODE{200}
Returned if the edge could be replaced.
@RESTRETURNCODE{404}
Returned if no graph with this name, no edge collection or no edge with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_edge_replace_http_examples
!SECTION Remove an edge
@RESTHEADER{DELETE /system/gharial/graph-name/edge/collection-name/edge-key, Remove an edge}
`DELETE /system/gharial/graph-name/edge/collection-name/edge-key`*(remove an edge)*
@RESTDESCRIPTION
Removes an edge from the collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the edge collection the edge belongs to.
`edge-key (string, required)`
The `_key` attribute of the edge.
@RESTPARAM{edge-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTRETURNCODES
Removes an edge from the collection.
@RESTRETURNCODE{200}
Returned if the edge could be removed.
@RESTRETURNCODE{404}
Returned if no graph with this name, no edge collection or no edge with this id could be found.
@EXAMPLES

122
Documentation/Books/Users/HttpGharial/Vertices.mdpp
View File

@ -1,112 +1,146 @@
!CHAPTER Handling Vertices
!SECTION Create a vertex
@RESTHEADER{POST /system/gharial/graph-name/vertex/collection-name, Create a vertex}
`POST /system/gharial/graph-name/vertex/collection-name`*(create a vertex)*
@RESTDESCRIPTION
Adds a vertex to the given collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
The name of the vertex collection the edge belongs to.
@RESTPARAM{collection-name, string, required}
The name of the vertex collection the vertex belongs to.
!SUBSECTION Description
@RESTBODYPARAMS
The body has to be the JSON object to be stored.
Adds a vertex to the given collection.
@RESTRETURNCODES
@RESTRETURNCODE{201}
Returned if the vertex could be added.
@RESTRETURNCODE{404}
Returned if no graph or no vertex collection with this name could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_vertex_create_http_examples
!SECTION Get a vertex
@RESTHEADER{GET /system/gharial/graph-name/vertex/collection-name/vertex-key, Get a vertex}
`GET /system/gharial/graph-name/vertex/collection-name/vertex-key`*(get a vertex)*
@RESTDESCRIPTION
Gets a vertex from the given collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the vertex collection the vertex belongs to.
`vertex-key (string, required)`
The `_key` attribute of the vertex.
@RESTPARAM{vertex-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTRETURNCODES
Gets a vertex from the given collection.
@RESTRETURNCODE{200}
Returned if the vertex could be found.
@RESTRETURNCODE{404}
Returned if no graph with this name, no vertex collection or no vertex with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_vertex_get_http_examples
!SECTION Modify an vertex
@RESTHEADER{PATCH /system/gharial/graph-name/vertex/collection-name/vertex-key, Modify a vertex}
`PATCH /system/gharial/graph-name/vertex/collection-name/vertex-key`*(modify a vertex)*
@RESTDESCRIPTION
Updates the data of the specific vertex in the collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the vertex collection the vertex belongs to.
`vertex-key (string, required)`
The `_key` attribute of the vertex.
@RESTPARAM{vertex-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTBODYPARAMS
The body has to contain a JSON object containing exactly the attributes that should be replaced.
Updates the data of the specific vertex in the collection.
@RESTRETURNCODES
@RESTRETURNCODE{200}
Returned if the vertex could be updated.
@RESTRETURNCODE{404}
Returned if no graph with this name, no vertex collection or no vertex with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_vertex_modify_http_examples
!SECTION Replace an vertex
@RESTHEADER{PUT /system/gharial/graph-name/vertex/collection-name/vertex-key, Replace a vertex}
`PATCH /system/gharial/graph-name/vertex/collection-name/vertex-key`*(replace a vertex)*
@RESTDESCRIPTION
Replaces the data of a vertex in the collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the vertex collection the vertex belongs to.
`vertex-key (string, required)`
The `_key` attribute of the vertex.
@RESTPARAM{vertex-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTBODYPARAMS
The body has to be the JSON object to be stored.
Replaces the data of a vertex in the collection.
@RESTRETURNCODES
@RESTRETURNCODE{200}
Returned if the vertex could be replaced.
@RESTRETURNCODE{404}
Returned if no graph with this name, no vertex collection or no vertex with this id could be found.
@EXAMPLES
@startDocuBlock JSF_general_graph_vertex_replace_http_examples
!SECTION Remove a vertex
@RESTHEADER{DELETE /system/gharial/graph-name/vertex/collection-name/vertex-key, Remove a vertex}
`DELETE /system/gharial/graph-name/vertex/collection-name/vertex-key`*(remove a vertex)*
@RESTDESCRIPTION
Removes a vertex from the collection.
!SUBSECTION URL parameters
@RESTURLPARAMS
`graph-name (string, required)`
@RESTPARAM{graph-name, string, required}
The name of the graph.
`collection-name (string, required)`
@RESTPARAM{collection-name, string, required}
The name of the vertex collection the vertex belongs to.
`vertex-key (string, required)`
The `_key` attribute of the vertex.
@RESTPARAM{vertex-key, string, required}
The *_key* attribute of the vertex.
!SUBSECTION Description
@RESTRETURNCODES
Removes a vertex from the collection.
@RESTRETURNCODE{200}
Returned if the vertex could be removed.
@RESTRETURNCODE{404}
Returned if no graph with this name, no vertex collection or no vertex with this id could be found.
@EXAMPLES

1
Documentation/UserManual/CommandLineTOC.md
View File

@ -74,4 +74,3 @@ TOC {#CommandLineTOC}
- @ref CommandLineSchedulerShowIoBackends "show-io-backends"
- @ref CommandLineRandom
- @ref CommandLineRandomGenerator "random.generator"
- @ref CommandLineRandomGenerator "random.no-seed"

1
UnitTests/Makefile.unittests
View File

@ -210,6 +210,7 @@ unittests-recovery:
@echo "================================================================================"
@echo
$(MAKE) execute-recovery-test PID=$(PID) RECOVERY_SCRIPT="collector-oom"
$(MAKE) execute-recovery-test PID=$(PID) RECOVERY_SCRIPT="transaction-no-abort"
$(MAKE) execute-recovery-test PID=$(PID) RECOVERY_SCRIPT="transaction-no-commit"
$(MAKE) execute-recovery-test PID=$(PID) RECOVERY_SCRIPT="multi-database-durability"

8
arangod/Ahuacatl/ahuacatl-codegen.cpp
View File

@ -1125,10 +1125,8 @@ static void GeneratePrimaryAccess (TRI_aql_codegen_js_t* const generator,
const TRI_aql_collection_t* const collection,
const char* const collectionName) {
TRI_aql_field_access_t* fieldAccess;
size_t n;
n = idx->_fieldAccesses->_length;
TRI_ASSERT(n == 1);
TRI_ASSERT(idx->_fieldAccesses->_length == 1);
fieldAccess = (TRI_aql_field_access_t*) TRI_AtVectorPointer(idx->_fieldAccesses, 0);
@ -1269,10 +1267,8 @@ static void GenerateEdgeAccess (TRI_aql_codegen_js_t* const generator,
const TRI_aql_collection_t* const collection,
const char* const collectionName) {
TRI_aql_field_access_t* fieldAccess;
size_t n;
n = idx->_fieldAccesses->_length;
TRI_ASSERT(n > 0);
TRI_ASSERT(idx->_fieldAccesses->_length > 0);
fieldAccess = (TRI_aql_field_access_t*) TRI_AtVectorPointer(idx->_fieldAccesses, 0);

33
arangod/Ahuacatl/ahuacatl-functions.cpp
View File

@ -389,33 +389,24 @@ static bool CheckPathRestriction (TRI_aql_field_access_t* fieldAccess,
static void OptimisePaths (const TRI_aql_node_t* const fcallNode,
TRI_aql_context_t* const context,
TRI_aql_field_access_t* fieldAccess) {
TRI_aql_node_t* args;
TRI_aql_node_t* vertexCollection;
TRI_aql_node_t* edgeCollection;
TRI_aql_node_t* direction;
char* directionValue;
char* name;
size_t n;
TRI_aql_node_t* args = TRI_AQL_NODE_MEMBER(fcallNode, 0);
args = TRI_AQL_NODE_MEMBER(fcallNode, 0);
if (args == NULL) {
if (args == nullptr) {
return;
}
vertexCollection = TRI_AQL_NODE_MEMBER(args, 0);
edgeCollection = TRI_AQL_NODE_MEMBER(args, 1);
direction = TRI_AQL_NODE_MEMBER(args, 2);
TRI_aql_node_t* vertexCollection = TRI_AQL_NODE_MEMBER(args, 0);
TRI_aql_node_t* direction = TRI_AQL_NODE_MEMBER(args, 2);
TRI_ASSERT(vertexCollection);
TRI_ASSERT(edgeCollection);
TRI_ASSERT(direction);
TRI_ASSERT(fieldAccess);
TRI_ASSERT(vertexCollection != nullptr);
TRI_ASSERT(TRI_AQL_NODE_MEMBER(args, 1) != nullptr); // edgeCollection
TRI_ASSERT(direction != nullptr);
TRI_ASSERT(fieldAccess != nullptr);
n = strlen(fieldAccess->_fullName);
name = fieldAccess->_fullName + fieldAccess->_variableNameLength;
size_t n = strlen(fieldAccess->_fullName);
char* name = fieldAccess->_fullName + fieldAccess->_variableNameLength;
directionValue = TRI_AQL_NODE_STRING(direction);
char* directionValue = TRI_AQL_NODE_STRING(direction);
// try to optimise the vertex collection access
if (TRI_EqualString(directionValue, "outbound")) {
CheckPathRestriction(fieldAccess, context, vertexCollection, ".source.", name, n);
@ -668,7 +659,7 @@ TRI_associative_pointer_t* TRI_CreateFunctionsAql (void) {
// graph functions
REGISTER_FUNCTION("PATHS", "GRAPH_PATHS", false, false, "c,h|s,b", &OptimisePaths);
REGISTER_FUNCTION("GRAPH_PATHS", "GENERAL_GRAPH_PATHS", false, false, "s|s,b,n,n", &OptimisePaths);
REGISTER_FUNCTION("GRAPH_PATHS", "GENERAL_GRAPH_PATHS", false, false, "s|a", NULL);
REGISTER_FUNCTION("SHORTEST_PATH", "GRAPH_SHORTEST_PATH", false, false, "h,h,s,s,s|a", NULL);
REGISTER_FUNCTION("GRAPH_SHORTEST_PATH", "GENERAL_GRAPH_SHORTEST_PATH", false, false, "s,als,als|a", NULL);
REGISTER_FUNCTION("GRAPH_DISTANCE_TO", "GENERAL_GRAPH_DISTANCE_TO", false, false, "s,als,als|a", NULL);

17
arangod/RestHandler/RestImportHandler.cpp
View File

@ -326,9 +326,11 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// Importing documents with heterogenous attributes from a JSON list:
///
/// @EXAMPLE_ARANGOSH_RUN{RestImportJsonList}
/// db._flushCache();
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn)
/// db._create(cn);
/// db._flushCache();
///
/// var body = [
/// { _key: "abc", value1: 25, value2: "test", allowed: true },
@ -351,9 +353,11 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// Importing documents from individual JSON lines:
///
/// @EXAMPLE_ARANGOSH_RUN{RestImportJsonLines}
/// db._flushCache();
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn);
/// db._flushCache();
///
/// var body = '{ "_key": "abc", "value1": 25, "value2": "test", "allowed": true }\n{ "_key": "foo", "name": "baz" }\n\n{ "name": { "detailed": "detailed name", "short": "short name" } }\n';
/// var response = logCurlRequestRaw('POST', "/_api/import?collection=" + cn + "&type=documents", body);
@ -371,9 +375,11 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// Using the auto type detection:
///
/// @EXAMPLE_ARANGOSH_RUN{RestImportJsonType}
/// db._flushCache();
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn);
/// db._flushCache();
///
/// var body = [
/// { _key: "abc", value1: 25, value2: "test", allowed: true },
@ -396,9 +402,11 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// Importing documents into a new collection from a JSON list:
///
/// @EXAMPLE_ARANGOSH_RUN{RestImportJsonCreate}
/// db._flushCache();
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn);
/// db._flushCache();
///
/// var body = [
/// { id: "12553", active: true },
@ -421,11 +429,13 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// Importing into an edge collection, with attributes `_from`, `_to` and `name`:
///
/// @EXAMPLE_ARANGOSH_RUN{RestImportJsonEdge}
/// db._flushCache();
/// var cn = "links";
/// db._drop(cn);
/// db._createEdgeCollection(cn);
/// db._drop("products");
/// db._create("products");
/// db._flushCache();
///
/// var body = '{ "_from": "products/123", "_to": "products/234" }\n{ "_from": "products/332", "_to": "products/abc", "name": "other name" }';
///
@ -445,9 +455,11 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// Importing into an edge collection, omitting `_from` or `_to`:
///
/// @EXAMPLE_ARANGOSH_RUN{RestImportJsonEdgeInvalid}
/// db._flushCache();
/// var cn = "links";
/// db._drop(cn);
/// db._createEdgeCollection(cn);
/// db._flushCache();
///
/// var body = [ { name: "some name" } ];
///
@ -469,6 +481,7 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn);
/// db._flushCache();
///
/// var body = '{ "_key": "abc", "value1": 25, "value2": "test" }\n{ "_key": "abc", "value1": "bar", "value2": "baz" }';
///
@ -490,6 +503,7 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn);
/// db._flushCache();
///
/// var body = '{ "_key": "abc", "value1": 25, "value2": "test" }\n{ "_key": "abc", "value1": "bar", "value2": "baz" }';
///
@ -522,6 +536,7 @@ int RestImportHandler::handleSingleDocument (RestImportTransaction& trx,
/// var cn = "products";
/// db._drop(cn);
/// db._create(cn);
/// db._flushCache();
///
/// var body = '{ }';
///

8
arangod/RestHandler/RestReplicationHandler.cpp
View File

@ -1247,13 +1247,11 @@ void RestReplicationHandler::handleTrampolineCoordinator () {
///
/// @EXAMPLE_ARANGOSH_RUN{RestReplicationLoggerFollowEmpty}
/// var re = require("org/arangodb/replication");
/// re.logger.start();
/// var lastTick = re.logger.state().state.lastLogTick;
///
/// var url = "/_api/replication/logger-follow?from=" + lastTick;
/// var response = logCurlRequest('GET', url);
///
/// re.logger.stop();
/// assert(response.code === 204);
///
/// logRawResponse(response);
@ -1265,7 +1263,6 @@ void RestReplicationHandler::handleTrampolineCoordinator () {
/// var re = require("org/arangodb/replication");
/// db._drop("products");
///
/// re.logger.start();
/// var lastTick = re.logger.state().state.lastLogTick;
///
/// db._create("products");
@ -1275,10 +1272,10 @@ void RestReplicationHandler::handleTrampolineCoordinator () {
/// db.products.update("p2", { "name" : "broken hovercraft" });
/// db.products.drop();
///
/// require("internal").wait(1);
/// var url = "/_api/replication/logger-follow?from=" + lastTick;
/// var response = logCurlRequest('GET', url);
///
/// re.logger.stop();
/// assert(response.code === 200);
///
/// logRawResponse(response);
@ -1290,7 +1287,6 @@ void RestReplicationHandler::handleTrampolineCoordinator () {
/// var re = require("org/arangodb/replication");
/// db._drop("products");
///
/// re.logger.start();
/// var lastTick = re.logger.state().state.lastLogTick;
///
/// db._create("products");
@ -1300,10 +1296,10 @@ void RestReplicationHandler::handleTrampolineCoordinator () {
/// db.products.update("p2", { "name" : "broken hovercraft" });
/// db.products.drop();
///
/// require("internal").wait(1);
/// var url = "/_api/replication/logger-follow?from=" + lastTick + "&chunkSize=400";
/// var response = logCurlRequest('GET', url);
///
/// re.logger.stop();
/// assert(response.code === 200);
///
/// logRawResponse(response);

12
arangod/RestServer/ArangoServer.cpp
View File

@ -284,9 +284,7 @@ ArangoServer::ArangoServer (int argc, char** argv)
_defaultMaximalSize(TRI_JOURNAL_DEFAULT_MAXIMAL_SIZE),
_defaultWaitForSync(false),
_forceSyncProperties(true),
_unusedForceSyncShapes(false),
_disableReplicationApplier(false),
_removeOnDrop(true),
_server(nullptr) {
char* p = TRI_GetTempPath();
@ -417,6 +415,8 @@ void ArangoServer::buildApplicationServer () {
("ruby.modules-path", &ignoreOpt, "one or more directories separated by (semi-) colons")
("ruby.startup-directory", &ignoreOpt, "path to the directory containing alternate Ruby startup scripts")
("server.disable-replication-logger", &ignoreOpt, "start with replication logger turned off")
("database.force-sync-shapes", &ignoreOpt, "force syncing of shape data to disk, will use waitForSync value of collection when turned off (deprecated)")
("database.remove-on-drop", &ignoreOpt, "wipe a collection from disk after dropping")
;
// .............................................................................
@ -501,18 +501,11 @@ void ArangoServer::buildApplicationServer () {
;
additional["DATABASE Options:help-admin"]
("database.remove-on-drop", &_removeOnDrop, "wipe a collection from disk after dropping")
("database.maximal-journal-size", &_defaultMaximalSize, "default maximal journal size, can be overwritten when creating a collection")
("database.wait-for-sync", &_defaultWaitForSync, "default wait-for-sync behavior, can be overwritten when creating a collection")
("database.force-sync-properties", &_forceSyncProperties, "force syncing of collection properties to disk, will use waitForSync value of collection when turned off")
;
// deprecated options
additional[ApplicationServer::OPTIONS_HIDDEN]
("database.force-sync-shapes", &_unusedForceSyncShapes, "force syncing of shape data to disk, will use waitForSync value of collection when turned off (deprecated)")
;
// .............................................................................
// cluster options
// .............................................................................
@ -1098,7 +1091,6 @@ void ArangoServer::openDatabases (bool checkVersion,
// override with command-line options
defaults.defaultMaximalSize = _defaultMaximalSize;
defaults.removeOnDrop = _removeOnDrop;
defaults.defaultWaitForSync = _defaultWaitForSync;
defaults.forceSyncProperties = _forceSyncProperties;
defaults.requireAuthentication = ! _disableAuthentication;

28
arangod/RestServer/ArangoServer.h
View File

@ -393,17 +393,6 @@ namespace triagens {
bool _forceSyncProperties;
////////////////////////////////////////////////////////////////////////////////
/// @brief force syncing of shapes
///
/// @CMDOPT{\--database.force-sync-shapes @CA{boolean}}
///
/// This option is not used in this version of ArangoDB. It only remains here
/// for compatibility reasons.
////////////////////////////////////////////////////////////////////////////////
bool _unusedForceSyncShapes;
////////////////////////////////////////////////////////////////////////////////
/// @brief disable the replication applier on server startup
///
@ -424,23 +413,6 @@ namespace triagens {
bool _disableReplicationApplier;
////////////////////////////////////////////////////////////////////////////////
/// @brief remove on drop
///
/// @CMDOPT{\--database.remove-on-drop @CA{flag}}
///
/// If @LIT{true} and you drop a collection, then they directory and all
/// associated datafiles will be removed from disk. If @LIT{false}, then they
/// collection directory will be renamed to @LIT{deleted-...}, but remains on
/// hard disk. To restore such a dropped collection, you can rename the
/// directory back to @LIT{collection-...}, but you must also edit the file
/// @LIT{parameter.json} inside the directory.
///
/// The default is @LIT{true}.
////////////////////////////////////////////////////////////////////////////////
bool _removeOnDrop;
////////////////////////////////////////////////////////////////////////////////
/// @brief unit tests
///

12
arangod/V8Server/v8-vocbase.cpp
View File

@ -95,6 +95,8 @@ using namespace triagens::rest;
static v8::Handle<v8::Value> WrapGeneralCursor (void* cursor);
extern bool TRI_ENABLE_STATISTICS;
// -----------------------------------------------------------------------------
// --SECTION-- private defines
// -----------------------------------------------------------------------------
@ -7769,7 +7771,7 @@ static v8::Handle<v8::Value> MapGetVocBase (v8::Local<v8::String> const name,
/// @startDocuBlock TODO
/// `db._changeMode(<mode>)`
///
/// Sets the sever to the given mode.
/// Sets the server to the given mode.
/// Possible parameters for mode are:
/// - Normal
/// - ReadOnly
@ -8989,7 +8991,6 @@ static v8::Handle<v8::Value> JS_CreateDatabase (v8::Arguments const& argv) {
TRI_vocbase_defaults_t defaults;
TRI_GetDatabaseDefaultsServer((TRI_server_t*) v8g->_server, &defaults);
v8::Local<v8::String> keyRemoveOnDrop = v8::String::New("removeOnDrop");
v8::Local<v8::String> keyDefaultMaximalSize = v8::String::New("defaultMaximalSize");
v8::Local<v8::String> keyDefaultWaitForSync = v8::String::New("defaultWaitForSync");
v8::Local<v8::String> keyForceSyncProperties = v8::String::New("forceSyncProperties");
@ -9001,10 +9002,6 @@ static v8::Handle<v8::Value> JS_CreateDatabase (v8::Arguments const& argv) {
if (argv.Length() > 1 && argv[1]->IsObject()) {
v8::Handle<v8::Object> options = argv[1]->ToObject();
if (options->Has(keyRemoveOnDrop)) {
defaults.removeOnDrop = options->Get(keyRemoveOnDrop)->BooleanValue();
}
if (options->Has(keyDefaultMaximalSize)) {
defaults.defaultMaximalSize = (TRI_voc_size_t) options->Get(keyDefaultMaximalSize)->IntegerValue();
}
@ -10178,6 +10175,9 @@ void TRI_InitV8VocBridge (v8::Handle<v8::Context> context,
// current thread number
context->Global()->Set(TRI_V8_SYMBOL("THREAD_NUMBER"), v8::Number::New((double) threadNumber), v8::ReadOnly);
// whether or not statistics are enabled
context->Global()->Set(TRI_V8_SYMBOL("ENABLE_STATISTICS"), v8::Boolean::New(TRI_ENABLE_STATISTICS), v8::ReadOnly);
}
// -----------------------------------------------------------------------------

5
arangod/VocBase/index.cpp
View File

@ -2476,10 +2476,9 @@ TRI_index_t* TRI_CreateBitarrayIndex (TRI_document_collection_t* document,
for (i = j + 1; i < paths->_length; ++i) {
TRI_shape_pid_t* rightShape = (TRI_shape_pid_t*)(TRI_AtVector(paths,i));
if (*leftShape == *rightShape) {
LOG_WARNING("bitarray index creation failed -- duplicate keys in index");
*errorNum = TRI_ERROR_ARANGO_INDEX_BITARRAY_CREATION_FAILURE_DUPLICATE_ATTRIBUTES;
*errorStr = TRI_DuplicateString("bitarray index creation failed -- duplicate keys in index");
return NULL;
*errorStr = TRI_DuplicateString("bitarray index creation failed - duplicate keys in index");
return nullptr;
}
}
}

20
arangod/VocBase/transaction.cpp
View File

@ -653,9 +653,7 @@ template<typename T> static T* CreateMarker () {
TRI_transaction_t* TRI_CreateTransaction (TRI_vocbase_t* vocbase,
double timeout,
bool waitForSync) {
TRI_transaction_t* trx;
trx = (TRI_transaction_t*) TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, sizeof(TRI_transaction_t), false);
TRI_transaction_t* trx = static_cast<TRI_transaction_t*>(TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, sizeof(TRI_transaction_t), false));
if (trx == nullptr) {
// out of memory
@ -1017,6 +1015,22 @@ int TRI_BeginTransaction (TRI_transaction_t* trx,
if (nestingLevel == 0) {
TRI_ASSERT(trx->_status == TRI_TRANSACTION_CREATED);
if (trx->_type == TRI_TRANSACTION_WRITE &&
triagens::wal::LogfileManager::instance()->canBeThrottled()) {
// write-throttling?
static uint64_t const WaitTime = 50000;
uint64_t const maxIterations = triagens::wal::LogfileManager::instance()->maxThrottleWait() / (WaitTime / 1000);
uint64_t iterations = 0;
while (triagens::wal::LogfileManager::instance()->isThrottled()) {
if (++iterations == maxIterations) {
return TRI_ERROR_LOCK_TIMEOUT;
}
usleep(WaitTime);
}
}
// get a new id
trx->_id = TRI_NewTickServer();

2
arangod/VocBase/voc-shaper.cpp
View File

@ -209,7 +209,7 @@ static TRI_shape_aid_t FindOrCreateAttributeByName (TRI_shaper_t* shaper,
THROW_ARANGO_EXCEPTION(slotInfo.errorCode);
}
void* f = TRI_InsertKeyAssociativeSynced(&s->_attributeIds, &aid, const_cast<void*>(slotInfo.mem), false);
void* f TRI_UNUSED = TRI_InsertKeyAssociativeSynced(&s->_attributeIds, &aid, const_cast<void*>(slotInfo.mem), false);
TRI_ASSERT(f == nullptr);
// enter into the dictionaries

11
arangod/VocBase/vocbase-defaults.cpp
View File

@ -42,7 +42,6 @@
void TRI_ApplyVocBaseDefaults (TRI_vocbase_t* vocbase,
TRI_vocbase_defaults_t const* defaults) {
vocbase->_settings.defaultMaximalSize = defaults->defaultMaximalSize;
vocbase->_settings.removeOnDrop = defaults->removeOnDrop;
vocbase->_settings.defaultWaitForSync = defaults->defaultWaitForSync;
vocbase->_settings.forceSyncProperties = defaults->forceSyncProperties;
vocbase->_settings.requireAuthentication = defaults->requireAuthentication;
@ -64,7 +63,6 @@ TRI_json_t* TRI_JsonVocBaseDefaults (TRI_memory_zone_t* zone,
return NULL;
}
TRI_Insert3ArrayJson(zone, json, "removeOnDrop", TRI_CreateBooleanJson(zone, defaults->removeOnDrop));
TRI_Insert3ArrayJson(zone, json, "waitForSync", TRI_CreateBooleanJson(zone, defaults->defaultWaitForSync));
TRI_Insert3ArrayJson(zone, json, "forceSyncProperties", TRI_CreateBooleanJson(zone, defaults->forceSyncProperties));
TRI_Insert3ArrayJson(zone, json, "requireAuthentication", TRI_CreateBooleanJson(zone, defaults->requireAuthentication));
@ -81,18 +79,11 @@ TRI_json_t* TRI_JsonVocBaseDefaults (TRI_memory_zone_t* zone,
void TRI_FromJsonVocBaseDefaults (TRI_vocbase_defaults_t* defaults,
TRI_json_t const* json) {
TRI_json_t* optionJson;
if (! TRI_IsArrayJson(json)) {
return;
}
optionJson = TRI_LookupArrayJson(json, "removeOnDrop");
if (TRI_IsBooleanJson(optionJson)) {
defaults->removeOnDrop = optionJson->_value._boolean;
}
TRI_json_t* optionJson;
optionJson = TRI_LookupArrayJson(json, "waitForSync");
if (TRI_IsBooleanJson(optionJson)) {

1
arangod/VocBase/vocbase-defaults.h
View File

@ -46,7 +46,6 @@ struct TRI_vocbase_s;
typedef struct TRI_vocbase_defaults_s {
TRI_voc_size_t defaultMaximalSize;
bool removeOnDrop;
bool defaultWaitForSync;
bool forceSyncProperties;
bool requireAuthentication;

62
arangod/VocBase/vocbase.cpp
View File

@ -409,23 +409,15 @@ static bool DropCollectionCallback (TRI_collection_t* col,
newFilename);
}
else {
if (collection->_vocbase->_settings.removeOnDrop) {
LOG_DEBUG("wiping dropped collection '%s' from disk",
collection->_name);
LOG_DEBUG("wiping dropped collection '%s' from disk",
collection->_name);
res = TRI_RemoveDirectory(newFilename);
res = TRI_RemoveDirectory(newFilename);
if (res != TRI_ERROR_NO_ERROR) {
LOG_ERROR("cannot wipe dropped collecton '%s' from disk: %s",
collection->_name,
TRI_last_error());
}
}
else {
LOG_DEBUG("renamed dropped collection '%s' from '%s' to '%s'",
if (res != TRI_ERROR_NO_ERROR) {
LOG_ERROR("cannot wipe dropped collection '%s' from disk: %s",
collection->_name,
collection->_path,
newFilename);
TRI_last_error());
}
}
@ -906,45 +898,13 @@ static int ScanPath (TRI_vocbase_t* vocbase,
}
else if (info._deleted) {
// we found a collection that is marked as deleted.
// it depends on the configuration what will happen with these collections
// deleted collections should be removed on startup. this is the default
LOG_DEBUG("collection '%s' was deleted, wiping it", name);
if (vocbase->_settings.removeOnDrop) {
// deleted collections should be removed on startup. this is the default
LOG_DEBUG("collection '%s' was deleted, wiping it", name);
res = TRI_RemoveDirectory(file);
res = TRI_RemoveDirectory(file);
if (res != TRI_ERROR_NO_ERROR) {
LOG_WARNING("cannot wipe deleted collection: %s", TRI_last_error());
}
}
else {
// deleted collections shout not be removed on startup
char* newFile;
char* tmp1;
char* tmp2;
char const* first = name + matches[1].rm_so;
size_t firstLen = matches[1].rm_eo - matches[1].rm_so;
tmp1 = TRI_DuplicateString2(first, firstLen);
tmp2 = TRI_Concatenate2String("deleted-", tmp1);
TRI_FreeString(TRI_CORE_MEM_ZONE, tmp1);
newFile = TRI_Concatenate2File(path, tmp2);
TRI_FreeString(TRI_CORE_MEM_ZONE, tmp2);
LOG_WARNING("collection '%s' was deleted, renaming it to '%s'", name, newFile);
res = TRI_RenameFile(file, newFile);
if (res != TRI_ERROR_NO_ERROR) {
LOG_WARNING("cannot rename deleted collection: %s", TRI_last_error());
}
TRI_FreeString(TRI_CORE_MEM_ZONE, newFile);
if (res != TRI_ERROR_NO_ERROR) {
LOG_WARNING("cannot wipe deleted collection: %s", TRI_last_error());
}
}
else {

58
arangod/Wal/CollectorThread.cpp
View File

@ -274,6 +274,9 @@ CollectorThread::CollectorThread (LogfileManager* logfileManager,
_logfileManager(logfileManager),
_server(server),
_condition(),
_operationsQueueLock(),
_operationsQueue(),
_numPendingOperations(0),
_stop(0),
_inRecovery(true) {
@ -458,6 +461,20 @@ bool CollectorThread::processQueuedOperations () {
}
if (res == TRI_ERROR_NO_ERROR) {
uint64_t numOperations = (*it2)->operations->size();
uint64_t maxNumPendingOperations = _logfileManager->throttleWhenPending();
if (maxNumPendingOperations > 0 &&
_numPendingOperations >= maxNumPendingOperations &&
(_numPendingOperations - numOperations) < maxNumPendingOperations) {
// write-throttling was active, but can be turned off now
_logfileManager->deactivateWriteThrottling();
LOG_INFO("deactivating write-throttling");
}
_numPendingOperations -= numOperations;
// delete the object
delete (*it2);
@ -467,6 +484,7 @@ bool CollectorThread::processQueuedOperations () {
_logfileManager->decreaseCollectQueueSize(logfile);
}
else {
// do not delete the object but advance in the operations vector
++it2;
}
}
@ -1064,20 +1082,40 @@ int CollectorThread::executeTransferMarkers (TRI_document_collection_t* document
int CollectorThread::queueOperations (triagens::wal::Logfile* logfile,
CollectorCache*& cache) {
TRI_voc_cid_t cid = cache->collectionId;
uint64_t maxNumPendingOperations = _logfileManager->throttleWhenPending();
MUTEX_LOCKER(_operationsQueueLock);
TRI_IF_FAILURE("CollectorThreadQueueOperations") {
throw std::bad_alloc();
}
auto it = _operationsQueue.find(cid);
if (it == _operationsQueue.end()) {
std::vector<CollectorCache*> ops;
ops.push_back(cache);
_operationsQueue.insert(it, std::make_pair(cid, ops));
_logfileManager->increaseCollectQueueSize(logfile);
{
MUTEX_LOCKER(_operationsQueueLock);
auto it = _operationsQueue.find(cid);
if (it == _operationsQueue.end()) {
std::vector<CollectorCache*> ops;
ops.push_back(cache);
_operationsQueue.insert(it, std::make_pair(cid, ops));
_logfileManager->increaseCollectQueueSize(logfile);
}
else {
(*it).second.push_back(cache);
_logfileManager->increaseCollectQueueSize(logfile);
}
}
else {
(*it).second.push_back(cache);
_logfileManager->increaseCollectQueueSize(logfile);
uint64_t numOperations = cache->operations->size();
if (maxNumPendingOperations > 0 &&
_numPendingOperations < maxNumPendingOperations &&
(_numPendingOperations + numOperations) >= maxNumPendingOperations) {
// activate write-throttling!
_logfileManager->activateWriteThrottling();
LOG_WARNING("queued more than %llu pending WAL collector operations. now activating write-throttling",
(unsigned long long) maxNumPendingOperations);
}
_numPendingOperations += numOperations;
// we have put the object into the queue successfully
// now set the original pointer to null so it isn't double-freed

6
arangod/Wal/CollectorThread.h
View File

@ -415,6 +415,12 @@ namespace triagens {
std::unordered_map<TRI_voc_cid_t, std::vector<CollectorCache*>> _operationsQueue;
////////////////////////////////////////////////////////////////////////////////
/// @brief number of pending operations in collector queue
////////////////////////////////////////////////////////////////////////////////
uint64_t _numPendingOperations;
////////////////////////////////////////////////////////////////////////////////
/// @brief stop flag
////////////////////////////////////////////////////////////////////////////////

238
arangod/Wal/LogfileManager.cpp
View File

@ -38,6 +38,7 @@
#include "Basics/ReadLocker.h"
#include "Basics/StringUtils.h"
#include "Basics/WriteLocker.h"
#include "Utils/Exception.h"
#include "VocBase/server.h"
#include "Wal/AllocatorThread.h"
#include "Wal/CollectorThread.h"
@ -56,6 +57,22 @@ static LogfileManager* Instance = nullptr;
// --SECTION-- helper functions
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
/// @brief minimum value for --wal.throttle-when-pending
////////////////////////////////////////////////////////////////////////////////
static inline uint64_t MinThrottleWhenPending () {
return 1024 * 1024;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief minimum value for --wal.sync-interval
////////////////////////////////////////////////////////////////////////////////
static inline uint64_t MinSyncInterval () {
return 5;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief minimum value for --wal.open-logfiles
////////////////////////////////////////////////////////////////////////////////
@ -96,22 +113,6 @@ static inline uint32_t MaxSlots () {
return 1024 * 1024 * 16;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief callback to handle one marker during logfile inspection
////////////////////////////////////////////////////////////////////////////////
static bool ScanMarkerTick (TRI_df_marker_t const* marker,
void* data,
TRI_datafile_t* datafile) {
RecoverState* state = reinterpret_cast<RecoverState*>(data);
TRI_ASSERT(marker != nullptr);
TRI_ASSERT(marker->_tick >= state->lastTick);
state->lastTick = marker->_tick;
return true;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief callback to handle one marker during recovery
////////////////////////////////////////////////////////////////////////////////
@ -170,7 +171,7 @@ static bool ScanMarker (TRI_df_marker_t const* marker,
// insert this transaction into the list of failed transactions
// we do this because if we don't find a commit marker for this transaction,
// we'll have it in the failed list at the end of the scan and can ignore it
state->failedTransactions.insert(m->_transactionId);
state->failedTransactions.insert(std::make_pair(m->_transactionId, std::make_pair(m->_databaseId, false)));
break;
}
case TRI_WAL_MARKER_COMMIT_TRANSACTION: {
@ -183,7 +184,15 @@ static bool ScanMarker (TRI_df_marker_t const* marker,
case TRI_WAL_MARKER_ABORT_TRANSACTION: {
// insert this transaction into the list of failed transactions
transaction_abort_marker_t const* m = reinterpret_cast<transaction_abort_marker_t const*>(marker);
state->failedTransactions.insert(m->_transactionId);
auto it = state->failedTransactions.find(m->_transactionId);
if (it != state->failedTransactions.end()) {
// delete previous element if present
state->failedTransactions.erase(m->_transactionId);
}
// and (re-)insert
state->failedTransactions.insert(std::make_pair(m->_transactionId, std::make_pair(m->_databaseId, true)));
break;
}
@ -221,12 +230,15 @@ LogfileManager::LogfileManager (TRI_server_t* server,
_server(server),
_databasePath(databasePath),
_directory(),
_recoverState(new RecoverState()),
_filesize(32 * 1024 * 1024),
_reserveLogfiles(4),
_historicLogfiles(10),
_maxOpenLogfiles(10),
_numberOfSlots(1048576),
_syncInterval(100),
_maxThrottleWait(15000),
_throttleWhenPending(0),
_allowOversizeEntries(true),
_ignoreLogfileErrors(false),
_allowWrites(false), // start in read-only mode
@ -245,10 +257,12 @@ LogfileManager::LogfileManager (TRI_server_t* server,
_failedTransactions(),
_droppedCollections(),
_droppedDatabases(),
_writeThrottled(0),
_filenameRegex(),
_shutdown(0) {
LOG_TRACE("creating WAL logfile manager");
TRI_ASSERT(! _allowWrites);
int res = regcomp(&_filenameRegex, "^logfile-([0-9][0-9]*)\\.db$", REG_EXTENDED);
@ -268,6 +282,11 @@ LogfileManager::~LogfileManager () {
regfree(&_filenameRegex);
if (_recoverState != nullptr) {
delete _recoverState;
_recoverState = nullptr;
}
if (_slots != nullptr) {
delete _slots;
_slots = nullptr;
@ -317,6 +336,8 @@ void LogfileManager::setupOptions (std::map<std::string, triagens::basics::Progr
("wal.reserve-logfiles", &_reserveLogfiles, "maximum number of reserve logfiles to maintain")
("wal.slots", &_numberOfSlots, "number of logfile slots to use")
("wal.sync-interval", &_syncInterval, "interval for automatic, non-requested disk syncs (in milliseconds)")
("wal.throttle-when-pending", &_throttleWhenPending, "throttle writes when at least this many operations are waiting for collection (set to 0 to deactivate write-throttling)")
("wal.throttle-wait", &_maxThrottleWait, "maximum wait time per operation when write-throttled (in milliseconds)")
;
}
@ -373,9 +394,13 @@ bool LogfileManager::prepare () {
if (_maxOpenLogfiles < MinOpenLogfiles()) {
LOG_FATAL_AND_EXIT("invalid value for --wal.open-logfiles. Please use a value of at least %lu", (unsigned long) MinOpenLogfiles());
}
if (_throttleWhenPending > 0 && _throttleWhenPending < MinThrottleWhenPending()) {
LOG_FATAL_AND_EXIT("invalid value for --wal.throttle-when-pending. Please use a value of at least %llu", (unsigned long long) MinThrottleWhenPending());
}
if (_syncInterval < 5) {
LOG_FATAL_AND_EXIT("invalid sync interval.");
if (_syncInterval < MinSyncInterval()) {
LOG_FATAL_AND_EXIT("invalid value for --wal.sync-interval. Please use a value of at least %llu", (unsigned long long) MinSyncInterval());
}
// sync interval is specified in milliseconds by the user, but internally
@ -399,6 +424,8 @@ bool LogfileManager::start () {
return true;
}
TRI_ASSERT(! _allowWrites);
int res = inventory();
if (res != TRI_ERROR_NO_ERROR) {
@ -410,6 +437,8 @@ bool LogfileManager::start () {
bool const shutdownFileExists = basics::FileUtils::exists(shutdownFile);
if (shutdownFileExists) {
LOG_TRACE("shutdown file found");
res = readShutdownInfo();
if (res != TRI_ERROR_NO_ERROR) {
@ -419,6 +448,9 @@ bool LogfileManager::start () {
return false;
}
}
else {
LOG_TRACE("no shutdown file found");
}
res = openLogfiles();
@ -470,9 +502,11 @@ bool LogfileManager::open () {
// we were already started
return true;
}
TRI_ASSERT(! _allowWrites);
opened = true;
return runRecovery();
}
@ -535,11 +569,11 @@ void LogfileManager::stop () {
/// @brief registers a transaction
////////////////////////////////////////////////////////////////////////////////
bool LogfileManager::registerTransaction (TRI_voc_tid_t id) {
bool LogfileManager::registerTransaction (TRI_voc_tid_t transactionId) {
{
WRITE_LOCKER(_logfilesLock);
_transactions.insert(make_pair(id, make_pair(_lastCollectedId, _lastSealedId)));
_transactions.insert(make_pair(transactionId, make_pair(_lastCollectedId, _lastSealedId)));
TRI_ASSERT(_lastCollectedId <= _lastSealedId);
}
@ -550,14 +584,14 @@ bool LogfileManager::registerTransaction (TRI_voc_tid_t id) {
/// @brief unregisters a transaction
////////////////////////////////////////////////////////////////////////////////
void LogfileManager::unregisterTransaction (TRI_voc_tid_t id,
void LogfileManager::unregisterTransaction (TRI_voc_tid_t transactionId,
bool markAsFailed) {
{
WRITE_LOCKER(_logfilesLock);
_transactions.erase(id);
_transactions.erase(transactionId);
if (markAsFailed) {
_failedTransactions.insert(id);
_failedTransactions.insert(transactionId);
}
}
}
@ -1346,30 +1380,50 @@ void LogfileManager::waitForCollector (Logfile::IdType logfileId) {
}
}
////////////////////////////////////////////////////////////////////////////////
/// @brief scan a single logfile for the max tick
////////////////////////////////////////////////////////////////////////////////
bool LogfileManager::scanLogfileTick (Logfile const* logfile,
RecoverState& state) {
TRI_ASSERT(logfile != nullptr);
LOG_TRACE("scanning logfile %llu (%s)", (unsigned long long) logfile->id(), logfile->statusText().c_str());
return TRI_IterateDatafile(logfile->df(), &ScanMarkerTick, static_cast<void*>(&state));
}
////////////////////////////////////////////////////////////////////////////////
/// @brief scan a single logfile
////////////////////////////////////////////////////////////////////////////////
bool LogfileManager::scanLogfile (Logfile const* logfile,
RecoverState& state) {
bool LogfileManager::scanLogfile (Logfile const* logfile) {
TRI_ASSERT(logfile != nullptr);
LOG_TRACE("scanning logfile %llu (%s)", (unsigned long long) logfile->id(), logfile->statusText().c_str());
return TRI_IterateDatafile(logfile->df(), &ScanMarker, static_cast<void*>(&state));
return TRI_IterateDatafile(logfile->df(), &ScanMarker, static_cast<void*>(_recoverState));
}
////////////////////////////////////////////////////////////////////////////////
/// @brief write abort markers for all open transactions
////////////////////////////////////////////////////////////////////////////////
void LogfileManager::closeOpenTransactions () {
TRI_ASSERT(_recoverState != nullptr);
if (! _recoverState->failedTransactions.empty()) {
LOG_TRACE("writing abort markers for previously failed transactions");
// write an abort marker for all transactions
for (auto it = _recoverState->failedTransactions.begin(); it != _recoverState->failedTransactions.end(); ++it) {
TRI_voc_tid_t transactionId = (*it).first;
if ((*it).second.second) {
// already handled
continue;
}
TRI_voc_tick_t databaseId = (*it).second.first;
// only write abort markers for databases that haven't been deleted yet
if (_recoverState->droppedDatabases.find(databaseId) == _recoverState->droppedDatabases.end()) {
AbortTransactionMarker marker(databaseId, transactionId);
SlotInfoCopy slotInfo = allocateAndWrite(marker.mem(), marker.size(), false);
if (slotInfo.errorCode != TRI_ERROR_NO_ERROR) {
THROW_ARANGO_EXCEPTION(slotInfo.errorCode);
}
}
}
}
}
////////////////////////////////////////////////////////////////////////////////
@ -1379,51 +1433,9 @@ bool LogfileManager::scanLogfile (Logfile const* logfile,
bool LogfileManager::runRecovery () {
LOG_TRACE("running WAL recovery");
RecoverState state;
state.lastTick = 0;
int logfilesToCollect = 0;
// we're the only ones that access the logfiles at this point
// nevertheless, get the read-lock
{
READ_LOCKER(_logfilesLock);
// first iterate the logfiles in order and track which collections are in use
// this also populates a list of failed transactions
for (auto it = _logfiles.begin(); it != _logfiles.end(); ++it) {
Logfile* logfile = (*it).second;
if (logfile != nullptr) {
if (logfile->status() == Logfile::StatusType::OPEN ||
logfile->status() == Logfile::StatusType::SEALED) {
++logfilesToCollect;
}
if (! scanLogfile(logfile, state)) {
LOG_TRACE("WAL recovery failed when scanning logfile '%s'", logfile->filename().c_str());
return false;
}
}
}
}
// update the tick with the max tick we found in the WAL
TRI_UpdateTickServer(state.lastTick);
// note all failed transactions that we found plus the list
// of collections and databases that we can ignore
{
WRITE_LOCKER(_logfilesLock);
_failedTransactions = state.failedTransactions;
_droppedDatabases = state.droppedDatabases;
_droppedCollections = state.droppedCollections;
}
// "seal" any open logfiles so the collector can copy over everything
this->setAllSealed();
int res = startCollectorThread();
if (res != TRI_ERROR_NO_ERROR) {
@ -1436,6 +1448,24 @@ bool LogfileManager::runRecovery () {
LOG_TRACE("waiting for collector to catch up");
waitForCollector(_lastOpenedId);
// from now on, we allow writes to the logfile
allowWrites(true);
try {
closeOpenTransactions();
}
catch (triagens::arango::Exception const& ex) {
LOG_ERROR("could not abort previously open transactions: %s", TRI_errno_string(ex.code()));
return false;
}
catch (...) {
LOG_ERROR("could not abort previously open transactions");
return false;
}
// write the current state into the shutdown file
writeShutdownInfo(true);
{
// reset the list of failed transactions
WRITE_LOCKER(_logfilesLock);
@ -1444,10 +1474,9 @@ bool LogfileManager::runRecovery () {
_droppedCollections.clear();
}
// finished recovery
_inRecovery = false;
// from now on, we allow writes to the logfile
allowWrites(true);
// tell the collector that the recovery is over now
_collectorThread->recoveryDone();
@ -1462,10 +1491,14 @@ bool LogfileManager::runRecovery () {
return false;
}
if (logfilesToCollect > 0) {
if (_recoverState->logfilesToCollect > 0) {
LOG_INFO("WAL recovery finished successfully");
}
// recover state is not needed anymore
delete _recoverState;
_recoverState = nullptr;
return true;
}
@ -1520,6 +1553,13 @@ int LogfileManager::readShutdownInfo () {
lastSealedId = lastCollectedId;
}
std::string const shutdownTime(basics::JsonHelper::getStringValue(json, "shutdownTime"));
if (shutdownTime.empty()) {
LOG_TRACE("no previous shutdown time found");
}
else {
LOG_TRACE("previous shutdown was at '%s'", shutdownTime.c_str());
}
{
WRITE_LOCKER(_logfilesLock);
@ -1734,8 +1774,7 @@ int LogfileManager::inventory () {
int LogfileManager::inspectLogfiles () {
LOG_TRACE("inspecting WAL logfiles");
RecoverState state;
state.lastTick = 0;
TRI_ASSERT(_recoverState != nullptr);
// we're the only ones that access the logfiles at this point
// nevertheless, get the read-lock
@ -1748,7 +1787,12 @@ int LogfileManager::inspectLogfiles () {
Logfile* logfile = (*it).second;
if (logfile != nullptr) {
if (! scanLogfileTick(logfile, state)) {
if (logfile->status() == Logfile::StatusType::OPEN ||
logfile->status() == Logfile::StatusType::SEALED) {
++_recoverState->logfilesToCollect;
}
if (! scanLogfile(logfile)) {
LOG_TRACE("WAL inspection failed when scanning logfile '%s'", logfile->filename().c_str());
return TRI_ERROR_INTERNAL;
}
@ -1759,7 +1803,19 @@ int LogfileManager::inspectLogfiles () {
}
// update the tick with the max tick we found in the WAL
TRI_UpdateTickServer(state.lastTick);
TRI_UpdateTickServer(_recoverState->lastTick);
// note all failed transactions that we found plus the list
// of collections and databases that we can ignore
{
WRITE_LOCKER(_logfilesLock);
for (auto it = _recoverState->failedTransactions.begin(); it != _recoverState->failedTransactions.end(); ++it) {
_failedTransactions.insert((*it).first);
}
_droppedDatabases = _recoverState->droppedDatabases;
_droppedCollections = _recoverState->droppedCollections;
}
return TRI_ERROR_NO_ERROR;
}

101
arangod/Wal/LogfileManager.h
View File

@ -60,11 +60,21 @@ namespace triagens {
////////////////////////////////////////////////////////////////////////////////
struct RecoverState {
RecoverState ()
: collections(),
failedTransactions(),
droppedCollections(),
droppedDatabases(),
lastTick(0),
logfilesToCollect(0) {
}
std::unordered_map<TRI_voc_cid_t, TRI_voc_tick_t> collections;
std::unordered_set<TRI_voc_tid_t> failedTransactions;
std::unordered_map<TRI_voc_tid_t, std::pair<TRI_voc_tick_t, bool>> failedTransactions;
std::unordered_set<TRI_voc_cid_t> droppedCollections;
std::unordered_set<TRI_voc_tick_t> droppedDatabases;
TRI_voc_tick_t lastTick;
int logfilesToCollect;
};
// -----------------------------------------------------------------------------
@ -228,6 +238,46 @@ namespace triagens {
return _slots;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief whether or not write-throttling can be enabled
////////////////////////////////////////////////////////////////////////////////
inline bool canBeThrottled () const {
return (_throttleWhenPending > 0);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief maximum wait time when write-throttled (in milliseconds)
////////////////////////////////////////////////////////////////////////////////
inline uint64_t maxThrottleWait () const {
return _maxThrottleWait;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief whether or not write-throttling is currently enabled
////////////////////////////////////////////////////////////////////////////////
inline bool isThrottled () {
return (_writeThrottled != 0);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief activate write-throttling
////////////////////////////////////////////////////////////////////////////////
void activateWriteThrottling () {
_writeThrottled = 1;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief deactivate write-throttling
////////////////////////////////////////////////////////////////////////////////
void deactivateWriteThrottling () {
_writeThrottled = 0;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief allow or disallow writes to the WAL
////////////////////////////////////////////////////////////////////////////////
@ -236,6 +286,14 @@ namespace triagens {
_allowWrites = value;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief get the value of --wal.throttle-when-pending
////////////////////////////////////////////////////////////////////////////////
inline uint64_t throttleWhenPending () const {
return _throttleWhenPending;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief whether or not we are in the recovery mode
////////////////////////////////////////////////////////////////////////////////
@ -498,19 +556,17 @@ namespace triagens {
void waitForCollector (Logfile::IdType);
////////////////////////////////////////////////////////////////////////////////
/// @brief scan a single logfile for the max tick
////////////////////////////////////////////////////////////////////////////////
bool scanLogfileTick (Logfile const*,
RecoverState&);
////////////////////////////////////////////////////////////////////////////////
/// @brief scan a single logfile
////////////////////////////////////////////////////////////////////////////////
bool scanLogfile (Logfile const*,
RecoverState&);
bool scanLogfile (Logfile const*);
////////////////////////////////////////////////////////////////////////////////
/// @brief write abort markers for all open transactions
////////////////////////////////////////////////////////////////////////////////
void closeOpenTransactions ();
////////////////////////////////////////////////////////////////////////////////
/// @brief run the recovery procedure
@ -651,6 +707,12 @@ namespace triagens {
std::string _directory;
////////////////////////////////////////////////////////////////////////////////
/// @brief state during recovery
////////////////////////////////////////////////////////////////////////////////
RecoverState* _recoverState;
////////////////////////////////////////////////////////////////////////////////
/// @brief the size of each logfile
////////////////////////////////////////////////////////////////////////////////
@ -687,6 +749,19 @@ namespace triagens {
uint64_t _syncInterval;
////////////////////////////////////////////////////////////////////////////////
/// @brief maximum wait time for write-throttling
////////////////////////////////////////////////////////////////////////////////
uint64_t _maxThrottleWait;
////////////////////////////////////////////////////////////////////////////////
/// @brief throttle writes to WAL when at least such many operations are
/// waiting for garbage collection
////////////////////////////////////////////////////////////////////////////////
uint64_t _throttleWhenPending;
////////////////////////////////////////////////////////////////////////////////
/// @brief allow entries that are bigger than a single logfile
////////////////////////////////////////////////////////////////////////////////
@ -799,6 +874,12 @@ namespace triagens {
std::unordered_set<TRI_voc_tick_t> _droppedDatabases;
////////////////////////////////////////////////////////////////////////////////
/// @brief whether or not write-throttling is currently enabled
////////////////////////////////////////////////////////////////////////////////
alignas(64) int _writeThrottled;
////////////////////////////////////////////////////////////////////////////////
/// @brief regex to match logfiles
////////////////////////////////////////////////////////////////////////////////

34
js/actions/api-transaction.js
View File

@ -1,4 +1,4 @@
/*jslint indent: 2, nomen: true, maxlen: 100, sloppy: true, vars: true, white: true, plusplus: true */
/*jslint indent: 2, nomen: true, maxlen: 180, sloppy: true, vars: true, white: true, plusplus: true */
/*global require, TRANSACTION */
////////////////////////////////////////////////////////////////////////////////
@ -35,11 +35,6 @@ var actions = require("org/arangodb/actions");
// --SECTION-- private functions
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
/// @addtogroup ArangoAPI
/// @{
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @brief execute a server-side transaction
///
@ -78,9 +73,6 @@ var actions = require("org/arangodb/actions");
/// value will be used. Setting `lockTimeout` to `0` will make ArangoDB
/// not time out waiting for a lock.
///
/// - `replicate`: whether or not to replicate the operations from this
/// transaction. If not specified, the default value is `true`.
///
/// - `params`: optional arguments passed to `action`.
///
/// If the transaction is fully executed and committed on the server,
@ -148,10 +140,7 @@ var actions = require("org/arangodb/actions");
/// collections: {
/// write : "products"
/// },
/// action: "function () { " +
/// "var db = require('internal').db; " +
/// "db.products.save({}); " +
/// "return db.products.count(); }"
/// action: "function () { var db = require('internal').db; db.products.save({}); return db.products.count(); }"
/// };
///
/// var response = logCurlRequest('POST', url, body);
@ -177,11 +166,7 @@ var actions = require("org/arangodb/actions");
/// collections: {
/// write : [ "products", "materials" ]
/// },
/// action: "function () { " +
/// "var db = require('internal').db; " +
/// "db.products.save({}); " +
/// "db.materials.save({}); " +
/// "return 'worked!'; }"
/// action: "function () { var db = require('internal').db; db.products.save({}); db.materials.save({}); return 'worked!'; }"
/// };
///
/// var response = logCurlRequest('POST', url, body);
@ -203,10 +188,7 @@ var actions = require("org/arangodb/actions");
/// collections: {
/// write : "products"
/// },
/// action : "function () { " +
/// "var db = require('internal').db; " +
/// "db.products.save({ _key: 'abc'}); " +
/// "db.products.save({ _key: 'abc'}); }"
/// action : "function () { var db = require('internal').db; db.products.save({ _key: 'abc'}); db.products.save({ _key: 'abc'}); }"
/// };
///
/// var response = logCurlRequest('POST', url, body);
@ -271,10 +253,6 @@ function post_api_transaction(req, res) {
actions.resultOk(req, res, actions.HTTP_OK, { result : result });
}
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////
// -----------------------------------------------------------------------------
// --SECTION-- initialiser
// -----------------------------------------------------------------------------
@ -304,10 +282,6 @@ actions.defineHttp({
}
});
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////
// Local Variables:
// mode: outline-minor
// outline-regexp: "^\\(/// @brief\\|/// @addtogroup\\|// --SECTION--\\|/// @page\\|/// @}\\)"

37
js/apps/system/aardvark/frontend/js/bootstrap/module-internal.js
View File

@ -1,15 +1,16 @@
/*jslint indent: 2, nomen: true, maxlen: 120, vars: true, white: true, plusplus: true, nonpropdel: true, proto: true */
/*jslint sloppy: true, regexp: true */
/*global require, module, Module, ArangoError, SleepAndRequeue,
REPLICATION_LOGGER_STATE, REPLICATION_LOGGER_CONFIGURE, REPLICATION_APPLIER_CONFIGURE, REPLICATION_APPLIER_START,
REPLICATION_LOGGER_STATE, REPLICATION_LOGGER_CONFIGURE, REPLICATION_APPLIER_CONFIGURE, REPLICATION_APPLIER_START,
REPLICATION_APPLIER_STOP, REPLICATION_APPLIER_FORGET, REPLICATION_APPLIER_STATE,
REPLICATION_SYNCHRONISE, REPLICATION_SERVER_ID, CONFIGURE_ENDPOINT, REMOVE_ENDPOINT, LIST_ENDPOINTS,
SYS_BASE64DECODE, SYS_BASE64ENCODE, SYS_DEBUG_SEGFAULT,
SYS_DEBUG_CAN_USE_FAILAT, SYS_DEBUG_SET_FAILAT, SYS_DEBUG_REMOVE_FAILAT, SYS_DEBUG_CLEAR_FAILAT,
SYS_DOWNLOAD, SYS_EXECUTE, SYS_GET_CURRENT_REQUEST, SYS_GET_CURRENT_RESPONSE,
SYS_DEBUG_CAN_USE_FAILAT, SYS_DEBUG_SET_FAILAT, SYS_DEBUG_REMOVE_FAILAT, SYS_DEBUG_CLEAR_FAILAT,
SYS_DOWNLOAD, SYS_EXECUTE, SYS_GET_CURRENT_REQUEST, SYS_GET_CURRENT_RESPONSE,
SYS_LOAD, SYS_LOG_LEVEL, SYS_MD5, SYS_OUTPUT, SYS_PROCESS_STATISTICS,
SYS_RAND, SYS_SERVER_STATISTICS, SYS_SPRINTF, SYS_TIME, SYS_START_PAGER, SYS_STOP_PAGER,
SYS_HMAC, SYS_SHA256, SYS_SLEEP, SYS_WAIT, SYS_PARSE, SYS_IMPORT_CSV_FILE, SYS_IMPORT_JSON_FILE, SYS_LOG,
SYS_RAND, SYS_SERVER_STATISTICS, SYS_SPRINTF, SYS_TIME, SYS_START_PAGER, SYS_STOP_PAGER,
SYS_HMAC, SYS_SHA256, SYS_SHA224, SYS_SHA1, SYS_SLEEP, SYS_WAIT,
SYS_PARSE, SYS_IMPORT_CSV_FILE, SYS_IMPORT_JSON_FILE, SYS_LOG,
SYS_GEN_RANDOM_NUMBERS, SYS_GEN_RANDOM_ALPHA_NUMBERS, SYS_GEN_RANDOM_SALT, SYS_CREATE_NONCE,
SYS_CHECK_AND_MARK_NONCE, SYS_CLIENT_STATISTICS, SYS_HTTP_STATISTICS, SYS_UNIT_TESTS, SYS_UNIT_TESTS_RESULT:true,
SYS_PROCESS_CSV_FILE, SYS_PROCESS_JSON_FILE, ARANGO_QUIET, COLORS, COLOR_OUTPUT,
@ -81,10 +82,10 @@
this.message = this.toString();
};
exports.ArangoError.prototype = new Error();
}
exports.ArangoError.prototype = new Error();
}
exports.ArangoError.prototype._PRINT = function (context) {
context.output += this.toString();
};
@ -613,6 +614,24 @@
delete SYS_SHA256;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief sha224
////////////////////////////////////////////////////////////////////////////////
if (typeof SYS_SHA224 !== "undefined") {
exports.sha224 = SYS_SHA224;
delete SYS_SHA224;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief sha1
////////////////////////////////////////////////////////////////////////////////
if (typeof SYS_SHA1 !== "undefined") {
exports.sha1 = SYS_SHA1;
delete SYS_SHA1;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief serverStatistics
////////////////////////////////////////////////////////////////////////////////

261
js/apps/system/aardvark/frontend/js/modules/org/arangodb/general-graph.js
View File

@ -1539,6 +1539,44 @@ var _extendEdgeDefinitions = function (edgeDefinition) {
////////////////////////////////////////////////////////////////////////////////
/// @brief create a new graph
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_how_to_create
///
/// * Create a graph
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphCreateGraphHowTo1}
/// var graph_module = require("org/arangodb/general-graph");
/// var graph = graph_module._create("myGraph");
/// graph;
/// ~ graph_module._drop("myGraph");
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// * Add some vertex collections
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphCreateGraphHowTo2}
/// ~ var graph_module = require("org/arangodb/general-graph");
/// ~ var graph = graph_module._create("myGraph");
/// graph._addVertexCollection("shop");
/// graph._addVertexCollection("customer");
/// graph._addVertexCollection("pet");
/// graph;
/// ~ graph_module._drop("myGraph");
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// * Define relations on the
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphCreateGraphHowTo3}
/// ~ var graph_module = require("org/arangodb/general-graph");
/// ~ var graph = graph_module._create("myGraph");
/// var rel = graph_module._directedRelation("isCustomer", ["shop"], ["customer"]);
/// graph._extendEdgeDefinitions(rel);
/// graph;
/// ~ graph_module._drop("myGraph");
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// @endDocuBlock
///
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_create
@ -2939,6 +2977,215 @@ Graph.prototype._countCommonProperties = function(vertex1Example, vertex2Example
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_paths
///
/// `graph._paths (options)`
/// *The _paths function returns all paths of a graph.*
///
/// This function determines all available paths in a graph.
///
/// *Parameters*
///
/// * *options* (optional) : An object containing options, see below:
/// * *direction* : The direction of the edges. Possible values are *any*,
/// *inbound* and *outbound* (default).
/// * *followCycles* (optional) : If set to *true* the query follows cycles in the graph,
/// default is false.
/// * *minLength* (optional) : Defines the minimal length a path must
/// have to be returned (default is 0).
/// * *maxLength* (optional) : Defines the maximal length a path must
/// have to be returned (default is 10).
///
/// *Examples*
///
/// Return all paths of the graph "social":
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModulePaths}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("social");
/// g._paths();
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// Return all inbound paths of the graph "social" with a maximal
/// length of 1 and a minimal length of 2:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModulePaths2}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("social");
/// g._paths({direction : 'inbound', minLength : 1, maxLength : 2});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
Graph.prototype._paths = function(options) {
var query = "RETURN"
+ " GRAPH_PATHS(@graphName"
+ ',@options'
+ ')';
options = options || {};
var bindVars = {
"graphName": this.__name,
"options": options
};
var result = db._query(query, bindVars).toArray();
return result;
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_shortest_path
///
/// `graph._shortestPath (startVertexExample, endVertexExample, options)`
/// *The _shortestPath function returns all shortest paths of a graph.*
///
/// This function determines all shortest paths in a graph.
/// The function accepts an id, an example, a list of examples
/// or even an empty example as parameter for
/// start and end vertex. If one wants to call this function to receive nearly all
/// shortest paths for a graph the option *algorithm* should be set to
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// to increase performance.
/// If no algorithm is provided in the options the function chooses the appropriate
/// one (either [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// or [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm)) according to its parameters.
/// The length of a path is by default the amount of edges from one start vertex to
/// an end vertex. The option weight allows the user to define an edge attribute
/// representing the length.
///
/// *Parameters*
///
/// * *startVertexExample* (optional) : An example for the desired start Vertices
/// (see [Definition of examples](#definition_of_examples)).
/// * *endVertexExample* (optional) : An example for the desired
/// end Vertices (see [Definition of examples](#definition_of_examples)).
/// * *options* (optional) : An object containing options, see below:
/// * *direction* : The direction of the edges as a string.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *edgeCollectionRestriction* : One or multiple edge
/// collection names. Only edges from these collections will be considered for the path.
/// * *startVertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// start vertex of a path.
/// * *endVertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// end vertex of a path.
/// * *edgeExamples* : A filter example for the
/// edges in the shortest paths
/// (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *algorithm* : The algorithm to calculate
/// the shortest paths. If both start and end vertex examples are empty
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) is
/// used, otherwise the default is [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm)
/// * *weight* : The name of the attribute of
/// the edges containing the length as a string.
/// * *defaultWeight* : Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
/// is used as length.
/// If no default is supplied the default would be positive Infinity so the path could
/// not be calculated.
///
/// *Examples*
///
/// A route planner example, shortest path from all german to all french cities:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleShortestPaths1}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._shortestPath({}, {}, {weight : 'distance', endVertexCollectionRestriction : 'frenchCity',
/// startVertexCollectionRestriction : 'germanCity'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// A route planner example, shortest path from Hamburg and Cologne to Lyon:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleShortestPaths2}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._shortestPath([{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], 'frenchCity/Lyon',
/// {weight : 'distance'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
Graph.prototype._shortestPath = function(startVertexExample, endVertexExample, options) {
var ex1 = transformExample(startVertexExample);
var ex2 = transformExample(endVertexExample);
var query = "RETURN"
+ " GRAPH_SHORTEST_PATH(@graphName"
+ ',@ex1'
+ ',@ex2'
+ ',@options'
+ ')';
options = options || {};
var bindVars = {
"graphName": this.__name,
"options": options,
"ex1": ex1,
"ex2": ex2
};
var result = db._query(query, bindVars).toArray();
return result;
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_distance_to
///
/// `graph._distanceTo (startVertexExample, endVertexExample, options)`
/// *The _distanceTo function returns all paths and there distance within a graph.*
///
/// This function is a wrapper of [graph._shortestPath](#_shortestpath).
/// It does not return the actual path but only the distance between two vertices.
///
/// *Examples*
///
/// A route planner example, shortest distance from all german to all french cities:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleDistanceTo1}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._distanceTo({}, {}, {weight : 'distance', endVertexCollectionRestriction : 'frenchCity',
/// startVertexCollectionRestriction : 'germanCity'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// A route planner example, shortest distance from Hamburg and Cologne to Lyon:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleDistanceTo2}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._distanceTo([{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], 'frenchCity/Lyon',
/// {weight : 'distance'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
Graph.prototype._distanceTo = function(startVertexExample, endVertexExample, options) {
var ex1 = transformExample(startVertexExample);
var ex2 = transformExample(endVertexExample);
var query = "RETURN"
+ " GRAPH_DISTANCE_TO(@graphName"
+ ',@ex1'
+ ',@ex2'
+ ',@options'
+ ')';
options = options || {};
var bindVars = {
"graphName": this.__name,
"options": options,
"ex1": ex1,
"ex2": ex2
};
var result = db._query(query, bindVars).toArray();
return result;
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_absolute_eccentricity
///
@ -2963,7 +3210,8 @@ Graph.prototype._countCommonProperties = function(vertex1Example, vertex2Example
/// considered for target vertices.
/// * *edgeExamples*: Filter the edges to be followed, see [Definition of examples](#definition_of_examples)
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3091,7 +3339,8 @@ Graph.prototype._eccentricity = function(options) {
/// considered for target vertices.
/// * *edgeExamples*: Filter the edges to be followed, see [Definition of examples](#definition_of_examples)
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3339,7 +3588,8 @@ Graph.prototype._betweenness = function(options) {
/// * *options* (optional): An object defining further options. Can have the following values:
/// * *direction*: The direction of the edges. Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3407,7 +3657,8 @@ Graph.prototype._radius = function(options) {
/// * *options* (optional): An object defining further options. Can have the following values:
/// * *direction*: The direction of the edges. Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3930,6 +4181,8 @@ Graph.prototype._PRINT = function(context) {
context.output += name;
context.output += " EdgeDefinitions: ";
internal.printRecursive(edgeDefs, context);
context.output += " VertexCollections: ";
internal.printRecursive(this.__orphanCollections, context);
context.output += " ]";
};

2
js/apps/system/aardvark/frontend/js/routers/router.js
View File

@ -20,7 +20,7 @@
"databases": "databases",
"applications": "applications",
"application/documentation/:key": "appDocumentation",
"graph": "graph",
"graph": "graphManagement",
"graphManagement": "graphManagement",
"userManagement": "userManagement",
"userProfile": "userProfile",

81
js/apps/system/aardvark/frontend/js/views/graphManagementView.js
View File

@ -19,7 +19,7 @@
addNewGraph: function(e) {
e.preventDefault();
this.createNewGraphModal();
this.createNewGraphModal2();
},
deleteGraph: function(e) {
@ -54,7 +54,7 @@
this.collection.fetch();
this.graphToEdit = this.evaluateGraphName($(e.currentTarget).attr("id"), '_settings');
var graph = this.collection.findWhere({_key: this.graphToEdit});
this.createEditGraphModal(this.graphToEdit, graph.get("vertices"), graph.get("edges"));
this.createEditGraphModal2(this.graphToEdit, graph.get("vertices"), graph.get("edges"));
},
info : function(e) {
@ -263,6 +263,83 @@
window.modalView.show("modalTable.ejs", "Graph Properties", buttons, tableContent);
},
createNewGraphModal2: function() {
var buttons = [],
tableContent = [];
tableContent.push(
window.modalView.createTextEntry(
"createNewGraphName",
"Name",
"",
"The name to identify the graph. Has to be unique.",
"graphName",
true
)
);
tableContent.push(
window.modalView.createTextEntry(
"newEdgeDefinitions",
"Edge definitions",
"",
"Some info for edge definitions",
"Edge definitions",
true
)
);
tableContent.push(
window.modalView.createSelect2Entry(
"newVertexCollections",
"Vertex collections",
"",
"Some info for vertex collections",
"Vertex Collections",
false
)
);
buttons.push(
window.modalView.createSuccessButton("Create", this.createNewGraph.bind(this))
);
window.modalView.show("modalTable.ejs", "Add new Graph", buttons, tableContent);
},
createEditGraphModal2: function(name, vertices, edges) {
var buttons = [],
tableContent = [];
tableContent.push(
window.modalView.createReadOnlyEntry(
"editGraphName",
"Name",
name,
false
)
);
tableContent.push(
window.modalView.createReadOnlyEntry(
"editVertices",
"Vertices",
vertices,
false
)
);
tableContent.push(
window.modalView.createReadOnlyEntry(
"editEdges",
"Edges",
edges,
false
)
);
buttons.push(
window.modalView.createDeleteButton("Delete", this.deleteGraph.bind(this))
);
window.modalView.show("modalTable.ejs", "Edit Graph", buttons, tableContent);
}
});

37
js/common/bootstrap/module-internal.js
View File

@ -1,15 +1,16 @@
/*jslint indent: 2, nomen: true, maxlen: 120, vars: true, white: true, plusplus: true, nonpropdel: true, proto: true */
/*jslint sloppy: true, regexp: true */
/*global require, module, Module, ArangoError, SleepAndRequeue,
REPLICATION_LOGGER_STATE, REPLICATION_LOGGER_CONFIGURE, REPLICATION_APPLIER_CONFIGURE, REPLICATION_APPLIER_START,
REPLICATION_LOGGER_STATE, REPLICATION_LOGGER_CONFIGURE, REPLICATION_APPLIER_CONFIGURE, REPLICATION_APPLIER_START,
REPLICATION_APPLIER_STOP, REPLICATION_APPLIER_FORGET, REPLICATION_APPLIER_STATE,
REPLICATION_SYNCHRONISE, REPLICATION_SERVER_ID, CONFIGURE_ENDPOINT, REMOVE_ENDPOINT, LIST_ENDPOINTS,
SYS_BASE64DECODE, SYS_BASE64ENCODE, SYS_DEBUG_SEGFAULT,
SYS_DEBUG_CAN_USE_FAILAT, SYS_DEBUG_SET_FAILAT, SYS_DEBUG_REMOVE_FAILAT, SYS_DEBUG_CLEAR_FAILAT,
SYS_DOWNLOAD, SYS_EXECUTE, SYS_GET_CURRENT_REQUEST, SYS_GET_CURRENT_RESPONSE,
SYS_DEBUG_CAN_USE_FAILAT, SYS_DEBUG_SET_FAILAT, SYS_DEBUG_REMOVE_FAILAT, SYS_DEBUG_CLEAR_FAILAT,
SYS_DOWNLOAD, SYS_EXECUTE, SYS_GET_CURRENT_REQUEST, SYS_GET_CURRENT_RESPONSE,
SYS_LOAD, SYS_LOG_LEVEL, SYS_MD5, SYS_OUTPUT, SYS_PROCESS_STATISTICS,
SYS_RAND, SYS_SERVER_STATISTICS, SYS_SPRINTF, SYS_TIME, SYS_START_PAGER, SYS_STOP_PAGER,
SYS_HMAC, SYS_SHA256, SYS_SLEEP, SYS_WAIT, SYS_PARSE, SYS_IMPORT_CSV_FILE, SYS_IMPORT_JSON_FILE, SYS_LOG,
SYS_RAND, SYS_SERVER_STATISTICS, SYS_SPRINTF, SYS_TIME, SYS_START_PAGER, SYS_STOP_PAGER,
SYS_HMAC, SYS_SHA256, SYS_SHA224, SYS_SHA1, SYS_SLEEP, SYS_WAIT,
SYS_PARSE, SYS_IMPORT_CSV_FILE, SYS_IMPORT_JSON_FILE, SYS_LOG,
SYS_GEN_RANDOM_NUMBERS, SYS_GEN_RANDOM_ALPHA_NUMBERS, SYS_GEN_RANDOM_SALT, SYS_CREATE_NONCE,
SYS_CHECK_AND_MARK_NONCE, SYS_CLIENT_STATISTICS, SYS_HTTP_STATISTICS, SYS_UNIT_TESTS, SYS_UNIT_TESTS_RESULT:true,
SYS_PROCESS_CSV_FILE, SYS_PROCESS_JSON_FILE, ARANGO_QUIET, COLORS, COLOR_OUTPUT,
@ -81,10 +82,10 @@
this.message = this.toString();
};
exports.ArangoError.prototype = new Error();
}
exports.ArangoError.prototype = new Error();
}
exports.ArangoError.prototype._PRINT = function (context) {
context.output += this.toString();
};
@ -613,6 +614,24 @@
delete SYS_SHA256;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief sha224
////////////////////////////////////////////////////////////////////////////////
if (typeof SYS_SHA224 !== "undefined") {
exports.sha224 = SYS_SHA224;
delete SYS_SHA224;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief sha1
////////////////////////////////////////////////////////////////////////////////
if (typeof SYS_SHA1 !== "undefined") {
exports.sha1 = SYS_SHA1;
delete SYS_SHA1;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief serverStatistics
////////////////////////////////////////////////////////////////////////////////

16
js/common/modules/org/arangodb/crypto.js
View File

@ -81,6 +81,22 @@ exports.sha256 = function (value) {
return internal.sha256(value);
};
////////////////////////////////////////////////////////////////////////////////
/// @brief apply an SHA 224 hash
////////////////////////////////////////////////////////////////////////////////
exports.sha224 = function (value) {
return internal.sha224(value);
};
////////////////////////////////////////////////////////////////////////////////
/// @brief apply an SHA 1 hash
////////////////////////////////////////////////////////////////////////////////
exports.sha1 = function (value) {
return internal.sha1(value);
};
////////////////////////////////////////////////////////////////////////////////
/// @brief Generates a string of a given length containing numbers.
////////////////////////////////////////////////////////////////////////////////

261
js/common/modules/org/arangodb/general-graph.js
View File

@ -1538,6 +1538,44 @@ var _extendEdgeDefinitions = function (edgeDefinition) {
////////////////////////////////////////////////////////////////////////////////
/// @brief create a new graph
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_how_to_create
///
/// * Create a graph
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphCreateGraphHowTo1}
/// var graph_module = require("org/arangodb/general-graph");
/// var graph = graph_module._create("myGraph");
/// graph;
/// ~ graph_module._drop("myGraph");
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// * Add some vertex collections
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphCreateGraphHowTo2}
/// ~ var graph_module = require("org/arangodb/general-graph");
/// ~ var graph = graph_module._create("myGraph");
/// graph._addVertexCollection("shop");
/// graph._addVertexCollection("customer");
/// graph._addVertexCollection("pet");
/// graph;
/// ~ graph_module._drop("myGraph");
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// * Define relations on the
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphCreateGraphHowTo3}
/// ~ var graph_module = require("org/arangodb/general-graph");
/// ~ var graph = graph_module._create("myGraph");
/// var rel = graph_module._directedRelation("isCustomer", ["shop"], ["customer"]);
/// graph._extendEdgeDefinitions(rel);
/// graph;
/// ~ graph_module._drop("myGraph");
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// @endDocuBlock
///
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_create
@ -2938,6 +2976,215 @@ Graph.prototype._countCommonProperties = function(vertex1Example, vertex2Example
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_paths
///
/// `graph._paths (options)`
/// *The _paths function returns all paths of a graph.*
///
/// This function determines all available paths in a graph.
///
/// *Parameters*
///
/// * *options* (optional) : An object containing options, see below:
/// * *direction* : The direction of the edges. Possible values are *any*,
/// *inbound* and *outbound* (default).
/// * *followCycles* (optional) : If set to *true* the query follows cycles in the graph,
/// default is false.
/// * *minLength* (optional) : Defines the minimal length a path must
/// have to be returned (default is 0).
/// * *maxLength* (optional) : Defines the maximal length a path must
/// have to be returned (default is 10).
///
/// *Examples*
///
/// Return all paths of the graph "social":
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModulePaths}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("social");
/// g._paths();
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// Return all inbound paths of the graph "social" with a maximal
/// length of 1 and a minimal length of 2:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModulePaths2}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("social");
/// g._paths({direction : 'inbound', minLength : 1, maxLength : 2});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
Graph.prototype._paths = function(options) {
var query = "RETURN"
+ " GRAPH_PATHS(@graphName"
+ ',@options'
+ ')';
options = options || {};
var bindVars = {
"graphName": this.__name,
"options": options
};
var result = db._query(query, bindVars).toArray();
return result;
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_shortest_path
///
/// `graph._shortestPath (startVertexExample, endVertexExample, options)`
/// *The _shortestPath function returns all shortest paths of a graph.*
///
/// This function determines all shortest paths in a graph.
/// The function accepts an id, an example, a list of examples
/// or even an empty example as parameter for
/// start and end vertex. If one wants to call this function to receive nearly all
/// shortest paths for a graph the option *algorithm* should be set to
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// to increase performance.
/// If no algorithm is provided in the options the function chooses the appropriate
/// one (either [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// or [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm)) according to its parameters.
/// The length of a path is by default the amount of edges from one start vertex to
/// an end vertex. The option weight allows the user to define an edge attribute
/// representing the length.
///
/// *Parameters*
///
/// * *startVertexExample* (optional) : An example for the desired start Vertices
/// (see [Definition of examples](#definition_of_examples)).
/// * *endVertexExample* (optional) : An example for the desired
/// end Vertices (see [Definition of examples](#definition_of_examples)).
/// * *options* (optional) : An object containing options, see below:
/// * *direction* : The direction of the edges as a string.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *edgeCollectionRestriction* : One or multiple edge
/// collection names. Only edges from these collections will be considered for the path.
/// * *startVertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// start vertex of a path.
/// * *endVertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// end vertex of a path.
/// * *edgeExamples* : A filter example for the
/// edges in the shortest paths
/// (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *algorithm* : The algorithm to calculate
/// the shortest paths. If both start and end vertex examples are empty
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) is
/// used, otherwise the default is [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm)
/// * *weight* : The name of the attribute of
/// the edges containing the length as a string.
/// * *defaultWeight* : Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
/// is used as length.
/// If no default is supplied the default would be positive Infinity so the path could
/// not be calculated.
///
/// *Examples*
///
/// A route planner example, shortest path from all german to all french cities:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleShortestPaths1}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._shortestPath({}, {}, {weight : 'distance', endVertexCollectionRestriction : 'frenchCity',
/// startVertexCollectionRestriction : 'germanCity'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// A route planner example, shortest path from Hamburg and Cologne to Lyon:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleShortestPaths2}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._shortestPath([{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], 'frenchCity/Lyon',
/// {weight : 'distance'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
Graph.prototype._shortestPath = function(startVertexExample, endVertexExample, options) {
var ex1 = transformExample(startVertexExample);
var ex2 = transformExample(endVertexExample);
var query = "RETURN"
+ " GRAPH_SHORTEST_PATH(@graphName"
+ ',@ex1'
+ ',@ex2'
+ ',@options'
+ ')';
options = options || {};
var bindVars = {
"graphName": this.__name,
"options": options,
"ex1": ex1,
"ex2": ex2
};
var result = db._query(query, bindVars).toArray();
return result;
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_distance_to
///
/// `graph._distanceTo (startVertexExample, endVertexExample, options)`
/// *The _distanceTo function returns all paths and there distance within a graph.*
///
/// This function is a wrapper of [graph._shortestPath](#_shortestpath).
/// It does not return the actual path but only the distance between two vertices.
///
/// *Examples*
///
/// A route planner example, shortest distance from all german to all french cities:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleDistanceTo1}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._distanceTo({}, {}, {weight : 'distance', endVertexCollectionRestriction : 'frenchCity',
/// startVertexCollectionRestriction : 'germanCity'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// A route planner example, shortest distance from Hamburg and Cologne to Lyon:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphModuleDistanceTo2}
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("routeplanner");
/// | g._distanceTo([{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], 'frenchCity/Lyon',
/// {weight : 'distance'});
/// @END_EXAMPLE_ARANGOSH_OUTPUT
///
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
Graph.prototype._distanceTo = function(startVertexExample, endVertexExample, options) {
var ex1 = transformExample(startVertexExample);
var ex2 = transformExample(endVertexExample);
var query = "RETURN"
+ " GRAPH_DISTANCE_TO(@graphName"
+ ',@ex1'
+ ',@ex2'
+ ',@options'
+ ')';
options = options || {};
var bindVars = {
"graphName": this.__name,
"options": options,
"ex1": ex1,
"ex2": ex2
};
var result = db._query(query, bindVars).toArray();
return result;
};
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_general_graph_absolute_eccentricity
///
@ -2962,7 +3209,8 @@ Graph.prototype._countCommonProperties = function(vertex1Example, vertex2Example
/// considered for target vertices.
/// * *edgeExamples*: Filter the edges to be followed, see [Definition of examples](#definition_of_examples)
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3090,7 +3338,8 @@ Graph.prototype._eccentricity = function(options) {
/// considered for target vertices.
/// * *edgeExamples*: Filter the edges to be followed, see [Definition of examples](#definition_of_examples)
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3338,7 +3587,8 @@ Graph.prototype._betweenness = function(options) {
/// * *options* (optional): An object defining further options. Can have the following values:
/// * *direction*: The direction of the edges. Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3406,7 +3656,8 @@ Graph.prototype._radius = function(options) {
/// * *options* (optional): An object defining further options. Can have the following values:
/// * *direction*: The direction of the edges. Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm*: The algorithm to calculate the shortest paths, possible values are
/// *"Floyd-Warshall"* and *"Dijkstra"*.
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) and
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight*: The name of the attribute of the edges containing the weight.
/// * *defaultWeight*: Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -3929,6 +4180,8 @@ Graph.prototype._PRINT = function(context) {
context.output += name;
context.output += " EdgeDefinitions: ";
internal.printRecursive(edgeDefs, context);
context.output += " VertexCollections: ";
internal.printRecursive(this.__orphanCollections, context);
context.output += " ]";
};

2
js/common/modules/org/arangodb/testrunner.js
View File

@ -63,7 +63,7 @@ runCommandLineTests = function (opts) {
options = opts || {},
jasmineReportFormat = options.jasmineReportFormat || 'progress',
unitTests = internal.unitTests(),
isSpecRegEx = /.+spec.js/,
isSpecRegEx = /.+spec\.js/,
isSpec = function (unitTest) {
return isSpecRegEx.test(unitTest);
},

68
js/common/tests/shell-crypto.js
View File

@ -105,6 +105,72 @@ function CryptoSuite () {
});
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test sha224, invalid values
////////////////////////////////////////////////////////////////////////////////
testSha224Invalid : function () {
[ undefined, null, true, false, 0, 1, -1, 32.5, [ ], { } ].forEach(function (value) {
try {
crypto.sha224(value);
fail();
}
catch (err) {
}
});
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test sha224
////////////////////////////////////////////////////////////////////////////////
testSha224 : function () {
var data = [
[ "", "d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f" ],
[ " ", "ca17734c016e36b898af29c1aeb142e774abf4b70bac55ec98a27ba8"],
[ "arangodb", "deeb6d8f9b6c316e7f5a601fb2549a2ebc857bee78df38b1977c9989" ],
[ "Arangodb", "6d7a8a7fb22537dab437c8c2915874a170b2b14eb1aa787df32d6999" ],
[ "ArangoDB is a database", "9a3e02d47eb686c67f6b9a51efe16e8b4f88b0ee14248636d6163f1d" ]
];
data.forEach(function (value) {
assertEqual(value[1], crypto.sha224(value[0]));
});
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test sha1, invalid values
////////////////////////////////////////////////////////////////////////////////
testSha1Invalid : function () {
[ undefined, null, true, false, 0, 1, -1, 32.5, [ ], { } ].forEach(function (value) {
try {
crypto.sha1(value);
fail();
}
catch (err) {
}
});
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test sha1
////////////////////////////////////////////////////////////////////////////////
testSha1 : function () {
var data = [
[ "", "da39a3ee5e6b4b0d3255bfef95601890afd80709" ],
[ " ", "b858cb282617fb0956d960215c8e84d1ccf909c6"],
[ "arangodb", "1f9aa6577198dd5fcab487d08d45258608dac9b5" ],
[ "Arangodb", "5fc1b451c5cd4770df14bd3ae362b5587a195311" ],
[ "ArangoDB is a database", "9e45475b50ea3e8438c55919238aa5b0736bda43" ]
];
data.forEach(function (value) {
assertEqual(value[1], crypto.sha1(value[0]));
});
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test hmac, invalid values
////////////////////////////////////////////////////////////////////////////////
@ -142,6 +208,8 @@ function CryptoSuite () {
[ "secret", "Arangodb", "sha256", "95144e880bbc4a4bf10a2b683603c763a38817b544e1c2f6ff1bd3523bf60f9e" ],
[ "secret", "ArangoDB is a database", "sha256", "4888d586d3208ca18ebaf78569949a13f3c03585edb007771cd820820a351b0f" ],
[ "SECRET", "ArangoDB is a database", "sha256", "a04df5ce362f49439db5e30032b20e0fa64d01c60ceb32a9150e58d3c2c929af" ],
[ "secret", "ArangoDB is a database", "sha224", "b55c13e25227abf919b510cf2289f4501fa13584676e7e4d56108172" ],
[ "secret", "ArangoDB is a database", "SHA224", "b55c13e25227abf919b510cf2289f4501fa13584676e7e4d56108172" ],
[ "secret", "ArangoDB is a database", "sha1", "f39d7a76e502ba3f79d663cfbc9ac43eb6fd323e" ],
[ "secret", "ArangoDB is a database", "SHA1", "f39d7a76e502ba3f79d663cfbc9ac43eb6fd323e" ],
[ "secret", "ArangoDB is a database", "md5", "6eecfc947725974efc24bbaaafe15a13" ],

16
js/common/tests/shell-general-graph.js
View File

@ -2623,7 +2623,7 @@ function MeasurementsSuite() {
var ids = {};
var vertex = g[vc1].save({first_name: "Tam"});
ids.vId11 = vertex._id;
vertex = g[vc1].save({first_name: "Tem"});
vertex = g[vc1].save({first_name: "Tem", age : 20});
ids.vId12 = vertex._id;
vertex = g[vc1].save({first_name: "Tim"});
ids.vId13 = vertex._id;
@ -2635,7 +2635,7 @@ function MeasurementsSuite() {
ids.vId31 = vertex._id;
vertex = g[vc3].save({first_name: "Tem"});
ids.vId32 = vertex._id;
vertex = g[vc3].save({first_name: "Tim"});
vertex = g[vc3].save({first_name: "Tim", age : 24});
ids.vId33 = vertex._id;
vertex = g[vc3].save({first_name: "Tom"});
ids.vId34 = vertex._id;
@ -2726,6 +2726,18 @@ function MeasurementsSuite() {
test_diameter : function () {
var a = g._diameter({});
assertEqual(a[0] , 2);
},
test_paths : function () {
var a = g._paths({maxLength : 2});
assertEqual(a[0].length , 50);
},
test_shortestPaths : function () {
var a = g._shortestPath([{first_name: 'Tim',age : 24}, {first_name: 'Tom'}], [{first_name: 'Tam'}, {first_name: 'Tem',age : 20}]);
assertEqual(a[0].length , 9);
},
test_distanceTo : function () {
var a = g._distanceTo([{first_name: 'Tim',age : 24}, {first_name: 'Tom'}], [{first_name: 'Tam'}, {first_name: 'Tem' ,age : 20}]);
assertEqual(a[0].length , 9);
}

166
js/server/modules/org/arangodb/ahuacatl.js
View File

@ -4342,22 +4342,20 @@ function GRAPH_PATHS (vertices, edgeCollection, direction, followCycles, minLeng
////////////////////////////////////////////////////////////////////////////////
/// @startDocuBlock JSF_ahuacatl_general_graph_paths
///
/// `GRAPH_PATHS (graphName, direction, followCycles, minLength, maxLength)`
/// `GRAPH_PATHS (graphName, options)`
/// *The GRAPH\_PATHS function returns all paths of a graph.*
///
/// This function determines all available paths in a graph identified by *graphName*.
/// Except for *graphName* every other parameter is optional.
///
/// *Parameters*
///
/// * *graphName* : The name of the graph as a string.
/// * *direction* (optional) : The direction of the edges as a string.
/// Possible values are *any*, *inbound* and *outbound* (default).
/// * *followCycles* (optional) : If set to *true* the query follows cycles in the graph,
/// * *options* : An object containing options, see below:
/// * *direction* : The direction of the edges. Possible values are *any*,
/// *inbound* and *outbound* (default).
/// * *followCycles* (optional) : If set to *true* the query follows cycles in the graph,
/// default is false.
/// * *minLength* (optional) : Defines the minimal length a path must
/// * *minLength* (optional) : Defines the minimal length a path must
/// have to be returned (default is 0).
/// * *maxLength* (optional) : Defines the maximal length a path must
/// * *maxLength* (optional) : Defines the maximal length a path must
/// have to be returned (default is 10).
///
/// *Examples*
@ -4378,20 +4376,25 @@ function GRAPH_PATHS (vertices, edgeCollection, direction, followCycles, minLeng
/// ~ var db = require("internal").db;
/// var examples = require("org/arangodb/graph-examples/example-graph.js");
/// var g = examples.loadGraph("social");
/// db._query("RETURN GRAPH_PATHS('social', 'inbound', false, 1, 2)").toArray();
/// | db._query(
/// | "RETURN GRAPH_PATHS('social', {direction : 'inbound', minLength : 1, maxLength : 2})"
/// ).toArray();
/// @END_EXAMPLE_ARANGOSH_OUTPUT
/// @endDocuBlock
//
////////////////////////////////////////////////////////////////////////////////
function GENERAL_GRAPH_PATHS (graphName, direction, followCycles, minLength, maxLength) {
function GENERAL_GRAPH_PATHS (graphName, options) {
"use strict";
var searchDirection;
direction = direction || "outbound";
followCycles = followCycles || false;
minLength = minLength || 0;
maxLength = maxLength !== undefined ? maxLength : 10;
if (! options) {
options = {};
}
var direction = options.direction || "outbound";
var followCycles = options.followCycles || false;
var minLength = options.minLength || 0;
var maxLength = options.maxLength || 10;
// check graph exists and load edgeDefintions
var graph = DOCUMENT_HANDLE("_graphs/" + graphName);
@ -5236,13 +5239,13 @@ function IS_EXAMPLE_SET (example) {
/// *The GRAPH\_SHORTEST\_PATH function returns all shortest paths of a graph.*
///
/// This function determines all shortest paths in a graph identified by *graphName*.
/// The function accepts an id, an example, a list of examples
/// or even an empty example as parameter for
/// start and end vertex. If one wants to call this function to receive nearly all
/// shortest paths for a graph the
/// option *algorithm* should be set to *Floyd-Warshall* to increase performance.
/// If one wants to call this function to receive nearly all shortest paths for a
/// graph the option *algorithm* should be set to
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) to
/// increase performance.
/// If no algorithm is provided in the options the function chooses the appropriate
/// one (either *Floyd-Warshall* or *Dijsktra*) according to its parameters.
/// one (either [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// or [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm)) according to its parameters.
/// The length of a path is by default the amount of edges from one start vertex to
/// an end vertex. The option weight allows the user to define an edge attribute
/// representing the length.
@ -5258,17 +5261,20 @@ function IS_EXAMPLE_SET (example) {
/// * *direction* : The direction of the edges as a string.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *edgeCollectionRestriction* : One or multiple edge
/// collections that should be considered.
/// collection names. Only edges from these collections will be considered for the path.
/// * *startVertexCollectionRestriction* : One or multiple vertex
/// collections that should be considered.
/// collection names. Only vertices from these collections will be considered as
/// start vertex of a path.
/// * *endVertexCollectionRestriction* : One or multiple vertex
/// collections that should be considered.
/// collection names. Only vertices from these collections will be considered as
/// end vertex of a path.
/// * *edgeExamples* : A filter example for the
/// edges in the shortest paths
/// (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *algorithm* : The algorithm to calculate
/// the shortest paths. If both start and end vertex examples are empty *Floyd-Warshall* is
/// used, otherwise the default is *Dijkstra*
/// the shortest paths. If both start and end vertex examples are empty
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) is
/// used, otherwise the default is [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight* : The name of the attribute of
/// the edges containing the length as a string.
/// * *defaultWeight* : Only used with the option *weight*.
@ -5279,7 +5285,7 @@ function IS_EXAMPLE_SET (example) {
///
/// *Examples*
///
/// A route planner example, shortest distance from all German to all french cities:
/// A route planner example, shortest distance from all german to all french cities:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphShortestPaths1}
/// ~ var db = require("internal").db;
@ -5489,7 +5495,7 @@ function GRAPH_TRAVERSAL_TREE (vertexCollection,
///
/// *Examples*
///
/// A route planner example, distance from all french to all German cities:
/// A route planner example, distance from all french to all german cities:
///
/// @EXAMPLE_ARANGOSH_OUTPUT{generalGraphDistanceTo1}
/// ~ var db = require("internal").db;
@ -5709,8 +5715,8 @@ function GRAPH_NEIGHBORS (vertexCollection,
/// `GRAPH_NEIGHBORS (graphName, vertexExample, options)`
/// *The GRAPH\_NEIGHBORS function returns all neighbors of vertices.*
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertex.
/// By default only the direct neighbors (path length equals 1) are returned, but one can define
/// the range of the path length to the neighbors with the options *minDepth* and *maxDepth*.
///
/// *Parameters*
///
@ -5724,10 +5730,11 @@ function GRAPH_NEIGHBORS (vertexCollection,
/// the neighbors (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *neighborExamples* : An example for the desired neighbors
/// (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *edgeCollectionRestriction* : One or multiple
/// edge collections that should be considered.
/// * *vertexCollectionRestriction* : One or multiple
/// vertex collections that should be considered.
/// * *edgeCollectionRestriction* : One or multiple edge
/// collection names. Only edges from these collections will be considered for the path.
/// * *vertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// neighbor.
/// * *minDepth* : Defines the minimal
/// depth a path to a neighbor must have to be returned (default is 1).
/// * *maxDepth* : Defines the maximal
@ -5821,10 +5828,8 @@ function GENERAL_GRAPH_NEIGHBORS (graphName,
/// @startDocuBlock JSF_ahuacatl_general_graph_edges
///
/// `GRAPH_EDGES (graphName, vertexExample, options)`
/// *The GRAPH\_EDGES function returns all edges of vertices.*
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertex.
/// *The GRAPH\_EDGES function returns all edges of the graph connected to the vertices
/// defined by the example.*
///
/// *Parameters*
///
@ -5834,20 +5839,24 @@ function GENERAL_GRAPH_NEIGHBORS (graphName,
/// * *options* (optional) : An object containing options, see below:
/// * *direction* : The direction
/// of the edges as a string. Possible values are *outbound*, *inbound* and *any* (default).
/// * *edgeCollectionRestriction* : One or multiple
/// edge collections that should be considered.
/// * *startVertexCollectionRestriction* : One or multiple
/// vertex collections that should be considered.
/// * *endVertexCollectionRestriction* : One or multiple
/// vertex collections that should be considered.
/// * *edgeCollectionRestriction* : One or multiple edge
/// collection names. Only edges from these collections will be considered for the path.
/// * *startVertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// start vertex of a path.
/// * *endVertexCollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered as
/// end vertex of a path.
/// * *edgeExamples* : A filter example
/// for the edges (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *neighborExamples* : An example for
/// the desired neighbors (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *minDepth* : Defines the minimal
/// depth a path to a neighbor must have to be returned (default is 1).
/// * *maxDepth* : Defines the maximal
/// depth a path to a neighbor must have to be returned (default is 1).
/// * *minDepth* : Defines the minimal length of a path from an edge
/// to a vertex (default is 1, which means only the edges directly connected to a vertex would
/// be returned).
/// * *maxDepth* : Defines the maximal length of a path from an edge
/// to a vertex (default is 1, which means only the edges directly connected to a vertex would
/// be returned).
///
/// *Examples*
///
@ -5901,8 +5910,6 @@ function GENERAL_GRAPH_EDGES (
/// `GRAPH_VERTICES (graphName, vertexExample, options)`
/// *The GRAPH\_VERTICES function returns all vertices.*
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertex.
/// According to the optional filters it will only return vertices that have
/// outbound, onbound or any (default) edges.
///
@ -5996,9 +6003,6 @@ function TRANSFER_GENERAL_GRAPH_NEIGHBORS_RESULT (result) {
/// *The GRAPH\_COMMON\_NEIGHBORS function returns all common neighbors of the vertices
/// defined by the examples.*
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertex1Example and vertex2Example.
///
/// This function returns the intersection of *GRAPH_NEIGHBORS(vertex1Example, optionsVertex1)*
/// and *GRAPH_NEIGHBORS(vertex2Example, optionsVertex2)*.
/// For parameter documentation read the documentation of
@ -6096,9 +6100,6 @@ function GENERAL_GRAPH_COMMON_NEIGHBORS (
/// *The GRAPH\_COMMON\_PROPERTIES function returns all vertices
/// defined by the examples that share common properties
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertex1Example and vertex2Example.
///
/// *Parameters*
///
/// * *graphName* : The name of the graph as a string.
@ -6107,10 +6108,10 @@ function GENERAL_GRAPH_COMMON_NEIGHBORS (
/// * *vertex2Example* : An example for the desired
/// vertices (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *options* (optional) : An object containing options, see below:
/// * *vertex1CollectionRestriction* : One or multiple
/// vertex collections that should be considered.
/// * *vertex2CollectionRestriction* : One or multiple
/// vertex collections that should be considered.
/// * *vertex1CollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered.
/// * *vertex2CollectionRestriction* : One or multiple vertex
/// collection names. Only vertices from these collections will be considered.
/// * *ignoreProperties* : One or multiple
/// attributes of a document that should be ignored, either a string or an array..
///
@ -6235,9 +6236,6 @@ function GENERAL_GRAPH_COMMON_PROPERTIES (
/// [eccentricity](http://en.wikipedia.org/wiki/Distance_%28graph_theory%29)
/// of the vertices defined by the examples.*
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertexExample.
///
/// *Parameters*
///
/// * *graphName* : The name of the graph as a string.
@ -6247,16 +6245,20 @@ function GENERAL_GRAPH_COMMON_PROPERTIES (
/// * *direction* : The direction of the edges as a string.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *edgeCollectionRestriction* : One or multiple edge
/// collections that should be considered.
/// collection names. Only edges from these collections will be considered for the path.
/// * *startVertexCollectionRestriction* : One or multiple vertex
/// collections that should be considered.
/// collection names. Only vertices from these collections will be considered as
/// start vertex of a path.
/// * *endVertexCollectionRestriction* : One or multiple vertex
/// collections that should be considered.
/// collection names. Only vertices from these collections will be considered as
/// end vertex of a path.
/// * *edgeExamples* : A filter example for the edges in the
/// shortest paths (see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *algorithm* : The algorithm to calculate
/// the shortest paths as a string. If vertex example is empty *Floyd-Warshall* is
/// used as default, otherwise the default is *Dijkstra*
/// the shortest paths as a string. If vertex example is empty
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) is
/// used as default, otherwise the default is
/// [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm)
/// * *weight* : The name of the attribute of
/// the edges containing the length as a string.
/// * *defaultWeight* : Only used with the option *weight*.
@ -6342,7 +6344,8 @@ function GENERAL_GRAPH_ABSOLUTE_ECCENTRICITY (graphName, vertexExample, options)
/// * *direction* : The direction of the edges as a string.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm* : The algorithm to calculate the shortest paths as a string. Possible
/// values are *Floyd-Warshall* (default) and *Dijkstra*.
/// values are [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// (default) and [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight* : The name of the attribute of the edges containing the length as a string.
/// * *defaultWeight* : Only used with the option *weight*.
/// If an edge does not have the attribute named as defined in option *weight* this default
@ -6409,9 +6412,6 @@ function GENERAL_GRAPH_ECCENTRICITY (graphName, options) {
/// [closeness](http://en.wikipedia.org/wiki/Centrality#Closeness_centrality)
/// of the vertices defined by the examples.*
///
/// The function accepts an id, an example, a list of examples or even an empty
/// example as parameter for vertexExample.
///
/// *Parameters*
///
/// * *graphName* : The name of the graph as a string.
@ -6421,16 +6421,20 @@ function GENERAL_GRAPH_ECCENTRICITY (graphName, options) {
/// * *direction* : The direction of the edges.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *edgeCollectionRestriction* : One or multiple edge
/// collections that should be considered.
/// collection names. Only edges from these collections will be considered for the path.
/// * *startVertexCollectionRestriction* : One or multiple vertex
/// collections that should be considered.
/// collection names. Only vertices from these collections will be considered as
/// start vertex of a path.
/// * *endVertexCollectionRestriction* : One or multiple vertex
/// collections that should be considered.
/// collection names. Only vertices from these collections will be considered as
/// end vertex of a path.
/// * *edgeExamples* : A filter example for the
/// edges in the shortest paths (
/// see [example](#short_explaination_of_the_vertex_example_parameter)).
/// * *algorithm* : The algorithm to calculate
/// the shortest paths. Possible values are *Floyd-Warshall* (default) and *Dijkstra*.
/// the shortest paths. Possible values are
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) (default)
/// and [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight* : The name of the attribute of
/// the edges containing the length.
/// * *defaultWeight* : Only used with the option *weight*.
@ -6516,7 +6520,9 @@ function GENERAL_GRAPH_ABSOLUTE_CLOSENESS (graphName, vertexExample, options) {
/// * *direction* : The direction of the edges.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm* : The algorithm to calculate
/// the shortest paths. Possible values are *Floyd-Warshall* (default) and *Dijkstra*.
/// the shortest paths. Possible values are
/// [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) (default)
/// and [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight* : The name of the attribute of
/// the edges containing the length.
/// * *defaultWeight* : Only used with the option *weight*.
@ -6790,7 +6796,8 @@ function GENERAL_GRAPH_BETWEENNESS (graphName, options) {
/// * *direction* : The direction of the edges.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm* : The algorithm to calculate the shortest paths as a string. Possible
/// values are *Floyd-Warshall* (default) and *Dijkstra*.
/// values are [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// (default) and [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight* : The name of the attribute of
/// the edges containing the length.
/// * *defaultWeight* : Only used with the option *weight*.
@ -6878,7 +6885,8 @@ function GENERAL_GRAPH_RADIUS (graphName, options) {
/// * *direction* : The direction of the edges.
/// Possible values are *outbound*, *inbound* and *any* (default).
/// * *algorithm* : The algorithm to calculate the shortest paths as a string. Possible
/// values are *Floyd-Warshall* (default) and *Dijkstra*.
/// values are [Floyd-Warshall](http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm)
/// (default) and [Dijkstra](http://en.wikipedia.org/wiki/Dijkstra's_algorithm).
/// * *weight* : The name of the attribute of
/// the edges containing the length.
/// * *defaultWeight* : Only used with the option *weight*.

7
js/server/server.js
View File

@ -1,5 +1,5 @@
/*jslint indent: 2, nomen: true, maxlen: 120, sloppy: true, vars: true, white: true, plusplus: true */
/*global require */
/*global require, ENABLE_STATISTICS */
////////////////////////////////////////////////////////////////////////////////
/// @brief JavaScript server functions
@ -50,6 +50,11 @@ var Buffer = require("buffer").Buffer;
////////////////////////////////////////////////////////////////////////////////
(function () {
if (! ENABLE_STATISTICS) {
// statistics are turned off
return;
}
var internal = require("internal");
var interval = require('org/arangodb/statistics').STATISTICS_INTERVAL;
var interval15 = require('org/arangodb/statistics').STATISTICS_HISTORY_INTERVAL;

4
js/server/tests/ahuacatl-general-graph.js
View File

@ -560,7 +560,7 @@ function ahuacatlQueryGeneralPathsTestSuite() {
testPathsWithDirectionAnyAndMaxLength1: function () {
var actual, result = {}, i = 0, ed;
actual = getQueryResults("FOR e IN GRAPH_PATHS('bla3', 'any', false , 1 , 1) SORT e.source._key,e.destination._key RETURN [e.source._key,e.destination._key,e.edges]");
actual = getQueryResults("FOR e IN GRAPH_PATHS('bla3', {direction :'any', minLength : 1 , maxLength: 1}) SORT e.source._key,e.destination._key RETURN [e.source._key,e.destination._key,e.edges]");
actual.forEach(function (p) {
i++;
ed = "";
@ -589,7 +589,7 @@ function ahuacatlQueryGeneralPathsTestSuite() {
testInBoundPaths: function () {
var actual, result = {}, i = 0, ed;
actual = getQueryResults("FOR e IN GRAPH_PATHS('bla3', 'inbound', false, 1) SORT e.source._key,e.destination._key RETURN [e.source._key,e.destination._key,e.edges]");
actual = getQueryResults("FOR e IN GRAPH_PATHS('bla3', {direction : 'inbound', minLength : 1}) SORT e.source._key,e.destination._key RETURN [e.source._key,e.destination._key,e.edges]");
actual.forEach(function (p) {
i++;

69
js/server/tests/recovery/collector-oom.js Normal file
View File

@ -0,0 +1,69 @@
var db = require("org/arangodb").db;
var internal = require("internal");
var jsunity = require("jsunity");
function runSetup () {
internal.debugClearFailAt();
db._drop("UnitTestsRecovery");
var c = db._create("UnitTestsRecovery"), i;
for (i = 0; i < 10000; ++i) {
c.save({ _key: "test" + i, value1: "test" + i, value2: i });
}
internal.debugSetFailAt("CollectorThreadQueueOperations");
internal.flushWal(true, false);
internal.wait(5);
internal.debugSegfault("crashing server");
}
////////////////////////////////////////////////////////////////////////////////
/// @brief test suite
////////////////////////////////////////////////////////////////////////////////
function recoverySuite () {
jsunity.jsUnity.attachAssertions();
return {
setUp: function () {
},
tearDown: function () {
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test whether we can restore the data
////////////////////////////////////////////////////////////////////////////////
testCollectorOom : function () {
var i, c = db._collection("UnitTestsRecovery");
assertEqual(10000, c.count());
for (i = 0; i < 10000; ++i) {
var doc = c.document("test" + i);
assertEqual("test" + i, doc.value1);
assertEqual(i, doc.value2);
}
}
};
}
////////////////////////////////////////////////////////////////////////////////
/// @brief executes the test suite
////////////////////////////////////////////////////////////////////////////////
function main (argv) {
if (argv[1] === "setup") {
runSetup();
return 0;
}
else {
jsunity.run(recoverySuite);
return jsunity.done() ? 0 : 1;
}
}

61
js/server/tests/shell-wal-noncluster.js
View File

@ -30,6 +30,60 @@ var arangodb = require("org/arangodb");
var testHelper = require("org/arangodb/test-helper").Helper;
var db = arangodb.db;
var internal = require("internal");
// -----------------------------------------------------------------------------
// --SECTION-- wal failures
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
/// @brief test suite
////////////////////////////////////////////////////////////////////////////////
function walFailureSuite () {
var cn = "UnitTestsWal";
var c;
return {
setUp: function () {
internal.debugClearFailAt();
db._drop(cn);
c = db._create(cn);
},
tearDown: function () {
db._drop(cn);
c = null;
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test bad alloc in collector
////////////////////////////////////////////////////////////////////////////////
testCollectorBadAlloc : function () {
internal.flushWal(true, true);
internal.debugSetFailAt("CollectorThreadQueueOperations");
var i = 0;
for (i = 0; i < 1000; ++i) {
c.save({ _key: "test" + i });
}
assertEqual(1000, c.count());
internal.flushWal(true, false);
assertEqual(1000, c.count());
internal.wait(6);
internal.debugClearFailAt();
internal.wait(6);
testHelper.waitUnload(c);
assertEqual(1000, c.count());
}
};
}
// -----------------------------------------------------------------------------
// --SECTION-- wal functions
@ -39,7 +93,7 @@ var internal = require("internal");
/// @brief test suite
////////////////////////////////////////////////////////////////////////////////
function WalSuite () {
function walSuite () {
var cn = "UnitTestsWal";
var c;
@ -123,7 +177,10 @@ function WalSuite () {
/// @brief executes the test suites
////////////////////////////////////////////////////////////////////////////////
jsunity.run(WalSuite);
if (internal.debugCanUseFailAt()) {
jsunity.run(walFailureSuite);
}
jsunity.run(walSuite);
return jsunity.done();

1
lib/ApplicationServer/ApplicationServer.cpp
View File

@ -820,7 +820,6 @@ void ApplicationServer::setupOptions (map<string, ProgramOptionsDescription>& op
// .............................................................................
options[OPTIONS_SERVER + ":help-extended"]
("random.no-seed", "do not seed the random generator")
("random.generator", &_randomGenerator, "1 = mersenne, 2 = random, 3 = urandom, 4 = combined")
#ifdef TRI_HAVE_SETUID
("server.uid", &_uid, "switch to user-id after reading config files")

7
lib/ApplicationServer/ApplicationServer.h
View File

@ -755,13 +755,6 @@ namespace triagens {
/// number generator using an implication of the Mersenne Twister MT19937
/// algorithm. Algorithm 4 is a combination of the blocking random number
/// generator and the Mersenne Twister.
///
/// @CMDOPT{\--random.no-seed}
///
/// By default, the random generator is seeded. Setting this option causes the
/// random number generator not to be seeded. (Seeding the random number
/// generator only occurs if the generator is set to Mersenne, refer to
/// random.generator for details.)
////////////////////////////////////////////////////////////////////////////////
uint32_t _randomGenerator;

3
lib/Rest/SslInterface.cpp
View File

@ -190,6 +190,9 @@ namespace triagens {
if (algorithm == Algorithm::ALGORITHM_SHA1) {
evp_md = const_cast<EVP_MD*>(EVP_sha1());
}
else if (algorithm == Algorithm::ALGORITHM_SHA224) {
evp_md = const_cast<EVP_MD*>(EVP_sha224());
}
else if (algorithm == Algorithm::ALGORITHM_MD5) {
evp_md = const_cast<EVP_MD*>(EVP_md5());
}

3
lib/Rest/SslInterface.h
View File

@ -39,7 +39,8 @@ namespace triagens {
enum Algorithm {
ALGORITHM_SHA256 = 0,
ALGORITHM_SHA1 = 1,
ALGORITHM_MD5 = 2
ALGORITHM_MD5 = 2,
ALGORITHM_SHA224 = 3
};
//////////////////////////////////////////////////////////////////////////

85
lib/V8/v8-utils.cpp
View File

@ -2273,6 +2273,86 @@ static v8::Handle<v8::Value> JS_Sha256 (v8::Arguments const& argv) {
return scope.Close(hashStr);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief computes the sha224 sum
///
/// @FUN{internal.sha224(@FA{text})}
///
/// Computes an sha224 for the @FA{text}.
////////////////////////////////////////////////////////////////////////////////
static v8::Handle<v8::Value> JS_Sha224 (v8::Arguments const& argv) {
v8::HandleScope scope;
// extract arguments
if (argv.Length() != 1 || ! argv[0]->IsString()) {
TRI_V8_EXCEPTION_USAGE(scope, "sha224(<text>)");
}
string key = TRI_ObjectToString(argv[0]);
// create sha224
char* hash = 0;
size_t hashLen;
SslInterface::sslSHA224(key.c_str(), key.size(), hash, hashLen);
// as hex
char* hex = 0;
size_t hexLen;
SslInterface::sslHEX(hash, hashLen, hex, hexLen);
delete[] hash;
// and return
v8::Handle<v8::String> hashStr = v8::String::New(hex, (int) hexLen);
delete[] hex;
return scope.Close(hashStr);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief computes the sha1 sum
///
/// @FUN{internal.sha1(@FA{text})}
///
/// Computes an sha1 for the @FA{text}.
////////////////////////////////////////////////////////////////////////////////
static v8::Handle<v8::Value> JS_Sha1 (v8::Arguments const& argv) {
v8::HandleScope scope;
// extract arguments
if (argv.Length() != 1 || ! argv[0]->IsString()) {
TRI_V8_EXCEPTION_USAGE(scope, "sha1(<text>)");
}
string key = TRI_ObjectToString(argv[0]);
// create sha1
char* hash = 0;
size_t hashLen;
SslInterface::sslSHA1(key.c_str(), key.size(), hash, hashLen);
// as hex
char* hex = 0;
size_t hexLen;
SslInterface::sslHEX(hash, hashLen, hex, hexLen);
delete[] hash;
// and return
v8::Handle<v8::String> hashStr = v8::String::New(hex, (int) hexLen);
delete[] hex;
return scope.Close(hashStr);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief sleeps
///
@ -2565,6 +2645,9 @@ static v8::Handle<v8::Value> JS_HMAC (v8::Arguments const& argv) {
else if (algorithm == "sha256") {
al = SslInterface::Algorithm::ALGORITHM_SHA256;
}
else if (algorithm == "sha224") {
al = SslInterface::Algorithm::ALGORITHM_SHA224;
}
else if (algorithm == "md5") {
al = SslInterface::Algorithm::ALGORITHM_MD5;
}
@ -3391,6 +3474,8 @@ void TRI_InitV8Utils (v8::Handle<v8::Context> context,
TRI_AddGlobalFunctionVocbase(context, "SYS_SAVE", JS_Save);
TRI_AddGlobalFunctionVocbase(context, "SYS_SERVER_STATISTICS", JS_ServerStatistics);
TRI_AddGlobalFunctionVocbase(context, "SYS_SHA256", JS_Sha256);
TRI_AddGlobalFunctionVocbase(context, "SYS_SHA224", JS_Sha224);
TRI_AddGlobalFunctionVocbase(context, "SYS_SHA1", JS_Sha1);
TRI_AddGlobalFunctionVocbase(context, "SYS_SLEEP", JS_Sleep);
TRI_AddGlobalFunctionVocbase(context, "SYS_SPRINTF", JS_SPrintF);
TRI_AddGlobalFunctionVocbase(context, "SYS_STATUS_EXTERNAL", JS_StatusExternal);