Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Feature/aql traversal prune (#8255) 

Allow PRUNE in the AQL Traverser.
This commit is contained in:
Michael Hackstein 2019-02-27 11:43:06 +01:00 committed by GitHub
parent b04cd607f4
commit 6e8d43b2da
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
31 changed files with 3357 additions and 2229 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
CHANGELOG
View File

@ -1,6 +1,17 @@
devel devel
----- -----
* add "PRUNE <condition>" to AQL Traversals. This allows to early abort searching of
unnecessary branches within a traversal.
PRUNE is only allowed in the Traversal statement and only between the graphdefinition
and the options of the traversal.
e.g.:
FOR v, e, p IN 1..3 OUTBOUND @source GRAPH "myGraph"
PRUNE v.value == "bar"
OPTIONS {} /* These options remain optional */
RETURN v
for more details refer to the documentation chapter.
* fixed a display issues when editing a graph within the web ui * fixed a display issues when editing a graph within the web ui
* fixed some escaping issues within the web ui. * fixed some escaping issues within the web ui.

74
Documentation/Books/AQL/Graphs/Traversals.md
View File

@ -17,6 +17,7 @@ FOR vertex[, edge[, path]]
IN [min[..max]] IN [min[..max]]
OUTBOUND|INBOUND|ANY startVertex OUTBOUND|INBOUND|ANY startVertex
GRAPH graphName GRAPH graphName
[PRUNE pruneCondition]
[OPTIONS options] [OPTIONS options]
``` ```
- `WITH`: optional for single server instances, but required for - `WITH`: optional for single server instances, but required for
@ -48,6 +49,22 @@ FOR vertex[, edge[, path]]
- `GRAPH` **graphName** (string): the name identifying the named graph. - `GRAPH` **graphName** (string): the name identifying the named graph.
Its vertex and edge collections will be looked up. Note that the graph name Its vertex and edge collections will be looked up. Note that the graph name
is like a regular string, hence it must be enclosed by quote marks. is like a regular string, hence it must be enclosed by quote marks.
- `PRUNE` **condition** (AQL condition, *optional*, (since version 3.4.5)):
A condition, like in a FILTER statement, which will be evaluated in every step
of the traversal, as early as possible. The semantics of this condition is as follows:
* If the condition evaluates to `true` this path will be considered as a result,
it might still be post filtered or ignored due to depth constraints. However
the search will not continue from this path, namely there will be no
result having this path as a prefix.
e.g.: Take the path: `(A) -> (B) -> (C)` starting at `A` and PRUNE on `B`
will result in `(A)` and `(A) -> (B)` beeing valid paths, and `(A) -> (B) -> (C)`
not returned, it got pruned on B.
* If the condition evaluates to `false` we will continue our search beyond
this path.
There is only one `PRUNE` condition possible, but it can contain an arbitrary amount
of `AND` or `OR` statements.
Also note that you can use the output variables of this traversal in the `PRUNE`,
as well as all variables defined before this Traversal statement.
- `OPTIONS` **options** (object, *optional*): used to modify the execution of the - `OPTIONS` **options** (object, *optional*): used to modify the execution of the
traversal. Only the following attributes have an effect, all others are ignored: traversal. Only the following attributes have an effect, all others are ignored:
- **uniqueVertices** (string): optionally ensure vertex uniqueness - **uniqueVertices** (string): optionally ensure vertex uniqueness
@ -79,6 +96,7 @@ FOR vertex[, edge[, path]]
IN [min[..max]] IN [min[..max]]
OUTBOUND|INBOUND|ANY startVertex OUTBOUND|INBOUND|ANY startVertex
edgeCollection1, ..., edgeCollectionN edgeCollection1, ..., edgeCollectionN
[PRUNE pruneCondition]
[OPTIONS options] [OPTIONS options]
``` ```
@ -138,10 +156,62 @@ combined filters cannot.
The following examples are based on the [traversal graph](../../Manual/Graphs/index.html#the-traversal-graph). The following examples are based on the [traversal graph](../../Manual/Graphs/index.html#the-traversal-graph).
### Pruning (since version 3.4.5)
Pruning is the easiest variant to formulate conditions to reduce the amount of data
to be checked during a search. So it allows to improve query performance and reduces
the amount of overhead generated by the query. Pruning can be executed on the
vertex, the edge and the path and any variable defined before.
See examples:
@startDocuBlockInline GRAPHTRAV_graphPruneEdges
@EXAMPLE_AQL{GRAPHTRAV_graphFilterEdges}
@DATASET{traversalGraph}
FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'
PRUNE e.theTruth == true
RETURN { vertices: p.vertices[*]._key, edges: p.edges[*].label }
@END_EXAMPLE_AQL
@endDocuBlock GRAPHTRAV_graphPruneEdges
This will search until it sees an edge having `theTruth == true`.
The path with this edge will be returned, the search will not
continue after this edge.
Namely all responses either have no edge with `theTruth == true`
or the last edge on the path has `theTruth == true`.
@startDocuBlockInline GRAPHTRAV_graphPruneVertices
@EXAMPLE_AQL{GRAPHTRAV_graphFilterEdges}
@DATASET{traversalGraph}
FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'
PRUNE v._key == 'G'
FILTER v._key == 'G'
RETURN { vertices: p.vertices[*]._key, edges: p.edges[*].label }
@END_EXAMPLE_AQL
@endDocuBlock GRAPHTRAV_graphPruneVertices
This will search for all paths from the source `circles/A` to the vertex `circles/G`.
This is done with first the PRUNE which makes sure we stop search as soon as we have found
`G` and we will not go beyond `G` and via a loop return to it.
With the second filter, we remove all paths that do not end in `G` namely
all shorter ones that have not been cut out by prune.
Hence the list of all paths from `A` to `G` are returned.
Note you can also prune as soon as you reach a certain collection with the following
example:
@startDocuBlockInline GRAPHTRAV_graphPruneCollection
@EXAMPLE_AQL{GRAPHTRAV_graphFilterEdges}
@DATASET{traversalGraph}
FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'
PRUNE IS_SAME_COLLECTION('circles', v)
RETURN { vertices: p.vertices[*]._key, edges: p.edges[*].label }
@END_EXAMPLE_AQL
@endDocuBlock GRAPHTRAV_graphPruneCollection
### Filtering on paths ### Filtering on paths
Filtering on paths allows for the most powerful filtering and may have the Filtering on paths allows for the second most powerful filtering and may have the
highest impact on performance. Using the path variable you can filter on second highest impact on performance. Using the path variable you can filter on
specific iteration depths. You can filter for absolute positions in the path specific iteration depths. You can filter for absolute positions in the path
by specifying a positive number (which then qualifies for the optimizations), by specifying a positive number (which then qualifies for the optimizations),
or relative positions to the end of the path by specifying a negative number. or relative positions to the end of the path by specifying a negative number.

140
arangod/Aql/Ast.cpp
View File

@ -1304,129 +1304,65 @@ AstNode* Ast::createNodeCollectionDirection(uint64_t direction, AstNode const* c
return node; return node;
} }
/// @brief create an AST traversal node with only vertex variable /// @brief create an AST traversal node
AstNode* Ast::createNodeTraversal(char const* vertexVarName, size_t vertexVarLength, AstNode* Ast::createNodeTraversal(AstNode const* outVars, AstNode const* graphInfo) {
AstNode const* direction, AstNode const* start, TRI_ASSERT(outVars->type == NODE_TYPE_ARRAY);
AstNode const* graph, AstNode const* options) { TRI_ASSERT(graphInfo->type == NODE_TYPE_ARRAY);
if (vertexVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
AstNode* node = createNode(NODE_TYPE_TRAVERSAL); AstNode* node = createNode(NODE_TYPE_TRAVERSAL);
node->reserve(5); node->reserve(outVars->numMembers() + graphInfo->numMembers());
if (options == nullptr) { TRI_ASSERT(graphInfo->numMembers() == 5);
// no options given. now use default options TRI_ASSERT(outVars->numMembers() > 0);
options = &NopNode; TRI_ASSERT(outVars->numMembers() < 4);
// Add GraphInfo
for (size_t i = 0; i < graphInfo->numMembers(); ++i) {
node->addMember(graphInfo->getMemberUnchecked(i));
} }
node->addMember(direction); // Add variables
node->addMember(start); for (size_t i = 0; i < outVars->numMembers(); ++i) {
node->addMember(graph); node->addMember(outVars->getMemberUnchecked(i));
node->addMember(options); }
TRI_ASSERT(node->numMembers() == graphInfo->numMembers() + outVars->numMembers());
AstNode* vertexVar = createNodeVariable(vertexVarName, vertexVarLength, false);
node->addMember(vertexVar);
TRI_ASSERT(node->numMembers() == 5);
_containsTraversal = true; _containsTraversal = true;
return node; return node;
} }
/// @brief create an AST traversal node with vertex and edge variable /// @brief create an AST shortest path node
AstNode* Ast::createNodeTraversal(char const* vertexVarName, size_t vertexVarLength, AstNode* Ast::createNodeShortestPath(AstNode const* outVars, AstNode const* graphInfo) {
char const* edgeVarName, size_t edgeVarLength, TRI_ASSERT(outVars->type == NODE_TYPE_ARRAY);
AstNode const* direction, AstNode const* start, TRI_ASSERT(graphInfo->type == NODE_TYPE_ARRAY);
AstNode const* graph, AstNode const* options) {
if (edgeVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
AstNode* node = createNodeTraversal(vertexVarName, vertexVarLength, direction,
start, graph, options);
AstNode* edgeVar = createNodeVariable(edgeVarName, edgeVarLength, false);
node->addMember(edgeVar);
TRI_ASSERT(node->numMembers() == 6);
_containsTraversal = true;
return node;
}
/// @brief create an AST traversal node with vertex, edge and path variable
AstNode* Ast::createNodeTraversal(char const* vertexVarName, size_t vertexVarLength,
char const* edgeVarName, size_t edgeVarLength,
char const* pathVarName, size_t pathVarLength,
AstNode const* direction, AstNode const* start,
AstNode const* graph, AstNode const* options) {
if (pathVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
AstNode* node = createNodeTraversal(vertexVarName, vertexVarLength, edgeVarName,
edgeVarLength, direction, start, graph, options);
AstNode* pathVar = createNodeVariable(pathVarName, pathVarLength, false);
node->addMember(pathVar);
TRI_ASSERT(node->numMembers() == 7);
_containsTraversal = true;
return node;
}
/// @brief create an AST shortest path node with only vertex variable
AstNode* Ast::createNodeShortestPath(char const* vertexVarName,
size_t vertexVarLength, uint64_t direction,
AstNode const* start, AstNode const* target,
AstNode const* graph, AstNode const* options) {
if (vertexVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
AstNode* node = createNode(NODE_TYPE_SHORTEST_PATH); AstNode* node = createNode(NODE_TYPE_SHORTEST_PATH);
node->reserve(outVars->numMembers() + graphInfo->numMembers());
node->reserve(6); TRI_ASSERT(graphInfo->numMembers() == 5);
TRI_ASSERT(outVars->numMembers() > 0);
TRI_ASSERT(outVars->numMembers() < 3);
if (options == nullptr) { // Add GraphInfo
// no options given. now use default options for (size_t i = 0; i < graphInfo->numMembers(); ++i) {
options = &NopNode; node->addMember(graphInfo->getMemberUnchecked(i));
} }
AstNode* dir = createNodeValueInt(direction);
node->addMember(dir);
node->addMember(start);
node->addMember(target);
node->addMember(graph);
node->addMember(options);
AstNode* vertexVar = createNodeVariable(vertexVarName, vertexVarLength, false); // Add variables
node->addMember(vertexVar); for (size_t i = 0; i < outVars->numMembers(); ++i) {
node->addMember(outVars->getMemberUnchecked(i));
}
TRI_ASSERT(node->numMembers() == graphInfo->numMembers() + outVars->numMembers());
TRI_ASSERT(node->numMembers() == 6); _containsTraversal = true;
return node; return node;
} }
/// @brief create an AST shortest path node with vertex and edge variable AstNode const* Ast::createNodeOptions(AstNode const* options) const {
AstNode* Ast::createNodeShortestPath(char const* vertexVarName, if (options != nullptr) {
size_t vertexVarLength, char const* edgeVarName, return options;
size_t edgeVarLength, uint64_t direction,
AstNode const* start, AstNode const* target,
AstNode const* graph, AstNode const* options) {
if (edgeVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
} }
return &NopNode;
AstNode* node = createNodeShortestPath(vertexVarName, vertexVarLength, direction,
start, target, graph, options);
AstNode* edgeVar = createNodeVariable(edgeVarName, edgeVarLength, false);
node->addMember(edgeVar);
TRI_ASSERT(node->numMembers() == 7);
return node;
} }
/// @brief create an AST function call node /// @brief create an AST function call node

27
arangod/Aql/Ast.h
View File

@ -355,27 +355,16 @@ class Ast {
/// @brief create an AST direction node /// @brief create an AST direction node
AstNode* createNodeCollectionDirection(uint64_t, AstNode const*); AstNode* createNodeCollectionDirection(uint64_t, AstNode const*);
/// @brief create an AST traversal node with only vertex variable /// @brief create an AST options node:
AstNode* createNodeTraversal(char const*, size_t, AstNode const*, // Will either return Noop noed, if the input is nullptr
AstNode const*, AstNode const*, AstNode const*); // Otherwise return the input node.
AstNode const* createNodeOptions(AstNode const*) const;
/// @brief create an AST traversal node with vertex and edge variable /// @brief create an AST traversal node
AstNode* createNodeTraversal(char const*, size_t, char const*, size_t, AstNode const*, AstNode* createNodeTraversal(AstNode const*, AstNode const*);
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST traversal node with vertex, edge and path variable /// @brief create an AST shortest path node
AstNode* createNodeTraversal(char const*, size_t, char const*, size_t, AstNode* createNodeShortestPath(AstNode const*, AstNode const*);
char const*, size_t, AstNode const*,
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST shortest path node with only vertex variable
AstNode* createNodeShortestPath(char const*, size_t, uint64_t, AstNode const*,
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST shortest path node with vertex and edge variable
AstNode* createNodeShortestPath(char const*, size_t, char const*, size_t,
uint64_t, AstNode const*, AstNode const*,
AstNode const*, AstNode const*);
/// @brief create an AST function call node /// @brief create an AST function call node
AstNode* createNodeFunctionCall(char const* functionName, AstNode const* arguments) { AstNode* createNodeFunctionCall(char const* functionName, AstNode const* arguments) {

33
arangod/Aql/ExecutionPlan.cpp
View File

@ -206,6 +206,13 @@ std::unique_ptr<graph::BaseOptions> createShortestPathOptions(arangodb::aql::Que
return ret; return ret;
} }
std::unique_ptr<Expression> createPruneExpression(ExecutionPlan* plan, Ast* ast, AstNode* node) {
if (node->type == NODE_TYPE_NOP) {
return nullptr;
}
return std::make_unique<Expression>(plan, ast, node);
}
} // namespace } // namespace
/// @brief create the plan /// @brief create the plan
@ -980,11 +987,11 @@ ExecutionNode* ExecutionPlan::fromNodeForView(ExecutionNode* previous, AstNode c
/// @brief create an execution plan element from an AST FOR TRAVERSAL node /// @brief create an execution plan element from an AST FOR TRAVERSAL node
ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode const* node) { ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode const* node) {
TRI_ASSERT(node != nullptr && node->type == NODE_TYPE_TRAVERSAL); TRI_ASSERT(node != nullptr && node->type == NODE_TYPE_TRAVERSAL);
TRI_ASSERT(node->numMembers() >= 5); TRI_ASSERT(node->numMembers() >= 6);
TRI_ASSERT(node->numMembers() <= 7); TRI_ASSERT(node->numMembers() <= 8);
// the first 3 members are used by traversal internally. // the first 5 members are used by traversal internally.
// The members 4-6, where 5 and 6 are optional, are used // The members 6-8, where 5 and 6 are optional, are used
// as out variables. // as out variables.
AstNode const* direction = node->getMember(0); AstNode const* direction = node->getMember(0);
AstNode const* start = node->getMember(1); AstNode const* start = node->getMember(1);
@ -1009,8 +1016,11 @@ ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode
previous = calc; previous = calc;
} }
// Prune Expression
std::unique_ptr<Expression> pruneExpression = createPruneExpression(this, _ast, node->getMember(3));
auto options = auto options =
createTraversalOptions(getAst()->query(), direction, node->getMember(3)); createTraversalOptions(getAst()->query(), direction, node->getMember(4));
TRI_ASSERT(direction->type == NODE_TYPE_DIRECTION); TRI_ASSERT(direction->type == NODE_TYPE_DIRECTION);
TRI_ASSERT(direction->numMembers() == 2); TRI_ASSERT(direction->numMembers() == 2);
@ -1019,24 +1029,25 @@ ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode
// First create the node // First create the node
auto travNode = new TraversalNode(this, nextId(), &(_ast->query()->vocbase()), auto travNode = new TraversalNode(this, nextId(), &(_ast->query()->vocbase()),
direction, start, graph, std::move(options)); direction, start, graph,
std::move(pruneExpression), std::move(options));
auto variable = node->getMember(4); auto variable = node->getMember(5);
TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE); TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE);
auto v = static_cast<Variable*>(variable->getData()); auto v = static_cast<Variable*>(variable->getData());
TRI_ASSERT(v != nullptr); TRI_ASSERT(v != nullptr);
travNode->setVertexOutput(v); travNode->setVertexOutput(v);
if (node->numMembers() > 5) { if (node->numMembers() > 6) {
// return the edge as well // return the edge as well
variable = node->getMember(5); variable = node->getMember(6);
TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE); TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE);
v = static_cast<Variable*>(variable->getData()); v = static_cast<Variable*>(variable->getData());
TRI_ASSERT(v != nullptr); TRI_ASSERT(v != nullptr);
travNode->setEdgeOutput(v); travNode->setEdgeOutput(v);
if (node->numMembers() > 6) { if (node->numMembers() > 7) {
// return the path as well // return the path as well
variable = node->getMember(6); variable = node->getMember(7);
TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE); TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE);
v = static_cast<Variable*>(variable->getData()); v = static_cast<Variable*>(variable->getData());
TRI_ASSERT(v != nullptr); TRI_ASSERT(v != nullptr);

123
arangod/Aql/InAndOutRowExpressionContext.cpp Normal file
View File

@ -0,0 +1,123 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#include "InAndOutRowExpressionContext.h"
#include "Aql/AqlItemBlock.h"
#include "Aql/AqlValue.h"
#include "Aql/Variable.h"
#include "Basics/Exceptions.h"
#include <limits.h>
using namespace arangodb;
using namespace arangodb::aql;
InAndOutRowExpressionContext::InAndOutRowExpressionContext(
Query* query, std::vector<Variable const*> const&& vars,
std::vector<RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx)
: QueryExpressionContext(query),
_input{CreateInvalidInputRowHint()},
_vars(std::move(vars)),
_regs(std::move(regs)),
_vertexVarIdx(vertexVarIdx),
_edgeVarIdx(edgeVarIdx),
_pathVarIdx(pathVarIdx) {
TRI_ASSERT(_vars.size() == _regs.size());
TRI_ASSERT(_vertexVarIdx < _vars.size() ||
_vertexVarIdx == std::numeric_limits<std::size_t>::max());
TRI_ASSERT(_edgeVarIdx < _vars.size() ||
_edgeVarIdx == std::numeric_limits<std::size_t>::max());
TRI_ASSERT(_pathVarIdx < _vars.size() ||
_pathVarIdx == std::numeric_limits<std::size_t>::max());
}
void InAndOutRowExpressionContext::setInputRow(InputAqlItemRow input) {
TRI_ASSERT(input.isInitialized());
_input = input;
}
AqlValue const& InAndOutRowExpressionContext::getRegisterValue(size_t i) const {
TRI_ASSERT(_input.isInitialized());
TRI_ASSERT(i < _regs.size());
if (i == _vertexVarIdx) {
return _vertexValue;
}
if (i == _edgeVarIdx) {
return _edgeValue;
}
if (i == _pathVarIdx) {
return _pathValue;
}
// Search InputRow
RegisterId const& regId = _regs[i];
TRI_ASSERT(regId < _input.getNrRegisters());
return _input.getValue(regId);
}
AqlValue InAndOutRowExpressionContext::getVariableValue(Variable const* variable, bool doCopy,
bool& mustDestroy) const {
TRI_ASSERT(_input.isInitialized());
for (size_t i = 0; i < _vars.size(); ++i) {
auto const& v = _vars[i];
if (v->id == variable->id) {
TRI_ASSERT(i < _regs.size());
if (doCopy) {
mustDestroy = true;
if (i == _vertexVarIdx) {
return _vertexValue.clone();
}
if (i == _edgeVarIdx) {
return _edgeValue.clone();
}
if (i == _pathVarIdx) {
return _pathValue.clone();
}
// Search InputRow
RegisterId const& regId = _regs[i];
TRI_ASSERT(regId < _input.getNrRegisters());
return _input.getValue(regId).clone();
} else {
mustDestroy = false;
if (i == _vertexVarIdx) {
return _vertexValue;
}
if (i == _edgeVarIdx) {
return _edgeValue;
}
if (i == _pathVarIdx) {
return _pathValue;
}
// Search InputRow
RegisterId const& regId = _regs[i];
TRI_ASSERT(regId < _input.getNrRegisters());
return _input.getValue(regId);
}
}
}
std::string msg("variable not found '");
msg.append(variable->name);
// NOTE: PRUNE is the only feature using this context.
msg.append("' in PRUNE statement");
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_INTERNAL, msg.c_str());
}

99
arangod/Aql/InAndOutRowExpressionContext.h Normal file
View File

@ -0,0 +1,99 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#ifndef ARANGOD_AQL_IN_AND_OUT_ROW_EXPRESSION_CONTEXT_H
#define ARANGOD_AQL_IN_AND_OUT_ROW_EXPRESSION_CONTEXT_H 1
#include "QueryExpressionContext.h"
#include "Aql/AqlValue.h"
#include "Aql/InputAqlItemRow.h"
namespace arangodb {
namespace aql {
class Query;
/**
* @brief Context for expression evaluation that allows
* to read registers from Input and Output.
* This is useful for expressions that need
* to evaluate variables created by the active block.
* User needs to make sure that outputblock is written to!
*/
class InAndOutRowExpressionContext final : public QueryExpressionContext {
public:
InAndOutRowExpressionContext(Query* query, std::vector<Variable const*> const&& vars,
std::vector<RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx);
~InAndOutRowExpressionContext() {}
void setInputRow(InputAqlItemRow input);
size_t numRegisters() const override { return _regs.size(); }
AqlValue const& getRegisterValue(size_t i) const override;
Variable const* getVariable(size_t i) const override { return _vars[i]; }
AqlValue getVariableValue(Variable const* variable, bool doCopy,
bool& mustDestroy) const override;
bool needsVertexValue() const { return _vertexVarIdx < _regs.size(); }
bool needsEdgeValue() const { return _edgeVarIdx < _regs.size(); }
bool needsPathValue() const { return _pathVarIdx < _regs.size(); }
/// @brief inject the result value when asked for the Vertex data
/// This will not copy ownership of slice content. caller needs to make sure
/// that the buffer stays valid until evaluate is called
void setVertexValue(velocypack::Slice v) {
_vertexValue = AqlValue(AqlValueHintDocumentNoCopy(v.begin()));
}
/// @brief inject the result value when asked for the Edge data
/// This will not copy ownership of slice content. caller needs to make sure
/// that the buffer stays valid until evaluate is called
void setEdgeValue(velocypack::Slice e) {
_edgeValue = AqlValue(AqlValueHintDocumentNoCopy(e.begin()));
}
/// @brief inject the result value when asked for the Path data
/// This will not copy ownership of slice content. caller needs to make sure
/// that the buffer stays valid until evaluate is called
void setPathValue(velocypack::Slice p) {
_pathValue = AqlValue(AqlValueHintDocumentNoCopy(p.begin()));
}
private:
InputAqlItemRow _input;
std::vector<Variable const*> const _vars;
std::vector<RegisterId> const _regs;
size_t const _vertexVarIdx;
size_t const _edgeVarIdx;
size_t const _pathVarIdx;
AqlValue _vertexValue;
AqlValue _edgeValue;
AqlValue _pathValue;
};
} // namespace aql
} // namespace arangodb
#endif

16
arangod/Aql/OptimizerRules.cpp
View File

@ -22,6 +22,7 @@
/// @author Jan Steemann /// @author Jan Steemann
//////////////////////////////////////////////////////////////////////////////// ////////////////////////////////////////////////////////////////////////////////
#include "OptimizerRules.h"
#include "Aql/AqlItemBlock.h" #include "Aql/AqlItemBlock.h"
#include "Aql/ClusterNodes.h" #include "Aql/ClusterNodes.h"
#include "Aql/CollectNode.h" #include "Aql/CollectNode.h"
@ -54,7 +55,6 @@
#include "GeoIndex/Index.h" #include "GeoIndex/Index.h"
#include "Graph/TraverserOptions.h" #include "Graph/TraverserOptions.h"
#include "Indexes/Index.h" #include "Indexes/Index.h"
#include "OptimizerRules.h"
#include "StorageEngine/EngineSelectorFeature.h" #include "StorageEngine/EngineSelectorFeature.h"
#include "StorageEngine/StorageEngine.h" #include "StorageEngine/StorageEngine.h"
#include "Transaction/Methods.h" #include "Transaction/Methods.h"
@ -5884,14 +5884,19 @@ void arangodb::aql::optimizeTraversalsRule(Optimizer* opt,
// yet, as many traversal internals depend on the number of vertices // yet, as many traversal internals depend on the number of vertices
// found/built // found/built
auto outVariable = traversal->edgeOutVariable(); auto outVariable = traversal->edgeOutVariable();
if (outVariable != nullptr && !n->isVarUsedLater(outVariable)) { std::vector<Variable const*> pruneVars;
traversal->getPruneVariables(pruneVars);
if (outVariable != nullptr && !n->isVarUsedLater(outVariable) &&
std::find(pruneVars.begin(), pruneVars.end(), outVariable) == pruneVars.end()) {
// traversal edge outVariable not used later // traversal edge outVariable not used later
traversal->setEdgeOutput(nullptr); traversal->setEdgeOutput(nullptr);
modified = true; modified = true;
} }
outVariable = traversal->pathOutVariable(); outVariable = traversal->pathOutVariable();
if (outVariable != nullptr && !n->isVarUsedLater(outVariable)) { if (outVariable != nullptr && !n->isVarUsedLater(outVariable) &&
std::find(pruneVars.begin(), pruneVars.end(), outVariable) == pruneVars.end()) {
// traversal path outVariable not used later // traversal path outVariable not used later
traversal->setPathOutput(nullptr); traversal->setPathOutput(nullptr);
modified = true; modified = true;
@ -6038,8 +6043,11 @@ void arangodb::aql::removeTraversalPathVariable(Optimizer* opt,
for (auto const& n : tNodes) { for (auto const& n : tNodes) {
TraversalNode* traversal = ExecutionNode::castTo<TraversalNode*>(n); TraversalNode* traversal = ExecutionNode::castTo<TraversalNode*>(n);
std::vector<Variable const*> pruneVars;
traversal->getPruneVariables(pruneVars);
auto outVariable = traversal->pathOutVariable(); auto outVariable = traversal->pathOutVariable();
if (outVariable != nullptr && !n->isVarUsedLater(outVariable)) { if (outVariable != nullptr && !n->isVarUsedLater(outVariable) &&
std::find(pruneVars.begin(), pruneVars.end(), outVariable) == pruneVars.end()) {
// traversal path outVariable not used later // traversal path outVariable not used later
traversal->setPathOutput(nullptr); traversal->setPathOutput(nullptr);
modified = true; modified = true;

48
arangod/Aql/PruneExpressionEvaluator.cpp Normal file
View File

@ -0,0 +1,48 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#include "PruneExpressionEvaluator.h"
#include "Aql/AqlValue.h"
#include "Aql/Expression.h"
#include "Transaction/Methods.h"
using namespace arangodb;
using namespace arangodb::aql;
PruneExpressionEvaluator::PruneExpressionEvaluator(
transaction::Methods* trx, Query* query,
std::vector<Variable const*> const&& vars, std::vector<RegisterId> const&& regs,
size_t vertexVarIdx, size_t edgeVarIdx, size_t pathVarIdx, Expression* expr)
: _trx(trx),
_pruneExpression(expr),
_ctx(query, std::move(vars), std::move(regs), vertexVarIdx, edgeVarIdx, pathVarIdx) {}
PruneExpressionEvaluator::~PruneExpressionEvaluator() = default;
bool PruneExpressionEvaluator::evaluate() {
bool mustDestroy = false;
aql::AqlValue res = _pruneExpression->execute(_trx, &_ctx, mustDestroy);
arangodb::aql::AqlValueGuard guard(res, mustDestroy);
TRI_ASSERT(res.isBoolean());
return res.toBoolean();
}

74
arangod/Aql/PruneExpressionEvaluator.h Normal file
View File

@ -0,0 +1,74 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#ifndef ARANGOD_AQL_PRUNE_EXPRESSION_EVALUATOR_H
#define ARANGOD_AQL_PRUNE_EXPRESSION_EVALUATOR_H 1
#include "Aql/InAndOutRowExpressionContext.h"
namespace arangodb {
namespace velocypack {
class Slice;
}
namespace transaction {
class Methods;
}
namespace aql {
class Expression;
class Query;
class InputAqlItemRow;
class PruneExpressionEvaluator {
public:
PruneExpressionEvaluator(transaction::Methods* trx, Query* query,
std::vector<Variable const*> const&& vars,
std::vector<RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx, Expression* expr);
~PruneExpressionEvaluator();
bool evaluate();
void prepareContext(InputAqlItemRow input) { _ctx.setInputRow(input); }
bool needsVertex() const { return _ctx.needsVertexValue(); }
void injectVertex(velocypack::Slice v) { _ctx.setVertexValue(v); }
bool needsEdge() const { return _ctx.needsEdgeValue(); }
void injectEdge(velocypack::Slice e) { _ctx.setEdgeValue(e); }
bool needsPath() const { return _ctx.needsPathValue(); }
void injectPath(velocypack::Slice p) { _ctx.setPathValue(p); }
private:
transaction::Methods* _trx;
/// @brief The condition given in PRUNE (might be empty)
/// The Node keeps responsibility
aql::Expression* _pruneExpression;
/// @brief The context used to inject variables
InAndOutRowExpressionContext _ctx;
};
} // namespace aql
} // namespace arangodb
#endif

8
arangod/Aql/TraversalExecutor.cpp
View File

@ -22,6 +22,7 @@
#include "TraversalExecutor.h" #include "TraversalExecutor.h"
#include "Aql/OutputAqlItemRow.h" #include "Aql/OutputAqlItemRow.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Aql/Query.h" #include "Aql/Query.h"
#include "Aql/SingleRowFetcher.h" #include "Aql/SingleRowFetcher.h"
#include "Graph/Traverser.h" #include "Graph/Traverser.h"
@ -149,7 +150,6 @@ std::pair<ExecutionState, TraversalStats> TraversalExecutor::produceRow(OutputAq
s.addScannedIndex(_traverser.getAndResetReadDocuments()); s.addScannedIndex(_traverser.getAndResetReadDocuments());
return {_rowState, s}; return {_rowState, s};
} }
if (!resetTraverser()) { if (!resetTraverser()) {
// Could not start here, (invalid) // Could not start here, (invalid)
// Go to next // Go to next
@ -203,7 +203,11 @@ bool TraversalExecutor::resetTraverser() {
for (auto const& pair : _infos.filterConditionVariables()) { for (auto const& pair : _infos.filterConditionVariables()) {
opts->setVariableValue(pair.first, _input.getValue(pair.second)); opts->setVariableValue(pair.first, _input.getValue(pair.second));
} }
if (opts->usesPrune()) {
auto* evaluator = opts->getPruneEvaluator();
// Replace by inputRow
evaluator->prepareContext(_input);
}
// Now reset the traverser // Now reset the traverser
if (_infos.usesFixedSource()) { if (_infos.usesFixedSource()) {
auto pos = _infos.getFixedSource().find('/'); auto pos = _infos.getFixedSource().find('/');

67
arangod/Aql/TraversalNode.cpp
View File

@ -91,13 +91,15 @@ void TraversalNode::TraversalEdgeConditionBuilder::toVelocyPack(VPackBuilder& bu
TraversalNode::TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocbase, TraversalNode::TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocbase,
AstNode const* direction, AstNode const* start, AstNode const* direction, AstNode const* start,
AstNode const* graph, std::unique_ptr<BaseOptions> options) AstNode const* graph, std::unique_ptr<Expression> pruneExpression,
std::unique_ptr<BaseOptions> options)
: GraphNode(plan, id, vocbase, direction, graph, std::move(options)), : GraphNode(plan, id, vocbase, direction, graph, std::move(options)),
_pathOutVariable(nullptr), _pathOutVariable(nullptr),
_inVariable(nullptr), _inVariable(nullptr),
_condition(nullptr), _condition(nullptr),
_fromCondition(nullptr), _fromCondition(nullptr),
_toCondition(nullptr) { _toCondition(nullptr),
_pruneExpression(std::move(pruneExpression)) {
TRI_ASSERT(start != nullptr); TRI_ASSERT(start != nullptr);
auto ast = _plan->getAst(); auto ast = _plan->getAst();
@ -143,6 +145,10 @@ TraversalNode::TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocb
"_id string or an object with _id."); "_id string or an object with _id.");
} }
if (_pruneExpression) {
_pruneExpression->variables(_pruneVariables);
}
#ifdef TRI_ENABLE_MAINTAINER_MODE #ifdef TRI_ENABLE_MAINTAINER_MODE
checkConditionsDefined(); checkConditionsDefined();
#endif #endif
@ -247,6 +253,17 @@ TraversalNode::TraversalNode(ExecutionPlan* plan, arangodb::velocypack::Slice co
} }
} }
list = base.get("expression");
if (!list.isNone()) {
_pruneExpression = std::make_unique<aql::Expression>(plan, plan->getAst(), base);
TRI_ASSERT(base.hasKey("pruneVariables"));
list = base.get("pruneVariables");
TRI_ASSERT(list.isArray());
for (auto const& varinfo : VPackArrayIterator(list)) {
_pruneVariables.emplace(plan->getAst()->variables()->createVariable(varinfo));
}
}
#ifdef TRI_ENABLE_MAINTAINER_MODE #ifdef TRI_ENABLE_MAINTAINER_MODE
checkConditionsDefined(); checkConditionsDefined();
#endif #endif
@ -379,6 +396,17 @@ void TraversalNode::toVelocyPackHelper(VPackBuilder& nodes, unsigned flags) cons
nodes.add(VPackValue("indexes")); nodes.add(VPackValue("indexes"));
_options->toVelocyPackIndexes(nodes); _options->toVelocyPackIndexes(nodes);
if (_pruneExpression != nullptr) {
// The Expression constructor expects only this name
nodes.add(VPackValue("expression"));
_pruneExpression->toVelocyPack(nodes, flags);
nodes.add(VPackValue("pruneVariables"));
nodes.openArray();
for (auto const& var : _pruneVariables) {
var->toVelocyPack(nodes);
}
nodes.close();
}
// And close it: // And close it:
nodes.close(); nodes.close();
} }
@ -428,6 +456,33 @@ std::unique_ptr<ExecutionBlock> TraversalNode::createBlock(
auto opts = static_cast<TraverserOptions*>(options()); auto opts = static_cast<TraverserOptions*>(options());
std::unique_ptr<Traverser> traverser; std::unique_ptr<Traverser> traverser;
auto trx = engine.getQuery()->trx(); auto trx = engine.getQuery()->trx();
if (pruneExpression() != nullptr) {
std::vector<Variable const*> pruneVars;
getPruneVariables(pruneVars);
std::vector<RegisterId> pruneRegs;
// Create List for _pruneVars
pruneRegs.reserve(pruneVars.size());
size_t vertexRegIdx = std::numeric_limits<std::size_t>::max();
size_t edgeRegIdx = std::numeric_limits<std::size_t>::max();
size_t pathRegIdx = std::numeric_limits<std::size_t>::max();
for (auto const v : pruneVars) {
auto it = varInfo.find(v->id);
TRI_ASSERT(it != varInfo.end());
if (v == vertexOutVariable()) {
vertexRegIdx = pruneRegs.size();
} else if (v == edgeOutVariable()) {
edgeRegIdx = pruneRegs.size();
} else if (v == pathOutVariable()) {
pathRegIdx = pruneRegs.size();
}
pruneRegs.emplace_back(it->second.registerId);
}
opts->activatePrune(std::move(pruneVars), std::move(pruneRegs),
vertexRegIdx, edgeRegIdx, pathRegIdx, pruneExpression());
}
if (arangodb::ServerState::instance()->isCoordinator()) { if (arangodb::ServerState::instance()->isCoordinator()) {
#ifdef USE_ENTERPRISE #ifdef USE_ENTERPRISE
if (isSmart()) { if (isSmart()) {
@ -699,6 +754,14 @@ void TraversalNode::getConditionVariables(std::vector<Variable const*>& res) con
} }
} }
void TraversalNode::getPruneVariables(std::vector<Variable const*>& res) const {
if (_pruneExpression) {
for (auto const& it : _pruneVariables) {
res.emplace_back(it);
}
}
}
#ifdef TRI_ENABLE_MAINTAINER_MODE #ifdef TRI_ENABLE_MAINTAINER_MODE
void TraversalNode::checkConditionsDefined() const { void TraversalNode::checkConditionsDefined() const {
TRI_ASSERT(_tmpObjVariable != nullptr); TRI_ASSERT(_tmpObjVariable != nullptr);

18
arangod/Aql/TraversalNode.h
View File

@ -78,7 +78,8 @@ class TraversalNode : public GraphNode {
public: public:
TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocbase, TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocbase,
AstNode const* direction, AstNode const* start, AstNode const* direction, AstNode const* start,
AstNode const* graph, std::unique_ptr<graph::BaseOptions> options); AstNode const* graph, std::unique_ptr<Expression> pruneExpression,
std::unique_ptr<graph::BaseOptions> options);
TraversalNode(ExecutionPlan* plan, arangodb::velocypack::Slice const& base); TraversalNode(ExecutionPlan* plan, arangodb::velocypack::Slice const& base);
@ -118,6 +119,9 @@ class TraversalNode : public GraphNode {
result.emplace(condVar); result.emplace(condVar);
} }
} }
for (auto const& pruneVar : _pruneVariables) {
result.emplace(pruneVar);
}
if (usesInVariable()) { if (usesInVariable()) {
result.emplace(_inVariable); result.emplace(_inVariable);
} }
@ -178,11 +182,17 @@ class TraversalNode : public GraphNode {
void getConditionVariables(std::vector<Variable const*>&) const override; void getConditionVariables(std::vector<Variable const*>&) const override;
void getPruneVariables(std::vector<Variable const*>&) const;
/// @brief Compute the traversal options containing the expressions /// @brief Compute the traversal options containing the expressions
/// MUST! be called after optimization and before creation /// MUST! be called after optimization and before creation
/// of blocks. /// of blocks.
void prepareOptions() override; void prepareOptions() override;
// @brief Get reference to the Prune expression.
// You are not responsible for it!
Expression* pruneExpression() const { return _pruneExpression.get(); }
private: private:
#ifdef TRI_ENABLE_MAINTAINER_MODE #ifdef TRI_ENABLE_MAINTAINER_MODE
void checkConditionsDefined() const; void checkConditionsDefined() const;
@ -210,6 +220,9 @@ class TraversalNode : public GraphNode {
/// @brief The hard coded condition on _to /// @brief The hard coded condition on _to
AstNode* _toCondition; AstNode* _toCondition;
/// @brief The condition given in PRUNE (might be empty)
std::unique_ptr<Expression> _pruneExpression;
/// @brief The global edge condition. Does not contain /// @brief The global edge condition. Does not contain
/// _from and _to checks /// _from and _to checks
std::vector<AstNode const*> _globalEdgeConditions; std::vector<AstNode const*> _globalEdgeConditions;
@ -222,6 +235,9 @@ class TraversalNode : public GraphNode {
/// @brief List of all depth specific conditions for vertices /// @brief List of all depth specific conditions for vertices
std::unordered_map<uint64_t, AstNode*> _vertexConditions; std::unordered_map<uint64_t, AstNode*> _vertexConditions;
/// @brief the hashSet for variables used in pruning
arangodb::HashSet<Variable const*> _pruneVariables;
}; };
} // namespace aql } // namespace aql

2970
arangod/Aql/grammar.cpp
View File

File diff suppressed because it is too large Load Diff

4
arangod/Aql/grammar.h
View File

@ -125,7 +125,7 @@ extern int Aqldebug;
union YYSTYPE union YYSTYPE
{ {
#line 35 "Aql/grammar.y" /* yacc.c:1909 */ #line 35 "Aql/grammar.y" /* yacc.c:1915 */
arangodb::aql::AstNode* node; arangodb::aql::AstNode* node;
struct { struct {
@ -135,7 +135,7 @@ union YYSTYPE
bool boolval; bool boolval;
int64_t intval; int64_t intval;
#line 139 "Aql/grammar.hpp" /* yacc.c:1909 */ #line 139 "Aql/grammar.hpp" /* yacc.c:1915 */
}; };
typedef union YYSTYPE YYSTYPE; typedef union YYSTYPE YYSTYPE;

4
arangod/Aql/grammar.hpp
View File

@ -125,7 +125,7 @@ extern int Aqldebug;
union YYSTYPE union YYSTYPE
{ {
#line 35 "Aql/grammar.y" /* yacc.c:1909 */ #line 35 "Aql/grammar.y" /* yacc.c:1915 */
arangodb::aql::AstNode* node; arangodb::aql::AstNode* node;
struct { struct {
@ -135,7 +135,7 @@ union YYSTYPE
bool boolval; bool boolval;
int64_t intval; int64_t intval;
#line 139 "Aql/grammar.hpp" /* yacc.c:1909 */ #line 139 "Aql/grammar.hpp" /* yacc.c:1915 */
}; };
typedef union YYSTYPE YYSTYPE; typedef union YYSTYPE YYSTYPE;

222
arangod/Aql/grammar.y
View File

@ -200,6 +200,17 @@ static AstNode const* GetIntoExpression(AstNode const* node) {
return node->getMember(1); return node->getMember(1);
} }
static AstNode* TransformOutputVariables(Parser* parser, AstNode const* names) {
auto wrapperNode = parser->ast()->createNodeArray();
for (size_t i = 0; i < names->numMembers(); ++i) {
AstNode* variableNameNode = names->getMemberUnchecked(i);
TRI_ASSERT(variableNameNode->isStringValue());
AstNode* variableNode = parser->ast()->createNodeVariable(variableNameNode->getStringValue(), variableNameNode->getStringLength(), true);
wrapperNode->addMember(variableNode);
}
return wrapperNode;
}
%} %}
/* define tokens and "nice" token names */ /* define tokens and "nice" token names */
@ -338,6 +349,9 @@ static AstNode const* GetIntoExpression(AstNode const* node) {
%type <node> function_arguments_list; %type <node> function_arguments_list;
%type <node> compound_value; %type <node> compound_value;
%type <node> array; %type <node> array;
%type <node> for_output_variables;
%type <node> traversal_graph_info;
%type <node> shortest_path_graph_info;
%type <node> optional_array_elements; %type <node> optional_array_elements;
%type <node> array_elements_list; %type <node> array_elements_list;
%type <node> for_options; %type <node> for_options;
@ -479,19 +493,145 @@ statement_block_statement:
} }
; ;
more_output_variables:
variable_name {
auto wrapperNode = parser->ast()->createNodeArray();
parser->pushArray(wrapperNode);
// This is guaranteed to be called on the first variable
AstNode* node = parser->ast()->createNodeValueString($1.value, $1.length);
parser->pushArrayElement(node);
}
| more_output_variables T_COMMA variable_name {
AstNode* node = parser->ast()->createNodeValueString($3.value, $3.length);
parser->pushArrayElement(node);
}
;
for_output_variables:
more_output_variables {
$$ = parser->popArray();
}
;
prune_and_options:
/* empty no prune, no options, add two NOPS */ {
auto node = static_cast<AstNode*>(parser->peekStack());
// Prune
node->addMember(parser->ast()->createNodeNop());
// Options
node->addMember(parser->ast()->createNodeNop());
}
| T_STRING expression {
auto node = static_cast<AstNode*>(parser->peekStack());
if (TRI_CaseEqualString($1.value, "PRUNE")) {
/* Only Prune */
if ($2 == nullptr) {
ABORT_OOM
}
// Prune
node->addMember($2);
// Options
node->addMember(parser->ast()->createNodeNop());
} else if (TRI_CaseEqualString($1.value, "OPTIONS")) {
/* Only Options */
if ($2 == nullptr) {
ABORT_OOM
}
if (!$2->isObject()) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "traversal 'OPTIONS' have to be an object", $1.value, yylloc.first_line, yylloc.first_column);
}
// Prune
node->addMember(parser->ast()->createNodeNop());
// Options
node->addMember($2);
} else {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'PRUNE' or 'OPTIONS'", $1.value, yylloc.first_line, yylloc.first_column);
}
}
| T_STRING expression T_STRING object {
/* prune and options */
auto node = static_cast<AstNode*>(parser->peekStack());
if (!TRI_CaseEqualString($1.value, "PRUNE")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'PRUNE'", $1.value, yylloc.first_line, yylloc.first_column);
}
if ($2 == nullptr) {
ABORT_OOM
}
if (!TRI_CaseEqualString($3.value, "OPTIONS")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'OPTIONS'", $3.value, yylloc.first_line, yylloc.first_column);
}
if ($4 == nullptr) {
ABORT_OOM
}
// Prune
node->addMember($2);
// Options
node->addMember($4);
}
;
traversal_graph_info:
graph_direction_steps expression graph_subject {
auto infoNode = parser->ast()->createNodeArray();
// Direction
infoNode->addMember($1);
// Source
infoNode->addMember($2);
// Graph
infoNode->addMember($3);
$$ = infoNode;
}
;
shortest_path_graph_info:
graph_direction T_SHORTEST_PATH expression T_STRING expression graph_subject options {
if (!TRI_CaseEqualString($4.value, "TO")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'TO'", $4.value, yylloc.first_line, yylloc.first_column);
}
auto infoNode = parser->ast()->createNodeArray();
auto dirNode = parser->ast()->createNodeValueInt($1);
// Direction
infoNode->addMember(dirNode);
// Source
infoNode->addMember($3);
// Target
infoNode->addMember($5);
// Graph
infoNode->addMember($6);
// Opts
auto opts = parser->ast()->createNodeOptions($7);
TRI_ASSERT(opts != nullptr);
infoNode->addMember(opts);
$$ = infoNode;
}
;
for_statement: for_statement:
T_FOR variable_name T_IN expression { T_FOR for_output_variables T_IN expression {
// first open a new scope // first open a new scope (after expression is evaluated)
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR); parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
AstNode* variablesNode = static_cast<AstNode*>($2);
TRI_ASSERT(variablesNode != nullptr);
TRI_ASSERT(variablesNode->type == NODE_TYPE_ARRAY);
if (variablesNode->numMembers() != 1) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "Collections and Views only have one return variable", yylloc.first_line, yylloc.first_column);
}
// now create an out variable for the FOR statement // now create an out variable for the FOR statement
// this prepares us to handle the optional SEARCH condition, which may // this prepares us to handle the optional SEARCH condition, which may
// or may not refer to the FOR's variable // or may not refer to the FOR's variable
parser->pushStack(parser->ast()->createNodeVariable($2.value, $2.length, true)); AstNode* variableNameNode = variablesNode->getMemberUnchecked(0);
TRI_ASSERT(variableNameNode->isStringValue());
AstNode* variableNode = parser->ast()->createNodeVariable(variableNameNode->getStringValue(), variableNameNode->getStringLength(), true);
parser->pushStack(variableNode);
} for_options { } for_options {
// now we can handle the optional SEARCH condition and OPTIONS. // now we can handle the optional SEARCH condition and OPTIONS.
AstNode* variableNode = static_cast<AstNode*>(parser->popStack()); AstNode* variableNode = static_cast<AstNode*>(parser->popStack());
TRI_ASSERT(variableNode != nullptr);
Variable* variable = static_cast<Variable*>(variableNode->getData()); Variable* variable = static_cast<Variable*>(variableNode->getData());
AstNode* node = nullptr; AstNode* node = nullptr;
@ -528,46 +668,46 @@ for_statement:
parser->ast()->addOperation(node); parser->ast()->addOperation(node);
} }
| T_FOR traversal_statement { | T_FOR for_output_variables T_IN traversal_graph_info {
// first open a new scope (after expression is evaluated)
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
// Traversal
auto variableNamesNode = static_cast<AstNode*>($2);
TRI_ASSERT(variableNamesNode != nullptr);
TRI_ASSERT(variableNamesNode->type == NODE_TYPE_ARRAY);
if (variableNamesNode->numMembers() > 3) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "Traversals only have one, two or three return variables", yylloc.first_line, yylloc.first_column);
} }
| T_FOR shortest_path_statement { auto variablesNode = TransformOutputVariables(parser, variableNamesNode);
auto graphInfoNode = static_cast<AstNode*>($4);
TRI_ASSERT(graphInfoNode != nullptr);
TRI_ASSERT(graphInfoNode->type == NODE_TYPE_ARRAY);
parser->pushStack(variablesNode);
parser->pushStack(graphInfoNode);
// This stack push/pop magic is necessary to allow v, e, and p in the prune condition
} prune_and_options {
auto graphInfoNode = static_cast<AstNode*>(parser->popStack());
auto variablesNode = static_cast<AstNode*>(parser->popStack());
auto node = parser->ast()->createNodeTraversal(variablesNode, graphInfoNode);
parser->ast()->addOperation(node);
} }
; | T_FOR for_output_variables T_IN shortest_path_graph_info {
// first open a new scope (after expression is evaluated)
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
// Shortest Path
auto variableNamesNode = static_cast<AstNode*>($2);
TRI_ASSERT(variableNamesNode != nullptr);
TRI_ASSERT(variableNamesNode->type == NODE_TYPE_ARRAY);
if (variableNamesNode->numMembers() > 2) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "ShortestPath only has one or two return variables", yylloc.first_line, yylloc.first_column);
}
auto variablesNode = TransformOutputVariables(parser, variableNamesNode);
auto graphInfoNode = static_cast<AstNode*>($4);
TRI_ASSERT(graphInfoNode != nullptr);
TRI_ASSERT(graphInfoNode->type == NODE_TYPE_ARRAY);
auto node = parser->ast()->createNodeShortestPath(variablesNode, graphInfoNode);
parser->ast()->addOperation(node);
traversal_statement:
variable_name T_IN graph_direction_steps expression graph_subject options {
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeTraversal($1.value, $1.length, $3, $4, $5, $6);
parser->ast()->addOperation(node);
}
| variable_name T_COMMA variable_name T_IN graph_direction_steps expression graph_subject options {
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeTraversal($1.value, $1.length, $3.value, $3.length, $5, $6, $7, $8);
parser->ast()->addOperation(node);
}
| variable_name T_COMMA variable_name T_COMMA variable_name T_IN graph_direction_steps expression graph_subject options {
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeTraversal($1.value, $1.length, $3.value, $3.length, $5.value, $5.length, $7, $8, $9, $10);
parser->ast()->addOperation(node);
}
;
shortest_path_statement:
variable_name T_IN graph_direction T_SHORTEST_PATH expression T_STRING expression graph_subject options {
if (!TRI_CaseEqualString($6.value, "TO")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'TO'", $6.value, yylloc.first_line, yylloc.first_column);
}
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeShortestPath($1.value, $1.length, $3, $5, $7, $8, $9);
parser->ast()->addOperation(node);
}
| variable_name T_COMMA variable_name T_IN graph_direction T_SHORTEST_PATH expression T_STRING expression graph_subject options {
if (!TRI_CaseEqualString($8.value, "TO")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'TO'", $8.value, yylloc.first_line, yylloc.first_column);
}
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeShortestPath($1.value, $1.length, $3.value, $3.length, $5, $7, $9, $10, $11);
parser->ast()->addOperation(node);
} }
; ;

2
arangod/CMakeLists.txt
View File

@ -224,6 +224,7 @@ SET(ARANGOD_SOURCES
Aql/Functions.cpp Aql/Functions.cpp
Aql/GraphNode.cpp Aql/GraphNode.cpp
Aql/Graphs.cpp Aql/Graphs.cpp
Aql/InAndOutRowExpressionContext.cpp
Aql/IdExecutor.cpp Aql/IdExecutor.cpp
Aql/IndexExecutor.cpp Aql/IndexExecutor.cpp
Aql/IndexNode.cpp Aql/IndexNode.cpp
@ -242,6 +243,7 @@ SET(ARANGOD_SOURCES
Aql/OutputAqlItemRow.cpp Aql/OutputAqlItemRow.cpp
Aql/Parser.cpp Aql/Parser.cpp
Aql/PlanCache.cpp Aql/PlanCache.cpp
Aql/PruneExpressionEvaluator.cpp
Aql/Quantifier.cpp Aql/Quantifier.cpp
Aql/Query.cpp Aql/Query.cpp
Aql/QueryCache.cpp Aql/QueryCache.cpp

1
arangod/Graph/BaseOptions.h
View File

@ -38,6 +38,7 @@ struct AstNode;
class ExecutionPlan; class ExecutionPlan;
class Expression; class Expression;
class Query; class Query;
} // namespace aql } // namespace aql
namespace velocypack { namespace velocypack {

109
arangod/Graph/BreadthFirstEnumerator.cpp
View File

@ -23,6 +23,7 @@
#include "BreadthFirstEnumerator.h" #include "BreadthFirstEnumerator.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Graph/EdgeCursor.h" #include "Graph/EdgeCursor.h"
#include "Graph/EdgeDocumentToken.h" #include "Graph/EdgeDocumentToken.h"
#include "Graph/Traverser.h" #include "Graph/Traverser.h"
@ -51,12 +52,13 @@ BreadthFirstEnumerator::PathStep::PathStep(PathStep& other)
BreadthFirstEnumerator::BreadthFirstEnumerator(Traverser* traverser, VPackSlice startVertex, BreadthFirstEnumerator::BreadthFirstEnumerator(Traverser* traverser, VPackSlice startVertex,
TraverserOptions* opts) TraverserOptions* opts)
: PathEnumerator(traverser, startVertex.copyString(), opts), : PathEnumerator(traverser, startVertex.copyString(), opts),
_schreierIndex(1), _schreierIndex(0),
_lastReturned(0), _lastReturned(0),
_currentDepth(0), _currentDepth(0),
_toSearchPos(0) { _toSearchPos(0) {
_schreier.reserve(32); _schreier.reserve(32);
arangodb::velocypack::StringRef startVId = _opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex)); arangodb::velocypack::StringRef startVId =
_opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex));
_schreier.emplace_back(std::make_unique<PathStep>(startVId)); _schreier.emplace_back(std::make_unique<PathStep>(startVId));
_toSearch.emplace_back(NextStep(0)); _toSearch.emplace_back(NextStep(0));
@ -67,6 +69,13 @@ BreadthFirstEnumerator::~BreadthFirstEnumerator() {}
bool BreadthFirstEnumerator::next() { bool BreadthFirstEnumerator::next() {
if (_isFirst) { if (_isFirst) {
_isFirst = false; _isFirst = false;
if (shouldPrune()) {
TRI_ASSERT(_toSearch.size() == 1);
// Throw the next one away
_toSearch.clear();
}
// We have faked the 0 position in schreier for pruning
_schreierIndex++;
if (_opts->minDepth == 0) { if (_opts->minDepth == 0) {
return true; return true;
} }
@ -92,22 +101,10 @@ bool BreadthFirstEnumerator::next() {
while (true) { while (true) {
if (_toSearchPos >= _toSearch.size()) { if (_toSearchPos >= _toSearch.size()) {
// This depth is done. GoTo next // This depth is done. GoTo next
if (_nextDepth.empty()) { if (!prepareSearchOnNextDepth()) {
// That's it. we are done. // That's it. we are done.
return false; return false;
} }
// Save copies:
// We clear current
// we swap current and next.
// So now current is filled
// and next is empty.
_toSearch.clear();
_toSearchPos = 0;
_toSearch.swap(_nextDepth);
_currentDepth++;
TRI_ASSERT(_toSearchPos < _toSearch.size());
TRI_ASSERT(_nextDepth.empty());
TRI_ASSERT(_currentDepth < _opts->maxDepth);
} }
// This access is always safe. // This access is always safe.
// If not it should have bailed out before. // If not it should have bailed out before.
@ -152,8 +149,11 @@ bool BreadthFirstEnumerator::next() {
_schreier.emplace_back(std::make_unique<PathStep>(nextIdx, std::move(eid), vId)); _schreier.emplace_back(std::make_unique<PathStep>(nextIdx, std::move(eid), vId));
if (_currentDepth < _opts->maxDepth - 1) { if (_currentDepth < _opts->maxDepth - 1) {
// Prune here
if (!shouldPrune()) {
_nextDepth.emplace_back(NextStep(_schreierIndex)); _nextDepth.emplace_back(NextStep(_schreierIndex));
} }
}
_schreierIndex++; _schreierIndex++;
didInsert = true; didInsert = true;
} }
@ -180,30 +180,41 @@ bool BreadthFirstEnumerator::next() {
// entry. We compute the path to it. // entry. We compute the path to it.
return true; return true;
} }
arangodb::aql::AqlValue BreadthFirstEnumerator::lastVertexToAqlValue() { arangodb::aql::AqlValue BreadthFirstEnumerator::lastVertexToAqlValue() {
TRI_ASSERT(_lastReturned < _schreier.size()); return vertexToAqlValue(_lastReturned);
return _traverser->fetchVertexData(_schreier[_lastReturned]->vertex);
} }
arangodb::aql::AqlValue BreadthFirstEnumerator::lastEdgeToAqlValue() { arangodb::aql::AqlValue BreadthFirstEnumerator::lastEdgeToAqlValue() {
TRI_ASSERT(_lastReturned < _schreier.size()); return edgeToAqlValue(_lastReturned);
if (_lastReturned == 0) {
// This is the first Vertex. No Edge Pointing to it
return arangodb::aql::AqlValue(arangodb::aql::AqlValueHintNull());
}
return _opts->cache()->fetchEdgeAqlResult(_schreier[_lastReturned]->edge);
} }
arangodb::aql::AqlValue BreadthFirstEnumerator::pathToAqlValue(arangodb::velocypack::Builder& result) { arangodb::aql::AqlValue BreadthFirstEnumerator::pathToAqlValue(arangodb::velocypack::Builder& result) {
return pathToIndexToAqlValue(result, _lastReturned);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::vertexToAqlValue(size_t index) {
TRI_ASSERT(index < _schreier.size());
return _traverser->fetchVertexData(_schreier[index]->vertex);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::edgeToAqlValue(size_t index) {
TRI_ASSERT(index < _schreier.size());
if (index == 0) {
// This is the first Vertex. No Edge Pointing to it
return arangodb::aql::AqlValue(arangodb::aql::AqlValueHintNull());
}
return _opts->cache()->fetchEdgeAqlResult(_schreier[index]->edge);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::pathToIndexToAqlValue(
arangodb::velocypack::Builder& result, size_t index) {
// TODO make deque class variable // TODO make deque class variable
std::deque<size_t> fullPath; std::deque<size_t> fullPath;
size_t cur = _lastReturned; while (index != 0) {
while (cur != 0) {
// Walk backwards through the path and push everything found on the local // Walk backwards through the path and push everything found on the local
// stack // stack
fullPath.emplace_front(cur); fullPath.emplace_front(index);
cur = _schreier[cur]->sourceIdx; index = _schreier[index]->sourceIdx;
} }
result.clear(); result.clear();
@ -226,7 +237,8 @@ arangodb::aql::AqlValue BreadthFirstEnumerator::pathToAqlValue(arangodb::velocyp
return arangodb::aql::AqlValue(result.slice()); return arangodb::aql::AqlValue(result.slice());
} }
bool BreadthFirstEnumerator::pathContainsVertex(size_t index, arangodb::velocypack::StringRef vertex) const { bool BreadthFirstEnumerator::pathContainsVertex(size_t index,
arangodb::velocypack::StringRef vertex) const {
while (true) { while (true) {
TRI_ASSERT(index < _schreier.size()); TRI_ASSERT(index < _schreier.size());
auto const& step = _schreier[index]; auto const& step = _schreier[index];
@ -262,3 +274,42 @@ bool BreadthFirstEnumerator::pathContainsEdge(size_t index,
} }
return false; return false;
} }
bool BreadthFirstEnumerator::prepareSearchOnNextDepth() {
if (_nextDepth.empty()) {
// Nothing left to search
return false;
}
// Save copies:
// We clear current
// we swap current and next.
// So now current is filled
// and next is empty.
_toSearch.clear();
_toSearchPos = 0;
_toSearch.swap(_nextDepth);
_currentDepth++;
TRI_ASSERT(_toSearchPos < _toSearch.size());
TRI_ASSERT(_nextDepth.empty());
TRI_ASSERT(_currentDepth < _opts->maxDepth);
return true;
}
bool BreadthFirstEnumerator::shouldPrune() {
if (_opts->usesPrune()) {
auto* evaluator = _opts->getPruneEvaluator();
if (evaluator->needsVertex()) {
evaluator->injectVertex(vertexToAqlValue(_schreierIndex).slice());
}
if (evaluator->needsEdge()) {
evaluator->injectEdge(edgeToAqlValue(_schreierIndex).slice());
}
transaction::BuilderLeaser builder(_opts->trx());
if (evaluator->needsPath()) {
aql::AqlValue val = pathToIndexToAqlValue(*builder.get(), _schreierIndex);
evaluator->injectPath(val.slice());
}
return evaluator->evaluate();
}
return false;
}

15
arangod/Graph/BreadthFirstEnumerator.h
View File

@ -172,6 +172,21 @@ class BreadthFirstEnumerator final : public arangodb::traverser::PathEnumerator
* @return true if the edge is already in the path * @return true if the edge is already in the path
*/ */
bool pathContainsEdge(size_t index, graph::EdgeDocumentToken const& edge) const; bool pathContainsEdge(size_t index, graph::EdgeDocumentToken const& edge) const;
/**
* @brief Reset iterators to search within next depth
* Also honors pruned paths
* @return true if we can continue searching. False if we are done
*/
bool prepareSearchOnNextDepth();
aql::AqlValue vertexToAqlValue(size_t index);
aql::AqlValue edgeToAqlValue(size_t index);
aql::AqlValue pathToIndexToAqlValue(arangodb::velocypack::Builder& result, size_t index);
bool shouldPrune();
}; };
} // namespace graph } // namespace graph
} // namespace arangodb } // namespace arangodb

39
arangod/Graph/NeighborsEnumerator.cpp
View File

@ -23,6 +23,7 @@
#include "NeighborsEnumerator.h" #include "NeighborsEnumerator.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Basics/VelocyPackHelper.h" #include "Basics/VelocyPackHelper.h"
#include "Graph/EdgeCursor.h" #include "Graph/EdgeCursor.h"
#include "Graph/Traverser.h" #include "Graph/Traverser.h"
@ -36,7 +37,8 @@ using namespace arangodb::traverser;
NeighborsEnumerator::NeighborsEnumerator(Traverser* traverser, VPackSlice const& startVertex, NeighborsEnumerator::NeighborsEnumerator(Traverser* traverser, VPackSlice const& startVertex,
TraverserOptions* opts) TraverserOptions* opts)
: PathEnumerator(traverser, startVertex.copyString(), opts), _searchDepth(0) { : PathEnumerator(traverser, startVertex.copyString(), opts), _searchDepth(0) {
arangodb::velocypack::StringRef vId = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(startVertex)); arangodb::velocypack::StringRef vId = _traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(startVertex));
_allFound.insert(vId); _allFound.insert(vId);
_currentDepth.insert(vId); _currentDepth.insert(vId);
_iterator = _currentDepth.begin(); _iterator = _currentDepth.begin();
@ -58,8 +60,7 @@ bool NeighborsEnumerator::next() {
return false; return false;
} }
_lastDepth.swap(_currentDepth); swapLastAndCurrentDepth();
_currentDepth.clear();
for (auto const& nextVertex : _lastDepth) { for (auto const& nextVertex : _lastDepth) {
auto callback = [&](EdgeDocumentToken&& eid, VPackSlice other, size_t cursorId) { auto callback = [&](EdgeDocumentToken&& eid, VPackSlice other, size_t cursorId) {
if (_opts->hasEdgeFilter(_searchDepth, cursorId)) { if (_opts->hasEdgeFilter(_searchDepth, cursorId)) {
@ -90,8 +91,11 @@ bool NeighborsEnumerator::next() {
if (_allFound.find(v) == _allFound.end()) { if (_allFound.find(v) == _allFound.end()) {
if (_traverser->vertexMatchesConditions(v, _searchDepth + 1)) { if (_traverser->vertexMatchesConditions(v, _searchDepth + 1)) {
_currentDepth.emplace(v);
_allFound.emplace(v); _allFound.emplace(v);
if (shouldPrune(v)) {
_toPrune.emplace(v);
}
_currentDepth.emplace(v);
} }
} else { } else {
_opts->cache()->increaseFilterCounter(); _opts->cache()->increaseFilterCounter();
@ -132,3 +136,30 @@ arangodb::aql::AqlValue NeighborsEnumerator::pathToAqlValue(arangodb::velocypack
TRI_ASSERT(false); TRI_ASSERT(false);
THROW_ARANGO_EXCEPTION(TRI_ERROR_NOT_IMPLEMENTED); THROW_ARANGO_EXCEPTION(TRI_ERROR_NOT_IMPLEMENTED);
} }
void NeighborsEnumerator::swapLastAndCurrentDepth() {
// Filter all in _toPrune
if (!_toPrune.empty()) {
for (auto const& it : _toPrune) {
_currentDepth.erase(it);
}
_toPrune.clear();
}
_lastDepth.swap(_currentDepth);
_currentDepth.clear();
}
bool NeighborsEnumerator::shouldPrune(arangodb::velocypack::StringRef v) {
// Prune here
if (_opts->usesPrune()) {
auto* evaluator = _opts->getPruneEvaluator();
if (evaluator->needsVertex()) {
evaluator->injectVertex(_traverser->fetchVertexData(v).slice());
}
// We cannot support these two here
TRI_ASSERT(!evaluator->needsEdge());
TRI_ASSERT(!evaluator->needsPath());
return evaluator->evaluate();
}
return false;
}

6
arangod/Graph/NeighborsEnumerator.h
View File

@ -38,6 +38,7 @@ class NeighborsEnumerator final : public arangodb::traverser::PathEnumerator {
std::unordered_set<arangodb::velocypack::StringRef> _currentDepth; std::unordered_set<arangodb::velocypack::StringRef> _currentDepth;
std::unordered_set<arangodb::velocypack::StringRef> _lastDepth; std::unordered_set<arangodb::velocypack::StringRef> _lastDepth;
std::unordered_set<arangodb::velocypack::StringRef>::iterator _iterator; std::unordered_set<arangodb::velocypack::StringRef>::iterator _iterator;
std::unordered_set<arangodb::velocypack::StringRef> _toPrune;
uint64_t _searchDepth; uint64_t _searchDepth;
@ -65,6 +66,11 @@ class NeighborsEnumerator final : public arangodb::traverser::PathEnumerator {
aql::AqlValue lastEdgeToAqlValue() override; aql::AqlValue lastEdgeToAqlValue() override;
aql::AqlValue pathToAqlValue(arangodb::velocypack::Builder& result) override; aql::AqlValue pathToAqlValue(arangodb::velocypack::Builder& result) override;
private:
void swapLastAndCurrentDepth();
bool shouldPrune(arangodb::velocypack::StringRef v);
}; };
} // namespace graph } // namespace graph

42
arangod/Graph/PathEnumerator.cpp
View File

@ -22,6 +22,8 @@
//////////////////////////////////////////////////////////////////////////////// ////////////////////////////////////////////////////////////////////////////////
#include "PathEnumerator.h" #include "PathEnumerator.h"
#include "Aql/AqlValue.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Basics/VelocyPackHelper.h" #include "Basics/VelocyPackHelper.h"
#include "Graph/EdgeCursor.h" #include "Graph/EdgeCursor.h"
#include "Graph/Traverser.h" #include "Graph/Traverser.h"
@ -36,7 +38,8 @@ using TraverserOptions = arangodb::traverser::TraverserOptions;
PathEnumerator::PathEnumerator(Traverser* traverser, std::string const& startVertex, PathEnumerator::PathEnumerator(Traverser* traverser, std::string const& startVertex,
TraverserOptions* opts) TraverserOptions* opts)
: _traverser(traverser), _isFirst(true), _opts(opts) { : _traverser(traverser), _isFirst(true), _opts(opts) {
arangodb::velocypack::StringRef svId = _opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex)); arangodb::velocypack::StringRef svId =
_opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex));
// Guarantee that this vertex _id does not run away // Guarantee that this vertex _id does not run away
_enumeratedPath.vertices.push_back(svId); _enumeratedPath.vertices.push_back(svId);
TRI_ASSERT(_enumeratedPath.vertices.size() == 1); TRI_ASSERT(_enumeratedPath.vertices.size() == 1);
@ -44,13 +47,16 @@ PathEnumerator::PathEnumerator(Traverser* traverser, std::string const& startVer
DepthFirstEnumerator::DepthFirstEnumerator(Traverser* traverser, std::string const& startVertex, DepthFirstEnumerator::DepthFirstEnumerator(Traverser* traverser, std::string const& startVertex,
TraverserOptions* opts) TraverserOptions* opts)
: PathEnumerator(traverser, startVertex, opts) {} : PathEnumerator(traverser, startVertex, opts), _pruneNext(false) {}
DepthFirstEnumerator::~DepthFirstEnumerator() {} DepthFirstEnumerator::~DepthFirstEnumerator() {}
bool DepthFirstEnumerator::next() { bool DepthFirstEnumerator::next() {
if (_isFirst) { if (_isFirst) {
_isFirst = false; _isFirst = false;
if (shouldPrune()) {
_pruneNext = true;
}
if (_opts->minDepth == 0) { if (_opts->minDepth == 0) {
return true; return true;
} }
@ -61,11 +67,12 @@ bool DepthFirstEnumerator::next() {
} }
while (true) { while (true) {
if (_enumeratedPath.edges.size() < _opts->maxDepth) { if (_enumeratedPath.edges.size() < _opts->maxDepth && !_pruneNext) {
// We are not done with this path, so // We are not done with this path, so
// we reserve the cursor for next depth // we reserve the cursor for next depth
auto cursor = _opts->nextCursor(_traverser->mmdr(), auto cursor = _opts->nextCursor(_traverser->mmdr(),
arangodb::velocypack::StringRef(_enumeratedPath.vertices.back()), arangodb::velocypack::StringRef(
_enumeratedPath.vertices.back()),
_enumeratedPath.edges.size()); _enumeratedPath.edges.size());
if (cursor != nullptr) { if (cursor != nullptr) {
_edgeCursors.emplace(cursor); _edgeCursors.emplace(cursor);
@ -77,6 +84,7 @@ bool DepthFirstEnumerator::next() {
_enumeratedPath.edges.pop_back(); _enumeratedPath.edges.pop_back();
} }
} }
_pruneNext = false;
bool foundPath = false; bool foundPath = false;
@ -145,6 +153,9 @@ bool DepthFirstEnumerator::next() {
if (cursor->next(callback)) { if (cursor->next(callback)) {
if (foundPath) { if (foundPath) {
if (shouldPrune()) {
_pruneNext = true;
}
if (_enumeratedPath.edges.size() < _opts->minDepth) { if (_enumeratedPath.edges.size() < _opts->minDepth) {
// We have a valid prefix, but do NOT return this path // We have a valid prefix, but do NOT return this path
break; break;
@ -171,7 +182,8 @@ bool DepthFirstEnumerator::next() {
} }
arangodb::aql::AqlValue DepthFirstEnumerator::lastVertexToAqlValue() { arangodb::aql::AqlValue DepthFirstEnumerator::lastVertexToAqlValue() {
return _traverser->fetchVertexData(arangodb::velocypack::StringRef(_enumeratedPath.vertices.back())); return _traverser->fetchVertexData(
arangodb::velocypack::StringRef(_enumeratedPath.vertices.back()));
} }
arangodb::aql::AqlValue DepthFirstEnumerator::lastEdgeToAqlValue() { arangodb::aql::AqlValue DepthFirstEnumerator::lastEdgeToAqlValue() {
@ -202,3 +214,23 @@ arangodb::aql::AqlValue DepthFirstEnumerator::pathToAqlValue(arangodb::velocypac
result.close(); result.close();
return arangodb::aql::AqlValue(result.slice()); return arangodb::aql::AqlValue(result.slice());
} }
bool DepthFirstEnumerator::shouldPrune() {
// We need to call prune here
if (_opts->usesPrune()) {
auto* evaluator = _opts->getPruneEvaluator();
if (evaluator->needsVertex()) {
evaluator->injectVertex(lastVertexToAqlValue().slice());
}
if (evaluator->needsEdge()) {
evaluator->injectEdge(lastEdgeToAqlValue().slice());
}
transaction::BuilderLeaser builder(_opts->trx());
if (evaluator->needsPath()) {
aql::AqlValue val = pathToAqlValue(*builder.get());
evaluator->injectPath(val.slice());
}
return evaluator->evaluate();
}
return false;
}

13
arangod/Graph/PathEnumerator.h
View File

@ -108,6 +108,11 @@ class DepthFirstEnumerator final : public PathEnumerator {
std::stack<std::unique_ptr<graph::EdgeCursor>> _edgeCursors; std::stack<std::unique_ptr<graph::EdgeCursor>> _edgeCursors;
//////////////////////////////////////////////////////////////////////////////
/// @brief Flag if we need to prune the next path
//////////////////////////////////////////////////////////////////////////////
bool _pruneNext;
public: public:
DepthFirstEnumerator(Traverser* traverser, std::string const& startVertex, DepthFirstEnumerator(Traverser* traverser, std::string const& startVertex,
TraverserOptions* opts); TraverserOptions* opts);
@ -120,16 +125,14 @@ class DepthFirstEnumerator final : public PathEnumerator {
bool next() override; bool next() override;
//////////////////////////////////////////////////////////////////////////////
/// @brief Prunes the current path prefix, the next function should not return
/// any path having this prefix anymore.
//////////////////////////////////////////////////////////////////////////////
aql::AqlValue lastVertexToAqlValue() override; aql::AqlValue lastVertexToAqlValue() override;
aql::AqlValue lastEdgeToAqlValue() override; aql::AqlValue lastEdgeToAqlValue() override;
aql::AqlValue pathToAqlValue(arangodb::velocypack::Builder& result) override; aql::AqlValue pathToAqlValue(arangodb::velocypack::Builder& result) override;
private:
bool shouldPrune();
}; };
} // namespace traverser } // namespace traverser
} // namespace arangodb } // namespace arangodb

34
arangod/Graph/Traverser.cpp
View File

@ -38,7 +38,8 @@ using namespace arangodb;
using namespace arangodb::traverser; using namespace arangodb::traverser;
using namespace arangodb::graph; using namespace arangodb::graph;
bool Traverser::VertexGetter::getVertex(VPackSlice edge, std::vector<arangodb::velocypack::StringRef>& result) { bool Traverser::VertexGetter::getVertex(VPackSlice edge,
std::vector<arangodb::velocypack::StringRef>& result) {
VPackSlice res = edge; VPackSlice res = edge;
if (!res.isString()) { if (!res.isString()) {
res = transaction::helpers::extractFromFromDocument(edge); res = transaction::helpers::extractFromFromDocument(edge);
@ -48,15 +49,19 @@ bool Traverser::VertexGetter::getVertex(VPackSlice edge, std::vector<arangodb::v
} }
TRI_ASSERT(res.isString()); TRI_ASSERT(res.isString());
if (!_traverser->vertexMatchesConditions(arangodb::velocypack::StringRef(res), result.size())) { if (!_traverser->vertexMatchesConditions(arangodb::velocypack::StringRef(res),
result.size())) {
return false; return false;
} }
result.emplace_back(_traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(res))); result.emplace_back(_traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(res)));
return true; return true;
} }
bool Traverser::VertexGetter::getSingleVertex(arangodb::velocypack::Slice edge, arangodb::velocypack::StringRef cmp, bool Traverser::VertexGetter::getSingleVertex(arangodb::velocypack::Slice edge,
uint64_t depth, arangodb::velocypack::StringRef& result) { arangodb::velocypack::StringRef cmp,
uint64_t depth,
arangodb::velocypack::StringRef& result) {
VPackSlice resSlice = edge; VPackSlice resSlice = edge;
if (!resSlice.isString()) { if (!resSlice.isString()) {
VPackSlice from = transaction::helpers::extractFromFromDocument(edge); VPackSlice from = transaction::helpers::extractFromFromDocument(edge);
@ -67,7 +72,8 @@ bool Traverser::VertexGetter::getSingleVertex(arangodb::velocypack::Slice edge,
} }
TRI_ASSERT(resSlice.isString()); TRI_ASSERT(resSlice.isString());
} }
result = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(resSlice)); result = _traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(resSlice));
return _traverser->vertexMatchesConditions(result, depth); return _traverser->vertexMatchesConditions(result, depth);
} }
@ -86,7 +92,8 @@ bool Traverser::UniqueVertexGetter::getVertex(VPackSlice edge,
TRI_ASSERT(toAdd.isString()); TRI_ASSERT(toAdd.isString());
} }
arangodb::velocypack::StringRef toAddStr = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(toAdd)); arangodb::velocypack::StringRef toAddStr =
_traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(toAdd));
// First check if we visited it. If not, then mark // First check if we visited it. If not, then mark
if (_returnedVertices.find(toAddStr) != _returnedVertices.end()) { if (_returnedVertices.find(toAddStr) != _returnedVertices.end()) {
// This vertex is not unique. // This vertex is not unique.
@ -104,7 +111,8 @@ bool Traverser::UniqueVertexGetter::getVertex(VPackSlice edge,
} }
bool Traverser::UniqueVertexGetter::getSingleVertex(arangodb::velocypack::Slice edge, bool Traverser::UniqueVertexGetter::getSingleVertex(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef cmp, uint64_t depth, arangodb::velocypack::StringRef cmp,
uint64_t depth,
arangodb::velocypack::StringRef& result) { arangodb::velocypack::StringRef& result) {
VPackSlice resSlice = edge; VPackSlice resSlice = edge;
if (!resSlice.isString()) { if (!resSlice.isString()) {
@ -115,7 +123,8 @@ bool Traverser::UniqueVertexGetter::getSingleVertex(arangodb::velocypack::Slice
TRI_ASSERT(resSlice.isString()); TRI_ASSERT(resSlice.isString());
} }
result = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(resSlice)); result = _traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(resSlice));
// First check if we visited it. If not, then mark // First check if we visited it. If not, then mark
if (_returnedVertices.find(result) != _returnedVertices.end()) { if (_returnedVertices.find(result) != _returnedVertices.end()) {
// This vertex is not unique. // This vertex is not unique.
@ -141,7 +150,6 @@ Traverser::Traverser(arangodb::traverser::TraverserOptions* opts, transaction::M
: _trx(trx), : _trx(trx),
_mmdr(new arangodb::ManagedDocumentResult()), _mmdr(new arangodb::ManagedDocumentResult()),
_startIdBuilder(), _startIdBuilder(),
_pruneNext(false),
_done(true), _done(true),
_opts(opts), _opts(opts),
_canUseOptimizedNeighbors(false) { _canUseOptimizedNeighbors(false) {
@ -154,7 +162,8 @@ Traverser::Traverser(arangodb::traverser::TraverserOptions* opts, transaction::M
Traverser::~Traverser() {} Traverser::~Traverser() {}
bool arangodb::traverser::Traverser::edgeMatchesConditions(VPackSlice e, arangodb::velocypack::StringRef vid, bool arangodb::traverser::Traverser::edgeMatchesConditions(VPackSlice e,
arangodb::velocypack::StringRef vid,
uint64_t depth, size_t cursorId) { uint64_t depth, size_t cursorId) {
if (!_opts->evaluateEdgeExpression(e, vid, depth, cursorId)) { if (!_opts->evaluateEdgeExpression(e, vid, depth, cursorId)) {
return false; return false;
@ -162,7 +171,8 @@ bool arangodb::traverser::Traverser::edgeMatchesConditions(VPackSlice e, arangod
return true; return true;
} }
bool arangodb::traverser::Traverser::vertexMatchesConditions(arangodb::velocypack::StringRef v, uint64_t depth) { bool arangodb::traverser::Traverser::vertexMatchesConditions(arangodb::velocypack::StringRef v,
uint64_t depth) {
if (_opts->vertexHasFilter(depth)) { if (_opts->vertexHasFilter(depth)) {
// We always need to destroy this vertex // We always need to destroy this vertex
aql::AqlValue vertex = fetchVertexData(v); aql::AqlValue vertex = fetchVertexData(v);

21
arangod/Graph/Traverser.h
View File

@ -134,7 +134,8 @@ class Traverser {
virtual ~VertexGetter() = default; virtual ~VertexGetter() = default;
virtual bool getVertex(arangodb::velocypack::Slice, std::vector<arangodb::velocypack::StringRef>&); virtual bool getVertex(arangodb::velocypack::Slice,
std::vector<arangodb::velocypack::StringRef>&);
virtual bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef, virtual bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef,
uint64_t, arangodb::velocypack::StringRef&); uint64_t, arangodb::velocypack::StringRef&);
@ -156,9 +157,11 @@ class Traverser {
~UniqueVertexGetter() = default; ~UniqueVertexGetter() = default;
bool getVertex(arangodb::velocypack::Slice, std::vector<arangodb::velocypack::StringRef>&) override; bool getVertex(arangodb::velocypack::Slice,
std::vector<arangodb::velocypack::StringRef>&) override;
bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef, uint64_t, arangodb::velocypack::StringRef&) override; bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef,
uint64_t, arangodb::velocypack::StringRef&) override;
void reset(arangodb::velocypack::StringRef const&) override; void reset(arangodb::velocypack::StringRef const&) override;
@ -218,7 +221,8 @@ class Traverser {
/// @brief Function to load the other sides vertex of an edge /// @brief Function to load the other sides vertex of an edge
/// Returns true if the vertex passes filtering conditions /// Returns true if the vertex passes filtering conditions
virtual bool getSingleVertex(arangodb::velocypack::Slice edge, virtual bool getSingleVertex(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef const sourceVertexId, uint64_t depth, arangodb::velocypack::StringRef const sourceVertexId,
uint64_t depth,
arangodb::velocypack::StringRef& targetVertexId) = 0; arangodb::velocypack::StringRef& targetVertexId) = 0;
public: public:
@ -276,7 +280,8 @@ class Traverser {
bool hasMore() { return !_done; } bool hasMore() { return !_done; }
bool edgeMatchesConditions(arangodb::velocypack::Slice edge, arangodb::velocypack::StringRef vid, bool edgeMatchesConditions(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef vid,
uint64_t depth, size_t cursorId); uint64_t depth, size_t cursorId);
bool vertexMatchesConditions(arangodb::velocypack::StringRef vid, uint64_t depth); bool vertexMatchesConditions(arangodb::velocypack::StringRef vid, uint64_t depth);
@ -306,9 +311,6 @@ class Traverser {
/// @brief Builder for the start value slice. Leased from transaction /// @brief Builder for the start value slice. Leased from transaction
velocypack::Builder _startIdBuilder; velocypack::Builder _startIdBuilder;
/// @brief toggle if this path should be pruned on next step
bool _pruneNext;
/// @brief indicator if this traversal is done /// @brief indicator if this traversal is done
bool _done; bool _done;
@ -321,7 +323,8 @@ class Traverser {
virtual aql::AqlValue fetchVertexData(arangodb::velocypack::StringRef vid) = 0; virtual aql::AqlValue fetchVertexData(arangodb::velocypack::StringRef vid) = 0;
/// @brief Function to add the real data of a vertex into a velocypack builder /// @brief Function to add the real data of a vertex into a velocypack builder
virtual void addVertexToVelocyPack(arangodb::velocypack::StringRef vid, arangodb::velocypack::Builder&) = 0; virtual void addVertexToVelocyPack(arangodb::velocypack::StringRef vid,
arangodb::velocypack::Builder&) = 0;
}; };
} // namespace traverser } // namespace traverser
} // namespace arangodb } // namespace arangodb

23
arangod/Graph/TraverserOptions.cpp
View File

@ -25,6 +25,7 @@
#include "Aql/Ast.h" #include "Aql/Ast.h"
#include "Aql/Expression.h" #include "Aql/Expression.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Aql/Query.h" #include "Aql/Query.h"
#include "Basics/VelocyPackHelper.h" #include "Basics/VelocyPackHelper.h"
#include "Cluster/ClusterEdgeCursor.h" #include "Cluster/ClusterEdgeCursor.h"
@ -431,8 +432,8 @@ bool TraverserOptions::hasEdgeFilter(int64_t depth, size_t cursorId) const {
} }
bool TraverserOptions::evaluateEdgeExpression(arangodb::velocypack::Slice edge, bool TraverserOptions::evaluateEdgeExpression(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef vertexId, uint64_t depth, arangodb::velocypack::StringRef vertexId,
size_t cursorId) const { uint64_t depth, size_t cursorId) const {
arangodb::aql::Expression* expression = nullptr; arangodb::aql::Expression* expression = nullptr;
auto specific = _depthLookupInfo.find(depth); auto specific = _depthLookupInfo.find(depth);
@ -493,9 +494,8 @@ bool TraverserOptions::evaluateVertexExpression(arangodb::velocypack::Slice vert
return evaluateExpression(expression, vertex); return evaluateExpression(expression, vertex);
} }
EdgeCursor* arangodb::traverser::TraverserOptions::nextCursor(ManagedDocumentResult* mmdr, EdgeCursor* arangodb::traverser::TraverserOptions::nextCursor(
arangodb::velocypack::StringRef vid, ManagedDocumentResult* mmdr, arangodb::velocypack::StringRef vid, uint64_t depth) {
uint64_t depth) {
if (_isCoordinator) { if (_isCoordinator) {
return nextCursorCoordinator(vid, depth); return nextCursorCoordinator(vid, depth);
} }
@ -510,7 +510,8 @@ EdgeCursor* arangodb::traverser::TraverserOptions::nextCursor(ManagedDocumentRes
return nextCursorLocal(mmdr, vid, list); return nextCursorLocal(mmdr, vid, list);
} }
EdgeCursor* TraverserOptions::nextCursorCoordinator(arangodb::velocypack::StringRef vid, uint64_t depth) { EdgeCursor* TraverserOptions::nextCursorCoordinator(arangodb::velocypack::StringRef vid,
uint64_t depth) {
TRI_ASSERT(_traverser != nullptr); TRI_ASSERT(_traverser != nullptr);
auto cursor = std::make_unique<ClusterEdgeCursor>(vid, depth, this); auto cursor = std::make_unique<ClusterEdgeCursor>(vid, depth, this);
return cursor.release(); return cursor.release();
@ -548,3 +549,13 @@ double TraverserOptions::estimateCost(size_t& nrItems) const {
nrItems = count; nrItems = count;
return cost; return cost;
} }
void TraverserOptions::activatePrune(std::vector<aql::Variable const*> const&& vars,
std::vector<aql::RegisterId> const&& regs,
size_t vertexVarIdx, size_t edgeVarIdx,
size_t pathVarIdx, aql::Expression* expr) {
_pruneExpression =
std::make_unique<aql::PruneExpressionEvaluator>(_trx, _query, std::move(vars),
std::move(regs), vertexVarIdx,
edgeVarIdx, pathVarIdx, expr);
}

23
arangod/Graph/TraverserOptions.h
View File

@ -43,8 +43,10 @@ class Slice;
namespace aql { namespace aql {
struct AstNode; struct AstNode;
class Expression; class Expression;
class PruneExpressionEvaluator;
class Query; class Query;
class TraversalNode; class TraversalNode;
struct Variable;
} // namespace aql } // namespace aql
namespace graph { namespace graph {
@ -71,6 +73,10 @@ struct TraverserOptions : public graph::BaseOptions {
arangodb::traverser::ClusterTraverser* _traverser; arangodb::traverser::ClusterTraverser* _traverser;
/// @brief The condition given in PRUNE (might be empty)
/// The Node keeps responsibility
std::unique_ptr<aql::PruneExpressionEvaluator> _pruneExpression;
public: public:
uint64_t minDepth; uint64_t minDepth;
@ -114,17 +120,30 @@ struct TraverserOptions : public graph::BaseOptions {
bool hasEdgeFilter(int64_t, size_t) const; bool hasEdgeFilter(int64_t, size_t) const;
bool evaluateEdgeExpression(arangodb::velocypack::Slice, arangodb::velocypack::StringRef vertexId, bool evaluateEdgeExpression(arangodb::velocypack::Slice,
arangodb::velocypack::StringRef vertexId,
uint64_t, size_t) const; uint64_t, size_t) const;
bool evaluateVertexExpression(arangodb::velocypack::Slice, uint64_t) const; bool evaluateVertexExpression(arangodb::velocypack::Slice, uint64_t) const;
graph::EdgeCursor* nextCursor(ManagedDocumentResult*, arangodb::velocypack::StringRef vid, uint64_t); graph::EdgeCursor* nextCursor(ManagedDocumentResult*,
arangodb::velocypack::StringRef vid, uint64_t);
void linkTraverser(arangodb::traverser::ClusterTraverser*); void linkTraverser(arangodb::traverser::ClusterTraverser*);
double estimateCost(size_t& nrItems) const override; double estimateCost(size_t& nrItems) const override;
void activatePrune(std::vector<aql::Variable const*> const&& vars,
std::vector<aql::RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx, aql::Expression* expr);
bool usesPrune() const { return _pruneExpression != nullptr; }
aql::PruneExpressionEvaluator* getPruneEvaluator() {
TRI_ASSERT(usesPrune());
return _pruneExpression.get();
}
private: private:
graph::EdgeCursor* nextCursorCoordinator(arangodb::velocypack::StringRef vid, uint64_t); graph::EdgeCursor* nextCursorCoordinator(arangodb::velocypack::StringRef vid, uint64_t);
}; };

350
js/common/modules/@arangodb/aql/explainer.js
View File

@ -6,14 +6,14 @@ var db = require('@arangodb').db,
_ = require('lodash'), _ = require('lodash'),
systemColors = internal.COLORS, systemColors = internal.COLORS,
print = internal.print, print = internal.print,
colors = { }; colors = {};
// max elements to print from array/objects // max elements to print from array/objects
const maxMembersToPrint = 20; const maxMembersToPrint = 20;
let uniqueValue = 0; let uniqueValue = 0;
const anonymize = function(doc) { const anonymize = function (doc) {
if (Array.isArray(doc)) { if (Array.isArray(doc)) {
return doc.map(anonymize); return doc.map(anonymize);
} }
@ -26,7 +26,7 @@ const anonymize = function(doc) {
} }
if (typeof doc === 'object') { if (typeof doc === 'object') {
let result = {}; let result = {};
Object.keys(doc).forEach(function(key) { Object.keys(doc).forEach(function (key) {
if (key.startsWith('_') || key.startsWith('@')) { if (key.startsWith('_') || key.startsWith('@')) {
// This excludes system attributes in examples // This excludes system attributes in examples
// and collections in bindVars // and collections in bindVars
@ -67,20 +67,20 @@ let stringBuilder = {
}; };
/* set colors for output */ /* set colors for output */
function setColors (useSystemColors) { function setColors(useSystemColors) {
'use strict'; 'use strict';
[ 'COLOR_RESET', ['COLOR_RESET',
'COLOR_CYAN', 'COLOR_BLUE', 'COLOR_GREEN', 'COLOR_MAGENTA', 'COLOR_YELLOW', 'COLOR_RED', 'COLOR_WHITE', 'COLOR_CYAN', 'COLOR_BLUE', 'COLOR_GREEN', 'COLOR_MAGENTA', 'COLOR_YELLOW', 'COLOR_RED', 'COLOR_WHITE',
'COLOR_BOLD_CYAN', 'COLOR_BOLD_BLUE', 'COLOR_BOLD_GREEN', 'COLOR_BOLD_MAGENTA', 'COLOR_BOLD_YELLOW', 'COLOR_BOLD_CYAN', 'COLOR_BOLD_BLUE', 'COLOR_BOLD_GREEN', 'COLOR_BOLD_MAGENTA', 'COLOR_BOLD_YELLOW',
'COLOR_BOLD_RED', 'COLOR_BOLD_WHITE' ].forEach(function (c) { 'COLOR_BOLD_RED', 'COLOR_BOLD_WHITE'].forEach(function (c) {
colors[c] = useSystemColors ? systemColors[c] : ''; colors[c] = useSystemColors ? systemColors[c] : '';
}); });
} }
/* colorizer and output helper functions */ /* colorizer and output helper functions */
function bracketize (node, v) { function bracketize(node, v) {
'use strict'; 'use strict';
if (node && node.subNodes && node.subNodes.length > 1) { if (node && node.subNodes && node.subNodes.length > 1) {
return '(' + v + ')'; return '(' + v + ')';
@ -88,22 +88,22 @@ function bracketize (node, v) {
return v; return v;
} }
function attributeUncolored (v) { function attributeUncolored(v) {
'use strict'; 'use strict';
return '`' + v + '`'; return '`' + v + '`';
} }
function keyword (v) { function keyword(v) {
'use strict'; 'use strict';
return colors.COLOR_CYAN + v + colors.COLOR_RESET; return colors.COLOR_CYAN + v + colors.COLOR_RESET;
} }
function annotation (v) { function annotation(v) {
'use strict'; 'use strict';
return colors.COLOR_BLUE + v + colors.COLOR_RESET; return colors.COLOR_BLUE + v + colors.COLOR_RESET;
} }
function value (v) { function value(v) {
'use strict'; 'use strict';
if (typeof v === 'string' && v.length > 1024) { if (typeof v === 'string' && v.length > 1024) {
return colors.COLOR_GREEN + v.substr(0, 1024) + '...' + colors.COLOR_RESET; return colors.COLOR_GREEN + v.substr(0, 1024) + '...' + colors.COLOR_RESET;
@ -111,7 +111,7 @@ function value (v) {
return colors.COLOR_GREEN + v + colors.COLOR_RESET; return colors.COLOR_GREEN + v + colors.COLOR_RESET;
} }
function variable (v) { function variable(v) {
'use strict'; 'use strict';
if (v[0] === '#') { if (v[0] === '#') {
return colors.COLOR_MAGENTA + v + colors.COLOR_RESET; return colors.COLOR_MAGENTA + v + colors.COLOR_RESET;
@ -119,38 +119,38 @@ function variable (v) {
return colors.COLOR_YELLOW + v + colors.COLOR_RESET; return colors.COLOR_YELLOW + v + colors.COLOR_RESET;
} }
function func (v) { function func(v) {
'use strict'; 'use strict';
return colors.COLOR_GREEN + v + colors.COLOR_RESET; return colors.COLOR_GREEN + v + colors.COLOR_RESET;
} }
function collection (v) { function collection(v) {
'use strict'; 'use strict';
return colors.COLOR_RED + v + colors.COLOR_RESET; return colors.COLOR_RED + v + colors.COLOR_RESET;
} }
function view (v) { function view(v) {
'use strict'; 'use strict';
return colors.COLOR_RED + v + colors.COLOR_RESET; return colors.COLOR_RED + v + colors.COLOR_RESET;
} }
function attribute (v) { function attribute(v) {
'use strict'; 'use strict';
return '`' + colors.COLOR_YELLOW + v + colors.COLOR_RESET + '`'; return '`' + colors.COLOR_YELLOW + v + colors.COLOR_RESET + '`';
} }
function header (v) { function header(v) {
'use strict'; 'use strict';
return colors.COLOR_MAGENTA + v + colors.COLOR_RESET; return colors.COLOR_MAGENTA + v + colors.COLOR_RESET;
} }
function section (v) { function section(v) {
'use strict'; 'use strict';
return colors.COLOR_BOLD_BLUE + v + colors.COLOR_RESET; return colors.COLOR_BOLD_BLUE + v + colors.COLOR_RESET;
} }
// return n times ' ' // return n times ' '
function pad (n) { function pad(n) {
'use strict'; 'use strict';
if (n < 0) { if (n < 0) {
// value seems invalid... // value seems invalid...
@ -162,7 +162,7 @@ function pad (n) {
/* print functions */ /* print functions */
/* print query string */ /* print query string */
function printQuery (query) { function printQuery(query) {
'use strict'; 'use strict';
// restrict max length of printed query to avoid endless printing for // restrict max length of printed query to avoid endless printing for
// very long query strings // very long query strings
@ -178,7 +178,7 @@ function printQuery (query) {
} }
/* print write query modification flags */ /* print write query modification flags */
function printModificationFlags (flags) { function printModificationFlags(flags) {
'use strict'; 'use strict';
if (flags === undefined) { if (flags === undefined) {
return; return;
@ -198,7 +198,7 @@ function printModificationFlags (flags) {
} }
/* print optimizer rules */ /* print optimizer rules */
function printRules (rules) { function printRules(rules) {
'use strict'; 'use strict';
stringBuilder.appendLine(section('Optimization rules applied:')); stringBuilder.appendLine(section('Optimization rules applied:'));
@ -215,7 +215,7 @@ function printRules (rules) {
} }
/* print warnings */ /* print warnings */
function printWarnings (warnings) { function printWarnings(warnings) {
'use strict'; 'use strict';
if (!Array.isArray(warnings) || warnings.length === 0) { if (!Array.isArray(warnings) || warnings.length === 0) {
return; return;
@ -231,7 +231,7 @@ function printWarnings (warnings) {
} }
/* print stats */ /* print stats */
function printStats (stats) { function printStats(stats) {
'use strict'; 'use strict';
if (!stats) { if (!stats) {
return; return;
@ -257,7 +257,7 @@ function printStats (stats) {
stringBuilder.appendLine(); stringBuilder.appendLine();
} }
function printProfile (profile) { function printProfile(profile) {
'use strict'; 'use strict';
if (!profile) { if (!profile) {
return; return;
@ -282,7 +282,7 @@ function printProfile (profile) {
} }
/* print indexes used */ /* print indexes used */
function printIndexes (indexes) { function printIndexes(indexes) {
'use strict'; 'use strict';
stringBuilder.appendLine(section('Indexes used:')); stringBuilder.appendLine(section('Indexes used:'));
@ -357,11 +357,11 @@ function printIndexes (indexes) {
} }
} }
function printFunctions (functions) { function printFunctions(functions) {
'use strict'; 'use strict';
let funcArray = []; let funcArray = [];
Object.keys(functions).forEach(function(f) { Object.keys(functions).forEach(function (f) {
funcArray.push(functions[f]); funcArray.push(functions[f]);
}); });
@ -404,8 +404,72 @@ function printFunctions (functions) {
} }
} }
/* create a table with a given amount of columns and arbitrary many rows */
class PrintedTable {
constructor(numColumns) {
this.content = [];
for (let i = 0; i < numColumns; ++i) {
this.content.push({
header: "",
cells: [],
size: 0
});
}
}
setHeader(index, value) {
this.content[index].header = value;
this.content[index].size = Math.max(this.content[index].size, value.length);
print(this.content[index].size, value);
}
addCell(index, value, valueLength) {
valueLength = valueLength || value.length;
this.content[index].cells.push({ formatted: value, size: valueLength });
this.content[index].size = Math.max(this.content[index].size, valueLength);
print(this.content[index].size, value);
}
alignNewEntry() {
let rowsNeeded = Math.max(...this.content.map(c => c.cells.length));
for (let c of this.content) {
while (c.cells.length < rowsNeeded) {
c.cells.push({ formatted: '', size: 0 });
}
}
}
print(builder) {
let rowsNeeded = Math.max(...this.content.map(c => c.cells.length));
// Print the header
let line = ' ';
let isFirst = true;
for (let c of this.content) {
print(c.size, c.header.length);
line += (isFirst ? '' : pad(3)) + header(c.header) + pad(1 + c.size - c.header.length);
isFirst = false;
}
builder.appendLine(line);
// Print the cells
for (let i = 0; i < rowsNeeded; ++i) {
let line = ' ';
let isFirst = true;
for (let c of this.content) {
if (c.cells.length > i) {
line += (isFirst ? '' : pad(3)) + c.cells[i].formatted + pad(1 + c.size - c.cells[i].size);
} else {
line += (isFirst ? '' : pad(3)) + pad(1 + c.size);
}
isFirst = false;
}
builder.appendLine(line);
}
}
}
/* print traversal info */ /* print traversal info */
function printTraversalDetails (traversals) { function printTraversalDetails(traversals) {
'use strict'; 'use strict';
if (traversals.length === 0) { if (traversals.length === 0) {
return; return;
@ -414,14 +478,16 @@ function printTraversalDetails (traversals) {
stringBuilder.appendLine(); stringBuilder.appendLine();
stringBuilder.appendLine(section('Traversals on graphs:')); stringBuilder.appendLine(section('Traversals on graphs:'));
var maxIdLen = String('Id').length; let outTable = new PrintedTable(6);
var maxMinMaxDepth = String('Depth').length; outTable.setHeader(0, 'Id');
var maxVertexCollectionNameStrLen = String('Vertex collections').length; outTable.setHeader(1, 'Depth');
var maxEdgeCollectionNameStrLen = String('Edge collections').length; outTable.setHeader(2, 'Vertex collections');
var maxOptionsLen = String('Options').length; outTable.setHeader(3, 'Edge collections');
var maxConditionsLen = String('Filter conditions').length; outTable.setHeader(4, 'Options');
outTable.setHeader(5, 'Filter / Prune Conditions');
var optify = function(options, colorize) {
var optify = function (options, colorize) {
var opts = { var opts = {
bfs: options.bfs || undefined, /* only print if set to true to space room */ bfs: options.bfs || undefined, /* only print if set to true to space room */
uniqueVertices: options.uniqueVertices, uniqueVertices: options.uniqueVertices,
@ -446,7 +512,7 @@ function printTraversalDetails (traversals) {
} else { } else {
result += att + ': '; result += att + ': ';
if (typeof opts[att] === 'boolean') { if (typeof opts[att] === 'boolean') {
result += (opts[att] ? 'true' : 'false'); result += opts[att] ? 'true' : 'false';
} else { } else {
result += String(opts[att]); result += String(opts[att]);
} }
@ -455,93 +521,30 @@ function printTraversalDetails (traversals) {
return result; return result;
}; };
traversals.forEach(function (node) { traversals.forEach(node => {
var l = String(node.id).length; outTable.alignNewEntry();
if (l > maxIdLen) { outTable.addCell(0, String(node.id));
maxIdLen = l; outTable.addCell(1, node.minMaxDepth);
} outTable.addCell(2, node.vertexCollectionNameStr, node.vertexCollectionNameStrLen);
outTable.addCell(3, node.edgeCollectionNameStr, node.edgeCollectionNameStrLen);
if (node.minMaxDepthLen > maxMinMaxDepth) {
maxMinMaxDepth = node.minMaxDepthLen;
}
if (node.hasOwnProperty('ConditionStr')) {
if (node.ConditionStr.length > maxConditionsLen) {
maxConditionsLen = node.ConditionStr.length;
}
}
if (node.hasOwnProperty('vertexCollectionNameStr')) {
if (node.vertexCollectionNameStrLen > maxVertexCollectionNameStrLen) {
maxVertexCollectionNameStrLen = node.vertexCollectionNameStrLen;
}
}
if (node.hasOwnProperty('edgeCollectionNameStr')) {
if (node.edgeCollectionNameStrLen > maxEdgeCollectionNameStrLen) {
maxEdgeCollectionNameStrLen = node.edgeCollectionNameStrLen;
}
}
if (node.hasOwnProperty('options')) { if (node.hasOwnProperty('options')) {
let opts = optify(node.options); outTable.addCell(4, optify(node.options, true), optify(node.options, false).length);
if (opts.length > maxOptionsLen) {
maxOptionsLen = opts.length;
}
} else if (node.hasOwnProperty('traversalFlags')) { } else if (node.hasOwnProperty('traversalFlags')) {
// Backwards compatibility for < 3.2 outTable.addCell(4, optify(node.traversalFlags, true), optify(node.options, false).length);
let opts = optify(node.traversalFlags);
if (opts.length > maxOptionsLen) {
maxOptionsLen = opts.length;
} }
// else do not add a cell in 4
if (node.hasOwnProperty('ConditionStr')) {
outTable.addCell(5, 'FILTER ' + node.ConditionStr);
}
if (node.hasOwnProperty('PruneConditionStr')) {
outTable.addCell(5, 'PRUNE ' + node.PruneConditionStr);
} }
}); });
outTable.print(stringBuilder);
var line = ' ' + pad(1 + maxIdLen - String('Id').length) + header('Id') + ' ' +
header('Depth') + pad(1 + maxMinMaxDepth - String('Depth').length) + ' ' +
header('Vertex collections') + pad(1 + maxVertexCollectionNameStrLen - 'Vertex collections'.length) + ' ' +
header('Edge collections') + pad(1 + maxEdgeCollectionNameStrLen - 'Edge collections'.length) + ' ' +
header('Options') + pad(1 + maxOptionsLen - 'Options'.length) + ' ' +
header('Filter conditions');
stringBuilder.appendLine(line);
for (var i = 0; i < traversals.length; ++i) {
line = ' ' + pad(1 + maxIdLen - String(traversals[i].id).length) +
traversals[i].id + ' ';
line += traversals[i].minMaxDepth + pad(1 + maxMinMaxDepth - traversals[i].minMaxDepthLen) + ' ';
if (traversals[i].hasOwnProperty('vertexCollectionNameStr')) {
line += traversals[i].vertexCollectionNameStr +
pad(1 + maxVertexCollectionNameStrLen - traversals[i].vertexCollectionNameStrLen) + ' ';
} else {
line += pad(1 + maxVertexCollectionNameStrLen) + ' ';
}
if (traversals[i].hasOwnProperty('edgeCollectionNameStr')) {
line += traversals[i].edgeCollectionNameStr +
pad(1 + maxEdgeCollectionNameStrLen - traversals[i].edgeCollectionNameStrLen) + ' ';
} else {
line += pad(1 + maxEdgeCollectionNameStrLen) + ' ';
}
if (traversals[i].hasOwnProperty('options')) {
line += optify(traversals[i].options, true) + pad(1 + maxOptionsLen - optify(traversals[i].options, false).length) + ' ';
} else if (traversals[i].hasOwnProperty('traversalFlags')) {
line += optify(traversals[i].traversalFlags, true) + pad(1 + maxOptionsLen - optify(traversals[i].traversalFlags, false).length) + ' ';
} else {
line += pad(1 + maxOptionsLen) + ' ';
}
if (traversals[i].hasOwnProperty('ConditionStr')) {
line += traversals[i].ConditionStr;
}
stringBuilder.appendLine(line);
}
} }
/* print shortest_path info */ /* print shortest_path info */
function printShortestPathDetails (shortestPaths) { function printShortestPathDetails(shortestPaths) {
'use strict'; 'use strict';
if (shortestPaths.length === 0) { if (shortestPaths.length === 0) {
return; return;
@ -604,10 +607,10 @@ function printShortestPathDetails (shortestPaths) {
} }
/* analyze and print execution plan */ /* analyze and print execution plan */
function processQuery (query, explain, planIndex) { function processQuery(query, explain, planIndex) {
'use strict'; 'use strict';
var nodes = { }, var nodes = {},
parents = { }, parents = {},
rootNode = null, rootNode = null,
maxTypeLen = 0, maxTypeLen = 0,
maxSiteLen = 0, maxSiteLen = 0,
@ -714,9 +717,9 @@ function processQuery (query, explain, planIndex) {
}); });
} }
var references = { }, var references = {},
collectionVariables = { }, collectionVariables = {},
usedVariables = { }, usedVariables = {},
indexes = [], indexes = [],
traversalDetails = [], traversalDetails = [],
shortestPathDetails = [], shortestPathDetails = [],
@ -755,7 +758,7 @@ function processQuery (query, explain, planIndex) {
return lhs + ' ' + name + ' ' + rhs; return lhs + ' ' + name + ' ' + rhs;
}; };
isConst = isConst && ([ 'value', 'object', 'object element', 'array' ].indexOf(node.type) !== -1); isConst = isConst && (['value', 'object', 'object element', 'array'].indexOf(node.type) !== -1);
switch (node.type) { switch (node.type) {
case 'reference': case 'reference':
@ -823,7 +826,7 @@ function processQuery (query, explain, planIndex) {
case 'expansion': case 'expansion':
if (node.subNodes.length > 2) { if (node.subNodes.length > 2) {
// [FILTER ...] // [FILTER ...]
references[node.subNodes[0].subNodes[0].name] = [ node.levels, node.subNodes[0].subNodes[1], node.subNodes[2], node.subNodes[3], node.subNodes[4]]; references[node.subNodes[0].subNodes[0].name] = [node.levels, node.subNodes[0].subNodes[1], node.subNodes[2], node.subNodes[3], node.subNodes[4]];
} else { } else {
// [*] // [*]
references[node.subNodes[0].subNodes[0].name] = node.subNodes[0].subNodes[1]; references[node.subNodes[0].subNodes[0].name] = node.subNodes[0].subNodes[1];
@ -934,27 +937,27 @@ function processQuery (query, explain, planIndex) {
if (range.equality) { if (range.equality) {
if (range.lowConst.hasOwnProperty('bound')) { if (range.lowConst.hasOwnProperty('bound')) {
results.push(buildBound(attr, [ '==', '==' ], range.lowConst)); results.push(buildBound(attr, ['==', '=='], range.lowConst));
} else if (range.hasOwnProperty('lows')) { } else if (range.hasOwnProperty('lows')) {
range.lows.forEach(function (bound) { range.lows.forEach(function (bound) {
results.push(buildBound(attr, [ '==', '==' ], bound)); results.push(buildBound(attr, ['==', '=='], bound));
}); });
} }
} else { } else {
if (range.lowConst.hasOwnProperty('bound')) { if (range.lowConst.hasOwnProperty('bound')) {
results.push(buildBound(attr, [ '>', '>=' ], range.lowConst)); results.push(buildBound(attr, ['>', '>='], range.lowConst));
} }
if (range.highConst.hasOwnProperty('bound')) { if (range.highConst.hasOwnProperty('bound')) {
results.push(buildBound(attr, [ '<', '<=' ], range.highConst)); results.push(buildBound(attr, ['<', '<='], range.highConst));
} }
if (range.hasOwnProperty('lows')) { if (range.hasOwnProperty('lows')) {
range.lows.forEach(function (bound) { range.lows.forEach(function (bound) {
results.push(buildBound(attr, [ '>', '>=' ], bound)); results.push(buildBound(attr, ['>', '>='], bound));
}); });
} }
if (range.hasOwnProperty('highs')) { if (range.hasOwnProperty('highs')) {
range.highs.forEach(function (bound) { range.highs.forEach(function (bound) {
results.push(buildBound(attr, [ '<', '<=' ], bound)); results.push(buildBound(attr, ['<', '<='], bound));
}); });
} }
} }
@ -1023,7 +1026,7 @@ function processQuery (query, explain, planIndex) {
var scorers = ''; var scorers = '';
if (node.scorers && node.scorers.length > 0) { if (node.scorers && node.scorers.length > 0) {
scorers = keyword(' LET ' ) + node.scorers.map(function(scorer) { scorers = keyword(' LET ') + node.scorers.map(function (scorer) {
return variableName(scorer) + ' = ' + buildExpression(scorer.node); return variableName(scorer) + ' = ' + buildExpression(scorer.node);
}).join(', '); }).join(', ');
} }
@ -1031,8 +1034,8 @@ function processQuery (query, explain, planIndex) {
return keyword('FOR') + ' ' + variableName(node.outVariable) + ' ' + keyword('IN') + ' ' + view(node.view) + condition + scorers + ' ' + annotation('/* view query */'); return keyword('FOR') + ' ' + variableName(node.outVariable) + ' ' + keyword('IN') + ' ' + view(node.view) + condition + scorers + ' ' + annotation('/* view query */');
case 'IndexNode': case 'IndexNode':
collectionVariables[node.outVariable.id] = node.collection; collectionVariables[node.outVariable.id] = node.collection;
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, false); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, false); });
return `${keyword('FOR')} ${variableName(node.outVariable)} ${keyword('IN')} ${collection(node.collection)} ${annotation(`/* ${types.join(', ')}${projection(node)}${node.satellite ? ', satellite':''}${restriction(node)}`)} */`; return `${keyword('FOR')} ${variableName(node.outVariable)} ${keyword('IN')} ${collection(node.collection)} ${annotation(`/* ${types.join(', ')}${projection(node)}${node.satellite ? ', satellite' : ''}${restriction(node)}`)} */`;
//` //`
case 'TraversalNode': case 'TraversalNode':
if (node.hasOwnProperty("options")) { if (node.hasOwnProperty("options")) {
@ -1099,7 +1102,7 @@ function processQuery (query, explain, planIndex) {
} }
} }
allIndexes.sort(function(l, r) { allIndexes.sort(function (l, r) {
if (l.collection !== r.collection) { if (l.collection !== r.collection) {
return l.collection < r.collection ? -1 : 1; return l.collection < r.collection ? -1 : 1;
} }
@ -1133,6 +1136,9 @@ function processQuery (query, explain, planIndex) {
if (node.hasOwnProperty('condition')) { if (node.hasOwnProperty('condition')) {
node.ConditionStr = buildExpression(node.condition); node.ConditionStr = buildExpression(node.condition);
} }
if (node.hasOwnProperty('expression')) {
node.PruneConditionStr = buildExpression(node.expression);
}
e = []; e = [];
if (node.hasOwnProperty('graphDefinition')) { if (node.hasOwnProperty('graphDefinition')) {
@ -1172,7 +1178,7 @@ function processQuery (query, explain, planIndex) {
} }
translate = ['ANY', 'INBOUND', 'OUTBOUND']; translate = ['ANY', 'INBOUND', 'OUTBOUND'];
var defaultDirection = node.directions[0]; var defaultDirection = node.directions[0];
rc = `${keyword("FOR")} ${parts.join(", ")} ${keyword("IN") } ${keyword(translate[defaultDirection])} ${keyword("SHORTEST_PATH") } `; rc = `${keyword("FOR")} ${parts.join(", ")} ${keyword("IN")} ${keyword(translate[defaultDirection])} ${keyword("SHORTEST_PATH")} `;
if (node.hasOwnProperty('startVertexId')) { if (node.hasOwnProperty('startVertexId')) {
rc += `'${value(node.startVertexId)}'`; rc += `'${value(node.startVertexId)}'`;
} else { } else {
@ -1226,7 +1232,7 @@ function processQuery (query, explain, planIndex) {
} }
return rc; return rc;
case 'CalculationNode': case 'CalculationNode':
(node.functions || []).forEach(function(f) { (node.functions || []).forEach(function (f) {
functions[f.name] = f; functions[f.name] = f;
}); });
return keyword('LET') + ' ' + variableName(node.outVariable) + ' = ' + buildExpression(node.expression) + ' ' + annotation('/* ' + node.expressionType + ' expression */'); return keyword('LET') + ' ' + variableName(node.outVariable) + ' = ' + buildExpression(node.expression) + ' ' + annotation('/* ' + node.expressionType + ' expression */');
@ -1294,7 +1300,7 @@ function processQuery (query, explain, planIndex) {
if (node.restrictedTo) { if (node.restrictedTo) {
restrictString = annotation('/* ' + restriction(node) + ' */'); restrictString = annotation('/* ' + restriction(node) + ' */');
} }
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
return `${keyword('UPDATE')} ${inputExplain} ${keyword('IN')} ${collection(node.collection)} ${restrictString}`; return `${keyword('UPDATE')} ${inputExplain} ${keyword('IN')} ${collection(node.collection)} ${restrictString}`;
} }
case 'ReplaceNode': { case 'ReplaceNode': {
@ -1311,13 +1317,13 @@ function processQuery (query, explain, planIndex) {
if (node.restrictedTo) { if (node.restrictedTo) {
restrictString = annotation('/* ' + restriction(node) + ' */'); restrictString = annotation('/* ' + restriction(node) + ' */');
} }
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
return `${keyword('REPLACE')} ${inputExplain} ${keyword('IN')} ${collection(node.collection)} ${restrictString}`; return `${keyword('REPLACE')} ${inputExplain} ${keyword('IN')} ${collection(node.collection)} ${restrictString}`;
} }
case 'UpsertNode': case 'UpsertNode':
modificationFlags = node.modificationFlags; modificationFlags = node.modificationFlags;
let indexRef = `${variableName(node.inDocVariable)}`; let indexRef = `${variableName(node.inDocVariable)}`;
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
return keyword('UPSERT') + ' ' + variableName(node.inDocVariable) + ' ' + keyword('INSERT') + ' ' + variableName(node.insertVariable) + ' ' + keyword(node.isReplace ? 'REPLACE' : 'UPDATE') + ' ' + variableName(node.updateVariable) + ' ' + keyword('IN') + ' ' + collection(node.collection); return keyword('UPSERT') + ' ' + variableName(node.inDocVariable) + ' ' + keyword('INSERT') + ' ' + variableName(node.insertVariable) + ' ' + keyword(node.isReplace ? 'REPLACE' : 'UPDATE') + ' ' + variableName(node.updateVariable) + ' ' + keyword('IN') + ' ' + collection(node.collection);
case 'RemoveNode': { case 'RemoveNode': {
modificationFlags = node.modificationFlags; modificationFlags = node.modificationFlags;
@ -1326,7 +1332,7 @@ function processQuery (query, explain, planIndex) {
restrictString = annotation('/* ' + restriction(node) + ' */'); restrictString = annotation('/* ' + restriction(node) + ' */');
} }
let indexRef = `${variableName(node.inVariable)}`; let indexRef = `${variableName(node.inVariable)}`;
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
return `${keyword('REMOVE')} ${variableName(node.inVariable)} ${keyword('IN')} ${collection(node.collection)} ${restrictString}`; return `${keyword('REMOVE')} ${variableName(node.inVariable)} ${keyword('IN')} ${collection(node.collection)} ${restrictString}`;
} }
case 'SingleRemoteOperationNode': { case 'SingleRemoteOperationNode': {
@ -1334,7 +1340,7 @@ function processQuery (query, explain, planIndex) {
case "IndexNode": { case "IndexNode": {
collectionVariables[node.outVariable.id] = node.collection; collectionVariables[node.outVariable.id] = node.collection;
let indexRef = `${variable(JSON.stringify(node.key))}`; let indexRef = `${variable(JSON.stringify(node.key))}`;
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
return `${keyword('FOR')} ${variableName(node.outVariable)} ${keyword('IN')} ${collection(node.collection)} ${keyword('FILTER')} ${variable('_key')} == ${indexRef} ${annotation(`/* primary index scan */`)}`; return `${keyword('FOR')} ${variableName(node.outVariable)} ${keyword('IN')} ${collection(node.collection)} ${keyword('FILTER')} ${variable('_key')} == ${indexRef} ${annotation(`/* primary index scan */`)}`;
// ` // `
} }
@ -1343,13 +1349,13 @@ function processQuery (query, explain, planIndex) {
collectionVariables[node.inVariable.id] = node.collection; collectionVariables[node.inVariable.id] = node.collection;
let indexRef = `${variableName(node.inVariable)}`; let indexRef = `${variableName(node.inVariable)}`;
if (node.hasOwnProperty('indexes')) { if (node.hasOwnProperty('indexes')) {
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
} }
return `${keyword('INSERT')} ${variableName(node.inVariable)} ${keyword('IN')} ${collection(node.collection)}`; return `${keyword('INSERT')} ${variableName(node.inVariable)} ${keyword('IN')} ${collection(node.collection)}`;
} }
case 'UpdateNode': { case 'UpdateNode': {
modificationFlags = node.modificationFlags; modificationFlags = node.modificationFlags;
let OLD=""; let OLD = "";
if (node.hasOwnProperty('inVariable')) { if (node.hasOwnProperty('inVariable')) {
collectionVariables[node.inVariable.id] = node.collection; collectionVariables[node.inVariable.id] = node.collection;
OLD = `${keyword('WITH')} ${variableName(node.inVariable)} `; OLD = `${keyword('WITH')} ${variableName(node.inVariable)} `;
@ -1369,7 +1375,7 @@ function processQuery (query, explain, planIndex) {
indexRef = "<UNSUPPORTED>"; indexRef = "<UNSUPPORTED>";
} }
if (node.hasOwnProperty('indexes')) { if (node.hasOwnProperty('indexes')) {
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
} }
let forStatement = ""; let forStatement = "";
if (node.replaceIndexNode) { if (node.replaceIndexNode) {
@ -1380,7 +1386,7 @@ function processQuery (query, explain, planIndex) {
} }
case 'ReplaceNode': { case 'ReplaceNode': {
modificationFlags = node.modificationFlags; modificationFlags = node.modificationFlags;
let OLD=""; let OLD = "";
if (node.hasOwnProperty('inVariable')) { if (node.hasOwnProperty('inVariable')) {
collectionVariables[node.inVariable.id] = node.collection; collectionVariables[node.inVariable.id] = node.collection;
OLD = `${keyword('WITH')} ${variableName(node.inVariable)} `; OLD = `${keyword('WITH')} ${variableName(node.inVariable)} `;
@ -1400,7 +1406,7 @@ function processQuery (query, explain, planIndex) {
indexRef = "<UNSUPPORTED>"; indexRef = "<UNSUPPORTED>";
} }
if (node.hasOwnProperty('indexes')) { if (node.hasOwnProperty('indexes')) {
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
} }
let forStatement = ""; let forStatement = "";
if (node.replaceIndexNode) { if (node.replaceIndexNode) {
@ -1429,7 +1435,7 @@ function processQuery (query, explain, planIndex) {
indexRef = "<UNSUPPORTED>"; indexRef = "<UNSUPPORTED>";
} }
if (node.hasOwnProperty('indexes')) { if (node.hasOwnProperty('indexes')) {
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, indexRef); }); node.indexes.forEach(function (idx, i) { iterateIndexes(idx, i, node, types, indexRef); });
} }
let forStatement = ""; let forStatement = "";
if (node.replaceIndexNode) { if (node.replaceIndexNode) {
@ -1450,7 +1456,7 @@ function processQuery (query, explain, planIndex) {
case 'GatherNode': case 'GatherNode':
return keyword('GATHER') + ' ' + node.elements.map(function (node) { return keyword('GATHER') + ' ' + node.elements.map(function (node) {
if (node.path && node.path.length) { if (node.path && node.path.length) {
return variableName(node.inVariable) + node.path.map(function(n) { return '.' + attribute(n); }) + ' ' + keyword(node.ascending ? 'ASC' : 'DESC'); return variableName(node.inVariable) + node.path.map(function (n) { return '.' + attribute(n); }) + ' ' + keyword(node.ascending ? 'ASC' : 'DESC');
} }
return variableName(node.inVariable) + ' ' + keyword(node.ascending ? 'ASC' : 'DESC'); return variableName(node.inVariable) + ' ' + keyword(node.ascending ? 'ASC' : 'DESC');
}).join(', ') + (node.sortmode === 'unset' ? '' : ' ' + annotation('/* sort mode: ' + node.sortmode + ' */')); }).join(', ') + (node.sortmode === 'unset' ? '' : ' ' + annotation('/* sort mode: ' + node.sortmode + ' */'));
@ -1465,7 +1471,7 @@ function processQuery (query, explain, planIndex) {
}; };
var preHandle = function (node) { var preHandle = function (node) {
usedVariables = { }; usedVariables = {};
currentNode = node.id; currentNode = node.id;
isConst = true; isConst = true;
if (node.type === 'SubqueryNode') { if (node.type === 'SubqueryNode') {
@ -1476,13 +1482,13 @@ function processQuery (query, explain, planIndex) {
var postHandle = function (node) { var postHandle = function (node) {
var isLeafNode = !parents.hasOwnProperty(node.id); var isLeafNode = !parents.hasOwnProperty(node.id);
if ([ 'EnumerateCollectionNode', if (['EnumerateCollectionNode',
'EnumerateListNode', 'EnumerateListNode',
'EnumerateViewNode', 'EnumerateViewNode',
'IndexRangeNode', 'IndexRangeNode',
'IndexNode', 'IndexNode',
'TraversalNode', 'TraversalNode',
'SubqueryNode' ].indexOf(node.type) !== -1) { 'SubqueryNode'].indexOf(node.type) !== -1) {
level++; level++;
} else if (isLeafNode && subqueries.length > 0) { } else if (isLeafNode && subqueries.length > 0) {
level = subqueries.pop(); level = subqueries.pop();
@ -1566,7 +1572,7 @@ function processQuery (query, explain, planIndex) {
stringBuilder.appendLine(line); stringBuilder.appendLine(line);
var walk = [ rootNode ]; var walk = [rootNode];
while (walk.length > 0) { while (walk.length > 0) {
var id = walk.pop(); var id = walk.pop();
var node = nodes[id]; var node = nodes[id];
@ -1575,7 +1581,7 @@ function processQuery (query, explain, planIndex) {
walk = walk.concat(parents[id]); walk = walk.concat(parents[id]);
} }
if (node.type === 'SubqueryNode') { if (node.type === 'SubqueryNode') {
walk = walk.concat([ node.subquery.nodes[0].id ]); walk = walk.concat([node.subquery.nodes[0].id]);
} }
} }
@ -1599,7 +1605,7 @@ function processQuery (query, explain, planIndex) {
function explain(data, options, shouldPrint) { function explain(data, options, shouldPrint) {
'use strict'; 'use strict';
if (typeof data === 'string') { if (typeof data === 'string') {
data = { query: data, options:options }; data = { query: data, options: options };
} }
if (!(data instanceof Object)) { if (!(data instanceof Object)) {
throw 'ArangoStatement needs initial data'; throw 'ArangoStatement needs initial data';
@ -1608,7 +1614,7 @@ function explain(data, options, shouldPrint) {
if (options === undefined) { if (options === undefined) {
options = data.options; options = data.options;
} }
options = options || { }; options = options || {};
options.verbosePlans = true; options.verbosePlans = true;
setColors(options.colors === undefined ? true : options.colors); setColors(options.colors === undefined ? true : options.colors);
@ -1647,7 +1653,7 @@ function profileQuery(data, shouldPrint) {
if (!(data instanceof Object) || !data.hasOwnProperty("options")) { if (!(data instanceof Object) || !data.hasOwnProperty("options")) {
throw 'ArangoStatement needs initial data'; throw 'ArangoStatement needs initial data';
} }
let options = data.options || { }; let options = data.options || {};
options.silent = true; options.silent = true;
options.allPlans = false; // always turn this off, as it will not work with profiling options.allPlans = false; // always turn this off, as it will not work with profiling
setColors(options.colors === undefined ? true : options.colors); setColors(options.colors === undefined ? true : options.colors);
@ -1704,36 +1710,36 @@ function debug(query, bindVars, options) {
let graphs = {}; let graphs = {};
let collections = result.explain.plan.collections; let collections = result.explain.plan.collections;
let map = {}; let map = {};
collections.forEach(function(c) { collections.forEach(function (c) {
map[c.name] = true; map[c.name] = true;
}); });
// export graphs // export graphs
let findGraphs = function(nodes) { let findGraphs = function (nodes) {
nodes.forEach(function(node) { nodes.forEach(function (node) {
if (node.type === 'TraversalNode') { if (node.type === 'TraversalNode') {
if (node.graph) { if (node.graph) {
try { try {
graphs[node.graph] = db._graphs.document(node.graph); graphs[node.graph] = db._graphs.document(node.graph);
} catch (err) {} } catch (err) { }
} }
if (node.graphDefinition) { if (node.graphDefinition) {
try { try {
node.graphDefinition.vertexCollectionNames.forEach(function(c) { node.graphDefinition.vertexCollectionNames.forEach(function (c) {
if (!map.hasOwnProperty(c)) { if (!map.hasOwnProperty(c)) {
map[c] = true; map[c] = true;
collections.push({ name: c }); collections.push({ name: c });
} }
}); });
} catch (err) {} } catch (err) { }
try { try {
node.graphDefinition.edgeCollectionNames.forEach(function(c) { node.graphDefinition.edgeCollectionNames.forEach(function (c) {
if (!map.hasOwnProperty(c)) { if (!map.hasOwnProperty(c)) {
map[c] = true; map[c] = true;
collections.push({ name: c }); collections.push({ name: c });
} }
}); });
} catch (err) {} } catch (err) { }
} }
} else if (node.type === 'SubqueryNode') { } else if (node.type === 'SubqueryNode') {
// recurse into subqueries // recurse into subqueries
@ -1745,7 +1751,7 @@ function debug(query, bindVars, options) {
findGraphs(result.explain.plan.nodes); findGraphs(result.explain.plan.nodes);
// add collection information // add collection information
collections.forEach(function(collection) { collections.forEach(function (collection) {
let c = db._collection(collection.name); let c = db._collection(collection.name);
if (c === null) { if (c === null) {
// probably a view... // probably a view...
@ -1815,7 +1821,7 @@ function inspectDump(filename, outfile) {
print("/* graphs */"); print("/* graphs */");
let graphs = data.graphs || {}; let graphs = data.graphs || {};
Object.keys(graphs).forEach(function(graph) { Object.keys(graphs).forEach(function (graph) {
let details = graphs[graph]; let details = graphs[graph];
print("try { db._graphs.remove(" + JSON.stringify(graph) + "); } catch (err) {}"); print("try { db._graphs.remove(" + JSON.stringify(graph) + "); } catch (err) {}");
print("db._graphs.insert(" + JSON.stringify(details) + ");"); print("db._graphs.insert(" + JSON.stringify(details) + ");");
@ -1824,7 +1830,7 @@ function inspectDump(filename, outfile) {
// all collections and indexes first, as data insertion may go wrong later // all collections and indexes first, as data insertion may go wrong later
print("/* collections and indexes setup */"); print("/* collections and indexes setup */");
Object.keys(data.collections).forEach(function(collection) { Object.keys(data.collections).forEach(function (collection) {
let details = data.collections[collection]; let details = data.collections[collection];
print("db._drop(" + JSON.stringify(collection) + ");"); print("db._drop(" + JSON.stringify(collection) + ");");
if (details.type === false || details.type === 3) { if (details.type === false || details.type === 3) {
@ -1832,7 +1838,7 @@ function inspectDump(filename, outfile) {
} else { } else {
print("db._create(" + JSON.stringify(collection) + ", " + JSON.stringify(details.properties) + ");"); print("db._create(" + JSON.stringify(collection) + ", " + JSON.stringify(details.properties) + ");");
} }
details.indexes.forEach(function(index) { details.indexes.forEach(function (index) {
delete index.figures; delete index.figures;
delete index.selectivityEstimate; delete index.selectivityEstimate;
if (index.type !== 'primary' && index.type !== 'edge') { if (index.type !== 'primary' && index.type !== 'edge') {
@ -1845,10 +1851,10 @@ function inspectDump(filename, outfile) {
// insert example data // insert example data
print("/* example data */"); print("/* example data */");
Object.keys(data.collections).forEach(function(collection) { Object.keys(data.collections).forEach(function (collection) {
let details = data.collections[collection]; let details = data.collections[collection];
if (details.examples) { if (details.examples) {
details.examples.forEach(function(example) { details.examples.forEach(function (example) {
print("db[" + JSON.stringify(collection) + "].insert(" + JSON.stringify(example) + ");"); print("db[" + JSON.stringify(collection) + "].insert(" + JSON.stringify(example) + ");");
}); });
} }
@ -1865,7 +1871,7 @@ function inspectDump(filename, outfile) {
// views // views
print("/* views */"); print("/* views */");
Object.keys(data.views || {}).forEach(function(view) { Object.keys(data.views || {}).forEach(function (view) {
let details = data.views[view]; let details = data.views[view];
print("db._dropView(" + JSON.stringify(view) + ");"); print("db._dropView(" + JSON.stringify(view) + ");");
print("db._createView(" + JSON.stringify(view) + ", " + JSON.stringify(details.type) + ", " + JSON.stringify(details.properties) + ");"); print("db._createView(" + JSON.stringify(view) + ", " + JSON.stringify(details.type) + ", " + JSON.stringify(details.properties) + ");");
@ -1873,7 +1879,7 @@ function inspectDump(filename, outfile) {
print(); print();
print("/* explain result */"); print("/* explain result */");
print(data.fancy.trim().split(/\n/).map(function(line) { return "// " + line; }).join("\n")); print(data.fancy.trim().split(/\n/).map(function (line) { return "// " + line; }).join("\n"));
print(); print();
print("/* explain command */"); print("/* explain command */");

826
tests/js/server/aql/aql-graph-traverser.js
View File

File diff suppressed because it is too large Load Diff