Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Feature/aql traversal prune (#8255) 

Allow PRUNE in the AQL Traverser.
This commit is contained in:
Michael Hackstein 2019-02-27 11:43:06 +01:00 committed by GitHub
parent b04cd607f4
commit 6e8d43b2da
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
31 changed files with 3357 additions and 2229 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
CHANGELOG
View File

@ -1,6 +1,17 @@
devel
-----
* add "PRUNE <condition>" to AQL Traversals. This allows to early abort searching of
unnecessary branches within a traversal.
PRUNE is only allowed in the Traversal statement and only between the graphdefinition
and the options of the traversal.
e.g.:
FOR v, e, p IN 1..3 OUTBOUND @source GRAPH "myGraph"
PRUNE v.value == "bar"
OPTIONS {} /* These options remain optional */
RETURN v
for more details refer to the documentation chapter.
* fixed a display issues when editing a graph within the web ui
* fixed some escaping issues within the web ui.

74
Documentation/Books/AQL/Graphs/Traversals.md
View File

@ -17,6 +17,7 @@ FOR vertex[, edge[, path]]
IN [min[..max]]
OUTBOUND|INBOUND|ANY startVertex
GRAPH graphName
[PRUNE pruneCondition]
[OPTIONS options]
```
- `WITH`: optional for single server instances, but required for
@ -48,6 +49,22 @@ FOR vertex[, edge[, path]]
- `GRAPH` **graphName** (string): the name identifying the named graph.
Its vertex and edge collections will be looked up. Note that the graph name
is like a regular string, hence it must be enclosed by quote marks.
- `PRUNE` **condition** (AQL condition, *optional*, (since version 3.4.5)):
A condition, like in a FILTER statement, which will be evaluated in every step
of the traversal, as early as possible. The semantics of this condition is as follows:
* If the condition evaluates to `true` this path will be considered as a result,
it might still be post filtered or ignored due to depth constraints. However
the search will not continue from this path, namely there will be no
result having this path as a prefix.
e.g.: Take the path: `(A) -> (B) -> (C)` starting at `A` and PRUNE on `B`
will result in `(A)` and `(A) -> (B)` beeing valid paths, and `(A) -> (B) -> (C)`
not returned, it got pruned on B.
* If the condition evaluates to `false` we will continue our search beyond
this path.
There is only one `PRUNE` condition possible, but it can contain an arbitrary amount
of `AND` or `OR` statements.
Also note that you can use the output variables of this traversal in the `PRUNE`,
as well as all variables defined before this Traversal statement.
- `OPTIONS` **options** (object, *optional*): used to modify the execution of the
traversal. Only the following attributes have an effect, all others are ignored:
- **uniqueVertices** (string): optionally ensure vertex uniqueness
@ -79,6 +96,7 @@ FOR vertex[, edge[, path]]
IN [min[..max]]
OUTBOUND|INBOUND|ANY startVertex
edgeCollection1, ..., edgeCollectionN
[PRUNE pruneCondition]
[OPTIONS options]
```
@ -138,10 +156,62 @@ combined filters cannot.
The following examples are based on the [traversal graph](../../Manual/Graphs/index.html#the-traversal-graph).
### Pruning (since version 3.4.5)
Pruning is the easiest variant to formulate conditions to reduce the amount of data
to be checked during a search. So it allows to improve query performance and reduces
the amount of overhead generated by the query. Pruning can be executed on the
vertex, the edge and the path and any variable defined before.
See examples:
@startDocuBlockInline GRAPHTRAV_graphPruneEdges
@EXAMPLE_AQL{GRAPHTRAV_graphFilterEdges}
@DATASET{traversalGraph}
FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'
PRUNE e.theTruth == true
RETURN { vertices: p.vertices[*]._key, edges: p.edges[*].label }
@END_EXAMPLE_AQL
@endDocuBlock GRAPHTRAV_graphPruneEdges
This will search until it sees an edge having `theTruth == true`.
The path with this edge will be returned, the search will not
continue after this edge.
Namely all responses either have no edge with `theTruth == true`
or the last edge on the path has `theTruth == true`.
@startDocuBlockInline GRAPHTRAV_graphPruneVertices
@EXAMPLE_AQL{GRAPHTRAV_graphFilterEdges}
@DATASET{traversalGraph}
FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'
PRUNE v._key == 'G'
FILTER v._key == 'G'
RETURN { vertices: p.vertices[*]._key, edges: p.edges[*].label }
@END_EXAMPLE_AQL
@endDocuBlock GRAPHTRAV_graphPruneVertices
This will search for all paths from the source `circles/A` to the vertex `circles/G`.
This is done with first the PRUNE which makes sure we stop search as soon as we have found
`G` and we will not go beyond `G` and via a loop return to it.
With the second filter, we remove all paths that do not end in `G` namely
all shorter ones that have not been cut out by prune.
Hence the list of all paths from `A` to `G` are returned.
Note you can also prune as soon as you reach a certain collection with the following
example:
@startDocuBlockInline GRAPHTRAV_graphPruneCollection
@EXAMPLE_AQL{GRAPHTRAV_graphFilterEdges}
@DATASET{traversalGraph}
FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'
PRUNE IS_SAME_COLLECTION('circles', v)
RETURN { vertices: p.vertices[*]._key, edges: p.edges[*].label }
@END_EXAMPLE_AQL
@endDocuBlock GRAPHTRAV_graphPruneCollection
### Filtering on paths
Filtering on paths allows for the most powerful filtering and may have the
highest impact on performance. Using the path variable you can filter on
Filtering on paths allows for the second most powerful filtering and may have the
second highest impact on performance. Using the path variable you can filter on
specific iteration depths. You can filter for absolute positions in the path
by specifying a positive number (which then qualifies for the optimizations),
or relative positions to the end of the path by specifying a negative number.

140
arangod/Aql/Ast.cpp
View File

@ -1304,129 +1304,65 @@ AstNode* Ast::createNodeCollectionDirection(uint64_t direction, AstNode const* c
return node;
}
/// @brief create an AST traversal node with only vertex variable
AstNode* Ast::createNodeTraversal(char const* vertexVarName, size_t vertexVarLength,
AstNode const* direction, AstNode const* start,
AstNode const* graph, AstNode const* options) {
if (vertexVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
/// @brief create an AST traversal node
AstNode* Ast::createNodeTraversal(AstNode const* outVars, AstNode const* graphInfo) {
TRI_ASSERT(outVars->type == NODE_TYPE_ARRAY);
TRI_ASSERT(graphInfo->type == NODE_TYPE_ARRAY);
AstNode* node = createNode(NODE_TYPE_TRAVERSAL);
node->reserve(5);
node->reserve(outVars->numMembers() + graphInfo->numMembers());
if (options == nullptr) {
// no options given. now use default options
options = &NopNode;
TRI_ASSERT(graphInfo->numMembers() == 5);
TRI_ASSERT(outVars->numMembers() > 0);
TRI_ASSERT(outVars->numMembers() < 4);
// Add GraphInfo
for (size_t i = 0; i < graphInfo->numMembers(); ++i) {
node->addMember(graphInfo->getMemberUnchecked(i));
}
node->addMember(direction);
node->addMember(start);
node->addMember(graph);
node->addMember(options);
AstNode* vertexVar = createNodeVariable(vertexVarName, vertexVarLength, false);
node->addMember(vertexVar);
TRI_ASSERT(node->numMembers() == 5);
// Add variables
for (size_t i = 0; i < outVars->numMembers(); ++i) {
node->addMember(outVars->getMemberUnchecked(i));
}
TRI_ASSERT(node->numMembers() == graphInfo->numMembers() + outVars->numMembers());
_containsTraversal = true;
return node;
}
/// @brief create an AST traversal node with vertex and edge variable
AstNode* Ast::createNodeTraversal(char const* vertexVarName, size_t vertexVarLength,
char const* edgeVarName, size_t edgeVarLength,
AstNode const* direction, AstNode const* start,
AstNode const* graph, AstNode const* options) {
if (edgeVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
AstNode* node = createNodeTraversal(vertexVarName, vertexVarLength, direction,
start, graph, options);
AstNode* edgeVar = createNodeVariable(edgeVarName, edgeVarLength, false);
node->addMember(edgeVar);
TRI_ASSERT(node->numMembers() == 6);
_containsTraversal = true;
return node;
}
/// @brief create an AST traversal node with vertex, edge and path variable
AstNode* Ast::createNodeTraversal(char const* vertexVarName, size_t vertexVarLength,
char const* edgeVarName, size_t edgeVarLength,
char const* pathVarName, size_t pathVarLength,
AstNode const* direction, AstNode const* start,
AstNode const* graph, AstNode const* options) {
if (pathVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
AstNode* node = createNodeTraversal(vertexVarName, vertexVarLength, edgeVarName,
edgeVarLength, direction, start, graph, options);
AstNode* pathVar = createNodeVariable(pathVarName, pathVarLength, false);
node->addMember(pathVar);
TRI_ASSERT(node->numMembers() == 7);
_containsTraversal = true;
return node;
}
/// @brief create an AST shortest path node with only vertex variable
AstNode* Ast::createNodeShortestPath(char const* vertexVarName,
size_t vertexVarLength, uint64_t direction,
AstNode const* start, AstNode const* target,
AstNode const* graph, AstNode const* options) {
if (vertexVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
}
/// @brief create an AST shortest path node
AstNode* Ast::createNodeShortestPath(AstNode const* outVars, AstNode const* graphInfo) {
TRI_ASSERT(outVars->type == NODE_TYPE_ARRAY);
TRI_ASSERT(graphInfo->type == NODE_TYPE_ARRAY);
AstNode* node = createNode(NODE_TYPE_SHORTEST_PATH);
node->reserve(outVars->numMembers() + graphInfo->numMembers());
node->reserve(6);
TRI_ASSERT(graphInfo->numMembers() == 5);
TRI_ASSERT(outVars->numMembers() > 0);
TRI_ASSERT(outVars->numMembers() < 3);
if (options == nullptr) {
// no options given. now use default options
options = &NopNode;
// Add GraphInfo
for (size_t i = 0; i < graphInfo->numMembers(); ++i) {
node->addMember(graphInfo->getMemberUnchecked(i));
}
AstNode* dir = createNodeValueInt(direction);
node->addMember(dir);
node->addMember(start);
node->addMember(target);
node->addMember(graph);
node->addMember(options);
AstNode* vertexVar = createNodeVariable(vertexVarName, vertexVarLength, false);
node->addMember(vertexVar);
// Add variables
for (size_t i = 0; i < outVars->numMembers(); ++i) {
node->addMember(outVars->getMemberUnchecked(i));
}
TRI_ASSERT(node->numMembers() == graphInfo->numMembers() + outVars->numMembers());
TRI_ASSERT(node->numMembers() == 6);
_containsTraversal = true;
return node;
}
/// @brief create an AST shortest path node with vertex and edge variable
AstNode* Ast::createNodeShortestPath(char const* vertexVarName,
size_t vertexVarLength, char const* edgeVarName,
size_t edgeVarLength, uint64_t direction,
AstNode const* start, AstNode const* target,
AstNode const* graph, AstNode const* options) {
if (edgeVarName == nullptr) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_OUT_OF_MEMORY);
AstNode const* Ast::createNodeOptions(AstNode const* options) const {
if (options != nullptr) {
return options;
}
AstNode* node = createNodeShortestPath(vertexVarName, vertexVarLength, direction,
start, target, graph, options);
AstNode* edgeVar = createNodeVariable(edgeVarName, edgeVarLength, false);
node->addMember(edgeVar);
TRI_ASSERT(node->numMembers() == 7);
return node;
return &NopNode;
}
/// @brief create an AST function call node

27
arangod/Aql/Ast.h
View File

@ -355,27 +355,16 @@ class Ast {
/// @brief create an AST direction node
AstNode* createNodeCollectionDirection(uint64_t, AstNode const*);
/// @brief create an AST traversal node with only vertex variable
AstNode* createNodeTraversal(char const*, size_t, AstNode const*,
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST options node:
// Will either return Noop noed, if the input is nullptr
// Otherwise return the input node.
AstNode const* createNodeOptions(AstNode const*) const;
/// @brief create an AST traversal node with vertex and edge variable
AstNode* createNodeTraversal(char const*, size_t, char const*, size_t, AstNode const*,
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST traversal node
AstNode* createNodeTraversal(AstNode const*, AstNode const*);
/// @brief create an AST traversal node with vertex, edge and path variable
AstNode* createNodeTraversal(char const*, size_t, char const*, size_t,
char const*, size_t, AstNode const*,
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST shortest path node with only vertex variable
AstNode* createNodeShortestPath(char const*, size_t, uint64_t, AstNode const*,
AstNode const*, AstNode const*, AstNode const*);
/// @brief create an AST shortest path node with vertex and edge variable
AstNode* createNodeShortestPath(char const*, size_t, char const*, size_t,
uint64_t, AstNode const*, AstNode const*,
AstNode const*, AstNode const*);
/// @brief create an AST shortest path node
AstNode* createNodeShortestPath(AstNode const*, AstNode const*);
/// @brief create an AST function call node
AstNode* createNodeFunctionCall(char const* functionName, AstNode const* arguments) {

33
arangod/Aql/ExecutionPlan.cpp
View File

@ -206,6 +206,13 @@ std::unique_ptr<graph::BaseOptions> createShortestPathOptions(arangodb::aql::Que
return ret;
}
std::unique_ptr<Expression> createPruneExpression(ExecutionPlan* plan, Ast* ast, AstNode* node) {
if (node->type == NODE_TYPE_NOP) {
return nullptr;
}
return std::make_unique<Expression>(plan, ast, node);
}
} // namespace
/// @brief create the plan
@ -980,11 +987,11 @@ ExecutionNode* ExecutionPlan::fromNodeForView(ExecutionNode* previous, AstNode c
/// @brief create an execution plan element from an AST FOR TRAVERSAL node
ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode const* node) {
TRI_ASSERT(node != nullptr && node->type == NODE_TYPE_TRAVERSAL);
TRI_ASSERT(node->numMembers() >= 5);
TRI_ASSERT(node->numMembers() <= 7);
TRI_ASSERT(node->numMembers() >= 6);
TRI_ASSERT(node->numMembers() <= 8);
// the first 3 members are used by traversal internally.
// The members 4-6, where 5 and 6 are optional, are used
// the first 5 members are used by traversal internally.
// The members 6-8, where 5 and 6 are optional, are used
// as out variables.
AstNode const* direction = node->getMember(0);
AstNode const* start = node->getMember(1);
@ -1009,8 +1016,11 @@ ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode
previous = calc;
}
// Prune Expression
std::unique_ptr<Expression> pruneExpression = createPruneExpression(this, _ast, node->getMember(3));
auto options =
createTraversalOptions(getAst()->query(), direction, node->getMember(3));
createTraversalOptions(getAst()->query(), direction, node->getMember(4));
TRI_ASSERT(direction->type == NODE_TYPE_DIRECTION);
TRI_ASSERT(direction->numMembers() == 2);
@ -1019,24 +1029,25 @@ ExecutionNode* ExecutionPlan::fromNodeTraversal(ExecutionNode* previous, AstNode
// First create the node
auto travNode = new TraversalNode(this, nextId(), &(_ast->query()->vocbase()),
direction, start, graph, std::move(options));
direction, start, graph,
std::move(pruneExpression), std::move(options));
auto variable = node->getMember(4);
auto variable = node->getMember(5);
TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE);
auto v = static_cast<Variable*>(variable->getData());
TRI_ASSERT(v != nullptr);
travNode->setVertexOutput(v);
if (node->numMembers() > 5) {
if (node->numMembers() > 6) {
// return the edge as well
variable = node->getMember(5);
variable = node->getMember(6);
TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE);
v = static_cast<Variable*>(variable->getData());
TRI_ASSERT(v != nullptr);
travNode->setEdgeOutput(v);
if (node->numMembers() > 6) {
if (node->numMembers() > 7) {
// return the path as well
variable = node->getMember(6);
variable = node->getMember(7);
TRI_ASSERT(variable->type == NODE_TYPE_VARIABLE);
v = static_cast<Variable*>(variable->getData());
TRI_ASSERT(v != nullptr);

123
arangod/Aql/InAndOutRowExpressionContext.cpp Normal file
View File

@ -0,0 +1,123 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#include "InAndOutRowExpressionContext.h"
#include "Aql/AqlItemBlock.h"
#include "Aql/AqlValue.h"
#include "Aql/Variable.h"
#include "Basics/Exceptions.h"
#include <limits.h>
using namespace arangodb;
using namespace arangodb::aql;
InAndOutRowExpressionContext::InAndOutRowExpressionContext(
Query* query, std::vector<Variable const*> const&& vars,
std::vector<RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx)
: QueryExpressionContext(query),
_input{CreateInvalidInputRowHint()},
_vars(std::move(vars)),
_regs(std::move(regs)),
_vertexVarIdx(vertexVarIdx),
_edgeVarIdx(edgeVarIdx),
_pathVarIdx(pathVarIdx) {
TRI_ASSERT(_vars.size() == _regs.size());
TRI_ASSERT(_vertexVarIdx < _vars.size() ||
_vertexVarIdx == std::numeric_limits<std::size_t>::max());
TRI_ASSERT(_edgeVarIdx < _vars.size() ||
_edgeVarIdx == std::numeric_limits<std::size_t>::max());
TRI_ASSERT(_pathVarIdx < _vars.size() ||
_pathVarIdx == std::numeric_limits<std::size_t>::max());
}
void InAndOutRowExpressionContext::setInputRow(InputAqlItemRow input) {
TRI_ASSERT(input.isInitialized());
_input = input;
}
AqlValue const& InAndOutRowExpressionContext::getRegisterValue(size_t i) const {
TRI_ASSERT(_input.isInitialized());
TRI_ASSERT(i < _regs.size());
if (i == _vertexVarIdx) {
return _vertexValue;
}
if (i == _edgeVarIdx) {
return _edgeValue;
}
if (i == _pathVarIdx) {
return _pathValue;
}
// Search InputRow
RegisterId const& regId = _regs[i];
TRI_ASSERT(regId < _input.getNrRegisters());
return _input.getValue(regId);
}
AqlValue InAndOutRowExpressionContext::getVariableValue(Variable const* variable, bool doCopy,
bool& mustDestroy) const {
TRI_ASSERT(_input.isInitialized());
for (size_t i = 0; i < _vars.size(); ++i) {
auto const& v = _vars[i];
if (v->id == variable->id) {
TRI_ASSERT(i < _regs.size());
if (doCopy) {
mustDestroy = true;
if (i == _vertexVarIdx) {
return _vertexValue.clone();
}
if (i == _edgeVarIdx) {
return _edgeValue.clone();
}
if (i == _pathVarIdx) {
return _pathValue.clone();
}
// Search InputRow
RegisterId const& regId = _regs[i];
TRI_ASSERT(regId < _input.getNrRegisters());
return _input.getValue(regId).clone();
} else {
mustDestroy = false;
if (i == _vertexVarIdx) {
return _vertexValue;
}
if (i == _edgeVarIdx) {
return _edgeValue;
}
if (i == _pathVarIdx) {
return _pathValue;
}
// Search InputRow
RegisterId const& regId = _regs[i];
TRI_ASSERT(regId < _input.getNrRegisters());
return _input.getValue(regId);
}
}
}
std::string msg("variable not found '");
msg.append(variable->name);
// NOTE: PRUNE is the only feature using this context.
msg.append("' in PRUNE statement");
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_INTERNAL, msg.c_str());
}

99
arangod/Aql/InAndOutRowExpressionContext.h Normal file
View File

@ -0,0 +1,99 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#ifndef ARANGOD_AQL_IN_AND_OUT_ROW_EXPRESSION_CONTEXT_H
#define ARANGOD_AQL_IN_AND_OUT_ROW_EXPRESSION_CONTEXT_H 1
#include "QueryExpressionContext.h"
#include "Aql/AqlValue.h"
#include "Aql/InputAqlItemRow.h"
namespace arangodb {
namespace aql {
class Query;
/**
* @brief Context for expression evaluation that allows
* to read registers from Input and Output.
* This is useful for expressions that need
* to evaluate variables created by the active block.
* User needs to make sure that outputblock is written to!
*/
class InAndOutRowExpressionContext final : public QueryExpressionContext {
public:
InAndOutRowExpressionContext(Query* query, std::vector<Variable const*> const&& vars,
std::vector<RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx);
~InAndOutRowExpressionContext() {}
void setInputRow(InputAqlItemRow input);
size_t numRegisters() const override { return _regs.size(); }
AqlValue const& getRegisterValue(size_t i) const override;
Variable const* getVariable(size_t i) const override { return _vars[i]; }
AqlValue getVariableValue(Variable const* variable, bool doCopy,
bool& mustDestroy) const override;
bool needsVertexValue() const { return _vertexVarIdx < _regs.size(); }
bool needsEdgeValue() const { return _edgeVarIdx < _regs.size(); }
bool needsPathValue() const { return _pathVarIdx < _regs.size(); }
/// @brief inject the result value when asked for the Vertex data
/// This will not copy ownership of slice content. caller needs to make sure
/// that the buffer stays valid until evaluate is called
void setVertexValue(velocypack::Slice v) {
_vertexValue = AqlValue(AqlValueHintDocumentNoCopy(v.begin()));
}
/// @brief inject the result value when asked for the Edge data
/// This will not copy ownership of slice content. caller needs to make sure
/// that the buffer stays valid until evaluate is called
void setEdgeValue(velocypack::Slice e) {
_edgeValue = AqlValue(AqlValueHintDocumentNoCopy(e.begin()));
}
/// @brief inject the result value when asked for the Path data
/// This will not copy ownership of slice content. caller needs to make sure
/// that the buffer stays valid until evaluate is called
void setPathValue(velocypack::Slice p) {
_pathValue = AqlValue(AqlValueHintDocumentNoCopy(p.begin()));
}
private:
InputAqlItemRow _input;
std::vector<Variable const*> const _vars;
std::vector<RegisterId> const _regs;
size_t const _vertexVarIdx;
size_t const _edgeVarIdx;
size_t const _pathVarIdx;
AqlValue _vertexValue;
AqlValue _edgeValue;
AqlValue _pathValue;
};
} // namespace aql
} // namespace arangodb
#endif

16
arangod/Aql/OptimizerRules.cpp
View File

@ -22,6 +22,7 @@
/// @author Jan Steemann
////////////////////////////////////////////////////////////////////////////////
#include "OptimizerRules.h"
#include "Aql/AqlItemBlock.h"
#include "Aql/ClusterNodes.h"
#include "Aql/CollectNode.h"
@ -54,7 +55,6 @@
#include "GeoIndex/Index.h"
#include "Graph/TraverserOptions.h"
#include "Indexes/Index.h"
#include "OptimizerRules.h"
#include "StorageEngine/EngineSelectorFeature.h"
#include "StorageEngine/StorageEngine.h"
#include "Transaction/Methods.h"
@ -5884,14 +5884,19 @@ void arangodb::aql::optimizeTraversalsRule(Optimizer* opt,
// yet, as many traversal internals depend on the number of vertices
// found/built
auto outVariable = traversal->edgeOutVariable();
if (outVariable != nullptr && !n->isVarUsedLater(outVariable)) {
std::vector<Variable const*> pruneVars;
traversal->getPruneVariables(pruneVars);
if (outVariable != nullptr && !n->isVarUsedLater(outVariable) &&
std::find(pruneVars.begin(), pruneVars.end(), outVariable) == pruneVars.end()) {
// traversal edge outVariable not used later
traversal->setEdgeOutput(nullptr);
modified = true;
}
outVariable = traversal->pathOutVariable();
if (outVariable != nullptr && !n->isVarUsedLater(outVariable)) {
if (outVariable != nullptr && !n->isVarUsedLater(outVariable) &&
std::find(pruneVars.begin(), pruneVars.end(), outVariable) == pruneVars.end()) {
// traversal path outVariable not used later
traversal->setPathOutput(nullptr);
modified = true;
@ -6038,8 +6043,11 @@ void arangodb::aql::removeTraversalPathVariable(Optimizer* opt,
for (auto const& n : tNodes) {
TraversalNode* traversal = ExecutionNode::castTo<TraversalNode*>(n);
std::vector<Variable const*> pruneVars;
traversal->getPruneVariables(pruneVars);
auto outVariable = traversal->pathOutVariable();
if (outVariable != nullptr && !n->isVarUsedLater(outVariable)) {
if (outVariable != nullptr && !n->isVarUsedLater(outVariable) &&
std::find(pruneVars.begin(), pruneVars.end(), outVariable) == pruneVars.end()) {
// traversal path outVariable not used later
traversal->setPathOutput(nullptr);
modified = true;

48
arangod/Aql/PruneExpressionEvaluator.cpp Normal file
View File

@ -0,0 +1,48 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#include "PruneExpressionEvaluator.h"
#include "Aql/AqlValue.h"
#include "Aql/Expression.h"
#include "Transaction/Methods.h"
using namespace arangodb;
using namespace arangodb::aql;
PruneExpressionEvaluator::PruneExpressionEvaluator(
transaction::Methods* trx, Query* query,
std::vector<Variable const*> const&& vars, std::vector<RegisterId> const&& regs,
size_t vertexVarIdx, size_t edgeVarIdx, size_t pathVarIdx, Expression* expr)
: _trx(trx),
_pruneExpression(expr),
_ctx(query, std::move(vars), std::move(regs), vertexVarIdx, edgeVarIdx, pathVarIdx) {}
PruneExpressionEvaluator::~PruneExpressionEvaluator() = default;
bool PruneExpressionEvaluator::evaluate() {
bool mustDestroy = false;
aql::AqlValue res = _pruneExpression->execute(_trx, &_ctx, mustDestroy);
arangodb::aql::AqlValueGuard guard(res, mustDestroy);
TRI_ASSERT(res.isBoolean());
return res.toBoolean();
}

74
arangod/Aql/PruneExpressionEvaluator.h Normal file
View File

@ -0,0 +1,74 @@
////////////////////////////////////////////////////////////////////////////////
/// DISCLAIMER
///
/// Copyright 2019-2019 ArangoDB GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is ArangoDB GmbH, Cologne, Germany
///
/// @author Michael Hackstein
////////////////////////////////////////////////////////////////////////////////
#ifndef ARANGOD_AQL_PRUNE_EXPRESSION_EVALUATOR_H
#define ARANGOD_AQL_PRUNE_EXPRESSION_EVALUATOR_H 1
#include "Aql/InAndOutRowExpressionContext.h"
namespace arangodb {
namespace velocypack {
class Slice;
}
namespace transaction {
class Methods;
}
namespace aql {
class Expression;
class Query;
class InputAqlItemRow;
class PruneExpressionEvaluator {
public:
PruneExpressionEvaluator(transaction::Methods* trx, Query* query,
std::vector<Variable const*> const&& vars,
std::vector<RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx, Expression* expr);
~PruneExpressionEvaluator();
bool evaluate();
void prepareContext(InputAqlItemRow input) { _ctx.setInputRow(input); }
bool needsVertex() const { return _ctx.needsVertexValue(); }
void injectVertex(velocypack::Slice v) { _ctx.setVertexValue(v); }
bool needsEdge() const { return _ctx.needsEdgeValue(); }
void injectEdge(velocypack::Slice e) { _ctx.setEdgeValue(e); }
bool needsPath() const { return _ctx.needsPathValue(); }
void injectPath(velocypack::Slice p) { _ctx.setPathValue(p); }
private:
transaction::Methods* _trx;
/// @brief The condition given in PRUNE (might be empty)
/// The Node keeps responsibility
aql::Expression* _pruneExpression;
/// @brief The context used to inject variables
InAndOutRowExpressionContext _ctx;
};
} // namespace aql
} // namespace arangodb
#endif

8
arangod/Aql/TraversalExecutor.cpp
View File

@ -22,6 +22,7 @@
#include "TraversalExecutor.h"
#include "Aql/OutputAqlItemRow.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Aql/Query.h"
#include "Aql/SingleRowFetcher.h"
#include "Graph/Traverser.h"
@ -149,7 +150,6 @@ std::pair<ExecutionState, TraversalStats> TraversalExecutor::produceRow(OutputAq
s.addScannedIndex(_traverser.getAndResetReadDocuments());
return {_rowState, s};
}
if (!resetTraverser()) {
// Could not start here, (invalid)
// Go to next
@ -203,7 +203,11 @@ bool TraversalExecutor::resetTraverser() {
for (auto const& pair : _infos.filterConditionVariables()) {
opts->setVariableValue(pair.first, _input.getValue(pair.second));
}
if (opts->usesPrune()) {
auto* evaluator = opts->getPruneEvaluator();
// Replace by inputRow
evaluator->prepareContext(_input);
}
// Now reset the traverser
if (_infos.usesFixedSource()) {
auto pos = _infos.getFixedSource().find('/');

67
arangod/Aql/TraversalNode.cpp
View File

@ -91,13 +91,15 @@ void TraversalNode::TraversalEdgeConditionBuilder::toVelocyPack(VPackBuilder& bu
TraversalNode::TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocbase,
AstNode const* direction, AstNode const* start,
AstNode const* graph, std::unique_ptr<BaseOptions> options)
AstNode const* graph, std::unique_ptr<Expression> pruneExpression,
std::unique_ptr<BaseOptions> options)
: GraphNode(plan, id, vocbase, direction, graph, std::move(options)),
_pathOutVariable(nullptr),
_inVariable(nullptr),
_condition(nullptr),
_fromCondition(nullptr),
_toCondition(nullptr) {
_toCondition(nullptr),
_pruneExpression(std::move(pruneExpression)) {
TRI_ASSERT(start != nullptr);
auto ast = _plan->getAst();
@ -143,6 +145,10 @@ TraversalNode::TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocb
"_id string or an object with _id.");
}
if (_pruneExpression) {
_pruneExpression->variables(_pruneVariables);
}
#ifdef TRI_ENABLE_MAINTAINER_MODE
checkConditionsDefined();
#endif
@ -247,6 +253,17 @@ TraversalNode::TraversalNode(ExecutionPlan* plan, arangodb::velocypack::Slice co
}
}
list = base.get("expression");
if (!list.isNone()) {
_pruneExpression = std::make_unique<aql::Expression>(plan, plan->getAst(), base);
TRI_ASSERT(base.hasKey("pruneVariables"));
list = base.get("pruneVariables");
TRI_ASSERT(list.isArray());
for (auto const& varinfo : VPackArrayIterator(list)) {
_pruneVariables.emplace(plan->getAst()->variables()->createVariable(varinfo));
}
}
#ifdef TRI_ENABLE_MAINTAINER_MODE
checkConditionsDefined();
#endif
@ -379,6 +396,17 @@ void TraversalNode::toVelocyPackHelper(VPackBuilder& nodes, unsigned flags) cons
nodes.add(VPackValue("indexes"));
_options->toVelocyPackIndexes(nodes);
if (_pruneExpression != nullptr) {
// The Expression constructor expects only this name
nodes.add(VPackValue("expression"));
_pruneExpression->toVelocyPack(nodes, flags);
nodes.add(VPackValue("pruneVariables"));
nodes.openArray();
for (auto const& var : _pruneVariables) {
var->toVelocyPack(nodes);
}
nodes.close();
}
// And close it:
nodes.close();
}
@ -428,6 +456,33 @@ std::unique_ptr<ExecutionBlock> TraversalNode::createBlock(
auto opts = static_cast<TraverserOptions*>(options());
std::unique_ptr<Traverser> traverser;
auto trx = engine.getQuery()->trx();
if (pruneExpression() != nullptr) {
std::vector<Variable const*> pruneVars;
getPruneVariables(pruneVars);
std::vector<RegisterId> pruneRegs;
// Create List for _pruneVars
pruneRegs.reserve(pruneVars.size());
size_t vertexRegIdx = std::numeric_limits<std::size_t>::max();
size_t edgeRegIdx = std::numeric_limits<std::size_t>::max();
size_t pathRegIdx = std::numeric_limits<std::size_t>::max();
for (auto const v : pruneVars) {
auto it = varInfo.find(v->id);
TRI_ASSERT(it != varInfo.end());
if (v == vertexOutVariable()) {
vertexRegIdx = pruneRegs.size();
} else if (v == edgeOutVariable()) {
edgeRegIdx = pruneRegs.size();
} else if (v == pathOutVariable()) {
pathRegIdx = pruneRegs.size();
}
pruneRegs.emplace_back(it->second.registerId);
}
opts->activatePrune(std::move(pruneVars), std::move(pruneRegs),
vertexRegIdx, edgeRegIdx, pathRegIdx, pruneExpression());
}
if (arangodb::ServerState::instance()->isCoordinator()) {
#ifdef USE_ENTERPRISE
if (isSmart()) {
@ -699,6 +754,14 @@ void TraversalNode::getConditionVariables(std::vector<Variable const*>& res) con
}
}
void TraversalNode::getPruneVariables(std::vector<Variable const*>& res) const {
if (_pruneExpression) {
for (auto const& it : _pruneVariables) {
res.emplace_back(it);
}
}
}
#ifdef TRI_ENABLE_MAINTAINER_MODE
void TraversalNode::checkConditionsDefined() const {
TRI_ASSERT(_tmpObjVariable != nullptr);

18
arangod/Aql/TraversalNode.h
View File

@ -78,7 +78,8 @@ class TraversalNode : public GraphNode {
public:
TraversalNode(ExecutionPlan* plan, size_t id, TRI_vocbase_t* vocbase,
AstNode const* direction, AstNode const* start,
AstNode const* graph, std::unique_ptr<graph::BaseOptions> options);
AstNode const* graph, std::unique_ptr<Expression> pruneExpression,
std::unique_ptr<graph::BaseOptions> options);
TraversalNode(ExecutionPlan* plan, arangodb::velocypack::Slice const& base);
@ -118,6 +119,9 @@ class TraversalNode : public GraphNode {
result.emplace(condVar);
}
}
for (auto const& pruneVar : _pruneVariables) {
result.emplace(pruneVar);
}
if (usesInVariable()) {
result.emplace(_inVariable);
}
@ -178,11 +182,17 @@ class TraversalNode : public GraphNode {
void getConditionVariables(std::vector<Variable const*>&) const override;
void getPruneVariables(std::vector<Variable const*>&) const;
/// @brief Compute the traversal options containing the expressions
/// MUST! be called after optimization and before creation
/// of blocks.
void prepareOptions() override;
// @brief Get reference to the Prune expression.
// You are not responsible for it!
Expression* pruneExpression() const { return _pruneExpression.get(); }
private:
#ifdef TRI_ENABLE_MAINTAINER_MODE
void checkConditionsDefined() const;
@ -210,6 +220,9 @@ class TraversalNode : public GraphNode {
/// @brief The hard coded condition on _to
AstNode* _toCondition;
/// @brief The condition given in PRUNE (might be empty)
std::unique_ptr<Expression> _pruneExpression;
/// @brief The global edge condition. Does not contain
/// _from and _to checks
std::vector<AstNode const*> _globalEdgeConditions;
@ -222,6 +235,9 @@ class TraversalNode : public GraphNode {
/// @brief List of all depth specific conditions for vertices
std::unordered_map<uint64_t, AstNode*> _vertexConditions;
/// @brief the hashSet for variables used in pruning
arangodb::HashSet<Variable const*> _pruneVariables;
};
} // namespace aql

2970
arangod/Aql/grammar.cpp
View File

File diff suppressed because it is too large Load Diff

4
arangod/Aql/grammar.h
View File

@ -125,7 +125,7 @@ extern int Aqldebug;
union YYSTYPE
{
#line 35 "Aql/grammar.y" /* yacc.c:1909 */
#line 35 "Aql/grammar.y" /* yacc.c:1915 */
arangodb::aql::AstNode* node;
struct {
@ -135,7 +135,7 @@ union YYSTYPE
bool boolval;
int64_t intval;
#line 139 "Aql/grammar.hpp" /* yacc.c:1909 */
#line 139 "Aql/grammar.hpp" /* yacc.c:1915 */
};
typedef union YYSTYPE YYSTYPE;

4
arangod/Aql/grammar.hpp
View File

@ -125,7 +125,7 @@ extern int Aqldebug;
union YYSTYPE
{
#line 35 "Aql/grammar.y" /* yacc.c:1909 */
#line 35 "Aql/grammar.y" /* yacc.c:1915 */
arangodb::aql::AstNode* node;
struct {
@ -135,7 +135,7 @@ union YYSTYPE
bool boolval;
int64_t intval;
#line 139 "Aql/grammar.hpp" /* yacc.c:1909 */
#line 139 "Aql/grammar.hpp" /* yacc.c:1915 */
};
typedef union YYSTYPE YYSTYPE;

216
arangod/Aql/grammar.y
View File

@ -200,6 +200,17 @@ static AstNode const* GetIntoExpression(AstNode const* node) {
return node->getMember(1);
}
static AstNode* TransformOutputVariables(Parser* parser, AstNode const* names) {
auto wrapperNode = parser->ast()->createNodeArray();
for (size_t i = 0; i < names->numMembers(); ++i) {
AstNode* variableNameNode = names->getMemberUnchecked(i);
TRI_ASSERT(variableNameNode->isStringValue());
AstNode* variableNode = parser->ast()->createNodeVariable(variableNameNode->getStringValue(), variableNameNode->getStringLength(), true);
wrapperNode->addMember(variableNode);
}
return wrapperNode;
}
%}
/* define tokens and "nice" token names */
@ -338,6 +349,9 @@ static AstNode const* GetIntoExpression(AstNode const* node) {
%type <node> function_arguments_list;
%type <node> compound_value;
%type <node> array;
%type <node> for_output_variables;
%type <node> traversal_graph_info;
%type <node> shortest_path_graph_info;
%type <node> optional_array_elements;
%type <node> array_elements_list;
%type <node> for_options;
@ -479,19 +493,145 @@ statement_block_statement:
}
;
more_output_variables:
variable_name {
auto wrapperNode = parser->ast()->createNodeArray();
parser->pushArray(wrapperNode);
// This is guaranteed to be called on the first variable
AstNode* node = parser->ast()->createNodeValueString($1.value, $1.length);
parser->pushArrayElement(node);
}
| more_output_variables T_COMMA variable_name {
AstNode* node = parser->ast()->createNodeValueString($3.value, $3.length);
parser->pushArrayElement(node);
}
;
for_output_variables:
more_output_variables {
$$ = parser->popArray();
}
;
prune_and_options:
/* empty no prune, no options, add two NOPS */ {
auto node = static_cast<AstNode*>(parser->peekStack());
// Prune
node->addMember(parser->ast()->createNodeNop());
// Options
node->addMember(parser->ast()->createNodeNop());
}
| T_STRING expression {
auto node = static_cast<AstNode*>(parser->peekStack());
if (TRI_CaseEqualString($1.value, "PRUNE")) {
/* Only Prune */
if ($2 == nullptr) {
ABORT_OOM
}
// Prune
node->addMember($2);
// Options
node->addMember(parser->ast()->createNodeNop());
} else if (TRI_CaseEqualString($1.value, "OPTIONS")) {
/* Only Options */
if ($2 == nullptr) {
ABORT_OOM
}
if (!$2->isObject()) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "traversal 'OPTIONS' have to be an object", $1.value, yylloc.first_line, yylloc.first_column);
}
// Prune
node->addMember(parser->ast()->createNodeNop());
// Options
node->addMember($2);
} else {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'PRUNE' or 'OPTIONS'", $1.value, yylloc.first_line, yylloc.first_column);
}
}
| T_STRING expression T_STRING object {
/* prune and options */
auto node = static_cast<AstNode*>(parser->peekStack());
if (!TRI_CaseEqualString($1.value, "PRUNE")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'PRUNE'", $1.value, yylloc.first_line, yylloc.first_column);
}
if ($2 == nullptr) {
ABORT_OOM
}
if (!TRI_CaseEqualString($3.value, "OPTIONS")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'OPTIONS'", $3.value, yylloc.first_line, yylloc.first_column);
}
if ($4 == nullptr) {
ABORT_OOM
}
// Prune
node->addMember($2);
// Options
node->addMember($4);
}
;
traversal_graph_info:
graph_direction_steps expression graph_subject {
auto infoNode = parser->ast()->createNodeArray();
// Direction
infoNode->addMember($1);
// Source
infoNode->addMember($2);
// Graph
infoNode->addMember($3);
$$ = infoNode;
}
;
shortest_path_graph_info:
graph_direction T_SHORTEST_PATH expression T_STRING expression graph_subject options {
if (!TRI_CaseEqualString($4.value, "TO")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'TO'", $4.value, yylloc.first_line, yylloc.first_column);
}
auto infoNode = parser->ast()->createNodeArray();
auto dirNode = parser->ast()->createNodeValueInt($1);
// Direction
infoNode->addMember(dirNode);
// Source
infoNode->addMember($3);
// Target
infoNode->addMember($5);
// Graph
infoNode->addMember($6);
// Opts
auto opts = parser->ast()->createNodeOptions($7);
TRI_ASSERT(opts != nullptr);
infoNode->addMember(opts);
$$ = infoNode;
}
;
for_statement:
T_FOR variable_name T_IN expression {
// first open a new scope
T_FOR for_output_variables T_IN expression {
// first open a new scope (after expression is evaluated)
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
AstNode* variablesNode = static_cast<AstNode*>($2);
TRI_ASSERT(variablesNode != nullptr);
TRI_ASSERT(variablesNode->type == NODE_TYPE_ARRAY);
if (variablesNode->numMembers() != 1) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "Collections and Views only have one return variable", yylloc.first_line, yylloc.first_column);
}
// now create an out variable for the FOR statement
// this prepares us to handle the optional SEARCH condition, which may
// or may not refer to the FOR's variable
parser->pushStack(parser->ast()->createNodeVariable($2.value, $2.length, true));
AstNode* variableNameNode = variablesNode->getMemberUnchecked(0);
TRI_ASSERT(variableNameNode->isStringValue());
AstNode* variableNode = parser->ast()->createNodeVariable(variableNameNode->getStringValue(), variableNameNode->getStringLength(), true);
parser->pushStack(variableNode);
} for_options {
// now we can handle the optional SEARCH condition and OPTIONS.
AstNode* variableNode = static_cast<AstNode*>(parser->popStack());
TRI_ASSERT(variableNode != nullptr);
Variable* variable = static_cast<Variable*>(variableNode->getData());
AstNode* node = nullptr;
@ -528,46 +668,46 @@ for_statement:
parser->ast()->addOperation(node);
}
| T_FOR traversal_statement {
}
| T_FOR shortest_path_statement {
}
;
traversal_statement:
variable_name T_IN graph_direction_steps expression graph_subject options {
| T_FOR for_output_variables T_IN traversal_graph_info {
// first open a new scope (after expression is evaluated)
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeTraversal($1.value, $1.length, $3, $4, $5, $6);
parser->ast()->addOperation(node);
}
| variable_name T_COMMA variable_name T_IN graph_direction_steps expression graph_subject options {
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeTraversal($1.value, $1.length, $3.value, $3.length, $5, $6, $7, $8);
parser->ast()->addOperation(node);
}
| variable_name T_COMMA variable_name T_COMMA variable_name T_IN graph_direction_steps expression graph_subject options {
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeTraversal($1.value, $1.length, $3.value, $3.length, $5.value, $5.length, $7, $8, $9, $10);
parser->ast()->addOperation(node);
}
;
shortest_path_statement:
variable_name T_IN graph_direction T_SHORTEST_PATH expression T_STRING expression graph_subject options {
if (!TRI_CaseEqualString($6.value, "TO")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'TO'", $6.value, yylloc.first_line, yylloc.first_column);
// Traversal
auto variableNamesNode = static_cast<AstNode*>($2);
TRI_ASSERT(variableNamesNode != nullptr);
TRI_ASSERT(variableNamesNode->type == NODE_TYPE_ARRAY);
if (variableNamesNode->numMembers() > 3) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "Traversals only have one, two or three return variables", yylloc.first_line, yylloc.first_column);
}
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeShortestPath($1.value, $1.length, $3, $5, $7, $8, $9);
auto variablesNode = TransformOutputVariables(parser, variableNamesNode);
auto graphInfoNode = static_cast<AstNode*>($4);
TRI_ASSERT(graphInfoNode != nullptr);
TRI_ASSERT(graphInfoNode->type == NODE_TYPE_ARRAY);
parser->pushStack(variablesNode);
parser->pushStack(graphInfoNode);
// This stack push/pop magic is necessary to allow v, e, and p in the prune condition
} prune_and_options {
auto graphInfoNode = static_cast<AstNode*>(parser->popStack());
auto variablesNode = static_cast<AstNode*>(parser->popStack());
auto node = parser->ast()->createNodeTraversal(variablesNode, graphInfoNode);
parser->ast()->addOperation(node);
}
| variable_name T_COMMA variable_name T_IN graph_direction T_SHORTEST_PATH expression T_STRING expression graph_subject options {
if (!TRI_CaseEqualString($8.value, "TO")) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "unexpected qualifier '%s', expecting 'TO'", $8.value, yylloc.first_line, yylloc.first_column);
}
| T_FOR for_output_variables T_IN shortest_path_graph_info {
// first open a new scope (after expression is evaluated)
parser->ast()->scopes()->start(arangodb::aql::AQL_SCOPE_FOR);
auto node = parser->ast()->createNodeShortestPath($1.value, $1.length, $3.value, $3.length, $5, $7, $9, $10, $11);
// Shortest Path
auto variableNamesNode = static_cast<AstNode*>($2);
TRI_ASSERT(variableNamesNode != nullptr);
TRI_ASSERT(variableNamesNode->type == NODE_TYPE_ARRAY);
if (variableNamesNode->numMembers() > 2) {
parser->registerParseError(TRI_ERROR_QUERY_PARSE, "ShortestPath only has one or two return variables", yylloc.first_line, yylloc.first_column);
}
auto variablesNode = TransformOutputVariables(parser, variableNamesNode);
auto graphInfoNode = static_cast<AstNode*>($4);
TRI_ASSERT(graphInfoNode != nullptr);
TRI_ASSERT(graphInfoNode->type == NODE_TYPE_ARRAY);
auto node = parser->ast()->createNodeShortestPath(variablesNode, graphInfoNode);
parser->ast()->addOperation(node);
}
;

2
arangod/CMakeLists.txt
View File

@ -224,6 +224,7 @@ SET(ARANGOD_SOURCES
Aql/Functions.cpp
Aql/GraphNode.cpp
Aql/Graphs.cpp
Aql/InAndOutRowExpressionContext.cpp
Aql/IdExecutor.cpp
Aql/IndexExecutor.cpp
Aql/IndexNode.cpp
@ -242,6 +243,7 @@ SET(ARANGOD_SOURCES
Aql/OutputAqlItemRow.cpp
Aql/Parser.cpp
Aql/PlanCache.cpp
Aql/PruneExpressionEvaluator.cpp
Aql/Quantifier.cpp
Aql/Query.cpp
Aql/QueryCache.cpp

1
arangod/Graph/BaseOptions.h
View File

@ -38,6 +38,7 @@ struct AstNode;
class ExecutionPlan;
class Expression;
class Query;
} // namespace aql
namespace velocypack {

111
arangod/Graph/BreadthFirstEnumerator.cpp
View File

@ -23,6 +23,7 @@
#include "BreadthFirstEnumerator.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Graph/EdgeCursor.h"
#include "Graph/EdgeDocumentToken.h"
#include "Graph/Traverser.h"
@ -51,12 +52,13 @@ BreadthFirstEnumerator::PathStep::PathStep(PathStep& other)
BreadthFirstEnumerator::BreadthFirstEnumerator(Traverser* traverser, VPackSlice startVertex,
TraverserOptions* opts)
: PathEnumerator(traverser, startVertex.copyString(), opts),
_schreierIndex(1),
_schreierIndex(0),
_lastReturned(0),
_currentDepth(0),
_toSearchPos(0) {
_schreier.reserve(32);
arangodb::velocypack::StringRef startVId = _opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex));
arangodb::velocypack::StringRef startVId =
_opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex));
_schreier.emplace_back(std::make_unique<PathStep>(startVId));
_toSearch.emplace_back(NextStep(0));
@ -67,6 +69,13 @@ BreadthFirstEnumerator::~BreadthFirstEnumerator() {}
bool BreadthFirstEnumerator::next() {
if (_isFirst) {
_isFirst = false;
if (shouldPrune()) {
TRI_ASSERT(_toSearch.size() == 1);
// Throw the next one away
_toSearch.clear();
}
// We have faked the 0 position in schreier for pruning
_schreierIndex++;
if (_opts->minDepth == 0) {
return true;
}
@ -92,22 +101,10 @@ bool BreadthFirstEnumerator::next() {
while (true) {
if (_toSearchPos >= _toSearch.size()) {
// This depth is done. GoTo next
if (_nextDepth.empty()) {
if (!prepareSearchOnNextDepth()) {
// That's it. we are done.
return false;
}
// Save copies:
// We clear current
// we swap current and next.
// So now current is filled
// and next is empty.
_toSearch.clear();
_toSearchPos = 0;
_toSearch.swap(_nextDepth);
_currentDepth++;
TRI_ASSERT(_toSearchPos < _toSearch.size());
TRI_ASSERT(_nextDepth.empty());
TRI_ASSERT(_currentDepth < _opts->maxDepth);
}
// This access is always safe.
// If not it should have bailed out before.
@ -152,7 +149,10 @@ bool BreadthFirstEnumerator::next() {
_schreier.emplace_back(std::make_unique<PathStep>(nextIdx, std::move(eid), vId));
if (_currentDepth < _opts->maxDepth - 1) {
_nextDepth.emplace_back(NextStep(_schreierIndex));
// Prune here
if (!shouldPrune()) {
_nextDepth.emplace_back(NextStep(_schreierIndex));
}
}
_schreierIndex++;
didInsert = true;
@ -180,30 +180,41 @@ bool BreadthFirstEnumerator::next() {
// entry. We compute the path to it.
return true;
}
arangodb::aql::AqlValue BreadthFirstEnumerator::lastVertexToAqlValue() {
TRI_ASSERT(_lastReturned < _schreier.size());
return _traverser->fetchVertexData(_schreier[_lastReturned]->vertex);
return vertexToAqlValue(_lastReturned);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::lastEdgeToAqlValue() {
TRI_ASSERT(_lastReturned < _schreier.size());
if (_lastReturned == 0) {
// This is the first Vertex. No Edge Pointing to it
return arangodb::aql::AqlValue(arangodb::aql::AqlValueHintNull());
}
return _opts->cache()->fetchEdgeAqlResult(_schreier[_lastReturned]->edge);
return edgeToAqlValue(_lastReturned);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::pathToAqlValue(arangodb::velocypack::Builder& result) {
return pathToIndexToAqlValue(result, _lastReturned);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::vertexToAqlValue(size_t index) {
TRI_ASSERT(index < _schreier.size());
return _traverser->fetchVertexData(_schreier[index]->vertex);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::edgeToAqlValue(size_t index) {
TRI_ASSERT(index < _schreier.size());
if (index == 0) {
// This is the first Vertex. No Edge Pointing to it
return arangodb::aql::AqlValue(arangodb::aql::AqlValueHintNull());
}
return _opts->cache()->fetchEdgeAqlResult(_schreier[index]->edge);
}
arangodb::aql::AqlValue BreadthFirstEnumerator::pathToIndexToAqlValue(
arangodb::velocypack::Builder& result, size_t index) {
// TODO make deque class variable
std::deque<size_t> fullPath;
size_t cur = _lastReturned;
while (cur != 0) {
while (index != 0) {
// Walk backwards through the path and push everything found on the local
// stack
fullPath.emplace_front(cur);
cur = _schreier[cur]->sourceIdx;
fullPath.emplace_front(index);
index = _schreier[index]->sourceIdx;
}
result.clear();
@ -226,7 +237,8 @@ arangodb::aql::AqlValue BreadthFirstEnumerator::pathToAqlValue(arangodb::velocyp
return arangodb::aql::AqlValue(result.slice());
}
bool BreadthFirstEnumerator::pathContainsVertex(size_t index, arangodb::velocypack::StringRef vertex) const {
bool BreadthFirstEnumerator::pathContainsVertex(size_t index,
arangodb::velocypack::StringRef vertex) const {
while (true) {
TRI_ASSERT(index < _schreier.size());
auto const& step = _schreier[index];
@ -262,3 +274,42 @@ bool BreadthFirstEnumerator::pathContainsEdge(size_t index,
}
return false;
}
bool BreadthFirstEnumerator::prepareSearchOnNextDepth() {
if (_nextDepth.empty()) {
// Nothing left to search
return false;
}
// Save copies:
// We clear current
// we swap current and next.
// So now current is filled
// and next is empty.
_toSearch.clear();
_toSearchPos = 0;
_toSearch.swap(_nextDepth);
_currentDepth++;
TRI_ASSERT(_toSearchPos < _toSearch.size());
TRI_ASSERT(_nextDepth.empty());
TRI_ASSERT(_currentDepth < _opts->maxDepth);
return true;
}
bool BreadthFirstEnumerator::shouldPrune() {
if (_opts->usesPrune()) {
auto* evaluator = _opts->getPruneEvaluator();
if (evaluator->needsVertex()) {
evaluator->injectVertex(vertexToAqlValue(_schreierIndex).slice());
}
if (evaluator->needsEdge()) {
evaluator->injectEdge(edgeToAqlValue(_schreierIndex).slice());
}
transaction::BuilderLeaser builder(_opts->trx());
if (evaluator->needsPath()) {
aql::AqlValue val = pathToIndexToAqlValue(*builder.get(), _schreierIndex);
evaluator->injectPath(val.slice());
}
return evaluator->evaluate();
}
return false;
}

15
arangod/Graph/BreadthFirstEnumerator.h
View File

@ -172,6 +172,21 @@ class BreadthFirstEnumerator final : public arangodb::traverser::PathEnumerator
* @return true if the edge is already in the path
*/
bool pathContainsEdge(size_t index, graph::EdgeDocumentToken const& edge) const;
/**
* @brief Reset iterators to search within next depth
* Also honors pruned paths
* @return true if we can continue searching. False if we are done
*/
bool prepareSearchOnNextDepth();
aql::AqlValue vertexToAqlValue(size_t index);
aql::AqlValue edgeToAqlValue(size_t index);
aql::AqlValue pathToIndexToAqlValue(arangodb::velocypack::Builder& result, size_t index);
bool shouldPrune();
};
} // namespace graph
} // namespace arangodb

39
arangod/Graph/NeighborsEnumerator.cpp
View File

@ -23,6 +23,7 @@
#include "NeighborsEnumerator.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Basics/VelocyPackHelper.h"
#include "Graph/EdgeCursor.h"
#include "Graph/Traverser.h"
@ -36,7 +37,8 @@ using namespace arangodb::traverser;
NeighborsEnumerator::NeighborsEnumerator(Traverser* traverser, VPackSlice const& startVertex,
TraverserOptions* opts)
: PathEnumerator(traverser, startVertex.copyString(), opts), _searchDepth(0) {
arangodb::velocypack::StringRef vId = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(startVertex));
arangodb::velocypack::StringRef vId = _traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(startVertex));
_allFound.insert(vId);
_currentDepth.insert(vId);
_iterator = _currentDepth.begin();
@ -58,8 +60,7 @@ bool NeighborsEnumerator::next() {
return false;
}
_lastDepth.swap(_currentDepth);
_currentDepth.clear();
swapLastAndCurrentDepth();
for (auto const& nextVertex : _lastDepth) {
auto callback = [&](EdgeDocumentToken&& eid, VPackSlice other, size_t cursorId) {
if (_opts->hasEdgeFilter(_searchDepth, cursorId)) {
@ -90,8 +91,11 @@ bool NeighborsEnumerator::next() {
if (_allFound.find(v) == _allFound.end()) {
if (_traverser->vertexMatchesConditions(v, _searchDepth + 1)) {
_currentDepth.emplace(v);
_allFound.emplace(v);
if (shouldPrune(v)) {
_toPrune.emplace(v);
}
_currentDepth.emplace(v);
}
} else {
_opts->cache()->increaseFilterCounter();
@ -132,3 +136,30 @@ arangodb::aql::AqlValue NeighborsEnumerator::pathToAqlValue(arangodb::velocypack
TRI_ASSERT(false);
THROW_ARANGO_EXCEPTION(TRI_ERROR_NOT_IMPLEMENTED);
}
void NeighborsEnumerator::swapLastAndCurrentDepth() {
// Filter all in _toPrune
if (!_toPrune.empty()) {
for (auto const& it : _toPrune) {
_currentDepth.erase(it);
}
_toPrune.clear();
}
_lastDepth.swap(_currentDepth);
_currentDepth.clear();
}
bool NeighborsEnumerator::shouldPrune(arangodb::velocypack::StringRef v) {
// Prune here
if (_opts->usesPrune()) {
auto* evaluator = _opts->getPruneEvaluator();
if (evaluator->needsVertex()) {
evaluator->injectVertex(_traverser->fetchVertexData(v).slice());
}
// We cannot support these two here
TRI_ASSERT(!evaluator->needsEdge());
TRI_ASSERT(!evaluator->needsPath());
return evaluator->evaluate();
}
return false;
}

6
arangod/Graph/NeighborsEnumerator.h
View File

@ -38,6 +38,7 @@ class NeighborsEnumerator final : public arangodb::traverser::PathEnumerator {
std::unordered_set<arangodb::velocypack::StringRef> _currentDepth;
std::unordered_set<arangodb::velocypack::StringRef> _lastDepth;
std::unordered_set<arangodb::velocypack::StringRef>::iterator _iterator;
std::unordered_set<arangodb::velocypack::StringRef> _toPrune;
uint64_t _searchDepth;
@ -65,6 +66,11 @@ class NeighborsEnumerator final : public arangodb::traverser::PathEnumerator {
aql::AqlValue lastEdgeToAqlValue() override;
aql::AqlValue pathToAqlValue(arangodb::velocypack::Builder& result) override;
private:
void swapLastAndCurrentDepth();
bool shouldPrune(arangodb::velocypack::StringRef v);
};
} // namespace graph

42
arangod/Graph/PathEnumerator.cpp
View File

@ -22,6 +22,8 @@
////////////////////////////////////////////////////////////////////////////////
#include "PathEnumerator.h"
#include "Aql/AqlValue.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Basics/VelocyPackHelper.h"
#include "Graph/EdgeCursor.h"
#include "Graph/Traverser.h"
@ -36,7 +38,8 @@ using TraverserOptions = arangodb::traverser::TraverserOptions;
PathEnumerator::PathEnumerator(Traverser* traverser, std::string const& startVertex,
TraverserOptions* opts)
: _traverser(traverser), _isFirst(true), _opts(opts) {
arangodb::velocypack::StringRef svId = _opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex));
arangodb::velocypack::StringRef svId =
_opts->cache()->persistString(arangodb::velocypack::StringRef(startVertex));
// Guarantee that this vertex _id does not run away
_enumeratedPath.vertices.push_back(svId);
TRI_ASSERT(_enumeratedPath.vertices.size() == 1);
@ -44,13 +47,16 @@ PathEnumerator::PathEnumerator(Traverser* traverser, std::string const& startVer
DepthFirstEnumerator::DepthFirstEnumerator(Traverser* traverser, std::string const& startVertex,
TraverserOptions* opts)
: PathEnumerator(traverser, startVertex, opts) {}
: PathEnumerator(traverser, startVertex, opts), _pruneNext(false) {}
DepthFirstEnumerator::~DepthFirstEnumerator() {}
bool DepthFirstEnumerator::next() {
if (_isFirst) {
_isFirst = false;
if (shouldPrune()) {
_pruneNext = true;
}
if (_opts->minDepth == 0) {
return true;
}
@ -61,11 +67,12 @@ bool DepthFirstEnumerator::next() {
}
while (true) {
if (_enumeratedPath.edges.size() < _opts->maxDepth) {
if (_enumeratedPath.edges.size() < _opts->maxDepth && !_pruneNext) {
// We are not done with this path, so
// we reserve the cursor for next depth
auto cursor = _opts->nextCursor(_traverser->mmdr(),
arangodb::velocypack::StringRef(_enumeratedPath.vertices.back()),
arangodb::velocypack::StringRef(
_enumeratedPath.vertices.back()),
_enumeratedPath.edges.size());
if (cursor != nullptr) {
_edgeCursors.emplace(cursor);
@ -77,6 +84,7 @@ bool DepthFirstEnumerator::next() {
_enumeratedPath.edges.pop_back();
}
}
_pruneNext = false;
bool foundPath = false;
@ -145,6 +153,9 @@ bool DepthFirstEnumerator::next() {
if (cursor->next(callback)) {
if (foundPath) {
if (shouldPrune()) {
_pruneNext = true;
}
if (_enumeratedPath.edges.size() < _opts->minDepth) {
// We have a valid prefix, but do NOT return this path
break;
@ -171,7 +182,8 @@ bool DepthFirstEnumerator::next() {
}
arangodb::aql::AqlValue DepthFirstEnumerator::lastVertexToAqlValue() {
return _traverser->fetchVertexData(arangodb::velocypack::StringRef(_enumeratedPath.vertices.back()));
return _traverser->fetchVertexData(
arangodb::velocypack::StringRef(_enumeratedPath.vertices.back()));
}
arangodb::aql::AqlValue DepthFirstEnumerator::lastEdgeToAqlValue() {
@ -202,3 +214,23 @@ arangodb::aql::AqlValue DepthFirstEnumerator::pathToAqlValue(arangodb::velocypac
result.close();
return arangodb::aql::AqlValue(result.slice());
}
bool DepthFirstEnumerator::shouldPrune() {
// We need to call prune here
if (_opts->usesPrune()) {
auto* evaluator = _opts->getPruneEvaluator();
if (evaluator->needsVertex()) {
evaluator->injectVertex(lastVertexToAqlValue().slice());
}
if (evaluator->needsEdge()) {
evaluator->injectEdge(lastEdgeToAqlValue().slice());
}
transaction::BuilderLeaser builder(_opts->trx());
if (evaluator->needsPath()) {
aql::AqlValue val = pathToAqlValue(*builder.get());
evaluator->injectPath(val.slice());
}
return evaluator->evaluate();
}
return false;
}

13
arangod/Graph/PathEnumerator.h
View File

@ -108,6 +108,11 @@ class DepthFirstEnumerator final : public PathEnumerator {
std::stack<std::unique_ptr<graph::EdgeCursor>> _edgeCursors;
//////////////////////////////////////////////////////////////////////////////
/// @brief Flag if we need to prune the next path
//////////////////////////////////////////////////////////////////////////////
bool _pruneNext;
public:
DepthFirstEnumerator(Traverser* traverser, std::string const& startVertex,
TraverserOptions* opts);
@ -120,16 +125,14 @@ class DepthFirstEnumerator final : public PathEnumerator {
bool next() override;
//////////////////////////////////////////////////////////////////////////////
/// @brief Prunes the current path prefix, the next function should not return
/// any path having this prefix anymore.
//////////////////////////////////////////////////////////////////////////////
aql::AqlValue lastVertexToAqlValue() override;
aql::AqlValue lastEdgeToAqlValue() override;
aql::AqlValue pathToAqlValue(arangodb::velocypack::Builder& result) override;
private:
bool shouldPrune();
};
} // namespace traverser
} // namespace arangodb

36
arangod/Graph/Traverser.cpp
View File

@ -38,7 +38,8 @@ using namespace arangodb;
using namespace arangodb::traverser;
using namespace arangodb::graph;
bool Traverser::VertexGetter::getVertex(VPackSlice edge, std::vector<arangodb::velocypack::StringRef>& result) {
bool Traverser::VertexGetter::getVertex(VPackSlice edge,
std::vector<arangodb::velocypack::StringRef>& result) {
VPackSlice res = edge;
if (!res.isString()) {
res = transaction::helpers::extractFromFromDocument(edge);
@ -48,15 +49,19 @@ bool Traverser::VertexGetter::getVertex(VPackSlice edge, std::vector<arangodb::v
}
TRI_ASSERT(res.isString());
if (!_traverser->vertexMatchesConditions(arangodb::velocypack::StringRef(res), result.size())) {
if (!_traverser->vertexMatchesConditions(arangodb::velocypack::StringRef(res),
result.size())) {
return false;
}
result.emplace_back(_traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(res)));
result.emplace_back(_traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(res)));
return true;
}
bool Traverser::VertexGetter::getSingleVertex(arangodb::velocypack::Slice edge, arangodb::velocypack::StringRef cmp,
uint64_t depth, arangodb::velocypack::StringRef& result) {
bool Traverser::VertexGetter::getSingleVertex(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef cmp,
uint64_t depth,
arangodb::velocypack::StringRef& result) {
VPackSlice resSlice = edge;
if (!resSlice.isString()) {
VPackSlice from = transaction::helpers::extractFromFromDocument(edge);
@ -67,7 +72,8 @@ bool Traverser::VertexGetter::getSingleVertex(arangodb::velocypack::Slice edge,
}
TRI_ASSERT(resSlice.isString());
}
result = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(resSlice));
result = _traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(resSlice));
return _traverser->vertexMatchesConditions(result, depth);
}
@ -86,7 +92,8 @@ bool Traverser::UniqueVertexGetter::getVertex(VPackSlice edge,
TRI_ASSERT(toAdd.isString());
}
arangodb::velocypack::StringRef toAddStr = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(toAdd));
arangodb::velocypack::StringRef toAddStr =
_traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(toAdd));
// First check if we visited it. If not, then mark
if (_returnedVertices.find(toAddStr) != _returnedVertices.end()) {
// This vertex is not unique.
@ -104,7 +111,8 @@ bool Traverser::UniqueVertexGetter::getVertex(VPackSlice edge,
}
bool Traverser::UniqueVertexGetter::getSingleVertex(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef cmp, uint64_t depth,
arangodb::velocypack::StringRef cmp,
uint64_t depth,
arangodb::velocypack::StringRef& result) {
VPackSlice resSlice = edge;
if (!resSlice.isString()) {
@ -115,7 +123,8 @@ bool Traverser::UniqueVertexGetter::getSingleVertex(arangodb::velocypack::Slice
TRI_ASSERT(resSlice.isString());
}
result = _traverser->traverserCache()->persistString(arangodb::velocypack::StringRef(resSlice));
result = _traverser->traverserCache()->persistString(
arangodb::velocypack::StringRef(resSlice));
// First check if we visited it. If not, then mark
if (_returnedVertices.find(result) != _returnedVertices.end()) {
// This vertex is not unique.
@ -141,7 +150,6 @@ Traverser::Traverser(arangodb::traverser::TraverserOptions* opts, transaction::M
: _trx(trx),
_mmdr(new arangodb::ManagedDocumentResult()),
_startIdBuilder(),
_pruneNext(false),
_done(true),
_opts(opts),
_canUseOptimizedNeighbors(false) {
@ -154,7 +162,8 @@ Traverser::Traverser(arangodb::traverser::TraverserOptions* opts, transaction::M
Traverser::~Traverser() {}
bool arangodb::traverser::Traverser::edgeMatchesConditions(VPackSlice e, arangodb::velocypack::StringRef vid,
bool arangodb::traverser::Traverser::edgeMatchesConditions(VPackSlice e,
arangodb::velocypack::StringRef vid,
uint64_t depth, size_t cursorId) {
if (!_opts->evaluateEdgeExpression(e, vid, depth, cursorId)) {
return false;
@ -162,7 +171,8 @@ bool arangodb::traverser::Traverser::edgeMatchesConditions(VPackSlice e, arangod
return true;
}
bool arangodb::traverser::Traverser::vertexMatchesConditions(arangodb::velocypack::StringRef v, uint64_t depth) {
bool arangodb::traverser::Traverser::vertexMatchesConditions(arangodb::velocypack::StringRef v,
uint64_t depth) {
if (_opts->vertexHasFilter(depth)) {
// We always need to destroy this vertex
aql::AqlValue vertex = fetchVertexData(v);
@ -198,4 +208,4 @@ size_t arangodb::traverser::Traverser::getAndResetFilteredPaths() {
void arangodb::traverser::Traverser::allowOptimizedNeighbors() {
_canUseOptimizedNeighbors = true;
}
}

21
arangod/Graph/Traverser.h
View File

@ -134,7 +134,8 @@ class Traverser {
virtual ~VertexGetter() = default;
virtual bool getVertex(arangodb::velocypack::Slice, std::vector<arangodb::velocypack::StringRef>&);
virtual bool getVertex(arangodb::velocypack::Slice,
std::vector<arangodb::velocypack::StringRef>&);
virtual bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef,
uint64_t, arangodb::velocypack::StringRef&);
@ -156,9 +157,11 @@ class Traverser {
~UniqueVertexGetter() = default;
bool getVertex(arangodb::velocypack::Slice, std::vector<arangodb::velocypack::StringRef>&) override;
bool getVertex(arangodb::velocypack::Slice,
std::vector<arangodb::velocypack::StringRef>&) override;
bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef, uint64_t, arangodb::velocypack::StringRef&) override;
bool getSingleVertex(arangodb::velocypack::Slice, arangodb::velocypack::StringRef,
uint64_t, arangodb::velocypack::StringRef&) override;
void reset(arangodb::velocypack::StringRef const&) override;
@ -218,7 +221,8 @@ class Traverser {
/// @brief Function to load the other sides vertex of an edge
/// Returns true if the vertex passes filtering conditions
virtual bool getSingleVertex(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef const sourceVertexId, uint64_t depth,
arangodb::velocypack::StringRef const sourceVertexId,
uint64_t depth,
arangodb::velocypack::StringRef& targetVertexId) = 0;
public:
@ -276,7 +280,8 @@ class Traverser {
bool hasMore() { return !_done; }
bool edgeMatchesConditions(arangodb::velocypack::Slice edge, arangodb::velocypack::StringRef vid,
bool edgeMatchesConditions(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef vid,
uint64_t depth, size_t cursorId);
bool vertexMatchesConditions(arangodb::velocypack::StringRef vid, uint64_t depth);
@ -306,9 +311,6 @@ class Traverser {
/// @brief Builder for the start value slice. Leased from transaction
velocypack::Builder _startIdBuilder;
/// @brief toggle if this path should be pruned on next step
bool _pruneNext;
/// @brief indicator if this traversal is done
bool _done;
@ -321,7 +323,8 @@ class Traverser {
virtual aql::AqlValue fetchVertexData(arangodb::velocypack::StringRef vid) = 0;
/// @brief Function to add the real data of a vertex into a velocypack builder
virtual void addVertexToVelocyPack(arangodb::velocypack::StringRef vid, arangodb::velocypack::Builder&) = 0;
virtual void addVertexToVelocyPack(arangodb::velocypack::StringRef vid,
arangodb::velocypack::Builder&) = 0;
};
} // namespace traverser
} // namespace arangodb

23
arangod/Graph/TraverserOptions.cpp
View File

@ -25,6 +25,7 @@
#include "Aql/Ast.h"
#include "Aql/Expression.h"
#include "Aql/PruneExpressionEvaluator.h"
#include "Aql/Query.h"
#include "Basics/VelocyPackHelper.h"
#include "Cluster/ClusterEdgeCursor.h"
@ -431,8 +432,8 @@ bool TraverserOptions::hasEdgeFilter(int64_t depth, size_t cursorId) const {
}
bool TraverserOptions::evaluateEdgeExpression(arangodb::velocypack::Slice edge,
arangodb::velocypack::StringRef vertexId, uint64_t depth,
size_t cursorId) const {
arangodb::velocypack::StringRef vertexId,
uint64_t depth, size_t cursorId) const {
arangodb::aql::Expression* expression = nullptr;
auto specific = _depthLookupInfo.find(depth);
@ -493,9 +494,8 @@ bool TraverserOptions::evaluateVertexExpression(arangodb::velocypack::Slice vert
return evaluateExpression(expression, vertex);
}
EdgeCursor* arangodb::traverser::TraverserOptions::nextCursor(ManagedDocumentResult* mmdr,
arangodb::velocypack::StringRef vid,
uint64_t depth) {
EdgeCursor* arangodb::traverser::TraverserOptions::nextCursor(
ManagedDocumentResult* mmdr, arangodb::velocypack::StringRef vid, uint64_t depth) {
if (_isCoordinator) {
return nextCursorCoordinator(vid, depth);
}
@ -510,7 +510,8 @@ EdgeCursor* arangodb::traverser::TraverserOptions::nextCursor(ManagedDocumentRes
return nextCursorLocal(mmdr, vid, list);
}
EdgeCursor* TraverserOptions::nextCursorCoordinator(arangodb::velocypack::StringRef vid, uint64_t depth) {
EdgeCursor* TraverserOptions::nextCursorCoordinator(arangodb::velocypack::StringRef vid,
uint64_t depth) {
TRI_ASSERT(_traverser != nullptr);
auto cursor = std::make_unique<ClusterEdgeCursor>(vid, depth, this);
return cursor.release();
@ -548,3 +549,13 @@ double TraverserOptions::estimateCost(size_t& nrItems) const {
nrItems = count;
return cost;
}
void TraverserOptions::activatePrune(std::vector<aql::Variable const*> const&& vars,
std::vector<aql::RegisterId> const&& regs,
size_t vertexVarIdx, size_t edgeVarIdx,
size_t pathVarIdx, aql::Expression* expr) {
_pruneExpression =
std::make_unique<aql::PruneExpressionEvaluator>(_trx, _query, std::move(vars),
std::move(regs), vertexVarIdx,
edgeVarIdx, pathVarIdx, expr);
}

23
arangod/Graph/TraverserOptions.h
View File

@ -43,8 +43,10 @@ class Slice;
namespace aql {
struct AstNode;
class Expression;
class PruneExpressionEvaluator;
class Query;
class TraversalNode;
struct Variable;
} // namespace aql
namespace graph {
@ -71,6 +73,10 @@ struct TraverserOptions : public graph::BaseOptions {
arangodb::traverser::ClusterTraverser* _traverser;
/// @brief The condition given in PRUNE (might be empty)
/// The Node keeps responsibility
std::unique_ptr<aql::PruneExpressionEvaluator> _pruneExpression;
public:
uint64_t minDepth;
@ -114,17 +120,30 @@ struct TraverserOptions : public graph::BaseOptions {
bool hasEdgeFilter(int64_t, size_t) const;
bool evaluateEdgeExpression(arangodb::velocypack::Slice, arangodb::velocypack::StringRef vertexId,
bool evaluateEdgeExpression(arangodb::velocypack::Slice,
arangodb::velocypack::StringRef vertexId,
uint64_t, size_t) const;
bool evaluateVertexExpression(arangodb::velocypack::Slice, uint64_t) const;
graph::EdgeCursor* nextCursor(ManagedDocumentResult*, arangodb::velocypack::StringRef vid, uint64_t);
graph::EdgeCursor* nextCursor(ManagedDocumentResult*,
arangodb::velocypack::StringRef vid, uint64_t);
void linkTraverser(arangodb::traverser::ClusterTraverser*);
double estimateCost(size_t& nrItems) const override;
void activatePrune(std::vector<aql::Variable const*> const&& vars,
std::vector<aql::RegisterId> const&& regs, size_t vertexVarIdx,
size_t edgeVarIdx, size_t pathVarIdx, aql::Expression* expr);
bool usesPrune() const { return _pruneExpression != nullptr; }
aql::PruneExpressionEvaluator* getPruneEvaluator() {
TRI_ASSERT(usesPrune());
return _pruneExpression.get();
}
private:
graph::EdgeCursor* nextCursorCoordinator(arangodb::velocypack::StringRef vid, uint64_t);
};

488
js/common/modules/@arangodb/aql/explainer.js
View File

File diff suppressed because it is too large Load Diff

834
tests/js/server/aql/aql-graph-traverser.js
View File

File diff suppressed because it is too large Load Diff