Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Bug fix/internal issue #316 (#7911) 

* allow using scorers outside ArangoSearch view context

Signed-off-by: Andrey Abramov <andrey@arangodb.com>

* ensure query is properly optimized after replacement of scorer functions

* do not apply `handleViewsRule` to queries without views

* simplify optimization rule for ArangoSearch views

* show ArangoSearch view scorers in query explanation

* fix tests

* fix tests

* add stub for scorer related tests

* reformat

* check variable depth in `ViewExpressionContext::getVariableValue`

* add some tests

* address js test failures

* address jslint errors

* ensure `IResearchViewNode` exposes variables used in scorers

* ensure scorers with expressions are deduplicated

* fix deduplication for indexed access

* more tests

* partially address review comments

* address review comments

* simplify code

* remove irrelevant, commented out code

* ensure array comparisons are properly handled

* update changelog & loki
This commit is contained in:
Andrey Abramov 2019-01-10 21:04:19 +03:00 committed by GitHub
parent aabb307295
commit d4a010baf9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
26 changed files with 2826 additions and 761 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGELOG
View File

@ -1,6 +1,9 @@
devel
-----
* fixed known issue #445: ArangoSearch ignores `_id` attribute even if `includeAllFields`
is set to `true`.
* upgraded bundled boost library to version 1.69.0
* upgraded bundled curl library to version 7.63

3
Documentation/Books/Manual/ReleaseNotes/KnownIssues35.md
View File

@ -11,13 +11,12 @@ ArangoSearch
| Issue |
|------------|
| **Date Added:** 2018-12-19 <br> **Component:** ArangoSearch <br> **Deployment Mode:** Single-server <br> **Description:** Value of `_id` attribute indexed by ArangoSearch view may become inconsistent after renaming a collection <br> **Affected Versions:** 3.4.2 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#514](https://github.com/arangodb/backlog/issues/514) (internal) |
| **Date Added:** 2018-12-19 <br> **Component:** ArangoSearch <br> **Deployment Mode:** Single-server <br> **Description:** Value of `_id` attribute indexed by ArangoSearch view may become inconsistent after renaming a collection <br> **Affected Versions:** >= 3.5.0 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#514](https://github.com/arangodb/backlog/issues/514) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** Cluster <br> **Description:** Score values evaluated by corresponding score functions (BM25/TFIDF) may differ in single-server and cluster with a collection having more than 1 shard <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#508](https://github.com/arangodb/backlog/issues/508) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** Cluster <br> **Description:** ArangoSearch index consolidation does not work during creation of a link on existing collection which may lead to massive file descriptors consumption <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#509](https://github.com/arangodb/backlog/issues/509) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** Cluster <br> **Description:** Long-running DML transactions on collections (linked with ArangoSearch view) block "ArangoDB flush thread" making impossible to refresh data "visible" by a view <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#510](https://github.com/arangodb/backlog/issues/510) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** All <br> **Description:** ArangoSearch index format included starting from 3.4.0-RC.4 is incompatible to earlier released 3.4.0 release candidates. Dump and restore is needed when upgrading from 3.4.0-RC.4 to a newer 3.4.0.x release <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** N/A |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** All <br> **Description:** RocksDB recovery fails sometimes after renaming a view <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#469](https://github.com/arangodb/backlog/issues/469) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** All <br> **Description:** ArangoSearch ignores `_id` attribute even if `includeAllFields` is set to `true` <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** 3.4.2 <br> **Reference:** [arangodb/backlog#445](https://github.com/arangodb/backlog/issues/445) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** All <br> **Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** All <br> **Description:** Using score functions (BM25/TFIDF) in ArangoDB expression is not supported <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/backlog#316](https://github.com/arangodb/backlog/issues/316) (internal) |
| **Date Added:** 2018-12-03 <br> **Component:** ArangoSearch <br> **Deployment Mode:** All <br> **Description:** ArangoSearch index format included starting from 3.4.0-RC.3 is incompatible to earlier released 3.4.0 release candidates. Dump and restore is needed when upgrading from 3.4.0-RC.2 to a newer 3.4.0.x release <br> **Affected Versions:** 3.4.0-RC.5 <br> **Fixed in Versions:** - <br> **Reference:** N/A |

5
arangod/Aql/ClusterBlocks.cpp
View File

@ -105,12 +105,7 @@ bool OurLessThan::operator()(std::pair<size_t, size_t> const& a,
int cmp;
if (attributePath.empty()) {
#if 0 // #ifdef USE_IRESEARCH
TRI_ASSERT(reg.comparator);
cmp = (*reg.comparator)(reg.scorer.get(), _trx, lhs, rhs);
#else
cmp = AqlValue::Compare(_trx, lhs, rhs, true);
#endif
} else {
// Take attributePath into consideration:
bool mustDestroyA;

3
arangod/Aql/Expression.h
View File

@ -82,6 +82,9 @@ class Expression {
}
}
/// @brief get the underlying AST
Ast* ast() const noexcept { return _ast; }
/// @brief get the underlying AST node
inline AstNode const* node() const { return _node; }

8
arangod/Aql/OptimizerRule.h
View File

@ -166,15 +166,15 @@ struct OptimizerRule {
// remove redundant filters statements
removeFiltersCoveredByTraversal,
// remove calculations that are redundant
// needs to run after filter removal
removeUnnecessaryCalculationsRule2,
#ifdef USE_IRESEARCH
// move filters and sort conditions into views and remove them
handleArangoSearchViewsRule,
#endif
// remove calculations that are redundant
// needs to run after filter removal
removeUnnecessaryCalculationsRule2,
// remove now obsolete path variables
removeTraversalPathVariable,
prepareTraversalsRule,

4
arangod/Aql/Query.h
View File

@ -296,6 +296,10 @@ class Query {
_views.emplace(name);
}
std::unordered_set<std::string> const& views() const noexcept {
return _views;
}
/// @brief look up a graph in the _graphs collection
graph::Graph const* lookupGraphByName(std::string const& name);

5
arangod/Aql/SortBlock.cpp
View File

@ -44,12 +44,7 @@ class OurLessThan {
auto const& lhs = _buffer[a.first]->getValueReference(a.second, reg.reg);
auto const& rhs = _buffer[b.first]->getValueReference(b.second, reg.reg);
#if 0 // #ifdef USE_IRESEARCH
TRI_ASSERT(reg.comparator);
int const cmp = (*reg.comparator)(reg.scorer.get(), _trx, lhs, rhs);
#else
int const cmp = AqlValue::Compare(_trx, lhs, rhs, true);
#endif
if (cmp < 0) {
return reg.asc;

82
arangod/Aql/SortRegister.cpp
View File

@ -28,45 +28,6 @@
#include "Aql/ExecutionPlan.h"
#include "Aql/SortNode.h"
#if 0 // #ifdef USE_IRESEARCH
#include "IResearch/IResearchOrderFactory.h"
#include "IResearch/IResearchViewNode.h"
namespace {
int compareIResearchScores(
irs::sort::prepared const* comparer,
arangodb::transaction::Methods*,
arangodb::aql::AqlValue const& lhs,
arangodb::aql::AqlValue const& rhs
) {
arangodb::velocypack::ValueLength tmp;
auto const* lhsScore = reinterpret_cast<irs::byte_type const*>(lhs.slice().getString(tmp));
auto const* rhsScore = reinterpret_cast<irs::byte_type const*>(rhs.slice().getString(tmp));
if (comparer->less(lhsScore, rhsScore)) {
return -1;
}
if (comparer->less(rhsScore, lhsScore)) {
return 1;
}
return 0;
}
int compareAqlValues(
irs::sort::prepared const*,
arangodb::transaction::Methods* trx,
arangodb::aql::AqlValue const& lhs,
arangodb::aql::AqlValue const& rhs) {
return arangodb::aql::AqlValue::Compare(trx, lhs, rhs, true);
}
}
#endif
namespace arangodb {
namespace aql {
@ -77,47 +38,6 @@ namespace aql {
SortRegister::SortRegister(RegisterId reg, SortElement const& element) noexcept
: attributePath(element.attributePath), reg(reg), asc(element.ascending) {}
#if 0 // #ifdef USE_IRESEARCH
void SortRegister::fill(
ExecutionPlan const& execPlan,
ExecutionNode::RegisterPlan const& regPlan,
std::vector<SortElement> const& elements,
std::vector<SortRegister>& sortRegisters
) {
sortRegisters.reserve(elements.size());
std::unordered_map<ExecutionNode const*, size_t> offsets(sortRegisters.capacity());
irs::sort::ptr comparer;
auto const& vars = regPlan.varInfo;
for (auto const& p : elements) {
auto const varId = p.var->id;
auto const it = vars.find(varId);
TRI_ASSERT(it != vars.end());
TRI_ASSERT(it->second.registerId < ExecutionNode::MaxRegisterId);
sortRegisters.emplace_back(it->second.registerId, p, &compareAqlValues);
auto const* setter = execPlan.getVarSetBy(varId);
if (setter && ExecutionNode::ENUMERATE_IRESEARCH_VIEW == setter->getType()) {
// sort condition is covered by `IResearchViewNode`
auto const* viewNode = ExecutionNode::castTo<iresearch::IResearchViewNode const*>(setter);
auto& offset = offsets[viewNode];
auto* node = viewNode->sortCondition()[offset++].node;
if (arangodb::iresearch::OrderFactory::comparer(&comparer, *node)) {
auto& reg = sortRegisters.back();
reg.scorer = comparer->prepare(); // set score comparer
reg.comparator = &compareIResearchScores; // set comparator
}
}
}
}
#else
void SortRegister::fill(ExecutionPlan const& /*execPlan*/,
ExecutionNode::RegisterPlan const& regPlan,
std::vector<SortElement> const& elements,
@ -134,7 +54,5 @@ void SortRegister::fill(ExecutionPlan const& /*execPlan*/,
}
}
#endif // USE_IRESEARCH
} // namespace aql
} // namespace arangodb

25
arangod/Aql/SortRegister.h
View File

@ -27,43 +27,18 @@
#include "Aql/ExecutionNode.h"
#include "types.h"
#if 0 // #ifdef USE_IRESEARCH
#include "search/sort.hpp"
#endif
namespace arangodb {
namespace aql {
/// @brief sort element for block, consisting of register, sort direction,
/// and a possible attribute path to dig into the document
struct SortRegister {
#if 0 // #ifdef USE_IRESEARCH
typedef int(*CompareFunc)(
irs::sort::prepared const* scorer,
transaction::Methods* trx,
AqlValue const& lhs,
AqlValue const& rhs
);
irs::sort::prepared::ptr scorer;
CompareFunc comparator;
#endif
std::vector<std::string> const& attributePath;
RegisterId reg;
bool asc;
SortRegister(RegisterId reg, SortElement const& element) noexcept;
#if 0 // #ifdef USE_IRESEARCH
SortRegister(
RegisterId reg,
SortElement const& element,
CompareFunc comparator) noexcept
: SortRegister(reg, element) {
this->comparator = comparator;
}
#endif
static void fill(ExecutionPlan const& /*execPlan*/,
ExecutionNode::RegisterPlan const& regPlan,
std::vector<SortElement> const& elements,

319
arangod/IResearch/AqlHelper.cpp
View File

@ -21,10 +21,13 @@
////////////////////////////////////////////////////////////////////////////////
#include "AqlHelper.h"
#include "Aql/Ast.h"
#include "Aql/ExecutionPlan.h"
#include "Aql/Expression.h"
#include "Aql/ExpressionContext.h"
#include "Aql/Function.h"
#include "Aql/Variable.h"
#include "Basics/fasthash.h"
#include "IResearchCommon.h"
#include "IResearchDocument.h"
#include "Logger/LogMacros.h"
@ -36,24 +39,298 @@
namespace {
arangodb::aql::AstNodeType const CmpMap[]{
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_EQ, // NODE_TYPE_OPERATOR_BINARY_EQ:
// 3 == a <==> a == 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_NE, // NODE_TYPE_OPERATOR_BINARY_NE:
// 3 != a <==> a != 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_GT, // NODE_TYPE_OPERATOR_BINARY_LT:
// 3 < a <==> a > 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_GE, // NODE_TYPE_OPERATOR_BINARY_LE:
// 3 <= a <==> a >= 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_LT, // NODE_TYPE_OPERATOR_BINARY_GT:
// 3 > a <==> a < 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_LE // NODE_TYPE_OPERATOR_BINARY_GE:
// 3 >= a <==> a <= 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_EQ, // NODE_TYPE_OPERATOR_BINARY_EQ: 3 == a <==> a == 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_NE, // NODE_TYPE_OPERATOR_BINARY_NE: 3 != a <==> a != 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_GT, // NODE_TYPE_OPERATOR_BINARY_LT: 3 < a <==> a > 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_GE, // NODE_TYPE_OPERATOR_BINARY_LE: 3 <= a <==> a >= 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_LT, // NODE_TYPE_OPERATOR_BINARY_GT: 3 > a <==> a < 3
arangodb::aql::NODE_TYPE_OPERATOR_BINARY_LE // NODE_TYPE_OPERATOR_BINARY_GE: 3 >= a <==> a <= 3
};
}
namespace arangodb {
namespace iresearch {
bool equalTo(aql::AstNode const* lhs, aql::AstNode const* rhs) {
if (lhs == rhs) {
return true;
}
if ((!lhs && rhs) || (lhs && !rhs)) {
return false;
}
if (lhs->type != rhs->type) {
return false;
}
size_t const n = lhs->numMembers();
if (n != rhs->numMembers()) {
return false;
}
// check members for equality
for (size_t i = 0; i < n; ++i) {
if (!equalTo(lhs->getMemberUnchecked(i), rhs->getMemberUnchecked(i))) {
return false;
}
}
switch (lhs->type) {
case aql::NODE_TYPE_VARIABLE: {
return lhs->getData() == rhs->getData();
}
case aql::NODE_TYPE_OPERATOR_UNARY_PLUS:
case aql::NODE_TYPE_OPERATOR_UNARY_MINUS:
case aql::NODE_TYPE_OPERATOR_UNARY_NOT:
case aql::NODE_TYPE_OPERATOR_BINARY_AND:
case aql::NODE_TYPE_OPERATOR_BINARY_OR:
case aql::NODE_TYPE_OPERATOR_BINARY_PLUS:
case aql::NODE_TYPE_OPERATOR_BINARY_MINUS:
case aql::NODE_TYPE_OPERATOR_BINARY_TIMES:
case aql::NODE_TYPE_OPERATOR_BINARY_DIV:
case aql::NODE_TYPE_OPERATOR_BINARY_MOD:
case aql::NODE_TYPE_OPERATOR_BINARY_EQ:
case aql::NODE_TYPE_OPERATOR_BINARY_NE:
case aql::NODE_TYPE_OPERATOR_BINARY_LT:
case aql::NODE_TYPE_OPERATOR_BINARY_LE:
case aql::NODE_TYPE_OPERATOR_BINARY_GT:
case aql::NODE_TYPE_OPERATOR_BINARY_GE:
case aql::NODE_TYPE_OPERATOR_BINARY_IN:
case aql::NODE_TYPE_OPERATOR_BINARY_NIN:
case aql::NODE_TYPE_OPERATOR_TERNARY:
case aql::NODE_TYPE_OBJECT:
case aql::NODE_TYPE_CALCULATED_OBJECT_ELEMENT:
case aql::NODE_TYPE_ARRAY:
case aql::NODE_TYPE_RANGE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_EQ:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_NE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_LT:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_LE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_GT:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_GE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_IN:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_NIN: {
return true;
}
case aql::NODE_TYPE_ATTRIBUTE_ACCESS:
case aql::NODE_TYPE_INDEXED_ACCESS:
case aql::NODE_TYPE_EXPANSION: {
return attributeAccessEqual(lhs, rhs, nullptr);
}
case aql::NODE_TYPE_VALUE: {
return 0 == aql::CompareAstNodes(lhs, rhs, true);
}
case aql::NODE_TYPE_OBJECT_ELEMENT: {
irs::string_ref lhsValue, rhsValue;
iresearch::parseValue(lhsValue, *lhs);
iresearch::parseValue(rhsValue, *rhs);
return lhsValue == rhsValue;
}
case aql::NODE_TYPE_REFERENCE: {
return lhs->getData() == rhs->getData();
}
case aql::NODE_TYPE_FCALL: {
return lhs->getData() == rhs->getData();
}
case aql::NODE_TYPE_FCALL_USER: {
irs::string_ref lhsName, rhsName;
iresearch::parseValue(lhsName, *lhs);
iresearch::parseValue(rhsName, *rhs);
return lhsName == rhsName;
}
case aql::NODE_TYPE_QUANTIFIER: {
return lhs->value.value._int == rhs->value.value._int;
}
default: {
return false;
}
}
}
size_t hash(aql::AstNode const* node, size_t hash /*= 0*/) noexcept {
if (!node) {
return hash;
}
// hash node type
auto const& typeString = node->getTypeString();
hash = fasthash64(
static_cast<const void*>(typeString.c_str()),
typeString.size(),
hash
);
// hash node members
for (size_t i = 0, n = node->numMembers(); i < n; ++i) {
auto sub = node->getMemberUnchecked(i);
if (sub) {
hash = iresearch::hash(sub, hash);
}
}
switch (node->type) {
case aql::NODE_TYPE_VARIABLE: {
return fasthash64(node->getData(), sizeof(void*), hash);
}
case aql::NODE_TYPE_OPERATOR_UNARY_PLUS:
case aql::NODE_TYPE_OPERATOR_UNARY_MINUS:
case aql::NODE_TYPE_OPERATOR_UNARY_NOT:
case aql::NODE_TYPE_OPERATOR_BINARY_AND:
case aql::NODE_TYPE_OPERATOR_BINARY_OR:
case aql::NODE_TYPE_OPERATOR_BINARY_PLUS:
case aql::NODE_TYPE_OPERATOR_BINARY_MINUS:
case aql::NODE_TYPE_OPERATOR_BINARY_TIMES:
case aql::NODE_TYPE_OPERATOR_BINARY_DIV:
case aql::NODE_TYPE_OPERATOR_BINARY_MOD:
case aql::NODE_TYPE_OPERATOR_BINARY_EQ:
case aql::NODE_TYPE_OPERATOR_BINARY_NE:
case aql::NODE_TYPE_OPERATOR_BINARY_LT:
case aql::NODE_TYPE_OPERATOR_BINARY_LE:
case aql::NODE_TYPE_OPERATOR_BINARY_GT:
case aql::NODE_TYPE_OPERATOR_BINARY_GE:
case aql::NODE_TYPE_OPERATOR_BINARY_IN:
case aql::NODE_TYPE_OPERATOR_BINARY_NIN:
case aql::NODE_TYPE_OPERATOR_TERNARY:
case aql::NODE_TYPE_INDEXED_ACCESS:
case aql::NODE_TYPE_EXPANSION:
case aql::NODE_TYPE_ARRAY:
case aql::NODE_TYPE_OBJECT:
case aql::NODE_TYPE_CALCULATED_OBJECT_ELEMENT:
case aql::NODE_TYPE_RANGE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_EQ:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_NE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_LT:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_LE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_GT:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_GE:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_IN:
case aql::NODE_TYPE_OPERATOR_BINARY_ARRAY_NIN: {
return hash;
}
case aql::NODE_TYPE_ATTRIBUTE_ACCESS: {
return aql::AstNode(node->value).hashValue(hash);
}
case aql::NODE_TYPE_VALUE: {
switch (node->value.type) {
case aql::VALUE_TYPE_NULL:
return fasthash64(static_cast<const void*>("null"), 4, hash);
case aql::VALUE_TYPE_BOOL:
if (node->value.value._bool) {
return fasthash64(static_cast<const void*>("true"), 4, hash);
}
return fasthash64(static_cast<const void*>("false"), 5, hash);
case aql::VALUE_TYPE_INT:
return fasthash64(static_cast<const void*>(&node->value.value._int),
sizeof(node->value.value._int), hash);
case aql::VALUE_TYPE_DOUBLE:
return fasthash64(static_cast<const void*>(&node->value.value._double),
sizeof(node->value.value._double), hash);
case aql::VALUE_TYPE_STRING:
return fasthash64(static_cast<const void*>(node->getStringValue()),
node->getStringLength(), hash);
}
}
case aql::NODE_TYPE_OBJECT_ELEMENT: {
return fasthash64(static_cast<const void*>(node->getStringValue()),
node->getStringLength(), hash);
}
case aql::NODE_TYPE_REFERENCE: {
return fasthash64(node->getData(), sizeof(void*), hash);
}
case aql::NODE_TYPE_FCALL: {
auto* fn = static_cast<aql::Function*>(node->getData());
hash = fasthash64(node->getData(), sizeof(void*), hash);
return fasthash64(fn->name.c_str(), fn->name.size(), hash);
}
case aql::NODE_TYPE_FCALL_USER: {
return fasthash64(static_cast<const void*>(node->getStringValue()),
node->getStringLength(), hash);
}
case aql::NODE_TYPE_QUANTIFIER: {
return fasthash64(static_cast<const void*>(&node->value.value._int),
sizeof(node->value.value._int), hash);
}
default: {
return fasthash64(static_cast<void const*>(&node), sizeof(&node), hash);
}
}
}
irs::string_ref getFuncName(aql::AstNode const& node) {
irs::string_ref fname;
switch (node.type) {
case aql::NODE_TYPE_FCALL:
fname = reinterpret_cast<aql::Function const*>(node.getData())->name;
break;
case aql::NODE_TYPE_FCALL_USER:
parseValue(fname, node);
break;
default:
TRI_ASSERT(false);
}
return fname;
}
void visitReferencedVariables(
aql::AstNode const& root,
std::function<void(aql::Variable const&)> const& visitor) {
auto preVisitor = [](aql::AstNode const* node) -> bool {
return !node->isConstant();
};
auto postVisitor = [&visitor](aql::AstNode const* node) {
if (node == nullptr) {
return;
}
// reference to a variable
if (node->type == aql::NODE_TYPE_REFERENCE) {
auto variable = static_cast<aql::Variable const*>(node->getData());
if (!variable) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_INTERNAL,
"invalid reference in AST");
}
if (variable->needsRegister()) {
visitor(*variable);
}
}
};
aql::Ast::traverseReadOnly(&root, preVisitor, postVisitor);
}
// ----------------------------------------------------------------------------
// --SECTION-- AqlValueTraits implementation
// ----------------------------------------------------------------------------
@ -199,14 +476,16 @@ bool attributeAccessEqual(arangodb::aql::AstNode const* lhs,
if (root && offset) {
aqlValue.reset(*offset);
if (!ctx) {
// can't evaluate expression at compile time
return true;
}
if (!aqlValue.isConstant()) {
if (!ctx) {
// can't evaluate expression at compile time
return true;
}
if (!aqlValue.execute(*ctx)) {
// failed to execute expression
return false;
if (!aqlValue.execute(*ctx)) {
// failed to execute expression
return false;
}
}
switch (aqlValue.type()) {
@ -387,4 +666,4 @@ arangodb::aql::AstNode const* checkAttributeAccess(arangodb::aql::AstNode const*
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------

79
arangod/IResearch/AqlHelper.h
View File

@ -58,6 +58,16 @@ class Methods; // forward declaration
namespace iresearch {
//////////////////////////////////////////////////////////////////////////////
/// @returns true if both nodes are equal, false otherwise
//////////////////////////////////////////////////////////////////////////////
bool equalTo(aql::AstNode const* lhs, aql::AstNode const* rhs);
//////////////////////////////////////////////////////////////////////////////
/// @returns computed hash value for a specified node
//////////////////////////////////////////////////////////////////////////////
size_t hash(aql::AstNode const* node, size_t hash = 0) noexcept;
//////////////////////////////////////////////////////////////////////////////
/// @brief extracts string_ref from an AstNode, note that provided 'node'
/// must be an arangodb::aql::VALUE_TYPE_STRING
@ -69,6 +79,12 @@ inline irs::string_ref getStringRef(aql::AstNode const& node) {
return irs::string_ref(node.getStringValue(), node.getStringLength());
}
//////////////////////////////////////////////////////////////////////////////
/// @returns name of function denoted by a specified AstNode
/// @note applicable for nodes of type NODE_TYPE_FCALL, NODE_TYPE_FCALL_USER
//////////////////////////////////////////////////////////////////////////////
irs::string_ref getFuncName(aql::AstNode const& node);
////////////////////////////////////////////////////////////////////////////////
/// @brief tries to extract 'size_t' value from the specified AstNode 'node'
/// @returns true on success, false otherwise
@ -126,6 +142,13 @@ bool visit(aql::SortCondition const& sort, Visitor const& visitor) {
return true;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief visits variables referenced in a specified expression
////////////////////////////////////////////////////////////////////////////////
void visitReferencedVariables(
aql::AstNode const& root,
std::function<void(aql::Variable const&)> const& visitor);
////////////////////////////////////////////////////////////////////////////////
/// @brief visits the specified node using the provided 'visitor' according
/// to the specified visiting strategy (preorder/postorder)
@ -386,62 +409,6 @@ bool visitAttributeAccess(aql::AstNode const*& head, aql::AstNode const* node, T
}
}
////////////////////////////////////////////////////////////////////////////////
/// @brief interprets the specified node as an attribute path description and
/// visits the members in attribute path order calling the provided
/// 'visitor' on each path sub-index, expecting the following signatures:
/// bool operator()(irs::string_ref) - string keys
/// bool operator()(int64_t) - array offsets
/// bool operator()() - any string key or numeric offset
/// @return success and set head the the starting node of path (reference/value)
////////////////////////////////////////////////////////////////////////////////
template <typename T>
bool visitAttributePath(aql::AstNode const*& head, aql::AstNode const& node, T& visitor) {
if (node.numMembers() >= 2 && aql::NODE_TYPE_EXPANSION == node.type) { // [*]
auto* itr = node.getMemberUnchecked(0);
auto* ref = node.getMemberUnchecked(1);
if (itr && itr->numMembers() == 2) {
auto* root = itr->getMemberUnchecked(1);
auto* var = itr->getMemberUnchecked(0);
return ref && aql::NODE_TYPE_ITERATOR == itr->type &&
aql::NODE_TYPE_REFERENCE == ref->type && root && var &&
aql::NODE_TYPE_VARIABLE == var->type &&
visitAttributePath(head, *root, visitor) // 1st visit root
&& visitor(); // 2nd visit current node
}
} else if (node.numMembers() == 2 && aql::NODE_TYPE_INDEXED_ACCESS == node.type) { // [<something>]
auto* root = node.getMemberUnchecked(0);
auto* offset = node.getMemberUnchecked(1);
if (offset && offset->isIntValue()) {
return root && offset->getIntValue() >= 0 &&
visitAttributePath(head, *root, visitor) // 1st visit root
&& visitor(offset->getIntValue()); // 2nd visit current node
}
return root && offset && offset->isStringValue() &&
visitAttributePath(head, *root, visitor) // 1st visit root
&& visitor(iresearch::getStringRef(*offset)); // 2nd visit current node
} else if (node.numMembers() == 1 && aql::NODE_TYPE_ATTRIBUTE_ACCESS == node.type) {
auto* root = node.getMemberUnchecked(0);
return root && aql::VALUE_TYPE_STRING == node.value.type &&
visitAttributePath(head, *root, visitor) // 1st visit root
&& visitor(iresearch::getStringRef(node)); // 2nd visit current node
} else if (!node.numMembers()) { // end of attribute path (base case)
head = &node;
return aql::NODE_TYPE_REFERENCE == node.type ||
(aql::NODE_TYPE_VALUE == node.type &&
aql::VALUE_TYPE_STRING == node.value.type &&
visitor(iresearch::getStringRef(node)));
}
return false;
}
struct NormalizedCmpNode {
aql::AstNode const* attribute;
aql::AstNode const* value;

31
arangod/IResearch/IResearchExpressionContext.cpp
View File

@ -24,19 +24,7 @@
#include "IResearch/IResearchExpressionContext.h"
#include "Aql/AqlItemBlock.h"
#include "IResearch/IResearchViewNode.h"
namespace {
inline arangodb::aql::RegisterId getRegister(arangodb::aql::Variable const& var,
arangodb::aql::ExecutionNode const& node) noexcept {
auto const& vars = node.getRegisterPlan()->varInfo;
auto const it = vars.find(var.id);
return vars.end() == it ? arangodb::aql::ExecutionNode::MaxRegisterId
: it->second.registerId;
}
} // namespace
#include "Basics/StaticStrings.h"
namespace arangodb {
namespace iresearch {
@ -82,13 +70,24 @@ AqlValue ViewExpressionContext::getVariableValue(Variable const* var, bool doCop
}
mustDestroy = false;
auto const reg = getRegister(*var, *_node);
if (reg == arangodb::aql::ExecutionNode::MaxRegisterId) {
auto const& vars = _node->getRegisterPlan()->varInfo;
auto const it = vars.find(var->id);
if (vars.end() == it) {
THROW_ARANGO_EXCEPTION(TRI_ERROR_INTERNAL);
}
auto& value = _data->getValueReference(_pos, reg);
auto const& varInfo = it->second;
if (varInfo.depth > decltype(varInfo.depth)(_node->getDepth())) {
THROW_ARANGO_EXCEPTION_FORMAT(
TRI_ERROR_BAD_PARAMETER,
"Variable '%s' is used before being assigned",
var->name.c_str());
}
auto& value = _data->getValueReference(_pos, varInfo.registerId);
if (doCopy) {
mustDestroy = true;

30
arangod/IResearch/IResearchFeature.cpp
View File

@ -333,33 +333,6 @@ bool iresearchViewUpgradeVersion0_1(TRI_vocbase_t& vocbase,
return true;
}
void registerFunctions(arangodb::aql::AqlFunctionFeature& /*functions*/) {
#if 0
arangodb::iresearch::addFunction(functions, {
"__ARANGOSEARCH_SCORE_DEBUG", // name
".", // value to convert
arangodb::aql::Function::makeFlags(
arangodb::aql::Function::Flags::Deterministic,
arangodb::aql::Function::Flags::Cacheable,
arangodb::aql::Function::Flags::CanRunOnDBServer
),
[](arangodb::aql::ExpressionContext*,
arangodb::transaction::Methods*,
arangodb::SmallVector<arangodb::aql::AqlValue> const& args) {
if (args.empty()) {
// no such input parameter. return NaN
return arangodb::aql::AqlValue(arangodb::aql::AqlValueHintDouble(double_t(std::nan(""))));
} else {
// unsafe
VPackValueLength length;
auto const floatValue = *reinterpret_cast<float_t const*>(args[0].slice().getString(length));
return arangodb::aql::AqlValue(arangodb::aql::AqlValueHintDouble(double_t(floatValue)));
}
}
});
#endif
}
void registerFilters(arangodb::aql::AqlFunctionFeature& functions) {
using arangodb::iresearch::addFunction;
auto flags =
@ -969,7 +942,6 @@ void IResearchFeature::start() {
if (functions) {
registerFilters(*functions);
registerScorers(*functions);
registerFunctions(*functions);
} else {
LOG_TOPIC(WARN, arangodb::iresearch::TOPIC)
<< "failure to find feature 'AQLFunctions' while registering "
@ -1009,4 +981,4 @@ NS_END // iresearch
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------

128
arangod/IResearch/IResearchOrderFactory.cpp
View File

@ -21,8 +21,7 @@
/// @author Vasiliy Nabatchikov
////////////////////////////////////////////////////////////////////////////////
// otherwise define conflict between 3rdParty\date\include\date\date.h and
// 3rdParty\iresearch\core\shared.hpp
// otherwise define conflict between 3rdParty\date\include\date\date.h and 3rdParty\iresearch\core\shared.hpp
#if defined(_MSC_VER)
#include "date/date.h"
#undef NOEXCEPT
@ -30,10 +29,12 @@
#include "search/scorers.hpp"
#include "Aql/Ast.h"
#include "Aql/AstNode.h"
#include "Aql/ExecutionNode.h"
#include "Aql/Function.h"
#include "Aql/SortCondition.h"
#include "AqlHelper.h"
#include "Basics/fasthash.h"
#include "IResearchFeature.h"
#include "IResearchOrderFactory.h"
#include "VelocyPackHelper.h"
@ -46,23 +47,24 @@ NS_LOCAL
arangodb::aql::AstNode const EMPTY_ARGS(arangodb::aql::NODE_TYPE_ARRAY);
bool validateFuncArgs(arangodb::aql::AstNode const* args, arangodb::aql::Variable const& ref) {
// checks a specified args to be deterministic
// and retuns reference to a loop variable
arangodb::aql::Variable const* getScorerRef(arangodb::aql::AstNode const* args) noexcept {
if (!args || arangodb::aql::NODE_TYPE_ARRAY != args->type) {
return false;
return nullptr;
}
size_t const size = args->numMembers();
if (size < 1) {
return false; // invalid args
return nullptr; // invalid args
}
// 1st argument has to be reference to `ref`
auto const* arg0 = args->getMemberUnchecked(0);
if (!arg0 || arangodb::aql::NODE_TYPE_REFERENCE != arg0->type ||
reinterpret_cast<void const*>(&ref) != arg0->getData()) {
return false;
if (!arg0 || arangodb::aql::NODE_TYPE_REFERENCE != arg0->type) {
return nullptr;
}
for (size_t i = 1, size = args->numMembers(); i < size; ++i) {
@ -70,11 +72,11 @@ bool validateFuncArgs(arangodb::aql::AstNode const* args, arangodb::aql::Variabl
if (!arg || !arg->isDeterministic()) {
// we don't support non-deterministic arguments for scorers
return false;
return nullptr;
}
}
return true;
return reinterpret_cast<arangodb::aql::Variable const*>(arg0->getData());
}
bool makeScorer(irs::sort::ptr& scorer, irs::string_ref const& name,
@ -87,15 +89,12 @@ bool makeScorer(irs::sort::ptr& scorer, irs::string_ref const& name,
case 0:
break;
case 1: {
// ArangoDB, for API consistency, only supports scorers configurable via
// jSON
// ArangoDB, for API consistency, only supports scorers configurable via jSON
scorer = irs::scorers::get(name, irs::text_format::json, irs::string_ref::NIL);
if (!scorer) {
// ArangoDB, for API consistency, only supports scorers configurable via
// jSON
scorer = irs::scorers::get(name, irs::text_format::json,
"[]"); // pass arg as json array
// ArangoDB, for API consistency, only supports scorers configurable via jSON
scorer = irs::scorers::get(name, irs::text_format::json, "[]"); // pass arg as json array
}
} break;
default: { // fall through
@ -123,10 +122,8 @@ bool makeScorer(irs::sort::ptr& scorer, irs::string_ref const& name,
builder.close();
// ArangoDB, for API consistency, only supports scorers configurable via
// jSON
scorer = irs::scorers::get(name, irs::text_format::json,
builder.toJson()); // pass arg as json
// ArangoDB, for API consistency, only supports scorers configurable via jSON
scorer = irs::scorers::get(name, irs::text_format::json, builder.toJson()); // pass arg as json
}
}
@ -136,15 +133,16 @@ bool makeScorer(irs::sort::ptr& scorer, irs::string_ref const& name,
bool fromFCall(irs::sort::ptr* scorer, irs::string_ref const& scorerName,
arangodb::aql::AstNode const* args,
arangodb::iresearch::QueryContext const& ctx) {
if (!validateFuncArgs(args, *ctx.ref)) {
auto const* ref = getScorerRef(args);
if (ref != ctx.ref) {
// invalid arguments
return false;
}
if (!scorer) {
// cheap shallow check
// ArangoDB, for API consistency, only supports scorers configurable via
// jSON
// ArangoDB, for API consistency, only supports scorers configurable via jSON
return irs::scorers::exists(scorerName, irs::text_format::json);
}
@ -207,6 +205,85 @@ NS_END
NS_BEGIN(arangodb)
NS_BEGIN(iresearch)
// ----------------------------------------------------------------------------
// --SECTION-- ScorerReplacer implementation
// ----------------------------------------------------------------------------
void ScorerReplacer::replace(aql::CalculationNode& node) {
if (!node.expression()) {
return;
}
auto& expr = *node.expression();
auto* ast = expr.ast();
if (!expr.ast()) {
// ast is not set
return;
}
auto* exprNode = expr.nodeForModification();
if (!exprNode) {
// node is not set
return;
}
auto replaceScorers = [this, ast](aql::AstNode* node) {
if (aql::NODE_TYPE_FCALL == node->type || aql::NODE_TYPE_FCALL_USER == node->type) {
auto* ref = getScorerRef(node->getMember(0));
if (!ref) {
// invalid arguments or reference
return node;
}
QueryContext const ctx{nullptr, nullptr, nullptr, nullptr, ref};
if (!OrderFactory::scorer(nullptr, *node, ctx)) {
// not a scorer function
return node;
}
HashedScorer const key(ref, node);
auto it = _dedup.find(key);
if (it == _dedup.end()) {
// create variable
auto* var = ast->variables()->createTemporaryVariable();
it = _dedup.emplace(key, var).first;
}
return ast->createNodeReference(it->second);
}
return node;
};
// Try to modify root node of the expression
auto newNode = replaceScorers(exprNode);
if (exprNode != newNode) {
// simple expression, e.g LET x = BM25(d)
expr.replaceNode(newNode);
} else {
aql::Ast::traverseAndModify(exprNode, replaceScorers);
}
}
void ScorerReplacer::extract(aql::Variable const& var, std::vector<Scorer>& scorers) {
for (auto it = _dedup.begin(), end = _dedup.end(); it != end;) {
if (it->first.var == &var) {
scorers.emplace_back(it->second, it->first.node);
it = _dedup.erase(it);
} else {
++it;
}
}
}
// ----------------------------------------------------------------------------
// --SECTION-- OrderFactory implementation
// ----------------------------------------------------------------------------
@ -252,8 +329,7 @@ NS_BEGIN(iresearch)
if (!comparer) {
// cheap shallow check
// ArangoDB, for API consistency, only supports scorers configurable via
// jSON
// ArangoDB, for API consistency, only supports scorers configurable via jSON
return irs::scorers::exists(scorerName, irs::text_format::json);
}

108
arangod/IResearch/IResearchOrderFactory.h
View File

@ -24,6 +24,8 @@
#ifndef ARANGOD_IRESEARCH__IRESEARCH_ORDER_FACTORY_H
#define ARANGOD_IRESEARCH__IRESEARCH_ORDER_FACTORY_H 1
#include "AqlHelper.h"
#include "VocBase/voc-types.h"
#include "search/sort.hpp"
@ -31,7 +33,10 @@
NS_BEGIN(arangodb)
NS_BEGIN(aql)
class Ast;
struct AstNode;
class CalculationNode;
struct Expression;
struct Variable;
NS_END // aql
@ -50,18 +55,111 @@ NS_END // transaction
NS_BEGIN(iresearch)
////////////////////////////////////////////////////////////////////////////////
/// @struct OrderFactory
////////////////////////////////////////////////////////////////////////////////
struct OrderFactory {
////////////////////////////////////////////////////////////////////////////////
/// @brief determine if the 'node' can be converted into an iresearch scorer
/// if 'scorer' != nullptr then also append build iresearch scorer
/// there
/// if 'scorer' != nullptr then also append build iresearch scorer there
////////////////////////////////////////////////////////////////////////////////
static bool scorer(irs::sort::ptr* scorer, arangodb::aql::AstNode const& node,
arangodb::iresearch::QueryContext const& ctx);
static bool scorer(irs::sort::ptr* scorer, aql::AstNode const& node,
iresearch::QueryContext const& ctx);
static bool comparer(irs::sort::ptr* scorer, arangodb::aql::AstNode const& node);
////////////////////////////////////////////////////////////////////////////////
/// @brief determine if the 'node' can be converted into an iresearch scorer
/// if 'scorer' != nullptr then also append build iresearch comparer there
////////////////////////////////////////////////////////////////////////////////
static bool comparer(irs::sort::ptr* scorer, aql::AstNode const& node);
OrderFactory() = delete;
}; // OrderFactory
////////////////////////////////////////////////////////////////////////////////
/// @struct Scorer
/// @brief represents IResearch scorer in AQL terms
////////////////////////////////////////////////////////////////////////////////
struct Scorer {
Scorer() = default;
constexpr Scorer(aql::Variable const* var, aql::AstNode const* node) noexcept
: var(var), node(node) {}
constexpr bool operator==(Scorer const& rhs) const noexcept {
return var == rhs.var && node == rhs.node;
}
constexpr bool operator!=(Scorer const& rhs) const noexcept {
return !(*this == rhs);
}
aql::Variable const* var{}; // scorer variable
aql::AstNode const* node{}; // scorer node
}; // Scorer
////////////////////////////////////////////////////////////////////////////////
/// @class ScorerReplacer
/// @brief utility class that replaces scorer function call with corresponding
/// reference access
////////////////////////////////////////////////////////////////////////////////
class ScorerReplacer {
public:
ScorerReplacer() = default;
////////////////////////////////////////////////////////////////////////////////
/// @brief replaces all occurences of IResearch scorers in a specified node with
/// corresponding reference access
////////////////////////////////////////////////////////////////////////////////
void replace(aql::CalculationNode& node);
////////////////////////////////////////////////////////////////////////////////
/// @brief extracts replacement results for a given variable
////////////////////////////////////////////////////////////////////////////////
void extract(aql::Variable const& var, std::vector<Scorer>& scorers);
////////////////////////////////////////////////////////////////////////////////
/// @returns true if no scorers were replaced
////////////////////////////////////////////////////////////////////////////////
bool empty() const noexcept { return _dedup.empty(); }
////////////////////////////////////////////////////////////////////////////////
/// @brief visits all replaced scorer entries
////////////////////////////////////////////////////////////////////////////////
template <typename Visitor>
bool visit(Visitor visitor) const {
for (auto& entry : _dedup) {
if (!visitor(entry.first)) {
return false;
}
}
return true;
}
private:
struct HashedScorer : Scorer {
HashedScorer(aql::Variable const* var, aql::AstNode const* node)
: Scorer(var, node), hash(iresearch::hash(node)) {}
size_t hash;
}; // HashedScorer
struct ScorerHash {
size_t operator()(HashedScorer const& key) const noexcept {
return key.hash;
}
}; // ScorerHash
struct ScorerEqualTo {
bool operator()(HashedScorer const& lhs, HashedScorer const& rhs) const {
return iresearch::equalTo(lhs.node, rhs.node);
}
}; // ScorerEqualTo
typedef std::unordered_map<HashedScorer, aql::Variable const*, ScorerHash, ScorerEqualTo> DedupScorers;
DedupScorers _dedup;
}; // ScorerReplacer
NS_END // iresearch
NS_END // arangodb

11
arangod/IResearch/IResearchViewBlock.cpp
View File

@ -202,15 +202,16 @@ void IResearchViewBlockBase::reset() {
irs::order order;
irs::sort::ptr scorer;
for (auto const& sort : viewNode.sortCondition()) {
TRI_ASSERT(sort.node);
for (auto const& scorerNode : viewNode.scorers()) {
TRI_ASSERT(scorerNode.node);
if (!arangodb::iresearch::OrderFactory::scorer(&scorer, *sort.node, queryCtx)) {
if (!arangodb::iresearch::OrderFactory::scorer(&scorer, *scorerNode.node, queryCtx)) {
// failed to append sort
THROW_ARANGO_EXCEPTION(TRI_ERROR_BAD_PARAMETER);
}
order.add(sort.asc, std::move(scorer));
// sorting order doesn't matter
order.add(true, std::move(scorer));
}
// compile order
@ -427,7 +428,7 @@ bool IResearchViewBlock::resetIterator() {
auto const& viewNode =
*ExecutionNode::castTo<IResearchViewNode const*>(getPlanNode());
TRI_ASSERT(numScores == viewNode.sortCondition().size());
TRI_ASSERT(numScores == viewNode.scorers().size());
#endif
} else {
_scr = &irs::score::no_score();

77
arangod/IResearch/IResearchViewNode.cpp
View File

@ -33,7 +33,6 @@
#include "Basics/StringUtils.h"
#include "Cluster/ClusterInfo.h"
#include "IResearchCommon.h"
#include "IResearchOrderFactory.h"
#include "IResearchView.h"
#include "IResearchViewBlock.h"
#include "IResearchViewCoordinator.h"
@ -59,19 +58,18 @@ inline bool filterConditionIsEmpty(aql::AstNode const* filterCondition) {
// -----------------------------------------------------------------------------
void toVelocyPack(velocypack::Builder& builder,
std::vector<arangodb::iresearch::IResearchSort> const& sorts,
bool verbose) {
std::vector<arangodb::iresearch::Scorer> const& scorers, bool verbose) {
VPackArrayBuilder arrayScope(&builder);
for (auto const sort : sorts) {
for (auto const& scorer : scorers) {
VPackObjectBuilder objectScope(&builder);
builder.add("varId", VPackValue(sort.var->id));
builder.add("asc", VPackValue(sort.asc));
builder.add("id", VPackValue(scorer.var->id));
builder.add("name", VPackValue(scorer.var->name)); // for explainer.js
builder.add(VPackValue("node"));
sort.node->toVelocyPack(builder, verbose);
scorer.node->toVelocyPack(builder, verbose);
}
}
std::vector<arangodb::iresearch::IResearchSort> fromVelocyPack(
std::vector<arangodb::iresearch::Scorer> fromVelocyPack(
arangodb::aql::ExecutionPlan& plan, arangodb::velocypack::Slice const& slice) {
if (!slice.isArray()) {
LOG_TOPIC(ERR, arangodb::iresearch::TOPIC)
@ -85,11 +83,11 @@ std::vector<arangodb::iresearch::IResearchSort> fromVelocyPack(
auto const* vars = plan.getAst()->variables();
TRI_ASSERT(vars);
std::vector<arangodb::iresearch::IResearchSort> sorts;
std::vector<arangodb::iresearch::Scorer> scorers;
size_t i = 0;
for (auto const sortSlice : velocypack::ArrayIterator(slice)) {
auto const varIdSlice = sortSlice.get("varId");
auto const varIdSlice = sortSlice.get("id");
if (!varIdSlice.isNumber()) {
LOG_TOPIC(ERR, arangodb::iresearch::TOPIC)
@ -107,24 +105,14 @@ std::vector<arangodb::iresearch::IResearchSort> fromVelocyPack(
return {};
}
auto const ascSlice = sortSlice.get("asc");
if (!ascSlice.isBoolean()) {
LOG_TOPIC(ERR, arangodb::iresearch::TOPIC)
<< "malformed order mark at line " << i << "', boolean expected";
return {};
}
bool const asc = ascSlice.getBoolean();
// will be owned by Ast
auto* node = new aql::AstNode(ast, sortSlice.get("node"));
sorts.emplace_back(var, node, asc);
scorers.emplace_back(var, node);
++i;
}
return sorts;
return scorers;
}
// -----------------------------------------------------------------------------
@ -224,7 +212,7 @@ bool parseOptions(aql::AstNode const* optionsNode,
// -----------------------------------------------------------------------------
// in loop or non-deterministic
bool hasDependecies(aql::ExecutionPlan const& plan, aql::AstNode const& node,
bool hasDependencies(aql::ExecutionPlan const& plan, aql::AstNode const& node,
aql::Variable const& ref,
std::unordered_set<aql::Variable const*>& vars) {
if (!node.isDeterministic()) {
@ -281,16 +269,16 @@ int evaluateVolatility(arangodb::iresearch::IResearchViewNode const& node) {
// evaluate filter condition volatility
auto& filterCondition = node.filterCondition();
if (!::filterConditionIsEmpty(&filterCondition) && inInnerLoop) {
irs::set_bit<0>(::hasDependecies(plan, filterCondition, outVariable, vars), mask);
irs::set_bit<0>(::hasDependencies(plan, filterCondition, outVariable, vars), mask);
}
// evaluate sort condition volatility
auto& sortCondition = node.sortCondition();
if (!sortCondition.empty() && inInnerLoop) {
auto& scorers = node.scorers();
if (!scorers.empty() && inInnerLoop) {
vars.clear();
for (auto const& sort : sortCondition) {
if (::hasDependecies(plan, *sort.node, outVariable, vars)) {
for (auto const& scorer : scorers) {
if (::hasDependencies(plan, *scorer.node, outVariable, vars)) {
irs::set_bit<1>(mask);
break;
}
@ -315,19 +303,18 @@ namespace iresearch {
IResearchViewNode::IResearchViewNode(aql::ExecutionPlan& plan, size_t id,
TRI_vocbase_t& vocbase,
std::shared_ptr<const arangodb::LogicalView> const& view,
arangodb::aql::Variable const& outVariable,
arangodb::aql::AstNode* filterCondition,
arangodb::aql::AstNode* options,
std::vector<IResearchSort>&& sortCondition)
: arangodb::aql::ExecutionNode(&plan, id),
std::shared_ptr<const LogicalView> const& view,
aql::Variable const& outVariable,
aql::AstNode* filterCondition,
aql::AstNode* options, std::vector<Scorer>&& scorers)
: aql::ExecutionNode(&plan, id),
_vocbase(vocbase),
_view(view),
_outVariable(&outVariable),
// in case if filter is not specified
// set it to surrogate 'RETURN ALL' node
_filterCondition(filterCondition ? filterCondition : &ALL),
_sortCondition(std::move(sortCondition)) {
_scorers(std::move(scorers)) {
TRI_ASSERT(_view);
TRI_ASSERT(iresearch::DATA_SOURCE_TYPE == _view->type());
TRI_ASSERT(LogicalView::category() == _view->category());
@ -348,7 +335,7 @@ IResearchViewNode::IResearchViewNode(aql::ExecutionPlan& plan, velocypack::Slice
// in case if filter is not specified
// set it to surrogate 'RETURN ALL' node
_filterCondition(&ALL),
_sortCondition(fromVelocyPack(plan, base.get("sortCondition"))) {
_scorers(fromVelocyPack(plan, base.get("scorers"))) {
// view
auto const viewIdSlice = base.get("viewId");
@ -442,10 +429,10 @@ void IResearchViewNode::planNodeRegisters(std::vector<aql::RegisterId>& nrRegsHe
// }
// plan registers for output scores
for (auto const& sort : _sortCondition) {
for (auto const& scorer : _scorers) {
++nrRegsHere[depth];
++nrRegs[depth];
varInfo.emplace(sort.var->id, VarInfo(depth, totalNrRegs++));
varInfo.emplace(scorer.var->id, VarInfo(depth, totalNrRegs++));
}
}
@ -484,8 +471,8 @@ void IResearchViewNode::toVelocyPackHelper(VPackBuilder& nodes, unsigned flags)
}
// sort condition
nodes.add(VPackValue("sortCondition"));
::toVelocyPack(nodes, _sortCondition, flags != 0);
nodes.add(VPackValue("scorers"));
::toVelocyPack(nodes, _scorers, flags != 0);
// shards
{
@ -544,8 +531,8 @@ aql::ExecutionNode* IResearchViewNode::clone(aql::ExecutionPlan* plan, bool with
auto node =
std::make_unique<IResearchViewNode>(*plan, _id, _vocbase, _view, *outVariable,
const_cast<aql::AstNode*>(_filterCondition), nullptr,
decltype(_sortCondition)(_sortCondition));
const_cast<aql::AstNode*>(_filterCondition),
nullptr, decltype(_scorers)(_scorers));
node->_shards = _shards;
node->_options = _options;
node->_volatilityMask = _volatilityMask;
@ -579,6 +566,10 @@ void IResearchViewNode::getVariablesUsedHere(std::unordered_set<aql::Variable co
if (!::filterConditionIsEmpty(_filterCondition)) {
aql::Ast::getReferencedVariables(_filterCondition, vars);
}
for (auto& scorer : _scorers) {
aql::Ast::getReferencedVariables(scorer.node, vars);
}
}
void IResearchViewNode::filterCondition(aql::AstNode const* node) noexcept {
@ -665,7 +656,7 @@ std::unique_ptr<aql::ExecutionBlock> IResearchViewNode::createBlock(
LOG_TOPIC(TRACE, arangodb::iresearch::TOPIC)
<< "Finish getting snapshot for view '" << view.name() << "'";
if (_sortCondition.empty()) {
if (_scorers.empty()) {
// unordered case
return std::make_unique<IResearchViewUnorderedBlock>(*reader, engine, *this);
}

41
arangod/IResearch/IResearchViewNode.h
View File

@ -24,6 +24,8 @@
#ifndef ARANGOD_IRESEARCH__IRESEARCH_VIEW_NODE_H
#define ARANGOD_IRESEARCH__IRESEARCH_VIEW_NODE_H 1
#include "IResearchOrderFactory.h"
#include "Aql/Collection.h"
#include "Aql/ExecutionNode.h"
@ -36,25 +38,6 @@ class ExecutionEngine;
namespace iresearch {
struct IResearchSort {
IResearchSort() = default;
IResearchSort(aql::Variable const* var, aql::AstNode const* node, bool asc) noexcept
: var(var), node(node), asc(asc) {}
bool operator==(IResearchSort const& rhs) const noexcept {
return var == rhs.var && node == rhs.node && asc == rhs.asc;
}
bool operator!=(IResearchSort const& rhs) const noexcept {
return !(*this == rhs);
}
aql::Variable const* var{};
aql::AstNode const* node{};
bool asc{};
}; // IResearchSort
/// @brief class EnumerateViewNode
class IResearchViewNode final : public arangodb::aql::ExecutionNode {
friend class arangodb::aql::RedundantCalculationsReplacer;
@ -69,7 +52,7 @@ class IResearchViewNode final : public arangodb::aql::ExecutionNode {
IResearchViewNode(aql::ExecutionPlan& plan, size_t id, TRI_vocbase_t& vocbase,
std::shared_ptr<const arangodb::LogicalView> const& view,
aql::Variable const& outVariable, aql::AstNode* filterCondition,
aql::AstNode* options, std::vector<IResearchSort>&& sortCondition);
aql::AstNode* options, std::vector<Scorer>&& scorers);
IResearchViewNode(aql::ExecutionPlan&, velocypack::Slice const& base);
@ -94,10 +77,10 @@ class IResearchViewNode final : public arangodb::aql::ExecutionNode {
/// @brief getVariablesSetHere
std::vector<arangodb::aql::Variable const*> getVariablesSetHere() const override final {
std::vector<arangodb::aql::Variable const*> vars(1 + _sortCondition.size());
std::vector<arangodb::aql::Variable const*> vars(1 + _scorers.size());
*std::transform(_sortCondition.begin(), sortCondition().end(), vars.begin(),
[](IResearchSort const& sort) { return sort.var; }) = _outVariable;
*std::transform(_scorers.begin(), _scorers.end(), vars.begin(),
[](auto const& scorer) { return scorer.var; }) = _outVariable;
return vars;
}
@ -134,13 +117,11 @@ class IResearchViewNode final : public arangodb::aql::ExecutionNode {
std::vector<std::string>& shards() noexcept { return _shards; }
/// @brief return the condition to pass to the view
std::vector<IResearchSort> const& sortCondition() const noexcept {
return _sortCondition;
}
std::vector<Scorer> const& scorers() const noexcept { return _scorers; }
/// @brief set the sort condition to pass to the view
void sortCondition(std::vector<IResearchSort>&& sortCondition) noexcept {
_sortCondition = std::move(sortCondition);
void scorers(std::vector<Scorer>&& scorers) noexcept {
_scorers = std::move(scorers);
}
/// @brief getVariablesUsedHere, returning a vector
@ -184,8 +165,8 @@ class IResearchViewNode final : public arangodb::aql::ExecutionNode {
/// @brief filter node to pass to view
aql::AstNode const* _filterCondition;
/// @brief sortCondition to pass to the view
std::vector<IResearchSort> _sortCondition;
/// @brief scorers related to the view
std::vector<Scorer> _scorers;
/// @brief list of shards involved, need this for the cluster
std::vector<std::string> _shards;

318
arangod/IResearch/IResearchViewOptimizerRules.cpp
View File

@ -26,6 +26,7 @@
#include "Aql/Condition.h"
#include "Aql/ExecutionNode.h"
#include "Aql/ExecutionPlan.h"
#include "Aql/Function.h"
#include "Aql/Optimizer.h"
#include "Aql/Query.h"
#include "Aql/SortNode.h"
@ -38,6 +39,8 @@
#include "Utils/CollectionNameResolver.h"
#include "VocBase/LogicalCollection.h"
#include "utils/misc.hpp"
using namespace arangodb::iresearch;
using namespace arangodb::aql;
using EN = arangodb::aql::ExecutionNode;
@ -58,51 +61,6 @@ size_t numberOfShards(arangodb::CollectionNameResolver const& resolver,
return numberOfShards;
}
std::vector<arangodb::iresearch::IResearchSort> buildSort(
ExecutionPlan const& plan, arangodb::aql::Variable const& ref,
std::vector<std::pair<Variable const*, bool>> const& sorts,
std::map<VariableId, AstNode const*> const& vars, bool scorersOnly) {
std::vector<IResearchSort> entries;
QueryContext const ctx{nullptr, nullptr, nullptr, nullptr, &ref};
for (auto& sort : sorts) {
auto const* var = sort.first;
auto varId = var->id;
AstNode const* rootNode = nullptr;
auto it = vars.find(varId);
if (it != vars.end()) {
auto const* node = rootNode = it->second;
while (node && NODE_TYPE_ATTRIBUTE_ACCESS == node->type) {
node = node->getMember(0);
}
if (node && NODE_TYPE_REFERENCE == node->type) {
var = reinterpret_cast<Variable const*>(node->getData());
}
} else {
auto const* setter = plan.getVarSetBy(varId);
if (setter && EN::CALCULATION == setter->getType()) {
auto const* expr =
ExecutionNode::castTo<CalculationNode const*>(setter)->expression();
if (expr) {
rootNode = expr->node();
}
}
}
if (var && rootNode && (!scorersOnly || OrderFactory::scorer(nullptr, *rootNode, ctx))) {
entries.emplace_back(var, rootNode, sort.second);
}
}
return entries;
}
bool addView(arangodb::LogicalView const& view, arangodb::aql::Query& query) {
auto* collections = query.collections();
@ -120,167 +78,57 @@ bool addView(arangodb::LogicalView const& view, arangodb::aql::Query& query) {
return view.visitCollections(visitor);
}
///////////////////////////////////////////////////////////////////////////////
/// @class IResearchViewConditionFinder
///////////////////////////////////////////////////////////////////////////////
class IResearchViewConditionHandler final
: public arangodb::aql::WalkerWorker<ExecutionNode> {
public:
IResearchViewConditionHandler(ExecutionPlan& plan,
std::set<arangodb::iresearch::IResearchViewNode const*>& processedViewNodes) noexcept
: _plan(&plan), _processedViewNodes(&processedViewNodes) {}
bool optimizeSearchCondition(IResearchViewNode& viewNode, Query& query, ExecutionPlan& plan) {
auto view = viewNode.view();
virtual bool before(ExecutionNode*) override;
virtual bool enterSubquery(ExecutionNode*, ExecutionNode*) override {
return false;
// add view and linked collections to the query
if (!addView(*view, query)) {
THROW_ARANGO_EXCEPTION_MESSAGE(
TRI_ERROR_QUERY_PARSE,
"failed to process all collections linked with the view '" +
view->name() + "'");
}
private:
bool handleFilterCondition(ExecutionNode* en, Condition& condition);
// build search condition
Condition searchCondition(plan.getAst());
ExecutionPlan* _plan;
std::vector<std::pair<Variable const*, bool>> _sorts;
// map and set are 25-30% faster than corresponding
// unordered_set for small number of elements
std::map<VariableId, AstNode const*> _variableDefinitions;
std::set<arangodb::iresearch::IResearchViewNode const*>* _processedViewNodes;
}; // IResearchViewConditionFinder
if (!viewNode.filterConditionIsEmpty()) {
searchCondition.andCombine(&viewNode.filterCondition());
searchCondition.normalize(&plan); // normalize the condition
bool IResearchViewConditionHandler::before(ExecutionNode* en) {
switch (en->getType()) {
case EN::LIMIT:
// LIMIT invalidates the sort expression we already found
_sorts.clear();
break;
case EN::SINGLETON:
case EN::NORESULTS:
// in all these cases we better abort
return true;
case EN::SORT: {
// register which variables are used in a SORT
if (_sorts.empty()) {
for (auto& it : EN::castTo<SortNode const*>(en)->elements()) {
_sorts.emplace_back(it.var, it.ascending);
TRI_IF_FAILURE("IResearchViewConditionFinder::sortNode") {
THROW_ARANGO_EXCEPTION(TRI_ERROR_DEBUG);
}
}
if (searchCondition.isEmpty()) {
// condition is always false
for (auto const& x : viewNode.getParents()) {
plan.insertDependency(x, plan.registerNode(
std::make_unique<NoResultsNode>(&plan, plan.nextId())));
}
break;
return false;
}
case EN::CALCULATION: {
auto outvars = en->getVariablesSetHere();
TRI_ASSERT(outvars.size() == 1);
auto const& varsValid = viewNode.getVarsValid();
_variableDefinitions.emplace(
outvars[0]->id, EN::castTo<CalculationNode const*>(en)->expression()->node());
TRI_IF_FAILURE("IResearchViewConditionFinder::variableDefinition") {
THROW_ARANGO_EXCEPTION(TRI_ERROR_DEBUG);
}
break;
// remove all invalid variables from the condition
if (searchCondition.removeInvalidVariables(varsValid)) {
// removing left a previously non-empty OR block empty...
// this means we can't use the index to restrict the results
return false;
}
case EN::ENUMERATE_IRESEARCH_VIEW: {
auto node = EN::castTo<IResearchViewNode*>(en);
auto& view = *node->view();
// add view and linked collections to the query
TRI_ASSERT(_plan && _plan->getAst() && _plan->getAst()->query());
if (!addView(view, *_plan->getAst()->query())) {
THROW_ARANGO_EXCEPTION_MESSAGE(
TRI_ERROR_QUERY_PARSE,
"failed to process all collections linked with the view '" +
view.name() + "'");
}
if (_processedViewNodes->find(node) != _processedViewNodes->end()) {
// already optimized this node
break;
}
Condition filterCondition(_plan->getAst());
if (!node->filterConditionIsEmpty()) {
filterCondition.andCombine(&node->filterCondition());
if (!handleFilterCondition(en, filterCondition)) {
break;
}
}
auto sortCondition =
buildSort(*_plan, node->outVariable(), _sorts, _variableDefinitions,
true // node->isInInnerLoop() // build scorers only in case
// if we're inside a loop
);
if (filterCondition.isEmpty() && sortCondition.empty()) {
// no conditions left
break;
}
auto const canUseView =
!filterCondition.root() ||
FilterFactory::filter(nullptr,
{nullptr, nullptr, nullptr, nullptr, &node->outVariable()},
*filterCondition.root());
if (!canUseView) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_QUERY_PARSE,
"unsupported SEARCH condition");
}
node->filterCondition(filterCondition.root());
node->sortCondition(std::move(sortCondition));
TRI_IF_FAILURE("IResearchViewConditionFinder::insertViewNode") {
THROW_ARANGO_EXCEPTION(TRI_ERROR_DEBUG);
}
_processedViewNodes->insert(node);
break;
}
default:
// in these cases we simply ignore the intermediate nodes, note
// that we have taken care of nodes that could throw exceptions
// above.
break;
}
return false;
}
// check filter condition
auto const conditionValid =
!searchCondition.root() ||
FilterFactory::filter(nullptr,
{nullptr, nullptr, nullptr, nullptr, &viewNode.outVariable()},
*searchCondition.root());
bool IResearchViewConditionHandler::handleFilterCondition(ExecutionNode* en,
Condition& condition) {
// normalize the condition
condition.normalize(_plan);
TRI_IF_FAILURE("ConditionFinder::normalizePlan") {
THROW_ARANGO_EXCEPTION(TRI_ERROR_DEBUG);
if (!conditionValid) {
THROW_ARANGO_EXCEPTION_MESSAGE(TRI_ERROR_QUERY_PARSE,
"unsupported SEARCH condition");
}
if (condition.isEmpty()) {
// condition is always false
for (auto const& x : en->getParents()) {
auto noRes = new NoResultsNode(_plan, _plan->nextId());
_plan->registerNode(noRes);
_plan->insertDependency(x, noRes);
}
return false;
}
auto const& varsValid = en->getVarsValid();
// remove all invalid variables from the condition
if (condition.removeInvalidVariables(varsValid)) {
// removing left a previously non-empty OR block empty...
// this means we can't use the index to restrict the results
return false;
if (!searchCondition.isEmpty()) {
viewNode.filterCondition(searchCondition.root());
}
return true;
@ -288,56 +136,72 @@ bool IResearchViewConditionHandler::handleFilterCondition(ExecutionNode* en,
} // namespace
NS_BEGIN(arangodb)
NS_BEGIN(iresearch)
namespace arangodb {
namespace iresearch {
/// @brief move filters and sort conditions into views
void handleViewsRule(arangodb::aql::Optimizer* opt,
std::unique_ptr<arangodb::aql::ExecutionPlan> plan,
arangodb::aql::OptimizerRule const* rule) {
TRI_ASSERT(plan && plan->getAst() && plan->getAst()->query());
auto& query = *plan->getAst()->query();
// ensure 'Optimizer::addPlan' will be called
bool modified = false;
auto addPlan = irs::make_finally([opt, &plan, rule, &modified]() {
opt->addPlan(std::move(plan), rule, modified);
});
if (query.views().empty()) {
// nothing to do in absence of views
return;
}
SmallVector<ExecutionNode*>::allocator_type::arena_type a;
SmallVector<ExecutionNode*> nodes{a};
// try to find `EnumerateViewNode`s and process corresponding filters and
// sorts
plan->findEndNodes(nodes, true);
// replace scorers in all calculation nodes with references
plan->findNodesOfType(nodes, EN::CALCULATION, true);
// set of processed view nodes
std::set<IResearchViewNode const*> processedViewNodes;
ScorerReplacer scorerReplacer;
IResearchViewConditionHandler handler(*plan, processedViewNodes);
for (auto const& n : nodes) {
n->walk(handler);
for (auto* node : nodes) {
TRI_ASSERT(node && EN::CALCULATION == node->getType());
scorerReplacer.replace(*EN::castTo<CalculationNode*>(node));
}
if (!processedViewNodes.empty()) {
std::unordered_set<ExecutionNode*> toUnlink;
// register replaced scorers to be evaluated by corresponding view nodes
nodes.clear();
plan->findNodesOfType(nodes, EN::ENUMERATE_IRESEARCH_VIEW, true);
// remove sort setters covered by a view internally
for (auto* viewNode : processedViewNodes) {
TRI_ASSERT(viewNode);
std::vector<Scorer> scorers;
for (auto const& sort : viewNode->sortCondition()) {
auto const* var = sort.var;
for (auto* node : nodes) {
TRI_ASSERT(node && EN::ENUMERATE_IRESEARCH_VIEW == node->getType());
auto& viewNode = *EN::castTo<IResearchViewNode*>(node);
if (!var) {
continue;
}
auto* setter = plan->getVarSetBy(var->id);
if (!setter || EN::CALCULATION != setter->getType()) {
continue;
}
toUnlink.emplace(setter);
}
if (!optimizeSearchCondition(viewNode, query, *plan)) {
continue;
}
plan->unlinkNodes(toUnlink);
// find scorers that have to be evaluated by a view
scorerReplacer.extract(viewNode.outVariable(), scorers);
viewNode.scorers(std::move(scorers));
modified = true;
}
opt->addPlan(std::move(plan), rule, !processedViewNodes.empty());
// ensure all replaced scorers are covered by corresponding view nodes
scorerReplacer.visit([](Scorer const& scorer) -> bool {
TRI_ASSERT(scorer.node);
auto const funcName = iresearch::getFuncName(*scorer.node);
THROW_ARANGO_EXCEPTION_FORMAT(
TRI_ERROR_QUERY_FUNCTION_ARGUMENT_TYPE_MISMATCH,
"Non ArangoSearch view variable '%s' is used in scorer function '%s'",
scorer.var->name.c_str(), funcName.c_str());
});
}
void scatterViewInClusterRule(arangodb::aql::Optimizer* opt,
@ -454,9 +318,9 @@ void scatterViewInClusterRule(arangodb::aql::Optimizer* opt,
opt->addPlan(std::move(plan), rule, wasModified);
}
NS_END // iresearch
NS_END // arangodb
} // namespace iresearch
} // namespace arangodb
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------

10
js/common/modules/@arangodb/aql/explainer.js
View File

@ -1020,7 +1020,15 @@ function processQuery (query, explain, planIndex) {
if (node.condition && node.condition.hasOwnProperty('type')) {
condition = ' ' + keyword('SEARCH') + ' ' + buildExpression(node.condition);
}
return keyword('FOR') + ' ' + variableName(node.outVariable) + ' ' + keyword('IN') + ' ' + view(node.view) + condition + ' ' + annotation('/* view query */');
var scorers = '';
if (node.scorers && node.scorers.length > 0) {
scorers = keyword(' LET ' ) + node.scorers.map(function(scorer) {
return variableName(scorer) + ' = ' + buildExpression(scorer.node);
}).join(', ');
}
return keyword('FOR') + ' ' + variableName(node.outVariable) + ' ' + keyword('IN') + ' ' + view(node.view) + condition + scorers + ' ' + annotation('/* view query */');
case 'IndexNode':
collectionVariables[node.outVariable.id] = node.collection;
node.indexes.forEach(function(idx, i) { iterateIndexes(idx, i, node, types, false); });

1
tests/CMakeLists.txt
View File

@ -44,6 +44,7 @@ if (USE_IRESEARCH)
IResearch/IResearchQueryNumericTerm-test.cpp
IResearch/IResearchQueryOr-test.cpp
IResearch/IResearchQueryPhrase-test.cpp
IResearch/IResearchQueryScorer-test.cpp
IResearch/IResearchQuerySelectAll-test.cpp
IResearch/IResearchQueryStartsWith-test.cpp
IResearch/IResearchQueryStringTerm-test.cpp

340
tests/IResearch/IResearchQueryJoin-test.cpp
View File

@ -68,7 +68,6 @@
#include "IResearch/VelocyPackHelper.h"
#include "analysis/analyzers.hpp"
#include "analysis/token_attributes.hpp"
#include "search/scorers.hpp"
#include "utils/utf8_path.hpp"
#include <velocypack/Iterator.h>
@ -77,101 +76,6 @@ extern const char* ARGV0; // defined in main.cpp
NS_LOCAL
struct CustomScorer : public irs::sort {
struct Prepared: public irs::sort::prepared_base<float_t> {
public:
DECLARE_FACTORY(Prepared);
Prepared(float_t i)
: i(i) {
}
virtual void add(irs::byte_type* dst, const irs::byte_type* src) const override {
score_cast(dst) += score_cast(src);
}
virtual irs::flags const& features() const override {
return irs::flags::empty_instance();
}
virtual bool less(const irs::byte_type* lhs, const irs::byte_type* rhs) const override {
return score_cast(lhs) < score_cast(rhs);
}
virtual irs::sort::collector::ptr prepare_collector() const override {
return nullptr;
}
virtual void prepare_score(irs::byte_type* score) const override {
score_cast(score) = 0.f;
}
virtual irs::sort::scorer::ptr prepare_scorer(
irs::sub_reader const&,
irs::term_reader const&,
irs::attribute_store const&,
irs::attribute_view const&
) const override {
struct Scorer : public irs::sort::scorer {
Scorer(float_t score): i(score) { }
virtual void score(irs::byte_type* score_buf) override {
*reinterpret_cast<score_t*>(score_buf) = i;
}
float_t i;
};
return irs::sort::scorer::make<Scorer>(i);
}
float_t i;
};
static ::iresearch::sort::type_id const& type() {
static ::iresearch::sort::type_id TYPE("customscorer");
return TYPE;
}
static irs::sort::ptr make(irs::string_ref const& args) {
if (args.null()) {
return std::make_shared<CustomScorer>(0.f);
}
// velocypack::Parser::fromJson(...) will throw exception on parse error
auto json = arangodb::velocypack::Parser::fromJson(args.c_str(), args.size());
auto slice = json ? json->slice() : arangodb::velocypack::Slice();
if (!slice.isArray()) {
return nullptr; // incorrect argument format
}
arangodb::velocypack::ArrayIterator itr(slice);
if (!itr.valid()) {
return nullptr;
}
auto const value = itr.value();
if (!value.isNumber()) {
return nullptr;
}
return std::make_shared<CustomScorer>(itr.value().getNumber<size_t>());
}
CustomScorer(size_t i) : irs::sort(CustomScorer::type()), i(i) {}
virtual irs::sort::prepared::ptr prepare() const override {
return irs::memory::make_unique<Prepared>(static_cast<float_t>(i));
}
size_t i;
}; // CustomScorer
REGISTER_SCORER_JSON(CustomScorer, CustomScorer::make);
// -----------------------------------------------------------------------------
// --SECTION-- setup / tear-down
// -----------------------------------------------------------------------------
@ -1362,12 +1266,6 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
}
// invalid reference in scorer
//
// FOR x IN 0..5
// FOR d IN testView
// SEARCH d.seq == x
// SORT customscorer(x,x)
// RETURN d;
{
std::string const query = "FOR d IN testView FOR i IN 0..5 SORT tfidf(i) DESC RETURN d";
@ -1376,7 +1274,7 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
));
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NOT_IMPLEMENTED == queryResult.code);
REQUIRE(TRI_ERROR_QUERY_FUNCTION_ARGUMENT_TYPE_MISMATCH == queryResult.code);
}
// FOR i IN 1..5
@ -1457,6 +1355,85 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
CHECK(expectedDoc == expectedDocs.end());
}
{
std::string const query = "LET attr = _NONDETERM_('seq') "
"FOR i IN 1..5 "
" FOR x IN collection_1 FILTER x.seq == i "
" FOR d IN testView SEARCH d.seq == x.seq AND d.name == x.name "
" SORT customscorer(d, x[attr]) DESC "
"RETURN d";
CHECK(arangodb::tests::assertRules(
vocbase, query,
{
arangodb::aql::OptimizerRule::handleArangoSearchViewsRule,
}
));
std::vector<arangodb::velocypack::Slice> expectedDocs {
arangodb::velocypack::Slice(insertedDocsView[4].vpack()),
arangodb::velocypack::Slice(insertedDocsView[2].vpack()),
};
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NO_ERROR == queryResult.code);
auto result = queryResult.result->slice();
CHECK(result.isArray());
arangodb::velocypack::ArrayIterator resultIt(result);
REQUIRE(expectedDocs.size() == resultIt.size());
// Check documents
auto expectedDoc = expectedDocs.begin();
for (;resultIt.valid(); resultIt.next(), ++expectedDoc) {
auto const actualDoc = resultIt.value();
auto const resolved = actualDoc.resolveExternals();
CHECK((0 == arangodb::basics::VelocyPackHelper::compare(arangodb::velocypack::Slice(*expectedDoc), resolved, true)));
}
CHECK(expectedDoc == expectedDocs.end());
}
// FOR i IN 1..5
// FOR x IN collection_0 SEARCH x.seq == i
// FOR d IN SEARCH d.seq == x.seq && d.name == x.name
// SORT customscorer(d, x.seq)
{
std::string const query = "FOR i IN 1..5 FOR x IN collection_1 FILTER x.seq == i FOR d IN testView SEARCH d.seq == x.seq AND d.name == x.name SORT customscorer(d, x['seq']) DESC RETURN d";
CHECK(arangodb::tests::assertRules(
vocbase, query,
{
arangodb::aql::OptimizerRule::handleArangoSearchViewsRule,
}
));
std::vector<arangodb::velocypack::Slice> expectedDocs {
arangodb::velocypack::Slice(insertedDocsView[4].vpack()),
arangodb::velocypack::Slice(insertedDocsView[2].vpack()),
};
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NO_ERROR == queryResult.code);
auto result = queryResult.result->slice();
CHECK(result.isArray());
arangodb::velocypack::ArrayIterator resultIt(result);
REQUIRE(expectedDocs.size() == resultIt.size());
// Check documents
auto expectedDoc = expectedDocs.begin();
for (;resultIt.valid(); resultIt.next(), ++expectedDoc) {
auto const actualDoc = resultIt.value();
auto const resolved = actualDoc.resolveExternals();
CHECK((0 == arangodb::basics::VelocyPackHelper::compare(arangodb::velocypack::Slice(*expectedDoc), resolved, true)));
}
CHECK(expectedDoc == expectedDocs.end());
}
// unable to retrieve `d.seq` from self-referenced variable
// FOR i IN 1..5
// FOR d IN SEARCH d.seq == i SORT customscorer(d, d.seq)
@ -1574,8 +1551,6 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
CHECK(expectedDoc == expectedDocs.end());
}
// we don't support scorers as a part of any expression (in sort or filter)
//
// FOR i IN 1..5
// FOR d IN testView SEARCH d.seq == i
// FOR x IN collection_1 FILTER x.seq == d.seq && x.seq == TFIDF(d)
@ -1583,7 +1558,7 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
std::string const query =
"FOR i IN 1..5 "
" FOR d IN testView SEARCH d.seq == i "
" FOR x IN collection_1 FILTER x.seq == d.seq && x.seq == TFIDF(d)"
" FOR x IN collection_1 FILTER x.seq == d.seq && x.seq == customscorer(d, i)"
"RETURN x";
CHECK(arangodb::tests::assertRules(
@ -1592,21 +1567,37 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
}
));
std::vector<arangodb::velocypack::Slice> expectedDocs {
arangodb::velocypack::Slice(insertedDocsView[2].vpack()),
arangodb::velocypack::Slice(insertedDocsView[4].vpack()),
};
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NOT_IMPLEMENTED == queryResult.code);
REQUIRE(TRI_ERROR_NO_ERROR == queryResult.code);
auto result = queryResult.result->slice();
CHECK(result.isArray());
arangodb::velocypack::ArrayIterator resultIt(result);
REQUIRE(expectedDocs.size() == resultIt.size());
// Check documents
auto expectedDoc = expectedDocs.begin();
for (;resultIt.valid(); resultIt.next(), ++expectedDoc) {
auto const actualDoc = resultIt.value();
auto const resolved = actualDoc.resolveExternals();
CHECK((0 == arangodb::basics::VelocyPackHelper::compare(arangodb::velocypack::Slice(*expectedDoc), resolved, true)));
}
CHECK(expectedDoc == expectedDocs.end());
}
// we don't support scorers as a part of any expression (in sort or filter)
//
// FOR i IN 1..5
// FOR d IN testView SEARCH d.seq == i
// FOR x IN collection_1 FILTER x.seq == d.seq && x.seq == TFIDF(d)
{
std::string const query =
"FOR i IN 1..5 "
" FOR d IN testView SEARCH d.seq == i "
" FOR x IN collection_1 FILTER x.seq == d.seq "
"SORT 1 + TFIDF(d)"
"SORT 1 + customscorer(d, i) DESC "
"RETURN d";
CHECK(arangodb::tests::assertRules(
@ -1615,41 +1606,36 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
}
));
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NOT_IMPLEMENTED == queryResult.code);
}
// we don't support scorers outside the view node
//
// FOR d IN (FOR c IN testView SEARCH c.name >= 'E' && c.seq < 10 SORT customscorer(c) DESC LIMIT 3 RETURN c)
// FOR x IN collection_1 FILTER x.seq == d.seq
// SORT customscorer(d, x.seq)
{
std::string const query =
"FOR d IN (FOR c IN testView SEARCH c.name >= 'E' && c.seq < 10 SORT customscorer(c) DESC LIMIT 3 RETURN c) "
" FOR x IN collection_1 FILTER x.seq == d.seq "
" SORT customscorer(d, x.seq) "
"RETURN x";
CHECK(arangodb::tests::assertRules(
vocbase, query, {
arangodb::aql::OptimizerRule::handleArangoSearchViewsRule,
}
));
std::vector<arangodb::velocypack::Slice> expectedDocs {
arangodb::velocypack::Slice(insertedDocsView[4].vpack()),
arangodb::velocypack::Slice(insertedDocsView[2].vpack()),
};
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NOT_IMPLEMENTED == queryResult.code);
REQUIRE(TRI_ERROR_NO_ERROR == queryResult.code);
auto result = queryResult.result->slice();
CHECK(result.isArray());
arangodb::velocypack::ArrayIterator resultIt(result);
REQUIRE(expectedDocs.size() == resultIt.size());
// Check documents
auto expectedDoc = expectedDocs.begin();
for (;resultIt.valid(); resultIt.next(), ++expectedDoc) {
auto const actualDoc = resultIt.value();
auto const resolved = actualDoc.resolveExternals();
CHECK((0 == arangodb::basics::VelocyPackHelper::compare(arangodb::velocypack::Slice(*expectedDoc), resolved, true)));
}
CHECK(expectedDoc == expectedDocs.end());
}
// multiple sorts (not supported now)
// FOR i IN 1..5
// FOR d IN SEARCH d.seq == i SORT customscorer(d, i) ASC
// FOR x IN collection_0 FILTER x.seq == d.seq && x.name == d.name
// SORT customscorer(d, i) DESC
// multiple sorts
{
std::string const query =
"FOR i IN 1..5 "
" FOR d IN testView SEARCH d.seq == i SORT tfidf(d, i) ASC "
" FOR d IN testView SEARCH d.seq == i SORT tfidf(d, i > 0) ASC "
" FOR x IN collection_1 FILTER x.seq == d.seq && x.name == d.name "
"SORT customscorer(d, i) DESC RETURN d";
@ -1665,26 +1651,64 @@ TEST_CASE("IResearchQueryTestJoin", "[iresearch][iresearch-query]") {
};
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_NOT_IMPLEMENTED == queryResult.code);
REQUIRE(TRI_ERROR_NO_ERROR == queryResult.code);
// auto result = queryResult.result->slice();
// CHECK(result.isArray());
//
// arangodb::velocypack::ArrayIterator resultIt(result);
// REQUIRE(expectedDocs.size() == resultIt.size());
//
// // Check documents
// auto expectedDoc = expectedDocs.begin();
// for (;resultIt.valid(); resultIt.next(), ++expectedDoc) {
// auto const actualDoc = resultIt.value();
// auto const resolved = actualDoc.resolveExternals();
//
// CHECK((0 == arangodb::basics::VelocyPackHelper::compare(arangodb::velocypack::Slice(*expectedDoc), resolved, true)));
// }
// CHECK(expectedDoc == expectedDocs.end());
auto result = queryResult.result->slice();
CHECK(result.isArray());
arangodb::velocypack::ArrayIterator resultIt(result);
REQUIRE(expectedDocs.size() == resultIt.size());
// Check documents
auto expectedDoc = expectedDocs.begin();
for (;resultIt.valid(); resultIt.next(), ++expectedDoc) {
auto const actualDoc = resultIt.value();
auto const resolved = actualDoc.resolveExternals();
CHECK((0 == arangodb::basics::VelocyPackHelper::compare(arangodb::velocypack::Slice(*expectedDoc), resolved, true)));
}
CHECK(expectedDoc == expectedDocs.end());
}
// x.seq is used before being assigned
{
std::string const query =
"FOR d IN testView SEARCH d.name >= 'E' && d.seq < 10 "
" SORT customscorer(d) DESC "
" LIMIT 3 "
" FOR x IN collection_1 FILTER x.seq == d.seq "
" SORT customscorer(d, x.seq) "
"RETURN x";
CHECK(arangodb::tests::assertRules(
vocbase, query, {
arangodb::aql::OptimizerRule::handleArangoSearchViewsRule,
}
));
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_BAD_PARAMETER == queryResult.code);
}
// x.seq is used before being assigned
{
std::string const query =
"FOR d IN (FOR c IN testView SEARCH c.name >= 'E' && c.seq < 10 SORT customscorer(c) DESC LIMIT 3 RETURN c) "
" FOR x IN collection_1 FILTER x.seq == d.seq "
" SORT customscorer(d, x.seq) "
"RETURN x";
CHECK(arangodb::tests::assertRules(
vocbase, query, {
arangodb::aql::OptimizerRule::handleArangoSearchViewsRule,
}
));
auto queryResult = arangodb::tests::executeQuery(vocbase, query);
REQUIRE(TRI_ERROR_BAD_PARAMETER == queryResult.code);
}
}
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------

1817
tests/IResearch/IResearchQueryScorer-test.cpp Normal file
View File

File diff suppressed because it is too large Load Diff

40
tests/IResearch/IResearchViewNode-test.cpp
View File

@ -200,7 +200,7 @@ SECTION("construct") {
CHECK(query.plan() == node.plan());
CHECK(42 == node.id());
CHECK(logicalView == node.view());
CHECK(node.sortCondition().empty());
CHECK(node.scorers().empty());
CHECK(!node.volatility().first); // filter volatility
CHECK(!node.volatility().second); // sort volatility
CHECK(node.getVariablesUsedHere().empty());
@ -245,7 +245,7 @@ SECTION("construct") {
CHECK(query.plan() == node.plan());
CHECK(42 == node.id());
CHECK(logicalView == node.view());
CHECK(node.sortCondition().empty());
CHECK(node.scorers().empty());
CHECK(!node.volatility().first); // filter volatility
CHECK(!node.volatility().second); // sort volatility
CHECK(node.getVariablesUsedHere().empty());
@ -316,7 +316,7 @@ SECTION("construct") {
CHECK(query.plan() == node.plan());
CHECK(42 == node.id());
CHECK(logicalView == node.view());
CHECK(node.sortCondition().empty());
CHECK(node.scorers().empty());
CHECK(!node.volatility().first); // filter volatility
CHECK(!node.volatility().second); // sort volatility
CHECK(node.getVariablesUsedHere().empty());
@ -418,7 +418,7 @@ SECTION("constructFromVPackSingleServer") {
CHECK(query.plan() == node.plan());
CHECK(42 == node.id());
CHECK(logicalView == node.view());
CHECK(node.sortCondition().empty());
CHECK(node.scorers().empty());
CHECK(!node.volatility().first); // filter volatility
CHECK(!node.volatility().second); // sort volatility
CHECK(node.getVariablesUsedHere().empty());
@ -453,7 +453,7 @@ SECTION("constructFromVPackSingleServer") {
CHECK(query.plan() == node.plan());
CHECK(42 == node.id());
CHECK(logicalView == node.view());
CHECK(node.sortCondition().empty());
CHECK(node.scorers().empty());
CHECK(!node.volatility().first); // filter volatility
CHECK(!node.volatility().second); // sort volatility
CHECK(node.getVariablesUsedHere().empty());
@ -488,7 +488,7 @@ SECTION("constructFromVPackSingleServer") {
CHECK(query.plan() == node.plan());
CHECK(42 == node.id());
CHECK(logicalView == node.view());
CHECK(node.sortCondition().empty());
CHECK(node.scorers().empty());
CHECK(!node.volatility().first); // filter volatility
CHECK(!node.volatility().second); // sort volatility
CHECK(node.getVariablesUsedHere().empty());
@ -553,7 +553,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.getCost() == cloned.getCost());
@ -581,7 +581,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -609,7 +609,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -658,7 +658,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.getCost() == cloned.getCost());
@ -686,7 +686,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -714,7 +714,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -758,7 +758,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -790,7 +790,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -821,7 +821,7 @@ SECTION("clone") {
CHECK(&node.vocbase() == &cloned.vocbase());
CHECK(node.view() == cloned.view());
CHECK(&node.filterCondition() == &cloned.filterCondition());
CHECK(node.sortCondition() == cloned.sortCondition());
CHECK(node.scorers() == cloned.scorers());
CHECK(node.volatility() == cloned.volatility());
CHECK(node.options().forceSync == cloned.options().forceSync);
@ -892,7 +892,7 @@ SECTION("serialize") {
CHECK(&node.vocbase() == &deserialized.vocbase());
CHECK(node.view() == deserialized.view());
CHECK(&node.filterCondition() == &deserialized.filterCondition());
CHECK(node.sortCondition() == deserialized.sortCondition());
CHECK(node.scorers() == deserialized.scorers());
CHECK(node.volatility() == deserialized.volatility());
CHECK(node.options().forceSync == deserialized.options().forceSync);
@ -916,7 +916,7 @@ SECTION("serialize") {
CHECK(&node.vocbase() == &deserialized.vocbase());
CHECK(node.view() == deserialized.view());
CHECK(&node.filterCondition() == &deserialized.filterCondition());
CHECK(node.sortCondition() == deserialized.sortCondition());
CHECK(node.scorers() == deserialized.scorers());
CHECK(node.volatility() == deserialized.volatility());
CHECK(node.options().forceSync == deserialized.options().forceSync);
@ -980,7 +980,7 @@ SECTION("serialize") {
CHECK(&node.vocbase() == &deserialized.vocbase());
CHECK(node.view() == deserialized.view());
CHECK(&node.filterCondition() == &deserialized.filterCondition());
CHECK(node.sortCondition() == deserialized.sortCondition());
CHECK(node.scorers() == deserialized.scorers());
CHECK(node.volatility() == deserialized.volatility());
CHECK(node.options().forceSync == deserialized.options().forceSync);
@ -1004,7 +1004,7 @@ SECTION("serialize") {
CHECK(&node.vocbase() == &deserialized.vocbase());
CHECK(node.view() == deserialized.view());
CHECK(&node.filterCondition() == &deserialized.filterCondition());
CHECK(node.sortCondition() == deserialized.sortCondition());
CHECK(node.scorers() == deserialized.scorers());
CHECK(node.volatility() == deserialized.volatility());
CHECK(node.options().forceSync == deserialized.options().forceSync);
@ -1220,4 +1220,4 @@ SECTION("createBlockCoordinator") {
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------

95
tests/IResearch/common.cpp
View File

@ -125,6 +125,101 @@ struct BoostScorer : public irs::sort {
REGISTER_SCORER_JSON(BoostScorer, BoostScorer::make);
struct CustomScorer : public irs::sort {
struct Prepared: public irs::sort::prepared_base<float_t> {
public:
DECLARE_FACTORY(Prepared);
Prepared(float_t i)
: i(i) {
}
virtual void add(irs::byte_type* dst, const irs::byte_type* src) const override {
score_cast(dst) += score_cast(src);
}
virtual irs::flags const& features() const override {
return irs::flags::empty_instance();
}
virtual bool less(const irs::byte_type* lhs, const irs::byte_type* rhs) const override {
return score_cast(lhs) < score_cast(rhs);
}
virtual irs::sort::collector::ptr prepare_collector() const override {
return nullptr;
}
virtual void prepare_score(irs::byte_type* score) const override {
score_cast(score) = 0.f;
}
virtual irs::sort::scorer::ptr prepare_scorer(
irs::sub_reader const&,
irs::term_reader const&,
irs::attribute_store const&,
irs::attribute_view const&
) const override {
struct Scorer : public irs::sort::scorer {
Scorer(float_t score): i(score) { }
virtual void score(irs::byte_type* score_buf) override {
*reinterpret_cast<score_t*>(score_buf) = i;
}
float_t i;
};
return irs::sort::scorer::make<Scorer>(i);
}
float_t i;
};
static ::iresearch::sort::type_id const& type() {
static ::iresearch::sort::type_id TYPE("customscorer");
return TYPE;
}
static irs::sort::ptr make(irs::string_ref const& args) {
if (args.null()) {
return std::make_shared<CustomScorer>(0.f);
}
// velocypack::Parser::fromJson(...) will throw exception on parse error
auto json = arangodb::velocypack::Parser::fromJson(args.c_str(), args.size());
auto slice = json ? json->slice() : arangodb::velocypack::Slice();
if (!slice.isArray()) {
return nullptr; // incorrect argument format
}
arangodb::velocypack::ArrayIterator itr(slice);
if (!itr.valid()) {
return nullptr;
}
auto const value = itr.value();
if (!value.isNumber()) {
return nullptr;
}
return std::make_shared<CustomScorer>(itr.value().getNumber<size_t>());
}
CustomScorer(size_t i) : irs::sort(CustomScorer::type()), i(i) {}
virtual irs::sort::prepared::ptr prepare() const override {
return irs::memory::make_unique<Prepared>(static_cast<float_t>(i));
}
size_t i;
}; // CustomScorer
REGISTER_SCORER_JSON(CustomScorer, CustomScorer::make);
}
namespace arangodb {

4
tests/js/server/aql/aql-optimizer-rule-inline-subqueries.js
View File

@ -280,7 +280,7 @@ function optimizerRuleViewTestSuite () {
assertEqual("v", viewNode.condition.subNodes[0].subNodes[0].subNodes[0].subNodes[0].name);
assertEqual("reference", viewNode.condition.subNodes[0].subNodes[0].subNodes[1].type);
assertEqual("outer", viewNode.condition.subNodes[0].subNodes[0].subNodes[1].name);
assertEqual([], viewNode.sortCondition);
assertEqual([], viewNode.scorers);
},
testNoVariableReplacementInSearchCondition : function () {
@ -305,7 +305,7 @@ function optimizerRuleViewTestSuite () {
assertEqual("reference", viewNode.condition.subNodes[0].subNodes[0].subNodes[0].subNodes[0].type);
assertEqual("v", viewNode.condition.subNodes[0].subNodes[0].subNodes[0].subNodes[0].name);
assertEqual("value", viewNode.condition.subNodes[0].subNodes[0].subNodes[1].type);
assertEqual([], viewNode.sortCondition);
assertEqual([], viewNode.scorers);
},
};