1
0
Fork 0
Commit Graph

16 Commits

Author SHA1 Message Date
Iurii Popov 26309c5c96 Feature/issue #642 (#10352)
* Index late materialization draft

* Index late materialization. Refactoring.

* Index late materialization. Expansion draft.

* HashIndexMock

* Refactoring.

* Hash index mock expansion fixes

* Fix compare of attributes.

* Hash index mock. Last expansion support.

* Hash index mock. Take value for array from document, not equal key.

* Optimizations.

* Query tests

* lateMaterialized improvements

* velocypack fixes and tests for index node

* Fix after merge.

* Const expression in a condition test

* Clone tests

* More tests

* Fix for materialized view (nullptr var)

* Fixed index vars registers count planning

* javascript tests

* Fixed cluster tests

* Cluster support

* MaterializeExecutor and MaterializerExecutorInfos refactoring

* More tests

* js tests rocksdb only

* js test. Removed print()

* js testd added skiplist

* C++ unit tests for invalid json

* js test var i

* js tests == -> ===

* iFix js tests ===

* jslint ;

* int unsigned int fix

* double deletion fix

* refactoring

* Added const for collection source

* override final, added materialize namespace, fixed femove in hash index mock

* Removed expansion support. Index can contain null value in case of several object with same fields

* isNumber<>

* Optimization for single collection case

* Renaming

* refactoring

* Unit tests. Added checking documents

* !sliceIt.valid() instead of !(!=)

* Fix primary index (object instead of array)

* Edge collection test

* Windows compilation fix

* IndexExecutor getCallback optimization

* ADB_UNLIKELY

* explainer.js

* removed unsused variable in js test

* fix explainer.js

* do not use several indexes with js test

* Removed several indexes implementation

* space

* fix explainer.js

* fix explainer.js

* code alignment
2019-11-07 19:16:33 +03:00
Dan Larkin-York 6fd419714f Move containers into their own folder and namespace. (#10244) 2019-10-14 10:55:55 +02:00
Dronplane a1014ac701 Bug fix/internal issue #600 - Late document materialization (#10006)
* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* adding plan traversal for rule

* Rule execution block developing

* Experiment with additional block

* LIMIT node modifications

* implemented late materialization base version.

* Code cleanup

* Added check for variable usage

* Implemented serialization and cloning

* Fixed build errors

* Implemented late materialization for  LimitNode

* Moved materialization to Sort Node

* Removed LimitNode modiications

* Implemented skipRows to speedup single server

* Added tests for optimizer rule

* fixed build

* Fix mac build

* Fixed mac build

* Fixed PR test run errors

* Added tmp fix for cluster test runs

* removed debug print

* Make materialization stick to last sort node. Add late materialization to explainer

* Updated optimization rule to search optimal SortNode to do materialization

* Refined optimization rules for cluster

* Added tests for SortExecutor and IResearchViewNode

* Fixed  mac buid. Removed misplaced assertion.

* Fixed build

* Updated tests. Coded cleanup

* Code cleanup

* made function name for SortedExecutor match name in ConstraintSortedExecutor

* cleanup

* Fixed test run

* made scoring results stable across cluster

* Make optimize rule stop if  sort node for limit node is not suitable

* Disabled constrained sort for materializing SortNode

* reverted SortNode modifications

* Implemented separate materializer node

* Fixed tests  to account new materialize node

* Code cleanup

* reverted debugging change

* Fixed test jslint error  and comments

* MaterializeNode renamed

* enum value renamed for materialize node

* Code cleanup

* code cleanup and optimization

* Fixed clang warning

* Applied review comments

* Out variable planning moved to materialize node

* inlined getters in ReadContex
2019-10-08 18:38:47 +03:00
Tobias Gödderz 82b0a5e745 De-inline in AQL (#10025)
* Enable IPO (LTO) via CMake

* Use separate IPO_ENABLED variable

* Enabled IPO for more binaries

* Enabled IPO for arangobackup

* Suppress an MSVC warning (though it is not unjustified...)

* Fix static builds, broken due to Policy CMP0060 introduced in cmake 3.3

* Removed policies superseded by minimum cmake version

* Disable IPO with google tests due to a g++ bug

* Disable IPO with google tests only on AUTO

* Disable warning for correct line

* Begin de-inlining Aql

* de-inlining Aql: 2nd batch

* de-inlining Aql: 3rd batch

* de-inlining Aql: 4th batch

* Moved code out of ifdef

* Enabled IPO for additional libs

* Fixed some problems found with IPO enabled

- Fixed ODR violation in Pregel
- Removed unused code in ClusterMethods
- Avoid possible uninitialized variable usage

* Set IPO globally, except for 3rdParty libs

* De-inlined AstHelper again (was undone due to a previous merge conflict)

* Added missing .cpp file

* Fixed compilation with MSVC

* Removed superfluous includes from deinlined AQL header files

* Added includes for size_t

* Deleted outdated files

* De-inlined files that were undone during the merge

* Removed includes from files that were undone during merge

* Fixed merge conflicts
2019-09-25 11:20:03 +02:00
Jan Christoph Uhde 0b8c75c7b7 one shard db - devel (#9395) 2019-09-23 15:48:37 +02:00
Jan 98fea326ea
handle K_SHORTEST_PATHS where it was forgotten (#9954) 2019-09-09 14:54:31 +02:00
Jan c52f2a8315
refactoring (#9411) 2019-07-09 11:15:52 +02:00
Andrey Abramov 420326be25
Bug fix/internal issue #414 (#9428)
* add more tests

* and more...

* address msvc build issue
2019-07-08 23:17:41 +03:00
Andrey Abramov 0fb8b348db
bug-fix/internal-issue-#574 (#8991)
* restrict number of buckets to compare

* do not use more sort buckets than requested

* cleanup
2019-06-04 18:55:25 +03:00
Jan d67513607d
fixed issue #9106 (#9116) 2019-05-29 18:27:38 +02:00
Andrey Abramov 56696e3ea6 Bug fix/internal issue #577 (#9035)
* properly handle shorthand of system analyzer

* don't allow accessing local analyzer from within another database

* ensure local analyzer are not acessible from outside

* fix duplicate ids
2019-05-19 18:00:13 +02:00
Jan Christoph Uhde d194cfc31e bug-fix/iresearch-error-propagation (#9001)
* use result instead of bool

* WIP: use arangodb::Results in more places

* WIP: fix unittests

* fix tests
2019-05-17 19:00:29 +03:00
Andrey Abramov 4e12fdb30b
ArangoSearch sorted indexes (#8888)
* move IResearchViewSort into separate unit

* ensure sort is propagated to IResearchLink

* update iresearch and extend some tests

* implement insertion of sorted values, store value comparator in link

* add tests for VPackComparer

* minor cleanup

* fix issue with sort definiton propagation

* propagate error message from index factory

* add integration tests for sorted indexes, single server

* initial implementation of IResearchViewMergeExecutor

* move iresearch::VPackComparer to separate translation unit

* store collection in IndexReadBuffer for IResearchViewMergeExecutor

* set primary sort from optimizer rule

* add some tests for sorted view case

* extend IResearchViewNode tests

* simplify primary sort definition

* pretty print order condition covered by arangosearch in query explanation

* fix compilation with g++-8

* address catch test failures

* fix tests

* adjust arangosearch optimizer rule to optimize sort in case of cluster

* make tests for sorted indexes work in cluster too

* fix tests

* extend js tests for sorted indices

* fix keyword in query explainer

* ensure sort-limit rule works with views

* backport ngram analyzer fix from upstream

* address jslint errors

* modify IResearchViewSort to support boolean direction specification format

* add some tests

* add tests

* add skip functionality

* extend tests

* add more catch tests

* inline some methods

* address jslint errors

* extend catch tests to reproduce the failure

* add some tests for check fullcount

* add more tests

* extend tests

* temporarily comment out broken tests

* extend js tests

* address review comments

* address jslint errors

* update iresearch
2019-05-11 17:58:56 +03:00
Jan 33c9e9e239 optimize away SortNode in case it is covered by an arangosearch view (#8701)
* optimize away SortNode in case it is covered by an arangosearch view

this implementation is a stub with hard-coded attribute names

* extend IResearchViewMeta with sorting order definition

* make 'IResearchViewMeta::Sort' compatible with 'SortCondition' API

* ensure ArangSearch sort is immutable after creation

* address review comments
2019-04-12 19:08:41 +03:00
Jan 771a1a646d
don't analyze all calculations in a query for views if there are no views contained in the query (#8723) 2019-04-10 13:56:40 +02:00
Tobias Gödderz d1416bed74 AQL by line: IResearch blocks as executors (#8387)
* Added RemoteExecutor skeleton

* Moved RemoteBlock implementations to ExecutionBlockImpl<RemoteExecutor>

* Remove unnecessary include to avoid unused function warnings

* Fixed gcc compile error

* Moved Scatter/Distribute block implementations to their new Executor versions

* Applied clang-format

* Added factory, infos and a skeleton for the unordered view executor

* Removed assert based on wrong assumption

* Added members from IResearchViewBlockBase to IResearchViewExecutor

* Moved more code into the ViewExecutor, hopefully enough to produce a working version now

* Added missing reset code, made produceRow work mostly correct

* Removed superfluous parentheses to get more useful output from Catch

* Ported fix 923b6e81ac723d1fe37f8e7bf1ab81149f3a08ef

Original commit message was:
Fixed a race condition in RemoteBlock which was triggered during
shutdown overtaking getSome.

* Applied review comments

* Inject input row instead of an item block + pos into the expression context, plus fixed some tests

* Adapted test. Search tests are now green.

* Do not ask upstream when already DONE

* Removed `limit` from next()

* Simplified code that could handle producing more than one document

* Minor readability change

* Solved two TODOs noted in the review

* Removed leftover references to DistributeNode members in the DistributeBlock

* Reverted removal of "exhausted"

* WIP: Implemented variant with scorers

* Fixed compile errors of the last commit

* Fixed some asserts and calculations

* Fixed violated assertions

* Moved files from IResearch/ to Aql/

* Replaced recursive call with a loop

* Worked on a few TODOs

* Removed IResearchViewBlock

* Set input registers correctly

* Eliminated dependency to the Node in the Executor

* Don't misuse the volatility variables for initialization

* Extended a TODO note

* Removed obsolete includes

* Removed an obsolete include from the tests

* Added missing include

* Read PKs in batches

* Fixed merge conflict

* Fixed merge conflict

* Restrict prefetching of PKs to the number of rows in the current output block

* Fixed merge

* Fix IResearch ASan errors

* Revert "Restrict prefetching of PKs to the number of rows in the current output block"

This reverts commit e0fd8698a3.

* Revert "Read PKs in batches"

This reverts commit c06c4d7a36.

* Began some small step refactoring to introduce batch-reading correctly

* Extracted method fillBuffer

* Extracted method evaluateScores

* Minor changes

* Read data from iresearch index in batches

* Replaced std::deque<IndexResult> buffer by a new class

* Solved minor TODOs

* Fixed last commit

* Fixed merge conflict

* Removed accidentally re-added view blocks

* Implemented review comments
2019-04-04 23:40:10 +03:00