1
0
Fork 0
Commit Graph

98 Commits

Author SHA1 Message Date
Iurii Popov 77f08b2ec7 decapitalize json in late materialization (#10610) 2019-12-03 13:35:32 +03:00
Kaveh Vahedipour d11c0954e8 Missing post failure on readlock in SynchronizeShard (#7860)
* when post cannot be delivered for readlock waiting for put meningless

* devel pull

* we can rely on POST alone

* clean up

* wrong errorCode tested

* fixed nullptr

* correct timeout

* fixed nullptr

* merge devel

* log level

* maintenace default log level raised to INFO, shard synchronisation only logs start/success all WARN/ERR demoted to DEBUG

* message unification

* devel merge

* fixed query registry

* tobias remarks

* mchacki part one

* static string for rebootid

* working tests

* use the reboot tracker from clusterinfo

* use the reboot tracker from clusterinfo

* rebootid be rebootid

* fixed serverid and all working

* static strings

* callguard out of reboottracker

* call callbacks, when query is ditched

* clear priv

* change log

* my oh my

* simon attention

* merge seems fine

* typo

* fix headers
2019-11-26 17:46:26 +01:00
Michael Hackstein c201dce948
Bug fix/input row serialization (#10502)
* Unified toVelocyPack of InputRow and AqlItemBlock. We can now velocypack any range within an AqlItemBlock as a new block.

* This is the fix required here. We now end exhausted false in initialize cursor. For some reason the other side checks this hard-coded value and if it is not the expected value it does ignore the send body ¯\_(ツ)_/¯

* Added test for range-based serialization
2019-11-22 08:08:53 +01:00
Tobias Gödderz 168103af88 Splicing gather cleanup (#10479)
Some cleanup and deprecation information
2019-11-21 13:00:31 +01:00
Markus Pfeiffer 8e13486687 Fix bug in SubqueryEndExecutor (#10485)
* Fix bug in SubqueryEndExecutor

If a subquery was spliced, and did not get any input rows, the SubqueryEnd
executor still expected a relevant ShadowRow to be handed through, and ran into
an assertion (or otherwise crashed the server) if it didn't get one.

* Add regression test for SubqueryEndExecutor fix
2019-11-20 17:03:28 +01:00
Tobias Gödderz 7a57a72620 Feature/aql subquery splicing with gather (#10341)
Allow Splicing with GATHER nodes
2019-11-18 19:05:44 +01:00
Iurii Popov 26309c5c96 Feature/issue #642 (#10352)
* Index late materialization draft

* Index late materialization. Refactoring.

* Index late materialization. Expansion draft.

* HashIndexMock

* Refactoring.

* Hash index mock expansion fixes

* Fix compare of attributes.

* Hash index mock. Last expansion support.

* Hash index mock. Take value for array from document, not equal key.

* Optimizations.

* Query tests

* lateMaterialized improvements

* velocypack fixes and tests for index node

* Fix after merge.

* Const expression in a condition test

* Clone tests

* More tests

* Fix for materialized view (nullptr var)

* Fixed index vars registers count planning

* javascript tests

* Fixed cluster tests

* Cluster support

* MaterializeExecutor and MaterializerExecutorInfos refactoring

* More tests

* js tests rocksdb only

* js test. Removed print()

* js testd added skiplist

* C++ unit tests for invalid json

* js test var i

* js tests == -> ===

* iFix js tests ===

* jslint ;

* int unsigned int fix

* double deletion fix

* refactoring

* Added const for collection source

* override final, added materialize namespace, fixed femove in hash index mock

* Removed expansion support. Index can contain null value in case of several object with same fields

* isNumber<>

* Optimization for single collection case

* Renaming

* refactoring

* Unit tests. Added checking documents

* !sliceIt.valid() instead of !(!=)

* Fix primary index (object instead of array)

* Edge collection test

* Windows compilation fix

* IndexExecutor getCallback optimization

* ADB_UNLIKELY

* explainer.js

* removed unsused variable in js test

* fix explainer.js

* do not use several indexes with js test

* Removed several indexes implementation

* space

* fix explainer.js

* fix explainer.js

* code alignment
2019-11-07 19:16:33 +03:00
Markus Pfeiffer 972e0b2ce0 Rewrite ModificationExecutor (#10221) 2019-11-07 14:48:26 +01:00
Simon Grätzer b0a2e207c1
Merge branch 'devel' of github.com:arangodb/arangodb into feature/parallel-aql-phase-one 2019-10-31 14:40:28 +01:00
Tobias Gödderz 6a3f50fb88 Enable AQL subquery optimization (only on getSome) (#10267)
Activate the Subquery Optimization rule in getSome only cases
2019-10-30 17:10:01 +01:00
jsteemann 125dcfffa9 Merge branch 'devel' of github.com:arangodb/arangodb into feature/parallel-aql-phase-one 2019-10-29 13:10:12 +01:00
Simon Grätzer aef6b87521
Merge branch 'devel' of github.com:arangodb/arangodb into feature/parallel-aql-phase-one 2019-10-29 11:46:52 +01:00
Michael Hackstein 1d8bad0a00
Feature/aql modificators cpp test (#10290)
* Adds some tests on AQL Modification blocks (C++)
2019-10-28 15:42:35 +01:00
Jan b124113190
abort write transactions (#10248) 2019-10-23 15:49:47 +02:00
Michael Hackstein bf2423d024
Feature/explain spliced subqueries (#10298)
* Implemented explain output for Subquery start and End nodes. Unfortunately i had to inject a temporary output variable for the Subquery start.
2019-10-22 15:38:33 +02:00
Simon Grätzer 7ced6b2064
fix wakeup 2019-10-17 19:34:10 +02:00
Simon d526805e81 Bug fix/fix suspicous stuff (#10273) 2019-10-17 15:34:22 +02:00
Dan Larkin-York 6fd419714f Move containers into their own folder and namespace. (#10244) 2019-10-14 10:55:55 +02:00
Tobias Gödderz 4e86de00d2 AQL Subquery: MultiDependencyRowFetcher (#10101)
Add ShadowRow support for MultiDependencyRowFetcher
2019-10-10 13:08:02 +02:00
Dronplane a1014ac701 Bug fix/internal issue #600 - Late document materialization (#10006)
* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* adding plan traversal for rule

* Rule execution block developing

* Experiment with additional block

* LIMIT node modifications

* implemented late materialization base version.

* Code cleanup

* Added check for variable usage

* Implemented serialization and cloning

* Fixed build errors

* Implemented late materialization for  LimitNode

* Moved materialization to Sort Node

* Removed LimitNode modiications

* Implemented skipRows to speedup single server

* Added tests for optimizer rule

* fixed build

* Fix mac build

* Fixed mac build

* Fixed PR test run errors

* Added tmp fix for cluster test runs

* removed debug print

* Make materialization stick to last sort node. Add late materialization to explainer

* Updated optimization rule to search optimal SortNode to do materialization

* Refined optimization rules for cluster

* Added tests for SortExecutor and IResearchViewNode

* Fixed  mac buid. Removed misplaced assertion.

* Fixed build

* Updated tests. Coded cleanup

* Code cleanup

* made function name for SortedExecutor match name in ConstraintSortedExecutor

* cleanup

* Fixed test run

* made scoring results stable across cluster

* Make optimize rule stop if  sort node for limit node is not suitable

* Disabled constrained sort for materializing SortNode

* reverted SortNode modifications

* Implemented separate materializer node

* Fixed tests  to account new materialize node

* Code cleanup

* reverted debugging change

* Fixed test jslint error  and comments

* MaterializeNode renamed

* enum value renamed for materialize node

* Code cleanup

* code cleanup and optimization

* Fixed clang warning

* Applied review comments

* Out variable planning moved to materialize node

* inlined getters in ReadContex
2019-10-08 18:38:47 +03:00
Michael Hackstein 1755b2b253
Feature/aql subquery single input tests (#10175)
Added DISABLED_ tests for the new Subquery Optimization Rule
2019-10-08 15:29:09 +02:00
Michael Hackstein cf99ff1586
Feature/aql subquery all rows fetcher shadow rows (#10079)
Allow handling of ShadowRows in AllRowsFetcher
2019-10-07 16:49:30 +02:00
Michael Hackstein 1d4584ffab
Add a way to avoid core-dumps on death tests (#10132)
* Added a way to execute DEATH tests in a core-free way, otherewise the Linux jenkins will produce core-files on those tests

* Update tests/Mocks/Death_Test.h

no new line at end of file

* Renamed the DeathTest macro as suggested.

* Yeah Windows Yeah, disabled death-test because i cannot get it to work.
2019-10-07 10:59:21 +02:00
Jan 6f14048bbe
implement early pruning of documents in EnumerateCollectionExecutor (#10159) 2019-10-05 13:38:23 +02:00
Tobias Gödderz e2c84acfaf Use explicit default destructors where possible (#10166) 2019-10-04 15:58:30 +02:00
Jan d29d3df698
improve test statements, fix memleaks and init-order-fiasco (#10142) 2019-10-02 16:02:56 +02:00
Markus Pfeiffer fc634a2fa0 Replace template <bool blockPassthrough> (#10127)
* Replace template <bool blockPassthrough>

by template <BlockPassthrough blockPassthrough> where BlockPassthrough is an
enum class.

This is mainly for better readability and for some type safety.

* Fixup some implicit conversions to bool
2019-10-02 09:47:09 +02:00
Markus Pfeiffer 32884568cb Feature/subquery end node executor (#10087)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Updates to testing code

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* More updates

* Add optimizer rule

* Fix optimizer rule

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fix optimizer rule (again)

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Attempt to fix :windows: compile warning

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Fixed mac compile issues

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Handle ShadowAqlItemRow in SingleRowFetcher

* Move some code from header to cpp

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Removed unused variable

* Added first test on ShadowRows in SingleRowFetcher. Found inconvenience in original code with it.

* Added a test for multi level shadow rows

* Added a test and fixed production for multiple consecutirve shadow rows.

* Added tests for pass through blocks and shadow rows

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Add SubqueryEndExecutor

* Adapted RowFetcherHelper to be able to work on ShadowRows

* Fixed minor error in RowFetcherHelper that causes LIMIT tests to error

* Temporarily disabled test suites (sorry there is only one 'test case' in this suites that does all or nothing tests ...) as it tests a feature we cannot support in this intermediate state, we need to default disable our new rule first.

* IMplemented missing function to copy over SubqueryDepth between blocks

* Seperated assert, to see faster which part is violated

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Disabled subquery splice rule by default

* Forgotten merge marker

* temporarily modified test

* Some Subquery End Executor work

* Fix RowFetcherHelper to work with ShadowRows

* Add tests for SubqueryEndExecutor

* Rewrite SubqueryEndExecutor

* Update test

* Add createBlock code for SubqueryEndExecutionNode

* Some fixups adding ExecutionBlockImpl for SubqueryEndExecutor

* Fix SubqueryEndExecutor

* Update SubqueryEndExecutorTest

* Update endexecutor test

* Fix the endnode executor once more

* Fix some merge fallout

* Update test

* Add missing advanceRow to SubqueryEndExecutor

* SubqueryEndExecutor Tests

* Activate a death test

* Better handling of death test

* RowFetcherTest refactoring Part 1, still ongoing. We use this for AqlItemMatrix and SingleRowFetcher

* Update and cleanup the SubqueryEndExecutor tests

* Check that expected shadow row depths are met

* Remove commented out code in AqlItemBlock.cpp

* Remove a misplaced comment

* Sort out SubqueryEndExecutor includes

* Remove superflous include

* Consistently use NoStats{} in SubqueryEndExecutor

* Make SubqueryEndExecutor properties constexpr

* Reactivate IResearchTests

* Do not copy the accumulator in SubqueryEndNode

* Return a more sensible value for expectedNumberOfRows of SubqueryEndNodeExecutor

* Constify an argument

* Fix ownership problems with the buffer for _accumulator

* Cleanup SubqueryEndExecutor tests

Also add a test that writes to a register other than 0

* Patch RowFetcherHelper to count the number of Items returned instead of number
of calls

* Make MSVC happy
2019-09-30 15:24:13 +02:00
Dan Larkin-York 5204e95512 Add log suppressor utility classes for cleaner test code. (#10088) 2019-09-27 12:57:38 +02:00
Michael Hackstein e89d72d02c
Feature/aql subquery start executor (#10040)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Updates to testing code

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* More updates

* Add optimizer rule

* Fix optimizer rule

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fix optimizer rule (again)

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Attempt to fix :windows: compile warning

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Fixed mac compile issues

* Initial commit, added class and test stubs

* Added Executor body stub, also added a test for no input. Passes now, as no input is read yet.

* Base implementation + Test of SubqueryStart Executor. First level shadowRows are now produced. Need to add tests for multilevel, and for non-fitting output blocks.

* Added more sophisticated test about shadow row relevance

* Added a test where ShadowRow does not fit into the open block. Also added disabled test on multi level shadowrows. Requires modifications on SingleRowFetcher.

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Handle ShadowAqlItemRow in SingleRowFetcher

* Move some code from header to cpp

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Removed unused variable

* Added first test on ShadowRows in SingleRowFetcher. Found inconvenience in original code with it.

* Added a test for multi level shadow rows

* Added a test and fixed production for multiple consecutirve shadow rows.

* Added tests for pass through blocks and shadow rows

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Adapted RowFetcherHelper to be able to work on ShadowRows

* Seperated assert, to see faster which part is violated

* Fixed minor error in RowFetcherHelper that causes LIMIT tests to error

* Temporarily disabled test suites (sorry there is only one 'test case' in this suites that does all or nothing tests ...) as it tests a feature we cannot support in this intermediate state, we need to default disable our new rule first.

* Improve check of ExecutionState

* Implemented shadow row handling in subquery start executor. Tests not happy yet

* IMplemented missing function to copy over SubqueryDepth between blocks

* Fixed test code of StartExecutor test. It asserted wrongly on releveance of subquery rows.

* Added instanciation of SubqueryStartExecutorTest. Code does not compile.

* Allow to create SubqueryStartExecutors.

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Disabled subquery splice rule by default

* Forgotten merge marker

* temporarily modified test

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning

* Removed incorrect optimization in SubqueryStartExecutor
2019-09-26 10:01:52 +02:00
Dan Larkin-York a83c2323c9 Refactor ApplicationServer stack (#9965) 2019-09-25 17:31:59 +02:00
Markus Pfeiffer 0b9dfb4b9b Subquery Start/End Nodes (#9983)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Updates to testing code

* More updates

* Add optimizer rule

* Fix optimizer rule

* Fix optimizer rule (again)

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Disabled subquery splice rule by default

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning
2019-09-25 12:35:18 +02:00
Tobias Gödderz 82b0a5e745 De-inline in AQL (#10025)
* Enable IPO (LTO) via CMake

* Use separate IPO_ENABLED variable

* Enabled IPO for more binaries

* Enabled IPO for arangobackup

* Suppress an MSVC warning (though it is not unjustified...)

* Fix static builds, broken due to Policy CMP0060 introduced in cmake 3.3

* Removed policies superseded by minimum cmake version

* Disable IPO with google tests due to a g++ bug

* Disable IPO with google tests only on AUTO

* Disable warning for correct line

* Begin de-inlining Aql

* de-inlining Aql: 2nd batch

* de-inlining Aql: 3rd batch

* de-inlining Aql: 4th batch

* Moved code out of ifdef

* Enabled IPO for additional libs

* Fixed some problems found with IPO enabled

- Fixed ODR violation in Pregel
- Removed unused code in ClusterMethods
- Avoid possible uninitialized variable usage

* Set IPO globally, except for 3rdParty libs

* De-inlined AstHelper again (was undone due to a previous merge conflict)

* Added missing .cpp file

* Fixed compilation with MSVC

* Removed superfluous includes from deinlined AQL header files

* Added includes for size_t

* Deleted outdated files

* De-inlined files that were undone during the merge

* Removed includes from files that were undone during merge

* Fixed merge conflicts
2019-09-25 11:20:03 +02:00
Michael Hackstein 265eb1549d
Feature/aql subquery opt shadow row interface (#9987)
* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Attempt to fix :windows: compile warning

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>
2019-09-24 09:09:52 +02:00
Jan Christoph Uhde 0b8c75c7b7 one shard db - devel (#9395) 2019-09-23 15:48:37 +02:00
Jan 5572675106
Bug fix/remove base directory from include path (#9885) 2019-09-04 17:39:01 +02:00
Tobias Gödderz 001550ada7 Bug fix/count collect should skip (#9511)
* Make CountCollectExecutor skip rather than fetch

* Cleanup, fix and test for top-level variables

* Added CHANGELOG entry and tests
2019-07-19 10:08:36 +02:00
Jan 1a58cc2213
add VelocyPackHelper::equal method (#9389) 2019-07-03 12:15:11 +02:00
Jan 9cb08ded92
make the comparison functions unambiguous (#9349)
* make the comparison functions unambiguous

* added @kaveh's suggestion
2019-07-01 16:35:28 +02:00
Wilfried Goesgens 42e3dcf5a1 Bug fix/wintendo compile sequence (#9295)
* move snappy up

* Fix compile warning with MSVC (#9291)

* one more warning

* use right type

* only define variable if we actually use it in the code

* only define variable if we actually use it in the code undo

* use right type
2019-06-19 12:00:48 +02:00
Tobias Gödderz c854d04864 Feature/limit executor passthrough (#9162)
* Some refactoring to implement helper methods iff necessary

* Updated comments

* Added static assertions

* Re-enabled check for maintainer mode

* Allow pass-through for LimitExecutor. This is not yet working with fullCount correctly!

* Get fullCount before returning the last row

* Adapted fullCount tests: LimitExecutor doesn't lie about the state of the last row anymore

* Use correct row

* Handle LimitExecutor-stats during fetchBlockForPassthrough

* Fixed LimitExecutorTest

* Removed _stats member

* Added more catch tests

* Bugfix for fullStats

* Removed an erroneous assertion

* Implemented LimitExecutor::skipRows

* Avoid name clash of class enum member to please MSVC

* Test MSVC

* Revert "Test MSVC"

This reverts commit d8325d95318bb83b79c8c4cc9f886d36e484c870.

* CREATE_NEW seems to be taken, too. Next try for MSVC

* Built VPack to ItemBlock helpers, and made SingleRowFetcherHelper more general

* Removed ostream operator because MSVC doesn't like it...

* Implemented SingleRowFetcherHelper::fetchBlockForPassthrough, plus a small framework to test executors

* Removed erroneous include

* Moved code in separate files

* Began writing the extended schema of runExecutor

* Added output operator for ExecutorCall

* Fixed removeWaiting

* Fixed expected output

* Added skipRows to runExecutor

* Added two tests

* Allow empty blocks

* Added two more tests

* Added another test

* Built two different parametrized classes for LimitExecutor

* Some cleanup

* Even more tests

* Fixed a bug found by the new tests

* Fix compile error on windows

* Use native matrix representation in tests

* Updated some comments
2019-06-18 17:33:58 +02:00
Wilfried Goesgens 863257f1d5 fix windows warnings / compile errors (#9215) 2019-06-06 18:08:44 +02:00
Michael Hackstein 8a4d0bc068
Performance improvement on Sort (#9198)
* Modified AqlItemMatrix to use two uint32 values to identify for in Block instead of one uint64 value encoding the two and calculation. Also now throw an error as soon as we exceed the limit there

* Removed new error code, thanks jsteemann for pointing out.
2019-06-05 12:45:17 +02:00
Tobias Gödderz 6a56130ff8 Reimplement ReturnExecutor with a simple pass-through (#8876) 2019-05-31 15:27:59 +02:00
Tobias Gödderz f1dd0659fa Fixed test usage of MultiDependencySingleRowFetcher (#9060)
* Fixed assertion to not have side effects

* init dependencies in test explicitly
2019-05-22 09:39:50 +02:00
Dan Larkin-York d5ecdd143a Convert unit tests to googletest framework (#9034) 2019-05-21 09:17:46 +02:00
Markus Pfeiffer c1d6a88da2 Fix a crash in ShortestPathExecutor (#8985)
* Fix a crash in ShortestPathExecutor

* Add a test that checks whether ShortestPathFinder is passed valid inputs
2019-05-17 16:40:22 +02:00
Wilfried Goesgens e36655dee7 Feature/have collection count type (#8942) 2019-05-14 13:19:09 +02:00
Heiko 58db1023f3 Feature/aql skip some (#8771)
* testing skipSome implementation, default and passThrough

* added simple implementation for default and passthrough skipsome methods

* added skipRows function to enumerate collection executor

* added tests for the enumerate collection skipRows function

* implemented and tested enumeratecollection skipSome

* skipSome tests

* fixed a test, prepared iresearch view exec for skipping

* rm logs

* gcc bug workaround

* changed to original test code as it has been before

* iresearch skipping, added ires skip test

* added index executor skipRows

* added skip blockfetcher, quick exit index

* test

* const dummy and singlerowfetcher skip

* forgot return value

* input wrong initialized

* trying to remove dynamic cast and solve with a different approach

* cleanup

* const skip

* jslint

* Handle skipSome of subqueries correctly

* Removed unused code

* Removed unused member

* Simplified skip variants and enabled IndexExecutor skipping

* A little cleanup, fixed DependencyProxy::skipSome

* Reverted test change

* Tried to make testLimitBlock3 clearer

* Extended test suite

* Bugfix

* Added stats when skipping and fixed a few other things

* Bugfixes

* Moar bugfixes

* Update arangod/Aql/IResearchViewExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* applied requested changes

* Fixed a bug in LimitExecutor::expectedNumberOfRows()

* Fix skipSome in catch test RowFetcherHelper

* Fixed a bug in the tests

* Two bugfixes in LimitExecutor::expectedNumberOfRows

* Avoid passing large batch sizes from skipSome to getSome

* Fixed Windows compile errors

* Fixed a skip bug with WAITING in unsorting gather blocks

* Make aql-skipping find some cluster bugs, like the last commit

* Bugfix and additional asserts

* Fixed skipSome counting in IndexExecutor

* Resolved merge conflicts
2019-05-08 15:02:43 +02:00
Tobias Gödderz 6dfb9995b4 Feature/aql by line cleanup (#8848)
* Removed useless parameters of buildCallback

* Renamed produceRow to produceRows and adapted a comment

* Renamed BlockFetcher to DependencyProxy

* Applied git-clang-format
2019-04-25 15:51:11 +02:00