1
0
Fork 0
Commit Graph

153 Commits

Author SHA1 Message Date
Frank Celler 393ffba276 fixed wrong defines 2019-09-12 14:46:23 +02:00
Michael Hackstein 6e8d43b2da
Feature/aql traversal prune (#8255)
Allow PRUNE in the AQL Traverser.
2019-02-27 11:43:06 +01:00
Vasiliy 69479f1dce issue 523.1: address build issues, ensure FlushFeature subscriptions are cleared during stop(), assert that they are deallocated (#8193)
* issue 523.1: address build issues, ensure FlushFeature subscriptions are cleared during stop(), assert that they are deallocated

* backport: account for Flush subscriptions validly surviving past FlushFeature::stop()

* fix comment typo
2019-02-22 00:08:17 +03:00
Tobias Gödderz 1adb438412 Calculate registersToKeep correctly to avoid redundant copying of empty registers (#8168) 2019-02-15 12:55:18 +01:00
Luschgy b7b84beebf Feature/aql blocks by line (#6553)
* Fixed hidden merge conflict

* Added BlockFetcher, added a minimal test for SingleRowFetcher

* move RegInfo in types.h

* fix getSomeLogic

* Added the AqlItemMatrix implementation. Not yet tested!

* Fixed nr called => Waiting ratio in AllBlocksFetcherMock

* fix compiling

* add RegInfo to produceRow

* BlockFetcherHelper now returns a matrix

* pass RegInfo as requested into AqlItemRow ctor

I would still keep it out as the ItemRow should be a thin view.

* allow to change the AqlItemRows cheaper / remove _sourceRow member

* implement copyRow for AqlItemRow

* organize cleanup for outputblock used in getsome / shrink if necessary

* Added tests for SingleRowFetcher, cleanup in AqlItemRow

* start mocking

* Added test helpers, began work on a more complex test for SingleRowFetcher

* Re-added RegInfo again to AqlItemRow

* helper, added test case for FilterExecutor

* merge fixed compile

* Removed trx name where it is actually not used

* Made some functions TEST_VIRTUAL for mocking.

* implemented Sort executor

* Fixed the BlockFetcherHelper API, to always return a matrix if it is not waiting

* Implemented a SortExecutor test with data

* disable borken test

* Fixed test on SortExecutor. Green now, needs some more testing

* change a bit how the AqlItemRow is used in a test

* Added another test for the SingleRowFetcher

* Added more tests for SingleRowFetcher

* fix getRegisterInfo

* Moved RegInfo into ExecutorInfos. Also now use the FilterExecutor implementation in production code instead of FilterBlock. Starting integration testing.

* add more asserts

* Fixed undefined behaviour in TestCode.

* Do not shrink an empty output block

* switch to block pointers in AqlItemRow

* return blocks in the row fetcher

* Updated grammar.cpp

* Updated (fixed) tests to verify returnBlock

* Improved const-ness

* Changed BlockFetcher constructor in order to make it mockable

* Implemented BlockFetcherMock and rewrote some tests with it

* Updated a comment

* Updated comments

* init commit - EnumerateListExecutor, including EnumerateListExecutorInfos

* add potential fix for ItemMatrix

* Replaced fakeit with BlockFetcherMock in another GIVEN block

* Replaced the last fakeit mocks with BlockFetcherMock and removed fakeit

* Updated function definition

* included missing arary included

* Renamed AqlItemRow to InputAqlItemRow

* Split AqlItemRow into Input~ and Output~ classes

* EnumerateListExecutor + Tests

* Added tests for AqlItemRows

* Added test file for AqlItemRows...

* Made the ostream operator for AqlItemBlocks available to Logger and Catch info

* build error

* Added a second test for AqlItemRows. Need to make input space invalid now, sth. acidential referencing will show up

* Introduce a block ID in the InputAqlItemRow

* Copy output values correctly, avoid InputAqlItemRow pointers

* Added a test for registerClearout

* Fixed tests

* Added a test for adding new values

* Added a test for an incomplete row write to not count as produced.

* Fixed getSome and a test

* Added a test for larger AQL values in AqlItemRows

* Fixed compile error in hidden merge conflict

* Added tests for Filter counter in FilterExecutor

* OutputAqlItemRow now takes responsibility of the block

* Handed responsibility of blocks in ExecutionBlockImpl to OutputAqlItemRow

* EnumerateListExecutor should be close to its finish. Tests need to be fixed, not ready yet.

* better swap

* OutputAqlItemRow now takes ExecutorInfos

- added several assert
- implemented a more strict and safe produced() logic

* enum list test

* Added a more specific error message

* Removed OutputAqlItemRow.changeRow()

* fix invalid block freeing

* changed order of input/output registers

* Fixed a test

* Fixed matrix width in test

* fix failure tests

* finished enumlist tests

* Implemented AllRowsFetcher and tests for it

* Bugfix in skipSome stub

* insert sort and enumerate list executors

into exectuion nodes' create functions

* fixed singlerowfetcherhelper, added tests

* small fixes

* add missing Infos

* restore accidentally deleted scorer

* make it compile

* fix executor info creation / usage

* make use of output var in EnumarateListNode

* Added Executor::Stats and FilterStats

* Extended FilterExecutor tests with stats, added node tracing to getSome

* Fixed Sort-&EnumerateList- Executor to use the new return type with stats

* Fixed compile error in tests

* Changed *ExecutorInfo constructors and classes to contain the necessary register informations

* Made an ExecutorInfo construction more readable

* Updated comments

* Fixed checks in OutputAqlItemRow::setValue()

* Add comparison method to SortRegister. Forbid construction without one.

* Implemented ExecutionBlockImpl::initializeCursor

* more mocking inside ExecutionBlockImplTest

* Minor changes

* Avoid AqlValueGuard destroying the value copied into the output block

* Disabled failure tests that do not apply to the new Sort implementation

* further mocking of test modules

* Removed constexpr for compatibility with outdated gcc versions

* add some failure tests - getsome works differently

therefor we do not test the old behavior

* fix sort node

* clean up alq item matrix

* Removed the transaction methods from read-only access to AqlValues (get and at) and repliced by CollectionNameResolver where necessary. Still left: some calls extract VPackOptions from the Transaction. Compare needs a BuilderLeaser only.

* textexecutionhelper, execimpl test

* Eliminated FilterBlock

* Eliminated the EnumerateListBlock

* first successful running test of executorblockimpl

* Eliminated SortBlock

* fixed testexecutor helper

* Made the ExecutorInfos private to the executors again.

* Implement and use AqlItemBlockShell (#6562)

* Added AqlItemBlockShell

* Minor changes

* Mostly replaced AqlItemBlock with ~Shell.

Current state:
- arangod compiles
- OutputAqlItemRow has yet to be changed
- catch tests have yet to be fixed

* Use AqlItemBlockShell in OutputAqlItemRow

* Added registersToKeep to AqlItemBlockShell

* Removed ExecutorInfos from OutputAqlItemRow

* Split AqlItemBlockShell into Input~ and Output~

* Changed tests so they compile again - they're still not yet working.

* Fixed tests

* fixed building, modified impl test

* Removed now superfluous BlockFetcherMock functions

* more execimpl testing

* Removed block IDs

* more impl testing

* Reduced dependencies of BlockFetcher construction

* Bugfix, cleared documentation

* Removed debug-output

* Adressed minor code review remarks

* Moved method definitions into .cpp files to avoid includes

* Added assertions

* Added comments

* Addressed several more review comments

* Added missing includes

* more testing, not finalized yet

* added missing header file

* added a test for execution block impl

* removed old comment

* Improved lookup algorithm to identify correct block/row combination in AqlItemMatrix

* added test for ExecutionBlockImpl

* move utility to the right position

* Bugfix and a TODO note

* Bugfix: use a uniform ResourceMonitor in tests

* Made OutputAqlItemRow::setValue clone by default and copy on move

* Replaced the getRow lookup with a binary search

*  enumerate tests

* Disable emplacing of AqlValues in ItemBlock

* Reenabled std::enable_if will warn until oskar is using c++17 as standard.

* Add helper function to create a shared pointer of an unordered_map<RegisterId>

* CalculationExecutor (#6570)

* add CalculationExecutor skeleton

* first shot of CalculationExecutor

* fix minor bugs

* handle references

* use new shared pointers

* commit to share work

* fix failure test

* reformat with clang-format

* add test for CalculationExecutor

* fix CalcualtionExecutor test

* clean up

* Removes unused code

* clean up includes

* fix/remove comments

* add changes requrested by hacki

* ReturnExecutor (#7932)

* add skeleton to CMakeLists.txt

* build most parts of the ReturnExecutor

* This commit removes ReturnBlock and adds some debug messages

removed inheritRegisters functionality

* finish ReturnExecutor

* add test for return executor

* remove unused code

* add stats and clean up includes

* verify result in ReturnExecutorTest and move *Stats to "Aql/Stats.h"

* Fixed algorithm to find correct row in a MultiRow Input

* fix full count and failure tests (#7969)

* fix full count and failure tests

* fix filter test

* WIP: fixes

* fixed

* NoResultsExecutor (#7953)

* add NoResultsExecutor

* clean up

* some review changes

* fix test

* fix

* fix WAIT -> DONE

* remove unused code (#7995)

* create skeleton for SingletonExecutor (#7957)

* create skeleton for SingletonExecutor

* WIP: continue implementation

* WIP: add fetcher skel

* WIP: add inject function to ConstFetcher

* rename SingletonExecutor to IdExecutor

* WIP: replace singleton node

* WIP: More Debugging on IdExecutor

* WIP: IdExecutor

* WIP: add failure tests

* make iresarch test work but not pass

* WIP: add ConstFetcherHelper

* fix test

* fix tests

* fix iresearch tests

* fix wording

* add review changes

* add review change to SingleRowFetcher as well

* fix test

* fix error when increasing index

* temporarily remove fold-expression

* clean up

* Fix gcc compile error

* Feature/aql limit executor (#7982)

* limit executor

* removed not needed tri assert

* rm _invariable in limitnode

* find out if we are in a subquery

* queryDepth in limit infos

* added missing queryDepth

* finalizing limit executor

* added more tests, fixed a condition

* add assert to test

* added missing fullcount check

* typo

* minor changes, fixed test

* test

* wrong check

* fixed tests

* false merge

* remove basic blocks

* remove basic block includes

* include sort order

* removed not needed basic blocks

* removed not needed basic blocks

* fix test

* Feature/remove input output shells (#8090)

* Added template parameters to allow for block pass-through to be chosen at compile time

* Adapted the tests to match the new template parameters

* Adapted ConstFetcher and IdExecutor to match the new templated fetcher

* Implemented pass-through, fixed SingleRowFetcherTest

* Fixed implicit merge conflicts

* Fix assumptions that are violated by block pass-through

* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fixed compile error

* Fixed a useless and leaking allocation

* Try a less implicit fix

* Allow IdExecutor and ConstFetcher to be used with pass-through

* Performance patches from @mchacki

* Fixed merge conflicts, and bugs in ConstFetcher

* Fixed atMost pass-through, and blocks that call their parents too often

* Removed InputAqlItemBlockShell (except from tests)

* Removed OutputAqlItemBlockShell (except for tests)

* Moved implementations of OutputAqlItemRow methods in the header

* Moved SingleRowFetcher::fetchRow to the header to allow inlining

* Reenabled returnInheritedResults optimization

* Enable passThrough for ReturnExecutor with inherited results

* Specialized CalculationExecutor to avoid unneccessary conditional jumps

* Made tests compile again (but are currently broken)

* Fixed ReturnExecutor test

* Move created InputRows out of AqlItemMatrix

* Limit- and EnumerateList Executors no longer fetch from upstream after DONE

* Added comments

* Added experimental AqlItemBlock optimizations

* Removed _numEntries, added comments

* Allow executors to pass an "atMost"-limit to the SingleRowFetcher (for LIMIT)

* Let the LimitExecutor pass atMost upstream

* Bugfix for FullCount

* Feature/aql enumeratecollection executor (#8028)

* initial commit

* enum coll executor

* buildable version of enumeratecollectionexecutor

* wrong usage of outVariable

* Added template parameters to allow for block pass-through to be chosen at compile time

* Adapted the tests to match the new template parameters

* Adapted ConstFetcher and IdExecutor to match the new templated fetcher

* more fixes, still some tests failing

* Implemented pass-through, fixed SingleRowFetcherTest

* Fixed implicit merge conflicts

* adding missing set value (remember optimize todo), optimized constructor

* removed old enumerate collection blocks

* Fix assumptions that are violated by block pass-through

* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fixed compile error

* Fixed a useless and leaking allocation

* added broken uncompilable executor test

* Try a less implicit fix

* fix mocking

* enumerate collection executor test

* remove class, use only static function.

* clean up test after merging with main branch

* Allow IdExecutor and ConstFetcher to be used with pass-through

* Performance patches from @mchacki

* Fixed merge conflicts, and bugs in ConstFetcher

* Fixed atMost pass-through, and blocks that call their parents too often

* Removed InputAqlItemBlockShell (except from tests)

* Removed OutputAqlItemBlockShell (except for tests)

* Moved implementations of OutputAqlItemRow methods in the header

* Moved SingleRowFetcher::fetchRow to the header to allow inlining

* Reenabled returnInheritedResults optimization

* Enable passThrough for ReturnExecutor with inherited results

* Specialized CalculationExecutor to avoid unneccessary conditional jumps

* Made tests compile again (but are currently broken)

* Fixed ReturnExecutor test

* Limit- and EnumerateList Executors no longer fetch from upstream after DONE

* Removed an assertion which seems to be incorrect during bootstrap.

* Inlined AQL ItemMatrix and removed unused elements in other Executors

* Now inline + Move in AqlItemMatrix, seems to be overall best performing

* Bugfix: Do not returnInheritedResults on DBServers

* Fixed LIMIT test, and made it more thorough.

* Fixed createion of NoResultBlock creation in IResearch Nodes. Also relaxed an assertion

* Reactivated failure point, now identical with anotherone in OutputRow, need to unified

* Bug fix: do not mutate the offset in the LimitExecutor (#8109)

* Introduced test

* Fixed LimitExecutor to not mutate its infos, so it works correctly in subqueries

* Fixed crashes in CatchTests

* Fixed non-maintainer build

* try to fix subqueries

* Fix isInSubQuery for non-singleton nodes.

* remove useless while true

* Feature/aql blocks by line dan sort (#8143)

* Add SortNode::SorterType to SortExecutorInfoss.

* prepare imple of sort executor

* WIP: add some code for dan sort

* WIP: working at pushRows

* WIP - try to add Dan's heap-sort

* re add missing test

* 2 fixes

* fix

* Dan's tests are working

* remove unused variable

* remove SortBlock.cpp from CMakeLists.txt

* remove hack and provide an extra method to set the _baseIndex

* Simplified ContrainedSortExecutor.

* fix forgotten variable

* update doc

* Assert that we do not use functions that might return wrong results.

Preconditions or invariants the functions rely on are broken. Therefore
the functions can not be used. This is asserted in maintainer mode.

* add function to restore row invariant

* do not emplace and pop_back if not necessary

* Use clear registers in case we can move the rows.

* WIndows opst zirkus

* Actually destroy AqlItemValue

* Feature/traversal executor by line (#8000)

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Further test cases

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Fixed leftovers from rebase

* Refixed mocks in different folder

* Fixed Mock import

* Fixed Calculators Exector Test to use the unified mocked server.

* Implemented logic for the TraversalExecutor. Also added tests for it. Not fully complete though

* Added validation of result values

* Added usage of constant register in traversal executor

* Included filter variables in traersal executor. Added test for const input. And moved MMDR into base traverser class for simplicity

* Added ostream operators on AQLValue and AqlInputRow. Not 100% done yet but good enough

* Added a shutdownable ExecutionBlock. Some blocks need to implement a specific shutdown still. I added this class to make it easier to eliminate them when the time is due.

* Removed ostream operator for INPUT row again, does for some reason not compile on Linux0o

* Fixed initialization order

* Removed classwrapper for shutdown.

* Applied review changes

* Refixed failing tests, API is not subject to change

* Fixed CatchTest (wrong usage of string literal in test case)

* Feature/shortest path executor (#8042)

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Further test cases

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Fixed leftovers from rebase

* Refixed mocks in different folder

* Fixed Mock import

* Fixed Calculators Exector Test to use the unified mocked server.

* Implemented logic for the TraversalExecutor. Also added tests for it. Not fully complete though

* Added validation of result values

* Added usage of constant register in traversal executor

* Included filter variables in traersal executor. Added test for const input. And moved MMDR into base traverser class for simplicity

* Added ostream operators on AQLValue and AqlInputRow. Not 100% done yet but good enough

* Added a shutdownable ExecutionBlock. Some blocks need to implement a specific shutdown still. I added this class to make it easier to eliminate them when the time is due.

* Removed ostream operator for INPUT row again, does for some reason not compile on Linux0o

* Fixed initialization order

* First draft of shortest path executor

* Fixed logic bugs, Local AQL tests succeed

* Added first version of ShortestPathExecutor tests. Edge output tests are missing yet.

* Added EDGE output tests

* Replaced TraversalStats with NoStats, they have never been tracked...

* Removed obsolete ShortestPathBlock.

* Removed classwrapper for shutdown.

* Applied review changes

* Refixed failing tests, API is not subject to change

* windows macro opst circus

* Clear registers per block, not per row (#8164)

* Fixed compilation under linux again...
2019-02-14 19:13:08 +01:00
Jan 9ce9312239
speed up containers that contain ExecutionNodes' variables (#7937) 2019-01-14 13:38:56 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Jan 0d48677619
Bug fix/aql optimizer improvements 2308 (#6241) 2018-08-24 13:42:22 +02:00
Jan edb24deafe
cleanup collection/vocbase interface of ExecutionNodes (#5557) 2018-06-08 15:28:48 +02:00
Andrey Abramov e117a97fb4
Feature/arangosearch scatter node (#5536)
* remove collection dependency from scatter

* simplify `createBlock` API

* remove useless IResearchViewScatter node

* some fixes

* code cleanup

* scatter arangosearch view

* cleanup

* fix compilation errors

* fix test issues

* store shared_ptr rather than raw pointer in IResearchViewNode

* refactor ScatterBlock

* refactor DistributeBlock

* ensure no leaks during execution node cloning

* some fixes

* code cleanup

* remove vocbase from `DistributeNode`

* add missing ifdef

* fixes after review
2018-06-06 20:42:53 +03:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
jsteemann ba81e7aa82 remove unnecessary casts 2018-05-11 17:54:36 +02:00
jsteemann 27e985d5d0 safer casts for ExecutionNode types 2018-05-10 20:08:37 +02:00
Wilfried Goesgens 7d6e580780 Refactoring & code cleanup (#5138) (#5142) 2018-04-24 14:42:23 +02:00
Jan 67e7e4181e
Bug fix/remove v8 executor (#4756) 2018-03-13 10:41:02 +01:00
Andrey Abramov f6c184df33 Factory method for execution block (#4729) 2018-03-05 15:02:30 +01:00
Jan 86ef7d4018 Bug fix/fixes 1711 obi (#3730) 2017-11-17 16:34:33 +01:00
Jan 5abf0c1185 Bug fix/fixes 1511 (#3711) 2017-11-16 14:18:51 +01:00
Jan 0a71b54e1f fix out-of-bounds attribute accessor calls (#3273) 2017-09-18 14:01:06 +02:00
Simon Grätzer ffc465433a No access collections Improvements (#3190)
* consolidated EdgeDocumentToken

* optimizing cluster traversal

* adding skip collection checks

* API cleanup

* copying AQLValue to avoid use-after-free bugs

* Fixing rocksdb SingleServerEdgeCursor

* Fixing a collection resolving issue
2017-09-07 14:55:07 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Michael Hackstein 43c4987023 Adapted the traverser code to make conditions for the used index and for post-filtering non-overlapping. If the post-filtering is empty it is not evaluated at all. 2017-06-09 17:01:26 +02:00
jsteemann 2994988210 attempt to fix broken graph stuff 2017-04-20 22:14:30 +02:00
Michael Hackstein e3febed2b5 ShortestPath and TraversalNode now call GraphNode::toVelocyPack as expected, not the Base ExecuteionNode::toVelocypackGeneric by accident 2017-04-20 17:19:30 +02:00
Michael Hackstein 4f899bfd2c Integreated a ShortestPathEngine on DBServers and wired it up with ShortestPathNode and Options. The path seems to be computed correctly in clsuter now, but the vertices are not yet retrieved. 2017-04-20 13:57:14 +02:00
Michael Hackstein 0fa1980064 More unified merge of Traverser- and ShortestPath- into GraphNode. Now the ShortestPathNode creates TraverserEngines on the Cluster. Finally everything ready to wire computation together 2017-04-19 15:23:37 +02:00
Michael Hackstein 50ce09cd98 Added a generic GraphNode as a base-class for Traversal and ShortestPath. It should handle Graph parsing and Engine creation. 2017-04-19 10:10:47 +02:00
Michael Hackstein 1087065ea5 Merge branch 'devel' of github.com:arangodb/arangodb into engine-api 2017-04-13 17:34:25 +02:00
Michael Hackstein 177a3de0fd Squashed commit of the following:
commit ba04aa66ce874fc67644ef2c7b8ee6ef3397e13e
Merge: 8ea1ad2 c6d65ea
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Apr 13 16:51:17 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 8ea1ad2ff0325fc014479376b5258d1b3fa4f730
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Apr 3 14:18:11 2017 +0200

    Deactivated cache in single-server mode

commit 2c68eaeb841aebccccd245ef78bab919adf39a2f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Apr 3 09:18:36 2017 +0200

    Activating document cache in cluster mode for traversals.

commit 3943ca4f8257fe54534949c46749e40c28c4d597
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri Mar 31 16:10:39 2017 +0200

    Created two implementations of TraverserCache. One that takes responsibility for strings only, the other one also caches Documents.

commit 0951f906f18d3631cec412cd0b00504e6d00fc3e
Merge: 0f9970b 87fbbe8
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri Mar 31 14:59:16 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 0f9970b7bab478c4b591b3f4c7fb18915753df30
Merge: d546fd1 8a42089
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 30 17:09:20 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit d546fd13c6a0a3bb4622f6aefd44ea3a149cc8cb
Merge: 873bc59 bd4a2eb
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 29 15:16:54 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 873bc5950028248bf209d0c53fc9eb93938e3610
Merge: 8e4457a 3487097
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Mar 27 14:00:27 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 8e4457a96754bea3437e8d44c0ef9db2366ee86d
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Mar 27 13:59:44 2017 +0200

    Removed old VPackSlice based getVertex API in traversers.

commit 2e0716eeeec1ec56b96571a18074539f5e27cb40
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri Mar 24 09:09:02 2017 +0100

    Fixed logic error in SingleServerEdgeCursor readAll. The cursorId was not propageted properly, which cased filter-evaluations to fail

commit cea85c2ac1163ed0a989c492b8039ed4f7622325
Merge: 538f216 45a4fac
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri Mar 24 08:17:31 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 538f216cbefcdfb4d06fe16a42f223cecae8598b
Merge: 5f4afee 8bfcb49
Author: Simon Grätzer <simon@graetzer.org>
Date:   Thu Mar 23 16:33:44 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

commit 5f4afee54094ea6d2a0cfec1abfb75451f228a23
Author: Simon Grätzer <simon@graetzer.org>
Date:   Thu Mar 23 16:33:38 2017 +0100

    VertexGetter StringRef methods

commit 8bfcb490d7a907ebd7dcc9e6f38a8d29baa7c7e5
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 16:32:38 2017 +0100

    Fixed AqlValues created by traverser-cache. They are now not using externals anymore which cased everything to crash

commit 614cdedb7e30e35f65f289b310c38b2585eb03e4
Merge: 773b0a5 416777b
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 12:52:19 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache

    This state is still red

commit 773b0a5e51f87875fa9bf1440dd002b2e1fcfd2a
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 12:48:53 2017 +0100

    Added Implementation or readAll in Cluster EdgeCursor. This commit is still red.

commit 82ebbd6ec84918d43bbc64174bc251efa548a4de
Merge: e52b210 35dffc4
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 11:37:31 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache

    This commit is not green.

commit 416777b60a8a0c741d11e9c654745866d0dfa15d
Author: Simon Grätzer <simon@graetzer.org>
Date:   Thu Mar 23 11:25:35 2017 +0100

    Fixed errornous ClusterEdgeCursor implementation of `readAll

commit e52b21025a5f56e2cfc03e3c129da83793f19846
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 11:01:17 2017 +0100

    Fixed API for Traversals: All functions now use StringRefs to identify vertices and edges which are presisted within the traverser-cache.

commit 35dffc439c156366b1110c6770c1565021934632
Author: Simon Grätzer <simon@graetzer.org>
Date:   Wed Mar 22 20:26:41 2017 +0100

    Fixed a few cluster issues

commit 73cf4fe41c363f6c034a1daeacee1630d7e827a5
Author: Simon Grätzer <simon@graetzer.org>
Date:   Wed Mar 22 17:26:12 2017 +0100

    Fixed EdgeCursor bug

commit 2f7329b4e9711740b423cf2963c6cf3f880c4bdf
Merge: bb9b129 8e5edf5
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 22 14:32:07 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 8e5edf5fbe93f34d3340099619baa9a56a738887
Author: Simon Grätzer <simon@graetzer.org>
Date:   Wed Mar 22 14:09:43 2017 +0100

    Fixing compile errors

commit bb9b1294c9ff91f62bc27775a4c7a69c5daca2fd
Merge: 048a4c6 8e51e3b
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 22 11:39:17 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 048a4c65e11d6fd9cb25a58ee97ed756a419a067
Merge: 693607e c6b177d
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 22 11:34:57 2017 +0100

    Solved merge conflicts. Build is still red.

commit c6b177dfc348df0f49f07cd67e83b873871fe6c5
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 23:04:28 2017 +0100

    Fixed off by one error, and crash

commit 5dbc9eb58f1d21e5a530806d7ad7e52058234b83
Merge: 463e352 db8c255
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 17:46:01 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

    # Conflicts:
    #	arangod/Graph/BreadthFirstEnumerator.cpp

commit 463e3520636c19ae7a7c1d445099f336c5bde31e
Merge: 4e75f29 b8cabbe
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 17:00:17 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

    # Conflicts:
    #	arangod/Graph/BreadthFirstEnumerator.cpp

commit 693607ea92ab3116a628c08c59a983a6fdf8b5f4
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 16:26:57 2017 +0100

    Moved specialized NeighborsEnumerator into its own files.

commit db8c25587ebcdeff8b32d08cf2d97d723bf2b3fe
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 15:16:39 2017 +0100

    Replaced SingleServerBreadthFirst with using Callbacks in readAll method of EdgeCursor

commit 4e75f29de95edeea35573b128a0a13e04c9e65d2
Merge: 8716936 6de86c5
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 14:32:12 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

commit b8cabbedebb16c09c8168ec8b5a4499e98c705f4
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 11:17:23 2017 +0100

    Removed _enumertedPath from BreadthFirstEnumerator. Less string copying expected now.

commit 8716936bd480f4b7c85ece6fc1a51335f1942258
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 11:05:04 2017 +0100

    Traverser: changed method definitions

commit 14af02fbc08b7e2ed6b0cbb918ff509ffaef76bf
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 10:45:13 2017 +0100

    Removed usage of _enumeratedPath in lastEdge/Vertex to AQL. Goal is to delete this struct in the class, as it does unneccessary copies

commit 06a20ab0549d97b47eda98d3428e3390619ca5ec
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 10:39:21 2017 +0100

    Intermediate changes

commit 3a9a5f7c7ad186c669c3986f6f89c5a26b835ee0
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 10:34:43 2017 +0100

    Logic of BreadthFirstEnumerator now points to the last path returned instead of the next path to return. This will allow to get rid of enumeratedPath in there.

commit 6de86c548220a3ad21c53fb1de691070774130ec
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 10:17:25 2017 +0100

    Extracted BreadthFirstEnumerator out of PathEnumerator files. Only moving of code.

commit acd1d9eba5d7ea23e0d6beab831a42dc61a385a5
Merge: a99ad3c 1ab8c44
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Mar 20 13:13:16 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit a99ad3c624d7ae4815776f165d65f19e43167e56
Author: Simon Grätzer <simon@graetzer.org>
Date:   Fri Mar 17 13:25:45 2017 +0100

    Added TraverserCache to SingleServerTraverser

commit e88ae1e53d99b55546329413e7b4a5edc48c4875
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 15 13:36:05 2017 +0100

    Fixed compiler Issues in TraverserCache

commit 9d89cdc0297813f24064b46a67a43cde9a3d0297
Merge: 9d41804 f18ad19
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 15 09:43:06 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 9d4180480c7bc37f1f64b6e3141dae611f8d399c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 15 09:42:53 2017 +0100

    Started implementing a Traverser Cache Abstraction that is used to store already fetched documents.
2017-04-13 17:00:44 +02:00
Michael Hackstein 212de2b0d1 Simplified AqlTraverser Setup code by moving shared logic into the TraverserOptions. 2017-04-11 13:47:06 +02:00
Michael Hackstein 052c4942fc Removed superflous aql::ShortestPathOptions and replaced it by traverser::ShortestPathOptions. Api is not fixed yet. 2017-04-10 11:02:01 +02:00
Michael Hackstein 1ab0295f54 Replaced Traverser estimateCost by a more precise version, it now does not check hardcoded on an edge-index estimate. 2017-03-16 09:45:09 +01:00
jsteemann f5c8932744 initial version of query execution plan cache, still very rough 2017-02-21 11:58:38 +01:00
jsteemann 8b273b5eaf Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-02-13 08:37:41 +01:00
jsteemann 1e78acad49 fix size_t vs uint64_t confusion 2017-02-10 23:43:47 +01:00
Michael Hackstein 48166a294c TraversalNode now does not know anything about EdgeIndexes any more. Removed isMMFilesEdgeIndex from Transaction, no obsolete 2017-02-10 18:18:23 +01:00
Michael Hackstein fd7341bbfa Moved a generic Aql Edge Condition builder out of the TraversalNode. It shall be used for all edge queries on other places. 2017-02-09 10:35:41 +01:00
jsteemann 62145baa2a Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-01-31 15:54:39 +01:00
Michael Hackstein 7ab8e823d6 Added a forgotten VPackBuilder.close() in Traversalnode to VPack. This caused the query planner to fail at certain cases 2017-01-31 14:33:10 +01:00
jsteemann 00b1632ece factored out AccessMode from transaction.h 2017-01-25 11:57:21 +01:00
jsteemann c4d2ff3dce renaming 2017-01-23 15:36:11 +01:00
jsteemann b96897e296 fixed some issues detected by coverity build tool 2016-12-12 15:41:54 +01:00
Michael Hackstein 7aea35e444 Fixed a bug when using a skiplist index in traversals like: [a, unused, _from]. If unused is not part of the condition the traverser ran into undef behaviour. 2016-11-29 10:01:34 +01:00
jsteemann fcacd76bca display index usage of traversals in AQL explainer output (previously missing) 2016-11-10 12:49:02 +01:00
Jan Steemann f0f90bc914 small fixes for traversal explain output 2016-11-08 16:08:58 +01:00
Michael Hackstein 69ebf08abe Fixed creation of SmartCollections with correct directions. 2016-10-10 17:49:01 +02:00
Michael Hackstein 9a91e410c2 Fixed a bug with the direction and isSmart check for traverser 2016-10-05 15:26:06 +02:00
Michael Hackstein 1830ea683e Now the traverser is able to work with SmartCollections without Graphs. 2016-10-05 14:47:18 +02:00