1
0
Fork 0
Commit Graph

45788 Commits

Author SHA1 Message Date
Wilfried Goesgens 1ad3d1a327 make sure we don't build paths that start with two \'s (#10144) 2019-10-02 18:35:14 +03:00
jsteemann 1258cdc2d9 use unique_ptr 2019-10-02 16:03:37 +02:00
Jan d29d3df698
improve test statements, fix memleaks and init-order-fiasco (#10142) 2019-10-02 16:02:56 +02:00
Kaveh Vahedipour dc6dba27a2 add repository address normalisation (#10113)
* add repository address normalisation
2019-10-02 11:38:38 +02:00
Michael Hackstein ccb2a3c62e Fixed invalid log ids 2019-10-02 10:26:38 +02:00
Markus Pfeiffer fc634a2fa0 Replace template <bool blockPassthrough> (#10127)
* Replace template <bool blockPassthrough>

by template <BlockPassthrough blockPassthrough> where BlockPassthrough is an
enum class.

This is mainly for better readability and for some type safety.

* Fixup some implicit conversions to bool
2019-10-02 09:47:09 +02:00
Jan c28f571b65
use malloc/realloc/free (#10119) 2019-10-01 19:47:02 +02:00
Jan c3423c7db0
Bug fix/disallow sharding on rev (#10123) 2019-10-01 19:42:55 +02:00
Jan 9898606223 fix logIds (#10135) 2019-10-01 20:19:38 +03:00
Dan Larkin-York c0f63856c2 Fix Windows compile error from network namespace. (#10133) 2019-10-01 18:47:00 +02:00
Wilfried Goesgens 2b743e0e61 Bug fix/speedup tests (#10109) 2019-10-01 18:23:00 +02:00
Kaveh Vahedipour a0be102d2c unitended multiple unlocks (#10114) 2019-10-01 16:21:44 +02:00
Max Neunhöffer 9b3f911cb5
Use execvp instead of execv and handle error. (#10102)
* Use execvp instead of execv and handle error.
* CHANGELOG.
2019-10-01 16:19:47 +02:00
Max Neunhöffer fb8bc4bc31
Fix a subtle special case of the shutdown hanger. (#10120)
* Another shutdown problem with CollectionKeysRepository.
2019-10-01 16:11:33 +02:00
Max Neunhöffer ca9affc798
Add backup sizes. (#10104)
* Single server backup size in META and create.
* More size info in backup.
* Finish size and nrfiles for create and list.
* Add nrDBServers and potentiallyInconsistent to BackupMeta.
* Fix API documentation.
* Make Windows compiler happier.
* One more fix for Windows.
* Added creation date.
2019-10-01 13:31:17 +02:00
Simon 9040f1d18a Fuerte + Pregel + Agency = 🥑 (#10110) 2019-10-01 11:19:18 +02:00
Michael Hackstein c6af396188 Make SubqueryEndExecutor compile again on all platforms again 2019-09-30 17:11:05 +02:00
Jan 1d84dd861d
use setUpAll and tearDownAll to reduce test durations (#10107) 2019-09-30 17:05:20 +02:00
Markus Pfeiffer 32884568cb Feature/subquery end node executor (#10087)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Updates to testing code

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* More updates

* Add optimizer rule

* Fix optimizer rule

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fix optimizer rule (again)

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Attempt to fix :windows: compile warning

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Fixed mac compile issues

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Handle ShadowAqlItemRow in SingleRowFetcher

* Move some code from header to cpp

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Removed unused variable

* Added first test on ShadowRows in SingleRowFetcher. Found inconvenience in original code with it.

* Added a test for multi level shadow rows

* Added a test and fixed production for multiple consecutirve shadow rows.

* Added tests for pass through blocks and shadow rows

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Add SubqueryEndExecutor

* Adapted RowFetcherHelper to be able to work on ShadowRows

* Fixed minor error in RowFetcherHelper that causes LIMIT tests to error

* Temporarily disabled test suites (sorry there is only one 'test case' in this suites that does all or nothing tests ...) as it tests a feature we cannot support in this intermediate state, we need to default disable our new rule first.

* IMplemented missing function to copy over SubqueryDepth between blocks

* Seperated assert, to see faster which part is violated

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Disabled subquery splice rule by default

* Forgotten merge marker

* temporarily modified test

* Some Subquery End Executor work

* Fix RowFetcherHelper to work with ShadowRows

* Add tests for SubqueryEndExecutor

* Rewrite SubqueryEndExecutor

* Update test

* Add createBlock code for SubqueryEndExecutionNode

* Some fixups adding ExecutionBlockImpl for SubqueryEndExecutor

* Fix SubqueryEndExecutor

* Update SubqueryEndExecutorTest

* Update endexecutor test

* Fix the endnode executor once more

* Fix some merge fallout

* Update test

* Add missing advanceRow to SubqueryEndExecutor

* SubqueryEndExecutor Tests

* Activate a death test

* Better handling of death test

* RowFetcherTest refactoring Part 1, still ongoing. We use this for AqlItemMatrix and SingleRowFetcher

* Update and cleanup the SubqueryEndExecutor tests

* Check that expected shadow row depths are met

* Remove commented out code in AqlItemBlock.cpp

* Remove a misplaced comment

* Sort out SubqueryEndExecutor includes

* Remove superflous include

* Consistently use NoStats{} in SubqueryEndExecutor

* Make SubqueryEndExecutor properties constexpr

* Reactivate IResearchTests

* Do not copy the accumulator in SubqueryEndNode

* Return a more sensible value for expectedNumberOfRows of SubqueryEndNodeExecutor

* Constify an argument

* Fix ownership problems with the buffer for _accumulator

* Cleanup SubqueryEndExecutor tests

Also add a test that writes to a register other than 0

* Patch RowFetcherHelper to count the number of Items returned instead of number
of calls

* Make MSVC happy
2019-09-30 15:24:13 +02:00
Dan Larkin-York 1d7225b289 Pass connection pool directly to network methods. (#10096) 2019-09-30 12:44:47 +02:00
Kaveh Vahedipour bbe7d424d5 corrected hot backup lock timings (#10076)
* corrected hot backup lock timings
* the lock timeout added to overall unlock timeout]
2019-09-30 10:22:24 +02:00
Jan 2e293b85ca
Bug fix/cppcheck issues (#10105) 2019-09-30 09:27:33 +02:00
Dan Larkin-York c21f4c92aa Fix handling of systemdb pointer (warnings). (#10106) 2019-09-30 09:20:25 +02:00
Simon cb7bf0314b Use fuerte in RemoteExecutor (#10077) 2019-09-27 16:20:38 +02:00
Jan a088678866
fix broken LDAP tests (#10099) 2019-09-27 16:17:19 +02:00
Jan c0136b49c7
use static_cast instead of dynamic_cast (#10095)
* use static_cast instead of dynamic_cast

* remove virtual inheritance
2019-09-27 12:58:52 +02:00
Dan Larkin-York 5204e95512 Add log suppressor utility classes for cleaner test code. (#10088) 2019-09-27 12:57:38 +02:00
Tobias Gödderz 4ea18d9031 Improved query tracing, especially useful for cluster queries (#10091)
* Improved query tracing, especially useful for cluster queries

* Fix duplicate log id
2019-09-27 10:52:22 +02:00
Tobias Gödderz f120919dfe Implement a constrained sorting gather variety (#10057)
* Added TODO notes

* Pass limit of constrained sort to sorting gather

* Count output rows, and added assertions

* Extracted row-producing code in a reusable way

* Implemented SortingGatherExecutor::skipRows

* Fixed signature

* Fixed an assertion

* Added limit to (de)serialization

* Fixed an assertion

* Fixed an atMost calculation

* In the sort-limit optimization, apply limit to sorting gather as well

* Fixed optimizer rule reporting modification

* Added missing initialization

* Fix: Do not produce more rows than the limit when skipping first

* Fix: pass limit when cloning

* Updated CHANGELOG

* Fixed atMost intricacies
2019-09-27 10:50:06 +02:00
Dan Larkin-York dc23896a01 Make count, figures, revision, and index warmup use non-blocking communication (#10048) 2019-09-27 09:54:01 +02:00
Jan 176cff3eac
remove unused AQL attribute (#10094) 2019-09-26 17:16:43 +02:00
Tobias Gödderz 5e8bdcbc24 Add definitions for static variables in Executor::Properties (#10093) 2019-09-26 14:48:29 +02:00
Jan 9533d1b71c
fix skipInaccessible (#10092) 2019-09-26 14:45:58 +02:00
Jan 9da2cadf83
Bug fix/issue 10078 (#10081) 2019-09-26 12:08:48 +02:00
Wilfried Goesgens 636b2e5d5f fixed version of arangosh result evaluation refactoring - properly count objects (#10080) 2019-09-26 12:07:52 +02:00
Michael Hackstein e89d72d02c
Feature/aql subquery start executor (#10040)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Updates to testing code

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* More updates

* Add optimizer rule

* Fix optimizer rule

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fix optimizer rule (again)

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Attempt to fix :windows: compile warning

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Fixed mac compile issues

* Initial commit, added class and test stubs

* Added Executor body stub, also added a test for no input. Passes now, as no input is read yet.

* Base implementation + Test of SubqueryStart Executor. First level shadowRows are now produced. Need to add tests for multilevel, and for non-fitting output blocks.

* Added more sophisticated test about shadow row relevance

* Added a test where ShadowRow does not fit into the open block. Also added disabled test on multi level shadowrows. Requires modifications on SingleRowFetcher.

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Handle ShadowAqlItemRow in SingleRowFetcher

* Move some code from header to cpp

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Removed unused variable

* Added first test on ShadowRows in SingleRowFetcher. Found inconvenience in original code with it.

* Added a test for multi level shadow rows

* Added a test and fixed production for multiple consecutirve shadow rows.

* Added tests for pass through blocks and shadow rows

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Adapted RowFetcherHelper to be able to work on ShadowRows

* Seperated assert, to see faster which part is violated

* Fixed minor error in RowFetcherHelper that causes LIMIT tests to error

* Temporarily disabled test suites (sorry there is only one 'test case' in this suites that does all or nothing tests ...) as it tests a feature we cannot support in this intermediate state, we need to default disable our new rule first.

* Improve check of ExecutionState

* Implemented shadow row handling in subquery start executor. Tests not happy yet

* IMplemented missing function to copy over SubqueryDepth between blocks

* Fixed test code of StartExecutor test. It asserted wrongly on releveance of subquery rows.

* Added instanciation of SubqueryStartExecutorTest. Code does not compile.

* Allow to create SubqueryStartExecutors.

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Disabled subquery splice rule by default

* Forgotten merge marker

* temporarily modified test

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning

* Removed incorrect optimization in SubqueryStartExecutor
2019-09-26 10:01:52 +02:00
Dan Larkin-York a83c2323c9 Refactor ApplicationServer stack (#9965) 2019-09-25 17:31:59 +02:00
Michael Hackstein ca30314c78 Fixed now missing include in test files 2019-09-25 13:24:27 +02:00
jsteemann d1116c1c72 fix compile error 2019-09-25 13:02:08 +02:00
Jan 8a56ed9a2c
fix sporadically failing one shard test (#10074) 2019-09-25 12:52:31 +02:00
Markus Pfeiffer 0b9dfb4b9b Subquery Start/End Nodes (#9983)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Updates to testing code

* More updates

* Add optimizer rule

* Fix optimizer rule

* Fix optimizer rule (again)

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Disabled subquery splice rule by default

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning
2019-09-25 12:35:18 +02:00
jsteemann d0324b8b08 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2019-09-25 11:46:12 +02:00
jsteemann 1aafa1be28 remove unused GraphCache 2019-09-25 11:45:58 +02:00
Tobias Gödderz 82b0a5e745 De-inline in AQL (#10025)
* Enable IPO (LTO) via CMake

* Use separate IPO_ENABLED variable

* Enabled IPO for more binaries

* Enabled IPO for arangobackup

* Suppress an MSVC warning (though it is not unjustified...)

* Fix static builds, broken due to Policy CMP0060 introduced in cmake 3.3

* Removed policies superseded by minimum cmake version

* Disable IPO with google tests due to a g++ bug

* Disable IPO with google tests only on AUTO

* Disable warning for correct line

* Begin de-inlining Aql

* de-inlining Aql: 2nd batch

* de-inlining Aql: 3rd batch

* de-inlining Aql: 4th batch

* Moved code out of ifdef

* Enabled IPO for additional libs

* Fixed some problems found with IPO enabled

- Fixed ODR violation in Pregel
- Removed unused code in ClusterMethods
- Avoid possible uninitialized variable usage

* Set IPO globally, except for 3rdParty libs

* De-inlined AstHelper again (was undone due to a previous merge conflict)

* Added missing .cpp file

* Fixed compilation with MSVC

* Removed superfluous includes from deinlined AQL header files

* Added includes for size_t

* Deleted outdated files

* De-inlined files that were undone during the merge

* Removed includes from files that were undone during merge

* Fixed merge conflicts
2019-09-25 11:20:03 +02:00
Wilfried Goesgens 41c07102c8 Feature/programm options new vector type (#10071) 2019-09-25 10:39:27 +02:00
Wilfried Goesgens 61cbff5e2e Fix windows path handling not to slip into UNC lookup (#10073)
* make sure that we don't get more than one leading directory separator in front of the path, so windows doesn't mistakenly look it up as a UNC.
2019-09-25 10:23:04 +02:00
Jan ca3c742652
improve binary search a bit, improve string comparisons (#10069) 2019-09-24 18:20:34 +02:00
jsteemann a2e0fc643b cppcheck 2019-09-24 16:59:00 +02:00
jsteemann de2ab1246e updated CHANGELOG 2019-09-24 15:43:22 +02:00
Jan 9f51c03dd7
re-open the PR, queue length counter still underflows (#10065) 2019-09-24 15:39:37 +02:00