1
0
Fork 0
Commit Graph

4599 Commits

Author SHA1 Message Date
jsteemann d6d8f321e7 move static variable from function into TU scope 2019-10-22 16:04:55 +02:00
Michael Hackstein bf2423d024
Feature/explain spliced subqueries (#10298)
* Implemented explain output for Subquery start and End nodes. Unfortunately i had to inject a temporary output variable for the Subquery start.
2019-10-22 15:38:33 +02:00
Simon Grätzer 02760bff67
ffffff 2019-10-22 12:48:18 +02:00
Simon Grätzer 5ce1b8bcc1
ffffff 2019-10-22 12:47:44 +02:00
Jan 09134ee8d4
micro optimizations for case conversion (#10291) 2019-10-22 09:39:16 +02:00
Simon Grätzer 5417e32beb
Merge branch 'devel' of github.com:arangodb/arangodb into feature/parallel-aql-phase-one 2019-10-21 13:51:45 +02:00
Jan Christoph Uhde ea20f7aeb2 use C++17 [[fallthrough]] (#10280) 2019-10-19 18:14:26 +02:00
Simon Grätzer c2d68609c5
fix shutdown 2019-10-18 16:38:04 +02:00
Simon Grätzer 162ea1c62a
fix gather 2019-10-17 23:10:59 +02:00
Simon Grätzer 7ced6b2064
fix wakeup 2019-10-17 19:34:10 +02:00
Simon Grätzer 1bbc2f144d
Merge branch 'devel' of github.com:arangodb/arangodb into feature/parallel-aql-phase-one 2019-10-17 18:45:02 +02:00
Tobias Gödderz e1961ebb66 Avoid a race and a use-after-free bug in the RemoteExecutor (#10275) 2019-10-17 18:22:13 +02:00
Simon d526805e81 Bug fix/fix suspicous stuff (#10273) 2019-10-17 15:34:22 +02:00
Jan cab9ea9132
fixed issue #10270 (#10271) 2019-10-17 12:00:09 +02:00
Jan Christoph Uhde b16a5e6393 C++17 (#10178)
* Squashed commit of the following:

commit dd4b85fcaa634ed829e79e266ee547c0522e71f6
Merge: 9fc1548e54 fa7de56cf8
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Thu Jan 10 10:21:26 2019 +0100

    Merge branch 'devel' into feature/cpp17

    * devel:
      upgrade to boost 1.69.0 (#7910)
      Doc - Add a paragraph that JS JS trxs are excluded from intermediate commits. (#7919)
      issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step (#7918)
      improve assertions (#7830)
      fix MSVC warning
      remove unused function
      fix #7900 - null bind values do not change to empty string anymore (#7917)

commit 9fc1548e54f8cfd2990e7ae0b2b7ae3da55a2cd2
Merge: f2ecee8136 e7810e1956
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Jan 9 09:36:19 2019 +0100

    Merge remote-tracking branch 'origin/devel' into feature/cpp17

    * origin/devel:
      Doc - Update remark about per-database replication (#7891)
      Doc - Known Issues - new links (#7908)
      support db._explain with all plans (#7895)
      JWT base64url encoding (#7899)
      cppcheck/Scheduler (#7909)
      multiplex REPLICATION-APPLIER-STATE files for RocksDB engine (#7898)
      fix compile warnings
      upgrade bundled curl library to version 7.63 (#7846)
      Feature/new server infra (#7733)
      Doc - check_function (#7885)
      simran (#7881)
      added AQL function CHECK_DOCUMENT (#7842)
      added arangorestore option `--cleanup-duplicate-attributes` (#7877)
      Doc - arangorestore improvements (#7878)
      added arangorestore options `--number-of-shards` and `--replication-f… (#7869)

commit f2ecee81360f0f0c19901accdbc9902304814bb2
Merge: 44adf97fa0 5c53ebceb0
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Thu Jan 3 10:51:20 2019 +0100

    Merge remote-tracking branch 'origin/devel' into feature/cpp17

    * origin/devel: (40 commits)
      Feature/jwt keyfile (#7863)
      prevent duplicate attributes being generated by AQL queries (#7837)
      fixed issue #7834 (#7843)
      Bug fix/issues 23122018 (#7847)
      Documents that arangorestore/arangodump threads option is from v3.4.0 on (#7701)
      Update ISSUE_TEMPLATE.md (#7103)
      Doc - Forwardports to devel - 2018-12-27 (#7858)
      Doc - Sync-external-repo-2018-12-25 (#7854)
      Doc - re-add switching storage engine section (#7781)
      Doc - Fix build (whitespaces, triple-backticks, comments) (#7857)
      Add http_server test for creating view with links (#7852)
      big reformat
      added new clang style and reformat script for clang-format 6.0
      Doc - Improve MacOS Installation page (#7744)
      Fix issue with geo iterator reset. (#7839)
      Doc - Single Instance vs. Cluster (diff + migration) (#7739)
      Rdb index background (preliminary) (#7644)
      iResearch asan issue (#7832)
      Feature/improve edgeindex covered (#7718)
      issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19 (#7819)
      ...

commit 44adf97fa0bc9cc7dc649c98916a91dba456b956
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Dec 18 13:39:03 2018 +0100

    Add COMPILER_* variables to VERSIONS file.

    The variables will be used to select a fitting compiler in the CI.
    Developers need to ensure themselves that their development environment
    supports the c++ standard.

commit 8124eb3b44fb2302cdd7250d1573b25135719e41
Merge: 127bb7d1cb 4423125a7f
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Dec 18 11:09:42 2018 +0100

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb: (381 commits)
      tools are now looking into the build to find the file (#7128)
      issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server (#7748)
      Fix S2 for aarch64. (#7770)
      equalising devel and 3.4 in agency/cluster (#7755)
      wording
      startLocalCluster in rr debugging mode cannot be started with --console on agents / db servers (#7776)
      fix cmake parameters coincidence (#7774)
      Fix a rare deadlock situation in Replication sync phase (#7759)
      add virtual dtor
      wait for procdump to exit too. (#7731)
      fix invalid handling of `_lastValue` in case of multiple coordinators (#7734)
      fixed item 3 of issue #7009 (#7747)
      updated 3rdparty licenses
      remove inclusion of valgrind.h
      remove now-unused file
      Bug fix/fix thread shutdown (#7728)
      Doc - fix build by adding missing SUMMARY entries (#7741)
      issue 511.2.1: use references instead of raw pointers for Index operations to avoid null pointer access (#7725)
      port 3.4 change to stopBackgroundThreads.  Move ClusterCommThread routines to same place in source file. (#7721)
      Ignore invaild geo coordinates when indexing. (#7724)
      ...

commit 127bb7d1cb56fb71b06613971c896aedc1817a98
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Oct 15 22:06:27 2018 +0200

    fix invoke for c++17

commit 65a514c34472a611167cd01e14efd5dccc9ae2d1
Author: Frank Celler <frank@arangodb.com>
Date:   Mon Oct 15 18:58:31 2018 +0200

    trying to use 17

commit 8344d96e43522757813bb37e10b95ae83ff35abf
Merge: 2822dae6e1 c7933bcb31
Author: Frank Celler <frank@arangodb.com>
Date:   Mon Oct 15 18:08:52 2018 +0200

    Merge branch 'devel' of ssh://github.com/arangodb/ArangoDB into feature/cpp17

commit 2822dae6e175e866106c49ed8e1f5248dcef0b62
Merge: 4001d11afd af2f18fcbf
Author: Frank Celler <frank@arangodb.com>
Date:   Fri Oct 12 17:44:04 2018 +0200

    Merge branch 'feature/boost-168' of ssh://github.com/arangodb/ArangoDB into feature/cpp17

commit 4001d11afd6ed691ce9dcce34b1ee7b75a08be4b
Author: Frank Celler <frank@arangodb.com>
Date:   Fri Oct 12 17:43:46 2018 +0200

    fixed merge conflict

commit 94cb4b7d4c43c7c12db7ccc3c7cb1c1885e9fafa
Merge: cbc2f34bb4 d3c446156c
Author: Frank Celler <frank@arangodb.com>
Date:   Fri Oct 12 14:35:42 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

commit af2f18fcbf709a86e76feb5165ea848f3d4eba67
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Fri Sep 28 14:44:16 2018 +0200

    add bcrypt.lib to make uuid work

commit 11efd1299a40a38ef92d717a8ee9e882fc77c82b
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 12:31:25 2018 +0200

    fix boost path for iresearch that is not able to use the provided
    information

commit 57ed116b6c7c5fda1180929c4f1d63895c25ea1f
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 10:24:10 2018 +0200

    add missing boost core includes

commit 5d52b322a0fa8fccb4c0d50f5677f7c4b0505284
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 09:22:43 2018 +0200

    upgrade boost to version 1.68.0

commit cbc2f34bb475f988fc407b2fd332d9984fc682db
Merge: 3de8d481e6 1b1142647a
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 12:08:13 2018 +0200

    Merge branch 'feature/cpp17' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'feature/cpp17' of https://github.com/arangodb/arangodb:
      allow MSVC to user autopointer with c++17
      nightly frontend build
      Remove unused variable check from arangodump. (#6529)
      fixed JavaScript error
      added a reserve call
      Bug fix/simplify things (#6516)
      catch 501 (not implemented) in the query editor (ui) (#6523)
      return error codes in case a babies remove operation fails (#6527)
      Use physical collection directly (#6526)
      Feature/aql item block compression (#6514)
      move tests to the right location
      take over selectivity estimates (#6505)
      Properly check syncer erros, catch more exceptions (#6520)
      prevent out-of-bounds array access
      Fix deduplication in IndexBlock (#6508)

commit 3de8d481e6c40ab1b160ca931b23882fe36048c2
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 12:05:56 2018 +0200

    add autopointer and tr1

commit 1b1142647ac79764c55c0688c1b22a0227c7fad6
Merge: 4b38f0a7a0 0dd277d325
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 10:06:01 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb:
      nightly frontend build
      Remove unused variable check from arangodump. (#6529)
      fixed JavaScript error
      added a reserve call
      Bug fix/simplify things (#6516)
      catch 501 (not implemented) in the query editor (ui) (#6523)
      return error codes in case a babies remove operation fails (#6527)
      Use physical collection directly (#6526)
      Feature/aql item block compression (#6514)
      move tests to the right location
      take over selectivity estimates (#6505)
      Properly check syncer erros, catch more exceptions (#6520)
      prevent out-of-bounds array access
      Fix deduplication in IndexBlock (#6508)

commit 4b38f0a7a003a2da813a363b91dc289419e0210c
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 10:04:46 2018 +0200

    allow MSVC to user autopointer with c++17

commit 9c9102341245f2bc4373d11cd69c8e4c3d9fb129
Merge: 1b9e580f14 e51359b68e
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 17 12:53:03 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb:
      port startup fix
      Bugfix/early out invalid links in view creation (#6502)

commit 1b9e580f14db737f005b7eed9926c222fa047754
Merge: b08bc2e9ba 8bd834bcf7
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 17 06:38:24 2018 +0200

    Merge remote-tracking branch 'origin/devel' into feature/cpp17

    * origin/devel: (38 commits)
      Maintenance delayed by incomplete hashing maintenance actions (#6448)
      Do not hide indexes (#6507)
      Fix WebUI Login with read-only (#6500)
      Improve use of single shard restriction (#6182)
      make cid values in /_api/replication/logger-follow strings as advertised (#6499)
      less dynamic memory allocations on index operations (#6497)
      Fix several TSan warnings (#6473)
      Copy installation files on startup (#6491)
      nightly frontend build
      fix cluster selectivity estimates (#6488)
      this moves aql::Query* out of the call parameter to any aql::Function (#6485)
      Add advertised endpoints. (#6104)
      nightly frontend build
      fix leak in regex cache (#6480)
      Removing ClusterComm ClientTransactionID (#6294)
      pass variables by const reference
      remove unused variable
      remove unused variable
      fix init-order fiasco with static strings (#6475)
      minor issues (#6471)
      ...

commit b08bc2e9ba1553d608ae1bd5dc4a42290fa2cabd
Merge: 03fabcc509 2eb3c2dee4
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Sep 11 12:46:51 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb:
      temporarily increase log verbosity for replication log topic
      Fix skipSome PR (#6455)
      bump version number to 3.5 (#6454)
      Feature/new devel versioning scheme (#6312)
      Bug fix/aql modify cleanup (#6380)
      Trace skipSome() calls (#6449)
      nightly frontend build
      Fix ArangoSearch authentication update task test (#6453)
      Forward port changelog changes and typo fixes from #6303 (#6409)
      fix some issues with sorted variant of COLLECT (#6433)
      added tests, updated docu (#6407)
      now that we are on c++14 and std::make_unique is available everywhere, we can safely remove our shim for std::make_unique (#6429)
      Fixed memory leak due to cyclic references of shared pointer. (#6442)
      do not create empty useless transaction data objects (#6430)
      add some debugging for unstable replication_sync test (#6436)
      Address leak sanitizer warnings (#6275)
      Fix Pregel Graph Loading Logic (#6419)
      remove unused variables
      fix profile output for nodes without calls to getSome (#6397)

commit 03fabcc5098f4e097d31546835fbadd373e854c1
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Sep 11 07:19:21 2018 +0200

    switch more libs to c++17

commit 335942038f5d0d817554f59a5943e041d928ec40
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 10 07:31:49 2018 +0200

    fix static asserts in iresearch

commit 90eab952765c26d165e57dc8e01ee42bc1d2d8e6
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 10 07:10:48 2018 +0200

    move from c++14 to c++17

* specify required clang and gcc versions

* fix openssl 1.3 usage

* change random_shuffle to shuffle as the former has been removed from the c++ standard

https://meetingcpp.com/blog/items/stdrandom_shuffle-is-deprecated.html

* disable uncaught_exceptions in catch2 for darwin

In file included from /Users/jenkins/Git/cpp17/tests/main.cpp:2:
/Users/jenkins/Git/cpp17/3rdParty/catch/catch.hpp:10877:21: error: 'uncaught_exceptions' is unavailable: introduced in macOS 10.12
        return std::uncaught_exceptions() > 0;
                    ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/exception:130:63: note: 'uncaught_exceptions' has been explicitly marked unavailable here
_LIBCPP_FUNC_VIS _LIBCPP_AVAILABILITY_UNCAUGHT_EXCEPTIONS int uncaught_exceptions() _NOEXCEPT;
                                                              ^

* boost locale now depends on boost thread

* add fix for VS2019

* try to fix build for windows

* add some change required by older macos

* add miserable hack to circumvent std::stack implementation problem on older Xcode

* try to fix windows build

* remove unary function as in v8 upstream

* clean up CMakeLists.txt

* Remove emplace hack that was introduced for OSX

* Revert "Remove emplace hack that was introduced for OSX"

This reverts commit 2b0a4cdd7881baa9ea8fbd34f2e89ef6fa9b9b63.

* Remove emplace hack that was introduced on OSX to support C++17 (#10184)

* Revert "Revert "Remove emplace hack that was introduced for OSX""

This reverts commit 7df9edde4097a951e95fcb204113d9c5289bf4bf.

* fixed MACOSX_DEPLOYMENT_TARGET

* Fix quotes usage in conditions

* Added MACOS_MIN

* Adjust variables

* Fix compilation

* remove iresearch patches

* Update CMakeLists.txt

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-10-16 20:28:27 +03:00
Simon Grätzer 1e241c16a9
fix test 2019-10-16 15:11:44 +02:00
Dronplane cfce3d8df8 Bug fix/internal issue #627 (#10233)
* Implemented Array IN operators

* First implementation for array comparsion operators for SEARCH clause

* WIP

* WIP

* simplified filter building

* Adding Unit tests

* Added tests for IN/NIN

* Debuggin Array comparsion operators. Fixed ViewNode redundand register planning

* Fixed log id duplicate

* Fixed jslin

* Fixed Mac build

* Code cleanup

* WIP

* WIP

* Code cleanup

* fixed applying boost to all/empty filters

* Removed redundancy in filter generation

* cleanup
2019-10-15 20:41:06 +03:00
Simon Grätzer b575c3a971
Support for fetching in parallel & skipping 2019-10-15 16:38:08 +02:00
Simon 4d8c909666 A mixed bag of goods (#10220) 2019-10-14 17:02:36 +02:00
Jan a3da5cec88
disallow subqueries in AQL traversal PRUNE conditions (#10232) 2019-10-14 16:01:55 +02:00
Dan Larkin-York 6fd419714f Move containers into their own folder and namespace. (#10244) 2019-10-14 10:55:55 +02:00
jsteemann 6d67f3dd4c unintentionally broke something. fixed... 2019-10-11 08:53:27 +02:00
Jan 46674831c9
Bug fix/improve stringutils performance (#10208) 2019-10-10 17:08:03 +02:00
jsteemann 184e8ae6be add missing include 2019-10-10 15:04:18 +02:00
Dan Larkin-York 13e24b2db9 Convert many uses of ClusterComm to Fuerte (#10154) 2019-10-10 14:03:33 +02:00
Tobias Gödderz 4e86de00d2 AQL Subquery: MultiDependencyRowFetcher (#10101)
Add ShadowRow support for MultiDependencyRowFetcher
2019-10-10 13:08:02 +02:00
Tobias Gödderz 6528c592c5 Bug fix/fix remote executor races (#10206)
* Fix races in RemoteExecutor

* Removed #ifdef
2019-10-10 12:36:53 +02:00
Tobias Gödderz d175dfb3b1 Bugfix for recurring Jenkins error in api-simple-modify-example-cluster-spec.rb (#10190)
DependencyProxy::skipSome did not work correctly with multiple
dependencies if atMost was reached, returning DONE early.
2019-10-08 19:18:01 +02:00
Dronplane a1014ac701 Bug fix/internal issue #600 - Late document materialization (#10006)
* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* adding plan traversal for rule

* Rule execution block developing

* Experiment with additional block

* LIMIT node modifications

* implemented late materialization base version.

* Code cleanup

* Added check for variable usage

* Implemented serialization and cloning

* Fixed build errors

* Implemented late materialization for  LimitNode

* Moved materialization to Sort Node

* Removed LimitNode modiications

* Implemented skipRows to speedup single server

* Added tests for optimizer rule

* fixed build

* Fix mac build

* Fixed mac build

* Fixed PR test run errors

* Added tmp fix for cluster test runs

* removed debug print

* Make materialization stick to last sort node. Add late materialization to explainer

* Updated optimization rule to search optimal SortNode to do materialization

* Refined optimization rules for cluster

* Added tests for SortExecutor and IResearchViewNode

* Fixed  mac buid. Removed misplaced assertion.

* Fixed build

* Updated tests. Coded cleanup

* Code cleanup

* made function name for SortedExecutor match name in ConstraintSortedExecutor

* cleanup

* Fixed test run

* made scoring results stable across cluster

* Make optimize rule stop if  sort node for limit node is not suitable

* Disabled constrained sort for materializing SortNode

* reverted SortNode modifications

* Implemented separate materializer node

* Fixed tests  to account new materialize node

* Code cleanup

* reverted debugging change

* Fixed test jslint error  and comments

* MaterializeNode renamed

* enum value renamed for materialize node

* Code cleanup

* code cleanup and optimization

* Fixed clang warning

* Applied review comments

* Out variable planning moved to materialize node

* inlined getters in ReadContex
2019-10-08 18:38:47 +03:00
Tobias Gödderz f2265f9306 Move warning-suppressing comment to the right place (#10191) 2019-10-08 16:49:13 +02:00
Michael Hackstein 1755b2b253
Feature/aql subquery single input tests (#10175)
Added DISABLED_ tests for the new Subquery Optimization Rule
2019-10-08 15:29:09 +02:00
Jan 35786aa517
decrease payload size of some lambdas (#10167) 2019-10-08 10:12:22 +02:00
Markus Pfeiffer e9ab15db03 Add a comment that subquery splicing should run last (#10181) 2019-10-08 09:13:50 +02:00
Michael Hackstein cf99ff1586
Feature/aql subquery all rows fetcher shadow rows (#10079)
Allow handling of ShadowRows in AllRowsFetcher
2019-10-07 16:49:30 +02:00
Jan 6f14048bbe
implement early pruning of documents in EnumerateCollectionExecutor (#10159) 2019-10-05 13:38:23 +02:00
Tobias Gödderz 8603ceb995 Implement copying/moving constructors of InputAqlItemRow (#10165) 2019-10-04 17:46:08 +02:00
Tobias Gödderz 4ec4218213 Replace union by separate members (#10168) 2019-10-04 17:38:29 +02:00
Tobias Gödderz e2c84acfaf Use explicit default destructors where possible (#10166) 2019-10-04 15:58:30 +02:00
Jan 0da109bf82
fixed issue #10158 (#10162) 2019-10-04 13:53:31 +02:00
Jan ae05fafa4e
Bug fix/fix parsing unterminated strings (#10155) 2019-10-03 13:38:05 +02:00
Jan a1775b7930
cleanup some of the internal APIs a bit (#10152) 2019-10-03 00:02:27 +02:00
Markus Pfeiffer fc634a2fa0 Replace template <bool blockPassthrough> (#10127)
* Replace template <bool blockPassthrough>

by template <BlockPassthrough blockPassthrough> where BlockPassthrough is an
enum class.

This is mainly for better readability and for some type safety.

* Fixup some implicit conversions to bool
2019-10-02 09:47:09 +02:00
Jan 9898606223 fix logIds (#10135) 2019-10-01 20:19:38 +03:00
Simon 9040f1d18a Fuerte + Pregel + Agency = 🥑 (#10110) 2019-10-01 11:19:18 +02:00
Michael Hackstein c6af396188 Make SubqueryEndExecutor compile again on all platforms again 2019-09-30 17:11:05 +02:00
Markus Pfeiffer 32884568cb Feature/subquery end node executor (#10087)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Updates to testing code

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* More updates

* Add optimizer rule

* Fix optimizer rule

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fix optimizer rule (again)

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Attempt to fix :windows: compile warning

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Fixed mac compile issues

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Handle ShadowAqlItemRow in SingleRowFetcher

* Move some code from header to cpp

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Removed unused variable

* Added first test on ShadowRows in SingleRowFetcher. Found inconvenience in original code with it.

* Added a test for multi level shadow rows

* Added a test and fixed production for multiple consecutirve shadow rows.

* Added tests for pass through blocks and shadow rows

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Add SubqueryEndExecutor

* Adapted RowFetcherHelper to be able to work on ShadowRows

* Fixed minor error in RowFetcherHelper that causes LIMIT tests to error

* Temporarily disabled test suites (sorry there is only one 'test case' in this suites that does all or nothing tests ...) as it tests a feature we cannot support in this intermediate state, we need to default disable our new rule first.

* IMplemented missing function to copy over SubqueryDepth between blocks

* Seperated assert, to see faster which part is violated

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Disabled subquery splice rule by default

* Forgotten merge marker

* temporarily modified test

* Some Subquery End Executor work

* Fix RowFetcherHelper to work with ShadowRows

* Add tests for SubqueryEndExecutor

* Rewrite SubqueryEndExecutor

* Update test

* Add createBlock code for SubqueryEndExecutionNode

* Some fixups adding ExecutionBlockImpl for SubqueryEndExecutor

* Fix SubqueryEndExecutor

* Update SubqueryEndExecutorTest

* Update endexecutor test

* Fix the endnode executor once more

* Fix some merge fallout

* Update test

* Add missing advanceRow to SubqueryEndExecutor

* SubqueryEndExecutor Tests

* Activate a death test

* Better handling of death test

* RowFetcherTest refactoring Part 1, still ongoing. We use this for AqlItemMatrix and SingleRowFetcher

* Update and cleanup the SubqueryEndExecutor tests

* Check that expected shadow row depths are met

* Remove commented out code in AqlItemBlock.cpp

* Remove a misplaced comment

* Sort out SubqueryEndExecutor includes

* Remove superflous include

* Consistently use NoStats{} in SubqueryEndExecutor

* Make SubqueryEndExecutor properties constexpr

* Reactivate IResearchTests

* Do not copy the accumulator in SubqueryEndNode

* Return a more sensible value for expectedNumberOfRows of SubqueryEndNodeExecutor

* Constify an argument

* Fix ownership problems with the buffer for _accumulator

* Cleanup SubqueryEndExecutor tests

Also add a test that writes to a register other than 0

* Patch RowFetcherHelper to count the number of Items returned instead of number
of calls

* Make MSVC happy
2019-09-30 15:24:13 +02:00
Jan 2e293b85ca
Bug fix/cppcheck issues (#10105) 2019-09-30 09:27:33 +02:00
Simon cb7bf0314b Use fuerte in RemoteExecutor (#10077) 2019-09-27 16:20:38 +02:00
Tobias Gödderz 4ea18d9031 Improved query tracing, especially useful for cluster queries (#10091)
* Improved query tracing, especially useful for cluster queries

* Fix duplicate log id
2019-09-27 10:52:22 +02:00
Tobias Gödderz f120919dfe Implement a constrained sorting gather variety (#10057)
* Added TODO notes

* Pass limit of constrained sort to sorting gather

* Count output rows, and added assertions

* Extracted row-producing code in a reusable way

* Implemented SortingGatherExecutor::skipRows

* Fixed signature

* Fixed an assertion

* Added limit to (de)serialization

* Fixed an assertion

* Fixed an atMost calculation

* In the sort-limit optimization, apply limit to sorting gather as well

* Fixed optimizer rule reporting modification

* Added missing initialization

* Fix: Do not produce more rows than the limit when skipping first

* Fix: pass limit when cloning

* Updated CHANGELOG

* Fixed atMost intricacies
2019-09-27 10:50:06 +02:00
Jan 176cff3eac
remove unused AQL attribute (#10094) 2019-09-26 17:16:43 +02:00
Tobias Gödderz 5e8bdcbc24 Add definitions for static variables in Executor::Properties (#10093) 2019-09-26 14:48:29 +02:00
Jan 9533d1b71c
fix skipInaccessible (#10092) 2019-09-26 14:45:58 +02:00
Michael Hackstein e89d72d02c
Feature/aql subquery start executor (#10040)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Updates to testing code

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* More updates

* Add optimizer rule

* Fix optimizer rule

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fix optimizer rule (again)

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Attempt to fix :windows: compile warning

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Fixed mac compile issues

* Initial commit, added class and test stubs

* Added Executor body stub, also added a test for no input. Passes now, as no input is read yet.

* Base implementation + Test of SubqueryStart Executor. First level shadowRows are now produced. Need to add tests for multilevel, and for non-fitting output blocks.

* Added more sophisticated test about shadow row relevance

* Added a test where ShadowRow does not fit into the open block. Also added disabled test on multi level shadowrows. Requires modifications on SingleRowFetcher.

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Handle ShadowAqlItemRow in SingleRowFetcher

* Move some code from header to cpp

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Removed unused variable

* Added first test on ShadowRows in SingleRowFetcher. Found inconvenience in original code with it.

* Added a test for multi level shadow rows

* Added a test and fixed production for multiple consecutirve shadow rows.

* Added tests for pass through blocks and shadow rows

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Adapted RowFetcherHelper to be able to work on ShadowRows

* Seperated assert, to see faster which part is violated

* Fixed minor error in RowFetcherHelper that causes LIMIT tests to error

* Temporarily disabled test suites (sorry there is only one 'test case' in this suites that does all or nothing tests ...) as it tests a feature we cannot support in this intermediate state, we need to default disable our new rule first.

* Improve check of ExecutionState

* Implemented shadow row handling in subquery start executor. Tests not happy yet

* IMplemented missing function to copy over SubqueryDepth between blocks

* Fixed test code of StartExecutor test. It asserted wrongly on releveance of subquery rows.

* Added instanciation of SubqueryStartExecutorTest. Code does not compile.

* Allow to create SubqueryStartExecutors.

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Disabled subquery splice rule by default

* Forgotten merge marker

* temporarily modified test

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning

* Removed incorrect optimization in SubqueryStartExecutor
2019-09-26 10:01:52 +02:00
Dan Larkin-York a83c2323c9 Refactor ApplicationServer stack (#9965) 2019-09-25 17:31:59 +02:00
jsteemann d1116c1c72 fix compile error 2019-09-25 13:02:08 +02:00
Markus Pfeiffer 0b9dfb4b9b Subquery Start/End Nodes (#9983)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Updates to testing code

* More updates

* Add optimizer rule

* Fix optimizer rule

* Fix optimizer rule (again)

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Disabled subquery splice rule by default

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning
2019-09-25 12:35:18 +02:00
Tobias Gödderz 82b0a5e745 De-inline in AQL (#10025)
* Enable IPO (LTO) via CMake

* Use separate IPO_ENABLED variable

* Enabled IPO for more binaries

* Enabled IPO for arangobackup

* Suppress an MSVC warning (though it is not unjustified...)

* Fix static builds, broken due to Policy CMP0060 introduced in cmake 3.3

* Removed policies superseded by minimum cmake version

* Disable IPO with google tests due to a g++ bug

* Disable IPO with google tests only on AUTO

* Disable warning for correct line

* Begin de-inlining Aql

* de-inlining Aql: 2nd batch

* de-inlining Aql: 3rd batch

* de-inlining Aql: 4th batch

* Moved code out of ifdef

* Enabled IPO for additional libs

* Fixed some problems found with IPO enabled

- Fixed ODR violation in Pregel
- Removed unused code in ClusterMethods
- Avoid possible uninitialized variable usage

* Set IPO globally, except for 3rdParty libs

* De-inlined AstHelper again (was undone due to a previous merge conflict)

* Added missing .cpp file

* Fixed compilation with MSVC

* Removed superfluous includes from deinlined AQL header files

* Added includes for size_t

* Deleted outdated files

* De-inlined files that were undone during the merge

* Removed includes from files that were undone during merge

* Fixed merge conflicts
2019-09-25 11:20:03 +02:00
jsteemann a2e0fc643b cppcheck 2019-09-24 16:59:00 +02:00
jsteemann 02f9eb06f8 make sleeping more accurate 2019-09-24 15:18:21 +02:00
Jan 4f2e1d4054
fixed issue #10062: AQL: Could not extract custom attribute (#10068) 2019-09-24 14:36:53 +02:00
Michael Hackstein 265eb1549d
Feature/aql subquery opt shadow row interface (#9987)
* Added first draft of ShadowRow Interface

* Refactored RegisterPlan and pulled it out of the ExecutionNode

* AqlItemBlock now has an additional hidden register to store subquery depth infromation. Right now no shadow rows are created, however thy could now make use of this situation

* Added API and test to insert new shadow rows into an OutputRow.

* Extrated test helper function

* Added API and tests to consume ShadowRows. Interface is there and compiles with templates, we might need to implement further underlying functions later on as we make progress on ShadowRows, it is only implemented in the minimal way (intentionally)

* Added a test for nested ShadowRows and adapted OutputRow accordingly.

* Added additional memory include, seems to be required under GCC not udner CLANG

* it actually helps to save fies before commiting them...

* Fixed serialization/Deserialization of AqlItemBlock to contain the hidden subquery register now

* Added a c++ test for AqlItemBlock. It just covers the basics thus far and needs to be improved.

* Fixed toVPack of AqlInputRow for subqueries. Also added serialization / deserialization tests for AqlItemRow

* Added a c++ test for inputAqlItem row serialization => AqlItemBlock deserialization

* Attempt to fix :windows: compile warning

* Added a serialization format to be backwards compatible to 3.5 whenever AQL item blocks need to be send over to remote nodes.

* Added more tests for serialization of shadow rows, and fixed a bug revealed by this

* Apply suggestions from code review

Thank you for review!

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review comments. Thanks for spotting

* Fixed hidden merge conflicts

* Attempt to explain windows on how one can actually cast numbers into other number types...

* Update arangod/Aql/AqlItemBlockSerializationFormat.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>
2019-09-24 09:09:52 +02:00
jsteemann 8a812ec8c0 use StaticString 2019-09-23 18:17:37 +02:00
Jan Christoph Uhde 0b8c75c7b7 one shard db - devel (#9395) 2019-09-23 15:48:37 +02:00
Tobias Gödderz fbcb4b7152 Avoid overfetch in sorting gather executor (#10047)
* Avoid overfetch in sorting gather executor

* Updated CHANGELOG

* Temporarily disabled sort-limit in cluster when fullCount is enabled, until we can fix SortingGather

* Added a cluster test for sort-limit

* Fixed non-maintainer compile

* Fixed jslint errors

* Disabled sort profiler tests until SortingGather is fixed
2019-09-20 17:00:18 +02:00
Jan 41b0717bc9
remove unused code (#10043) 2019-09-19 12:25:25 +02:00
Tobias Gödderz 2f59a79435 Bug fix/sort limit rule too lax (#10014)
* Added regression test for consecutive constrained sorts

* Bugfix: sort-limit rule skipped too many node types

* Added CHANGELOG entry

* Apply sort-limit rule before single document operations
2019-09-18 09:04:43 +02:00
Jan 7e36a9ad9b
Bug fix/validate transaction collection (#9986) 2019-09-13 17:00:30 +02:00
Tobias Gödderz a97000cd04 Additional sort-limit tests (#10010) 2019-09-13 15:18:01 +02:00
Tobias Gödderz e113e36138 Make AQL's constrained heap play nice with fullCount (#9981) 2019-09-12 18:56:59 +02:00
Jan 2dde7e2307
Bug fix/allow not like (#9989) 2019-09-12 12:54:02 +02:00
Simon 3d2952b23a Non block modify (#9963) 2019-09-11 15:37:02 +02:00
Jan aada04e75b
don't assert/crash when using an unknown collection/shard (#9959) 2019-09-11 12:03:13 +02:00
Jan 4cb08d2a0d
fix wrong assertion in AstHelper.h (#9964) 2019-09-10 15:46:41 +02:00
Frank Celler a0852d33a2
fixed typo in define (#9945) 2019-09-10 11:34:42 +02:00
Jan 3a59abd1dc
various issues reported by cppcheck (#9962) 2019-09-09 20:32:04 +02:00
Jan 98fea326ea
handle K_SHORTEST_PATHS where it was forgotten (#9954) 2019-09-09 14:54:31 +02:00
Jan 35526deddf
improve handling of FILTERs that are known to always produce true or false (#9941) 2019-09-09 10:15:15 +02:00
Tobias Gödderz b7f8f01a22 Feature/split libarangoserver (#9670)
* Made usage of named variables for targets a little more consistent

* Bumped minimum cmake version

- removed policies superseded by minimum cmake version
- fixed static linking due to new policy CMP0060

* Fixed library suffixes for windows

* Extracted libs arango_mmfiles and _rocksdb from arangoserver

* Replaced target variables by constants

* Extracted arango_cluster_engine from arangoserver

* Extracted llhttp from arangoserver

* First successful split of arangoserver

* Moved enterprise files to enterprise

* Again only optionally include RestTestHandler and AcceptorUnixDomain

* Cleaned source files from other libraries

* Removed old commented sources

* Split off a small third library

* Fixed boost dependency for cluster engine

* Added some missing dependencies

* Added arango_geo dep, and -J on windows

* Began to split off an arango_graph lib

* Moved more files to arango_graph

* Do not set /J globally for ATL

* Moved more files to arango_graph

* Moved graph-RestHandlers to arango_graph

* Added arango_geo dependency to _mmfiles and _rocksdb

* Updated graph dependencies

* Split off arango_pregel

* Split off arango_aql

* Added missing boost_system dependency to pregel

* Split off arango_vocbase

* Cleanup

* Added missing boost_system dependency for arango_vocbase

* Split off arango_v8server

* Split off arango_utils

* Minor cleanup

* Split of arango_storage_engine

* Split off arango_indexes and arango_cache

* Fixed some dependencies

* Split off arango_replication

* Resolved two todos

* Split off arango_agency

* Reordered some statements

* Ordered dependency definitions alphabetically

* Cleaned some deps

* Break one cycle, comment on another

* Merge the remaining arangoserver_part[123] sources

* Moved some utils to vocbase to break cycles

* Added missing backtrace dependency to iresearch-s

* Added missing boost dependency

* Added dependency arango_indexes -> arango_geo

* Added deps to arango_cluster_engine, cleaned duplicate deps

* Broke remaining dependency cycles

* Actually, missed one cycle...

* Re-added include for Mac

* arango_cache needs SharedPRNG
2019-09-05 09:37:12 +02:00
Jan 5572675106
Bug fix/remove base directory from include path (#9885) 2019-09-04 17:39:01 +02:00
Dronplane 3bba2cd7db Fix for internal issue #633 (#9884)
* Made ANALYZER BOOST and MIN_MATCH non deterministic to prevent premature calls

* Returned Deterministic flag. Added context function implementation for cons arguments calls.

* Fixed filter volatility detection

* Added assertion

* Fixed formatting.  Removed redundant checks.

* Applied review suggestion

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-09-03 15:03:49 +03:00
Jan f1ca1d11ff
Bug fix/make graphs transaction aware (#9840) 2019-09-02 16:53:00 +02:00
Jan d42490aa42
honor return values of important methods (#9859) 2019-08-30 23:11:19 +02:00
Simon 9a43b28f8f Improve ExecContext usability (#9806) 2019-08-28 19:05:23 +02:00
Jan 86a496b8a3
Bug fix/fix in not in workaround (#9813) 2019-08-27 12:51:10 +02:00
Jan 00bcc4954c
AQL date functions improvements (#9714) 2019-08-22 12:50:08 +02:00
Jan ba02dc4b4d
mark AQL functions FULLTEXT, NEAR, WITHIN, WITHIN_RECTANGLE as cacheable (#9771) 2019-08-21 18:10:55 +02:00
Dronplane 49dfb0a96e Bug fix/issue #9612 (#9764)
* Fixed ViewExecutionNode retrieval with deleted documents present in view

* Ported solution from 3.4 branch

* Changed index store in collection from vector to set. To make reversable indexes always last to execute

* Fixed re-enter hung

* Index storage fix

* Made index order deterministic

* Fix Mac build

* Added tests for index reversal

* Fixed Mac build

* Code cleanup

* Some cleanup

* Removed some redundand copy constructor calls

* Applied review comments

* Applied review comments
2019-08-20 19:34:23 +03:00
Jan 6ffb6067fb
slightly reorder boolean members to reduce struct sizes (#9762) 2019-08-20 18:28:09 +02:00
Dan Larkin-York f7d41ad8fb Check scheduler queue return value (#9754) 2019-08-19 19:31:55 +02:00
jsteemann 2735114c59 Revert "Check scheduler queue return value. (#9701)"
This reverts commit 41c1b571b9.
2019-08-19 10:13:48 +02:00
Dan Larkin-York 41c1b571b9 Check scheduler queue return value. (#9701) 2019-08-19 10:01:08 +02:00
Simon 8af83d5bd4 Auxilliary changes from timeseries branch (#9699) 2019-08-15 10:12:58 +02:00
Simon 36ebf6f56f Add area calculations (#9693) 2019-08-14 10:08:44 +02:00
Matthew Von-Maszewski 77fb9df2c0
Bugfix: Correct rocksdb statistics to report column family sums. (#9705)
* Update rocksdb statistics to properly report sums from column families.  Add throttle statistic.

* add cstddef include to allow clean compile with gcc 6.3.0 20170516
2019-08-13 14:35:09 -04:00
Jan 0610e544fe
fixed issue #9679 (#9681) 2019-08-12 18:26:48 +02:00
Simon 9d125ef5cb Add support for WGS84 on distances (#9672) 2019-08-12 13:05:56 +02:00
Frank Celler aa3d3f8e40
Feature/cleanup ccpcheck (#9665) 2019-08-12 11:11:49 +02:00
Jan 5576e44547 miscellaneous adjustments to pacify cppcheck (#9677) 2019-08-09 14:29:21 +02:00
Jan b57224f85c
remove unused instance variable (#9662) 2019-08-08 14:35:20 +02:00