1
0
Fork 0
Commit Graph

83 Commits

Author SHA1 Message Date
Jan Christoph Uhde 484e2971d6 C++17 clean up (#10287) 2019-11-11 08:49:54 +01:00
Tobias Gödderz e2c84acfaf Use explicit default destructors where possible (#10166) 2019-10-04 15:58:30 +02:00
Jan 9533d1b71c
fix skipInaccessible (#10092) 2019-09-26 14:45:58 +02:00
Jan d42490aa42
honor return values of important methods (#9859) 2019-08-30 23:11:19 +02:00
Dan Larkin-York 3d0246cb18 Decentralize includes (#9623) 2019-08-06 15:32:09 +02:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan 77c37723d6
some very small optimizations (#8484) 2019-03-21 09:57:05 +01:00
Jan 9d3327c6ea
Bug fix/rearm cursors (#8363) 2019-03-12 15:28:33 +01:00
Jan 1798036ea0
Bug fix/optimizations 18022019 (#8180) 2019-02-19 19:24:04 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Jan 5719a7fc99
remove unneeded nullptr checks (#7121) 2018-10-29 11:51:27 +01:00
jsteemann cc21a938c7 fixed typos 2018-10-02 18:19:12 +02:00
Dan Larkin-York 1f63f16396 Move some logging off of general topic. 2018-10-01 13:28:11 -04:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Vasiliy 012aaa9469 issue 383.4: push vocbase validity check up from Query constructor out into arangodb::consensus::State, StatisticsWorker and AQLUserFunctions calls (#5177) 2018-04-23 14:52:42 +03:00
Simon 8be273efb8 Replication cleanup (#5105) 2018-04-17 08:17:42 +02:00
Michael Hackstein c1650702bf
Feature/aql server based locking (#4783)
* Started Implementing the ServerBasedlocking. There now is a container that can contain multiple query snippets. It now has to setup the necessary calls to the Servers

* Added backwards linking of QueryEngines, sth. DBServers can contact their Coordinators.

* Added LogTopic AQL

* Made AccessMode::Type Hashable

* Created a Mapping Server => LockLevel => Shard and createad a JSON object containing the Lock information for a complete AQL query per server

* Added code to build coordinator engines

* Finished with first draft of Coordinator-side of new DBServer based locking.

* Added a _api/aql/setup route that creates and locks all snippets/collections for one DBServer in a single go

* Fixed some Coordinator parts

* Index node now gracefully reports if it could not find it's collection when created from vpack. Otherwise it just hardly crashed...

* Modified the Coordinator Snippet collector to be able to handle subqueries properly.

* Started adding GraphNode handling. WIP. Need to deploy engines properly. Coordinator crashes on Graph tests

* Fixed compiler errors

* WIP: EngineInfoContainer

* Separated the EngineInfoContainers for Coordinator and DBServer into different files. They diverged more than anticipated

* Added forgotten files. THe DBServer container now creates the TraverserEngine Mapping and moves it into the Infos. They are not keeping it yet and need to add it to the message as well.

* The DBServer engine infos now persist the TraverserEngine infos. Need to add them to messages though.

* The new aql exec-engine now sends out traverserEngines as well

* Formatting and adding DEBUG level output

* Made the RestAQLHandler aware of the TraverserEngineRegistry. Also created the engines now. Return format changed server-side coordinator side needs fix.

* Adapted the Coordinator side for the DBServer based Shard Locking

* The DBServer based Locking now honors restrictions to certain shards

* Fixed a strange double lock bug in the new AQL Server based locking technique. Add some DEBUG output

* Fixed usage of MAINTAINERMODE macro. The assertion was never active

* Added TestCase for ContainerCoordinatorTest to cmake

* Added -DTEST_VIRTUAL to CMAKE. This is used to define virtual functions for mocking ONLY on test-builds.

* Fixed usage of ENABLE_MAINTAINER_MODE ifdef. CLANG format

* On non-enterprise builds ENTERPRISE_VERT defaults to TEST_VIRTUAL => virtual in test else non-virtual

* Added TEST_VIRTUAL to ExecutionEngine, Query and QueryRegistry

* Added first testcase for EngineInfoContainerCoordinator not yet ready.

* Mode CreateBlock a member function of engine, we have the engine in our hands anyways no need to make it static. Included some more TEST_VIRTUAL functions.

* Fixed clang/MacOs compile error. Added some more TEST_VIRTUAL declarations

* Finally fixed the first buildEngines UnitTest \o/

* Added a unit-test for backward linking of dependencies in CoordinatorPlanner

* Added multi-snippet test for EngineInfoContainerCoordinator

* Removed QueryRegistry.h from central header files and replaced by a forward declartion.

* Added a createBlocks method on the ExecutionEngine. It should be responsible to create all those blocks at once. Adapted the UnitTests as well. Not included Tests for the new createBlocks functionality. Need to mock the options feature first

* Added another test that Coordinator Snippets of queries can be created correctly

* Fixed Coordinator-site cleanup of QueryRegistry, if any of the query creations fails with error, incl UnitTest.

* Added first test for RestAqlHandler::setup. It does only test the setup and gives prepartion for real testing.

* Added a assertion of http return code. Still no creation of queries is tested. Requires a huge amount of mocking.

* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* Allow the clustercomm to send nolock headers on count. This is used form within AQL

* IsLocked on transactions will now always yield true IF LOCK_NEVER is set. We simply assume someone else holds the lock for us. Also LOCK_NEVER is now set on collection/count if noLock header is send.

* Moved the flag if collections need to be locked into the TraverserEngines.

* Added enterprise-satellite hooks in EngineInfoContainerDBServer

* Removed now obsolete code

* Replaced throwing of Exception by an ResultObject

* Added some more tests and moved adding snippet to query engine more to the outside.

* Added the AQL result type

* Make the branch compile again

* Register WITH collections for Graphs in the new Collector.

* Fixed test code for failing query clone. Idea was to once clone successfully and second time to fail, we verify that first clone is cleaned up properly. However test failed on first clone...

* Removed a double builderClose

* Added Changelog entry

* Removed empty if

* Removed obsolete todo

* Properly initialize the AqlResult with nullptr on error case

* Updated comment

* Simplified Assertion

* Removed debug output object...

* Added additional catch case for std::exception to get some more error info

* Clarified evaluation order for move case

* Added Explicit

* Fixed cleanup of Coordinator if Registry fails to insert query.

* Allow to use other locks than Read/Write for AQL collections. Not yet in API.

* Updated Comments for other Locks on DBSide. Adapted Destruction CatchTests

* Fixed double builderClose and removed unnecessary double commits

* Added a comment to clarify the state

* Moved error output to trace. Leftover from debugging

* Added some tests for complex subquery patterns

* Added a 'fireAndForgetRequests' methods to cluster comm which allows to send out a bunch of messages but does not wait for their results

* Properly cleanup leftovers of queries if the instantiation step already failed

* Added code comment for fireAndForget

* Added indexes to subquery test to make the plan a bit easier

* The cleanup on DBServerEngines in error case now also cleans up traverser engines.

* Removed unnecessary includes

* Removed debug logging

* Fixed hidden merge conflict
2018-03-20 16:52:19 +01:00
Jan 9c76613e63
fix premature unlock (#3802)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 13:27:42 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 4ba38bf981 try to work around some assertions (#3296)
* remove obsolete values from relative config

* warn about using obsolete config parameters
2017-09-28 09:21:33 +02:00
Simon Grätzer ffc465433a No access collections Improvements (#3190)
* consolidated EdgeDocumentToken

* optimizing cluster traversal

* adding skip collection checks

* API cleanup

* copying AQLValue to avoid use-after-free bugs

* Fixing rocksdb SingleServerEdgeCursor

* Fixing a collection resolving issue
2017-09-07 14:55:07 +02:00
Jan cd2dfdfd0d fix deadlocks in cluster traversals (#3198) 2017-09-04 17:35:24 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Simon Grätzer f3eb0c2ac0 No access collections (#3088)
* Added virtual attributes for enterprise on Methods.cpp

* Working no access collections

* align a comment

* Documentation and test fix

* fixing community build
2017-08-25 13:59:03 +02:00
Simon Grätzer 87de810566 Edge index refactoring 2017-06-02 11:20:15 +02:00
jsteemann b91eab0ce8 honor transaction options 2017-06-01 02:01:33 +02:00
Michael Hackstein 52406567cd Replaced EdgeCursors callback to return cid,rev instead of _id. Does less costly copying and comparision internally 2017-05-31 17:49:46 +02:00
jsteemann d24455c682 pass transaction options into AQL queries 2017-05-31 16:23:04 +02:00
jsteemann a4fde59fd2 some refactoring 2017-05-23 13:18:51 +02:00
Michael Hackstein 851f772dc1 Fixed some counting/reporting for AQL statistics in Traverser. 2017-04-24 10:42:40 +02:00
Michael Hackstein 4f899bfd2c Integreated a ShortestPathEngine on DBServers and wired it up with ShortestPathNode and Options. The path seems to be computed correctly in clsuter now, but the vertices are not yet retrieved. 2017-04-20 13:57:14 +02:00
jsteemann f08a118dfc Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-04-19 12:33:05 +02:00
jsteemann e949b39d27 slightly improve error message 2017-04-19 11:07:40 +02:00
Michael Hackstein c2be40b4ab Moved some Edge Lookup logic around and moved the EdgeCursor to it's own class. This is still ongoing work. Expect shortest-path to fail. 2017-04-11 18:03:56 +02:00
Jan Christoph Uhde b83ae2ab82 refactor some code to make use of arangodb::Result 2017-03-30 09:39:21 +02:00
Michael Hackstein 544007a5c8 Squashed commit of the following:
commit 873bc5950028248bf209d0c53fc9eb93938e3610
Merge: 8e4457a 3487097
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Mar 27 14:00:27 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 8e4457a96754bea3437e8d44c0ef9db2366ee86d
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Mar 27 13:59:44 2017 +0200

    Removed old VPackSlice based getVertex API in traversers.

commit 2e0716eeeec1ec56b96571a18074539f5e27cb40
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri Mar 24 09:09:02 2017 +0100

    Fixed logic error in SingleServerEdgeCursor readAll. The cursorId was not propageted properly, which cased filter-evaluations to fail

commit cea85c2ac1163ed0a989c492b8039ed4f7622325
Merge: 538f216 45a4fac
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri Mar 24 08:17:31 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 538f216cbefcdfb4d06fe16a42f223cecae8598b
Merge: 5f4afee 8bfcb49
Author: Simon Grätzer <simon@graetzer.org>
Date:   Thu Mar 23 16:33:44 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

commit 5f4afee54094ea6d2a0cfec1abfb75451f228a23
Author: Simon Grätzer <simon@graetzer.org>
Date:   Thu Mar 23 16:33:38 2017 +0100

    VertexGetter StringRef methods

commit 8bfcb490d7a907ebd7dcc9e6f38a8d29baa7c7e5
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 16:32:38 2017 +0100

    Fixed AqlValues created by traverser-cache. They are now not using externals anymore which cased everything to crash

commit 614cdedb7e30e35f65f289b310c38b2585eb03e4
Merge: 773b0a5 416777b
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 12:52:19 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache

    This state is still red

commit 773b0a5e51f87875fa9bf1440dd002b2e1fcfd2a
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 12:48:53 2017 +0100

    Added Implementation or readAll in Cluster EdgeCursor. This commit is still red.

commit 82ebbd6ec84918d43bbc64174bc251efa548a4de
Merge: e52b210 35dffc4
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 11:37:31 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache

    This commit is not green.

commit 416777b60a8a0c741d11e9c654745866d0dfa15d
Author: Simon Grätzer <simon@graetzer.org>
Date:   Thu Mar 23 11:25:35 2017 +0100

    Fixed errornous ClusterEdgeCursor implementation of `readAll

commit e52b21025a5f56e2cfc03e3c129da83793f19846
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Mar 23 11:01:17 2017 +0100

    Fixed API for Traversals: All functions now use StringRefs to identify vertices and edges which are presisted within the traverser-cache.

commit 35dffc439c156366b1110c6770c1565021934632
Author: Simon Grätzer <simon@graetzer.org>
Date:   Wed Mar 22 20:26:41 2017 +0100

    Fixed a few cluster issues

commit 73cf4fe41c363f6c034a1daeacee1630d7e827a5
Author: Simon Grätzer <simon@graetzer.org>
Date:   Wed Mar 22 17:26:12 2017 +0100

    Fixed EdgeCursor bug

commit 2f7329b4e9711740b423cf2963c6cf3f880c4bdf
Merge: bb9b129 8e5edf5
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 22 14:32:07 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 8e5edf5fbe93f34d3340099619baa9a56a738887
Author: Simon Grätzer <simon@graetzer.org>
Date:   Wed Mar 22 14:09:43 2017 +0100

    Fixing compile errors

commit bb9b1294c9ff91f62bc27775a4c7a69c5daca2fd
Merge: 048a4c6 8e51e3b
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 22 11:39:17 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 048a4c65e11d6fd9cb25a58ee97ed756a419a067
Merge: 693607e c6b177d
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 22 11:34:57 2017 +0100

    Solved merge conflicts. Build is still red.

commit c6b177dfc348df0f49f07cd67e83b873871fe6c5
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 23:04:28 2017 +0100

    Fixed off by one error, and crash

commit 5dbc9eb58f1d21e5a530806d7ad7e52058234b83
Merge: 463e352 db8c255
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 17:46:01 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

    # Conflicts:
    #	arangod/Graph/BreadthFirstEnumerator.cpp

commit 463e3520636c19ae7a7c1d445099f336c5bde31e
Merge: 4e75f29 b8cabbe
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 17:00:17 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

    # Conflicts:
    #	arangod/Graph/BreadthFirstEnumerator.cpp

commit 693607ea92ab3116a628c08c59a983a6fdf8b5f4
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 16:26:57 2017 +0100

    Moved specialized NeighborsEnumerator into its own files.

commit db8c25587ebcdeff8b32d08cf2d97d723bf2b3fe
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 15:16:39 2017 +0100

    Replaced SingleServerBreadthFirst with using Callbacks in readAll method of EdgeCursor

commit 4e75f29de95edeea35573b128a0a13e04c9e65d2
Merge: 8716936 6de86c5
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 14:32:12 2017 +0100

    Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache

commit b8cabbedebb16c09c8168ec8b5a4499e98c705f4
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 11:17:23 2017 +0100

    Removed _enumertedPath from BreadthFirstEnumerator. Less string copying expected now.

commit 8716936bd480f4b7c85ece6fc1a51335f1942258
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 11:05:04 2017 +0100

    Traverser: changed method definitions

commit 14af02fbc08b7e2ed6b0cbb918ff509ffaef76bf
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 10:45:13 2017 +0100

    Removed usage of _enumeratedPath in lastEdge/Vertex to AQL. Goal is to delete this struct in the class, as it does unneccessary copies

commit 06a20ab0549d97b47eda98d3428e3390619ca5ec
Author: Simon Grätzer <simon@graetzer.org>
Date:   Tue Mar 21 10:39:21 2017 +0100

    Intermediate changes

commit 3a9a5f7c7ad186c669c3986f6f89c5a26b835ee0
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 10:34:43 2017 +0100

    Logic of BreadthFirstEnumerator now points to the last path returned instead of the next path to return. This will allow to get rid of enumeratedPath in there.

commit 6de86c548220a3ad21c53fb1de691070774130ec
Author: Michael Hackstein <michael@arangodb.com>
Date:   Tue Mar 21 10:17:25 2017 +0100

    Extracted BreadthFirstEnumerator out of PathEnumerator files. Only moving of code.

commit acd1d9eba5d7ea23e0d6beab831a42dc61a385a5
Merge: a99ad3c 1ab8c44
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon Mar 20 13:13:16 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit a99ad3c624d7ae4815776f165d65f19e43167e56
Author: Simon Grätzer <simon@graetzer.org>
Date:   Fri Mar 17 13:25:45 2017 +0100

    Added TraverserCache to SingleServerTraverser

commit e88ae1e53d99b55546329413e7b4a5edc48c4875
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 15 13:36:05 2017 +0100

    Fixed compiler Issues in TraverserCache

commit 9d89cdc0297813f24064b46a67a43cde9a3d0297
Merge: 9d41804 f18ad19
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 15 09:43:06 2017 +0100

    Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache

commit 9d4180480c7bc37f1f64b6e3141dae611f8d399c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed Mar 15 09:42:53 2017 +0100

    Started implementing a Traverser Cache Abstraction that is used to store already fetched documents.
2017-03-27 14:30:06 +02:00
Max Neunhoeffer 29338899fd Finally port TraverserEngine locking bug fix manually. 2017-03-23 12:44:51 +01:00
jsteemann 4086c65e45 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-23 12:36:58 +01:00
jsteemann 90d81af098 fix compile error 2017-03-23 12:33:47 +01:00
Max Neunhoeffer 32953a4a97 Revert "Fix locking of shards in TraverserEngines."
This reverts commit 467088b8af.
2017-03-23 12:33:11 +01:00
Max Neunhoeffer 467088b8af Fix locking of shards in TraverserEngines. 2017-03-23 12:23:40 +01:00
jsteemann 666b2f8da9 renaming 2017-02-27 14:38:27 +01:00
jsteemann 94ce6a65f8 renamed methods 2017-02-27 11:22:44 +01:00
jsteemann 92f61416ca Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-02-13 15:25:17 +01:00
jsteemann 7e690a0c61 renaming 2017-02-13 15:18:40 +01:00
Michael Hackstein e0bdc5c122 Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-13 14:58:54 +01:00
jsteemann f09ec85127 renaming 2017-02-13 12:27:26 +01:00