1
0
Fork 0
Commit Graph

992 Commits

Author SHA1 Message Date
Jan aca44b9792
disable variable tracking for tests file on Linux (#5572) 2018-06-11 13:25:57 +02:00
Jan edb24deafe
cleanup collection/vocbase interface of ExecutionNodes (#5557) 2018-06-08 15:28:48 +02:00
Andrey Abramov e117a97fb4
Feature/arangosearch scatter node (#5536)
* remove collection dependency from scatter

* simplify `createBlock` API

* remove useless IResearchViewScatter node

* some fixes

* code cleanup

* scatter arangosearch view

* cleanup

* fix compilation errors

* fix test issues

* store shared_ptr rather than raw pointer in IResearchViewNode

* refactor ScatterBlock

* refactor DistributeBlock

* ensure no leaks during execution node cloning

* some fixes

* code cleanup

* remove vocbase from `DistributeNode`

* add missing ifdef

* fixes after review
2018-06-06 20:42:53 +03:00
Vasiliy 3e983adb2d issue 381.6: enforce use of VIEW keyword for bound views and disallow for bound collections (#5522) 2018-06-05 14:45:53 +03:00
Wilfried Goesgens 5aa46ee0cd pass argv[0] into the setup again (#5526) 2018-06-05 13:39:16 +02:00
Vasiliy 4253dca6aa issue 381.5: ensure the LogicalView definition that is persisted to the Agency matches the definition that gets created (#5518)
* issue 381.5: ensure the LogicalView definition that is persisted to the Agency matches the definition that gets created

* backport: correct comment
2018-06-02 17:21:55 +03:00
jsteemann 0f5fe129c9 fix compiler warnings 2018-06-01 22:21:07 +02:00
jsteemann 1368c7e2f2 fix memleaks in geo tests 2018-06-01 22:20:58 +02:00
jsteemann 1283e0f629 fix compile warnings 2018-06-01 18:48:50 +02:00
Wilfried Goesgens 1172512622 Add ability to turn on line number logging to the catch tests (#5448)
- if the user has enabled extremeVerbosity on the unittest,
   line number logging will be turned on as well.
2018-06-01 18:11:33 +02:00
Wilfried Goesgens 230ac322ea fresh catch, more flags. (#5452) 2018-06-01 18:07:25 +02:00
Vasiliy ac1a6742e0 issue 381.4.1: create a separate IResearchLink implementation for use with ClusterEngine (#5505)
* issue 381.4.1: create a separate IResearchLink implementation for use with ClusterEngine

* backport: ensure StorageEngineMock assigns unique transaction ids

* delegate index normalization on the ClusterEngine to the 'actualEngine'
2018-06-01 19:05:53 +03:00
Jan cee529727c
Squashed commit of the following: (#4092) 2018-05-26 10:56:26 +02:00
Andrey Abramov 0f0235e8a9
Remove collection dependency from gather node (#5457)
* remove collection dependency

* some fixes

* fix test issues

* add tests for newly created functionality

* fix enterprise test fails

* add comment
2018-05-25 17:26:33 +03:00
Vasiliy e021085aac issue 381.3.1: remove redundant copy of CollectionNameResolver from TransactionState (#5459) 2018-05-25 16:48:43 +03:00
Jan 8e6d5df129
fixed minor several compiler complaints (#5406) 2018-05-23 11:50:00 +02:00
Jan 30d0d840c2
Bug fix/validate from to consistently (#5434) 2018-05-23 11:41:30 +02:00
Vasiliy 94ddd7803d issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424) 2018-05-23 00:59:08 +03:00
Andrey Abramov 4b47400af5
disable flushthread on a coordinator (#5421)
* disable flushthread on a coordinator

* some fixes after review
2018-05-22 21:12:25 +03:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
Vasiliy 370bde71d9 issue 389.7: expand workaround to include MSVC2015 debug code, backport from iresearch upstream: skip importing empty readers (#5391)
* issue 389.7: expand workaround to include MSVC2015 debug code, backport from iresearch upstream: skip importing empty readers

* backport: address MSVC V8/ICU link issues

* backport: fix typo
2018-05-20 22:43:37 +03:00
Jan 72ec463a8d
Bug fix/cleanup 2005 (#5404) 2018-05-20 20:57:43 +02:00
Wilfried Goesgens 37bd883507 disable this test for now. (#5393) 2018-05-18 10:56:37 +02:00
Vasiliy 907767f4e1 issue 389.6: ensure proper error handling in StatisticsFeature for missing feature (#5377) 2018-05-18 01:12:39 +03:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Vasiliy 5a66661f78 issue 389.4: implement IResearchViewDBServer::snapshot (#5358)
* issue 389.4: implement IResearchViewDBServer::snapshot
* backport: revert back namespace header changes
* backport: address review comments
2018-05-16 16:33:48 +02:00
Max Neunhöffer c1e35f7bb1
Make sure VPackObjectBuilder is closed at exit of function scope. (#5347) 2018-05-15 12:00:41 +02:00
Vasiliy b13a8da1ce issue 389.1: ensure GeneralClientConnection socket is valid for MSVC (#5328) 2018-05-14 14:55:51 +03:00
Jan fe586dac40
retire some old string functions, and make encode/decodeHex copy less data around (#5313) 2018-05-14 10:06:45 +02:00
Andrey Abramov 671734b32f
do not scatter empty view (#5318)
* do not distribute scatter views without links

* simplify view downcasting

* refactor & add some tests

* adjust comments
2018-05-11 11:45:22 +03:00
Vasiliy d48bdf72e1 issue 373.4: address issue with MSVC picking incorrect function, minor formatting changes, backport from iresearch upstream: ensure id_to_column contains enough slots for all column ids, address another issue with _T(...) macro not working when _MBCS is defined on MSVC (#5310) 2018-05-10 20:12:37 +03:00
Heiko e01603a4b1 fixed geo tests asan (#5296) 2018-05-09 09:20:23 +02:00
Vasiliy 5d8abce7bb issue 373.2: move toVelocyPack into LogicalDataSource (#5288)
* issue 373.2: move toVelocyPack into LogicalDataSource

* backport: move static DataSource related strings into StaticStrings, add support for registering snapshots on DBServer views
2018-05-08 17:28:39 +03:00
jsteemann 03acfe7cf5 Merge branch 'bug-fix/remove-jemalloc-feature' of https://github.com/arangodb/arangodb into devel 2018-05-08 08:31:56 +02:00
jsteemann 7f8a1cc614 Merge branch 'bug-fix/add-missing-overrides-and-final' of https://github.com/arangodb/arangodb into devel 2018-05-07 23:02:46 +02:00
Vasiliy 2ce20e86d7 issue 373.1: move globally-unique id generation from collection into data-source (#5182) 2018-05-07 22:14:40 +03:00
Tobias Gödderz 8c87f51429 Feature/fix inconsistent distribute shards like job (#4743) 2018-05-07 16:53:08 +02:00
Kaveh Vahedipour 012718d6b5 Bug fix/default stack size on osx too small (#5265) 2018-05-07 16:50:51 +02:00
Simon fdee0544b7 Using asio::io_context::strands instead of locks (#5266)
* initial try adding strands

* working, stable amount of threads

* improve shell_client cluster

* Fixing some accounting for the scheduler

* Fix accounting

* Fixing wrong strand usage

* add missing return

* Fixing thread accounting

* More scheduler accounting issues

* Fixing various things

* Fixing some stuff

* Fixing some stuff

* Some more subtle bugfixes

* Some cleanup code

* fixing some stuff

* adding some more fixes

* Fixing possible issues

* Fixing missing _storeResult

* Fixing some stuff

* Reducing lambda stack, perhaps fixing hangups

* Fix writeunlocker

* Fixing possible issues

* adding some debugging stuff

* refactor sockets

* possible fixes

* Adding more job guards

* Fixin possible bug

* cleaning up some stuff

* working impl

* Remove debugging output

* Fixing build

* fixing import

* Fixing another bug

* removing debug log

* Removing examples

* Reverting scheduler working code

* Cleanup

* Addressing review comments
2018-05-07 15:58:19 +02:00
Andrey Abramov b69b5bdfdf
Bug fix/issue #5186 (#5269)
* do not persist view on startup

* small refactoring

* ensure view is being opened after creation
2018-05-06 20:38:32 +03:00
jsteemann 52de92d334 add missing override specifiers, add final specifiers 2018-05-04 09:01:50 +02:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Jan 349e2c1681 forward-port AQL optimizations from 3.3 (#5020)
* forward-port AQL optimizations from 3.3

* fix restrict-to-single-shard rule

* added tests for specific cases

* ModificationNodes now can know if they are restricted or not

* OptimizerRule now updates the ModificationNode instead of setting ownName on remote node.

* The DBServerEngine container now only sends a subset of snippets (those which are relevant) It is  able to restrict any part of the query to a certain shard or even a list of shards

* ExecutionEngine CreateBlocks in ClusterMode now returns Result instead of throwing.

* Adapted Coordinator side to a bitter mapping of RemoteNode => QuerySnippets. This is still ongoing work.

* Fixed Coordinator enginecontainer test and finisehd implementation of EngineInfoContainerDBServer. Seems to work so far, need to adapt test to prove it.

* Adapted test for modification nodes and restrict to shard.

* restrictTo can now be de-/serialized on Modification Nodes

* Fixed Community Compile Error

* Added a test for Indexes/EnumerateCollection nodes

* EnumerateCollection and IndexNode now have options to be restricted to a single shard, opt rule not yet updated

* DBServer part of planning now checks Enumerate and IndexNodes for single shard optimization

* Added tests for simple single shard optimization on IndexNodes

* Adapted optimzierrule for SingleShard optimierung on IndexNodes

* Fixed an issue with externally restricted shard ids in combination with shard_id based optimization

* Added a test for multiple single shard optimizations on the same collection

* Removed collection/Shard duplicate check of optimize to single shard rules

* Updated Explainer to handle optimize to single shard

* Let more tests pass an optimized AQL variant

* Temporarily disabled a tests that can only be solved after some work in transaction
2018-04-30 11:25:09 +02:00
Jan 30b12e311b
Bug fix/remove most of aql js (#5223) 2018-04-30 11:17:11 +02:00
Wilfried Goesgens 357ec8ebef prohibit the use of sleep() in the source (#5178) 2018-04-30 10:23:22 +02:00
Wilfried Goesgens 7d6e580780 Refactoring & code cleanup (#5138) (#5142) 2018-04-24 14:42:23 +02:00
Vasiliy 012aaa9469 issue 383.4: push vocbase validity check up from Query constructor out into arangodb::consensus::State, StatisticsWorker and AQLUserFunctions calls (#5177) 2018-04-23 14:52:42 +03:00
Simon 45fbed497b Supervision Job for Active Failover (#5066) 2018-04-23 12:49:41 +02:00
Vasiliy 9062c41592 issue 383.3: implement remainder of IResearchViewDBServer tests, use the data-source id (primary key) instead of an arbitrary instance for dropCollection()/dropView(), backport from iresearch upstream: ensure block is flushed if key index is full (#5176) 2018-04-23 00:33:46 +03:00
jsteemann 88a8686654 remove ApplicationFeature "JemallocFeature" 2018-04-21 00:56:10 +02:00
Vasiliy da4519df43 issue 383.2: backport from iresearch upstream: address issue with _T(…) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC (#5152)
* issue 383.2: backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC

* backport: add more tests for IResearchViewDBServer
2018-04-20 18:20:31 +03:00
Vasiliy 6053056b6a issue 383.1: initial stub implementation of IResearchViewDBServer (#5132) 2018-04-18 13:51:21 +03:00
Wilfried Goesgens a6ddeb3782 Bugfix/fix windows warnings (#5121)
* fix windows warnings
* don't use <date/datetime.h> in global scope so it doesn't interfere with iresearch
2018-04-17 15:44:32 +02:00
Wilfried Goesgens d18e760150 Bugfix/fix windows warnings (#5116)
* fix windows warnings
2018-04-17 12:38:59 +02:00
Vasiliy 9a4ddee13e issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally (#5107)
* issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally

* remove invalid include
2018-04-16 18:30:51 +03:00
Simon 7677afabf1 Remove copy of request body in rest handlers (#5104) 2018-04-16 14:49:51 +02:00
Vasiliy d470371f7c issue 374.2: use a reference to vocbase instead of a pointer in StorageEngime LogicalDatasource related DDL operations, use vocbase from the found collection 2018-04-11 18:49:46 +03:00
Andrey Abramov 5dc01a3f34
Merge pull request #5059 from arangodb/bug-fix/internal-issue-#355.7
issue 355.7: ensure LogicalDataSource::vocbase() returns a reference
2018-04-10 13:34:54 +03:00
Simon 68442dae5a Fixing agency prefix in Agency/Job.cpp (#5039)
* Fixing some test issues and fixing the agency prefix in Agency/Job.cpp
* Making logic consistent in  failed- leader / follower job
* reverting condition back to == GOOD
2018-04-09 16:21:24 +02:00
Vasiliy d1ce3a97ef issue 355.7: ensure LogicalDataSource::vocbase() returns a reference 2018-04-09 15:38:24 +03:00
Vasiliy e4368b0991 issue 355.6: remove create() from LogicalView, remove IResearch dependency from IndexFactory, store vocbase reference in LogicalDataSource 2018-04-06 16:38:34 +03:00
Simon 3ec8bcfd97
Merge branch 'devel' into bug-fix/internal-issue-#355.5 2018-04-05 17:34:55 +02:00
Simon 9980067e73 fixing various warnings (#5013) 2018-04-05 17:01:16 +02:00
Vasiliy 99b83ba8c8 issue 355.5: remove more unused methods, move view-related storage engine functionality from vocbase into DBServerLogicalView, address MSVC cmake dependency issue 2018-04-05 16:17:07 +03:00
Vasiliy 635db3b409 issue 355.4: remove redundant methods and code, use 'cp' instead of 'cmake copy_directory' where possible, use vocbase reference instead of pointer 2018-04-04 10:53:48 +03:00
Vasiliy 7c25902b27 issue 355.3: allow IResearchLink creation, as opposed to IResearch View update, to direct which CIDs get included in an IResearchView snapshot 2018-04-02 15:27:48 +03:00
Andrey Abramov eaa91592e5
Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/remove-view-implementation 2018-03-27 14:58:55 +03:00
Andrey Abramov a7760cb3b7 fix tests 2018-03-27 14:44:10 +03:00
Andrey Abramov b641cf9b13
remove view implementation part 1 2018-03-26 23:18:08 +03:00
Dan Larkin-York 38f162e344 Parallelize arangodump (#4356) 2018-03-26 15:55:14 +02:00
Andrey Abramov a164b25039
Merge pull request #4942 from arangodb/feature/refactor-logical-view
refactor logical view
2018-03-26 12:49:43 +03:00
Tobias Gödderz 055e3f4b3d Split large string literal in parts <16k for MSVC++ (#4940) 2018-03-26 10:09:24 +02:00
Andrey Abramov f2703786f1
Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/refactor-logical-view 2018-03-23 15:17:17 +03:00
Andrey Abramov 6b9bffa343 remove PhysicalView part 3 2018-03-23 15:17:05 +03:00
Michael Hackstein 2082ea134e
Deactivated tests that mocked destructors. Does not seem to work with any optimization build and made jenkins red (#4935) 2018-03-23 11:03:23 +01:00
Tobias Gödderz 7e53d3ed75 Bugfix / Supervision: removeFollower should remove the last follower(s) first (#4923)
* Added a test asserting the last followers are removed first as required by moveShard
* Remove the last followers first
* Removed unused includes
* Updated CHANGELOG
2018-03-23 09:34:04 +01:00
Andrey Abramov 47fd5808c3 remove PhysicalView part 2 2018-03-22 22:41:28 +03:00
Andrey Abramov a874fda823 remove PhysicalView part 1 2018-03-22 20:42:42 +03:00
Manuel B 215777736e Move AQL Date Functions => CPP 2018-03-22 15:22:24 +01:00
Andrey Abramov 41eb649556
Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/internal-issue-#345 2018-03-21 21:02:17 +03:00
Max Neunhöffer b8ca34c746
Run catch tests in subthread to get larger stack size in libmusl. (#4913)
* Run catch tests in subthread to get larger stack size in libmusl.
2018-03-21 18:01:01 +01:00
Andrey Abramov 15304b705b
Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/internal-issue-#345 2018-03-21 14:37:07 +03:00
Michael Hackstein f9c2d7a0e6
Fixed catch test (#4910) 2018-03-21 10:19:03 +01:00
Jan 5105e2b3c0
Remove "atLeast" from the internal query APIs (#4848) 2018-03-21 09:10:28 +01:00
Michael Hackstein bb17ae172f
Removed fakeit namespace. Kills windows (#4909) 2018-03-21 09:02:05 +01:00
Andrey Abramov 04bb3da337
Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/internal-issue-#345 2018-03-20 19:04:54 +03:00
Michael Hackstein c1650702bf
Feature/aql server based locking (#4783)
* Started Implementing the ServerBasedlocking. There now is a container that can contain multiple query snippets. It now has to setup the necessary calls to the Servers

* Added backwards linking of QueryEngines, sth. DBServers can contact their Coordinators.

* Added LogTopic AQL

* Made AccessMode::Type Hashable

* Created a Mapping Server => LockLevel => Shard and createad a JSON object containing the Lock information for a complete AQL query per server

* Added code to build coordinator engines

* Finished with first draft of Coordinator-side of new DBServer based locking.

* Added a _api/aql/setup route that creates and locks all snippets/collections for one DBServer in a single go

* Fixed some Coordinator parts

* Index node now gracefully reports if it could not find it's collection when created from vpack. Otherwise it just hardly crashed...

* Modified the Coordinator Snippet collector to be able to handle subqueries properly.

* Started adding GraphNode handling. WIP. Need to deploy engines properly. Coordinator crashes on Graph tests

* Fixed compiler errors

* WIP: EngineInfoContainer

* Separated the EngineInfoContainers for Coordinator and DBServer into different files. They diverged more than anticipated

* Added forgotten files. THe DBServer container now creates the TraverserEngine Mapping and moves it into the Infos. They are not keeping it yet and need to add it to the message as well.

* The DBServer engine infos now persist the TraverserEngine infos. Need to add them to messages though.

* The new aql exec-engine now sends out traverserEngines as well

* Formatting and adding DEBUG level output

* Made the RestAQLHandler aware of the TraverserEngineRegistry. Also created the engines now. Return format changed server-side coordinator side needs fix.

* Adapted the Coordinator side for the DBServer based Shard Locking

* The DBServer based Locking now honors restrictions to certain shards

* Fixed a strange double lock bug in the new AQL Server based locking technique. Add some DEBUG output

* Fixed usage of MAINTAINERMODE macro. The assertion was never active

* Added TestCase for ContainerCoordinatorTest to cmake

* Added -DTEST_VIRTUAL to CMAKE. This is used to define virtual functions for mocking ONLY on test-builds.

* Fixed usage of ENABLE_MAINTAINER_MODE ifdef. CLANG format

* On non-enterprise builds ENTERPRISE_VERT defaults to TEST_VIRTUAL => virtual in test else non-virtual

* Added TEST_VIRTUAL to ExecutionEngine, Query and QueryRegistry

* Added first testcase for EngineInfoContainerCoordinator not yet ready.

* Mode CreateBlock a member function of engine, we have the engine in our hands anyways no need to make it static. Included some more TEST_VIRTUAL functions.

* Fixed clang/MacOs compile error. Added some more TEST_VIRTUAL declarations

* Finally fixed the first buildEngines UnitTest \o/

* Added a unit-test for backward linking of dependencies in CoordinatorPlanner

* Added multi-snippet test for EngineInfoContainerCoordinator

* Removed QueryRegistry.h from central header files and replaced by a forward declartion.

* Added a createBlocks method on the ExecutionEngine. It should be responsible to create all those blocks at once. Adapted the UnitTests as well. Not included Tests for the new createBlocks functionality. Need to mock the options feature first

* Added another test that Coordinator Snippets of queries can be created correctly

* Fixed Coordinator-site cleanup of QueryRegistry, if any of the query creations fails with error, incl UnitTest.

* Added first test for RestAqlHandler::setup. It does only test the setup and gives prepartion for real testing.

* Added a assertion of http return code. Still no creation of queries is tested. Requires a huge amount of mocking.

* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* Allow the clustercomm to send nolock headers on count. This is used form within AQL

* IsLocked on transactions will now always yield true IF LOCK_NEVER is set. We simply assume someone else holds the lock for us. Also LOCK_NEVER is now set on collection/count if noLock header is send.

* Moved the flag if collections need to be locked into the TraverserEngines.

* Added enterprise-satellite hooks in EngineInfoContainerDBServer

* Removed now obsolete code

* Replaced throwing of Exception by an ResultObject

* Added some more tests and moved adding snippet to query engine more to the outside.

* Added the AQL result type

* Make the branch compile again

* Register WITH collections for Graphs in the new Collector.

* Fixed test code for failing query clone. Idea was to once clone successfully and second time to fail, we verify that first clone is cleaned up properly. However test failed on first clone...

* Removed a double builderClose

* Added Changelog entry

* Removed empty if

* Removed obsolete todo

* Properly initialize the AqlResult with nullptr on error case

* Updated comment

* Simplified Assertion

* Removed debug output object...

* Added additional catch case for std::exception to get some more error info

* Clarified evaluation order for move case

* Added Explicit

* Fixed cleanup of Coordinator if Registry fails to insert query.

* Allow to use other locks than Read/Write for AQL collections. Not yet in API.

* Updated Comments for other Locks on DBSide. Adapted Destruction CatchTests

* Fixed double builderClose and removed unnecessary double commits

* Added a comment to clarify the state

* Moved error output to trace. Leftover from debugging

* Added some tests for complex subquery patterns

* Added a 'fireAndForgetRequests' methods to cluster comm which allows to send out a bunch of messages but does not wait for their results

* Properly cleanup leftovers of queries if the instantiation step already failed

* Added code comment for fireAndForget

* Added indexes to subquery test to make the plan a bit easier

* The cleanup on DBServerEngines in error case now also cleans up traverser engines.

* Removed unnecessary includes

* Removed debug logging

* Fixed hidden merge conflict
2018-03-20 16:52:19 +01:00
Vasiliy a553410296 backport: explicitly set the database path in test setup 2018-03-20 17:54:49 +03:00
Vasiliy 399c1ce2f0 manually-merge: add more tests, backport some miscellaneous fixes 2018-03-20 16:40:35 +03:00
Andrey Abramov 3a30c85e49
Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/internal-issue-#345 2018-03-20 15:59:44 +03:00
Max Neunhöffer 34fb55d024
Use a temporary directory for the DatabasePathFeature in catch tests. (#4896)
* Use a temporary directory for the DatabasePathFeature in catch tests.
2018-03-20 13:57:52 +01:00
Andrey Abramov 0b37cd1796 fix some issues, add more tests 2018-03-16 17:40:34 +03:00
Vasiliy 06eb8ade01 issue 344.7: remove more redundant functions (#4863)
* issue 344.7: remove more redundant functions

* backport: fix missed functions under USE_ENTERPRISE
2018-03-15 17:10:28 +01:00
Vasiliy c8739cd3cd manually-merge: cache data-sources in CollectionNameResolver 2018-03-14 10:11:50 +03:00
Andrey Abramov 90e0a9326b
remove ArangoSearch dependencies from StorageEngineMock 2018-03-12 13:34:57 +03:00
Wilfried Goesgens 42d32e08de Bug fix/fix mac wincompile (#4799) 2018-03-09 15:51:54 +01:00
Vasiliy bf0068e7f5 Changes since last PR: use a single map in vocbase for storing LogicalDataSource instances 2018-03-08 17:28:08 +03:00
Vasiliy daa1db659c manually-merge: differentiate data-source types by category e.g. LogicalCollection/LogicalView 2018-03-08 13:40:25 +03:00
Simon 102998dd65 Streaming Query Cursor (#4769) 2018-03-08 09:18:41 +01:00
Vasiliy f9a45032cd LogicalDataSource for Methods::StateRegistrationCallback instead of TRI_voc_cid_t to avoid unnecessary lookups (#4732)
* Changes since last PR: use LogicalDataSource for Methods::StateRegistrationCallback instead of TRI_voc_cid_t to avoid unnecessary lookups
* backport: address cluster LogicalDataSource resolution failure
2018-03-06 11:41:41 +01:00
jsteemann 37e2ef461a fix tests 2018-03-05 17:30:46 +01:00
Andrey Abramov 9f84c95f0a Execution block mock (#4710) 2018-03-05 16:24:15 +01:00
Vasiliy a988cb19d7 Changes since last PR: create a common ancestor to LogicalCollection/LogicalView (#4719) 2018-03-02 09:02:09 +01:00
Vasiliy f4dbfdbdbd Changes since last PR: partially revert commit bd86eea365737300a4e0e8d23938a88d54cfcbf4 to address test warnings, partially revert commit a1cfb3d72b to address test warnings, remove unused function (#4699) 2018-03-01 09:29:11 +01:00
Simon 345fc3c0b7 Refactor Authentication Layer (devel) (#4592)
* Cherry Picking LDAP changes

* Adding missing merges

* Fixing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

(cherry picked from commit 686d28a779)

* Fixing wrong assert

* Adding changelog entry, making requested changes from code review

* Fixing dump_authentication, fix typos

* improvements found during code review

* oops

* more use of sessionstorage

* fix tests

* Fixing broken handling, disallowing adding of local users when disabled

* Fixing testInvalidGrants

* Removing undefined auth level externally

* Fixing previous commit

* added tests for ldap search mode

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* forward port changes from 3.3

* added generated files

* forward port missing changes for web UI

* added generated files

* added generated files
2018-02-28 13:24:28 +01:00
Vasiliy 423ce33ed1 Changes since last PR: remove 'dataPath' IResearchView configuration property and use a hardcoded path similar to a LogicalCollection (#4683) 2018-02-27 18:10:32 +01:00
Jan 3cf03cfb21
Feature/cleanup internal (#4680) 2018-02-27 14:53:31 +01:00
Simon cde96d97aa RocksDB replication hardening & cleanup (#4639) 2018-02-23 18:15:04 +01:00
Andrey Abramov 1b64f03bf8 Bug fix/internal issue #327 (#4649) 2018-02-22 11:11:23 +01:00
Andrey Abramov d130e0f7c6 Bug fix/internal issue #350 (#4601) 2018-02-15 18:57:08 +01:00
Vasiliy 828f8fb48c locally tested only, Changes since last PR: ensure parser.cc dependency is tracked in the build directory to avoid bison invocation on MSVC, aquire IResearchView snapshot at transaction begin() and reuse it for all queries, remove unused callback registration from Methods (#4564) 2018-02-14 12:32:29 +01:00
Andrey Abramov 1b7dd672c9 Bug fix/internal issue #342: allow to bind a view name (#4554) 2018-02-13 21:53:01 +01:00
Dan Larkin 934e936bae Fix index selectivity estimate issues with RocksDB engine (#4429) 2018-02-09 16:05:33 +01:00
Kaveh Vahedipour 7f9786eb27 builder fixed for agency transaction. worked only for a single server. (#4436) 2018-02-06 23:14:53 +01:00
Vasiliy aa041ddfb9 locally tested only (#4476) 2018-02-05 11:24:34 +01:00
Jan 3889494c9c
removed obsolete comment (#4431) 2018-01-25 15:57:45 +01:00
Jan 64ad854e67
fix exit code for catch tests (#4427) 2018-01-24 18:10:09 +01:00
Jan 5cb29ee198
fix several memleaks in catch tests (#4409) 2018-01-24 17:26:49 +01:00
Andrey Abramov 373a5d1f4c fix a couple of issues (#4389) 2018-01-22 17:48:26 +01:00
Andrey Abramov ffbb00bbe2 standardize on the 'analyzer' and 'trackListPositions' naming (#4384) 2018-01-22 16:39:53 +01:00
Andrey Abramov a1cfb3d72b Feature iresearch (#4105) 2018-01-19 14:23:58 +01:00
jsteemann f7e34b2d7c fix crashes 2018-01-08 17:02:34 +01:00
jsteemann dee060e09a fix IResearch tests compilation 2018-01-05 19:28:39 +01:00
Jan b2b6c06cbf
Feature/efficiency (#3736) 2018-01-05 16:51:31 +01:00
Heiko 78213b3b4a fixed some macos ires related warnings and errors (#4196) 2018-01-05 14:51:13 +01:00
Heiko 61de1b6099 Bug fix/optimize shard distribution api and ui (#3921)
* UI: document/edge editor now remembering their modes (e.g. code or tree)

* changed shardDistribution api behaviour, added PUT route to only fetch collection based shard distribution

* ui: optimized shards view, added missing cleanup function in nodes view

* broken test

* shard distribution tests not fit the new api behaviour

* variables as reference

* CHANGELOG
2018-01-02 12:42:12 +01:00
jsteemann db4724811d test behaves differently when invoked via scripts/unittest or when invoked via the tests binary 2017-12-23 02:10:17 +01:00
Jan 2744d52691
added tests for file descriptors (#4130)
* added tests for file descriptors

* properly clean up generated test files
2017-12-22 11:00:00 +01:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Jan 4ce91ac16d
fix platform-dependent behavior of TRI_CopyFile (#3988) 2017-12-11 17:51:56 +01:00
Jan 5a4613c0bf
try to make the tests cope with async execution (#3982) 2017-12-11 11:28:51 +01:00
Dan Larkin 5e3b5526d0 Fixed return value of cache rebalancer in a few spots. (#3967) 2017-12-08 16:01:54 +01:00
Jan 282be208cc
remove TRI_usleep and TRI_sleep, and use std::this_thread::sleep_for … (#3817) 2017-12-06 18:43:49 +01:00
Jan 29ca87bb0c
fix an unnecessary downcast (#3774) 2017-12-06 16:36:10 +01:00
Jan 514374440c
attempt to reactivate test for ClusterComm::wait (#3785) 2017-11-27 23:07:48 +01:00
Michael Hackstein 1bac6dd414 Replaced usleep by this_thread::sleep_for as usleep can return to early letting the test fail (#3800) 2017-11-24 13:54:28 +01:00
Kaveh Vahedipour 2195ca8e35 ClusterComm tests commented out. Tests unreliable under load and also without load in OSX and Windows (#3739) 2017-11-18 16:35:55 +01:00
Matthew Von-Maszewski 0558b4372d Improve ClusterComm::wait() (#3722) 2017-11-17 22:46:54 +01:00
m0ppers 75e1bf31cd Feature/allow access to configured 🔐 in readonly mode (#3698)
* Special method to access configured access levels as opposed to only getting the effective access level
* Add integration test
* Allow grants on collection level
2017-11-17 16:11:22 +01:00
Jan Christoph Uhde 5eec20b021 delete using namesapace fakeit to resolve DWORD issue on Windows (#3694)
* delete using namesapace fakit to resolve DWORD issue on Windows

* Fixed a second catch test with the same error.
2017-11-15 10:16:26 +01:00
Kaveh Vahedipour c9621ff230 Feature/new agency checks for preconditions (#3612) 2017-11-11 22:48:23 +01:00
Michael Hackstein 5c633f9fae Bug fix/speedup shard distribution (#3645)
* Added a more sophisticated test for shardDistribution format

* Updated shard distribution test to use request instead of download

* Added a cxx reporter for the shard distribuation. WIP

* Added some virtual functions/classes for Mocking

* Added a unittest for the new CXX ShardDistribution Reporter.

* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red

* Shard distribution now uses the cxx variant

* The ShardDistribution reporter now tries to execute count on the shards

* Updated changelog

* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
2017-11-10 15:17:08 +01:00
Michael Hackstein 2b0aa318ec Bug fix/sparse graph register warning (#3569) 2017-11-10 14:33:15 +01:00
Jan 151c717d22 fix potential duplicate closing of typed buffer (#3587) 2017-11-07 10:27:51 +01:00
m0ppers bb1d303473 Cmake 5.0 complains about unused lambda captures (#3390) 2017-10-13 12:20:48 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Jan 2525a3a1bd Feature/mmfiles hash lookup performance (#3265)
* speed up full collection scans in mmfiles engine

* some API cleanup
2017-09-26 14:48:06 +02:00
m0ppers f0ae3d3de7 Bug fix/uaf communicator request abort (#3216)
* Fix abortion of requests

* Fix progress callback
2017-09-15 22:42:30 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Dan Larkin a775e37c76 Feature/unique vpack index lookup performance improvements with rocksdbkey optimizations (#3123)
* fixed issue #3075

* update the known issues document, as some issues have already been fixed

* speed up equality lookups in unique vpack indexes

* Allow for re-use of RocksDBKey objects through transaction leaser.

* Converted all RocksDBKey variants to non-static constructors.
2017-08-31 20:00:05 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Dan Larkin e2fdab431c Fix compiler warnings and needlessly failing catch tests. (#3171) 2017-08-31 17:27:25 +02:00
Kaveh Vahedipour e808867ddc Bug fix/unordered map changes order in catch tests (#3175)
* order of free and free2 changed with use of unordered_multimap vs multimap

* fixing order independance for maps in catch tests fixed?

* missing bits and pieces
2017-08-31 15:58:48 +02:00
Kaveh Vahedipour 94e9744208 order of free and free2 changed with use of unordered_multimap vs multimap (#3168) 2017-08-30 11:39:08 +02:00
Dan Larkin 18e8c19271 Improvements to CachedValue internals and handling. (#3126)
* Added checks everywhere for CachedValue::construct.

* Enforce 16MiB limit on keys and 4GiB limit on values.

* Protected members behind function calls in anticipation of format change.

* Shrunk CachedValue struct.

* Fixed alignment of CachedValues.

* Fixed improper comparison.
2017-08-28 15:07:16 +02:00
m0ppers 0dfea58d54 Feature/remove manual zippery (#3036) 2017-08-17 23:30:27 +02:00
Jan ec5dde49ec remove memset parameter from TRI_Allocate signature (#2993) 2017-08-08 16:07:32 +02:00
Frank Celler fa9246c9f9 fixed locator for icudtl.dat (#2950) 2017-08-04 09:43:49 +02:00
Jan 871ee6a85f added startsBefore() for ApplicationFeature (#2913) 2017-08-01 17:15:25 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713)
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
2017-07-05 10:18:52 +02:00
Dan Larkin a3ae2b7191 Augmented cache API and fixed a potential deadlock scenario. (#2579)
* Augmented cache API and fixed a potential deadlock scenario.

* The RocksDBEdgeIndex now tries to read up to 10 times form the cache on lock_timeout. Instead of rewriting everything after first attempt
2017-06-19 17:28:04 +02:00
Kaveh Vahedipour da0cc3490c Squashed commit of the following:
commit 3d9cf792912db1974b9ac5e00ca2b4c9245b7d34
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 15:32:43 2017 +0200

    optimise for single writes in agency log

commit 65056ab9026f9b4b211dda0f17c75602b978f2bf
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 15:01:15 2017 +0200

    More tests, taking agency log compaction interval into account.

commit 6600d707784e8fd5b62c0c75fd1826af09b8e13f
Merge: cf46882 02f00cc
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 14:50:38 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 02f00cc271d027f02b0625afb76745bfa76bf833
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 14:34:41 2017 +0200

    compaction step and keep size defaults for 3.2

commit 03fc8fbff8f0ac701f7d7f94521c0c3152dd6f92
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 14:32:46 2017 +0200

    Constituent fatally exists if eletion ballot cannot be persisted

commit cf4688226fc897e74bb2d9ebdfca3ce4578c3b70
Merge: c727fc4 724bd1e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 13:08:15 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 724bd1efe19e2e9dbfc14cd819f180816b6d62d0
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 13:02:51 2017 +0200

    persistence success in agency state is properly evaluated

commit c727fc48bb93e7b135b3ca929c03221c7bcaddb9
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 12:04:55 2017 +0200

    Set default compaction step size in agency to 20000 and 10000 keep size.

commit ded16ae6945e9c1479e99bc2e7ccb4d6feca19a6
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 11:11:12 2017 +0200

    Fix help page in startStandAloneAgency.sh for --use-persistence.

commit 13ae9f40f649a8f92eeca4b16bbb5647b540722d
Merge: 834c7c9 aa3e8c1
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 23:41:34 2017 +0200

    Merge remote-tracking branch 'origin/devel' into agency-log-compaction-overhaul

commit 834c7c920d36db3579def66c38fb04870936f8bd
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 16:56:03 2017 +0200

    Handle error in recvAppendEntriesRPC properly.

commit bd9c8d03b76ad25d4078740b5bf994fdba3845d0
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 16:55:35 2017 +0200

    Handle errors in persist() and log() properly.

commit 5b4d2c3d9af078d6a1b5626af20dc9abf2546baa
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 16:25:22 2017 +0200

    Improve error reporting.

commit d60697c5f26d6592eecefc9b9a43e9b699d1773d
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Thu Jun 1 12:16:39 2017 +0200

    Agency must not responds to any requests after startup util leader has RAFT commited up to pre shutdown state

commit 92b8ede5fa022ace1596607abcf8fad1130504c8
Merge: 9340e74 d24455c
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 16:54:45 2017 +0200

    Merge remote-tracking branch 'origin/devel' into agency-log-compaction-overhaul

commit 9340e7461130a4783c09ad8d91e5a07f9500a045
Merge: 7b7ce9d 63a9d60
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 12:13:55 2017 +0200

    agency tests to cover compaction

commit 63a9d604c474eda4302032629dff1f0f69fa0813
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 11:59:11 2017 +0200

    Set agency.compaction-keep-size to at least 0.

commit ef842260968a4769d9502a701b7251da32647e52
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 11:49:34 2017 +0200

    Fix agent size 1 case (thread already gone).

commit 7b7ce9d79f6e8208c13f153b1b9a395b780d6ce1
Merge: 24e2e7e ff306bf
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 11:39:58 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of https://github.com/arangodb/arangodb into agency-log-compaction-overhaul

commit ff306bf547bc4f528c9b66e222271ac143029508
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 11:11:06 2017 +0200

    Move compaction into the future when we take a snapshot from leader.

commit 24e2e7e00f960928a79ce4008b8031d6b9b07fd9
Merge: 84034ac b3ea17a
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 11:01:13 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of https://github.com/arangodb/arangodb into agency-log-compaction-overhaul

commit b3ea17a219baa2abd5892819012fb59f440cdeb8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 09:42:59 2017 +0200

    Get rid of double nonsense.

commit 035c8d1b34e1b73a381d5468422adf13b2ebc36a
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 09:25:28 2017 +0200

    Sort out Agent::load sequence.

commit 84034ac2809a77145d6b1d23bf44857b3a0c4651
Merge: eb34a2e 3180a9d
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Tue May 30 17:07:20 2017 +0200

    merging in

commit eb34a2e64e6ac8dc6571b92cb853c38b7022c833
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Tue May 30 16:58:05 2017 +0200

    keep persistence for restarts in standalone agency

commit 3180a9d9ce4a4401a55ef02606b020316d43cbe5
Merge: 5d60524 28b9580
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Tue May 30 16:56:56 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 5d60524429d8ddda4491beecb931c3b9e3cc1d8a
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Tue May 30 16:56:36 2017 +0200

    Implement snapshot sending in AppendEntriesRCV.

commit 28b958054f51c9cb36706df4e4345aa0f726ed15
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Tue May 30 14:20:13 2017 +0200

    state machine should not advance _committed if empty

commit df18f326acea7f5bc2660a37e22f1503952e4b41
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 23:39:47 2017 +0200

    Store term with compaction snapshot, recover again.

commit 2551a48b6fb513c9ea934bce755f8c364dae2f05
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Mon May 29 17:45:26 2017 +0200

    indices renamed to closely match RAFT documentation

commit e62dcdecf6e8650cfa5725d91b809d05591b48a4
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 16:37:43 2017 +0200

    More cleanup.

commit 9f4787c46621375f0361138a8961431eb21ce5c0
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 14:50:25 2017 +0200

    Revise loadLastCompactedSnapshot to return 0 without a snapshot.

commit 13285e1d70c8a4ac8c79a08de6f8fbc0f8d242bf
Merge: 3393c43 6c5f23e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 12:06:20 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 3393c43c75520c74d20df09c74fbbbd8b1af5976
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 12:03:47 2017 +0200

    More cleanup around Store::apply and friends.

commit 4ccb41d1839748c98e11403fa04f6a7d6af5e95b
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:41 2017 +0200

    Document apply methods in Store with comments.

commit ea05c4880fedb6fe535e24761ac5cb3c26ccfc20
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:18 2017 +0200

    Intentionally keep one log entry more to prevent empty log.

commit 67fb62f2259cc3c6368319917c7257ebcc177d3f
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:44:42 2017 +0200

    Improve plausibility checks at compaction time.

commit 0bafc368785b15a94f8783c4c929f4208f87d09c
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:51:51 2017 +0200

    Sort out (re-)building of agency K/V store(s) from compaction snapshots.

    This is in case of (re-)start, becoming a leader and when serving
    /_api/agency/store.

commit 46b0750bc6c597ec388aac0cdca32082c0cc54b8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:50:54 2017 +0200

    Set compaction interval to 200 and keep to 0.

    This is way to small but tests should run with it.
    Will later increase numbers again.

commit 024dc0846ae30248b464dd481a8bbc1134f56983
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:29:57 2017 +0200

    Add a trivial test for agency log compaction.

commit e12fd3b46833419d7b436eeadd7246304324b891
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:26:03 2017 +0200

    First part of cleanup of agency log compaction.

    Now the right compaction snapshots are taken and persisted.
    Furthermore, the right log entries and old snapshots are removed
    after compaction, both the volatile and the persisted ones.
    The readDB and spearHead stay unchanged at compaction time as it
    should be.

commit d59901aea0c3ca31ef253299d2adc3353b79e664
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 12:18:26 2017 +0200

    Remove unused member variable.

commit 6c5f23eb7b42d9f20d4dadb2932a63add99f9c76
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:41 2017 +0200

    Document apply methods in Store with comments.

commit 670899f72d215e0fcc0ca0389cea9250a291e83b
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:18 2017 +0200

    Intentionally keep one log entry more to prevent empty log.

commit 660f61029917bbc2ce1fae3e4fc903095b023297
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:44:42 2017 +0200

    Improve plausibility checks at compaction time.

commit e2802e4b36d1f67d8361c1d8b0c92fbff696f439
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:51:51 2017 +0200

    Sort out (re-)building of agency K/V store(s) from compaction snapshots.

    This is in case of (re-)start, becoming a leader and when serving
    /_api/agency/store.

commit 12b43f1b91284a1185390d6dcfbd1e838522d392
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:50:54 2017 +0200

    Set compaction interval to 200 and keep to 0.

    This is way to small but tests should run with it.
    Will later increase numbers again.

commit c8b9a37a690b8e7e8bfa1276a3f9ba4b6b5a9c27
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:29:57 2017 +0200

    Add a trivial test for agency log compaction.

commit cf0c8c1fff666f76411082f87efe685a412ecebb
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:26:03 2017 +0200

    First part of cleanup of agency log compaction.

    Now the right compaction snapshots are taken and persisted.
    Furthermore, the right log entries and old snapshots are removed
    after compaction, both the volatile and the persisted ones.
    The readDB and spearHead stay unchanged at compaction time as it
    should be.

commit 0a4255359a57b8686133e6014e2b82b8079f36fa
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 12:18:26 2017 +0200

    Remove unused member variable.
2017-06-02 16:13:03 +02:00
Mop 619eae9be5 Revert "Squashed commit of the following:"
This reverts commit 2252088572.
2017-06-01 18:37:45 +02:00
Andreas Streichardt 15e622d9c9 Fix naming 2017-06-01 17:50:23 +02:00
Andreas Streichardt 2252088572 Squashed commit of the following:
commit f3d0fd6584b0e451b8c97abcb4ba8d9f2fc6f560
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:31:36 2017 +0200

    fix unittest

commit 7cd3544a39e1b78af9d4175cb3b978799b9bbfff
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:10:00 2017 +0200

    Remove debug comment

commit fb6b10dac15be49a72dbff80030a7d22abdfc3e0
Merge: 055eb1d269 6b18cc64fe
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:00:21 2017 +0200

    Merge branch 'devel' into shardorganizer

commit 055eb1d2693a583d21ea59ec8b6ba95ab0db57ac
Merge: 1ff7998ebf 8ea89b7677
Author: Mop <andreas@arangodb.com>
Date:   Thu Jun 1 16:56:30 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 8ea89b76777c75b6a77bf695c3f074a0c4643c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 16:55:41 2017 +0200

    Fix shardmapping bug

commit 1ff7998ebfd691598ec5b455ca5bc2bfd7020fb4
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:26:08 2017 +0200

    more output

commit 68e88aa0e14316c4929d05b2c151bee6421d754d
Merge: 0978ad1d9e 44a6a78ec3
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:03:33 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 44a6a78ec338a1e7cabb15464500d96b84c68f1d
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 07:42:43 2017 -0700

    Fix namespace

commit 0978ad1d9e2f01b86204990e74b66958f25eba66
Merge: f98582ccff d74e5989ad
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:40:35 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit f98582ccff3448f6c2388dab4cc2dc38034271b0
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:39:03 2017 +0200

    Revert "Revert "Next attempt at merging ShardOrganizer...distributeShardsLike fixed""

    This reverts commit fed45b7b10.

commit d74e5989ad478efe7d66d196715c05f4f41c9c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 16:31:31 2017 +0200

    Make it an error

commit 0a6a9ef9464df4f24ad205bbab5b9f8ded50054f
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 12:42:51 2017 +0200

    distributeShardsLike has to be saved as a cidString
2017-06-01 17:32:40 +02:00
Jan Christoph Uhde bdf40fa6e4 add more tests for type conv test 2017-05-31 11:09:12 +02:00
Jan Christoph Uhde 868966a71a Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
* 'devel' of https://github.com/arangodb/arangodb: (25 commits)
  Revert "Implement shardorganizer...Cluster with 1 DBServer possible again"
  remove unused variable
  Move FM.listJson into aardvark
  Remove unused legacy FM test
  Implement shardorganizer...Cluster with 1 DBServer possible again
  Make sure the bundle doesn't exist before loading it from db
  More meaningful object logging
  Response -> IncomingResponse
  Updated plain cache to properly count evictions.
  More realistic binary document test
  Simplified Foxx self healing (#2511)
  fix superfluous re-creation attempts for collections
  Made shortest path OOM test deterministic
  added documentation
  sync ENGINE file
  adjusted ui to internal cluster api
  fix usage of su
  Fixing various issues
  Fixing catch test
  fixed vertexId getter
  ...
2017-05-31 09:04:25 +02:00
Jan Christoph Uhde 5872a4c2b1 add functions for endianess conversion 2017-05-31 09:04:07 +02:00
Frank Celler 73a4203413 Revert "Implement shardorganizer...Cluster with 1 DBServer possible again"
This reverts commit 1b2d6dca83.
2017-05-30 23:40:01 +02:00
Andreas Streichardt dded5abb6c Merge remote-tracking branch 'origin' into shardorganizer 2017-05-30 20:17:07 +02:00
Andreas Streichardt 1b2d6dca83 Implement shardorganizer...Cluster with 1 DBServer possible again 2017-05-30 20:13:00 +02:00
Simon Grätzer 6ecf733585 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/storage-format-refactoring
# Conflicts:
#	tests/RocksDBEngine/TypeConversionTest.cpp
2017-05-30 13:42:44 +02:00
Simon Grätzer 9f107123dd Fixing catch test 2017-05-30 13:41:48 +02:00
Jan Christoph Uhde ab1d11a5e5 fix test - pointer is now moved forward so we need a copy 2017-05-30 13:35:20 +02:00
Simon Grätzer c19bca0eeb fixing compiler warnings 2017-05-30 11:45:05 +02:00
Simon Grätzer 09e0545cbd Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/storage-format-refactoring 2017-05-30 11:31:15 +02:00
Jan Christoph Uhde 64c0b03a99 Add missing changes to geo index 2017-05-30 11:08:44 +02:00
Simon Grätzer 12d1b2d318 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/storage-format-refactoring 2017-05-30 10:12:47 +02:00
Simon Grätzer 3ad939ea41 changed storage format 2017-05-30 10:12:39 +02:00
Jan Christoph Uhde c4e7ae54da Prepare geodata to be more platform independent 2017-05-30 09:22:45 +02:00
Michael Hackstein 5aed709996 Fixed CacheTable test. Adjusted it to new upper and lower limit 2017-05-29 11:19:18 +02:00
Simon Grätzer b0e5f4adc8 Test fixes 2017-05-24 19:21:56 +02:00
Andreas Streichardt f2670f8040 Extract compareServerList and make it reuseable 2017-05-24 14:13:51 +02:00
Dan Larkin 78c80c3a3d Doubled size of cache buckets to increase associativity. 2017-05-23 17:51:47 -04:00
Dan Larkin f63b605f3e Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-15 13:24:33 -04:00
Dan Larkin 5817de528f Adjusted some cache constants and added eviction rate tracking. 2017-05-15 13:24:30 -04:00
Michael Hackstein d7d650e27d Implemented index estimates for RocksDB. Now the following indexes have
a proper estimate:
* HashIndex
* SkiplistIndex / Persistent
* EdgeIndex
2017-05-15 17:17:58 +02:00
Simon Grätzer f101e21328 Geo index update, renaming 2017-05-11 13:19:51 +02:00
Simon Grätzer d8053a9bcf Fixing fulltext deduplication 2017-05-10 16:17:46 +02:00
Simon Grätzer 42d230e4ae Fixing recovery for geo index 2017-05-09 22:29:29 +02:00
Simon Grätzer 6f69454c82 Added fulltext index 2017-05-05 15:44:15 +02:00
Andreas Streichardt fd3f8e6ad9 Remove rocksdb linking (implicitly done via arangoserver!) 2017-04-28 12:09:02 +02:00
Kaveh Vahedipour e7797d292e fixed shard ordering in Job::clones with consequences for unit tests 2017-04-27 13:37:47 +02:00
jsteemann b24f4a9435 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-04-21 18:14:31 +02:00
Andreas Streichardt 3bf2627f57 Remove ifdefs for non linuxes 2017-04-21 18:10:09 +02:00
Andreas Streichardt 7322e3bff3 Allow seeding of randomgenerator for tests 2017-04-21 18:08:49 +02:00
jsteemann 1df46f8923 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-04-21 16:59:32 +02:00
Kaveh Vahedipour 42caf5044c merge in cpp & js from 3.1.18 yet to do tests 2017-04-21 15:47:03 +02:00
Dan Larkin 97d525e9b7 Revert "Fixed primary index cluster bug."
This reverts commit 8c4214625c.
2017-04-17 17:06:23 -04:00
Dan Larkin 8c4214625c Fixed primary index cluster bug. 2017-04-13 21:44:51 -04:00
Michael Hackstein 1087065ea5 Merge branch 'devel' of github.com:arangodb/arangodb into engine-api 2017-04-13 17:34:25 +02:00
Andreas Streichardt f60a60a4b9 manually initialize logger 2017-04-12 14:48:31 +02:00
Dan Larkin eaac6c013c Fixed issue with cache transactions. 2017-04-07 16:58:22 -04:00
Michael Hackstein 535e4cd6cb Renamed PriorityQueue => ShortestPathPriorityQueue 2017-04-05 16:12:47 +02:00
Michael Hackstein 039c6fbd16 Moved all logic of lib/Basics/PathFinder into seperated files in arangod/Graph. Preparation to detemplatify this code as we need some arangodb interna in it for Slice mangement 2017-04-05 15:31:44 +02:00
jsteemann de769b534d split ManagedDocumentResult in .h and .cpp files 2017-04-04 15:47:05 +02:00
jsteemann b975d0ea1d added tests 2017-04-04 13:47:14 +02:00
jsteemann 4196594130 fixed crash 2017-04-04 13:27:35 +02:00
jsteemann a435b86b09 added very basic test 2017-04-03 17:58:22 +02:00
Simon Grätzer 901abc6705 Fixing some windows conversion warnings 2017-03-29 11:35:37 +02:00
jsteemann d0766688f3 properly tag geo index tests, and include the long running geo index tests by default 2017-03-29 00:54:24 +02:00
Dan Larkin c2f370f99b Fixed an alignment bug in cache. 2017-03-24 13:39:05 -04:00
Dan Larkin 22d17a318b Fixed a failing cache test. 2017-03-22 15:26:54 -04:00
jsteemann f8a25febe7 fix tests with g++5.4.1 and ASAN 2017-03-22 18:13:19 +01:00
jsteemann e270ddb262 fixed failing test 2017-03-22 14:46:10 +01:00
jsteemann 71bf18dd12 renaming 2017-03-22 13:01:40 +01:00
Simon Grätzer c2331729a2 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-21 14:39:30 +01:00
Simon Grätzer a01a69e560 Pregel: fixed catch test 2017-03-21 14:39:14 +01:00
jsteemann 7fb536387d move replication-dump into engine 2017-03-21 13:55:57 +01:00
Simon Grätzer 68f32177c6 Added test for mapped files in TypedBuffer.h 2017-03-16 17:17:05 +01:00
Dan Larkin 4026b844c7 Lowered more test thresholds to lessen environmental sensitivity. 2017-03-16 11:15:12 -04:00
Dan Larkin 773d489f55 Lowered a(n arbitrary) test threshold to account for testing environment. 2017-03-16 10:45:01 -04:00
Dan Larkin 3ac4495eac Moved Finding class out of Cache for convenience. 2017-03-14 11:51:25 -04:00
Dan Larkin 5ed14990ec Made a few tests thresholds less sensitive. 2017-03-13 14:09:12 -04:00
Dan Larkin 45ca4bca08 Fixed some compiler warnings. 2017-03-13 13:10:22 -04:00
jsteemann 7634bb5039 fixed some compile warnings 2017-03-13 17:44:46 +01:00
hkernbach ecfe18a6ce fixed clang build error 2017-03-13 12:47:09 +01:00
Wilfried Goesgens 690a3902a6 fix rocksdb submodule references 2017-03-13 11:43:47 +01:00
Dan Larkin 6a449efb06 Improved cache rebalancing and allocation strategies. 2017-03-11 12:15:08 -05:00
jsteemann fd7ad33a0c added -Wnon-virtual-dtor compile option 2017-03-08 12:56:58 +01:00
Andreas Streichardt 95f9d1e191 Remove fakeit for now 2017-03-08 11:33:42 +01:00
jsteemann aa870e9bce fix attribute name parser failure return code 2017-03-07 10:32:27 +01:00
Andreas Streichardt 0d02163ee5 Fix test compile on mac 2017-03-07 08:59:13 +01:00
Andreas Streichardt 8a79e5fed4 Move last test and remove pathenumerator...was not in use and broken 2017-03-06 19:04:45 +01:00
Andreas Streichardt 6f6e40140c Move icu helper 2017-03-06 18:51:12 +01:00
Andreas Streichardt 59b49485b5 Merge branch 'devel' into catch_tests_ng 2017-03-06 18:34:46 +01:00
Andreas Streichardt ce3c17e7de Merge branch 'devel' into catch_tests_ng 2017-03-06 17:59:26 +01:00
Andreas Streichardt cb8e5aa331 Migrate boost unittests to catch 2017-03-03 21:09:56 +01:00
Andreas Streichardt 1664c31529 First tests \o/ 2017-03-03 15:42:07 +01:00
Andreas Streichardt fe512ed526 Create a arangoserver lib which we link againts arangod and tests 2017-03-03 15:31:43 +01:00
Andreas Streichardt 6168e81b2e first steps 2017-03-02 19:23:32 +01:00