1
0
Fork 0
Commit Graph

1967 Commits

Author SHA1 Message Date
jsteemann 4e5c4413ea constify API 2018-06-26 23:37:25 +02:00
Simon ec0d2a1b7b Remove Coordinator DBs (#5661) 2018-06-25 19:18:11 +02:00
Andrey Abramov 5eef6cd618
Feature/test iresearch (#5610)
* start implementing arangosearch cluster tests.

* backport: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties

* first attempt to fix failing tests

* refactor cluster wide view creation logic

* if view is not found in the new plan then check the old plan too

* ensure the cluster-wide view is looked up in vocbase as well on startup/recovery

* do not store cluster-wide IResearchView in vocbase

* move stale view cleanup to the shared pointer deleter, address test failures

* do not print warning

* enable arangosearch tests by default

* fix catch tests

* address icorrect return value for cluster-wide links

* address some issues with test failures due to cluster-view allocated within TRI_vocbase_t

* simplify per-cid view name, address 'catch' test failures

* ensure IResearchViewNode volatility is properly calculated in cluster

* invoke callbacks directly in AgencyMock instead of waiting for timeout

* ensure view updates via JavaScript always use the latest view definition

* pass a list of shards to `IResearchViewDBServer::snapshot`

* extend cluster aql tests

* fixes after merge

* fix class/struct inconsistencies

* comment failing tests

* remove debug logging

* add debug function

* tests cleanup

* simplify upcoming merge: pass resolver from a side

* backport: move all transaction status callback logic to Methods

* add changes missed from previous commit

* fix js and ruby tests

* more tests for IResearchViewNode

* pass transaction to IResearchViewDBServer::snapshot, address IResearchViewDBServer tests segfault

* pass transaction to IResearchView::snapshot instead of transaction state

* temporarily add trace log output to tests to try to find the cause of the core dump on Jenkins

* add more temporary debug output to trace down the segfault on Jenkins

* add even more temporary debug output to trace down the segfault on Jenkins

* ensure Vieew related maps are cleared during shutdown

* reset ClusterInfo::instance() before DatabaseFeature::unprepare()

* remove extraneous debug output

* missed line from previous commit

* uncomment required line

* add nullptr checks to RocksDBIndexFactory::prepareIndexes(...) similar to the ones in MMFilesIndexFactory::prepareIndexes(...)

* attempt to fix deadlock in tests

* add comment as per reviewer request

* fix aql test suite name

* add some debug logging

* address deadlock between ClusterInfo::loadPlan() and CollectionNameResolver::localNameLookup(...)

* eplicitly state which index definition failed in the log message

* use vocbase from shard-view isntead just in case

* explicitly state which index definition failed in the log message

* do not create shard-view instances from cluster-link instances (only register existing ones)

* add some tests
2018-06-21 20:35:04 +03:00
Jan 8a25558e62
fix test (#5618) 2018-06-15 15:53:23 +02:00
Wilfried Goesgens c4c711f372 fix windows compile (#5611) 2018-06-14 19:18:54 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Jan 08b12942bd
fix AQL DOCUMENT lookup in case a collection has multiple shards and custom shard keys (#5602) 2018-06-14 12:20:06 +02:00
Jan 448a435713
clean up key generators a bit (#5573) 2018-06-12 11:28:38 +02:00
Vasiliy 4253dca6aa issue 381.5: ensure the LogicalView definition that is persisted to the Agency matches the definition that gets created (#5518)
* issue 381.5: ensure the LogicalView definition that is persisted to the Agency matches the definition that gets created

* backport: correct comment
2018-06-02 17:21:55 +03:00
jsteemann bc87837778 simplifications 2018-05-27 19:47:48 +02:00
jsteemann a14b67f584 use nullptr 2018-05-25 18:55:37 +02:00
Jan Christoph Uhde a2dcb6cc5d WIP - start adding optional overwrite to insert operation (RepSert) (#5268) 2018-05-24 19:47:15 +02:00
Simon 332a7958f5 Cleanup cluster selectivity (#5440) 2018-05-23 18:00:14 +02:00
Jan 8e6d5df129
fixed minor several compiler complaints (#5406) 2018-05-23 11:50:00 +02:00
Vasiliy 94ddd7803d issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424) 2018-05-23 00:59:08 +03:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
Vasiliy d9cda9666f issue 389.8: remove redundant function from Methods, convert Syncer API to user TRI_ocbase_t& wherever possible (#5408) 2018-05-22 16:10:24 +03:00
Kaveh Vahedipour c2c104d6b1 agency pool size had to be larger 1 (#5379) 2018-05-17 11:54:41 +02:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Vasiliy 6a53154160 issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344) 2018-05-14 19:06:24 +03:00
Simon 17b1a2aafb Rest middleware refactoring (#5332) 2018-05-14 17:43:10 +02:00
Simon f2b952134f Fixing agency pool update (#5316) 2018-05-14 14:56:19 +02:00
jsteemann cdfac2049a Merge branch 'bug-fix/remove-unused-class-in-cluster-selectivity-estimate' of https://github.com/arangodb/arangodb into devel 2018-05-09 18:15:54 +02:00
Vasiliy 0d1cf45097 issue 373.3: use TRI_vocbase_t& for Upgrade tasks, remove redundant checks for null TRI_vocbase_t (#5301) 2018-05-09 15:54:07 +03:00
Kaveh Vahedipour 6580ba11cd removal of unused class Cluster/SelectivityEstimate 2018-05-08 12:16:21 +02:00
Michael Hackstein 4ede998186
Bug fix/compile issues on mac (#5280)
* Mvoved unnecessary std::move

* Replaced size_t by uint64_t

* Fixed usage of <array>
2018-05-08 11:07:16 +02:00
Tobias Gödderz 8c87f51429 Feature/fix inconsistent distribute shards like job (#4743) 2018-05-07 16:53:08 +02:00
Simon fdee0544b7 Using asio::io_context::strands instead of locks (#5266)
* initial try adding strands

* working, stable amount of threads

* improve shell_client cluster

* Fixing some accounting for the scheduler

* Fix accounting

* Fixing wrong strand usage

* add missing return

* Fixing thread accounting

* More scheduler accounting issues

* Fixing various things

* Fixing some stuff

* Fixing some stuff

* Some more subtle bugfixes

* Some cleanup code

* fixing some stuff

* adding some more fixes

* Fixing possible issues

* Fixing missing _storeResult

* Fixing some stuff

* Reducing lambda stack, perhaps fixing hangups

* Fix writeunlocker

* Fixing possible issues

* adding some debugging stuff

* refactor sockets

* possible fixes

* Adding more job guards

* Fixin possible bug

* cleaning up some stuff

* working impl

* Remove debugging output

* Fixing build

* fixing import

* Fixing another bug

* removing debug log

* Removing examples

* Reverting scheduler working code

* Cleanup

* Addressing review comments
2018-05-07 15:58:19 +02:00
Andrey Abramov b69b5bdfdf
Bug fix/issue #5186 (#5269)
* do not persist view on startup

* small refactoring

* ensure view is being opened after creation
2018-05-06 20:38:32 +03:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
Matthew Von-Maszewski 01cc6d2159 adjust thread crash logging for simple db server case. (#5234) 2018-05-02 22:35:20 +02:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Simon a1416e1067 Make v8 optional on startup (#5220) 2018-04-30 12:48:57 +02:00
Jan 30b12e311b
Bug fix/remove most of aql js (#5223) 2018-04-30 11:17:11 +02:00
Simon 2e0e2574d4 Port from 3.3 (#5213) 2018-04-27 17:05:18 +02:00
Matthew Von-Maszewski a67df088b0 correct race condition leading to infinite job execution (#5201)
* fix infinite loop by setting _lastSyncTime within runBackgroundJob().  add code to make agency callback ignore _lastSyncTime limit.
* create change notes for this PR and previous PR 5114
2018-04-27 13:43:22 +02:00
Wilfried Goesgens ea507bb27c use std::chrono and std::date for date / time formatting (#5194) 2018-04-24 18:23:04 +02:00
Simon 468231efc5 AQL Profiling code (#5165)
* initial start of profiling

* adding profiling code

* Fixing remote block tracing, fixing width and units

* Fixing some tests

* Various fixes

* adressing review comments
2018-04-24 16:17:30 +02:00
Wilfried Goesgens 7d6e580780 Refactoring & code cleanup (#5138) (#5142) 2018-04-24 14:42:23 +02:00
Matthew Von-Maszewski a84f7805ad Feature/mv thread death logging (#5111)
* Initial low level interface for thread crash reporting (and management).
* Add a member version of isClusterRole()
* isolate heartbeat thread creation to new StartHeartbeatThread().  create heartbeat thread even if not a cluster or if an agent.
* update runDBServer() and runCoordinator() to shutdown more quickly by polling isStopping() at additional locations.
* copying updates from different branch / PR
* basic thread crash logging.  Not yet tied into Agency arangod or have any specific threads posting crashes
* make Supervision thread a CriticalThread
* sandwich CriticalThread between Thread and other classes to create long term, repeating thread crash reporting.
* restore code lost upon branch update relating to new startHeartbeatThread() function
* add CriticalThread.cpp to build
* add new runAgentServer() function to loop for Agents.  Make Heartbeat thread derive from CriticalThread.
* remove debug line
2018-04-23 15:50:14 +02:00
Vasiliy 012aaa9469 issue 383.4: push vocbase validity check up from Query constructor out into arangodb::consensus::State, StatisticsWorker and AQLUserFunctions calls (#5177) 2018-04-23 14:52:42 +03:00
Simon 45fbed497b Supervision Job for Active Failover (#5066) 2018-04-23 12:49:41 +02:00
Jan 2b84348b77
remove call to requiresElevatedPrivileges with default value (#5172) 2018-04-23 11:28:24 +02:00
Vasiliy 9062c41592 issue 383.3: implement remainder of IResearchViewDBServer tests, use the data-source id (primary key) instead of an arbitrary instance for dropCollection()/dropView(), backport from iresearch upstream: ensure block is flushed if key index is full (#5176) 2018-04-23 00:33:46 +03:00
Matthew Von-Maszewski dd03ca5dd8 shutdown quicker on db server and coordinator heartbeat threads (#5114)
* shutdown quicker on db server and coordinator heartbeat threads

* Adjust the new condition variable usage to follow normal coding patterns.  Probably was ok.  But why take the chance.  And simplify future maintenance.
2018-04-20 15:00:08 +02:00
Kaveh Vahedipour 3d043b35a3 Feature/supervsion maintenance mode (#5108)
* Supervision goes to Maintenance mode, when /arango/Supervision/Maintenance exists
* coordinator route stands
* stop updates in transient, when supervision off
2018-04-20 13:23:22 +02:00
Wilfried Goesgens 2b1ba8c524 fix windows warnings (#5115) 2018-04-17 11:44:45 +02:00
Simon 8be273efb8 Replication cleanup (#5105) 2018-04-17 08:17:42 +02:00
Simon 7677afabf1 Remove copy of request body in rest handlers (#5104) 2018-04-16 14:49:51 +02:00
Andrey Abramov 6eaaf6abd2
Merge pull request #5103 from arangodb/bug-fix/internal-issue-#374.3
issue 374.3: use a reference to vocbase instead of a pointer in DatabaseGuard
2018-04-13 15:52:01 +03:00