1
0
Fork 0
Commit Graph

225 Commits

Author SHA1 Message Date
Dan Larkin-York 663212ba19 [3.5] Check scheduler queue return value (#9759)
* Add to cmake.

* Backport changes from devel.

* added CHANGELOG entry, port adjustments from devel
2019-08-20 12:57:51 +03:00
Tobias Gödderz f501e00e9d Bug fix/add shard id to replication client identifier (#9366) 2019-07-08 14:03:42 +02:00
Jan c054cddfe5
fix race in _batchPingTimer shutdown (#8801) 2019-04-26 18:31:24 +02:00
Jan e36f7d429e
Bug fix/fix scheduler shutdown task assertion (#8727) 2019-04-10 19:51:45 +02:00
Lars Maier 12eebb15fe Feature/new server infra (#7733)
* Decoupled IO handling from Scheduler.

* Fixed SSL start up bug.

* Replaced Scheduler with new worker farm implementation.

* Added minimal statistics and info string for Scheduler.

* Added support for timed submissions.

* Updated delayed submission api. Updated code that used timers.

* Extracted new Scheduler into a virtual parent class. The implementation can now depend on the usecase.

* Signal handler now working.

* Changed threads names, `_stop` is atomic, check for failure during thread start + exception handling like old scheduler did.

* Commented on source code and added TODOs.

* Played around with start-stop-conditions

* Play around with start stop condition.

* start stop cond

* Sart Stop Conditions

* Removed bad cv_status check.

* Bug fix: now compare the actual objects instead of pointer values. Setup t1 and t2 depending on the thread id.

* Moved most of the stuff now unrelated to the Scheduler to GeneralServer. Got rid of JobGuard.

* Instead of waiting for a thread to terminate, put it on a clean up list and check for its termination in each supervisor run.

* Allow detaching long running threads.

* Fixed test mock.

* Updated the WorkHandle logic. Removed post functions.

* Fixed crash when obtaining shared_ptr from this in destructor.

* Added lost mutex.

* Fixed memory leak.

* Fixed merge bug.

* Changed a lot of code to optimize the scheduler.

* Fixed bug of invalidated iterator. Dont remove task on shutdown at different places. Let scheduler threads run until queue is empty.

* Only by value calls to queue.

* Added options again.

* Clean up of code.

* UI Request Lane added.

* Bug fixes in Scheduler.

* Applied reformat.

* Use sigaction.
2019-01-08 10:12:02 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Jan 2115abdce9
smaller changes for replication (#7200) 2018-11-02 18:38:32 +01:00
Simon b883026f45 Server stream cursor (#7186) 2018-11-02 16:19:48 +01:00
Simon 6628a4e55a Refactor stuff, add async batch extension task (#6875) (#6880) 2018-10-15 13:18:24 +02:00
Dan Larkin-York 820bfee329 Refactor syncer state and make notes for future parallelization (#5742) 2018-07-03 21:32:16 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Jan 5acaf9b5a3
Bug fix/carnival (#4562) 2018-02-12 09:01:57 +01:00
Jan 8da61a9875
Bug fix/more replication tests (#4500) 2018-02-09 13:06:47 +01:00
Jan 720e6df82e Bug fix/fixes 1910 (#3471)
* properly initialize all properties

* use faster comparison

* properly detect and handle "method not allowed"

* code-style

* remove unused variable

* narrow variable scope

* handle non-existance of AuthenticationFeature

* remove dead code

* replace some C string handling with std::strings

* moved assertion to the correct place

* honor number of array members for IN operator

* slightly adjust error messages

* slighty adjust some error messages

* try to fix issue with lingering replication contexts on shutdown

* clean up heartbeat thread a little bit

* small fixes
2017-10-23 09:17:36 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
m0ppers 9a0bc716d0 Do not allow replication to create/drop collections (#2898)
In the cluster case the only one who is allowed to do this is the schmutz
2017-07-28 14:24:02 +02:00
m0ppers 589ffd5c59 Feature/improve logging (#2881)
* Improve logging in various places

* Fix jslint
2017-07-28 09:23:18 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Max Neunhoeffer a97ec1cc29 Try to handle another shutdown hanger. 2017-06-09 00:42:31 +02:00
jsteemann 226920d7fa optimizations 2017-05-17 23:41:16 +02:00
jsteemann c5a195d62c attempt to fix races in replication 2017-05-10 14:32:16 +02:00
Dan Larkin d77efe38dc Added explicit WAL file lifecycle management to reduce space overhead from replication. 2017-05-04 15:16:24 -04:00
Jan Christoph Uhde 3cb1cc7a52 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
* 'devel' of https://github.com/arangodb/arangodb:
  Fixing index markers
  exclusive locks for indexes
  better incremental sync
  grunt build
  Avoid log spam.
  Removed code paths that wrote objectIds into the Agency. This did break replication.
  WAL: honor tick end value
  WAL fiter after collection
  Reactivated client-side filtering of unnecessary markers
  Added an Assert when persisting an index it's objectId is not allowed to be 0
  The RestReplicationHandler now inserts new ObjectIds when replicating collections
  moved files
  add dependencies for TransactionManager
  fix include
  move engine-specific test into engine test
2017-05-02 16:41:43 +02:00
Jan Christoph Uhde da02fd36c6 move engine specific syncer code into engines 2017-05-02 16:36:27 +02:00
jsteemann 45096dd503 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-02 16:09:35 +02:00
Simon Grätzer e335a29326 better incremental sync 2017-05-02 15:08:05 +02:00
jsteemann 910f5680b0 moved files 2017-05-02 12:34:28 +02:00
Simon Grätzer 14a3168e40 Working initial sync 2017-04-26 17:01:35 +02:00
Simon Grätzer bdd002d6ec Fixing incremental sync deletion 2017-04-26 14:11:51 +02:00
Simon Grätzer 02e41de608 Working on replication 2017-04-26 13:32:12 +02:00
Simon Grätzer 6472c4928d Various warnings 2017-04-26 11:26:08 +02:00
Simon Grätzer d228a94462 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/Replication/InitialSyncer.cpp
2017-04-26 11:06:09 +02:00
jsteemann 6b8c818e2d remove unused variable 2017-04-26 09:59:19 +02:00
jsteemann 404ef431ae cppcheck 2017-04-26 09:50:55 +02:00
Jan Christoph Uhde 26112be584 do checking before creating copy of slice 2017-04-26 08:33:23 +02:00
Dan Larkin 20dc8b4fa0 Fixed duplicate objectIds in InitialSyncer. 2017-04-25 16:51:38 -04:00
Simon Grätzer 3aada2afe0 Fixed double handled case 2017-04-25 16:03:51 +02:00
Jan Christoph Uhde e4b3f5c4ac fix warning 2017-04-25 15:12:22 +02:00
Jan Christoph Uhde e06efa6420 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
* 'devel' of https://github.com/arangodb/arangodb:
  fix compile warnings
  Fixed build
  add missing signaling of crashed state
2017-04-25 15:08:17 +02:00
Jan Christoph Uhde fca8f01df6 create logger state just in one place 2017-04-25 14:58:32 +02:00
jsteemann 26d4fd61db fix compile warnings 2017-04-25 14:56:00 +02:00
Simon Grätzer 2f968d649a Fixed build 2017-04-25 14:53:59 +02:00
Simon Grätzer 69c2faf592 Fixed synced 2017-04-25 14:27:34 +02:00
Simon Grätzer 5d20b79c0f Incremental Sync 2017-04-25 12:01:58 +02:00
Simon Grätzer 91fe0f01ea formatting 2017-04-24 15:55:21 +02:00
Simon Grätzer 7176d34169 Fixed incremental synced 2017-04-24 15:40:48 +02:00
Jan Christoph Uhde 2269ca7b0a add batchId and logging 2017-04-24 11:47:54 +02:00