1
0
Fork 0
Commit Graph

605 Commits

Author SHA1 Message Date
Jan Christoph Uhde 3f603f024f remove some containers from common.h (#9223)
* remove some containers from Common.h

* enterprise fixes
2019-06-07 13:27:24 +02:00
Dan Larkin-York 44a413a9af Miscellaneous fixes for named indices (#9100) 2019-05-31 17:00:56 +02:00
Tobias Gödderz 79cd45f89c Wait for replication before inserting documents (#9151)
* Wait for replication before inserting documents

Also, increased some timeouts and fixed a log message

* Fixed some log levels and a log message

* Removed repair-distribute-shards-like-spec from greylisted tests
2019-05-31 16:09:20 +03:00
Jan 59b67cad40
fix various small annoyances (#9079) 2019-05-23 17:36:38 +02:00
Jan 79258e072a
Bug fix/remove io task (#9056) 2019-05-22 14:34:49 +02:00
Wilfried Goesgens 1907a7211b Bug fix/cleanup system includes (#8962) 2019-05-15 15:12:59 +02:00
Jan 0cbdfe9289
Bug fix/vpack update (#8875) 2019-04-30 12:33:26 +02:00
Jan c054cddfe5
fix race in _batchPingTimer shutdown (#8801) 2019-04-26 18:31:24 +02:00
Simon 5bb9d2dc17 transaction manager tests (#8759) 2019-04-16 21:26:40 +02:00
jsteemann fe58c54b2d oops 2019-04-15 11:15:17 +02:00
Jan 69f7a8a00d
longer WAL file retention for test (#8736) 2019-04-15 10:16:34 +02:00
Jan e36f7d429e
Bug fix/fix scheduler shutdown task assertion (#8727) 2019-04-10 19:51:45 +02:00
Simon 2b594bdab5 Reduce # of memcpy from storage engine layer (#8685) 2019-04-08 22:43:29 +02:00
Jan 4f7923a971
remove replication clients handling out of vocbase code (#8676) 2019-04-08 19:15:28 +02:00
Dan Larkin-York b029f67e68 Fix replication index conflicts. (#8659) 2019-04-04 16:41:06 +02:00
Jan d1d1fe555b
Bug fix/better error message in case something goes wrong http wise (#8670) 2019-04-03 18:44:10 +02:00
Jan b6df220821
cleanup replication contexts (#8630) 2019-04-01 09:35:08 +02:00
Jan 7bf23bf238
add shutdown protection for PregelFeature (#8628) 2019-03-29 19:36:10 +01:00
Dan Larkin-York ffc05c05c0 Fix handling of index name conflicts during replication. (#8571) 2019-03-28 18:53:43 +01:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Dan Larkin-York 04ef4595e3 Fix replication issue with index ID/name conflicts (#8448) 2019-03-20 14:06:02 +01:00
Jan 12e11a5197
port of replication improvements from 3.4 (#8308) 2019-03-11 13:37:18 +01:00
Jan 4c6c61ca33
document `includeFoxxQueues` attribute and persist it (#8351) 2019-03-08 18:43:35 +01:00
Kaveh Vahedipour ee751e8ba3 [devel] clear compilation warnings (#8345) 2019-03-08 10:35:09 +01:00
Jan 2c1eee547d
abort ongoing transactions in all cases (#8291) 2019-02-28 14:36:56 +01:00
Jan 5d2ab0c901
port from 3.4 (#8275) 2019-02-28 14:36:29 +01:00
Simon dc24d7fa4e Sync Foxx Queues (#8250) 2019-02-25 17:13:14 +01:00
Jan Christoph Uhde 808fb694d0 WIP - improve auto-resync (#8020) 2019-02-22 17:42:04 +01:00
Jan 1798036ea0
Bug fix/optimizations 18022019 (#8180) 2019-02-19 19:24:04 +01:00
Jan 366af1782b
some more debug logging for replication (#8082) 2019-02-04 12:39:34 +01:00
Jan 0053a72a6a update velocypack (#8076) 2019-01-31 17:31:14 +01:00
Jan f7d94daba2
port some changes from 3.4 (#8011) 2019-01-29 09:26:57 +01:00
Manuel Pöter ecf4d9d62a Fix race conditions in thread management. (#8032) 2019-01-28 15:44:46 +01:00
Wilfried Goesgens 457e14b970 add windows suspend/resume hooks as @char101 sugested for others (#7833)
* add windows suspend/resume hooks

* disable procdump
2019-01-11 12:41:27 +01:00
Jan 8376fbb3d9
multiplex REPLICATION-APPLIER-STATE files for RocksDB engine (#7898) 2019-01-08 14:26:01 +01:00
Lars Maier 12eebb15fe Feature/new server infra (#7733)
* Decoupled IO handling from Scheduler.

* Fixed SSL start up bug.

* Replaced Scheduler with new worker farm implementation.

* Added minimal statistics and info string for Scheduler.

* Added support for timed submissions.

* Updated delayed submission api. Updated code that used timers.

* Extracted new Scheduler into a virtual parent class. The implementation can now depend on the usecase.

* Signal handler now working.

* Changed threads names, `_stop` is atomic, check for failure during thread start + exception handling like old scheduler did.

* Commented on source code and added TODOs.

* Played around with start-stop-conditions

* Play around with start stop condition.

* start stop cond

* Sart Stop Conditions

* Removed bad cv_status check.

* Bug fix: now compare the actual objects instead of pointer values. Setup t1 and t2 depending on the thread id.

* Moved most of the stuff now unrelated to the Scheduler to GeneralServer. Got rid of JobGuard.

* Instead of waiting for a thread to terminate, put it on a clean up list and check for its termination in each supervisor run.

* Allow detaching long running threads.

* Fixed test mock.

* Updated the WorkHandle logic. Removed post functions.

* Fixed crash when obtaining shared_ptr from this in destructor.

* Added lost mutex.

* Fixed memory leak.

* Fixed merge bug.

* Changed a lot of code to optimize the scheduler.

* Fixed bug of invalidated iterator. Dont remove task on shutdown at different places. Let scheduler threads run until queue is empty.

* Only by value calls to queue.

* Added options again.

* Clean up of code.

* UI Request Lane added.

* Bug fixes in Scheduler.

* Applied reformat.

* Use sigaction.
2019-01-08 10:12:02 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Jan 33b288e3dc
fix restrictCollections parameter on database level replication (#7810) 2018-12-19 19:07:31 +01:00
Jan 5bae3742e5
Feature/internal 3306 (#7683) 2018-12-06 16:19:28 +01:00
Michael Hackstein 2d73f04008
Bug fix 3.4/syncing of followers (#7377) (#7535)
* Added some DEBUG output for replication rest handler

* Some more debug logging.

* Increased the priority of the ReplicationHandler. This way we will not get stuck with locks that cannot be canceled. Also cancel the lock on the correct database.

* Added extensive log output for replication thins

* Added tombstones to RestReplicationHandler. In a very unlikely case the cancel of a lock can be executed BEFORE the code that actually registers the lock, in this case we will now write a tombstone and do not lock.

* Revert "Added extensive log output for replication thins"

This reverts commit 6d4e37ea1e59e3b3457336019cc7dbc4c979504d.

* Added extensive log output for replication things, now in ERR level instead of MAINTAINER only

* Now actually use hours for synchronization

* React to errors under soft lock if they show up.

* Added a retry loop to increase the read-lock timer.

* Added more timeing output in RocksDB collection internals to figure out why the followers are dropped

* Tweaked RocksDB options

* Revert "Tweaked RocksDB options"

This reverts commit 2bf9c43280beda4792c47d079387fe5154cdd896.

* Removed debug output

* Applied all requested changes by goedderz

* Deleted unused variable
2018-11-30 14:43:04 +01:00
Simon d0efd95a37 Bug fix/restore index refactor (#7470) 2018-11-27 20:22:36 +01:00
jsteemann 7213b80c6c increase shutdown time 2018-11-27 13:46:10 +01:00
Simon f90b48f792 switch default message for requireFromPresent (#7439) 2018-11-23 18:59:03 +01:00
Michael Hackstein 16d0874da5
Bug fix/synchronous replication catchup (#7146)
* merged fixes from 3.4

* odd fix

* Bug fix 3.4/sync repl release thread (#6784)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Bug fix 3.4/shorter foot in door (#7084)

* Implement `syncCollectionCatchup` in DatabaseTailingSyncer.

First stab, might not even compile.

* Fixed a typo.

* Fix a typo and a compilation problem.

* Further compilation fix.

* Implement two stage catchup.

* Two small corrections.

* Unified error messages in Synchronize shard job.

* Improved a code comment.

* Fixed autocasting bool->double and double->bool issue. That is truely one of the best features ever invented... </irony>

* Renamed doHardLock => toSoftLockOnly and inverted default value

* Merged soft/hard foot logic with Transaction splits

* Use scopeguards to cancel readlocks

* Bug fix 3.4/sync replication allow soft and hard lock (#6864)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Implemented optional 'doHardLock' parameter in the replication acquire read-lock call. A hard-lock guarntees to stop all writes, a soft-lock may not.

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Renamed doHardLock => toSoftLockOnly and inverted default value
2018-11-23 16:16:34 +01:00
Simon c584527d79 Fix restore of views, add --view option (#7425) 2018-11-22 19:24:24 +01:00
Simon cc55ef9f82 Faster index creation (#7348) (#7383) 2018-11-21 09:53:14 +01:00
Jan 3969016b6b
dont update lastProcessedTick too early (#7380) 2018-11-20 17:55:35 +01:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
Vasiliy facb4043a1 issue 496.5: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291 (#7216)
* issue 496.5: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291

* add workaround for shell-collection-rocksdb-noncluster.js::testSystemSpecial test failure

* fix typo
2018-11-06 13:59:18 +03:00
Simon 0eb5142df8 Use shared_ptr for LogicalCollection (#7220) 2018-11-05 18:31:57 +01:00