1
0
Fork 0
Commit Graph

1218 Commits

Author SHA1 Message Date
Lars Maier 642c5fd994 Bug fix 3.3/cleanup lost collections (#6721)
* Working draft: clean lost collections in supervision.
* Added early exit as in spec.
* Finished test. Fixed logging.
* Increase plan version when cleaning out a lost collection.
* Increase the current version rather than the plan version.
* Fixed test for 3.3
2018-10-08 16:35:18 +02:00
jsteemann 8a88060ae3 fixed misuse of bool value 2018-10-05 12:03:32 +02:00
Simon 8ae00c388d Debug logs for ActiveFailover, allow recovery tests on MacOS (#6686) 2018-10-03 18:34:39 +02:00
Kaveh Vahedipour fd800f1642 Bug fix 3.3/agency update endpoints (#6663)
* server building
* tests
2018-10-01 09:51:54 +02:00
Max Neunhöffer 2b59eca0ab
Fix a crash found by the agency torturer. (#6591) 2018-09-28 15:15:13 +02:00
Lars Maier 666235b0b1 Bug fix 3.3/cleanout server dropping wrong follower (#6575)
* Fixing bug: cleanoutServer will no longer add old leader as follower.

* Fixed rollback.
2018-09-21 14:38:04 +02:00
Simon fed9a45b20 Properly check syncer erros, catch more exceptions (#6521) 2018-09-17 16:39:10 +02:00
Matthew Von-Maszewski ef4a6b84fa backport PR 6150: unique function to block network retries during shutdown. (#6188) 2018-08-20 12:58:25 +02:00
Matthew Von-Maszewski ec5a2f62b8 3.3: Bring two key Agency bug fixes, plus some secondary stuff back to 3.3 (#6009) 2018-08-08 10:33:17 +02:00
Max Neunhoeffer 94ba83ae90 Keep failed follower in followers list in Plan. 2018-07-23 11:24:49 +02:00
Kaveh Vahedipour 3991525eea compaction index should not go beyond own log index ever (#5899) 2018-07-17 15:54:44 +02:00
Kaveh Vahedipour 773a4a231c Bug fix 3.3/mv faster agency gossip (#5422)
* backport of multiplier + version interaction bug from devel
* change log order
2018-07-16 15:08:48 +02:00
Kaveh Vahedipour c1d671dcb9 Bug fix 3.3/agency better log compaction (#5098)
* agency's log compaction according to internal issue #2249
2018-07-16 12:12:05 +02:00
Kaveh Vahedipour 73cf95166c backport devel fix for lastConfirmed bug fix (#5743) 2018-07-11 11:28:18 +02:00
Kaveh Vahedipour bc8e73fce0 agent can now join with disaster-recovery-id and changed endpoint (#5828) 2018-07-11 11:24:36 +02:00
Kaveh Vahedipour b92b6f378f inception was ignoring leader's configuration (#5415) 2018-05-22 12:57:00 +02:00
Tobias Gödderz a391088dbf [3.3] Fix inconsistent distributeShardsLike job (#5300) 2018-05-17 12:07:20 +02:00
Simon 4f5defc9ea Fixing agency pool update (#5327) 2018-05-14 18:07:47 +02:00
Simon c954841a4f Backport supervision for active failover job + testsuite (#5181) 2018-04-23 16:38:11 +02:00
Kaveh Vahedipour 507418d9a4 stop supervision on demand (#5109)
* stop supervision on demand
* adding tests
* Correct an error message.
2018-04-20 11:58:47 +02:00
Simon fa9da5e356 Fixing agency prefix in Agency/Job.cpp (3.3) (#5060) 2018-04-13 11:03:52 +02:00
Kaveh Vahedipour c07a706948 supervision fix for internal issue #2215 backport to 3.3 (#5063)
* supervision fix for internal issue #2215 backport to 3.3
2018-04-10 15:29:27 +02:00
Kaveh Vahedipour 3d8eb7541c internal issue #2215 (#5043) 2018-04-10 12:00:48 +02:00
Jan 765aed1368 bump version number for all local DDL changes and tell agency (#4684)
* bump version number for all local DDL changes and tell agency
  this allows other listeners (e.g. for DC2DC) to get notified when  
  DDL operations are carried out locally and need to be applied remotely
* Increase a log level.
2018-03-23 12:34:08 +01:00
Tobias Gödderz b34eb1c23f Bugfix / Supervision: removeFollower should remove the last follower(s) first (#4926)
* Added a test asserting the last followers are removed first as required by moveShard
* Remove the last followers first
* Removed unused includes
* Updated CHANGELOG
2018-03-23 09:33:04 +01:00
Kaveh Vahedipour 263fe1f6e9 3.3: fixed the missed changes to plan after agency callback is registred f… (#4777)
* fixed the missed changes to plan after agency callback is registred for create collection
* fixed agency inquire
2018-03-14 12:03:40 +01:00
jsteemann f68cd7f296 fix typo in variable name 2018-02-28 17:39:49 +01:00
Kaveh Vahedipour 4893e1ccca more information from inception, when agent is activated (#4691) 2018-02-28 16:11:58 +01:00
Kaveh Vahedipour a69c60d79b bug fix for jobs looking at distrubuteShardsLike and virtual collections (#4666) 2018-02-28 16:09:16 +01:00
Simon 1d57a46168 Refactor Authentication Layer (3.3) (#4588)
* added tests for revokeCollection and revokeDatabase

* optimized user permission test

* ui selection bugfix

* fixed ldap ui login

* login view

* Authentication refactoring

* localstorage fallback if user config is not available

* Fixing permission resolution test

* Adding missing import

* local storage queries now supported

* disabled collection task check for ldap

* added internal ldapEnabled function and ldap config to the ui

* more db creation tests

* removed console logs

* render fix

* Various authentication related fixes

* exec ldap test also for cluster

* Adding support to refresh user rights from external auth sources

* ldap test howto comment

* Handling roles more correctly

* jwt

* login view

* First part of rework of LDAP documentation.

* test roles in a ldap environment

* Changing role handling

* Finish revision of the LDAP chapter in the manual.

* Fixing user header

* Fixing some slight issues with LDAP users and roles

* Removing unused code

* Removing unused code

* added ldap test

* fixing a bug in restuserhandler

* more ldap tests

* ldap tests

* ldap tests

* optimized ldap testing, added cluster support, fixed some auth tests

* ldap cleanup test

* ldap tests

* auth tests

* ldap test

* Changing permission defaults

* revert change

* updated user helper test

* rm of try catch block connection

* Removing FeatureCacheFeature

* Changed permission resolution according to discussion

* updated the docs to clarify the permission resolution and the intricacies of LDAP users

* Fixing wrong permissions check in handling of PUT in ResUserHandler

* Using revision ID when replacing users

* Adding basic replace test

* tests

* Fixing some outstanding issues

* Fixing test setup, optimizing some stuff

* Fixing permission resolution rules, testsuite setup, etc

* Fix deadlock

* Adding error message for keyspace, slightly changing test setup

* Removing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

* Fixing an assert

* Cleanup, adding changelog entry

* fix typo

* Fixing dump_authentication test

* improvements found during code review

* oops

* updated CHANGELOG

* Fixing broken handling, disallowing adding of local users when disabled

* added tests for ldap search mode

* Fixing testInvalidGrants

(cherry picked from commit bc7ea2aaa29a9ed0974898f487e8a318f24912f1)

* Removing undefined auth level externally

(cherry picked from commit 70859f43ae6fd694fdbf70f669fbfdafc58e7913)

* Fixing previous commit

(cherry picked from commit 2fbcffd2ed657862ef9fb5e6d45201a6ec8ada69)

* more use of sessionstorage

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* fix LDAP test invocation

* Added roles transformation to ldap test suite

* Fix compilation of community version.

* Imrpved the ldap testsuites by unifying their options

* fix permission problems for system collections

* Improved LDAP configuration documentation.

* Grunt.

* fixed some ro/rw display issues

* fixed some ro/rw display issues part 2

* grunt build

* bump version number

* Fixed typos in LDAP manual
2018-02-28 13:24:18 +01:00
Simon e96e899fd3 Active Failover for Foxx Services (3.3) (#4593) 2018-02-15 09:36:25 +01:00
Kaveh Vahedipour cce5b2decb Bug fix 3.3/supervision to delete removed nodes from health (#4455) 2018-02-13 15:55:42 +01:00
Kaveh Vahedipour a14c4bd02f constituent correctly persisiting _votedFor and _term (#4248) (#4320) 2018-01-17 10:37:16 +01:00
Kaveh Vahedipour 56a9ad69b1 Bug fix 3.3/supervision no longer fails to remove server from failed when back to good (#4210)
* let's not miss failedserver removal
* remove resetting of FailedServers in test code
* Only call abortRequestsToFailedServers at most every 3 seconds.
2018-01-03 21:55:01 +01:00
Matthew Von-Maszewski 41d1bfce23 create independent executeLockedRead and executeLockedWrite to speed performance (#4178) 2017-12-29 13:36:48 +01:00
Kaveh Vahedipour d6ce7a1301 Agency read write locks ported from devel (#4175) 2017-12-28 11:28:11 +01:00
Matthew Von-Maszewski 392ddde251 Bug fix 3.3: Fix supervisor thread crash (#4165)
* port devel branch to 3.3 of supervisor thread death fix
2017-12-27 22:34:29 +01:00
Max Neunhöffer ef8fcd101c
Port to 3.3 of various fixes around leadership preparation in agency. (#4150)
* Add logging for _earliestPackage in Agent.
* Really enforce the hidden option --server.maximal-threads if given.
* Switch off --log.force-direct in scripts/startStandAloneAgency.sh
* Lower the timeout for sending AppendEntriesRPC to 150s.
* Erase _earliestPackage when becoming a leader.
* Challenge leadership in agent main loop.
* Use steady_clock for _earliestPackage.
* Change _lastAcked and _leaderSince to steady_clock as well.
* time difference calculations based on old readSystemClock to steadyClockToDouble
* All system_clock transitioned to steady_clock in Agent. Remaining system_clock are user input / output or timestamps
* Inception system_clock to steady_clock
2017-12-27 16:47:16 +01:00
Matthew Von-Maszewski f35215ea51 Have twice seen coordinator go into long loop on shutdown. Added two tests for isStopping() to break the loops. (#4139) 2017-12-21 20:56:14 +01:00
Jan b7ee607312
Bug fix 3.3/integer overflow when calculating waits in constituent (#4090)
* integer overflow in Constituent could seize operation of Agency

* less likely integer overflow on double conversion

* less likely integer overflow on double conversion

* changed comparison to integer comparison as suggested by @neunhoef
2017-12-19 10:10:05 +01:00
Jan 9c5893e7a7
fix premature unlock (#3802) (#4027)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 18:46:14 +01:00
Jan 7af86685e3
when upgrading from 3.1 LastHeartBeatAcked could also have been missing, when the 3.1 cluster had not run for long enough (#3974) 2017-12-08 17:33:37 +01:00
Jan bec83181be Bug fix 3.3/add security check end with failover (#3911)
* Add security check in AgencyComm::sendWithFailover.

* some cleanup

* added some more tests

* add typeName() to AgencyCommTransaction to make the transaction type printable in debug messages

* improve debuggability
2017-12-07 10:33:59 +01:00
Jan ba729150bf backporting inquire fixes (#3920) 2017-12-07 10:27:41 +01:00
Kaveh Vahedipour f7b4150b64 no clientId anymore in send/sendWithFailOver SPIs (#3819) 2017-11-28 10:47:36 +01:00
Kaveh Vahedipour c300eee5f0 minor (#3813) 2017-11-27 18:22:13 +01:00
Kaveh Vahedipour 27cd691bbf Bug fix/agencycomm validate methods broken (#3805) 2017-11-27 14:18:25 +01:00
Kaveh Vahedipour 2beaef41ff Bug fix/agencycomm validate methods broken (#3784) 2017-11-24 10:31:07 +01:00
Simon Grätzer 987daca85b Handle invalid endpoints in AgencyComm (#3729) 2017-11-17 16:35:59 +01:00
Kaveh Vahedipour 7b80deb5cc Fixed object assignment operator for agency's key value store (#3701)
* Fixed object assignment operator for agency's key value store
* Node's toJson is now actually toJson. getString should be used for string extractions
* adjust agency's documentation (clarify precondition)
2017-11-17 15:49:40 +01:00