1
0
Fork 0
Commit Graph

558 Commits

Author SHA1 Message Date
Max Neunhöffer f2f9fe4e3d
Repair arangorestore. (#7076)
* Improve logging on coordinator when doing `arangorestore`.

* Return more error information in `mergeResults`.

* Longer timeout for communication coordinator -> leader for writes.

This is taking into account possible write stops from followers needed
to get in sync.

* Fix compilation.

* Get rid of numbers in exception log messages.

* Fix a typo.
2018-10-31 14:39:20 +01:00
Max Neunhoeffer 3d1c5ca6a2
Revert "Improve logging on coordinator when doing `arangorestore`."
This reverts commit 1ce781c2a1.
2018-10-25 13:35:32 +02:00
Max Neunhoeffer 1ce781c2a1
Improve logging on coordinator when doing `arangorestore`. 2018-10-25 12:07:18 +02:00
Dan Larkin-York 069b456217 [3.3] Backport #6539. (#6569) 2018-09-24 13:00:08 +02:00
Simon fed9a45b20 Properly check syncer erros, catch more exceptions (#6521) 2018-09-17 16:39:10 +02:00
Michael Hackstein 0a9005e890
Bug fix/dump restore under authentication (#6156)
* Let authentication dump test execute the dump under authentication as well.

* Added a test for active-failover to ensure users are kept during failover scenarios

* Added temporary rights for Admin users on dump and restore for indexes and inventory.

* Updated changelog

* Added a test for replication of _users in single-server replication case

* Added a regression test in replication that validates users are replicated under authentication

* Execute replication ongoing tests under authentication
2018-08-14 22:18:26 +02:00
Simon f5116c4bc3 support truncate marker, add intermediate commit hints (#6125) 2018-08-09 15:13:06 +02:00
Wilfried Goesgens ccc339ca46 Feature 3.3/forward compatible sync (#6001) 2018-07-30 08:55:26 +02:00
Jan Christoph Uhde d536109760 propagate replication error messages (#5785) 2018-07-09 09:02:38 +02:00
Heiko fe0b88fb08 replication ui requested changes (#5562) 2018-06-08 11:19:41 +02:00
Heiko 51caa460f2 Feature/replication ui (#5282) 2018-06-07 15:54:27 +02:00
Tobias Gödderz a391088dbf [3.3] Fix inconsistent distributeShardsLike job (#5300) 2018-05-17 12:07:20 +02:00
Simon c954841a4f Backport supervision for active failover job + testsuite (#5181) 2018-04-23 16:38:11 +02:00
Michael Hackstein 27ac691447
Bug fix 3.3/drop smart collection restore (#5130)
* Added CHANGELOG

* Moved restore handler to use CollectionMethods like all other places.

* Replaced ensureIndex in restore case with helper method as well.
2018-04-18 12:14:43 +02:00
Michael Hackstein 5decb66d01
fix dump&restore of smart graphs (#5065)
* Added tests for dump-restore of SmartGraphs

* Arangosh will now expose isSmart and the smartGraphAttribute on properties

* RestReplicationHandler will now ignore smart-graph collections unless you execute it with force

* Added changelog

* Reactivated original mmfiles dump/restore test

* Skip hidden smart graph collections in arangodump

* Do not dump shadowCollections metadata of smart edge collections

* Cluster optimizer rules for soerted gather nodes now handle virtual edge collections correctly

* Added a dump/restore tests for smartgraphs in rocksdb as well

* Deactivated checks for writesExecuted statistics in dump/restore tests for smartgraphs mmfiles

* Really exclude shadowCollections

* Reduced loglevel

* Added tests

* Don't change single-server behaviour

* Fix tests for omitted shadowCollections and hidden collections

* Activated statistics in MMFIles dump test again and included isEnterprise in rocksdb dump test

* A modification node can now disableStatistics, which means it does not contribute to query->extras() this is only relevant in SmartGraph case so far.

* Added a test to dump&restore satellite collections

* Bugfix: restore satellite collections properly

* Added regression test for internal issue #2237

* Fix bug #2237

* Updated CHANGELOG

* Copied dump/restore tests to rockdsb

* Removed enterprise test

* Added inline comment for smart-edge collections in optimizer rules

* Removed duplicate CHANGELOG entry

* Simplified removal of shadowCollections
2018-04-10 17:21:03 +02:00
Jan 12decb1c29
increase timeout for index creation (#4914) 2018-03-21 18:17:26 +01:00
Simon 38a99d8f65 Fixing issues related to missing auth rights (#4753) 2018-03-05 13:41:22 +01:00
Simon 1d57a46168 Refactor Authentication Layer (3.3) (#4588)
* added tests for revokeCollection and revokeDatabase

* optimized user permission test

* ui selection bugfix

* fixed ldap ui login

* login view

* Authentication refactoring

* localstorage fallback if user config is not available

* Fixing permission resolution test

* Adding missing import

* local storage queries now supported

* disabled collection task check for ldap

* added internal ldapEnabled function and ldap config to the ui

* more db creation tests

* removed console logs

* render fix

* Various authentication related fixes

* exec ldap test also for cluster

* Adding support to refresh user rights from external auth sources

* ldap test howto comment

* Handling roles more correctly

* jwt

* login view

* First part of rework of LDAP documentation.

* test roles in a ldap environment

* Changing role handling

* Finish revision of the LDAP chapter in the manual.

* Fixing user header

* Fixing some slight issues with LDAP users and roles

* Removing unused code

* Removing unused code

* added ldap test

* fixing a bug in restuserhandler

* more ldap tests

* ldap tests

* ldap tests

* optimized ldap testing, added cluster support, fixed some auth tests

* ldap cleanup test

* ldap tests

* auth tests

* ldap test

* Changing permission defaults

* revert change

* updated user helper test

* rm of try catch block connection

* Removing FeatureCacheFeature

* Changed permission resolution according to discussion

* updated the docs to clarify the permission resolution and the intricacies of LDAP users

* Fixing wrong permissions check in handling of PUT in ResUserHandler

* Using revision ID when replacing users

* Adding basic replace test

* tests

* Fixing some outstanding issues

* Fixing test setup, optimizing some stuff

* Fixing permission resolution rules, testsuite setup, etc

* Fix deadlock

* Adding error message for keyspace, slightly changing test setup

* Removing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

* Fixing an assert

* Cleanup, adding changelog entry

* fix typo

* Fixing dump_authentication test

* improvements found during code review

* oops

* updated CHANGELOG

* Fixing broken handling, disallowing adding of local users when disabled

* added tests for ldap search mode

* Fixing testInvalidGrants

(cherry picked from commit bc7ea2aaa29a9ed0974898f487e8a318f24912f1)

* Removing undefined auth level externally

(cherry picked from commit 70859f43ae6fd694fdbf70f669fbfdafc58e7913)

* Fixing previous commit

(cherry picked from commit 2fbcffd2ed657862ef9fb5e6d45201a6ec8ada69)

* more use of sessionstorage

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* fix LDAP test invocation

* Added roles transformation to ldap test suite

* Fix compilation of community version.

* Imrpved the ldap testsuites by unifying their options

* fix permission problems for system collections

* Improved LDAP configuration documentation.

* Grunt.

* fixed some ro/rw display issues

* fixed some ro/rw display issues part 2

* grunt build

* bump version number

* Fixed typos in LDAP manual
2018-02-28 13:24:18 +01:00
Simon e96e899fd3 Active Failover for Foxx Services (3.3) (#4593) 2018-02-15 09:36:25 +01:00
Jan 6b9f31f1eb
add temporary rights that are required to make a slave (#4513) 2018-02-08 09:33:34 +01:00
Jan 4c6337611b
properly unlock after restore-indexes (#4369) 2018-01-25 15:56:13 +01:00
Jan 75953d69e8 do not reuse collection ids when restoring (#4218) 2018-01-03 23:48:47 +01:00
Max Neunhöffer f58301ed8c
clusterInventory: distinguish between isReady and allInSync. (#4214) 2018-01-03 22:18:15 +01:00
Jan 9c5893e7a7
fix premature unlock (#3802) (#4027)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 18:46:14 +01:00
Simon Grätzer cfd500c53c Fixing an issue with intermediate commits (#4006)
* Fixing an issue with intermediate commits

(cherry picked from commit 7f61fb9be0f22904be21e9dd26c3870035a5b318)

* Only check collection counts without intermediate commits

* Fixing assert
2017-12-12 23:15:41 +01:00
Simon Grätzer 0e485f7441 Fixing collection name collection handling in Syncer (#3710) 2017-11-17 16:36:57 +01:00
Jan b4f6ee9273 Feature/improved index api for unique constraints and replication (#3715) 2017-11-16 21:02:01 +01:00
Jan fce593b724 removed `--recycle-ids` option for arangorestore (#3713) 2017-11-16 14:25:54 +01:00
Simon Grätzer 161d7e8b5c Do not grant rights on system collections (#3627) 2017-11-10 18:01:17 +01:00
Jan 356751bd12 fix a non-deterministic replication test (#3623) 2017-11-09 09:10:19 +01:00
m0ppers b552853909 Feature/optional replication factor enforcement (#3570) 2017-11-02 21:41:25 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Simon Grätzer 05d6a9a9f4 Reuse buffers for HttpResponse (#3433) 2017-10-30 17:30:36 +01:00
Max Neunhöffer 5e2f17c04f Fast path for get-in-sync of empty shards. (#3470)
* Fast path for get-in-sync of empty shards.

* Fix short cut in synchronization.
2017-10-26 09:55:00 +02:00
Simon Grätzer 09840239c9 Fixed wait for sync in mmfiles (#3478) 2017-10-23 12:36:50 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Max Neunhöffer f3acea797b Feature/cluster inventory version (#3152)
* Get rid of a compiler warning in community edition.

* Teach /_api/replication/clusterInventory to report Plan/Version and readiness.

This is first implemented in the ClusterInfo library.
Then the clusterInventory code uses it and checks readiness.
Readiness of a collection means that it is created and all shards
and all replications have been created and are in sync.
2017-08-30 13:34:23 +02:00
Michael Hackstein 7ab117f5f8 Bug fix/dump restore auth (#2917)
* Added a backup test suite. This suite is supposed to entirely drop an ArangoDB _system and restore it into a fresh one. This also includes system collections.

* Added more test cases for backup suite. Now tests several authorization/rights cenarios

* Fixed RestReplication Handlers to restore _user collections Properly.

* Updated Changelog

* Added special handling of _users in Restore for MMFiles as well.

* Added JWT secret for cluster execution of this test, also added JWT secret to shutdown call
2017-08-09 12:59:51 +02:00
Dan Larkin af8e1889a1 Moved MMFiles replication handler. 2017-04-18 15:35:45 -04:00
jsteemann 6cd6ed192d Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-04-18 12:53:37 +02:00
Andreas Streichardt 4d7e6a8fba Properly generate an error message during import 2017-04-18 12:05:31 +02:00
Michael Hackstein 1087065ea5 Merge branch 'devel' of github.com:arangodb/arangodb into engine-api 2017-04-13 17:34:25 +02:00
Max Neunhoeffer 53b3a73486 Add a timeout option to <collection>.drop().
This is in particular used to allow the Schmutz to make progress
in the case of a long running transaction.
2017-04-07 15:52:19 +02:00
Dan Larkin f056aeee42 Fixed unloaded collection status reporting issue. 2017-04-06 16:21:51 -04:00
Jan Christoph Uhde 95007fe28e Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api-arango-result
* 'engine-api' of https://github.com/arangodb/arangodb: (46 commits)
  Merged RocksDBPathBasedIndex into RocksDBVPackIndex.
  fix handling of externals
  fix toArray()
  fix returning documents
  Added hash-, skip list, persistent indexes
  harmonize filtering - filter on full filename.
  mini modifications
  standardize starting of servers for replication tests
  Edge index compliant with design doc
  another fix for #2404
  Moved „removeLargeRange“ method to commons
  Added length byte to IndexValue keys.
  fixed issue #2404
  Added drop to primary index
  Added drop() to indexes
  Fixes a coordinator crash in Clustered Traversal. In case of failover during query planning.
  Fixed edge index
  Expand on CORS documentation
  Let ArangoDB handle the allow-credentials header
  make the authentication tests run on directories
  ...
2017-03-30 19:46:17 +02:00
jsteemann d563eb432f Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-03-30 15:50:53 +02:00
jsteemann ddb83c8075 fixed issue #2404 2017-03-30 14:18:38 +02:00
Jan Christoph Uhde b83ae2ab82 refactor some code to make use of arangodb::Result 2017-03-30 09:39:21 +02:00
jsteemann 2aa2fd65a4 more startup 2017-03-24 14:20:51 +01:00