1
0
Fork 0
Commit Graph

221 Commits

Author SHA1 Message Date
Tobias Gödderz 917ad26d0d [3.3] Bug fix 3.3/allow agency ops in active failover (#9961)
* Backported #9881

* Fixed merge
2019-10-21 16:07:12 +03:00
Tobias Gödderz 9acdeb6ca3 [3.3] Foxx queue jobs hanging after Foxxmaster crash (#7936)
* Fixed bug where the Foxxmaster doesn't reset jobs after a crash when it should, or a non-master coordinator removes jobs in progress during startup

* Added a regression test

* Updated CHANGELOG

* Fixed non-maintainer compile
2019-01-14 16:09:36 +01:00
Frank Celler ec27adc860 big reformat 2018-12-26 01:11:59 +01:00
Lars Maier 3404e554b7 [3.3] Feature/engine version added to agent configuration (#7481) (#7525)
* agents' is obtained from leader's configuration
* corrections in Supervision for advertised endpoints
* change log
* Updated Documentation for cluster/health.
* Unified naming convention.
* Fixed missing update of volatile fields.
* Set version in right order.
* Removed debug output.
* Fixed jslint - missing ;
2018-11-29 14:26:03 +01:00
Simon Grätzer eb0fb254a1
Copy installation files on startup 2018-09-26 14:09:33 +02:00
Tobias Gödderz a391088dbf [3.3] Fix inconsistent distributeShardsLike job (#5300) 2018-05-17 12:07:20 +02:00
Kaveh Vahedipour 507418d9a4 stop supervision on demand (#5109)
* stop supervision on demand
* adding tests
* Correct an error message.
2018-04-20 11:58:47 +02:00
Jan 29be817cf8
fixed internal issue 2102: Segfaults in arangod console when using ag… (#4875)
* fixed internal issue 2102: Segfaults in arangod console when using agency functions in single server mode

* make more ops fail when invoking them in non-cluster mode
2018-03-20 11:21:42 +01:00
Simon 1d57a46168 Refactor Authentication Layer (3.3) (#4588)
* added tests for revokeCollection and revokeDatabase

* optimized user permission test

* ui selection bugfix

* fixed ldap ui login

* login view

* Authentication refactoring

* localstorage fallback if user config is not available

* Fixing permission resolution test

* Adding missing import

* local storage queries now supported

* disabled collection task check for ldap

* added internal ldapEnabled function and ldap config to the ui

* more db creation tests

* removed console logs

* render fix

* Various authentication related fixes

* exec ldap test also for cluster

* Adding support to refresh user rights from external auth sources

* ldap test howto comment

* Handling roles more correctly

* jwt

* login view

* First part of rework of LDAP documentation.

* test roles in a ldap environment

* Changing role handling

* Finish revision of the LDAP chapter in the manual.

* Fixing user header

* Fixing some slight issues with LDAP users and roles

* Removing unused code

* Removing unused code

* added ldap test

* fixing a bug in restuserhandler

* more ldap tests

* ldap tests

* ldap tests

* optimized ldap testing, added cluster support, fixed some auth tests

* ldap cleanup test

* ldap tests

* auth tests

* ldap test

* Changing permission defaults

* revert change

* updated user helper test

* rm of try catch block connection

* Removing FeatureCacheFeature

* Changed permission resolution according to discussion

* updated the docs to clarify the permission resolution and the intricacies of LDAP users

* Fixing wrong permissions check in handling of PUT in ResUserHandler

* Using revision ID when replacing users

* Adding basic replace test

* tests

* Fixing some outstanding issues

* Fixing test setup, optimizing some stuff

* Fixing permission resolution rules, testsuite setup, etc

* Fix deadlock

* Adding error message for keyspace, slightly changing test setup

* Removing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

* Fixing an assert

* Cleanup, adding changelog entry

* fix typo

* Fixing dump_authentication test

* improvements found during code review

* oops

* updated CHANGELOG

* Fixing broken handling, disallowing adding of local users when disabled

* added tests for ldap search mode

* Fixing testInvalidGrants

(cherry picked from commit bc7ea2aaa29a9ed0974898f487e8a318f24912f1)

* Removing undefined auth level externally

(cherry picked from commit 70859f43ae6fd694fdbf70f669fbfdafc58e7913)

* Fixing previous commit

(cherry picked from commit 2fbcffd2ed657862ef9fb5e6d45201a6ec8ada69)

* more use of sessionstorage

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* fix LDAP test invocation

* Added roles transformation to ldap test suite

* Fix compilation of community version.

* Imrpved the ldap testsuites by unifying their options

* fix permission problems for system collections

* Improved LDAP configuration documentation.

* Grunt.

* fixed some ro/rw display issues

* fixed some ro/rw display issues part 2

* grunt build

* bump version number

* Fixed typos in LDAP manual
2018-02-28 13:24:18 +01:00
Simon e96e899fd3 Active Failover for Foxx Services (3.3) (#4593) 2018-02-15 09:36:25 +01:00
Heiko a16fd972eb optimized shard distribution api and ui (#4312) 2018-01-16 20:54:07 +01:00
Kaveh Vahedipour 27cd691bbf Bug fix/agencycomm validate methods broken (#3805) 2017-11-27 14:18:25 +01:00
Michael Hackstein 5c633f9fae Bug fix/speedup shard distribution (#3645)
* Added a more sophisticated test for shardDistribution format

* Updated shard distribution test to use request instead of download

* Added a cxx reporter for the shard distribuation. WIP

* Added some virtual functions/classes for Mocking

* Added a unittest for the new CXX ShardDistribution Reporter.

* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red

* Shard distribution now uses the cxx variant

* The ShardDistribution reporter now tries to execute count on the shards

* Updated changelog

* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
2017-11-10 15:17:08 +01:00
Jan 7840d3f824 Bug fix/fixes 1810 (#3460)
* improve error reporting in RestAqlHandler

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
2017-10-19 11:28:01 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Wilfried Goesgens 88fd685296 Bug fix/fix shard state restore display (#3385)
* don't add  items into the array if we have an empty list member.

* we mustn't display 100% here, since its a rounding error. If, the checkmark would be set.

* rather use string cutting to round so we don't get into the 100% trap.
2017-10-17 14:08:07 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Max Neunhöffer f3acea797b Feature/cluster inventory version (#3152)
* Get rid of a compiler warning in community edition.

* Teach /_api/replication/clusterInventory to report Plan/Version and readiness.

This is first implemented in the ClusterInfo library.
Then the clusterInventory code uses it and checks readiness.
Readiness of a collection means that it is created and all shards
and all replications have been created and are in sync.
2017-08-30 13:34:23 +02:00
Kaveh Vahedipour 1d1e0f5a50 Feature/cluster id and extended health (#3046)
* added unique id to cluster, added access to Health

* added agents to health api

* added agents to health api

* added agents to health api

* transaction information for api

* agents listed like other servers

* missing line through merge conflict
2017-08-18 11:13:23 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler bbe7484521 Feature/auth context (#2704)
* added read-only users
2017-07-02 23:15:57 +02:00
Kaveh Vahedipour c1abc0333d cluster documentation varnish (#2553) 2017-06-12 19:02:11 +02:00
jsteemann 8bc6b3d3a1 removed default parameters 2017-05-18 15:06:19 +02:00
Alan Plum 5d017d733c Improve Foxx cluster resilience (#2479)
* Improve Foxx cluster resilience

Fixes #2083
Fixes #2384
Fixes #2408
Addresses #1892

* Port old Foxx API

* Implement single-file services

* Add console.errorStack/warnStack/infoStack helpers

* Simplify serviceInfo validation

* Extract github/upload logic into Aardvark and old FM API
* Move generator logic into Aardvark
* Move zip/js buffer logic into FM core

* Add Foxxmanager tests

* Send empty response when no README

* Disambiguate script arg format

Historically we allow passing an array of positional arguments or an arbitrary first argument.
This is surprising behaviour, so we should just always treat the value as a first argument.

* Rebuild bundle in development mode

* Nicer HTTP docs formatting

* Create Foxx HTTP docs

* Simplify service upload handling

* Remove inline swagger docs

* Implement public download route

* Consistency

* Rebuild aardvark

* Move bundle route into /_api/foxx/_local

* Rebuild Swagger API docs

* Add changes to CHANGELOG

* More docs
2017-05-16 17:41:29 +02:00
Max Neunhoeffer f614439416 Squashed commit of the following:
commit 19a7210119cc284af64251b202f690ab62bf615c
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 23:20:10 2017 +0200

    Allow access to /_api/user/ without database check in VST as in HTTP.

    This is a try to fix #2465

commit f973c1335652540174bf6b78df42290bded357da
Merge: 30bdc4c 365de45
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 21:32:15 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 30bdc4ca38e028d58cebea8268683d9f94e87ad7
Merge: 14acaea 086f6a2
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 14:52:23 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 14acaea8cb5fff2c1c8c27eee32294bfd612f629
Merge: 5bd0e79 fb088d3
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:40:29 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 5bd0e79ab053447f94f2fd959a33b1e15302b6a8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:36:23 2017 +0200

    Fix bug in authentication in VST

commit 9a999ce5e629aee5816a4f61b194b086d11fd169
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:24:35 2017 +0200

    Open up endpoints /_open/* in VST without being authenticated

commit 32f62db42af6d5bfc24214ebb4ca7c1fb473b55e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:11:31 2017 +0200

    Simplify logic by not using boost::optional.

commit 925ce2f7b890c6f14205fd3e98553939d89bec61
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:11:20 2017 +0200

    Add JWT case for authentication for VST

commit f1d7f67a9fc20c91ca000e9a4dad91e0f50f1652
Merge: 040ace3 7a9ccc9
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 08:58:51 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 040ace3e9dce47ddea5f51d29f0153a70d257c9e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 08:23:16 2017 +0200

    Write out chunk buffer using little endian uints

commit 66ad4c0e8d3bc94091664505986b00e7cff39f2f
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 08:12:38 2017 +0200

    Move maxChunkSize query to constructor, remove dead code

commit f7b4c26cc09c6d512362340c6bf6061bbaae61a9
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 17:57:14 2017 +0200

    Do not take protocolVersion from request for VST.

commit c76d6685f9507701715abf882f0ac0e192ead59f
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:20:58 2017 +0200

    Remove dead code for compression which did never work well.

commit b823765b7c041dc5d143f7ab2e13d92f5f42d3eb
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:09:53 2017 +0200

    Fix renaming to make compiler happy.

commit b327830012d146f4f4e2d0f0467dab351fbd3bd0
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:07:36 2017 +0200

    Rename _defaultPortVpp to _defaultPortVst.

commit 5eb98a9e64d702476e8ec098b7506b1a63145d64
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:06:28 2017 +0200

    Rename header guards VPP -> VST.

commit a3e96be26d49d6151898d8d757ede6225ac6abd2
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:05:55 2017 +0200

    Rename VppRequest and VppResponse files to Vst.

commit 1d5f6f196490c3cf9312be5927b967b4abc03b67
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:04:19 2017 +0200

    Change names of header guards VPP -> VST.

commit f004b25fbef75050b34bcd4eae4eb17b8b467230
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:03:19 2017 +0200

    Rename files VppNetwork.h and VstMessage.h

commit c04d3aec19e8f0a8a3b90cd48e7736bff037359d
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 12:59:15 2017 +0200

    Implement correct chunk header for sending out VelocyStream 1.1 messages.

commit 73aeedfbbef24d5929c44bd26c16343480514270
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 01:11:23 2017 +0200

    A lot more renaming VPP -> VST.

commit 2acedc77f16a82251bc4f9e39526184e2a6f0dec
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:49:20 2017 +0200

    Rename header guard.

commit 2ade43946aa5ea0048dab81eed5ffd931ce35ea8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:48:01 2017 +0200

    Rename files VppCommTask.* to VstCommTask.*

commit 0ffcda02a51cbad35e5ec06117d397aac37323cd
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:46:31 2017 +0200

    Rename class VppCommTask to VstCommTask.

commit 9478c6c56c995b014b7a14408544b76fa676a317
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:30:47 2017 +0200

    Rename protocol versions to VST in ProtocolVersion enum.

commit 0b69b7add6d7b5155ac71b8257def282ab4a1637
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:29:29 2017 +0200

    Recognize incoming chunk headers in VST/1.1 format.

commit f40173c4fc6dc48613ae67ac3fd810d002c9b6cd
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:11:54 2017 +0200

    Recognise and remember VST/1.0 and VST/1.1 protocol versions.

commit 29c798dc1af2d650c41eb9ed06f635786faf10f6
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:11:23 2017 +0200

    Create protocol version VST/1.1 and disable vst+ prefix for endpoints.
2017-05-12 23:22:22 +02:00
Alan Plum bb6ffd70de
Cluster: expose body as V8 buffer in asyncRequest 2017-04-03 22:20:37 +02:00
Alan Plum a5a3542280
Cluster: expose getFoxxmaster in V8 2017-04-03 22:20:37 +02:00
jsteemann 645889949c move engine-specific V8 functions into storage engine 2017-03-09 12:59:44 +01:00
Michael Hackstein 86c34d1b88 ClusterInfo now uses the general toVPack function of LogicalCollection instead of building everything locally. 2017-02-28 17:05:30 +01:00
Michael Hackstein de31c68539 Moved isVolatile into MMfiles collection. 2017-02-23 14:56:54 +01:00
Jan Christoph Uhde 071137adf6 move doCompact into PhysicalColletion 2017-02-20 10:53:11 +01:00
Jan Christoph Uhde 2d5e74f7f6 move journaize() to PhysicalCollection
MMFilesCollection ctor might be buggy
2017-02-17 20:06:47 +01:00
Andreas Streichardt 39436dac89 Finally remove lock methods 2017-02-10 19:47:31 +01:00
Max Neunhoeffer 883c11ea45 Handle the case that ClusterComm is already shut down gracefully.
This touches every single place where ClusterComm is being used.
2017-02-07 15:31:40 +01:00
Max Neunhoeffer 71dae630c3 Store ClusterComm singleton in shared_ptr. 2017-02-06 15:25:18 +01:00
Max Neunhoeffer 20e2128a44 Fix shardDistribution in case of leader change. 2017-02-06 09:45:00 +01:00
Frank Celler 37805f6bc9 safety check 2017-02-04 13:08:13 +01:00
Frank Celler 602795deea safety check 2017-02-04 13:08:08 +01:00
Kaveh Vahedipour 8509200040 MoveShard with ShortNames 2017-01-26 11:35:44 +01:00
Andreas Streichardt b56621a01b First current tests 2017-01-23 15:29:25 +01:00
Kaveh Vahedipour fe48bcb982 fixed for short names in frontend shard view 2017-01-11 16:31:19 +01:00
Andreas Streichardt 466f932701 First steps to low level replication debugging 2017-01-06 17:19:07 +01:00
Kaveh Vahedipour a0af781630 to lock or not to lock 2017-01-06 14:46:57 +01:00
Kaveh Vahedipour 4ac74301d5 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-01-06 11:10:36 +01:00
Andreas Streichardt dafb5d0724 Allow setInitialized within DEBUG_SYNC_REPL mode 2017-01-06 10:44:35 +01:00
Kaveh Vahedipour e87182d971 more verbose output on failure to create plan 2017-01-06 09:38:37 +01:00
jsteemann 0070b40fbf attempt to fix Windows build 2017-01-04 09:08:13 +01:00
Kaveh Vahedipour f66688c397 visual studio ridiculous once again 2017-01-03 17:45:57 +01:00
Kaveh Vahedipour 9d5a5537ce remove deceased agents from AgencyComm 2017-01-02 17:12:00 +01:00
Kaveh Vahedipour a2ee40d4f3 restarting agents inform rest of their new endpoints 2017-01-02 15:58:38 +01:00