1
0
Fork 0
Commit Graph

118 Commits

Author SHA1 Message Date
Jan 455cccefbb Bug fix 3.5/honor sharding restrictions (#10140)
* honor sharding restrictions also when creating a graph with existing
collections

* add test

* additional validation for startup parameters

* fixed logIDs
2019-10-02 18:42:53 +03:00
Jan 13dae8cace Feature 3.5/sharding config (#10111)
* added options

`--cluster.max-number-of-shards`
`--cluster.default-replication-factor`
`--cluster.min-replication-factor`
`--cluster.max-replication-factor`

* fix creating satellite collections

* added CHANGELOG entry and tests

* guard against invalid replicationFactor changes when changing properties

* fix validation

* remove stray print

* Update arangod/Cluster/ClusterFeature.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>
2019-10-01 16:43:57 +03:00
Max Neunhöffer 4da1a6afdf
Coordinators do not unregister at every shutdown. (#9134)
* Coordinators do not unregister at every shutdown.

Instead they create a new short name with every start.
This is needed for the transactions.

* Always new short id for coordinators. Never for DBServers!
2019-05-29 22:55:22 +02:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Dan Larkin-York 4459cde000 Make coordinator short ID transient (#8234) 2019-03-01 09:48:36 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Wilfried Goesgens c0f9e8125a Bug fix/allow tcp connection to finish (#7635) 2018-12-10 10:38:34 +01:00
Jan 5bae3742e5
Feature/internal 3306 (#7683) 2018-12-06 16:19:28 +01:00
Jan a5db298c92
fix buffer overrun, remove unused variable (#7302) 2018-11-13 14:18:50 +01:00
Max Neunhöffer a74330250f
Port bug fix 3.4/cluster comm threads start stop (#6939) to devel. (#7253)
* Start ClusterComm threads in `ClusterFeature::start`. Stop ClusterComm threads in `ClusterFeature::stop`.

* Do not free objects in `Scheduler::shutdown`. Let the `unique_ptr` do their job. Stop ClusterComm threads in `ClusterFeature::stop`, but free instance in `ClusterFeature::unprepare`.

* `io_context` may contains lambdas that hold `shared_ptr`s to `Tasks` the required a functional `VocBase` in their destructor.
2018-11-07 21:42:34 +01:00
Simon cb4c07e0ed Replace engine equality feature (#6931)
* replace engine equality feature

* remove pointless code
2018-10-17 14:41:47 +02:00
Dan Larkin-York 1f63f16396 Move some logging off of general topic. 2018-10-01 13:28:11 -04:00
Max Neunhöffer 84735955ea Add advertised endpoints. (#6104) 2018-09-13 16:30:55 +02:00
Jan 5873f63a72
Bug fix/fixes 2908 (#6279) 2018-08-31 17:26:54 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Kaveh Vahedipour 54cc026d34 servers should be retrying registration until successful (#5919) 2018-08-17 10:57:45 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Jan 4d4135d25c
Feature/add dbserver as an alias for primary (#6072)
* add "DBSERVER" as an alias for "PRIMARY"

This allows specifying the value "DBSERVER" for `--cluster.my-role`.
"DBSERVER" is only treated as an alias for "PRIMARY", because several
other parts of the code and APIs use the string "PRIMARY".
Changing these from "PRIMARY" to "DBSERVER" would make the change
downwards-incompatible, which we do not want.

The downside of this alias-only solution is that even when specifying
a role value of "DBSERVER", the server will still report its role as
"PRIMARY", which may be a bit confusing. The server will also generate
its id as "PRMR-XXXX" as before:

    2018-08-03T15:23:09Z [9584] INFO {cluster} Starting up with role PRIMARY
    2018-08-03T15:23:09Z [9584] INFO {cluster} Cluster feature is turned on. Agency version: {"server":"arango","version":"3.4.devel","license":"enterprise"}, Agency endpoints: http+tcp://[::]:4001, server id: 'PRMR-f655b728-4cea-44ac-88e9-8b34baa80958', internal address: tcp://[::1]:8629, role: PRIMARY

* adjusted documentation to use "DBSERVER" instead of "PRIMARY"

* api doc

- secondary role not used anymore. stated.
- primary database is not clear. replaced with dbserver
- brief referenced only dbserver and coordinator - better to provide wider description, in line with what is described below, as other roles can be returned

* typo

* typo

* added starting from 3.4

* additional warning

* cited in the release note
2018-08-06 17:20:50 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York b2db010969 Register version and storage engine info in agency to facilitate rolling upgrades (#6061) 2018-08-03 09:46:04 +02:00
jsteemann 36f05c07e0 cleanup of server options 2018-07-16 21:38:35 +02:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
Max Neunhöffer 014c3f7f53
Only load Plan and Current in ClusterInfo when actually needed. (#5649)
* Only update Plan and Current from Agency if not already done.
* Add read protection for getPlanVersion and getCurrentVersion.
* Add a further check to loadPlan and loadCurrent.
* Fix tests to new behaviour.
* Try to increase Plan/Version and Current/Version with every change.
* Add two more increments of Plan/Version
* Add missing increments in tests for Plan/Version.
* Add changelog entry.
2018-07-16 12:20:13 +02:00
Simon 17b1a2aafb Rest middleware refactoring (#5332) 2018-05-14 17:43:10 +02:00
Simon a1416e1067 Make v8 optional on startup (#5220) 2018-04-30 12:48:57 +02:00
Matthew Von-Maszewski a84f7805ad Feature/mv thread death logging (#5111)
* Initial low level interface for thread crash reporting (and management).
* Add a member version of isClusterRole()
* isolate heartbeat thread creation to new StartHeartbeatThread().  create heartbeat thread even if not a cluster or if an agent.
* update runDBServer() and runCoordinator() to shutdown more quickly by polling isStopping() at additional locations.
* copying updates from different branch / PR
* basic thread crash logging.  Not yet tied into Agency arangod or have any specific threads posting crashes
* make Supervision thread a CriticalThread
* sandwich CriticalThread between Thread and other classes to create long term, repeating thread crash reporting.
* restore code lost upon branch update relating to new startHeartbeatThread() function
* add CriticalThread.cpp to build
* add new runAgentServer() function to loop for Agents.  Make Heartbeat thread derive from CriticalThread.
* remove debug line
2018-04-23 15:50:14 +02:00
Jan 2b84348b77
remove call to requiresElevatedPrivileges with default value (#5172) 2018-04-23 11:28:24 +02:00
Simon 8be273efb8 Replication cleanup (#5105) 2018-04-17 08:17:42 +02:00
jsteemann 690dd2186d remove unused instance variable 2018-04-13 12:34:31 +02:00
Jan 76dcd6ded5
added option `--cluster.require-persisted-id` (#5001) 2018-04-13 11:08:49 +02:00
Jan 7cb115a1a9
remove option `--cluster.my-local-info` (#4999) 2018-04-03 17:34:08 +02:00
Jan cd219bdfa1
increase timeout for index creation (#4915) 2018-03-21 18:17:38 +01:00
Simon 345fc3c0b7 Refactor Authentication Layer (devel) (#4592)
* Cherry Picking LDAP changes

* Adding missing merges

* Fixing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

(cherry picked from commit 686d28a779)

* Fixing wrong assert

* Adding changelog entry, making requested changes from code review

* Fixing dump_authentication, fix typos

* improvements found during code review

* oops

* more use of sessionstorage

* fix tests

* Fixing broken handling, disallowing adding of local users when disabled

* Fixing testInvalidGrants

* Removing undefined auth level externally

* Fixing previous commit

* added tests for ldap search mode

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* forward port changes from 3.3

* added generated files

* forward port missing changes for web UI

* added generated files

* added generated files
2018-02-28 13:24:28 +01:00
Matthew Von-Maszewski e6f7282e03 Make chooseTimeout() dynamic (#3996)
* add parameter to increase timeouts per 4096 size of total request package

* remove log output

* change initial timeout math from integer to double ... avoids reviewer confusion.
2017-12-15 18:25:18 +01:00
Jan 282be208cc
remove TRI_usleep and TRI_sleep, and use std::this_thread::sleep_for … (#3817) 2017-12-06 18:43:49 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Max Neunhöffer 9a2385b941 Add host id detection and show in /_admin/cluster/Health. (#3389) 2017-10-11 12:42:44 +02:00
Jan 1ace247273 Bug fix/scheduling et al (#3161)
* added V8 context lifetime control options `--javascript.v8-contexts-max-invocations` and `--javascript.v8-contexts-max-age`

* make thread scheduling take into account most of the tasks dispatched via the io service
2017-08-30 10:40:02 +02:00
Frank Celler bbe7484521 Feature/auth context (#2704)
* added read-only users
2017-07-02 23:15:57 +02:00
Kaveh Vahedipour c1abc0333d cluster documentation varnish (#2553) 2017-06-12 19:02:11 +02:00
Andreas Streichardt 749f4b980b Revert "Revert "Response can always be a nullptr""
This reverts commit c3c76e9a8f.
2017-06-08 15:56:01 +02:00
Andreas Streichardt 7250ba29ee Disable communicator and abort all requests when shutting down 2017-06-08 11:58:15 +02:00
Max Neunhoeffer ad5652ad1d Make two obsolete options hidden. 2017-05-24 23:33:55 +02:00
Max Neunhoeffer cbdf9563cb Add configuration option to multiply synchronous replication timeout. 2017-05-22 15:34:15 +02:00
jsteemann 68b4b2f393 fix shutdown order 2017-05-11 20:59:36 +02:00
Andreas Streichardt 74b721481c Wait for cluster sync when creating collections 2017-04-27 17:46:41 +02:00
Kaveh Vahedipour 0d96ba4f4b merge from 3.1 2017-04-26 12:06:43 +02:00
Andreas Streichardt dad5a1429e Add waitForSyncReplication as a _create() option 2017-04-26 09:57:40 +02:00
Kaveh Vahedipour 6ef775e52f fix bug with ioService gone before hearbeatthread 2017-04-25 08:54:33 +02:00