1
0
Fork 0
Commit Graph

177 Commits

Author SHA1 Message Date
Max Neunhöffer 328f46e3d6 This merges hotbackup and atomic-db-creation into 3.5. (#9968)
* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.
2019-09-11 13:13:54 +03:00
Tobias Gödderz da6e9da820 Measure time a request takes to get executed by the scheduler (#9224) 2019-06-18 17:34:33 +02:00
Markus Pfeiffer 422fd6057e Fix shebangs in scripts so that they work on non-linuxey systems (#9254) 2019-06-13 19:42:43 +02:00
Max Neunhöffer 80bfb85695
Port agency performance tuning for many shards to devel. (#8647)
* Port agency performance tuning for many shards to devel.
* Add more IDs to LOG_TOPIC calls.
* Even more IDs for LOG_TOPIC.
* Fix a duplicate LOG_TOPIC ID.
* Fix an old merging bug in devel.
* Don't hesitate between phases one and two for small clusters.
2019-04-11 11:14:56 +02:00
Simon 348bc5d2cb Revert change in scripts/startLocalCluster.sh (#8527) 2019-03-22 14:26:48 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Kaveh Vahedipour 94784e5639 startLocalCluster in rr debugging mode cannot be started with --console on agents / db servers (#7776) 2018-12-16 16:25:55 +01:00
Michael Hackstein 16d0874da5
Bug fix/synchronous replication catchup (#7146)
* merged fixes from 3.4

* odd fix

* Bug fix 3.4/sync repl release thread (#6784)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Bug fix 3.4/shorter foot in door (#7084)

* Implement `syncCollectionCatchup` in DatabaseTailingSyncer.

First stab, might not even compile.

* Fixed a typo.

* Fix a typo and a compilation problem.

* Further compilation fix.

* Implement two stage catchup.

* Two small corrections.

* Unified error messages in Synchronize shard job.

* Improved a code comment.

* Fixed autocasting bool->double and double->bool issue. That is truely one of the best features ever invented... </irony>

* Renamed doHardLock => toSoftLockOnly and inverted default value

* Merged soft/hard foot logic with Transaction splits

* Use scopeguards to cancel readlocks

* Bug fix 3.4/sync replication allow soft and hard lock (#6864)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Implemented optional 'doHardLock' parameter in the replication acquire read-lock call. A hard-lock guarntees to stop all writes, a soft-lock may not.

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Renamed doHardLock => toSoftLockOnly and inverted default value
2018-11-23 16:16:34 +01:00
Dan Larkin-York b447f98a7a Upgrade old MMFiles collections for compatibility with ArangoSearch. (#7248) 2018-11-07 15:52:37 +01:00
Tobias Gödderz 129d075d72 Added --rr parameter to startLocalCluster.sh (#6906) 2018-10-16 14:00:16 +02:00
Jan 9edb9dedef
fix logging of stdout for agency nodes (#6376)
also capture stderr as originally intended (was not logged)
2018-09-05 13:48:09 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Jan 4d4135d25c
Feature/add dbserver as an alias for primary (#6072)
* add "DBSERVER" as an alias for "PRIMARY"

This allows specifying the value "DBSERVER" for `--cluster.my-role`.
"DBSERVER" is only treated as an alias for "PRIMARY", because several
other parts of the code and APIs use the string "PRIMARY".
Changing these from "PRIMARY" to "DBSERVER" would make the change
downwards-incompatible, which we do not want.

The downside of this alias-only solution is that even when specifying
a role value of "DBSERVER", the server will still report its role as
"PRIMARY", which may be a bit confusing. The server will also generate
its id as "PRMR-XXXX" as before:

    2018-08-03T15:23:09Z [9584] INFO {cluster} Starting up with role PRIMARY
    2018-08-03T15:23:09Z [9584] INFO {cluster} Cluster feature is turned on. Agency version: {"server":"arango","version":"3.4.devel","license":"enterprise"}, Agency endpoints: http+tcp://[::]:4001, server id: 'PRMR-f655b728-4cea-44ac-88e9-8b34baa80958', internal address: tcp://[::1]:8629, role: PRIMARY

* adjusted documentation to use "DBSERVER" instead of "PRIMARY"

* api doc

- secondary role not used anymore. stated.
- primary database is not clear. replaced with dbserver
- brief referenced only dbserver and coordinator - better to provide wider description, in line with what is described below, as other roles can be returned

* typo

* typo

* added starting from 3.4

* additional warning

* cited in the release note
2018-08-06 17:20:50 +02:00
Jan 1b9fcb589d
make RocksDB the default engine (#5964) 2018-07-25 11:17:49 +02:00
Kaveh Vahedipour 5b307db85d Better log compaction 2018-07-16 12:09:58 +02:00
Jan d8c887ab61
remove hard-coded thread counts (#5816) 2018-07-10 14:39:09 +02:00
Jan Christoph Uhde 2829b3e77d moves -e to XTERMOPTIONS and adds delay (#5297)
* moves -e to XTERMOPTIONS and adds delay
* add delay only when rr is used
2018-06-01 10:07:59 +02:00
Simon a1416e1067 Make v8 optional on startup (#5220) 2018-04-30 12:48:57 +02:00
Jan 73bce34174
potentially fix send request timeout (#4026) 2017-12-13 17:54:12 +01:00
jsteemann 0aae349d1d do not restrict to 5 threads 2017-11-17 17:28:46 +01:00
Jan Christoph Uhde 89384dd767 fix startLocalCluster for darwin (#3641) 2017-11-09 20:08:35 +01:00
Jan Christoph Uhde 27460a92e1 Store parameters of startLocalCluster.sh (#3593)
Store parameters of startLocalCluster.sh

This should help you to bring the cluster up again when you
continue to work after some time without investigating the
port numbers.

Restore parameters if no arguments are given
2017-11-08 11:26:54 +01:00
m0ppers 19879ff568 Allow custom address to be set from the outside (#3469) 2017-10-20 15:03:22 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Kaveh Vahedipour 231a360b3b fixes for secondaries 2017-07-11 14:05:51 +02:00
Mark Volmary b5f69f9ca4 Add startLocalCluster.sh option --ongoing-ports 2017-06-09 16:20:42 +02:00
Kaveh Vahedipour 63b5cadcd0 fix startLocalCluster to run everywhere in file system 2017-06-08 14:34:23 +02:00
Kaveh Vahedipour 35492fa51f slice can throw 2017-06-07 16:40:16 +02:00
Max Neunhoeffer e2a40dd161 Set keep size by default to 1000. 2017-06-07 13:16:40 +02:00
Andreas Streichardt 14a65688db Fix startLocalCluster.sh jwt authentication, implement ps1 jwt auth 2017-05-17 16:38:49 +02:00
Kaveh Vahedipour 511fa4036d more smooth local cluster start and shutdown 2017-05-17 11:10:05 +02:00
Kaveh Vahedipour 499ef8d1b9 reduced code lines in startLocalCluster 2017-05-16 13:14:04 +02:00
Kaveh Vahedipour 90cf2becc3 Allow cluster to be started anywhere 2017-05-16 10:14:56 +02:00
Kaveh Vahedipour 6728e829b1 making startlocalcluster more independent 2017-05-15 16:17:56 +02:00
jsteemann 2b4338d85c disable special behavior for RocksDB 2017-05-02 09:04:45 +02:00
Kaveh Vahedipour a823a437b1 correct rocksdb cluster start 2017-04-24 15:34:07 +02:00
Kaveh Vahedipour e559ca63f9 unable to start rocksdb cluster right now 2017-04-24 15:32:16 +02:00
Max Neunhoeffer 4151087273 Fix some duplicated options. 2017-04-24 12:22:29 +02:00
Kaveh Vahedipour 97c9198b4d ipv4 is so yesterday 2017-04-24 11:51:38 +02:00
Kaveh Vahedipour 733d7260eb startRocksdbCluster.sh has gone in favour of a simple flag in startLocalCluster.sh 2017-04-24 11:34:54 +02:00
Kaveh Vahedipour b96f3be332 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-04-24 11:08:15 +02:00
Kaveh Vahedipour a211e5ee83 ipv4 is so yesterday 2017-04-24 11:08:04 +02:00
Max Neunhoeffer 6b67f0a550 Revert "Add -r option for rocksDB engine to startLocalCluster.sh"
This reverts commit 1f842312f5.
2017-04-24 11:07:04 +02:00
Max Neunhoeffer 1f842312f5 Add -r option for rocksDB engine to startLocalCluster.sh 2017-04-24 10:57:06 +02:00
Andreas Streichardt 0bf00036c6 Streamline shutdown and start 2017-03-29 11:55:02 +02:00
Simon Grätzer ad268697e3 Fix some stuff 2017-03-15 16:22:10 +01:00
Simon Grätzer 27c617fe10 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	3rdParty/V8/v8
#	arangod/Transaction/Methods.h
#	arangod/Utils/UserTransaction.h
#	arangod/V8Server/v8-collection.cpp
2017-03-01 14:52:35 +01:00
Andreas Streichardt 1bb8f97773 Fix secondaries 2017-02-13 14:00:19 +01:00
Simon Grätzer edab268572 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/Aql/FunctionDefinitions.cpp
#	arangod/Aql/Functions.h
#	arangod/Utils/ExplicitTransaction.h
2017-02-10 15:21:24 +01:00
Kaveh Vahedipour 29d73b2e9c sendAppendEntries does resonable estimation of follower time needs leading to less frequent spamming of followers 2017-02-10 11:25:55 +01:00