* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
* Pass config/deps on _install so setup can see them
* Only register service when install succeeded
* Don't throw after point of no return: breaky install is breaky
* Delete service files during install if service validation fail
* Delete service files during install if service execution fails
* Fix commitLocalState with invalide services
* Remove deprecated test
* add warmup documentation - #188
* warn when relinking build directory
* add warmup documentation - #188
* warn when relinking build directory
* Renamed warmup to loadIndexesInMemory which is a better name for the current implementation of this feature.
* Adapted WebUI to state 'load indexes in memory' instead of 'warmup'
* Added loadIndexesInMemory documentation.
* Renamed loadIndexesInMemory => loadIndexesIntoMemory
commit 3d9cf792912db1974b9ac5e00ca2b4c9245b7d34
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Fri Jun 2 15:32:43 2017 +0200
optimise for single writes in agency log
commit 65056ab9026f9b4b211dda0f17c75602b978f2bf
Author: Max Neunhoeffer <max@arangodb.com>
Date: Fri Jun 2 15:01:15 2017 +0200
More tests, taking agency log compaction interval into account.
commit 6600d707784e8fd5b62c0c75fd1826af09b8e13f
Merge: cf46882 02f00cc
Author: Max Neunhoeffer <max@arangodb.com>
Date: Fri Jun 2 14:50:38 2017 +0200
Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul
commit 02f00cc271d027f02b0625afb76745bfa76bf833
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Fri Jun 2 14:34:41 2017 +0200
compaction step and keep size defaults for 3.2
commit 03fc8fbff8f0ac701f7d7f94521c0c3152dd6f92
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Fri Jun 2 14:32:46 2017 +0200
Constituent fatally exists if eletion ballot cannot be persisted
commit cf4688226fc897e74bb2d9ebdfca3ce4578c3b70
Merge: c727fc4 724bd1e
Author: Max Neunhoeffer <max@arangodb.com>
Date: Fri Jun 2 13:08:15 2017 +0200
Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul
commit 724bd1efe19e2e9dbfc14cd819f180816b6d62d0
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Fri Jun 2 13:02:51 2017 +0200
persistence success in agency state is properly evaluated
commit c727fc48bb93e7b135b3ca929c03221c7bcaddb9
Author: Max Neunhoeffer <max@arangodb.com>
Date: Fri Jun 2 12:04:55 2017 +0200
Set default compaction step size in agency to 20000 and 10000 keep size.
commit ded16ae6945e9c1479e99bc2e7ccb4d6feca19a6
Author: Max Neunhoeffer <max@arangodb.com>
Date: Fri Jun 2 11:11:12 2017 +0200
Fix help page in startStandAloneAgency.sh for --use-persistence.
commit 13ae9f40f649a8f92eeca4b16bbb5647b540722d
Merge: 834c7c9 aa3e8c1
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu Jun 1 23:41:34 2017 +0200
Merge remote-tracking branch 'origin/devel' into agency-log-compaction-overhaul
commit 834c7c920d36db3579def66c38fb04870936f8bd
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu Jun 1 16:56:03 2017 +0200
Handle error in recvAppendEntriesRPC properly.
commit bd9c8d03b76ad25d4078740b5bf994fdba3845d0
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu Jun 1 16:55:35 2017 +0200
Handle errors in persist() and log() properly.
commit 5b4d2c3d9af078d6a1b5626af20dc9abf2546baa
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu Jun 1 16:25:22 2017 +0200
Improve error reporting.
commit d60697c5f26d6592eecefc9b9a43e9b699d1773d
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Thu Jun 1 12:16:39 2017 +0200
Agency must not responds to any requests after startup util leader has RAFT commited up to pre shutdown state
commit 92b8ede5fa022ace1596607abcf8fad1130504c8
Merge: 9340e74 d24455c
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Wed May 31 16:54:45 2017 +0200
Merge remote-tracking branch 'origin/devel' into agency-log-compaction-overhaul
commit 9340e7461130a4783c09ad8d91e5a07f9500a045
Merge: 7b7ce9d 63a9d60
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Wed May 31 12:13:55 2017 +0200
agency tests to cover compaction
commit 63a9d604c474eda4302032629dff1f0f69fa0813
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 31 11:59:11 2017 +0200
Set agency.compaction-keep-size to at least 0.
commit ef842260968a4769d9502a701b7251da32647e52
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 31 11:49:34 2017 +0200
Fix agent size 1 case (thread already gone).
commit 7b7ce9d79f6e8208c13f153b1b9a395b780d6ce1
Merge: 24e2e7e ff306bf
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Wed May 31 11:39:58 2017 +0200
Merge branch 'agency-log-compaction-overhaul' of https://github.com/arangodb/arangodb into agency-log-compaction-overhaul
commit ff306bf547bc4f528c9b66e222271ac143029508
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 31 11:11:06 2017 +0200
Move compaction into the future when we take a snapshot from leader.
commit 24e2e7e00f960928a79ce4008b8031d6b9b07fd9
Merge: 84034ac b3ea17a
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Wed May 31 11:01:13 2017 +0200
Merge branch 'agency-log-compaction-overhaul' of https://github.com/arangodb/arangodb into agency-log-compaction-overhaul
commit b3ea17a219baa2abd5892819012fb59f440cdeb8
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 31 09:42:59 2017 +0200
Get rid of double nonsense.
commit 035c8d1b34e1b73a381d5468422adf13b2ebc36a
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 31 09:25:28 2017 +0200
Sort out Agent::load sequence.
commit 84034ac2809a77145d6b1d23bf44857b3a0c4651
Merge: eb34a2e 3180a9d
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Tue May 30 17:07:20 2017 +0200
merging in
commit eb34a2e64e6ac8dc6571b92cb853c38b7022c833
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Tue May 30 16:58:05 2017 +0200
keep persistence for restarts in standalone agency
commit 3180a9d9ce4a4401a55ef02606b020316d43cbe5
Merge: 5d60524 28b9580
Author: Max Neunhoeffer <max@arangodb.com>
Date: Tue May 30 16:56:56 2017 +0200
Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul
commit 5d60524429d8ddda4491beecb931c3b9e3cc1d8a
Author: Max Neunhoeffer <max@arangodb.com>
Date: Tue May 30 16:56:36 2017 +0200
Implement snapshot sending in AppendEntriesRCV.
commit 28b958054f51c9cb36706df4e4345aa0f726ed15
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Tue May 30 14:20:13 2017 +0200
state machine should not advance _committed if empty
commit df18f326acea7f5bc2660a37e22f1503952e4b41
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 23:39:47 2017 +0200
Store term with compaction snapshot, recover again.
commit 2551a48b6fb513c9ea934bce755f8c364dae2f05
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date: Mon May 29 17:45:26 2017 +0200
indices renamed to closely match RAFT documentation
commit e62dcdecf6e8650cfa5725d91b809d05591b48a4
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 16:37:43 2017 +0200
More cleanup.
commit 9f4787c46621375f0361138a8961431eb21ce5c0
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 14:50:25 2017 +0200
Revise loadLastCompactedSnapshot to return 0 without a snapshot.
commit 13285e1d70c8a4ac8c79a08de6f8fbc0f8d242bf
Merge: 3393c43 6c5f23e
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 12:06:20 2017 +0200
Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul
commit 3393c43c75520c74d20df09c74fbbbd8b1af5976
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 12:03:47 2017 +0200
More cleanup around Store::apply and friends.
commit 4ccb41d1839748c98e11403fa04f6a7d6af5e95b
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 09:45:41 2017 +0200
Document apply methods in Store with comments.
commit ea05c4880fedb6fe535e24761ac5cb3c26ccfc20
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 09:45:18 2017 +0200
Intentionally keep one log entry more to prevent empty log.
commit 67fb62f2259cc3c6368319917c7257ebcc177d3f
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 09:44:42 2017 +0200
Improve plausibility checks at compaction time.
commit 0bafc368785b15a94f8783c4c929f4208f87d09c
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu May 25 00:51:51 2017 +0200
Sort out (re-)building of agency K/V store(s) from compaction snapshots.
This is in case of (re-)start, becoming a leader and when serving
/_api/agency/store.
commit 46b0750bc6c597ec388aac0cdca32082c0cc54b8
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu May 25 00:50:54 2017 +0200
Set compaction interval to 200 and keep to 0.
This is way to small but tests should run with it.
Will later increase numbers again.
commit 024dc0846ae30248b464dd481a8bbc1134f56983
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 24 23:29:57 2017 +0200
Add a trivial test for agency log compaction.
commit e12fd3b46833419d7b436eeadd7246304324b891
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 24 23:26:03 2017 +0200
First part of cleanup of agency log compaction.
Now the right compaction snapshots are taken and persisted.
Furthermore, the right log entries and old snapshots are removed
after compaction, both the volatile and the persisted ones.
The readDB and spearHead stay unchanged at compaction time as it
should be.
commit d59901aea0c3ca31ef253299d2adc3353b79e664
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 24 12:18:26 2017 +0200
Remove unused member variable.
commit 6c5f23eb7b42d9f20d4dadb2932a63add99f9c76
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 09:45:41 2017 +0200
Document apply methods in Store with comments.
commit 670899f72d215e0fcc0ca0389cea9250a291e83b
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 09:45:18 2017 +0200
Intentionally keep one log entry more to prevent empty log.
commit 660f61029917bbc2ce1fae3e4fc903095b023297
Author: Max Neunhoeffer <max@arangodb.com>
Date: Mon May 29 09:44:42 2017 +0200
Improve plausibility checks at compaction time.
commit e2802e4b36d1f67d8361c1d8b0c92fbff696f439
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu May 25 00:51:51 2017 +0200
Sort out (re-)building of agency K/V store(s) from compaction snapshots.
This is in case of (re-)start, becoming a leader and when serving
/_api/agency/store.
commit 12b43f1b91284a1185390d6dcfbd1e838522d392
Author: Max Neunhoeffer <max@arangodb.com>
Date: Thu May 25 00:50:54 2017 +0200
Set compaction interval to 200 and keep to 0.
This is way to small but tests should run with it.
Will later increase numbers again.
commit c8b9a37a690b8e7e8bfa1276a3f9ba4b6b5a9c27
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 24 23:29:57 2017 +0200
Add a trivial test for agency log compaction.
commit cf0c8c1fff666f76411082f87efe685a412ecebb
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 24 23:26:03 2017 +0200
First part of cleanup of agency log compaction.
Now the right compaction snapshots are taken and persisted.
Furthermore, the right log entries and old snapshots are removed
after compaction, both the volatile and the persisted ones.
The readDB and spearHead stay unchanged at compaction time as it
should be.
commit 0a4255359a57b8686133e6014e2b82b8079f36fa
Author: Max Neunhoeffer <max@arangodb.com>
Date: Wed May 24 12:18:26 2017 +0200
Remove unused member variable.
* Implement new self-heal
* Add error codes for 503, service missing/outdated
* Detect changes to service via rev
* Pretty print incoming response object in log
Squashed commit of the following:
commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon May 29 09:39:31 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup
commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon May 29 09:00:41 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup
commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 15:53:23 2017 +0200
Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)
commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 12:46:20 2017 +0200
Implemented client-side warmup. It can now be triggered via http and arangosh
commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:25:40 2017 +0200
Fixed assertion.
commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:08:46 2017 +0200
Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values
commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:06:43 2017 +0200
Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted
commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:04:47 2017 +0200
Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries
commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 10:45:38 2017 +0200
Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on
commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 10:44:21 2017 +0200
Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document
commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 07:56:58 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup
commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed May 24 09:52:01 2017 +0200
Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
* Improve Foxx cluster resilience
Fixes#2083Fixes#2384Fixes#2408
Addresses #1892
* Port old Foxx API
* Implement single-file services
* Add console.errorStack/warnStack/infoStack helpers
* Simplify serviceInfo validation
* Extract github/upload logic into Aardvark and old FM API
* Move generator logic into Aardvark
* Move zip/js buffer logic into FM core
* Add Foxxmanager tests
* Send empty response when no README
* Disambiguate script arg format
Historically we allow passing an array of positional arguments or an arbitrary first argument.
This is surprising behaviour, so we should just always treat the value as a first argument.
* Rebuild bundle in development mode
* Nicer HTTP docs formatting
* Create Foxx HTTP docs
* Simplify service upload handling
* Remove inline swagger docs
* Implement public download route
* Consistency
* Rebuild aardvark
* Move bundle route into /_api/foxx/_local
* Rebuild Swagger API docs
* Add changes to CHANGELOG
* More docs
* 'devel' of https://github.com/arangodb/arangodb:
fix enterprise branch detection so it doesn't trigger a full recompile
fix detection of partial cmake configure
fix detection of partial cmake configure
one more place to choose the propper directory for unittest result files.
remove debug print
fixed issue: #2412
fix crash
Max Neunhöffer
Jan
Manuel B
Frank Celler
Mark
Michael Hackstein
Kaveh Vahedipour
Kaveh Vahedipour
jsteemann
Alan Plum
Andreas Streichardt
Wilfried Goesgens
Jan Christoph Uhde
baslr
Dan Larkin