* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
* add warmup documentation - #188
* warn when relinking build directory
* add warmup documentation - #188
* warn when relinking build directory
* Renamed warmup to loadIndexesInMemory which is a better name for the current implementation of this feature.
* Adapted WebUI to state 'load indexes in memory' instead of 'warmup'
* Added loadIndexesInMemory documentation.
* Renamed loadIndexesInMemory => loadIndexesIntoMemory
Squashed commit of the following:
commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon May 29 09:39:31 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup
commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon May 29 09:00:41 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup
commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 15:53:23 2017 +0200
Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)
commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 12:46:20 2017 +0200
Implemented client-side warmup. It can now be triggered via http and arangosh
commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:25:40 2017 +0200
Fixed assertion.
commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:08:46 2017 +0200
Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values
commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:06:43 2017 +0200
Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted
commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 11:04:47 2017 +0200
Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries
commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 10:45:38 2017 +0200
Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on
commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 10:44:21 2017 +0200
Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document
commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri May 26 07:56:58 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup
commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed May 24 09:52:01 2017 +0200
Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
commit 873bc5950028248bf209d0c53fc9eb93938e3610
Merge: 8e4457a 3487097
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon Mar 27 14:00:27 2017 +0200
Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache
commit 8e4457a96754bea3437e8d44c0ef9db2366ee86d
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon Mar 27 13:59:44 2017 +0200
Removed old VPackSlice based getVertex API in traversers.
commit 2e0716eeeec1ec56b96571a18074539f5e27cb40
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri Mar 24 09:09:02 2017 +0100
Fixed logic error in SingleServerEdgeCursor readAll. The cursorId was not propageted properly, which cased filter-evaluations to fail
commit cea85c2ac1163ed0a989c492b8039ed4f7622325
Merge: 538f216 45a4fac
Author: Michael Hackstein <michael@arangodb.com>
Date: Fri Mar 24 08:17:31 2017 +0100
Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache
commit 538f216cbefcdfb4d06fe16a42f223cecae8598b
Merge: 5f4afee 8bfcb49
Author: Simon Grätzer <simon@graetzer.org>
Date: Thu Mar 23 16:33:44 2017 +0100
Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache
commit 5f4afee54094ea6d2a0cfec1abfb75451f228a23
Author: Simon Grätzer <simon@graetzer.org>
Date: Thu Mar 23 16:33:38 2017 +0100
VertexGetter StringRef methods
commit 8bfcb490d7a907ebd7dcc9e6f38a8d29baa7c7e5
Author: Michael Hackstein <michael@arangodb.com>
Date: Thu Mar 23 16:32:38 2017 +0100
Fixed AqlValues created by traverser-cache. They are now not using externals anymore which cased everything to crash
commit 614cdedb7e30e35f65f289b310c38b2585eb03e4
Merge: 773b0a5 416777b
Author: Michael Hackstein <michael@arangodb.com>
Date: Thu Mar 23 12:52:19 2017 +0100
Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache
This state is still red
commit 773b0a5e51f87875fa9bf1440dd002b2e1fcfd2a
Author: Michael Hackstein <michael@arangodb.com>
Date: Thu Mar 23 12:48:53 2017 +0100
Added Implementation or readAll in Cluster EdgeCursor. This commit is still red.
commit 82ebbd6ec84918d43bbc64174bc251efa548a4de
Merge: e52b210 35dffc4
Author: Michael Hackstein <michael@arangodb.com>
Date: Thu Mar 23 11:37:31 2017 +0100
Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache
This commit is not green.
commit 416777b60a8a0c741d11e9c654745866d0dfa15d
Author: Simon Grätzer <simon@graetzer.org>
Date: Thu Mar 23 11:25:35 2017 +0100
Fixed errornous ClusterEdgeCursor implementation of `readAll
commit e52b21025a5f56e2cfc03e3c129da83793f19846
Author: Michael Hackstein <michael@arangodb.com>
Date: Thu Mar 23 11:01:17 2017 +0100
Fixed API for Traversals: All functions now use StringRefs to identify vertices and edges which are presisted within the traverser-cache.
commit 35dffc439c156366b1110c6770c1565021934632
Author: Simon Grätzer <simon@graetzer.org>
Date: Wed Mar 22 20:26:41 2017 +0100
Fixed a few cluster issues
commit 73cf4fe41c363f6c034a1daeacee1630d7e827a5
Author: Simon Grätzer <simon@graetzer.org>
Date: Wed Mar 22 17:26:12 2017 +0100
Fixed EdgeCursor bug
commit 2f7329b4e9711740b423cf2963c6cf3f880c4bdf
Merge: bb9b129 8e5edf5
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed Mar 22 14:32:07 2017 +0100
Merge branch 'devel-feature/traversal-cache' of github.com:arangodb/arangodb into devel-feature/traversal-cache
commit 8e5edf5fbe93f34d3340099619baa9a56a738887
Author: Simon Grätzer <simon@graetzer.org>
Date: Wed Mar 22 14:09:43 2017 +0100
Fixing compile errors
commit bb9b1294c9ff91f62bc27775a4c7a69c5daca2fd
Merge: 048a4c6 8e51e3b
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed Mar 22 11:39:17 2017 +0100
Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache
commit 048a4c65e11d6fd9cb25a58ee97ed756a419a067
Merge: 693607e c6b177d
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed Mar 22 11:34:57 2017 +0100
Solved merge conflicts. Build is still red.
commit c6b177dfc348df0f49f07cd67e83b873871fe6c5
Author: Simon Grätzer <simon@graetzer.org>
Date: Tue Mar 21 23:04:28 2017 +0100
Fixed off by one error, and crash
commit 5dbc9eb58f1d21e5a530806d7ad7e52058234b83
Merge: 463e352 db8c255
Author: Simon Grätzer <simon@graetzer.org>
Date: Tue Mar 21 17:46:01 2017 +0100
Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache
# Conflicts:
# arangod/Graph/BreadthFirstEnumerator.cpp
commit 463e3520636c19ae7a7c1d445099f336c5bde31e
Merge: 4e75f29 b8cabbe
Author: Simon Grätzer <simon@graetzer.org>
Date: Tue Mar 21 17:00:17 2017 +0100
Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache
# Conflicts:
# arangod/Graph/BreadthFirstEnumerator.cpp
commit 693607ea92ab3116a628c08c59a983a6fdf8b5f4
Author: Michael Hackstein <michael@arangodb.com>
Date: Tue Mar 21 16:26:57 2017 +0100
Moved specialized NeighborsEnumerator into its own files.
commit db8c25587ebcdeff8b32d08cf2d97d723bf2b3fe
Author: Michael Hackstein <michael@arangodb.com>
Date: Tue Mar 21 15:16:39 2017 +0100
Replaced SingleServerBreadthFirst with using Callbacks in readAll method of EdgeCursor
commit 4e75f29de95edeea35573b128a0a13e04c9e65d2
Merge: 8716936 6de86c5
Author: Simon Grätzer <simon@graetzer.org>
Date: Tue Mar 21 14:32:12 2017 +0100
Merge branch 'devel-feature/traversal-cache' of https://github.com/arangodb/arangodb into devel-feature/traversal-cache
commit b8cabbedebb16c09c8168ec8b5a4499e98c705f4
Author: Michael Hackstein <michael@arangodb.com>
Date: Tue Mar 21 11:17:23 2017 +0100
Removed _enumertedPath from BreadthFirstEnumerator. Less string copying expected now.
commit 8716936bd480f4b7c85ece6fc1a51335f1942258
Author: Simon Grätzer <simon@graetzer.org>
Date: Tue Mar 21 11:05:04 2017 +0100
Traverser: changed method definitions
commit 14af02fbc08b7e2ed6b0cbb918ff509ffaef76bf
Author: Michael Hackstein <michael@arangodb.com>
Date: Tue Mar 21 10:45:13 2017 +0100
Removed usage of _enumeratedPath in lastEdge/Vertex to AQL. Goal is to delete this struct in the class, as it does unneccessary copies
commit 06a20ab0549d97b47eda98d3428e3390619ca5ec
Author: Simon Grätzer <simon@graetzer.org>
Date: Tue Mar 21 10:39:21 2017 +0100
Intermediate changes
commit 3a9a5f7c7ad186c669c3986f6f89c5a26b835ee0
Author: Michael Hackstein <michael@arangodb.com>
Date: Tue Mar 21 10:34:43 2017 +0100
Logic of BreadthFirstEnumerator now points to the last path returned instead of the next path to return. This will allow to get rid of enumeratedPath in there.
commit 6de86c548220a3ad21c53fb1de691070774130ec
Author: Michael Hackstein <michael@arangodb.com>
Date: Tue Mar 21 10:17:25 2017 +0100
Extracted BreadthFirstEnumerator out of PathEnumerator files. Only moving of code.
commit acd1d9eba5d7ea23e0d6beab831a42dc61a385a5
Merge: a99ad3c 1ab8c44
Author: Michael Hackstein <michael@arangodb.com>
Date: Mon Mar 20 13:13:16 2017 +0100
Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache
commit a99ad3c624d7ae4815776f165d65f19e43167e56
Author: Simon Grätzer <simon@graetzer.org>
Date: Fri Mar 17 13:25:45 2017 +0100
Added TraverserCache to SingleServerTraverser
commit e88ae1e53d99b55546329413e7b4a5edc48c4875
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed Mar 15 13:36:05 2017 +0100
Fixed compiler Issues in TraverserCache
commit 9d89cdc0297813f24064b46a67a43cde9a3d0297
Merge: 9d41804 f18ad19
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed Mar 15 09:43:06 2017 +0100
Merge branch 'devel' of github.com:arangodb/arangodb into devel-feature/traversal-cache
commit 9d4180480c7bc37f1f64b6e3141dae611f8d399c
Author: Michael Hackstein <michael@arangodb.com>
Date: Wed Mar 15 09:42:53 2017 +0100
Started implementing a Traverser Cache Abstraction that is used to store already fetched documents.
An invalid rev should lead to a 1200 ("conflict") error rather than a
1239 ("illegal document revision") error. This is more intuitive and
in line with the corresponding change in the HTTP API. No tests needed
adjustment.
- this sets option CURLOPT_PATH_AS_IS to keep ".." and other special components in paths
- this also changes the behavior of StringUtils::urlEncode() to always encode the "."
character to "%2e"
this changes the server startup option `--database.check-30-revisions` from a boolean (true/false)
parameter to a string parameter with the following possible values:
- "fail":
will validate _rev values of 3.0 collections on collection loading and throw an exception when invalid _rev values are found.
in this case collections with invalid _rev values are marked as corrupted and cannot be used in the ArangoDB 3.1 instance.
the fix procedure for such collections is to export the collections from 3.0 database with arangodump and restore them in 3.1 with arangorestore.
collections that do not contain invalid _rev values are marked as ok and will not be re-checked on following loads.
collections that contain invalid _rev values will be re-checked on following loads.
- "true":
will validate _rev values of 3.0 collections on collection loading and print a warning when invalid _rev values are found.
in this case collections with invalid _rev values can be used in the ArangoDB 3.1 instance.
however, subsequent operations on documents with invalid _rev values may silently fail or fail with explicit errors.
the fix procedure for such collections is to export the collections from 3.0 database with arangodump and restore them in 3.1 with arangorestore.
collections that do not contain invalid _rev values are marked as ok and will not be re-checked on following loads.
collections that contain invalid _rev values will be re-checked on following loads.
- "false":
will not validate _rev values on collection loading and not print warnings.
no hint is given when invalid _rev values are found.
subsequent operations on documents with invalid _rev values may silently fail or fail with explicit errors.
this setting does not affect whether collections are re-checked later.
collections will be re-checked on following loads if `--database.check-30-revisions` is later set to either `true` or `fail`.
The change also suppresses warnings that were printed when collections were restored using arangorestore, and the restore
data contained invalid _rev values. Now these warnings are suppressed, and new HLC _rev values are generated for these documents
as before.
* origin/devel: (47 commits)
grunt build [ci skip]
gv settings [ci skip]
attempt to fix issue #1943
Add missing documentation of the users database list and the hint that this should be used instead of the database api.
added dev button to graphs view [ci skip]
grunt build [ci skip]
Remove jshint
Mention updating license info
Prevent World War 3
Always use --global-style when installing npm deps
gv - editable & createable nodes, edges [ci skip]
Rebuild aardvark
Make sure Show Interface check asks for HTML
oops, @not supported here.
Fix phrase explaining when the collections are dropped alongside the graph definition.
fixed issue #1941
gv [ci skip]
added test for "fake" _key values
fixed issue #1942
In Farbe und in STEREREOOOUU!
...
* origin/devel: (43 commits)
added dev button to graphs view [ci skip]
grunt build [ci skip]
Remove jshint
Mention updating license info
Prevent World War 3
Always use --global-style when installing npm deps
gv - editable & createable nodes, edges [ci skip]
Rebuild aardvark
Make sure Show Interface check asks for HTML
oops, @not supported here.
Fix phrase explaining when the collections are dropped alongside the graph definition.
fixed issue #1941
gv [ci skip]
added test for "fake" _key values
fixed issue #1942
In Farbe und in STEREREOOOUU!
grunt [ci skip]
gv [ci skip]
Fix documentation.
Fix fix.
...
* origin/devel:
Start adding a hybrid logical clock.
Handle replicationFactor and numberOfShards in arangorestore.
upgrade script for homebrew
Fix namespace stuff for VC++
Use standard GNU install dirs
This adds an option --default-replication-factor to arangorestore and
makes it more convenient to specify the number of shards for restored
collections.
Simon Grätzer
Kaveh Vahedipour
m0ppers
Jan Christoph Uhde
Max Neunhöffer
Michael Hackstein
Andreas Streichardt
Frank Celler
jsteemann
Kaveh Vahedipour
Jan Christoph Uhde