1
0
Fork 0
Commit Graph

174 Commits

Author SHA1 Message Date
Michael Hackstein bc752a28c7 Cherry picked bug-fix for GharialAPI to return name in LIST (#10545) 2019-11-26 15:36:57 +01:00
Markus Pfeiffer 13df0d23de Bug fix 3.5/k shortest paths (#10390)
* Fix a bug in KShortestPathFinder

The finder was too eager to return a "shortest" path when there was actually
potentially a shorter path still to be discovered

* A little bit of cleanup

* Add a test that reproduces k Shortest Paths bug

* Add hacked up variant of optional

* Fixup tests

* Update CHANGELOG
2019-11-11 20:10:04 +03:00
Frank Celler 55ca6c8660 you cannot have an assigment operator and a const member 2019-10-17 10:19:45 +02:00
Heiko 1d264dea9f applied fix which is already present in devel (#10210) 2019-10-10 12:09:39 +03:00
Jan 68ff8d4366 fixed issue #10158 (#10163)
* fixed issue #10158

* Update arangod/Aql/TraversalExecutor.cpp
2019-10-09 12:22:23 +03:00
Jan 455cccefbb Bug fix 3.5/honor sharding restrictions (#10140)
* honor sharding restrictions also when creating a graph with existing
collections

* add test

* additional validation for startup parameters

* fixed logIDs
2019-10-02 18:42:53 +03:00
Jan 7e524f9af9 honor restrictions when creating graphs (#10136) 2019-10-01 23:55:58 +03:00
Max Neunhöffer 328f46e3d6 This merges hotbackup and atomic-db-creation into 3.5. (#9968)
* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.
2019-09-11 13:13:54 +03:00
Jan f8c156a44f make graphs transaction-aware (#9855)
* make graphs transaction-aware

* simranification

* fix tests with mmfiles
2019-09-02 22:06:10 +03:00
Heiko e40d7fadd1 Bug fix 3.5/graph create orphan collection properly (#9842)
* added ensureCollections to orphan create method

* do not move anymore

* rm trx finish. will be handled automatically in error case

* changelog
2019-08-30 13:55:08 +03:00
Jan 3dcc293224 make index selection more deterministic (#9736)
* make index selection more deterministic

* updated CHANGELOG

* serialize indexes used by traversal with their estimates

* serialize selectivity estimates for shortest path nodes too

* fix assertion that doesn't hold true in unit tests

* fix test
2019-08-20 12:55:21 +03:00
Michael Hackstein d5840c125a Bug fix 3.5/min replication factor (#9524)
* Cherry-pick minReplicationFactor

* Bug fix/failover with min replication factor (#9486)

* Improve collection time of IResearchQueryOptimizationTest

* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* Initial commit, old plan, does not yet work

* fixed ires tests

* Included FailoverCandidates key. Not fully implemented

* fixed wrong assert

* unified in sync follower reporting

* fixed compiler errors

* Cleanup locking, and fixed potential deadlocks

* Comments about locking order in FollowerInfo.

* properly check uint

* Keep old leader as potential failover candidate

* Transaction methods now use followerInfo to check if the leader can write, this might have the sideeffect that 'failoverCandidates' are updated

* Let agency check failoverCandidates if possible

* Initialize member variables

* Use unified follower reporting in DBServerAgencySync

* Removed obsolete variable, collecting it somewhere else

* repl factor attr check

* Reimplemented previous followers, second attempt now. PhaseOne and PhaseTwo can now synchronize on current.

* Fixed assertion, forgot an off-by-one

* adjusted test to be more preciese now

* Fixed failove candidates list

* Disable write on dropping too many followers

* Allow to run updateFailoerCandidates multiple times with same leader.

* Final fixes, resilience tests now green, crossing fingers for jenkins

* Fixed race on atomics comparison

* Fixed invalid number type

* added nullptr handling

* added nullptr handling

* Removed invalid assert

* Make takeover of leadership an atomic operation

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* Fixed creation code to use takeoverLeadership

* Update arangod/Cluster/FollowerInfo.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Applied review fixes

* There is no timeout

* Moved AQL + Pregel to INTERNAL_AQL lane, which is medium priority, to avoid deadlocks with Sync replication

* More review fixes

* Use difference if you want to compare two vectors...

* Use std::string ...

* Now check if we are in recovery mode

* Added documentation for minReplicationFactor

* Added readme update as well in documenation

* Removed merge conflict leftovers 0o, i should not trust the IDE

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/Architecture/Replication/README.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Collections/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Graph/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Adepted review requests, thanks for finding!

* Removed unnecessary const

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Moved initilization of variable more downwards

* Apply lock before notify_all()

* Remove documentation except DocuBlocks, covered by PR in docs repo

* Remove accidental indent
2019-07-22 17:48:34 +03:00
Jan c52f2a8315
refactoring (#9411) 2019-07-09 11:15:52 +02:00
Jan 9cb08ded92
make the comparison functions unambiguous (#9349)
* make the comparison functions unambiguous

* added @kaveh's suggestion
2019-07-01 16:35:28 +02:00
Heiko 56c969316f Bug fix/gharial edge from and tos verification (#9177)
* prepared new functions for more precise edge verification

* added tests

* more testing

* validate function added to replace edge functionality

* changelog

* applied requested mchackis refactor wishes

* adjustments for update and replace edge validation

* ruby testing adjustments due more prices update and replace edge verification mechanisms

* used wrong flag for the replacement of an edge process

* adjusted last failing tests

* rm print

* added edge to edge tests
2019-06-12 15:12:50 +02:00
Jan Christoph Uhde 3f603f024f remove some containers from common.h (#9223)
* remove some containers from Common.h

* enterprise fixes
2019-06-07 13:27:24 +02:00
Michael Hackstein d135d55d55
Bug fix/collection babies (#9124)
* Bug fix 3.4/collection babies (#9033)

* Prepare API to create multiple collections in a single request to ClusterMethods to improve speedup

* Added counter on how many collections are successfully created

* Allow multi collection creation one level higher

* CollectionMethods now allow batch createion of Collections

* Improved array size assertions

* Now a graph is createad within a single roundtrip in the agency.

* Added new header files

* Insert collections in the AGENCY with TTL and a isBuilding flag, collections with this flag should not be visisible in the coordinator

* Added forgotten C++ file

* Fixed a rare race condition, and the failing IResearch Tests

* readded callback on DONE, otherwise lists are out of sync

* Fixed assertions to let mocked tests pass...

* Fixed community cluster

* Started fixing IResearch analyzer test, catch-tests are failing ;(

* Solved missed merge-conflict

* Added helper functions in AnalyzerFeature-test

* Refactoring AnalyzerTest Section-Auth

* Refactoring AnalyzerTest Section-Emplace-Duplicates

* Refactoring AnalyzerTest Section-Emplace-Error-Cases. Recovery-Test is now red, it seemed to be green because of invalid test case before.

* Refactoring AnalyzerTest, split GET test into multiple parts, still left 'cluster simulation'.

* Attempt to extract Coordinator / DBServer tests a little bit. This commit starts to break all Coordinator tests. However i am convinced that earlier version did NOT test a cluster situation at all, but some hybrid of SingleServer with full local storage that got told to be a Coordinator from now on, but without any Coordinator setup...

* Temporarly disabled some tests in AnalyzerFeature, as discussed with @gnusi.

* Fixed include guard.

* Temporarily deactivated failing tests

* You shall save your files before you commit...

* Fixed test asserting on plan version, which is now higher than before
2019-06-03 17:11:22 +02:00
Simon a5a72ef6d4 port Pregel segmented buffers (#9112) 2019-05-28 18:23:20 +02:00
Jan 59b67cad40
fix various small annoyances (#9079) 2019-05-23 17:36:38 +02:00
Simon 1923d315e8 New tests for Transaction API (#8996) 2019-05-14 21:40:31 +02:00
Markus Pfeiffer f00a8e98a2 Fix ASAN failure (#8974) 2019-05-13 13:56:19 +02:00
Markus Pfeiffer 65d090be9d Fix warning about incompatible types in KShortestPathsFinder (#8936) 2019-05-08 15:29:51 +02:00
Simon 0502a97abb forwardport changes from 3.4 (#8894) 2019-05-08 14:34:25 +02:00
Markus Pfeiffer c9a35ea2fa Improve k Shortest Paths performance (#8889)
* Only consider spur paths from after the branchpoint
   from the previous path
 * Cache neighbourhoods around vertices
2019-05-07 13:21:52 +02:00
Jan 6ee9206e46
clear traverser cache in-between (#8772) 2019-04-17 15:27:37 +02:00
Markus Pfeiffer 304e8f861d Fix k Shortest Paths with weights, reactivate cluster tests (#8761)
* Make find in ShortestPathPriorityQueue const

* Fix k Shortest Paths with weights, reactivate cluster tests
2019-04-17 15:10:59 +02:00
Michael Hackstein b3205dada8
USe smart-pointer in Priority Queue, it seems the usages have been correct everywhere (so no leaks) but memory responibility was hard to understand, should now be much simpler. (#8768) 2019-04-16 12:06:42 +02:00
Jan 449ab1ed8e
Bug fix/cppcheck 13042019 (#8752) 2019-04-15 10:13:56 +02:00
Markus Pfeiffer a14386267a k Shortest Paths (#8715)
* Add k-shortest-paths finder
2019-04-12 21:58:18 +02:00
Markus Pfeiffer af400d0af7 Add hash and equals_to for EdgeDocumentToken (#8709) 2019-04-12 11:42:07 +02:00
Markus Pfeiffer b3224eed1d Add isQueryKilled callback to ShortestPathOptions and use it (#8739)
* Add isQueryKilled callback to ShortestPathOptions

* Use isQueryKilledCallback

in
 * ConstantWeightShortestPathFinder
 * AttributeWeightShortestPathFinder
 * ShortestPathExecutor
2019-04-12 10:39:01 +02:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Jan 4af7fa8f46
count HTTP requests from traversals (#8645) 2019-04-02 18:00:11 +02:00
Jan 616ea94f24
Bug fix/cleanup 31032019 (#8632) 2019-04-01 17:14:11 +02:00
Heiko 7a44359bb0 Bug fix/gharial rm vertex smart fix (#8622) 2019-03-29 11:11:01 +01:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Simon bcc26926de Support Transactions in RestCursorHandler (#8539) 2019-03-25 17:04:11 +01:00
jsteemann d5de286568 cppcheck 2019-03-23 22:29:42 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan f9fe1de82f
prevent assertion failures when creating smart graphs with invalid options (#8502) 2019-03-21 18:51:47 +01:00
Dan Larkin-York 2eadab33e7 Index hints (#8431) 2019-03-19 09:14:18 +01:00
Vasiliy 29b941a677 issue 526.2.1: add upgrade step for legacy static analyzers (#8417) 2019-03-18 23:22:23 +03:00
Jan 23f7fc1368
fix some memleaks (#8432) 2019-03-18 18:11:37 +01:00
Simon 49cc3bcd1e Refactorings from cluster trx improvement branch (#8391) 2019-03-14 23:13:17 +01:00
Tobias Gödderz 8d649903f7 Aql by line mixed fixes (#8349) 2019-03-14 19:38:41 +01:00
Jan 3156e481de
fix test (#8402) 2019-03-13 15:24:55 +01:00
Jan 9d3327c6ea
Bug fix/rearm cursors (#8363) 2019-03-12 15:28:33 +01:00
Michael Hackstein 0403d6cb6f
Bug fix/prune neighbors and explain (#8304)
* Fixed Neighbors prune on depth0

* Fixed the Explainer output

* Rmeoved debug output
2019-03-11 10:13:57 +01:00
Michael Hackstein 6e8d43b2da
Feature/aql traversal prune (#8255)
Allow PRUNE in the AQL Traverser.
2019-02-27 11:43:06 +01:00
Heiko 4557dfc0d5 do not allow edge definitions with empty from or to arrays, added als… (#8198) 2019-02-20 18:32:33 +01:00