1
0
Fork 0
Commit Graph

15092 Commits

Author SHA1 Message Date
Michael Hackstein d5840c125a Bug fix 3.5/min replication factor (#9524)
* Cherry-pick minReplicationFactor

* Bug fix/failover with min replication factor (#9486)

* Improve collection time of IResearchQueryOptimizationTest

* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* Initial commit, old plan, does not yet work

* fixed ires tests

* Included FailoverCandidates key. Not fully implemented

* fixed wrong assert

* unified in sync follower reporting

* fixed compiler errors

* Cleanup locking, and fixed potential deadlocks

* Comments about locking order in FollowerInfo.

* properly check uint

* Keep old leader as potential failover candidate

* Transaction methods now use followerInfo to check if the leader can write, this might have the sideeffect that 'failoverCandidates' are updated

* Let agency check failoverCandidates if possible

* Initialize member variables

* Use unified follower reporting in DBServerAgencySync

* Removed obsolete variable, collecting it somewhere else

* repl factor attr check

* Reimplemented previous followers, second attempt now. PhaseOne and PhaseTwo can now synchronize on current.

* Fixed assertion, forgot an off-by-one

* adjusted test to be more preciese now

* Fixed failove candidates list

* Disable write on dropping too many followers

* Allow to run updateFailoerCandidates multiple times with same leader.

* Final fixes, resilience tests now green, crossing fingers for jenkins

* Fixed race on atomics comparison

* Fixed invalid number type

* added nullptr handling

* added nullptr handling

* Removed invalid assert

* Make takeover of leadership an atomic operation

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* Fixed creation code to use takeoverLeadership

* Update arangod/Cluster/FollowerInfo.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Applied review fixes

* There is no timeout

* Moved AQL + Pregel to INTERNAL_AQL lane, which is medium priority, to avoid deadlocks with Sync replication

* More review fixes

* Use difference if you want to compare two vectors...

* Use std::string ...

* Now check if we are in recovery mode

* Added documentation for minReplicationFactor

* Added readme update as well in documenation

* Removed merge conflict leftovers 0o, i should not trust the IDE

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/Architecture/Replication/README.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Collections/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Graph/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Adepted review requests, thanks for finding!

* Removed unnecessary const

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Moved initilization of variable more downwards

* Apply lock before notify_all()

* Remove documentation except DocuBlocks, covered by PR in docs repo

* Remove accidental indent
2019-07-22 17:48:34 +03:00
Wilfried Goesgens 9781b9890c Bug fix 3.5/connect mocha results to testing js (#9493)
* connect the mocha testresults if we run it in an extra arangosh instance

* fix status handling, lint

* sometimes we have the string undefined to ignore

* fix filter forwarding for spawned arangoshs - add mocha version

* migrate mochaGrep into --testCase parametrizing
2019-07-18 19:49:03 +03:00
Tobias Gödderz 87e5fe7dd2 Bug fix 3.5/clean replication api wal tracking (#9503)
* Use int type for server id

Change serverId to an int

Pass syncerId only for synchronous replication

Added UrlBuilder

structs to classes, reordering

Added Location class, cleanup

Fixed initialization order

Use Location class

Use string for large ints

Documentation

Added clientInfo to ReplicationClientProgressTracker and corresponding rest handlers

Pass clientInfo string in sync replication

Pass clientInfo in addFollower, too

Updated docu

Renamed UrlBuilder to UrlHelper

Updated docu

Try to fix compile error on windows

Fixed a bug and a test

* Implemented @jsteeman's comments
2019-07-18 19:38:31 +03:00
Wilfried Goesgens 338e7ce087 jslint is your friend. (#9509) 2019-07-18 14:52:25 +03:00
Simran ef3aeb00e5 Documentation 3.5/example generation improvements (#9498)
* Documentation/switch example generation to rocksdb (#9478)

* improve the formatting of the batch request documentation

* split to linewise, check whether we have a json and format it.

* Add missing asterisk

* Minor fixes / improvements
2019-07-18 14:45:59 +03:00
Frank Celler 2beb731264 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-07-11 15:22:32 +02:00
Wilfried Goesgens 623048aff8 fix 6561f21536 - only abort on timeout, not regular exit failures (#9434)
* fix 6561f21536 - only abort on timeout, not regular exit failures

* make pre-check exception distinguishable, its got no ready existing test status
2019-07-10 18:48:46 +03:00
Wilfried Goesgens 0ad76a8185 fix location - while they work with the server these tests are run in arangosh so they need to be in client/ (#9435)
* fix location - while they work with the server these tests are run in arangosh so they need to be in client/

* rename testsuite

* rename file as its testsuite

* Rename permissions_server to server_permissions
2019-07-10 16:49:12 +03:00
Andrey Abramov b3690fc24f
bug-fix/internal-issue-#595 (#9440)
* address ASAN failures in tests

* Add --isAsan for testing.js

* Fix typo
2019-07-10 01:11:22 +03:00
KVS85 c05288fa59
Fix recovery tmp dir 2019-07-09 18:13:25 +03:00
Wilfried Goesgens c2b645062b if there is no testcase, we still 'splitted' the bucket (#9425) 2019-07-08 16:44:54 +03:00
Tobias Gödderz f501e00e9d Bug fix/add shard id to replication client identifier (#9366) 2019-07-08 14:03:42 +02:00
Wilfried Goesgens 3599aedda1 Don't wait eternaly; don't unload collections with rocksdb (#9415) 2019-07-05 17:13:24 +02:00
Jan 5c39692c20
remove several outdated error codes (#9402) 2019-07-05 13:36:16 +02:00
Jan Christoph Uhde 54169cd44f fix cluster-statistics (#9397) 2019-07-05 10:32:46 +02:00
Tobias Gödderz 96685618d6 Don't use non-existent variable (#9407) 2019-07-04 14:41:30 +02:00
Wilfried Goesgens 5bab188907 improve handling when procdump detects the process is dead (#9381) 2019-07-04 12:38:57 +02:00
Wilfried Goesgens 081e755b25 fix typo (#9400) 2019-07-03 16:17:55 +02:00
Wilfried Goesgens 61f3397e33 Feature/add tcpdump support (#9396) 2019-07-03 16:12:03 +02:00
Wilfried Goesgens 3f9b85967d apply filters before starting the server, so we can detect whether no test would be executed (#9387)
* apply filters before starting the server, so we can detect whether no test would be executed

* skipTimecritial and skipNondeterministic also qualify for non-errnous empty testcase

* simplify code, if  is issued no matching testcase is an error

* Update test-utils.js
2019-07-03 14:13:16 +02:00
Jan 11f5f33659
make sure all error code names are prefixed with ERROR_ @fceller @kvs85 (#9384) 2019-07-02 18:07:33 +02:00
Wilfried Goesgens 46479af842 keep coredump for reports (#9378) 2019-07-02 10:09:12 +02:00
Andrey Abramov cf7d5bfa95 fix arangosearch explanation (#9365) 2019-06-28 19:51:34 +02:00
Jan 18042e16ec
remove unused JS function (#9363) 2019-06-28 19:20:08 +02:00
Jan 1653e9698a
remove unused functionality (#9360) 2019-06-28 18:28:11 +02:00
Jan Christoph Uhde 782cb49105 Feature: Analyzer REST-API Tests (#9091)
* add analyser http API tests

* add more tests

* add tests and lint

* remove print

* add another test

* fix formattig

* add e6 expresseion as key in object

* WIP: add analyzer permissions test

* fix test

* lint

* WIP: try to add analyzer auth-test

* add sleepBeforeStart option

* use basic auth

* fix tests

* change test so it would work if the api was as expected

* fix authentication test

* fit shell client analyzer test

* fix shell-analyzer-rest-api.js

* fix auth-analyzer.js

* fix user-access-right-update-view-arangosearch.js

* remove print statements

* Fixed analyzer parameters for test
2019-06-28 16:21:09 +02:00
Wilfried Goesgens 51e687d4fa Feature/add output before http ing (#9361)
* make rspec timeouteable (on windows for now)

* add timeout state, fix windows implementation

* abort testrun if a timeout in rspec occured

* lint

* fix scope of procdump stopper

* fix include scope

* fix invokation

* use proper variable to wait for

* add output before we attempt to talk to the SUT, so we can better insulate startup lockups

* when we timeout a process dump out all we know about the SUT.

* when we timeout a process dump out all we know about the SUT.
2019-06-28 16:03:32 +02:00
Wilfried Goesgens 6561f21536 Feature/make rspec timeouteable (#9354)
* make rspec timeouteable (on windows for now)

* add timeout state, fix windows implementation

* abort testrun if a timeout in rspec occured

* lint

* fix scope of procdump stopper

* fix include scope

* fix invokation

* use proper variable to wait for
2019-06-28 15:01:04 +02:00
Wilfried Goesgens 3c6264b9b0 wait .25, then check whether procdump successfully starts, and if not, properly abort the operation (#9333) 2019-06-26 19:08:38 +02:00
Wilfried Goesgens 63568a2aac add a function that will use procdump to generate a core before attempting to kill the SUT with *nix coredump enabled signals (#9322) 2019-06-25 16:25:25 +02:00
Wilfried Goesgens bf9328cd18 properly forward corecheck option to the client tools launch invokations (#9313)
* properly forward corecheck option to the client tools launch invokations

* one param was missing
2019-06-25 11:24:55 +02:00
Wilfried Goesgens f947954f6c fix the option to specify a ruby executeable from the outside in instead of leaning on rspec.bat to do its job (#9315) 2019-06-24 20:39:30 +02:00
Lars Maier 7d98b5bde4 Info log when server is removed. Store remove date in agency. (#9285) 2019-06-24 14:04:57 +02:00
Wilfried Goesgens f3b1de9808 adopt option to do the right thing with gtests (#9280) 2019-06-18 11:15:09 +02:00
Wilfried Goesgens a866f8c6fd update gtest, use json output for reporting (#9193) 2019-06-17 13:46:40 +02:00
Frank Celler bb7c9082d1 release version 3.5.0-rc.4 2019-06-13 21:57:29 +02:00
Frank Celler f021ea68f7 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-06-13 20:47:41 +02:00
Heiko 7432c5f152 this out of scope inside the callback function (#9169)
* this out of scope inside the callback function

* changelog
2019-06-12 16:03:41 +02:00
Markus Pfeiffer a793f72448 Move roundCost function to utilities (#9247) 2019-06-12 16:02:09 +02:00
Heiko 677ffdc692 Bug fix/gravatar ui fix (#9211)
* fixed editing of users in the web ui when gravatar was enabled

* changelog

* Update userManagementView.js

rm console log
2019-06-12 15:20:15 +02:00
Alan Plum a6b49e3a7e Bug fix/buffer redux (#9229)
* Add failing tests

* Add tests for Buffer#slice

* Add tests for Buffer#new

* Skip failing tests

* Implement changes

* Unskip failing tests

This reverts commit 9f8a34bd74e152b7b3158fea79b12e6da8343001.

* Fix implementation

* Remove meaningless test

This test only succeeded because of a broken implementation of the Buffer constructor. It doesn't really do anything useful.

* Add note to CHANGELOG
2019-06-12 15:00:43 +02:00
Jan 59d84e8301 attempt to remove expired WAL files on agents (#9231) 2019-06-09 19:46:45 +02:00
Jan 091abbec77 don't make RocksDB startup fail with 0-byte log files (#9208) 2019-06-07 15:13:56 +02:00
Frank Celler 6b16d1c0a5 Revert "Fix Buffer methods (#9070)"
This reverts commit 6c96e5dd24.
2019-06-07 10:18:58 +02:00
Tobias Gödderz 429b3dcf3d Allow generic graph tests to work without enterprise checkout (#9202) 2019-06-06 15:15:53 +02:00
Jan 074397d367
allow "stream" attribute on top level (#9195) 2019-06-05 15:48:58 +02:00
Wilfried Goesgens 1d2071420b output setup failures to testfailures.txt (#9188) 2019-06-05 09:55:54 +02:00
Alan Plum 6c96e5dd24 Fix Buffer methods (#9070)
* Add failing tests

* Implement changes

* Add note to CHANGELOG

* Fix test failure
2019-06-03 17:07:52 +02:00
Dan Larkin-York 44a413a9af Miscellaneous fixes for named indices (#9100) 2019-05-31 17:00:56 +02:00
Frank Celler a2ee756c5e release version 3.5.0-rc.3 2019-05-30 08:46:27 +02:00