1
0
Fork 0
Commit Graph

899 Commits

Author SHA1 Message Date
Heiko a7b64bd815 Bug fix 3.5/fix internal issue 4451 (#10538)
* Fix dump_authentication suite

* Fix typos

* user the correct attribute name

* properly reload user permissions after _users collection restore

* fixed foxx restore test

* changelog

* changed the order of index creation during restore for _users collection
2019-11-26 14:54:36 +01:00
Jan bf187f4b50 Bug fix 3.5/issue 10193 (#10194)
* fixed issue #10193: Arangoexport does not handle line feeds when exporting as csvi

* escape \r too
2019-10-09 16:01:51 +03:00
Max Neunhöffer f511aff08b rclone path normalization (port from devel to 3.5). (#10139)
* rclone path normalization (port from devel to 3.5).

* Fix invalid thing in CMakeLists.txt (wrong port).
2019-10-02 13:45:35 +03:00
Jan 935a35a275 naive attempt to not build arangobackup in community version (#10013)
* naive attempt to not build arangobackup in community version

* fix testing.js

* try to fix enterprise detection
2019-09-13 20:26:35 +03:00
Heiko e5da8b1cef Bug fix 3.5/arangosh gharial vertex deletion (#9926)
* typo in route

* unused function removed

* added general graph standalone and cluster tests, formatting

* removed fail, removed print

* changelog

* Update CHANGELOG
2019-09-13 13:15:53 +03:00
Max Neunhöffer 328f46e3d6 This merges hotbackup and atomic-db-creation into 3.5. (#9968)
* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.
2019-09-11 13:13:54 +03:00
KVS85 f25baf1ecb Fix rspec timeout in Windows with disableMonitor 2019-09-04 17:36:18 +02:00
Wilfried Goesgens d9057ad695 fix runInArangosh result evaluation (#9589) 2019-07-29 15:56:22 +03:00
Matthew Von-Maszewski 0832c6d546 Feature 3.5: add gzip and encryption support to arangoexport and arangoimport (#9579)
* port gzip/encryption for arangoexport/import from devel branch

* revert Dump default use of gzip from true to false.

* Stylefix in changelog.
2019-07-27 00:06:33 +03:00
Jan 9777f53878 Bug fix 3.5/add db list transactions function (#9575)
* added missing function db._transactions(), and equivalent REST API route
GET /_api/transaction

* updated

* use performRequests

* updated CHANGELOG

* added tests, fixed segfault

* use throwNotRunning

* use read transactions for testing (mmfiles blocks with multiple write
transactions on the same collection)
2019-07-26 16:34:57 +03:00
Michael Hackstein d5840c125a Bug fix 3.5/min replication factor (#9524)
* Cherry-pick minReplicationFactor

* Bug fix/failover with min replication factor (#9486)

* Improve collection time of IResearchQueryOptimizationTest

* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* Initial commit, old plan, does not yet work

* fixed ires tests

* Included FailoverCandidates key. Not fully implemented

* fixed wrong assert

* unified in sync follower reporting

* fixed compiler errors

* Cleanup locking, and fixed potential deadlocks

* Comments about locking order in FollowerInfo.

* properly check uint

* Keep old leader as potential failover candidate

* Transaction methods now use followerInfo to check if the leader can write, this might have the sideeffect that 'failoverCandidates' are updated

* Let agency check failoverCandidates if possible

* Initialize member variables

* Use unified follower reporting in DBServerAgencySync

* Removed obsolete variable, collecting it somewhere else

* repl factor attr check

* Reimplemented previous followers, second attempt now. PhaseOne and PhaseTwo can now synchronize on current.

* Fixed assertion, forgot an off-by-one

* adjusted test to be more preciese now

* Fixed failove candidates list

* Disable write on dropping too many followers

* Allow to run updateFailoerCandidates multiple times with same leader.

* Final fixes, resilience tests now green, crossing fingers for jenkins

* Fixed race on atomics comparison

* Fixed invalid number type

* added nullptr handling

* added nullptr handling

* Removed invalid assert

* Make takeover of leadership an atomic operation

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* Fixed creation code to use takeoverLeadership

* Update arangod/Cluster/FollowerInfo.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Applied review fixes

* There is no timeout

* Moved AQL + Pregel to INTERNAL_AQL lane, which is medium priority, to avoid deadlocks with Sync replication

* More review fixes

* Use difference if you want to compare two vectors...

* Use std::string ...

* Now check if we are in recovery mode

* Added documentation for minReplicationFactor

* Added readme update as well in documenation

* Removed merge conflict leftovers 0o, i should not trust the IDE

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/Architecture/Replication/README.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Collections/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Graph/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Adepted review requests, thanks for finding!

* Removed unnecessary const

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Moved initilization of variable more downwards

* Apply lock before notify_all()

* Remove documentation except DocuBlocks, covered by PR in docs repo

* Remove accidental indent
2019-07-22 17:48:34 +03:00
Wilfried Goesgens 9781b9890c Bug fix 3.5/connect mocha results to testing js (#9493)
* connect the mocha testresults if we run it in an extra arangosh instance

* fix status handling, lint

* sometimes we have the string undefined to ignore

* fix filter forwarding for spawned arangoshs - add mocha version

* migrate mochaGrep into --testCase parametrizing
2019-07-18 19:49:03 +03:00
Wilfried Goesgens 623048aff8 fix 6561f21536 - only abort on timeout, not regular exit failures (#9434)
* fix 6561f21536 - only abort on timeout, not regular exit failures

* make pre-check exception distinguishable, its got no ready existing test status
2019-07-10 18:48:46 +03:00
Wilfried Goesgens 0ad76a8185 fix location - while they work with the server these tests are run in arangosh so they need to be in client/ (#9435)
* fix location - while they work with the server these tests are run in arangosh so they need to be in client/

* rename testsuite

* rename file as its testsuite

* Rename permissions_server to server_permissions
2019-07-10 16:49:12 +03:00
Andrey Abramov b3690fc24f
bug-fix/internal-issue-#595 (#9440)
* address ASAN failures in tests

* Add --isAsan for testing.js

* Fix typo
2019-07-10 01:11:22 +03:00
KVS85 c05288fa59
Fix recovery tmp dir 2019-07-09 18:13:25 +03:00
Wilfried Goesgens c2b645062b if there is no testcase, we still 'splitted' the bucket (#9425) 2019-07-08 16:44:54 +03:00
Wilfried Goesgens 3599aedda1 Don't wait eternaly; don't unload collections with rocksdb (#9415) 2019-07-05 17:13:24 +02:00
Tobias Gödderz 96685618d6 Don't use non-existent variable (#9407) 2019-07-04 14:41:30 +02:00
Wilfried Goesgens 5bab188907 improve handling when procdump detects the process is dead (#9381) 2019-07-04 12:38:57 +02:00
Wilfried Goesgens 081e755b25 fix typo (#9400) 2019-07-03 16:17:55 +02:00
Wilfried Goesgens 61f3397e33 Feature/add tcpdump support (#9396) 2019-07-03 16:12:03 +02:00
Wilfried Goesgens 3f9b85967d apply filters before starting the server, so we can detect whether no test would be executed (#9387)
* apply filters before starting the server, so we can detect whether no test would be executed

* skipTimecritial and skipNondeterministic also qualify for non-errnous empty testcase

* simplify code, if  is issued no matching testcase is an error

* Update test-utils.js
2019-07-03 14:13:16 +02:00
Wilfried Goesgens 46479af842 keep coredump for reports (#9378) 2019-07-02 10:09:12 +02:00
Jan Christoph Uhde 782cb49105 Feature: Analyzer REST-API Tests (#9091)
* add analyser http API tests

* add more tests

* add tests and lint

* remove print

* add another test

* fix formattig

* add e6 expresseion as key in object

* WIP: add analyzer permissions test

* fix test

* lint

* WIP: try to add analyzer auth-test

* add sleepBeforeStart option

* use basic auth

* fix tests

* change test so it would work if the api was as expected

* fix authentication test

* fit shell client analyzer test

* fix shell-analyzer-rest-api.js

* fix auth-analyzer.js

* fix user-access-right-update-view-arangosearch.js

* remove print statements

* Fixed analyzer parameters for test
2019-06-28 16:21:09 +02:00
Wilfried Goesgens 51e687d4fa Feature/add output before http ing (#9361)
* make rspec timeouteable (on windows for now)

* add timeout state, fix windows implementation

* abort testrun if a timeout in rspec occured

* lint

* fix scope of procdump stopper

* fix include scope

* fix invokation

* use proper variable to wait for

* add output before we attempt to talk to the SUT, so we can better insulate startup lockups

* when we timeout a process dump out all we know about the SUT.

* when we timeout a process dump out all we know about the SUT.
2019-06-28 16:03:32 +02:00
Wilfried Goesgens 6561f21536 Feature/make rspec timeouteable (#9354)
* make rspec timeouteable (on windows for now)

* add timeout state, fix windows implementation

* abort testrun if a timeout in rspec occured

* lint

* fix scope of procdump stopper

* fix include scope

* fix invokation

* use proper variable to wait for
2019-06-28 15:01:04 +02:00
Wilfried Goesgens 3c6264b9b0 wait .25, then check whether procdump successfully starts, and if not, properly abort the operation (#9333) 2019-06-26 19:08:38 +02:00
Wilfried Goesgens 63568a2aac add a function that will use procdump to generate a core before attempting to kill the SUT with *nix coredump enabled signals (#9322) 2019-06-25 16:25:25 +02:00
Wilfried Goesgens bf9328cd18 properly forward corecheck option to the client tools launch invokations (#9313)
* properly forward corecheck option to the client tools launch invokations

* one param was missing
2019-06-25 11:24:55 +02:00
Wilfried Goesgens f947954f6c fix the option to specify a ruby executeable from the outside in instead of leaning on rspec.bat to do its job (#9315) 2019-06-24 20:39:30 +02:00
Wilfried Goesgens f3b1de9808 adopt option to do the right thing with gtests (#9280) 2019-06-18 11:15:09 +02:00
Wilfried Goesgens a866f8c6fd update gtest, use json output for reporting (#9193) 2019-06-17 13:46:40 +02:00
Jan 59d84e8301 attempt to remove expired WAL files on agents (#9231) 2019-06-09 19:46:45 +02:00
Jan 091abbec77 don't make RocksDB startup fail with 0-byte log files (#9208) 2019-06-07 15:13:56 +02:00
Jan 074397d367
allow "stream" attribute on top level (#9195) 2019-06-05 15:48:58 +02:00
Wilfried Goesgens 1d2071420b output setup failures to testfailures.txt (#9188) 2019-06-05 09:55:54 +02:00
Dan Larkin-York 44a413a9af Miscellaneous fixes for named indices (#9100) 2019-05-31 17:00:56 +02:00
Frank Celler 7dbb26c0c0 added missing semicolon 2019-05-29 22:58:56 +02:00
Tobias Gödderz bdfa6c6951 Collect all tests that are skipped due to a previous crash (#9011) 2019-05-29 15:32:57 +02:00
Wilfried Goesgens 065b87f9a8 preserve the instance shutdown state, and if its bad, don't clean up the test directory (#9122)
* preserve the instance shutdown state, and if its bad, don't clean up the test directory

* Update js/client/modules/@arangodb/testing.js

Co-Authored-By: Tobias Gödderz <github@tobias.goedderz.info>

* Update js/client/modules/@arangodb/testsuites/permissions_server.js

Co-Authored-By: Tobias Gödderz <github@tobias.goedderz.info>

* Update js/client/modules/@arangodb/testsuites/permissions_server.js

Co-Authored-By: Tobias Gödderz <github@tobias.goedderz.info>

* Update js/client/modules/@arangodb/testsuites/single.js

Co-Authored-By: Tobias Gödderz <github@tobias.goedderz.info>

* fix handling in custom start/stop handlers

* remove shutdown state from reporting structure

* fix shutdown of second instance

* one more to adjust

* only add jwt if we run the agency with authentification

* lint
2019-05-29 15:59:34 +03:00
Wilfried Goesgens b7ad25b3fe only add whitespace if we have actual content to separate (#9062) 2019-05-23 14:52:18 +02:00
Wilfried Goesgens 8db0acae79 control the agency dumper via whether we launched an agency or not (#9058) 2019-05-23 14:50:44 +02:00
Dan Larkin-York d5ecdd143a Convert unit tests to googletest framework (#9034) 2019-05-21 09:17:46 +02:00
Wilfried Goesgens 3c5193231f we now support umlaut temp dirs; create multi-layer temp-dirs; add tests (#9008)
* we now support umlaut temp dirs; create multi-layer temp-dirs; add tests

* Fix log
2019-05-15 21:47:05 +02:00
jsteemann 8fabe21a79 fix jslint 2019-05-15 14:35:50 +02:00
Wilfried Goesgens 5171771dc8 switch the temp path too (#9000) 2019-05-15 12:16:28 +02:00
Andrey Abramov 99f3a1fb75
add missing quote (#8970) 2019-05-11 18:30:29 +03:00
Vasiliy 42b830c7a9 issue 561.1: add pretty-printing to analyzers in JavaScript (#8964) 2019-05-10 21:50:15 +03:00
jsteemann d620d5c720 jslint 2019-05-09 19:08:49 +02:00