1
0
Fork 0
Commit Graph

59 Commits

Author SHA1 Message Date
Jan 4f7923a971
remove replication clients handling out of vocbase code (#8676) 2019-04-08 19:15:28 +02:00
Jan f16b75440c
make ownership a bit more transparent, API-wise (#8653) 2019-04-03 18:47:04 +02:00
Jan 616ea94f24
Bug fix/cleanup 31032019 (#8632) 2019-04-01 17:14:11 +02:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Jan e078f35285
fixed typos, removed unneeded includes (#8547) 2019-03-25 12:09:37 +01:00
jsteemann d0dba4f941 make sure some tick invariants always hold true 2019-03-12 16:47:12 +01:00
Jan 12e11a5197
port of replication improvements from 3.4 (#8308) 2019-03-11 13:37:18 +01:00
Simon dc24d7fa4e Sync Foxx Queues (#8250) 2019-02-25 17:13:14 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon fd81f52ab4 Allow WAL logger to split up transactions (#6800) (#6866) 2018-10-12 17:50:58 +02:00
Jan 9b6d526942
Bug fix/increase replication timeouts (#6740) 2018-10-08 09:40:38 +02:00
Dan Larkin-York 6660cb3e75 Cleanup logging in MMFiles. 2018-10-01 16:30:10 -04:00
Jan 791eaba873 Bug fix/planning 2865 (#6252)
* potential bugfix for planning/#2865

* speed up dump tests setup

* enable authentication for backup tests

* make arangodump provide a "serverId" to the server

this allows the server to track arangodump as an active dump client
so any data required for the dumping may be retained while the
dump is ongoing

* don't log binary stuff into the logfile
2018-08-27 10:21:34 +02:00
Jan a5ef080a8a
attempt to make replication_sync more reliable in tests for MMFiles (#6184) 2018-08-17 14:20:40 +02:00
Jan 29ab2fdbe3
Bug fix/3107 (#6046) 2018-08-01 12:50:55 +02:00
Simon f47367a495 Allow dumping / restoring of views (#5774)
* allow dumping and restoring of views

* fix restore, adding some tests

* remove debug print

* fix global replication
2018-07-05 17:55:28 +02:00
Dan Larkin-York 820bfee329 Refactor syncer state and make notes for future parallelization (#5742) 2018-07-03 21:32:16 +02:00
Vasiliy 94ddd7803d issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424) 2018-05-23 00:59:08 +03:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Simon 08e355aed8 Simple dump speedup (#5298) 2018-05-09 12:51:04 +02:00
Simon 7677afabf1 Remove copy of request body in rest handlers (#5104) 2018-04-16 14:49:51 +02:00
Vasiliy 635db3b409 issue 355.4: remove redundant methods and code, use 'cp' instead of 'cmake copy_directory' where possible, use vocbase reference instead of pointer 2018-04-04 10:53:48 +03:00
Andrey Abramov 01d9baf359 remove TRI_ERROR_ARANGO_VIEW_NOT_FOUND, rename TRI_ERROR_ARANGO_COLLECTION_NOT_FOUND to TRI_ERROR_ARNANGO_DATA_SOURCE_NOT_FOUND 2018-03-17 19:36:14 +03:00
Vasiliy 148bdb7158 issue 344.6: remove some redundant functions (#4842) 2018-03-15 11:03:35 +01:00
Vasiliy c8739cd3cd manually-merge: cache data-sources in CollectionNameResolver 2018-03-14 10:11:50 +03:00
Simon 1b572c52c2 Active-Failover VelocyStream Fixes (#4715) 2018-03-05 15:54:19 +01:00
Simon 950e8c3e7b Fixing issues related to missing auth rights (#4753) (#4754)
(cherry picked from commit 38a99d8f65)
2018-03-05 13:48:51 +01:00
Wilfried Goesgens f489b7d5de fix dumping permission checks; (#4725) 2018-03-01 17:17:11 +01:00
Jan 935e2790ad
forward port of replication bugfixes from 3.2 (#4572) 2018-02-12 18:13:51 +01:00
Simon 35136a89c0 Fix some problems with active failover (#4540) 2018-02-09 15:11:53 +01:00
Jan 8da61a9875
Bug fix/more replication tests (#4500) 2018-02-09 13:06:47 +01:00
Jan 1e116a9f6d
slight cleanup of replication-related code (#4286) 2018-01-17 16:56:40 +01:00
Jan 9c76613e63
fix premature unlock (#3802)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 13:27:42 +01:00
Jan eb0190066b
allow passing a serverId value of "none" to replication APIs (#3909) 2017-12-08 11:35:53 +01:00
Simon Grätzer c6fe726901 Removing unsafe asserts in wal tailing v3.3 (#3672) 2017-11-13 17:41:57 +01:00
Jan 03daf9e9c9 make incremental sync replication API check if the client passed an "offset" request parameter (#3669)
if yes, the server may dynamically adapt the size of the response, in order to ensure that
HTTP responses do not get out of hand size-wise. This is a new feature in devel, and this
commit now makes it optional so that older clients do not need to be changed.
2017-11-13 15:08:31 +01:00
Simon Grätzer 87f441753b RocksDB WAL tailing fixes (#3595) 2017-11-10 09:31:53 +01:00
Jan 4d03d3bd82
Bug fix/fixes 2410 (#3511)
* slightly clean up indexes

* proper stringification of error code

* improve diagnostic messages

* improve memory usage for incremental sync
2017-10-30 22:49:24 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Jan 79d76c89fb Bug fix/use count for checksum (#3033)
* Count as checksum

* Make readLockId optional as well so upgrades are possible

* fix option name in startup script

* fix some replication issues with RocksDB engine
2017-08-15 11:24:54 +02:00
Michael Hackstein 7ab117f5f8 Bug fix/dump restore auth (#2917)
* Added a backup test suite. This suite is supposed to entirely drop an ArangoDB _system and restore it into a fresh one. This also includes system collections.

* Added more test cases for backup suite. Now tests several authorization/rights cenarios

* Fixed RestReplication Handlers to restore _user collections Properly.

* Updated Changelog

* Added special handling of _users in Restore for MMFiles as well.

* Added JWT secret for cluster execution of this test, also added JWT secret to shutdown call
2017-08-09 12:59:51 +02:00
Jan da6e63c696 fixed issue #2949 (#2951) 2017-08-04 13:36:37 +02:00
m0ppers 9a0bc716d0 Do not allow replication to create/drop collections (#2898)
In the cluster case the only one who is allowed to do this is the schmutz
2017-07-28 14:24:02 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
jsteemann a68c95d2b7 try to honor server shutdown while locks are held 2017-06-01 02:24:00 +02:00
jsteemann 0366fba276 try to fix thread joining for replication applier 2017-05-18 21:05:57 +02:00
Max Neunhoeffer f614439416 Squashed commit of the following:
commit 19a7210119cc284af64251b202f690ab62bf615c
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 23:20:10 2017 +0200

    Allow access to /_api/user/ without database check in VST as in HTTP.

    This is a try to fix #2465

commit f973c1335652540174bf6b78df42290bded357da
Merge: 30bdc4c 365de45
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 21:32:15 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 30bdc4ca38e028d58cebea8268683d9f94e87ad7
Merge: 14acaea 086f6a2
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 14:52:23 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 14acaea8cb5fff2c1c8c27eee32294bfd612f629
Merge: 5bd0e79 fb088d3
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:40:29 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 5bd0e79ab053447f94f2fd959a33b1e15302b6a8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:36:23 2017 +0200

    Fix bug in authentication in VST

commit 9a999ce5e629aee5816a4f61b194b086d11fd169
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:24:35 2017 +0200

    Open up endpoints /_open/* in VST without being authenticated

commit 32f62db42af6d5bfc24214ebb4ca7c1fb473b55e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:11:31 2017 +0200

    Simplify logic by not using boost::optional.

commit 925ce2f7b890c6f14205fd3e98553939d89bec61
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 12:11:20 2017 +0200

    Add JWT case for authentication for VST

commit f1d7f67a9fc20c91ca000e9a4dad91e0f50f1652
Merge: 040ace3 7a9ccc9
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 08:58:51 2017 +0200

    Merge remote-tracking branch 'origin/devel' into vst-cleanup

commit 040ace3e9dce47ddea5f51d29f0153a70d257c9e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 08:23:16 2017 +0200

    Write out chunk buffer using little endian uints

commit 66ad4c0e8d3bc94091664505986b00e7cff39f2f
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri May 12 08:12:38 2017 +0200

    Move maxChunkSize query to constructor, remove dead code

commit f7b4c26cc09c6d512362340c6bf6061bbaae61a9
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 17:57:14 2017 +0200

    Do not take protocolVersion from request for VST.

commit c76d6685f9507701715abf882f0ac0e192ead59f
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:20:58 2017 +0200

    Remove dead code for compression which did never work well.

commit b823765b7c041dc5d143f7ab2e13d92f5f42d3eb
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:09:53 2017 +0200

    Fix renaming to make compiler happy.

commit b327830012d146f4f4e2d0f0467dab351fbd3bd0
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:07:36 2017 +0200

    Rename _defaultPortVpp to _defaultPortVst.

commit 5eb98a9e64d702476e8ec098b7506b1a63145d64
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:06:28 2017 +0200

    Rename header guards VPP -> VST.

commit a3e96be26d49d6151898d8d757ede6225ac6abd2
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:05:55 2017 +0200

    Rename VppRequest and VppResponse files to Vst.

commit 1d5f6f196490c3cf9312be5927b967b4abc03b67
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:04:19 2017 +0200

    Change names of header guards VPP -> VST.

commit f004b25fbef75050b34bcd4eae4eb17b8b467230
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 13:03:19 2017 +0200

    Rename files VppNetwork.h and VstMessage.h

commit c04d3aec19e8f0a8a3b90cd48e7736bff037359d
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 12:59:15 2017 +0200

    Implement correct chunk header for sending out VelocyStream 1.1 messages.

commit 73aeedfbbef24d5929c44bd26c16343480514270
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 01:11:23 2017 +0200

    A lot more renaming VPP -> VST.

commit 2acedc77f16a82251bc4f9e39526184e2a6f0dec
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:49:20 2017 +0200

    Rename header guard.

commit 2ade43946aa5ea0048dab81eed5ffd931ce35ea8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:48:01 2017 +0200

    Rename files VppCommTask.* to VstCommTask.*

commit 0ffcda02a51cbad35e5ec06117d397aac37323cd
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:46:31 2017 +0200

    Rename class VppCommTask to VstCommTask.

commit 9478c6c56c995b014b7a14408544b76fa676a317
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:30:47 2017 +0200

    Rename protocol versions to VST in ProtocolVersion enum.

commit 0b69b7add6d7b5155ac71b8257def282ab4a1637
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:29:29 2017 +0200

    Recognize incoming chunk headers in VST/1.1 format.

commit f40173c4fc6dc48613ae67ac3fd810d002c9b6cd
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:11:54 2017 +0200

    Recognise and remember VST/1.0 and VST/1.1 protocol versions.

commit 29c798dc1af2d650c41eb9ed06f635786faf10f6
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 11 00:11:23 2017 +0200

    Create protocol version VST/1.1 and disable vst+ prefix for endpoints.
2017-05-12 23:22:22 +02:00