1
0
Fork 0
Commit Graph

94 Commits

Author SHA1 Message Date
Simon 0d955554f2 Use shared_ptr for LogicalCollection (#7220) (#7244) 2018-11-07 10:43:08 +01:00
Jan 452c70acdd
fix premature replicatin context deletion in case the context has no iterators (#7187) 2018-11-02 12:14:19 +01:00
Simon 9271a11441 RocksDB replication thread safety (#7088) (#7131) 2018-10-30 11:24:17 +01:00
Simon 010b54c81e Allow WAL logger to split up transactions (#6800) 2018-10-12 15:15:55 +02:00
Jan 791eaba873 Bug fix/planning 2865 (#6252)
* potential bugfix for planning/#2865

* speed up dump tests setup

* enable authentication for backup tests

* make arangodump provide a "serverId" to the server

this allows the server to track arangodump as an active dump client
so any data required for the dumping may be retained while the
dump is ongoing

* don't log binary stuff into the logfile
2018-08-27 10:21:34 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Simon Grätzer a7c78a3f37 Merge branch 'devel' of github.com:arangodb/arangodb into bug-fix/improve-intermediate-commits
# Conflicts:
#	arangod/RocksDBEngine/RocksDBIncrementalSync.cpp
#	arangod/RocksDBEngine/RocksDBIterators.cpp
2018-07-31 15:19:57 +02:00
Jan 2a416f2e33
Bug fix/3007 (#6019) 2018-07-30 17:16:47 +02:00
Simon Grätzer 1cfb328bcc fix intermediate commits 2018-07-27 17:40:38 +02:00
Jan 21064144c8
Bug fix/replication improvements (#5962) 2018-07-25 09:04:50 +02:00
Simon a13ec34053 keep replication batches alive for longer (#5932)
*  keep replication batches alive for longer

* latest changes
2018-07-20 11:17:20 +02:00
Dan Larkin-York 820bfee329 Refactor syncer state and make notes for future parallelization (#5742) 2018-07-03 21:32:16 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Jan 0d98c0f3ba
Bug fix/fix codescan issues (#5371) 2018-05-16 20:40:37 +02:00
Jan Christoph Uhde 18a3663746 speed up replication (#5226) 2018-05-14 18:32:26 +02:00
Simon 08e355aed8 Simple dump speedup (#5298) 2018-05-09 12:51:04 +02:00
Vasiliy 0abd46ad73 backport: use vocbase reference instead of pointer in arangodb::pregel::GraphStore 2018-04-13 11:23:34 +03:00
Vasiliy f392925903 issue 374.3: use a reference to vocbase instead of a pointer in DatabaseGuard 2018-04-13 09:56:49 +03:00
Dan Larkin-York 898f3e1dd0 Fix compiler warnings/issues from arangodump parallelization for Mac. (#4965) 2018-03-26 17:36:21 +02:00
Dan Larkin-York 38f162e344 Parallelize arangodump (#4356) 2018-03-26 15:55:14 +02:00
Vasiliy 06eb8ade01 issue 344.7: remove more redundant functions (#4863)
* issue 344.7: remove more redundant functions

* backport: fix missed functions under USE_ENTERPRISE
2018-03-15 17:10:28 +01:00
Vasiliy c8739cd3cd manually-merge: cache data-sources in CollectionNameResolver 2018-03-14 10:11:50 +03:00
Simon 1b572c52c2 Active-Failover VelocyStream Fixes (#4715) 2018-03-05 15:54:19 +01:00
Simon 345fc3c0b7 Refactor Authentication Layer (devel) (#4592)
* Cherry Picking LDAP changes

* Adding missing merges

* Fixing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

(cherry picked from commit 686d28a779)

* Fixing wrong assert

* Adding changelog entry, making requested changes from code review

* Fixing dump_authentication, fix typos

* improvements found during code review

* oops

* more use of sessionstorage

* fix tests

* Fixing broken handling, disallowing adding of local users when disabled

* Fixing testInvalidGrants

* Removing undefined auth level externally

* Fixing previous commit

* added tests for ldap search mode

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* forward port changes from 3.3

* added generated files

* forward port missing changes for web UI

* added generated files

* added generated files
2018-02-28 13:24:28 +01:00
Jan 8da61a9875
Bug fix/more replication tests (#4500) 2018-02-09 13:06:47 +01:00
Jan 1e116a9f6d
slight cleanup of replication-related code (#4286) 2018-01-17 16:56:40 +01:00
Jan b2b6c06cbf
Feature/efficiency (#3736) 2018-01-05 16:51:31 +01:00
Jan 82abb24388
add missing replication context fix (#4066) 2017-12-18 09:16:19 +01:00
Jan 9c76613e63
fix premature unlock (#3802)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 13:27:42 +01:00
Jan 73b3c65153
fix revision id vs local document id usage in incremental sync (#3887) 2017-12-06 16:33:57 +01:00
Jan 3b5b1e4eba
fix replication context locking (#3875) 2017-12-06 12:11:55 +01:00
Jan 3b0a8a9cdf make the replication applier auto-start for the RocksDB engine if con… (#3647) 2017-11-15 12:02:37 +01:00
Jan 03daf9e9c9 make incremental sync replication API check if the client passed an "offset" request parameter (#3669)
if yes, the server may dynamically adapt the size of the response, in order to ensure that
HTTP responses do not get out of hand size-wise. This is a new feature in devel, and this
commit now makes it optional so that older clients do not need to be changed.
2017-11-13 15:08:31 +01:00
Simon Grätzer 87f441753b RocksDB WAL tailing fixes (#3595) 2017-11-10 09:31:53 +01:00
Jan 4d03d3bd82
Bug fix/fixes 2410 (#3511)
* slightly clean up indexes

* proper stringification of error code

* improve diagnostic messages

* improve memory usage for incremental sync
2017-10-30 22:49:24 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Max Neunhöffer 98e6484d86 Fix bug in RocksDB incremental collection sync for empty colls. (#3207)
Also: Make hash computations for chunks independent.
2017-09-06 15:26:03 +02:00
Jan 39465193c2 fix replication (#3099) 2017-08-22 15:51:50 +02:00
Jan Christoph Uhde df76bbc690 Bug fix/planning issue 514 replication api (#3021)
* add "cluster selectivity estimates" to CHANGELOG

* add some documentation to RocksDBRestReplicationHandler

* fix building with relative paths

* add some more doc

* add some tests for the replication api

* fix RocksDBRestReplicationHandler and add tests

* update documentation

* remove obsolete parameter

* fix error message

* Implementing logger-first-tick, logger-tick-ranges. Fixing dump `chunkSize` documentation
2017-08-22 10:46:06 +02:00
Frank Celler a692577f5b Feature/auth context (#2815) 2017-07-16 09:39:03 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713)
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
2017-07-05 10:18:52 +02:00
Jan ebaeb639b2 Bug fix/incremental replication syncs too much (#2631)
* fix too much synchronization
* fix incremental sync for RocksDB engine
2017-06-21 14:56:33 +02:00
Simon Grätzer 72661d85c3 fixed incremental replication 2017-05-31 18:08:05 +02:00
Simon Grätzer 469e57773e workaround for replication 2017-05-31 16:37:58 +02:00
Simon Grätzer 87ee7a45fe Adding error checks 2017-05-31 15:10:17 +02:00