1
0
Fork 0
Commit Graph

419 Commits

Author SHA1 Message Date
Jan 8e65fbb539 Bug fix/issues 1708 (#3060)
* we must now ignore that datafiles are not sealed

this is because an unsealed datafile may have been produced by
renaming multiple journals to datafiles at server start

* acquire collection count after we have acquired the lock

* count the null byte as well

* fix count value acquisition

* send query fragments to the correct servers, even after failover or when a follower drops

the problem with using the previous shard-based approach is that responsibilities for shards may change at runtime
however, an AQL query must send all requests for the query to the initially used servers.
if there is a failover while the query is executing, we must still send all following requests to the same servers, and not the newly responsible servers
otherwise we potentially would try to get data from a query from server B while the query was only
instanciated on server A.
2017-08-17 21:48:27 +02:00
Jan 6180fcfdd1 Bug fix/prevent multiple journals (#3027)
* prevent multiple journals

* fix documentation

* remove _nrDesired, as it is not used anymore
2017-08-15 23:02:08 +02:00
Jan 79d76c89fb Bug fix/use count for checksum (#3033)
* Count as checksum

* Make readLockId optional as well so upgrades are possible

* fix option name in startup script

* fix some replication issues with RocksDB engine
2017-08-15 11:24:54 +02:00
Jan 8e4dac4fc4 fix fulltext index removal performance, simplified code (#3015)
* simplify index API a bit

* fix fulltext index removal performance, simplified code

* updated CHANGELOG

* fix hanging test

* try to fix shutdown problem

* improve fulltext query performance

* fixed duplicate var

* removed obsolete code

* fix some shutdown races

* do not call ensureIndex that often
2017-08-12 09:25:30 +02:00
Michael Hackstein 7ab117f5f8 Bug fix/dump restore auth (#2917)
* Added a backup test suite. This suite is supposed to entirely drop an ArangoDB _system and restore it into a fresh one. This also includes system collections.

* Added more test cases for backup suite. Now tests several authorization/rights cenarios

* Fixed RestReplication Handlers to restore _user collections Properly.

* Updated Changelog

* Added special handling of _users in Restore for MMFiles as well.

* Added JWT secret for cluster execution of this test, also added JWT secret to shutdown call
2017-08-09 12:59:51 +02:00
Jan ec5dde49ec remove memset parameter from TRI_Allocate signature (#2993) 2017-08-08 16:07:32 +02:00
Jan 9088257af3 specialize IndexIterator::skip for primary indexes (#2952)
this provides a speedup of about 10% for LIMIT offset, limit operations
2017-08-06 14:12:17 +02:00
Jan da6e63c696 fixed issue #2949 (#2951) 2017-08-04 13:36:37 +02:00
Jan 52d5941897 fix potential deadlock errors in collector thread (#2933)
* fix potential deadlock errors in collector thread

* add test
2017-08-03 22:47:24 +02:00
Simon Grätzer 3f58a44e3b Fix cases where the root user has access to excluded collections (#2856) 2017-08-03 21:16:08 +02:00
Jan ed9d15156e remove dependency on MMFiles features from non-MMFiles files (#2925) 2017-08-01 22:16:43 +02:00
Jan 4f5d5ff54d slightly move responsibility for recovery (#2922) 2017-08-01 14:25:13 +02:00
Jan Christoph Uhde ed8efe3566 Feature/issue 387 cluster index estimates (#2866) 2017-08-01 09:53:58 +02:00
Jan 87567c3c2a fixed issue #2876 (#2896) 2017-07-30 14:24:25 +02:00
Jan beea9033d3 fix a race on shutdown (#2897) 2017-07-29 10:30:24 +02:00
m0ppers 9a0bc716d0 Do not allow replication to create/drop collections (#2898)
In the cluster case the only one who is allowed to do this is the schmutz
2017-07-28 14:24:02 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Simon Grätzer 59dc8d0135 Update rocksdb options (#2880)
* changing some defaults

* Maybe fixing performance

* Modified changelog

* Updated documentation to reflect options
2017-07-27 16:05:31 +02:00
Jan 582d3e5fd1 only sync RocksDB WAL in MMFiles engine when there is a reason to do so (#2863) 2017-07-26 21:49:41 +02:00
Jan 0771a69138 remove only partly used AQL function argument types definitions (#2873) 2017-07-26 13:03:30 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan 381869e3ec fix WAL file tick updating (happened too late before) (#2762) 2017-07-10 17:41:08 +02:00
Jan 5c25b6934e fix issue #2744 (#2745) 2017-07-07 12:39:02 +02:00
Jan 27dc53b451 Bug fix/cleanup stuff (#2734)
* cppcheck

* remove unused files

* cppcheck just sucks

* add _FORTIFY_SOURCE
2017-07-05 19:34:16 +02:00
Frank Celler bbe7484521 Feature/auth context (#2704)
* added read-only users
2017-07-02 23:15:57 +02:00
Jan ccbe1ba792 Bug fix/fix rocksdb options (#2702)
* cleanup rocksdb options
2017-07-02 23:13:42 +02:00
Jan cf4368881d change init order of mmfiles logfile manager and database feature (#2697)
database feature now comes first. this ensures that the mmfiles collector thread
(owned by the mmfiles logfile manager) can always access the list of databases,
and that this list is not destroyed while the collector thread is still running.
2017-07-01 08:47:57 +02:00
Jan a5afdf9437 Bug fix/small issues (#2696)
* remove unused code

* print values of all options

* don't print "warning(s):" if there are none

* remove remainders of old file deletion functionality

* remove unused function
2017-07-01 00:30:45 +02:00
Jan 94b4a9ec4b added "deduplicate" attribute for array indexes (#2644) 2017-06-30 17:53:50 +02:00
Dan Larkin ee8cb749bb Added recovery tests for views and fixed a few related bugs. (#2682)
* Added recovery tests for views and fixed a few related bugs.

* Added more view recovery tests.

* Modified view recovery tests to add a waitForSync operation after.

* fixed usage of wrong view type

* fixed recovery of view change markers
2017-06-29 17:09:43 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Jan cfbd8ed93c Bug fix/rocksdb autoincrement (#2648)
* Added autoincrement keygen support to RocksDB engine with test.

* fixed key generator state keeping for mmfiles engine
2017-06-23 15:27:09 +02:00
Jan 3320052f5d do not include our datafiles in core files (#2630) 2017-06-22 12:34:27 +02:00
Jan ebaeb639b2 Bug fix/incremental replication syncs too much (#2631)
* fix too much synchronization
* fix incremental sync for RocksDB engine
2017-06-21 14:56:33 +02:00
Frank Celler ce8b1fb885 Feature/rocksdb update 5.6 (#2621) 2017-06-20 18:12:43 +02:00
Jan 67016ba560 potentially fixed issue #2559: Duplicate _key generated on insertion (#2593) 2017-06-19 17:44:59 +02:00
Jan 5cd9ad7ee1 fail with exception and not with undefined behavior when the expected primary index is not there! (#2590) 2017-06-18 22:48:15 +02:00
Jan 2f04981fe9 modified error message so it becomes clear that the mmfiles engine also uses rocksdb internally, but only for persistent indexes (#2545) 2017-06-12 11:34:13 +02:00
jsteemann ad9fa2bd78 cleanup error codes a bit 2017-06-09 14:50:05 +02:00
jsteemann b9c000253d fix tick assignment
arangod may report a "lastLogTick" value of "0" when a request to
/_api/replication/logger-state was made before the first WAL write
operation was synced to disk. After the first sync operation, the
"lastLogTick" values were ok then.
Now a "lastLogTick" value of "0" should not be returned even when
the logger state API is queried directly after the startup.
2017-06-08 22:12:23 +02:00
Jan 5ad9423f16 unify index struct return values in cluster and non-cluster for fulltext and geo indexes (#2534) 2017-06-08 13:35:00 +02:00
jsteemann 6acfb4ff97 fix invalid new/free pairings 2017-06-07 13:21:41 +02:00
jsteemann 3c1ecb03a4 slightly less allocations for MMFiles transaction ops 2017-06-07 00:30:18 +02:00
jsteemann e38705ec24 fixed logic error in index code 2017-06-06 23:59:06 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
jsteemann 12132c60c7 improve waitForSync performance for mmfiles 2017-06-06 12:44:39 +02:00
Simon Grätzer e5386aef30 fixing compiler warnings and edge cache test 2017-06-02 16:00:41 +02:00