1
0
Fork 0
Commit Graph

202 Commits

Author SHA1 Message Date
jsteemann 3883501001 fix compile error 2018-02-13 22:28:06 +01:00
Dan Larkin 96dbc693db Speed up condition normalization (DNF conversion) (#4574) 2018-02-13 22:13:52 +01:00
Simon 35136a89c0 Fix some problems with active failover (#4540) 2018-02-09 15:11:53 +01:00
Jan 7bb1c88663
speed up collection.any() for mmfiles (#4433) 2018-02-08 11:39:40 +01:00
Andrey Abramov a1cfb3d72b Feature iresearch (#4105) 2018-01-19 14:23:58 +01:00
Jan b2b6c06cbf
Feature/efficiency (#3736) 2018-01-05 16:51:31 +01:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Matthew Von-Maszewski e6f7282e03 Make chooseTimeout() dynamic (#3996)
* add parameter to increase timeouts per 4096 size of total request package

* remove log output

* change initial timeout math from integer to double ... avoids reviewer confusion.
2017-12-15 18:25:18 +01:00
Jan 9c76613e63
fix premature unlock (#3802)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 13:27:42 +01:00
Jan ec7da71287
do not swallow error messages when transactions fail (#3836) 2017-12-06 10:49:28 +01:00
m0ppers d9f006cb84 Propagate commit errors to AQL (#3831) 2017-11-29 16:56:01 +01:00
Jan b4f6ee9273 Feature/improved index api for unique constraints and replication (#3715) 2017-11-16 21:02:01 +01:00
Jan 5abf0c1185 Bug fix/fixes 1511 (#3711) 2017-11-16 14:18:51 +01:00
Jan cc1fc716a4 Bug fix/fix server mode test side effects (#3674) 2017-11-14 15:19:00 +01:00
Jan 8a467e74db try to not crash when encountering a collection that is not present (#3660) 2017-11-11 19:30:37 +01:00
jsteemann 571d4587ca Revert "Return offending key for unique constraint violations (#3624)"
This reverts commit 68bd31ac99.
2017-11-10 16:40:56 +01:00
Dan Larkin 68bd31ac99 Return offending key for unique constraint violations (#3624) 2017-11-10 16:03:10 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Simon Grätzer ee8209943f Missing things for active / passive (#3578)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* auth = off does not longer make everyone superuser

* Fixing cluster_sync and maybe resilience
2017-11-04 20:30:23 +01:00
Jan c569c1b816 fix test (#3524) 2017-10-26 14:04:41 +02:00
Simon Grätzer 09840239c9 Fixed wait for sync in mmfiles (#3478) 2017-10-23 12:36:50 +02:00
Jan 720e6df82e Bug fix/fixes 1910 (#3471)
* properly initialize all properties

* use faster comparison

* properly detect and handle "method not allowed"

* code-style

* remove unused variable

* narrow variable scope

* handle non-existance of AuthenticationFeature

* remove dead code

* replace some C string handling with std::strings

* moved assertion to the correct place

* honor number of array members for IN operator

* slightly adjust error messages

* slighty adjust some error messages

* try to fix issue with lingering replication contexts on shutdown

* clean up heartbeat thread a little bit

* small fixes
2017-10-23 09:17:36 +02:00
Jan 7840d3f824 Bug fix/fixes 1810 (#3460)
* improve error reporting in RestAqlHandler

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
2017-10-19 11:28:01 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 2fee652e77 Bug fix/fixes 1610 (#3425)
* remove memory zone remainders

* remove useless calls

* attempt to fix non-deterministic CRC test

* attempt to fix non-deterministic replication tests

* fix document removal with multiple documents in cluster

* attempt to fix non-determinism of tests

* fix privilege checks for accessing collections by collection id, not by name
2017-10-18 12:23:39 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Jan 2525a3a1bd Feature/mmfiles hash lookup performance (#3265)
* speed up full collection scans in mmfiles engine

* some API cleanup
2017-09-26 14:48:06 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Simon Grätzer ffc465433a No access collections Improvements (#3190)
* consolidated EdgeDocumentToken

* optimizing cluster traversal

* adding skip collection checks

* API cleanup

* copying AQLValue to avoid use-after-free bugs

* Fixing rocksdb SingleServerEdgeCursor

* Fixing a collection resolving issue
2017-09-07 14:55:07 +02:00
Jan cd2dfdfd0d fix deadlocks in cluster traversals (#3198) 2017-09-04 17:35:24 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Simon Grätzer f3eb0c2ac0 No access collections (#3088)
* Added virtual attributes for enterprise on Methods.cpp

* Working no access collections

* align a comment

* Documentation and test fix

* fixing community build
2017-08-25 13:59:03 +02:00
Simon Grätzer 4cc8686e82 RocksDB: Optimizing dropCollection / dropIndex / fillIndex (#2927)
* cleaning up several duplicate versions of Index::memory, Index::cleanup, Index::drop, integrating branch issue-383-ensure-delete

* fixing wrong column family
2017-08-05 21:09:41 +02:00
Jan Christoph Uhde ed8efe3566 Feature/issue 387 cluster index estimates (#2866) 2017-08-01 09:53:58 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Max Neunhoeffer 29921d32a8 Fix behaviour of synchronous replication in dropFollower case.
If we are already in shutdown, we do not drop a follower.
If we cannot drop a follower (no contact to agency), we error out.
2017-06-09 13:06:02 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
jsteemann c05421e1aa some small optimizations 2017-06-02 17:34:03 +02:00
jsteemann b91eab0ce8 honor transaction options 2017-06-01 02:01:33 +02:00
Simon Grätzer 53ece582a8 optimized all index iterator 2017-05-31 11:34:32 +02:00
jsteemann 53b150b206 added tests for intermediate commits 2017-05-30 12:03:58 +02:00
Max Neunhoeffer d45ac5536d Adjust lower limit for timeout in sync repl for RocksDB. 2017-05-24 11:52:39 +02:00
Max Neunhoeffer 11e6b9ea7f Merge branch 'devel' of ssh://github.com/ArangoDB/ArangoDB into devel 2017-05-23 20:38:38 +02:00
jsteemann a7b3839e6c renamed "getMore" to "next" 2017-05-23 08:53:13 +02:00
Max Neunhoeffer 03c5766a6d Merge branch 'devel' of ssh://github.com/ArangoDB/ArangoDB into devel 2017-05-22 15:37:12 +02:00
Max Neunhoeffer cbdf9563cb Add configuration option to multiply synchronous replication timeout. 2017-05-22 15:34:15 +02:00
jsteemann a91c54f6b3 avoid double-copying of shared ptrs 2017-05-22 11:28:18 +02:00
Simon Grätzer 9950fa7245 removing unused and unimplemented methods 2017-05-10 18:25:41 +02:00
Dan Larkin f840d05d64 Raise transaction replication timeout to adjust for sporadic RocksDB latencies. 2017-05-07 11:07:09 -04:00
jsteemann a15cf5bce6 added transaction commit hooks 2017-05-04 02:00:40 +02:00
Dan Larkin 841ef8b1a0 Changed index VPack building to include forPersistence flag. 2017-04-17 17:04:50 -04:00
Simon Grätzer 5822e77843 invokeOnAllElements 2017-04-05 13:38:43 +02:00
jsteemann 6598d94e3d fix assertion failure 2017-04-04 15:52:54 +02:00
Simon Grätzer 96dc714139 Fixed some tests in shell_server 2017-04-04 15:10:17 +02:00
jsteemann 4196594130 fixed crash 2017-04-04 13:27:35 +02:00
jsteemann ceec5c8bd3 fix result handling 2017-04-04 10:10:59 +02:00
jsteemann 81dcfbbe3d pass trx into numberDocuments 2017-03-31 11:33:16 +02:00
Jan Christoph Uhde 95007fe28e Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api-arango-result
* 'engine-api' of https://github.com/arangodb/arangodb: (46 commits)
  Merged RocksDBPathBasedIndex into RocksDBVPackIndex.
  fix handling of externals
  fix toArray()
  fix returning documents
  Added hash-, skip list, persistent indexes
  harmonize filtering - filter on full filename.
  mini modifications
  standardize starting of servers for replication tests
  Edge index compliant with design doc
  another fix for #2404
  Moved „removeLargeRange“ method to commons
  Added length byte to IndexValue keys.
  fixed issue #2404
  Added drop to primary index
  Added drop() to indexes
  Fixes a coordinator crash in Clustered Traversal. In case of failover during query planning.
  Fixed edge index
  Expand on CORS documentation
  Let ArangoDB handle the allow-credentials header
  make the authentication tests run on directories
  ...
2017-03-30 19:46:17 +02:00
jsteemann 677960785a fix returning documents 2017-03-30 16:22:30 +02:00
Jan Christoph Uhde b83ae2ab82 refactor some code to make use of arangodb::Result 2017-03-30 09:39:21 +02:00
jsteemann f728fcdbb8 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-03-29 02:41:00 +02:00
jsteemann e1aef394d8 delete useless DEV_TIMERS, fix rwlockcpp11 2017-03-28 22:38:27 +02:00
jsteemann 0d38d2c287 finalize insert() 2017-03-27 11:30:30 +02:00
jsteemann 2c54a6ac1e added some basic transaction handling 2017-03-24 17:23:32 +01:00
jsteemann 42fccb69c6 remove dependency on MMFilesLogfileManager 2017-03-01 15:50:10 +01:00
jsteemann ed4b5fb3f9 removed unused includes 2017-02-27 17:29:28 +01:00
jsteemann 666b2f8da9 renaming 2017-02-27 14:38:27 +01:00
jsteemann 94ce6a65f8 renamed methods 2017-02-27 11:22:44 +01:00
Michael Hackstein 5d452edec6 Moved ditches into MMFiles. 2017-02-23 09:02:34 +01:00
Jan Christoph Uhde 711a71ac4d add LogicalCollection::invokeOnAllElements()
and remove MMFilesPrimaryIndex.h from Transaction/Methods.cpp
2017-02-22 10:40:27 +01:00
Jan Christoph Uhde fcfe66e254 add getAll/AnyIterator to LogicalCollection 2017-02-22 09:24:38 +01:00
Jan Christoph Uhde d73d25a310 remove some includes 2017-02-21 14:43:21 +01:00
Michael Hackstein fbdbfdcb6d Moved primaryIndex knowledge out of Logical Collection. Also moved logic for close into MMFiles 2017-02-21 14:12:07 +01:00
Michael Hackstein 91f303e963 Removed readRevision from LogicalCollection. You only get readDocument with a token. Moved it's logic into MMFilesCollection. removed the ignoreWal parameter from readDocumentConditional. It was always true anyways... 2017-02-17 17:29:49 +01:00
jsteemann aa09e8cb27 moved properties 2017-02-16 10:34:14 +01:00
jsteemann 089ff78d2d moved collection lock state into its own struct 2017-02-16 10:27:14 +01:00
jsteemann fbd7f5d265 simplfied APIs 2017-02-16 10:00:46 +01:00
jsteemann 97728e51c1 simplifications 2017-02-15 15:19:40 +01:00
jsteemann 3f34efc1bb remove _resolver property 2017-02-15 13:13:43 +01:00
jsteemann 09da6bea7f simplify internal APIs 2017-02-15 12:57:43 +01:00
jsteemann d49c5c3fc0 simplified internal APIs a bit 2017-02-15 12:42:07 +01:00
jsteemann 112d07f6c2 protect all properties of TransactionState 2017-02-15 11:37:35 +01:00
jsteemann b6173827c9 remove unused method 2017-02-15 11:15:07 +01:00
jsteemann c69eb0a7d5 protect some properties 2017-02-15 11:14:56 +01:00
jsteemann b4027dc02c make property protected 2017-02-15 10:41:34 +01:00
jsteemann df3a3eff9f refactoring 2017-02-15 10:26:05 +01:00
jsteemann 01bf68549d simplifications 2017-02-14 22:48:51 +01:00
jsteemann 12b973dc4b fix my compile errors 2017-02-14 16:41:31 +01:00
jsteemann 60e0b03b06 remove unused method 2017-02-14 16:23:07 +01:00
Michael Hackstein dffd9fc807 Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-14 14:08:56 +01:00
Michael Hackstein d5e14a95f4 Moved truncate logic from Transaction::Methods to MMFilesCollection. It can do optimizations if close to the DB layer 2017-02-14 13:50:29 +01:00
jsteemann 0909ffe69b moved helper functions into their own file 2017-02-14 13:44:54 +01:00
jsteemann 8a2e353f3d move encoding functions into their own file 2017-02-14 12:57:15 +01:00
jsteemann be77decb2d remove an unused include 2017-02-14 12:39:50 +01:00
jsteemann 80be198a39 moved helper classes into their own file 2017-02-14 12:35:34 +01:00
jsteemann 9efc1c7b61 moved transaction status into its own file 2017-02-13 17:28:02 +01:00
jsteemann a184697072 move RocksDB-specific transactions into engine 2017-02-13 16:53:58 +01:00
jsteemann 7e690a0c61 renaming 2017-02-13 15:18:40 +01:00