1
0
Fork 0
Commit Graph

15716 Commits

Author SHA1 Message Date
Max Neunhöffer 1ca1b89deb
Port static-build fixes to the 3.3 branch (libmusl). (#4481)
* Changes for libmusl compilation. (#4294)
* Further fixes for libmusl completely static build. (#4474)
* Make crc assembler code position independent.
* Fix PIE compilation for libmusl in alpine linux.
* Use backtrace library if found.
2018-02-07 10:49:32 +01:00
Michael Hackstein d574cff1b9
Create javascript.app-path if it does not exist (#4510) 2018-02-06 15:41:59 +01:00
Jan c55374a75c
fix issue #4272 (#4418) 2018-01-25 15:57:06 +01:00
Jan 4c6337611b
properly unlock after restore-indexes (#4369) 2018-01-25 15:56:13 +01:00
Simon 6752e6bc38
Merge pull request #4363 from arangodb/bug-fix-3.3/fix-revoke-in-cluster
fix revoke operation
2018-01-22 15:31:59 +01:00
Jan cce1a6f55b
improved logging for MMFiles IO operations (#4365) 2018-01-19 18:36:49 +01:00
jsteemann fcad07dd7e fix revoke operation 2018-01-19 10:14:59 +01:00
Jan d227c83e29
explicitly control activation of RocksDB write-throttling feature (#4359)
also consistently use -DNDEBUG to avoid different class layouts when
the same class are used by different translation units
2018-01-19 09:32:47 +01:00
Kaveh Vahedipour a14c4bd02f constituent correctly persisiting _votedFor and _term (#4248) (#4320) 2018-01-17 10:37:16 +01:00
Heiko f7933aa0f7 Bug fix 3.3/issue 4222 (#4325) 2018-01-16 20:58:13 +01:00
Heiko a16fd972eb optimized shard distribution api and ui (#4312) 2018-01-16 20:54:07 +01:00
Michael Hackstein ffbc7058cd
Bug fix 3.3/rocksdb truncate (#4338) 2018-01-16 19:52:27 +01:00
Jan f31734a198
fixed issue #4185: On execution of FULLTEXT search / AQL query db is … (#4239) 2018-01-15 22:46:14 +01:00
Jan c1459ec5b9
fix missing events in RocksDB-based replication for transactions that… (#4276) 2018-01-15 22:44:54 +01:00
Jan f482b7aed4
Feature 3.3/small optimizations for query registry (#4259) 2018-01-15 22:44:17 +01:00
Jan 0a28878059
Bug fix 3.3/micro fixes (#4235) 2018-01-12 13:59:18 +01:00
Jan 7b370dbf23
fix memory hunger of cache (#4269) 2018-01-12 12:15:19 +01:00
Jan 15d21ced0b
Bug fix 3.3/proper shutdown of throttle (#4250) 2018-01-10 00:08:17 +01:00
Jan 402ed9a99d ensure proper shutdown of throttle (#4233) 2018-01-05 14:56:52 +01:00
Jan b8c043945f
try to not fail hard when a collection is dropped while the WAL is tailed (#4225) 2018-01-04 16:31:17 +01:00
Jan b2e3a6614a fixed issue #4197: AQL statement not working in 3.3.1 when upgraded from 3.2.10 (#4207) 2018-01-04 00:06:41 +01:00
Jan 75953d69e8 do not reuse collection ids when restoring (#4218) 2018-01-03 23:48:47 +01:00
Max Neunhöffer f58301ed8c
clusterInventory: distinguish between isReady and allInSync. (#4214) 2018-01-03 22:18:15 +01:00
Kaveh Vahedipour 56a9ad69b1 Bug fix 3.3/supervision no longer fails to remove server from failed when back to good (#4210)
* let's not miss failedserver removal
* remove resetting of FailedServers in test code
* Only call abortRequestsToFailedServers at most every 3 seconds.
2018-01-03 21:55:01 +01:00
Jan 1bcab5628d
Bug fix 3.3/fix issue 4199 (#4201) 2018-01-03 21:32:49 +01:00
Jan d80daeb450
Bug fix 3.3/pre release woes (#4084) 2018-01-03 16:02:30 +01:00
Matthew Von-Maszewski 41d1bfce23 create independent executeLockedRead and executeLockedWrite to speed performance (#4178) 2017-12-29 13:36:48 +01:00
Kaveh Vahedipour d6ce7a1301 Agency read write locks ported from devel (#4175) 2017-12-28 11:28:11 +01:00
Matthew Von-Maszewski 392ddde251 Bug fix 3.3: Fix supervisor thread crash (#4165)
* port devel branch to 3.3 of supervisor thread death fix
2017-12-27 22:34:29 +01:00
Max Neunhöffer ef8fcd101c
Port to 3.3 of various fixes around leadership preparation in agency. (#4150)
* Add logging for _earliestPackage in Agent.
* Really enforce the hidden option --server.maximal-threads if given.
* Switch off --log.force-direct in scripts/startStandAloneAgency.sh
* Lower the timeout for sending AppendEntriesRPC to 150s.
* Erase _earliestPackage when becoming a leader.
* Challenge leadership in agent main loop.
* Use steady_clock for _earliestPackage.
* Change _lastAcked and _leaderSince to steady_clock as well.
* time difference calculations based on old readSystemClock to steadyClockToDouble
* All system_clock transitioned to steady_clock in Agent. Remaining system_clock are user input / output or timestamps
* Inception system_clock to steady_clock
2017-12-27 16:47:16 +01:00
Matthew Von-Maszewski d35ebbe6a1 move from devel to 3.3 the dynamic chooseTimeout() feature. (#4166) 2017-12-27 16:36:42 +01:00
Jan 60ecfcfce4
fix wrong lock usage (did not have any effect except misleading the reader of the code) (#4086) 2017-12-23 00:19:46 +01:00
Jan cfe94a0913
add `--ignore-missing` option to arangoimp (#4128)
this option allows importing lines with less fields than specified in the CSV
header line
2017-12-22 11:09:03 +01:00
Jan e804f1d861
fix nullptr access (#4143)
* fix nullptr access

* fix compile error
2017-12-22 10:56:01 +01:00
Matthew Von-Maszewski f35215ea51 Have twice seen coordinator go into long loop on shutdown. Added two tests for isStopping() to break the loops. (#4139) 2017-12-21 20:56:14 +01:00
Jan 8b873bdf4a
prefer unique equality lookups, as they do not have O(log(n)) complexity (#4096) 2017-12-21 20:41:10 +01:00
Jan Christoph Uhde c91cfecded remove superfluous decrease memory usage (#4132) 2017-12-21 20:15:03 +01:00
Jan f83a1a0394
Feature 3.3/add recovery tests for intermediate commits (#4106)
* added several recovery tests that check intermediate commits

* small fixes

* fix tests
2017-12-20 09:46:12 +01:00
Jan 5230df9f72
Feature 3.3/fulltext optimizations (#3925) 2017-12-19 21:40:12 +01:00
Jan b7ee607312
Bug fix 3.3/integer overflow when calculating waits in constituent (#4090)
* integer overflow in Constituent could seize operation of Agency

* less likely integer overflow on double conversion

* less likely integer overflow on double conversion

* changed comparison to integer comparison as suggested by @neunhoef
2017-12-19 10:10:05 +01:00
Simon Grätzer 90e6c78845 Make sure recalculated counts are actually stored (#4079)
(cherry picked from commit ab9faf56ce81a214a20683f7512db063ef9b063b)
2017-12-18 22:39:04 +01:00
jsteemann f9a86e6cca remove debug message 2017-12-16 15:22:54 +01:00
Jan 08de96ffee
fixed issue #4056 (#4058)
* fixed issue #4056

* fix over eager santiation
2017-12-16 15:10:29 +01:00
Jan d78f4601ba
don't make the syncer fail when it is restarted (#4030) 2017-12-14 10:23:35 +01:00
Jan 5b392c027a
forgotten fix for globallyUniqueId (#4033) 2017-12-14 10:05:28 +01:00
Jan d0021ac71a
fix issue #planning/1511 (#4029) 2017-12-13 22:09:46 +01:00
Jan 9c5893e7a7
fix premature unlock (#3802) (#4027)
* fix some deadlocks found by evil lock manager (tm)

* fix duplicate lock

* fix indentation

* ensure proper lock dependencies

* fix lock acquisition

* removed useless comment

* do not lock twice

* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not

* AQL micro optimizations

* use explicit constructor

* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context

entering and exiting the selected context is then done later on without having to hold the ConditionLocker

* remove some recursive locks

* Disable custom deadlock detection when Thread Sanitizer is enabled

* Changing ifdef's

* grr

* broke gcc

* Using atomic for ApplicationServer::_server

* fix premature unlock

* add some asserts

* honor collection locking in cluster

* yet one more lock fix

* removed assertion

* some more bugfixes

* Fixing assert

(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
2017-12-13 18:46:14 +01:00
Jan e7f93d8b9d
potentially fix send request timeout (#4025)
* potentially fix send request timeout

* fix globallyUniqueIds
2017-12-13 18:45:31 +01:00
Simon Grätzer 3287d8706a Use uuid in initalsync to ensure proper collection mapping (#3972) 2017-12-13 14:29:21 +01:00
Simon Grätzer cfd500c53c Fixing an issue with intermediate commits (#4006)
* Fixing an issue with intermediate commits

(cherry picked from commit 7f61fb9be0f22904be21e9dd26c3870035a5b318)

* Only check collection counts without intermediate commits

* Fixing assert
2017-12-12 23:15:41 +01:00
Jan d080350afa
make replication abortable (#4015) 2017-12-12 23:15:22 +01:00
Jan 7af86685e3
when upgrading from 3.1 LastHeartBeatAcked could also have been missing, when the 3.1 cluster had not run for long enough (#3974) 2017-12-08 17:33:37 +01:00
Jan 3caa4c47bb
add a test for replication applier restarts (#3750) (#3933) 2017-12-08 15:48:15 +01:00
jsteemann e0b3917380 fix failing test on MSVC 2017-12-07 16:25:31 +01:00
Wilfried Goesgens 3a940eccdc release version 3.3.rc7 2017-12-07 14:48:38 +01:00
Jan 6ce073c2d5 fixed issue #3640: limit in subquery (#3903) 2017-12-07 10:41:46 +01:00
Jan 5f142fb601 fix issue #3745: Invalid result when using OLD object with array attribute in UPSERT statement (#3904)
* fix issue #3745: Invalid result when using OLD object with array attribute in UPSERT statement

* updated CHANGELOG
2017-12-07 10:40:59 +01:00
Jan 29d81302ce Bug fix 3.3/agency transactions (#3936)
* Added 503/0 inquire tests to resilience tests (#3839)

* replaced all AgencyGeneralTransactions by AgencyWriteTransaction (#3841)
2017-12-07 10:39:36 +01:00
Jan 08f9326eed make sure we do not dereference a nullptr (#3906) 2017-12-07 10:36:13 +01:00
Jan 291df05e08 Resolving externals before using them with VelocyPackHelper::VPackHash (#3907) 2017-12-07 10:35:25 +01:00
Jan 1789977d51 fix heap sort in GatherBlock (#3908) 2017-12-07 10:34:44 +01:00
Jan 7aed168a0d added missing guard object (#3910) 2017-12-07 10:34:17 +01:00
Jan bec83181be Bug fix 3.3/add security check end with failover (#3911)
* Add security check in AgencyComm::sendWithFailover.

* some cleanup

* added some more tests

* add typeName() to AgencyCommTransaction to make the transaction type printable in debug messages

* improve debuggability
2017-12-07 10:33:59 +01:00
Jan ad8de0e6bf allow passing a serverId value of "none" to replication APIs (#3912) 2017-12-07 10:33:14 +01:00
Jan 4397ebf3c7 improve compatibility when replicating from a 3.2 (#3915) 2017-12-07 10:30:27 +01:00
Jan ba729150bf backporting inquire fixes (#3920) 2017-12-07 10:27:41 +01:00
Jan 631b2dcb11 fix multiple binds to the same ip/port on Windows (#3923) 2017-12-07 10:26:25 +01:00
Jan 21e275d1a1 Propagate commit errors to AQL (#3831) (#3932) 2017-12-07 10:20:04 +01:00
Michael Hackstein f986a5f06c Improved the cost estimation of traversals to be independent from max search depth. This fixes #3917 (#3944)
LGTM
2017-12-07 10:05:35 +01:00
Matthew Von-Maszewski f6cdbcd4e8 add defensive coding and logging in response to a divide by zero. ColumnFamilyData::RecalculateWriteStallConditions() called SetBytePerSecond() with some value less than four. Could have been a race condition due to our SetThrottle() called without db mutex. No proof, just likely. (#3950) 2017-12-07 09:25:44 +01:00
Jan 4104296fd3 fix revision id vs local document id usage in incremental sync (#3916) 2017-12-07 09:24:20 +01:00
jsteemann b91e6da52e fix replication context locking 2017-12-04 12:09:57 +01:00
Frank Celler 75a494dc74 Merge branch 'devel' of github.com:arangodb/arangodb into 3.3 2017-11-28 12:49:25 +01:00
Kaveh Vahedipour f7b4150b64 no clientId anymore in send/sendWithFailOver SPIs (#3819) 2017-11-28 10:47:36 +01:00
Kaveh Vahedipour c300eee5f0 minor (#3813) 2017-11-27 18:22:13 +01:00
Jan 4eab985e90 fix wrong ifdefs (#3803) 2017-11-27 14:20:04 +01:00
Kaveh Vahedipour 27cd691bbf Bug fix/agencycomm validate methods broken (#3805) 2017-11-27 14:18:25 +01:00
Frank Celler a54da92fc7 Merge branch 'devel' of github.com:arangodb/arangodb into 3.3 2017-11-24 12:29:19 +01:00
Michael Hackstein 99b7b5080b Fixed CAS in AuthInfo update in cluster. Should now be more reliable. (#3787) 2017-11-24 11:47:24 +01:00
Jan bd1a2d6cce fix shutdown deadlock (#3796) 2017-11-24 11:42:09 +01:00
Kaveh Vahedipour 2beaef41ff Bug fix/agencycomm validate methods broken (#3784) 2017-11-24 10:31:07 +01:00
Frank Celler 0b609dc65e Merge branch 'devel' of github.com:arangodb/arangodb into 3.3 2017-11-22 16:11:52 +01:00
Matthew Von-Maszewski 50012d1ed8 Throttle adjustments for high speed arrays (#3755) 2017-11-22 12:03:55 +01:00
Michael Hackstein 7d95de46e8
Fixed a dead-lock issue in authinfo (#3764) 2017-11-22 10:33:39 +01:00
Frank Celler e963a4c39b Merge branch 'devel' of github.com:arangodb/arangodb into 3.3 2017-11-20 10:30:42 +01:00
Matthew Von-Maszewski 874e1d8eb0 correct stupid mistake: was erasing iterator in middle of same iterators loop (#3740) 2017-11-20 10:24:53 +01:00
Frank Celler cf742d984c Merge branch 'devel' of github.com:arangodb/arangodb into 3.3 2017-11-17 22:57:43 +01:00
Matthew Von-Maszewski 0558b4372d Improve ClusterComm::wait() (#3722) 2017-11-17 22:46:54 +01:00
Simon Grätzer 0e485f7441 Fixing collection name collection handling in Syncer (#3710) 2017-11-17 16:36:57 +01:00
Simon Grätzer 987daca85b Handle invalid endpoints in AgencyComm (#3729) 2017-11-17 16:35:59 +01:00
Jan 86ef7d4018 Bug fix/fixes 1711 obi (#3730) 2017-11-17 16:34:33 +01:00
m0ppers 75e1bf31cd Feature/allow access to configured 🔐 in readonly mode (#3698)
* Special method to access configured access levels as opposed to only getting the effective access level
* Add integration test
* Allow grants on collection level
2017-11-17 16:11:22 +01:00
Kaveh Vahedipour 7b80deb5cc Fixed object assignment operator for agency's key value store (#3701)
* Fixed object assignment operator for agency's key value store
* Node's toJson is now actually toJson. getString should be used for string extractions
* adjust agency's documentation (clarify precondition)
2017-11-17 15:49:40 +01:00
jsteemann 5394cc27a6 disable potentially unsafe socket option 2017-11-17 11:21:03 +01:00
Kaveh Vahedipour 255d90d26a cherry pick from 3.2 pull request for bug-fix/supervision-thread-exists-on-pre3.2-agency (#3709)
This is the HealthRecord upgrade patch.
2017-11-17 10:14:14 +01:00
Jan b4f6ee9273 Feature/improved index api for unique constraints and replication (#3715) 2017-11-16 21:02:01 +01:00
Jan 4a199e2fe3 make truncate a bit less obstrusive (#3721) 2017-11-16 20:28:33 +01:00
Jan Christoph Uhde 6f6d19b1b9 Stricter normalization for filter and sort conditions (#3718)
Moves attribute accesses to the LHS when using <,>,<=,>=
2017-11-16 19:17:12 +01:00
Jan 7eaaeaf392 fix some compile warnings (#3720) 2017-11-16 18:18:28 +01:00
Jan fce593b724 removed `--recycle-ids` option for arangorestore (#3713) 2017-11-16 14:25:54 +01:00
Jan 5abf0c1185 Bug fix/fixes 1511 (#3711) 2017-11-16 14:18:51 +01:00
Matthew Von-Maszewski 97944f012e Fix issues due to throttle started with too small a flush (#3707) 2017-11-16 09:13:08 +01:00
Jan Christoph Uhde 987aea2af3 implement heap merge for cluster blocks (#3617) 2017-11-15 18:05:43 +01:00
Jan 3b0a8a9cdf make the replication applier auto-start for the RocksDB engine if con… (#3647) 2017-11-15 12:02:37 +01:00
Max Neunhoeffer bc1891dfd0
Comment out RocksDBThrottle. 2017-11-14 16:53:42 +01:00
Michael Hackstein 3911a6a722 Fixed minor issue in shard distribution reporter thanks to @danielhlarkin for spotting (#3695) 2017-11-14 16:47:25 +01:00
Jan 15ad2fbd99 do not complain about missing datafile statistics (#3688) 2017-11-14 16:42:31 +01:00
Max Neunhöffer 766ab7c8cf
Fix agency shutdown bug. (#3683)
* Fix agency shutdown bug.
* Remove precondition that was not needed in AgencyComm::removeValues.
* Fail fatally if threads do not shut down.
2017-11-14 16:33:46 +01:00
Jan cc1fc716a4 Bug fix/fix server mode test side effects (#3674) 2017-11-14 15:19:00 +01:00
Jan 8ece5914a3 do not warn when the user attempts to drop a non-existing index (#3682) 2017-11-13 17:47:57 +01:00
Simon Grätzer c6fe726901 Removing unsafe asserts in wal tailing v3.3 (#3672) 2017-11-13 17:41:57 +01:00
Jan ba9bc41457 fix some typos in code and docs (#3671) 2017-11-13 17:33:36 +01:00
Matthew Von-Maszewski bbce4caefa Feature/mv basho smoothing1 (#3384) 2017-11-13 15:43:44 +01:00
Jan 03daf9e9c9 make incremental sync replication API check if the client passed an "offset" request parameter (#3669)
if yes, the server may dynamically adapt the size of the response, in order to ensure that
HTTP responses do not get out of hand size-wise. This is a new feature in devel, and this
commit now makes it optional so that older clients do not need to be changed.
2017-11-13 15:08:31 +01:00
Jan e1ecc6b02c fix some threading issues (#3659) 2017-11-12 22:34:51 +01:00
Frank Celler 813f5c8541
fixed output (#3668) 2017-11-12 18:06:56 +01:00
Kaveh Vahedipour c9621ff230 Feature/new agency checks for preconditions (#3612) 2017-11-11 22:48:23 +01:00
Jan e13a611047 fix cache for gather block (#3656) 2017-11-11 22:30:38 +01:00
Max Neunhöffer bff630b332 Handle leader resignation race with redirectRequst. (#3663) 2017-11-11 19:38:29 +01:00
Jan Christoph Uhde daf98e145e improve rocksdb logging (#3613) 2017-11-11 19:35:27 +01:00
Jan 8a467e74db try to not crash when encountering a collection that is not present (#3660) 2017-11-11 19:30:37 +01:00
Kaveh Vahedipour 7e816db51e Bug fix/agency restart enhancements (#3619)
* Removed unused active(...) method in Agent
* Inception's restart from persistence allows peer with empty active RAFT list to join
* Agency's UUID is persisted outside of the database comparable to coordinator and db server action.
* Publicized Methods to UUID stuff in ServerState
* Inception method documentation
* added --agency.disaster-recovery-id to allow for specification of known former agency id. this is a very dangerous option potentially.
* Delete a unused methods.
* separate _id and _recoveryId
* populating active list with entire pool
* Improve logging.
* reject gossip from unknown agent, if pool is complete
2017-11-10 23:40:26 +01:00
Simon Grätzer 161d7e8b5c Do not grant rights on system collections (#3627) 2017-11-10 18:01:17 +01:00
m0ppers 6cbf7159be Feature/server mode (#3590)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* Fixing some bugs

* Fixing tests

* Canceling ongoing operations

* removing some unused code + some asserts

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* Current work

* proper ifs

* Migrate resthandler to c++ and implement mode

* readonly server mode spec test

* Add changelog

* change code so it expects a full object and not just a string

* Fix jslint
2017-11-10 17:56:21 +01:00
jsteemann 571d4587ca Revert "Return offending key for unique constraint violations (#3624)"
This reverts commit 68bd31ac99.
2017-11-10 16:40:56 +01:00
Kaveh Vahedipour b09c329a7c fixed 10 min dead time, when bogus --cluster.my-local-info specification (#3643) 2017-11-10 16:05:03 +01:00
Dan Larkin 68bd31ac99 Return offending key for unique constraint violations (#3624) 2017-11-10 16:03:10 +01:00
Jan 733f27e997 Bug fix/fix compilation with gxx7 (#3637) 2017-11-10 16:00:57 +01:00
Michael Hackstein 5c633f9fae Bug fix/speedup shard distribution (#3645)
* Added a more sophisticated test for shardDistribution format

* Updated shard distribution test to use request instead of download

* Added a cxx reporter for the shard distribuation. WIP

* Added some virtual functions/classes for Mocking

* Added a unittest for the new CXX ShardDistribution Reporter.

* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red

* Shard distribution now uses the cxx variant

* The ShardDistribution reporter now tries to execute count on the shards

* Updated changelog

* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
2017-11-10 15:17:08 +01:00
Michael Hackstein 2b0aa318ec Bug fix/sparse graph register warning (#3569) 2017-11-10 14:33:15 +01:00
Jan bef52d7dc3
Bug fix/cleanup after cppcheck (#3639) 2017-11-10 13:53:28 +01:00
Jan 4a06599adf fixed issue #3618 (#3621) 2017-11-10 12:31:10 +01:00
Jan 057e87f919 fix shutdown in case no threads can be started (#3648) 2017-11-10 10:21:51 +01:00
Simon Grätzer 87f441753b RocksDB WAL tailing fixes (#3595) 2017-11-10 09:31:53 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Max Neunhöffer 3c0ee6908b Bug fix/lead to agent (#3541) 2017-11-09 11:10:09 +01:00
Jan 6b90029c92 remove misleading and not used geo index type enum values (#3626) 2017-11-09 09:12:44 +01:00
Jan 356751bd12 fix a non-deterministic replication test (#3623) 2017-11-09 09:10:19 +01:00
Jan 3f0dd63d2b added test for issue #3473 (#3608) 2017-11-08 17:09:07 +01:00
Jan Christoph Uhde 5a6046874d fixes: geoindex type 1 queries (#3575) 2017-11-07 15:03:45 +01:00
Jan 0195ede1d8 potential fix for issue #3581: Unexpected "rocksdb unique constraint violated" with unique hash index (#3596) 2017-11-07 10:42:41 +01:00
Jan 151c717d22 fix potential duplicate closing of typed buffer (#3587) 2017-11-07 10:27:51 +01:00
Jan f6a90c4879 some cleanup (#3583) 2017-11-07 10:27:21 +01:00
Jan 98eecaae20 bug fix for agency precondition checks (#3579) 2017-11-06 23:55:41 +01:00
Frank Celler 02eed22dc2 release version 3.3.beta1 2017-11-06 18:27:03 +01:00
Jan 8673c1c31d properly return `{"result":true}` for rotate API call (#3586) 2017-11-06 12:42:04 +01:00
Frank Celler 3d0ab7d396 fixed asserts and check password test 2017-11-05 13:10:47 +01:00
Simon Grätzer ee8209943f Missing things for active / passive (#3578)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* auth = off does not longer make everyone superuser

* Fixing cluster_sync and maybe resilience
2017-11-04 20:30:23 +01:00
m0ppers b552853909 Feature/optional replication factor enforcement (#3570) 2017-11-02 21:41:25 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Simon Grätzer 974fa47a87 Fixing Pregel module (#3557)
* Fixing wrong response logic

* fixing rest pregel handler
2017-10-31 20:15:51 +01:00
Jan 4d03d3bd82
Bug fix/fixes 2410 (#3511)
* slightly clean up indexes

* proper stringification of error code

* improve diagnostic messages

* improve memory usage for incremental sync
2017-10-30 22:49:24 +01:00
Simon Grätzer 05d6a9a9f4 Reuse buffers for HttpResponse (#3433) 2017-10-30 17:30:36 +01:00
Jan 3f1f2c3e81 potential fix for issue #3517: several "filesystem full" errors in log… (#3525) 2017-10-30 17:28:59 +01:00
Jan 636ba68466 fixed issue #3549: server reading ENGINE config file fails on common standard newline character (#3550) 2017-10-30 16:31:12 +01:00
jsteemann 4dc13142dc yet another fix for replicationFactor validation 2017-10-27 00:26:53 +02:00
Manuel B edba1e57ce Feature/cpp aql left trim right (#3004) (#3506)
* Feature/cpp left trim right (#3004)

* aql char length null returns now 0

* cpp aql trim and tests

* Update CHANGELOG
2017-10-26 20:21:41 +02:00
jsteemann 376cc7dcad fix replicationFactor value validation 2017-10-26 20:15:50 +02:00
jsteemann a5c777e565 fix broken inquiry results in AgencyComm 2017-10-26 20:10:54 +02:00
jsteemann d03fb7e01b fix validation for satellite collections 2017-10-26 19:27:25 +02:00
Simon Grätzer 36e51913b3 Changing satellite parameter check 2017-10-26 19:27:18 +02:00
Simon Grätzer d14710b683 Updating Replication Factor (#3528)
* Changing replication factor in 3.2 (#3513)

* Allow changing replicationFactor on coordinator

* Fixing logic

* Allowing change of replication factor

* Additional input validation

* grrr

* Testing invalid inputs

(cherry picked from commit 47600e1ea1)

* jslint

(cherry picked from commit 4d223597db)

* cherry-pick commits from 3.2: 47600e1ea1 4d223597db d684eaa7f8 a898af3723
2017-10-26 15:28:05 +02:00
Jan c569c1b816 fix test (#3524) 2017-10-26 14:04:41 +02:00
Max Neunhöffer cb05d33e17 Term is a number not a string. (#3520) 2017-10-26 12:02:38 +02:00
Jan e9ade3f02d increase timeout for truncate operation in cluster (#3515) 2017-10-26 10:29:18 +02:00
Max Neunhöffer 5e2f17c04f Fast path for get-in-sync of empty shards. (#3470)
* Fast path for get-in-sync of empty shards.

* Fix short cut in synchronization.
2017-10-26 09:55:00 +02:00
Jan 9eb5e1545f Bug fix/fix cluster foxx queue startup (#3507)
* slightly clean up indexes

* proper stringification of error code

* do not run AQL queries during Foxx queue startup

* always create keyspace
2017-10-25 18:07:27 +02:00
Max Neunhöffer ee96c37237 Fix agency restart problems. (#3493)
* Fix agency restart problems (port from a 3.2 fix).

* Further fixes after Craneware rescue.
2017-10-25 18:05:58 +02:00
Jan df5ea8ee2b try to fix another permissions problem when collections are accessed … (#3485)
* try to fix another permissions problem when collections are accessed by id

* added tests
2017-10-25 18:05:17 +02:00
Simon Grätzer f8260d20eb Fixing permission problem in createQueues (#3483)
* Fixing permission problem in createQueues

* Fixing queues test

* fixing jslint
2017-10-25 18:04:55 +02:00
Jan bb5b7a67a9 fix undefined behavior in RocksDB geo index (#3479) 2017-10-25 18:04:26 +02:00
Michael Hackstein 15d9a4be5f Reactivated the failover of the FoxxMaster, it was not modified anymore after the current master dies (#3510) 2017-10-25 18:03:24 +02:00
Manuel B 7d4b319c58 cpp aql substring, fixed v8 substring (#3489) 2017-10-23 17:12:45 +02:00
Simon Grätzer 09840239c9 Fixed wait for sync in mmfiles (#3478) 2017-10-23 12:36:50 +02:00
Simon Grätzer 3e211f0d9e Fix --cluster.my-local-info (#3481)
* Compatibility for old startup methods.

* removed my-local-info from docs
2017-10-23 12:20:17 +02:00
Manuel B be8c648620 cpp aql collections() and test (#3331)
* cpp aql collections() and test

* remview fixes

* fixed jslint

* fix for new ExecContext / auth api
2017-10-23 11:42:34 +02:00
Jan 720e6df82e Bug fix/fixes 1910 (#3471)
* properly initialize all properties

* use faster comparison

* properly detect and handle "method not allowed"

* code-style

* remove unused variable

* narrow variable scope

* handle non-existance of AuthenticationFeature

* remove dead code

* replace some C string handling with std::strings

* moved assertion to the correct place

* honor number of array members for IN operator

* slightly adjust error messages

* slighty adjust some error messages

* try to fix issue with lingering replication contexts on shutdown

* clean up heartbeat thread a little bit

* small fixes
2017-10-23 09:17:36 +02:00
Manuel B cf0578242f Feature/devel cpp aql matches (#3329)
* matches function header

* aql matches() C++ definition

* cpp aql matches() implementation

* review fixes

* test for v8 / cpp

* aql cpp matches matches behavior of js matches, more tests

* fix cpp matches()

* final fixes
2017-10-23 08:25:55 +02:00
Max Neunhöffer 67300f9d77 Add a hidden AGENCY_DUMP for agency emergency recovery. (#3474) 2017-10-21 00:24:32 +02:00
Simon Grätzer fd3f9d99d9 Fixing webinterface access (#3464)
* intermediate commit

* Refactoring the ExecContext

* Fixing authentication

* Added start script

* some fixes

* fixed access to nullptr

* some c++

* fixed misleading message

* Made DatabaseGuard movable. Also adapted map insertions to _vocbase in Syncer classes, which failed to compile under older GCC versions

* added support for global flag to replication handler

* Started Refactoring in replication-static

* Fixing syncer code

* store applier configuration

* Static replication tests now test replication in a non system Database

* added flags to replication feature

* Adding some extra checks

* Fixing issue with rocksdb rest replication handler

* replication static now runs _system and otherdatabase replication tests.

* Fixing crash on startup

* Replication_sync now tests _system as well as other Database

* Fixing up heartbeat thread, adding global flag to rest handler

* Fixing wrong assert

* some cleanup, probably some tests are broken

* Made non-system db version of replication-ongoing tests

* fix determine-open-transaction

* Fixed ongoing tests. And added a test where we drop a database on slave while replication is still ongoing

* test fixes

* Activated ongoing other db tests. Also added a test that drops the DB on master, while the slave is still syncing.

* some better error reporting

* gradually switch to Result

* createCollection -> create

* re-activate using of collection ids for now

* enable auto-start

* Fixed create collection in replication ongoing test

* Added first draft of a test for global replication

* move to Result

* use system database for global applier

* improved error reporting

* fixed invalid URLs

* add test case filter

* load existing global applier configuration

* improve error reporting

* Added further tests for global replication

* Fixed global replication test, it now properly waits for replication. Timeouts after 10 seconds.

* Removed erronious assertion

* improve error reporting

* intermediate commit

* Added a test-case for global replication where the Master already has some data and the slave is clean

* fix deletion of replication contexts

* Fixed JSLint

* compiling code

* fix typo

* do not fail for global applier when no database is configured

* intermediate commit

* syncer supports switch for 3.3 / 3.2

* fixed errors

* Fixing some replication bugs

* Fixing some assertions

* Fixed missing commit markers

* Fixing assertion on database drop

* Attempt to fix deadlock in applier and assertion

* Fixing some stupid things

* Support for collection parameter

* Acidentally turned off some tests

* Grrr

* Fixing wrong method call

* Fixed startscript

* Fixed assignmet instead of equality check typo

* Added a test far interrupted replication. For now it justs tests basics on _system database.

* Improved index tests on replication.

* properly initialize variable

* fixed some replication problems

* MMFiles wal access support

* fix replication issues

* Started mmfiles replication support

* fixing a bug

* Fixing an issue

* fixing some mmfiles stuff

* fix test

* reload users

* prevent pure virtual method call

* intermediate commit

* Making from exclusive

* do not call getMasterState if child syncer

* some reformatting

* Adding global support for handleCommandSync

* Fixing assertion

* removing some debug logs

* Changing return codes

* Fixing some issues in the rest handler

* Make replication less susceptible to errors

* remove some debug output

* return last log tick

* remove waits from tests

* fix two tests

* changing header for open-transactions call

* some fixes

* fix test

* invalidate cached databases

* merging request and execcontext

* try to fix assertion error

* renamed method

* fix compile warning

* small changes

* Always use execcontext

* Fixing an assert

* fix replication issues

* try to fix collection lookups

* try to fix master/slave start

* Changing comments in heartbeat thread

* fix wrong signature of READ_LOCKER_EVENTUAL

* log server role in testing mode

* Fixed authentication, removed execContext in favor of request context

* Adding cluster rest api

* Fixing cluster rest handler

* Fixing cluster callback

* Some refactoring

* Queue creation is not a single operation

* Allowed for leader redirects

* Setting start of batch

* Disabling 2.8 compat tests

* fix start/stop bugs

* jslint

* various little changes

* add flag for exposing jwt

* indentation

* cleanup

* Some changed to guid

* fixing tcp to http, vst

* changed endpoint header

* small fixes

* Reorder servers by health status

* Higher timeout

* Changing error messages

* update the fromTick when fetching multiple batches from the coordinator

* more debug info

* Reducing copy pasted code

* change uid generation

* reducing logspam

* more exceptions for redirects

* more exceptions

* attempt to fix uniqids in cluster

* centralize printing of HTTP errors in replication

* debug output

* fix messages for authentication

* cleanup

* removing --cluster.my-id, --cluster.my-local-info

* Added leadership race to bootstrap, determine foxxmaster on boostrap, removing obsolete code

* improve error reporting in RestAqlHandler

* Changing heartbeat thread, fixing cluster_sync

* some more debug output

* added master

* attempt to make tests more deterministic

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED

* Fixing broken webinterface access

* reverting groovy change

* Fixing read-only internal users

* Using superuser rights for dashboard now

* Adding mode field to _admin/server/role

* added mode TRYAGAIN

* remove inventory lock (does not seem necessary here)

* remove invalid assertion

* fixing agency bugs

* Removing debug output

* return proper errors in case of "method not allowed"

* Fixed up some info messages

* jslint
2017-10-20 18:06:59 +02:00
Kaveh Vahedipour 428e163db9 Return the result of the inquiry (#3465) 2017-10-20 15:01:32 +02:00
Simon Grätzer 9490f1b105 Fixing broken webinterface access 2017-10-19 12:31:24 +02:00
Jan 7840d3f824 Bug fix/fixes 1810 (#3460)
* improve error reporting in RestAqlHandler

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
2017-10-19 11:28:01 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 451abcb23d added non-destructive DISTINCT (#3320)
* added non-destructive DISTINCT

* added tests

* update documentation
2017-10-18 12:52:33 +02:00
Jan 2fee652e77 Bug fix/fixes 1610 (#3425)
* remove memory zone remainders

* remove useless calls

* attempt to fix non-deterministic CRC test

* attempt to fix non-deterministic replication tests

* fix document removal with multiple documents in cluster

* attempt to fix non-determinism of tests

* fix privilege checks for accessing collections by collection id, not by name
2017-10-18 12:23:39 +02:00
Simon Grätzer a7903495a5 Fix createQueue silent unique constraint violation, Fix asan detection in clang, (#3421) 2017-10-18 12:21:31 +02:00
Wilfried Goesgens 88fd685296 Bug fix/fix shard state restore display (#3385)
* don't add  items into the array if we have an empty list member.

* we mustn't display 100% here, since its a rounding error. If, the checkmark would be set.

* rather use string cutting to round so we don't get into the 100% trap.
2017-10-17 14:08:07 +02:00
Jan 9358cc0b8e fix setting minimum number of v8 contexts (#3435) 2017-10-16 21:17:43 +02:00
Wilfried Goesgens bd8d61c1fa use the c++11 way of sleeping (#3429)
* use the c++11 way of sleeping

* fix unit.
2017-10-16 17:40:51 +02:00
Jan f8a7a9b919 fixed issue #3408 (#3419) 2017-10-16 17:39:29 +02:00
Kaveh Vahedipour 46333a762f Bug fix/agency restart after compaction and holes in log (#3413)
* State fixes holes in RAFT index range
* Avoid application of entries older than compaction index _cur and guard for unsigned overflow
2017-10-13 16:01:41 +02:00
Wilfried Goesgens 8bd11eafef Fix the password dialog (#3405)
- make sure the question is only output after log messages are sent to the console
- make sure error messages are sent before re-quoting for the next try
- inform the user whether the password has been sent
2017-10-13 12:30:12 +02:00
m0ppers f9defbfb3f Database info was returning VPack none...was "none" before (#3402) 2017-10-13 12:28:17 +02:00
Jan f1497e0df7 fixed issue #3395 (#3399) 2017-10-13 12:25:46 +02:00
Jan 4174cd20b9 minimum number of V8 contexts in console mode must be 2, not 1 (#3397)
this is required to ensure the console gets one dedicated V8 context and all other
operations have at least one extra context. This requirement was not enforced
anymore.
2017-10-13 12:24:03 +02:00
m0ppers bb1d303473 Cmake 5.0 complains about unused lambda captures (#3390) 2017-10-13 12:20:48 +02:00
Jan 360184be87 fix misleading cluster AQL OOM messages (#3359) 2017-10-13 12:16:18 +02:00
Jan 1579eb9dfd show which unique constraint was violated in an AQL query (#3330)
this addresses https://stackoverflow.com/questions/46427126/arangodb-3-2-unique-constraint-violation-id-or-key
2017-10-13 12:13:44 +02:00
Max Neunhöffer 9a2385b941 Add host id detection and show in /_admin/cluster/Health. (#3389) 2017-10-11 12:42:44 +02:00