1
0
Fork 0
Commit Graph

16952 Commits

Author SHA1 Message Date
Jan f95281471c
fix lagging agency callbacks (#9621) 2019-08-02 11:43:04 +02:00
Jan Christoph Uhde 5b2e3f041b disable-foxx 3.4 (#9593) 2019-07-31 10:58:42 +02:00
KVS85 cc9aa74e1a
Fix ArangoSearch view creation errors (#9598) 2019-07-29 20:42:57 +03:00
Jan Christoph Uhde 6211a5fc4f nested collect backport 3.4 (#9590)
* stop optimization for nested collects (#9484)

* Update CHANGELOG
2019-07-29 19:42:50 +03:00
Wilfried Goesgens 823cc1d8ee don't keep the memory stats in the collections to save disk space (#9583) 2019-07-29 08:45:51 +02:00
Wilfried Goesgens 19887387d1 Feature 3.4/add js isolate mem statistics (#9570) 2019-07-26 17:51:53 +02:00
Jan 51921c4129
fixed a bug in RTRIM implementation, fixed issue #9558 (#9564) 2019-07-25 13:04:21 +02:00
Jan 950aefe6b8
Bug fix 3.4/temporary directory fixes (#9550)
* added startup error for bad temporary directory setting

if the temporary directory (--temp.path) setting is identical to the database directory (--database.directory)
this can eventually lead to data loss, as temporary files may be created inside the temporary directory, causing
overwrites of existing database files/directories with the same names.
Additionally the temporary directory may be cleaned at some point, and this would lead to an unintended cleanup
of the database files/directories as well.
Now, if the database directory and temporary directory are set to the same path, there will be a startup error
warning about potential data loss (though in ArangoDB 3.4 allowing to continue the startup - in 3.5 and higher
we will abort the startup).
2019-07-24 12:46:27 +02:00
Jan 0c9290ea1b
fix a query abort error in smart joins if both collections were (#9533) 2019-07-22 14:55:20 +02:00
Dan Larkin-York 07344d7694 Fix typo and restructure to match. (#9530) 2019-07-22 09:45:07 +02:00
Jan 1a812b4b4f
Bug fix 3.4/fix races in collection creation (#9504) 2019-07-19 13:29:24 +02:00
Tobias Gödderz 94a84b6233 [3.4] Fix query snippet responsibility for views (#9460)
* Fix query snippet responsibility for views

Select one snippet per node id, instead of parent query id

* Updated CHANGELOG

* Fixed compile error
2019-07-18 10:59:53 +02:00
Jan 83227cb7a1
prevent duplicate actions from popping up (#9453) 2019-07-16 10:02:59 +02:00
Matthew Von-Maszewski 4ef47fc7bd BugFix 3.4: Some error results have messages that are not reporting (#9455) 2019-07-11 13:16:15 +02:00
Max Neunhöffer 75f0a63549
Various error reporting fixes plus Maintenance Current fix. (#9398)
* Cleanup new logging.
* Hand on error message from getLocalCollections.
* Better behaviour if a database was announce but has vanished since.
* Fix catch tests.
* Switch on maintenance debugging output.
* Fix maintenance reporting bugs.
* CHANGELOG.
* No error if follower cannot be dropped.
* Improvement to avoid copying.
* Add preconditions to FollowerInfo agency operations.
* Adjust timeouts.
* Use isEqualString instead of compareString.
* Fix Windows compilation.
2019-07-05 13:38:44 +02:00
Jan 3cedbe4a67
replace potentially unsafe binary comparisons with logical ones (#9380) 2019-07-04 14:56:38 +02:00
Simon 5eb4951d29 use a lock when calling unload (#9367) 2019-07-03 16:21:00 +02:00
Jan c655a89db6
make sure all errors are prefixed with ERROR_ @fceller @kvs85 (#9383) 2019-07-02 18:06:52 +02:00
Lars Maier 8bd34fd1e3 [3.4]Precs to check if collection exists (#9283)
* Adding preconditions for jobs to check that the collection still exists.

* Make it compile.

* Fixed tests.
2019-07-01 13:26:01 +02:00
Max Neunhöffer 5f9423057d
Fix agency election lockstep bug. (#9348)
* Fix agency election lockstep bug.

Reset the base point for the random election timeout to now whenever we have
cast a vote, be it for us or for some other server.

* CHANGELOG.
2019-06-27 21:05:59 +02:00
Simon 17468f1dc9 do not buffer index estimates (#9338) 2019-06-26 17:27:01 +02:00
Kaveh Vahedipour b97a62c0de if collection is gone in meantine ... (#9331) 2019-06-26 15:11:40 +02:00
Wilfried Goesgens 1be89f8ce7 Bug fix 3.4/fix snappy dependency chain (#9316)
* fix snappy build dependencies

* fix snowball invocation and dependencies (#9275)
2019-06-25 11:20:18 +02:00
Michael Hackstein f06706e53d Bug fix 3.4/create collections better preconditions (#9306)
* Backward port of #9296

* Updated changelog

* Updated changelog
2019-06-24 08:43:46 +02:00
Heiko 5d5eb23e7f Bug fix/gharial 3 4 edge from and tos verification (#9263)
* attempt to apply PR: pull/#9177

* changelog
2019-06-21 14:43:43 +02:00
Lars Maier 40a72f8b43 Added special lock for local data. Use read and write locking. Do not hold read lock during agency transactions. (#9284) 2019-06-21 14:31:43 +02:00
Lars Maier b7d4b9ab5a Release _to server when abort because of dropped collection or `something serious went wrong`. (#9268) 2019-06-17 15:03:07 +02:00
Dan Larkin-York e4cc3ac776 [3.4] Add drop-check for index creation in cluster (#9220)
* Add drop-check for index creation in cluster.

* Fix return.

* Add changelog entry.

* Address review comments.

* Revert change of shared_ptr to plain atomic.
2019-06-12 20:02:01 +02:00
Jan 65e7557f42 Bug fix 3.4/improve arangorestore (#9156)
* always import system collections first, so if importing some other collection fails later, the system collections are already in place

* properly disable enterprise features when restoring an enterprise dump into a community cluster

* disable enterprise features when restoring an enterprise dump into the community version

* fix handling of "duplicate name" issues when restoring system
collections
2019-06-09 20:21:07 +02:00
Kaveh Vahedipour a46fc0690b fix agency's ttl handling in object assignments (#9180) 2019-06-08 22:23:05 +02:00
Jan 2de2d92054
remove archived RocksDB WAL files on agent servers (#9230) 2019-06-08 19:40:07 +02:00
Michael Hackstein b14259b55e
Fix missed callbacks race condition (#9183)
* If we miss a callback in CollectionCreation also test if we missed later callbacks.

* Updated changelog

* Fixed callback lockers

* Remove lockers vector alltogether, we are protected by mutex anyways

* Fixed include guard

* Do not recheck the Collection that has woken us up

* Fixed recursive lock gathering
2019-06-06 14:07:41 +02:00
Frank Celler 4982ca6c3d class ManagedDocumentResult is not a struct 2019-06-06 12:05:31 +02:00
Jan Christoph Uhde 19190dc963 add shardKeyAttribute to pregel start parameters (#9157) 2019-06-05 14:27:26 +02:00
Jan efc0e6c6f6
fix path length checks when copying js files (#9190) 2019-06-05 10:40:37 +02:00
Frank Celler ae3b956568 fix for windows 2019-06-04 11:51:58 +02:00
Jan 6fa651dbe3
Bug fix 3.4/pregel micro improvements (#9178) 2019-06-04 09:32:29 +02:00
Jan 3dc27af242
improve error message (#9155) 2019-06-03 17:06:44 +02:00
Tobias Gödderz 755c2c2db7 [3.4] Fixed incorrect tracing in TraversalBlock (#9152)
* Trace WAITING results in TraversalBlock

* Added CHANGELOG entry

* Fixed state type
2019-06-03 09:26:43 +02:00
Kaveh Vahedipour 75f0a51d67 [3.4] fix clientid register in state (#9065)
* fix flientid register in state

* remove the log erase locations

* iterator check

* also clear considered

* emplaces also handled

* don't add if clientId empty

* move semantics for log_t

* forward log_t into deque. bail out also as follower, when log_t cannot be emplaced

* whitespaces

* change log

* fix agency test for leadership change during long transactions with clientId

* log_t was as good as it gets

* white spaces

* add clientIds the inc/dec test

* client id for inc/dec test

* Fix a typo in a message.

* Add a comment for better readability.

* Fix agency test.

* Take debugging out.

* Activate another old agency test.

* Fix client id.

* Consistent behaviour across branches.
2019-05-29 21:43:45 +02:00
Simon 87977d1228 Pregel segmented buffers (#9109) 2019-05-28 18:21:56 +02:00
Simon e1fdc04193 Bug fix 3.4/jobs indexes (#9099) 2019-05-27 11:43:05 +02:00
Jan e440a1c7d2
fix various small annoyances (#9078) 2019-05-23 17:37:34 +02:00
Wilfried Goesgens 8e0ebb00c6 Bugfix/backport fs changes (#9024) 2019-05-23 12:46:47 +02:00
Jan 6d0e2e58e7 fix VERSION file handling on upgrade (#9077) 2019-05-22 22:43:37 +03:00
Jan 7ef180c71e
fix crash (#9059) 2019-05-21 18:32:46 +02:00
Jan 7ec209fc74
fix 3-way joins (#9051) 2019-05-21 17:33:50 +02:00
Jan ac5d7efcb0
fix handling of initial `null` values in AQL function `MIN` (#9052) 2019-05-21 17:32:45 +02:00
Max Neunhöffer 5acdf10735
Silence an info message. (#9055) 2019-05-21 15:24:58 +02:00
Michael Hackstein 4d4c23c302
Bug fix 3.4/collection babies (#9033)
* Prepare API to create multiple collections in a single request to ClusterMethods to improve speedup

* Added counter on how many collections are successfully created

* Allow multi collection creation one level higher

* CollectionMethods now allow batch createion of Collections

* Improved array size assertions

* Now a graph is createad within a single roundtrip in the agency.

* Added new header files

* Insert collections in the AGENCY with TTL and a isBuilding flag, collections with this flag should not be visisible in the coordinator

* Added forgotten C++ file

* Fixed a rare race condition, and the failing IResearch Tests

* readded callback on DONE, otherwise lists are out of sync

* Fixed assertions to let mocked tests pass...

* Fixed community cluster
2019-05-21 08:41:12 +02:00
Lars Maier 5b41e5a5c8 [3.4] Scheduler Logging (#9028)
* Log when queues become filled up or are completly filled.
* Added latency for pushing.
* lowered log level.
* clever logging. and mutex. :(
* Reset local clock.
* Try to fix mac compile.
* Improve logging logic for half full and full queue.
* CHANGELOG.
2019-05-20 16:30:52 +02:00
Jan 8925edc131
fix crash in async request handling when request queue is full (#9029) 2019-05-17 16:28:43 +02:00
Jan 4d86dd5fa8
handle incomplete reads (#9019) 2019-05-17 11:17:51 +02:00
Max Neunhöffer e53966c843
Try to fix agency problems with snapshots. (#8947)
* Try to fix agency problems with snapshots.
* Abort MoveShards jobs that have the failed server as fromServer.
* Report aborts.
2019-05-16 14:41:39 +02:00
Simon 30e785537a Bug fix 3.4/pregel gr (#9007) 2019-05-15 18:38:22 +02:00
Tobias Gödderz 398129f1d0 [3.4] Let only on shard per collection, not per DBServer, be responsible for initialize cursor (and shutdown) (#8952)
* Let only on shard per collection, not per DBServer, be responsible for initialize cursor (and shutdown)

Reverted assertion

Changed parameter to const&

* Fixed erroneous merge

* Style correction
2019-05-15 11:21:32 +02:00
Lars Maier 3c9cba32ae Retry updating user collection. (#8999) 2019-05-14 17:39:00 +02:00
Simon f20e3713d2 prgl switcheroo (#8995) 2019-05-14 17:18:08 +02:00
Jan c6f2fc5eb3
fix sortedness of sorted results returned by RETURN DISTINCT (#8934) 2019-05-09 20:10:43 +02:00
jsteemann 36b55e796f fix compile warning 2019-05-09 19:19:36 +02:00
Simon aae758fe53 fix issues with pregel (#8948) 2019-05-09 18:58:27 +02:00
Lars Maier 287dc8c223 Do not skip the check if there is only a single agent. (#8937)
* Do not skip the check if there is only a single agent.
* Updated changelog.
2019-05-09 11:37:23 +02:00
Simon 185c8724a3 Pregel various fixes (#8879) 2019-05-07 09:53:18 +02:00
Jan 4ea195f6a2
less locking for ClusterSelectivityEstimates (#8891) 2019-05-06 13:23:33 +02:00
Lars Maier a1bae63cf1 [3.4] Verbose Abort Reason (#8878)
* Added reason to job abort method.

* Additional abort that is not in devel.
2019-05-01 13:54:47 +02:00
Lars Maier 1c13296f86 [3.4] ClientID Agency Transaction (#8651)
* Changed clientId to format <serverid>:<uuid>.
* Changed behavior if id is not known.
2019-04-30 10:35:34 +02:00
Simon 11aa075747 Backport fix for truncate selectivity (#8863) 2019-04-29 17:10:02 +02:00
Jan 885c473ad7 fix assertion failure during documentation generation (#8858)
* fix assertion

* @mchacki
2019-04-29 13:29:36 +03:00
Jan 71d405915c fix invalid usage of VPackSlice (#8864) 2019-04-27 17:25:43 -04:00
Simon b58bf82871 Fix tick overflow (#8840) (#8856) 2019-04-26 19:20:38 +02:00
jsteemann 5b66fa7aa0 RocksDB returns certain statistics values as strings. honor that 2019-04-26 19:05:58 +02:00
Jan 1b36a68742
fix misleading instruction in error message (#8821) 2019-04-26 11:40:01 +02:00
Jan ddcb904acf
overload all document-producing functions in EmptyIndexIterator with noops (#8780) 2019-04-17 15:54:04 +02:00
Kaveh Vahedipour 60b49bb08b [3.4] db servers reported none slice in agency for newly created database (#8755)
* probably fixed a bug, where db servers reported none slice in agency for newly created databases

* exempt none case from user experience

* also harden local ]collection lookups
2019-04-17 13:17:37 +02:00
Jan 53f4a84e76 don't reveal secrets via log or to JavaScript apps (#8762) 2019-04-15 19:05:41 +02:00
Max Neunhöffer e234713f43
Avoid delays caused by invalidateCurrent and invalidatePlan. (#8747)
* Avoid delays caused by invalidateCurrent and invalidatePlan.
* Do loadPlan and loadCurrent asynchronously in the background.
* Reload server mappings on Plan Version change.
2019-04-15 17:42:32 +02:00
Jan 0133f3c48d
increase the timeout for more reliable test results (#8735) 2019-04-15 10:16:42 +02:00
jsteemann ae62cce45b initialize member 2019-04-12 10:17:28 +02:00
Simon 3f44a9bba6 fix the pregel code (#8741) 2019-04-12 10:07:16 +02:00
Dan Larkin-York ce82a512d1 [3.4] Fix RocksDB recovery ticks (#8654) 2019-04-11 11:37:29 +02:00
Jan 7a28877522
Bug fix 3.4/fix prune memleak (#8716) 2019-04-10 00:10:01 +02:00
jsteemann e3ca0c252b fix wrong assertion 2019-04-09 17:52:53 +02:00
Max Neunhöffer f97997ea0f
Port back timestamp replication from devel. (#8649)
* Port back timestamp replication from devel.
* Some more very minor backports from devel to 3.4.
* CHANGELOG.
2019-04-09 17:09:33 +02:00
Max Neunhöffer e5908fd384
Don't hesitate for fewish shards between phase 1 and phase 2. (#8702)
* Only hesitate between phase 1 and 2 if phase 1 took long.
* Get rid of warning in agency (_id attribute filtered out).
* Resolve externals.
* CHANGELOG.
2019-04-09 17:05:11 +02:00
Jan 22d572c543
Bug fix 3.4/fix test muell (#8704) 2019-04-09 11:27:07 +02:00
Jan bf2141b96a
finally fix AQL error messages in all contexts, so that they are usable (#8705) 2019-04-09 09:17:53 +02:00
Jan a540b795b5
this addresses an issue described in ES-260, and is a forward-port of https://github.com/facebook/rocksdb/pull/5160 (#8697) 2019-04-08 18:20:50 +02:00
Jan a3c2e88fc9
preserve smart-join attribute in DC2DC as well (#8686) 2019-04-08 08:45:38 +02:00
jsteemann 4e93000fcf added debugging variables 2019-04-04 10:18:37 +02:00
Max Neunhöffer 6ceb86acc7
Remove ttl when key overwritten without ttl. (#8667)
* Fix ttl deleted when overwritten without ttl.
* Fix the agency test.
* CHANGELOG.
2019-04-03 12:04:12 +02:00
Jan 17669474c2
Bug fix 3.4/replication fuzz debugging (#8668) 2019-04-03 11:43:36 +02:00
Jan b814a62f5d
optimize lookups by shard key for smart vertex collections (#8640) 2019-04-03 11:39:34 +02:00
Max Neunhöffer 0acb19b18a
Better logic for AddFollower/RemoveFollower scheduling. (#8655)
* Fail a MoveShard job to a FAILED server.
* Better logic for AddFollower/RemoveFollower scheduling.
* Abort MoveShard (leader) in case of a FAILED server in Plan.
* Wait for statistics collections before doing stuff in tests.

cleanOutServer, moveShard, failover and the like.

* Abort MoveShard for follower if FAILED server in Plan.
* Take resigned servers into account when checking for health.
* CHANGELOG.
2019-04-02 23:34:27 +02:00
Jan d29d65face
Bug fix 3.4/cleanup 31032019 (#8633) 2019-04-01 17:02:00 +02:00
Max Neunhöffer 54f84cab92 Performance tuning for many shards. (#8577) 2019-03-29 21:34:45 +01:00
Jan 3631d55146
prevent PregelFeature from shutting down while its workers are running (#8627) 2019-03-29 18:22:37 +01:00
Heiko 6b1d56a62c fixed smg collections within the transaction, added gharial decoding (#8621) 2019-03-29 11:10:36 +01:00
Jan 86599ee2fc acquire mutex when serializing conductor state (#8619) 2019-03-29 09:22:49 +01:00
KVS85 c88c4d1416 Remove USE_IRESEARCH definition and usage (#8612) 2019-03-28 17:22:11 +01:00
Jan cf517d0cb6
add AQL functions CRC32 and FNV64 (#8599) 2019-03-27 21:05:56 +01:00