1
0
Fork 0
Commit Graph

5471 Commits

Author SHA1 Message Date
Jan e19c478192 Bug fix 3.4/re add overcommit memory 2 warning (#9643)
* re-add overcommit_memory=2 warning

* updated CHANGELOG
2019-08-06 08:21:37 -04:00
Jan Christoph Uhde 5b2e3f041b disable-foxx 3.4 (#9593) 2019-07-31 10:58:42 +02:00
Wilfried Goesgens 19887387d1 Feature 3.4/add js isolate mem statistics (#9570) 2019-07-26 17:51:53 +02:00
Jan 950aefe6b8
Bug fix 3.4/temporary directory fixes (#9550)
* added startup error for bad temporary directory setting

if the temporary directory (--temp.path) setting is identical to the database directory (--database.directory)
this can eventually lead to data loss, as temporary files may be created inside the temporary directory, causing
overwrites of existing database files/directories with the same names.
Additionally the temporary directory may be cleaned at some point, and this would lead to an unintended cleanup
of the database files/directories as well.
Now, if the database directory and temporary directory are set to the same path, there will be a startup error
warning about potential data loss (though in ArangoDB 3.4 allowing to continue the startup - in 3.5 and higher
we will abort the startup).
2019-07-24 12:46:27 +02:00
Matthew Von-Maszewski 015f59f0b9
Feature 3.4: Use ManagedDirectory class to add gzip and encryption to import/export (#9472)
* preview code I dropped on jackie today.  only supports document collections.

* saving work in progress

* update arangoexport to fully use ManagedDirectory object (arangoimport still needs work).

* migrate remainder of import to using ManagedDirectory objects

* correct spelling within activateEncryption(): encription to encryption

* forgot to fix up graphExport for managed directory.  luckily unittest noticed.

* add test read of gzip json file

* add readGzip to our fs object

* add TRI_SlurpGzipFile function

* add code to bridge javascript fs.readGzip to TRI_SlurpGzipFile

* correct the return type used by JS_ReadGzip

* add gzip test for jsonl type export

* add remain gzip versions of existing export tests

* add import tests of gzip files

* activate encryption in enterprise to enable fs.readDecrypt()

* bug fix: allow encryption to enable in empty directory if EncryptionFeature active.

* add JS_ReadDecrypt to enterprise build.  Current used for unit tests.

* enterprise build needs EncryptionFeature added to build to support TRI_SlurpDecryptFile()

* add TRI_SlurpDecryptFile() to aid in unit test of encrypted stuff

* missing newline at end of file.

* add fs.readDecrypt when available

* add test of encrypted json array export

* correct progress meter when gzip involved

* move from using ManagedDirectory::File to ManagedDirectory::readableFile calls

* create version of readableFile that uses file descriptor as param
2019-07-23 14:58:34 -04:00
Jan 2d9ee2281c
fix leak in thread shutdown (#9527) 2019-07-22 09:40:04 +02:00
Jan c655a89db6
make sure all errors are prefixed with ERROR_ @fceller @kvs85 (#9383) 2019-07-02 18:06:52 +02:00
Wilfried Goesgens b7f933c260 use two simple regexes to workaround macos compilers producing endless loops (#9281) 2019-06-25 09:12:02 +02:00
Jan 81686eebb9
fix handling of comments in ini files (#9170) 2019-06-04 14:47:02 +02:00
Jan e440a1c7d2
fix various small annoyances (#9078) 2019-05-23 17:37:34 +02:00
Wilfried Goesgens 8e0ebb00c6 Bugfix/backport fs changes (#9024) 2019-05-23 12:46:47 +02:00
Kaveh Vahedipour abbba376ad [3.4] clean up container madness (#8791) 2019-05-23 12:43:39 +02:00
Michael Hackstein 4d4c23c302
Bug fix 3.4/collection babies (#9033)
* Prepare API to create multiple collections in a single request to ClusterMethods to improve speedup

* Added counter on how many collections are successfully created

* Allow multi collection creation one level higher

* CollectionMethods now allow batch createion of Collections

* Improved array size assertions

* Now a graph is createad within a single roundtrip in the agency.

* Added new header files

* Insert collections in the AGENCY with TTL and a isBuilding flag, collections with this flag should not be visisible in the coordinator

* Added forgotten C++ file

* Fixed a rare race condition, and the failing IResearch Tests

* readded callback on DONE, otherwise lists are out of sync

* Fixed assertions to let mocked tests pass...

* Fixed community cluster
2019-05-21 08:41:12 +02:00
Jan 4d86dd5fa8
handle incomplete reads (#9019) 2019-05-17 11:17:51 +02:00
Simon 185c8724a3 Pregel various fixes (#8879) 2019-05-07 09:53:18 +02:00
Jan 53f4a84e76 don't reveal secrets via log or to JavaScript apps (#8762) 2019-04-15 19:05:41 +02:00
Simon 3f44a9bba6 fix the pregel code (#8741) 2019-04-12 10:07:16 +02:00
Jan d29d65face
Bug fix 3.4/cleanup 31032019 (#8633) 2019-04-01 17:02:00 +02:00
Dan Larkin-York 455b6b6518 Fix bug in thread creation failure handling. (#8616) 2019-03-28 18:46:00 +01:00
KVS85 c88c4d1416 Remove USE_IRESEARCH definition and usage (#8612) 2019-03-28 17:22:11 +01:00
Jan cf517d0cb6
add AQL functions CRC32 and FNV64 (#8599) 2019-03-27 21:05:56 +01:00
Simran 1d65339002 Doc - setIntroducedIn for log.file-group/mode (#8487)
* Add introducedIn version information

* Update program option dumps
2019-03-21 16:30:05 +01:00
Jan 3f3f0c6fe3
Feature 3.4/ncc1701 (#8441) 2019-03-21 14:53:28 +01:00
Jan dbfa483374
add RocksDB options `--rocksdb.allow-fallocate` and `--rocksdb.limit-open-files-at-startup` (#8492) 2019-03-21 12:05:57 +01:00
jsteemann e7cd25b264 pretend to use result marked as warn-unused 2019-03-20 14:19:36 +01:00
Frank Celler cbacd40e02
added options log.file-mode and log.file-group (#8449) 2019-03-19 09:05:45 +01:00
Frank Celler aca1673b62 Revert "added options log.file-mode and log.file-group"
This reverts commit 07a04dc49b.
2019-03-18 21:23:49 +01:00
Frank Celler 07a04dc49b added options log.file-mode and log.file-group 2019-03-18 21:22:39 +01:00
Jan 6941eb941c
added option `--console.history` to arangosh (#8328) 2019-03-07 13:14:51 +01:00
jsteemann f437b83832 add thread name to messages 2019-03-07 11:42:58 +01:00
Max Neunhöffer 46e479376d
Further supervision fixes. (#8259)
* Do not schedule Coordinators in Plan.

* Finish failed server when server is no longer in health.

* Fix removeServer checks.

Check that server is no longer in use before removing it. Give 60s
waiting time for condition to be met. Also observer agency lock.

* Finish FailedFollower job if server no longer follower.

This can happen because RemoveFollower was faster.

* Only use GOOD servers as replacement followers.

* Fix AddFollower for satellite collections.

* Fix RemoveServer for satellite collections.

* MoveShard handles moves from leader to followers

* Prepare CleanoutServer and FailedServer for satellite collections.

* More sorting out of AddFollower and RemoveFollower.

* Fix RemoveFollower job w.r.t. choice of follower to remove.

* Fix message.

* kill you own sub jobs, please

* Added preconditions to payloads for supervision's job finishers

* Improve logging.

* Add agency diagnostics to failed move shard test, start.

* Add coordinator agency diagnostics.

* Remove warning.

* Add changelog entry.

* Add agency diagnostics if things go sour with move shard.

* Add agency diags when things go wrong 2.

* API /_api/agency/state: back to old format.

* Fix Windows compilation.

* handle aborts in supervision and wait for the last Raft log to be committed

* tests compiling, 2 failing for valid reasons

* Correctly report TRI_ERROR_CLUSTER_CONNECTION_LOST as 503.

* FailedLeader /FailedFollower cannot continue, when aborting blocks
2019-03-04 11:43:35 +01:00
Jan 8e3fb5dfc7
Feature 3.4/improve replication speed (#8268) 2019-02-28 14:37:40 +01:00
Max Neunhöffer b87f362f27
The big supervision fix. (#8243)
* Updated CleanoutServerTests. Exclude servers in ToBeCleanedServers. Allow bad servers as new follower.
* Prefer good servers.
* Removed copy, sort and binary_search for a list of ~10 elements.
* Fix move shard bug with compare.
* MoveShard fixes, expansion of doForAllShards
* Count only GOOD servers in actualReplicationFactor.
* Make RemoveFollower remove broken servers.
* Precondition on Plan Version for updating Current as leader.
* CleanupServer to evict server from ToBeCleaned, when aborting
* cleanoutserver with payload in finish
* Use static string for ToBeCleanedOut.
* Fixed typo in log message.
* Change warning level. If a MoveShard job is aborted and we can no longer roll back, then we issue a WARNING rather than a DEBUG log message.
* Another typo and log level.
* Start to fix unit tests.
* Does not make sense for AddFollowerTest to have a FAILED leader.
* Only count GOOD followers in AddFollower.
* Fix AddFollowerTest.
* Report precondition failed in MoveShard follower case.
* Add CHANGELOG.
2019-02-25 08:12:18 -05:00
Jan 8a16a4b3ae update velocypack (#8075) 2019-01-31 17:31:54 +01:00
Frank Celler 84802fdc0f Feature/maskings (#8006) 2019-01-28 15:04:23 +01:00
Jan 15852cb491
Bug fix 3.4/address jenkins fails (#7985) 2019-01-22 12:32:17 +01:00
Jan e19817fa6c
upgrade bundled version of curl library to 7.63.0 (#7924) 2019-01-19 18:54:53 +01:00
Jan 3c828347dc
do not simplify non-deterministic conditions (#7927) 2019-01-19 18:52:17 +01:00
KVS85 dfad8906d9 Bug fix/active failover fix windows 3.4 (#7959)
* Backport active-failover fix for Windows into 3.4

* Backport stop/resume for Windows from devel

* Backport changes from devel into tests also

* Fix tests

* Remove forgotten whitespaces
2019-01-16 11:08:48 +01:00
Tobias Gödderz d48495c195 [3.4] Foxx queue jobs hanging after Foxxmaster crash (#7921)
* Fixed bug where the Foxxmaster doesn't reset jobs after a crash when it should, or a non-master coordinator removes jobs in progress during startup

* Added a regression test

* Removed foxxmaster test from greylist

* Updated CHANGELOG

* Fixed non-maintainer compile
2019-01-14 16:06:48 +01:00
Jan d5592def42
fix issue #7933: Regression on ISO8601 string compatibility in AQL (#7934) 2019-01-14 13:50:50 +01:00
Kaveh Vahedipour 7b37922f92 releveling logging in maintenance module (#7925) 2019-01-10 12:15:48 +01:00
Matthew Von-Maszewski 474f0cde31 Bug fix 3.4/scheduler empty reformat (#7872)
* added check for empty scheduler

* removed log, old is 1 not 0

* require running in this thread

* test

* added isDirect to callback

* signature fixed

* added drain

* added allowDirectHandling

* disabled for testing

* Add ExecContextScope object to direct call.

* try alternate initialization of ExecContextScope

* remove ExecContextScope, no help.  try _fifoSize as part of direct decision.

* strand management to minimize reuse of same strand per listen socket

* blind attempt to address Jenkins shutdown lock up.  may remove quickly.

* add filename and line to existing error log message

* Adjust queueOperation() to stop accepting items once isStopping() becomes true.

* revert previous check-in to MMFilesCollectorThread.cpp

* big reformat

* fixed merge conflicts

* Add CHANGELOG entry.
2019-01-08 20:39:42 +01:00
Jan 4df4b418f3
Bug fix 3.4/simran (#7883) 2019-01-04 17:08:37 +01:00
Frank Celler 9477af198b big reformat 2018-12-26 00:57:05 +01:00
Frank Celler 8b294742b1 added new clang style and reformat script for clang-format 6.0 2018-12-26 00:55:17 +01:00
Jan 0574393d52
fixed issue #7757 (#7806) 2018-12-19 14:06:11 +01:00
Wilfried Goesgens 7fa0cdc41b Feature/drop before win7 support in compiler (#6681) (#7751) 2018-12-17 12:06:53 +01:00
jsteemann b2c9d80d29 add virtual dtor 2018-12-12 22:00:33 +01:00
Wilfried Goesgens 656fd89bcf Bug fix 3.4/wait for procdump (#7753)
* wait for procdump to exit too.

* move running process check up, propagate status. First check whether processes are still running before attempting to force kill them

* only list processes, don't change their state

* add stopping procdump in all places
2018-12-12 18:06:48 +01:00