1
0
Fork 0
Commit Graph

44 Commits

Author SHA1 Message Date
Jan c655a89db6
make sure all errors are prefixed with ERROR_ @fceller @kvs85 (#9383) 2019-07-02 18:06:52 +02:00
Matthew Von-Maszewski 474f0cde31 Bug fix 3.4/scheduler empty reformat (#7872)
* added check for empty scheduler

* removed log, old is 1 not 0

* require running in this thread

* test

* added isDirect to callback

* signature fixed

* added drain

* added allowDirectHandling

* disabled for testing

* Add ExecContextScope object to direct call.

* try alternate initialization of ExecContextScope

* remove ExecContextScope, no help.  try _fifoSize as part of direct decision.

* strand management to minimize reuse of same strand per listen socket

* blind attempt to address Jenkins shutdown lock up.  may remove quickly.

* add filename and line to existing error log message

* Adjust queueOperation() to stop accepting items once isStopping() becomes true.

* revert previous check-in to MMFilesCollectorThread.cpp

* big reformat

* fixed merge conflicts

* Add CHANGELOG entry.
2019-01-08 20:39:42 +01:00
Frank Celler 9477af198b big reformat 2018-12-26 00:57:05 +01:00
Jan 8f1ce5b396
fix issues found by codescan (#7198) 2018-11-02 18:53:34 +01:00
Matthew Von-Maszewski 87c4d99eea Bugfix 3.4: Allow libcurl time to retry DNS request (#7126)
* open up connect limit to allow one DNS retry

* add CHANGELOG entry for dns retry fix

* update warning message wording

* Add lock line that was missed in recent scheduler merge
2018-10-30 18:39:48 +01:00
Matthew Von-Maszewski cfaedce493 defensive coding to ensure lambda function is not dependent upon Communicator object (#7144) 2018-10-30 18:25:11 +01:00
Tobias Gödderz e9388ab710 [3.4] Stop curl from trying to POST stdin (#7097)
* Stop libcurl from trying to POST stdin

* Stop relocking every iteration in wait

* Remove unimplemented function

* Restrict setting of empty POSTFIELDS to POST requests

* Revert locking change
2018-10-29 14:41:23 +01:00
Matthew Von-Maszewski 43016cf04f Bugfix 3.4: address concerns from prior scheduler PR (#7005) 2018-10-23 11:30:45 +02:00
Matthew Von-Maszewski 887822afa6 Bug fix 3.4: libcurl threading changes (#6829)
* enable the ability to push results processing to threads
* have ClusterComm push libcurl response processing to Scheduler threads
* tuning changes from Matthew and Michael
* give new defaults to minimum thread count
* create multiple ClusterCommThreads, each with own Communicator object
* put PR notes in change log
* correct speling
* Also drain V8 queue.
* Add prio V8 to switch in canPostDirectly.
* Accept --server.minimal-threads even if maximal threads is not set.
* Reactivate stopping of threads.
2018-10-12 17:00:55 +02:00
Matthew Von-Maszewski 276da923cf Bugfix 3.4: dynamic management of libcurl's open connections (#6711)
* untested connection count management code.
* add digit to identify location of identical error messages.
* unit tests created, code working and integrated
* subtle changes to smoothing algorithm, close ... but still not perfect
* use scheduler queue() instead of post() to avoid occassional extreme latencies. throughput improved!
* temporary hack to hardcode queue size much lower
* force direct calls to post() to instead route through queue()
* correct issue with previous check-in, but performance has dropped with this on
* revert hardcoded value in SchedulerFeature.cpp
* queue to high priority to be consistent with other Aql actions
* backout all edits to Scheduler.cpp and Scheduler.h
* oops, forgot to remove some testing code.
* 3.4 merge/update
* add changelog entry
2018-10-10 09:11:11 +02:00
jsteemann 2d6dc9b8bc make performance great again 2018-07-10 14:54:41 +02:00
Jan 208f1297e1
Bug fix/fixes 1007 (#5815) 2018-07-10 13:47:04 +02:00
Jan 0b7c74ca96
make curl not send "Expect: 100-continue" headers (#5730) 2018-07-10 09:17:01 +02:00
Simon fdee0544b7 Using asio::io_context::strands instead of locks (#5266)
* initial try adding strands

* working, stable amount of threads

* improve shell_client cluster

* Fixing some accounting for the scheduler

* Fix accounting

* Fixing wrong strand usage

* add missing return

* Fixing thread accounting

* More scheduler accounting issues

* Fixing various things

* Fixing some stuff

* Fixing some stuff

* Some more subtle bugfixes

* Some cleanup code

* fixing some stuff

* adding some more fixes

* Fixing possible issues

* Fixing missing _storeResult

* Fixing some stuff

* Reducing lambda stack, perhaps fixing hangups

* Fix writeunlocker

* Fixing possible issues

* adding some debugging stuff

* refactor sockets

* possible fixes

* Adding more job guards

* Fixin possible bug

* cleaning up some stuff

* working impl

* Remove debugging output

* Fixing build

* fixing import

* Fixing another bug

* removing debug log

* Removing examples

* Reverting scheduler working code

* Cleanup

* Addressing review comments
2018-05-07 15:58:19 +02:00
Jan b2b6c06cbf
Feature/efficiency (#3736) 2018-01-05 16:51:31 +01:00
Jan 73bce34174
potentially fix send request timeout (#4026) 2017-12-13 17:54:12 +01:00
Matthew Von-Maszewski 7e3dc8afb8 connection timeout treated as "send complete" (#3866) 2017-12-08 11:36:43 +01:00
Max Neunhöffer 74458d9d34 Add security check in AgencyComm::sendWithFailover. (#3838) 2017-12-06 10:50:40 +01:00
m0ppers 325065a0c0 show warnings if curl work thread is blocked by error/success cbs (#3316) 2017-09-26 15:12:09 +02:00
m0ppers f0ae3d3de7 Bug fix/uaf communicator request abort (#3216)
* Fix abortion of requests

* Fix progress callback
2017-09-15 22:42:30 +02:00
Max Neunhöffer f458a7d10a Wow...wrong type for httpstatuscode...it would zero out parts of the mutex on the stack (#2800) 2017-07-14 11:36:30 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Jan 6a35df3992 should fix https://github.com/arangodb/planning/issues/413 (#2742) 2017-07-06 18:03:59 +02:00
Andreas Streichardt 7250ba29ee Disable communicator and abort all requests when shutting down 2017-06-08 11:58:15 +02:00
Andreas Streichardt dbf9e61b20 Add more debug output 2017-06-07 14:50:46 +02:00
Andreas Streichardt 4664c958f9 Disable proxy communication alltogether 2017-06-02 14:19:09 +02:00
jsteemann 422b06211f fix some replication issues 2017-05-11 19:00:32 +02:00
Simon Grätzer b304d35ac2 Added rocksdb background thread 2017-04-20 18:50:52 +02:00
jsteemann d024a6d00a remove logging for non-topics 2017-02-10 09:32:50 +01:00
jsteemann fa917937c4 do not use namespaces in header files 2017-02-01 13:41:31 +01:00
jsteemann b96897e296 fixed some issues detected by coverity build tool 2016-12-12 15:41:54 +01:00
Andreas Streichardt 42e5e4318d Fix communicator loglevel 2016-12-07 18:38:14 +01:00
Kaveh Vahedipour 029ff44bb0 Merge back FMH to devel 2016-11-25 16:03:13 +01:00
Frank Celler 73e71e7252 fixed blocking for windows 2016-11-24 09:20:20 +01:00
Kaveh Vahedipour b464b378bf new transact api for agency 2016-11-23 17:59:48 +01:00
Frank Celler 14b134d0b8 filer internal function 2016-11-23 15:46:01 +01:00
Kaveh Vahedipour ab34880bea Merge remote-tracking branch 'origin/devel' into FMH 2016-11-22 10:30:25 +01:00
Andreas Streichardt 98c2a7190d Fix useless abort message 2016-11-21 17:47:05 +01:00
Kaveh Vahedipour ffe7f9f3ad merged in devel 2016-11-16 14:59:53 +01:00
Andreas Streichardt e90cc9df6a Fix windows warning 2016-10-27 10:29:12 +02:00
Jan Christoph Uhde 5b818c3243 enable switching from http to vst fix vst authentication 2016-10-26 23:23:35 +02:00
Frank Celler e4ba82e8e9 rewrite of AgencyComm 2016-10-23 00:46:30 +02:00
jsteemann 0bfe4473ee fix compile warning 2016-10-10 17:10:55 +02:00
Andreas Streichardt e75b8f5503 Implement cURL based inter server communication 2016-10-10 14:56:04 +02:00