1
0
Fork 0
Commit Graph

5473 Commits

Author SHA1 Message Date
jsteemann b2c9d80d29 add virtual dtor 2018-12-12 22:00:33 +01:00
Wilfried Goesgens 656fd89bcf Bug fix 3.4/wait for procdump (#7753)
* wait for procdump to exit too.

* move running process check up, propagate status. First check whether processes are still running before attempting to force kill them

* only list processes, don't change their state

* add stopping procdump in all places
2018-12-12 18:06:48 +01:00
Jan 9dbbd1cc9f
Bug fix 3.4/fix thread shutdown (#7729) 2018-12-11 22:17:17 +01:00
hellower fa0e1c438c Update messages.h (#7360)
Korean bye message
2018-12-08 01:44:01 +01:00
Jan 1a4a6e7d2b
minor fixes (#7691) 2018-12-07 16:22:32 +01:00
Jan 677522991e
Feature/internal 3306 (#7683) (#7688) 2018-12-06 17:46:58 +01:00
Wilfried Goesgens d5a64629f8 Feature 3.4/kill em all (#7590) 2018-12-04 09:03:39 +01:00
Jan 836954b8e3
allow using UTF8 filenames for UUID directory (#7569) 2018-11-30 17:25:50 +01:00
Jan 92616c1dca
fix path normalization for Windows (#7565) 2018-11-30 16:21:13 +01:00
Dan Larkin-York dbd59e19cd [3.4] Persist and check default language selection (#7490) 2018-11-29 19:51:19 +01:00
Jan f1086bac4f
added option `--rocksdb.enforce-block-cache-size-limit` (#7508) 2018-11-28 20:40:20 +01:00
Jan ffc823e1c8
Bug fix 3.4/backport optimizations (#7434) 2018-11-26 19:16:05 +01:00
jsteemann 9658300f11 revert Scheduler changes 2018-11-26 09:54:41 +01:00
Jan b363372c63
Bug fix 3.4/remove shutdown assertion (#7387) 2018-11-22 15:36:06 +01:00
Jan 19dc2ca0b7
yet more micro optimizations (#7399) 2018-11-21 17:09:16 +01:00
Simon ef239cbe4e Make recovery more reliable (#7297) (#7367) 2018-11-21 16:51:38 +01:00
Kaveh Vahedipour 860fa21219 Bug fix 3.4/index readiness (#6716)
* backport of test data generation for maintenance from devel
* 3.4 working
* fixing index use in cluster while still being built
* fixed broken views
* correct 200 for ensureIndex
* merge with 3.4
* agency comm to handle replace in array
* supervision changes
* cluster info's exsureIndex
* 3.4 ready
* timeout
* missing files from origin
* neunhoef complaints
* bogus entry
* no need to wait for current once again
* no longer necessary. done in IndexFactory now
* correct comments
* left overs
* dead code revived
* Move CHANGELOG entry to the right place.
2018-11-21 14:41:36 +01:00
Jan d27c4cc113
Bug fix 3.4/aql speedup (#7378) 2018-11-20 16:08:16 +01:00
Max Neunhöffer 805f7a7621
Fix timeout in cluster operation in create and drop collections. (#7300)
* Fix loophole.
* Fix inquiry case of id not found: 404.
* Also handle correctly in AgencyComm.
* Fix agency tests.
* Fix error handling in dropCollectionOnCoordinator.
2018-11-14 10:02:26 +01:00
jsteemann 046dac4234 fix define name 2018-11-12 13:09:48 +01:00
Wilfried Goesgens ccc43cd932 Bug fix 3.4/remove enterprise goto (#7274)
* add more information when timeout failing the index creation tests

* rather use null-pointers than try/catch for control flow
2018-11-08 16:27:51 +01:00
Jan 1af9443efb
turn off warnings about `overcommit_memory` settings (#7235) 2018-11-07 11:07:21 +01:00
Dan Larkin-York e5243f9254 [3.4] Upgrade old MMFiles collections for compatibility with ArangoSearch (#7133) 2018-11-06 11:12:54 +01:00
Frank Celler 07ee9c48e1 release version 3.4.0-rc.4 2018-11-05 00:12:28 +01:00
Vasiliy d644561f1f issue 496.4.1: backport 3.4: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7213)
* issue 496.4.1: backport 3.4: move StorageEngine-specific flag out of the genric API and closer to the storage engine

* address merge issue
2018-11-04 16:52:54 +03:00
Simon 9c53d045be Server stream cursor (#7186) (#7210) 2018-11-03 20:17:52 +01:00
Jan 8f1ce5b396
fix issues found by codescan (#7198) 2018-11-02 18:53:34 +01:00
Jan 8420f21376
add GE goodbye (#7206) 2018-11-02 16:23:21 +01:00
Jan aa385c2bbc
increase the maximum allowed number of collections in an AQL query (#7155) 2018-10-31 10:08:32 +01:00
Matthew Von-Maszewski 87c4d99eea Bugfix 3.4: Allow libcurl time to retry DNS request (#7126)
* open up connect limit to allow one DNS retry

* add CHANGELOG entry for dns retry fix

* update warning message wording

* Add lock line that was missed in recent scheduler merge
2018-10-30 18:39:48 +01:00
Matthew Von-Maszewski cfaedce493 defensive coding to ensure lambda function is not dependent upon Communicator object (#7144) 2018-10-30 18:25:11 +01:00
Simon c073b9dbbe Make ensureIndexOnCoordinator more robust (#7110) (#7130) 2018-10-30 11:25:06 +01:00
Jan 889cee0dae
speed up query string parsing (#7116) 2018-10-29 14:42:39 +01:00
Tobias Gödderz e9388ab710 [3.4] Stop curl from trying to POST stdin (#7097)
* Stop libcurl from trying to POST stdin

* Stop relocking every iteration in wait

* Remove unimplemented function

* Restrict setting of empty POSTFIELDS to POST requests

* Revert locking change
2018-10-29 14:41:23 +01:00
jsteemann 253b00aba1 change banner 2018-10-27 18:50:24 +02:00
Michael Hackstein e05880895a
Bug fix 3.4/shorter foot in door (#7084)
* Implement `syncCollectionCatchup` in DatabaseTailingSyncer.

First stab, might not even compile.

* Fixed a typo.

* Fix a typo and a compilation problem.

* Further compilation fix.

* Implement two stage catchup.

* Two small corrections.

* Unified error messages in Synchronize shard job.

* Improved a code comment.

* Fixed autocasting bool->double and double->bool issue. That is truely one of the best features ever invented... </irony>

* Renamed doHardLock => toSoftLockOnly and inverted default value

* Merged soft/hard foot logic with Transaction splits

* Use scopeguards to cancel readlocks
2018-10-26 16:16:52 +02:00
Jan 2d0fce0fec
add option `--rocksdb.total-write-buffer-size` (#7024) 2018-10-25 13:12:27 +02:00
Simon e87b42a0c3 Silence tsan warnings (#7051) 2018-10-24 23:58:47 +02:00
Vasiliy 52e2c97693 backport missed changes (#7016) 2018-10-24 15:43:45 +03:00
Simon 8b19d40136 Properly compare velocypack objects in Agency operations (#6922) 2018-10-23 11:52:22 +02:00
Matthew Von-Maszewski 43016cf04f Bugfix 3.4: address concerns from prior scheduler PR (#7005) 2018-10-23 11:30:45 +02:00
Jan 29383d1082
don't fiddle with thread structs that were not initialized yet (#6979) 2018-10-22 16:49:13 +02:00
Jan 169fb8edb7
do not copy node_modules directory on startup (#6946) 2018-10-22 16:43:26 +02:00
Simon c2b6fb99ba Propper commit Sequence Numbers (#6958) 2018-10-18 18:25:34 +02:00
Jan 18de63c7c8
Feature 3.4/medium priority (#6910) 2018-10-18 17:08:39 +02:00
Simon f5392d3c53 Fix SimpleHttpClient assertions (#6913) 2018-10-16 20:31:29 +02:00
Simon 7eda6768ab Refactor stuff, add async batch extension task (#6875)
* Refactor stuff, add async batch extension task

* fix compilation
2018-10-15 11:43:45 +02:00
Matthew Von-Maszewski 887822afa6 Bug fix 3.4: libcurl threading changes (#6829)
* enable the ability to push results processing to threads
* have ClusterComm push libcurl response processing to Scheduler threads
* tuning changes from Matthew and Michael
* give new defaults to minimum thread count
* create multiple ClusterCommThreads, each with own Communicator object
* put PR notes in change log
* correct speling
* Also drain V8 queue.
* Add prio V8 to switch in canPostDirectly.
* Accept --server.minimal-threads even if maximal threads is not set.
* Reactivate stopping of threads.
2018-10-12 17:00:55 +02:00
Simon 010b54c81e Allow WAL logger to split up transactions (#6800) 2018-10-12 15:15:55 +02:00
jsteemann fb051ca0bf fix cast 2018-10-12 14:17:44 +02:00
Jan 3a552cbc4f
Bug fix 3.4/fixes 111018 (#6844) 2018-10-12 12:49:43 +02:00
Jan 3adaf001c5
velocypack library update (#6850) 2018-10-12 12:46:52 +02:00
Matthew Von-Maszewski 276da923cf Bugfix 3.4: dynamic management of libcurl's open connections (#6711)
* untested connection count management code.
* add digit to identify location of identical error messages.
* unit tests created, code working and integrated
* subtle changes to smoothing algorithm, close ... but still not perfect
* use scheduler queue() instead of post() to avoid occassional extreme latencies. throughput improved!
* temporary hack to hardcode queue size much lower
* force direct calls to post() to instead route through queue()
* correct issue with previous check-in, but performance has dropped with this on
* revert hardcoded value in SchedulerFeature.cpp
* queue to high priority to be consistent with other Aql actions
* backout all edits to Scheduler.cpp and Scheduler.h
* oops, forgot to remove some testing code.
* 3.4 merge/update
* add changelog entry
2018-10-10 09:11:11 +02:00
Lars Maier 34ec28c6b9 Reset retry counter only when previously connected in order to not disable the retry check. (#6753)
Abort the doRequest loop when in shutdown.
2018-10-08 12:54:09 +02:00
Jan ac3ca432fa Bug fix 3.4/even more ldap debugging (#6735)
* forward-port changes from 3.3

* remove useless log output
2018-10-05 18:44:37 +02:00
Wilfried Goesgens 26bb818d5c Bug fix 3.4/fix windows cp defaults (#6682) 2018-10-02 18:59:42 +02:00
jsteemann fac87b9d6b upgrade banner 2018-10-02 12:09:56 +02:00
Dan Larkin-York 3d15b96f66 [3.4] Additional and cleaned-up logging (#6648) 2018-10-01 19:03:05 +02:00
Wilfried Goesgens fbd2ca2e79 Feature 3.4/windows utf16 fileaccess (#6600) 2018-10-01 17:19:01 +02:00
Frank Celler 6611854e72 release version 3.4.0-rc.2 2018-09-30 14:05:06 +02:00
Jan 68e81da38e
handle DeleteRange markers on RocksDB recovery (#6565) 2018-09-28 11:10:11 +02:00
Jan 1ed723604b fix race on Logger shutdown (#6635) 2018-09-27 19:30:08 +02:00
jsteemann 202bea538d banner 2018-09-27 19:10:50 +02:00
Heiko cf07af9b1b Fix the recovery tests under mac os x (#6593)
* this pr will fix the recovery tests under mac os x

* removed unused define
2018-09-26 21:57:10 +02:00
Jan bf80d4c040 modify msync code so that only base address is page-aligned (#6585)
according to man msync, there is no need for the end address to be page-aligned
also use uintptr_t consistently instead of intptr_t
2018-09-26 09:29:17 +02:00
jsteemann 8a7ad7c779 silence a compile warning about potential string value truncation 2018-09-20 01:15:23 +02:00
Simon 3c965ee48a Resilience test failure points (#6545) 2018-09-20 01:04:38 +02:00
Simon 1b1fbde2c8 Copy installation files on startup (#6494) 2018-09-14 11:14:50 +02:00
Kaveh Vahedipour 2041e56f44 advertised endpoints (#6493) 2018-09-14 10:05:46 +02:00
Jan b029186edc
Bug fix/aql modify cleanup (#6380) (#6458) 2018-09-12 13:56:59 +02:00
Simon 3eed525481 Hide links (#6348) 2018-09-03 15:36:37 +02:00
Frank Celler d160c982ef Merge branch 'devel' of https://github.com/arangodb/arangodb into 3.4 2018-09-02 11:39:10 +02:00
Jan c8ff719665 Bug fix/always remove lock file (#6314)
* always remove LOCK file

* disable useless messages about environment for upgrade check and password reset
2018-09-01 12:09:23 +02:00
Vasiliy e862efdc3b issue 458.4: retrieve the system database via the SystemDatabaseFeature (#6299) 2018-08-31 19:45:10 +02:00
Jan 840ad8c20d
added option `--rocksdb.use-file-logging` (#6308) 2018-08-31 17:23:00 +02:00
Frank Celler aa3f0eab05 Merge branch 'devel' of https://github.com/arangodb/arangodb into 3.4 2018-08-28 00:04:29 +02:00
jsteemann 1093c00cd1 add path to error message 2018-08-27 22:31:57 +02:00
jsteemann 1aa162d6d6 add error message 2018-08-27 22:20:34 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Lars Maier 5555bd2fad Schmutz++ Improved (#6259)
* Fixed startup order. Don't start maintenance threads in single-server or agent.
Added range check for `--server.maintenance-threads`.
Fixed invalid array access, when shard exists locally but not in plan.
* Removed unused header imports.
* Added CHANGELOG entry
* Fixed shutdown bug. Startup fixed.
* Fixed catch test.
* Add Maintenance improvements to NewFeature34.md.
2018-08-27 20:25:09 +02:00
Jan 601e2ccbc6
Feature/aql 1808 (#6191) 2018-08-27 14:56:57 +02:00
Frank Celler ca48ba43fc release version 3.4.preview1 2018-08-24 12:19:28 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Simon 948820e484 Various small changes (#6234) 2018-08-24 09:39:03 +02:00
Simon 247d41287b More fuerte changes (#6212) 2018-08-22 10:08:33 +02:00
Jan cb12be3e4e
fix crash when pressing CTRL+c in arangosh (#6187) 2018-08-21 11:12:11 +02:00
Simon 229c09d434 Allow dirty-reads from passive (#6136) 2018-08-20 16:26:14 +02:00
jsteemann eca02eaaf1 added "license" return attribute 2018-08-18 16:22:04 +02:00
Matthew Von-Maszewski 86ea784372 bugfix: establish unique function name & implementation for communication retry status (#6150)
* initial checkin of isRetryOK().  Includes fixes to known code that has previously hung shutdowns by performing infinite retries.

* slight help on getting out of a loop faster during shutdown.  not essential.
2018-08-17 14:57:12 +02:00
Wilfried Goesgens e5843d32f8 Improve windows installer (#6045) 2018-08-17 12:37:47 +02:00
Jan d4d0f44f87
add `enterpriseOnly` documentation sections (#6175) 2018-08-17 10:28:25 +02:00
jsteemann 34869f9324 add kernel ASLR detection 2018-08-16 15:13:44 +02:00
Jan d6a3b66e2a
micro optimizations (#6162) 2018-08-16 08:50:16 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Frank Celler a688dc0962
Feature/remove job queue thread (#5986)
limiting V8 calls in flight
2018-08-10 12:17:43 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Simon 606c15ce48 VST support in arangosh (#6047) 2018-08-08 12:18:54 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York b2db010969 Register version and storage engine info in agency to facilitate rolling upgrades (#6061) 2018-08-03 09:46:04 +02:00
Jan dc2a6d3066
always look in etc/relative first for config files (#6049) 2018-08-03 09:38:08 +02:00