1
0
Fork 0
Commit Graph

5549 Commits

Author SHA1 Message Date
Simon 4cfff35cef Use SSE42 for UTF-8 string validation (#6991) 2018-10-23 11:32:06 +02:00
Jan 87d5d29e5f
Feature/optimizations 211018 (#6994) 2018-10-23 09:57:24 +02:00
Jan a161fe78a0 don't fiddle with thread structs that were not initialized yet (#6980) 2018-10-23 09:53:11 +02:00
Jan 360888c3ef
do not copy node_modules directory on startup (#6945) 2018-10-22 16:43:13 +02:00
Simon 23b8e5c735 Propper commit Sequence Numbers (#6958) (#6964) 2018-10-19 14:19:57 +02:00
Simon 8b7a4099b8 Properly compare velocypack objects in Agency operations (#6921)
* Properly compare velocypack objects in Agency operations

* Add changelog

* added option for VPackDumper
2018-10-17 20:03:53 +02:00
Simon 63cfdad52e Fix SimpleHttpClient assertions (#6914)
(cherry picked from commit 0d36790cbaaf8b9f7492534e3c9e5318da909e0f)
2018-10-16 20:31:59 +02:00
Wilfried Goesgens 5b020e465f add missing include (#6896) 2018-10-15 16:31:12 +02:00
Simon 6628a4e55a Refactor stuff, add async batch extension task (#6875) (#6880) 2018-10-15 13:18:24 +02:00
Wilfried Goesgens 67439484e4 add file and linenumber to windows stacktraces (#6865) 2018-10-15 09:35:07 +02:00
Simon fd81f52ab4 Allow WAL logger to split up transactions (#6800) (#6866) 2018-10-12 17:50:58 +02:00
Jan a0980ed2f2
Bug fix/fixes 111018 (#6843) 2018-10-12 12:51:11 +02:00
Jan 96edc0f6ff
Bug fix/fixes 121018 (#6853) 2018-10-12 12:47:30 +02:00
Matthew Von-Maszewski e2bc7e10e3 port of 3.4 libcurl connection management to devel (#6775) 2018-10-10 09:10:34 +02:00
jsteemann 63136348b0 honor our coding guidelines, remove useless debug log message 2018-10-08 16:36:59 +02:00
Lars Maier fac7b48c74 [3.5] Feature/decoupled io (#6281)
* Decoupled IO from Scheduler.
* Fixed SSL start up bug.
* Updated messages and thread names. Fixed missing code from cherry-pick.
* Reintroduced checks for executing thread to be correct. Modifed default value for io-context depending on cores.
* Fixed memory leak caused by cyclic references.
* Actually distribute endpoints. Move handlers into function and do not copy them for each encapsulation.
* Inserted debug output.
* BUG FIXED! One has to call drain() on every queue as temporary work around.
* Added some flags and output for testing.
* More debug output!!!
* Manuel is right.
* Removed debug output.
2018-10-08 13:05:12 +02:00
Jan e78d1aa541
Bug fix/even more ldap debugging (#6736) 2018-10-08 09:42:11 +02:00
Lars Maier 090024daed Reset retry counter only when previously connected in order to not disable the retry check. (#6737)
Abort the doRequest loop when in shutdown.
2018-10-05 17:10:44 +02:00
Jan 3414825696
Feature/improvements 031018 (#6712) 2018-10-04 15:10:54 +02:00
Wilfried Goesgens c1b6426e92 Feature/drop before win7 support in compiler (#6681) 2018-10-03 18:33:52 +02:00
jsteemann 319c05c55d remove unused variables 2018-10-03 13:25:07 +02:00
jsteemann 56147843c2 Merge branch 'feature/additional-logging' of https://github.com/arangodb/arangodb into devel 2018-10-02 17:53:31 +02:00
Jan c06f2d77da
Feature/velocypack update (#6678) 2018-10-02 14:04:14 +02:00
Dan Larkin-York b922d260bc More cleanup and additional logging. 2018-10-02 07:50:26 -04:00
Wilfried Goesgens 73ee5cb40e Fix ICU path building, windows codepage correctness (#6679) 2018-10-02 12:38:24 +02:00
Wilfried Goesgens a1e32118db Bug fix/fix console codepage (#6646) 2018-10-01 17:25:07 +02:00
Jan 4b2588d74f
Bug fix/remove valgrind (#6644) 2018-09-28 17:51:53 +02:00
jsteemann eac5f3edfd better add this include 2018-09-28 15:21:50 +02:00
Jan dade5b1bb1 fix race on Logger shutdown (#6637) 2018-09-27 19:30:27 +02:00
Simon 25f5c841b9 Fix recovery tests on MacOS (#6610) 2018-09-27 18:52:28 +02:00
jsteemann 805a1c5d51 cache pid for logging 2018-09-27 17:38:41 +02:00
jsteemann 39cbd8adcc Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2018-09-25 15:57:36 +02:00
jsteemann 3e41f9ca2b dont reveal passwd 2018-09-25 15:57:22 +02:00
Jan 5a13afd96d
modify msync code so that only base address is page-aligned (#6585)
according to man msync, there is no need for the end address to be page-aligned
also use uintptr_t consistently instead of intptr_t
2018-09-25 09:37:03 +02:00
Wilfried Goesgens a477df49cf Feature/windows utf16 fileaccess (#6534) 2018-09-24 19:41:17 +02:00
Jan abb2ffd98f
centralize some redundant SSL options handling (#6576) 2018-09-24 13:03:48 +02:00
Simon 912f109968 Add simple Future library (#6464) 2018-09-21 16:14:17 +02:00
jsteemann 98a3fecb6a silence a compile warning about potential string value truncation 2018-09-20 01:18:57 +02:00
Simon 0fa7f01c66 Resilience test failure points (#6539) 2018-09-20 01:05:10 +02:00
Simon 82aa24ad7e Copy installation files on startup (#6491) 2018-09-14 11:15:21 +02:00
Max Neunhöffer 84735955ea Add advertised endpoints. (#6104) 2018-09-13 16:30:55 +02:00
Simon 22b9c31c13 Removing ClusterComm ClientTransactionID (#6294) 2018-09-12 22:15:16 +02:00
Jan 9e7fa8420d
align mmfiles index elements to at least pointer alignment (#6460)
this is required for systems with strict alignment requirements
2018-09-12 13:56:06 +02:00
Heiko 2ec1a5cc26 Feature/new devel versioning scheme (#6312) 2018-09-11 08:57:38 +02:00
Jan d1241b1deb
Bug fix/aql modify cleanup (#6380) 2018-09-11 08:54:45 +02:00
Jan d9f669e085
now that we are on c++14 and std::make_unique is available everywhere, we can safely remove our shim for std::make_unique (#6429) 2018-09-10 16:19:06 +02:00
Simon da4585ddad Address leak sanitizer warnings (#6275) 2018-09-10 12:10:55 +02:00
Vasiliy 5329f34771 issue 465.2.2: remove redudnant heap allocations and simplify API (#6349)
* issue 465.2.2: remove redudnant heap allocations and simplify API

* address merge issue

* address more merge issues

* address more merge issues

* address review comments

* do not deallocate non-allocated instances
2018-09-05 13:37:37 +03:00
Simon 0661a4c1fe Hide Links from getIndexes() (#6325) 2018-09-03 15:17:24 +02:00
Jan 07abfca588
Bug fix/cleanup 020918 (#6338) 2018-09-03 12:56:41 +02:00
Jan c8ff719665 Bug fix/always remove lock file (#6314)
* always remove LOCK file

* disable useless messages about environment for upgrade check and password reset
2018-09-01 12:09:23 +02:00
Vasiliy e862efdc3b issue 458.4: retrieve the system database via the SystemDatabaseFeature (#6299) 2018-08-31 19:45:10 +02:00
Jan 840ad8c20d
added option `--rocksdb.use-file-logging` (#6308) 2018-08-31 17:23:00 +02:00
jsteemann 1093c00cd1 add path to error message 2018-08-27 22:31:57 +02:00
jsteemann 1aa162d6d6 add error message 2018-08-27 22:20:34 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Lars Maier 5555bd2fad Schmutz++ Improved (#6259)
* Fixed startup order. Don't start maintenance threads in single-server or agent.
Added range check for `--server.maintenance-threads`.
Fixed invalid array access, when shard exists locally but not in plan.
* Removed unused header imports.
* Added CHANGELOG entry
* Fixed shutdown bug. Startup fixed.
* Fixed catch test.
* Add Maintenance improvements to NewFeature34.md.
2018-08-27 20:25:09 +02:00
Jan 601e2ccbc6
Feature/aql 1808 (#6191) 2018-08-27 14:56:57 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Simon 948820e484 Various small changes (#6234) 2018-08-24 09:39:03 +02:00
Simon 247d41287b More fuerte changes (#6212) 2018-08-22 10:08:33 +02:00
Jan cb12be3e4e
fix crash when pressing CTRL+c in arangosh (#6187) 2018-08-21 11:12:11 +02:00
Simon 229c09d434 Allow dirty-reads from passive (#6136) 2018-08-20 16:26:14 +02:00
jsteemann eca02eaaf1 added "license" return attribute 2018-08-18 16:22:04 +02:00
Matthew Von-Maszewski 86ea784372 bugfix: establish unique function name & implementation for communication retry status (#6150)
* initial checkin of isRetryOK().  Includes fixes to known code that has previously hung shutdowns by performing infinite retries.

* slight help on getting out of a loop faster during shutdown.  not essential.
2018-08-17 14:57:12 +02:00
Wilfried Goesgens e5843d32f8 Improve windows installer (#6045) 2018-08-17 12:37:47 +02:00
Jan d4d0f44f87
add `enterpriseOnly` documentation sections (#6175) 2018-08-17 10:28:25 +02:00
jsteemann 34869f9324 add kernel ASLR detection 2018-08-16 15:13:44 +02:00
Jan d6a3b66e2a
micro optimizations (#6162) 2018-08-16 08:50:16 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Frank Celler a688dc0962
Feature/remove job queue thread (#5986)
limiting V8 calls in flight
2018-08-10 12:17:43 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Simon 606c15ce48 VST support in arangosh (#6047) 2018-08-08 12:18:54 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York b2db010969 Register version and storage engine info in agency to facilitate rolling upgrades (#6061) 2018-08-03 09:46:04 +02:00
Jan dc2a6d3066
always look in etc/relative first for config files (#6049) 2018-08-03 09:38:08 +02:00
Frank Celler da7ef0a025
added __attribute__((unused)) and other minor tweaks to silence some of the warnings (#6035) 2018-08-01 13:22:45 +02:00
shivamdave24 d4c6431ee5 Regex Split Implementation (unfinished) (#5991) 2018-08-01 12:09:36 +02:00
jsteemann ac94de2e96 fix some replication issues 2018-07-27 19:03:43 +02:00
Jan 1f16898027
added more RocksDB options and stats (#6006) 2018-07-26 16:48:25 +02:00
Jan e4d7f1c5f0
Bug fix/wenn der shard mann 2mal klingelt (#5890) 2018-07-26 15:37:40 +02:00
Simon 2dd8593609 View Replication (#5915) 2018-07-26 10:28:46 +02:00
shivamdave24 7db28da251 Levenshtein Distance function implementation with documentation and i… (#5922) 2018-07-25 12:48:58 +02:00
Jan 21064144c8
Bug fix/replication improvements (#5962) 2018-07-25 09:04:50 +02:00
Jan a5bb50b0bf
remove methods from VelocyPackHelper that are also in VPackSlice (#5946) 2018-07-25 09:01:29 +02:00
jsteemann 7d57dead38 use pthread_attr_init instead of memset 2018-07-24 18:09:04 +02:00
Michael Hackstein f7373fe103
Bug fix/lock file cleanup (#5960)
* Added cleanup functions class that allows to execute functions on unexpected shutdown (like FATAL_EXIT). First use: delete the LOCK file. (#5218)

* Removed superfluous log message
2018-07-24 09:43:15 +02:00
Jan ac1d5aac9b
allow starting agency with --console again (requires V8 then) (#5927) 2018-07-24 09:34:22 +02:00
Jan 3319829432
do not run storage engine equality check too early (#5935) 2018-07-24 09:34:11 +02:00
Dan Larkin-York 718e3f7bfa Add load balancing support and user restriction for async jobs API (#5906) 2018-07-23 15:23:18 +02:00
jsteemann ed167ddcd4 fix warning 2018-07-23 14:59:05 +02:00
jsteemann 2138dc0479 Merge branch 'bug-fix/fixes-1707' of https://github.com/arangodb/arangodb into devel 2018-07-19 16:02:30 +02:00
jsteemann d5be86d9ce Merge branch 'bug-fix/rw-lock-changes' of https://github.com/arangodb/arangodb into devel 2018-07-19 11:17:00 +02:00
jsteemann 5c10fc24da fix typo in comment 2018-07-19 10:58:16 +02:00
Simon Grätzer 80baf2bb11 update copyright, use uin32_t everywhere 2018-07-18 23:43:44 +02:00
jsteemann c2ffc3d21d Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/fixes-1707 2018-07-17 13:30:18 +02:00
jsteemann abf41c9f6f be more graceful when options are added to a yet-unknown section 2018-07-17 12:08:07 +02:00
Jan f4b99bb5ed
Bug fix/meierloch (#5895) 2018-07-17 11:39:50 +02:00
Jan 1a128799f2
finalize `--rocksdb.sync-interval` feature (#5856) 2018-07-17 11:09:57 +02:00
jsteemann e512ad98ef remove unused error code 2018-07-16 17:26:43 +02:00
jsteemann 44c7b1b476 remove tabstops 2018-07-16 15:00:12 +02:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
Jan 201a6a308b
allow turning off statistics feature (#5883) 2018-07-16 10:46:43 +02:00
Jan 2710b19487
properly report out-of-range coordinates in geo index queries (#5880) 2018-07-13 21:26:42 +02:00
Jan 2c83454066
Bug fix/fixes 1307 (#5878) 2018-07-13 20:48:21 +02:00
shivamdave24 c45a1b6ac1 Soundex function implementation with integration tests and minor bug fixes to feature/add-aql-tobase64-tohex-encodeuricomponent-uuid branch (#5851) 2018-07-13 17:47:25 +02:00
Jan 45f500c1df
bring back colors when showing option parsing error messages (#5654) 2018-07-13 15:27:19 +02:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
jsteemann 2d6dc9b8bc make performance great again 2018-07-10 14:54:41 +02:00
Jan 208f1297e1
Bug fix/fixes 1007 (#5815) 2018-07-10 13:47:04 +02:00
Jan 0b7c74ca96
make curl not send "Expect: 100-continue" headers (#5730) 2018-07-10 09:17:01 +02:00
Tobias Gödderz fc3e11dbbc Async AQL (#5806)
* Modified header to new initializeCursor API

* Adapted initializeCursor to DONE/WAITING API. Compiles but not tested and no one reacts to WAITING state, it is not returned anywhere yet

* Subqueries now expect a WAITING return from initilize cursor. However they will just return a nullptr and pretend the query is empty, this will be fixed later

* First attempt to simulate thread waiting over information within the query

* Small fix to allow for isDirect handlers to go to sleep.

* Waiting in the necessary places now for the async request to be send.

* Thank you auto-casting compiler, you are totally right i absolutely wanted to use this bool value as an index in may Array. How could i possibly not want to use it here?

* Include cond-var header

* Fixed mutex/cond_var usage

* Added oldAPI wrappers in AQL Blocks for get/skip some variants. This Commit compiles but is NOT tested

* Let getSome now return unique_ptr of AqlItemsBlocks. Also implemented the async variant of getSome in subqueries.

* Removed all references to OLD implementations in AQL. only the base wrappers are allowed to call OLD functions from now on. Now the testing part starts

* Fixed endless virtual recursion

* Implemented new getOrSkip API in SortBlock

* Implemented new getOrSkip API in LimitBlock

* Initilaize all variables

* Fixed logic bug in SubqueryBlock

* getBlock in ExecutionBlock now returns a state. All blocks need to handle this properly!

* Createad a wrapper getBlockOld that servers the old sync api and is used now in AQL. To be replaced overtime.

* Added IndexBlock::skipSome and IndexBlock::getSome

* getBlock now returns its old return value along with the state

* Switch from getBlockOld to getBlock in IndexBlock::skipSome

* Switch from getBlockOld to getBlock in IndexBlock::getSome

* ShortestPathBlock::skipSome is not implemented! Added a regression test

* Attempt to fix SubQueryResult memory management

* Fixed LIMIT Block

* Moved from ShortestPathBlock::getSomeOld to ::getSome

* Implemented ASYNC api on SingletonBlock

* Adapted EnumerateCollectionBlock to new async API

* Fixed FilterBlock and adapted return block to async API

* Adapted NORESULTS block to async AQL api.

* Adapted Modification Blocks to async API

* Fixed some initialize cursor functions to reset values required during get/skipSome

* First steps to adapt ClusterNodes to Async AQL api. Not there yet, need to implement the core still

* Added asnyc implementation for xxxForShard in ClusterBlocks. This commit changes internal logic of _doneForShard. Needs additional testing as soon as everything is in place.

* Adapted CalculationBlock to async API

* Adapted TraversalBlocks to ASYNC Aql. This is not optimal yet, we need a better decission if we are DONE or not on RETURN

* Adapted EnumerateListBlock to Async AQL api

* Adapted RemoteBlock to ASYNC API in getSome/skipSome. The whole thing is now LIVE in the cluster. Exetensive testing to be started now

* Fixed IndexBlock WAITING behaviour if Waiting occurs during a index processing

* Adapted IReasearchViewBlock to ASYNC AQL API

* Fixed SortingGatherBlock in WAITING state.

* Adapted IResearch ExecutionBlockMock to Async API

* Unified the HASMORE/DONE distinction. Code is much more readable now and harder to get incorrect 👍

* Implemented tonly heoretically reachable function of non void function.

* Fixed last commit

* Added inline TODO comments

* fix warning

* Fixed a clearing logic bug in RemoveNodes

* Fixed Error Handling in RemoteBlocks. Also fixed a logic bug (true/false simply has a 50% chance of getting it wrong) in Distribute and Scatter.

* remove unused methods

* Fixed failure test

* implement skipping

* Moved the Query Waiting out of the ExecutionEngine.

* changed one of the collect blocks

* Removed _upstreamState from ExecutionBlockMock, that is in the base-class now

* Added a Test Mock for a an ExecutionBlock that simulates the WAITING/HASMORE/DONE api.

* do not check "hasMore" if not necessary

* Added DistinctCollectBlock::getOrSkipSome from ~Old and changed its return type

(still uses getBlockOld)

* Save state to resume in DistinctCollectBlock::getOrSkipSome

* Extracted redundant code

* fixed some ops

* added one more test

* fix endless blocking

* fix compile error

* fix test

* Refactored HashedCollectBlock::getOrSkipSome

* Return blocks to the manager

* Replaced usage of getBlockOld in HashedCollectBlock::getOrSkipSome

* remove unused shutdown calls, simplify ownership for expressions

* Removed superfluous variable

* Capture const variable by value

* Removed SortedCollectBlock::getOrSkipSomeOld in favour of getOrSkipSome

* Added a working version of SortedCollectBlock::getOrSkipSome

Has yet to be cleaned up

* Removed isTotalAggregation special treatment

* On no input, return a group of nulls (instead of no group at all)

* Bugfixes

* Simplified code

* Move return to the end, eliminate duplicate code

* Corrected skipped count in HashedCollectBlock

* Aligned getNextRow() implementations

* Added comments

* some cleanup

* fix potential memleak

* Bugfix

* Fixed failure tests

* Removed usage of getBlockOld in ExecutionBlock::getOrSkipSome

* Replaced hasMore with an async implementation (mostly)

* Removed getBlockOld()

* Added hasMoreState to the AQL API (and renamed hasMore methods to hasMoreState)

* RemoteBlock now uses the async hasMoreState route

* remove job queue

* options

* Bugfixes in the async implementation of LimitBlock

* LimitBlock::getOrSkipSome now always skips when calculating the fullcount

* fix compile warnings

* restrict threads

* Fixed api of Waiting ExecBlockMock. Unused yet

* Made SortedGatherBlock async-capable

* Removed nonEmptyIndex hack

* Removed duplicate traceGetSome~ calls, moved all to getSome

* Added asserts before replacing getNr*Registers

* Added a TODO note and a comment

* Removed getSomeWithoutRegisterClearoutOld()

* Removed skip()

* Removed common code by using getNr*Registers()

* Use getNr*Registers() in the TraversalBlock as well

* started to add lane

* started to add lane

* added lane

* completed lane

* removed debug output

* fixed merge

* Began working on a test suite for AQL tracing/profiling

* Added more tests and asserts in aql-profiler

* Made some ExecutionBlocks final

* Added a type enum to all blocks and the per-block stats

* Add block type to stats nodes when tracing AQL on block level

* Removed initializeCursor call from instantiateFromPlan

* Avoided additional getSome calls after DONE

* Added more profiler tests

* Refactored ExecutionBlock::getOrSkipSome and fixed two bugs

- set _upstreamState also when skipping
- explicitly use xecutionBlock::getHasMoreState()

* Bugfix: update state

* Reuse parent _skipped wherever possible; rename where not (LimitBlock)

* Simplified SortedCollectBlock::getOrSkipSome and reused general pattern & code

* Implemented missing virtual function (with USE_FAILURE)

* Reset neccessary values during initializeCursor

* Simplified code in EnumerateListBlock a little

* Added a test for DistinctCollectBlock in aql-profiler

* Avoid redundant getSome calls in DistinctCollectBlock

* fix compilation

* Fixed DistinctCollectBlock profiler test

* Added a second profiler test for the DistinctCollectBlock

* Added a profiler test for EnumerateCollectionBlock

* Bugfix in EnumerateListBlock

* added --server.fifoN-size

* Simplified EnumerateCollectionBlock::getSome

* Simplified EnumerateCollectionBlock::getSome, and return HASMORE less often when DONE

* Fix testEnumerateCollectionBlock1 for mmfiles

* do not pass by reference

* Fixed compile error

* fixed merge conflicts

* Added profiler tests for EnumerateCollectionBlock

* Test fix for mmfiles

* Fixed IResearch tests

* Bugfix in DistinctCollectBlock and a regression test

* Updated comment

* Bugfix for query statistics in cluster

* Check plan in distinct test

* Fix aql-profiler tests in cluster

* Removed unused line / bugfix for single server test runs

* This commit implements waking up of AQL queries. (#5651)

* Non-compiling intermediate commit for handover.

* Make branch compile again

* Started implementation of continueable rest cursor handler by moving the callbacks to the outer part. This is not yet fully tested!

* Made finalizeExecute noexcept. We cannot react to this errors as the response was potentially written before. Also introduced continueExecution in the RestHandler engine.

* First successful query wakeup.

* The wakeup callback now posts on the scheduler directly. A resthandler only needs to provide a callback that encapsulates the continueExecution call on this handler

* renamed finalizeExecute to shutdownExecute

* Added a differentiation between Handler and Callback in Query continuation. Handler will be posted in IO service. Callback will be executed directly

* fix audit log

* Removed callback from deleteQueryCursor. This cannot be waiting

* use CONDITION_LOCKER

* removed yet another thread-local variable

* Fixed forward declaration

* Made RestAqlHandler repeatable

* Use defer to close the query in RestAqlHandler. Now waiting will close the query as well.

* Added a mutex in the RestHandlers to make sure if the callback over network is too fast that there is only one Thread running in the RestHandler

* Captured the GeneralCommTask if it is posted to a RestHandler. This is necessary in the PAUSED case

* Refactoring of _noLockHeader responsibilities. Now the BaseHandler selects them and resets them after it is done. Only Coordinators are allowed to define them if a query is loaded.

* Removed reaction to existing nolockheaders in Coordinator Query Planning Phase

* Removed incorrect assertion.

* Further refactoring of NoLockHeaders. Now there is a wrapper class around it which allows for debugging and logging. The state now seems to be better. Also all non-rest-handler triggered queries clean up the NoLockHeaders properly.

* Fixed UserManager, now deletes nolock headers properly

* Swing to the Symphony of Destruction

* Forgot about community build...

* Fixed compiling of Catch tests

* Fixed community build

* need thread for size

* Made the restSimpleHndler repeatable

* Implemented dump and dumpSync in Cursors, Sync will block a thread, dump allows to wait, only relevant for Streaming cursor

* Reactivated StreamingCursors

* Removed debug output.

* Fixed false query continuation

* Reset thread output to non-debug

* Added missing return statements

* Allow some CollectionMethods to hand-in a context that may contain a transaction. This is meant to honor nolock headers.

* Fixed hidden merge conflict

* Bugfix in aql-profiler.js: use plan.nodes order, not stats

* Added two profiler tests for filter

* Avoid too many getBlock calls in the FilterBlock

* Removed debug output

* RemoteBlock API will now send a done(bool) flag whenever we request documents from remote Servers. It is possible that we are DONE and have a result. The pre 3.4.0 API uses exhausted which is exclusive to a result. This API is still implemented for beckwards compatibility.

* Implemented an executeSync function in AqlQuery. This will block the thread until query execution is complete

* Added another test for FILTER, and one test for the HashedCollectBlock

* Added more tests for HashedCollectBlock; avoid unneccessary getSome calls

* Added an profiler IndexBlock test

* IndexBlock: avoid redundant getSome calls, added missing traceGetSomeEnd calls

* Added a second test profiling IndexBlock

* Added a third test for IndexBlock

* Moved general code to module

* Moved noncluster tests into a separate file

* Split aql-profiler testsuite into three files

* Added profiler tests for LimitBlock

* Added a test for NoResultsBlock

* Added profiler tests for TraversalBlock

* Shutdown of an AQL query is now asynchronous. However in Error-Cases it will be executed in a blocking way still

* Optimized TraversalBlock getSome calls due to new (nightly) test results

* Fixed std::min calls I broke

* Let shutdown calls in AQL wait, if the query is executed successfully.

* Fixed queryResult going out of scope

* fix compile error through merge conflict with devel

* Fixed compiler warning "mismatching tags"

* Removed debug log output

* Added TODO notes

* Fixed test fail due to devel merge

* Fixed some invalid sync waiting implementations

* Added a profiler test for SortBlock

* Added profiler tests for SortedCollectBlock

* Fixed bug introduced by devel merge

* Fixed Remoteblocks ignoring errors!

* Added some more continue Callbacks in used places. And removed debug log

* Removed debug log output

* Suppress clang warnings

* Bugfix: use of invalid stack pointer

* Bugfix: RemoteBlock::shutdown now sends code as int, not string

* Revert "Suppress clang warnings"

This reverts commit 05591649c59743c992edd5e78814edc8ca2a83e0.

* Bugfix: cleanup state in RemoteBlock ::shutdown, ::getSome and ::skipSome

* Bugfix in Subquery shutdown: don't skip subquery shutdown when main query shutdown failed

* Allow copy elision
2018-07-09 14:24:10 +02:00
Dan Larkin-York a5ef1e8e86 Finishing touches for cursor load balancer support. (#5787) 2018-07-05 22:48:32 +02:00
Jan 3abc76c4d8
always pass -DNDEBUG for all components (#5769) 2018-07-04 20:55:40 +02:00
Simon b8e140493a Remove redundant geo code, remove WITHIN_RECTANGLE (#5757) 2018-07-04 20:21:00 +02:00
jsteemann 4e78063929 fix small typo 2018-07-03 21:32:39 +02:00
Jan b88fde9659
added key generators "uuid" and "padded" (#5740) 2018-07-03 15:23:47 +02:00
Dan Larkin-York 21e16a8a24 Add load balancer awareness for cursor API (#5682) 2018-07-03 14:29:09 +02:00
Simon 439b12331f Big endian document keys for rocksdb (#5712) 2018-07-03 14:11:22 +02:00
jsteemann a788b81de3 remove logging of warning here. should be handled by AQL query instead 2018-06-28 23:21:17 +02:00
Simon 5ee96fe208 fix multipolygon parser (#5713) 2018-06-28 20:26:30 +02:00
Jan 5af37176d5
add ✓ (#5717) 2018-06-28 20:15:33 +02:00
Simon 52f44596ba Adding MultiPolygon support, fixing some outstanding bugs (#5613)
* Adding MultiPolygon support, fixing some outstanding bugs

* removing dead code

* Adding MultiPolygon tests

* Added multi-polygon docs

* adjust the docs

* fixing remaining occurrences of geo1 / geo2, added note into UpgradingChanges34.md

* some clarification

* Improve Geo docs, reformatting

* Release notes: responses are JSON, reformatting

* Update release notes: /_api/aqlfunction has an isNewlyCreated attribute now

* Update Geo.md
2018-06-22 10:03:27 +02:00
Jan 825ec2b79f
dont use memory for nonces if not needed (#5638) 2018-06-20 14:28:41 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Wilfried Goesgens 0470f945fa Bug fix/improve test status handling (#5591)
* add missing documentation about argument handled in unittest.js

* improve error handling on fs.write

* fix option name

* improve result handling

- add exception handling
- write status bad even before we start the test so we really know when
  something went wrong.

* SYS exceptions will already handle system exceptions

* rethrow

* only return true if the file object has no error bit set

* don't double-close on error.

* add note that we disable exceptions explicitely
2018-06-13 14:10:44 +02:00
Frank Celler efc030ea87 Feature/remove event loop (#5565) 2018-06-11 11:46:17 +02:00
Max Neunhöffer 7bed3f2df0 Try openssl 1.1 (#5123) 2018-06-08 13:21:31 +02:00
Frank Celler c5ac519d1c Bug fix for Read/Write race [WIP] (#5534)
* added wrapper, added asio_ns
* Temporarily fix condition variable bug in job queue.
* preparation for 3.3 back-port
* clang-format
* removed unecessary check, this is now fixed by stand
* added missing RequestStatistics::SET_READ_END
* cosmetics
2018-06-08 10:51:54 +02:00
Jan e6dcff8360
Bug fix/cleanup 2805 (#5477) 2018-05-29 10:16:48 +02:00
Andrey Abramov 0f0235e8a9
Remove collection dependency from gather node (#5457)
* remove collection dependency

* some fixes

* fix test issues

* add tests for newly created functionality

* fix enterprise test fails

* add comment
2018-05-25 17:26:33 +03:00
Simon ef87bb04a4 Allow RestHandler to pause execution (#5458) 2018-05-25 12:27:14 +02:00
Wilfried Goesgens ee9352c0ce Bug fix/refactor read write locks (#5215) 2018-05-24 21:18:00 +02:00
Jan Christoph Uhde a2dcb6cc5d WIP - start adding optional overwrite to insert operation (RepSert) (#5268) 2018-05-24 19:47:15 +02:00
Jan bf5caf210c
Bug fix/cleanup 2305 (#5437) 2018-05-23 15:45:55 +02:00
jsteemann 37c58e1b40 fix define kirmes 2018-05-23 11:52:30 +02:00
Jan d9373531d5
added new ScopeGuard implementation (#5413) 2018-05-23 11:50:52 +02:00
Jan 8e6d5df129
fixed minor several compiler complaints (#5406) 2018-05-23 11:50:00 +02:00
Jan 30d0d840c2
Bug fix/validate from to consistently (#5434) 2018-05-23 11:41:30 +02:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
Jan 16707535db
velocypack micro optimizations (#5402) 2018-05-22 13:27:28 +02:00
Jan fd1ebd2ee9
remove unused defines from directories.h.in (#5407) 2018-05-22 11:32:46 +02:00
Dan Larkin-York 2a8c7f9aef Fix typo in conf search for arangoimp. (#5409)
* Fix typo in conf search for arangoimp.

* Fixed log formatting.
2018-05-21 14:38:22 +02:00
jsteemann cf27c695f9 fix left shift of signed values 2018-05-19 21:59:04 +02:00
Vasiliy 907767f4e1 issue 389.6: ensure proper error handling in StatisticsFeature for missing feature (#5377) 2018-05-18 01:12:39 +03:00
Jan 0d98c0f3ba
Bug fix/fix codescan issues (#5371) 2018-05-16 20:40:37 +02:00
jsteemann d46d01c7fb fix date test 2018-05-16 16:20:45 +02:00
Jan 7147ff9006
apply fixes found by cppcheck (#5360) 2018-05-15 20:06:36 +02:00
Jan 76e2c1b087
abort startup when using SSLv2 for a server endpoint (#5339) 2018-05-15 18:38:34 +02:00
Jan 3e4a595137
remove sscanf calls from datetime functions (#5350) 2018-05-15 14:46:29 +02:00
Jan Christoph Uhde 18a3663746 speed up replication (#5226) 2018-05-14 18:32:26 +02:00
Vasiliy 6a53154160 issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344) 2018-05-14 19:06:24 +03:00
Simon 17b1a2aafb Rest middleware refactoring (#5332) 2018-05-14 17:43:10 +02:00
Jan 2810b60130
avoid redundant creation of the same regexes (#5343) 2018-05-14 17:36:22 +02:00
Jan fe586dac40
retire some old string functions, and make encode/decodeHex copy less data around (#5313) 2018-05-14 10:06:45 +02:00
Jan 4703049c4e
added startup option `--log.escape` for unescaped logging (#5315) 2018-05-11 19:12:16 +02:00
Andrey Abramov 671734b32f
do not scatter empty view (#5318)
* do not distribute scatter views without links

* simplify view downcasting

* refactor & add some tests

* adjust comments
2018-05-11 11:45:22 +03:00
Simran 59de3403c1 Doc - Administration & Programs Refactor (#4907) 2018-05-10 13:05:22 +02:00
Simon 08e355aed8 Simple dump speedup (#5298) 2018-05-09 12:51:04 +02:00
Vasiliy 5d8abce7bb issue 373.2: move toVelocyPack into LogicalDataSource (#5288)
* issue 373.2: move toVelocyPack into LogicalDataSource

* backport: move static DataSource related strings into StaticStrings, add support for registering snapshots on DBServer views
2018-05-08 17:28:39 +03:00
jsteemann 03acfe7cf5 Merge branch 'bug-fix/remove-jemalloc-feature' of https://github.com/arangodb/arangodb into devel 2018-05-08 08:31:56 +02:00
jsteemann 2694fc9f40 more overrides 2018-05-07 23:51:20 +02:00
jsteemann 7f8a1cc614 Merge branch 'bug-fix/add-missing-overrides-and-final' of https://github.com/arangodb/arangodb into devel 2018-05-07 23:02:46 +02:00
Tobias Gödderz 8c87f51429 Feature/fix inconsistent distribute shards like job (#4743) 2018-05-07 16:53:08 +02:00
Kaveh Vahedipour 012718d6b5 Bug fix/default stack size on osx too small (#5265) 2018-05-07 16:50:51 +02:00
Wilfried Goesgens 545019ce37 Bug fix/downgrade s2 2 (#5276)
* downgrade s2 for windows compatibility

* undefine constants if they're already there

*     Fix compilation on Windows for S2 and VS 2017. (#5267)

    * Fix compilation on Windows VS 2017 (s2).

    * Improve compilation fix for windows.

* downgrade reference
2018-05-07 16:24:47 +02:00
Simon fdee0544b7 Using asio::io_context::strands instead of locks (#5266)
* initial try adding strands

* working, stable amount of threads

* improve shell_client cluster

* Fixing some accounting for the scheduler

* Fix accounting

* Fixing wrong strand usage

* add missing return

* Fixing thread accounting

* More scheduler accounting issues

* Fixing various things

* Fixing some stuff

* Fixing some stuff

* Some more subtle bugfixes

* Some cleanup code

* fixing some stuff

* adding some more fixes

* Fixing possible issues

* Fixing missing _storeResult

* Fixing some stuff

* Reducing lambda stack, perhaps fixing hangups

* Fix writeunlocker

* Fixing possible issues

* adding some debugging stuff

* refactor sockets

* possible fixes

* Adding more job guards

* Fixin possible bug

* cleaning up some stuff

* working impl

* Remove debugging output

* Fixing build

* fixing import

* Fixing another bug

* removing debug log

* Removing examples

* Reverting scheduler working code

* Cleanup

* Addressing review comments
2018-05-07 15:58:19 +02:00
jsteemann 52de92d334 add missing override specifiers, add final specifiers 2018-05-04 09:01:50 +02:00
Jan 4807fdc638
Bug fix/make query string owning (#5255)
* make QueryString owning, to avoid programming errors

* remove unused class
2018-05-03 16:40:50 +02:00
Mark 34709327ef Fix windows compile errors (#5254) 2018-05-03 11:48:55 +02:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
Matthew Von-Maszewski 9ff6a41236 Add pacing logic to arangoimp (#5238)
* initial check-in of working auto block size tuning for import.  Needs clean-up.
* partial fix in this branch to enable testing, awaiting better fix in different branch.
* move pacing code from findIdleSender to AutoTuneThread object.
* move pacing code from findIdleSender to AutoTuneThread object.  clean up dead code.  add comments about algorithm.
* edits to algorithm discussion
* correct logging level.  add CHANGELOG entry.
* update comment concerning proper usage of shutdown()
* initialize new static member MaxBatchSize.  Correct sendCsvBuffer() call to addPeriodByteCount() since _outputBuffer drained by sendData().
* remove redundant semicolons.  add new static member MaxBatchSize.
* move MaxBatchSize to ImportHelper so both ImportFeature object and AutoTuneThread object can access it.
* minor updates to comments and statics
* add pacing discussion to import manual page
* minor edit
2018-05-02 22:34:04 +02:00
jsteemann c3efc6eab5 fix compilation 2018-05-02 22:01:11 +02:00
Kaveh Vahedipour affe5e1edf missing entry for error 1498 in dat file (#5244) 2018-05-02 17:07:38 +02:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Dan Larkin-York b9d05351fb Parallelize arangorestore using the tools developed for arangodump (#5206) 2018-05-01 10:56:11 +02:00
Tobias Gödderz 192fc46486 Removed ERROR_ARANGO_COLLECTION_NOT_FOUND (#5231) 2018-04-30 18:34:02 +02:00
Jan a7555e0bf3
make arangovpack work without any config file specified (#5224) 2018-04-30 18:33:42 +02:00
Wilfried Goesgens 357ec8ebef prohibit the use of sleep() in the source (#5178) 2018-04-30 10:23:22 +02:00
Heiko bcd33b8ea1 Bug fix/issue 3811 (#4984)
* now checking existence of _from and _to vertices during edge creation
2018-04-25 14:24:24 +02:00
Matthew Von-Maszewski a84f7805ad Feature/mv thread death logging (#5111)
* Initial low level interface for thread crash reporting (and management).
* Add a member version of isClusterRole()
* isolate heartbeat thread creation to new StartHeartbeatThread().  create heartbeat thread even if not a cluster or if an agent.
* update runDBServer() and runCoordinator() to shutdown more quickly by polling isStopping() at additional locations.
* copying updates from different branch / PR
* basic thread crash logging.  Not yet tied into Agency arangod or have any specific threads posting crashes
* make Supervision thread a CriticalThread
* sandwich CriticalThread between Thread and other classes to create long term, repeating thread crash reporting.
* restore code lost upon branch update relating to new startHeartbeatThread() function
* add CriticalThread.cpp to build
* add new runAgentServer() function to loop for Agents.  Make Heartbeat thread derive from CriticalThread.
* remove debug line
2018-04-23 15:50:14 +02:00
Simon 45fbed497b Supervision Job for Active Failover (#5066) 2018-04-23 12:49:41 +02:00
Jan 646db8ca0a
added `--dump-option` startup option to print all available options i… (#5174) 2018-04-23 11:31:08 +02:00
Jan 2b84348b77
remove call to requiresElevatedPrivileges with default value (#5172) 2018-04-23 11:28:24 +02:00
jsteemann 88a8686654 remove ApplicationFeature "JemallocFeature" 2018-04-21 00:56:10 +02:00
Matthew Von-Maszewski dd03ca5dd8 shutdown quicker on db server and coordinator heartbeat threads (#5114)
* shutdown quicker on db server and coordinator heartbeat threads

* Adjust the new condition variable usage to follow normal coding patterns.  Probably was ok.  But why take the chance.  And simplify future maintenance.
2018-04-20 15:00:08 +02:00
Wilfried Goesgens ef6a294a3b if the process to spawn doesn't contain a path, do a lookup in the host path for it. (#5127) 2018-04-18 16:37:47 +02:00
Wilfried Goesgens a6ddeb3782 Bugfix/fix windows warnings (#5121)
* fix windows warnings
* don't use <date/datetime.h> in global scope so it doesn't interfere with iresearch
2018-04-17 15:44:32 +02:00
Tobias Gödderz 089b21e3de fix dump&restore of smart graphs (#5093)
* fix dump&restore of smart graphs (#5093)

* Added tests for dump-restore of SmartGraphs

* Arangosh will now expose isSmart and the smartGraphAttribute on properties

* RestReplicationHandler will now ignore smart-graph collections unless you execute it with force

* Added changelog

* Reactivated original mmfiles dump/restore test

* Skip hidden smart graph collections in arangodump

* Do not dump shadowCollections metadata of smart edge collections

* Cluster optimizer rules for soerted gather nodes now handle virtual edge collections correctly

* Added a dump/restore tests for smartgraphs in rocksdb as well

* Deactivated checks for writesExecuted statistics in dump/restore tests for smartgraphs mmfiles

* Really exclude shadowCollections

* Reduced loglevel

* Added tests

* Don't change single-server behaviour

* Fix tests for omitted shadowCollections and hidden collections

* Activated statistics in MMFIles dump test again and included isEnterprise in rocksdb dump test

* A modification node can now disableStatistics, which means it does not contribute to query->extras() this is only relevant in SmartGraph case so far.

* Added a test to dump&restore satellite collections

* Bugfix: restore satellite collections properly

* Added regression test for internal issue #2237

* Fix bug #2237

* Updated CHANGELOG

* Copied dump/restore tests to rockdsb

* Removed enterprise test

* Added inline comment for smart-edge collections in optimizer rules

* Removed duplicate CHANGELOG entry

* Simplified removal of shadowCollections

* fix jslint

* Made internal.isEnterprise() available in js client context. Taken from 3.3/59f655fa

* fix broken dump_encrypted test

* Removed accidentally copied CHANGELOG entry
2018-04-16 12:28:13 +02:00
Jan Christoph Uhde 8ca63d6655 Feature/check for equal engines (#4887) 2018-04-13 12:02:34 +02:00
Jan 0e9fa1a08f
make file locking a bit more sensible (#5012) 2018-04-13 11:06:47 +02:00
Jan 8b9eb13925
forward port branch feature-3.3/allow-jwt-in-arangosh (#5009) 2018-04-03 17:37:01 +02:00
Jan 85479a5c99
simplify code a bit (#4936) 2018-04-03 17:32:46 +02:00
Jan e567392e49
improve startup resilience in case there are datafile errors (#4975) 2018-04-03 17:32:30 +02:00
Jan 65bfeb7054
Bug fix/forward port 3.3.5 changes (#4956) 2018-04-03 17:31:34 +02:00
Max Neunhoeffer 5d5a44b01b
Do not store name in properties. 2018-03-28 23:29:43 +02:00
Max Neunhoeffer c7b77076a9
Merge remote-tracking branch 'origin/devel' into feature/arangosearch-cluster-views 2018-03-28 23:26:48 +02:00
Wilfried Goesgens a07a0fe990 Bug fix/date function portability (#4972)
* fix casts to use defined bit width
* adjust regex so it also works on elderly compiler infrastructure
* optimized regex
* work around MSVC bit shortage
* howto manipulate logging of the SUT
* improve regexp readability
2018-03-27 16:32:54 +02:00
Dan Larkin-York 38f162e344 Parallelize arangodump (#4356) 2018-03-26 15:55:14 +02:00
Max Neunhoeffer 41661662c3
Check in errors change. 2018-03-26 11:23:30 +02:00