1
0
Fork 0
Commit Graph

15897 Commits

Author SHA1 Message Date
Simon 8d5a30cca6 S2 upgrade (#5258) 2018-05-03 18:18:09 +02:00
Jan 9c0460acff
Feature/aql optimizations (#5237) 2018-05-03 16:55:58 +02:00
Jan 4807fdc638
Bug fix/make query string owning (#5255)
* make QueryString owning, to avoid programming errors

* remove unused class
2018-05-03 16:40:50 +02:00
Jan Christoph Uhde 1c88902962 Fix AQL LIKE function - escapes * when building pattern. (#5239) 2018-05-03 13:14:58 +02:00
Mark 34709327ef Fix windows compile errors (#5254) 2018-05-03 11:48:55 +02:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
Matthew Von-Maszewski 01cc6d2159 adjust thread crash logging for simple db server case. (#5234) 2018-05-02 22:35:20 +02:00
Andrey Abramov 4aa3ba0d59
fix undefined macro expansion (#5246) 2018-05-02 20:15:47 +03:00
Andrey Abramov 6b72cb1a81
fix clang-802.0.38 build issue (#5245) 2018-05-02 17:15:58 +03:00
Simon c95522ca67 fix iresearch (#5243) 2018-05-02 16:05:27 +03:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
jsteemann cb51e561b7 fix catch tests 2018-04-30 23:13:50 +02:00
Wilfried Goesgens ba278a64c9 add native implementation of the FAIL aql function (#5232) 2018-04-30 18:35:14 +02:00
jsteemann 2870b8f586 fix potential compile error 2018-04-30 14:18:47 +02:00
Jan 5357adad95
Feature/count plan nodes by type (#5225) 2018-04-30 12:52:32 +02:00
Simon a1416e1067 Make v8 optional on startup (#5220) 2018-04-30 12:48:57 +02:00
Jan 349e2c1681 forward-port AQL optimizations from 3.3 (#5020)
* forward-port AQL optimizations from 3.3

* fix restrict-to-single-shard rule

* added tests for specific cases

* ModificationNodes now can know if they are restricted or not

* OptimizerRule now updates the ModificationNode instead of setting ownName on remote node.

* The DBServerEngine container now only sends a subset of snippets (those which are relevant) It is  able to restrict any part of the query to a certain shard or even a list of shards

* ExecutionEngine CreateBlocks in ClusterMode now returns Result instead of throwing.

* Adapted Coordinator side to a bitter mapping of RemoteNode => QuerySnippets. This is still ongoing work.

* Fixed Coordinator enginecontainer test and finisehd implementation of EngineInfoContainerDBServer. Seems to work so far, need to adapt test to prove it.

* Adapted test for modification nodes and restrict to shard.

* restrictTo can now be de-/serialized on Modification Nodes

* Fixed Community Compile Error

* Added a test for Indexes/EnumerateCollection nodes

* EnumerateCollection and IndexNode now have options to be restricted to a single shard, opt rule not yet updated

* DBServer part of planning now checks Enumerate and IndexNodes for single shard optimization

* Added tests for simple single shard optimization on IndexNodes

* Adapted optimzierrule for SingleShard optimierung on IndexNodes

* Fixed an issue with externally restricted shard ids in combination with shard_id based optimization

* Added a test for multiple single shard optimizations on the same collection

* Removed collection/Shard duplicate check of optimize to single shard rules

* Updated Explainer to handle optimize to single shard

* Let more tests pass an optimized AQL variant

* Temporarily disabled a tests that can only be solved after some work in transaction
2018-04-30 11:25:09 +02:00
Jan 30b12e311b
Bug fix/remove most of aql js (#5223) 2018-04-30 11:17:11 +02:00
Wilfried Goesgens 357ec8ebef prohibit the use of sleep() in the source (#5178) 2018-04-30 10:23:22 +02:00
Simon 2e0e2574d4 Port from 3.3 (#5213) 2018-04-27 17:05:18 +02:00
Matthew Von-Maszewski a67df088b0 correct race condition leading to infinite job execution (#5201)
* fix infinite loop by setting _lastSyncTime within runBackgroundJob().  add code to make agency callback ignore _lastSyncTime limit.
* create change notes for this PR and previous PR 5114
2018-04-27 13:43:22 +02:00
Wilfried Goesgens ea507bb27c use std::chrono and std::date for date / time formatting (#5194) 2018-04-24 18:23:04 +02:00
Simon 468231efc5 AQL Profiling code (#5165)
* initial start of profiling

* adding profiling code

* Fixing remote block tracing, fixing width and units

* Fixing some tests

* Various fixes

* adressing review comments
2018-04-24 16:17:30 +02:00
Wilfried Goesgens 7d6e580780 Refactoring & code cleanup (#5138) (#5142) 2018-04-24 14:42:23 +02:00
Matthew Von-Maszewski a84f7805ad Feature/mv thread death logging (#5111)
* Initial low level interface for thread crash reporting (and management).
* Add a member version of isClusterRole()
* isolate heartbeat thread creation to new StartHeartbeatThread().  create heartbeat thread even if not a cluster or if an agent.
* update runDBServer() and runCoordinator() to shutdown more quickly by polling isStopping() at additional locations.
* copying updates from different branch / PR
* basic thread crash logging.  Not yet tied into Agency arangod or have any specific threads posting crashes
* make Supervision thread a CriticalThread
* sandwich CriticalThread between Thread and other classes to create long term, repeating thread crash reporting.
* restore code lost upon branch update relating to new startHeartbeatThread() function
* add CriticalThread.cpp to build
* add new runAgentServer() function to loop for Agents.  Make Heartbeat thread derive from CriticalThread.
* remove debug line
2018-04-23 15:50:14 +02:00
Vasiliy 012aaa9469 issue 383.4: push vocbase validity check up from Query constructor out into arangodb::consensus::State, StatisticsWorker and AQLUserFunctions calls (#5177) 2018-04-23 14:52:42 +03:00
Simon 45fbed497b Supervision Job for Active Failover (#5066) 2018-04-23 12:49:41 +02:00
Jan 2b84348b77
remove call to requiresElevatedPrivileges with default value (#5172) 2018-04-23 11:28:24 +02:00
Vasiliy 9062c41592 issue 383.3: implement remainder of IResearchViewDBServer tests, use the data-source id (primary key) instead of an arbitrary instance for dropCollection()/dropView(), backport from iresearch upstream: ensure block is flushed if key index is full (#5176) 2018-04-23 00:33:46 +03:00
Vasiliy da4519df43 issue 383.2: backport from iresearch upstream: address issue with _T(…) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC (#5152)
* issue 383.2: backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC

* backport: add more tests for IResearchViewDBServer
2018-04-20 18:20:31 +03:00
Matthew Von-Maszewski dd03ca5dd8 shutdown quicker on db server and coordinator heartbeat threads (#5114)
* shutdown quicker on db server and coordinator heartbeat threads

* Adjust the new condition variable usage to follow normal coding patterns.  Probably was ok.  But why take the chance.  And simplify future maintenance.
2018-04-20 15:00:08 +02:00
Kaveh Vahedipour 3d043b35a3 Feature/supervsion maintenance mode (#5108)
* Supervision goes to Maintenance mode, when /arango/Supervision/Maintenance exists
* coordinator route stands
* stop updates in transient, when supervision off
2018-04-20 13:23:22 +02:00
Jan 1cc5d3a071
remove some unused AQL APIs (#5144) 2018-04-19 11:40:58 +02:00
Vasiliy 6053056b6a issue 383.1: initial stub implementation of IResearchViewDBServer (#5132) 2018-04-18 13:51:21 +03:00
Jan 0972befd87 fix lookups by _id (#5125) 2018-04-18 09:26:29 +02:00
Kaveh Vahedipour 8bbe256633 Bug fix/node smapping logs (#5126)
* Node was spamming the logs for only bad reasons
* lets not spam the customers
2018-04-17 17:01:38 +02:00
Simon fad4c0f9f4 Moving out more from #5066 (#5124) 2018-04-17 16:55:19 +02:00
Wilfried Goesgens 9f5323bc53 Bugfix/fix windows warnings (#5117)
* fix windows warning
2018-04-17 13:35:03 +02:00
Wilfried Goesgens d18e760150 Bugfix/fix windows warnings (#5116)
* fix windows warnings
2018-04-17 12:38:59 +02:00
Wilfried Goesgens 2b1ba8c524 fix windows warnings (#5115) 2018-04-17 11:44:45 +02:00
Matthew Von-Maszewski c0c149cf5b Create non-throwing wrappers for Node access in Agency (#4598)
* safety checkin of Node throw reduction.
* final round of Node throw protection.  Common accessors now protected to force code to hasAsXXX() functions.
2018-04-17 10:21:14 +02:00
Simon 8be273efb8 Replication cleanup (#5105) 2018-04-17 08:17:42 +02:00
Vasiliy 9a4ddee13e issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally (#5107)
* issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally

* remove invalid include
2018-04-16 18:30:51 +03:00
Wilfried Goesgens 2a12e0a02f Fix windows portability (#5113) 2018-04-16 17:20:26 +02:00
Simon 7677afabf1 Remove copy of request body in rest handlers (#5104) 2018-04-16 14:49:51 +02:00
Wilfried Goesgens 5e48d5cd9e if we re-enter a js context make sure we load the AQL functions (#5110) 2018-04-16 13:22:32 +02:00
Tobias Gödderz 089b21e3de fix dump&restore of smart graphs (#5093)
* fix dump&restore of smart graphs (#5093)

* Added tests for dump-restore of SmartGraphs

* Arangosh will now expose isSmart and the smartGraphAttribute on properties

* RestReplicationHandler will now ignore smart-graph collections unless you execute it with force

* Added changelog

* Reactivated original mmfiles dump/restore test

* Skip hidden smart graph collections in arangodump

* Do not dump shadowCollections metadata of smart edge collections

* Cluster optimizer rules for soerted gather nodes now handle virtual edge collections correctly

* Added a dump/restore tests for smartgraphs in rocksdb as well

* Deactivated checks for writesExecuted statistics in dump/restore tests for smartgraphs mmfiles

* Really exclude shadowCollections

* Reduced loglevel

* Added tests

* Don't change single-server behaviour

* Fix tests for omitted shadowCollections and hidden collections

* Activated statistics in MMFIles dump test again and included isEnterprise in rocksdb dump test

* A modification node can now disableStatistics, which means it does not contribute to query->extras() this is only relevant in SmartGraph case so far.

* Added a test to dump&restore satellite collections

* Bugfix: restore satellite collections properly

* Added regression test for internal issue #2237

* Fix bug #2237

* Updated CHANGELOG

* Copied dump/restore tests to rockdsb

* Removed enterprise test

* Added inline comment for smart-edge collections in optimizer rules

* Removed duplicate CHANGELOG entry

* Simplified removal of shadowCollections

* fix jslint

* Made internal.isEnterprise() available in js client context. Taken from 3.3/59f655fa

* fix broken dump_encrypted test

* Removed accidentally copied CHANGELOG entry
2018-04-16 12:28:13 +02:00
Wilfried Goesgens ac2a8721e6 Feature/aql native call apply (#5100) 2018-04-13 16:07:06 +02:00
Jan 79eef177ce
remove sorts from GatherNode if not required (#5019) 2018-04-13 16:02:00 +02:00
Andrey Abramov 6eaaf6abd2
Merge pull request #5103 from arangodb/bug-fix/internal-issue-#374.3
issue 374.3: use a reference to vocbase instead of a pointer in DatabaseGuard
2018-04-13 15:52:01 +03:00