1
0
Fork 0
Commit Graph

79 Commits

Author SHA1 Message Date
Jan 3f3f0c6fe3
Feature 3.4/ncc1701 (#8441) 2019-03-21 14:53:28 +01:00
Michael Hackstein fb956ccc67 PRUNE in AQL Traversal (#8286) 2019-03-07 21:13:26 +01:00
Jan ec242818f8
allow explaining with allPlans: true (#7977) 2019-01-19 18:52:00 +01:00
Dan Larkin-York 4210d9eab6 [3.4] Updates to collection versioning. (#7260) 2018-11-19 09:45:09 +01:00
Dan Larkin-York 9eb005b8bf Fix query profiling discrepancy in console. (#7240) 2018-11-06 14:49:46 +01:00
Jan 9b7258a4d1
added several query optimizations (#6781) 2018-10-12 17:41:45 +02:00
Jan 1a303dd43d
fix some issues with sorted variant of COLLECT (#6432)
* fix some issues with sorted variant of COLLECT

* honor review comments
2018-09-10 16:20:57 +02:00
Jan 1810ec9909
fix profile output for nodes without calls to getSome (#6396) 2018-09-05 20:25:18 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Andrey Abramov 615e1cf7de
Feature/add options to view (#6255)
* added OPTIONS for views

* pass options too

* validation for OPTIONS

* fix OPTIONS validation

* removed debug log message

* add forceSync options

* add forceSync view option

* fix comments

* fix some memleaks

* fix typo

* fix integration tests

* fix integration tests

* fix another integration test

* rename 'forceSync' option to 'waitForSync'

* another attempt to fix test

* ensure query caches are properly invalidated

* address integration test failures

* another attempt to fix test
2018-08-26 22:30:23 +03:00
Jan 0d48677619
Bug fix/aql optimizer improvements 2308 (#6241) 2018-08-24 13:42:22 +02:00
Simon fc204a2b3e fix profiling code (#6190) 2018-08-20 12:59:07 +02:00
Jan 99f176bed5 Feature/add search keyword (#6166)
* added SEARCH keyword

* fixes jslint errors

* add incompatibility notice

* remove VIEW keyword (does not work yet!!)

* add initial support for SEARCH keyword to optimizer rules

* replace FILTER with SEARCH keyword in tests

* removed VIEW keyword, make shell_server_aql tests pass again in single server

* cleanup

* handle SEARCH clause for views

* make SEARCH a non-keyword

* fixed an issue with duplicate variables

* fixed 3 tests

* fix SEARCH statement support for views

* minor refactoring

* fix integration tests

* optimize SEARCH conditions a bit more

* fix jslint error

* fixed wrong comments and typo in class name

* fix documentation

* adjust recovery tests to use SEARCH instead of FILTER

* fix another failing test
2018-08-18 20:22:58 +03:00
Jan d7e631eddf
Bug fix/more sparsung (#6177) 2018-08-17 08:47:45 +02:00
Jan 42fdef45d4
Bug fix/fix profile query (#6073) 2018-08-06 17:02:27 +02:00
Jan e3ce2e8cfb
fix anonymization with unique indexes (#6068) 2018-08-03 12:17:05 +02:00
Heiko 434cfce699 Feature/query profiling ui (#6027) 2018-08-03 10:58:22 +02:00
Jan Christoph Uhde bcf9e30a4d Feature/cluster single operations (#5585) 2018-07-12 20:18:36 +02:00
jsteemann c0517b69f2 nicify output 2018-07-03 16:53:27 +02:00
Wilfried Goesgens 2613ca56b9 add the primary indices to the AQL modification nodes serialization and their explain output (#5628) 2018-06-21 11:45:15 +02:00
Jan edb24deafe
cleanup collection/vocbase interface of ExecutionNodes (#5557) 2018-06-08 15:28:48 +02:00
Jan dc97eb0111
Subquery optimizations (#5398) 2018-05-28 13:56:03 +02:00
Jan a295eaa120
show AQL functions used in query in explain output (#5451) 2018-05-24 21:17:51 +02:00
Jan 4b1ef1dd3a
add support for named graphs (#5385) 2018-05-22 16:40:51 +02:00
Jan 8cd52ae95b
remove obsolete printBrowser function (#5259) 2018-05-04 11:30:39 +02:00
Jan 9c0460acff
Feature/aql optimizations (#5237) 2018-05-03 16:55:58 +02:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Jan 349e2c1681 forward-port AQL optimizations from 3.3 (#5020)
* forward-port AQL optimizations from 3.3

* fix restrict-to-single-shard rule

* added tests for specific cases

* ModificationNodes now can know if they are restricted or not

* OptimizerRule now updates the ModificationNode instead of setting ownName on remote node.

* The DBServerEngine container now only sends a subset of snippets (those which are relevant) It is  able to restrict any part of the query to a certain shard or even a list of shards

* ExecutionEngine CreateBlocks in ClusterMode now returns Result instead of throwing.

* Adapted Coordinator side to a bitter mapping of RemoteNode => QuerySnippets. This is still ongoing work.

* Fixed Coordinator enginecontainer test and finisehd implementation of EngineInfoContainerDBServer. Seems to work so far, need to adapt test to prove it.

* Adapted test for modification nodes and restrict to shard.

* restrictTo can now be de-/serialized on Modification Nodes

* Fixed Community Compile Error

* Added a test for Indexes/EnumerateCollection nodes

* EnumerateCollection and IndexNode now have options to be restricted to a single shard, opt rule not yet updated

* DBServer part of planning now checks Enumerate and IndexNodes for single shard optimization

* Added tests for simple single shard optimization on IndexNodes

* Adapted optimzierrule for SingleShard optimierung on IndexNodes

* Fixed an issue with externally restricted shard ids in combination with shard_id based optimization

* Added a test for multiple single shard optimizations on the same collection

* Removed collection/Shard duplicate check of optimize to single shard rules

* Updated Explainer to handle optimize to single shard

* Let more tests pass an optimized AQL variant

* Temporarily disabled a tests that can only be solved after some work in transaction
2018-04-30 11:25:09 +02:00
Simon 95c8710dec Fixing subqueries in aql profiler (#5195)
* Fixing subqueries in aql profiler

* Adding additional testsuite

* Fix AQL profiling example name, add API example

Also use different query for explain and profiling with SLEEP() for
non-zero runtimes.

* New/updated examples for query explain/profiling

* Fix runtime calculation
2018-04-25 13:35:55 +02:00
Simon 468231efc5 AQL Profiling code (#5165)
* initial start of profiling

* adding profiling code

* Fixing remote block tracing, fixing width and units

* Fixing some tests

* Various fixes

* adressing review comments
2018-04-24 16:17:30 +02:00
Michael Hackstein 0d2d61ca9c
Bug fix 3.3/anonymize in debug dump (#5190) (#5191)
* Fixed the amount of Xs generated by debug dump

* Fixed typo in debugDump information
2018-04-24 13:26:21 +02:00
Michael Hackstein a8e737f16f
Feature/debugdump ui (#5187)
* Adeded a route to aardvark to export a debug dump, it can optinally contain anonymized example data

* Added the Download Debug Button to UI

* fixed zip file download of query dump file

* removed not needed vars

* Added JWT token authentication and error display on debug dump button

* Fixed Error reporting on download debug dump.

* Anonymization does not anonymize system attributes any more.

* Anonymize in explainer debugDump will now anonymize bindVars and will not anonymize system document attributes.
2018-04-24 09:43:04 +02:00
Jan 2eb6d719ca
added ArangoShell helper function for packaging all information about an AQL query so it can be run and analyzed elsewhere: (#5023)
query = "FOR doc IN @@collection FILTER doc.value > @value RETURN doc";
    bind = { value: 42, "@collection": "mycollection" };
    options = { examples: 10, anonymize: true };
    require("@arangodb/aql/explainer").debugDump("/tmp/query-debug-info", query, bind, options);

Entitled users can send the generated file to the ArangoDB support to facilitate
reproduction and debugging.

The data from the generated file can be restored and analyzed via the *inspectDump*
function:

    require("@arangodb/aql/explainer").inspectDump("/tmp/query-debug-info");
2018-04-05 14:06:59 +02:00
Jan 167dfcdf7f
Feature/index only scans (#5005) 2018-04-03 17:34:28 +02:00
Jan 11cfbf5039
Feature/optimize cluster collect (#4897) 2018-04-03 17:28:07 +02:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan 9bebd33ba3 fix invalid first group results for sorted AQL COLLECT when LIMIT was used (#2577) 2017-06-16 09:44:14 +02:00
Michael Hackstein 2e417a1576 db._explain should now be backwards-compatible if we access a 3.1 instance with a 3.2 arangosh 2017-04-26 11:01:05 +02:00
Wilfried Goesgens 05a46e4781 fix db._explain() to work with the new interface of the graph traverser 2017-04-25 20:33:17 +02:00
jsteemann f5c8932744 initial version of query execution plan cache, still very rough 2017-02-21 11:58:38 +01:00
jsteemann c9a2c028fa emit sort registers for GatherNode in explain output 2017-01-18 11:58:31 +01:00
Andreas Streichardt 11bd9381d5 Add satellite collections 2016-12-06 16:40:50 +01:00
jsteemann fcacd76bca display index usage of traversals in AQL explainer output (previously missing) 2016-11-10 12:49:02 +01:00
Jan Steemann f0f90bc914 small fixes for traversal explain output 2016-11-08 16:08:58 +01:00
Michael Hackstein 4f50911c88 Fixed a bug in Traversal Explainer. Also fixed undefined behaviour when directly checking for an attribute omitting == true in early traversal filter. 2016-09-06 18:46:40 +02:00
jsteemann 5c9d015a28 fixed some explainer errors 2016-09-06 13:27:04 +02:00
Jan Steemann 23dc618311 fix a glitch in explainer output 2016-07-21 12:08:50 +02:00