1
0
Fork 0
Commit Graph

86 Commits

Author SHA1 Message Date
Jan 39a3f5bc4e
reintroduce smart joins after temporarily reverting them in devel (#8543) 2019-03-23 20:36:02 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)"
This reverts commit 59ad583796.
2019-03-21 19:18:46 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Dan Larkin-York 2eadab33e7 Index hints (#8431) 2019-03-19 09:14:18 +01:00
Michael Hackstein 0403d6cb6f
Bug fix/prune neighbors and explain (#8304)
* Fixed Neighbors prune on depth0

* Fixed the Explainer output

* Rmeoved debug output
2019-03-11 10:13:57 +01:00
Michael Hackstein 6e8d43b2da
Feature/aql traversal prune (#8255)
Allow PRUNE in the AQL Traverser.
2019-02-27 11:43:06 +01:00
Dan Larkin-York efb8364d99 Optimize memory usage in AQL by using constrained heap sort when reasonable (#7990) 2019-01-31 22:46:57 +01:00
Jan 6944115de6
various optimizations (#7994) 2019-01-21 13:40:39 +01:00
Andrey Abramov d4a010baf9
Bug fix/internal issue #316 (#7911)
* allow using scorers outside ArangoSearch view context

Signed-off-by: Andrey Abramov <andrey@arangodb.com>

* ensure query is properly optimized after replacement of scorer functions

* do not apply `handleViewsRule` to queries without views

* simplify optimization rule for ArangoSearch views

* show ArangoSearch view scorers in query explanation

* fix tests

* fix tests

* add stub for scorer related tests

* reformat

* check variable depth in `ViewExpressionContext::getVariableValue`

* add some tests

* address js test failures

* address jslint errors

* ensure `IResearchViewNode` exposes variables used in scorers

* ensure scorers with expressions are deduplicated

* fix deduplication for indexed access

* more tests

* partially address review comments

* address review comments

* simplify code

* remove irrelevant, commented out code

* ensure array comparisons are properly handled

* update changelog & loki
2019-01-10 21:04:19 +03:00
Jan fe81007438
support db._explain with all plans (#7895) 2019-01-08 18:06:46 +01:00
Dan Larkin-York 0bc6a4d7a2 Fix query profiling discrepancy in console. (#7241) 2018-11-06 14:50:32 +01:00
Jan 6d54518145
forward port changes from 3.4 (#6849) 2018-10-12 17:49:17 +02:00
Jan 46efcff7d7
micro improvements (#6674) 2018-10-05 10:25:13 +02:00
Jan dbe4559ed1
fix some issues with sorted variant of COLLECT (#6433) 2018-09-10 17:01:51 +02:00
Jan 3752c7deab
fix profile output for nodes without calls to getSome (#6397) 2018-09-10 09:19:01 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Andrey Abramov 615e1cf7de
Feature/add options to view (#6255)
* added OPTIONS for views

* pass options too

* validation for OPTIONS

* fix OPTIONS validation

* removed debug log message

* add forceSync options

* add forceSync view option

* fix comments

* fix some memleaks

* fix typo

* fix integration tests

* fix integration tests

* fix another integration test

* rename 'forceSync' option to 'waitForSync'

* another attempt to fix test

* ensure query caches are properly invalidated

* address integration test failures

* another attempt to fix test
2018-08-26 22:30:23 +03:00
Jan 0d48677619
Bug fix/aql optimizer improvements 2308 (#6241) 2018-08-24 13:42:22 +02:00
Simon fc204a2b3e fix profiling code (#6190) 2018-08-20 12:59:07 +02:00
Jan 99f176bed5 Feature/add search keyword (#6166)
* added SEARCH keyword

* fixes jslint errors

* add incompatibility notice

* remove VIEW keyword (does not work yet!!)

* add initial support for SEARCH keyword to optimizer rules

* replace FILTER with SEARCH keyword in tests

* removed VIEW keyword, make shell_server_aql tests pass again in single server

* cleanup

* handle SEARCH clause for views

* make SEARCH a non-keyword

* fixed an issue with duplicate variables

* fixed 3 tests

* fix SEARCH statement support for views

* minor refactoring

* fix integration tests

* optimize SEARCH conditions a bit more

* fix jslint error

* fixed wrong comments and typo in class name

* fix documentation

* adjust recovery tests to use SEARCH instead of FILTER

* fix another failing test
2018-08-18 20:22:58 +03:00
Jan d7e631eddf
Bug fix/more sparsung (#6177) 2018-08-17 08:47:45 +02:00
Jan 42fdef45d4
Bug fix/fix profile query (#6073) 2018-08-06 17:02:27 +02:00
Jan e3ce2e8cfb
fix anonymization with unique indexes (#6068) 2018-08-03 12:17:05 +02:00
Heiko 434cfce699 Feature/query profiling ui (#6027) 2018-08-03 10:58:22 +02:00
Jan Christoph Uhde bcf9e30a4d Feature/cluster single operations (#5585) 2018-07-12 20:18:36 +02:00
jsteemann c0517b69f2 nicify output 2018-07-03 16:53:27 +02:00
Wilfried Goesgens 2613ca56b9 add the primary indices to the AQL modification nodes serialization and their explain output (#5628) 2018-06-21 11:45:15 +02:00
Jan edb24deafe
cleanup collection/vocbase interface of ExecutionNodes (#5557) 2018-06-08 15:28:48 +02:00
Jan dc97eb0111
Subquery optimizations (#5398) 2018-05-28 13:56:03 +02:00
Jan a295eaa120
show AQL functions used in query in explain output (#5451) 2018-05-24 21:17:51 +02:00
Jan 4b1ef1dd3a
add support for named graphs (#5385) 2018-05-22 16:40:51 +02:00
Jan 8cd52ae95b
remove obsolete printBrowser function (#5259) 2018-05-04 11:30:39 +02:00
Jan 9c0460acff
Feature/aql optimizations (#5237) 2018-05-03 16:55:58 +02:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Jan 349e2c1681 forward-port AQL optimizations from 3.3 (#5020)
* forward-port AQL optimizations from 3.3

* fix restrict-to-single-shard rule

* added tests for specific cases

* ModificationNodes now can know if they are restricted or not

* OptimizerRule now updates the ModificationNode instead of setting ownName on remote node.

* The DBServerEngine container now only sends a subset of snippets (those which are relevant) It is  able to restrict any part of the query to a certain shard or even a list of shards

* ExecutionEngine CreateBlocks in ClusterMode now returns Result instead of throwing.

* Adapted Coordinator side to a bitter mapping of RemoteNode => QuerySnippets. This is still ongoing work.

* Fixed Coordinator enginecontainer test and finisehd implementation of EngineInfoContainerDBServer. Seems to work so far, need to adapt test to prove it.

* Adapted test for modification nodes and restrict to shard.

* restrictTo can now be de-/serialized on Modification Nodes

* Fixed Community Compile Error

* Added a test for Indexes/EnumerateCollection nodes

* EnumerateCollection and IndexNode now have options to be restricted to a single shard, opt rule not yet updated

* DBServer part of planning now checks Enumerate and IndexNodes for single shard optimization

* Added tests for simple single shard optimization on IndexNodes

* Adapted optimzierrule for SingleShard optimierung on IndexNodes

* Fixed an issue with externally restricted shard ids in combination with shard_id based optimization

* Added a test for multiple single shard optimizations on the same collection

* Removed collection/Shard duplicate check of optimize to single shard rules

* Updated Explainer to handle optimize to single shard

* Let more tests pass an optimized AQL variant

* Temporarily disabled a tests that can only be solved after some work in transaction
2018-04-30 11:25:09 +02:00
Simon 95c8710dec Fixing subqueries in aql profiler (#5195)
* Fixing subqueries in aql profiler

* Adding additional testsuite

* Fix AQL profiling example name, add API example

Also use different query for explain and profiling with SLEEP() for
non-zero runtimes.

* New/updated examples for query explain/profiling

* Fix runtime calculation
2018-04-25 13:35:55 +02:00
Simon 468231efc5 AQL Profiling code (#5165)
* initial start of profiling

* adding profiling code

* Fixing remote block tracing, fixing width and units

* Fixing some tests

* Various fixes

* adressing review comments
2018-04-24 16:17:30 +02:00
Michael Hackstein 0d2d61ca9c
Bug fix 3.3/anonymize in debug dump (#5190) (#5191)
* Fixed the amount of Xs generated by debug dump

* Fixed typo in debugDump information
2018-04-24 13:26:21 +02:00
Michael Hackstein a8e737f16f
Feature/debugdump ui (#5187)
* Adeded a route to aardvark to export a debug dump, it can optinally contain anonymized example data

* Added the Download Debug Button to UI

* fixed zip file download of query dump file

* removed not needed vars

* Added JWT token authentication and error display on debug dump button

* Fixed Error reporting on download debug dump.

* Anonymization does not anonymize system attributes any more.

* Anonymize in explainer debugDump will now anonymize bindVars and will not anonymize system document attributes.
2018-04-24 09:43:04 +02:00
Jan 2eb6d719ca
added ArangoShell helper function for packaging all information about an AQL query so it can be run and analyzed elsewhere: (#5023)
query = "FOR doc IN @@collection FILTER doc.value > @value RETURN doc";
    bind = { value: 42, "@collection": "mycollection" };
    options = { examples: 10, anonymize: true };
    require("@arangodb/aql/explainer").debugDump("/tmp/query-debug-info", query, bind, options);

Entitled users can send the generated file to the ArangoDB support to facilitate
reproduction and debugging.

The data from the generated file can be restored and analyzed via the *inspectDump*
function:

    require("@arangodb/aql/explainer").inspectDump("/tmp/query-debug-info");
2018-04-05 14:06:59 +02:00
Jan 167dfcdf7f
Feature/index only scans (#5005) 2018-04-03 17:34:28 +02:00
Jan 11cfbf5039
Feature/optimize cluster collect (#4897) 2018-04-03 17:28:07 +02:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan 9bebd33ba3 fix invalid first group results for sorted AQL COLLECT when LIMIT was used (#2577) 2017-06-16 09:44:14 +02:00
Michael Hackstein 2e417a1576 db._explain should now be backwards-compatible if we access a 3.1 instance with a 3.2 arangosh 2017-04-26 11:01:05 +02:00
Wilfried Goesgens 05a46e4781 fix db._explain() to work with the new interface of the graph traverser 2017-04-25 20:33:17 +02:00
jsteemann f5c8932744 initial version of query execution plan cache, still very rough 2017-02-21 11:58:38 +01:00