1
0
Fork 0
Commit Graph

4422 Commits

Author SHA1 Message Date
Heiko 442d59a3f8 added MultiPolygon GeoJSON constructor function (#7633) 2018-12-04 14:40:13 +01:00
jsteemann 4ab4d7d7ab improve debug messages 2018-12-03 16:42:15 +01:00
Andrey Abramov d6de637cb7
treat all iresearch scores as float_t for 3.4 (#7573) (#7578)
* treat all iresearch scores as float_t for 3.4

* cleanup

* attempt to fix tests

* another attempt to fix tests
2018-12-02 15:55:42 +03:00
Jan 8c8472be48
Bug fix/issue 7522 (#7556) 2018-11-29 19:25:49 +01:00
Simon 17a95710bd Micro-Optimize AQL CXX calls (#7486) (#7489)
(cherry picked from commit 8ddb9a063b)
2018-11-29 14:00:41 +01:00
Jan b2924057e7
cleanup (#7507) 2018-11-28 19:42:37 +01:00
Dan Larkin-York 6d542e11c8 Fix end condition (hasMore) for EnumerateViewNode. (#7279)
* Fix end condition (hasMore) for EnumerateViewNode.

* Fix crashes.

* Some more fixes.

* eliminate code duplication
2018-11-26 23:41:58 +03:00
jsteemann d5acd724f1 remove dead code 2018-11-26 09:58:24 +01:00
Michael Hackstein 16d0874da5
Bug fix/synchronous replication catchup (#7146)
* merged fixes from 3.4

* odd fix

* Bug fix 3.4/sync repl release thread (#6784)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Bug fix 3.4/shorter foot in door (#7084)

* Implement `syncCollectionCatchup` in DatabaseTailingSyncer.

First stab, might not even compile.

* Fixed a typo.

* Fix a typo and a compilation problem.

* Further compilation fix.

* Implement two stage catchup.

* Two small corrections.

* Unified error messages in Synchronize shard job.

* Improved a code comment.

* Fixed autocasting bool->double and double->bool issue. That is truely one of the best features ever invented... </irony>

* Renamed doHardLock => toSoftLockOnly and inverted default value

* Merged soft/hard foot logic with Transaction splits

* Use scopeguards to cancel readlocks

* Bug fix 3.4/sync replication allow soft and hard lock (#6864)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Implemented optional 'doHardLock' parameter in the replication acquire read-lock call. A hard-lock guarntees to stop all writes, a soft-lock may not.

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Renamed doHardLock => toSoftLockOnly and inverted default value
2018-11-23 16:16:34 +01:00
Simon d5cb94d2d0 Minor refactoring (#7408) 2018-11-22 16:16:05 +01:00
Jan c7869f1c46
Bug fix/remove shutdown assertion (#7388) 2018-11-22 15:35:55 +01:00
Jan 33f18942df
yet more micro optimizations (#7400) 2018-11-21 17:09:09 +01:00
Wilfried Goesgens 0a7c7446af Bug fix/less exceptions (#7385) 2018-11-21 12:00:14 +01:00
Jan bf2eeb16cd
Bug fix/aql speedup (#7379) 2018-11-20 16:07:40 +01:00
Wilfried Goesgens 05a7d4e96e add alternative to ClusterInfo::getCollection() that doesn't throw (#7339) 2018-11-20 16:05:57 +01:00
Jan Christoph Uhde 8703615acd Feature/disable indexing (#7362) 2018-11-20 15:00:55 +01:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
Jan f5cf42e4f2
Bug fix/refactor query optimizer part2 (#7337) 2018-11-15 18:19:07 +01:00
Jan b475f27148
fix queries that refer to COLLECT variables from inside COLLECT (#7332) 2018-11-15 15:11:11 +01:00
Jan bb09e357dc
Bug fix/aql micro optimizations (#7287) 2018-11-13 11:54:07 +01:00
jsteemann 02cf9013e3 fix define name 2018-11-12 09:49:22 +01:00
Simon 311fb717f2 Bug fix/refactor query optimizer (#7228) 2018-11-05 18:30:12 +01:00
Jan c7d76c5da4
fix some issues reported by codescan (#7195) 2018-11-02 19:04:39 +01:00
Jan 1973022d00
Bug fix/refactor find emplace (#7197) 2018-11-02 17:18:47 +01:00
Simon b883026f45 Server stream cursor (#7186) 2018-11-02 16:19:48 +01:00
Jan 7e78f85f61
add separate option `--query.slow-streaming-threshold` (#7181) 2018-10-31 21:31:52 +01:00
Jan 1e27f47b50
Truncate Fix (Nr 6) (#7176) (#7184) 2018-10-31 21:30:54 +01:00
Jan 4bc646f207
increase the maximum allowed number of collections in an AQL query (#7154) 2018-10-31 17:20:30 +01:00
Jan 7765a48bb6
speed up query string parsing (#7115) 2018-10-29 15:45:42 +01:00
Simon 10dc287eb3 Silence Tsan warnings (#7075) 2018-10-25 15:50:39 +02:00
Jan 412b5671a3
fix release-3.4 issue 109 (#7031) 2018-10-23 22:59:24 +02:00
Simon 4cfff35cef Use SSE42 for UTF-8 string validation (#6991) 2018-10-23 11:32:06 +02:00
Jan 87d5d29e5f
Feature/optimizations 211018 (#6994) 2018-10-23 09:57:24 +02:00
Tobias Gödderz 9d8858d75c Fixed optimize-traversals and added a regression test (#6974)
* Fixed optimize-traversals and added a regression test

* Updated CHANGELOG
2018-10-23 09:45:54 +02:00
Jan 158876a74f
Bug fix/fix array sorting (#6989) 2018-10-22 12:27:59 +02:00
jsteemann adc36c729f increase query setup timeout 2018-10-19 17:41:58 +02:00
Dan Larkin-York 9952cfbc95 Fix issues with ArangoSearch link permissions in cluster mode (#6925) 2018-10-16 20:42:11 +02:00
Jan 6d54518145
forward port changes from 3.4 (#6849) 2018-10-12 17:49:17 +02:00
Jan d3c446156c
slightly refactor the handling of optimizer rules (#6855) 2018-10-12 13:49:46 +02:00
Jan a0980ed2f2
Bug fix/fixes 111018 (#6843) 2018-10-12 12:51:11 +02:00
jsteemann cbd44e0e2e cppcheck 2018-10-12 02:25:27 +02:00
Jan 8f44d6b658
Bug fix 3.4/views in aql collections (#6714) (#6785) 2018-10-11 10:52:38 +02:00
Matthew Von-Maszewski e2bc7e10e3 port of 3.4 libcurl connection management to devel (#6775) 2018-10-10 09:10:34 +02:00
Jan 7b5d163aee fix variable replacements in view search conditions (#6756)
* fix variable replacements in view search conditions

* added reference to internal issue

* add catch test

* Added AQL test
2018-10-08 20:53:29 +03:00
Simon 877c9d9eb3 Do not load counters until recovey is done (#6719) 2018-10-05 12:52:00 +02:00
Jan 46efcff7d7
micro improvements (#6674) 2018-10-05 10:25:13 +02:00
Wilfried Goesgens c1b6426e92 Feature/drop before win7 support in compiler (#6681) 2018-10-03 18:33:52 +02:00
Jan f59761dcc2
small issues (#6675) 2018-10-02 17:47:22 +02:00
Jan c06f2d77da
Feature/velocypack update (#6678) 2018-10-02 14:04:14 +02:00
Jan fbf2904ab0
fixed issue #6601 (#6608) 2018-09-26 13:42:50 +02:00
jsteemann 2cf85fa239 added missing move 2018-09-25 10:13:10 +02:00
Jan ae85fbc4ea
much better diagnostics for AQL query results cache (#6580) 2018-09-25 09:49:11 +02:00
Simon b16af5ac71 Fix superfluous QueryRegistry::close, cleanup (#6579) 2018-09-24 13:10:07 +02:00
Simon 912f109968 Add simple Future library (#6464) 2018-09-21 16:14:17 +02:00
Jan a93db9a6b9 fix undefined behavior in query cache result handling (#6544) 2018-09-20 16:58:29 +02:00
jsteemann 04846b74f5 follow compiler's recommendation 2018-09-20 01:16:03 +02:00
Dan Larkin-York 282159ec2c Fix single shard restriction with binary and. (#6549) 2018-09-19 23:40:55 +02:00
Jan 203b141a6f
Bug fix/fix stats on skip (#6531) 2018-09-19 22:02:32 +02:00
Jan f8aae2811d
Feature/aql item block compression (#6514) 2018-09-18 14:33:33 +02:00
Simon 02b6c6ea90 Fix deduplication in IndexBlock (#6508) 2018-09-17 12:53:01 +02:00
Dan Larkin-York 52e0face29 Improve use of single shard restriction (#6182) 2018-09-14 14:31:49 +02:00
Dan Larkin-York 0dfabd8f04 Fix several TSan warnings (#6473) 2018-09-14 11:16:45 +02:00
Jan 6f10211f09
this moves aql::Query* out of the call parameter to any aql::Function (#6485) 2018-09-13 23:37:46 +02:00
Jan ac0a49fe64
fix leak in regex cache (#6480) 2018-09-12 23:26:42 +02:00
Simon 22b9c31c13 Removing ClusterComm ClientTransactionID (#6294) 2018-09-12 22:15:16 +02:00
jsteemann a3c9d6b5f8 remove unused variable 2018-09-12 21:34:19 +02:00
jsteemann eeb5f37f8e fix error message (backlog/#479) 2018-09-12 16:33:37 +02:00
Jan faac94a5c9
forward-ported from 3.4 branch feature-3.4/aql-extensions-080908 (#6462) 2018-09-12 13:58:20 +02:00
Simon 9fb3b31516 Fix skipSome PR (#6455)
* address review

* fix SortingGatherBlock
2018-09-11 10:58:28 +02:00
Jan d1241b1deb
Bug fix/aql modify cleanup (#6380) 2018-09-11 08:54:45 +02:00
Simon ec17a38fc8 Trace skipSome() calls (#6449) 2018-09-11 08:53:58 +02:00
Jan dbe4559ed1
fix some issues with sorted variant of COLLECT (#6433) 2018-09-10 17:01:51 +02:00
Jan 3752c7deab
fix profile output for nodes without calls to getSome (#6397) 2018-09-10 09:19:01 +02:00
Vasiliy d365c7cfd5 issue 465.4: remove unused functions (#6398) 2018-09-06 14:12:39 +03:00
Vasiliy 5329f34771 issue 465.2.2: remove redudnant heap allocations and simplify API (#6349)
* issue 465.2.2: remove redudnant heap allocations and simplify API

* address merge issue

* address more merge issues

* address more merge issues

* address review comments

* do not deallocate non-allocated instances
2018-09-05 13:37:37 +03:00
Jan 09bf296545
Bug fix/cache fullcount in query cache (#6364) 2018-09-04 16:33:13 +02:00
Jan be637cc1a2
Feature/relax aql limit grammar (#6363) 2018-09-04 16:02:33 +02:00
Simon 0661a4c1fe Hide Links from getIndexes() (#6325) 2018-09-03 15:17:24 +02:00
Dan Larkin-York 18d827688a Fix condition finders moving filters past modifications into index nodes (#6327)
* Fixed issue with condition finder pulling filter past modification node into index.

* Fixed issue with traversal condition finder pulling filter past modification node into index.

* fixed jslint
2018-09-01 13:34:23 +02:00
Jan 5873f63a72
Bug fix/fixes 2908 (#6279) 2018-08-31 17:26:54 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Jan 601e2ccbc6
Feature/aql 1808 (#6191) 2018-08-27 14:56:57 +02:00
Andrey Abramov 615e1cf7de
Feature/add options to view (#6255)
* added OPTIONS for views

* pass options too

* validation for OPTIONS

* fix OPTIONS validation

* removed debug log message

* add forceSync options

* add forceSync view option

* fix comments

* fix some memleaks

* fix typo

* fix integration tests

* fix integration tests

* fix another integration test

* rename 'forceSync' option to 'waitForSync'

* another attempt to fix test

* ensure query caches are properly invalidated

* address integration test failures

* another attempt to fix test
2018-08-26 22:30:23 +03:00
jsteemann c154dd95ce fix overrun 2018-08-25 11:28:22 +02:00
Simon 568a09f177 Disable JS on DBServer, fix race in UserManager (#6244) 2018-08-24 22:20:49 +02:00
Jan 73e44bc605
invalidate query cache after modification of views (#6198) 2018-08-24 19:54:10 +02:00
Jan 0d48677619
Bug fix/aql optimizer improvements 2308 (#6241) 2018-08-24 13:42:22 +02:00
Jan 2bce75eb86
add AQL function CONTAINS_ARRAY as alias for POSITION (#6242) 2018-08-24 12:50:50 +02:00
Jan 5f0403ed1c
Bug fix/add aql collection count cache (#6227) 2018-08-23 16:05:51 +02:00
jsteemann 9e1e6c20e2 preparations for some more index usage 2018-08-22 09:53:33 +02:00
jsteemann 9959218d1d revert unification of collection name validation 2018-08-21 18:46:13 +02:00
Jan d2c2f21c85
remove unnecessary parameter validation (#6210)
the validation has already been done before, so it does not need
to be repeated for each function call invocation
2018-08-21 13:51:28 +02:00
Jan 2aa879a53b
unify validation of collection names in AQL (#6203) 2018-08-21 11:11:15 +02:00
Simon 97b7e15792 Fixing crashes on UPDATE / INSERT queries (#6207) 2018-08-21 11:10:44 +02:00
Jan 68dac3cdfe
fix a crash when creating a resolver (#6206) 2018-08-20 18:22:48 +02:00
Simon fc204a2b3e fix profiling code (#6190) 2018-08-20 12:59:07 +02:00
Jan 7cb1da12cf
make AQL functions construction easier (#6195) 2018-08-20 11:56:38 +02:00
Jan 99f176bed5 Feature/add search keyword (#6166)
* added SEARCH keyword

* fixes jslint errors

* add incompatibility notice

* remove VIEW keyword (does not work yet!!)

* add initial support for SEARCH keyword to optimizer rules

* replace FILTER with SEARCH keyword in tests

* removed VIEW keyword, make shell_server_aql tests pass again in single server

* cleanup

* handle SEARCH clause for views

* make SEARCH a non-keyword

* fixed an issue with duplicate variables

* fixed 3 tests

* fix SEARCH statement support for views

* minor refactoring

* fix integration tests

* optimize SEARCH conditions a bit more

* fix jslint error

* fixed wrong comments and typo in class name

* fix documentation

* adjust recovery tests to use SEARCH instead of FILTER

* fix another failing test
2018-08-18 20:22:58 +03:00
Jan d7e631eddf
Bug fix/more sparsung (#6177) 2018-08-17 08:47:45 +02:00
Jan b5eaf1443b
make optimizer rule "patch-update-statements" fire for REPLACE too (#6181) 2018-08-17 08:45:40 +02:00
jsteemann 478dc56104 improve estimate for CollectNode in case it does an aggregation to a single row 2018-08-16 17:37:51 +02:00
Jan 63efe2a634
fixed issue #6128 (#6137) 2018-08-16 11:01:44 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
jsteemann b09638a883 fixthedocs.com 2018-08-14 12:02:53 +02:00
jsteemann 9141855b16 some adjustments to the docs for `REGEX_MATCHES` 2018-08-14 11:04:48 +02:00
jsteemann d634e0166b Merge branch 'feature/add-aql-function-regexmatches' of https://github.com/arangodb/arangodb into devel 2018-08-14 10:18:53 +02:00
Frank Celler a688dc0962
Feature/remove job queue thread (#5986)
limiting V8 calls in flight
2018-08-10 12:17:43 +02:00
Jan 8267c61ec2
Merge branch 'devel' into feature/add-aql-function-regexmatches 2018-08-09 20:28:06 +02:00
Jan 95d1892f18
remove canThrow attribute from AQL functions (#6123)
Its was somehow overlapping with the isDeterministic attribute (but with the opposite meaning)
2018-08-09 16:45:54 +02:00
Jan bf3ce0b704
fixed issue #6106: Arithmetic operator type casting documentation inc… (#6113) 2018-08-09 11:30:54 +02:00
Jan 9abf1b1562
make tests non-timecritical (#6129) 2018-08-09 11:28:29 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
jsteemann 1573f83693 small fixes for RegexMatches 2018-08-08 18:12:45 +02:00
jsteemann c1006b9c13 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-aql-function-regexmatches 2018-08-08 17:55:41 +02:00
Shivam Dave b8bf25b2d2 Regex Matches implementation- having trouble finding matches in an input string using the ICU regex matcher 2018-08-07 09:35:02 -07:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Simon a79c8af310 fix non-blocking stream cursors (#6064) 2018-08-03 12:41:52 +02:00
Jan Christoph Uhde a60e04b251 fix timeout issue next try: no subquery nodes on dbservers (#6037) 2018-08-03 09:43:11 +02:00
jsteemann 44d7a11bd9 Merge branch 'bug-fix/improve-intermediate-commits' of https://github.com/arangodb/arangodb into devel 2018-08-01 17:49:17 +02:00
Jan 29ab2fdbe3
Bug fix/3107 (#6046) 2018-08-01 12:50:55 +02:00
shivamdave24 d4c6431ee5 Regex Split Implementation (unfinished) (#5991) 2018-08-01 12:09:36 +02:00
Jan 7f01b7f728
fixed issue #6031: Broken LIMIT in nested list iterations (#6033) 2018-08-01 08:32:00 +02:00
Simon Grätzer d64649a513 adding hint to other constructor 2018-07-31 16:17:23 +02:00
Simon Grätzer a7c78a3f37 Merge branch 'devel' of github.com:arangodb/arangodb into bug-fix/improve-intermediate-commits
# Conflicts:
#	arangod/RocksDBEngine/RocksDBIncrementalSync.cpp
#	arangod/RocksDBEngine/RocksDBIterators.cpp
2018-07-31 15:19:57 +02:00
jsteemann 22088520b5 fix crashes in catch tests 2018-07-31 11:38:59 +02:00
Jan 2a416f2e33
Bug fix/3007 (#6019) 2018-07-30 17:16:47 +02:00
Simon Grätzer 1cfb328bcc fix intermediate commits 2018-07-27 17:40:38 +02:00
Shivam Dave eea9fce609 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-aql-function-regexsplit 2018-07-26 09:36:53 -07:00
Jan e4d7f1c5f0
Bug fix/wenn der shard mann 2mal klingelt (#5890) 2018-07-26 15:37:40 +02:00
jsteemann 5369afe5fc fix compilation 2018-07-26 13:39:39 +02:00
jsteemann 2bfec8ecee use unique_ptrs for QueryRegistry 2018-07-26 11:54:38 +02:00
jsteemann 91756ef9dd remove unnecessary dynamic cast 2018-07-26 11:53:52 +02:00
jsteemann 01afabd93e test for nullptr 2018-07-26 11:53:37 +02:00
Jan Christoph Uhde 6210b9be34 bug-fix/double-modification-bug (#5979) 2018-07-26 10:35:42 +02:00
Jan 562372ee8c
fix single-shard optimization checks (only affects devel) (#5985) 2018-07-26 10:32:10 +02:00
jsteemann 2256bad159 fix REGEX_SPLIT 2018-07-26 01:20:11 +02:00
jsteemann 66f174fbe3 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-aql-function-regexsplit 2018-07-26 01:20:06 +02:00
Shivam Dave 917c069c19 Regex Split implementation (unfinished) 2018-07-25 12:00:42 -07:00
Jan 213c42d2f6
fixed issue #5977: Unexpected execution plan when subquery contains COLLECT (#5984) 2018-07-25 17:33:55 +02:00
shivamdave24 7db28da251 Levenshtein Distance function implementation with documentation and i… (#5922) 2018-07-25 12:48:58 +02:00
Tobias Gödderz aa31cc3f1e Bugfix: Fix `_rev` handling in `UPDATE`/`REPLACE` `WITH` clauses (#5966) 2018-07-25 12:13:09 +02:00
Jan Christoph Uhde 4c0f881764 compatibilty 3.3 <-> 3.4 (#5963) 2018-07-25 09:05:34 +02:00
Jan 21064144c8
Bug fix/replication improvements (#5962) 2018-07-25 09:04:50 +02:00
Jan a5bb50b0bf
remove methods from VelocyPackHelper that are also in VPackSlice (#5946) 2018-07-25 09:01:29 +02:00
jsteemann 16e6a04176 fixed out of bounds access 2018-07-20 21:00:26 +02:00
jsteemann a6af188757 use nullptr instead of NULL 2018-07-20 19:50:07 +02:00
jsteemann 425050a8e7 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-aql-function-levenshteindistance 2018-07-19 10:54:30 +02:00
Shivam Dave 644adc47f1 Levenshtein Distance function implementation with documentation and integration tests 2018-07-18 13:24:14 -07:00
jsteemann cdd7967bad fix "hasMore" route 2018-07-17 21:46:44 +02:00
jsteemann 5e0011a9af add specific messages for queries issued by 3.3 2018-07-17 20:31:35 +02:00
jsteemann 6a35c1b29f added assertions 2018-07-17 20:31:21 +02:00
jsteemann 609ca145a4 ensure rest handler compatibility with 3.3 2018-07-17 20:30:56 +02:00
Tobias Gödderz 24dccfee90 Streaming cursor API stops lying about hasMore (#5867)
* Make StreamCursor prefetch at least one row, so it can answer hasMore with surety.

* Removed #pragmas

* Removed unused parameter
2018-07-17 12:56:29 +02:00
jsteemann 44c7b1b476 remove tabstops 2018-07-16 15:00:12 +02:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
Jan 7fb985d573
Bug fix/fix codescan issues (#5872) 2018-07-16 10:58:14 +02:00
Jan 2c83454066
Bug fix/fixes 1307 (#5878) 2018-07-13 20:48:21 +02:00
Jan e8e7358a79
use ExecutionNode::castTo (#5875) 2018-07-13 20:09:16 +02:00
jsteemann db3ca147ec forward-port extra condition for single-shard optimization from 3.3 2018-07-13 19:23:59 +02:00
shivamdave24 c45a1b6ac1 Soundex function implementation with integration tests and minor bug fixes to feature/add-aql-tobase64-tohex-encodeuricomponent-uuid branch (#5851) 2018-07-13 17:47:25 +02:00
Jan 8a659cd16f Bug fix/fix bison error message leak (#5864)
* define YYSTACK_USE_ALLOCA for Bison

    we are now using alloca here explicitly because we may
    otherwise leak error messages that are generated by Bison.
    Bison reports all its errors via the function `Aqlerror`, which
    will receive the error message as a constant string. So we
    must not free the string inside `Aqlerror`, and we cannot even
    tell if the error message is a dynamically allocated error
    message or a hard-coded error message that resides in some
    static part of the program.
    Even worse, `Aqlerror` does not return control to Bison but throws
    an exception... So the best thing we can do here is to not use
    dynamically memory allocation by Bison, but make it use alloca.

* added generated files
2018-07-13 14:06:52 +02:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
jsteemann 2d9e5288ab try to fix windows compile warnings and errors 2018-07-13 11:48:05 +02:00
Jan 606ea478c2
fix AQL leak (#5862) 2018-07-13 11:28:44 +02:00
jsteemann 2d795c1f8a fix some undefined behavior 2018-07-12 21:54:51 +02:00
jsteemann 3918c66399 fix annoying compiler warning 2018-07-12 20:27:59 +02:00
Jan 0905a7eab6
fix undefined behavior in some AstNodes (#5858) 2018-07-12 20:23:03 +02:00
Jan Christoph Uhde bcf9e30a4d Feature/cluster single operations (#5585) 2018-07-12 20:18:36 +02:00
Vasiliy 57d2517bd6 issue 399.1: store only transient link information by IResearchViewCoordinator (#5834)
* issue 399.1: store only transient link information by IResearchViewCoordinator

* some fixes
2018-07-11 21:21:51 +03:00
Wilfried Goesgens 67e8089cd4 don't use rumpelstilskin-lookup to map block strings to block types (#5819) 2018-07-10 16:09:20 +02:00
Jan 2cac8b8a51
honor some cppcheck recommendations (#5817) 2018-07-10 13:50:30 +02:00
jsteemann 2aa17551f3 fix non-deterministic "excludesNull" flags 2018-07-10 13:38:07 +02:00
Jan 9ab26ba972
Bug fix/fix hangs (#5808) 2018-07-10 11:39:37 +02:00
Jan 1525f49d50
Bug fix/execution block cleanup (#5814) 2018-07-10 11:36:57 +02:00
Jan Christoph Uhde 1e3e3623da Bug fix/always distribute restrict to single shard later (#5791) 2018-07-10 11:11:26 +02:00
jsteemann 20bdf0c55c fix compile warning 2018-07-10 11:09:20 +02:00
Tobias Gödderz 1800a26297 Remove hasMore (#5810)
* Fixed hasMoreState remote communication; cleanup in RemoteBlock::getSome and ::skipSome

* Removed /_api/aql/hasMore and all ExecutionBlock::hasMoreState() methods

* Removed now useless method RestAqlHandler::getInfoQuery
2018-07-10 09:50:08 +02:00
Jan ddd6b73d85
Bug fix/fixes 0907 (#5812) 2018-07-10 08:48:14 +02:00
Tobias Gödderz fc3e11dbbc Async AQL (#5806)
* Modified header to new initializeCursor API

* Adapted initializeCursor to DONE/WAITING API. Compiles but not tested and no one reacts to WAITING state, it is not returned anywhere yet

* Subqueries now expect a WAITING return from initilize cursor. However they will just return a nullptr and pretend the query is empty, this will be fixed later

* First attempt to simulate thread waiting over information within the query

* Small fix to allow for isDirect handlers to go to sleep.

* Waiting in the necessary places now for the async request to be send.

* Thank you auto-casting compiler, you are totally right i absolutely wanted to use this bool value as an index in may Array. How could i possibly not want to use it here?

* Include cond-var header

* Fixed mutex/cond_var usage

* Added oldAPI wrappers in AQL Blocks for get/skip some variants. This Commit compiles but is NOT tested

* Let getSome now return unique_ptr of AqlItemsBlocks. Also implemented the async variant of getSome in subqueries.

* Removed all references to OLD implementations in AQL. only the base wrappers are allowed to call OLD functions from now on. Now the testing part starts

* Fixed endless virtual recursion

* Implemented new getOrSkip API in SortBlock

* Implemented new getOrSkip API in LimitBlock

* Initilaize all variables

* Fixed logic bug in SubqueryBlock

* getBlock in ExecutionBlock now returns a state. All blocks need to handle this properly!

* Createad a wrapper getBlockOld that servers the old sync api and is used now in AQL. To be replaced overtime.

* Added IndexBlock::skipSome and IndexBlock::getSome

* getBlock now returns its old return value along with the state

* Switch from getBlockOld to getBlock in IndexBlock::skipSome

* Switch from getBlockOld to getBlock in IndexBlock::getSome

* ShortestPathBlock::skipSome is not implemented! Added a regression test

* Attempt to fix SubQueryResult memory management

* Fixed LIMIT Block

* Moved from ShortestPathBlock::getSomeOld to ::getSome

* Implemented ASYNC api on SingletonBlock

* Adapted EnumerateCollectionBlock to new async API

* Fixed FilterBlock and adapted return block to async API

* Adapted NORESULTS block to async AQL api.

* Adapted Modification Blocks to async API

* Fixed some initialize cursor functions to reset values required during get/skipSome

* First steps to adapt ClusterNodes to Async AQL api. Not there yet, need to implement the core still

* Added asnyc implementation for xxxForShard in ClusterBlocks. This commit changes internal logic of _doneForShard. Needs additional testing as soon as everything is in place.

* Adapted CalculationBlock to async API

* Adapted TraversalBlocks to ASYNC Aql. This is not optimal yet, we need a better decission if we are DONE or not on RETURN

* Adapted EnumerateListBlock to Async AQL api

* Adapted RemoteBlock to ASYNC API in getSome/skipSome. The whole thing is now LIVE in the cluster. Exetensive testing to be started now

* Fixed IndexBlock WAITING behaviour if Waiting occurs during a index processing

* Adapted IReasearchViewBlock to ASYNC AQL API

* Fixed SortingGatherBlock in WAITING state.

* Adapted IResearch ExecutionBlockMock to Async API

* Unified the HASMORE/DONE distinction. Code is much more readable now and harder to get incorrect 👍

* Implemented tonly heoretically reachable function of non void function.

* Fixed last commit

* Added inline TODO comments

* fix warning

* Fixed a clearing logic bug in RemoveNodes

* Fixed Error Handling in RemoteBlocks. Also fixed a logic bug (true/false simply has a 50% chance of getting it wrong) in Distribute and Scatter.

* remove unused methods

* Fixed failure test

* implement skipping

* Moved the Query Waiting out of the ExecutionEngine.

* changed one of the collect blocks

* Removed _upstreamState from ExecutionBlockMock, that is in the base-class now

* Added a Test Mock for a an ExecutionBlock that simulates the WAITING/HASMORE/DONE api.

* do not check "hasMore" if not necessary

* Added DistinctCollectBlock::getOrSkipSome from ~Old and changed its return type

(still uses getBlockOld)

* Save state to resume in DistinctCollectBlock::getOrSkipSome

* Extracted redundant code

* fixed some ops

* added one more test

* fix endless blocking

* fix compile error

* fix test

* Refactored HashedCollectBlock::getOrSkipSome

* Return blocks to the manager

* Replaced usage of getBlockOld in HashedCollectBlock::getOrSkipSome

* remove unused shutdown calls, simplify ownership for expressions

* Removed superfluous variable

* Capture const variable by value

* Removed SortedCollectBlock::getOrSkipSomeOld in favour of getOrSkipSome

* Added a working version of SortedCollectBlock::getOrSkipSome

Has yet to be cleaned up

* Removed isTotalAggregation special treatment

* On no input, return a group of nulls (instead of no group at all)

* Bugfixes

* Simplified code

* Move return to the end, eliminate duplicate code

* Corrected skipped count in HashedCollectBlock

* Aligned getNextRow() implementations

* Added comments

* some cleanup

* fix potential memleak

* Bugfix

* Fixed failure tests

* Removed usage of getBlockOld in ExecutionBlock::getOrSkipSome

* Replaced hasMore with an async implementation (mostly)

* Removed getBlockOld()

* Added hasMoreState to the AQL API (and renamed hasMore methods to hasMoreState)

* RemoteBlock now uses the async hasMoreState route

* remove job queue

* options

* Bugfixes in the async implementation of LimitBlock

* LimitBlock::getOrSkipSome now always skips when calculating the fullcount

* fix compile warnings

* restrict threads

* Fixed api of Waiting ExecBlockMock. Unused yet

* Made SortedGatherBlock async-capable

* Removed nonEmptyIndex hack

* Removed duplicate traceGetSome~ calls, moved all to getSome

* Added asserts before replacing getNr*Registers

* Added a TODO note and a comment

* Removed getSomeWithoutRegisterClearoutOld()

* Removed skip()

* Removed common code by using getNr*Registers()

* Use getNr*Registers() in the TraversalBlock as well

* started to add lane

* started to add lane

* added lane

* completed lane

* removed debug output

* fixed merge

* Began working on a test suite for AQL tracing/profiling

* Added more tests and asserts in aql-profiler

* Made some ExecutionBlocks final

* Added a type enum to all blocks and the per-block stats

* Add block type to stats nodes when tracing AQL on block level

* Removed initializeCursor call from instantiateFromPlan

* Avoided additional getSome calls after DONE

* Added more profiler tests

* Refactored ExecutionBlock::getOrSkipSome and fixed two bugs

- set _upstreamState also when skipping
- explicitly use xecutionBlock::getHasMoreState()

* Bugfix: update state

* Reuse parent _skipped wherever possible; rename where not (LimitBlock)

* Simplified SortedCollectBlock::getOrSkipSome and reused general pattern & code

* Implemented missing virtual function (with USE_FAILURE)

* Reset neccessary values during initializeCursor

* Simplified code in EnumerateListBlock a little

* Added a test for DistinctCollectBlock in aql-profiler

* Avoid redundant getSome calls in DistinctCollectBlock

* fix compilation

* Fixed DistinctCollectBlock profiler test

* Added a second profiler test for the DistinctCollectBlock

* Added a profiler test for EnumerateCollectionBlock

* Bugfix in EnumerateListBlock

* added --server.fifoN-size

* Simplified EnumerateCollectionBlock::getSome

* Simplified EnumerateCollectionBlock::getSome, and return HASMORE less often when DONE

* Fix testEnumerateCollectionBlock1 for mmfiles

* do not pass by reference

* Fixed compile error

* fixed merge conflicts

* Added profiler tests for EnumerateCollectionBlock

* Test fix for mmfiles

* Fixed IResearch tests

* Bugfix in DistinctCollectBlock and a regression test

* Updated comment

* Bugfix for query statistics in cluster

* Check plan in distinct test

* Fix aql-profiler tests in cluster

* Removed unused line / bugfix for single server test runs

* This commit implements waking up of AQL queries. (#5651)

* Non-compiling intermediate commit for handover.

* Make branch compile again

* Started implementation of continueable rest cursor handler by moving the callbacks to the outer part. This is not yet fully tested!

* Made finalizeExecute noexcept. We cannot react to this errors as the response was potentially written before. Also introduced continueExecution in the RestHandler engine.

* First successful query wakeup.

* The wakeup callback now posts on the scheduler directly. A resthandler only needs to provide a callback that encapsulates the continueExecution call on this handler

* renamed finalizeExecute to shutdownExecute

* Added a differentiation between Handler and Callback in Query continuation. Handler will be posted in IO service. Callback will be executed directly

* fix audit log

* Removed callback from deleteQueryCursor. This cannot be waiting

* use CONDITION_LOCKER

* removed yet another thread-local variable

* Fixed forward declaration

* Made RestAqlHandler repeatable

* Use defer to close the query in RestAqlHandler. Now waiting will close the query as well.

* Added a mutex in the RestHandlers to make sure if the callback over network is too fast that there is only one Thread running in the RestHandler

* Captured the GeneralCommTask if it is posted to a RestHandler. This is necessary in the PAUSED case

* Refactoring of _noLockHeader responsibilities. Now the BaseHandler selects them and resets them after it is done. Only Coordinators are allowed to define them if a query is loaded.

* Removed reaction to existing nolockheaders in Coordinator Query Planning Phase

* Removed incorrect assertion.

* Further refactoring of NoLockHeaders. Now there is a wrapper class around it which allows for debugging and logging. The state now seems to be better. Also all non-rest-handler triggered queries clean up the NoLockHeaders properly.

* Fixed UserManager, now deletes nolock headers properly

* Swing to the Symphony of Destruction

* Forgot about community build...

* Fixed compiling of Catch tests

* Fixed community build

* need thread for size

* Made the restSimpleHndler repeatable

* Implemented dump and dumpSync in Cursors, Sync will block a thread, dump allows to wait, only relevant for Streaming cursor

* Reactivated StreamingCursors

* Removed debug output.

* Fixed false query continuation

* Reset thread output to non-debug

* Added missing return statements

* Allow some CollectionMethods to hand-in a context that may contain a transaction. This is meant to honor nolock headers.

* Fixed hidden merge conflict

* Bugfix in aql-profiler.js: use plan.nodes order, not stats

* Added two profiler tests for filter

* Avoid too many getBlock calls in the FilterBlock

* Removed debug output

* RemoteBlock API will now send a done(bool) flag whenever we request documents from remote Servers. It is possible that we are DONE and have a result. The pre 3.4.0 API uses exhausted which is exclusive to a result. This API is still implemented for beckwards compatibility.

* Implemented an executeSync function in AqlQuery. This will block the thread until query execution is complete

* Added another test for FILTER, and one test for the HashedCollectBlock

* Added more tests for HashedCollectBlock; avoid unneccessary getSome calls

* Added an profiler IndexBlock test

* IndexBlock: avoid redundant getSome calls, added missing traceGetSomeEnd calls

* Added a second test profiling IndexBlock

* Added a third test for IndexBlock

* Moved general code to module

* Moved noncluster tests into a separate file

* Split aql-profiler testsuite into three files

* Added profiler tests for LimitBlock

* Added a test for NoResultsBlock

* Added profiler tests for TraversalBlock

* Shutdown of an AQL query is now asynchronous. However in Error-Cases it will be executed in a blocking way still

* Optimized TraversalBlock getSome calls due to new (nightly) test results

* Fixed std::min calls I broke

* Let shutdown calls in AQL wait, if the query is executed successfully.

* Fixed queryResult going out of scope

* fix compile error through merge conflict with devel

* Fixed compiler warning "mismatching tags"

* Removed debug log output

* Added TODO notes

* Fixed test fail due to devel merge

* Fixed some invalid sync waiting implementations

* Added a profiler test for SortBlock

* Added profiler tests for SortedCollectBlock

* Fixed bug introduced by devel merge

* Fixed Remoteblocks ignoring errors!

* Added some more continue Callbacks in used places. And removed debug log

* Removed debug log output

* Suppress clang warnings

* Bugfix: use of invalid stack pointer

* Bugfix: RemoteBlock::shutdown now sends code as int, not string

* Revert "Suppress clang warnings"

This reverts commit 05591649c59743c992edd5e78814edc8ca2a83e0.

* Bugfix: cleanup state in RemoteBlock ::shutdown, ::getSome and ::skipSome

* Bugfix in Subquery shutdown: don't skip subquery shutdown when main query shutdown failed

* Allow copy elision
2018-07-09 14:24:10 +02:00
Jan 1c6ded6724
use sparse indexes in more cases (#5790) 2018-07-09 09:01:10 +02:00
Simon c84fbcfb24 Fix enterprise auth tests (#5721) 2018-07-05 18:03:54 +02:00
Simon f47367a495 Allow dumping / restoring of views (#5774)
* allow dumping and restoring of views

* fix restore, adding some tests

* remove debug print

* fix global replication
2018-07-05 17:55:28 +02:00
Jan f9dc610c1e
Bug fix/internal issue 2658 (#5760) (#5778) 2018-07-05 10:19:28 +02:00
Jan 222f77249f
for some reasons fortify complained about these parts (#5777) 2018-07-05 09:38:03 +02:00
jsteemann 208914e3d7 fixed test failures 2018-07-04 23:55:55 +02:00
jsteemann fe9b2d5bb5 honor cppcheck comments 2018-07-04 22:25:33 +02:00
Simon b8e140493a Remove redundant geo code, remove WITHIN_RECTANGLE (#5757) 2018-07-04 20:21:00 +02:00
Jan b88fde9659
added key generators "uuid" and "padded" (#5740) 2018-07-03 15:23:47 +02:00
Jan b7cc85349c
slightly simplify syncing API (#5726) 2018-06-29 14:03:47 +02:00
jsteemann b1002075b3 Merge branch 'feature/add-distinct-aggregator' of https://github.com/arangodb/arangodb into devel 2018-06-28 19:03:16 +02:00
Simon 4387fabacf Converting Pregel AQL function to c++ and fixing a bug (#5620) 2018-06-28 10:46:16 +02:00
Jan 47db35af98
fix potential duplicate setting of memoryLimit (#5680) 2018-06-27 11:33:57 +02:00
Simon 5eea1d4199 adding ttl option for the query registry (#5677) 2018-06-27 09:05:43 +02:00
Kaveh Vahedipour 137085c902 Feature/make the boolean true again (#5646) 2018-06-27 08:49:51 +02:00
jsteemann c755c7927b finalize distinct aggregator code 2018-06-26 22:46:36 +02:00
jsteemann 2610af55f8 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-distinct-aggregator 2018-06-26 20:16:46 +02:00
Simon 8c48fdf4ab Replace rocksdb export cursor with query (#5657) 2018-06-25 18:14:27 +02:00
Andrey Abramov 59ea4970d4
fix broken tests (#5662) 2018-06-25 04:08:38 +03:00
Simon 52f44596ba Adding MultiPolygon support, fixing some outstanding bugs (#5613)
* Adding MultiPolygon support, fixing some outstanding bugs

* removing dead code

* Adding MultiPolygon tests

* Added multi-polygon docs

* adjust the docs

* fixing remaining occurrences of geo1 / geo2, added note into UpgradingChanges34.md

* some clarification

* Improve Geo docs, reformatting

* Release notes: responses are JSON, reformatting

* Update release notes: /_api/aqlfunction has an isNewlyCreated attribute now

* Update Geo.md
2018-06-22 10:03:27 +02:00
Andrey Abramov 5eef6cd618
Feature/test iresearch (#5610)
* start implementing arangosearch cluster tests.

* backport: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties

* first attempt to fix failing tests

* refactor cluster wide view creation logic

* if view is not found in the new plan then check the old plan too

* ensure the cluster-wide view is looked up in vocbase as well on startup/recovery

* do not store cluster-wide IResearchView in vocbase

* move stale view cleanup to the shared pointer deleter, address test failures

* do not print warning

* enable arangosearch tests by default

* fix catch tests

* address icorrect return value for cluster-wide links

* address some issues with test failures due to cluster-view allocated within TRI_vocbase_t

* simplify per-cid view name, address 'catch' test failures

* ensure IResearchViewNode volatility is properly calculated in cluster

* invoke callbacks directly in AgencyMock instead of waiting for timeout

* ensure view updates via JavaScript always use the latest view definition

* pass a list of shards to `IResearchViewDBServer::snapshot`

* extend cluster aql tests

* fixes after merge

* fix class/struct inconsistencies

* comment failing tests

* remove debug logging

* add debug function

* tests cleanup

* simplify upcoming merge: pass resolver from a side

* backport: move all transaction status callback logic to Methods

* add changes missed from previous commit

* fix js and ruby tests

* more tests for IResearchViewNode

* pass transaction to IResearchViewDBServer::snapshot, address IResearchViewDBServer tests segfault

* pass transaction to IResearchView::snapshot instead of transaction state

* temporarily add trace log output to tests to try to find the cause of the core dump on Jenkins

* add more temporary debug output to trace down the segfault on Jenkins

* add even more temporary debug output to trace down the segfault on Jenkins

* ensure Vieew related maps are cleared during shutdown

* reset ClusterInfo::instance() before DatabaseFeature::unprepare()

* remove extraneous debug output

* missed line from previous commit

* uncomment required line

* add nullptr checks to RocksDBIndexFactory::prepareIndexes(...) similar to the ones in MMFilesIndexFactory::prepareIndexes(...)

* attempt to fix deadlock in tests

* add comment as per reviewer request

* fix aql test suite name

* add some debug logging

* address deadlock between ClusterInfo::loadPlan() and CollectionNameResolver::localNameLookup(...)

* eplicitly state which index definition failed in the log message

* use vocbase from shard-view isntead just in case

* explicitly state which index definition failed in the log message

* do not create shard-view instances from cluster-link instances (only register existing ones)

* add some tests
2018-06-21 20:35:04 +03:00
Wilfried Goesgens 2613ca56b9 add the primary indices to the AQL modification nodes serialization and their explain output (#5628) 2018-06-21 11:45:15 +02:00
jsteemann 8af5e06223 refactoring of aggregators 2018-06-20 02:33:20 +02:00
jsteemann 69dec8ba17 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-distinct-aggregator 2018-06-19 21:53:19 +02:00
Wilfried Goesgens c4c711f372 fix windows compile (#5611) 2018-06-14 19:18:54 +02:00
Jan 5df12917d6
prevent an optimization from destroying variables (#5604) 2018-06-14 18:09:10 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
jsteemann 2712f9ce14 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/add-distinct-aggregator 2018-06-14 00:46:22 +02:00
Jan Christoph Uhde f81e42af36 Add optimizer rule that replaces js function calls (NEAR/WITHIN/FULLTEXT) with pure AQL (#5529) 2018-06-12 13:48:31 +02:00
Jan 448a435713
clean up key generators a bit (#5573) 2018-06-12 11:28:38 +02:00
Jan 983078ff1e
disable a test that leaks memory (#5571) 2018-06-11 13:26:25 +02:00
jsteemann 396d98a1cc make dtor always virtual
this is required because Query has virtual functions
2018-06-11 10:39:12 +02:00
Wilfried Goesgens 0be84ee138 make msvc happy (#5568) 2018-06-08 17:22:22 +02:00
Andrey Abramov 153d25d3ab
fix ms compilation (#5566) 2018-06-08 17:40:51 +03:00
Jan af9d2c05bc port COLLECT bugfix from 3.3 (#5519) 2018-06-08 15:45:41 +02:00
Jan edb24deafe
cleanup collection/vocbase interface of ExecutionNodes (#5557) 2018-06-08 15:28:48 +02:00
Max Neunhöffer 7bed3f2df0 Try openssl 1.1 (#5123) 2018-06-08 13:21:31 +02:00
jsteemann e92ac9264b fix compile warning 2018-06-07 13:40:29 +02:00
Vasiliy 6fae53fb1b issue 380.1: reuse resolver used during injectBindParameters(...) in the query transaction (#5548) 2018-06-07 00:23:05 +03:00
Andrey Abramov e117a97fb4
Feature/arangosearch scatter node (#5536)
* remove collection dependency from scatter

* simplify `createBlock` API

* remove useless IResearchViewScatter node

* some fixes

* code cleanup

* scatter arangosearch view

* cleanup

* fix compilation errors

* fix test issues

* store shared_ptr rather than raw pointer in IResearchViewNode

* refactor ScatterBlock

* refactor DistributeBlock

* ensure no leaks during execution node cloning

* some fixes

* code cleanup

* remove vocbase from `DistributeNode`

* add missing ifdef

* fixes after review
2018-06-06 20:42:53 +03:00
Vasiliy 3e983adb2d issue 381.6: enforce use of VIEW keyword for bound views and disallow for bound collections (#5522) 2018-06-05 14:45:53 +03:00
Andrey Abramov 1a83809b84
split gather block (#5506)
* initial split into sorted and unsorted blocks

* some fixes

* move gather blocks out of header

* cleanup

* cleanup

* ensure ExecutionBlock constructor is noexcept

* extract sorting logic into separate SortingStrategy class

* ensure we not rebuild heap each getSome/skipSome call

* cleanup API attempt

* Revert "cleanup API attempt"

This reverts commit c824de1a1d29e2cb7f405138f830cc0e02e7667b.

* Revert "Revert "cleanup API attempt""

This reverts commit 65a2c95f6fdbbcbd00d6582af4d354fe339d4738.

* fix comments

* fixes after review

* more fixes after review
2018-06-01 22:26:51 +03:00
Jan 0dc9896718
more eager "remove-filter-covered-by-index" rule (#5498) 2018-06-01 14:40:24 +02:00
Andrey Abramov c41b056209
add missing ifdefs (#5499) 2018-05-30 17:22:27 +03:00
Andrey Abramov f708e46973
Feature/arangosearch gather node (#5487)
* integrate ArangoSearch into GatherNode/GatherBlock logic

* fix inconsistency after merge

* add serialization/deserealization logic for IResearchViewNode sorting

* fix optimizer rule order

* fix copy/past typo

* fix tests

* ensure `GatherNode` is properly set up for ArangoSearch views

* adjust optimizer rule order
2018-05-30 02:53:21 +03:00
jsteemann f5d7053bb1 fix call/apply 2018-05-29 22:07:37 +02:00
Jan e6dcff8360
Bug fix/cleanup 2805 (#5477) 2018-05-29 10:16:48 +02:00
Jan dc97eb0111
Subquery optimizations (#5398) 2018-05-28 13:56:03 +02:00
jsteemann e5ee8e0573 added new aggregators 2018-05-28 08:48:01 +02:00
jsteemann 56e03af4c4 pass "isModificationQuery" around in cluster 2018-05-27 22:41:08 +02:00
Jan cee529727c
Squashed commit of the following: (#4092) 2018-05-26 10:56:26 +02:00
Jan edfdd49a3f
Bug fix/simplify skip hasmore (#5466)
* simplify skip and hasMore

* include less dependencies in header files
2018-05-26 09:27:28 +02:00
Andrey Abramov 0f0235e8a9
Remove collection dependency from gather node (#5457)
* remove collection dependency

* some fixes

* fix test issues

* add tests for newly created functionality

* fix enterprise test fails

* add comment
2018-05-25 17:26:33 +03:00
Jan f55df6786a
blind commit of work, as I managed to break my local git repository (#5456) 2018-05-25 13:07:28 +02:00
Jan a295eaa120
show AQL functions used in query in explain output (#5451) 2018-05-24 21:17:51 +02:00
Jan Christoph Uhde a2dcb6cc5d WIP - start adding optional overwrite to insert operation (RepSert) (#5268) 2018-05-24 19:47:15 +02:00
Andrey Abramov 0489322cd1
Distributed ArangoSearch query execution (part 2) (#5397)
* further implementation of arangosearch query distribution

* do not store `aql::Collection` in `IResearchViewNode`, use `Query::collections()` instead

* some fixes

* cleanup

* some fixes after review

* more fixes after review

* use assertion instead of condition
2018-05-23 19:29:18 +03:00
Jan bf5caf210c
Bug fix/cleanup 2305 (#5437) 2018-05-23 15:45:55 +02:00
Jan 8e6d5df129
fixed minor several compiler complaints (#5406) 2018-05-23 11:50:00 +02:00
Vasiliy 94ddd7803d issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424) 2018-05-23 00:59:08 +03:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
Vasiliy 27f7f4a0aa issue 389.9: return TRI_vocbase_t& from Query::vocbase() (#5420)
* issue 389.9: return TRI_vocbase_t& from Query::vocbase()

* backport: use view name instead of view id in error messages
2018-05-22 19:03:14 +03:00
Jan 245c759523
fix incompatibility after fullCount change (#5399) 2018-05-22 11:28:17 +02:00
Jan 72ec463a8d
Bug fix/cleanup 2005 (#5404) 2018-05-20 20:57:43 +02:00
Wilfried Goesgens 7b8fce07a4 aql current_user in cpp, test (#5302) 2018-05-19 21:26:53 +02:00
Jan 379e848919
clean up AQL functions a little bit (#5401) 2018-05-19 21:15:14 +02:00
jsteemann 6139ae6225 fix broken error message 2018-05-18 17:17:44 +02:00
Jan df23ed092e
Bug fix/cleanup 1705 (#5376) 2018-05-18 09:21:50 +02:00
Wilfried Goesgens 4e805e6896 if the direction is wrong in production builds, divert to throw instead of assert. (#5383) 2018-05-18 09:21:08 +02:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Jan 0d98c0f3ba
Bug fix/fix codescan issues (#5371) 2018-05-16 20:40:37 +02:00