1
0
Fork 0
Commit Graph

528 Commits

Author SHA1 Message Date
Alan Plum 6c96e5dd24 Fix Buffer methods (#9070)
* Add failing tests

* Implement changes

* Add note to CHANGELOG

* Fix test failure
2019-06-03 17:07:52 +02:00
Max Neunhöffer 06054dfb64
Fix yet another agency test with clientid. (#9150) 2019-05-31 17:23:42 +02:00
Dan Larkin-York 44a413a9af Miscellaneous fixes for named indices (#9100) 2019-05-31 17:00:56 +02:00
Tobias Gödderz 79cd45f89c Wait for replication before inserting documents (#9151)
* Wait for replication before inserting documents

Also, increased some timeouts and fixed a log message

* Fixed some log levels and a log message

* Removed repair-distribute-shards-like-spec from greylisted tests
2019-05-31 16:09:20 +03:00
Tobias Gödderz 90abe3ba93 Increased timeouts, added a little more helpful error messages for assertions (#9135) 2019-05-30 04:30:39 +02:00
Wilfried Goesgens ec0cc82bf6 test files and their content were swapped (#9136) 2019-05-30 04:29:37 +02:00
Kaveh Vahedipour 773f3c8422 [devel] fix state clientlookuptable (#9066) 2019-05-30 04:24:46 +02:00
Jan d67513607d
fixed issue #9106 (#9116) 2019-05-29 18:27:38 +02:00
Markus Pfeiffer de9ef00896 Round estimatedCost in some tests (#9126)
Otherise slightly different floating point results in estimatedCosts cause
tests to fail.
2019-05-29 15:29:52 +02:00
Simon a5a72ef6d4 port Pregel segmented buffers (#9112) 2019-05-28 18:23:20 +02:00
Jan 033f411420
Bug fix/issues 27052019 (#9117) 2019-05-28 12:08:25 +02:00
Simon 27112a3486 Bug fix/jobs indexes (#9107) 2019-05-28 09:38:14 +02:00
Tobias Gödderz 5da9fcb0ce Bugfix of DFS path uniqueness implementation (#9041)
* Began work on generic traversal testing framework

* Translate shard ids to smart attributes

* Minor fixes

* Added a TODO note

* fixed tests

* cleanup

* first tests inside new traversal framework working

* added single server diamond tests

* Enable sharding in aql-graph-traversal-generic-graphs.js

* Tried to fix smart sharding

* Fixed smart graph name clashes, some cleanup

* Added meta tests

* Added TODO note

* Fixed single server

* Added a BFS order check

* Renamed test

* added more dfs and bfs tests for the openDiamond graph

* Added new graph "largeBinTree"

* Fix Map usage

* added smallCircle graph, added three dfs tests

* added smallCircle graph, added three dfs tests

* more smallCircle dfs tests

* Fixed largeBinTree shardings, and long graph names

* Fix long suffixes

* Provided names for shardings

* Added a log message

* more bfs smallCircle tests

* more bfs smallCircle tests - global

* smallCircle picture

* added completeGraph to graph test suite

* added more dfs tests for the complete graph up to depth 3

* Removed duplicate function definitions

* added depth 2 dfs complete graph test, added missing test to the exports

* Added test for largeBinTree

* added three path graphs, easy, advanced and more advanced

* Added a working check for bfs with global uniqueness

* Fixed usage of bfs checks

* Fixed expected input for two tests

* Added some static assertions to largeBinTree

* Oops

* added more advancedPath dfs tests

* better formatting of the visual graph

* more dfs advanced graph tests

* Implemented all easy path test combinations

* Fixed testAqlGraphQuery

* Fix in bfs check error reports

* Removed duplicate test

* Added advancedPath BFS tests

* Updated comments

* Fixed new tests

* Fixed graph description

* added more complete graph bfs tests

* added more complete graph bfs tests

* added testCompleteGraphBfsUniqueEdgesNoneD1 and testCompleteGraphBfsUniqueEdgesNoneD2

* added testCompleteGraphBfsUniqueVerticesUniqueEdgesPathD3 test

* added a few more complete graph bfs checks

* Extended a comment
2019-05-27 12:57:02 +02:00
Dronplane 3d697d8584 Added filtering analyzers before adding them to view link - duplica… (#9094)
* Added filtering analyzers before adding them to view link   - duplicates are not allowed any more

* Updated test testAddDuplicateAnalyzers - now it tests analyzers with same names but in different databases
2019-05-27 13:20:51 +03:00
KVS85 8d1885d0cb Bug fix/greylist timecritical and nondeterministic (#9047)
* Mark all timecritical and nondeterministic as greylisted

* Mark description to Greylist.txt

* Fix typo
2019-05-24 11:52:15 +02:00
Jan de717924ac
fix crash (#9057) 2019-05-21 18:19:11 +02:00
Jan 6cea7528fb
fix result of AQL function `MIN` in case the first input element is (#9053)
`null`
2019-05-21 15:43:33 +02:00
Alan Plum e8b7e1860a Bug fix/unbreak foxx security (#9042)
* Unbreak aardvark when --foxx.api false

The easiest way to disable the aardvark foxx routes is to just skip the
import and mount altogether. The previous approach caused the service to
throw an error because the module doesn't actually export anything.
There's no need to prevent the export if we just don't import the module
to begin with.

* Add an error code for service api disabled

* Throw an actual error when accessing Foxx API routes

* Throw an error when using legacy Foxx API

There's no easy way to use a catch-all here so disable them one by one. No new routes will be added to this API and it'll go away in 4.0 so there's no risk of forgetting to add the throw-check in new routes.

* Render subnav without delay

The subnav doesn't need the perm check results but we need the subnav to exist so we can populate it in other parts of the app. This fixes the subnav not showing up when reloading certain views (e.g. install service via upload).

* Hide ONLY the Foxx store when it is disabled

We still want to allow installing and upgrading/replacing services. We just don't want to show the store.

* Hide Foxx services in frontend if Foxx API is disabled

* Build aardvark

* fixed typo in disable check, fixed tests
2019-05-21 14:45:23 +02:00
jsteemann c640e07ada next attempt for fixing unreliable test 2019-05-20 11:45:25 +02:00
Lars Maier 4fc2790863 [devel] Direct Exec Scheduler (#9004) 2019-05-20 11:38:57 +02:00
Frank Celler 0bb8ec2e06 Revert "test is broken"
This reverts commit 469b94235b.
2019-05-18 09:55:01 +02:00
Frank Celler de566b405d Revert "fixed typo in disable check, fixed tests"
This reverts commit 068bbc9143.
2019-05-18 09:54:49 +02:00
Frank Celler 068bbc9143 fixed typo in disable check, fixed tests 2019-05-18 09:42:11 +02:00
Frank Celler 469b94235b test is broken 2019-05-18 08:04:49 +02:00
Simon 93b2e64f37 Port pregel fixes (#9022) 2019-05-17 16:32:58 +02:00
Jan Christoph Uhde 970f732dec add exceptions for tokens function. (#8984)
* Provide users of the TOKENS function with errors.

* remove newlines

* fix compare

* fix as asked for

* improve fix
2019-05-16 22:12:43 +03:00
Tobias Gödderz c9fa5fba11 Additional AQL skipSome tests for IndexExecutor (#8938)
* Began work on additional index skipSome tests

* Added more index skip tests
2019-05-16 11:30:24 +02:00
Jan 0ea4d35a17
fix failing test (#9006) 2019-05-15 22:58:14 +02:00
KVS85 d727cc7f9b Greylist tests/js/server/recovery/write-throttling-mmfiles.js 2019-05-15 12:27:04 +02:00
Tobias Gödderz 82f5978f30 Bugfix: Do not call getSome on remote after DONE (#8978)
* Bugfix: Do not call getSome on remote after DONE

* Added profiler tests for Sorting- and UnsortingGather

* Fixed jslint error
2019-05-15 09:13:44 +02:00
Vasiliy 12bd4c785b issue 555.1.1: issue 555.1.1: try loading previous data store snapshots if checkpoint file unavailable, ehnace error message, address some warnings (#8975) 2019-05-13 18:33:44 +03:00
Andrey Abramov 404bcc8b0c
switch to tiered consolidation by default (#8981)
* switch to tiered consolidation by default

* fix copy/paste error

* fix tests

* fix tests
2019-05-13 18:25:46 +03:00
Andrey Abramov 4e12fdb30b
ArangoSearch sorted indexes (#8888)
* move IResearchViewSort into separate unit

* ensure sort is propagated to IResearchLink

* update iresearch and extend some tests

* implement insertion of sorted values, store value comparator in link

* add tests for VPackComparer

* minor cleanup

* fix issue with sort definiton propagation

* propagate error message from index factory

* add integration tests for sorted indexes, single server

* initial implementation of IResearchViewMergeExecutor

* move iresearch::VPackComparer to separate translation unit

* store collection in IndexReadBuffer for IResearchViewMergeExecutor

* set primary sort from optimizer rule

* add some tests for sorted view case

* extend IResearchViewNode tests

* simplify primary sort definition

* pretty print order condition covered by arangosearch in query explanation

* fix compilation with g++-8

* address catch test failures

* fix tests

* adjust arangosearch optimizer rule to optimize sort in case of cluster

* make tests for sorted indexes work in cluster too

* fix tests

* extend js tests for sorted indices

* fix keyword in query explainer

* ensure sort-limit rule works with views

* backport ngram analyzer fix from upstream

* address jslint errors

* modify IResearchViewSort to support boolean direction specification format

* add some tests

* add tests

* add skip functionality

* extend tests

* add more catch tests

* inline some methods

* address jslint errors

* extend catch tests to reproduce the failure

* add some tests for check fullcount

* add more tests

* extend tests

* temporarily comment out broken tests

* extend js tests

* address review comments

* address jslint errors

* update iresearch
2019-05-11 17:58:56 +03:00
Simon 6075fec35e Forward port some changes (#8949) 2019-05-09 19:42:06 +02:00
jsteemann 60a19e0ab3 fix jslint warning 2019-05-09 19:11:21 +02:00
Jan Christoph Uhde 192ecd2f75 add test for unicode paths (#8909) 2019-05-09 19:00:48 +02:00
Jan c1d48c0fef
honor index hints in optimizer rule (#8940) 2019-05-09 18:47:36 +02:00
KVS85 2a66b53591 Greylist tests/js/client/shell/shell-foxx-paths-spec.js 2019-05-08 17:44:19 +02:00
KVS85 7847e8e9d8 Small fix of test setUp 2019-05-08 15:54:36 +02:00
Heiko 58db1023f3 Feature/aql skip some (#8771)
* testing skipSome implementation, default and passThrough

* added simple implementation for default and passthrough skipsome methods

* added skipRows function to enumerate collection executor

* added tests for the enumerate collection skipRows function

* implemented and tested enumeratecollection skipSome

* skipSome tests

* fixed a test, prepared iresearch view exec for skipping

* rm logs

* gcc bug workaround

* changed to original test code as it has been before

* iresearch skipping, added ires skip test

* added index executor skipRows

* added skip blockfetcher, quick exit index

* test

* const dummy and singlerowfetcher skip

* forgot return value

* input wrong initialized

* trying to remove dynamic cast and solve with a different approach

* cleanup

* const skip

* jslint

* Handle skipSome of subqueries correctly

* Removed unused code

* Removed unused member

* Simplified skip variants and enabled IndexExecutor skipping

* A little cleanup, fixed DependencyProxy::skipSome

* Reverted test change

* Tried to make testLimitBlock3 clearer

* Extended test suite

* Bugfix

* Added stats when skipping and fixed a few other things

* Bugfixes

* Moar bugfixes

* Update arangod/Aql/IResearchViewExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* applied requested changes

* Fixed a bug in LimitExecutor::expectedNumberOfRows()

* Fix skipSome in catch test RowFetcherHelper

* Fixed a bug in the tests

* Two bugfixes in LimitExecutor::expectedNumberOfRows

* Avoid passing large batch sizes from skipSome to getSome

* Fixed Windows compile errors

* Fixed a skip bug with WAITING in unsorting gather blocks

* Make aql-skipping find some cluster bugs, like the last commit

* Bugfix and additional asserts

* Fixed skipSome counting in IndexExecutor

* Resolved merge conflicts
2019-05-08 15:02:43 +02:00
Simon 0502a97abb forwardport changes from 3.4 (#8894) 2019-05-08 14:34:25 +02:00
jsteemann d50eeb0410 fix test 2019-05-08 10:24:54 +02:00
KVS85 068270fff9 Small speedup of setup in dump suite 2019-05-07 23:49:17 +02:00
Jan Christoph Uhde f3a2eaf6d9 Bug fix/permission tests (#8890) 2019-05-07 15:07:59 +02:00
Heiko 4837c7b864 Bug fix/collect aggregate undefined (#8853)
* allow 3.4 behaviour regarding aggregate collect and undefined variable

* added test to check undefined var

* rm print

* remove duplicate doCopyRow

* wrongly removed due merge

* initialize _allowSourceRowUninitialized always as false as default

* try to fix another case

* added missing parenthesis

* optimized if, also added compiler hint
2019-05-07 15:05:49 +02:00
Jan eed6729789
fix an issue, add tests (#8920) 2019-05-07 12:15:43 +02:00
jsteemann 8b245d25a4 remove print statement 2019-05-06 17:54:51 +02:00
KVS85 62d207edd2 Add debug print of installed service path 2019-05-06 16:50:27 +02:00
Simon d737c19302 address issues with active failover (#8908) 2019-05-06 15:52:49 +02:00
KVS85 9364f56594 Greylist tests/js/server/recovery/view-arangosearch-link-populate-no-flushthread.js 2019-05-06 14:33:32 +02:00
KVS85 0ef3056d65 Disable resilience_move_view test till it's reworked 2019-05-03 16:50:47 +02:00
Frank Celler 7c79e7359b greylisted view-arangosearch-link-populate-transaction-grey.js 2019-05-03 11:30:03 +02:00
Wilfried Goesgens 7c7f69d473 greylist view-arangosearch-link-populate-no-callbacks.js (#8885) 2019-05-02 15:07:52 +03:00
KVS85 5406cb7447 Add forgotten greylisted files 2019-04-30 22:18:48 +02:00
KVS85 ed14f3b650 Greylist tests/js/client/resilience/foxxmaster.js 2019-04-30 16:03:44 +02:00
KVS85 9f90a8347b Greylist tests/js/server/recovery/view-arangosearch-link-populate-transaction-no-flushthread.js 2019-04-30 16:01:00 +02:00
Simon 2f1cfa40e0 Fix tick overflow (#8840) 2019-04-26 11:07:04 +02:00
Tobias Gödderz 6dfb9995b4 Feature/aql by line cleanup (#8848)
* Removed useless parameters of buildCallback

* Renamed produceRow to produceRows and adapted a comment

* Renamed BlockFetcher to DependencyProxy

* Applied git-clang-format
2019-04-25 15:51:11 +02:00
Jan Christoph Uhde 677a79026c Foxx Security (#8845) 2019-04-25 09:56:29 +02:00
KVS85 e6b73726fd Remove several tests from greylisting 2019-04-24 21:38:49 +02:00
KVS85 3c972ef695 Fix db object visibility for ArangoSearch tests (#8819) 2019-04-23 09:31:29 +02:00
Markus Pfeiffer 1f2f5ec983 K_SHORTEST_PATHS queries only support one variable in FOR (#8784)
* K_SHORTEST_PATHS queries only support one variable in FOR

 * catch if the user gives more than one variable in grammar.y
 * also give a correct error message

* Test code that catches too many variables for K_SHORTEST_PATHS
2019-04-23 08:48:16 +02:00
Frank Celler 17fb0b795f removed print 2019-04-22 20:54:32 +02:00
Simon 569198a089 Abort el-cheapo transactions if servers fail (#8799) 2019-04-22 19:31:24 +02:00
KVS85 9027cb2418 Remove nondeterministic mark from 2 arangosearch recovery tests 2019-04-22 17:40:06 +02:00
KVS85 31ac3deee7 Remove moving-shards-cluster.js from greylisting 2019-04-22 17:10:20 +02:00
KVS85 7dc2ea0535 Greylist more frequently failed tests 2019-04-22 16:48:06 +02:00
KVS85 6911ee47f4 Remove view-arangosearch-link-populate-transaction-no-flushthread from greylisting 2019-04-22 16:01:19 +02:00
Frank Celler 499e97c78b added missing result 2019-04-21 20:24:37 +02:00
Vasiliy 40ebdd0b58 issue 545.1.1: output short analyzer prefixes for user definitions (#8814) 2019-04-21 15:10:28 +02:00
Frank Celler ed83084d5a splitted aql-view-arangosearch 2019-04-21 13:04:27 +02:00
Vasiliy 0375218ca2 issue 541.1: fix incorrect test filename (#8810) 2019-04-20 23:19:23 +03:00
KVS85 9cfaf511c8 Fix resilience_move location 2019-04-20 16:51:59 +02:00
KVS85 878415ef8e Change resilience and replication testsuits layouts required for new tests and greylisting 2019-04-19 01:14:34 +02:00
Alan Plum 8692f19e99 Handle extra args to db._query when using aql templates (#8789) 2019-04-18 10:50:46 +02:00
Markus Pfeiffer 304e8f861d Fix k Shortest Paths with weights, reactivate cluster tests (#8761)
* Make find in ShortestPathPriorityQueue const

* Fix k Shortest Paths with weights, reactivate cluster tests
2019-04-17 15:10:59 +02:00
Alan Plum 40aff3fdcd Upgrade JS deps (#8773)
* Make jsunity hash function avoid recursion

This guards against traversals of recursive data structures causing a stack overflow. This is crucial when using assertEqual with unknown inputs.

* Update deps

accepts: 1.3.4 -> 1.3.5
ansi_up: 2.0.2 -> 4.0.3
content-disposition: 0.5.2 -> 0.5.3
dedent: 0.6.0 -> 0.7.0
error-stack-parser: 1.3.6 -> 2.0.2
eslint: 2.13.1 -> 5.16.0
eslint-config-semistandard: 6.0.2 -> removed
eslint-config-standard: 5.3.1 -> removed
eslint-plugin-promise: 1.3.2 -> removed
eslint-plugin-standard: 1.3.2 -> removed
highlight.js: 9.12.0 -> 9.15.6
http-errors: 1.6.2 -> 1.7.2
iconv-lite: 0.4.19 -> 0.4.24
joi: 9.2.0 -> 14.3.1
joi-to-json-schema: 2.3.0 -> 4.0.1
js-yaml: 3.10.0 -> 3.13.1
marked: 0.3.9 -> 0.6.2
mime-types: 2.1.12 -> 2.1.22
mocha: 2.5.3 -> 6.1.3
qs: 6.5.1 -> 6.7.0
semver: 5.4.1 -> 6.0.0
statuses: 1.4.0 -> 1.5.0
timezone: 1.0.13 -> 1.0.22
type-is: 1.6.15 -> 1.6.16
underscore: 1.8.3 -> 1.9.1

* Inline eslint semistandard config

The eslint standard config has become extremely opinionated and incompatible with the existing coding style. This subset matches the existing coding style most closely. In the future we should migrate to an autoformatter like prettier and avoid this problem altogether.

* Linting

* Fix mocha runner

* New joi has additional property
2019-04-17 13:36:26 +02:00
Vasiliy 1a22d1360c issue 526.9.1: implement swagger interface, add documentation (#8730)
* issue 526.9.1: implement swagger interface, add documentation

* address review comments

* add ngram

* Formatting

* Move REST description to new Analyzers top chapter in HTTP book

* Missed a DocuBlock

* Add Analyzers chapter to Manual SUMMARY.md

* Move REST API description back to Manual

Headlines were broken

* Add n-gram example
2019-04-16 18:54:30 +03:00
Jan 7c0c01c401
use the primary index when full-scanning a collection (#8548) 2019-04-16 16:50:03 +02:00
Jan 1d175f446d
do not use the TTL index for queries which it does not support (#8766) 2019-04-15 18:55:19 +02:00
Markus Pfeiffer a14386267a k Shortest Paths (#8715)
* Add k-shortest-paths finder
2019-04-12 21:58:18 +02:00
Alan Plum b76f8adc0f Add SyntheticRequest#auth (#8750)
* Add SyntheticRequest#auth

* Fix tests
2019-04-12 21:18:35 +02:00
Alan Plum 3007c928aa Add collection.documentId method (#8749) 2019-04-12 19:53:01 +02:00
jsteemann ca8b81b52b make test more reliable 2019-04-12 13:09:10 +02:00
Jan 4f62dd8dae
added AQL function DECODE_REV (#8606) 2019-04-11 15:59:04 +02:00
Simon 937d743ba6 Bug fix/pregel stuff (#8733) 2019-04-11 15:58:28 +02:00
Frank Celler 9c4af6cb7d greylisted view-arangosearch-link-populate-transaction-no-flushthread.js 2019-04-10 12:46:47 +02:00
Frank Celler 6ec20cddee greylisted active-failover/basic.js 2019-04-10 12:44:54 +02:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Frank Celler 04d979e26e greylisted shell-foxx-api-spec 2019-04-10 12:20:21 +02:00
Vasiliy 030637c1c7 issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields (#8689)
* issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields

* address test failures

* remove debug logging
2019-04-10 13:10:25 +03:00
Simon 4e80e1b351 Simplify recovery code (#8699) 2019-04-09 21:48:04 +02:00
Jan c7500c0c95
finally fix AQL error messages in all contexts, so that they are usable (#8706) 2019-04-09 17:46:23 +02:00
Frank Celler b7c5e620fe still failing test 2019-04-09 14:57:31 +02:00
Frank Celler 3e26f66aa8 added pregel to greylist and raised log level 2019-04-09 14:45:08 +02:00
Jan 5897baa984
added db.<collection>.getResponsibleShard() (#8683) 2019-04-08 16:10:45 +02:00
Frank Celler 44b25e2e88 greylisted because it takes too long 2019-04-08 10:36:15 +02:00
Vasiliy 0410588723 issue 526.7.1: remove deprecated functions, update tests to set up required analyzers (#8688)
* issue 526.7.1: remove deprecated functions, update tests to set up required analyzers

* address merge issues

* address more merge issues
2019-04-05 18:10:42 +03:00
Jan 1160961af8
added debugging for replication-fuzz fails (#8674) 2019-04-04 10:20:49 +02:00
Jan 98aaa82048
added log output for pregel job status (#8673) 2019-04-03 17:17:20 +02:00
Max Neunhöffer 02281d3be4
Handle InitDone correctly. (#8552)
* precondition plan / version in compaction / store TTL removal independent of local _ttl set
* Agency init loops break when shutting down.
* assertion failures in store on restarting following agents
* Minor porting fixes from 3.4
2019-04-01 17:01:05 +02:00
jsteemann a5232f60aa fix dump-multiple with mmfiles in cluster 2019-04-01 12:15:17 +02:00
Jan 14720b85be
add AQL functions CRC32 and FNV64 (#8600) 2019-03-28 03:06:33 +01:00
Andrey Abramov 930b09cd93
[3.4] bug-fix/issue-#8294 (#8430) (#8585)
* [3.4] bug-fix/issue-#8294 (#8430)

* fix invalid optimizations for multi-valued attributes

* fix broken optimizations for multivalued attributes

* adjust tests

* add `IN_RANGE` function

* add tests

* add some shallow integration tests

* fix optimization for IN operator

# Conflicts:
#	arangod/IResearch/IResearchFeature.cpp
#	arangod/IResearch/IResearchViewOptimizerRules.cpp
#	tests/IResearch/IResearchFilterBoolean-test.cpp

* fix compilation errors

* fix another compilation issue

* address compilation errors

* fix tests
2019-03-27 18:33:16 +03:00
jsteemann e4594b72a8 revert to old behavior 2019-03-27 14:20:31 +01:00
Wilfried Goesgens c5587f1c30 fix enterprise tests - we mustn't restore smart collections over exising collections (#8581)
* fix enterprise tests - we mustn't restore smart collections over existing collections

* un-blacklist test
2019-03-27 09:08:38 +01:00
KVS85 4ea1675e74 Allow replication suite filtering (#8586) 2019-03-26 20:25:14 +03:00
Jan 207c978d1e
Bug fix/move a few actions from js to cpp (#8549) 2019-03-26 10:02:21 +01:00
KVS85 0212d56fb3 Greylist tests/js/server/replication/replication-fuzz.js (#8557) 2019-03-25 18:09:29 +01:00
Simon bcc26926de Support Transactions in RestCursorHandler (#8539) 2019-03-25 17:04:11 +01:00
Wilfried Goesgens 0634b24dde add tests for dump/restore of multiple databases (#8530) 2019-03-25 15:52:37 +01:00
Jan Christoph Uhde 5d5b77abb8 add ModificationExecutor (#8179)
All modification operator as Executors
2019-03-24 20:48:38 +01:00
Simon 329acea21c Add more El-Cheapo client tests (#8533) 2019-03-23 22:09:28 +01:00
Jan cfd3418115
microoptimizations (#8519) 2019-03-22 16:53:21 +01:00
jsteemann 868b58c7d7 reformatting, removed potentially undefined behavior 2019-03-22 13:03:37 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan 962a080399
attempt to make replication fuzz tests more stable (#8497) 2019-03-21 15:00:27 +01:00
jsteemann fbecc9df34 attempt to fix test 2019-03-21 09:27:05 +01:00
Heiko ce51797609 Feature/aql hashed collect (#8337)
Implementation of Hashed Collect Executor
2019-03-21 07:19:26 +01:00
Kaveh Vahedipour 5038dfe685 supervision must not copy snapshots into jobs (#8425)
* supervision must not copy snapshots into jobs
* CHANGELOG.
2019-03-20 17:07:54 +01:00
KVS85 84abf37435 Add testTemplateStringBuilderEmptyInlines test for https://github.com/arangodb/arangodb/pull/8141 (#8206) 2019-03-20 15:56:40 +01:00
Dan Larkin-York 04ef4595e3 Fix replication issue with index ID/name conflicts (#8448) 2019-03-20 14:06:02 +01:00
Dan Larkin-York 2eadab33e7 Index hints (#8431) 2019-03-19 09:14:18 +01:00
Vasiliy 29b941a677 issue 526.2.1: add upgrade step for legacy static analyzers (#8417) 2019-03-18 23:22:23 +03:00
Kaveh Vahedipour 888a20bcf7 also compaction step size test grey listed (#8435) 2019-03-18 11:17:14 +01:00
Kaveh Vahedipour a2ca779e71 log compaction excluded for no (#8423) 2019-03-14 17:05:28 +01:00
Wilfried Goesgens d88bc376a6 When updating documents with array indices and unique values we mustn't disable indexing in rocksdb (#8404) 2019-03-14 13:35:38 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
jsteemann cdb4b46554 added debugging data for testing 2019-03-13 18:08:45 +01:00
Jan dd4938598c
use requireFromPresent and barriers in more cases (#8398) 2019-03-13 15:16:22 +01:00
jsteemann 3e75acf67f fixed wrong usage of JavaScript sort function 2019-03-11 16:33:25 +01:00
Jan 12e11a5197
port of replication improvements from 3.4 (#8308) 2019-03-11 13:37:18 +01:00
Heiko def7b613db code now fits to the documentation of the arangoUser object inside th… (#8352)
* code now fits to the documentation of the arangoUser object inside the foxx request object

* more tests
2019-03-11 12:43:59 +01:00
Jan 85227e00e4
make geo-index-optimizer rule work with multiple loops (#8353) 2019-03-11 12:26:07 +01:00
jsteemann aec1abd673 try to fix test 2019-03-08 12:32:51 +01:00
Jan 9f9426e36c
Bug fix/allow creating foxx queues inside transactions (#8339)
* allow creating a foxx queue from within a transaction

* added tests
2019-03-07 14:28:39 +01:00
Vasiliy b1e30937bb issue 520.1.1: decrease link commit interval to 1 second, add workaround for MMFiles not recovering document insertions via WAL (#8313) 2019-03-05 14:50:54 +03:00
Kaveh Vahedipour 68178ba165 [devel] supervision bug fix backports (#8314)
* back ports for supervision fixes from 3.4 part 1

* back ports for supervision fixes from 3.4 part 2
2019-03-04 19:27:24 +01:00
Jan 9855dbad29
fix test (#8307) 2019-03-04 13:31:29 +01:00
jsteemann 9ee60c3b38 attempt to fix test failure 2019-03-04 09:54:34 +01:00
Jan 08a6561adc
Bug fix/small issues 21022019 (#8229) 2019-03-01 17:20:10 +01:00
Michael Hackstein 6e8d43b2da
Feature/aql traversal prune (#8255)
Allow PRUNE in the AQL Traverser.
2019-02-27 11:43:06 +01:00
Andrey Abramov 804c6e21e3
backport: Fix bug #8213 (#8273)
* backport: Fix bug #8213

* rename variable

* fix compilation error

* fix tests
2019-02-26 23:47:15 +03:00
Jan d46acbb156
Bug fix/make ttl index work with datestrings too (#8208) 2019-02-22 16:09:51 +01:00
KVS85 12c2739f04 Greylist ./tests/js/server/resilience/moving-shards-with-arangosearch-view-grey-cluster.js (#8227) 2019-02-21 16:50:35 +01:00
Heiko 4557dfc0d5 do not allow edge definitions with empty from or to arrays, added als… (#8198) 2019-02-20 18:32:33 +01:00
Jan 2b956d5b79
fixed issue #8165 (#8194) 2019-02-19 18:20:01 +01:00
Jan 44c6a2d732
Feature/ttl index (#8169) 2019-02-19 14:12:21 +01:00
Wilfried Goesgens 492d05c1f1 Feature/upgrade v8 7.1.302.28 (#8088) 2019-02-19 11:15:34 +01:00
Jan dd6329da25
when creating a new database with an initial user, set the database p… (#8184) 2019-02-18 16:16:53 +01:00
Tobias Gödderz c16bf52065 Resilience tests with data (#7497)
* Added data to resilience tests. Forward-port of #7193.

* Begin work on repair-dsl suite to run with data, too

* Use and check data in all tests

* Fixed jslint errors

* Added data to moving-shards-cluster test

* Added additional asserts during createBrokenClusterState()

* Improved failure messages

* Minor cleanup

* Greylist affected tests

* Un-greylist resilience tests, as the fix for moving leaders is now merged

* Prevent "Duplicate testsuite" error

* Added missing require
2019-02-15 12:50:28 +01:00
Luschgy b7b84beebf Feature/aql blocks by line (#6553)
* Fixed hidden merge conflict

* Added BlockFetcher, added a minimal test for SingleRowFetcher

* move RegInfo in types.h

* fix getSomeLogic

* Added the AqlItemMatrix implementation. Not yet tested!

* Fixed nr called => Waiting ratio in AllBlocksFetcherMock

* fix compiling

* add RegInfo to produceRow

* BlockFetcherHelper now returns a matrix

* pass RegInfo as requested into AqlItemRow ctor

I would still keep it out as the ItemRow should be a thin view.

* allow to change the AqlItemRows cheaper / remove _sourceRow member

* implement copyRow for AqlItemRow

* organize cleanup for outputblock used in getsome / shrink if necessary

* Added tests for SingleRowFetcher, cleanup in AqlItemRow

* start mocking

* Added test helpers, began work on a more complex test for SingleRowFetcher

* Re-added RegInfo again to AqlItemRow

* helper, added test case for FilterExecutor

* merge fixed compile

* Removed trx name where it is actually not used

* Made some functions TEST_VIRTUAL for mocking.

* implemented Sort executor

* Fixed the BlockFetcherHelper API, to always return a matrix if it is not waiting

* Implemented a SortExecutor test with data

* disable borken test

* Fixed test on SortExecutor. Green now, needs some more testing

* change a bit how the AqlItemRow is used in a test

* Added another test for the SingleRowFetcher

* Added more tests for SingleRowFetcher

* fix getRegisterInfo

* Moved RegInfo into ExecutorInfos. Also now use the FilterExecutor implementation in production code instead of FilterBlock. Starting integration testing.

* add more asserts

* Fixed undefined behaviour in TestCode.

* Do not shrink an empty output block

* switch to block pointers in AqlItemRow

* return blocks in the row fetcher

* Updated grammar.cpp

* Updated (fixed) tests to verify returnBlock

* Improved const-ness

* Changed BlockFetcher constructor in order to make it mockable

* Implemented BlockFetcherMock and rewrote some tests with it

* Updated a comment

* Updated comments

* init commit - EnumerateListExecutor, including EnumerateListExecutorInfos

* add potential fix for ItemMatrix

* Replaced fakeit with BlockFetcherMock in another GIVEN block

* Replaced the last fakeit mocks with BlockFetcherMock and removed fakeit

* Updated function definition

* included missing arary included

* Renamed AqlItemRow to InputAqlItemRow

* Split AqlItemRow into Input~ and Output~ classes

* EnumerateListExecutor + Tests

* Added tests for AqlItemRows

* Added test file for AqlItemRows...

* Made the ostream operator for AqlItemBlocks available to Logger and Catch info

* build error

* Added a second test for AqlItemRows. Need to make input space invalid now, sth. acidential referencing will show up

* Introduce a block ID in the InputAqlItemRow

* Copy output values correctly, avoid InputAqlItemRow pointers

* Added a test for registerClearout

* Fixed tests

* Added a test for adding new values

* Added a test for an incomplete row write to not count as produced.

* Fixed getSome and a test

* Added a test for larger AQL values in AqlItemRows

* Fixed compile error in hidden merge conflict

* Added tests for Filter counter in FilterExecutor

* OutputAqlItemRow now takes responsibility of the block

* Handed responsibility of blocks in ExecutionBlockImpl to OutputAqlItemRow

* EnumerateListExecutor should be close to its finish. Tests need to be fixed, not ready yet.

* better swap

* OutputAqlItemRow now takes ExecutorInfos

- added several assert
- implemented a more strict and safe produced() logic

* enum list test

* Added a more specific error message

* Removed OutputAqlItemRow.changeRow()

* fix invalid block freeing

* changed order of input/output registers

* Fixed a test

* Fixed matrix width in test

* fix failure tests

* finished enumlist tests

* Implemented AllRowsFetcher and tests for it

* Bugfix in skipSome stub

* insert sort and enumerate list executors

into exectuion nodes' create functions

* fixed singlerowfetcherhelper, added tests

* small fixes

* add missing Infos

* restore accidentally deleted scorer

* make it compile

* fix executor info creation / usage

* make use of output var in EnumarateListNode

* Added Executor::Stats and FilterStats

* Extended FilterExecutor tests with stats, added node tracing to getSome

* Fixed Sort-&EnumerateList- Executor to use the new return type with stats

* Fixed compile error in tests

* Changed *ExecutorInfo constructors and classes to contain the necessary register informations

* Made an ExecutorInfo construction more readable

* Updated comments

* Fixed checks in OutputAqlItemRow::setValue()

* Add comparison method to SortRegister. Forbid construction without one.

* Implemented ExecutionBlockImpl::initializeCursor

* more mocking inside ExecutionBlockImplTest

* Minor changes

* Avoid AqlValueGuard destroying the value copied into the output block

* Disabled failure tests that do not apply to the new Sort implementation

* further mocking of test modules

* Removed constexpr for compatibility with outdated gcc versions

* add some failure tests - getsome works differently

therefor we do not test the old behavior

* fix sort node

* clean up alq item matrix

* Removed the transaction methods from read-only access to AqlValues (get and at) and repliced by CollectionNameResolver where necessary. Still left: some calls extract VPackOptions from the Transaction. Compare needs a BuilderLeaser only.

* textexecutionhelper, execimpl test

* Eliminated FilterBlock

* Eliminated the EnumerateListBlock

* first successful running test of executorblockimpl

* Eliminated SortBlock

* fixed testexecutor helper

* Made the ExecutorInfos private to the executors again.

* Implement and use AqlItemBlockShell (#6562)

* Added AqlItemBlockShell

* Minor changes

* Mostly replaced AqlItemBlock with ~Shell.

Current state:
- arangod compiles
- OutputAqlItemRow has yet to be changed
- catch tests have yet to be fixed

* Use AqlItemBlockShell in OutputAqlItemRow

* Added registersToKeep to AqlItemBlockShell

* Removed ExecutorInfos from OutputAqlItemRow

* Split AqlItemBlockShell into Input~ and Output~

* Changed tests so they compile again - they're still not yet working.

* Fixed tests

* fixed building, modified impl test

* Removed now superfluous BlockFetcherMock functions

* more execimpl testing

* Removed block IDs

* more impl testing

* Reduced dependencies of BlockFetcher construction

* Bugfix, cleared documentation

* Removed debug-output

* Adressed minor code review remarks

* Moved method definitions into .cpp files to avoid includes

* Added assertions

* Added comments

* Addressed several more review comments

* Added missing includes

* more testing, not finalized yet

* added missing header file

* added a test for execution block impl

* removed old comment

* Improved lookup algorithm to identify correct block/row combination in AqlItemMatrix

* added test for ExecutionBlockImpl

* move utility to the right position

* Bugfix and a TODO note

* Bugfix: use a uniform ResourceMonitor in tests

* Made OutputAqlItemRow::setValue clone by default and copy on move

* Replaced the getRow lookup with a binary search

*  enumerate tests

* Disable emplacing of AqlValues in ItemBlock

* Reenabled std::enable_if will warn until oskar is using c++17 as standard.

* Add helper function to create a shared pointer of an unordered_map<RegisterId>

* CalculationExecutor (#6570)

* add CalculationExecutor skeleton

* first shot of CalculationExecutor

* fix minor bugs

* handle references

* use new shared pointers

* commit to share work

* fix failure test

* reformat with clang-format

* add test for CalculationExecutor

* fix CalcualtionExecutor test

* clean up

* Removes unused code

* clean up includes

* fix/remove comments

* add changes requrested by hacki

* ReturnExecutor (#7932)

* add skeleton to CMakeLists.txt

* build most parts of the ReturnExecutor

* This commit removes ReturnBlock and adds some debug messages

removed inheritRegisters functionality

* finish ReturnExecutor

* add test for return executor

* remove unused code

* add stats and clean up includes

* verify result in ReturnExecutorTest and move *Stats to "Aql/Stats.h"

* Fixed algorithm to find correct row in a MultiRow Input

* fix full count and failure tests (#7969)

* fix full count and failure tests

* fix filter test

* WIP: fixes

* fixed

* NoResultsExecutor (#7953)

* add NoResultsExecutor

* clean up

* some review changes

* fix test

* fix

* fix WAIT -> DONE

* remove unused code (#7995)

* create skeleton for SingletonExecutor (#7957)

* create skeleton for SingletonExecutor

* WIP: continue implementation

* WIP: add fetcher skel

* WIP: add inject function to ConstFetcher

* rename SingletonExecutor to IdExecutor

* WIP: replace singleton node

* WIP: More Debugging on IdExecutor

* WIP: IdExecutor

* WIP: add failure tests

* make iresarch test work but not pass

* WIP: add ConstFetcherHelper

* fix test

* fix tests

* fix iresearch tests

* fix wording

* add review changes

* add review change to SingleRowFetcher as well

* fix test

* fix error when increasing index

* temporarily remove fold-expression

* clean up

* Fix gcc compile error

* Feature/aql limit executor (#7982)

* limit executor

* removed not needed tri assert

* rm _invariable in limitnode

* find out if we are in a subquery

* queryDepth in limit infos

* added missing queryDepth

* finalizing limit executor

* added more tests, fixed a condition

* add assert to test

* added missing fullcount check

* typo

* minor changes, fixed test

* test

* wrong check

* fixed tests

* false merge

* remove basic blocks

* remove basic block includes

* include sort order

* removed not needed basic blocks

* removed not needed basic blocks

* fix test

* Feature/remove input output shells (#8090)

* Added template parameters to allow for block pass-through to be chosen at compile time

* Adapted the tests to match the new template parameters

* Adapted ConstFetcher and IdExecutor to match the new templated fetcher

* Implemented pass-through, fixed SingleRowFetcherTest

* Fixed implicit merge conflicts

* Fix assumptions that are violated by block pass-through

* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fixed compile error

* Fixed a useless and leaking allocation

* Try a less implicit fix

* Allow IdExecutor and ConstFetcher to be used with pass-through

* Performance patches from @mchacki

* Fixed merge conflicts, and bugs in ConstFetcher

* Fixed atMost pass-through, and blocks that call their parents too often

* Removed InputAqlItemBlockShell (except from tests)

* Removed OutputAqlItemBlockShell (except for tests)

* Moved implementations of OutputAqlItemRow methods in the header

* Moved SingleRowFetcher::fetchRow to the header to allow inlining

* Reenabled returnInheritedResults optimization

* Enable passThrough for ReturnExecutor with inherited results

* Specialized CalculationExecutor to avoid unneccessary conditional jumps

* Made tests compile again (but are currently broken)

* Fixed ReturnExecutor test

* Move created InputRows out of AqlItemMatrix

* Limit- and EnumerateList Executors no longer fetch from upstream after DONE

* Added comments

* Added experimental AqlItemBlock optimizations

* Removed _numEntries, added comments

* Allow executors to pass an "atMost"-limit to the SingleRowFetcher (for LIMIT)

* Let the LimitExecutor pass atMost upstream

* Bugfix for FullCount

* Feature/aql enumeratecollection executor (#8028)

* initial commit

* enum coll executor

* buildable version of enumeratecollectionexecutor

* wrong usage of outVariable

* Added template parameters to allow for block pass-through to be chosen at compile time

* Adapted the tests to match the new template parameters

* Adapted ConstFetcher and IdExecutor to match the new templated fetcher

* more fixes, still some tests failing

* Implemented pass-through, fixed SingleRowFetcherTest

* Fixed implicit merge conflicts

* adding missing set value (remember optimize todo), optimized constructor

* removed old enumerate collection blocks

* Fix assumptions that are violated by block pass-through

* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fixed compile error

* Fixed a useless and leaking allocation

* added broken uncompilable executor test

* Try a less implicit fix

* fix mocking

* enumerate collection executor test

* remove class, use only static function.

* clean up test after merging with main branch

* Allow IdExecutor and ConstFetcher to be used with pass-through

* Performance patches from @mchacki

* Fixed merge conflicts, and bugs in ConstFetcher

* Fixed atMost pass-through, and blocks that call their parents too often

* Removed InputAqlItemBlockShell (except from tests)

* Removed OutputAqlItemBlockShell (except for tests)

* Moved implementations of OutputAqlItemRow methods in the header

* Moved SingleRowFetcher::fetchRow to the header to allow inlining

* Reenabled returnInheritedResults optimization

* Enable passThrough for ReturnExecutor with inherited results

* Specialized CalculationExecutor to avoid unneccessary conditional jumps

* Made tests compile again (but are currently broken)

* Fixed ReturnExecutor test

* Limit- and EnumerateList Executors no longer fetch from upstream after DONE

* Removed an assertion which seems to be incorrect during bootstrap.

* Inlined AQL ItemMatrix and removed unused elements in other Executors

* Now inline + Move in AqlItemMatrix, seems to be overall best performing

* Bugfix: Do not returnInheritedResults on DBServers

* Fixed LIMIT test, and made it more thorough.

* Fixed createion of NoResultBlock creation in IResearch Nodes. Also relaxed an assertion

* Reactivated failure point, now identical with anotherone in OutputRow, need to unified

* Bug fix: do not mutate the offset in the LimitExecutor (#8109)

* Introduced test

* Fixed LimitExecutor to not mutate its infos, so it works correctly in subqueries

* Fixed crashes in CatchTests

* Fixed non-maintainer build

* try to fix subqueries

* Fix isInSubQuery for non-singleton nodes.

* remove useless while true

* Feature/aql blocks by line dan sort (#8143)

* Add SortNode::SorterType to SortExecutorInfoss.

* prepare imple of sort executor

* WIP: add some code for dan sort

* WIP: working at pushRows

* WIP - try to add Dan's heap-sort

* re add missing test

* 2 fixes

* fix

* Dan's tests are working

* remove unused variable

* remove SortBlock.cpp from CMakeLists.txt

* remove hack and provide an extra method to set the _baseIndex

* Simplified ContrainedSortExecutor.

* fix forgotten variable

* update doc

* Assert that we do not use functions that might return wrong results.

Preconditions or invariants the functions rely on are broken. Therefore
the functions can not be used. This is asserted in maintainer mode.

* add function to restore row invariant

* do not emplace and pop_back if not necessary

* Use clear registers in case we can move the rows.

* WIndows opst zirkus

* Actually destroy AqlItemValue

* Feature/traversal executor by line (#8000)

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Further test cases

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Fixed leftovers from rebase

* Refixed mocks in different folder

* Fixed Mock import

* Fixed Calculators Exector Test to use the unified mocked server.

* Implemented logic for the TraversalExecutor. Also added tests for it. Not fully complete though

* Added validation of result values

* Added usage of constant register in traversal executor

* Included filter variables in traersal executor. Added test for const input. And moved MMDR into base traverser class for simplicity

* Added ostream operators on AQLValue and AqlInputRow. Not 100% done yet but good enough

* Added a shutdownable ExecutionBlock. Some blocks need to implement a specific shutdown still. I added this class to make it easier to eliminate them when the time is due.

* Removed ostream operator for INPUT row again, does for some reason not compile on Linux0o

* Fixed initialization order

* Removed classwrapper for shutdown.

* Applied review changes

* Refixed failing tests, API is not subject to change

* Fixed CatchTest (wrong usage of string literal in test case)

* Feature/shortest path executor (#8042)

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Further test cases

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Fixed leftovers from rebase

* Refixed mocks in different folder

* Fixed Mock import

* Fixed Calculators Exector Test to use the unified mocked server.

* Implemented logic for the TraversalExecutor. Also added tests for it. Not fully complete though

* Added validation of result values

* Added usage of constant register in traversal executor

* Included filter variables in traersal executor. Added test for const input. And moved MMDR into base traverser class for simplicity

* Added ostream operators on AQLValue and AqlInputRow. Not 100% done yet but good enough

* Added a shutdownable ExecutionBlock. Some blocks need to implement a specific shutdown still. I added this class to make it easier to eliminate them when the time is due.

* Removed ostream operator for INPUT row again, does for some reason not compile on Linux0o

* Fixed initialization order

* First draft of shortest path executor

* Fixed logic bugs, Local AQL tests succeed

* Added first version of ShortestPathExecutor tests. Edge output tests are missing yet.

* Added EDGE output tests

* Replaced TraversalStats with NoStats, they have never been tracked...

* Removed obsolete ShortestPathBlock.

* Removed classwrapper for shutdown.

* Applied review changes

* Refixed failing tests, API is not subject to change

* windows macro opst circus

* Clear registers per block, not per row (#8164)

* Fixed compilation under linux again...
2019-02-14 19:13:08 +01:00
Jan a911ea64bf
issue #8137: NULL input field generates U_ILLEGAL_ARGUMENT_ERROR (#8140) 2019-02-12 17:56:01 +01:00
Vasiliy 98b024d787 issue 466.6: remove unused code, remove hardcoded view meta values (#8130) 2019-02-11 16:32:04 +03:00
Frank Celler 5277f7744e fixed typo in version check 2019-02-08 16:51:52 +01:00
Jan 2bb1e1d3e1
fix issue #8108 (#8122) 2019-02-07 13:03:13 +01:00
Simon 581f9f362b test for broken indexes (#8103) 2019-02-05 09:38:31 +01:00
Jan 635ff0d934
more mmfiles replication tests changes (#8097) 2019-02-04 16:53:46 +01:00
Tobias Gödderz 4b7dc8cd28 Sorted COLLECT: avoid nullptr deref when skipping and fix non-invalidated input variables (#8038)
* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Try a less implicit fix

* Updated CHANGELOG
2019-02-04 16:38:13 +01:00
Simon 1123bc10c0 Check authentication for cluster APIs (#8051) 2019-02-04 14:47:44 +01:00
Jan 366af1782b
some more debug logging for replication (#8082) 2019-02-04 12:39:34 +01:00
Dan Larkin-York efb8364d99 Optimize memory usage in AQL by using constrained heap sort when reasonable (#7990) 2019-01-31 22:46:57 +01:00
KVS85 bbfae0ceef Update shell-database-{rocksdb|mmfiles}-noncluster.js (#8083) 2019-01-31 22:08:43 +01:00
Jan f466cdda6a
return V8 context after usage not only on DB servers, but also for st… (#8079) 2019-01-31 17:26:38 +01:00
Simon 655d594949 Fix background indexing (#8065) 2019-01-31 11:49:57 +01:00
Simon fd70b6fc34 Background Indexing (#8039) 2019-01-29 09:31:32 +01:00
Jan f7d94daba2
port some changes from 3.4 (#8011) 2019-01-29 09:26:57 +01:00
Andrey Abramov d1e9373244
improve logging and add tests (#8027) 2019-01-24 13:32:21 +03:00
Frank Celler 9927b3a281
Feature/maskings (#8006) 2019-01-22 22:23:25 +01:00
Jan cabeb8fc7b
make use of projections if the projected data is used by an IndexNode's condition (#8001) 2019-01-21 23:11:17 +01:00
Andrey Abramov a2d0b9a3d1
Feature/arangosearch restrict collections (#7989)
* add ability to restrict collections scope of ArangoSearch view in query

* fail if collection which is not registered with a view was specified in options

* allow bound parameters to be used with ArangoSearch view options, add unit tests

* fix tests

* add tests for cluster

* fix issue in deseralization logic

* fix cluster tests, extend integration tests

* more tests

* fix failing cluster test

* address review comments

* address test issues
2019-01-21 17:49:37 +03:00
Jan f3597510cf
fix scrambling of AstNodes that were in use multiple times (#7979) 2019-01-18 18:30:51 +01:00
Jan 9635df56eb
added "peakMemoryUsage" query result figure (#7952) 2019-01-17 13:58:18 +01:00
Simon 86fe11024b Fix stream cursor bug (#7956) 2019-01-16 13:24:41 +01:00
Tobias Gödderz a1d3bc3e94 Foxx queue jobs hanging after Foxxmaster crash (#7922)
* Fixed bug where the Foxxmaster doesn't reset jobs after a crash when it should, or a non-master coordinator removes jobs in progress during startup

* Added a regression test

* Updated CHANGELOG

* Fixed non-maintainer compile
2019-01-14 16:08:08 +01:00
Vasiliy fac48ddbc1 issue 511.9.1: track DataSource instance instead of only its name in QueryCache (#7943)
* issue 511.9.1: track DataSource instance instead of only its name in QueryCache

* address use of invalid function

* fix expected test result

* address merge issue
2019-01-14 16:25:18 +03:00
Jan 5670698d7b
fix issue #7933: Regression on ISO8601 string compatibility in AQL (#7935) 2019-01-14 13:51:13 +01:00
Jan e42befdc52
do not simplify non-deterministic conditions (#7926) 2019-01-11 14:48:34 +01:00
jsteemann 79257310c0 increase test timeouts 2019-01-11 14:22:17 +01:00
Wilfried Goesgens 457e14b970 add windows suspend/resume hooks as @char101 sugested for others (#7833)
* add windows suspend/resume hooks

* disable procdump
2019-01-11 12:41:27 +01:00
Jan Christoph Uhde f65853e30f make use of sortedness of rocksdb primary index (#7788) 2019-01-11 09:40:22 +01:00
Andrey Abramov d4a010baf9
Bug fix/internal issue #316 (#7911)
* allow using scorers outside ArangoSearch view context

Signed-off-by: Andrey Abramov <andrey@arangodb.com>

* ensure query is properly optimized after replacement of scorer functions

* do not apply `handleViewsRule` to queries without views

* simplify optimization rule for ArangoSearch views

* show ArangoSearch view scorers in query explanation

* fix tests

* fix tests

* add stub for scorer related tests

* reformat

* check variable depth in `ViewExpressionContext::getVariableValue`

* add some tests

* address js test failures

* address jslint errors

* ensure `IResearchViewNode` exposes variables used in scorers

* ensure scorers with expressions are deduplicated

* fix deduplication for indexed access

* more tests

* partially address review comments

* address review comments

* simplify code

* remove irrelevant, commented out code

* ensure array comparisons are properly handled

* update changelog & loki
2019-01-10 21:04:19 +03:00
Simon 5a0dd57c3a improve assertions (#7830)
(cherry picked from commit 44890c98583ed16e23990b8987fb808bff89057f)
2019-01-09 10:29:53 +01:00
Jan f66bac4f86
added AQL function CHECK_DOCUMENT (#7842) 2019-01-04 15:31:29 +01:00
Jan b3a3e35f3d added arangorestore options `--number-of-shards` and `--replication-f… (#7869)
* added arangorestore options `--number-of-shards` and `--replication-factor`

* updated documentation

* added version numbers
2019-01-03 11:07:02 +01:00
Jan 98ff9621bf
prevent duplicate attributes being generated by AQL queries (#7837) 2019-01-02 12:37:17 +01:00
Jan 74f0bbed5a
fixed issue #7834 (#7843) 2019-01-02 11:22:42 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Michael Hackstein 4965cea454 Feature/improve edgeindex covered (#7718)
* The Rocks Edge index now states that it covers _from and _to and can be used as a covering index for both.

* Moved the ExecutionState checkmark in query to a later point. It actually still included parts of Instantiation in cluster

* Updated test for the new feature, it now covers more states

* Updated changelog

* Updated ClusterIndex to support RocksDBEdgeIndex covering

* Removed assertion, during startup this assertion is not hold

* Replaced inplace builder by a leased builder

* Update CHANGELOG

Co-Authored-By: jsteemann <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: jsteemann <jsteemann@users.noreply.github.com>
2018-12-21 13:41:25 +01:00
Jan 33b288e3dc
fix restrictCollections parameter on database level replication (#7810) 2018-12-19 19:07:31 +01:00
Jan ea39dfa3d2
fixed issue #7757 (#7805)
* fixed issue #7757

* fix compilation error
2018-12-19 14:06:52 +01:00
Jan 529e1a5579
fix issue #7743: properly invalidate variables after COLLECT statemen… (#7780) 2018-12-18 18:13:34 +01:00
Jan 93d624d217
issue #7749 (#7794) 2018-12-18 17:16:14 +01:00
Jan 259170f1c2
fixed issue #7763 (#7793) 2018-12-18 17:02:01 +01:00
Jan a19f90c771
fix invalid handling of `_lastValue` in case of multiple coordinators (#7734) 2018-12-12 11:49:54 +01:00
Michael Hackstein 170e3520be
Improved stability of dump restore test (#7593)
* Improved stability of dump restore test, the assumption tested was not correct

* Fixed options in recovery test suite
2018-12-10 15:48:34 +01:00
Alan Plum c74583b089 Fix windows test for Foxx socket request URL (#7536)
* Fix windows test for Foxx socket request URL

* Fix unix abs path test
2018-12-10 15:44:00 +01:00
Max Neunhöffer c78e3c89e0
Call license key check. (#7594) (#7704)
* Call license key check. (#7594)

* Add CHANGELOG entry.

* Fix super user JWT token behaviour with non-ex. db. (#7656)
2018-12-10 10:33:00 +01:00
Jan 21d98efa91
do not optimize away sort clauses when it is unsafe to do so (#7693) 2018-12-07 16:23:52 +01:00
Jan cec6d3292a
fix test failures (#7659) 2018-12-05 18:14:57 +01:00
Jan 762c23bf77
add more tests and improve queries a bit for single remote AQL operations (#7658) 2018-12-05 17:37:25 +01:00
Jan ff2e43f374
Bug fix/aql allow floating point values without leading zero (#7647) 2018-12-05 16:13:04 +01:00
Jan 6fc8bb4a45
speed up remove ops for RocksDB engine (#7639) 2018-12-04 19:23:49 +01:00
Max Neunhöffer f9dd500132
Wait for 5 dbservers before starting resilience tests. (#7542)
* Wait for 5 dbservers before starting test.
* Fix jslint.
2018-12-03 13:17:07 +01:00
Andrey Abramov b016ba3255
re-enable arangosearch scoring tests in cluster (#7581) 2018-12-02 17:07:01 +03:00
Jan 8c8472be48
Bug fix/issue 7522 (#7556) 2018-11-29 19:25:49 +01:00
Dan Larkin-York 6d542e11c8 Fix end condition (hasMore) for EnumerateViewNode. (#7279)
* Fix end condition (hasMore) for EnumerateViewNode.

* Fix crashes.

* Some more fixes.

* eliminate code duplication
2018-11-26 23:41:58 +03:00
Max Neunhöffer d72e51ed8f
Fix move leader shard. (#7445)
* Ungreylist move shard test.
* Move leader shard: wait until all but the old leader are in sync.
* Increate moveShard timeout to 10000 seconds.
* Add CHANGELOG entry.
* Fix compilation.
* Fix a misleading comment.
2018-11-26 15:04:04 +01:00
Simon f90b48f792 switch default message for requireFromPresent (#7439) 2018-11-23 18:59:03 +01:00
Heiko 3322cc5ae2 Bug fix/update foxx services after restore (#6774)
* triggers foxx queue update which leads to the internal selfHeal function of a foxx app to make foxx apps publicy visible after a foxx app is restored

* this will also add the recognition of imported foxx services

* Testing JS refactoring

* Added test case for Foxx dump -> restore cases

* Fixed check, sth- i tries to contact to the foxx service. Test is red now.

* Fixed tests for dump / restore using Foxx. Also first draft to fix the fox side. Blocked by sth. fixed in the Main Repositorx already

* Fixed typo in dump/restore testsuite

* Removed old version of the fix

* Fixed JSLint
2018-11-23 17:06:09 +01:00
Max Neunhöffer d772253b89
Make error handling in move shards tests more robust. (#7419)
* Make getCleanedOutServers error handling more robust.

* Fix two more instances of getCleanedOutServers.
2018-11-22 22:53:24 +01:00
Simon c584527d79 Fix restore of views, add --view option (#7425) 2018-11-22 19:24:24 +01:00
Jan e1e0f62647
add some more intermediate commit tests (#7416) 2018-11-22 15:29:27 +01:00
Vasiliy 1a0b9b9261 issue 153: ensure views are dropped in Agency when database is dropped in cluster, minor fixes (#7370)
* issue 153: ensure views are dropped in Agency when database is dropped in cluster, minor fixes

* backport: add test to ensure views are dropped when database is dropped from plan, fix some issues in ClusterInfo

* optimize primary key lookups in ArangoSearch

* fix test

* Add JS tests

* temporary comment optimizations
2018-11-21 19:18:34 +03:00
jsteemann 0025f82cda cleanup 2018-11-21 10:07:35 +01:00
Simon cc55ef9f82 Faster index creation (#7348) (#7383) 2018-11-21 09:53:14 +01:00
Alan Plum c36c8c48ee Teach Foxx req.makeAbsolute about unix sockets (#7327)
* Expose rawReq.server.endpoint in JS

* Make rawReq.portType enumerable

This is not strictly necessary but we don't gain much from hiding its existence.

* Allow using http: protocol in local requests

This helps distinguishing local request URLs from file paths in ambiguous contexts.
The alternative would require the user to specify the local endpoint but this doesn't work with
sockets (and for non-sockets would incur the penalty of a real network roundtrip).

* Teach Foxx req.makeAbsolute about unix sockets

* Add Foxx socket tests
2018-11-20 14:20:56 +01:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
Max Neunhöffer 71a0550b04
Another place with a resigned leader. (#7349) 2018-11-19 12:10:09 +01:00
Jan b475f27148
fix queries that refer to COLLECT variables from inside COLLECT (#7332) 2018-11-15 15:11:11 +01:00
Markus Pfeiffer 39bdebf851 Port bug-fix-3.4/timeout-create-coll to devel (#7307)
* Fix loophole in error handling.
* Fix inquiry case of id not found: 404.
* Also handle correctly in AgencyComm.
* Fix agency tests.
* Fix error handling in dropCollectionOnCoordinator.
2018-11-14 10:03:55 +01:00
Jan f1760bf01b
fix failing non-deterministic query-stream test (#7295) 2018-11-13 11:36:41 +01:00
Alan Plum fe7eae33bd Fix Foxx Queues infinite maxFailures (#7223) 2018-11-12 13:20:10 +01:00
jsteemann 285cff52c0 fix "duplicate test suite name" error 2018-11-12 11:32:33 +01:00
Max Neunhöffer ffd40088ed
Greylist moving-shards-with-arangosearch-view-cluster. (#7273)
* Greylist moving-shards-with-arangosearch-view-cluster.

* Add documentation for greylisted tests.
2018-11-08 16:26:15 +01:00
Wilfried Goesgens bfde87751c add more information when timeout failing the index creation tests (#7265) 2018-11-08 11:50:30 +01:00
Dan Larkin-York b447f98a7a Upgrade old MMFiles collections for compatibility with ArangoSearch. (#7248) 2018-11-07 15:52:37 +01:00
KVS85 e67690c554 Fix update-view-arangosearch-spec tests (#7245) 2018-11-07 00:30:21 +03:00
KVS85 4646ff28b2 Add view profiling tests (#7229)
* Add view profiling tests

* Fix jslint errors
2018-11-06 17:15:46 +03:00
Tobias Gödderz 1de42118b7 Fix "Duplicate testsuite" error (#7238) 2018-11-06 13:10:29 +01:00
Tobias Gödderz cc30bc1585 Fix teardown, improved assertion & error message (#7219) 2018-11-06 13:09:51 +01:00
Jan 2115abdce9
smaller changes for replication (#7200) 2018-11-02 18:38:32 +01:00
Simon b883026f45 Server stream cursor (#7186) 2018-11-02 16:19:48 +01:00
Vasiliy f088733420 issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7166)
* issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address potential deadlock between link creation and FlushThread

* remove code causing nullptr access

* add back lock around reader reopen

* revert: address potential deadlock between link creation and FlushThread

* invalidate payload for each field in FieldIterator before setting a value
2018-11-01 23:10:01 +03:00
Jan 4bc646f207
increase the maximum allowed number of collections in an AQL query (#7154) 2018-10-31 17:20:30 +01:00
Jan 7cfc63f007
patch collection counts on the slave in case it is off (#7147) 2018-10-31 17:20:14 +01:00
Vasiliy 8f44afb6cf issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion (#7101)
* issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion

* backport: address test failures

* backport: ensure arangosearch links get exported in the dump

* backport: ensure view is created during restore on the coordinator

* Updates for ArangoSearch DDL tests, IResearchView unregistration and known issues

* Add fix for internal issue 483
2018-10-30 12:50:35 +03:00
Simon 5b71dff64f RocksDB replication thread safety (#7088) 2018-10-29 18:09:46 +01:00
Max Neunhöffer 15268fd051
Consider case of resigned leader. (#7059) 2018-10-29 09:19:50 +01:00
Wilfried Goesgens a95d117396 if we run in maintainer mode, be more patient (#7050) 2018-10-25 15:39:53 +02:00
Michael Hackstein 83c1b08c9f
Removed incorrect skipping of Batches in RocksDB Tailing syncer. (#7022)
* Removed incorrect skipping of Batches in RocksDB Tailing syncer. This caused issues, whenever one transaction was spiltted.

* Added a test for Splitting a large transaction in RocksDB

* Reactivated skipping in RocksDB Wal Tailing (reverts initial fix)

* Actually include lastScannedTick in CollectionFinalize. Proper fix, kudos to @jsteemann.

* Fixed healFollower task in split-large-transaction test
2018-10-25 12:31:25 +02:00
KVS85 6623b8824a Execute all ArangoSearch DDL tests on both cluster and single (#7053) 2018-10-24 22:18:11 +03:00
Wilfried Goesgens 2cfe9126f2 Bug fix/windows process handling (#7012) 2018-10-23 15:38:24 +02:00
Alan Plum 4e25350f6b Fix Foxx API handling of broken services (#6459)
* Add tests for service failing on mount

* Don't install broken services

* Restore error page for broken services

* Prefer JSON error pages

* Expose service upsert in API

* Add Foxx fixes to CHANGELOG

* Update CHANGELOG

Typo
2018-10-23 13:05:03 +02:00
jsteemann a0b0385554 jslint 2018-10-23 11:21:35 +02:00
Wilfried Goesgens 15a366b17f Bug fix/fix duplicate testcasenames (#7003) 2018-10-23 11:18:22 +02:00
Tobias Gödderz 9d8858d75c Fixed optimize-traversals and added a regression test (#6974)
* Fixed optimize-traversals and added a regression test

* Updated CHANGELOG
2018-10-23 09:45:54 +02:00
Jan 158876a74f
Bug fix/fix array sorting (#6989) 2018-10-22 12:27:59 +02:00
Simon 4c1e8819c2 Add engine specific collection APIs (#6977) 2018-10-19 17:46:33 +02:00
Simon 23b8e5c735 Propper commit Sequence Numbers (#6958) (#6964) 2018-10-19 14:19:57 +02:00