1
0
Fork 0
Commit Graph

532 Commits

Author SHA1 Message Date
Jan Christoph Uhde fb839d1c40 stop optimization for nested collects (#9484) 2019-07-29 09:00:03 +02:00
Jan 50f41cec59
added missing function db._transactions(), and equivalent REST API route GET /_api/transaction (#9571) 2019-07-26 16:20:28 +02:00
Matthew Von-Maszewski 91b56a50a3
Feature: Add gzip and encryption to import/export (#9560)
* port of feature-3.4/mv-gzip-export to devel branch

* add explicit namespaces so gcc 6.3.0 would successfully compile

* add conditional cleanup of ENCRYPTION file from another branch

* use _lseek() for Windows build to avoid deprecated warning.

* change from ifdef for lseek variants to TRI_LSEEK.

* force Windows lseek to return Linux expected type.
2019-07-26 07:53:39 -04:00
Jan b62eee034b
fixed a bug in RTRIM implementation, fixed issue #9558 (#9562) 2019-07-25 13:04:46 +02:00
Andrey Abramov d91fb35e39
Bug fix/iresearch trx (#9536)
* add tescase for trx and iresearch

* iresearch transaction integration

* adjust test

* reverse operation for correct index

* remove debug output

* address jsling errors

* address review comments

* forgot something

* added missing fail
2019-07-23 17:44:29 +03:00
Jan b5932c2edf
make TTL indexes behave like other indexes on creation (#9546)
* make TTL indexes behave like other indexes on creation

if a TTL index is already present on a collection, the previous behavior
was to make subsequent calls to `ensureIndex` fail unconditionally with
the error "there can only be one ttl index per collection".

now, we are comparing the attributes of the to-be-created index with the
attributes of the existing TTL index and make it only fail when the
attributes differ. if the attributes are identical, the `ensureIndex`
call succeeds and returns the existing index.
2019-07-23 13:53:58 +02:00
Dronplane 11ef4bf18c Bug fix/internal issue #566 (#9497)
* Implemented TOKENS with array as parameter

* Binary terms encoded as Base64

* Fixed tests for base64 encoded values

* Code cleanup

* Fixed error codes

* Fixed Mac build issue

* Changed error code in case of type error. Code cleanup.

* Reverted MSVC formatting

* added changelog message

* Applied review comments

* Applied review comments

* numeric terms moved to raw pointer.

* Fix *nix build issue
2019-07-22 14:23:35 +03:00
Michael Hackstein cbcf561450
Feature/min replication factor (#9433)
* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* fixed ires tests

* fixed wrong assert

* properly check uint

* repl factor attr check

* adjusted test to be more preciese now

* Fixed race on atomics comparison

* Fixed invalid number type

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* More review fixes
2019-07-19 13:02:28 +02:00
Andrey Abramov 5bc6eac636
bug-fix/internal-issue-#609 (#9464)
* allow to access last operation tick of the transaction

* modify IResearchLink to rely on last commtted tick

* get rid of writing arangosearch markers to WAL

* further implementation

* local changes in iresearch

* add recovery states

* properly handle link creation during recovery

* adjust test cases

* properly handle nested transactions

* ungreylist recovery tests

* add more recovery tests

* do not use transaction to pass recovery tick

* do not store recoveryTick in MMFilesRecoveryState

* adjust tests

* fix mmfiles

* cleanup

* add context validity check

* fix tests

* fix more tests

* ensure subscription is not being released during commit

* address review comment

* final cleanup

* fix crash

* fix tests

* address test failures

* address review comments

* address review comments

* properly set recovery tick even if no recovery happened
2019-07-19 11:10:56 +03:00
Tobias Gödderz 001550ada7 Bug fix/count collect should skip (#9511)
* Make CountCollectExecutor skip rather than fetch

* Cleanup, fix and test for top-level variables

* Added CHANGELOG entry and tests
2019-07-19 10:08:36 +02:00
Wilfried Goesgens 2da0fb356d Feature/cleanup j client api (#9457) 2019-07-19 09:23:24 +02:00
Simon e5507d840f Feature/comm task refactor (#9426) 2019-07-16 09:43:25 +02:00
Andrey Abramov f2ff365206
bug-fix/fix-arangosearch-recovery-mmfiles (#9448)
* start maintenance arangosearch threads after recovery is done

* ensure flush subscription is not being deallocated while in use

* add some tests

* properly determine storage engine

* adjust default view options

* stick to old index meta format in 3.5

* address test failures

* and cluster tests
2019-07-11 11:42:08 +03:00
Wilfried Goesgens 0ad76a8185 fix location - while they work with the server these tests are run in arangosh so they need to be in client/ (#9435)
* fix location - while they work with the server these tests are run in arangosh so they need to be in client/

* rename testsuite

* rename file as its testsuite

* Rename permissions_server to server_permissions
2019-07-10 16:49:12 +03:00
KVS85 8897cd8f31 Fix wrong ars test failure suppression 2019-07-10 15:08:53 +02:00
KVS85 e4d76e087f
Temporarily suppress failure (#9443) 2019-07-10 01:10:56 +03:00
Dronplane 3330d9bb0f Bug fix/internal issue #524 (#9436)
* Added test for inBackground arangosearch link creation

* Updated flag for index check

* Reformatted  too long line

* Fixed comment

* Fixed test run on server-side

* fixed jslint issue

* Added task unregistering in teardown

* Made transaction sync with filesystem
2019-07-10 01:06:25 +03:00
Jan c52f2a8315
refactoring (#9411) 2019-07-09 11:15:52 +02:00
Tobias Gödderz f501e00e9d Bug fix/add shard id to replication client identifier (#9366) 2019-07-08 14:03:42 +02:00
Dronplane 22d5df16e8 Bug fix/internal issue #606 (#9413)
* Fixed stem, norm, text analyzer creation without properties. Added tests.

* Removed non-determenistic test. Same functionality is tested by gtest (IResearchAnalyzerFeatureTest.test_remove)

* Added check for analyzer type existence.

* Made error messages for analyzer creation more specific and readable

* Fixed test

* Applied review suggestions
2019-07-08 13:11:22 +03:00
Jan Christoph Uhde e605c9cd2e add test for oasis statistics (#9412) 2019-07-05 16:11:25 +02:00
Andrey Abramov 595728ec1b
Bug fix/internal issue #605 (#9405)
* fix recovery

* update iresearch

* more fixes

* address review comments
2019-07-05 13:14:37 +03:00
Jan Christoph Uhde 54169cd44f fix cluster-statistics (#9397) 2019-07-05 10:32:46 +02:00
Wilfried Goesgens f9c93c23b1 relax test condition for windows (#9393) 2019-07-04 14:19:24 +02:00
Dronplane b70d7372ef Bug fix/internal issue #586 (#9401)
* Removet lazy creation ob analyzers collection. Test fixes (added explicit creation of analyzers collection for test enviroments)

* Fixed test runs. Removed cluster tests for analyzer DDL

* Analyzers collection name moved to tests common

* Disbaled load_library in normalize calls

* Legacy code cleanup

* added db analyzers collection checks in analyzer creation test to cover functions removed from gtest

* Reverted analyzer properties comparsion to non-utf8 as that must be binary equal
2019-07-04 13:12:06 +03:00
Simon b3ffaff86c Pregel additional test & TSan error fix (#9357) 2019-07-04 09:36:40 +02:00
KVS85 75d7687417 Fix interferention of bob user in test 2019-07-02 12:47:19 +02:00
KVS85 38d714087b
Fix authentication and catch tests in devel (#9374)
* Fixed LDAP authentication issue

* Fixed undefined _system db id in storage engine mock

* Fix auth-analyzer.js to work with LDAP

* Fixed mac warning
2019-07-02 10:11:10 +02:00
Jan fe19b8aaae
don't unfairly favor the edge index when comparing it to more specialized indexes on `_from`/`_to` plus other attributes (#9362) 2019-07-01 16:32:42 +02:00
Jan d842b877a2
actually honor the return value of FollowerInfo::addFollower (#9358) 2019-06-28 18:31:15 +02:00
Dronplane d561221201 Bug fix/fixed analyzer removal from db (#9359)
* Added missing commit. Updated tests to check actual deletion from db.

* Added check for commit result
2019-06-28 17:29:49 +02:00
Jan Christoph Uhde 782cb49105 Feature: Analyzer REST-API Tests (#9091)
* add analyser http API tests

* add more tests

* add tests and lint

* remove print

* add another test

* fix formattig

* add e6 expresseion as key in object

* WIP: add analyzer permissions test

* fix test

* lint

* WIP: try to add analyzer auth-test

* add sleepBeforeStart option

* use basic auth

* fix tests

* change test so it would work if the api was as expected

* fix authentication test

* fit shell client analyzer test

* fix shell-analyzer-rest-api.js

* fix auth-analyzer.js

* fix user-access-right-update-view-arangosearch.js

* remove print statements

* Fixed analyzer parameters for test
2019-06-28 16:21:09 +02:00
Jan 64c8079186
Forbid indexes on _id (#9337) 2019-06-28 11:11:22 +02:00
Andrey Abramov 40cce2cee0
bug-fix/internal-issue-#565.1 (#9257)
* validate analyzer properties, do not return `null` for identity analyzer properties

* fix some tests

* Fixed tests for new analyzer parameters validation. Added explicit test for analyzer parameter validation

* update iresearch, fix analyzer tests

* fix more tests

* fix tests

* store analyzer properties as vpack

* more compilation fixes

* Updated iresearch

* Tests compilation fixed

* Test run fixes

* Test run fixes

* Fix test run

* Fixed all IresearchTests

* Fixed V8Analyzers tests

* Added ngram and delimeter analyzers vpack config

* Added ngram and stem analyzers vpack parsers. Added tests

* Fixed internal issue #593 Fixed build issue

* Fixed tests

* Fixed Gtest tests

* Changed test run to fix Mac failure

* Mac tests debugging

* Fixed jslint errors

* test tracing added

* Taken account for VPackSlice operator== false negatives
2019-06-28 10:49:01 +02:00
Andrey Abramov 33ce94379f
bug-fix/internal-issue-#597 (#9324)
* ensure immutable properties of ArangoSearch are properly handled in cluster

* address jslint warnings
2019-06-25 17:15:07 +02:00
Simon cf7cf0131b Try to fix corruption error (#9258) 2019-06-25 10:18:26 +02:00
jsteemann 086c4e7b82 fix tests 2019-06-25 09:44:01 +02:00
Dronplane 89aa67dcf4 Updated names for text and norm analyzer parameters. Added tests for new parameter names. (#9287) 2019-06-19 09:28:46 +02:00
Tobias Gödderz c854d04864 Feature/limit executor passthrough (#9162)
* Some refactoring to implement helper methods iff necessary

* Updated comments

* Added static assertions

* Re-enabled check for maintainer mode

* Allow pass-through for LimitExecutor. This is not yet working with fullCount correctly!

* Get fullCount before returning the last row

* Adapted fullCount tests: LimitExecutor doesn't lie about the state of the last row anymore

* Use correct row

* Handle LimitExecutor-stats during fetchBlockForPassthrough

* Fixed LimitExecutorTest

* Removed _stats member

* Added more catch tests

* Bugfix for fullStats

* Removed an erroneous assertion

* Implemented LimitExecutor::skipRows

* Avoid name clash of class enum member to please MSVC

* Test MSVC

* Revert "Test MSVC"

This reverts commit d8325d95318bb83b79c8c4cc9f886d36e484c870.

* CREATE_NEW seems to be taken, too. Next try for MSVC

* Built VPack to ItemBlock helpers, and made SingleRowFetcherHelper more general

* Removed ostream operator because MSVC doesn't like it...

* Implemented SingleRowFetcherHelper::fetchBlockForPassthrough, plus a small framework to test executors

* Removed erroneous include

* Moved code in separate files

* Began writing the extended schema of runExecutor

* Added output operator for ExecutorCall

* Fixed removeWaiting

* Fixed expected output

* Added skipRows to runExecutor

* Added two tests

* Allow empty blocks

* Added two more tests

* Added another test

* Built two different parametrized classes for LimitExecutor

* Some cleanup

* Even more tests

* Fixed a bug found by the new tests

* Fix compile error on windows

* Use native matrix representation in tests

* Updated some comments
2019-06-18 17:33:58 +02:00
KVS85 c2bdf80532 Greylist resilience-synchronous-repl-with-arangosearch-view-cluster.js 2019-06-14 15:08:12 +02:00
Markus Pfeiffer a793f72448 Move roundCost function to utilities (#9247) 2019-06-12 16:02:09 +02:00
Heiko 56c969316f Bug fix/gharial edge from and tos verification (#9177)
* prepared new functions for more precise edge verification

* added tests

* more testing

* validate function added to replace edge functionality

* changelog

* applied requested mchackis refactor wishes

* adjustments for update and replace edge validation

* ruby testing adjustments due more prices update and replace edge verification mechanisms

* used wrong flag for the replacement of an edge process

* adjusted last failing tests

* rm print

* added edge to edge tests
2019-06-12 15:12:50 +02:00
Alan Plum a6b49e3a7e Bug fix/buffer redux (#9229)
* Add failing tests

* Add tests for Buffer#slice

* Add tests for Buffer#new

* Skip failing tests

* Implement changes

* Unskip failing tests

This reverts commit 9f8a34bd74e152b7b3158fea79b12e6da8343001.

* Fix implementation

* Remove meaningless test

This test only succeeded because of a broken implementation of the Buffer constructor. It doesn't really do anything useful.

* Add note to CHANGELOG
2019-06-12 15:00:43 +02:00
Andrey Abramov 384c4310b1
Bug fix/internal issue #589 (#9244)
* update iresearch from upstream

* adjust tfidf/bm25 to original behaviour

* update iresearch from upstream

* fix compilation issues

* reduce number of virtual calls

* upgrade iresearch from upstream

* add js test
2019-06-11 17:42:30 +03:00
Jan 59d84e8301 attempt to remove expired WAL files on agents (#9231) 2019-06-09 19:46:45 +02:00
Jan 091abbec77 don't make RocksDB startup fail with 0-byte log files (#9208) 2019-06-07 15:13:56 +02:00
Frank Celler 6b16d1c0a5 Revert "Fix Buffer methods (#9070)"
This reverts commit 6c96e5dd24.
2019-06-07 10:18:58 +02:00
Kaveh Vahedipour 8a511934df [devel] fix ttl handling for object assigments (#9207)
* fix ttl handling for object assigments
* completed test handling
* CHANGELOG.
2019-06-06 15:48:40 +02:00
Jan 074397d367
allow "stream" attribute on top level (#9195) 2019-06-05 15:48:58 +02:00
Jan Christoph Uhde 12ec86c2b1 add shardKeyAttribute to pregel start parameters (#9149) 2019-06-05 14:25:47 +02:00
jsteemann 0359a0acda fixed jslint warning 2019-06-05 10:56:53 +02:00
Jan Christoph Uhde 7ac857784f add endpoint check to reconnect (#9196) 2019-06-05 10:53:57 +02:00
Andrey Abramov b2a2bd100e
Bug fix/internal issue #590 (#9176)
* disallow creation of analyzer by a full name

* fix tests
2019-06-04 00:28:20 +03:00
Dronplane 1e24c79ca3 Bug fix/internal issue #565 (#9154)
* Merged latest iresearch. Removed creation of build-in analyzers from tests

* Updated tests for new analyzer parameter names

* Merged fix for format from iresearch

* Apply suggestions from code review. Corrected typo in variable name.

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* backport changes from IResearch upstream

* fix compilation issues

* backport fix from IResarch upstream

* Fixed analyzer parameter name in tests
2019-06-03 18:20:23 +03:00
Alan Plum 6c96e5dd24 Fix Buffer methods (#9070)
* Add failing tests

* Implement changes

* Add note to CHANGELOG

* Fix test failure
2019-06-03 17:07:52 +02:00
Max Neunhöffer 06054dfb64
Fix yet another agency test with clientid. (#9150) 2019-05-31 17:23:42 +02:00
Dan Larkin-York 44a413a9af Miscellaneous fixes for named indices (#9100) 2019-05-31 17:00:56 +02:00
Tobias Gödderz 79cd45f89c Wait for replication before inserting documents (#9151)
* Wait for replication before inserting documents

Also, increased some timeouts and fixed a log message

* Fixed some log levels and a log message

* Removed repair-distribute-shards-like-spec from greylisted tests
2019-05-31 16:09:20 +03:00
Tobias Gödderz 90abe3ba93 Increased timeouts, added a little more helpful error messages for assertions (#9135) 2019-05-30 04:30:39 +02:00
Wilfried Goesgens ec0cc82bf6 test files and their content were swapped (#9136) 2019-05-30 04:29:37 +02:00
Kaveh Vahedipour 773f3c8422 [devel] fix state clientlookuptable (#9066) 2019-05-30 04:24:46 +02:00
Jan d67513607d
fixed issue #9106 (#9116) 2019-05-29 18:27:38 +02:00
Markus Pfeiffer de9ef00896 Round estimatedCost in some tests (#9126)
Otherise slightly different floating point results in estimatedCosts cause
tests to fail.
2019-05-29 15:29:52 +02:00
Simon a5a72ef6d4 port Pregel segmented buffers (#9112) 2019-05-28 18:23:20 +02:00
Jan 033f411420
Bug fix/issues 27052019 (#9117) 2019-05-28 12:08:25 +02:00
Simon 27112a3486 Bug fix/jobs indexes (#9107) 2019-05-28 09:38:14 +02:00
Tobias Gödderz 5da9fcb0ce Bugfix of DFS path uniqueness implementation (#9041)
* Began work on generic traversal testing framework

* Translate shard ids to smart attributes

* Minor fixes

* Added a TODO note

* fixed tests

* cleanup

* first tests inside new traversal framework working

* added single server diamond tests

* Enable sharding in aql-graph-traversal-generic-graphs.js

* Tried to fix smart sharding

* Fixed smart graph name clashes, some cleanup

* Added meta tests

* Added TODO note

* Fixed single server

* Added a BFS order check

* Renamed test

* added more dfs and bfs tests for the openDiamond graph

* Added new graph "largeBinTree"

* Fix Map usage

* added smallCircle graph, added three dfs tests

* added smallCircle graph, added three dfs tests

* more smallCircle dfs tests

* Fixed largeBinTree shardings, and long graph names

* Fix long suffixes

* Provided names for shardings

* Added a log message

* more bfs smallCircle tests

* more bfs smallCircle tests - global

* smallCircle picture

* added completeGraph to graph test suite

* added more dfs tests for the complete graph up to depth 3

* Removed duplicate function definitions

* added depth 2 dfs complete graph test, added missing test to the exports

* Added test for largeBinTree

* added three path graphs, easy, advanced and more advanced

* Added a working check for bfs with global uniqueness

* Fixed usage of bfs checks

* Fixed expected input for two tests

* Added some static assertions to largeBinTree

* Oops

* added more advancedPath dfs tests

* better formatting of the visual graph

* more dfs advanced graph tests

* Implemented all easy path test combinations

* Fixed testAqlGraphQuery

* Fix in bfs check error reports

* Removed duplicate test

* Added advancedPath BFS tests

* Updated comments

* Fixed new tests

* Fixed graph description

* added more complete graph bfs tests

* added more complete graph bfs tests

* added testCompleteGraphBfsUniqueEdgesNoneD1 and testCompleteGraphBfsUniqueEdgesNoneD2

* added testCompleteGraphBfsUniqueVerticesUniqueEdgesPathD3 test

* added a few more complete graph bfs checks

* Extended a comment
2019-05-27 12:57:02 +02:00
Dronplane 3d697d8584 Added filtering analyzers before adding them to view link - duplica… (#9094)
* Added filtering analyzers before adding them to view link   - duplicates are not allowed any more

* Updated test testAddDuplicateAnalyzers - now it tests analyzers with same names but in different databases
2019-05-27 13:20:51 +03:00
KVS85 8d1885d0cb Bug fix/greylist timecritical and nondeterministic (#9047)
* Mark all timecritical and nondeterministic as greylisted

* Mark description to Greylist.txt

* Fix typo
2019-05-24 11:52:15 +02:00
Jan de717924ac
fix crash (#9057) 2019-05-21 18:19:11 +02:00
Jan 6cea7528fb
fix result of AQL function `MIN` in case the first input element is (#9053)
`null`
2019-05-21 15:43:33 +02:00
Alan Plum e8b7e1860a Bug fix/unbreak foxx security (#9042)
* Unbreak aardvark when --foxx.api false

The easiest way to disable the aardvark foxx routes is to just skip the
import and mount altogether. The previous approach caused the service to
throw an error because the module doesn't actually export anything.
There's no need to prevent the export if we just don't import the module
to begin with.

* Add an error code for service api disabled

* Throw an actual error when accessing Foxx API routes

* Throw an error when using legacy Foxx API

There's no easy way to use a catch-all here so disable them one by one. No new routes will be added to this API and it'll go away in 4.0 so there's no risk of forgetting to add the throw-check in new routes.

* Render subnav without delay

The subnav doesn't need the perm check results but we need the subnav to exist so we can populate it in other parts of the app. This fixes the subnav not showing up when reloading certain views (e.g. install service via upload).

* Hide ONLY the Foxx store when it is disabled

We still want to allow installing and upgrading/replacing services. We just don't want to show the store.

* Hide Foxx services in frontend if Foxx API is disabled

* Build aardvark

* fixed typo in disable check, fixed tests
2019-05-21 14:45:23 +02:00
jsteemann c640e07ada next attempt for fixing unreliable test 2019-05-20 11:45:25 +02:00
Lars Maier 4fc2790863 [devel] Direct Exec Scheduler (#9004) 2019-05-20 11:38:57 +02:00
Frank Celler 0bb8ec2e06 Revert "test is broken"
This reverts commit 469b94235b.
2019-05-18 09:55:01 +02:00
Frank Celler de566b405d Revert "fixed typo in disable check, fixed tests"
This reverts commit 068bbc9143.
2019-05-18 09:54:49 +02:00
Frank Celler 068bbc9143 fixed typo in disable check, fixed tests 2019-05-18 09:42:11 +02:00
Frank Celler 469b94235b test is broken 2019-05-18 08:04:49 +02:00
Simon 93b2e64f37 Port pregel fixes (#9022) 2019-05-17 16:32:58 +02:00
Jan Christoph Uhde 970f732dec add exceptions for tokens function. (#8984)
* Provide users of the TOKENS function with errors.

* remove newlines

* fix compare

* fix as asked for

* improve fix
2019-05-16 22:12:43 +03:00
Tobias Gödderz c9fa5fba11 Additional AQL skipSome tests for IndexExecutor (#8938)
* Began work on additional index skipSome tests

* Added more index skip tests
2019-05-16 11:30:24 +02:00
Jan 0ea4d35a17
fix failing test (#9006) 2019-05-15 22:58:14 +02:00
KVS85 d727cc7f9b Greylist tests/js/server/recovery/write-throttling-mmfiles.js 2019-05-15 12:27:04 +02:00
Tobias Gödderz 82f5978f30 Bugfix: Do not call getSome on remote after DONE (#8978)
* Bugfix: Do not call getSome on remote after DONE

* Added profiler tests for Sorting- and UnsortingGather

* Fixed jslint error
2019-05-15 09:13:44 +02:00
Vasiliy 12bd4c785b issue 555.1.1: issue 555.1.1: try loading previous data store snapshots if checkpoint file unavailable, ehnace error message, address some warnings (#8975) 2019-05-13 18:33:44 +03:00
Andrey Abramov 404bcc8b0c
switch to tiered consolidation by default (#8981)
* switch to tiered consolidation by default

* fix copy/paste error

* fix tests

* fix tests
2019-05-13 18:25:46 +03:00
Andrey Abramov 4e12fdb30b
ArangoSearch sorted indexes (#8888)
* move IResearchViewSort into separate unit

* ensure sort is propagated to IResearchLink

* update iresearch and extend some tests

* implement insertion of sorted values, store value comparator in link

* add tests for VPackComparer

* minor cleanup

* fix issue with sort definiton propagation

* propagate error message from index factory

* add integration tests for sorted indexes, single server

* initial implementation of IResearchViewMergeExecutor

* move iresearch::VPackComparer to separate translation unit

* store collection in IndexReadBuffer for IResearchViewMergeExecutor

* set primary sort from optimizer rule

* add some tests for sorted view case

* extend IResearchViewNode tests

* simplify primary sort definition

* pretty print order condition covered by arangosearch in query explanation

* fix compilation with g++-8

* address catch test failures

* fix tests

* adjust arangosearch optimizer rule to optimize sort in case of cluster

* make tests for sorted indexes work in cluster too

* fix tests

* extend js tests for sorted indices

* fix keyword in query explainer

* ensure sort-limit rule works with views

* backport ngram analyzer fix from upstream

* address jslint errors

* modify IResearchViewSort to support boolean direction specification format

* add some tests

* add tests

* add skip functionality

* extend tests

* add more catch tests

* inline some methods

* address jslint errors

* extend catch tests to reproduce the failure

* add some tests for check fullcount

* add more tests

* extend tests

* temporarily comment out broken tests

* extend js tests

* address review comments

* address jslint errors

* update iresearch
2019-05-11 17:58:56 +03:00
Simon 6075fec35e Forward port some changes (#8949) 2019-05-09 19:42:06 +02:00
jsteemann 60a19e0ab3 fix jslint warning 2019-05-09 19:11:21 +02:00
Jan Christoph Uhde 192ecd2f75 add test for unicode paths (#8909) 2019-05-09 19:00:48 +02:00
Jan c1d48c0fef
honor index hints in optimizer rule (#8940) 2019-05-09 18:47:36 +02:00
KVS85 2a66b53591 Greylist tests/js/client/shell/shell-foxx-paths-spec.js 2019-05-08 17:44:19 +02:00
KVS85 7847e8e9d8 Small fix of test setUp 2019-05-08 15:54:36 +02:00
Heiko 58db1023f3 Feature/aql skip some (#8771)
* testing skipSome implementation, default and passThrough

* added simple implementation for default and passthrough skipsome methods

* added skipRows function to enumerate collection executor

* added tests for the enumerate collection skipRows function

* implemented and tested enumeratecollection skipSome

* skipSome tests

* fixed a test, prepared iresearch view exec for skipping

* rm logs

* gcc bug workaround

* changed to original test code as it has been before

* iresearch skipping, added ires skip test

* added index executor skipRows

* added skip blockfetcher, quick exit index

* test

* const dummy and singlerowfetcher skip

* forgot return value

* input wrong initialized

* trying to remove dynamic cast and solve with a different approach

* cleanup

* const skip

* jslint

* Handle skipSome of subqueries correctly

* Removed unused code

* Removed unused member

* Simplified skip variants and enabled IndexExecutor skipping

* A little cleanup, fixed DependencyProxy::skipSome

* Reverted test change

* Tried to make testLimitBlock3 clearer

* Extended test suite

* Bugfix

* Added stats when skipping and fixed a few other things

* Bugfixes

* Moar bugfixes

* Update arangod/Aql/IResearchViewExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* applied requested changes

* Fixed a bug in LimitExecutor::expectedNumberOfRows()

* Fix skipSome in catch test RowFetcherHelper

* Fixed a bug in the tests

* Two bugfixes in LimitExecutor::expectedNumberOfRows

* Avoid passing large batch sizes from skipSome to getSome

* Fixed Windows compile errors

* Fixed a skip bug with WAITING in unsorting gather blocks

* Make aql-skipping find some cluster bugs, like the last commit

* Bugfix and additional asserts

* Fixed skipSome counting in IndexExecutor

* Resolved merge conflicts
2019-05-08 15:02:43 +02:00
Simon 0502a97abb forwardport changes from 3.4 (#8894) 2019-05-08 14:34:25 +02:00
jsteemann d50eeb0410 fix test 2019-05-08 10:24:54 +02:00
KVS85 068270fff9 Small speedup of setup in dump suite 2019-05-07 23:49:17 +02:00
Jan Christoph Uhde f3a2eaf6d9 Bug fix/permission tests (#8890) 2019-05-07 15:07:59 +02:00
Heiko 4837c7b864 Bug fix/collect aggregate undefined (#8853)
* allow 3.4 behaviour regarding aggregate collect and undefined variable

* added test to check undefined var

* rm print

* remove duplicate doCopyRow

* wrongly removed due merge

* initialize _allowSourceRowUninitialized always as false as default

* try to fix another case

* added missing parenthesis

* optimized if, also added compiler hint
2019-05-07 15:05:49 +02:00
Jan eed6729789
fix an issue, add tests (#8920) 2019-05-07 12:15:43 +02:00
jsteemann 8b245d25a4 remove print statement 2019-05-06 17:54:51 +02:00
KVS85 62d207edd2 Add debug print of installed service path 2019-05-06 16:50:27 +02:00
Simon d737c19302 address issues with active failover (#8908) 2019-05-06 15:52:49 +02:00
KVS85 9364f56594 Greylist tests/js/server/recovery/view-arangosearch-link-populate-no-flushthread.js 2019-05-06 14:33:32 +02:00
KVS85 0ef3056d65 Disable resilience_move_view test till it's reworked 2019-05-03 16:50:47 +02:00
Frank Celler 7c79e7359b greylisted view-arangosearch-link-populate-transaction-grey.js 2019-05-03 11:30:03 +02:00
Wilfried Goesgens 7c7f69d473 greylist view-arangosearch-link-populate-no-callbacks.js (#8885) 2019-05-02 15:07:52 +03:00
KVS85 5406cb7447 Add forgotten greylisted files 2019-04-30 22:18:48 +02:00
KVS85 ed14f3b650 Greylist tests/js/client/resilience/foxxmaster.js 2019-04-30 16:03:44 +02:00
KVS85 9f90a8347b Greylist tests/js/server/recovery/view-arangosearch-link-populate-transaction-no-flushthread.js 2019-04-30 16:01:00 +02:00
Simon 2f1cfa40e0 Fix tick overflow (#8840) 2019-04-26 11:07:04 +02:00
Tobias Gödderz 6dfb9995b4 Feature/aql by line cleanup (#8848)
* Removed useless parameters of buildCallback

* Renamed produceRow to produceRows and adapted a comment

* Renamed BlockFetcher to DependencyProxy

* Applied git-clang-format
2019-04-25 15:51:11 +02:00
Jan Christoph Uhde 677a79026c Foxx Security (#8845) 2019-04-25 09:56:29 +02:00
KVS85 e6b73726fd Remove several tests from greylisting 2019-04-24 21:38:49 +02:00
KVS85 3c972ef695 Fix db object visibility for ArangoSearch tests (#8819) 2019-04-23 09:31:29 +02:00
Markus Pfeiffer 1f2f5ec983 K_SHORTEST_PATHS queries only support one variable in FOR (#8784)
* K_SHORTEST_PATHS queries only support one variable in FOR

 * catch if the user gives more than one variable in grammar.y
 * also give a correct error message

* Test code that catches too many variables for K_SHORTEST_PATHS
2019-04-23 08:48:16 +02:00
Frank Celler 17fb0b795f removed print 2019-04-22 20:54:32 +02:00
Simon 569198a089 Abort el-cheapo transactions if servers fail (#8799) 2019-04-22 19:31:24 +02:00
KVS85 9027cb2418 Remove nondeterministic mark from 2 arangosearch recovery tests 2019-04-22 17:40:06 +02:00
KVS85 31ac3deee7 Remove moving-shards-cluster.js from greylisting 2019-04-22 17:10:20 +02:00
KVS85 7dc2ea0535 Greylist more frequently failed tests 2019-04-22 16:48:06 +02:00
KVS85 6911ee47f4 Remove view-arangosearch-link-populate-transaction-no-flushthread from greylisting 2019-04-22 16:01:19 +02:00
Frank Celler 499e97c78b added missing result 2019-04-21 20:24:37 +02:00
Vasiliy 40ebdd0b58 issue 545.1.1: output short analyzer prefixes for user definitions (#8814) 2019-04-21 15:10:28 +02:00
Frank Celler ed83084d5a splitted aql-view-arangosearch 2019-04-21 13:04:27 +02:00
Vasiliy 0375218ca2 issue 541.1: fix incorrect test filename (#8810) 2019-04-20 23:19:23 +03:00
KVS85 9cfaf511c8 Fix resilience_move location 2019-04-20 16:51:59 +02:00
KVS85 878415ef8e Change resilience and replication testsuits layouts required for new tests and greylisting 2019-04-19 01:14:34 +02:00
Alan Plum 8692f19e99 Handle extra args to db._query when using aql templates (#8789) 2019-04-18 10:50:46 +02:00
Markus Pfeiffer 304e8f861d Fix k Shortest Paths with weights, reactivate cluster tests (#8761)
* Make find in ShortestPathPriorityQueue const

* Fix k Shortest Paths with weights, reactivate cluster tests
2019-04-17 15:10:59 +02:00
Alan Plum 40aff3fdcd Upgrade JS deps (#8773)
* Make jsunity hash function avoid recursion

This guards against traversals of recursive data structures causing a stack overflow. This is crucial when using assertEqual with unknown inputs.

* Update deps

accepts: 1.3.4 -> 1.3.5
ansi_up: 2.0.2 -> 4.0.3
content-disposition: 0.5.2 -> 0.5.3
dedent: 0.6.0 -> 0.7.0
error-stack-parser: 1.3.6 -> 2.0.2
eslint: 2.13.1 -> 5.16.0
eslint-config-semistandard: 6.0.2 -> removed
eslint-config-standard: 5.3.1 -> removed
eslint-plugin-promise: 1.3.2 -> removed
eslint-plugin-standard: 1.3.2 -> removed
highlight.js: 9.12.0 -> 9.15.6
http-errors: 1.6.2 -> 1.7.2
iconv-lite: 0.4.19 -> 0.4.24
joi: 9.2.0 -> 14.3.1
joi-to-json-schema: 2.3.0 -> 4.0.1
js-yaml: 3.10.0 -> 3.13.1
marked: 0.3.9 -> 0.6.2
mime-types: 2.1.12 -> 2.1.22
mocha: 2.5.3 -> 6.1.3
qs: 6.5.1 -> 6.7.0
semver: 5.4.1 -> 6.0.0
statuses: 1.4.0 -> 1.5.0
timezone: 1.0.13 -> 1.0.22
type-is: 1.6.15 -> 1.6.16
underscore: 1.8.3 -> 1.9.1

* Inline eslint semistandard config

The eslint standard config has become extremely opinionated and incompatible with the existing coding style. This subset matches the existing coding style most closely. In the future we should migrate to an autoformatter like prettier and avoid this problem altogether.

* Linting

* Fix mocha runner

* New joi has additional property
2019-04-17 13:36:26 +02:00
Vasiliy 1a22d1360c issue 526.9.1: implement swagger interface, add documentation (#8730)
* issue 526.9.1: implement swagger interface, add documentation

* address review comments

* add ngram

* Formatting

* Move REST description to new Analyzers top chapter in HTTP book

* Missed a DocuBlock

* Add Analyzers chapter to Manual SUMMARY.md

* Move REST API description back to Manual

Headlines were broken

* Add n-gram example
2019-04-16 18:54:30 +03:00
Jan 7c0c01c401
use the primary index when full-scanning a collection (#8548) 2019-04-16 16:50:03 +02:00
Jan 1d175f446d
do not use the TTL index for queries which it does not support (#8766) 2019-04-15 18:55:19 +02:00
Markus Pfeiffer a14386267a k Shortest Paths (#8715)
* Add k-shortest-paths finder
2019-04-12 21:58:18 +02:00
Alan Plum b76f8adc0f Add SyntheticRequest#auth (#8750)
* Add SyntheticRequest#auth

* Fix tests
2019-04-12 21:18:35 +02:00
Alan Plum 3007c928aa Add collection.documentId method (#8749) 2019-04-12 19:53:01 +02:00
jsteemann ca8b81b52b make test more reliable 2019-04-12 13:09:10 +02:00
Jan 4f62dd8dae
added AQL function DECODE_REV (#8606) 2019-04-11 15:59:04 +02:00
Simon 937d743ba6 Bug fix/pregel stuff (#8733) 2019-04-11 15:58:28 +02:00
Frank Celler 9c4af6cb7d greylisted view-arangosearch-link-populate-transaction-no-flushthread.js 2019-04-10 12:46:47 +02:00
Frank Celler 6ec20cddee greylisted active-failover/basic.js 2019-04-10 12:44:54 +02:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Frank Celler 04d979e26e greylisted shell-foxx-api-spec 2019-04-10 12:20:21 +02:00
Vasiliy 030637c1c7 issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields (#8689)
* issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields

* address test failures

* remove debug logging
2019-04-10 13:10:25 +03:00
Simon 4e80e1b351 Simplify recovery code (#8699) 2019-04-09 21:48:04 +02:00
Jan c7500c0c95
finally fix AQL error messages in all contexts, so that they are usable (#8706) 2019-04-09 17:46:23 +02:00
Frank Celler b7c5e620fe still failing test 2019-04-09 14:57:31 +02:00
Frank Celler 3e26f66aa8 added pregel to greylist and raised log level 2019-04-09 14:45:08 +02:00
Jan 5897baa984
added db.<collection>.getResponsibleShard() (#8683) 2019-04-08 16:10:45 +02:00
Frank Celler 44b25e2e88 greylisted because it takes too long 2019-04-08 10:36:15 +02:00
Vasiliy 0410588723 issue 526.7.1: remove deprecated functions, update tests to set up required analyzers (#8688)
* issue 526.7.1: remove deprecated functions, update tests to set up required analyzers

* address merge issues

* address more merge issues
2019-04-05 18:10:42 +03:00
Jan 1160961af8
added debugging for replication-fuzz fails (#8674) 2019-04-04 10:20:49 +02:00
Jan 98aaa82048
added log output for pregel job status (#8673) 2019-04-03 17:17:20 +02:00
Max Neunhöffer 02281d3be4
Handle InitDone correctly. (#8552)
* precondition plan / version in compaction / store TTL removal independent of local _ttl set
* Agency init loops break when shutting down.
* assertion failures in store on restarting following agents
* Minor porting fixes from 3.4
2019-04-01 17:01:05 +02:00
jsteemann a5232f60aa fix dump-multiple with mmfiles in cluster 2019-04-01 12:15:17 +02:00
Jan 14720b85be
add AQL functions CRC32 and FNV64 (#8600) 2019-03-28 03:06:33 +01:00
Andrey Abramov 930b09cd93
[3.4] bug-fix/issue-#8294 (#8430) (#8585)
* [3.4] bug-fix/issue-#8294 (#8430)

* fix invalid optimizations for multi-valued attributes

* fix broken optimizations for multivalued attributes

* adjust tests

* add `IN_RANGE` function

* add tests

* add some shallow integration tests

* fix optimization for IN operator

# Conflicts:
#	arangod/IResearch/IResearchFeature.cpp
#	arangod/IResearch/IResearchViewOptimizerRules.cpp
#	tests/IResearch/IResearchFilterBoolean-test.cpp

* fix compilation errors

* fix another compilation issue

* address compilation errors

* fix tests
2019-03-27 18:33:16 +03:00
jsteemann e4594b72a8 revert to old behavior 2019-03-27 14:20:31 +01:00
Wilfried Goesgens c5587f1c30 fix enterprise tests - we mustn't restore smart collections over exising collections (#8581)
* fix enterprise tests - we mustn't restore smart collections over existing collections

* un-blacklist test
2019-03-27 09:08:38 +01:00
KVS85 4ea1675e74 Allow replication suite filtering (#8586) 2019-03-26 20:25:14 +03:00
Jan 207c978d1e
Bug fix/move a few actions from js to cpp (#8549) 2019-03-26 10:02:21 +01:00
KVS85 0212d56fb3 Greylist tests/js/server/replication/replication-fuzz.js (#8557) 2019-03-25 18:09:29 +01:00
Simon bcc26926de Support Transactions in RestCursorHandler (#8539) 2019-03-25 17:04:11 +01:00
Wilfried Goesgens 0634b24dde add tests for dump/restore of multiple databases (#8530) 2019-03-25 15:52:37 +01:00
Jan Christoph Uhde 5d5b77abb8 add ModificationExecutor (#8179)
All modification operator as Executors
2019-03-24 20:48:38 +01:00
Simon 329acea21c Add more El-Cheapo client tests (#8533) 2019-03-23 22:09:28 +01:00
Jan cfd3418115
microoptimizations (#8519) 2019-03-22 16:53:21 +01:00
jsteemann 868b58c7d7 reformatting, removed potentially undefined behavior 2019-03-22 13:03:37 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan 962a080399
attempt to make replication fuzz tests more stable (#8497) 2019-03-21 15:00:27 +01:00
jsteemann fbecc9df34 attempt to fix test 2019-03-21 09:27:05 +01:00
Heiko ce51797609 Feature/aql hashed collect (#8337)
Implementation of Hashed Collect Executor
2019-03-21 07:19:26 +01:00
Kaveh Vahedipour 5038dfe685 supervision must not copy snapshots into jobs (#8425)
* supervision must not copy snapshots into jobs
* CHANGELOG.
2019-03-20 17:07:54 +01:00
KVS85 84abf37435 Add testTemplateStringBuilderEmptyInlines test for https://github.com/arangodb/arangodb/pull/8141 (#8206) 2019-03-20 15:56:40 +01:00
Dan Larkin-York 04ef4595e3 Fix replication issue with index ID/name conflicts (#8448) 2019-03-20 14:06:02 +01:00
Dan Larkin-York 2eadab33e7 Index hints (#8431) 2019-03-19 09:14:18 +01:00
Vasiliy 29b941a677 issue 526.2.1: add upgrade step for legacy static analyzers (#8417) 2019-03-18 23:22:23 +03:00
Kaveh Vahedipour 888a20bcf7 also compaction step size test grey listed (#8435) 2019-03-18 11:17:14 +01:00
Kaveh Vahedipour a2ca779e71 log compaction excluded for no (#8423) 2019-03-14 17:05:28 +01:00
Wilfried Goesgens d88bc376a6 When updating documents with array indices and unique values we mustn't disable indexing in rocksdb (#8404) 2019-03-14 13:35:38 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
jsteemann cdb4b46554 added debugging data for testing 2019-03-13 18:08:45 +01:00
Jan dd4938598c
use requireFromPresent and barriers in more cases (#8398) 2019-03-13 15:16:22 +01:00
jsteemann 3e75acf67f fixed wrong usage of JavaScript sort function 2019-03-11 16:33:25 +01:00
Jan 12e11a5197
port of replication improvements from 3.4 (#8308) 2019-03-11 13:37:18 +01:00
Heiko def7b613db code now fits to the documentation of the arangoUser object inside th… (#8352)
* code now fits to the documentation of the arangoUser object inside the foxx request object

* more tests
2019-03-11 12:43:59 +01:00
Jan 85227e00e4
make geo-index-optimizer rule work with multiple loops (#8353) 2019-03-11 12:26:07 +01:00
jsteemann aec1abd673 try to fix test 2019-03-08 12:32:51 +01:00
Jan 9f9426e36c
Bug fix/allow creating foxx queues inside transactions (#8339)
* allow creating a foxx queue from within a transaction

* added tests
2019-03-07 14:28:39 +01:00
Vasiliy b1e30937bb issue 520.1.1: decrease link commit interval to 1 second, add workaround for MMFiles not recovering document insertions via WAL (#8313) 2019-03-05 14:50:54 +03:00
Kaveh Vahedipour 68178ba165 [devel] supervision bug fix backports (#8314)
* back ports for supervision fixes from 3.4 part 1

* back ports for supervision fixes from 3.4 part 2
2019-03-04 19:27:24 +01:00
Jan 9855dbad29
fix test (#8307) 2019-03-04 13:31:29 +01:00
jsteemann 9ee60c3b38 attempt to fix test failure 2019-03-04 09:54:34 +01:00
Jan 08a6561adc
Bug fix/small issues 21022019 (#8229) 2019-03-01 17:20:10 +01:00
Michael Hackstein 6e8d43b2da
Feature/aql traversal prune (#8255)
Allow PRUNE in the AQL Traverser.
2019-02-27 11:43:06 +01:00
Andrey Abramov 804c6e21e3
backport: Fix bug #8213 (#8273)
* backport: Fix bug #8213

* rename variable

* fix compilation error

* fix tests
2019-02-26 23:47:15 +03:00
Jan d46acbb156
Bug fix/make ttl index work with datestrings too (#8208) 2019-02-22 16:09:51 +01:00
KVS85 12c2739f04 Greylist ./tests/js/server/resilience/moving-shards-with-arangosearch-view-grey-cluster.js (#8227) 2019-02-21 16:50:35 +01:00
Heiko 4557dfc0d5 do not allow edge definitions with empty from or to arrays, added als… (#8198) 2019-02-20 18:32:33 +01:00