1
0
Fork 0
Commit Graph

256 Commits

Author SHA1 Message Date
Jan 00bcc4954c
AQL date functions improvements (#9714) 2019-08-22 12:50:08 +02:00
Dronplane 49dfb0a96e Bug fix/issue #9612 (#9764)
* Fixed ViewExecutionNode retrieval with deleted documents present in view

* Ported solution from 3.4 branch

* Changed index store in collection from vector to set. To make reversable indexes always last to execute

* Fixed re-enter hung

* Index storage fix

* Made index order deterministic

* Fix Mac build

* Added tests for index reversal

* Fixed Mac build

* Code cleanup

* Some cleanup

* Removed some redundand copy constructor calls

* Applied review comments

* Applied review comments
2019-08-20 19:34:23 +03:00
Jan fda4eea43f
make index selection more deterministic (#9735)
* make index selection more deterministic

* serialize indexes used by traversal with their estimates

* serialize selectivity estimates for shortest path nodes too

* fix assertion that doesn't hold true in unit tests

* fix test
2019-08-20 14:45:13 +02:00
Lars Maier 492057d4f4 [devel] Resign Leadership (#9427)
* First version of ResignLeadership Job.
* Port some performance optimizations from CleanOutServerJob.
* Draft of resigning leadership on shutdown.
* Moved code into Maintenance Feature. Fixed beginShutdown.
2019-08-07 15:02:17 +02:00
Lars Maier ed496fe5dd Feature/hotbackup devel (#9495)
Hotbackup
2019-08-02 11:39:46 +02:00
Jan 7d829de89e added internal function getResponsibleServers() (#9604)
* added internal function getResponsibleServers()

* forgot to commit

* honor review comments

* Update arangod/Cluster/ClusterInfo.cpp

Potentially Fixed Unique logID usage. (let Jenkins test it)
2019-07-31 10:18:37 +02:00
Jan Christoph Uhde fb839d1c40 stop optimization for nested collects (#9484) 2019-07-29 09:00:03 +02:00
Matthew Von-Maszewski 91b56a50a3
Feature: Add gzip and encryption to import/export (#9560)
* port of feature-3.4/mv-gzip-export to devel branch

* add explicit namespaces so gcc 6.3.0 would successfully compile

* add conditional cleanup of ENCRYPTION file from another branch

* use _lseek() for Windows build to avoid deprecated warning.

* change from ifdef for lseek variants to TRI_LSEEK.

* force Windows lseek to return Linux expected type.
2019-07-26 07:53:39 -04:00
Jan b62eee034b
fixed a bug in RTRIM implementation, fixed issue #9558 (#9562) 2019-07-25 13:04:46 +02:00
Andrey Abramov d91fb35e39
Bug fix/iresearch trx (#9536)
* add tescase for trx and iresearch

* iresearch transaction integration

* adjust test

* reverse operation for correct index

* remove debug output

* address jsling errors

* address review comments

* forgot something

* added missing fail
2019-07-23 17:44:29 +03:00
Michael Hackstein cbcf561450
Feature/min replication factor (#9433)
* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* fixed ires tests

* fixed wrong assert

* properly check uint

* repl factor attr check

* adjusted test to be more preciese now

* Fixed race on atomics comparison

* Fixed invalid number type

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* More review fixes
2019-07-19 13:02:28 +02:00
Andrey Abramov 5bc6eac636
bug-fix/internal-issue-#609 (#9464)
* allow to access last operation tick of the transaction

* modify IResearchLink to rely on last commtted tick

* get rid of writing arangosearch markers to WAL

* further implementation

* local changes in iresearch

* add recovery states

* properly handle link creation during recovery

* adjust test cases

* properly handle nested transactions

* ungreylist recovery tests

* add more recovery tests

* do not use transaction to pass recovery tick

* do not store recoveryTick in MMFilesRecoveryState

* adjust tests

* fix mmfiles

* cleanup

* add context validity check

* fix tests

* fix more tests

* ensure subscription is not being released during commit

* address review comment

* final cleanup

* fix crash

* fix tests

* address test failures

* address review comments

* address review comments

* properly set recovery tick even if no recovery happened
2019-07-19 11:10:56 +03:00
Tobias Gödderz 001550ada7 Bug fix/count collect should skip (#9511)
* Make CountCollectExecutor skip rather than fetch

* Cleanup, fix and test for top-level variables

* Added CHANGELOG entry and tests
2019-07-19 10:08:36 +02:00
Andrey Abramov f2ff365206
bug-fix/fix-arangosearch-recovery-mmfiles (#9448)
* start maintenance arangosearch threads after recovery is done

* ensure flush subscription is not being deallocated while in use

* add some tests

* properly determine storage engine

* adjust default view options

* stick to old index meta format in 3.5

* address test failures

* and cluster tests
2019-07-11 11:42:08 +03:00
Wilfried Goesgens 0ad76a8185 fix location - while they work with the server these tests are run in arangosh so they need to be in client/ (#9435)
* fix location - while they work with the server these tests are run in arangosh so they need to be in client/

* rename testsuite

* rename file as its testsuite

* Rename permissions_server to server_permissions
2019-07-10 16:49:12 +03:00
KVS85 8897cd8f31 Fix wrong ars test failure suppression 2019-07-10 15:08:53 +02:00
KVS85 e4d76e087f
Temporarily suppress failure (#9443) 2019-07-10 01:10:56 +03:00
Jan c52f2a8315
refactoring (#9411) 2019-07-09 11:15:52 +02:00
Tobias Gödderz f501e00e9d Bug fix/add shard id to replication client identifier (#9366) 2019-07-08 14:03:42 +02:00
Jan Christoph Uhde e605c9cd2e add test for oasis statistics (#9412) 2019-07-05 16:11:25 +02:00
Andrey Abramov 595728ec1b
Bug fix/internal issue #605 (#9405)
* fix recovery

* update iresearch

* more fixes

* address review comments
2019-07-05 13:14:37 +03:00
Jan Christoph Uhde 54169cd44f fix cluster-statistics (#9397) 2019-07-05 10:32:46 +02:00
Simon b3ffaff86c Pregel additional test & TSan error fix (#9357) 2019-07-04 09:36:40 +02:00
Jan d842b877a2
actually honor the return value of FollowerInfo::addFollower (#9358) 2019-06-28 18:31:15 +02:00
Simon cf7cf0131b Try to fix corruption error (#9258) 2019-06-25 10:18:26 +02:00
jsteemann 086c4e7b82 fix tests 2019-06-25 09:44:01 +02:00
Tobias Gödderz c854d04864 Feature/limit executor passthrough (#9162)
* Some refactoring to implement helper methods iff necessary

* Updated comments

* Added static assertions

* Re-enabled check for maintainer mode

* Allow pass-through for LimitExecutor. This is not yet working with fullCount correctly!

* Get fullCount before returning the last row

* Adapted fullCount tests: LimitExecutor doesn't lie about the state of the last row anymore

* Use correct row

* Handle LimitExecutor-stats during fetchBlockForPassthrough

* Fixed LimitExecutorTest

* Removed _stats member

* Added more catch tests

* Bugfix for fullStats

* Removed an erroneous assertion

* Implemented LimitExecutor::skipRows

* Avoid name clash of class enum member to please MSVC

* Test MSVC

* Revert "Test MSVC"

This reverts commit d8325d95318bb83b79c8c4cc9f886d36e484c870.

* CREATE_NEW seems to be taken, too. Next try for MSVC

* Built VPack to ItemBlock helpers, and made SingleRowFetcherHelper more general

* Removed ostream operator because MSVC doesn't like it...

* Implemented SingleRowFetcherHelper::fetchBlockForPassthrough, plus a small framework to test executors

* Removed erroneous include

* Moved code in separate files

* Began writing the extended schema of runExecutor

* Added output operator for ExecutorCall

* Fixed removeWaiting

* Fixed expected output

* Added skipRows to runExecutor

* Added two tests

* Allow empty blocks

* Added two more tests

* Added another test

* Built two different parametrized classes for LimitExecutor

* Some cleanup

* Even more tests

* Fixed a bug found by the new tests

* Fix compile error on windows

* Use native matrix representation in tests

* Updated some comments
2019-06-18 17:33:58 +02:00
KVS85 c2bdf80532 Greylist resilience-synchronous-repl-with-arangosearch-view-cluster.js 2019-06-14 15:08:12 +02:00
Markus Pfeiffer a793f72448 Move roundCost function to utilities (#9247) 2019-06-12 16:02:09 +02:00
Jan 091abbec77 don't make RocksDB startup fail with 0-byte log files (#9208) 2019-06-07 15:13:56 +02:00
Dronplane 1e24c79ca3 Bug fix/internal issue #565 (#9154)
* Merged latest iresearch. Removed creation of build-in analyzers from tests

* Updated tests for new analyzer parameter names

* Merged fix for format from iresearch

* Apply suggestions from code review. Corrected typo in variable name.

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* backport changes from IResearch upstream

* fix compilation issues

* backport fix from IResarch upstream

* Fixed analyzer parameter name in tests
2019-06-03 18:20:23 +03:00
Tobias Gödderz 79cd45f89c Wait for replication before inserting documents (#9151)
* Wait for replication before inserting documents

Also, increased some timeouts and fixed a log message

* Fixed some log levels and a log message

* Removed repair-distribute-shards-like-spec from greylisted tests
2019-05-31 16:09:20 +03:00
Tobias Gödderz 90abe3ba93 Increased timeouts, added a little more helpful error messages for assertions (#9135) 2019-05-30 04:30:39 +02:00
Jan d67513607d
fixed issue #9106 (#9116) 2019-05-29 18:27:38 +02:00
Markus Pfeiffer de9ef00896 Round estimatedCost in some tests (#9126)
Otherise slightly different floating point results in estimatedCosts cause
tests to fail.
2019-05-29 15:29:52 +02:00
Simon a5a72ef6d4 port Pregel segmented buffers (#9112) 2019-05-28 18:23:20 +02:00
Jan 033f411420
Bug fix/issues 27052019 (#9117) 2019-05-28 12:08:25 +02:00
Simon 27112a3486 Bug fix/jobs indexes (#9107) 2019-05-28 09:38:14 +02:00
Tobias Gödderz 5da9fcb0ce Bugfix of DFS path uniqueness implementation (#9041)
* Began work on generic traversal testing framework

* Translate shard ids to smart attributes

* Minor fixes

* Added a TODO note

* fixed tests

* cleanup

* first tests inside new traversal framework working

* added single server diamond tests

* Enable sharding in aql-graph-traversal-generic-graphs.js

* Tried to fix smart sharding

* Fixed smart graph name clashes, some cleanup

* Added meta tests

* Added TODO note

* Fixed single server

* Added a BFS order check

* Renamed test

* added more dfs and bfs tests for the openDiamond graph

* Added new graph "largeBinTree"

* Fix Map usage

* added smallCircle graph, added three dfs tests

* added smallCircle graph, added three dfs tests

* more smallCircle dfs tests

* Fixed largeBinTree shardings, and long graph names

* Fix long suffixes

* Provided names for shardings

* Added a log message

* more bfs smallCircle tests

* more bfs smallCircle tests - global

* smallCircle picture

* added completeGraph to graph test suite

* added more dfs tests for the complete graph up to depth 3

* Removed duplicate function definitions

* added depth 2 dfs complete graph test, added missing test to the exports

* Added test for largeBinTree

* added three path graphs, easy, advanced and more advanced

* Added a working check for bfs with global uniqueness

* Fixed usage of bfs checks

* Fixed expected input for two tests

* Added some static assertions to largeBinTree

* Oops

* added more advancedPath dfs tests

* better formatting of the visual graph

* more dfs advanced graph tests

* Implemented all easy path test combinations

* Fixed testAqlGraphQuery

* Fix in bfs check error reports

* Removed duplicate test

* Added advancedPath BFS tests

* Updated comments

* Fixed new tests

* Fixed graph description

* added more complete graph bfs tests

* added more complete graph bfs tests

* added testCompleteGraphBfsUniqueEdgesNoneD1 and testCompleteGraphBfsUniqueEdgesNoneD2

* added testCompleteGraphBfsUniqueVerticesUniqueEdgesPathD3 test

* added a few more complete graph bfs checks

* Extended a comment
2019-05-27 12:57:02 +02:00
KVS85 8d1885d0cb Bug fix/greylist timecritical and nondeterministic (#9047)
* Mark all timecritical and nondeterministic as greylisted

* Mark description to Greylist.txt

* Fix typo
2019-05-24 11:52:15 +02:00
Jan de717924ac
fix crash (#9057) 2019-05-21 18:19:11 +02:00
Jan 6cea7528fb
fix result of AQL function `MIN` in case the first input element is (#9053)
`null`
2019-05-21 15:43:33 +02:00
Alan Plum e8b7e1860a Bug fix/unbreak foxx security (#9042)
* Unbreak aardvark when --foxx.api false

The easiest way to disable the aardvark foxx routes is to just skip the
import and mount altogether. The previous approach caused the service to
throw an error because the module doesn't actually export anything.
There's no need to prevent the export if we just don't import the module
to begin with.

* Add an error code for service api disabled

* Throw an actual error when accessing Foxx API routes

* Throw an error when using legacy Foxx API

There's no easy way to use a catch-all here so disable them one by one. No new routes will be added to this API and it'll go away in 4.0 so there's no risk of forgetting to add the throw-check in new routes.

* Render subnav without delay

The subnav doesn't need the perm check results but we need the subnav to exist so we can populate it in other parts of the app. This fixes the subnav not showing up when reloading certain views (e.g. install service via upload).

* Hide ONLY the Foxx store when it is disabled

We still want to allow installing and upgrading/replacing services. We just don't want to show the store.

* Hide Foxx services in frontend if Foxx API is disabled

* Build aardvark

* fixed typo in disable check, fixed tests
2019-05-21 14:45:23 +02:00
jsteemann c640e07ada next attempt for fixing unreliable test 2019-05-20 11:45:25 +02:00
Frank Celler 0bb8ec2e06 Revert "test is broken"
This reverts commit 469b94235b.
2019-05-18 09:55:01 +02:00
Frank Celler de566b405d Revert "fixed typo in disable check, fixed tests"
This reverts commit 068bbc9143.
2019-05-18 09:54:49 +02:00
Frank Celler 068bbc9143 fixed typo in disable check, fixed tests 2019-05-18 09:42:11 +02:00
Frank Celler 469b94235b test is broken 2019-05-18 08:04:49 +02:00
Simon 93b2e64f37 Port pregel fixes (#9022) 2019-05-17 16:32:58 +02:00
Tobias Gödderz c9fa5fba11 Additional AQL skipSome tests for IndexExecutor (#8938)
* Began work on additional index skipSome tests

* Added more index skip tests
2019-05-16 11:30:24 +02:00