1
0
Fork 0
Commit Graph

288 Commits

Author SHA1 Message Date
Tobias Gödderz 9c92287707 Bug fix/fix simple example dep proxy skip some regression test (#10213)
* Added regression test for #10190
2019-10-10 14:30:07 +02:00
Tobias Gödderz 4e86de00d2 AQL Subquery: MultiDependencyRowFetcher (#10101)
Add ShadowRow support for MultiDependencyRowFetcher
2019-10-10 13:08:02 +02:00
Jan 79c3187036
fixed issue #10193 (#10195) 2019-10-09 10:54:40 +02:00
Dronplane a1014ac701 Bug fix/internal issue #600 - Late document materialization (#10006)
* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* Applied review cAdded sceleton optimizer functionomments

* Added mask for rule

* adding plan traversal for rule

* Rule execution block developing

* Experiment with additional block

* LIMIT node modifications

* implemented late materialization base version.

* Code cleanup

* Added check for variable usage

* Implemented serialization and cloning

* Fixed build errors

* Implemented late materialization for  LimitNode

* Moved materialization to Sort Node

* Removed LimitNode modiications

* Implemented skipRows to speedup single server

* Added tests for optimizer rule

* fixed build

* Fix mac build

* Fixed mac build

* Fixed PR test run errors

* Added tmp fix for cluster test runs

* removed debug print

* Make materialization stick to last sort node. Add late materialization to explainer

* Updated optimization rule to search optimal SortNode to do materialization

* Refined optimization rules for cluster

* Added tests for SortExecutor and IResearchViewNode

* Fixed  mac buid. Removed misplaced assertion.

* Fixed build

* Updated tests. Coded cleanup

* Code cleanup

* made function name for SortedExecutor match name in ConstraintSortedExecutor

* cleanup

* Fixed test run

* made scoring results stable across cluster

* Make optimize rule stop if  sort node for limit node is not suitable

* Disabled constrained sort for materializing SortNode

* reverted SortNode modifications

* Implemented separate materializer node

* Fixed tests  to account new materialize node

* Code cleanup

* reverted debugging change

* Fixed test jslint error  and comments

* MaterializeNode renamed

* enum value renamed for materialize node

* Code cleanup

* code cleanup and optimization

* Fixed clang warning

* Applied review comments

* Out variable planning moved to materialize node

* inlined getters in ReadContex
2019-10-08 18:38:47 +03:00
Jan 21b0311d57
rename minReplicationFactor to writeConcern (#10118) 2019-10-07 15:12:15 +02:00
Jan 0da109bf82
fixed issue #10158 (#10162) 2019-10-04 13:53:31 +02:00
Jan ae05fafa4e
Bug fix/fix parsing unterminated strings (#10155) 2019-10-03 13:38:05 +02:00
Wilfried Goesgens 2b743e0e61 Bug fix/speedup tests (#10109) 2019-10-01 18:23:00 +02:00
Jan 1d84dd861d
use setUpAll and tearDownAll to reduce test durations (#10107) 2019-09-30 17:05:20 +02:00
Tobias Gödderz f120919dfe Implement a constrained sorting gather variety (#10057)
* Added TODO notes

* Pass limit of constrained sort to sorting gather

* Count output rows, and added assertions

* Extracted row-producing code in a reusable way

* Implemented SortingGatherExecutor::skipRows

* Fixed signature

* Fixed an assertion

* Added limit to (de)serialization

* Fixed an assertion

* Fixed an atMost calculation

* In the sort-limit optimization, apply limit to sorting gather as well

* Fixed optimizer rule reporting modification

* Added missing initialization

* Fix: Do not produce more rows than the limit when skipping first

* Fix: pass limit when cloning

* Updated CHANGELOG

* Fixed atMost intricacies
2019-09-27 10:50:06 +02:00
Jan 9da2cadf83
Bug fix/issue 10078 (#10081) 2019-09-26 12:08:48 +02:00
Markus Pfeiffer 0b9dfb4b9b Subquery Start/End Nodes (#9983)
* Add Start/End subquery node skeleton

* Added a test query that triggers complex interna in the future and needs to proof that our work is side-effect free.

* Add template function to statically assert size of type

* Add Start/End subquery node skeleton

* Add Start/End subquery node creation to ExecutionNode

* Cleanup SubqueryEndNode

* Add test file

* Add isEqualTo to ExecutionNode

* Subquery nodes cleanup

* Update ExecutionNode tests

* Updates to testing code

* More updates

* Add optimizer rule

* Fix optimizer rule

* Fix optimizer rule (again)

* Add a public function to get plan from query

* Fix register planning

* Add test for splicing subquery nodes

* add static asserts for sizes of SubqueryStartNode and SubqueryEndNode

* Add comment about statically asserted size

* Fix isEqualTo virtual and override dance.

* Fix isEqualTo, remove static_assert_size

* Remove a TODO; the plan owns the ExecutionNodes

* Only add stealPlan to Query when testing

* Remove another static_assert_size

* Disabled subquery splice rule by default

* temporarily modified test

* Fixed windows compile

* Fixed unused variable warning
2019-09-25 12:35:18 +02:00
Jan 4f2e1d4054
fixed issue #10062: AQL: Could not extract custom attribute (#10068) 2019-09-24 14:36:53 +02:00
jsteemann dd9989a2e8 renamed and fixed a test 2019-09-23 18:17:17 +02:00
Jan Christoph Uhde 0b8c75c7b7 one shard db - devel (#9395) 2019-09-23 15:48:37 +02:00
Tobias Gödderz fbcb4b7152 Avoid overfetch in sorting gather executor (#10047)
* Avoid overfetch in sorting gather executor

* Updated CHANGELOG

* Temporarily disabled sort-limit in cluster when fullCount is enabled, until we can fix SortingGather

* Added a cluster test for sort-limit

* Fixed non-maintainer compile

* Fixed jslint errors

* Disabled sort profiler tests until SortingGather is fixed
2019-09-20 17:00:18 +02:00
Tobias Gödderz 2f59a79435 Bug fix/sort limit rule too lax (#10014)
* Added regression test for consecutive constrained sorts

* Bugfix: sort-limit rule skipped too many node types

* Added CHANGELOG entry

* Apply sort-limit rule before single document operations
2019-09-18 09:04:43 +02:00
Dan Larkin-York 8c573549b3 Make truncate use non-blocking communication. (#9980) 2019-09-16 10:46:49 +02:00
Tobias Gödderz a97000cd04 Additional sort-limit tests (#10010) 2019-09-13 15:18:01 +02:00
KVS85 554ca3874e
Fix jslint 2019-09-12 22:25:15 +03:00
Tobias Gödderz e113e36138 Make AQL's constrained heap play nice with fullCount (#9981) 2019-09-12 18:56:59 +02:00
KVS85 c892fe05b4 Add more stability to moving-shards-cluster.js 2019-09-12 18:23:44 +02:00
Jan 2dde7e2307
Bug fix/allow not like (#9989) 2019-09-12 12:54:02 +02:00
KVS85 4fc39dd4b3
Debug segfault reimplementation (#9940)
* Changed debugSegfault to debugTerminate

* Fix *nix compilation

* More data for broken reconnect

* Remove circumventCores completely

* Fix forgotten calls
2019-09-09 23:07:45 +03:00
Jan 35526deddf
improve handling of FILTERs that are known to always produce true or false (#9941) 2019-09-09 10:15:15 +02:00
jsteemann f0305a0760 fix zirkus 2019-09-06 16:22:25 +02:00
jsteemann fc86824638 @dothebart 2019-09-06 16:19:49 +02:00
Jan 7e500f5597
drop views on test teardown (#9919) 2019-09-05 15:03:02 +02:00
Markus Pfeiffer 753ff4aa67 Feature/atomic database creation 2 (#9826) 2019-09-05 12:38:07 +02:00
Jan 68291ce617
fixed issue #9862 (#9864) 2019-08-30 14:46:42 +02:00
Jan 7220af9602
cover more cases of "unique constraint violated" issues during replication (#9830) 2019-08-30 10:37:32 +02:00
Jan 86a496b8a3
Bug fix/fix in not in workaround (#9813) 2019-08-27 12:51:10 +02:00
Jan 00bcc4954c
AQL date functions improvements (#9714) 2019-08-22 12:50:08 +02:00
Dronplane 49dfb0a96e Bug fix/issue #9612 (#9764)
* Fixed ViewExecutionNode retrieval with deleted documents present in view

* Ported solution from 3.4 branch

* Changed index store in collection from vector to set. To make reversable indexes always last to execute

* Fixed re-enter hung

* Index storage fix

* Made index order deterministic

* Fix Mac build

* Added tests for index reversal

* Fixed Mac build

* Code cleanup

* Some cleanup

* Removed some redundand copy constructor calls

* Applied review comments

* Applied review comments
2019-08-20 19:34:23 +03:00
Jan fda4eea43f
make index selection more deterministic (#9735)
* make index selection more deterministic

* serialize indexes used by traversal with their estimates

* serialize selectivity estimates for shortest path nodes too

* fix assertion that doesn't hold true in unit tests

* fix test
2019-08-20 14:45:13 +02:00
Lars Maier 492057d4f4 [devel] Resign Leadership (#9427)
* First version of ResignLeadership Job.
* Port some performance optimizations from CleanOutServerJob.
* Draft of resigning leadership on shutdown.
* Moved code into Maintenance Feature. Fixed beginShutdown.
2019-08-07 15:02:17 +02:00
Lars Maier ed496fe5dd Feature/hotbackup devel (#9495)
Hotbackup
2019-08-02 11:39:46 +02:00
Jan 7d829de89e added internal function getResponsibleServers() (#9604)
* added internal function getResponsibleServers()

* forgot to commit

* honor review comments

* Update arangod/Cluster/ClusterInfo.cpp

Potentially Fixed Unique logID usage. (let Jenkins test it)
2019-07-31 10:18:37 +02:00
Jan Christoph Uhde fb839d1c40 stop optimization for nested collects (#9484) 2019-07-29 09:00:03 +02:00
Matthew Von-Maszewski 91b56a50a3
Feature: Add gzip and encryption to import/export (#9560)
* port of feature-3.4/mv-gzip-export to devel branch

* add explicit namespaces so gcc 6.3.0 would successfully compile

* add conditional cleanup of ENCRYPTION file from another branch

* use _lseek() for Windows build to avoid deprecated warning.

* change from ifdef for lseek variants to TRI_LSEEK.

* force Windows lseek to return Linux expected type.
2019-07-26 07:53:39 -04:00
Jan b62eee034b
fixed a bug in RTRIM implementation, fixed issue #9558 (#9562) 2019-07-25 13:04:46 +02:00
Andrey Abramov d91fb35e39
Bug fix/iresearch trx (#9536)
* add tescase for trx and iresearch

* iresearch transaction integration

* adjust test

* reverse operation for correct index

* remove debug output

* address jsling errors

* address review comments

* forgot something

* added missing fail
2019-07-23 17:44:29 +03:00
Michael Hackstein cbcf561450
Feature/min replication factor (#9433)
* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* fixed ires tests

* fixed wrong assert

* properly check uint

* repl factor attr check

* adjusted test to be more preciese now

* Fixed race on atomics comparison

* Fixed invalid number type

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* More review fixes
2019-07-19 13:02:28 +02:00
Andrey Abramov 5bc6eac636
bug-fix/internal-issue-#609 (#9464)
* allow to access last operation tick of the transaction

* modify IResearchLink to rely on last commtted tick

* get rid of writing arangosearch markers to WAL

* further implementation

* local changes in iresearch

* add recovery states

* properly handle link creation during recovery

* adjust test cases

* properly handle nested transactions

* ungreylist recovery tests

* add more recovery tests

* do not use transaction to pass recovery tick

* do not store recoveryTick in MMFilesRecoveryState

* adjust tests

* fix mmfiles

* cleanup

* add context validity check

* fix tests

* fix more tests

* ensure subscription is not being released during commit

* address review comment

* final cleanup

* fix crash

* fix tests

* address test failures

* address review comments

* address review comments

* properly set recovery tick even if no recovery happened
2019-07-19 11:10:56 +03:00
Tobias Gödderz 001550ada7 Bug fix/count collect should skip (#9511)
* Make CountCollectExecutor skip rather than fetch

* Cleanup, fix and test for top-level variables

* Added CHANGELOG entry and tests
2019-07-19 10:08:36 +02:00
Andrey Abramov f2ff365206
bug-fix/fix-arangosearch-recovery-mmfiles (#9448)
* start maintenance arangosearch threads after recovery is done

* ensure flush subscription is not being deallocated while in use

* add some tests

* properly determine storage engine

* adjust default view options

* stick to old index meta format in 3.5

* address test failures

* and cluster tests
2019-07-11 11:42:08 +03:00
Wilfried Goesgens 0ad76a8185 fix location - while they work with the server these tests are run in arangosh so they need to be in client/ (#9435)
* fix location - while they work with the server these tests are run in arangosh so they need to be in client/

* rename testsuite

* rename file as its testsuite

* Rename permissions_server to server_permissions
2019-07-10 16:49:12 +03:00
KVS85 8897cd8f31 Fix wrong ars test failure suppression 2019-07-10 15:08:53 +02:00
KVS85 e4d76e087f
Temporarily suppress failure (#9443) 2019-07-10 01:10:56 +03:00
Jan c52f2a8315
refactoring (#9411) 2019-07-09 11:15:52 +02:00