1
0
Fork 0
Commit Graph

17262 Commits

Author SHA1 Message Date
Vasiliy 83a26cb003 issue 546.1: do not load analyzers in cluster for non-user requests (#8822) 2019-04-23 16:01:01 +02:00
Vasiliy 80c508c296 issue 526.11.1: limit maximum analyzer properties size (#8830) 2019-04-23 14:14:21 +02:00
Markus Pfeiffer 1f2f5ec983 K_SHORTEST_PATHS queries only support one variable in FOR (#8784)
* K_SHORTEST_PATHS queries only support one variable in FOR

 * catch if the user gives more than one variable in grammar.y
 * also give a correct error message

* Test code that catches too many variables for K_SHORTEST_PATHS
2019-04-23 08:48:16 +02:00
Simon 569198a089 Abort el-cheapo transactions if servers fail (#8799) 2019-04-22 19:31:24 +02:00
Vasiliy 40ebdd0b58 issue 545.1.1: output short analyzer prefixes for user definitions (#8814) 2019-04-21 15:10:28 +02:00
Vasiliy 2de9510051 issue 539.1: backport from iresearch upstream: add support for text normalizing analyzer, make loading shared object ptional in the registry (#8813) 2019-04-20 23:20:15 +03:00
jsteemann 90f17e2ee1 ok, cannot simply make these SmallVectors :sad: 2019-04-20 11:15:06 +02:00
Dan Larkin-York 91ed0d6568 Fix issue with truncate and index selectivity estimates (#8805) 2019-04-20 10:14:15 +02:00
Vasiliy 52aa8a9be4 issue 526.10.1: store 'forPersistence' link definitions in the agency, manually query for analyzer definitions across the cluster (#8797) 2019-04-19 16:24:02 +03:00
Vasiliy cad86bcde0 issue 535.1: ensure recovery success if link recovery starts right at the previous marker (#8796)
* issue 535.1: ensure recovery success if link recovery starts right at the previous marker

* backport: initialize members

* backport: use string_ref instead if string copies
2019-04-19 02:00:49 +03:00
Tobias Gödderz 425b40d9a2 Feature/improve refcount performance (#8783) 2019-04-18 14:32:30 +02:00
Tobias Gödderz 748ba14071 Avoid deleting default move constructor of SortedCollectExecutor (#8778) 2019-04-18 10:48:41 +02:00
Dan Larkin-York 48fb15e94f Auth-related tests for audit logging (#8790) 2019-04-18 10:45:41 +02:00
Jan 2669544944
Bug fix/fix skip inaccessible (#8779) 2019-04-17 15:54:15 +02:00
Jan 6ee9206e46
clear traverser cache in-between (#8772) 2019-04-17 15:27:37 +02:00
Markus Pfeiffer 304e8f861d Fix k Shortest Paths with weights, reactivate cluster tests (#8761)
* Make find in ShortestPathPriorityQueue const

* Fix k Shortest Paths with weights, reactivate cluster tests
2019-04-17 15:10:59 +02:00
Kaveh Vahedipour dbc5e0e2b4 fixing maintenance exception handling local resources (#8758) 2019-04-17 13:34:41 +02:00
Simon 5bb9d2dc17 transaction manager tests (#8759) 2019-04-16 21:26:40 +02:00
Vasiliy 1a22d1360c issue 526.9.1: implement swagger interface, add documentation (#8730)
* issue 526.9.1: implement swagger interface, add documentation

* address review comments

* add ngram

* Formatting

* Move REST description to new Analyzers top chapter in HTTP book

* Missed a DocuBlock

* Add Analyzers chapter to Manual SUMMARY.md

* Move REST API description back to Manual

Headlines were broken

* Add n-gram example
2019-04-16 18:54:30 +03:00
Jan 80b021f915
speed up projections with RocksDBPrimaryIndex if there is no FILTER condition (#8770) 2019-04-16 17:40:30 +02:00
Jan 7c0c01c401
use the primary index when full-scanning a collection (#8548) 2019-04-16 16:50:03 +02:00
Michael Hackstein b3205dada8
USe smart-pointer in Priority Queue, it seems the usages have been correct everywhere (so no leaks) but memory responibility was hard to understand, should now be much simpler. (#8768) 2019-04-16 12:06:42 +02:00
Jan 7fd456a83b
don't reveal information via logs or to JavaScript apps (#8763) 2019-04-15 19:10:03 +02:00
Jan c4d8091f45
remove unused attributeExcludeHandler (#8760) 2019-04-15 18:55:59 +02:00
Jan 1d175f446d
do not use the TTL index for queries which it does not support (#8766) 2019-04-15 18:55:19 +02:00
Dan Larkin-York 777ba1a364 Improved audit logging (#8740) 2019-04-15 14:51:32 +02:00
Tobias Gödderz 62ed576c8e Resolved or removed some TODOs (#8756) 2019-04-15 14:49:12 +02:00
Jan 340e3c79b0
try to force cleanup on shutdown (#8754) 2019-04-15 12:40:09 +02:00
Jan 13b2d40aef
fix compile warnings (#8743) 2019-04-15 11:22:24 +02:00
jsteemann fe58c54b2d oops 2019-04-15 11:15:17 +02:00
Jan 69f7a8a00d
longer WAL file retention for test (#8736) 2019-04-15 10:16:34 +02:00
Jan 449ab1ed8e
Bug fix/cppcheck 13042019 (#8752) 2019-04-15 10:13:56 +02:00
Markus Pfeiffer a14386267a k Shortest Paths (#8715)
* Add k-shortest-paths finder
2019-04-12 21:58:18 +02:00
Tobias Gödderz 08afad7296 Remove AqlItemBlockShell (#8681)
* Added RemoteExecutor skeleton

* Moved RemoteBlock implementations to ExecutionBlockImpl<RemoteExecutor>

* Remove unnecessary include to avoid unused function warnings

* Fixed gcc compile error

* Moved Scatter/Distribute block implementations to their new Executor versions

* Applied clang-format

* Added factory, infos and a skeleton for the unordered view executor

* Removed assert based on wrong assumption

* Added members from IResearchViewBlockBase to IResearchViewExecutor

* Moved more code into the ViewExecutor, hopefully enough to produce a working version now

* Added missing reset code, made produceRow work mostly correct

* Removed superfluous parentheses to get more useful output from Catch

* Ported fix 923b6e81ac723d1fe37f8e7bf1ab81149f3a08ef

Original commit message was:
Fixed a race condition in RemoteBlock which was triggered during
shutdown overtaking getSome.

* Applied review comments

* Inject input row instead of an item block + pos into the expression context, plus fixed some tests

* Adapted test. Search tests are now green.

* Do not ask upstream when already DONE

* Removed `limit` from next()

* Simplified code that could handle producing more than one document

* Minor readability change

* Solved two TODOs noted in the review

* Removed leftover references to DistributeNode members in the DistributeBlock

* Reverted removal of "exhausted"

* WIP: Implemented variant with scorers

* Fixed compile errors of the last commit

* Fixed some asserts and calculations

* Fixed violated assertions

* Moved files from IResearch/ to Aql/

* Replaced recursive call with a loop

* Worked on a few TODOs

* Removed IResearchViewBlock

* Set input registers correctly

* Eliminated dependency to the Node in the Executor

* Don't misuse the volatility variables for initialization

* Extended a TODO note

* Removed obsolete includes

* Removed an obsolete include from the tests

* Added missing include

* Read PKs in batches

* Fixed merge conflict

* Fixed merge conflict

* Restrict prefetching of PKs to the number of rows in the current output block

* Fixed merge

* Fix IResearch ASan errors

* Revert "Restrict prefetching of PKs to the number of rows in the current output block"

This reverts commit e0fd8698a3.

* Revert "Read PKs in batches"

This reverts commit c06c4d7a36.

* Began some small step refactoring to introduce batch-reading correctly

* Extracted method fillBuffer

* Extracted method evaluateScores

* Minor changes

* Read data from iresearch index in batches

* Replaced std::deque<IndexResult> buffer by a new class

* Solved minor TODOs

* Fixed last commit

* Fixed merge conflict

* Removed accidentally re-added view blocks

* Implemented SharedAqlItemBlockPtr

* Replaced all AqlItemBlockShell, shared_ptr<AqlItemBlock>, unique_ptr<AqlItemBlock> with SharedAqlItemBlockPtr

* Removed AqlItemBlockShell

* Bugfixes

* Added missing noexcept (used in returnBlock())

* Added nullptr constructor/operator= and noexcept specs

* Removed references to the shell

* Implemented review comments

* Fixed a compile error clang somehow ignored

* Two bugfixes and additional asserts

* Fixed ASan error

* protected AqlItemBlock destructor, and some cleanup

* added SharedAqlItemBlockPtr include in QueryCursor.h

* Added some asserts

* Fixed merge conflicts

* protected returnBlock in AqlItemBlockManager
2019-04-12 19:52:01 +02:00
Jan 33c9e9e239 optimize away SortNode in case it is covered by an arangosearch view (#8701)
* optimize away SortNode in case it is covered by an arangosearch view

this implementation is a stub with hard-coded attribute names

* extend IResearchViewMeta with sorting order definition

* make 'IResearchViewMeta::Sort' compatible with 'SortCondition' API

* ensure ArangSearch sort is immutable after creation

* address review comments
2019-04-12 19:08:41 +03:00
Heiko 8e308680d8 Bug fix/exec block cleanup (#8710)
* remove unused functions

* more cleanup, virtual functions

* rm old functions

* more cleanup

* more cleanup, rm unused code

* rm of inheritRegisters and cleaerRegisters

* more cleanup, move functions into IMPL and cluster blocks

* used wrong shutdown function

* rm not reachable code

* moved lots of ExecutionBlock stuff to Impl and ClusterBlocks

* trace functions back to executionblock

* rm trace

* rm empty protected

* this getBlock addition, might be useless

* more fixes

* fixes for the distribute executor, hopefully almost done now

* removed obsolete todos

* shutdown var order

* applied requested changes

* re added const

* suppress a warning

* added forgotten test changes

* default destructor, removed not needed function

* refactor name

* Update tests/CMakeLists.txt

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>
2019-04-12 16:11:03 +02:00
Markus Pfeiffer af400d0af7 Add hash and equals_to for EdgeDocumentToken (#8709) 2019-04-12 11:42:07 +02:00
Markus Pfeiffer b3224eed1d Add isQueryKilled callback to ShortestPathOptions and use it (#8739)
* Add isQueryKilled callback to ShortestPathOptions

* Use isQueryKilledCallback

in
 * ConstantWeightShortestPathFinder
 * AttributeWeightShortestPathFinder
 * ShortestPathExecutor
2019-04-12 10:39:01 +02:00
Michael Hackstein e5ce10cd60
Fixed compile warning of false-positive return-type (#8737)
* Fixed compile warning of false-positive return-type

* Actually fallback to the non-optimal but correct variant
2019-04-12 08:29:58 +02:00
jsteemann a07d272b1f Merge branch 'devel' of github.com:arangodb/arangodb into devel 2019-04-11 19:13:40 +02:00
jsteemann a15dd41113 initialize member 2019-04-11 19:12:53 +02:00
Michael Hackstein d2287b1b23
Bug fix/collect prefetch (#8611)
* Allow Collect Executors to prefetch rows

* Alphabthical odering of includes

* API change to prefetch the number of rows in use.
2019-04-11 17:33:26 +02:00
jsteemann c5571858ab fixed log uniqueness violation 2019-04-11 16:16:31 +02:00
Jan 4f62dd8dae
added AQL function DECODE_REV (#8606) 2019-04-11 15:59:04 +02:00
Simon 937d743ba6 Bug fix/pregel stuff (#8733) 2019-04-11 15:58:28 +02:00
Max Neunhöffer 80bfb85695
Port agency performance tuning for many shards to devel. (#8647)
* Port agency performance tuning for many shards to devel.
* Add more IDs to LOG_TOPIC calls.
* Even more IDs for LOG_TOPIC.
* Fix a duplicate LOG_TOPIC ID.
* Fix an old merging bug in devel.
* Don't hesitate between phases one and two for small clusters.
2019-04-11 11:14:56 +02:00
Jan e36f7d429e
Bug fix/fix scheduler shutdown task assertion (#8727) 2019-04-10 19:51:45 +02:00
Simon 676012ca3a snafu (#8728) 2019-04-10 19:14:34 +02:00
Jan 771a1a646d
don't analyze all calculations in a query for views if there are no views contained in the query (#8723) 2019-04-10 13:56:40 +02:00
Jan 44bc625317
improve documentation a bit (#8722) 2019-04-10 12:50:10 +02:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Vasiliy 030637c1c7 issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields (#8689)
* issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields

* address test failures

* remove debug logging
2019-04-10 13:10:25 +03:00
Jan 892446ed23
keep g++ from complaining about potential nullptr dereference (#8721) 2019-04-10 11:03:41 +02:00
Simon 4e80e1b351 Simplify recovery code (#8699) 2019-04-09 21:48:04 +02:00
Jan c7500c0c95
finally fix AQL error messages in all contexts, so that they are usable (#8706) 2019-04-09 17:46:23 +02:00
Jan 675b006ebc
this addresses an issue described in ES-260, and is a forward-port of… (#8696) 2019-04-09 11:53:20 +02:00
Jan 6b9b9b0946
Bug fix/fix test muell (#8703) 2019-04-09 11:27:53 +02:00
jsteemann ed1e0336cb fix memleaks 2019-04-09 11:24:03 +02:00
Simon 2b594bdab5 Reduce # of memcpy from storage engine layer (#8685) 2019-04-08 22:43:29 +02:00
Jan 4f7923a971
remove replication clients handling out of vocbase code (#8676) 2019-04-08 19:15:28 +02:00
Jan 5897baa984
added db.<collection>.getResponsibleShard() (#8683) 2019-04-08 16:10:45 +02:00
Jan 5a83e4832a
preserve smart-join attribute in DC2DC as well (#8687) 2019-04-08 08:47:12 +02:00
Vasiliy 0410588723 issue 526.7.1: remove deprecated functions, update tests to set up required analyzers (#8688)
* issue 526.7.1: remove deprecated functions, update tests to set up required analyzers

* address merge issues

* address more merge issues
2019-04-05 18:10:42 +03:00
jsteemann 04fe5f816c make tools complain less 2019-04-04 23:20:15 +02:00
jsteemann 83ed1d85eb use prefix increment to please cppcheck 2019-04-04 23:07:50 +02:00
jsteemann 9d7f893511 suppress "potential null pointer dereference" warning 2019-04-04 23:07:04 +02:00
jsteemann 1b03991eec fix compile warning 2019-04-04 22:53:25 +02:00
Tobias Gödderz d1416bed74 AQL by line: IResearch blocks as executors (#8387)
* Added RemoteExecutor skeleton

* Moved RemoteBlock implementations to ExecutionBlockImpl<RemoteExecutor>

* Remove unnecessary include to avoid unused function warnings

* Fixed gcc compile error

* Moved Scatter/Distribute block implementations to their new Executor versions

* Applied clang-format

* Added factory, infos and a skeleton for the unordered view executor

* Removed assert based on wrong assumption

* Added members from IResearchViewBlockBase to IResearchViewExecutor

* Moved more code into the ViewExecutor, hopefully enough to produce a working version now

* Added missing reset code, made produceRow work mostly correct

* Removed superfluous parentheses to get more useful output from Catch

* Ported fix 923b6e81ac723d1fe37f8e7bf1ab81149f3a08ef

Original commit message was:
Fixed a race condition in RemoteBlock which was triggered during
shutdown overtaking getSome.

* Applied review comments

* Inject input row instead of an item block + pos into the expression context, plus fixed some tests

* Adapted test. Search tests are now green.

* Do not ask upstream when already DONE

* Removed `limit` from next()

* Simplified code that could handle producing more than one document

* Minor readability change

* Solved two TODOs noted in the review

* Removed leftover references to DistributeNode members in the DistributeBlock

* Reverted removal of "exhausted"

* WIP: Implemented variant with scorers

* Fixed compile errors of the last commit

* Fixed some asserts and calculations

* Fixed violated assertions

* Moved files from IResearch/ to Aql/

* Replaced recursive call with a loop

* Worked on a few TODOs

* Removed IResearchViewBlock

* Set input registers correctly

* Eliminated dependency to the Node in the Executor

* Don't misuse the volatility variables for initialization

* Extended a TODO note

* Removed obsolete includes

* Removed an obsolete include from the tests

* Added missing include

* Read PKs in batches

* Fixed merge conflict

* Fixed merge conflict

* Restrict prefetching of PKs to the number of rows in the current output block

* Fixed merge

* Fix IResearch ASan errors

* Revert "Restrict prefetching of PKs to the number of rows in the current output block"

This reverts commit e0fd8698a3.

* Revert "Read PKs in batches"

This reverts commit c06c4d7a36.

* Began some small step refactoring to introduce batch-reading correctly

* Extracted method fillBuffer

* Extracted method evaluateScores

* Minor changes

* Read data from iresearch index in batches

* Replaced std::deque<IndexResult> buffer by a new class

* Solved minor TODOs

* Fixed last commit

* Fixed merge conflict

* Removed accidentally re-added view blocks

* Implemented review comments
2019-04-04 23:40:10 +03:00
Dan Larkin-York 8f49fde98f Fix RocksDB recovery ticks (#8682) 2019-04-04 20:38:44 +02:00
Dan Larkin-York b029f67e68 Fix replication index conflicts. (#8659) 2019-04-04 16:41:06 +02:00
Heiko 066ed742af Bug fix/aql line cleanup (#8598) 2019-04-04 15:57:41 +02:00
Lars Maier 06526b0a32 Feature devel/allowed paths jwt (#8678) 2019-04-04 13:55:02 +02:00
jsteemann 2b3e18418a added static variables for debugging 2019-04-04 10:27:05 +02:00
Jan 1160961af8
added debugging for replication-fuzz fails (#8674) 2019-04-04 10:20:49 +02:00
Jan 5a19a0320b
remove/fix parameters in crud methods (#8675) 2019-04-03 19:55:15 +02:00
Jan f16b75440c
make ownership a bit more transparent, API-wise (#8653) 2019-04-03 18:47:04 +02:00
Jan d1d1fe555b
Bug fix/better error message in case something goes wrong http wise (#8670) 2019-04-03 18:44:10 +02:00
jsteemann 557637fa46 turn down log level for unimportant message 2019-04-03 16:08:40 +02:00
Jan 0764e045fe
optimize lookups by shard key for smart vertex collections (#8641) 2019-04-03 15:15:46 +02:00
Simon 7cd84a785a Remove Obsolete code (#8657) 2019-04-03 13:40:44 +02:00
Jan 744845c9bb
fix small issues (#8646) 2019-04-03 11:06:05 +02:00
Jan ec6a52fb0e
fix smart join optimization when another optimizer rule (reduce-extraction-to-projection) turned full scans into index accesses (#8660) 2019-04-03 10:42:21 +02:00
Vasiliy f4919dc173 issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature (#8626)
* issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature

* address typo

* remove excess comments

* temporarily comment out tests failing on MacOS

* temporarily comment out more MacOS-only test failures
2019-04-02 19:23:28 +03:00
Jan 4af7fa8f46
count HTTP requests from traversals (#8645) 2019-04-02 18:00:11 +02:00
Jan 616ea94f24
Bug fix/cleanup 31032019 (#8632) 2019-04-01 17:14:11 +02:00
Max Neunhöffer 02281d3be4
Handle InitDone correctly. (#8552)
* precondition plan / version in compaction / store TTL removal independent of local _ttl set
* Agency init loops break when shutting down.
* assertion failures in store on restarting following agents
* Minor porting fixes from 3.4
2019-04-01 17:01:05 +02:00
Jan 24bcc7cc9f
create _frontend collection lazily (#8631) 2019-04-01 10:36:14 +02:00
Jan b6df220821
cleanup replication contexts (#8630) 2019-04-01 09:35:08 +02:00
jsteemann ef7e6ec85c initialize variable 2019-03-29 23:43:45 +01:00
Jan 7bf23bf238
add shutdown protection for PregelFeature (#8628) 2019-03-29 19:36:10 +01:00
Tobias Gödderz e42a097d54 Calculation executor v8 performance (#8511) 2019-03-29 18:24:59 +01:00
Jan 531b55f225
introduce option `--log.time-format` (#8604) 2019-03-29 17:15:33 +01:00
Tobias Gödderz 1ac2c314de Fixed several ASan errors in the catch tests (#8624) 2019-03-29 16:00:53 +01:00
Jan 9ab9cc7857
disambiguate internal exceptions (#8623) 2019-03-29 15:59:37 +01:00
Vasiliy 3a3c360765 issue 526.5: add analyzer retrieval functionality (#8592) 2019-03-29 13:23:36 +01:00
Heiko 7a44359bb0 Bug fix/gharial rm vertex smart fix (#8622) 2019-03-29 11:11:01 +01:00
Jan bccd2f3d58 acquire mutex when serializing conductor state (#8620) 2019-03-29 09:23:34 +01:00
Dan Larkin-York ffc05c05c0 Fix handling of index name conflicts during replication. (#8571) 2019-03-28 18:53:43 +01:00
KVS85 2d5bf58c94 Remove USE_IRESEARCH definition and usage (#8613) 2019-03-28 17:23:51 +01:00
Jan a5e80fa7ed
remove redundant parameter from update API (#8607) 2019-03-28 12:50:20 +01:00
Jan 0106da4ba3
spring cleaning (#8605) 2019-03-28 10:36:47 +01:00
Jan 14720b85be
add AQL functions CRC32 and FNV64 (#8600) 2019-03-28 03:06:33 +01:00
Michael Hackstein a12a28b094
Bug fix/fixes asan uninitialized by line (#8588)
* Fixed maybe-uninitialized warnings by removing unnecessary boost:optionals

* Fixed use after free

* Update arangod/Aql/SingleRemoteModificationExecutor.cpp

* Fixed another autocast number=>bool thanks c !

* Fixed wrong usage of almost identically named variables
2019-03-27 20:54:36 +01:00
Andrey Abramov 930b09cd93
[3.4] bug-fix/issue-#8294 (#8430) (#8585)
* [3.4] bug-fix/issue-#8294 (#8430)

* fix invalid optimizations for multi-valued attributes

* fix broken optimizations for multivalued attributes

* adjust tests

* add `IN_RANGE` function

* add tests

* add some shallow integration tests

* fix optimization for IN operator

# Conflicts:
#	arangod/IResearch/IResearchFeature.cpp
#	arangod/IResearch/IResearchViewOptimizerRules.cpp
#	tests/IResearch/IResearchFilterBoolean-test.cpp

* fix compilation errors

* fix another compilation issue

* address compilation errors

* fix tests
2019-03-27 18:33:16 +03:00
Simon 417ee266d4 Fuse transaction begin request for non baby operations (#8566) 2019-03-27 11:31:39 +01:00
Jan d6d3e3daa4
initialize some member variables, added TODOs (#8545) 2019-03-26 12:57:32 +01:00
Vasiliy 778417cdf0 issue 526.4.1: add analyzer removal and feature validation functionality (#8575) 2019-03-26 14:17:40 +03:00
Jan 207c978d1e
Bug fix/move a few actions from js to cpp (#8549) 2019-03-26 10:02:21 +01:00
Jan 80a6e621ee
don't allocate memory so often in ClusterComm requests (#8550) 2019-03-26 00:31:56 +01:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Simon 75c3cadd67 Hide in progress indexes (#8564) 2019-03-25 18:31:12 +01:00
Simon bcc26926de Support Transactions in RestCursorHandler (#8539) 2019-03-25 17:04:11 +01:00
Michael Hackstein 48b709ae16
Feature/subquery by line (#8553)
Subquery Executor by line
2019-03-25 17:02:07 +01:00
Jan Christoph Uhde 763cf1168c stop after receiving first error message (#8555) 2019-03-25 13:46:02 +01:00
Michael Hackstein 863af5f3d9
Fixed sequence point warning in all rows fetcher (#8556) 2019-03-25 13:29:16 +01:00
Jan e078f35285
fixed typos, removed unneeded includes (#8547) 2019-03-25 12:09:37 +01:00
Jan f2ab6da5f8
dont whine about --server.jwt-secret if it isn't used (#8546) 2019-03-25 12:08:43 +01:00
Jan Christoph Uhde 5711854531 add SingleRemoteModificationExecutor (#8245) 2019-03-25 07:07:57 +01:00
Jan Christoph Uhde 5d5b77abb8 add ModificationExecutor (#8179)
All modification operator as Executors
2019-03-24 20:48:38 +01:00
Michael Hackstein f1e4f370c3
Feature/aql sorted collect (#8524)
Sorted Collect Executor
2019-03-24 19:51:59 +01:00
jsteemann 871c987b68 eliminate dead store 2019-03-24 18:45:10 +01:00
Michael Hackstein 20d3141486
Feature/aql gather executors (#8514)
Implemented AQL Gather Executors by line
2019-03-24 17:09:07 +01:00
jsteemann d5de286568 cppcheck 2019-03-23 22:29:42 +01:00
Jan 0a1ab07c64
address review concerns (#8544) 2019-03-23 22:10:19 +01:00
jsteemann 22fecdd3b7 fix compile warning 2019-03-23 20:39:40 +01:00
Jan 39a3f5bc4e
reintroduce smart joins after temporarily reverting them in devel (#8543) 2019-03-23 20:36:02 +01:00
Tobias Gödderz ff06c5244f Suppress false positives about "potential null pointer dereference" (#8501) 2019-03-23 17:41:33 +01:00
Dan Larkin-York 44810a1218 Improved index error messages. (#8532) 2019-03-22 20:26:09 +01:00
Dan Larkin-York be2cccfa28 Add RocksDB background error listener. (#8535) 2019-03-22 20:25:04 +01:00
Jan cfd3418115
microoptimizations (#8519) 2019-03-22 16:53:21 +01:00
Jan 4812ad9146
fix /_api/cluster/endpoints URL for active failover (#8520) 2019-03-22 16:52:26 +01:00
Jan Christoph Uhde 0b14599b67 better AQL error messages (#8503) 2019-03-22 16:49:08 +01:00
Simon f4ecf8cd9e Fix asan error in truncate (#8523) 2019-03-22 15:22:28 +01:00
jsteemann 6a6947d268 fix compile warning 2019-03-22 11:55:45 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan a35f6ccf10
fix type of hidden startup option `--rocksdb.recycle-log-file-num` (#8517) 2019-03-22 11:21:53 +01:00
Michael Hackstein 14bcc37e36 Fix a race in RemoteExecutor (#8516) 2019-03-22 10:52:16 +01:00
jsteemann 864a7581e0 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2019-03-21 19:57:47 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)"
This reverts commit 59ad583796.
2019-03-21 19:18:46 +01:00
jsteemann d300000bf2 Revert "fix a compile warning"
This reverts commit 45d460f934.
2019-03-21 19:18:41 +01:00
Jan f9fe1de82f
prevent assertion failures when creating smart graphs with invalid options (#8502) 2019-03-21 18:51:47 +01:00
jsteemann 45d460f934 fix a compile warning 2019-03-21 16:53:17 +01:00
Jan Christoph Uhde 679ac12395 use abort as SIGABRT has CORE as default action (#8510) 2019-03-21 15:55:21 +01:00
Max Neunhöffer 55706e3c74
Make addfollower jobs less aggressive. (#8490)
* Make addfollower jobs less aggressive.
* CHANGELOG.
2019-03-21 15:24:31 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Jan 00ac1db850
add RocksDB options `--rocksdb.allow-fallocate` and `--rocksdb.limit-open-files-at-startup` (#8493) 2019-03-21 14:21:35 +01:00
Vasiliy 405b60c2b7 issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections (#8452)
* issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections

* address merge issues

* address another merge issue
2019-03-21 14:23:36 +03:00
Jan 77c37723d6
some very small optimizations (#8484) 2019-03-21 09:57:05 +01:00
Lars Maier 4d49285754 [devel]agency appends entries with leaders timestamp (#8478)
* agency's append entries with leader's timestamp
* compatibility with appendEntries protocol without timestamps
* Updated changelog.
2019-03-21 09:52:58 +01:00
Heiko ce51797609 Feature/aql hashed collect (#8337)
Implementation of Hashed Collect Executor
2019-03-21 07:19:26 +01:00
Jan 0bef1c1098
dont attempt to remove non-existing WAL files (#8474) 2019-03-20 17:58:28 +01:00
Kaveh Vahedipour 5038dfe685 supervision must not copy snapshots into jobs (#8425)
* supervision must not copy snapshots into jobs
* CHANGELOG.
2019-03-20 17:07:54 +01:00
Kaveh Vahedipour 237e079614 leader check needs to sit inside waitfor loop (#8445)
* leader check needs to sit inside waitfor loop
* Do not wait in Supervision for commits of new writes.
* CHANGELOG.
2019-03-20 16:34:54 +01:00
Jan 6d107beeda
don't run compact() on a collection after a truncate() was done in th… (#8468)
* don't run compact() on a collection after a truncate() was done in the same transaction

running compact() in the same transaction will only increase the data size on disk due to RocksDB not being able to remove
any documents physically due to the snapshot we take at transaction start.
Decoupling the truncate transaction from the compact operation allows finishing the truncate transaction first, so we can
get rid of the snapshot. Running compact afterwards is then free to physically remove all the data.
As a nice side effect this change will also speed up the truncation of larger collections, because the compact will run
faster.

This change also exposes db.<collection>.compact() in the arangosh, in order to manually run a compaction on the data
range of a collection should it be needed for maintenance.

* fix documentation anchors
2019-03-20 16:27:54 +01:00
Tobias Gödderz 9203579218 Initialize alternativeRegId (#8480) 2019-03-20 15:00:08 +01:00
jsteemann 11efed7510 fix compile warnings 2019-03-20 14:28:19 +01:00
Lars Maier e598148707 Bug fix/jwt secret file logging (#7976)
* Print with correct log topic.

* Moved message to a place where logger is enabled.
2019-03-20 14:14:13 +01:00
Dan Larkin-York 04ef4595e3 Fix replication issue with index ID/name conflicts (#8448) 2019-03-20 14:06:02 +01:00
Tobias Gödderz f97d0a211f Implement ScatterExecutor and DistributeExecutor (#8292) 2019-03-20 13:40:20 +01:00
Jan 2c4accdf34
micro improvements for rocksdb options (#8433) 2019-03-19 12:47:57 +01:00
Jan 5dbd67c8fd
fix compile warnings about operator<< (#8456) 2019-03-19 12:33:26 +01:00
Dan Larkin-York 2eadab33e7 Index hints (#8431) 2019-03-19 09:14:18 +01:00
Vasiliy 29b941a677 issue 526.2.1: add upgrade step for legacy static analyzers (#8417) 2019-03-18 23:22:23 +03:00
Jan 23f7fc1368
fix some memleaks (#8432) 2019-03-18 18:11:37 +01:00
Simon 49cc3bcd1e Refactorings from cluster trx improvement branch (#8391) 2019-03-14 23:13:17 +01:00
Tobias Gödderz 8d649903f7 Aql by line mixed fixes (#8349) 2019-03-14 19:38:41 +01:00
Wilfried Goesgens d88bc376a6 When updating documents with array indices and unique values we mustn't disable indexing in rocksdb (#8404) 2019-03-14 13:35:38 +01:00
Jan a1897d4e84
expose ARANGODB_USE_CATCH_TESTS define, make TEST_VIRTUAL be based on it (#8410) 2019-03-14 12:53:57 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
Jan 3156e481de
fix test (#8402) 2019-03-13 15:24:55 +01:00
Kaveh Vahedipour fa98e94d23 Supervision must not waitfor if no longer leading (#8403)
* Supervision must not waitfor if no longer leading

* Supervision must not waitfor if no longer leading
2019-03-13 13:18:10 +01:00
Jan be53ccda97
avoid a few dynamic memory allocations in the optimizer (#8393) 2019-03-13 12:55:19 +01:00
jsteemann d0dba4f941 make sure some tick invariants always hold true 2019-03-12 16:47:12 +01:00
Jan 9d3327c6ea
Bug fix/rearm cursors (#8363) 2019-03-12 15:28:33 +01:00
Max Neunhöffer 2a4f606df2
Various agency improvements. (#8380)
* Ignore satellite collections in shrinkCluster in agency.
* Abort RemoveFollower job if not enough in-sync followers or leader failure.
* Break quick wait loop in supervision if leadership is lost.
* In case of resigned leader, set isReady=false in clusterInventory.
* Fix catch tests.
2019-03-12 15:25:16 +01:00
Max Neunhöffer 30adf5e2d9
Fix an agency crash. (#8381)
* Check if transaction failed before accessing the result.
* FailedFollower has the same bug.
* Add CHANGELOG.
2019-03-12 15:24:37 +01:00
Vasiliy f5f572dd29 issue 527.1: ensure the cause of the failure is reported to the user (#8371) 2019-03-11 23:42:34 +03:00
Jan 12e11a5197
port of replication improvements from 3.4 (#8308) 2019-03-11 13:37:18 +01:00
Jan 85227e00e4
make geo-index-optimizer rule work with multiple loops (#8353) 2019-03-11 12:26:07 +01:00
Tobias Gödderz fc5f75a452 Implement RemoteExecutor (#8280) 2019-03-11 10:16:55 +01:00
Michael Hackstein 0403d6cb6f
Bug fix/prune neighbors and explain (#8304)
* Fixed Neighbors prune on depth0

* Fixed the Explainer output

* Rmeoved debug output
2019-03-11 10:13:57 +01:00
Jan 4c6c61ca33
document `includeFoxxQueues` attribute and persist it (#8351) 2019-03-08 18:43:35 +01:00
Jan f6d817b8ba
remove some unused stuff (#8350) 2019-03-08 15:52:39 +01:00
Kaveh Vahedipour 098d2d086c fix compaction behaviour of followers (#8348) 2019-03-08 10:39:49 +01:00
Kaveh Vahedipour ee751e8ba3 [devel] clear compilation warnings (#8345) 2019-03-08 10:35:09 +01:00
Jan 9f9426e36c
Bug fix/allow creating foxx queues inside transactions (#8339)
* allow creating a foxx queue from within a transaction

* added tests
2019-03-07 14:28:39 +01:00
Jan e28b908bbe
fix an assertion failure when plans are shut down in an invalid state (#8334) 2019-03-07 14:21:08 +01:00
Jan c82e0c2ca0
add option `--console.history` to arangosh (#8327) 2019-03-07 13:05:54 +01:00
jsteemann c8a53d9771 fix ASan warnings 2019-03-06 21:11:45 +01:00
m0ppers f4abe08b21 🚨 Fix struct vs class warning (#8329) 2019-03-06 14:42:56 +01:00
Vasiliy e97cd8a0fb issue 526.1: initial support for vocbase-prefixed analyzer names (#8319)
* issue 526.1: initial support for vocbase-prefixed analyzer names

* backport: add tests for new functionality, add temporary workaround for legacy analyzers

* address review comments

* fix typo
2019-03-06 15:13:19 +03:00
Kaveh Vahedipour 4b464aeb97 oversight (#8324)
* oversight of an abort
* fix waitFor trap in supervision
2019-03-05 23:31:18 +01:00
Vasiliy b1e30937bb issue 520.1.1: decrease link commit interval to 1 second, add workaround for MMFiles not recovering document insertions via WAL (#8313) 2019-03-05 14:50:54 +03:00
Kaveh Vahedipour 68178ba165 [devel] supervision bug fix backports (#8314)
* back ports for supervision fixes from 3.4 part 1

* back ports for supervision fixes from 3.4 part 2
2019-03-04 19:27:24 +01:00
jsteemann 29496d72e3 remove unused method 2019-03-04 09:54:26 +01:00
Michael Hackstein e84c71b62f
Prefetch in Filter Executor (#8212)
Prefetch number of rows in filter executor as well. Fixes the GO Test
2019-03-01 17:53:21 +01:00
Jan 08a6561adc
Bug fix/small issues 21022019 (#8229) 2019-03-01 17:20:10 +01:00
Jan 43c2850ec3
fix test in active failover mode (#8305) 2019-03-01 17:01:12 +01:00
Vasiliy 7130ffa427 issue 525.1: ensure RocksDB CreateIndex/DropIndex WAL markers are properly written during recovery (#8282)
* issue 525.1: ensure RocksDB CreateIndex/DropIndex WAL markers are properly writen during recovery

* account for changes in devel

* skip writing DropIndex marker in recovery
2019-03-01 15:58:11 +03:00
Andrey Abramov d86fb726e6
ArangoSearch index encryption (#8214)
* update iresearch

* add index encryption support

* update iresearch

* encrypt ArangoSearch index using rocksdb encryption provider

* fix merge issue

* update iresearch

* fix cmake

* minor fixes after update

* update iresearch

* partially revert changes from commit 492d05c1f1 since they cause iresearch configuration failures

* Update arangod/IResearch/IResearchRocksDBLink.cpp

Co-Authored-By: gnusi <gnusmas@kvolitek.ru>

* partially revert changes from commit 492d05c1f1 since they cause iresearch configuration failures

* fix cmake configuration for MAX

* another attempt to fix cmake on mac

* Fix 3rdparty cmake for Windows
2019-03-01 15:56:40 +03:00