1
0
Fork 0
Commit Graph

1210 Commits

Author SHA1 Message Date
Jan 0cbdfe9289
Bug fix/vpack update (#8875) 2019-04-30 12:33:26 +02:00
Frank Celler 6abda70550 recheck if mkdir fails 2019-04-21 14:05:30 +02:00
Vasiliy 2de9510051 issue 539.1: backport from iresearch upstream: add support for text normalizing analyzer, make loading shared object ptional in the registry (#8813) 2019-04-20 23:20:15 +03:00
Vasiliy 371509000b issue 537.1: backport from iresearch upstream: ensure key is cached too (#8807) 2019-04-19 16:20:16 +03:00
Tobias Gödderz 11eb393631 Prefer make_shared over new; move buffer in move constructor and -ass… (#8746) 2019-04-12 17:33:02 +02:00
Andrey Abramov e784ded6de
fix invalid assertion (#8731) 2019-04-11 16:26:20 +03:00
Andrey Abramov b5c6af5ab1
bug-fix/internal-issue-#536 (#8726)
* backport fix from iresearch upstream

* update changelog
2019-04-11 14:12:53 +03:00
Jan 675b006ebc
this addresses an issue described in ES-260, and is a forward-port of… (#8696) 2019-04-09 11:53:20 +02:00
Simon 2b594bdab5 Reduce # of memcpy from storage engine layer (#8685) 2019-04-08 22:43:29 +02:00
Jan 5897baa984
added db.<collection>.getResponsibleShard() (#8683) 2019-04-08 16:10:45 +02:00
Jan 86458580d8
updated jemalloc version to 5.2.0 (#8684) 2019-04-05 22:01:56 +02:00
Tobias Gödderz d1416bed74 AQL by line: IResearch blocks as executors (#8387)
* Added RemoteExecutor skeleton

* Moved RemoteBlock implementations to ExecutionBlockImpl<RemoteExecutor>

* Remove unnecessary include to avoid unused function warnings

* Fixed gcc compile error

* Moved Scatter/Distribute block implementations to their new Executor versions

* Applied clang-format

* Added factory, infos and a skeleton for the unordered view executor

* Removed assert based on wrong assumption

* Added members from IResearchViewBlockBase to IResearchViewExecutor

* Moved more code into the ViewExecutor, hopefully enough to produce a working version now

* Added missing reset code, made produceRow work mostly correct

* Removed superfluous parentheses to get more useful output from Catch

* Ported fix 923b6e81ac723d1fe37f8e7bf1ab81149f3a08ef

Original commit message was:
Fixed a race condition in RemoteBlock which was triggered during
shutdown overtaking getSome.

* Applied review comments

* Inject input row instead of an item block + pos into the expression context, plus fixed some tests

* Adapted test. Search tests are now green.

* Do not ask upstream when already DONE

* Removed `limit` from next()

* Simplified code that could handle producing more than one document

* Minor readability change

* Solved two TODOs noted in the review

* Removed leftover references to DistributeNode members in the DistributeBlock

* Reverted removal of "exhausted"

* WIP: Implemented variant with scorers

* Fixed compile errors of the last commit

* Fixed some asserts and calculations

* Fixed violated assertions

* Moved files from IResearch/ to Aql/

* Replaced recursive call with a loop

* Worked on a few TODOs

* Removed IResearchViewBlock

* Set input registers correctly

* Eliminated dependency to the Node in the Executor

* Don't misuse the volatility variables for initialization

* Extended a TODO note

* Removed obsolete includes

* Removed an obsolete include from the tests

* Added missing include

* Read PKs in batches

* Fixed merge conflict

* Fixed merge conflict

* Restrict prefetching of PKs to the number of rows in the current output block

* Fixed merge

* Fix IResearch ASan errors

* Revert "Restrict prefetching of PKs to the number of rows in the current output block"

This reverts commit e0fd8698a3.

* Revert "Read PKs in batches"

This reverts commit c06c4d7a36.

* Began some small step refactoring to introduce batch-reading correctly

* Extracted method fillBuffer

* Extracted method evaluateScores

* Minor changes

* Read data from iresearch index in batches

* Replaced std::deque<IndexResult> buffer by a new class

* Solved minor TODOs

* Fixed last commit

* Fixed merge conflict

* Removed accidentally re-added view blocks

* Implemented review comments
2019-04-04 23:40:10 +03:00
KVS85 2d5bf58c94 Remove USE_IRESEARCH definition and usage (#8613) 2019-03-28 17:23:51 +01:00
Jan e0ebfefddd
added missing fix for NowNanos from upstream RocksDB (#8602) 2019-03-28 02:25:47 +01:00
Jan Christoph Uhde 523019c8d3 update rocksdb to 6.0.fb (#8576) 2019-03-26 15:52:35 +01:00
Wilfried Goesgens 6f9091878c where did the % go? (#8342) 2019-03-07 14:38:29 +01:00
Wilfried Goesgens 19abe0d63a use the specified python interpreter here too (#8330) 2019-03-06 15:41:08 +01:00
Andrey Abramov d86fb726e6
ArangoSearch index encryption (#8214)
* update iresearch

* add index encryption support

* update iresearch

* encrypt ArangoSearch index using rocksdb encryption provider

* fix merge issue

* update iresearch

* fix cmake

* minor fixes after update

* update iresearch

* partially revert changes from commit 492d05c1f1 since they cause iresearch configuration failures

* Update arangod/IResearch/IResearchRocksDBLink.cpp

Co-Authored-By: gnusi <gnusmas@kvolitek.ru>

* partially revert changes from commit 492d05c1f1 since they cause iresearch configuration failures

* fix cmake configuration for MAX

* another attempt to fix cmake on mac

* Fix 3rdparty cmake for Windows
2019-03-01 15:56:40 +03:00
Frank Celler 8e814744a5
fix buffer overflow (#8188) 2019-02-27 18:02:20 +01:00
Jan 1798036ea0
Bug fix/optimizations 18022019 (#8180) 2019-02-19 19:24:04 +01:00
Wilfried Goesgens 08c3710c29 when we DO_NOTHING tell the implementer he may stop invoking us (#8201) 2019-02-19 19:20:18 +01:00
Wilfried Goesgens 492d05c1f1 Feature/upgrade v8 7.1.302.28 (#8088) 2019-02-19 11:15:34 +01:00
Jan 820ba5dd7c
micro optimizations (#8156) 2019-02-13 16:14:28 +01:00
Vasiliy ea48789c54 issue 466.4: implement persistence of IResearchLink WAL flush marker (#8059)
* issue 466.4: implement persistence of IResearchLink WAL flush marker

* address enterprise test failure

* address test failures

* treat missing collections/indexes as potentially removed

* change view asertios to match collection assertions

* convert assertions to exceptions since they are possible

* revert assertion removal since assertions are actually valid for coordinator

* address scenario where link is dropped twice

* check for data store state before returning error

* revert last change since it's not valid for MMFiles
2019-02-04 16:36:13 +03:00
Jan 0053a72a6a update velocypack (#8076) 2019-01-31 17:31:14 +01:00
Vasiliy 22ea8bc293 issue 466.2: update iresearch to revision 9f179cea9037c702115d0884c88e742a30921cd1 (#8010)
* issue 466.2: update iresearch to revision ee15310f1450035c7d3ba664c6886ef7a39eb6bb

* update iresearch to commit 9f179cea9037c702115d0884c88e742a30921cd1
2019-01-24 16:11:04 +03:00
Jan 6944115de6
various optimizations (#7994) 2019-01-21 13:40:39 +01:00
Simon f748aee240 Added collectAll, updated fuerte (#7949) 2019-01-16 11:31:08 +01:00
Jan 2ae738792a
willi (#7948) 2019-01-15 09:41:00 +01:00
Jan fa7de56cf8
upgrade to boost 1.69.0 (#7910) 2019-01-09 17:17:33 +01:00
Jan b40b88d73b
upgrade bundled curl library to version 7.63 (#7846) 2019-01-08 11:27:59 +01:00
Jan 98ff9621bf
prevent duplicate attributes being generated by AQL queries (#7837) 2019-01-02 12:37:17 +01:00
Jan f4f61e2947
Bug fix/issues 23122018 (#7847) 2019-01-02 11:11:03 +01:00
Vasiliy db07e1ca5e issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19 (#7819)
* issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19

* backport: add back required MSVC command replacement

* try to address MSVC link issues

* backport: move post-build step into a separate custom target

* another attempt to address cmake build issues
2018-12-21 15:00:35 +03:00
Wilfried Goesgens c3e73ba45b upgrade rocksdb to V5.18.X (#7792) 2018-12-19 19:34:55 +01:00
Andrey Abramov 7f4740b335
update ArangoSearch consolidation policy (#7801) 2018-12-19 01:55:08 +03:00
Vasiliy 8e7f0df775 issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup (#7786)
* issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup

* remove unnessesary checks

* revert last change due to test failures

* fix typo in change revert

* try to address random test failures
2018-12-18 18:45:11 +03:00
Max Neunhöffer 2846bdc393
Fix S2 for aarch64. (#7770) 2018-12-17 09:59:33 +01:00
Andrey Abramov 43dbe065d5
fix cmake parameters coincidence (#7774) 2018-12-15 01:56:32 +03:00
jsteemann 884f085917 remove now-unused file 2018-12-12 10:28:38 +01:00
Vasiliy dff5d53c31 issue 511.1: move iresearch data-store from IResearchView to IResearchLink (#7695)
* issue 511.1: move iresearch data-store from IResearchView to IResearchLink

* backport: address build failures on some compilers

* address MacOS build issue
2018-12-10 14:00:35 +03:00
Simon 8b0e0b0b75 Upgrade S2 Library (#7519) 2018-11-29 15:49:17 +01:00
Jan 389b677161
use jemalloc for client tools as well (#7506) 2018-11-28 19:42:52 +01:00
Andrey Abramov 95fd5793a1
Bug fix/internal issue #502 (#7480)
* update iresearch

* update iresearch
2018-11-27 22:53:19 +03:00
Wilfried Goesgens 285c842983 Bug fix/copy schmutz (#7441)
* don't copy files, rather source them from their actual location directly

* also locate the stemmer modules header
2018-11-26 16:35:03 +01:00
Vasiliy 1a0b9b9261 issue 153: ensure views are dropped in Agency when database is dropped in cluster, minor fixes (#7370)
* issue 153: ensure views are dropped in Agency when database is dropped in cluster, minor fixes

* backport: add test to ensure views are dropped when database is dropped from plan, fix some issues in ClusterInfo

* optimize primary key lookups in ArangoSearch

* fix test

* Add JS tests

* temporary comment optimizations
2018-11-21 19:18:34 +03:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
KVS85 8655a34239 Bug fix/fix vs15.9 codecvt macro (#7328)
* Fix std::codecvt<charXX_t...> marco to be used with VS 15.9

* Extend MSVC2017_ONLY macro with VS 15.9 vesion
2018-11-15 00:13:49 +03:00
Andrey Abramov de30a5f04f
Feature/arangosearch pk endianness (#7306)
* refactor arangosearch pks

* minor refactoring

* store PK as BigEndian since it leads to more compact index representation

* force iresearch to not to use libbfd

* fix tests
2018-11-14 01:42:38 +03:00
jsteemann 62645c1370 fix file privileges 2018-11-10 23:17:12 +01:00
Andrey Abramov 779809d3e1
Feature/arangosearch optimize documents reading (#7280)
* optimize reading documents from arangosearch index

* simplify code

* get rid of useless interface

* even more simplifications

* update iresearch to commit 40128bf50cea3546313fbfd71e5a32bb88e418a2

* optimize PK reading

* cleanup

* minor refactoring

* address review comments

* micro optimization
2018-11-09 18:10:22 +03:00
Vasiliy facb4043a1 issue 496.5: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291 (#7216)
* issue 496.5: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291

* add workaround for shell-collection-rocksdb-noncluster.js::testSystemSpecial test failure

* fix typo
2018-11-06 13:59:18 +03:00
Vasiliy f088733420 issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7166)
* issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address potential deadlock between link creation and FlushThread

* remove code causing nullptr access

* add back lock around reader reopen

* revert: address potential deadlock between link creation and FlushThread

* invalidate payload for each field in FieldIterator before setting a value
2018-11-01 23:10:01 +03:00
jsteemann 95ce366e27 libev not used anymore - remove it from .gitignore 2018-10-30 19:31:39 +01:00
Andrey Abramov 937a7eee5f
Feature/arangosearch speedup removals (#7134)
* speedup document removals and optimize data model

* fix invalid constexpr
2018-10-30 13:26:30 +03:00
Simon 84de3f6052 Fix undefined behaviour (#7108) 2018-10-29 19:02:24 +01:00
Andrey Abramov d30cf315fd
update codebase (#7125) 2018-10-29 16:59:39 +03:00
Simon 7113e779fb Next try at truncate (#7090) 2018-10-26 13:00:33 +02:00
Andrey Abramov 936af6f6ea
fix ASAN issues in arangosearch (#7026)
* update iresearch

* update iresearch
2018-10-23 22:30:30 +03:00
Jan df54403db6
const-expressify some VPack functions (#6993) 2018-10-23 11:32:46 +02:00
Simon 4cfff35cef Use SSE42 for UTF-8 string validation (#6991) 2018-10-23 11:32:06 +02:00
Vasiliy 5d2062ffbe issue 2921: make error message more detailed (#7004)
* issue 2921: make error message more detailed

* upgrade iresearch
2018-10-23 01:06:44 +03:00
Vasiliy 554770576c issue 494: update iresearch to changeset 6ff909851c760c2111dca5074ab17d35114bee97 (20181022) 2018-10-22 13:51:37 +03:00
Andrey Abramov ec3f9c6be9
Feature/update iresearch 2 (#6984)
* update iresearch sources

* another update

* update iresearch

* fix segfault in log message

* update iresearch

* update iresearch

* update iresearch

* remove noexcept

* improve logging

* update iresearch
2018-10-19 23:37:38 +03:00
Simon 23b8e5c735 Propper commit Sequence Numbers (#6958) (#6964) 2018-10-19 14:19:57 +02:00
Simon 8b7a4099b8 Properly compare velocypack objects in Agency operations (#6921)
* Properly compare velocypack objects in Agency operations

* Add changelog

* added option for VPackDumper
2018-10-17 20:03:53 +02:00
Vasiliy 9b36a94d89 issue 493.1: update iresearch to changeset 15f561dee5865ead83663ef056d7bbcc38636fb2 (#6947) 2018-10-17 18:13:15 +03:00
Tobias Gödderz f003d72323 Fix compile error on clang (#6884) 2018-10-15 15:58:26 +03:00
Vasiliy 78567bef09 update iresearch to codebase as of 20181011 (#6858)
* update iresearch to codebase as of 20181011

* backport: address cluster test failures

* backport: address dump test failures

* backport: address discrepency in view creation between single-server and cluster

* backport: address test failure on cluster (revert change)

* backport: address test failures

* backport: address MSVC build issues

* backport: address issue with LogicalDatasource destructing after TRI_vocbase_t

* Revert "backport: address issue with LogicalDatasource destructing after TRI_vocbase_t"

This reverts commit 4f9880bbaa22194dfbb604b5a54658de1d447ac1.
2018-10-12 21:07:12 +03:00
Simon fd81f52ab4 Allow WAL logger to split up transactions (#6800) (#6866) 2018-10-12 17:50:58 +02:00
jsteemann 544d57fb95 remove sddsd 2018-10-12 12:25:34 +02:00
jsteemann 4bcbbcb520 honor @graetzer's suggestion 2018-10-12 11:15:12 +02:00
Andrey Abramov 4177bd15b1
backport recent changes in iresearch from upstream (#6794) 2018-10-10 16:53:43 +03:00
Jan f140aa6941
added missing override specifiers (#6729)
* added missing override specifiers

* added two more occurrences of "override"
2018-10-05 12:03:18 +02:00
Andrey Abramov 647949e556
Feature/update iresearch (#6727)
* update iresearch part 1

* fix tests

* update iresearch again

* update iresearch again

* cleanup log messages

* update iresearch sources

* another update

* and another update

* add workaround for the bug in msvc2017

* fix typo

* address msvc build issues

* Fix typo
2018-10-04 23:56:17 +03:00
jsteemann 68a44490d6 try to fix MacOS build 2018-10-04 11:01:26 +02:00
jsteemann 8800780bc0 copy over some velocypack fixes 2018-10-02 18:44:27 +02:00
Jan c06f2d77da
Feature/velocypack update (#6678) 2018-10-02 14:04:14 +02:00
jsteemann 8667a8e001 when returning memory to the OS, use some memory protection flags as when initializing the memory
this prevents "hole punching" and keeps the OS from splitting one memory mapping into multiple mappings with different protection settings
2018-10-02 11:29:03 +02:00
Jan ce8d30c18f
forward-port some RocksDB bugfix (#6672) 2018-10-01 18:44:00 +02:00
Wilfried Goesgens a477df49cf Feature/windows utf16 fileaccess (#6534) 2018-09-24 19:41:17 +02:00
Simon 6db05b8b12 Do not retry connect on canceled connections (#6559) 2018-09-20 17:13:38 +02:00
Jan 6a2ad813bf
forward-port range deletion fix from upstream RocksDB (#6535) 2018-09-19 13:56:42 +02:00
KVS85 f18d575e65 Bug fix/vs2017 15.8.2 iresearch build (#6428)
* Add workaround

* Wrong check
2018-09-07 18:39:34 +03:00
Max Neunhöffer 0eab3c46f8 Upgrade to jemalloc-5.1.0 with patch for static linking. (#6167) 2018-09-07 16:13:58 +02:00
Jan 92ab03daf9
change compiler options to use C++14 (#6339) 2018-09-06 16:49:35 +02:00
jsteemann 0c1040874a remove unused submodule 2018-09-03 15:21:11 +02:00
Simon 0661a4c1fe Hide Links from getIndexes() (#6325) 2018-09-03 15:17:24 +02:00
Jan 3c638849bd
this fixes the "undefined" HTTP status codes in the generated documentation (#6322) 2018-08-31 20:15:17 +02:00
Wilfried Goesgens 975ada6d20 upgrade rocksdb to 5.16-fb (#6247) 2018-08-28 19:07:37 +02:00
Wilfried Goesgens 430f5229dd upgrade snappy (#6245) 2018-08-28 19:07:25 +02:00
Simon 7d1c0a1b17 Fix possible hang on OSX (#6263) 2018-08-27 19:24:54 +02:00
Jan 73e44bc605
invalidate query cache after modification of views (#6198) 2018-08-24 19:54:10 +02:00
Simon 948820e484 Various small changes (#6234) 2018-08-24 09:39:03 +02:00
Simon Grätzer 73640d6b7c Merge branch 'devel' of github.com:arangodb/arangodb into bug-fix/add-arangosearch-view-to-resilience-tests
# Conflicts:
#	3rdParty/fuerte/src/HttpConnection.cpp
2018-08-22 14:54:26 +02:00
Simon Grätzer d7b773f2c3 Fixing a race in fuerte 2018-08-22 14:48:42 +02:00
Simon 736e12e89c Fuerte retries connections (#6216) 2018-08-22 10:10:53 +02:00
Simon 247d41287b More fuerte changes (#6212) 2018-08-22 10:08:33 +02:00
Jan cb12be3e4e
fix crash when pressing CTRL+c in arangosh (#6187) 2018-08-21 11:12:11 +02:00
Jan e5864838d5
use rocksdb::Transaction::PopSavePoint() (#6192)
this prevents stacking many useless SavePoints in transactions with
many operations.

The PopSavePoint method in RocksDB is forward-ported from upstream
RocksDB commit 90f744941d48f15d5d070dfcab5d51b77abac8ad.
This is likely to be released with a future release of RocksDB.
2018-08-20 12:13:26 +02:00