1
0
Fork 0
Commit Graph

58 Commits

Author SHA1 Message Date
Vasiliy 5959f7f8e8 issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step (#7918)
* issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step

* address merge issue

* address another merge issue

* and another merge issue

* run ./utils/reformat.sh on the source tree

* address review comments
2019-01-09 17:16:02 +03:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Vasiliy db07e1ca5e issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19 (#7819)
* issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19

* backport: add back required MSVC command replacement

* try to address MSVC link issues

* backport: move post-build step into a separate custom target

* another attempt to address cmake build issues
2018-12-21 15:00:35 +03:00
Vasiliy 00a4c34576 issue 511.5: remove cid from IResearchPrimaryKey since iresearch data-stores are per-collction, minor code cleanup (#7803)
* issue 511.5: remove cid from IResearchPrimaryKey since iresearch data-stores are per-collction, minr code cleanup

* get rid of CID field in a document

* make ArangoSearch PK encoding/decoding logic explicit
2018-12-19 16:10:58 +03:00
Andrey Abramov 0399379233
ensure all system attributes are properly indexed by ArangoSearch view (#7784) 2018-12-19 15:00:49 +03:00
Vasiliy 8e7f0df775 issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup (#7786)
* issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup

* remove unnessesary checks

* revert last change due to test failures

* fix typo in change revert

* try to address random test failures
2018-12-18 18:45:11 +03:00
Vasiliy f1bf5cc9cf issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server (#7748)
* issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server

* ensure objectId is set in RocksDBLink
2018-12-17 21:11:20 +03:00
Vasiliy f1245af554 issue 511.2.1: use references instead of raw pointers for Index operations to avoid null pointer access (#7725) 2018-12-11 14:40:49 +03:00
Vasiliy dff5d53c31 issue 511.1: move iresearch data-store from IResearchView to IResearchLink (#7695)
* issue 511.1: move iresearch data-store from IResearchView to IResearchLink

* backport: address build failures on some compilers

* address MacOS build issue
2018-12-10 14:00:35 +03:00
Vasiliy e16be8f3db issue 505.1: move index json comparison into index-type-specific implementations to address issue with incorrect definition comparison (#7430)
* issue 505: move index json comparison into index-type-specific implementations to address issue with incorrect definition comparison

* remove commented-out code

* fix typo

* backport: add additional logging and fix typo
2018-12-03 16:26:08 +03:00
Vasiliy facb4043a1 issue 496.5: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291 (#7216)
* issue 496.5: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291

* add workaround for shell-collection-rocksdb-noncluster.js::testSystemSpecial test failure

* fix typo
2018-11-06 13:59:18 +03:00
Vasiliy 68953ae33a issue 496.4.1: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7212) 2018-11-04 16:52:28 +03:00
Vasiliy f088733420 issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7166)
* issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address potential deadlock between link creation and FlushThread

* remove code causing nullptr access

* add back lock around reader reopen

* revert: address potential deadlock between link creation and FlushThread

* invalidate payload for each field in FieldIterator before setting a value
2018-11-01 23:10:01 +03:00
Vasiliy 8f44afb6cf issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion (#7101)
* issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion

* backport: address test failures

* backport: ensure arangosearch links get exported in the dump

* backport: ensure view is created during restore on the coordinator

* Updates for ArangoSearch DDL tests, IResearchView unregistration and known issues

* Add fix for internal issue 483
2018-10-30 12:50:35 +03:00
Simon 7113e779fb Next try at truncate (#7090) 2018-10-26 13:00:33 +02:00
Andrey Abramov 647949e556
Feature/update iresearch (#6727)
* update iresearch part 1

* fix tests

* update iresearch again

* update iresearch again

* cleanup log messages

* update iresearch sources

* another update

* and another update

* add workaround for the bug in msvc2017

* fix typo

* address msvc build issues

* Fix typo
2018-10-04 23:56:17 +03:00
jsteemann cc21a938c7 fixed typos 2018-10-02 18:19:12 +02:00
Dan Larkin-York 302f6d1852 Fix compile mistakes. 2018-10-02 08:55:49 -04:00
Dan Larkin-York fd38db0f2e Cleanup and additional logging in ArangoSearch. 2018-10-01 14:25:39 -04:00
Vasiliy 19ee42377e issue 456.6: use same code for link modification bot for single-server and cluster cases (#6414) 2018-09-07 18:09:38 +03:00
Vasiliy 1586b5128e issue 458.3: syntactic cleanup (#6286)
* issue 458.3: syntactic cleanup

* remove extraneous debug output
2018-08-29 17:20:00 +03:00
Vasiliy f7d7ce93fe issue 450.2: remove redundant functions from IResearchView API (#6197) 2018-08-20 16:38:17 +03:00
Simon 2d29232c59 fix restoring iresearch view in cluster (#6159) 2018-08-17 09:28:20 +02:00
Simon 392118bd62 Use RangeDelete where possible (#6121) 2018-08-15 18:52:09 +02:00
Simon 2dd8593609 View Replication (#5915) 2018-07-26 10:28:46 +02:00
Vasiliy c3279dedca issue 428.1: ensure IResearchLink::drop() releases locks on view (#5971) 2018-07-25 13:53:33 +03:00
Vasiliy 08d3cb3802 issue 399.4: use LogicalCollection& instead of LogicalCollection* in Index (#5929)
* issue 399.4: use LogicalCollection& instead of LogicalCollection* in Index

* backport: address enterprise build issues
2018-07-20 14:55:58 +03:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
Vasiliy 64666d0edd issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter (#5813)
* issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter

* set flag before notify, fix failures in new tests from 'devel'

* revert set flag before notify, fix more failures in new tests from 'devel'

* fix even more failures in new tests from 'devel', backport: do not reset notification flag if pending tasks were seen

* fix still more failures in new tests from 'devel'

* try to work around buggy test failures
2018-07-10 18:15:01 +03:00
Vasiliy 7aaeab50fb issue 402.1: share sync thread between IResearchView and IResearchViewDBServer (#5733) 2018-07-02 15:03:00 +03:00
Andrey Abramov 5eef6cd618
Feature/test iresearch (#5610)
* start implementing arangosearch cluster tests.

* backport: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties

* first attempt to fix failing tests

* refactor cluster wide view creation logic

* if view is not found in the new plan then check the old plan too

* ensure the cluster-wide view is looked up in vocbase as well on startup/recovery

* do not store cluster-wide IResearchView in vocbase

* move stale view cleanup to the shared pointer deleter, address test failures

* do not print warning

* enable arangosearch tests by default

* fix catch tests

* address icorrect return value for cluster-wide links

* address some issues with test failures due to cluster-view allocated within TRI_vocbase_t

* simplify per-cid view name, address 'catch' test failures

* ensure IResearchViewNode volatility is properly calculated in cluster

* invoke callbacks directly in AgencyMock instead of waiting for timeout

* ensure view updates via JavaScript always use the latest view definition

* pass a list of shards to `IResearchViewDBServer::snapshot`

* extend cluster aql tests

* fixes after merge

* fix class/struct inconsistencies

* comment failing tests

* remove debug logging

* add debug function

* tests cleanup

* simplify upcoming merge: pass resolver from a side

* backport: move all transaction status callback logic to Methods

* add changes missed from previous commit

* fix js and ruby tests

* more tests for IResearchViewNode

* pass transaction to IResearchViewDBServer::snapshot, address IResearchViewDBServer tests segfault

* pass transaction to IResearchView::snapshot instead of transaction state

* temporarily add trace log output to tests to try to find the cause of the core dump on Jenkins

* add more temporary debug output to trace down the segfault on Jenkins

* add even more temporary debug output to trace down the segfault on Jenkins

* ensure Vieew related maps are cleared during shutdown

* reset ClusterInfo::instance() before DatabaseFeature::unprepare()

* remove extraneous debug output

* missed line from previous commit

* uncomment required line

* add nullptr checks to RocksDBIndexFactory::prepareIndexes(...) similar to the ones in MMFilesIndexFactory::prepareIndexes(...)

* attempt to fix deadlock in tests

* add comment as per reviewer request

* fix aql test suite name

* add some debug logging

* address deadlock between ClusterInfo::loadPlan() and CollectionNameResolver::localNameLookup(...)

* eplicitly state which index definition failed in the log message

* use vocbase from shard-view isntead just in case

* explicitly state which index definition failed in the log message

* do not create shard-view instances from cluster-link instances (only register existing ones)

* add some tests
2018-06-21 20:35:04 +03:00
Vasiliy 3e983adb2d issue 381.6: enforce use of VIEW keyword for bound views and disallow for bound collections (#5522) 2018-06-05 14:45:53 +03:00
Vasiliy c26496c44c issue 381.1: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties, remove redundant functions (#5439) 2018-05-23 19:29:53 +03:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
Vasiliy 5a66661f78 issue 389.4: implement IResearchViewDBServer::snapshot (#5358)
* issue 389.4: implement IResearchViewDBServer::snapshot
* backport: revert back namespace header changes
* backport: address review comments
2018-05-16 16:33:48 +02:00
Andrey Abramov 671734b32f
do not scatter empty view (#5318)
* do not distribute scatter views without links

* simplify view downcasting

* refactor & add some tests

* adjust comments
2018-05-11 11:45:22 +03:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Vasiliy da4519df43 issue 383.2: backport from iresearch upstream: address issue with _T(…) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC (#5152)
* issue 383.2: backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC

* backport: add more tests for IResearchViewDBServer
2018-04-20 18:20:31 +03:00
Vasiliy 6053056b6a issue 383.1: initial stub implementation of IResearchViewDBServer (#5132) 2018-04-18 13:51:21 +03:00
Vasiliy 9a4ddee13e issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally (#5107)
* issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally

* remove invalid include
2018-04-16 18:30:51 +03:00
Vasiliy d1ce3a97ef issue 355.7: ensure LogicalDataSource::vocbase() returns a reference 2018-04-09 15:38:24 +03:00
Vasiliy e4368b0991 issue 355.6: remove create() from LogicalView, remove IResearch dependency from IndexFactory, store vocbase reference in LogicalDataSource 2018-04-06 16:38:34 +03:00
Vasiliy 635db3b409 issue 355.4: remove redundant methods and code, use 'cp' instead of 'cmake copy_directory' where possible, use vocbase reference instead of pointer 2018-04-04 10:53:48 +03:00
Vasiliy 7c25902b27 issue 355.3: allow IResearchLink creation, as opposed to IResearch View update, to direct which CIDs get included in an IResearchView snapshot 2018-04-02 15:27:48 +03:00
Andrey Abramov 65ea2bfbb9 code cleanup 2018-03-27 18:39:59 +03:00
Andrey Abramov b641cf9b13
remove view implementation part 1 2018-03-26 23:18:08 +03:00
Vasiliy 0823f23764 backport: address merge conflicts 2018-03-23 12:04:25 +03:00
Vasiliy eae3d0ade8 issue 348.2: hold view lock directly instead of via an Ayncself pointer 2018-03-23 10:28:00 +03:00
Andrey Abramov 01d9baf359 remove TRI_ERROR_ARANGO_VIEW_NOT_FOUND, rename TRI_ERROR_ARANGO_COLLECTION_NOT_FOUND to TRI_ERROR_ARNANGO_DATA_SOURCE_NOT_FOUND 2018-03-17 19:36:14 +03:00