1
0
Fork 0
Commit Graph

62 Commits

Author SHA1 Message Date
Simon 2dd8593609 View Replication (#5915) 2018-07-26 10:28:46 +02:00
Vasiliy ec45ea1101 issue 430.1: add dedicated catch handlers for arangodb::basics::Exception (#5978) 2018-07-25 17:17:56 +03:00
Andrey Abramov 9205e2d9f8
allow user to explicitly track values for "exists" query (#5959)
* allow user to explicitly track values for "exists" query

* fix broken catch tests

* rename view link property

* fix tests after merge
2018-07-25 16:55:02 +03:00
Vasiliy fae47715eb issue 410.3: remove the extra neting of ArangoSearch view definition into 'properties' (#5956)
* issue 410.3: remove the extra neting of ArangoSearch view definition into 'properties'

* address merge issue

* backport: address test failures

* backport: address dump test failure
2018-07-24 16:27:28 +03:00
Vasiliy 3c384bf3e8 issue 410.1: ensure both JavaScript and Rest API for views matches API for collections (#5903)
* issue 410.1: ensure both JavaScript and Rest API for views matches API for collections
* backport: address some FIXMEs
* backport: address test failures
* backport: address more test failures
* backport: fix some Ruby tests
* temporarily comment out broken Ruby tests
* backport: fix more broken Ruby tests
* temporarily comment out failing Ruby tests
* backport: match result structure for View instances with Collection instances
* backport: return different result structure for view listing and view definition
* backport: fix even more broken Ruby tests
* backport: do not depend on unordered_set stable order
* update documentation
2018-07-19 15:07:17 +02:00
Vasiliy a401934d62 issue 399.3: remove internal persisted proerty 'collections' from externally generated IResearchView jSON (#5874)
* issue 399.3: remove internal persisted proerty 'collections' from externally generated IResearchView jSON

* backport: address test failure

* backport: address another test failure
2018-07-17 15:21:07 +03:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
Vasiliy 6ba3d98e39 issue 400.3: remove redundant asynchronous execution parameter (#5852) 2018-07-12 17:46:30 +03:00
Vasiliy 57d2517bd6 issue 399.1: store only transient link information by IResearchViewCoordinator (#5834)
* issue 399.1: store only transient link information by IResearchViewCoordinator

* some fixes
2018-07-11 21:21:51 +03:00
Vasiliy 64666d0edd issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter (#5813)
* issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter

* set flag before notify, fix failures in new tests from 'devel'

* revert set flag before notify, fix more failures in new tests from 'devel'

* fix even more failures in new tests from 'devel', backport: do not reset notification flag if pending tasks were seen

* fix still more failures in new tests from 'devel'

* try to work around buggy test failures
2018-07-10 18:15:01 +03:00
Simon f47367a495 Allow dumping / restoring of views (#5774)
* allow dumping and restoring of views

* fix restore, adding some tests

* remove debug print

* fix global replication
2018-07-05 17:55:28 +02:00
Vasiliy 9c04a6059c issue 401.1.1: implement an asynchronous task execution pool (thread pool) in IResearchFeature (#5761) 2018-07-04 15:18:25 +03:00
Vasiliy 7aaeab50fb issue 402.1: share sync thread between IResearchView and IResearchViewDBServer (#5733) 2018-07-02 15:03:00 +03:00
Vasiliy bf1f2eab29 issue 401.2: split IResearchViewMeta into the user configurable part and internal state (#5683)
* issue 401.2: split IResearchViewMeta into the user configurable part and internal state

* backport: update documentation and integration tests to reflect the IResearchViewMeta changes

* backport: addresss typo

* backport: address another typo
2018-06-27 16:40:51 +03:00
Vasiliy 532b4e613e issue 401.1: move IResearchView sync thread logic into its own class (#5668)
* issue 401.1: move IResearchView sync thread logic into its own class

* backport: use arangodb::Thread for ArangoSearch Sync
2018-06-26 15:47:56 +03:00
Andrey Abramov 5eef6cd618
Feature/test iresearch (#5610)
* start implementing arangosearch cluster tests.

* backport: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties

* first attempt to fix failing tests

* refactor cluster wide view creation logic

* if view is not found in the new plan then check the old plan too

* ensure the cluster-wide view is looked up in vocbase as well on startup/recovery

* do not store cluster-wide IResearchView in vocbase

* move stale view cleanup to the shared pointer deleter, address test failures

* do not print warning

* enable arangosearch tests by default

* fix catch tests

* address icorrect return value for cluster-wide links

* address some issues with test failures due to cluster-view allocated within TRI_vocbase_t

* simplify per-cid view name, address 'catch' test failures

* ensure IResearchViewNode volatility is properly calculated in cluster

* invoke callbacks directly in AgencyMock instead of waiting for timeout

* ensure view updates via JavaScript always use the latest view definition

* pass a list of shards to `IResearchViewDBServer::snapshot`

* extend cluster aql tests

* fixes after merge

* fix class/struct inconsistencies

* comment failing tests

* remove debug logging

* add debug function

* tests cleanup

* simplify upcoming merge: pass resolver from a side

* backport: move all transaction status callback logic to Methods

* add changes missed from previous commit

* fix js and ruby tests

* more tests for IResearchViewNode

* pass transaction to IResearchViewDBServer::snapshot, address IResearchViewDBServer tests segfault

* pass transaction to IResearchView::snapshot instead of transaction state

* temporarily add trace log output to tests to try to find the cause of the core dump on Jenkins

* add more temporary debug output to trace down the segfault on Jenkins

* add even more temporary debug output to trace down the segfault on Jenkins

* ensure Vieew related maps are cleared during shutdown

* reset ClusterInfo::instance() before DatabaseFeature::unprepare()

* remove extraneous debug output

* missed line from previous commit

* uncomment required line

* add nullptr checks to RocksDBIndexFactory::prepareIndexes(...) similar to the ones in MMFilesIndexFactory::prepareIndexes(...)

* attempt to fix deadlock in tests

* add comment as per reviewer request

* fix aql test suite name

* add some debug logging

* address deadlock between ClusterInfo::loadPlan() and CollectionNameResolver::localNameLookup(...)

* eplicitly state which index definition failed in the log message

* use vocbase from shard-view isntead just in case

* explicitly state which index definition failed in the log message

* do not create shard-view instances from cluster-link instances (only register existing ones)

* add some tests
2018-06-21 20:35:04 +03:00
Vasiliy b7f7711d30 issue 380.4: allow transaction status change and data-source registration callbacks to be registered with a transaction (#5624) 2018-06-18 13:16:32 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Vasiliy a5f274234a issue 380.2.2: remove extraneous pointers from StorageEngine API (#5589) 2018-06-13 15:22:51 +03:00
Vasiliy 1bd4947629 issue 381.7: move per-transaction IResearchView state from the view into the TransactionState (#5530) 2018-06-06 18:35:15 +03:00
Vasiliy ac1a6742e0 issue 381.4.1: create a separate IResearchLink implementation for use with ClusterEngine (#5505)
* issue 381.4.1: create a separate IResearchLink implementation for use with ClusterEngine

* backport: ensure StorageEngineMock assigns unique transaction ids

* delegate index normalization on the ClusterEngine to the 'actualEngine'
2018-06-01 19:05:53 +03:00
Vasiliy e021085aac issue 381.3.1: remove redundant copy of CollectionNameResolver from TransactionState (#5459) 2018-05-25 16:48:43 +03:00
Vasiliy c26496c44c issue 381.1: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties, remove redundant functions (#5439) 2018-05-23 19:29:53 +03:00
Simon 35992ad67b Coordinator storage engine (#5405) 2018-05-22 19:30:27 +02:00
Vasiliy 907767f4e1 issue 389.6: ensure proper error handling in StatisticsFeature for missing feature (#5377) 2018-05-18 01:12:39 +03:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Vasiliy 5a66661f78 issue 389.4: implement IResearchViewDBServer::snapshot (#5358)
* issue 389.4: implement IResearchViewDBServer::snapshot
* backport: revert back namespace header changes
* backport: address review comments
2018-05-16 16:33:48 +02:00
Vasiliy 5d8abce7bb issue 373.2: move toVelocyPack into LogicalDataSource (#5288)
* issue 373.2: move toVelocyPack into LogicalDataSource

* backport: move static DataSource related strings into StaticStrings, add support for registering snapshots on DBServer views
2018-05-08 17:28:39 +03:00
Andrey Abramov b69b5bdfdf
Bug fix/issue #5186 (#5269)
* do not persist view on startup

* small refactoring

* ensure view is being opened after creation
2018-05-06 20:38:32 +03:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Vasiliy da4519df43 issue 383.2: backport from iresearch upstream: address issue with _T(…) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC (#5152)
* issue 383.2: backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC, backport from iresearch upstream: address issue with _T(...) macro not working when _MBCS is defined on MSVC

* backport: add more tests for IResearchViewDBServer
2018-04-20 18:20:31 +03:00
Vasiliy 9a4ddee13e issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally (#5107)
* issue 374.4: move iresearch related log topic and type into a separate file so as not to expose it externally

* remove invalid include
2018-04-16 18:30:51 +03:00
Vasiliy d470371f7c issue 374.2: use a reference to vocbase instead of a pointer in StorageEngime LogicalDatasource related DDL operations, use vocbase from the found collection 2018-04-11 18:49:46 +03:00
Vasiliy d1ce3a97ef issue 355.7: ensure LogicalDataSource::vocbase() returns a reference 2018-04-09 15:38:24 +03:00
Vasiliy e4368b0991 issue 355.6: remove create() from LogicalView, remove IResearch dependency from IndexFactory, store vocbase reference in LogicalDataSource 2018-04-06 16:38:34 +03:00
Vasiliy 99b83ba8c8 issue 355.5: remove more unused methods, move view-related storage engine functionality from vocbase into DBServerLogicalView, address MSVC cmake dependency issue 2018-04-05 16:17:07 +03:00
Vasiliy 635db3b409 issue 355.4: remove redundant methods and code, use 'cp' instead of 'cmake copy_directory' where possible, use vocbase reference instead of pointer 2018-04-04 10:53:48 +03:00
Vasiliy fa13c72950 backport: address review comments 2018-04-03 10:12:38 +03:00
Vasiliy 7c25902b27 issue 355.3: allow IResearchLink creation, as opposed to IResearch View update, to direct which CIDs get included in an IResearchView snapshot 2018-04-02 15:27:48 +03:00
Andrey Abramov 9addef6419 small fixes after review 2018-03-28 18:45:14 +03:00
Andrey Abramov 65ea2bfbb9 code cleanup 2018-03-27 18:39:59 +03:00
Andrey Abramov a7760cb3b7 fix tests 2018-03-27 14:44:10 +03:00
Andrey Abramov b641cf9b13
remove view implementation part 1 2018-03-26 23:18:08 +03:00
Andrey Abramov f2703786f1
Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/refactor-logical-view 2018-03-23 15:17:17 +03:00
Andrey Abramov 47fd5808c3 remove PhysicalView part 2 2018-03-22 22:41:28 +03:00
Andrey Abramov 614cf73ff8
Merge pull request #4922 from arangodb/bug-fix/internal-issue-#348.1
issue 348.1: use a view validity lock instead of a view member lock for snapshot readers
2018-03-22 18:48:31 +03:00
Vasiliy a9b22ad36b issue 348.1: use a view validity lock instead of a view member lock for snapshot readers 2018-03-22 12:45:29 +03:00
Andrey Abramov 01d9baf359 remove TRI_ERROR_ARANGO_VIEW_NOT_FOUND, rename TRI_ERROR_ARANGO_COLLECTION_NOT_FOUND to TRI_ERROR_ARNANGO_DATA_SOURCE_NOT_FOUND 2018-03-17 19:36:14 +03:00
Vasiliy 148bdb7158 issue 344.6: remove some redundant functions (#4842) 2018-03-15 11:03:35 +01:00
Vasiliy c8739cd3cd manually-merge: cache data-sources in CollectionNameResolver 2018-03-14 10:11:50 +03:00