1
0
Fork 0
Commit Graph

117 Commits

Author SHA1 Message Date
Vasiliy fac48ddbc1 issue 511.9.1: track DataSource instance instead of only its name in QueryCache (#7943)
* issue 511.9.1: track DataSource instance instead of only its name in QueryCache

* address use of invalid function

* fix expected test result

* address merge issue
2019-01-14 16:25:18 +03:00
Vasiliy 5959f7f8e8 issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step (#7918)
* issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step

* address merge issue

* address another merge issue

* and another merge issue

* run ./utils/reformat.sh on the source tree

* address review comments
2019-01-09 17:16:02 +03:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Simon 52be735244 iResearch asan issue (#7832) 2018-12-21 15:51:57 +01:00
Vasiliy db07e1ca5e issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19 (#7819)
* issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19

* backport: add back required MSVC command replacement

* try to address MSVC link issues

* backport: move post-build step into a separate custom target

* another attempt to address cmake build issues
2018-12-21 15:00:35 +03:00
Vasiliy 00a4c34576 issue 511.5: remove cid from IResearchPrimaryKey since iresearch data-stores are per-collction, minor code cleanup (#7803)
* issue 511.5: remove cid from IResearchPrimaryKey since iresearch data-stores are per-collction, minr code cleanup

* get rid of CID field in a document

* make ArangoSearch PK encoding/decoding logic explicit
2018-12-19 16:10:58 +03:00
Vasiliy 8e7f0df775 issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup (#7786)
* issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup

* remove unnessesary checks

* revert last change due to test failures

* fix typo in change revert

* try to address random test failures
2018-12-18 18:45:11 +03:00
Vasiliy f1bf5cc9cf issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server (#7748)
* issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server

* ensure objectId is set in RocksDBLink
2018-12-17 21:11:20 +03:00
Vasiliy dff5d53c31 issue 511.1: move iresearch data-store from IResearchView to IResearchLink (#7695)
* issue 511.1: move iresearch data-store from IResearchView to IResearchLink

* backport: address build failures on some compilers

* address MacOS build issue
2018-12-10 14:00:35 +03:00
Vasiliy e16be8f3db issue 505.1: move index json comparison into index-type-specific implementations to address issue with incorrect definition comparison (#7430)
* issue 505: move index json comparison into index-type-specific implementations to address issue with incorrect definition comparison

* remove commented-out code

* fix typo

* backport: add additional logging and fix typo
2018-12-03 16:26:08 +03:00
Jan adc651e338
Bug fix/cxxcheck 28112018 (#7518) 2018-11-29 13:39:04 +01:00
Andrey Abramov 95fd5793a1
Bug fix/internal issue #502 (#7480)
* update iresearch

* update iresearch
2018-11-27 22:53:19 +03:00
Vasiliy 59e002841f issue 506.3: use camel-case configuration parameter names consistntly, add a configuration version property to iresearch view meta (#7475)
* issue 506.3: use camel-case configuration parameter names consistntly, add a configuration version property to iresearch view meta

* backport: ensure meta version is supported

* backport: hide 'version' property from non-persistence json
2018-11-27 18:35:33 +03:00
Vasiliy 2d0a75f0d5 issue 506.2: add optimization to not reexecute a primary-key filter if a match was already found (#7460)
* issue 506.2: add optimization to not reexecute a primary-key filter if a match was already found

* backport: explicitly check type of instance of the primary-key filter

* backport: return non-null prepared filter and convert check to assert
2018-11-27 18:29:35 +03:00
Dan Larkin-York 452acf2a11 Port #7209. (#7386) 2018-11-21 09:35:18 +01:00
Andrey Abramov de30a5f04f
Feature/arangosearch pk endianness (#7306)
* refactor arangosearch pks

* minor refactoring

* store PK as BigEndian since it leads to more compact index representation

* force iresearch to not to use libbfd

* fix tests
2018-11-14 01:42:38 +03:00
Andrey Abramov 779809d3e1
Feature/arangosearch optimize documents reading (#7280)
* optimize reading documents from arangosearch index

* simplify code

* get rid of useless interface

* even more simplifications

* update iresearch to commit 40128bf50cea3546313fbfd71e5a32bb88e418a2

* optimize PK reading

* cleanup

* minor refactoring

* address review comments

* micro optimization
2018-11-09 18:10:22 +03:00
Vasiliy 68953ae33a issue 496.4.1: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7212) 2018-11-04 16:52:28 +03:00
Jan c7d76c5da4
fix some issues reported by codescan (#7195) 2018-11-02 19:04:39 +01:00
Vasiliy f088733420 issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7166)
* issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address potential deadlock between link creation and FlushThread

* remove code causing nullptr access

* add back lock around reader reopen

* revert: address potential deadlock between link creation and FlushThread

* invalidate payload for each field in FieldIterator before setting a value
2018-11-01 23:10:01 +03:00
Andrey Abramov 6a4f462fa2
reduce number of heap allocations for removals (#7157) 2018-10-31 01:21:25 +03:00
Vasiliy 21a89c9270 issue 496.2: allow customization of segmentsize lmts on arangosearch view creation, minor code cleanup (#7140)
* issue 496.2: allow customization of segmentsize lmts on arangosearch view creation, minor code clanup

* fix incorrect condition
2018-10-30 23:41:30 +03:00
Andrey Abramov 937a7eee5f
Feature/arangosearch speedup removals (#7134)
* speedup document removals and optimize data model

* fix invalid constexpr
2018-10-30 13:26:30 +03:00
Vasiliy 8f44afb6cf issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion (#7101)
* issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion

* backport: address test failures

* backport: ensure arangosearch links get exported in the dump

* backport: ensure view is created during restore on the coordinator

* Updates for ArangoSearch DDL tests, IResearchView unregistration and known issues

* Add fix for internal issue 483
2018-10-30 12:50:35 +03:00
Jan 7765a48bb6
speed up query string parsing (#7115) 2018-10-29 15:45:42 +01:00
Vasiliy 5d2062ffbe issue 2921: make error message more detailed (#7004)
* issue 2921: make error message more detailed

* upgrade iresearch
2018-10-23 01:06:44 +03:00
Vasiliy 554770576c issue 494: update iresearch to changeset 6ff909851c760c2111dca5074ab17d35114bee97 (20181022) 2018-10-22 13:51:37 +03:00
Andrey Abramov ec3f9c6be9
Feature/update iresearch 2 (#6984)
* update iresearch sources

* another update

* update iresearch

* fix segfault in log message

* update iresearch

* update iresearch

* update iresearch

* remove noexcept

* improve logging

* update iresearch
2018-10-19 23:37:38 +03:00
Vasiliy 6ae3bb056c issue 493.2: trigger termination of long-running consolidation on shutdown request (#6955) 2018-10-18 23:21:23 +03:00
Vasiliy 46ee6d94b0 issue 485.1: reduce scope of lock in FlushFeature (#6943) 2018-10-17 14:33:48 +03:00
Dan Larkin-York 9952cfbc95 Fix issues with ArangoSearch link permissions in cluster mode (#6925) 2018-10-16 20:42:11 +02:00
Vasiliy 18b2eb6d98 issue 485: ensure LogicalDataSource::drop() is called on vocbase drop (#6895)
* issue 485: ensure LogicalDataSource::drop() is called on vocbase drop

* add missed change

* backport: address race between make(...) and async job

* add another missed change

* backport: ensure recursive lock reports itself as locked correctly

* backport: address test failure on mmfiles

* backport: remove redundant lock already held by async task

* backport: reset reader before unlinking directory
2018-10-16 17:03:51 +03:00
Vasiliy 78567bef09 update iresearch to codebase as of 20181011 (#6858)
* update iresearch to codebase as of 20181011

* backport: address cluster test failures

* backport: address dump test failures

* backport: address discrepency in view creation between single-server and cluster

* backport: address test failure on cluster (revert change)

* backport: address test failures

* backport: address MSVC build issues

* backport: address issue with LogicalDatasource destructing after TRI_vocbase_t

* Revert "backport: address issue with LogicalDatasource destructing after TRI_vocbase_t"

This reverts commit 4f9880bbaa22194dfbb604b5a54658de1d447ac1.
2018-10-12 21:07:12 +03:00
Andrey Abramov b8dcf51631
Bug fix/arangosearch lock issue (#6871)
* ensure no concurrent links midifications

* another attempt to fix

* reduce scope of the lock

* Revert "reduce scope of the lock"

This reverts commit 2eed8eb75c372c1723bcd46b05018843edcc51e0.
2018-10-12 20:50:28 +03:00
Andrey Abramov 287b44d66f
Bug fix/avoid cache double invalidation (#6801)
* avoid query cache double invalidation

* do not invalidate cache twice

* can't use CAS on directory_reader for the time being, protect with mutex instead

* do not lock properties during index consolidation

* reduce scope of _callbacksLock in FlushFeature

* prevent concurrent reader reopen
2018-10-11 14:23:34 +03:00
Andrey Abramov 752714de24
Feature/arangosearch overhaul (#6743)
* get rid of memory stores

* do not use 'waitForSync' transaction option to synchronize a view

* honor `Methods::isSingleOperationTransaction()` hint

* explicitly specify index_writer's properties

* ensure data is synced

* fix tests

* another fix

* code cleanup

* remove code duplication

* some fixes

* fix typo
2018-10-10 01:13:08 +03:00
Andrey Abramov 647949e556
Feature/update iresearch (#6727)
* update iresearch part 1

* fix tests

* update iresearch again

* update iresearch again

* cleanup log messages

* update iresearch sources

* another update

* and another update

* add workaround for the bug in msvc2017

* fix typo

* address msvc build issues

* Fix typo
2018-10-04 23:56:17 +03:00
Dan Larkin-York fd38db0f2e Cleanup and additional logging in ArangoSearch. 2018-10-01 14:25:39 -04:00
Lars Maier e2930ccba6 Completely export views dump restore (#6466) 2018-09-12 17:21:22 +02:00
Vasiliy 1586b5128e issue 458.3: syntactic cleanup (#6286)
* issue 458.3: syntactic cleanup

* remove extraneous debug output
2018-08-29 17:20:00 +03:00
Andrey Abramov 615e1cf7de
Feature/add options to view (#6255)
* added OPTIONS for views

* pass options too

* validation for OPTIONS

* fix OPTIONS validation

* removed debug log message

* add forceSync options

* add forceSync view option

* fix comments

* fix some memleaks

* fix typo

* fix integration tests

* fix integration tests

* fix another integration test

* rename 'forceSync' option to 'waitForSync'

* another attempt to fix test

* ensure query caches are properly invalidated

* address integration test failures

* another attempt to fix test
2018-08-26 22:30:23 +03:00
Vasiliy 5d14775de8 issue 459.3: ensure collection permissions are checked before updating/dropping an IResearch view (#6253)
* issue 459.3: ensure collection permissions are checked before updating/dropping an IResearch view

* backport: ensure collection permissions are checked before updating/dropping an IResearch view on cluster

* backport: address test failures

* backport: address more test failures

* reuse existing classes for scoping ExecContext
2018-08-26 18:00:16 +03:00
Jan 73e44bc605
invalidate query cache after modification of views (#6198) 2018-08-24 19:54:10 +02:00
Vasiliy b55546225f issue 450.6: simplify consolidation policy typing (#6235) 2018-08-24 01:25:23 +03:00
Vasiliy e04b5318b4 issue 450.5: rename consolidation-related configuration properties and remove locale (#6232) 2018-08-23 16:04:25 +03:00
Vasiliy 61574755b6 issue 450.4: address issues with impropper comparison (#6217) 2018-08-22 15:49:56 +03:00
Vasiliy 325e61b4ed issue 450.3: use a single configuration policy differentiated by 'type' with arbitrary number of configurable parameters (#6205)
* issue 450.3: use a single configuration policy differentiated by 'type' with arbitrary number of configurable parameters

* ensure test data is persisted to the test directory

* add missed declaration

* fix typo

* add full namespace and includes
2018-08-21 14:46:28 +03:00
Vasiliy f7d7ce93fe issue 450.2: remove redundant functions from IResearchView API (#6197) 2018-08-20 16:38:17 +03:00
Vasiliy c08ab01f67 issue 450.1: remove the "commit" configuration layer from ArangoSearch properties (#6185)
* issue 450.1: remove the "commit" configuration layer from ArangoSearch properties

* fix broken tests
2018-08-20 02:18:02 +03:00