1
0
Fork 0
Commit Graph

135 Commits

Author SHA1 Message Date
Andrey Abramov 8741d338d0 allow in-place analyzer creation via link definition (#10466) (#10481)
* allow in-place analyzer creation via link definition (#10466)

* allow in-place analyzer creation via link definition

* add special handling for _analyzers collection

* modify initial syncer

* address review commments

* fix accidentally broken test

* address compilation errors
2019-11-19 21:49:28 +01:00
Jan e7db6d3097 don't return any in-progress indexes (#10431)
* don't return any in-progress indexes

* fix handling of in-progress indexes

* add test

* address review comment
2019-11-14 12:09:41 +01:00
Andrey Abramov f4e6538edd Bug fix/internal issue #647 (#10292) (#10334)
* Bug fix/internal issue #647 (#10292)

* extend replication tests

* ensure proper replication order

* fix tests

* address review comments

* address test failures

* extend dump tests

* fix analyzers tests

* more fixes

* extend tests

* enhance tests

* adjust tests

* use enum instead of flags (part 1)

* cleanup

* use enum instead of flags (part 2)

* get rid of flags for views

* get rid of flags for collections

* completely get rid of collection flags

* fix replication test

* refactor index flags

* fix tests and move AnalyzerPool out of class scope

* fix tests

* adjust log levels

* add tests

* remove debug logging

* remove noexcept from `equalAanalyzer`

* extend cluster tests

* fix cluster tests

* add tests for views and smart graphs

* address jslint errors

# Conflicts:
#	arangod/Cluster/ClusterMethods.cpp
#	arangod/Cluster/v8-cluster.cpp
#	arangod/IResearch/IResearchAnalyzerFeature.cpp
#	arangod/IResearch/IResearchAnalyzerFeature.h
#	arangod/IResearch/IResearchLinkMeta.cpp
#	arangod/RestHandler/RestAnalyzerHandler.cpp
#	arangod/StorageEngine/PhysicalCollection.cpp
#	arangod/VocBase/Methods/Indexes.cpp
#	tests/IResearch/IResearchAnalyzerFeature-test.cpp
#	tests/IResearch/IResearchFeature-test.cpp
#	tests/IResearch/IResearchLinkHelper-test.cpp
#	tests/IResearch/IResearchLinkMeta-test.cpp
#	tests/IResearch/IResearchQueryOptimization-test.cpp
#	tests/IResearch/IResearchViewDBServer-test.cpp
#	tests/IResearch/IResearchViewSort-test.cpp
#	tests/V8Server/v8-analyzers-test.cpp
#	tests/VocBase/LogicalDataSource-test.cpp

* Update CHANGELOG

* Update CHANGELOG
2019-10-30 17:01:18 +03:00
Andrey Abramov 33ce94379f
bug-fix/internal-issue-#597 (#9324)
* ensure immutable properties of ArangoSearch are properly handled in cluster

* address jslint warnings
2019-06-25 17:15:07 +02:00
Simon cf7cf0131b Try to fix corruption error (#9258) 2019-06-25 10:18:26 +02:00
Andrey Abramov e8b38bfa8e
bug-fix/internal-issue-#549 (#9086)
* do not persist legacy analyzers into _analyzers table

* fix arangosearch upgrade in cluster

* get rid of Vasiliy's shit

* address review comments

* ensure link is synchronized after creation in upgrade

* fix compilation error

* minor cleanup

* fix tests

* distribute '_analyzers' collection as '_graphs'

* comment out Vasiliy's shit part 2
2019-05-30 22:00:06 +03:00
Andrey Abramov f5a14414e1 Bug fix/disable upgrade for views again 2 (#9084)
* temporary disable upgrade for views

* temporary comment out test for upgrade
2019-05-23 15:31:26 +02:00
Frank Celler fdbaeec732 Revert "temporary disable upgrade for views (#9083)"
This reverts commit b459a85498.
2019-05-23 15:20:09 +02:00
Andrey Abramov b459a85498 temporary disable upgrade for views (#9083) 2019-05-23 15:18:55 +02:00
Andrey Abramov b7cfa24518
bug-fix/arangosearch-upgrade (#9010)
* fix upgrade

* fix catch tests

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-05-16 10:38:39 +03:00
Dan Larkin-York 777ba1a364 Improved audit logging (#8740) 2019-04-15 14:51:32 +02:00
Jan 33c9e9e239 optimize away SortNode in case it is covered by an arangosearch view (#8701)
* optimize away SortNode in case it is covered by an arangosearch view

this implementation is a stub with hard-coded attribute names

* extend IResearchViewMeta with sorting order definition

* make 'IResearchViewMeta::Sort' compatible with 'SortCondition' API

* ensure ArangSearch sort is immutable after creation

* address review comments
2019-04-12 19:08:41 +03:00
Vasiliy 0410588723 issue 526.7.1: remove deprecated functions, update tests to set up required analyzers (#8688)
* issue 526.7.1: remove deprecated functions, update tests to set up required analyzers

* address merge issues

* address more merge issues
2019-04-05 18:10:42 +03:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Vasiliy 405b60c2b7 issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections (#8452)
* issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections

* address merge issues

* address another merge issue
2019-03-21 14:23:36 +03:00
Vasiliy 98b024d787 issue 466.6: remove unused code, remove hardcoded view meta values (#8130) 2019-02-11 16:32:04 +03:00
Vasiliy c52911e9f5 issue 466.5: switch to using WAL markers instead of FlushThread for recovery (#8124) 2019-02-07 17:43:01 +03:00
Andrey Abramov a2d0b9a3d1
Feature/arangosearch restrict collections (#7989)
* add ability to restrict collections scope of ArangoSearch view in query

* fail if collection which is not registered with a view was specified in options

* allow bound parameters to be used with ArangoSearch view options, add unit tests

* fix tests

* add tests for cluster

* fix issue in deseralization logic

* fix cluster tests, extend integration tests

* more tests

* fix failing cluster test

* address review comments

* address test issues
2019-01-21 17:49:37 +03:00
Vasiliy fac48ddbc1 issue 511.9.1: track DataSource instance instead of only its name in QueryCache (#7943)
* issue 511.9.1: track DataSource instance instead of only its name in QueryCache

* address use of invalid function

* fix expected test result

* address merge issue
2019-01-14 16:25:18 +03:00
Vasiliy 5959f7f8e8 issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step (#7918)
* issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step

* address merge issue

* address another merge issue

* and another merge issue

* run ./utils/reformat.sh on the source tree

* address review comments
2019-01-09 17:16:02 +03:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Simon 52be735244 iResearch asan issue (#7832) 2018-12-21 15:51:57 +01:00
Vasiliy db07e1ca5e issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19 (#7819)
* issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19

* backport: add back required MSVC command replacement

* try to address MSVC link issues

* backport: move post-build step into a separate custom target

* another attempt to address cmake build issues
2018-12-21 15:00:35 +03:00
Vasiliy 00a4c34576 issue 511.5: remove cid from IResearchPrimaryKey since iresearch data-stores are per-collction, minor code cleanup (#7803)
* issue 511.5: remove cid from IResearchPrimaryKey since iresearch data-stores are per-collction, minr code cleanup

* get rid of CID field in a document

* make ArangoSearch PK encoding/decoding logic explicit
2018-12-19 16:10:58 +03:00
Vasiliy 8e7f0df775 issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup (#7786)
* issue 511.4: ensure IResearchLink can be initialized without view as required for db-server, minor code fixes and cleanup

* remove unnessesary checks

* revert last change due to test failures

* fix typo in change revert

* try to address random test failures
2018-12-18 18:45:11 +03:00
Vasiliy f1bf5cc9cf issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server (#7748)
* issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server

* ensure objectId is set in RocksDBLink
2018-12-17 21:11:20 +03:00
Vasiliy dff5d53c31 issue 511.1: move iresearch data-store from IResearchView to IResearchLink (#7695)
* issue 511.1: move iresearch data-store from IResearchView to IResearchLink

* backport: address build failures on some compilers

* address MacOS build issue
2018-12-10 14:00:35 +03:00
Vasiliy e16be8f3db issue 505.1: move index json comparison into index-type-specific implementations to address issue with incorrect definition comparison (#7430)
* issue 505: move index json comparison into index-type-specific implementations to address issue with incorrect definition comparison

* remove commented-out code

* fix typo

* backport: add additional logging and fix typo
2018-12-03 16:26:08 +03:00
Jan adc651e338
Bug fix/cxxcheck 28112018 (#7518) 2018-11-29 13:39:04 +01:00
Andrey Abramov 95fd5793a1
Bug fix/internal issue #502 (#7480)
* update iresearch

* update iresearch
2018-11-27 22:53:19 +03:00
Vasiliy 59e002841f issue 506.3: use camel-case configuration parameter names consistntly, add a configuration version property to iresearch view meta (#7475)
* issue 506.3: use camel-case configuration parameter names consistntly, add a configuration version property to iresearch view meta

* backport: ensure meta version is supported

* backport: hide 'version' property from non-persistence json
2018-11-27 18:35:33 +03:00
Vasiliy 2d0a75f0d5 issue 506.2: add optimization to not reexecute a primary-key filter if a match was already found (#7460)
* issue 506.2: add optimization to not reexecute a primary-key filter if a match was already found

* backport: explicitly check type of instance of the primary-key filter

* backport: return non-null prepared filter and convert check to assert
2018-11-27 18:29:35 +03:00
Dan Larkin-York 452acf2a11 Port #7209. (#7386) 2018-11-21 09:35:18 +01:00
Andrey Abramov de30a5f04f
Feature/arangosearch pk endianness (#7306)
* refactor arangosearch pks

* minor refactoring

* store PK as BigEndian since it leads to more compact index representation

* force iresearch to not to use libbfd

* fix tests
2018-11-14 01:42:38 +03:00
Andrey Abramov 779809d3e1
Feature/arangosearch optimize documents reading (#7280)
* optimize reading documents from arangosearch index

* simplify code

* get rid of useless interface

* even more simplifications

* update iresearch to commit 40128bf50cea3546313fbfd71e5a32bb88e418a2

* optimize PK reading

* cleanup

* minor refactoring

* address review comments

* micro optimization
2018-11-09 18:10:22 +03:00
Vasiliy 68953ae33a issue 496.4.1: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7212) 2018-11-04 16:52:28 +03:00
Jan c7d76c5da4
fix some issues reported by codescan (#7195) 2018-11-02 19:04:39 +01:00
Vasiliy f088733420 issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7166)
* issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address potential deadlock between link creation and FlushThread

* remove code causing nullptr access

* add back lock around reader reopen

* revert: address potential deadlock between link creation and FlushThread

* invalidate payload for each field in FieldIterator before setting a value
2018-11-01 23:10:01 +03:00
Andrey Abramov 6a4f462fa2
reduce number of heap allocations for removals (#7157) 2018-10-31 01:21:25 +03:00
Vasiliy 21a89c9270 issue 496.2: allow customization of segmentsize lmts on arangosearch view creation, minor code cleanup (#7140)
* issue 496.2: allow customization of segmentsize lmts on arangosearch view creation, minor code clanup

* fix incorrect condition
2018-10-30 23:41:30 +03:00
Andrey Abramov 937a7eee5f
Feature/arangosearch speedup removals (#7134)
* speedup document removals and optimize data model

* fix invalid constexpr
2018-10-30 13:26:30 +03:00
Vasiliy 8f44afb6cf issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion (#7101)
* issue 496.1: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion

* backport: address test failures

* backport: ensure arangosearch links get exported in the dump

* backport: ensure view is created during restore on the coordinator

* Updates for ArangoSearch DDL tests, IResearchView unregistration and known issues

* Add fix for internal issue 483
2018-10-30 12:50:35 +03:00
Jan 7765a48bb6
speed up query string parsing (#7115) 2018-10-29 15:45:42 +01:00
Vasiliy 5d2062ffbe issue 2921: make error message more detailed (#7004)
* issue 2921: make error message more detailed

* upgrade iresearch
2018-10-23 01:06:44 +03:00
Vasiliy 554770576c issue 494: update iresearch to changeset 6ff909851c760c2111dca5074ab17d35114bee97 (20181022) 2018-10-22 13:51:37 +03:00
Andrey Abramov ec3f9c6be9
Feature/update iresearch 2 (#6984)
* update iresearch sources

* another update

* update iresearch

* fix segfault in log message

* update iresearch

* update iresearch

* update iresearch

* remove noexcept

* improve logging

* update iresearch
2018-10-19 23:37:38 +03:00
Vasiliy 6ae3bb056c issue 493.2: trigger termination of long-running consolidation on shutdown request (#6955) 2018-10-18 23:21:23 +03:00
Vasiliy 46ee6d94b0 issue 485.1: reduce scope of lock in FlushFeature (#6943) 2018-10-17 14:33:48 +03:00
Dan Larkin-York 9952cfbc95 Fix issues with ArangoSearch link permissions in cluster mode (#6925) 2018-10-16 20:42:11 +02:00