1
0
Fork 0
Commit Graph

57 Commits

Author SHA1 Message Date
Andrey Abramov a793a4749f
Bug fix/rename iresearch analyzer feature (#9424)
* rename feature

* ensure 'MMFilesLogFileManager::SafeToUseInstance' is safe to use in multithreaded context

* address mac compilation issue
2019-07-08 15:46:57 +03:00
Dronplane 22d5df16e8 Bug fix/internal issue #606 (#9413)
* Fixed stem, norm, text analyzer creation without properties. Added tests.

* Removed non-determenistic test. Same functionality is tested by gtest (IResearchAnalyzerFeatureTest.test_remove)

* Added check for analyzer type existence.

* Made error messages for analyzer creation more specific and readable

* Fixed test

* Applied review suggestions
2019-07-08 13:11:22 +03:00
Dronplane b70d7372ef Bug fix/internal issue #586 (#9401)
* Removet lazy creation ob analyzers collection. Test fixes (added explicit creation of analyzers collection for test enviroments)

* Fixed test runs. Removed cluster tests for analyzer DDL

* Analyzers collection name moved to tests common

* Disbaled load_library in normalize calls

* Legacy code cleanup

* added db analyzers collection checks in analyzer creation test to cover functions removed from gtest

* Reverted analyzer properties comparsion to non-utf8 as that must be binary equal
2019-07-04 13:12:06 +03:00
Dronplane d561221201 Bug fix/fixed analyzer removal from db (#9359)
* Added missing commit. Updated tests to check actual deletion from db.

* Added check for commit result
2019-06-28 17:29:49 +02:00
Andrey Abramov 40cce2cee0
bug-fix/internal-issue-#565.1 (#9257)
* validate analyzer properties, do not return `null` for identity analyzer properties

* fix some tests

* Fixed tests for new analyzer parameters validation. Added explicit test for analyzer parameter validation

* update iresearch, fix analyzer tests

* fix more tests

* fix tests

* store analyzer properties as vpack

* more compilation fixes

* Updated iresearch

* Tests compilation fixed

* Test run fixes

* Test run fixes

* Fix test run

* Fixed all IresearchTests

* Fixed V8Analyzers tests

* Added ngram and delimeter analyzers vpack config

* Added ngram and stem analyzers vpack parsers. Added tests

* Fixed internal issue #593 Fixed build issue

* Fixed tests

* Fixed Gtest tests

* Changed test run to fix Mac failure

* Mac tests debugging

* Fixed jslint errors

* test tracing added

* Taken account for VPackSlice operator== false negatives
2019-06-28 10:49:01 +02:00
Andrey Abramov 0fb8b348db
bug-fix/internal-issue-#574 (#8991)
* restrict number of buckets to compare

* do not use more sort buckets than requested

* cleanup
2019-06-04 18:55:25 +03:00
Dronplane 1e24c79ca3 Bug fix/internal issue #565 (#9154)
* Merged latest iresearch. Removed creation of build-in analyzers from tests

* Updated tests for new analyzer parameter names

* Merged fix for format from iresearch

* Apply suggestions from code review. Corrected typo in variable name.

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* backport changes from IResearch upstream

* fix compilation issues

* backport fix from IResarch upstream

* Fixed analyzer parameter name in tests
2019-06-03 18:20:23 +03:00
Andrey Abramov e8b38bfa8e
bug-fix/internal-issue-#549 (#9086)
* do not persist legacy analyzers into _analyzers table

* fix arangosearch upgrade in cluster

* get rid of Vasiliy's shit

* address review comments

* ensure link is synchronized after creation in upgrade

* fix compilation error

* minor cleanup

* fix tests

* distribute '_analyzers' collection as '_graphs'

* comment out Vasiliy's shit part 2
2019-05-30 22:00:06 +03:00
Andrey Abramov 56696e3ea6 Bug fix/internal issue #577 (#9035)
* properly handle shorthand of system analyzer

* don't allow accessing local analyzer from within another database

* ensure local analyzer are not acessible from outside

* fix duplicate ids
2019-05-19 18:00:13 +02:00
Jan Christoph Uhde 970f732dec add exceptions for tokens function. (#8984)
* Provide users of the TOKENS function with errors.

* remove newlines

* fix compare

* fix as asked for

* improve fix
2019-05-16 22:12:43 +03:00
Andrey Abramov b7cfa24518
bug-fix/arangosearch-upgrade (#9010)
* fix upgrade

* fix catch tests

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-05-16 10:38:39 +03:00
Andrey Abramov 8358b978be disable arangosearch analyzers in agency (#8989)
* disable arangosearch analyzers

* do nothing in `stop()` if feature is disabled
2019-05-14 17:37:38 +02:00
Andrey Abramov 404bcc8b0c
switch to tiered consolidation by default (#8981)
* switch to tiered consolidation by default

* fix copy/paste error

* fix tests

* fix tests
2019-05-13 18:25:46 +03:00
Vasiliy 9f1340eeeb issue 563.1: include static analyzers in analyzer listing (#8925)
* issue 563.1: include static analyzers in analyzer listing

* address macos test failures
2019-05-11 14:52:48 +03:00
Vasiliy 6a5785be40 issue 568.1: restrict analyzer name to characters allowed for collection names (#8923) 2019-05-07 16:10:00 +03:00
Vasiliy 934313846a issue 526.12: store analyzer definitions in a separate entry of the serialized link meta (#8832) 2019-04-24 12:15:20 +02:00
Vasiliy 83a26cb003 issue 546.1: do not load analyzers in cluster for non-user requests (#8822) 2019-04-23 16:01:01 +02:00
Vasiliy 80c508c296 issue 526.11.1: limit maximum analyzer properties size (#8830) 2019-04-23 14:14:21 +02:00
Vasiliy 2de9510051 issue 539.1: backport from iresearch upstream: add support for text normalizing analyzer, make loading shared object ptional in the registry (#8813) 2019-04-20 23:20:15 +03:00
Vasiliy 52aa8a9be4 issue 526.10.1: store 'forPersistence' link definitions in the agency, manually query for analyzer definitions across the cluster (#8797) 2019-04-19 16:24:02 +03:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Vasiliy 030637c1c7 issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields (#8689)
* issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields

* address test failures

* remove debug logging
2019-04-10 13:10:25 +03:00
Vasiliy 0410588723 issue 526.7.1: remove deprecated functions, update tests to set up required analyzers (#8688)
* issue 526.7.1: remove deprecated functions, update tests to set up required analyzers

* address merge issues

* address more merge issues
2019-04-05 18:10:42 +03:00
Vasiliy f4919dc173 issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature (#8626)
* issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature

* address typo

* remove excess comments

* temporarily comment out tests failing on MacOS

* temporarily comment out more MacOS-only test failures
2019-04-02 19:23:28 +03:00
Vasiliy 3a3c360765 issue 526.5: add analyzer retrieval functionality (#8592) 2019-03-29 13:23:36 +01:00
Vasiliy 778417cdf0 issue 526.4.1: add analyzer removal and feature validation functionality (#8575) 2019-03-26 14:17:40 +03:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Vasiliy 405b60c2b7 issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections (#8452)
* issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections

* address merge issues

* address another merge issue
2019-03-21 14:23:36 +03:00
Vasiliy 29b941a677 issue 526.2.1: add upgrade step for legacy static analyzers (#8417) 2019-03-18 23:22:23 +03:00
Vasiliy e97cd8a0fb issue 526.1: initial support for vocbase-prefixed analyzer names (#8319)
* issue 526.1: initial support for vocbase-prefixed analyzer names

* backport: add tests for new functionality, add temporary workaround for legacy analyzers

* address review comments

* fix typo
2019-03-06 15:13:19 +03:00
Andrey Abramov 360b9f456c
reformat arangosearch sources (#7941) 2019-01-11 20:32:54 +03:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon 17a95710bd Micro-Optimize AQL CXX calls (#7486) (#7489)
(cherry picked from commit 8ddb9a063b)
2018-11-29 14:00:41 +01:00
Vasiliy 78567bef09 update iresearch to codebase as of 20181011 (#6858)
* update iresearch to codebase as of 20181011

* backport: address cluster test failures

* backport: address dump test failures

* backport: address discrepency in view creation between single-server and cluster

* backport: address test failure on cluster (revert change)

* backport: address test failures

* backport: address MSVC build issues

* backport: address issue with LogicalDatasource destructing after TRI_vocbase_t

* Revert "backport: address issue with LogicalDatasource destructing after TRI_vocbase_t"

This reverts commit 4f9880bbaa22194dfbb604b5a54658de1d447ac1.
2018-10-12 21:07:12 +03:00
Andrey Abramov 647949e556
Feature/update iresearch (#6727)
* update iresearch part 1

* fix tests

* update iresearch again

* update iresearch again

* cleanup log messages

* update iresearch sources

* another update

* and another update

* add workaround for the bug in msvc2017

* fix typo

* address msvc build issues

* Fix typo
2018-10-04 23:56:17 +03:00
Jan 6f10211f09
this moves aql::Query* out of the call parameter to any aql::Function (#6485) 2018-09-13 23:37:46 +02:00
Vasiliy a6f3444601 issue 458.2.1: ensure LogicalCollection presence is checked before granting/revoking permissions (#6274)
* issue 458.2.1: ensure LogicalCollection presence is checked before granting/revoking permissions

* try to address test failures

* backport: support wildcard for database too

* create collection before granting

* adjust ruby tests to expect behaviour as defined by issue #458

* adjust expected Ruby test result

* create required collection in Ruby test

* revert back to previous test code since Ruby refuses to create required collection

* missed revert
2018-08-29 14:57:56 +03:00
Vasiliy 953027ba63 issue 458.1: move SystemDatabaseFeature from IResearch to RestServer (#6260)
* issue 458.1: move SystemDatabaseFeature from IResearch to RestServer

* Add modified tests.
2018-08-28 09:48:09 +02:00
Vadim Kondratyev b6f2efebf1 Added document type to _iresearch_analyzers collection 2018-08-21 12:51:29 +02:00
Jan 7cb1da12cf
make AQL functions construction easier (#6195) 2018-08-20 11:56:38 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Jan 95d1892f18
remove canThrow attribute from AQL functions (#6123)
Its was somehow overlapping with the isDeterministic attribute (but with the opposite meaning)
2018-08-09 16:45:54 +02:00
Andrey Abramov b8e7752787 Feature/arangosearch stabilization (#6069)
* Added authentication tests for ArangoSearch views creation (client and task)

* Added authentication tests for ArangoSearch views querying (client and task)

* Added authentication tests for ArangoSearch views update (client and task) and correct other tests

* Added authentication tests for ArangoSearch views drop (client and task) and correct other tests

* Fix tests to support additional 'properties' layer of a view

* Add data to dumped/restored collection with a link from a view to single server test and view dump/restore to cluster tests

* backport from iresearch upstream: use a bitvector instead of an acceptor lambda for consolidation policies

* ensure view access permissions are checked for REST calls

* ensure view access permissions are checked for V8 calls

* do not allow LogicalView property modification if reading the properties fails

* Fixed authentication tests for ArangoSearch views to pass regardless of errorNum and possible multiple REST codes

* add missing ldap feature for enterprise version

* do not store useless data in ArangoSearch index (see #289, #441)
2018-08-05 20:47:22 +02:00
Vasiliy ec45ea1101 issue 430.1: add dedicated catch handlers for arangodb::basics::Exception (#5978) 2018-07-25 17:17:56 +03:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
Vasiliy 907767f4e1 issue 389.6: ensure proper error handling in StatisticsFeature for missing feature (#5377) 2018-05-18 01:12:39 +03:00
Vasiliy 843e584746 issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370)
* issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t&

* backport: address build issues
2018-05-17 01:15:50 +03:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Jan 30b12e311b
Bug fix/remove most of aql js (#5223) 2018-04-30 11:17:11 +02:00
Jan 2b84348b77
remove call to requiresElevatedPrivileges with default value (#5172) 2018-04-23 11:28:24 +02:00