1
0
Fork 0
Commit Graph

69 Commits

Author SHA1 Message Date
Andrey Abramov 3a0ed2f8b9 Bug fix 3.5/internal issue #651 (#10388)
* extract recursive part of IResearchLinkMeta

* avoid saving analyzer definition during link meta initialization

* add tests to check cluster deadlock

* cleanup

* code cleanup

* add moar tests

* mooooaaar checks

* make jslint happy

* backport fix for cluster startup from 3.5.2 branch

* backport changes from 3.5.2

* fix tests

* fix tests

* moar fixes

* refactor analyzer definitions tracking

* add some tests

* extend dump tests

* add tests for replication

* fix tests

* more fixes

* address jslint errors

* address review comments

* address test failures

* reflect changes in changelog
2019-11-12 18:26:09 +03:00
Max Neunhöffer b186aebd8d Fix immediate shutdown after startup failure. (#10365)
* Add missing dependencies to IResearchAnalyzerFeature

* First all features beginShutdown, then stop.

This is for the case of immediate shutdown if the startup did not work.
2019-11-05 18:32:33 +03:00
Andrey Abramov f4e6538edd Bug fix/internal issue #647 (#10292) (#10334)
* Bug fix/internal issue #647 (#10292)

* extend replication tests

* ensure proper replication order

* fix tests

* address review comments

* address test failures

* extend dump tests

* fix analyzers tests

* more fixes

* extend tests

* enhance tests

* adjust tests

* use enum instead of flags (part 1)

* cleanup

* use enum instead of flags (part 2)

* get rid of flags for views

* get rid of flags for collections

* completely get rid of collection flags

* fix replication test

* refactor index flags

* fix tests and move AnalyzerPool out of class scope

* fix tests

* adjust log levels

* add tests

* remove debug logging

* remove noexcept from `equalAanalyzer`

* extend cluster tests

* fix cluster tests

* add tests for views and smart graphs

* address jslint errors

# Conflicts:
#	arangod/Cluster/ClusterMethods.cpp
#	arangod/Cluster/v8-cluster.cpp
#	arangod/IResearch/IResearchAnalyzerFeature.cpp
#	arangod/IResearch/IResearchAnalyzerFeature.h
#	arangod/IResearch/IResearchLinkMeta.cpp
#	arangod/RestHandler/RestAnalyzerHandler.cpp
#	arangod/StorageEngine/PhysicalCollection.cpp
#	arangod/VocBase/Methods/Indexes.cpp
#	tests/IResearch/IResearchAnalyzerFeature-test.cpp
#	tests/IResearch/IResearchFeature-test.cpp
#	tests/IResearch/IResearchLinkHelper-test.cpp
#	tests/IResearch/IResearchLinkMeta-test.cpp
#	tests/IResearch/IResearchQueryOptimization-test.cpp
#	tests/IResearch/IResearchViewDBServer-test.cpp
#	tests/IResearch/IResearchViewSort-test.cpp
#	tests/V8Server/v8-analyzers-test.cpp
#	tests/VocBase/LogicalDataSource-test.cpp

* Update CHANGELOG

* Update CHANGELOG
2019-10-30 17:01:18 +03:00
Jan 845701de6d fix broken & duplicate logIds (#10134) 2019-10-01 20:15:07 +03:00
Max Neunhöffer 328f46e3d6 This merges hotbackup and atomic-db-creation into 3.5. (#9968)
* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.
2019-09-11 13:13:54 +03:00
Dronplane 5bee042b65 Bug fix 3.5/internal issue #622 (#9787)
* Bug fix/internal issue #622 (#9781)

* Added analyzer cache invalidation for dropped database

* Fixed jslint reported errors

* Update CHANGELOG
2019-08-22 14:25:14 +03:00
Dronplane 5effbfd35a Porting fix for issue #9652 (#9727)
* Porting fix for issue #9652

* Update CHANGELOG
2019-08-16 13:27:51 +03:00
Frank Celler 2beb731264 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-07-11 15:22:32 +02:00
Andrey Abramov a793a4749f
Bug fix/rename iresearch analyzer feature (#9424)
* rename feature

* ensure 'MMFilesLogFileManager::SafeToUseInstance' is safe to use in multithreaded context

* address mac compilation issue
2019-07-08 15:46:57 +03:00
Dronplane 22d5df16e8 Bug fix/internal issue #606 (#9413)
* Fixed stem, norm, text analyzer creation without properties. Added tests.

* Removed non-determenistic test. Same functionality is tested by gtest (IResearchAnalyzerFeatureTest.test_remove)

* Added check for analyzer type existence.

* Made error messages for analyzer creation more specific and readable

* Fixed test

* Applied review suggestions
2019-07-08 13:11:22 +03:00
Dronplane b70d7372ef Bug fix/internal issue #586 (#9401)
* Removet lazy creation ob analyzers collection. Test fixes (added explicit creation of analyzers collection for test enviroments)

* Fixed test runs. Removed cluster tests for analyzer DDL

* Analyzers collection name moved to tests common

* Disbaled load_library in normalize calls

* Legacy code cleanup

* added db analyzers collection checks in analyzer creation test to cover functions removed from gtest

* Reverted analyzer properties comparsion to non-utf8 as that must be binary equal
2019-07-04 13:12:06 +03:00
Dronplane d561221201 Bug fix/fixed analyzer removal from db (#9359)
* Added missing commit. Updated tests to check actual deletion from db.

* Added check for commit result
2019-06-28 17:29:49 +02:00
Andrey Abramov 40cce2cee0
bug-fix/internal-issue-#565.1 (#9257)
* validate analyzer properties, do not return `null` for identity analyzer properties

* fix some tests

* Fixed tests for new analyzer parameters validation. Added explicit test for analyzer parameter validation

* update iresearch, fix analyzer tests

* fix more tests

* fix tests

* store analyzer properties as vpack

* more compilation fixes

* Updated iresearch

* Tests compilation fixed

* Test run fixes

* Test run fixes

* Fix test run

* Fixed all IresearchTests

* Fixed V8Analyzers tests

* Added ngram and delimeter analyzers vpack config

* Added ngram and stem analyzers vpack parsers. Added tests

* Fixed internal issue #593 Fixed build issue

* Fixed tests

* Fixed Gtest tests

* Changed test run to fix Mac failure

* Mac tests debugging

* Fixed jslint errors

* test tracing added

* Taken account for VPackSlice operator== false negatives
2019-06-28 10:49:01 +02:00
Frank Celler f021ea68f7 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-06-13 20:47:41 +02:00
Andrey Abramov 0fb8b348db
bug-fix/internal-issue-#574 (#8991)
* restrict number of buckets to compare

* do not use more sort buckets than requested

* cleanup
2019-06-04 18:55:25 +03:00
Dronplane 1e24c79ca3 Bug fix/internal issue #565 (#9154)
* Merged latest iresearch. Removed creation of build-in analyzers from tests

* Updated tests for new analyzer parameter names

* Merged fix for format from iresearch

* Apply suggestions from code review. Corrected typo in variable name.

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* backport changes from IResearch upstream

* fix compilation issues

* backport fix from IResarch upstream

* Fixed analyzer parameter name in tests
2019-06-03 18:20:23 +03:00
KVS85 fab06069db
Bug fix/adjust 3.5 rc3 (#9146)
* fix for windows

* Fix some uses of atomics (#9141)

* Remove std::move as it prevents copy elision (#9144)

* bug-fix/internal-issue-#549 (#9086)

* do not persist legacy analyzers into _analyzers table

* fix arangosearch upgrade in cluster

* get rid of Vasiliy's shit

* address review comments

* ensure link is synchronized after creation in upgrade

* fix compilation error

* minor cleanup

* fix tests

* distribute '_analyzers' collection as '_graphs'

* comment out Vasiliy's shit part 2

* Remove ArangoSearch views upgrade limit
2019-05-30 23:20:33 +03:00
Andrey Abramov e8b38bfa8e
bug-fix/internal-issue-#549 (#9086)
* do not persist legacy analyzers into _analyzers table

* fix arangosearch upgrade in cluster

* get rid of Vasiliy's shit

* address review comments

* ensure link is synchronized after creation in upgrade

* fix compilation error

* minor cleanup

* fix tests

* distribute '_analyzers' collection as '_graphs'

* comment out Vasiliy's shit part 2
2019-05-30 22:00:06 +03:00
Frank Celler 446a025960 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-05-22 09:54:14 +02:00
Andrey Abramov 56696e3ea6 Bug fix/internal issue #577 (#9035)
* properly handle shorthand of system analyzer

* don't allow accessing local analyzer from within another database

* ensure local analyzer are not acessible from outside

* fix duplicate ids
2019-05-19 18:00:13 +02:00
Jan Christoph Uhde 970f732dec add exceptions for tokens function. (#8984)
* Provide users of the TOKENS function with errors.

* remove newlines

* fix compare

* fix as asked for

* improve fix
2019-05-16 22:12:43 +03:00
Andrey Abramov b7cfa24518
bug-fix/arangosearch-upgrade (#9010)
* fix upgrade

* fix catch tests

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-05-16 10:38:39 +03:00
Jan bacb0886dc Bug fix/fix analyzer upgrade (#8990) 2019-05-14 23:29:21 +02:00
Andrey Abramov 8358b978be disable arangosearch analyzers in agency (#8989)
* disable arangosearch analyzers

* do nothing in `stop()` if feature is disabled
2019-05-14 17:37:38 +02:00
Andrey Abramov 404bcc8b0c
switch to tiered consolidation by default (#8981)
* switch to tiered consolidation by default

* fix copy/paste error

* fix tests

* fix tests
2019-05-13 18:25:46 +03:00
Vasiliy 9f1340eeeb issue 563.1: include static analyzers in analyzer listing (#8925)
* issue 563.1: include static analyzers in analyzer listing

* address macos test failures
2019-05-11 14:52:48 +03:00
Vasiliy 6a5785be40 issue 568.1: restrict analyzer name to characters allowed for collection names (#8923) 2019-05-07 16:10:00 +03:00
Vasiliy 934313846a issue 526.12: store analyzer definitions in a separate entry of the serialized link meta (#8832) 2019-04-24 12:15:20 +02:00
Vasiliy 83a26cb003 issue 546.1: do not load analyzers in cluster for non-user requests (#8822) 2019-04-23 16:01:01 +02:00
Vasiliy 80c508c296 issue 526.11.1: limit maximum analyzer properties size (#8830) 2019-04-23 14:14:21 +02:00
Vasiliy 2de9510051 issue 539.1: backport from iresearch upstream: add support for text normalizing analyzer, make loading shared object ptional in the registry (#8813) 2019-04-20 23:20:15 +03:00
Vasiliy 52aa8a9be4 issue 526.10.1: store 'forPersistence' link definitions in the agency, manually query for analyzer definitions across the cluster (#8797) 2019-04-19 16:24:02 +03:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Vasiliy 030637c1c7 issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields (#8689)
* issue 526.8: remove remaining deprecated code from analyzer feature, use simplified analyzer names for fields

* address test failures

* remove debug logging
2019-04-10 13:10:25 +03:00
Vasiliy 0410588723 issue 526.7.1: remove deprecated functions, update tests to set up required analyzers (#8688)
* issue 526.7.1: remove deprecated functions, update tests to set up required analyzers

* address merge issues

* address more merge issues
2019-04-05 18:10:42 +03:00
Vasiliy f4919dc173 issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature (#8626)
* issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature

* address typo

* remove excess comments

* temporarily comment out tests failing on MacOS

* temporarily comment out more MacOS-only test failures
2019-04-02 19:23:28 +03:00
Vasiliy 3a3c360765 issue 526.5: add analyzer retrieval functionality (#8592) 2019-03-29 13:23:36 +01:00
Vasiliy 778417cdf0 issue 526.4.1: add analyzer removal and feature validation functionality (#8575) 2019-03-26 14:17:40 +03:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Vasiliy 405b60c2b7 issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections (#8452)
* issue 526.3: update analyzer feature to store analyzer definitions in per-vocbase system collections

* address merge issues

* address another merge issue
2019-03-21 14:23:36 +03:00
Vasiliy 29b941a677 issue 526.2.1: add upgrade step for legacy static analyzers (#8417) 2019-03-18 23:22:23 +03:00
Vasiliy e97cd8a0fb issue 526.1: initial support for vocbase-prefixed analyzer names (#8319)
* issue 526.1: initial support for vocbase-prefixed analyzer names

* backport: add tests for new functionality, add temporary workaround for legacy analyzers

* address review comments

* fix typo
2019-03-06 15:13:19 +03:00
Andrey Abramov 360b9f456c
reformat arangosearch sources (#7941) 2019-01-11 20:32:54 +03:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon 17a95710bd Micro-Optimize AQL CXX calls (#7486) (#7489)
(cherry picked from commit 8ddb9a063b)
2018-11-29 14:00:41 +01:00
Vasiliy 78567bef09 update iresearch to codebase as of 20181011 (#6858)
* update iresearch to codebase as of 20181011

* backport: address cluster test failures

* backport: address dump test failures

* backport: address discrepency in view creation between single-server and cluster

* backport: address test failure on cluster (revert change)

* backport: address test failures

* backport: address MSVC build issues

* backport: address issue with LogicalDatasource destructing after TRI_vocbase_t

* Revert "backport: address issue with LogicalDatasource destructing after TRI_vocbase_t"

This reverts commit 4f9880bbaa22194dfbb604b5a54658de1d447ac1.
2018-10-12 21:07:12 +03:00
Andrey Abramov 647949e556
Feature/update iresearch (#6727)
* update iresearch part 1

* fix tests

* update iresearch again

* update iresearch again

* cleanup log messages

* update iresearch sources

* another update

* and another update

* add workaround for the bug in msvc2017

* fix typo

* address msvc build issues

* Fix typo
2018-10-04 23:56:17 +03:00
Jan 6f10211f09
this moves aql::Query* out of the call parameter to any aql::Function (#6485) 2018-09-13 23:37:46 +02:00
Vasiliy a6f3444601 issue 458.2.1: ensure LogicalCollection presence is checked before granting/revoking permissions (#6274)
* issue 458.2.1: ensure LogicalCollection presence is checked before granting/revoking permissions

* try to address test failures

* backport: support wildcard for database too

* create collection before granting

* adjust ruby tests to expect behaviour as defined by issue #458

* adjust expected Ruby test result

* create required collection in Ruby test

* revert back to previous test code since Ruby refuses to create required collection

* missed revert
2018-08-29 14:57:56 +03:00
Vasiliy 953027ba63 issue 458.1: move SystemDatabaseFeature from IResearch to RestServer (#6260)
* issue 458.1: move SystemDatabaseFeature from IResearch to RestServer

* Add modified tests.
2018-08-28 09:48:09 +02:00