1
0
Fork 0
Commit Graph

262 Commits

Author SHA1 Message Date
Jan e7db6d3097 don't return any in-progress indexes (#10431)
* don't return any in-progress indexes

* fix handling of in-progress indexes

* add test

* address review comment
2019-11-14 12:09:41 +01:00
Andrey Abramov f4e6538edd Bug fix/internal issue #647 (#10292) (#10334)
* Bug fix/internal issue #647 (#10292)

* extend replication tests

* ensure proper replication order

* fix tests

* address review comments

* address test failures

* extend dump tests

* fix analyzers tests

* more fixes

* extend tests

* enhance tests

* adjust tests

* use enum instead of flags (part 1)

* cleanup

* use enum instead of flags (part 2)

* get rid of flags for views

* get rid of flags for collections

* completely get rid of collection flags

* fix replication test

* refactor index flags

* fix tests and move AnalyzerPool out of class scope

* fix tests

* adjust log levels

* add tests

* remove debug logging

* remove noexcept from `equalAanalyzer`

* extend cluster tests

* fix cluster tests

* add tests for views and smart graphs

* address jslint errors

# Conflicts:
#	arangod/Cluster/ClusterMethods.cpp
#	arangod/Cluster/v8-cluster.cpp
#	arangod/IResearch/IResearchAnalyzerFeature.cpp
#	arangod/IResearch/IResearchAnalyzerFeature.h
#	arangod/IResearch/IResearchLinkMeta.cpp
#	arangod/RestHandler/RestAnalyzerHandler.cpp
#	arangod/StorageEngine/PhysicalCollection.cpp
#	arangod/VocBase/Methods/Indexes.cpp
#	tests/IResearch/IResearchAnalyzerFeature-test.cpp
#	tests/IResearch/IResearchFeature-test.cpp
#	tests/IResearch/IResearchLinkHelper-test.cpp
#	tests/IResearch/IResearchLinkMeta-test.cpp
#	tests/IResearch/IResearchQueryOptimization-test.cpp
#	tests/IResearch/IResearchViewDBServer-test.cpp
#	tests/IResearch/IResearchViewSort-test.cpp
#	tests/V8Server/v8-analyzers-test.cpp
#	tests/VocBase/LogicalDataSource-test.cpp

* Update CHANGELOG

* Update CHANGELOG
2019-10-30 17:01:18 +03:00
Dronplane 2c72655dd5 Bug fix 3.5/issue #9612 (#9778)
* Bug fix/issue #9612 (#9764)

* Fixed ViewExecutionNode retrieval with deleted documents present in view

* Ported solution from 3.4 branch

* Changed index store in collection from vector to set. To make reversable indexes always last to execute

* Fixed re-enter hung

* Index storage fix

* Made index order deterministic

* Fix Mac build

* Added tests for index reversal

* Fixed Mac build

* Code cleanup

* Some cleanup

* Removed some redundand copy constructor calls

* Applied review comments

* Applied review comments

* Update CHANGELOG

* Update CHANGELOG
2019-08-21 17:59:59 +03:00
Jan d2b01a4c5e Bug fix 3.5/make ttl indexes behave like others (#9547)
* make TTL indexes behave like other indexes on creation

if a TTL index is already present on a collection, the previous behavior
was to make subsequent calls to `ensureIndex` fail unconditionally with
the error "there can only be one ttl index per collection".

now, we are comparing the attributes of the to-be-created index with the
attributes of the existing TTL index and make it only fail when the
attributes differ. if the attributes are identical, the `ensureIndex`
call succeeds and returns the existing index.

* added tests

* updated CHANGELOG
2019-07-23 15:46:50 +03:00
Andrey Abramov be88d8d499 bug-fix/internal-issue-#609 (#9464) (#9520)
* allow to access last operation tick of the transaction

* modify IResearchLink to rely on last commtted tick

* get rid of writing arangosearch markers to WAL

* further implementation

* local changes in iresearch

* add recovery states

* properly handle link creation during recovery

* adjust test cases

* properly handle nested transactions

* ungreylist recovery tests

* add more recovery tests

* do not use transaction to pass recovery tick

* do not store recoveryTick in MMFilesRecoveryState

* adjust tests

* fix mmfiles

* cleanup

* add context validity check

* fix tests

* fix more tests

* ensure subscription is not being released during commit

* address review comment

* final cleanup

* fix crash

* fix tests

* address test failures

* address review comments

* address review comments

* properly set recovery tick even if no recovery happened

# Conflicts:
#	arangod/IResearch/IResearchLink.cpp
2019-07-19 14:01:33 +03:00
Jan 64c8079186
Forbid indexes on _id (#9337) 2019-06-28 11:11:22 +02:00
Simon cf7cf0131b Try to fix corruption error (#9258) 2019-06-25 10:18:26 +02:00
Wilfried Goesgens 863257f1d5 fix windows warnings / compile errors (#9215) 2019-06-06 18:08:44 +02:00
Jan 3364ea6273
refactor index APIs (#9145) 2019-05-31 11:12:39 +02:00
Jan 445d020bac
fix a rare recovery error, add diagnostic output (#8931) 2019-05-08 14:24:26 +02:00
Jan 976dc2b726
Bug fix/issues 2019 05 06 (#8913) 2019-05-07 12:17:16 +02:00
Jan 0cbdfe9289
Bug fix/vpack update (#8875) 2019-04-30 12:33:26 +02:00
Dan Larkin-York 777ba1a364 Improved audit logging (#8740) 2019-04-15 14:51:32 +02:00
Simon 2b594bdab5 Reduce # of memcpy from storage engine layer (#8685) 2019-04-08 22:43:29 +02:00
Jan 5a19a0320b
remove/fix parameters in crud methods (#8675) 2019-04-03 19:55:15 +02:00
jsteemann 557637fa46 turn down log level for unimportant message 2019-04-03 16:08:40 +02:00
Jan 616ea94f24
Bug fix/cleanup 31032019 (#8632) 2019-04-01 17:14:11 +02:00
Dan Larkin-York ffc05c05c0 Fix handling of index name conflicts during replication. (#8571) 2019-03-28 18:53:43 +01:00
Jan a5e80fa7ed
remove redundant parameter from update API (#8607) 2019-03-28 12:50:20 +01:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Jan 39a3f5bc4e
reintroduce smart joins after temporarily reverting them in devel (#8543) 2019-03-23 20:36:02 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)"
This reverts commit 59ad583796.
2019-03-21 19:18:46 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Jan 6d107beeda
don't run compact() on a collection after a truncate() was done in th… (#8468)
* don't run compact() on a collection after a truncate() was done in the same transaction

running compact() in the same transaction will only increase the data size on disk due to RocksDB not being able to remove
any documents physically due to the snapshot we take at transaction start.
Decoupling the truncate transaction from the compact operation allows finishing the truncate transaction first, so we can
get rid of the snapshot. Running compact afterwards is then free to physically remove all the data.
As a nice side effect this change will also speed up the truncation of larger collections, because the compact will run
faster.

This change also exposes db.<collection>.compact() in the arangosh, in order to manually run a compaction on the data
range of a collection should it be needed for maintenance.

* fix documentation anchors
2019-03-20 16:27:54 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
Jan 9d3327c6ea
Bug fix/rearm cursors (#8363) 2019-03-12 15:28:33 +01:00
Jan 1798036ea0
Bug fix/optimizations 18022019 (#8180) 2019-02-19 19:24:04 +01:00
Jan 44c6a2d732
Feature/ttl index (#8169) 2019-02-19 14:12:21 +01:00
Jan f7d94daba2
port some changes from 3.4 (#8011) 2019-01-29 09:26:57 +01:00
Simon ba102323cc Refactoring index interface (#7971) 2019-01-17 16:33:46 +01:00
Lars Maier 12eebb15fe Feature/new server infra (#7733)
* Decoupled IO handling from Scheduler.

* Fixed SSL start up bug.

* Replaced Scheduler with new worker farm implementation.

* Added minimal statistics and info string for Scheduler.

* Added support for timed submissions.

* Updated delayed submission api. Updated code that used timers.

* Extracted new Scheduler into a virtual parent class. The implementation can now depend on the usecase.

* Signal handler now working.

* Changed threads names, `_stop` is atomic, check for failure during thread start + exception handling like old scheduler did.

* Commented on source code and added TODOs.

* Played around with start-stop-conditions

* Play around with start stop condition.

* start stop cond

* Sart Stop Conditions

* Removed bad cv_status check.

* Bug fix: now compare the actual objects instead of pointer values. Setup t1 and t2 depending on the thread id.

* Moved most of the stuff now unrelated to the Scheduler to GeneralServer. Got rid of JobGuard.

* Instead of waiting for a thread to terminate, put it on a clean up list and check for its termination in each supervisor run.

* Allow detaching long running threads.

* Fixed test mock.

* Updated the WorkHandle logic. Removed post functions.

* Fixed crash when obtaining shared_ptr from this in destructor.

* Added lost mutex.

* Fixed memory leak.

* Fixed merge bug.

* Changed a lot of code to optimize the scheduler.

* Fixed bug of invalidated iterator. Dont remove task on shutdown at different places. Let scheduler threads run until queue is empty.

* Only by value calls to queue.

* Added options again.

* Clean up of code.

* UI Request Lane added.

* Bug fixes in Scheduler.

* Applied reformat.

* Use sigaction.
2019-01-08 10:12:02 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Jan 7b571d311d
suppress a warning message about non-optimal MMFiles collection data structures while doing WAL recovery, not just while upgrading. (#7811) 2018-12-20 16:52:25 +01:00
Vasiliy f1245af554 issue 511.2.1: use references instead of raw pointers for Index operations to avoid null pointer access (#7725) 2018-12-11 14:40:49 +03:00
Dan Larkin-York d621387051 Disable warning about persistent IDs during upgrade procedure. (#7612) 2018-12-04 09:02:19 +01:00
Jan 1bef7e842c
do not allow MMFiles single document operations overtaking each other… (#7469) 2018-11-28 13:03:47 +01:00
Simon d0efd95a37 Bug fix/restore index refactor (#7470) 2018-11-27 20:22:36 +01:00
Jan 33f18942df
yet more micro optimizations (#7400) 2018-11-21 17:09:09 +01:00
Dan Larkin-York 452acf2a11 Port #7209. (#7386) 2018-11-21 09:35:18 +01:00
Tobias Gödderz c61ed1d77a MMFiles-replication-get-followers-under-lock (forward-port) (#7343)
* Forward-port of bug-fix-3.4/mmfiles-replication-get-followers-under-lock

Fix resign order

Fixed a typo

Get followers later, add TODOs

Added a callback parameter to collection insert methods

Get followers under the lock if necessary

Extracted the replication of inserts into a separate method

Move shortcut into replicate method

Added callbacks for remove, replace and update

Added missing overrides

Extracted replication code from modifyLocal and removeLocal

Update followers under lock also during replace, update, remove

Fix changes from the last commit for update/replace

Update comments, add asserts

Remove changes for document-level locks that will be done in another PR

Unify replication

Adapt log messages to the devel ones

Move common methods from its descendants to TransactionCollection, fix Mock on the way

More IResearch test / mock fixes

Relax asserts for nested transactions

Reformat

Fix non-babies remove and modify replication

* Remove some changes introduced by the merge

* Fixed compile errors introduced by merge
2018-11-20 09:43:26 +01:00
jsteemann 5c208c85ba fix assertion 2018-11-19 15:13:50 +01:00
Dan Larkin-York 32db9157bf Updates to collection versioning. (#7261) 2018-11-19 09:45:52 +01:00
jsteemann 5b2a557f94 removed spam comment 2018-11-15 17:39:21 +01:00
Dan Larkin-York b447f98a7a Upgrade old MMFiles collections for compatibility with ArangoSearch. (#7248) 2018-11-07 15:52:37 +01:00
Vasiliy 68953ae33a issue 496.4.1: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7212) 2018-11-04 16:52:28 +03:00
Matthew Von-Maszewski 97ba8ca2be Bugfix: More 3.4 scheduler changes backported (#7091) 2018-10-26 17:09:20 +02:00
Simon 7113e779fb Next try at truncate (#7090) 2018-10-26 13:00:33 +02:00
jsteemann 56147843c2 Merge branch 'feature/additional-logging' of https://github.com/arangodb/arangodb into devel 2018-10-02 17:53:31 +02:00