1
0
Fork 0
Commit Graph

305 Commits

Author SHA1 Message Date
Matthew Von-Maszewski d1d2edba57 Bugfix 3.4: Calculate Write Amplification / Flag to disable StatisticsWorker (#9874)
* release version 3.4.8

* [3.4] fix agency lockup when removing 404-ed callbacks (#9839)

* add write amplification statistic

* add --server.statistics-history flag that blocks StatisticsWorker

* no longer disable statistics for Agency

* update for statistics changes

* correct grammar

* correct grammar and only force statistical history off on Agent when option not explicitly used.

* correct grammar and only force statistical history off on Agent when option not explicitly used.

* correct variable naming
2019-09-11 12:54:33 +03:00
Tobias Gödderz 4ada35f20c [3.4] Bug fix 3.4/add shard id to replication client identifier (#9466)
* Bug fix/add shard id to replication client identifier (#9366)

* Fixed compile (but not linker) errors

* Backported ReplicationClientProgressTracker

* Fixed compile errors, fixed hash function

* No longer use SyncerId for real asynchronous replication

* Updated docu

* Try to fix compile error on windows

* Fixed a bug

* Removed old code

* Fixed CHANGELOG entry
2019-09-11 12:45:57 +03:00
Jan eb5f4241a5 issue #9654 (#9752) 2019-08-19 23:18:57 +03:00
Matthew Von-Maszewski 38b9ec80b8
Bugfix 3.4: Clean up rocksdb getStatistics() (#9703)
* clean up basic rocksdb statistics.  More clean-up needed.

* add rocksdb throttle value to statistics

* update comments about rocksdb properties

* rename function per PR comments
2019-08-13 13:10:45 -04:00
Jan fdbf2eb1e8
various replication improvements: (#9674)
- better debuggability (more log details)
- shorter minimum wait delay in active failover
- fixed too early pruning of WAL files on leaders
2019-08-12 10:44:50 +02:00
Jan 2de2d92054
remove archived RocksDB WAL files on agent servers (#9230) 2019-06-08 19:40:07 +02:00
jsteemann 5b66fa7aa0 RocksDB returns certain statistics values as strings. honor that 2019-04-26 19:05:58 +02:00
Jan a540b795b5
this addresses an issue described in ES-260, and is a forward-port of https://github.com/facebook/rocksdb/pull/5160 (#8697) 2019-04-08 18:20:50 +02:00
Jan d29d65face
Bug fix 3.4/cleanup 31032019 (#8633) 2019-04-01 17:02:00 +02:00
KVS85 c88c4d1416 Remove USE_IRESEARCH definition and usage (#8612) 2019-03-28 17:22:11 +01:00
Dan Larkin-York ea6a854b45 Add RocksDB background error listener. (#8537) 2019-03-23 20:46:07 +01:00
Jan dbfa483374
add RocksDB options `--rocksdb.allow-fallocate` and `--rocksdb.limit-open-files-at-startup` (#8492) 2019-03-21 12:05:57 +01:00
Jan 1c54830310
don't attempt to remove non-existing WAL files, because such attempt will trigger unnecessary error log messages in the RocksDB library (#8476) 2019-03-20 17:02:34 +01:00
Jan cd5c9edce1
various replication improvements (#8300) 2019-03-11 13:07:43 +01:00
Frank Celler 9477af198b big reformat 2018-12-26 00:57:05 +01:00
Jan 677522991e
Feature/internal 3306 (#7683) (#7688) 2018-12-06 17:46:58 +01:00
Jan b708edd059
fix assertion (#7526) 2018-11-29 15:43:06 +01:00
Jan f1086bac4f
added option `--rocksdb.enforce-block-cache-size-limit` (#7508) 2018-11-28 20:40:20 +01:00
Simon ef239cbe4e Make recovery more reliable (#7297) (#7367) 2018-11-21 16:51:38 +01:00
Simon 5124633e6a Faster index creation (#7348) 2018-11-20 13:41:01 +01:00
Vasiliy d644561f1f issue 496.4.1: backport 3.4: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7213)
* issue 496.4.1: backport 3.4: move StorageEngine-specific flag out of the genric API and closer to the storage engine

* address merge issue
2018-11-04 16:52:54 +03:00
Vasiliy e6a6025818 backport: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion (#7106)
* backport: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion

* backport: ensure arangosearch links get exported in the dump

* backport: ensure view is created during restore on the coordinator

* Updates for ArangoSearch DDL tests, IResearchView unregistration and known issues

* Add fix for internal issue 483
2018-10-30 12:50:29 +03:00
Jan 2d0fce0fec
add option `--rocksdb.total-write-buffer-size` (#7024) 2018-10-25 13:12:27 +02:00
Simon c2b6fb99ba Propper commit Sequence Numbers (#6958) 2018-10-18 18:25:34 +02:00
Andrey Abramov 4ef4e1119a
backport new arangosearch to 3.4 (#6874)
* backport new arangosearch

* backport tests

* backport another test

* backport changes in cmake
2018-10-13 01:27:31 +03:00
Jan 5a347017da
backport some fixes to 3.4 (#6739) 2018-10-08 09:43:14 +02:00
jsteemann dfba4b6381 added missing ifdef 2018-10-03 19:41:32 +02:00
Dan Larkin-York 3d15b96f66 [3.4] Additional and cleaned-up logging (#6648) 2018-10-01 19:03:05 +02:00
Jan 68e81da38e
handle DeleteRange markers on RocksDB recovery (#6565) 2018-09-28 11:10:11 +02:00
Jan b029186edc
Bug fix/aql modify cleanup (#6380) (#6458) 2018-09-12 13:56:59 +02:00
Jan 840ad8c20d
added option `--rocksdb.use-file-logging` (#6308) 2018-08-31 17:23:00 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Jan d78adb7ae1
Bug fix/turn off range deletes in cluster (#6178) 2018-08-17 08:41:00 +02:00
Simon 392118bd62 Use RangeDelete where possible (#6121) 2018-08-15 18:52:09 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Matthew Von-Maszewski 9f880dd529 (devel) add bool option rocksdb.debug-logging to easily enable rocksdb logging (#6016) 2018-07-30 14:30:20 +02:00
Jan 1f16898027
added more RocksDB options and stats (#6006) 2018-07-26 16:48:25 +02:00
Jan 21023945ba
disable RocksDB background syncing on Windows (#6004) 2018-07-26 16:48:11 +02:00
Simon 2dd8593609 View Replication (#5915) 2018-07-26 10:28:46 +02:00
Jan 21064144c8
Bug fix/replication improvements (#5962) 2018-07-25 09:04:50 +02:00
Jan 1a128799f2
finalize `--rocksdb.sync-interval` feature (#5856) 2018-07-17 11:09:57 +02:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
Vasiliy 1240090cd8 issue 399.2: use a LogicalCollection& instead of LogicalCollection* in PhysicalCollection (#5861) 2018-07-13 15:02:43 +03:00
Jan b88fde9659
added key generators "uuid" and "padded" (#5740) 2018-07-03 15:23:47 +02:00
Jan 297279f4f3
turn off MMFiles collection and compaction (#5734) 2018-07-03 15:23:27 +02:00
Simon 439b12331f Big endian document keys for rocksdb (#5712) 2018-07-03 14:11:22 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Vasiliy a5f274234a issue 380.2.2: remove extraneous pointers from StorageEngine API (#5589) 2018-06-13 15:22:51 +03:00
Jan Christoph Uhde f81e42af36 Add optimizer rule that replaces js function calls (NEAR/WITHIN/FULLTEXT) with pure AQL (#5529) 2018-06-12 13:48:31 +02:00