History

jsteemann 56147843c2 Merge branch 'feature/additional-logging' of https://github.com/arangodb/arangodb into devel		2018-10-02 17:53:31 +02:00
..
CMakeLists.txt	Add optimizer rule that replaces js function calls (NEAR/WITHIN/FULLTEXT) with pure AQL (#5529 )	2018-06-12 13:48:31 +02:00
MMFilesAllocatorThread.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesAllocatorThread.h	…
MMFilesCleanupThread.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesCleanupThread.h	Remove Coordinator DBs (#5661 )	2018-06-25 19:18:11 +02:00
MMFilesCollection.cpp	Merge branch 'feature/additional-logging' of https://github.com/arangodb/arangodb into devel	2018-10-02 17:53:31 +02:00
MMFilesCollection.h	Resilience test failure points (#6539 )	2018-09-20 01:05:10 +02:00
MMFilesCollectionExport.cpp	Bug fix/add aql collection count cache (#6227 )	2018-08-23 16:05:51 +02:00
MMFilesCollectionExport.h	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionKeys.cpp	Bug fix/simplify things (#6516 )	2018-09-18 17:47:01 +02:00
MMFilesCollectionKeys.h	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionWriteLocker.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesCollectorCache.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesCollectorThread.cpp	allow master & slave to work in parallel for RocksDB WAL tailing (#6059 )	2018-08-03 13:37:53 +02:00
MMFilesCollectorThread.h	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
MMFilesCompactionFeature.cpp	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesCompactionFeature.h	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesCompactionLocker.h	…
MMFilesCompactorThread.cpp	issue 465.2.2: remove redudnant heap allocations and simplify API (#6349 )	2018-09-05 13:37:37 +03:00
MMFilesCompactorThread.h	Bug fix/add aql collection count cache (#6227 )	2018-08-23 16:05:51 +02:00
MMFilesDatafile.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesDatafile.h	fixed some rare assertion failures complaining about non-sealed datafiles (#6139 )	2018-08-09 21:55:13 +02:00
MMFilesDatafileHelper.h	View Replication (#5915 )	2018-07-26 10:28:46 +02:00
MMFilesDatafileStatistics.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesDatafileStatistics.h	do not complain about missing datafile statistics (#3688 )	2017-11-14 16:42:31 +01:00
MMFilesDatafileStatisticsContainer.cpp	…
MMFilesDatafileStatisticsContainer.h	…
MMFilesDitch.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesDitch.h	issue 399.2: use a LogicalCollection& instead of LogicalCollection* in PhysicalCollection (#5861 )	2018-07-13 15:02:43 +03:00
MMFilesDocumentOperation.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesDocumentOperation.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesDocumentPosition.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesEdgeIndex.cpp	Bug fix/simplify things (#6516 )	2018-09-18 17:47:01 +02:00
MMFilesEdgeIndex.h	take over selectivity estimates (#6505 )	2018-09-17 16:41:32 +02:00
MMFilesEngine.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesEngine.h	Bug fix/fixes 2508 (#6254 )	2018-08-27 21:36:39 +02:00
MMFilesExportCursor.cpp	fix non-blocking stream cursors (#6064 )	2018-08-03 12:41:52 +02:00
MMFilesExportCursor.h	fix non-blocking stream cursors (#6064 )	2018-08-03 12:41:52 +02:00
MMFilesFulltextIndex.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesFulltextIndex.h	Hide Links from getIndexes() (#6325 )	2018-09-03 15:17:24 +02:00
MMFilesGeoIndex.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesGeoIndex.h	Hide Links from getIndexes() (#6325 )	2018-09-03 15:17:24 +02:00
MMFilesHashIndex.cpp	Bug fix/simplify things (#6516 )	2018-09-18 17:47:01 +02:00
MMFilesHashIndex.h	take over selectivity estimates (#6505 )	2018-09-17 16:41:32 +02:00
MMFilesIncrementalSync.h	Bug fix/simplify things (#6516 )	2018-09-18 17:47:01 +02:00
MMFilesIndex.h	Use RangeDelete where possible (#6121 )	2018-08-15 18:52:09 +02:00
MMFilesIndexElement.cpp	allow master & slave to work in parallel for RocksDB WAL tailing (#6059 )	2018-08-03 13:37:53 +02:00
MMFilesIndexElement.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesIndexFactory.cpp	issue 399.4: use LogicalCollection& instead of LogicalCollection* in Index (#5929 )	2018-07-20 14:55:58 +03:00
MMFilesIndexFactory.h	issue 399.4: use LogicalCollection& instead of LogicalCollection* in Index (#5929 )	2018-07-20 14:55:58 +03:00
MMFilesLogfileManager.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesLogfileManager.h	attempt to make replication_sync more reliable in tests for MMFiles (#6184 )	2018-08-17 14:20:40 +02:00
MMFilesOptimizerRules.cpp	Disable JS on DBServer, fix race in UserManager (#6244 )	2018-08-24 22:20:49 +02:00
MMFilesOptimizerRules.h	Working geo index + tests	2017-05-09 15:04:51 +02:00
MMFilesPathBasedIndex.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesPathBasedIndex.h	Hide Links from getIndexes() (#6325 )	2018-09-03 15:17:24 +02:00
MMFilesPersistentIndex.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesPersistentIndex.h	take over selectivity estimates (#6505 )	2018-09-17 16:41:32 +02:00
MMFilesPersistentIndexFeature.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesPersistentIndexFeature.h	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesPersistentIndexKeyComparator.cpp	…
MMFilesPersistentIndexKeyComparator.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesPrimaryIndex.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesPrimaryIndex.h	take over selectivity estimates (#6505 )	2018-09-17 16:41:32 +02:00
MMFilesRemoverThread.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesRemoverThread.h	…
MMFilesRestExportHandler.cpp	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestExportHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestHandlers.cpp	Moved MMFiles replication handler.	2017-04-18 15:35:45 -04:00
MMFilesRestHandlers.h	…
MMFilesRestReplicationHandler.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesRestReplicationHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestWalHandler.cpp	attempt to make MMFiles replication tests more deterministic (#6219 )	2018-08-22 14:54:27 +02:00
MMFilesRestWalHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRevisionsCache.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesRevisionsCache.h	allow master & slave to work in parallel for RocksDB WAL tailing (#6059 )	2018-08-03 13:37:53 +02:00
MMFilesSkiplist.h	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
MMFilesSkiplistIndex.cpp	Bug fix/simplify things (#6516 )	2018-09-18 17:47:01 +02:00
MMFilesSkiplistIndex.h	take over selectivity estimates (#6505 )	2018-09-17 16:41:32 +02:00
MMFilesSynchronizerThread.cpp	improved logging for MMFiles IO operations (#4366 )	2018-01-19 18:37:01 +01:00
MMFilesSynchronizerThread.h	…
MMFilesTransactionCollection.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesTransactionCollection.h	fix premature unlock (#3802 )	2017-12-13 13:27:42 +01:00
MMFilesTransactionContextData.cpp	issue 344.6: remove some redundant functions (#4842 )	2018-03-15 11:03:35 +01:00
MMFilesTransactionContextData.h	…
MMFilesTransactionManager.cpp	do not create empty useless transaction data objects (#6430 )	2018-09-10 14:47:05 +02:00
MMFilesTransactionManager.h	optimize away own transaction management for RocksDB engine	2017-05-04 23:16:53 +02:00
MMFilesTransactionState.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesTransactionState.h	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesV8Functions.cpp	issue 465.2.2: remove redudnant heap allocations and simplify API (#6349 )	2018-09-05 13:37:37 +03:00
MMFilesV8Functions.h	fix db._engineStats()	2017-05-04 00:33:36 +02:00
MMFilesWalAccess.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalAccess.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesWalLogfile.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalLogfile.h	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalMarker.h	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
MMFilesWalRecoverState.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalRecoverState.h	renaming	2017-03-22 13:01:40 +01:00
MMFilesWalRecoveryFeature.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalRecoveryFeature.h	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesWalSlot.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalSlot.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSlots.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
MMFilesWalSlots.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSyncRegion.h	…
README.md	…
mmfiles-fulltext-common.h	move replication-dump into engine	2017-03-21 13:55:57 +01:00
mmfiles-fulltext-index.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
mmfiles-fulltext-index.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
mmfiles-fulltext-list.cpp	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
mmfiles-fulltext-list.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
mmfiles-fulltext-query.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-fulltext-query.h	move replication-dump into engine	2017-03-21 13:55:57 +01:00
mmfiles-replication-common.cpp	View Replication (#5915 )	2018-07-26 10:28:46 +02:00
mmfiles-replication-common.h	slight cleanup of replication-related code (#4286 )	2018-01-17 16:56:40 +01:00
mmfiles-replication-dump.cpp	Cleanup logging in MMFiles.	2018-10-01 16:30:10 -04:00
mmfiles-replication-dump.h	issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370 )	2018-05-17 01:15:50 +03:00

README.md

MMFilesEngine

How operations are stored - Overview

All operations like insert or remove are written as markers to a write ahead log (WAL). This WAL consists of files of a certain size and if such a file is full (or is manually flushed), all relevant markers are transferred (transferMarkers()) to the journals of the respective collections. During the transfer any obsolete markers will be thrown away: a sequence of insert, remove, insert on the same document will result in the last insert discarding the previous operations. When a journal file of size (journalSize()) is full, it will be sealed and renamed. By applying these operations it will become a datafile that is read-only. Datafiles will eventually be merged by a compactor thread that does about the same work as the transferMarkers function, reducing the size of the stored data.

Ditches

Ditches are used to pin objects in WAL or journal as long as they are used by other operations.