History

Jan f4b99bb5ed Bug fix/meierloch (#5895 )		2018-07-17 11:39:50 +02:00
..
CMakeLists.txt	Add optimizer rule that replaces js function calls (NEAR/WITHIN/FULLTEXT) with pure AQL (#5529 )	2018-06-12 13:48:31 +02:00
MMFilesAllocatorThread.cpp	…
MMFilesAllocatorThread.h	…
MMFilesCleanupThread.cpp	Remove Coordinator DBs (#5661 )	2018-06-25 19:18:11 +02:00
MMFilesCleanupThread.h	Remove Coordinator DBs (#5661 )	2018-06-25 19:18:11 +02:00
MMFilesCollection.cpp	Bug fix/meierloch (#5895 )	2018-07-17 11:39:50 +02:00
MMFilesCollection.h	Bug fix/meierloch (#5895 )	2018-07-17 11:39:50 +02:00
MMFilesCollectionExport.cpp	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionExport.h	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionKeys.cpp	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesCollectionKeys.h	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionWriteLocker.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesCollectorCache.h	…
MMFilesCollectorThread.cpp	turn off MMFiles collection and compaction (#5734 )	2018-07-03 15:23:27 +02:00
MMFilesCollectorThread.h	…
MMFilesCompactionFeature.cpp	Feature/feature phases (#5272 )	2018-07-16 14:09:36 +02:00
MMFilesCompactionFeature.h	…
MMFilesCompactionLocker.h	…
MMFilesCompactorThread.cpp	renamed MMFiles compactor thread from "Compactor" to "MMFilesCompactor"	2018-07-14 01:10:14 +02:00
MMFilesCompactorThread.h	issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370 )	2018-05-17 01:15:50 +03:00
MMFilesDatafile.cpp	slightly simplify syncing API (#5726 )	2018-06-29 14:03:47 +02:00
MMFilesDatafile.h	slightly simplify syncing API (#5726 )	2018-06-29 14:03:47 +02:00
MMFilesDatafileHelper.h	…
MMFilesDatafileStatistics.cpp	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesDatafileStatistics.h	…
MMFilesDatafileStatisticsContainer.cpp	…
MMFilesDatafileStatisticsContainer.h	…
MMFilesDitch.cpp	issue 399.2: use a LogicalCollection& instead of LogicalCollection* in PhysicalCollection (#5861 )	2018-07-13 15:02:43 +03:00
MMFilesDitch.h	issue 399.2: use a LogicalCollection& instead of LogicalCollection* in PhysicalCollection (#5861 )	2018-07-13 15:02:43 +03:00
MMFilesDocumentOperation.cpp	…
MMFilesDocumentOperation.h	…
MMFilesDocumentPosition.h	…
MMFilesEdgeIndex.cpp	Cleanup cluster selectivity (#5440 )	2018-05-23 18:00:14 +02:00
MMFilesEdgeIndex.h	Cleanup cluster selectivity (#5440 )	2018-05-23 18:00:14 +02:00
MMFilesEngine.cpp	finalize `--rocksdb.sync-interval` feature (#5856 )	2018-07-17 11:09:57 +02:00
MMFilesEngine.h	finalize `--rocksdb.sync-interval` feature (#5856 )	2018-07-17 11:09:57 +02:00
MMFilesExportCursor.cpp	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesExportCursor.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesFulltextIndex.cpp	issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344 )	2018-05-14 19:06:24 +03:00
MMFilesFulltextIndex.h	fixed issue #5252 (#5427 )	2018-05-23 08:30:27 +02:00
MMFilesGeoIndex.cpp	Remove redundant geo code, remove WITHIN_RECTANGLE (#5757 )	2018-07-04 20:21:00 +02:00
MMFilesGeoIndex.h	Remove redundant geo code, remove WITHIN_RECTANGLE (#5757 )	2018-07-04 20:21:00 +02:00
MMFilesHashIndex.cpp	Cleanup cluster selectivity (#5440 )	2018-05-23 18:00:14 +02:00
MMFilesHashIndex.h	Cleanup cluster selectivity (#5440 )	2018-05-23 18:00:14 +02:00
MMFilesIncrementalSync.h	Refactor syncer state and make notes for future parallelization (#5742 )	2018-07-03 21:32:16 +02:00
MMFilesIndex.h	…
MMFilesIndexElement.cpp	…
MMFilesIndexElement.h	…
MMFilesIndexFactory.cpp	Feature/test iresearch (#5610 )	2018-06-21 20:35:04 +03:00
MMFilesIndexFactory.h	Coordinator storage engine (#5405 )	2018-05-22 19:30:27 +02:00
MMFilesLogfileManager.cpp	Feature/feature phases (#5272 )	2018-07-16 14:09:36 +02:00
MMFilesLogfileManager.h	…
MMFilesOptimizerRules.cpp	use ExecutionNode::castTo (#5875 )	2018-07-13 20:09:16 +02:00
MMFilesOptimizerRules.h	…
MMFilesPathBasedIndex.cpp	Coordinator storage engine (#5405 )	2018-05-22 19:30:27 +02:00
MMFilesPathBasedIndex.h	Coordinator storage engine (#5405 )	2018-05-22 19:30:27 +02:00
MMFilesPersistentIndex.cpp	Coordinator storage engine (#5405 )	2018-05-22 19:30:27 +02:00
MMFilesPersistentIndex.h	Coordinator storage engine (#5405 )	2018-05-22 19:30:27 +02:00
MMFilesPersistentIndexFeature.cpp	Feature/feature phases (#5272 )	2018-07-16 14:09:36 +02:00
MMFilesPersistentIndexFeature.h	…
MMFilesPersistentIndexKeyComparator.cpp	…
MMFilesPersistentIndexKeyComparator.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesPrimaryIndex.cpp	issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344 )	2018-05-14 19:06:24 +03:00
MMFilesPrimaryIndex.h	Cleanup cluster selectivity (#5440 )	2018-05-23 18:00:14 +02:00
MMFilesRemoverThread.cpp	…
MMFilesRemoverThread.h	…
MMFilesRestExportHandler.cpp	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestExportHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestHandlers.cpp	…
MMFilesRestHandlers.h	…
MMFilesRestReplicationHandler.cpp	Allow dumping / restoring of views (#5774 )	2018-07-05 17:55:28 +02:00
MMFilesRestReplicationHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestWalHandler.cpp	remove useless flush from test, bump wal flush default timeouts (#4845 )	2018-03-14 21:59:48 +01:00
MMFilesRestWalHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRevisionsCache.cpp	…
MMFilesRevisionsCache.h	…
MMFilesSkiplist.h	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
MMFilesSkiplistIndex.cpp	Bug fix/fixes 0907 (#5812 )	2018-07-10 08:48:14 +02:00
MMFilesSkiplistIndex.h	Coordinator storage engine (#5405 )	2018-05-22 19:30:27 +02:00
MMFilesSynchronizerThread.cpp	…
MMFilesSynchronizerThread.h	…
MMFilesTransactionCollection.cpp	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesTransactionCollection.h	…
MMFilesTransactionContextData.cpp	issue 344.6: remove some redundant functions (#4842 )	2018-03-15 11:03:35 +01:00
MMFilesTransactionContextData.h	…
MMFilesTransactionManager.cpp	…
MMFilesTransactionManager.h	…
MMFilesTransactionState.cpp	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesTransactionState.h	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesV8Functions.cpp	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesV8Functions.h	…
MMFilesWalAccess.cpp	issue 373.1: move globally-unique id generation from collection into data-source (#5182 )	2018-05-07 22:14:40 +03:00
MMFilesWalAccess.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesWalLogfile.cpp	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesWalLogfile.h	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesWalMarker.h	…
MMFilesWalRecoverState.cpp	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesWalRecoverState.h	…
MMFilesWalRecoveryFeature.cpp	Feature/feature phases (#5272 )	2018-07-16 14:09:36 +02:00
MMFilesWalRecoveryFeature.h	…
MMFilesWalSlot.cpp	…
MMFilesWalSlot.h	…
MMFilesWalSlots.cpp	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
MMFilesWalSlots.h	…
MMFilesWalSyncRegion.h	…
README.md	…
mmfiles-fulltext-common.h	…
mmfiles-fulltext-index.cpp	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
mmfiles-fulltext-index.h	…
mmfiles-fulltext-list.cpp	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
mmfiles-fulltext-list.h	…
mmfiles-fulltext-query.cpp	…
mmfiles-fulltext-query.h	…
mmfiles-replication-common.cpp	…
mmfiles-replication-common.h	…
mmfiles-replication-dump.cpp	issue 355.7: ensure LogicalDataSource::vocbase() returns a reference	2018-04-09 15:38:24 +03:00
mmfiles-replication-dump.h	issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370 )	2018-05-17 01:15:50 +03:00

README.md

MMFilesEngine

How operations are stored - Overview

All operations like insert or remove are written as markers to a write ahead log (WAL). This WAL consists of files of a certain size and if such a file is full (or is manually flushed), all relevant markers are transferred (transferMarkers()) to the journals of the respective collections. During the transfer any obsolete markers will be thrown away: a sequence of insert, remove, insert on the same document will result in the last insert discarding the previous operations. When a journal file of size (journalSize()) is full, it will be sealed and renamed. By applying these operations it will become a datafile that is read-only. Datafiles will eventually be merged by a compactor thread that does about the same work as the transferMarkers function, reducing the size of the stored data.

Ditches

Ditches are used to pin objects in WAL or journal as long as they are used by other operations.