History

Wilfried Goesgens 0faf9be695 Feature/compaction configuration (#3217 ) * make the different values influencing the compaction run configurable * Compaction statistics handling - we mustn't keep the number of dead objects on the compacted datafiles statistics, else it will be compacted again. - keep statistics of the compaction runs on the DatafileStatistics object - add the new statistics on DatafileStatistics to the figures api - implement test that assures only one compaction is run, and the statistic values are maintained * don't mention the version number * Implement review - fix documentation - allow 0 maxfiles to enable users to disable combined of datafiles - add statistic element that counts the number of combined datafiles * Implement review - fix documentation - use locks to make statistic values consistent. - fix typo in variable name * fix temporary variable unnecessary. * changelog		2017-09-15 14:37:30 +02:00
..
CMakeLists.txt	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesAllocatorThread.cpp	…
MMFilesAllocatorThread.h	…
MMFilesAqlFunctions.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesAqlFunctions.h	…
MMFilesCleanupThread.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesCleanupThread.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesCollection.cpp	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesCollection.h	Optimizations for Caches and Graph Traversals (#3169 )	2017-08-31 18:33:10 +02:00
MMFilesCollectionExport.cpp	move common code into its own file	2017-05-22 17:46:12 +02:00
MMFilesCollectionExport.h	move common code into its own file	2017-05-22 17:46:12 +02:00
MMFilesCollectionKeys.cpp	removing unused and unimplemented methods	2017-05-10 18:25:41 +02:00
MMFilesCollectionKeys.h	…
MMFilesCollectionReadLocker.h	…
MMFilesCollectionWriteLocker.h	…
MMFilesCollectorCache.h	…
MMFilesCollectorThread.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesCollectorThread.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesCompactionFeature.cpp	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesCompactionFeature.h	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesCompactionLocker.h	…
MMFilesCompactorThread.cpp	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesCompactorThread.h	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesDatafile.cpp	do not include our datafiles in core files (#2630 )	2017-06-22 12:34:27 +02:00
MMFilesDatafile.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesDatafileHelper.h	…
MMFilesDatafileStatistics.cpp	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesDatafileStatistics.h	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesDatafileStatisticsContainer.cpp	…
MMFilesDatafileStatisticsContainer.h	…
MMFilesDitch.cpp	…
MMFilesDitch.h	…
MMFilesDocumentOperation.cpp	…
MMFilesDocumentOperation.h	…
MMFilesDocumentPosition.h	…
MMFilesEdgeIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesEdgeIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesEngine.cpp	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesEngine.h	Feature/reduce extraction to projection (#2792 )	2017-07-14 08:40:29 +02:00
MMFilesExportCursor.cpp	move common code into its own file	2017-05-22 17:46:12 +02:00
MMFilesExportCursor.h	…
MMFilesFulltextIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesFulltextIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesGeoIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesGeoIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesHashIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesHashIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesIncrementalSync.h	Bug fix/adjust agency comm timeouts (#2765 )	2017-07-13 00:44:28 +02:00
MMFilesIndexElement.cpp	…
MMFilesIndexElement.h	…
MMFilesIndexFactory.cpp	Feature/reduce extraction to projection (#2792 )	2017-07-14 08:40:29 +02:00
MMFilesIndexFactory.h	…
MMFilesLogfileManager.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesLogfileManager.h	fix a race on shutdown (#2897 )	2017-07-29 10:30:24 +02:00
MMFilesOptimizerRules.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesOptimizerRules.h	…
MMFilesPathBasedIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesPathBasedIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesPersistentIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesPersistentIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesPersistentIndexFeature.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesPersistentIndexFeature.h	…
MMFilesPersistentIndexKeyComparator.cpp	…
MMFilesPersistentIndexKeyComparator.h	…
MMFilesPrimaryIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesPrimaryIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesRemoverThread.cpp	…
MMFilesRemoverThread.h	…
MMFilesRestExportHandler.cpp	move common code into its own file	2017-05-22 17:46:12 +02:00
MMFilesRestExportHandler.h	move common code into its own file	2017-05-22 17:46:12 +02:00
MMFilesRestHandlers.cpp	…
MMFilesRestHandlers.h	…
MMFilesRestReplicationHandler.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesRestReplicationHandler.h	Bug fix/use count for checksum (#3033 )	2017-08-15 11:24:54 +02:00
MMFilesRestWalHandler.cpp	issue #2505	2017-05-29 10:29:47 +02:00
MMFilesRestWalHandler.h	…
MMFilesRevisionsCache.cpp	…
MMFilesRevisionsCache.h	…
MMFilesSkiplist.h	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesSkiplistIndex.cpp	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesSkiplistIndex.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesSynchronizerThread.cpp	…
MMFilesSynchronizerThread.h	…
MMFilesToken.h	…
MMFilesTransactionCollection.cpp	a few simplifications and extensions (#3115 )	2017-08-24 14:46:16 +02:00
MMFilesTransactionCollection.h	a few simplifications and extensions (#3115 )	2017-08-24 14:46:16 +02:00
MMFilesTransactionContextData.cpp	…
MMFilesTransactionContextData.h	…
MMFilesTransactionManager.cpp	…
MMFilesTransactionManager.h	…
MMFilesTransactionState.cpp	only sync RocksDB WAL in MMFiles engine when there is a reason to do so (#2863 )	2017-07-26 21:49:41 +02:00
MMFilesTransactionState.h	added tests for intermediate commits	2017-05-30 12:03:58 +02:00
MMFilesV8Functions.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesV8Functions.h	…
MMFilesView.cpp	…
MMFilesView.h	…
MMFilesWalLogfile.cpp	…
MMFilesWalLogfile.h	…
MMFilesWalMarker.h	…
MMFilesWalRecoverState.cpp	Added recovery tests for views and fixed a few related bugs. (#2682 )	2017-06-29 17:09:43 +02:00
MMFilesWalRecoverState.h	…
MMFilesWalRecoveryFeature.cpp	remove dependency on MMFiles features from non-MMFiles files (#2925 )	2017-08-01 22:16:43 +02:00
MMFilesWalRecoveryFeature.h	…
MMFilesWalSlot.cpp	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSlot.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSlots.cpp	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSlots.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSyncRegion.h	…
README.md	…
mmfiles-fulltext-common.h	…
mmfiles-fulltext-index.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-fulltext-index.h	Bug fix/issues 2308 (#3117 )	2017-08-24 14:47:22 +02:00
mmfiles-fulltext-list.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-fulltext-list.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
mmfiles-fulltext-query.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-fulltext-query.h	…
mmfiles-geo-index.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-geo-index.h	…
mmfiles-replication-dump.cpp	Squashed commit of the following:	2017-05-12 23:22:22 +02:00
mmfiles-replication-dump.h	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00

README.md

MMFilesEngine

How operations are stored - Overview

All operations like insert or remove are written as markers to a write ahead log (WAL). This WAL consists of files of a certain size and if such a file is full (or is manually flushed), all relevant markers are transferred (transferMarkers()) to the journals of the respective collections. During the transfer any obsolete markers will be thrown away: a sequence of insert, remove, insert on the same document will result in the last insert discarding the previous operations. When a journal file of size (journalSize()) is full, it will be sealed and renamed. By applying these operations it will become a datafile that is read-only. Datafiles will eventually be merged by a compactor thread that does about the same work as the transferMarkers function, reducing the size of the stored data.

Ditches

Ditches are used to pin objects in WAL or journal as long as they are used by other operations.