History

Dan Larkin-York 743ede87ea Disable warning about persistent IDs during upgrade procedure. (#7611 )		2018-12-04 09:02:41 +01:00
..
CMakeLists.txt	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesAllocatorThread.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesAllocatorThread.h	…
MMFilesCleanupThread.cpp	backport missed changes (#7016 )	2018-10-24 15:43:45 +03:00
MMFilesCleanupThread.h	Remove Coordinator DBs (#5661 )	2018-06-25 19:18:11 +02:00
MMFilesCollection.cpp	Disable warning about persistent IDs during upgrade procedure. (#7611 )	2018-12-04 09:02:41 +01:00
MMFilesCollection.h	Bug fix/restore index refactor (#7470 ) (#7491 )	2018-11-29 14:08:29 +01:00
MMFilesCollectionExport.cpp	Bug fix/add aql collection count cache (#6227 )	2018-08-23 16:05:51 +02:00
MMFilesCollectionExport.h	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionKeys.cpp	issue 430.3: remove redundant constructor from SingleCollectionTransaction (#5996 )	2018-07-26 16:54:53 +03:00
MMFilesCollectionKeys.h	issue 389.10: refactor CollectionNameResolver to use TRI_vocbase_t& (#5424 )	2018-05-23 00:59:08 +03:00
MMFilesCollectionWriteLocker.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesCollectorCache.h	Bug fix 3.4/backport optimizations (#7434 )	2018-11-26 19:16:05 +01:00
MMFilesCollectorThread.cpp	fix assertion failures for MMFiles operationsQueue state on shutdown (#7389 )	2018-11-21 09:33:38 +01:00
MMFilesCollectorThread.h	fix assertion failures for MMFiles operationsQueue state on shutdown (#7389 )	2018-11-21 09:33:38 +01:00
MMFilesCompactionFeature.cpp	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesCompactionFeature.h	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesCompactionLocker.h	…
MMFilesCompactorThread.cpp	[3.4] Upgrade old MMFiles collections for compatibility with ArangoSearch (#7133 )	2018-11-06 11:12:54 +01:00
MMFilesCompactorThread.h	Bug fix/add aql collection count cache (#6227 )	2018-08-23 16:05:51 +02:00
MMFilesDatafile.cpp	[3.4] Upgrade old MMFiles collections for compatibility with ArangoSearch (#7133 )	2018-11-06 11:12:54 +01:00
MMFilesDatafile.h	[3.4] Upgrade old MMFiles collections for compatibility with ArangoSearch (#7133 )	2018-11-06 11:12:54 +01:00
MMFilesDatafileHelper.h	View Replication (#5915 )	2018-07-26 10:28:46 +02:00
MMFilesDatafileStatistics.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesDatafileStatistics.h	do not complain about missing datafile statistics (#3688 )	2017-11-14 16:42:31 +01:00
MMFilesDatafileStatisticsContainer.cpp	…
MMFilesDatafileStatisticsContainer.h	…
MMFilesDitch.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesDitch.h	issue 399.2: use a LogicalCollection& instead of LogicalCollection* in PhysicalCollection (#5861 )	2018-07-13 15:02:43 +03:00
MMFilesDocumentOperation.cpp	…
MMFilesDocumentOperation.h	…
MMFilesDocumentPosition.h	…
MMFilesEdgeIndex.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesEdgeIndex.h	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesEngine.cpp	Bug fix 3.4/backport optimizations (#7434 )	2018-11-26 19:16:05 +01:00
MMFilesEngine.h	properly initialize member	2018-11-07 15:49:57 +01:00
MMFilesExportCursor.cpp	Server stream cursor (#7186 ) (#7210 )	2018-11-03 20:17:52 +01:00
MMFilesExportCursor.h	fix non-blocking stream cursors (#6064 )	2018-08-03 12:41:52 +02:00
MMFilesFulltextIndex.cpp	Bug fix 3.4/backport optimizations (#7434 )	2018-11-26 19:16:05 +01:00
MMFilesFulltextIndex.h	Hide links (#6348 )	2018-09-03 15:36:37 +02:00
MMFilesGeoIndex.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesGeoIndex.h	Hide links (#6348 )	2018-09-03 15:36:37 +02:00
MMFilesHashIndex.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesHashIndex.h	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesIncrementalSync.h	Bug fix 3.4/use proper log topic for replication test messages (#6920 )	2018-10-17 15:40:59 +02:00
MMFilesIndex.h	Next try at truncate (#7090 ) (#7096 )	2018-10-26 15:06:36 +02:00
MMFilesIndexElement.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesIndexElement.h	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesIndexFactory.cpp	issue 399.4: use LogicalCollection& instead of LogicalCollection* in Index (#5929 )	2018-07-20 14:55:58 +03:00
MMFilesIndexFactory.h	issue 399.4: use LogicalCollection& instead of LogicalCollection* in Index (#5929 )	2018-07-20 14:55:58 +03:00
MMFilesIndexLookupContext.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesIndexLookupContext.h	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesLogfileManager.cpp	[3.4] Upgrade old MMFiles collections for compatibility with ArangoSearch (#7133 )	2018-11-06 11:12:54 +01:00
MMFilesLogfileManager.h	attempt to make replication_sync more reliable in tests for MMFiles (#6184 )	2018-08-17 14:20:40 +02:00
MMFilesOptimizerRules.cpp	Disable JS on DBServer, fix race in UserManager (#6244 )	2018-08-24 22:20:49 +02:00
MMFilesOptimizerRules.h	…
MMFilesPathBasedIndex.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesPathBasedIndex.h	Hide links (#6348 )	2018-09-03 15:36:37 +02:00
MMFilesPersistentIndex.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesPersistentIndex.h	take over selectivity estimates (#6512 )	2018-09-17 16:42:24 +02:00
MMFilesPersistentIndexFeature.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesPersistentIndexFeature.h	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesPersistentIndexKeyComparator.cpp	…
MMFilesPersistentIndexKeyComparator.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesPrimaryIndex.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesPrimaryIndex.h	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesRemoverThread.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesRemoverThread.h	…
MMFilesRestCollectionHandler.cpp	Add engine specific collection APIs (#6962 )	2018-10-19 15:23:55 +02:00
MMFilesRestCollectionHandler.h	Add engine specific collection APIs (#6962 )	2018-10-19 15:23:55 +02:00
MMFilesRestExportHandler.cpp	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestExportHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestHandlers.cpp	Add engine specific collection APIs (#6962 )	2018-10-19 15:23:55 +02:00
MMFilesRestHandlers.h	…
MMFilesRestReplicationHandler.cpp	Allow WAL logger to split up transactions (#6800 )	2018-10-12 15:15:55 +02:00
MMFilesRestReplicationHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRestWalHandler.cpp	attempt to make MMFiles replication tests more deterministic (#6219 )	2018-08-22 14:54:27 +02:00
MMFilesRestWalHandler.h	Async AQL (#5806 )	2018-07-09 14:24:10 +02:00
MMFilesRevisionsCache.cpp	…
MMFilesRevisionsCache.h	allow master & slave to work in parallel for RocksDB WAL tailing (#6059 )	2018-08-03 13:37:53 +02:00
MMFilesSkiplist.h	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
MMFilesSkiplistIndex.cpp	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesSkiplistIndex.h	yet more micro optimizations (#7399 )	2018-11-21 17:09:16 +01:00
MMFilesSynchronizerThread.cpp	improved logging for MMFiles IO operations (#4366 )	2018-01-19 18:37:01 +01:00
MMFilesSynchronizerThread.h	…
MMFilesTransactionCollection.cpp	[3.4] MMFiles replication: get followers under lock (#7298 )	2018-11-19 13:03:07 +01:00
MMFilesTransactionCollection.h	[3.4] MMFiles replication: get followers under lock (#7298 )	2018-11-19 13:03:07 +01:00
MMFilesTransactionContextData.cpp	issue 344.6: remove some redundant functions (#4842 )	2018-03-15 11:03:35 +01:00
MMFilesTransactionContextData.h	…
MMFilesTransactionManager.cpp	…
MMFilesTransactionManager.h	…
MMFilesTransactionState.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesTransactionState.h	TransactionState::addCollection refactoring (#5606 )	2018-06-14 15:34:58 +02:00
MMFilesV8Functions.cpp	Add engine specific collection APIs (#6962 )	2018-10-19 15:23:55 +02:00
MMFilesV8Functions.h	…
MMFilesWalAccess.cpp	Allow WAL logger to split up transactions (#6800 )	2018-10-12 15:15:55 +02:00
MMFilesWalAccess.h	Allow WAL logger to split up transactions (#6800 )	2018-10-12 15:15:55 +02:00
MMFilesWalLogfile.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesWalLogfile.h	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesWalMarker.h	[3.4] Upgrade old MMFiles collections for compatibility with ArangoSearch (#7133 )	2018-11-06 11:12:54 +01:00
MMFilesWalRecoverState.cpp	Bug fix/restore index refactor (#7470 ) (#7491 )	2018-11-29 14:08:29 +01:00
MMFilesWalRecoverState.h	Use shared_ptr for LogicalCollection (#7220 ) (#7244 )	2018-11-07 10:43:08 +01:00
MMFilesWalRecoveryFeature.cpp	issue 496.2: backport 3.4: allow customization of segment size limits on arangosearch view creation, minor code cleanup (#7141 )	2018-10-30 23:41:15 +03:00
MMFilesWalRecoveryFeature.h	issue 427.5: use ApplicationServer reference instead of pointer (#6145 )	2018-08-15 12:16:02 +03:00
MMFilesWalSlot.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesWalSlot.h	…
MMFilesWalSlots.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
MMFilesWalSlots.h	…
MMFilesWalSyncRegion.h	…
README.md	…
mmfiles-fulltext-common.h	…
mmfiles-fulltext-index.cpp	[3.4] Additional and cleaned-up logging (#6648 )	2018-10-01 19:03:05 +02:00
mmfiles-fulltext-index.h	…
mmfiles-fulltext-list.cpp	Feature/misc spelling corrections (#5164 )	2018-07-13 13:06:20 +02:00
mmfiles-fulltext-list.h	…
mmfiles-fulltext-query.cpp	…
mmfiles-fulltext-query.h	…
mmfiles-replication-common.cpp	View Replication (#5915 )	2018-07-26 10:28:46 +02:00
mmfiles-replication-common.h	slight cleanup of replication-related code (#4286 )	2018-01-17 16:56:40 +01:00
mmfiles-replication-dump.cpp	backport some replication debugging from devel (#7069 )	2018-10-25 14:08:26 +02:00
mmfiles-replication-dump.h	issue 389.5: refactor StandaloneContext to be constructed with a TRI_vocbase_t& (#5370 )	2018-05-17 01:15:50 +03:00

README.md

MMFilesEngine

How operations are stored - Overview

All operations like insert or remove are written as markers to a write ahead log (WAL). This WAL consists of files of a certain size and if such a file is full (or is manually flushed), all relevant markers are transferred (transferMarkers()) to the journals of the respective collections. During the transfer any obsolete markers will be thrown away: a sequence of insert, remove, insert on the same document will result in the last insert discarding the previous operations. When a journal file of size (journalSize()) is full, it will be sealed and renamed. By applying these operations it will become a datafile that is read-only. Datafiles will eventually be merged by a compactor thread that does about the same work as the transferMarkers function, reducing the size of the stored data.

Ditches

Ditches are used to pin objects in WAL or journal as long as they are used by other operations.