mirror of https://gitee.com/bigwinds/arangodb
8e65fbb539
* we must now ignore that datafiles are not sealed this is because an unsealed datafile may have been produced by renaming multiple journals to datafiles at server start * acquire collection count after we have acquired the lock * count the null byte as well * fix count value acquisition * send query fragments to the correct servers, even after failover or when a follower drops the problem with using the previous shard-based approach is that responsibilities for shards may change at runtime however, an AQL query must send all requests for the query to the initially used servers. if there is a failover while the query is executing, we must still send all following requests to the same servers, and not the newly responsible servers otherwise we potentially would try to get data from a query from server B while the query was only instanciated on server A. |
||
|---|---|---|
| .. | ||
| CMakeLists.txt | ||
| MMFilesAllocatorThread.cpp | ||
| MMFilesAllocatorThread.h | ||
| MMFilesAqlFunctions.cpp | ||
| MMFilesAqlFunctions.h | ||
| MMFilesCleanupThread.cpp | ||
| MMFilesCleanupThread.h | ||
| MMFilesCollection.cpp | ||
| MMFilesCollection.h | ||
| MMFilesCollectionExport.cpp | ||
| MMFilesCollectionExport.h | ||
| MMFilesCollectionKeys.cpp | ||
| MMFilesCollectionKeys.h | ||
| MMFilesCollectionReadLocker.h | ||
| MMFilesCollectionWriteLocker.h | ||
| MMFilesCollectorCache.h | ||
| MMFilesCollectorThread.cpp | ||
| MMFilesCollectorThread.h | ||
| MMFilesCompactionLocker.h | ||
| MMFilesCompactorThread.cpp | ||
| MMFilesCompactorThread.h | ||
| MMFilesDatafile.cpp | ||
| MMFilesDatafile.h | ||
| MMFilesDatafileHelper.h | ||
| MMFilesDatafileStatistics.cpp | ||
| MMFilesDatafileStatistics.h | ||
| MMFilesDatafileStatisticsContainer.cpp | ||
| MMFilesDatafileStatisticsContainer.h | ||
| MMFilesDitch.cpp | ||
| MMFilesDitch.h | ||
| MMFilesDocumentOperation.cpp | ||
| MMFilesDocumentOperation.h | ||
| MMFilesDocumentPosition.h | ||
| MMFilesEdgeIndex.cpp | ||
| MMFilesEdgeIndex.h | ||
| MMFilesEngine.cpp | ||
| MMFilesEngine.h | ||
| MMFilesExportCursor.cpp | ||
| MMFilesExportCursor.h | ||
| MMFilesFulltextIndex.cpp | ||
| MMFilesFulltextIndex.h | ||
| MMFilesGeoIndex.cpp | ||
| MMFilesGeoIndex.h | ||
| MMFilesHashIndex.cpp | ||
| MMFilesHashIndex.h | ||
| MMFilesIncrementalSync.h | ||
| MMFilesIndexElement.cpp | ||
| MMFilesIndexElement.h | ||
| MMFilesIndexFactory.cpp | ||
| MMFilesIndexFactory.h | ||
| MMFilesLogfileManager.cpp | ||
| MMFilesLogfileManager.h | ||
| MMFilesOptimizerRules.cpp | ||
| MMFilesOptimizerRules.h | ||
| MMFilesPathBasedIndex.cpp | ||
| MMFilesPathBasedIndex.h | ||
| MMFilesPersistentIndex.cpp | ||
| MMFilesPersistentIndex.h | ||
| MMFilesPersistentIndexFeature.cpp | ||
| MMFilesPersistentIndexFeature.h | ||
| MMFilesPersistentIndexKeyComparator.cpp | ||
| MMFilesPersistentIndexKeyComparator.h | ||
| MMFilesPrimaryIndex.cpp | ||
| MMFilesPrimaryIndex.h | ||
| MMFilesRemoverThread.cpp | ||
| MMFilesRemoverThread.h | ||
| MMFilesRestExportHandler.cpp | ||
| MMFilesRestExportHandler.h | ||
| MMFilesRestHandlers.cpp | ||
| MMFilesRestHandlers.h | ||
| MMFilesRestReplicationHandler.cpp | ||
| MMFilesRestReplicationHandler.h | ||
| MMFilesRestWalHandler.cpp | ||
| MMFilesRestWalHandler.h | ||
| MMFilesRevisionsCache.cpp | ||
| MMFilesRevisionsCache.h | ||
| MMFilesSkiplist.h | ||
| MMFilesSkiplistIndex.cpp | ||
| MMFilesSkiplistIndex.h | ||
| MMFilesSynchronizerThread.cpp | ||
| MMFilesSynchronizerThread.h | ||
| MMFilesToken.h | ||
| MMFilesTransactionCollection.cpp | ||
| MMFilesTransactionCollection.h | ||
| MMFilesTransactionContextData.cpp | ||
| MMFilesTransactionContextData.h | ||
| MMFilesTransactionManager.cpp | ||
| MMFilesTransactionManager.h | ||
| MMFilesTransactionState.cpp | ||
| MMFilesTransactionState.h | ||
| MMFilesV8Functions.cpp | ||
| MMFilesV8Functions.h | ||
| MMFilesView.cpp | ||
| MMFilesView.h | ||
| MMFilesWalLogfile.cpp | ||
| MMFilesWalLogfile.h | ||
| MMFilesWalMarker.h | ||
| MMFilesWalRecoverState.cpp | ||
| MMFilesWalRecoverState.h | ||
| MMFilesWalRecoveryFeature.cpp | ||
| MMFilesWalRecoveryFeature.h | ||
| MMFilesWalSlot.cpp | ||
| MMFilesWalSlot.h | ||
| MMFilesWalSlots.cpp | ||
| MMFilesWalSlots.h | ||
| MMFilesWalSyncRegion.h | ||
| README.md | ||
| mmfiles-fulltext-common.h | ||
| mmfiles-fulltext-index.cpp | ||
| mmfiles-fulltext-index.h | ||
| mmfiles-fulltext-list.cpp | ||
| mmfiles-fulltext-list.h | ||
| mmfiles-fulltext-query.cpp | ||
| mmfiles-fulltext-query.h | ||
| mmfiles-geo-index.cpp | ||
| mmfiles-geo-index.h | ||
| mmfiles-replication-dump.cpp | ||
| mmfiles-replication-dump.h | ||
README.md
MMFilesEngine
How operations are stored - Overview
All operations like insert or remove are written as markers to a write ahead log (WAL). This WAL consists of files of a certain size and if such a file is full (or is manually flushed), all relevant markers are transferred (transferMarkers()) to the journals of the respective collections. During the transfer any obsolete markers will be thrown away: a sequence of insert, remove, insert on the same document will result in the last insert discarding the previous operations. When a journal file of size (journalSize()) is full, it will be sealed and renamed. By applying these operations it will become a datafile that is read-only. Datafiles will eventually be merged by a compactor thread that does about the same work as the transferMarkers function, reducing the size of the stored data.
Ditches
Ditches are used to pin objects in WAL or journal as long as they are used by other operations.