History

Jan 4df1c81e9b Feature/mmfiles index lookup speedup (#5329 )		2018-05-12 10:21:12 +02:00
..
CMakeLists.txt	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesAllocatorThread.cpp	remove logging for non-topics	2017-02-10 09:32:50 +01:00
MMFilesAllocatorThread.h	…
MMFilesAqlFunctions.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesAqlFunctions.h	functionality for plugging REST handlers	2017-03-09 17:12:00 +01:00
MMFilesCleanupThread.cpp	issue 355.7: ensure LogicalDataSource::vocbase() returns a reference	2018-04-09 15:38:24 +03:00
MMFilesCleanupThread.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
MMFilesCollection.cpp	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesCollection.h	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesCollectionExport.cpp	issue 355.7: ensure LogicalDataSource::vocbase() returns a reference	2018-04-09 15:38:24 +03:00
MMFilesCollectionExport.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesCollectionKeys.cpp	issue 355.7: ensure LogicalDataSource::vocbase() returns a reference	2018-04-09 15:38:24 +03:00
MMFilesCollectionKeys.h	fix premature unlock (#3802 )	2017-12-13 13:27:42 +01:00
MMFilesCollectionWriteLocker.h	Deadlock detection uses TransactionID instead of Thread ID (#4787 )	2018-03-09 18:35:38 +01:00
MMFilesCollectorCache.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesCollectorThread.cpp	Bug fix/remove most of aql js (#5223 )	2018-04-30 11:17:11 +02:00
MMFilesCollectorThread.h	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
MMFilesCompactionFeature.cpp	remove call to requiresElevatedPrivileges with default value (#5172 )	2018-04-23 11:28:24 +02:00
MMFilesCompactionFeature.h	fix dropping of system collections and retrieving list of user databa… (#3304 )	2017-09-21 10:05:46 +02:00
MMFilesCompactionLocker.h	renamed classes	2017-02-28 17:59:48 +01:00
MMFilesCompactorThread.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesCompactorThread.h	Feature/compaction configuration (#3217 )	2017-09-15 14:37:30 +02:00
MMFilesDatafile.cpp	Bugfix/fix windows warnings (#5116 )	2018-04-17 12:38:59 +02:00
MMFilesDatafile.h	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesDatafileHelper.h	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
MMFilesDatafileStatistics.cpp	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesDatafileStatistics.h	do not complain about missing datafile statistics (#3688 )	2017-11-14 16:42:31 +01:00
MMFilesDatafileStatisticsContainer.cpp	moved files	2017-03-16 16:45:02 +01:00
MMFilesDatafileStatisticsContainer.h	moved files	2017-03-16 16:45:02 +01:00
MMFilesDitch.cpp	renamed classes	2017-02-27 11:40:15 +01:00
MMFilesDitch.h	renamed classes	2017-02-27 11:40:15 +01:00
MMFilesDocumentOperation.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesDocumentOperation.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesDocumentPosition.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesEdgeIndex.cpp	Feature/mmfiles index lookup speedup (#5329 )	2018-05-12 10:21:12 +02:00
MMFilesEdgeIndex.h	Feature/mmfiles index lookup speedup (#5329 )	2018-05-12 10:21:12 +02:00
MMFilesEngine.cpp	issue 373.1: move globally-unique id generation from collection into data-source (#5182 )	2018-05-07 22:14:40 +03:00
MMFilesEngine.h	issue 374.2: use a reference to vocbase instead of a pointer in StorageEngime LogicalDatasource related DDL operations, use vocbase from the found collection	2018-04-11 18:49:46 +03:00
MMFilesExportCursor.cpp	backport: use vocbase reference instead of pointer in arangodb::pregel::GraphStore	2018-04-13 11:23:34 +03:00
MMFilesExportCursor.h	backport: use vocbase reference instead of pointer in arangodb::pregel::GraphStore	2018-04-13 11:23:34 +03:00
MMFilesFulltextIndex.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesFulltextIndex.h	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesGeoIndex.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesGeoIndex.h	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesHashIndex.cpp	Feature/mmfiles index lookup speedup (#5329 )	2018-05-12 10:21:12 +02:00
MMFilesHashIndex.h	Feature/mmfiles index lookup speedup (#5329 )	2018-05-12 10:21:12 +02:00
MMFilesIncrementalSync.h	issue 344.6: remove some redundant functions (#4842 )	2018-03-15 11:03:35 +01:00
MMFilesIndex.h	Bug fix/fixes 0211 (#3568 )	2017-11-09 12:48:15 +01:00
MMFilesIndexElement.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesIndexElement.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesIndexFactory.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesIndexFactory.h	issue 355.6: remove create() from LogicalView, remove IResearch dependency from IndexFactory, store vocbase reference in LogicalDataSource	2018-04-06 16:38:34 +03:00
MMFilesLogfileManager.cpp	remove call to requiresElevatedPrivileges with default value (#5172 )	2018-04-23 11:28:24 +02:00
MMFilesLogfileManager.h	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
MMFilesOptimizerRules.cpp	Feature/aql optimizations (#5237 )	2018-05-03 16:55:58 +02:00
MMFilesOptimizerRules.h	Working geo index + tests	2017-05-09 15:04:51 +02:00
MMFilesPathBasedIndex.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesPathBasedIndex.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesPersistentIndex.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesPersistentIndex.h	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesPersistentIndexFeature.cpp	remove call to requiresElevatedPrivileges with default value (#5172 )	2018-04-23 11:28:24 +02:00
MMFilesPersistentIndexFeature.h	change --rocksdb.enabled to be a obosolete option	2017-04-19 14:54:10 +02:00
MMFilesPersistentIndexKeyComparator.cpp	class renaming	2017-03-08 17:54:00 +01:00
MMFilesPersistentIndexKeyComparator.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesPrimaryIndex.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesPrimaryIndex.h	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
MMFilesRemoverThread.cpp	remove logging for non-topics	2017-02-10 09:32:50 +01:00
MMFilesRemoverThread.h	…
MMFilesRestExportHandler.cpp	Remove copy of request body in rest handlers (#5104 )	2018-04-16 14:49:51 +02:00
MMFilesRestExportHandler.h	move common code into its own file	2017-05-22 17:46:12 +02:00
MMFilesRestHandlers.cpp	Moved MMFiles replication handler.	2017-04-18 15:35:45 -04:00
MMFilesRestHandlers.h	make RestWalHandler a handler of the MMFiles engine	2017-03-10 09:00:37 +01:00
MMFilesRestReplicationHandler.cpp	Simple dump speedup (#5298 )	2018-05-09 12:51:04 +02:00
MMFilesRestReplicationHandler.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesRestWalHandler.cpp	remove useless flush from test, bump wal flush default timeouts (#4845 )	2018-03-14 21:59:48 +01:00
MMFilesRestWalHandler.h	make RestWalHandler a handler of the MMFiles engine	2017-03-10 09:00:37 +01:00
MMFilesRevisionsCache.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesRevisionsCache.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesSkiplist.h	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
MMFilesSkiplistIndex.cpp	Feature/mmfiles index lookup speedup (#5329 )	2018-05-12 10:21:12 +02:00
MMFilesSkiplistIndex.h	Feature/mmfiles index lookup speedup (#5329 )	2018-05-12 10:21:12 +02:00
MMFilesSynchronizerThread.cpp	improved logging for MMFiles IO operations (#4366 )	2018-01-19 18:37:01 +01:00
MMFilesSynchronizerThread.h	…
MMFilesTransactionCollection.cpp	remove TRI_ERROR_ARANGO_VIEW_NOT_FOUND, rename TRI_ERROR_ARANGO_COLLECTION_NOT_FOUND to TRI_ERROR_ARNANGO_DATA_SOURCE_NOT_FOUND	2018-03-17 19:36:14 +03:00
MMFilesTransactionCollection.h	fix premature unlock (#3802 )	2017-12-13 13:27:42 +01:00
MMFilesTransactionContextData.cpp	issue 344.6: remove some redundant functions (#4842 )	2018-03-15 11:03:35 +01:00
MMFilesTransactionContextData.h	removed unused includes	2017-02-27 17:29:28 +01:00
MMFilesTransactionManager.cpp	optimize away own transaction management for RocksDB engine	2017-05-04 23:16:53 +02:00
MMFilesTransactionManager.h	optimize away own transaction management for RocksDB engine	2017-05-04 23:16:53 +02:00
MMFilesTransactionState.cpp	issue 344.6: remove some redundant functions (#4842 )	2018-03-15 11:03:35 +01:00
MMFilesTransactionState.h	Feature/async failover (#3451 )	2017-10-18 23:59:29 +02:00
MMFilesV8Functions.cpp	issue 373.3: use TRI_vocbase_t& for Upgrade tasks, remove redundant checks for null TRI_vocbase_t (#5301 )	2018-05-09 15:54:07 +03:00
MMFilesV8Functions.h	fix db._engineStats()	2017-05-04 00:33:36 +02:00
MMFilesWalAccess.cpp	issue 373.1: move globally-unique id generation from collection into data-source (#5182 )	2018-05-07 22:14:40 +03:00
MMFilesWalAccess.h	add missing override specifiers, add final specifiers	2018-05-04 09:01:50 +02:00
MMFilesWalLogfile.cpp	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesWalLogfile.h	improve startup resilience in case there are datafile errors (#4975 )	2018-04-03 17:32:30 +02:00
MMFilesWalMarker.h	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
MMFilesWalRecoverState.cpp	issue 383.3: implement remainder of IResearchViewDBServer tests, use the data-source id (primary key) instead of an arbitrary instance for dropCollection()/dropView(), backport from iresearch upstream: ensure block is flushed if key index is full (#5176 )	2018-04-23 00:33:46 +03:00
MMFilesWalRecoverState.h	renaming	2017-03-22 13:01:40 +01:00
MMFilesWalRecoveryFeature.cpp	remove call to requiresElevatedPrivileges with default value (#5172 )	2018-04-23 11:28:24 +02:00
MMFilesWalRecoveryFeature.h	…
MMFilesWalSlot.cpp	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
MMFilesWalSlot.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSlots.cpp	remove TRI_usleep and TRI_sleep, and use std::this_thread::sleep_for … (#3817 )	2017-12-06 18:43:49 +01:00
MMFilesWalSlots.h	fix WAL file tick updating (happened too late before) (#2762 )	2017-07-10 17:41:08 +02:00
MMFilesWalSyncRegion.h	…
README.md	remove transferMarkers from StorageEngine.h	2017-02-21 16:00:37 +01:00
mmfiles-fulltext-common.h	move replication-dump into engine	2017-03-21 13:55:57 +01:00
mmfiles-fulltext-index.cpp	S2 based Geo-Spatial index (#5249 )	2018-05-02 23:54:41 +02:00
mmfiles-fulltext-index.h	Bug fix/isrestore (#3283 )	2017-10-03 11:53:49 +02:00
mmfiles-fulltext-list.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-fulltext-list.h	fix fulltext index removal performance, simplified code (#3015 )	2017-08-12 09:25:30 +02:00
mmfiles-fulltext-query.cpp	Bug fix/fixes 0609 (#3227 )	2017-09-13 16:28:21 +02:00
mmfiles-fulltext-query.h	move replication-dump into engine	2017-03-21 13:55:57 +01:00
mmfiles-replication-common.cpp	Feature iresearch (#4071 )	2017-12-18 15:04:59 +01:00
mmfiles-replication-common.h	slight cleanup of replication-related code (#4286 )	2018-01-17 16:56:40 +01:00
mmfiles-replication-dump.cpp	issue 355.7: ensure LogicalDataSource::vocbase() returns a reference	2018-04-09 15:38:24 +03:00
mmfiles-replication-dump.h	forward port of replication bugfixes from 3.2 (#4572 )	2018-02-12 18:13:51 +01:00

README.md

MMFilesEngine

How operations are stored - Overview

All operations like insert or remove are written as markers to a write ahead log (WAL). This WAL consists of files of a certain size and if such a file is full (or is manually flushed), all relevant markers are transferred (transferMarkers()) to the journals of the respective collections. During the transfer any obsolete markers will be thrown away: a sequence of insert, remove, insert on the same document will result in the last insert discarding the previous operations. When a journal file of size (journalSize()) is full, it will be sealed and renamed. By applying these operations it will become a datafile that is read-only. Datafiles will eventually be merged by a compactor thread that does about the same work as the transferMarkers function, reducing the size of the stored data.

Ditches

Ditches are used to pin objects in WAL or journal as long as they are used by other operations.