done ---- - create new branch - factor out transactions from LogfileManager - concept "collection locks" - 3 states: READ / WRITE / EXCLUSIVE for locks - index implementations => moved & renamed to StorageEngine - move engine files into MMFiles directory - split IndexElement - rename *IndexElement to MMFiles* - move fulltext & geoindex & skiplist helper code into MMFiles - rename "RocksDBIndex" to "PersistentIndex" - Delete OperationCursor->getMore. Replace by getMoreTokens - remove ReadCache - Index API - Indexes always return std::vector - Rename OperationCursor->getMoreMptr => getMoreTokens, "returns" std::vector& - GeoIndex hands out TRI_revision instead of Tokens - FulltextIndex hands out TRI_revision instead of Tokens - trx::InvokeOnAllElements : uses SimpleIndexLookupElements => Has to use Tokens instead? - remove TransactionState include from Transaction.h MMFiles reference removals from files: * arangod/V8Server/v8-query.cpp - StorageEngine specific IndexFactory: - create indexes from VPack - enhance/validate given VPack index definitions (set default values, reject illegal ones) - Implement new IndexIterator API next(callback, limit) - Primary - Hash - Skiplist - Persistent - Geo - Fulltext - index API - StorageEngine specific AQL functions - Register for specific function names => branches to StorageEngine impl - Storage Engine can implement these functions with specific code and interna - e.g.: Geo, Fulltext - Replace Usage of new callback-based IndexIterator in progress ----------- - move engine-specific parts of transaction.cpp into engine - transaction API to do ----- - rename TRI_df_marker_* to something storage-engine specific - check for illegal includes - fix includes during API conversion - DML API - DDL API - add new serialization RW lock to LogicalCollection. all DML ops must acquire it in read mode, the explicit lock command must acquire it in write mode. - StorageEngineAPI readDocument requires 2 functions: - void readDocument(TOKEN id, VPackBuilder& result) => Collects the document and inserts it asis into result. Does NOT clear result. - void readDocument(TOKEN id, std::vector const& attributePath, VPackBuilder& result) => Collects the document and writes only the value at the given attributePath (e.g. `a.b.c`) into result. Does NOT clear result. - AqlValue needs a (lazy evaluated) type TOKEN that handles collection ID and TOKEN inplace. - slice() => looksup the value in the Database - We need to keep in mind the cluster. If a DBServer creates this token-type it has to be translated BEFORE the register is teleported to coordinator - Remove temporary wrapper LogCol::readDocument() - InitialySyncer.cpp knows details of StorageEngine MMFiles MMFiles are known to the following files: * arangod/Aql/Functions.cpp * arangod/Replication/InitialSyncer.cpp * arangod/RestHandler/RestEdgesHandler.cpp * arangod/RestHandler/RestExportHandler.cpp * arangod/RestHandler/RestReplicationHandler.cpp * arangod/RestHandler/RestWalHandler.cpp * arangod/RestServer/arangod.cpp * arangod/RestServer/DatabaseFeature.cpp * arangod/RestServer/UpgradeFeature.cpp * arangod/StorageEngine/EngineSelectorFeature.cpp * arangod/StorageEngine/RocksDBEngine.cpp * arangod/StorageEngine/StorageEngine.h * arangod/Utils/CollectionExport.cpp * arangod/Utils/CollectionKeys.cpp * arangod/Utils/TransactionContext.cpp * arangod/Utils/Transaction.cpp * arangod/V8Server/v8-collection.cpp * arangod/V8Server/v8-replication.cpp * arangod/V8Server/v8-vocbase.cpp * arangod/V8Server/v8-vocindex.cpp * arangod/VocBase/EdgeCollectionInfo.cpp * arangod/VocBase/Ditch.cpp * arangod/VocBase/LogicalCollection.cpp * arangod/VocBase/replication-dump.cpp * arangod/VocBase/transaction.cpp * arangod/VocBase/vocbase.cpp - IndexFactory needs a function to stringifyIndexes and rename idxFoo => IdxZirkusBar - Implement new IndexIterator API nextExtra(callback, limit) - Geo - OperationOptions - revoryMarker - re-enable RocksDB storage engine (e.g arangod.cpp) - implement RocksDB storage engine Questions --------- * For GeoIndex `ignoreNull: true` and `constraint: false` are only set in Cluster mode. Is that art or can it go away? OpenIssues Hacki ---------------- * batchSize == 0 ? What Happens? ASSERT ok? React gracefully? * HashIndex Lookup into a still local buffer, could be replaced by callback as well. * SingleServerTraverser API does NOT takeover responsibility for slice data. getMore() hopes slices to not go away * This API can be improved if we make better use of those callbacks. * ShortestPathBlock does assume that slices do not walk away. * EdgeCollectionInfos in ShortestPath could share one conditionBuilder.