1
0
Fork 0
Commit Graph

597 Commits

Author SHA1 Message Date
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713)
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
2017-07-05 10:18:52 +02:00
Jan 94b4a9ec4b added "deduplicate" attribute for array indexes (#2644) 2017-06-30 17:53:50 +02:00
jsteemann 441e7ef37d only use one dash in error message 2017-06-07 00:30:49 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
Simon Grätzer 634254b80f adding load and unload 2017-06-02 14:44:38 +02:00
Simon Grätzer 53ece582a8 optimized all index iterator 2017-05-31 11:34:32 +02:00
Michael Hackstein b9d2faa7cc Added warmup function for indexes.
Squashed commit of the following:

commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:39:31 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:00:41 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 15:53:23 2017 +0200

    Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)

commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 12:46:20 2017 +0200

    Implemented client-side warmup. It can now be triggered via http and arangosh

commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:25:40 2017 +0200

    Fixed assertion.

commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:08:46 2017 +0200

    Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values

commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:06:43 2017 +0200

    Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted

commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:04:47 2017 +0200

    Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries

commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:45:38 2017 +0200

    Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on

commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:44:21 2017 +0200

    Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document

commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 07:56:58 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed May 24 09:52:01 2017 +0200

    Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
2017-05-29 09:40:58 +02:00
jsteemann a7b3839e6c renamed "getMore" to "next" 2017-05-23 08:53:13 +02:00
jsteemann 686a964718 adjusted error message 2017-05-22 13:53:22 +02:00
Michael Hackstein cf12573507 Removed code paths that wrote objectIds into the Agency. This did break replication. 2017-05-02 14:59:19 +02:00
jsteemann 7344a03623 fix crashes 2017-04-25 15:47:11 +02:00
jsteemann 4300c77d3e fixes for non-array IN lookups, added tests 2017-04-25 01:47:07 +02:00
jsteemann 4d6c21f3c5 fixed issue #2450 2017-04-24 23:06:45 +02:00
Simon Grätzer c31d4c1590 Fixed iterator 2017-04-18 17:17:28 +02:00
Dan Larkin 841ef8b1a0 Changed index VPack building to include forPersistence flag. 2017-04-17 17:04:50 -04:00
Jan Christoph Uhde 2700a6d79d fix rocksdb engine index reporting but break report for mmfiles engine 2017-04-12 14:50:55 +02:00
Michael Hackstein 3cb9f0b805 Fixed a bug with SortedIndexes. Some sort functions for AstNodes ordered upperbounds before lower bounds. This caused the index iterator to ignore the upperbound in sparse index cases. 2017-04-11 17:57:59 +02:00
jsteemann f59c78b587 add engine capabilities 2017-04-06 14:50:27 +02:00
jsteemann 49551127b9 de-constified lookupKey 2017-03-31 10:16:14 +02:00
jsteemann f17ff29518 added stubs for indexes 2017-03-23 15:24:00 +01:00
jsteemann 6ba59eda5b index API change 2017-03-20 14:40:54 +01:00
jsteemann fd07af5216 renamed index type 2017-03-16 17:00:19 +01:00
Michael Hackstein 91f303e963 Removed readRevision from LogicalCollection. You only get readDocument with a token. Moved it's logic into MMFilesCollection. removed the ignoreWal parameter from readDocumentConditional. It was always true anyways... 2017-02-17 17:29:49 +01:00
Michael Hackstein b84b05244f Moved pre-commit document modifications out of logical-collection into the physical collection. They are only relevant when actually writing to disc. 2017-02-17 16:07:02 +01:00
jsteemann 7e690a0c61 renaming 2017-02-13 15:18:40 +01:00
jsteemann eb73dbf960 renaming 2017-02-13 13:30:57 +01:00
jsteemann f09ec85127 renaming 2017-02-13 12:27:26 +01:00
Michael Hackstein 07154391a2 Removed iteratorForSlice on Indexes and adapted index scan on transaction, because it can now only be used for ANY or ALL scans. 2017-02-10 15:47:24 +01:00
Michael Hackstein 7083671116 Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-03 13:22:43 +01:00
Michael Hackstein 5e4b9e42b3 Fixed minor bugs in usage of callback-based index-api. Remove unused old next and nextBabies index iterator functions. 2017-02-03 13:22:06 +01:00
jsteemann eb89982279 reserve space 2017-02-03 11:17:47 +01:00
Michael Hackstein d920788a7f The new index-callback API returns if it has more elements or not 2017-02-01 17:33:52 +01:00
Michael Hackstein 0c3cada5b8 Added a new callback based option for IndexIterators to hand out DocumentTokens. This feature is not yet used anywhere 2017-02-01 16:07:37 +01:00
jsteemann 6598828449 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-01-31 00:33:42 +01:00
Dan Larkin 5b736dc319 Changed index-filling to dispatch batch insertion tasks to scheduler, obsoleted index-threads option. 2017-01-30 13:05:10 -05:00
Michael Hackstein 55e0f63fec Moved major Index-creation out of LogicalCollection into the StorageEngine Specific IndexFactory. 2017-01-29 15:08:26 +01:00
Michael Hackstein 0027bc54c8 Create a StorageEngine-specific IndexFactory. Moved EnhanceIndexJson v8 -> IndexFactory. 2017-01-28 21:19:50 +01:00
Michael Hackstein ce6d3b3492 The Fulltext index now return DocumentIdentifierTokens instead of revisions. Removed one inderection rid <-> Index_doc which were a typedef. 2017-01-25 13:20:15 +01:00
Michael Hackstein 2133b1448c Moved index API to return DocumentIdentifierTokens which are independent of the underlying storage engine. The agreement is, that the Engine gives a uint64_t sized identifier which allows to fetch any document back again. For MMFiles Engine this identifier is the revision. 2017-01-24 18:19:25 +01:00
jsteemann e3bcb90916 remove revisions cache 2017-01-24 10:48:07 +01:00
jsteemann c4d2ff3dce renaming 2017-01-23 15:36:11 +01:00
Michael Hackstein 92ad574c97 Moved Indexes to StorageEngine 2017-01-23 13:32:23 +01:00
jsteemann f9fb701505 moved LogfileManager into engine 2017-01-23 10:30:55 +01:00
jsteemann a9f749f549 micro optimizations for geo index queries 2017-01-04 12:34:24 +01:00
Andreas Streichardt c3ae91b405 Remove unused variable 2017-01-03 16:41:45 +01:00
jsteemann 16f16b768d optimizations for geo index 2017-01-03 14:17:49 +01:00
jsteemann 035305ff69 fix nullptr dereference 2016-12-27 18:42:40 +01:00