1
0
Fork 0
Commit Graph

69 Commits

Author SHA1 Message Date
Jan 49babbc66b
handle incomplete read errors (#9021) 2019-05-20 14:13:41 +02:00
Simon 7cd84a785a Remove Obsolete code (#8657) 2019-04-03 13:40:44 +02:00
Simon fd70b6fc34 Background Indexing (#8039) 2019-01-29 09:31:32 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
Jan d78adb7ae1
Bug fix/turn off range deletes in cluster (#6178) 2018-08-17 08:41:00 +02:00
Simon 392118bd62 Use RangeDelete where possible (#6121) 2018-08-15 18:52:09 +02:00
Simon 439b12331f Big endian document keys for rocksdb (#5712) 2018-07-03 14:11:22 +02:00
Jan 72ec463a8d
Bug fix/cleanup 2005 (#5404) 2018-05-20 20:57:43 +02:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
Simon cde96d97aa RocksDB replication hardening & cleanup (#4639) 2018-02-23 18:15:04 +01:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan a212f45a86 Bug fix/issues 2308 (#3117)
* remove API docs for endpoints that were deleted in 3.0

* remove unused variable

* set all scalar values in the ctor, so check tools do not complain

* move Result objects

* prevent potential nullptr access

* remove ambiguous boolean parameter with no clear meaning

* make code check tools happy

* don't allow copying cache Findings, allow moving Results

* the debug functionality for counting RocksDB key ranges was broken

this produced sporadic test failures

* remove superfluous iostream include

* fix range checks

* don't leak if an error happens during database removal

* try to simplify Result a bit

* try to deletion checks a bit more robust
2017-08-24 14:47:22 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan bee32b6325 properly compact ranges so data actually gets deleted after drop etc. (#2771) 2017-07-11 21:47:16 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713)
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
2017-07-05 10:18:52 +02:00
Michael Hackstein f7e40a14ca RocksDBBounds now give info about their columnFamily. Adapted convenience methods that used those bounds to use correct column family now 2017-06-08 11:47:02 +02:00
Simon Grätzer 87ee7a45fe Adding error checks 2017-05-31 15:10:17 +02:00
Jan Christoph Uhde e0f608f219 check endianess when storing ints 2017-05-31 10:52:32 +02:00
Simon Grätzer d0eafe420a fixed vertexId getter 2017-05-30 12:46:18 +02:00
Jan Christoph Uhde 64c0b03a99 Add missing changes to geo index 2017-05-30 11:08:44 +02:00
Jan Christoph Uhde c4e7ae54da Prepare geodata to be more platform independent 2017-05-30 09:22:45 +02:00
Michael Hackstein b9d2faa7cc Added warmup function for indexes.
Squashed commit of the following:

commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:39:31 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:00:41 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 15:53:23 2017 +0200

    Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)

commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 12:46:20 2017 +0200

    Implemented client-side warmup. It can now be triggered via http and arangosh

commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:25:40 2017 +0200

    Fixed assertion.

commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:08:46 2017 +0200

    Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values

commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:06:43 2017 +0200

    Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted

commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:04:47 2017 +0200

    Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries

commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:45:38 2017 +0200

    Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on

commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:44:21 2017 +0200

    Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document

commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 07:56:58 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed May 24 09:52:01 2017 +0200

    Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
2017-05-29 09:40:58 +02:00
Simon Grätzer 21dd147ad6 Fixing issues 2017-05-26 14:03:12 +02:00
Simon Grätzer 555a7fcecd Merge branch 'devel' of https://github.com/arangodb/arangodb into column-family 2017-05-24 19:04:20 +02:00
Michael Hackstein 91e19071a9 The Cuckoo index estimator now uses 32bit counters instead of 16bit. It should be more relyable now. 2017-05-23 12:56:24 +02:00
Simon Grätzer 9679aecf53 Adding bounds back 2017-05-22 17:39:19 +02:00
Simon Grätzer a99611e7c0 Various changes 2017-05-19 16:43:00 +02:00
jsteemann 226920d7fa optimizations 2017-05-17 23:41:16 +02:00
Simon Grätzer 6472d7cc94 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/RocksDBEngine/RocksDBCollection.cpp
2017-05-15 17:36:09 +02:00
Simon Grätzer 70a1d27121 intermediate commits 2017-05-15 17:35:16 +02:00
Michael Hackstein d7d650e27d Implemented index estimates for RocksDB. Now the following indexes have
a proper estimate:
* HashIndex
* SkiplistIndex / Persistent
* EdgeIndex
2017-05-15 17:17:58 +02:00
jsteemann 422b06211f fix some replication issues 2017-05-11 19:00:32 +02:00
Simon Grätzer ddf7ae60cf WAL tailing 2017-04-27 17:38:02 +02:00
Jan Christoph Uhde 26112be584 do checking before creating copy of slice 2017-04-26 08:33:23 +02:00
Dan Larkin 9dd275b497 Fixed dump/restore tests. 2017-04-25 13:54:56 -04:00
Dan Larkin 57701ff766 Added (rough) logger-follow handler. 2017-04-20 14:59:49 -04:00
Dan Larkin dfd1e78f35 Added objectId-to-database+collection mapping. 2017-04-19 13:56:17 -04:00
jsteemann 8e779aab24 cppcheck 2017-04-19 13:20:46 +02:00
Jan Christoph Uhde d67aa33fe3 limit transaction size and work on intermediate commits 2017-04-12 09:01:13 +02:00
Jan Christoph Uhde 8df9fe1d21 fix error in insert 2017-04-10 21:27:52 +02:00
Jan Christoph Uhde c82fadc863 prepare limitation of transaction size 2017-04-10 14:00:55 +02:00
jsteemann 2b0673c3fe removed type "Index" 2017-04-07 12:05:56 +02:00
Simon Grätzer 96dc714139 Fixed some tests in shell_server 2017-04-04 15:10:17 +02:00
Jan Christoph Uhde 34aa22948d add helper function for non-transactional iteration of bounds 2017-04-04 09:54:58 +02:00
Jan Christoph Uhde 5773a028c8 add more code for dropping databases 2017-04-03 21:29:25 +02:00
Jan Christoph Uhde 256b2b2d6a construct rocksdbkey from slice 2017-04-03 17:50:36 +02:00