1
0
Fork 0
Commit Graph

200 Commits

Author SHA1 Message Date
Simon Grätzer 4cc8686e82 RocksDB: Optimizing dropCollection / dropIndex / fillIndex (#2927)
* cleaning up several duplicate versions of Index::memory, Index::cleanup, Index::drop, integrating branch issue-383-ensure-delete

* fixing wrong column family
2017-08-05 21:09:41 +02:00
Jan 4f5d5ff54d slightly move responsibility for recovery (#2922) 2017-08-01 14:25:13 +02:00
Simon Grätzer 59dc8d0135 Update rocksdb options (#2880)
* changing some defaults

* Maybe fixing performance

* Modified changelog

* Updated documentation to reflect options
2017-07-27 16:05:31 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Jan 722870b70c fix access to cache object if cache is turned off (#2782) 2017-07-12 22:11:50 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan bee32b6325 properly compact ranges so data actually gets deleted after drop etc. (#2771) 2017-07-11 21:47:16 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713)
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
2017-07-05 10:18:52 +02:00
Jan ccbe1ba792 Bug fix/fix rocksdb options (#2702)
* cleanup rocksdb options
2017-07-02 23:13:42 +02:00
Dan Larkin 05c8d3ae71 Converted Get calls to use PinnableSlice where possible. (#2683) 2017-07-01 00:27:27 +02:00
Dan Larkin ee8cb749bb Added recovery tests for views and fixed a few related bugs. (#2682)
* Added recovery tests for views and fixed a few related bugs.

* Added more view recovery tests.

* Modified view recovery tests to add a waitForSync operation after.

* fixed usage of wrong view type

* fixed recovery of view change markers
2017-06-29 17:09:43 +02:00
Jan cfbd8ed93c Bug fix/rocksdb autoincrement (#2648)
* Added autoincrement keygen support to RocksDB engine with test.

* fixed key generator state keeping for mmfiles engine
2017-06-23 15:27:09 +02:00
Frank Celler ce8b1fb885 Feature/rocksdb update 5.6 (#2621) 2017-06-20 18:12:43 +02:00
Dan Larkin a0c4e3d9df Improved RocksDB engine statistics with docs and rest endpoint (#2598)
* Improved statistics reporting for RocksDB engine (including cache info).

* Added documentation about db._engineStats().

* Added rest endpoint for engine stats.
2017-06-18 22:55:25 +02:00
Michael Hackstein f21962dd03 Merged in Column Family for Views
Squashed commit of the following:

commit 860ddfc7a74866919ce5139ac3d874f09501ec97
Merge: d9ec0e3 bf20939
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Jun 8 12:45:14 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/views-column-family

commit d9ec0e3c88122c6b9ee8acdbc6e69ca6af62e19f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Jun 8 12:44:43 2017 +0200

    Added a column family for views. It is no yet used anywhere. Just preparation.
2017-06-08 12:46:01 +02:00
jsteemann 427fc35f35 fix MSVC compile warnings 2017-06-08 10:48:39 +02:00
Ewout Prangsma 4d6fa5bb87 Import RocksDB encryption support (#2491)
* Import RocksDB encryption support

* Adding enterprise hooks

* Allocate scratch space in BlockCipher, used for all subsequent calls to EncryptBlock

* Adding more enterprise hooks

* Added encryption test script

* Added (commented) entry for encryption-keyfile

* Use correct env

* Updated ARANGODB value

* Log startup at debug level

* Removed TTY flag

* Fixed build

* Added prefix block support with unique initial counter & IV for CTR implementation
2017-06-08 08:51:02 +02:00
Dan Larkin c48eb92c1e Added some more pass-through options for RocksDB and optimized a couple defaults. 2017-06-06 10:51:01 -04:00
jsteemann 6b05b39825 expose currently used block cache size 2017-06-06 11:19:11 +02:00
Simon Grätzer 66f10870dd Renaming files 2017-05-31 18:12:47 +02:00
Frank Celler f45325fe3f comment out new options. Needs further investigation into performance impact 2017-05-31 09:15:45 +02:00
jsteemann 3886aece76 updated manual 2017-05-30 13:09:06 +02:00
jsteemann 53b150b206 added tests for intermediate commits 2017-05-30 12:03:58 +02:00
jsteemann c7e1877fd6 workaround for compile issue 2017-05-29 13:02:22 +02:00
Frank Celler 4b61f5b8bb added --rocksdb.num-uncompressed-levels 2017-05-29 11:56:48 +02:00
jsteemann 76f2ce5b2c nicer error message on startup when RocksDB data version does not fit 2017-05-29 11:36:05 +02:00
Frank Celler b7961ef3dd Reenabled snappy (#2504)
* fixed snappy config

* added numactl and GLIBCXX_FORCE_NEW

* enabled snappy

* fix snappy include path

* windows paths
2017-05-26 20:53:23 +02:00
Simon Grätzer 21dd147ad6 Fixing issues 2017-05-26 14:03:12 +02:00
Simon Grätzer d5f85d5bbb Adding more colum families 2017-05-26 10:43:17 +02:00
Simon Grätzer 555a7fcecd Merge branch 'devel' of https://github.com/arangodb/arangodb into column-family 2017-05-24 19:04:20 +02:00
jsteemann d6bb581b18 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-24 11:33:53 +02:00
jsteemann 81786c0aa6 print compression support 2017-05-24 11:33:46 +02:00
jsteemann 631453a1f3 more options 2017-05-24 11:15:36 +02:00
jsteemann b23285df81 added more options, but not effective yet 2017-05-24 11:02:24 +02:00
Simon Grätzer 9679aecf53 Adding bounds back 2017-05-22 17:39:19 +02:00
Simon Grätzer ad3da410b2 Pregel improvements 2017-05-22 11:10:28 +02:00
Simon Grätzer 45d7c5c715 Various debugging changes 2017-05-20 17:44:28 +02:00
Simon Grätzer e73f446c5b Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-18 22:08:39 +02:00
Simon Grätzer 2bd2a15cb3 Column family support 2017-05-18 22:08:34 +02:00
Michael Hackstein e4050f9804 Temporary Commit, modifying some RocksDB Options and the EdgeIndex Cache size. This commit is used for NoSQL Performance Banchmark validation. 2017-05-18 15:45:49 +02:00
jsteemann a8320e99dc don't force compaction after every collection drop operation 2017-05-16 12:22:30 +02:00
Michael Hackstein d7d650e27d Implemented index estimates for RocksDB. Now the following indexes have
a proper estimate:
* HashIndex
* SkiplistIndex / Persistent
* EdgeIndex
2017-05-15 17:17:58 +02:00
jsteemann 257e3a05e3 update rocksdb logging to error level 2017-05-12 17:27:22 +02:00
jsteemann b1aae2efdd remove unused options 2017-05-12 12:12:09 +02:00
jsteemann b848030818 enable rocksdb logger 2017-05-12 12:03:54 +02:00
Simon Grätzer e11ef2bf54 Addding locking, fixing conversion warning 2017-05-11 17:38:50 +02:00
Simon Grätzer 9950fa7245 removing unused and unimplemented methods 2017-05-10 18:25:41 +02:00
Simon Grätzer 5fa85761a2 adding transactions to the geo index 2017-05-10 14:54:39 +02:00
Dan Larkin 9662b26002 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-09 18:44:15 -04:00