Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
42,127 Commits 358 Branches 495 Tags 994 MiB
Commit Graph

243 Commits

Author SHA1 Message Date
Jan bee32b6325 properly compact ranges so data actually gets deleted after drop etc. (#2771) 2017-07-11 21:47:16 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713) 
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
 2017-07-05 10:18:52 +02:00
Jan ccbe1ba792 Bug fix/fix rocksdb options (#2702) 
* cleanup rocksdb options
 2017-07-02 23:13:42 +02:00
Dan Larkin 05c8d3ae71 Converted Get calls to use PinnableSlice where possible. (#2683) 2017-07-01 00:27:27 +02:00
Dan Larkin ee8cb749bb Added recovery tests for views and fixed a few related bugs. (#2682) 
* Added recovery tests for views and fixed a few related bugs.

* Added more view recovery tests.

* Modified view recovery tests to add a waitForSync operation after.

* fixed usage of wrong view type

* fixed recovery of view change markers
 2017-06-29 17:09:43 +02:00
Jan cfbd8ed93c Bug fix/rocksdb autoincrement (#2648) 
* Added autoincrement keygen support to RocksDB engine with test.

* fixed key generator state keeping for mmfiles engine
 2017-06-23 15:27:09 +02:00
Frank Celler ce8b1fb885 Feature/rocksdb update 5.6 (#2621) 2017-06-20 18:12:43 +02:00
Dan Larkin a0c4e3d9df Improved RocksDB engine statistics with docs and rest endpoint (#2598) 
* Improved statistics reporting for RocksDB engine (including cache info).

* Added documentation about db._engineStats().

* Added rest endpoint for engine stats.
 2017-06-18 22:55:25 +02:00
Michael Hackstein f21962dd03 Merged in Column Family for Views 
Squashed commit of the following:

commit 860ddfc7a74866919ce5139ac3d874f09501ec97
Merge: d9ec0e3 bf20939
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Jun 8 12:45:14 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/views-column-family

commit d9ec0e3c88122c6b9ee8acdbc6e69ca6af62e19f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Thu Jun 8 12:44:43 2017 +0200

    Added a column family for views. It is no yet used anywhere. Just preparation.
 2017-06-08 12:46:01 +02:00
jsteemann 427fc35f35 fix MSVC compile warnings 2017-06-08 10:48:39 +02:00
Ewout Prangsma 4d6fa5bb87 Import RocksDB encryption support (#2491) 
* Import RocksDB encryption support

* Adding enterprise hooks

* Allocate scratch space in BlockCipher, used for all subsequent calls to EncryptBlock

* Adding more enterprise hooks

* Added encryption test script

* Added (commented) entry for encryption-keyfile

* Use correct env

* Updated ARANGODB value

* Log startup at debug level

* Removed TTY flag

* Fixed build

* Added prefix block support with unique initial counter & IV for CTR implementation
 2017-06-08 08:51:02 +02:00
Dan Larkin c48eb92c1e Added some more pass-through options for RocksDB and optimized a couple defaults. 2017-06-06 10:51:01 -04:00
jsteemann 6b05b39825 expose currently used block cache size 2017-06-06 11:19:11 +02:00
Simon Grätzer 66f10870dd Renaming files 2017-05-31 18:12:47 +02:00
Frank Celler f45325fe3f comment out new options. Needs further investigation into performance impact 2017-05-31 09:15:45 +02:00
jsteemann 3886aece76 updated manual 2017-05-30 13:09:06 +02:00
jsteemann 53b150b206 added tests for intermediate commits 2017-05-30 12:03:58 +02:00
jsteemann c7e1877fd6 workaround for compile issue 2017-05-29 13:02:22 +02:00
Frank Celler 4b61f5b8bb added --rocksdb.num-uncompressed-levels 2017-05-29 11:56:48 +02:00
jsteemann 76f2ce5b2c nicer error message on startup when RocksDB data version does not fit 2017-05-29 11:36:05 +02:00
Frank Celler b7961ef3dd Reenabled snappy (#2504) 
* fixed snappy config

* added numactl and GLIBCXX_FORCE_NEW

* enabled snappy

* fix snappy include path

* windows paths
 2017-05-26 20:53:23 +02:00
Simon Grätzer 21dd147ad6 Fixing issues 2017-05-26 14:03:12 +02:00
Simon Grätzer d5f85d5bbb Adding more colum families 2017-05-26 10:43:17 +02:00
Simon Grätzer 555a7fcecd Merge branch 'devel' of https://github.com/arangodb/arangodb into column-family 2017-05-24 19:04:20 +02:00
jsteemann d6bb581b18 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-24 11:33:53 +02:00
jsteemann 81786c0aa6 print compression support 2017-05-24 11:33:46 +02:00
jsteemann 631453a1f3 more options 2017-05-24 11:15:36 +02:00
jsteemann b23285df81 added more options, but not effective yet 2017-05-24 11:02:24 +02:00
Simon Grätzer 9679aecf53 Adding bounds back 2017-05-22 17:39:19 +02:00
Simon Grätzer ad3da410b2 Pregel improvements 2017-05-22 11:10:28 +02:00
Simon Grätzer 45d7c5c715 Various debugging changes 2017-05-20 17:44:28 +02:00
Simon Grätzer e73f446c5b Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-18 22:08:39 +02:00
Simon Grätzer 2bd2a15cb3 Column family support 2017-05-18 22:08:34 +02:00
Michael Hackstein e4050f9804 Temporary Commit, modifying some RocksDB Options and the EdgeIndex Cache size. This commit is used for NoSQL Performance Banchmark validation. 2017-05-18 15:45:49 +02:00
jsteemann a8320e99dc don't force compaction after every collection drop operation 2017-05-16 12:22:30 +02:00
Michael Hackstein d7d650e27d Implemented index estimates for RocksDB. Now the following indexes have 
a proper estimate:
* HashIndex
* SkiplistIndex / Persistent
* EdgeIndex
 2017-05-15 17:17:58 +02:00
jsteemann 257e3a05e3 update rocksdb logging to error level 2017-05-12 17:27:22 +02:00
jsteemann b1aae2efdd remove unused options 2017-05-12 12:12:09 +02:00
jsteemann b848030818 enable rocksdb logger 2017-05-12 12:03:54 +02:00
Simon Grätzer e11ef2bf54 Addding locking, fixing conversion warning 2017-05-11 17:38:50 +02:00
Simon Grätzer 9950fa7245 removing unused and unimplemented methods 2017-05-10 18:25:41 +02:00
Simon Grätzer 5fa85761a2 adding transactions to the geo index 2017-05-10 14:54:39 +02:00
Dan Larkin 9662b26002 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-09 18:44:15 -04:00
Dan Larkin ca91c0f7c9 Added some optimizations enabled by prefix extraction. 2017-05-09 18:36:12 -04:00
Simon Grätzer 3b7ef439c8 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/rocksdb-geoindex 2017-05-09 17:42:27 +02:00
jsteemann 488e4be485 preparation for RocksDB logger 2017-05-09 12:42:25 +02:00
jsteemann 587418fd1b constified 2017-05-09 02:13:51 +02:00
Dan Larkin e601e1efd9 Added prefix extractor. 2017-05-08 16:47:36 -04:00
jsteemann 12b46d2086 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-08 18:27:07 +02:00
jsteemann 7577fed6fb added logger 2017-05-08 18:24:40 +02:00
Dan Larkin f6a20e1f41 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-08 12:09:10 -04:00
Dan Larkin 284bb6fb51 Fixed some issues with RocksDB passthrough options. 2017-05-08 12:08:41 -04:00
Jan Christoph Uhde a4a04350a3 fix replication 2017-05-08 17:43:36 +02:00
Jan Christoph Uhde acee270075 fix small bugs revealed by tests 2017-05-08 15:43:09 +02:00
Jan Christoph Uhde 8ee0406c64 move last logger creation to engines 2017-05-08 15:18:19 +02:00
Simon Grätzer a610d81e3d fixing compiler warnings 2017-05-08 14:14:23 +02:00
Jan Christoph Uhde cf333f3b0a move creation of tickranges and first ticks into storage engines 2017-05-08 14:11:41 +02:00
Dan Larkin 9924fa1480 Added more RocksDB passthrough options. 2017-05-07 11:08:25 -04:00
Simon Grätzer dd0b0c6b28 RocksDB Fülltext index 2017-05-06 01:14:49 +02:00
jsteemann 8c2be23d73 do not call SyncWAL() on Windows, as it does not work 2017-05-05 15:37:38 +02:00
jsteemann 6a30df3363 add some safety padding for WAL file deletion 2017-05-05 15:31:57 +02:00
jsteemann d352794360 optimize away own transaction management for RocksDB engine 2017-05-04 23:16:53 +02:00
Dan Larkin 037f3f38b9 Removed accidental log spam. 2017-05-04 15:30:09 -04:00
Dan Larkin d77efe38dc Added explicit WAL file lifecycle management to reduce space overhead from replication. 2017-05-04 15:16:24 -04:00
Simon Grätzer 40b8eb2824 Trigger compaction on drop and truncate 2017-05-04 13:45:30 +02:00
Simon Grätzer 1c82d4448c Adding compaction and estimateSize method for rocksdb 2017-05-04 13:08:02 +02:00
jsteemann 1e72f8ccdc allow configuration of use_fsync from the outside 2017-05-04 01:15:09 +02:00
jsteemann f1763e491d fix db._engineStats() 2017-05-04 00:33:36 +02:00
jsteemann 2e3d704d10 fix MSVC compile warnings 2017-05-03 15:47:45 +02:00
jsteemann 9a48bd3411 fix unintended change of option value 2017-05-03 15:18:44 +02:00
jsteemann c716fcacd3 write settings on shutdown 2017-05-03 15:18:17 +02:00
Simon Grätzer e62d82f1d0 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 
# Conflicts:
#	arangod/RocksDBEngine/RocksDBEngine.cpp
#	arangod/RocksDBEngine/RocksDBV8Functions.cpp
 2017-05-03 12:27:58 +02:00
Simon Grätzer b5df720870 Added rocksdb properties 2017-05-03 12:23:19 +02:00
Jan Christoph Uhde b7f42f638e add empty impl for recalculating counts in rocksdb engine 2017-05-02 17:54:36 +02:00
Jan Christoph Uhde da02fd36c6 move engine specific syncer code into engines 2017-05-02 16:36:27 +02:00
Dan Larkin e6611ddc4f Fixed ongoing replication bugs and split the test. 2017-04-28 16:01:29 -04:00
Simon Grätzer 84ed7ec100 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 
# Conflicts:
#	arangod/RocksDBEngine/RocksDBRestReplicationHandler.cpp
 2017-04-28 14:40:17 +02:00
Simon Grätzer 0a00a7eb67 Fixed broken JSON output 2017-04-28 14:28:34 +02:00
Michael Hackstein 360d2b3b99 Fixed restoreIndox for RocksDB. Also removed Primary and EdgeIndexes from Inventory, which is identical to MMFiles 2017-04-28 11:09:53 +02:00
Dan Larkin e2853e4b52 Enabled saving applier config and fixed some replication tests/behavior. 2017-04-27 22:39:48 -04:00
Simon Grätzer 9e498851a9 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-04-27 18:23:14 +02:00
Michael Hackstein 70f5413fe8 RocksDB inventory now includes indexees as well 2017-04-27 18:11:06 +02:00
Simon Grätzer 88c62c05a9 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-04-27 17:38:06 +02:00
Simon Grätzer ddf7ae60cf WAL tailing 2017-04-27 17:38:02 +02:00
Michael Hackstein e0d9fd3cc3 Redesigned Collection.drop() in RocksDB. Does not write LogData and has a different workflow to create a consistent user-view 2017-04-27 12:58:51 +02:00
Dan Larkin f8efe6e627 Added create index and change collection marker creation. 2017-04-26 15:23:25 -04:00
Michael Hackstein e423a5fde1 Added WAL MARKERS for CreateCollection, RenameCollection and DropIndex, also adjested WalMarkerType constructors. 2017-04-26 18:32:28 +02:00
Simon Grätzer 6472c4928d Various warnings 2017-04-26 11:26:08 +02:00
jsteemann f36a889709 try to fix shutdown issues 2017-04-26 09:58:19 +02:00
Jan Christoph Uhde fca8f01df6 create logger state just in one place 2017-04-25 14:58:32 +02:00
jsteemann 0434d88ab4 rocksdb options 2017-04-25 02:22:00 +02:00
Simon Grätzer 88dc999001 Counter manager fixes 2017-04-24 12:43:44 +02:00
jsteemann 361d513087 re-enable user settings for RocksDB 2017-04-21 15:28:29 +02:00
jsteemann 0444a5a068 disable usage of (bad) default values 2017-04-21 14:02:50 +02:00
jsteemann 862ec7bae3 remove log garbage 2017-04-21 12:57:20 +02:00
Simon Grätzer 98df9c21a2 Fixing warnings 2017-04-20 19:07:30 +02:00
Simon Grätzer b304d35ac2 Added rocksdb background thread 2017-04-20 18:50:52 +02:00
jsteemann 569450321d cppcheck 2017-04-20 16:47:10 +02:00
Simon Grätzer 6dd02bab0d Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-04-20 12:47:28 +02:00
Simon Grätzer 3319a5d607 Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 
# Conflicts:
#	arangod/RocksDBEngine/RocksDBRestReplicationHandler.cpp
 2017-04-20 12:47:22 +02:00