1
0
Fork 0
Commit Graph

415 Commits

Author SHA1 Message Date
Jan b2b6c06cbf
Feature/efficiency (#3736) 2018-01-05 16:51:31 +01:00
Max Neunhöffer 49082d9b0e
clusterInventory: distinguish between isReady and allInSync. (#4213) 2018-01-03 22:17:57 +01:00
Jan a93773c3d0
forgotten fix for globallyUniqueId (#4032) 2017-12-14 10:05:34 +01:00
Simon Grätzer 19f68df395 Use uuid in Initial sync to ensure proper collection mapping (#3965)
* Use uuid in initalsync to ensure proper collection mapping

* fix compile error after merge
2017-12-13 22:12:47 +01:00
Jan 17986ebc08
return error context for "some agency operation failed" (#3760) 2017-12-06 11:16:19 +01:00
Simon Grätzer 0e485f7441 Fixing collection name collection handling in Syncer (#3710) 2017-11-17 16:36:57 +01:00
Jan fce593b724 removed `--recycle-ids` option for arangorestore (#3713) 2017-11-16 14:25:54 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
jsteemann 4dc13142dc yet another fix for replicationFactor validation 2017-10-27 00:26:53 +02:00
jsteemann 376cc7dcad fix replicationFactor value validation 2017-10-26 20:15:50 +02:00
jsteemann d03fb7e01b fix validation for satellite collections 2017-10-26 19:27:25 +02:00
Simon Grätzer 36e51913b3 Changing satellite parameter check 2017-10-26 19:27:18 +02:00
Simon Grätzer d14710b683 Updating Replication Factor (#3528)
* Changing replication factor in 3.2 (#3513)

* Allow changing replicationFactor on coordinator

* Fixing logic

* Allowing change of replication factor

* Additional input validation

* grrr

* Testing invalid inputs

(cherry picked from commit 47600e1ea1)

* jslint

(cherry picked from commit 4d223597db)

* cherry-pick commits from 3.2: 47600e1ea1 4d223597db d684eaa7f8 a898af3723
2017-10-26 15:28:05 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
jsteemann 5ee5890649 increase cache lifetime for cluster selectivity estimates 2017-09-07 16:54:01 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Max Neunhöffer f3acea797b Feature/cluster inventory version (#3152)
* Get rid of a compiler warning in community edition.

* Teach /_api/replication/clusterInventory to report Plan/Version and readiness.

This is first implemented in the ClusterInfo library.
Then the clusterInventory code uses it and checks readiness.
Readiness of a collection means that it is created and all shards
and all replications have been created and are in sync.
2017-08-30 13:34:23 +02:00
Frank Celler 6d08d4f4aa Bug fix/scheduler delete (#3077)
* removed delete call

* cleanup

* lower cpu activity of log thread too

* fix log messages

* do not enter threads into unordered_set, as it is unneeded

* do not compile in calls to disabled plan cache

* moved AQL regex cache from thread local variables to a class of its own

* more sensible thread creation and destruction
2017-08-25 12:00:17 +02:00
Jan a212f45a86 Bug fix/issues 2308 (#3117)
* remove API docs for endpoints that were deleted in 3.0

* remove unused variable

* set all scalar values in the ctor, so check tools do not complain

* move Result objects

* prevent potential nullptr access

* remove ambiguous boolean parameter with no clear meaning

* make code check tools happy

* don't allow copying cache Findings, allow moving Results

* the debug functionality for counting RocksDB key ranges was broken

this produced sporadic test failures

* remove superfluous iostream include

* fix range checks

* don't leak if an error happens during database removal

* try to simplify Result a bit

* try to deletion checks a bit more robust
2017-08-24 14:47:22 +02:00
Jan 79d76c89fb Bug fix/use count for checksum (#3033)
* Count as checksum

* Make readLockId optional as well so upgrades are possible

* fix option name in startup script

* fix some replication issues with RocksDB engine
2017-08-15 11:24:54 +02:00
Jan 8e4dac4fc4 fix fulltext index removal performance, simplified code (#3015)
* simplify index API a bit

* fix fulltext index removal performance, simplified code

* updated CHANGELOG

* fix hanging test

* try to fix shutdown problem

* improve fulltext query performance

* fixed duplicate var

* removed obsolete code

* fix some shutdown races

* do not call ensureIndex that often
2017-08-12 09:25:30 +02:00
Jan a10832d41e do not allow the creation of volatile collections in the RocksDB stor… (#2900)
* do not allow the creation of volatile collections in the RocksDB storage engine

* only create a volatile collection for mmfiles in test

* fix tests
2017-08-03 22:45:17 +02:00
Kaveh Vahedipour 0b6d6d9287 Fixed distributeShardsLike bug in creating collections. numberOfShard… (#2895) 2017-08-03 19:38:16 +02:00
Jan 62f9231aa5 Bug fix/predictable results data modifcation multiple fors (#2921) 2017-08-01 17:08:35 +02:00
Jan Christoph Uhde ed8efe3566 Feature/issue 387 cluster index estimates (#2866) 2017-08-01 09:53:58 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan Christoph Uhde 562cc77b59 Bug fix/issue 2499 insert 9 instead of 1 (#2727)
* Add testcase to demonstrate #2499

  - In subqueries inserts get pushed to the DB-Servers

  - these create documents documents on each DB-Server that don't respect the shard-key
  
- subsequent deletes will fail to detect the shard correctly and will also fail.

* fix for #2499
2017-07-06 09:53:39 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
Simon Grätzer 634254b80f adding load and unload 2017-06-02 14:44:38 +02:00
jsteemann 8bc6b3d3a1 removed default parameters 2017-05-18 15:06:19 +02:00
Simon Grätzer c21bd1eeb8 Rocksb: index locking issue 2017-05-11 15:12:05 +02:00
Simon Grätzer 9950fa7245 removing unused and unimplemented methods 2017-05-10 18:25:41 +02:00
Max Neunhoeffer b2e6ca3aa2 Translate distributeShardsLike into collection name in dump. 2017-05-02 22:36:15 +02:00
jsteemann 45096dd503 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-02 16:09:35 +02:00
Michael Hackstein cf12573507 Removed code paths that wrote objectIds into the Agency. This did break replication. 2017-05-02 14:59:19 +02:00
jsteemann 910f5680b0 moved files 2017-05-02 12:34:28 +02:00
Kaveh Vahedipour e778604cef distributeShardsLike is not ignored when prototype does not exist 2017-04-27 10:38:07 +02:00
Dan Larkin 841ef8b1a0 Changed index VPack building to include forPersistence flag. 2017-04-17 17:04:50 -04:00
Jan Christoph Uhde 5e63ba1e6f Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api
* 'engine-api' of https://github.com/arangodb/arangodb:
  fix objectId handling
  Fixing windows build
  Fixing windows build
  Bugfix: we want the common _and_ the server tests.
  Conversion warnings
  Conversion errors
  Renaming test files
  fix windows build
  fix if clause
  exclude longrunning tests
  add slash here too.
  ...
2017-04-06 11:20:32 +02:00
Jan Christoph Uhde cc39030042 move key options ot logical collection 2017-04-06 11:19:30 +02:00
jsteemann 0e98a32455 fix objectId handling 2017-04-06 11:19:27 +02:00
jsteemann 5880daf2a5 Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-04-05 13:40:25 +02:00
jsteemann 1c43920bfc preparations for revision() 2017-04-05 13:40:18 +02:00
Simon Grätzer 5822e77843 invokeOnAllElements 2017-04-05 13:38:43 +02:00
jsteemann 81dcfbbe3d pass trx into numberDocuments 2017-03-31 11:33:16 +02:00
jsteemann f728fcdbb8 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-03-29 02:41:00 +02:00
jsteemann e1aef394d8 delete useless DEV_TIMERS, fix rwlockcpp11 2017-03-28 22:38:27 +02:00
jsteemann 7bd6bd6e45 slightly adjusted index creation 2017-03-28 08:46:01 +02:00
jsteemann 2c54a6ac1e added some basic transaction handling 2017-03-24 17:23:32 +01:00
jsteemann b8e011eee5 move attributes into engine 2017-03-21 10:31:50 +01:00
jsteemann 1932fd1975 moved files 2017-03-16 16:45:02 +01:00
Simon Grätzer 1c5c616875 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-15 14:12:18 +01:00
Michael Hackstein 1ebf562c4e Moved MMFiles out of vocbase.cpp 2017-03-14 10:29:17 +01:00
Simon Grätzer 7b35f75b23 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/CMakeLists.txt
#	arangod/GeneralServer/GeneralServerFeature.cpp
2017-03-13 12:25:28 +01:00
Michael Hackstein 83663c47b2 Deleted CollectionResult. It is identical to arangodb::Result introduced later. 2017-03-10 17:18:12 +02:00
Michael Hackstein 046793815c Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-03-09 16:37:51 +02:00
Michael Hackstein a7965945d9 Just comment fixes 2017-03-09 12:20:22 +01:00
Michael Hackstein 90b4876d3c Moved WriteMarker code Vocbase => MMFilesEngine. Also removed the decission to write a marker out of Vocbase-API 2017-03-09 11:12:39 +01:00
Simon Grätzer 67066c64ad Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-08 18:20:44 +01:00
jsteemann 3ed19e291d added "shardIds" option for AQL queries to restrict queries to specific shards 2017-03-08 15:07:10 +01:00
Michael Hackstein 580d2a4336 Moved indexes down into the physical collections 2017-03-08 11:03:45 +01:00
Simon Grätzer 2c77f42486 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-07 15:08:06 +01:00
Michael Hackstein e6ab0afb50 Moved indexBuckets from Logical => MMFiles collection. 2017-03-07 14:07:32 +01:00
Michael Hackstein 31df7eb527 Fixed error in recovery for duplicate collection name 2017-03-06 17:34:54 +01:00
Simon Grätzer 080048b080 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-01 15:05:25 +01:00
Simon Grätzer 27c617fe10 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	3rdParty/V8/v8
#	arangod/Transaction/Methods.h
#	arangod/Utils/UserTransaction.h
#	arangod/V8Server/v8-collection.cpp
2017-03-01 14:52:35 +01:00
Michael Hackstein 26d95cfe80 Fixed dead-lock situation in dump tests 2017-03-01 11:10:51 +01:00
Michael Hackstein 7950270134 Removed obsolete code in LogicalCollection. 2017-02-28 16:08:54 +01:00
Michael Hackstein 900b15dd3a API cleanup in LogicalCollection. Removed some unused exposed toVelocypack functions. Removed toVelocyPack with maxTick variant. That is available in the StorageEngine instead 2017-02-28 15:35:31 +01:00
Michael Hackstein cac32a4d1d Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-28 14:59:13 +01:00
Michael Hackstein df4eadac75 Replaced several specialized toVelocyPack functions of LogicalCollection by toVelocyPackIgnore(). Fixed distributeLike bug that occured through this. 2017-02-28 14:54:32 +01:00
jsteemann 5e0fd0c8b2 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-02-27 22:27:51 +01:00
jsteemann b872aa59fd simplified performance logging 2017-02-27 22:22:41 +01:00
jsteemann e37e94adbe Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-02-27 14:38:35 +01:00
jsteemann 666b2f8da9 renaming 2017-02-27 14:38:27 +01:00
Michael Hackstein ae14ae99a5 Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-27 13:01:12 +01:00
Michael Hackstein a884ee2bfb Intermediate commit. Moved many toVPack functions of LogicalCollection to a unified vpack + ignore-keys alternative. This commit is used to run all tests everywhere. May be red 2017-02-27 13:01:07 +01:00
jsteemann d0df1952e8 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-02-24 17:51:59 +01:00
Michael Hackstein c8f03d16e3 updateProperties now returns an CollectionResult container with errorMessage and code. Moved error reporting logic for cluster-case into this function. 2017-02-24 17:00:41 +01:00
Michael Hackstein 076198106d Now also the cluster variant of collection properties uses the shared toVPackVariant 2017-02-23 16:40:33 +01:00
Michael Hackstein cc78b0131f v8-collection-properties now reuses propertiesVPack on logical collection (SingleServerCase) 2017-02-23 15:31:13 +01:00
Michael Hackstein de31c68539 Moved isVolatile into MMfiles collection. 2017-02-23 14:56:54 +01:00
jsteemann 3679050a31 fixed a race when renaming collections 2017-02-23 13:41:48 +01:00
Michael Hackstein daf7cc73e9 Added a propertiesVPack to PhysicalCollection / MMFiles collection for engine specific properties 2017-02-23 11:27:16 +01:00
Michael Hackstein 853394da79 Moved keyGenerator into physicalCollection. Only used when actually storing data. 2017-02-23 11:10:58 +01:00
Michael Hackstein df59411710 Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-22 14:43:10 +01:00
Jan Christoph Uhde 3a201b4362 add persistProperties() to PysicalCollection 2017-02-22 12:47:03 +01:00
Michael Hackstein a359424af0 Moved logic of write Marker out of the dropIndex API from Logical collection. Can be derived from the engine internally. 2017-02-22 12:32:54 +01:00
Jan Christoph Uhde 711a71ac4d add LogicalCollection::invokeOnAllElements()
and remove MMFilesPrimaryIndex.h from Transaction/Methods.cpp
2017-02-22 10:40:27 +01:00
Jan Christoph Uhde fcfe66e254 add getAll/AnyIterator to LogicalCollection 2017-02-22 09:24:38 +01:00
Jan Christoph Uhde d73d25a310 remove some includes 2017-02-21 14:43:21 +01:00
Michael Hackstein 0f7d50765d Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-21 14:12:16 +01:00
Michael Hackstein fbdbfdcb6d Moved primaryIndex knowledge out of Logical Collection. Also moved logic for close into MMFiles 2017-02-21 14:12:07 +01:00
Jan Christoph Uhde 133961ff8d remove applyForTickRange from LogicalCollection 2017-02-21 13:42:59 +01:00
jsteemann f5c8932744 initial version of query execution plan cache, still very rough 2017-02-21 11:58:38 +01:00