1
0
Fork 0
Commit Graph

1810 Commits

Author SHA1 Message Date
Lars Maier d8bc3685d3 [3.4] Fast Leader Change (#9642)
* Intermediate state.

* Finished.
2019-08-09 14:02:04 +02:00
KVS85 cc9aa74e1a
Fix ArangoSearch view creation errors (#9598) 2019-07-29 20:42:57 +03:00
Jan 65e7557f42 Bug fix 3.4/improve arangorestore (#9156)
* always import system collections first, so if importing some other collection fails later, the system collections are already in place

* properly disable enterprise features when restoring an enterprise dump into a community cluster

* disable enterprise features when restoring an enterprise dump into the community version

* fix handling of "duplicate name" issues when restoring system
collections
2019-06-09 20:21:07 +02:00
Wilfried Goesgens 8e0ebb00c6 Bugfix/backport fs changes (#9024) 2019-05-23 12:46:47 +02:00
Michael Hackstein 4d4c23c302
Bug fix 3.4/collection babies (#9033)
* Prepare API to create multiple collections in a single request to ClusterMethods to improve speedup

* Added counter on how many collections are successfully created

* Allow multi collection creation one level higher

* CollectionMethods now allow batch createion of Collections

* Improved array size assertions

* Now a graph is createad within a single roundtrip in the agency.

* Added new header files

* Insert collections in the AGENCY with TTL and a isBuilding flag, collections with this flag should not be visisible in the coordinator

* Added forgotten C++ file

* Fixed a rare race condition, and the failing IResearch Tests

* readded callback on DONE, otherwise lists are out of sync

* Fixed assertions to let mocked tests pass...

* Fixed community cluster
2019-05-21 08:41:12 +02:00
Simon 30e785537a Bug fix 3.4/pregel gr (#9007) 2019-05-15 18:38:22 +02:00
Jan d29d65face
Bug fix 3.4/cleanup 31032019 (#8633) 2019-04-01 17:02:00 +02:00
Jan 3631d55146
prevent PregelFeature from shutting down while its workers are running (#8627) 2019-03-29 18:22:37 +01:00
Heiko 6b1d56a62c fixed smg collections within the transaction, added gharial decoding (#8621) 2019-03-29 11:10:36 +01:00
Jan 3f3f0c6fe3
Feature 3.4/ncc1701 (#8441) 2019-03-21 14:53:28 +01:00
Jan db7fcdce7a
don't run compact() on a collection after a truncate() was done in the same transaction (#8471)
running compact() in the same transaction will only increase the data size on disk due to
  RocksDB not being able to remove any documents physically due to the snapshot that is
  taken at transaction start.

  This change also exposes db.<collection>.compact() in the arangosh, in order to manually
  run a compaction on the data range of a collection should it be needed for maintenance.
2019-03-20 17:44:06 +01:00
Max Neunhöffer 54c533c5f6
Some more agency fixes. (#8377)
* Ignore satellite collections in shrinkCluster in agency.
* Abort RemoveFollower job if not enough in-sync followers or leader failure.
* Break quick wait loop in supervision if leadership is lost.
* In case of resigned leader, set isReady=false in clusterInventory.
* Fix catch tests.
2019-03-12 14:09:56 +01:00
Jan cd5c9edce1
various replication improvements (#8300) 2019-03-11 13:07:43 +01:00
Jan 6941eb941c
added option `--console.history` to arangosh (#8328) 2019-03-07 13:14:51 +01:00
Simon a52e6fa3d3 Sync Foxx Queues (#8254) 2019-02-25 17:16:26 +01:00
Jan 30c61a5c82
improve error messages when restoring from invalid JSON data (#8211) 2019-02-20 18:34:43 +01:00
Tobias Gödderz a83d42d0e9 [3.4] Forbid ambiguous casts to and from ResultT (#7368)
* Forbid ambiguous casts to and from ResultT

* Reformat

* Changed enabled_if checks to check for implicit casts to Result

* Added comments
2019-02-19 12:50:39 +01:00
Jan 75ab4ac3dc
Bug fix 3.4/misc issues (#8012)
* added missing return statements

* only spend up to 10 seconds for initially fetching the list of collections in arangosh

fetching the list of collections is a blocking operation, and the default timeout for this is very high.
If the server is blocked by whatever reason, then the shell is unusable until the collections list request returns.
To avoid this, the initial request is limited to 10 seconds, so the shell can be used afterwards.

* if an index cannot be used for sorting, its sort

cost was previously returned as 0. this will in fact favor
indexes that can be used for filtering but not for sorting
over indexes that can be used for both.

this change is to report the sort cost for indexes that
cannot be used for sorting to n * log(n), where n is the
number of documents that optimizer expects to come out of the
index after filtering
2019-01-28 08:55:06 +01:00
Matthew Von-Maszewski 474f0cde31 Bug fix 3.4/scheduler empty reformat (#7872)
* added check for empty scheduler

* removed log, old is 1 not 0

* require running in this thread

* test

* added isDirect to callback

* signature fixed

* added drain

* added allowDirectHandling

* disabled for testing

* Add ExecContextScope object to direct call.

* try alternate initialization of ExecContextScope

* remove ExecContextScope, no help.  try _fifoSize as part of direct decision.

* strand management to minimize reuse of same strand per listen socket

* blind attempt to address Jenkins shutdown lock up.  may remove quickly.

* add filename and line to existing error log message

* Adjust queueOperation() to stop accepting items once isStopping() becomes true.

* revert previous check-in to MMFilesCollectorThread.cpp

* big reformat

* fixed merge conflicts

* Add CHANGELOG entry.
2019-01-08 20:39:42 +01:00
Jan a4a7867451
Bug fix 3.4/arangorestore add cleanup duplicate attributes (#7876) 2019-01-04 15:26:11 +01:00
Jan a14f6dd573
prevent duplicate attributes being generated by AQL queries (#7836) 2019-01-02 12:37:47 +01:00
Frank Celler 9477af198b big reformat 2018-12-26 00:57:05 +01:00
Michael Hackstein bc65e79b02
Bug fix 3.4/fix tombstones (#7758)
* fixes a rare situation where replication could require identical lock twice

* Updated changelog
2018-12-12 22:01:27 +01:00
Simon 933ca8a775 Bug fix/restore index refactor (#7470) (#7491)
(cherry picked from commit d0efd95a37)
2018-11-29 14:08:29 +01:00
Jan ffc823e1c8
Bug fix 3.4/backport optimizations (#7434) 2018-11-26 19:16:05 +01:00
Michael Hackstein 8098bb4eed
Bug fix 3.4/syncing of followers (#7377)
* Added some DEBUG output for replication rest handler

* Some more debug logging.

* Increased the priority of the ReplicationHandler. This way we will not get stuck with locks that cannot be canceled. Also cancel the lock on the correct database.

* Added extensive log output for replication thins

* Added tombstones to RestReplicationHandler. In a very unlikely case the cancel of a lock can be executed BEFORE the code that actually registers the lock, in this case we will now write a tombstone and do not lock.

* Revert "Added extensive log output for replication thins"

This reverts commit 6d4e37ea1e59e3b3457336019cc7dbc4c979504d.

* Added extensive log output for replication things, now in ERR level instead of MAINTAINER only

* Now actually use hours for synchronization

* React to errors under soft lock if they show up.

* Added a retry loop to increase the read-lock timer.

* Added more timeing output in RocksDB collection internals to figure out why the followers are dropped

* Tweaked RocksDB options

* Revert "Tweaked RocksDB options"

This reverts commit 2bf9c43280beda4792c47d079387fe5154cdd896.

* Removed debug output

* Applied all requested changes by goedderz

* Deleted unused variable
2018-11-23 16:08:27 +01:00
Wilfried Goesgens c50d346453 add alternative to ClusterInfo::getCollection() that doesn't throw (#7413)
* add alternative to ClusterInfo::getCollection() that doesn't throw (#7339)

* handle more potential nullptrs, fix try/catch scope
2018-11-23 11:15:25 +01:00
Simon ef239cbe4e Make recovery more reliable (#7297) (#7367) 2018-11-21 16:51:38 +01:00
Jan eda236f968
AuthenticationFeature::isEnabled() is not doing what is expected (#7373) 2018-11-20 11:30:10 +01:00
Jan 8bcb1a310c
fix failing non-deterministic query-stream test (#7296) 2018-11-13 11:37:09 +01:00
Vasiliy 1ba23cd39b issue 496.5: backport 3.4: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291 (#7217)
* issue 496.5: backport 3.4: minor API cleanup and error reportin enhancements, update iresearch to commit d69f7bd184e009da7bf0a478efd34a0c85b74291

* add workaround for shell-collection-rocksdb-noncluster.js::testSystemSpecial test failure

* fix typo
2018-11-05 16:17:41 +03:00
Vasiliy d644561f1f issue 496.4.1: backport 3.4: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7213)
* issue 496.4.1: backport 3.4: move StorageEngine-specific flag out of the genric API and closer to the storage engine

* address merge issue
2018-11-04 16:52:54 +03:00
Simon 9c53d045be Server stream cursor (#7186) (#7210) 2018-11-03 20:17:52 +01:00
Jan eb3deb578f
smaller changes for replication (#7201) 2018-11-02 15:47:49 +01:00
Jan 5ea7d20863
add error checking (#7199) 2018-11-02 15:23:01 +01:00
Vasiliy 850919178f issue 496.3: backport 3.4: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7167)
* issue 496.3: backport 3.4: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address merge issue

* backport: remove code causing nullptr access

* invalidate payload for each field in FieldIterator before setting a value

* address compilation issues
2018-11-01 23:12:39 +03:00
Jan 023ca6dc33
Bug fix 3.4/patch slave collection count (#7156) 2018-10-31 17:20:46 +01:00
Simon f2f4a0fdaf Truncate Fix (Nr 6) (#7176) 2018-10-31 16:32:00 +01:00
Max Neunhöffer 42fd0825ab
Fix timeouts for write operations from coordinator to leader. (#7081)
* Improve logging on coordinator when doing `arangorestore`.

* Return more error information in `mergeResults`.

* Longer timeout for communication coordinator -> leader for writes.

This is taking into account possible write stops from followers needed
to get in sync.

* Fix compilation.

* Get rid of numbers in exception log messages.

* Fix compilation.

* Fix indentation.
2018-10-31 14:39:48 +01:00
Vasiliy 2458705607 issue 496.2: backport 3.4: allow customization of segment size limits on arangosearch view creation, minor code cleanup (#7141)
* issue 496.2: backport 3.4: allow customization of segmentsize lmts on arangosearch view creation, minor code clanup

* backport: fix incorrect condition
2018-10-30 23:41:15 +03:00
Simon c073b9dbbe Make ensureIndexOnCoordinator more robust (#7110) (#7130) 2018-10-30 11:25:06 +01:00
Vasiliy e6a6025818 backport: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion (#7106)
* backport: switch scope of responsibility between a TRI_vocbase_t and a LogicalView in respect to view creation/deletion

* backport: ensure arangosearch links get exported in the dump

* backport: ensure view is created during restore on the coordinator

* Updates for ArangoSearch DDL tests, IResearchView unregistration and known issues

* Add fix for internal issue 483
2018-10-30 12:50:29 +03:00
Michael Hackstein e05880895a
Bug fix 3.4/shorter foot in door (#7084)
* Implement `syncCollectionCatchup` in DatabaseTailingSyncer.

First stab, might not even compile.

* Fixed a typo.

* Fix a typo and a compilation problem.

* Further compilation fix.

* Implement two stage catchup.

* Two small corrections.

* Unified error messages in Synchronize shard job.

* Improved a code comment.

* Fixed autocasting bool->double and double->bool issue. That is truely one of the best features ever invented... </irony>

* Renamed doHardLock => toSoftLockOnly and inverted default value

* Merged soft/hard foot logic with Transaction splits

* Use scopeguards to cancel readlocks
2018-10-26 16:16:52 +02:00
Jan fac788bb89
fix nullptr access to usermanager object (#7094) 2018-10-26 12:59:00 +02:00
jsteemann 435c014a15 Merge branch '3.4' of https://github.com/arangodb/arangodb into 3.4 2018-10-25 00:00:00 +02:00
jsteemann 5e6a267296 remove unneeded string value 2018-10-24 23:59:51 +02:00
Simon e87b42a0c3 Silence tsan warnings (#7051) 2018-10-24 23:58:47 +02:00
Vasiliy 52e2c97693 backport missed changes (#7016) 2018-10-24 15:43:45 +03:00
Andrey Abramov dc98852dda
Bug fix 3.4/internal issue #2921 (#7008)
* issue 494: update iresearch to changeset 6ff909851c760c2111dca5074ab17d35114bee97 (20181022)

* issue 2921: make error message more detailed (#7004)

* issue 2921: make error message more detailed

* upgrade iresearch

# Conflicts:
#	arangod/RestHandler/RestViewHandler.cpp

* fix compilation

* and another fix
2018-10-23 02:41:25 +03:00
Simon c0455e9c60 Add engine specific collection APIs (#6962) 2018-10-19 15:23:55 +02:00