1
0
Fork 0
Commit Graph

224 Commits

Author SHA1 Message Date
Jan 1653e9698a
remove unused functionality (#9360) 2019-06-28 18:28:11 +02:00
Max Neunhöffer 80bfb85695
Port agency performance tuning for many shards to devel. (#8647)
* Port agency performance tuning for many shards to devel.
* Add more IDs to LOG_TOPIC calls.
* Even more IDs for LOG_TOPIC.
* Fix a duplicate LOG_TOPIC ID.
* Fix an old merging bug in devel.
* Don't hesitate between phases one and two for small clusters.
2019-04-11 11:14:56 +02:00
Simon c8961b2faa Remove unused bindings (#7143) 2018-10-30 19:42:45 +01:00
Kaveh Vahedipour 1eab18be2a Feature/generate test data for maintenance script (#6249)
* test data script being fixed
* test data can be generated with expectations
* starting / stopping is done by script
2018-09-24 11:04:00 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
Jan 46e36d55f1
replace busy-waiting with just waiting (#5722) 2018-07-03 09:00:54 +02:00
Jan 36e8feb681
fixed internal issue #2559 (#5635) 2018-06-20 11:45:57 +02:00
Max Neunhöffer a84d9f7335
Add an API to query for status of moveShard and cleanOutServer jobs. (#5593)
This is so far intentionally undocumented, since we want to collect
experience with it first.
2018-06-15 16:27:47 +02:00
Simon f6d3032f59 Enabling cache (#4880) 2018-04-03 14:18:57 +02:00
Andrey Abramov 294c36f778 use ERROR_ARANGO_DATA_SOURCE_NOT_FOUND instead of ERROR_ARANGO_COLLECTION_NOT_FOUND/ERROR_ARANGO_VIEW_NOT_FOUND in js part too 2018-03-19 13:55:25 +03:00
Heiko 61de1b6099 Bug fix/optimize shard distribution api and ui (#3921)
* UI: document/edge editor now remembering their modes (e.g. code or tree)

* changed shardDistribution api behaviour, added PUT route to only fetch collection based shard distribution

* ui: optimized shards view, added missing cleanup function in nodes view

* broken test

* shard distribution tests not fit the new api behaviour

* variables as reference

* CHANGELOG
2018-01-02 12:42:12 +01:00
jsteemann b60159725a fixed jslint error 2017-11-17 17:28:16 +01:00
m0ppers 278e51e5be Fix stalling inserts...follower was not up-to-date in some cases (#3727) 2017-11-17 17:18:59 +01:00
Michael Hackstein 5c633f9fae Bug fix/speedup shard distribution (#3645)
* Added a more sophisticated test for shardDistribution format

* Updated shard distribution test to use request instead of download

* Added a cxx reporter for the shard distribuation. WIP

* Added some virtual functions/classes for Mocking

* Added a unittest for the new CXX ShardDistribution Reporter.

* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red

* Shard distribution now uses the cxx variant

* The ShardDistribution reporter now tries to execute count on the shards

* Updated changelog

* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
2017-11-10 15:17:08 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Simon Grätzer ee8209943f Missing things for active / passive (#3578)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* auth = off does not longer make everyone superuser

* Fixing cluster_sync and maybe resilience
2017-11-04 20:30:23 +01:00
Simon Grätzer d14710b683 Updating Replication Factor (#3528)
* Changing replication factor in 3.2 (#3513)

* Allow changing replicationFactor on coordinator

* Fixing logic

* Allowing change of replication factor

* Additional input validation

* grrr

* Testing invalid inputs

(cherry picked from commit 47600e1ea1)

* jslint

(cherry picked from commit 4d223597db)

* cherry-pick commits from 3.2: 47600e1ea1 4d223597db d684eaa7f8 a898af3723
2017-10-26 15:28:05 +02:00
Max Neunhöffer 5e2f17c04f Fast path for get-in-sync of empty shards. (#3470)
* Fast path for get-in-sync of empty shards.

* Fix short cut in synchronization.
2017-10-26 09:55:00 +02:00
Jan 9eb5e1545f Bug fix/fix cluster foxx queue startup (#3507)
* slightly clean up indexes

* proper stringification of error code

* do not run AQL queries during Foxx queue startup

* always create keyspace
2017-10-25 18:07:27 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 43e0fd318e Bug fix/fixes 1909 (#3285)
* remove TRI_ERROR_INTERNAL usage

* prevent assertion failure from occurring

* ensure downwards-compatibility for explain handler

* allow schmutz to drop system collections as well - needed for arangojs tests
2017-09-19 21:56:32 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Kaveh Vahedipour 15a87b2982 removed database loop around rebalancing of shards; thus rebalance shards now only affects current database (#3091)
This is UI only
2017-08-21 17:13:27 +02:00
Jan 79d76c89fb Bug fix/use count for checksum (#3033)
* Count as checksum

* Make readLockId optional as well so upgrades are possible

* fix option name in startup script

* fix some replication issues with RocksDB engine
2017-08-15 11:24:54 +02:00
m0ppers 9a0bc716d0 Do not allow replication to create/drop collections (#2898)
In the cluster case the only one who is allowed to do this is the schmutz
2017-07-28 14:24:02 +02:00
m0ppers 589ffd5c59 Feature/improve logging (#2881)
* Improve logging in various places

* Fix jslint
2017-07-28 09:23:18 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Kaveh Vahedipour 231a360b3b fixes for secondaries 2017-07-11 14:05:51 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Alan Plum 5d017d733c Improve Foxx cluster resilience (#2479)
* Improve Foxx cluster resilience

Fixes #2083
Fixes #2384
Fixes #2408
Addresses #1892

* Port old Foxx API

* Implement single-file services

* Add console.errorStack/warnStack/infoStack helpers

* Simplify serviceInfo validation

* Extract github/upload logic into Aardvark and old FM API
* Move generator logic into Aardvark
* Move zip/js buffer logic into FM core

* Add Foxxmanager tests

* Send empty response when no README

* Disambiguate script arg format

Historically we allow passing an array of positional arguments or an arbitrary first argument.
This is surprising behaviour, so we should just always treat the value as a first argument.

* Rebuild bundle in development mode

* Nicer HTTP docs formatting

* Create Foxx HTTP docs

* Simplify service upload handling

* Remove inline swagger docs

* Implement public download route

* Consistency

* Rebuild aardvark

* Move bundle route into /_api/foxx/_local

* Rebuild Swagger API docs

* Add changes to CHANGELOG

* More docs
2017-05-16 17:41:29 +02:00
jsteemann 134b4f80ae do not care about collection attributes that are not used in an engine 2017-05-04 13:11:21 +02:00
Kaveh Vahedipour 1f81ce28b0 merge in cpp & js from 3.1.18 yet to do tests 2017-04-21 15:41:05 +02:00
Kaveh Vahedipour 001c2b4092 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-04-11 11:46:40 +02:00
Kaveh Vahedipour 28fbf8ec12 sensitivityEstimates do not belong in Current 2017-04-11 11:46:20 +02:00
Max Neunhoeffer 53b3a73486 Add a timeout option to <collection>.drop().
This is in particular used to allow the Schmutz to make progress
in the case of a long running transaction.
2017-04-07 15:52:19 +02:00
Max Neunhoeffer 4017bfb296 Revise /_api/cluster/endpoints API and Documentation. 2017-03-16 11:33:46 +01:00
Max Neunhoeffer c8a205b1aa New /_api/cluster/endpoints.
Also fix documentation (and deprecate) /_api/endpoint.
2017-03-15 13:33:50 +01:00
Andreas Streichardt 488b71cf54 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-02-10 19:40:51 +01:00
Andreas Streichardt bc1df86ddd remove lock queries 2017-02-10 19:34:29 +01:00
Max Neunhoeffer f72af6a06a Increase timeout for waiting for sync repl to settle.
This is mainly for the case of the parallel unittests, where the heavy
load leads to delays.
2017-02-09 22:06:39 +01:00
Max Neunhoeffer 20e2128a44 Fix shardDistribution in case of leader change. 2017-02-06 09:45:00 +01:00
Max Neunhoeffer 72fa45cc94 Port better reporting of longish Schmutz behaviour to devel. 2017-02-03 10:47:13 +01:00
Max Neunhoeffer 7c0575b27d Fix timeout in cluster operation and wal flush bugs. 2017-02-03 10:42:06 +01:00
Max Neunhoeffer 105389702a Lower log level of sync replication events. 2017-02-01 16:04:04 +01:00
Max Neunhoeffer a5b4eb3c3d Lower log level for local shard operations to debug. 2017-01-31 09:31:52 +01:00
Max Neunhoeffer f09ecd21b2 Logging level for background job debugging down to DEBUG. 2017-01-29 23:05:33 +01:00
Max Neunhoeffer 6993ca32c7 Add more debugging output on log level info around Schmutz script. 2017-01-28 20:28:58 +01:00
Andreas Streichardt 2476cf7146 Fix drop and add reporting in agency current 2017-01-26 14:00:56 +01:00