Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 

* 'devel' of https://github.com/arangodb/arangodb:
  Add bye bye messages
  update rocksdb logging to error level
  protect vector of isolate data with a mutex
  avoid nullptr in Inception
  remove satelites
This commit is contained in:
Jan Christoph Uhde 2017-05-12 21:39:22 +02:00
parent 4a85f93814 365de45f07
commit 235a82a7dc
8 changed files with 111 additions and 116 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
Documentation/Books/Manual/Administration/Replication/README.md
View File

@ -13,14 +13,6 @@ Synchronous replication only works in in a cluster and is typically used for mis
Synchronous replication is organized in a way that every shard has a leader and r-1 followers. The number of followers can be controlled using the `replicationFactor` whenever you create a collection, the `replicationFactor` is the total number of copies being kept, that is, it is one plus the number of followers.
### Satellite collections
Satellite collections are synchronously replicated collections having a dynamic replicationFactor.
They will replicate all data to all database servers allowing the database servers to join data
locally instead of doing heavy network operations.
Satellite collections are an enterprise only feature.
### Asynchronous replication
In ArangoDB any write operation will be logged to the write-ahead log. When using Asynchronous replication slaves will connect to a master and apply all the events from the log in the same order locally. After that, they will have the same state of data as the master database.

6
Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.md
View File

@ -140,12 +140,6 @@ to the [naming conventions](../NamingConventions/README.md).
servers holding copies take over, usually without an error being
reported.
When using the *Enterprise* version of ArangoDB the replicationFactor
may be set to "satellite" making the collection locally joinable
on every database server. This reduces the number of network hops
dramatically when using joins in AQL at the costs of reduced write
performance on these collections.
`db._create(collection-name, properties, type)`
Specifies the optional *type* of the collection, it can either be *document*

6
Documentation/DocuBlocks/Rest/Cursors/JSF_post_api_cursor.md
View File

@ -73,12 +73,6 @@ If set to *true*, then the additional query profiling information will be return
in the sub-attribute *profile* of the *extra* return attribute if the query result
is not served from the query cache.
@RESTSTRUCT{satelliteSyncWait,JSF_post_api_cursor_opts,boolean,optional,}
This *enterprise* parameter allows to configure how long a DBServer will have time
to bring the satellite collections involved in the query into sync.
The default value is *60.0* (seconds). When the max time has been reached the query
will be stopped.
@RESTDESCRIPTION
The query details include the query string plus optional query options and
bind parameters. These values need to be passed in a JSON representation in

190
arangod/Agency/Inception.cpp
View File

@ -101,10 +101,12 @@ void Inception::gossip() {
std::make_unique<std::unordered_map<std::string, std::string>>();
LOG_TOPIC(DEBUG, Logger::AGENCY) << "Sending gossip message: "
<< out->toJson() << " to peer " << clientid;
cc->asyncRequest(
clientid, 1, p, rest::RequestType::POST, path,
std::make_shared<std::string>(out->toJson()), hf,
std::make_shared<GossipCallback>(_agent, version), 1.0, true, 0.5);
if (cc != nullptr) {
cc->asyncRequest(
clientid, 1, p, rest::RequestType::POST, path,
std::make_shared<std::string>(out->toJson()), hf,
std::make_shared<GossipCallback>(_agent, version), 1.0, true, 0.5);
}
}
}
@ -124,10 +126,12 @@ void Inception::gossip() {
std::make_unique<std::unordered_map<std::string, std::string>>();
LOG_TOPIC(DEBUG, Logger::AGENCY) << "Sending gossip message: "
<< out->toJson() << " to pool member " << clientid;
cc->asyncRequest(
clientid, 1, pair.second, rest::RequestType::POST, path,
std::make_shared<std::string>(out->toJson()), hf,
std::make_shared<GossipCallback>(_agent, version), 1.0, true, 0.5);
if (cc != nullptr) {
cc->asyncRequest(
clientid, 1, pair.second, rest::RequestType::POST, path,
std::make_shared<std::string>(out->toJson()), hf,
std::make_shared<GossipCallback>(_agent, version), 1.0, true, 0.5);
}
}
}
@ -213,17 +217,19 @@ bool Inception::restartingActiveAgent() {
std::vector<std::string> informed;
for (auto& p : gp) {
auto comres = cc->syncRequest(
clientId, 1, p, rest::RequestType::POST, path, greetstr,
std::unordered_map<std::string, std::string>(), 2.0);
if (comres->status == CL_COMM_SENT) {
auto const theirConfigVP = comres->result->getBodyVelocyPack();
auto const& theirConfig = theirConfigVP->slice();
auto const& tcc = theirConfig.get("configuration");
auto const& theirId = tcc.get("id").copyString();
_agent->updatePeerEndpoint(theirId, p);
informed.push_back(p);
if (cc != nullptr) {
auto comres = cc->syncRequest(
clientId, 1, p, rest::RequestType::POST, path, greetstr,
std::unordered_map<std::string, std::string>(), 2.0);
if (comres->status == CL_COMM_SENT) {
auto const theirConfigVP = comres->result->getBodyVelocyPack();
auto const& theirConfig = theirConfigVP->slice();
auto const& tcc = theirConfig.get("configuration");
auto const& theirId = tcc.get("id").copyString();
_agent->updatePeerEndpoint(theirId, p);
informed.push_back(p);
}
}
}
@ -236,82 +242,84 @@ bool Inception::restartingActiveAgent() {
for (auto& p : pool) {
if (p.first != myConfig.id() && p.first != "") {
auto comres = cc->syncRequest(
clientId, 1, p.second, rest::RequestType::POST, path, greetstr,
std::unordered_map<std::string, std::string>(), 2.0);
if (comres->status == CL_COMM_SENT) {
try {
auto const theirConfigVP = comres->result->getBodyVelocyPack();
auto const& theirConfig = theirConfigVP->slice();
auto const& theirLeaderId = theirConfig.get("leaderId").copyString();
auto const& tcc = theirConfig.get("configuration");
auto const& theirId = tcc.get("id").copyString();
// Found RAFT with leader
if (!theirLeaderId.empty()) {
LOG_TOPIC(INFO, Logger::AGENCY) <<
"Found active RAFTing agency lead by " << theirLeaderId <<
". Finishing startup sequence.";
auto const theirLeaderEp =
tcc.get(
std::vector<std::string>({"pool", theirLeaderId})).copyString();
// Contact leader to update endpoint
if (theirLeaderId != theirId) {
comres = cc->syncRequest(
clientId, 1, theirLeaderEp, rest::RequestType::POST, path,
greetstr, std::unordered_map<std::string, std::string>(), 2.0);
// Failed to contact leader move on until we do. This way at
// least we inform everybody individually of the news.
if (comres->status != CL_COMM_SENT) {
continue;
if (cc != nullptr) {
auto comres = cc->syncRequest(
clientId, 1, p.second, rest::RequestType::POST, path, greetstr,
std::unordered_map<std::string, std::string>(), 2.0);
if (comres->status == CL_COMM_SENT) {
try {
auto const theirConfigVP = comres->result->getBodyVelocyPack();
auto const& theirConfig = theirConfigVP->slice();
auto const& theirLeaderId = theirConfig.get("leaderId").copyString();
auto const& tcc = theirConfig.get("configuration");
auto const& theirId = tcc.get("id").copyString();
// Found RAFT with leader
if (!theirLeaderId.empty()) {
LOG_TOPIC(INFO, Logger::AGENCY) <<
"Found active RAFTing agency lead by " << theirLeaderId <<
". Finishing startup sequence.";
auto const theirLeaderEp =
tcc.get(
std::vector<std::string>({"pool", theirLeaderId})).copyString();
// Contact leader to update endpoint
if (theirLeaderId != theirId) {
comres = cc->syncRequest(
clientId, 1, theirLeaderEp, rest::RequestType::POST, path,
greetstr, std::unordered_map<std::string, std::string>(), 2.0);
// Failed to contact leader move on until we do. This way at
// least we inform everybody individually of the news.
if (comres->status != CL_COMM_SENT) {
continue;
}
}
auto agency = std::make_shared<Builder>();
agency->openObject();
agency->add("term", theirConfig.get("term"));
agency->add("id", VPackValue(theirLeaderId));
agency->add("active", tcc.get("active"));
agency->add("pool", tcc.get("pool"));
agency->add("min ping", tcc.get("min ping"));
agency->add("max ping", tcc.get("max ping"));
agency->close();
_agent->notify(agency);
return true;
}
auto const theirActive = tcc.get("active").toJson();
auto const myActive = myConfig.activeToBuilder()->toJson();
auto i = std::find(active.begin(),active.end(),p.first);
if (i != active.end()) {
if (theirActive != myActive) {
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "Assumed active RAFT peer and I disagree on active membership:";
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "Their active list is " << theirActive;
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "My active list is " << myActive;
FATAL_ERROR_EXIT();
return false;
} else {
*i = "";
}
}
auto agency = std::make_shared<Builder>();
agency->openObject();
agency->add("term", theirConfig.get("term"));
agency->add("id", VPackValue(theirLeaderId));
agency->add("active", tcc.get("active"));
agency->add("pool", tcc.get("pool"));
agency->add("min ping", tcc.get("min ping"));
agency->add("max ping", tcc.get("max ping"));
agency->close();
_agent->notify(agency);
return true;
} catch (std::exception const& e) {
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "Assumed active RAFT peer has no active agency list: "
<< e.what() << "Administrative intervention needed.";
FATAL_ERROR_EXIT();
return false;
}
auto const theirActive = tcc.get("active").toJson();
auto const myActive = myConfig.activeToBuilder()->toJson();
auto i = std::find(active.begin(),active.end(),p.first);
if (i != active.end()) {
if (theirActive != myActive) {
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "Assumed active RAFT peer and I disagree on active membership:";
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "Their active list is " << theirActive;
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "My active list is " << myActive;
FATAL_ERROR_EXIT();
return false;
} else {
*i = "";
}
}
} catch (std::exception const& e) {
LOG_TOPIC(FATAL, Logger::AGENCY)
<< "Assumed active RAFT peer has no active agency list: "
<< e.what() << "Administrative intervention needed.";
FATAL_ERROR_EXIT();
return false;
}
}
}
}
}
}

2
arangod/RocksDBEngine/RocksDBEngine.cpp
View File

@ -220,7 +220,7 @@ void RocksDBEngine::start() {
// intentionally set the RocksDB logger to warning because it will
// log lots of things otherwise
_options.info_log_level = rocksdb::InfoLogLevel::WARN_LEVEL;
_options.info_log_level = rocksdb::InfoLogLevel::ERROR_LEVEL;
auto logger = std::make_shared<RocksDBLogger>(_options.info_log_level);
_options.info_log = logger;
logger->disable();

4
lib/ApplicationFeatures/V8PlatformFeature.cpp
View File

@ -22,6 +22,7 @@
#include "ApplicationFeatures/V8PlatformFeature.h"
#include "Basics/MutexLocker.h"
#include "Basics/StringUtils.h"
#include "Logger/Logger.h"
#include "ProgramOptions/ProgramOptions.h"
@ -175,7 +176,8 @@ v8::Isolate* V8PlatformFeature::createIsolate() {
isolate->AddGCEpilogueCallback(gcEpilogueCallback);
auto data = std::make_unique<IsolateData>();
MUTEX_LOCKER(guard, _lock);
_isolateData.emplace_back(std::move(data));
isolate->SetData(V8_INFO, _isolateData.back().get());

3
lib/ApplicationFeatures/V8PlatformFeature.h
View File

@ -24,6 +24,7 @@
#define ARANGODB_APPLICATION_FEATURES_V8PLATFORM_FEATURE_H 1
#include "ApplicationFeatures/ApplicationFeature.h"
#include "Basics/Mutex.h"
#include <libplatform/libplatform.h>
#include <v8.h>
@ -78,6 +79,8 @@ class V8PlatformFeature final
std::unique_ptr<v8::Platform> _platform;
std::unique_ptr<v8::ArrayBuffer::Allocator> _allocator;
std::string _v8CombinedOptions;
arangodb::Mutex _lock; // to protect vector _isolateData
std::vector<std::unique_ptr<IsolateData>> _isolateData;
};
}

8
lib/Basics/messages.h
View File

@ -44,6 +44,7 @@
#define TRI_UNICODE_LRM "\xE2\x80\x8E"
#define TRI_UNICODE_RLM "\xE2\x80\x8F"
#define TRI_BYE_MESSAGE_CH "Uf wiederluege!"
#define TRI_BYE_MESSAGE_CN "\xE5\x86\x8D\xE8\xA7\x81\xEF\xBC\x81"
#define TRI_BYE_MESSAGE_CZ "Na shledanou!"
#define TRI_BYE_MESSAGE_DE "Auf Wiedersehen!"
#define TRI_BYE_MESSAGE_EN "Bye Bye!"
@ -57,6 +58,7 @@
//#define TRI_BYE_MESSAGE_IL "\xd7\xaa\xd7\x95\xd7\x90\xd7\xa8\xd7\xaa\xd7\x94\xd7\x9c\x21"
#define TRI_BYE_MESSAGE_IT "Arrivederci!"
#define TRI_BYE_MESSAGE_JP "\xe3\x81\x95\xe3\x82\x88\xe3\x81\x86\xe3\x81\xaa\xe3\x82\x89"
#define TRI_BYE_MESSAGE_LV "\x55\x7a\x20\x72\x65\x64\x7a\xC4\x93\xC5\xA1\x61\x6e\x6f\x73\x21"
#define TRI_BYE_MESSAGE_NL "Tot ziens!"
#define TRI_BYE_MESSAGE_RU "\xd0\x94\xd0\xbe\x20\xd1\x81\xd0\xb2\xd0\xb8\xd0\xb4\xd0\xb0\xd0\xbd\xd0\xb8\xd1\x8f\x21"
#define TRI_BYE_MESSAGE_SV "\x41\x64\x6a\xc3\xb6\x21"
@ -64,11 +66,11 @@
#define TRI_BYE_MESSAGE_FA "\xd8\xae\xd8\xaf\xd8\xa7\xd8\xad\xd8\xa7\xd9\x81\xd8\xb8\x21"
#define TRI_BYE_MESSAGE \
TRI_BYE_MESSAGE_CH \
TRI_BYE_MESSAGE_CH " " TRI_BYE_MESSAGE_CN \
" " TRI_BYE_MESSAGE_CZ " " TRI_BYE_MESSAGE_DE " " TRI_BYE_MESSAGE_EN \
" " TRI_BYE_MESSAGE_EO " " TRI_BYE_MESSAGE_ES " " TRI_BYE_MESSAGE_GR \
"\n" TRI_BYE_MESSAGE_IL " " TRI_BYE_MESSAGE_IT " " TRI_BYE_MESSAGE_NL \
" " TRI_BYE_MESSAGE_SV " " TRI_BYE_MESSAGE_FR " " TRI_BYE_MESSAGE_JP \
" " TRI_BYE_MESSAGE_RU " " TRI_BYE_MESSAGE_PT " " TRI_BYE_MESSAGE_FA
" " TRI_BYE_MESSAGE_RU " " TRI_BYE_MESSAGE_PT " " TRI_BYE_MESSAGE_FA \
" " TRI_BYE_MESSAGE_LV
#endif