mirror of https://gitee.com/bigwinds/arangodb
fixed incremental sync method
This commit is contained in:
Jan Steemann
parent
e08d840b47
commit
75ccd7e2cc
|
|
@ -1,8 +1,8 @@
|
|||
!CHAPTER Example Setup
|
||||
|
||||
Setting up a working master-slave replication requires two ArangoDB instances:
|
||||
* _master_: this is the instance that all data-modification operations should be directed to
|
||||
* _slave_: on this instance, we'll start a replication applier, and this will fetch data from the
|
||||
* **master**: this is the instance that all data-modification operations should be directed to
|
||||
* **slave**: on this instance, we'll start a replication applier, and this will fetch data from the
|
||||
master database's write-ahead log and apply its operations locally
|
||||
|
||||
For the following example setup, we'll use the instance *tcp://master.domain.org:8529* as the
|
||||
|
|
@ -11,18 +11,18 @@ master, and the instance *tcp://slave.domain.org:8530* as a slave.
|
|||
The goal is to have all data from the database *_system* on master *tcp://master.domain.org:8529*
|
||||
be replicated to the database *_system* on the slave *tcp://slave.domain.org:8530*.
|
||||
|
||||
On the *master*, nothing special needs to be done, as all write operations will automatically be
|
||||
On the **master**, nothing special needs to be done, as all write operations will automatically be
|
||||
logged in the master's write-ahead log.
|
||||
|
||||
To start replication, make sure there currently is no replication applier running in the slave's
|
||||
*_system* database:
|
||||
To start replication on the **slave**, make sure there currently is no replication applier running
|
||||
in the slave's *_system* database:
|
||||
|
||||
```js
|
||||
db._useDatabase("_system");
|
||||
require("org/arangodb/replication").applier.stop();
|
||||
```
|
||||
|
||||
The stop operation will terminate any replication activity in the _system database on the slave.
|
||||
The *stop* operation will terminate any replication activity in the _system database on the slave.
|
||||
|
||||
After that, do an initial sync of the slave with data from the master. Execute the following
|
||||
commands on the slave:
|
||||
|
|
@ -30,7 +30,7 @@ commands on the slave:
|
|||
```js
|
||||
db._useDatabase("_system");
|
||||
require("org/arangodb/replication").sync({
|
||||
endpoint: "tcp://master.example.org:8529",
|
||||
endpoint: "tcp://master.domain.org:8529",
|
||||
username: "myuser",
|
||||
password: "mypasswd"
|
||||
});
|
||||
|
|
@ -39,7 +39,7 @@ require("org/arangodb/replication").sync({
|
|||
Username and password only need to be specified when the master server requires authentication.
|
||||
|
||||
**Warning**: The sync command will replace data in the slave database with data from the master
|
||||
database! Only execute the commands if you have verified you are on the correct server, in the
|
||||
database! Only execute these commands if you have verified you are on the correct server, in the
|
||||
correct database!
|
||||
|
||||
The sync operation will return an attribute named *lastLogTick* which we'll need to note. The
|
||||
|
|
@ -59,6 +59,22 @@ slave gets shut down. When the slave server gets restarted, replication will be
|
|||
To change this, we first need to configure the slave's replication applier and set its
|
||||
*autoStart* attribute.
|
||||
|
||||
Here's the command to configure the replication applier with several options:
|
||||
|
||||
```js
|
||||
db._useDatabase("_system");
|
||||
require("org/arangodb/replication").applier.properties({
|
||||
endpoint: "tcp://master.domain.org:8529",
|
||||
username: "myuser",
|
||||
password: "mypasswd",
|
||||
autoStart: true,
|
||||
adaptivePolling: true,
|
||||
includeSystem: false,
|
||||
requireFromPresent: false,
|
||||
verbose: false
|
||||
});
|
||||
```
|
||||
|
||||
An important consideration for replication is whether data from system collections (such as
|
||||
*_graphs* or *_users*) should be applied. The *includeSystem* option controls that. If set to
|
||||
*true*, changes in system collections will be replicated. Otherwise, they will not be replicated.
|
||||
|
|
@ -75,22 +91,6 @@ or resumes replication is still present on the master. If not, then there would
|
|||
*requireFromPresent* is *true*, the replication applier will abort with an appropriate error message.
|
||||
If set to *false*, then the replication applier will still start, and ignore the data loss.
|
||||
|
||||
Here's the command to configure the replication applier with several options:
|
||||
|
||||
```js
|
||||
db._useDatabase("_system");
|
||||
require("org/arangodb/replication").applier.properties({
|
||||
endpoint: "tcp://master.example.org:8529",
|
||||
username: "myuser",
|
||||
password: "mypasswd",
|
||||
autoStart: true,
|
||||
adaptivePolling: true,
|
||||
includeSystem: false,
|
||||
requireFromPresent: false,
|
||||
verbose: false
|
||||
});
|
||||
```
|
||||
|
||||
Now it's time to start the replication applier on the slave using the last log tick we got
|
||||
before:
|
||||
|
||||
|
|
@ -120,5 +120,6 @@ have been replicated fully.
|
|||
Note that while a slave has only partly executed a transaction from the master, it might keep
|
||||
a write lock on the collections involved in the transaction.
|
||||
|
||||
You may also want to check the master and slave states via the HTTP APIs (see [HTTP Interface for Replication](../HttpReplications/README.md)).
|
||||
You may also want to check the master and slave states via the HTTP APIs
|
||||
(see [HTTP Interface for Replication](../HttpReplications/README.md)).
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,37 @@
|
|||
!CHAPTER Syncing Collections
|
||||
|
||||
In order to synchronize data for a single collection from a master to a slave instance, there
|
||||
is the *syncCollection* function:
|
||||
|
||||
It will fetch all documents of the specified collection from the master database and store
|
||||
them in the local instance. After the synchronization, the collection data on the slave will be
|
||||
identical to the data on the master, provided no further data changes happen on the master.
|
||||
Any data changes that are performed on the master after the synchronization was started will
|
||||
not be captured by *syncCollection*, but need to be replicated using the regular replication
|
||||
applier mechanism.
|
||||
|
||||
For the following example setup, we'll use the instance *tcp://master.domain.org:8529* as the
|
||||
master, and the instance *tcp://slave.domain.org:8530* as a slave.
|
||||
|
||||
The goal is to have all data from the collection *test* in database *_system* on master
|
||||
*tcp://master.domain.org:8529* be replicated to the collection *test* in database *_system* on
|
||||
the slave *tcp://slave.domain.org:8530*.
|
||||
|
||||
On the **master**, the collection *test* needs to be present in the *_system* database, with
|
||||
any data in it.
|
||||
|
||||
To transfer this collection to the **slave**, issue the following commands there:
|
||||
|
||||
```js
|
||||
db._useDatabase("_system");
|
||||
require("org/arangodb/replication").syncCollection("test", {
|
||||
endpoint: "tcp://master.domain.org:8529",
|
||||
username: "myuser",
|
||||
password: "mypasswd"
|
||||
});
|
||||
```
|
||||
|
||||
**Warning**: The syncCollection command will replace the collection's data in the slave database
|
||||
with data from the master database! Only execute these commands if you have verified you are on
|
||||
the correct server, in the correct database!
|
||||
|
||||
|
|
@ -135,6 +135,7 @@
|
|||
* [Replication](Replication/README.md)
|
||||
* [Components](Replication/Components.md)
|
||||
* [Example Setup](Replication/ExampleSetup.md)
|
||||
* [Syncing Collections](Replication/SyncingCollections.md)
|
||||
* [Replication Limitations](Replication/Limitations.md)
|
||||
* [Replication Overhead](Replication/Overhead.md)
|
||||
* [Sharding](Sharding/README.md)
|
||||
|
|
|
|||
|
|
@ -172,9 +172,17 @@ int InitialSyncer::run (string& errorMsg,
|
|||
return TRI_ERROR_INTERNAL;
|
||||
}
|
||||
|
||||
int res = _vocbase->_replicationApplier->preventStart();
|
||||
|
||||
if (res != TRI_ERROR_NO_ERROR) {
|
||||
return res;
|
||||
}
|
||||
|
||||
TRI_DEFER(_vocbase->_replicationApplier->allowStart());
|
||||
|
||||
setProgress("fetching master state");
|
||||
|
||||
int res = getMasterState(errorMsg);
|
||||
res = getMasterState(errorMsg);
|
||||
|
||||
if (res != TRI_ERROR_NO_ERROR) {
|
||||
return res;
|
||||
|
|
@ -186,7 +194,6 @@ int InitialSyncer::run (string& errorMsg,
|
|||
return res;
|
||||
}
|
||||
|
||||
|
||||
string url = BaseUrl + "/inventory?serverId=" + _localServerIdString;
|
||||
if (_includeSystem) {
|
||||
url += "&includeSystem=true";
|
||||
|
|
@ -636,9 +643,13 @@ int InitialSyncer::handleCollectionSync (std::string const& cid,
|
|||
string& errorMsg) {
|
||||
|
||||
string const baseUrl = BaseUrl + "/keys";
|
||||
string url = baseUrl + "?collection=" + cid + "&to=" + std::to_string(maxTick);
|
||||
|
||||
std::string progress = "fetching collection keys from " + url;
|
||||
setProgress(progress);
|
||||
|
||||
std::unique_ptr<SimpleHttpResult> response(_client->request(HttpRequest::HTTP_REQUEST_POST,
|
||||
baseUrl + "?collection=" + cid + "&to=" + std::to_string(maxTick),
|
||||
url,
|
||||
nullptr,
|
||||
0));
|
||||
|
||||
|
|
@ -718,9 +729,13 @@ int InitialSyncer::handleCollectionSync (std::string const& cid,
|
|||
}
|
||||
|
||||
{
|
||||
url = baseUrl + "/" + id;
|
||||
string progress = "deleting remote collection keys object from " + url;
|
||||
setProgress(progress);
|
||||
|
||||
// now delete the keys we ordered
|
||||
std::unique_ptr<SimpleHttpResult> response(_client->request(HttpRequest::HTTP_REQUEST_DELETE,
|
||||
baseUrl + "/" + id,
|
||||
url,
|
||||
nullptr,
|
||||
0));
|
||||
|
||||
|
|
@ -750,6 +765,9 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
auto shaper = trx.documentCollection()->getShaper();
|
||||
|
||||
bool const isEdge = (trx.documentCollection()->_info._type == TRI_COL_TYPE_EDGE);
|
||||
|
||||
string progress = "collecting local keys";
|
||||
setProgress(progress);
|
||||
|
||||
// fetch all local keys from primary index
|
||||
std::vector<TRI_df_marker_t const*> markers;
|
||||
|
|
@ -771,13 +789,11 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
|
||||
void const* marker = ptr->getDataPtr();
|
||||
auto df = static_cast<TRI_df_marker_t const*>(marker);
|
||||
|
||||
if (df->_tick >= maxTick) {
|
||||
continue;
|
||||
}
|
||||
|
||||
markers.emplace_back(df);
|
||||
}
|
||||
|
||||
string progress = "sorting " + std::to_string(markers.size()) + " local key(s)";
|
||||
setProgress(progress);
|
||||
|
||||
// sort all our local keys
|
||||
std::sort(markers.begin(), markers.end(), [] (TRI_df_marker_t const* lhs, TRI_df_marker_t const* rhs) -> bool {
|
||||
|
|
@ -791,9 +807,13 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
|
||||
TRI_voc_tick_t const chunkSize = 5000;
|
||||
string const baseUrl = BaseUrl + "/keys";
|
||||
|
||||
string url = baseUrl + "/" + keysId + "?chunkSize=" + std::to_string(chunkSize);
|
||||
progress = "fetching remote keys chunks from " + url;
|
||||
setProgress(progress);
|
||||
|
||||
std::unique_ptr<SimpleHttpResult> response(_client->request(HttpRequest::HTTP_REQUEST_GET,
|
||||
baseUrl + "/" + keysId + "?chunkSize=" + std::to_string(chunkSize),
|
||||
url,
|
||||
nullptr,
|
||||
0));
|
||||
|
||||
|
|
@ -927,9 +947,13 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
else {
|
||||
// no match
|
||||
// must transfer keys for non-matching range
|
||||
|
||||
std::string url = baseUrl + "/" + keysId + "?type=keys&chunk=" + std::to_string(i) + "&chunkSize=" + std::to_string(chunkSize);
|
||||
progress = "fetching keys from " + url;
|
||||
setProgress(progress);
|
||||
|
||||
std::unique_ptr<SimpleHttpResult> response(_client->request(HttpRequest::HTTP_REQUEST_PUT,
|
||||
baseUrl + "/" + keysId + "?type=keys&chunk=" + std::to_string(i) + "&chunkSize=" + std::to_string(chunkSize),
|
||||
url,
|
||||
nullptr,
|
||||
0));
|
||||
|
||||
|
|
@ -997,8 +1021,12 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
// key
|
||||
auto keyJson = static_cast<TRI_json_t const*>(TRI_AtVector(&chunk->_value._objects, 0));
|
||||
// rid
|
||||
auto ridJson = static_cast<TRI_json_t const*>(TRI_AtVector(&chunk->_value._objects, 1));
|
||||
|
||||
if (markers.empty()) {
|
||||
// no local markers
|
||||
toFetch.emplace_back(i);
|
||||
continue;
|
||||
}
|
||||
|
||||
while (nextStart < markers.size()) {
|
||||
auto df = markers[nextStart];
|
||||
char const* localKey = TRI_EXTRACT_MARKER_KEY(df);
|
||||
|
|
@ -1016,6 +1044,7 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
}
|
||||
}
|
||||
|
||||
auto ridJson = static_cast<TRI_json_t const*>(TRI_AtVector(&chunk->_value._objects, 1));
|
||||
auto mptr = idx->lookupKey(keyJson->_value._string.data);
|
||||
|
||||
if (mptr == nullptr) {
|
||||
|
|
@ -1032,16 +1061,18 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
}
|
||||
|
||||
// calculate next starting point
|
||||
BinarySearch(markers, highJson->_value._string.data, nextStart);
|
||||
while (nextStart < markers.size()) {
|
||||
TRI_ASSERT(nextStart < markers.size());
|
||||
char const* key = TRI_EXTRACT_MARKER_KEY(markers.at(nextStart));
|
||||
int res = strcmp(key, highJson->_value._string.data);
|
||||
if (res <= 0) {
|
||||
++nextStart;
|
||||
}
|
||||
else {
|
||||
break;
|
||||
if (! markers.empty()) {
|
||||
BinarySearch(markers, highJson->_value._string.data, nextStart);
|
||||
while (nextStart < markers.size()) {
|
||||
TRI_ASSERT(nextStart < markers.size());
|
||||
char const* key = TRI_EXTRACT_MARKER_KEY(markers.at(nextStart));
|
||||
int res = strcmp(key, highJson->_value._string.data);
|
||||
if (res <= 0) {
|
||||
++nextStart;
|
||||
}
|
||||
else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1058,10 +1089,14 @@ int InitialSyncer::handleSyncKeys (std::string const& keysId,
|
|||
keysJson.add(triagens::basics::Json(static_cast<double>(it)));
|
||||
}
|
||||
|
||||
std::string url = baseUrl + "/" + keysId + "?type=docs&chunk=" + std::to_string(currentChunkId) + "&chunkSize=" + std::to_string(chunkSize);
|
||||
progress = "fetching documents from " + url;
|
||||
setProgress(progress);
|
||||
|
||||
auto const keyJsonString = triagens::basics::JsonHelper::toString(keysJson.json());
|
||||
|
||||
std::unique_ptr<SimpleHttpResult> response(_client->request(HttpRequest::HTTP_REQUEST_PUT,
|
||||
baseUrl + "/" + keysId + "?type=docs&chunk=" + std::to_string(currentChunkId) + "&chunkSize=" + std::to_string(chunkSize),
|
||||
url,
|
||||
keyJsonString.c_str(),
|
||||
keyJsonString.size()));
|
||||
|
||||
|
|
@ -1395,7 +1430,7 @@ int InitialSyncer::handleCollection (TRI_json_t const* parameters,
|
|||
|
||||
if (n > 0) {
|
||||
string const progress = "creating indexes for " + collectionMsg;
|
||||
setProgress(progress.c_str());
|
||||
setProgress(progress);
|
||||
|
||||
READ_LOCKER(_vocbase->_inventoryLock);
|
||||
|
||||
|
|
|
|||
|
|
@ -3997,9 +3997,10 @@ void RestReplicationHandler::handleCommandMakeSlave () {
|
|||
/// if set to *true*, then an incremental synchronization method will be used
|
||||
/// for synchronizing data in collections. This method is useful when
|
||||
/// collections already exist locally, and only the remaining differences need
|
||||
/// to be transferred from the remote endpoint. The default value is *false*,
|
||||
/// meaning that the complete data from the remote collection will be
|
||||
/// transferred.
|
||||
/// to be transferred from the remote endpoint. In this case, the incremental
|
||||
/// synchronization can be faster than a full synchronization.
|
||||
/// The default value is *false*, meaning that the complete data from the remote
|
||||
/// collection will be transferred.
|
||||
///
|
||||
/// @RESTBODYPARAM{restrictType,string,optional,string}
|
||||
/// an optional string value for collection filtering. When
|
||||
|
|
|
|||
|
|
@ -1112,6 +1112,10 @@ int TRI_replication_applier_t::start (TRI_voc_tick_t initialTick,
|
|||
|
||||
WRITE_LOCKER(_statusLock);
|
||||
|
||||
if (_state._preventStart) {
|
||||
return TRI_ERROR_LOCKED;
|
||||
}
|
||||
|
||||
if (_state._active) {
|
||||
return TRI_ERROR_NO_ERROR;
|
||||
}
|
||||
|
|
@ -1158,6 +1162,52 @@ int TRI_replication_applier_t::start (TRI_voc_tick_t initialTick,
|
|||
return TRI_ERROR_NO_ERROR;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief test if the replication applier is running
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
bool TRI_replication_applier_t::isRunning () const {
|
||||
READ_LOCKER(_statusLock);
|
||||
|
||||
return _state._active;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief block the replication applier from starting
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
int TRI_replication_applier_t::preventStart () {
|
||||
WRITE_LOCKER(_statusLock);
|
||||
|
||||
if (_state._active) {
|
||||
// already running
|
||||
return TRI_ERROR_REPLICATION_RUNNING;
|
||||
}
|
||||
|
||||
if (_state._preventStart) {
|
||||
// someone else requested start prevention
|
||||
return TRI_ERROR_LOCKED;
|
||||
}
|
||||
|
||||
return TRI_ERROR_NO_ERROR;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief unblock the replication applier from starting
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
int TRI_replication_applier_t::allowStart () {
|
||||
WRITE_LOCKER(_statusLock);
|
||||
|
||||
if (! _state._preventStart) {
|
||||
return TRI_ERROR_INTERNAL;
|
||||
}
|
||||
|
||||
_state._preventStart = false;
|
||||
|
||||
return TRI_ERROR_NO_ERROR;
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief stop the replication applier
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
|
|
|||
|
|
@ -99,6 +99,7 @@ struct TRI_replication_applier_state_t {
|
|||
TRI_voc_tick_t _lastAvailableContinuousTick;
|
||||
TRI_voc_tick_t _safeResumeTick;
|
||||
bool _active;
|
||||
bool _preventStart;
|
||||
char* _progressMsg;
|
||||
char _progressTime[24];
|
||||
TRI_server_id_t _serverId;
|
||||
|
|
@ -146,6 +147,24 @@ class TRI_replication_applier_t {
|
|||
return _databaseName.c_str();
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief test if the replication applier is running
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
bool isRunning () const;
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief block the replication applier from starting
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
int preventStart ();
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief unblock the replication applier from starting
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
int allowStart ();
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief start the replication applier
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
|
@ -206,7 +225,7 @@ class TRI_replication_applier_t {
|
|||
|
||||
TRI_server_t* _server;
|
||||
TRI_vocbase_t* _vocbase;
|
||||
triagens::basics::ReadWriteLock _statusLock;
|
||||
mutable triagens::basics::ReadWriteLock _statusLock;
|
||||
std::atomic<bool> _terminateThread;
|
||||
TRI_replication_applier_state_t _state;
|
||||
TRI_replication_applier_configuration_t _configuration;
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@
|
|||
"ERROR_IP_ADDRESS_INVALID" : { "code" : 25, "message" : "IP address is invalid" },
|
||||
"ERROR_LEGEND_NOT_IN_WAL_FILE" : { "code" : 26, "message" : "internal error if a legend for a marker does not yet exist in the same WAL file" },
|
||||
"ERROR_FILE_EXISTS" : { "code" : 27, "message" : "file exists" },
|
||||
"ERROR_LOCKED" : { "code" : 28, "message" : "locked" },
|
||||
"ERROR_HTTP_BAD_PARAMETER" : { "code" : 400, "message" : "bad parameter" },
|
||||
"ERROR_HTTP_UNAUTHORIZED" : { "code" : 401, "message" : "unauthorized" },
|
||||
"ERROR_HTTP_FORBIDDEN" : { "code" : 403, "message" : "forbidden" },
|
||||
|
|
@ -116,9 +117,8 @@
|
|||
"ERROR_REPLICATION_UNEXPECTED_MARKER" : { "code" : 1406, "message" : "unexpected marker" },
|
||||
"ERROR_REPLICATION_INVALID_APPLIER_STATE" : { "code" : 1407, "message" : "invalid applier state" },
|
||||
"ERROR_REPLICATION_UNEXPECTED_TRANSACTION" : { "code" : 1408, "message" : "invalid transaction" },
|
||||
"ERROR_REPLICATION_INVALID_LOGGER_CONFIGURATION" : { "code" : 1409, "message" : "invalid replication logger configuration" },
|
||||
"ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATION" : { "code" : 1410, "message" : "invalid replication applier configuration" },
|
||||
"ERROR_REPLICATION_RUNNING" : { "code" : 1411, "message" : "cannot change applier configuration while running" },
|
||||
"ERROR_REPLICATION_RUNNING" : { "code" : 1411, "message" : "cannot perform operation while applier is running" },
|
||||
"ERROR_REPLICATION_APPLIER_STOPPED" : { "code" : 1412, "message" : "replication stopped" },
|
||||
"ERROR_REPLICATION_NO_START_TICK" : { "code" : 1413, "message" : "no start tick" },
|
||||
"ERROR_REPLICATION_START_TICK_NOT_PRESENT" : { "code" : 1414, "message" : "start tick not present" },
|
||||
|
|
|
|||
|
|
@ -178,6 +178,27 @@ var sync = function (config) {
|
|||
return requestResult;
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief performs a one-time synchronization with a remote endpoint, for
|
||||
/// a single collection
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
var syncCollection = function (collection, config) {
|
||||
var db = internal.db;
|
||||
|
||||
var body = JSON.stringify(config || { });
|
||||
body.restrictType = "include";
|
||||
body.restrictCollections = [ collection ];
|
||||
body.includeSystem = true;
|
||||
body.incremental = true;
|
||||
|
||||
var requestResult = db._connection.PUT("/_api/replication/sync", body);
|
||||
|
||||
arangosh.checkRequestResult(requestResult);
|
||||
|
||||
return requestResult;
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief fetches a server's id
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
|
@ -196,10 +217,11 @@ var serverId = function () {
|
|||
// --SECTION-- module exports
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
exports.logger = logger;
|
||||
exports.applier = applier;
|
||||
exports.sync = sync;
|
||||
exports.serverId = serverId;
|
||||
exports.logger = logger;
|
||||
exports.applier = applier;
|
||||
exports.sync = sync;
|
||||
exports.syncCollection = syncCollection;
|
||||
exports.serverId = serverId;
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// --SECTION-- END-OF-FILE
|
||||
|
|
|
|||
|
|
@ -177,6 +177,27 @@ var sync = function (config) {
|
|||
return requestResult;
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief performs a one-time synchronization with a remote endpoint, for
|
||||
/// a single collection
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
var syncCollection = function (collection, config) {
|
||||
var db = internal.db;
|
||||
|
||||
var body = JSON.stringify(config || { });
|
||||
body.restrictType = "include";
|
||||
body.restrictCollections = [ collection ];
|
||||
body.includeSystem = true;
|
||||
body.incremental = true;
|
||||
|
||||
var requestResult = db._connection.PUT("/_api/replication/sync", body);
|
||||
|
||||
arangosh.checkRequestResult(requestResult);
|
||||
|
||||
return requestResult;
|
||||
};
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief fetches a server's id
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
|
@ -195,10 +216,11 @@ var serverId = function () {
|
|||
// --SECTION-- module exports
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
exports.logger = logger;
|
||||
exports.applier = applier;
|
||||
exports.sync = sync;
|
||||
exports.serverId = serverId;
|
||||
exports.logger = logger;
|
||||
exports.applier = applier;
|
||||
exports.sync = sync;
|
||||
exports.syncCollection = syncCollection;
|
||||
exports.serverId = serverId;
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// --SECTION-- END-OF-FILE
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@
|
|||
"ERROR_IP_ADDRESS_INVALID" : { "code" : 25, "message" : "IP address is invalid" },
|
||||
"ERROR_LEGEND_NOT_IN_WAL_FILE" : { "code" : 26, "message" : "internal error if a legend for a marker does not yet exist in the same WAL file" },
|
||||
"ERROR_FILE_EXISTS" : { "code" : 27, "message" : "file exists" },
|
||||
"ERROR_LOCKED" : { "code" : 28, "message" : "locked" },
|
||||
"ERROR_HTTP_BAD_PARAMETER" : { "code" : 400, "message" : "bad parameter" },
|
||||
"ERROR_HTTP_UNAUTHORIZED" : { "code" : 401, "message" : "unauthorized" },
|
||||
"ERROR_HTTP_FORBIDDEN" : { "code" : 403, "message" : "forbidden" },
|
||||
|
|
@ -116,9 +117,8 @@
|
|||
"ERROR_REPLICATION_UNEXPECTED_MARKER" : { "code" : 1406, "message" : "unexpected marker" },
|
||||
"ERROR_REPLICATION_INVALID_APPLIER_STATE" : { "code" : 1407, "message" : "invalid applier state" },
|
||||
"ERROR_REPLICATION_UNEXPECTED_TRANSACTION" : { "code" : 1408, "message" : "invalid transaction" },
|
||||
"ERROR_REPLICATION_INVALID_LOGGER_CONFIGURATION" : { "code" : 1409, "message" : "invalid replication logger configuration" },
|
||||
"ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATION" : { "code" : 1410, "message" : "invalid replication applier configuration" },
|
||||
"ERROR_REPLICATION_RUNNING" : { "code" : 1411, "message" : "cannot change applier configuration while running" },
|
||||
"ERROR_REPLICATION_RUNNING" : { "code" : 1411, "message" : "cannot perform operation while applier is running" },
|
||||
"ERROR_REPLICATION_APPLIER_STOPPED" : { "code" : 1412, "message" : "replication stopped" },
|
||||
"ERROR_REPLICATION_NO_START_TICK" : { "code" : 1413, "message" : "no start tick" },
|
||||
"ERROR_REPLICATION_START_TICK_NOT_PRESENT" : { "code" : 1414, "message" : "start tick not present" },
|
||||
|
|
|
|||
|
|
@ -120,7 +120,23 @@ applier.properties = function (config) {
|
|||
/// @brief performs a one-time synchronization with a remote endpoint
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
function sync(config) {
|
||||
function sync (config) {
|
||||
return internal.synchronizeReplication(config);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief performs a one-time synchronization with a remote endpoint
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
function syncCollection (collection, config) {
|
||||
config = config || { };
|
||||
config.restrictType = "include";
|
||||
config.restrictCollections = [ collection ];
|
||||
config.includeSystem = true;
|
||||
if (! config.hasOwnProperty('verbose')) {
|
||||
config.verbose = false;
|
||||
}
|
||||
|
||||
return internal.synchronizeReplication(config);
|
||||
}
|
||||
|
||||
|
|
@ -128,7 +144,7 @@ function sync(config) {
|
|||
/// @brief returns the server's id
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
function serverId() {
|
||||
function serverId () {
|
||||
return internal.serverId();
|
||||
}
|
||||
|
||||
|
|
@ -136,10 +152,11 @@ function serverId() {
|
|||
// --SECTION-- module exports
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
exports.logger = logger;
|
||||
exports.applier = applier;
|
||||
exports.sync = sync;
|
||||
exports.serverId = serverId;
|
||||
exports.logger = logger;
|
||||
exports.applier = applier;
|
||||
exports.sync = sync;
|
||||
exports.syncCollection = syncCollection;
|
||||
exports.serverId = serverId;
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// --SECTION-- END-OF-FILE
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ ERROR_LEGEND_INCOMPLETE,24,"internal error if a legend could not be created","Wi
|
|||
ERROR_IP_ADDRESS_INVALID,25,"IP address is invalid","Will be raised when the structure of an IP address is invalid."
|
||||
ERROR_LEGEND_NOT_IN_WAL_FILE,26,"internal error if a legend for a marker does not yet exist in the same WAL file","Will be raised internally, then fixed internally, and never come out to the user."
|
||||
ERROR_FILE_EXISTS,27,"file exists","Will be raised when a file already exists."
|
||||
ERROR_LOCKED,28,"locked","Will be raised when a resource or an operation is locked."
|
||||
|
||||
################################################################################
|
||||
## HTTP standard errors
|
||||
|
|
@ -147,9 +148,8 @@ ERROR_REPLICATION_LOOP,1405,"loop detected","Will be raised when the replication
|
|||
ERROR_REPLICATION_UNEXPECTED_MARKER,1406,"unexpected marker","Will be raised when an unexpected marker is found in the replication log stream."
|
||||
ERROR_REPLICATION_INVALID_APPLIER_STATE,1407,"invalid applier state","Will be raised when an invalid replication applier state file is found."
|
||||
ERROR_REPLICATION_UNEXPECTED_TRANSACTION,1408,"invalid transaction","Will be raised when an unexpected transaction id is found."
|
||||
ERROR_REPLICATION_INVALID_LOGGER_CONFIGURATION,1409,"invalid replication logger configuration","Will be raised when the configuration for the replication logger is invalid."
|
||||
ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATION,1410,"invalid replication applier configuration","Will be raised when the configuration for the replication applier is invalid."
|
||||
ERROR_REPLICATION_RUNNING,1411,"cannot change applier configuration while running","Will be raised when there is an attempt to change the configuration for the replication applier while it is running."
|
||||
ERROR_REPLICATION_RUNNING,1411,"cannot perform operation while applier is running","Will be raised when there is an attempt to perform an operation while the replication applier is running."
|
||||
ERROR_REPLICATION_APPLIER_STOPPED,1412,"replication stopped","Special error code used to indicate the replication applier was stopped by a user."
|
||||
ERROR_REPLICATION_NO_START_TICK,1413,"no start tick","Will be raised when the replication applier is started without a known start tick value."
|
||||
ERROR_REPLICATION_START_TICK_NOT_PRESENT,1414,"start tick not present","Will be raised when the replication applier fetches data using a start tick, but that start tick is not present on the logger server anymore."
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ void TRI_InitializeErrorMessages () {
|
|||
REG_ERROR(ERROR_IP_ADDRESS_INVALID, "IP address is invalid");
|
||||
REG_ERROR(ERROR_LEGEND_NOT_IN_WAL_FILE, "internal error if a legend for a marker does not yet exist in the same WAL file");
|
||||
REG_ERROR(ERROR_FILE_EXISTS, "file exists");
|
||||
REG_ERROR(ERROR_LOCKED, "locked");
|
||||
REG_ERROR(ERROR_HTTP_BAD_PARAMETER, "bad parameter");
|
||||
REG_ERROR(ERROR_HTTP_UNAUTHORIZED, "unauthorized");
|
||||
REG_ERROR(ERROR_HTTP_FORBIDDEN, "forbidden");
|
||||
|
|
@ -112,9 +113,8 @@ void TRI_InitializeErrorMessages () {
|
|||
REG_ERROR(ERROR_REPLICATION_UNEXPECTED_MARKER, "unexpected marker");
|
||||
REG_ERROR(ERROR_REPLICATION_INVALID_APPLIER_STATE, "invalid applier state");
|
||||
REG_ERROR(ERROR_REPLICATION_UNEXPECTED_TRANSACTION, "invalid transaction");
|
||||
REG_ERROR(ERROR_REPLICATION_INVALID_LOGGER_CONFIGURATION, "invalid replication logger configuration");
|
||||
REG_ERROR(ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATION, "invalid replication applier configuration");
|
||||
REG_ERROR(ERROR_REPLICATION_RUNNING, "cannot change applier configuration while running");
|
||||
REG_ERROR(ERROR_REPLICATION_RUNNING, "cannot perform operation while applier is running");
|
||||
REG_ERROR(ERROR_REPLICATION_APPLIER_STOPPED, "replication stopped");
|
||||
REG_ERROR(ERROR_REPLICATION_NO_START_TICK, "no start tick");
|
||||
REG_ERROR(ERROR_REPLICATION_START_TICK_NOT_PRESENT, "start tick not present");
|
||||
|
|
|
|||
|
|
@ -66,6 +66,8 @@
|
|||
/// the user.
|
||||
/// - 27: @LIT{file exists}
|
||||
/// Will be raised when a file already exists.
|
||||
/// - 28: @LIT{locked}
|
||||
/// Will be raised when a resource or an operation is locked.
|
||||
/// - 400: @LIT{bad parameter}
|
||||
/// Will be raised when the HTTP request does not fulfill the requirements.
|
||||
/// - 401: @LIT{unauthorized}
|
||||
|
|
@ -255,15 +257,12 @@
|
|||
/// Will be raised when an invalid replication applier state file is found.
|
||||
/// - 1408: @LIT{invalid transaction}
|
||||
/// Will be raised when an unexpected transaction id is found.
|
||||
/// - 1409: @LIT{invalid replication logger configuration}
|
||||
/// Will be raised when the configuration for the replication logger is
|
||||
/// invalid.
|
||||
/// - 1410: @LIT{invalid replication applier configuration}
|
||||
/// Will be raised when the configuration for the replication applier is
|
||||
/// invalid.
|
||||
/// - 1411: @LIT{cannot change applier configuration while running}
|
||||
/// Will be raised when there is an attempt to change the configuration for
|
||||
/// the replication applier while it is running.
|
||||
/// - 1411: @LIT{cannot perform operation while applier is running}
|
||||
/// Will be raised when there is an attempt to perform an operation while the
|
||||
/// replication applier is running.
|
||||
/// - 1412: @LIT{replication stopped}
|
||||
/// Special error code used to indicate the replication applier was stopped
|
||||
/// by a user.
|
||||
|
|
@ -957,6 +956,16 @@ void TRI_InitializeErrorMessages ();
|
|||
|
||||
#define TRI_ERROR_FILE_EXISTS (27)
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief 28: ERROR_LOCKED
|
||||
///
|
||||
/// locked
|
||||
///
|
||||
/// Will be raised when a resource or an operation is locked.
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#define TRI_ERROR_LOCKED (28)
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief 400: ERROR_HTTP_BAD_PARAMETER
|
||||
///
|
||||
|
|
@ -1765,16 +1774,6 @@ void TRI_InitializeErrorMessages ();
|
|||
|
||||
#define TRI_ERROR_REPLICATION_UNEXPECTED_TRANSACTION (1408)
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief 1409: ERROR_REPLICATION_INVALID_LOGGER_CONFIGURATION
|
||||
///
|
||||
/// invalid replication logger configuration
|
||||
///
|
||||
/// Will be raised when the configuration for the replication logger is invalid.
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#define TRI_ERROR_REPLICATION_INVALID_LOGGER_CONFIGURATION (1409)
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief 1410: ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATION
|
||||
///
|
||||
|
|
@ -1789,10 +1788,10 @@ void TRI_InitializeErrorMessages ();
|
|||
////////////////////////////////////////////////////////////////////////////////
|
||||
/// @brief 1411: ERROR_REPLICATION_RUNNING
|
||||
///
|
||||
/// cannot change applier configuration while running
|
||||
/// cannot perform operation while applier is running
|
||||
///
|
||||
/// Will be raised when there is an attempt to change the configuration for the
|
||||
/// replication applier while it is running.
|
||||
/// Will be raised when there is an attempt to perform an operation while the
|
||||
/// replication applier is running.
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#define TRI_ERROR_REPLICATION_RUNNING (1411)
|
||||
|
|
|
|||
Loading…
Reference in New Issue