Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

fixed merge conflicts

This commit is contained in:
Frank Celler 2013-01-02 17:52:12 +01:00
parent ed19fccc65 f188357f83
commit 503e0ff8fe
109 changed files with 6366 additions and 1275 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -4,6 +4,7 @@ mr-*.h
.dirstamp
*.o
*.a
*~
.libev-build-64
.v8-build-64

5
CHANGELOG
View File

@ -135,6 +135,11 @@ v1.2.alpha (XXXX-XX-XX)
v1.1.2 (XXXX-XX-XX)
-------------------
* backported issue #300: Extend arangoImp to Allow importing resultset-like
(list of documents) formatted files
* fixed issue #332: arangoimp --use-ids parameter seems to have no impact
* fixed issue #336: Collections REST API docs
* fixed issue #335: mmap errors due to wrong memory address calculation

5
Documentation/InstallationManual/Installing.md
View File

@ -7,9 +7,8 @@ Installing ArangoDB {#Installing}
Linux {#InstallingLinux}
========================
You can find binary packages for various Linux distributions here:
@EXTREF{http://www.arangodb.org/download/,http://www.arangodb.org/download/}
You can find binary packages for various Linux distributions
@EXTREF{http://www.arangodb.org/download/,here}.
We provide packages for

1
Documentation/Makefile.files
View File

@ -161,6 +161,7 @@ Doxygen/xml/%.md: Doxygen/xml/%.xml
doxygen-toc:
python @top_srcdir@/Documentation/Scripts/generateTOC.py $(DOXYGEN_TOC) >> Doxygen/toc.doxy.tmp
cmp -s Doxygen/toc.doxy Doxygen/toc.doxy.tmp || mv Doxygen/toc.doxy.tmp Doxygen/toc.doxy
@rm -f Doxygen/toc.doxy.tmp
Doxygen/toc.doxy: doxygen-toc

647
Documentation/UserManual/Actions.md Normal file
View File

@ -0,0 +1,647 @@
Arango Actions {#UserManualActions}
===================================
@NAVIGATE_UserManualActions
@EMBEDTOC{UserManualActionsTOC}
Please note, that user Actions in ArangoDB are still preliminary and details
are subject to change.
Introduction to User Actions {#UserManualActionsIntro}
======================================================
In some ways the communication layer of the ArangoDB server behaves like a Web
server. Unlike a Web server, it normally responds to HTTP requests by delivering
JSON objects. Remember, documents in the database are just JSON objects. So,
most of the time the HTTP response will contain a JSON document from the
database as body. You can extract the documents stored in the database using
HTTP `GET`. You can store documents using HTTP `POST`.
However, there is something more. You can write small sniplets - so called
actions - to extend the database. The idea of actions is that sometimes it is
better to store parts of the business logic within AnrangoDB.
The simplest example is the age of a person. Assume you store information about
people in your database. It is an anti-pattern to store the age, because it
changes every now and then. Therefore, you normally store the birthday and let
the client decide what to do with it. However, if you have many different
clients, it might be easier to enrich the person document with the age using
actions once on the server side.
Or, for instance, if you want to apply some statistics to large data-sets and
you cannot easily express this as query. You can define a action instead of
transferring the whole data to the client and do the computation on the client.
Actions are also useful if you want to restrict and filter data according to
some complex permission system.
The ArangoDB server can deliver all kinds of information, JSON being only one
possible format. You can also generate HTML or images. However, a Web server is
normally better suited for the task as it also implements various caching
strategies, language selection, compression and so on. Having said that, there
are still situations where it might be suitable to use the ArangoDB to deliver
HTML pages - static or dynamic. An simple example is the built-in administration
interface. You can access it using any modern browser and there is no need for a
separate Apache or IIS.
The following sections will explain actions within ArangoDB and show how to
define them. The examples start with delivering static HTML pages - even if this
is not the primary use-case for actions. The later sections will then show you
how to code some pieces of your business logic and return JSON objects.
The interface is loosely modelled after the JavaScript classes for HTTP request
and responses found in node.js and the middleware/routing aspects of connect.js
and express.js.
Note that unlike node.js, ArangoDB is multi-threaded and there is no easy way to
share state between queries inside the JavaScript engine. If such state
information is required, you need to use the database itself.
A Hello World Example {#UserManualActionsHelloWorld}
====================================================
The client API or browser sends a HTTP request to the ArangoDB server and the
server returns a HTTP response to the client. A HTTP request consists of a
method, normally `GET` or `POST` when using a browser, and a request path like
`/hello/world`. For a real Web server there are a zillion of other thing to
consider, we will ignore this for the moment. The HTTP response contains a
content type, describing how to interpret the returned data, and the data
itself.
In the following example, we want to define an action in ArangoDB, so that the
server returns the HTML document
<html>
<body>
Hello World
</body>
</html>
if asked `GET /hello/world`.
The server needs to know what function to call or what document to deliver if it
receives a request. This is called routing. All the routing information of
ArangoDB is stored in a collection `_routing`. Each entry in this collections
describes how to deal with a particular request path.
For the above example, add the following document to the @{_routing} collection:
arangosh> db._routing.save({
........> url: { match: "/hello/world" },
........> content: {
........> contentType: "text/html",
........> body: "<html><body>Hello World</body></html>" }});
In order to activate the new routing, you must either restart the server or call
the internal reload function.
arangosh> require("internal").reloadRouting()
Now use the browser and access
http://localhost:8529/hello/world
You should see the `Hello World` in our browser.
Matching an URL {#UserManualActionsMatches}
===========================================
There are a lot of options for the `url` attribute. If you define different
routing for the same path, then the following simple rule is applied in order to
determine which match wins: If there are two matches, then the more specific
wins. I. e, if there is a wildcard match and an exact match, the exact match is
prefered. If there is a short and a long match, the longer match wins.
Exact Match {#UserManualActionsMatchesExact}
--------------------------------------------
If the definition is
{ url: { match: "/hello/world" } }
then the match must be exact. Only the request for `/hello/world` will match,
everything else, e. g. `/hello/world/my` or `/hello/world2`, will not match.
The following definition is a short-cut for an exact match.
{ url: "/hello/world" }
Prefix Match {#UserManualActionsMatchesPrefix}
----------------------------------------------
If the definition is
{ url: { match: "/hello/world/*" } }
then the match can be a prefix match. The requests for `/hello/world`,
`/hello/world/my`, and `/hello/world/how/are/you` will all match. However
`/hello/world2` does not match. Prefix matches within an URL part,
i. e. `/hello/world*`, are not allowed. The wildcard must occur at the end,
i. e.
/hello/*/world
is also disallowed.
If you define two routes
{ url: { match: "/hello/world/*" } }
{ url: { match: "/hello/world/emil" } }
then the second route will be used for `/hello/world/emil` because it is more
specific.
Parameterized Match {#UserManualActionsMatchesParameterized}
------------------------------------------------------------
A parameterized match is similar to a prefix match, but the parameters are also
allowed inside the URL path.
If the definition is
{ url: { match: "/hello/:name/world" } }
then the URL must have three parts, the first part being `hello` and the third
part `world`. For example, `/hello/emil/world` will match, while
`/hello/emil/meyer/world` will not.
Constraint Match {#UserManualActionsMatchesConstraint}
------------------------------------------------------
A constraint match is similar to a parameterized match, but the parameters can
carry constraints.
If the definition is
{ url: { match: "/hello/:name/world", constraint: { name: "/[a-z]+/" } }
then the URL must have three parts, the first part being `hello` and the third
part `world`. The second part must be all lowercase.
It is possible to use more then one constraint for the same URL part.
{ url: { match: "/hello/:name|:id/world",
constraint: { name: "/[a-z]+/", id: "/[0-9]+/" } }
Optional Match {#UserManualActionsMatchesOptional}
--------------------------------------------------
An optional match is similar to a parameterized match, but the last parameter is
optional.
If the definition is
{ url: { match: "/hello/:name?", constraint: { name: "/[a-z]+/" } }
then the URL `/hello` and `/hello/emil` will match.
If the definitions are
{ url: { match: "/hello/world" } }
{ url: { match: "/hello/:name", constraint: { name: "/[a-z]+/" } }
{ url: { match: "/hello/*" } }
then the URL `/hello/world` will be matched by the first route, because it is
the most specific. The URL `/hello/you` will be matched by the second route,
because it is more specific than the prefix match.
Method Restriction {#UserManualActionsMatchesMethod}
----------------------------------------------------
You can restrict the match to specific methods.
If the definition is
{ url: { match: "/hello/world", methods: [ "post", "put" ] }
then only `POST` and `PUT` requests will match.
More on Matching {#UserManualActionsMatching}
---------------------------------------------
Remember that the more specific match wins.
- A match without parameter or wildcard is more specific than a match with
parameters or wildcard.
- A match with parameter is more specific than a match with a wildcard.
- If there is more than one parameter, specificity is applied from left to
right.
Consider the following definitions
(1) { url: { match: "/hello/world" } }
(2) { url: { match: "/hello/:name", constraint: { name: "/[a-z]+/" } }
(3) { url: { match: "/:something/world" }
(4) { url: { match: "/hello/*" } }
Then
- `/hello/world` is match by (1)
- `/hello/emil` is match by (2)
- `/your/world` is match by (3)
- `/hello/you` is match by (4)
You can write the following document into the `_routing` collection
to test the above examples.
{
routes: [
{ url: { match: "/hello/world" }, content: "route 1" },
{ url: { match: "/hello/:name|:id", constraint: { name: "/[a-z]+/", id: "/[0-9]+/" } }, content: "route 2" },
{ url: { match: "/:something/world" }, content: "route 3" },
{ url: { match: "/hello/*" }, content: "route 4" },
]
}
A Hello World Example for JSON {#UserManualActionsHelloJson}
============================================================
If you change the example slightly, then a JSON object will be delivered.
arangosh> db._routing.save({
........> url: "/hello/json",
........> content: {
........> contentType: "application/json",
........> body: "{ \"hello\" : \"world\" }" }});
arangosh> require("internal").reloadRouting()
Again check with your browser
http://localhost:8529/hello/json
Depending on your browser and installed add-ons you will either see the JSON
object or a download dialog. If your browser wants to open an external
application to display the JSON object, you can change the `contentType` to
`"text/plain"` for the example. This makes it easier to check the example using
a browser. Or use `curl` to access the server.
bash> curl "http://127.0.0.1:8529/hello/json" && echo
{ "hello" : "world" }
Delivering Content {#UserManualActionsContent}
==============================================
There are a lot of different ways on how to deliver content. We have already
seen the simplest one, where static content is delivered. The fun, however,
starts when delivering dynamic content.
Static Content {#UserManualActionsContentStatic}
------------------------------------------------
You can specify a body and a content-type.
{ content: {
contentType: "text/html",
body: "<html><body>Hallo World</body></html>"
}
}
If the content type is `text/plain` then you can use the short-cut
{ content: "Hallo World" }
A Simple Action {#UserManualActionsContentAction}
=================================================
The simplest dynamic action is:
{ action: { controller: "org/arangodb/actions", do: "echoRequest" } }
It is not possible to store functions directly in the routing table, but you can
call functions defined in modules. In the above example the function can be
accessed from JavaScript as:
require("org/arangodb/actions").echoRequest
The function `echoRequest` is pre-defined. It takes the request objects and
echos it in the response.
The signature of such a function must be
function (req, res, options, next)
For example
arangosh> db._routing.save({
........> url: "/hello/echo",
........> action: { controller: "org/arangodb/actions", do: "echoRequest" } });
Reload the routing and check
http://127.0.0.1:8529/hello/echo
You should see something like
{
"request": {
"path": "/hello/echo",
"headers": {
"accept-encoding": "gzip, deflate",
"accept-language": "de-de,de;q=0.8,en-us;q=0.5,en;q=0.3",
"connection": "keep-alive",
"content-length": "0",
"host": "localhost:8529",
"user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0"
},
"requestType": "GET",
"parameters": { }
},
"options": { }
}
The request might contain `path`, `prefix`, `suffix`, and `urlParameters`
attributes. `path` is the complete path as supplied by the user and always
available. If a prefix was matched, then this prefix is stored in the attribute
`prefix` and the remaining URL parts are stored as an array in `suffix`. If one
or more parameters were matched, then the parameter values are stored in
`urlParameters`.
For example, if the url description is
{ url: { match: "/hello/:name/:action" } }
and you request the path `/hello/emil/jump`, then the request object
will contain the following attribute
urlParameters: { name: "emil", action: "jump" } }
Action Controller {#UserManualActionsContentController}
-------------------------------------------------------
As an alternative to the simple action, you can use controllers. A
controller is a module, defines the function `get`, `put`,
`post`, `delete`, `head`, `patch`. If a request of
the corresponding type is matched, the function will be called.
For example
arangosh> db._routing.save({
........> url: "/hello/echo",
........> action: { controller: "org/arangodb/actions/echoController" } });
Prefix Action Controller {#UserManualActionsContentPrefix}
----------------------------------------------------------
The controller is selected when the definition is read. There is a
more flexible, but slower and maybe insecure variant, the prefix
controller.
Assume that the url is a prefix match
{ url: { match: /hello/*" } }
You can use
{ action: { prefixController: "org/arangodb/actions" } }
to define a prefix controller. If the URL `/hello/echoController` is
given, then the module `org/arangodb/actions/echoController` is used.
If you use an prefix controller, you should make certain that no unwanted
actions are available under the prefix.
The definition
{ action: "org/arangodb/actions" }
is a short-cut for a prefix controller definition.
Requests and Responses {#UserManualActionsReqRes}
=================================================
The controller must define handler functions which take a request object and
fill the response object.
A very simple example is the function `echoRequest` defined in
the module `org/arangodb/actions`.
function (req, res, options, next) {
var result;
result = { request: req, options: options };
res.responseCode = exports.HTTP_OK;
res.contentType = "application/json";
res.body = JSON.stringify(result);
}
Install it as
arangosh> db._routing.save({
........> url: "/echo",
........> action: { controller: "org/arangodb/actions", do: "echoRequest" } });
Reload the routing and check
http://127.0.0.1:8529/hello/echo
You should see something like
{
"request": {
"prefix": "/hello/echo",
"suffix": [
"hello",
"echo"
],
"path": "/hello/echo",
"headers": {
"accept-encoding": "gzip, deflate",
"accept-language": "de-de,de;q=0.8,en-us;q=0.5,en;q=0.3",
"connection": "keep-alive",
"content-length": "0",
"host": "localhost:8529",
"user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0"
},
"requestType": "GET",
"parameters": { }
},
"options": { }
}
You may also pass options to the called function:
arangosh> db._routing.save({
........> url: "/echo",
........> action: {
........> controller: "org/arangodb/actions",
........> do: "echoRequest",
........> options: { "Hallo": "World" } } });
You should now see the options in the result.
{
"request": {
...
},
"options": {
"Hallo": "World"
}
}
Modifying Request and Response {#UserManualActionsModify}
=========================================================
As we've seen in the previous examples, actions get called with the request and
response objects (named `req` and `res` in the examples) passed as parameters to
their handler functions.
The `req` object contains the incoming HTTP request, which might or might not
have been modified by a previous action (if actions were chained).
A handler can modify the request object in place if desired. This might be
useful when writing middleware (see below) that is used to intercept incoming
requests, modify them and pass them to the actual handlers.
While modifying the request object might not be that relevant for non-middleware
actions, modifying the response object definitely is. Modifying the response
object is an action's only way to return data to the caller of the action.
We've already seen how to set the HTTP status code, the content type, and the
result body. The `res` object has the following properties for these:
- contentType: MIME type of the body as defined in the HTTP standard (e.g.
`text/html`, `text/plain`, `application/json`, ...)
- responsecode: the HTTP status code of the response as defined in the HTTP
standard. Common values for actions that succeed are `200` or `201`.
Please refer to the HTTP standard for more information.
- body: the actual response data
To set or modify arbitrary headers of the response object, the `headers`
property can be used. For example, to add a user-defined header to the response,
the following code will do:
res.headers = res.headers || { }; // headers might or might not be present
res.headers['X-Test'] = 'someValue'; // set header X-Test to "someValue"
This will set the additional HTTP header `X-Test` to value `someValue`. Other
headers can be set as well. Note that ArangoDB might change the case of the
header names to lower case when assembling the overall response that is sent to
the caller.
It is not necessary to explicitly set a `Content-Length` header for the response
as ArangoDB will calculate the content length automatically and add this header
itself. ArangoDB might also add a `Connection` header itself to handle HTTP
keep-alive.
ArangoDB also supports automatic transformation of the body data to another
format. Currently, the only supported transformations are base64-encoding and
base64-decoding. Using the transformations, an action can create a base64
encoded body and still let ArangoDB send the non-encoded version, for example:
res.body = 'VGhpcyBpcyBhIHRlc3Q=';
res.transformations = res.transformations || [ ]; // initialise
res.transformations.push('base64decode'); // will base64 decode the response body
When ArangoDB processes the response, it will base64-decode what's in `res.body`
and set the HTTP header `Content-Encoding: binary`. The opposite can be achieved
with the `base64encode` transformation: ArangoDB will then automatically
base64-encode the body and set a `Content-Encoding: base64` HTTP header.
Writing dynamic action handlers {#UserManualActionsHandlers}
============================================================
To write your own dynamic action handlers, you must put them into modules.
Modules are a means of organising action handlers and making them loadable under
specific names.
To start, we'll define a simple action handler in a module `/own/test`:
arangosh> db._modules.save({
........> path: "/own/test",
........> content: "exports.do = function(req, res, options, next) { res.body = 'test'; res.responseCode = 200; res.contentType = 'text/html'; };",
........> autoload: true });
This does nothing but register a do action handler in a module `/own/test`. The
action handler is not yet callable, but must be mapped to a route first. To map
the action to the route `/ourtest`, execute the following command:
arangosh> db._routing.save({
........> url: "/ourtest",
........> action: { controller: "/own/test" } });
In order to see the module in action, you must either restart the server or call
the internal reload function.
arangosh> require("internal").reloadRouting()
Now use the browser and access
http://localhost:8529/ourtest
You will see that the module's do function has been executed.
Advanced Usages {#UserManualActionsAdvanced}
============================================
For detailed information see the reference manual.
Redirects {#UserManualActionsAdvancedRedirects}
-----------------------------------------------
Use the following for a permanent redirect:
arangosh> db._routing.save({
........> url: "/",
........> action: {
........> controller: "org/arangodb/actions",
........> do: "redirectRequest",
........> options: {
........> permanently: true,
........> destination: "http://somewhere.else/" } } });
Routing Bundles {#UserManualActionsAdvancedBundles}
---------------------------------------------------
Instead of adding all routes for package separately, you can
specify a bundle.
{
routes: [
{ url: "/url1", content: "..." },
{ url: "/url2", content: "..." },
{ url: "/url3", content: "..." },
...
]
}
The advantage is, that you can put all your routes into one document
and use a common prefix.
{
urlPrefix: "/test",
routes: [
{ url: "/url1", content: "..." },
{ url: "/url2", content: "..." },
{ url: "/url3", content: "..." },
...
]
}
will define the URL `/test/url1`, `/test/url2`, and `/test/url3`.
Writing Middleware {#UserManualActionsAdvancedMiddleware}
---------------------------------------------------------
Assume, you want to log every request. In this case you can easily define an
action for the whole url-space `/`. This action simply logs the requests, calls
the next in line, and logs the response.
exports.logRequest = function (req, res, options, next) {
console.log("received request: %s", JSON.stringify(req));
next();
console.log("produced response: %s", JSON.stringify(res));
};
This function is available as `org/arangodb/actions/logRequest`. You need to
tell ArangoDB that it is should use a prefix match and that the shortest match
should win in this case:
arangosh> db._routing.save({
........> middleware: [
........> { url: { match: "/*" }, action: { controller: "org/arangodb/actions", do: "logRequest" } }
........> ]
........> });

27
Documentation/UserManual/ActionsTOC.md Normal file
View File

@ -0,0 +1,27 @@
TOC {#UserManualActionsTOC}
===========================
- @ref UserManualActions
- @ref UserManualActionsIntro
- @ref UserManualActionsHelloWorld
- @ref UserManualActionsMatches
- @ref UserManualActionsMatchesExact
- @ref UserManualActionsMatchesPrefix
- @ref UserManualActionsMatchesParameterized
- @ref UserManualActionsMatchesConstraint
- @ref UserManualActionsMatchesOptional
- @ref UserManualActionsMatchesMethod
- @ref UserManualActionsMatching
- @ref UserManualActionsHelloJson
- @ref UserManualActionsContent
- @ref UserManualActionsContentStatic
- @ref UserManualActionsContentAction
- @ref UserManualActionsContentController
- @ref UserManualActionsContentPrefix
- @ref UserManualActionsReqRes
- @ref UserManualActionsModify
- @ref UserManualActionsHandlers
- @ref UserManualActionsAdvanced
- @ref UserManualActionsAdvancedRedirects
- @ref UserManualActionsAdvancedBundles
- @ref UserManualActionsAdvancedMiddleware

8
Makefile.am
View File

@ -8,7 +8,7 @@
### @brief source to build before compile
################################################################################
BUILT_SOURCES = build.h
BUILT_SOURCES = build_posix.h
################################################################################
### @brief man pages to install
@ -253,8 +253,8 @@ install-data-local:
### @brief version number
################################################################################
build.h: configure.ac
@echo '#define TRIAGENS_VERSION "@PACKAGE_VERSION@"' > build.h
build_posix.h: configure.ac
@echo '#define TRIAGENS_VERSION "@PACKAGE_VERSION@"' > build_posix.h
################################################################################
### @brief local modifications
@ -323,7 +323,7 @@ clean-local:
.PHONY: built-sources
built-sources: \
build.h \
build_posix.h \
@top_srcdir@/js/common/bootstrap/errors.js \
$(JAVASCRIPT_HEADER)

2
arangod/Actions/RestActionHandler.cpp
View File

@ -54,6 +54,8 @@ RestActionHandler::RestActionHandler (HttpRequest* request, action_options_t* da
_action(0),
_queue(),
_allowed(false) {
_action = TRI_LookupActionVocBase(request);
// check if the action is allowed

1
arangod/Actions/actions.cpp
View File

@ -158,6 +158,7 @@ TRI_action_t* TRI_LookupActionVocBase (triagens::rest::HttpRequest* request) {
string name = StringUtils::join(suffix, '/');
map<string, TRI_action_t*>::iterator i = Actions.find(name);
if (i != Actions.end()) {
return i->second;
}

2
arangod/Ahuacatl/ahuacatl-explain.c
View File

@ -146,7 +146,7 @@ static inline bool AddRow (TRI_aql_explain_t* const explain, TRI_json_t* value)
////////////////////////////////////////////////////////////////////////////////
static inline TRI_json_t* GetRowProtoType (TRI_aql_explain_t* const explain,
const TRI_aql_node_type_e const type) {
const TRI_aql_node_type_e type) {
TRI_json_t* row;
row = TRI_CreateArrayJson(TRI_UNKNOWN_MEM_ZONE);

4012
arangod/Ahuacatl/ahuacatl-grammar.c.old Normal file
View File

File diff suppressed because it is too large Load Diff

2
arangod/Ahuacatl/ahuacatl-index.c
View File

@ -290,7 +290,7 @@ void TRI_FreeIndexAql (TRI_aql_index_t* const idx) {
TRI_aql_index_t* TRI_DetermineIndexAql (TRI_aql_context_t* const context,
const TRI_vector_pointer_t* const availableIndexes,
const char* const collectionName,
const TRI_vector_pointer_t* const candidates) {
const TRI_vector_pointer_t* candidates) {
TRI_aql_index_t* picked = NULL;
TRI_vector_pointer_t matches;
size_t i, n;

2
arangod/Ahuacatl/ahuacatl-statement-walker.c
View File

@ -369,7 +369,7 @@ void TRI_FreeStatementWalkerAql (TRI_aql_statement_walker_t* const walker) {
////////////////////////////////////////////////////////////////////////////////
void TRI_WalkStatementsAql (TRI_aql_statement_walker_t* const walker,
TRI_aql_statement_list_t* const list) {
TRI_aql_statement_list_t* list) {
assert(walker);
assert(list);

2
arangod/BitIndexes/bitarrayIndex.c
View File

@ -199,7 +199,7 @@ int BitarrayIndex_new(BitarrayIndex** baIndex,
// ...........................................................................
// Sime simple checks
// Some simple checks
// ...........................................................................
if (baIndex == NULL) {

779
arangod/Documentation/user-manual.dox
View File

@ -100,785 +100,6 @@
/// JSON output format.
////////////////////////////////////////////////////////////////////////////////
// -----------------------------------------------------------------------------
// --SECTION-- USER MANUAL ACTIONS
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
/// @page UserManualActionsTOC
///
/// <ul>
/// <li>@ref UserManualActions
/// <ul>
/// <li>@ref UserManualActionsIntro</li>
/// <li>@ref UserManualActionsHelloWorld</li>
/// <li>@ref UserManualActionsMatches
/// <ul>
/// <li>@ref UserManualActionsMatchesExact</li>
/// <li>@ref UserManualActionsMatchesPrefix</li>
/// <li>@ref UserManualActionsMatchesParameterized</li>
/// <li>@ref UserManualActionsMatchesConstraint</li>
/// <li>@ref UserManualActionsMatchesOptional</li>
/// <li>@ref UserManualActionsMatchesMethod</li>
/// <li>@ref UserManualActionsMatching</li>
/// </ul>
/// </li>
/// <li>@ref UserManualActionsHelloJson</li>
/// <li>@ref UserManualActionsContent
/// <ul>
/// <li>@ref UserManualActionsContentStatic</li>
/// <li>@ref UserManualActionsContentAction</li>
/// <li>@ref UserManualActionsContentController</li>
/// <li>@ref UserManualActionsContentPrefix</li>
/// </ul>
/// </li>
/// <li>@ref UserManualActionsReqRes</li>
/// <li>@ref UserManualActionsModify</li>
/// <li>@ref UserManualActionsHandlers</li>
/// <li>@ref UserManualActionsAdvanced
/// <ul>
/// <li>@ref UserManualActionsAdvancedRedirects</li>
/// <li>@ref UserManualActionsAdvancedBundles</li>
/// <li>@ref UserManualActionsAdvancedMiddleware</li>
/// </ul>
/// </li>
/// </ul>
/// </li>
/// </ul>
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @page UserManualActions Arango Actions
///
/// @NAVIGATE_UserManualActions
///
/// Please note, that user Actions in ArangoDB are still preliminary and details
/// are subject to change.
///
/// @EMBEDTOC{UserManualActionsTOC}
///
/// @section UserManualActionsIntro Introduction to User Actions
////////////////////////////////////////////////////////////////
///
/// In some ways the communication layer of the ArangoDB server behaves like a
/// Web server. Unlike a Web server, it normally responds to HTTP requests by
/// delivering JSON objects. Remember, documents in the database are just JSON
/// objects. So, most of the time the HTTP response will contain a JSON document
/// from the database as body. You can extract the documents stored in the
/// database using HTTP @LIT{GET}. You can store documents using HTTP
/// @LIT{POST}.
///
/// However, there is something more. You can write small sniplets - so called
/// actions - to extend the database. The idea of actions is that sometimes it
/// is better to store parts of the business logic within AnrangoDB.
///
/// The simplest example is the age of a person. Assume you store information
/// about people in your database. It is an anti-pattern to store the age,
/// because it changes every now and then. Therefore, you normally store the
/// birthday and let the client decide what to do with it. However, if you have
/// many different clients, it might be easier to enrich the person document
/// with the age using actions once on the server side.
///
/// Or, for instance, if you want to apply some statistics to large data-sets
/// and you cannot easily express this as query. You can define a action instead
/// of transferring the whole data to the client and do the computation on the
/// client.
///
/// Actions are also useful if you want to restrict and filter data according to
/// some complex permission system.
///
/// The ArangoDB server can deliver all kinds of information, JSON being only
/// one possible format. You can also generate HTML or images. However, a Web
/// server is normally better suited for the task as it also implements various
/// caching strategies, language selection, compression and so on. Having said
/// that, there are still situations where it might be suitable to use the
/// ArangoDB to deliver HTML pages - static or dynamic. An simple example is the
/// built-in administration interface. You can access it using any modern
/// browser and there is no need for a separate Apache or IIS.
///
/// The following sections will explain actions within ArangoDB and show how to
/// define them. The examples start with delivering static HTML pages - even if
/// this is not the primary use-case for actions. The later sections will then
/// show you how to code some pieces of your business logic and return JSON
/// objects.
///
/// The interface is loosely modelled after the JavaScript classes for HTTP
/// request and responses found in node.js and the middleware/routing aspects
/// of connect.js and express.js.
///
/// Note that unlike node.js, ArangoDB is multi-threaded and there is no easy
/// way to share state between queries inside the JavaScript engine. If such
/// state information is required, you need to use the database itself.
///
/// @section UserManualActionsHelloWorld A Hello World Example
//////////////////////////////////////////////////////////////
///
/// The client API or browser sends a HTTP request to the ArangoDB server and
/// the server returns a HTTP response to the client. A HTTP request consists
/// of a method, normally @LIT{GET} or @LIT{POST} when using a browser, and a
/// request path like @LIT{/hello/world}. For a real Web server there are a zillion
/// of other thing to consider, we will ignore this for the moment. The HTTP
/// response contains a content type, describing how to interpret the returned
/// data, and the data itself.
///
/// In the following example, we want to define an action in ArangoDB, so that the
/// server returns the HTML document
///
/// @code
/// <html>
/// <body>
/// Hello World
/// </body>
/// </html>
/// @endcode
///
/// if asked @LIT{GET /hello/world}.
///
/// The server needs to know what function to call or what document to deliver
/// if it receives a request. This is called routing. All the routing information
/// of ArangoDB is stored in a collection @LIT{_routing}. Each entry in this
/// collections describes how to deal with a particular request path.
///
/// For the above example, add the following document to the @{_routing}
/// collection:
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: { match: "/hello/world" },
/// ........> content: {
/// ........> contentType: "text/html",
/// ........> body: "<html><body>Hello World</body></html>" }});
/// @endcode
///
/// In order to activate the new routing, you must either restart the server
/// or call the internal reload function.
///
/// @code
/// arangosh> require("internal").reloadRouting()
/// @endcode
///
/// Now use the browser and access
///
/// @LIT{http://localhost:8529/hello/world}
///
/// You should see the @LIT{Hello World} in our browser.
///
/// @section UserManualActionsMatches Matching an URL
/////////////////////////////////////////////////////
///
/// There are a lot of options for the @LIT{url} attribute. If you define
/// different routing for the same path, then the following simple rule is
/// applied in order to determine which match wins: If there are two matches,
/// then the more specific wins. I. e, if there is a wildcard match and an exact
/// match, the exact match is prefered. If there is a short and a long match,
/// the longer match wins.
///
/// @subsection UserManualActionsMatchesExact Exact Match
///
/// If the definition is
///
/// @code
/// { url: { match: "/hello/world" } }
/// @endcode
///
/// then the match must be exact. Only the request for @LIT{/hello/world} will
/// match, everything else, e. g. @LIT{/hello/world/my} or @LIT{/hello/world2},
/// will not match.
///
/// The following definition is a short-cut for an exact match.
///
/// @code
/// { url: "/hello/world" }
/// @endcode
///
/// @subsection UserManualActionsMatchesPrefix Prefix Match
///
/// If the definition is
///
/// @code
/// { url: { match: "/hello/world/*" } }
/// @endcode
///
/// then the match can be a prefix match. The requests for @LIT{/hello/world},
/// @LIT{/hello/world/my}, and @LIT{/hello/world/how/are/you} will all
/// match. However @LIT{/hello/world2} does not match. Prefix matches within an
/// URL part, i. e. @LIT{/hello/world*}, are not allowed. The wildcard must
/// occur at the end, i. e.
///
/// @code
/// /hello/*/world
/// @endcode
///
/// is also disallowed.
///
/// If you define two routes
///
/// @code
/// { url: { match: "/hello/world/*" } }
/// { url: { match: "/hello/world/emil" } }
/// @endcode
///
/// then the second route will be used for @LIT{/hello/world/emil} because it is
/// more specific.
///
/// @subsection UserManualActionsMatchesParameterized Parameterized Match
///
/// A parameterized match is similar to a prefix match, but the parameters are
/// also allowed inside the URL path.
///
/// If the definition is
///
/// @code
/// { url: { match: "/hello/:name/world" } }
/// @endcode
///
/// then the URL must have three parts, the first part being @LIT{hello} and the
/// third part @LIT{world}. For example, @LIT{/hello/emil/world} will match,
/// while @LIT{/hello/emil/meyer/world} will not.
///
/// @subsection UserManualActionsMatchesConstraint Constraint Match
///
/// A constraint match is similar to a parameterized match, but the parameters
/// can carry constraints.
///
/// If the definition is
///
/// @code
/// { url: { match: "/hello/:name/world", constraint: { name: "/[a-z]+/" } }
/// @endcode
///
/// then the URL must have three parts, the first part being @LIT{hello} and the
/// third part @LIT{world}. The second part must be all lowercase.
///
/// It is possible to use more then one constraint for the same URL part.
///
/// @code
/// { url: { match: "/hello/:name|:id/world",
/// constraint: { name: "/[a-z]+/", id: "/[0-9]+/" } }
/// @endcode
///
/// @subsection UserManualActionsMatchesOptional Optional Match
///
/// An optional match is similar to a parameterized match, but the last
/// parameter is optional.
///
/// If the definition is
///
/// @code
/// { url: { match: "/hello/:name?", constraint: { name: "/[a-z]+/" } }
/// @endcode
///
/// then the URL @LIT{/hello} and @LIT{/hello/emil} will match.
///
/// If the definitions are
///
/// @code
/// { url: { match: "/hello/world" } }
/// { url: { match: "/hello/:name", constraint: { name: "/[a-z]+/" } }
/// { url: { match: "/hello/*" } }
/// @endcode
///
/// then the URL @LIT{/hello/world} will be matched by the first route, because it
/// is the most specific. The URL @LIT{/hello/you} will be matched by the second
/// route, because it is more specific than the prefix match.
///
/// @subsection UserManualActionsMatchesMethod Method Restriction
///
/// You can restrict the match to specific methods.
///
/// If the definition is
///
/// @code
/// { url: { match: "/hello/world", methods: [ "post", "put" ] }
/// @endcode
///
/// then only @LIT{POST} and @LIT{PUT} requests will match.
///
/// @subsection UserManualActionsMatching More on Matching
///
/// Remember that the more specific match wins.
///
/// - A match without parameter or wildcard is more specific than a match with
/// parameters or wildcard.
/// - A match with parameter is more specific than a match with a wildcard.
/// - If there is more than one parameter, specificity is applied from left to
/// right.
///
/// Consider the following definitions
///
/// @code
/// (1) { url: { match: "/hello/world" } }
/// (2) { url: { match: "/hello/:name", constraint: { name: "/[a-z]+/" } }
/// (3) { url: { match: "/:something/world" }
/// (4) { url: { match: "/hello/*" } }
/// @endcode
///
/// Then
///
/// - @LIT{/hello/world} is match by (1)
/// - @LIT{/hello/emil} is match by (2)
/// - @LIT{/your/world} is match by (3)
/// - @LIT{/hello/you} is match by (4)
///
/// You can write the following document into the @LIT{_routing} collection
/// to test the above examples.
///
/// @code
/// {
/// routes: [
/// { url: { match: "/hello/world" }, content: "route 1" },
/// { url: { match: "/hello/:name|:id", constraint: { name: "/[a-z]+/", id: "/[0-9]+/" } }, content: "route 2" },
/// { url: { match: "/:something/world" }, content: "route 3" },
/// { url: { match: "/hello/*" }, content: "route 4" },
/// ]
/// }
/// @endcode
///
/// @section UserManualActionsHelloJson A Hello World Example for JSON
//////////////////////////////////////////////////////////////////////
///
/// If you change the example slightly, then a JSON object will be delivered.
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/hello/json",
/// ........> content: {
/// ........> contentType: "application/json",
/// ........> body: "{ \"hello\" : \"world\" }" }});
/// arangosh> require("internal").reloadRouting()
/// @endcode
///
/// Again check with your browser
///
/// @LIT{http://localhost:8529/hello/json}
///
/// Depending on your browser and installed add-ons you will either see the
/// JSON object or a download dialog. If your browser wants to open an external
/// application to display the JSON object, you can change the @LIT{contentType}
/// to @LIT{"text/plain"} for the example. This makes it easier to check the
/// example using a browser. Or use @LIT{curl} to access the server.
///
/// @code
/// bash> curl "http://127.0.0.1:8529/hello/json" && echo
/// { "hello" : "world" }
/// @endcode
///
/// @section UserManualActionsContent Delivering Content
////////////////////////////////////////////////////////
///
/// There are a lot of different ways on how to deliver content. We have already
/// seen the simplest one, where static content is delivered. The fun, however,
/// starts when delivering dynamic content.
///
/// @subsection UserManualActionsContentStatic Static Content
///
/// You can specify a body and a content-type.
///
/// @code
/// { content: {
/// contentType: "text/html",
/// body: "<html><body>Hallo World</body></html>"
/// }
/// }
/// @endcode
///
/// If the content type is @LIT{text/plain} then you can use the short-cut
///
/// @code
/// { content: "Hallo World" }
/// @endcode
///
/// @subsection UserManualActionsContentAction A Simple Action
///
/// The simplest dynamic action is:
///
/// @code
/// { action: { controller: "org/arangodb/actions", do: "echoRequest" } }
/// @endcode
///
/// It is not possible to store functions directly in the routing table, but you
/// can call functions defined in modules. In the above example the function can
/// be accessed from JavaScript as:
///
/// @LIT{require("org/arangodb/actions").echoRequest}
///
/// The function @LIT{echoRequest} is pre-defined. It takes the request objects
/// and echos it in the response.
///
/// The signature of such a function must be
///
/// @code
/// function (req, res, options, next)
/// @endcode
///
/// For example
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/hello/echo",
/// ........> action: { controller: "org/arangodb/actions", do: "echoRequest" } });
/// @endcode
///
/// Reload the routing and check
///
/// @LIT{http://127.0.0.1:8529/hello/echo}
///
/// You should see something like
///
/// @code
/// {
/// "request": {
/// "path": "/hello/echo",
/// "headers": {
/// "accept-encoding": "gzip, deflate",
/// "accept-language": "de-de,de;q=0.8,en-us;q=0.5,en;q=0.3",
/// "connection": "keep-alive",
/// "content-length": "0",
/// "host": "localhost:8529",
/// "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0"
/// },
/// "requestType": "GET",
/// "parameters": { }
/// },
/// "options": { }
/// }
/// @endcode
///
/// The request might contain @LIT{path}, @LIT{prefix}, @LIT{suffix}, and
/// @LIT{urlParameters} attributes. @LIT{path} is the complete path as supplied
/// by the user and always available. If a prefix was matched, then this prefix
/// is stored in the attribute @LIT{prefix} and the remaining URL parts are
/// stored as an array in @LIT{suffix}. If one or more parameters were matched,
/// then the parameter values are stored in @LIT{urlParameters}.
///
/// For example, if the url description is
///
/// @code
/// { url: { match: "/hello/:name/:action" } }
/// @endcode
///
/// and you request the path @LIT{/hello/emil/jump}, then the request object
/// will contain the following attribute
///
/// @code
/// urlParameters: { name: "emil", action: "jump" } }
/// @endcode
///
/// @subsection UserManualActionsContentController Action Controller
///
/// As an alternative to the simple action, you can use controllers. A
/// controller is a module, defines the function @LIT{get}, @LIT{put},
/// @LIT{post}, @LIT{delete}, @LIT{head}, @LIT{patch}. If a request of
/// the corresponding type is matched, the function will be called.
///
/// For example
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/hello/echo",
/// ........> action: { controller: "org/arangodb/actions/echoController" } });
/// @endcode
///
/// @subsection UserManualActionsContentPrefix Prefix Action Controller
///
/// The controller is selected when the definition is read. There is a
/// more flexible, but slower and maybe insecure variant, the prefix
/// controller.
///
/// Assume that the url is a prefix match
///
/// @code
/// { url: { match: /hello/*" } }
/// @endcode
///
/// You can use
///
/// @code
/// { action: { prefixController: "org/arangodb/actions" } }
/// @endcode
///
/// to define a prefix controller. If the URL @LIT{/hello/echoController} is
/// given, then the module @LIT{org/arangodb/actions/echoController} is used.
///
/// If you use an prefix controller, you should make certain that no unwanted
/// actions are available under the prefix.
///
/// The definition
///
/// @code
/// { action: "org/arangodb/actions" }
/// @endcode
///
/// is a short-cut for a prefix controller definition.
///
/// @section UserManualActionsReqRes Requests and Responses
///////////////////////////////////////////////////////////
///
/// The controller must define handler functions which take a request object and
/// fill the response object.
///
/// A very simple example is the function @LIT{echoRequest} defined in
/// the module @LIT{org/arangodb/actions}.
///
/// @code
/// function (req, res, options, next) {
/// var result;
///
/// result = { request: req, options: options };
///
/// res.responseCode = exports.HTTP_OK;
/// res.contentType = "application/json";
/// res.body = JSON.stringify(result);
/// }
/// @endcode
///
/// Install it as
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/echo",
/// ........> action: { controller: "org/arangodb/actions", do: "echoRequest" } });
/// @endcode
///
/// Reload the routing and check
///
/// @LIT{http://127.0.0.1:8529/hello/echo}
///
/// You should see something like
///
/// @code
/// {
/// "request": {
/// "prefix": "/hello/echo",
/// "suffix": [
/// "hello",
/// "echo"
/// ],
/// "path": "/hello/echo",
/// "headers": {
/// "accept-encoding": "gzip, deflate",
/// "accept-language": "de-de,de;q=0.8,en-us;q=0.5,en;q=0.3",
/// "connection": "keep-alive",
/// "content-length": "0",
/// "host": "localhost:8529",
/// "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0"
/// },
/// "requestType": "GET",
/// "parameters": { }
/// },
/// "options": { }
/// }
/// @endcode
///
/// You may also pass options to the called function:
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/echo",
/// ........> action: {
/// ........> controller: "org/arangodb/actions",
/// ........> do: "echoRequest",
/// ........> options: { "Hallo": "World" } } });
/// @endcode
///
/// You should now see the options in the result.
///
/// @code
/// {
/// "request": {
/// ...
/// },
/// "options": {
/// "Hallo": "World"
/// }
/// }
/// @endcode
///
/// @section UserManualActionsModify Modifying Request and Response
///
/// As we've seen in the previous examples, actions get called
/// with the request and response objects (named @LIT{req} and @LIT{res} in the
/// examples) passed as parameters to their handler functions.
///
/// The @LIT{req} object contains the incoming HTTP request, which
/// might or might not have been modified by a previous action (if actions were
/// chained).
///
/// A handler can modify the request object in place if desired. This might
/// be useful when writing middleware (see below) that is used to intercept
/// incoming requests, modify them and pass them to the actual handlers.
///
/// While modifying the request object might not be that relevant for
/// non-middleware actions, modifying the response object definitely is. Modifying
/// the response object is an action's only way to return data to the caller of
/// the action.
///
/// We've already seen how to set the HTTP status code, the content type, and the
/// result body. The @LIT{res} object has the following properties for these:
/// - contentType: MIME type of the body as defined in the HTTP standard (e.g.
/// @LIT{text/html}, @LIT{text/plain}, @LIT{application/json}, ...)
/// - responsecode: the HTTP status code of the response as defined in the HTTP
/// standard. Common values for actions that succeed are @LIT{200} or @LIT{201}.
/// Please refer to the HTTP standard for more information.
/// - body: the actual response data
///
/// To set or modify arbitrary headers of the response object, the @LIT{headers}
/// property can be used. For example, to add a user-defined header to the response,
/// the following code will do:
///
/// @code
/// res.headers = res.headers || { }; // headers might or might not be present
/// res.headers['X-Test'] = 'someValue'; // set header X-Test to "someValue"
/// @endcode
///
/// This will set the additional HTTP header @LIT{X-Test} to value @LIT{someValue}.
/// Other headers can be set as well. Note that ArangoDB might change the case
/// of the header names to lower case when assembling the overall response that
/// is sent to the caller.
///
/// It is not necessary to explicitly set a @LIT{Content-Length} header for the
/// response as ArangoDB will calculate the content length automatically and add
/// this header itself. ArangoDB might also add a @LIT{Connection} header itself
/// to handle HTTP keep-alive.
///
/// ArangoDB also supports automatic transformation of the body data to another
/// format. Currently, the only supported transformations are base64-encoding and
/// base64-decoding. Using the transformations, an action can create a base64
/// encoded body and still let ArangoDB send the non-encoded version, for example:
///
/// @code
/// res.body = 'VGhpcyBpcyBhIHRlc3Q=';
/// res.transformations = res.transformations || [ ]; // initialise
/// res.transformations.push('base64decode'); // will base64 decode the response body
/// @endcode
///
/// When ArangoDB processes the response, it will base64-decode what's in @LIT{res.body}
/// and set the HTTP header @LIT{Content-Encoding: binary}. The opposite can be
/// achieved with the @LIT{base64encode} transformation: ArangoDB will then automatically
/// base64-encode the body and set a @LIT{Content-Encoding: base64} HTTP header.
///
/// @section UserManualActionsHandlers Writing dynamic action handlers
/////////////////////////////////////////////////////////////////////
///
/// To write your own dynamic action handlers, you must put them into modules.
///
/// Modules are a means of organising action handlers and making them loadable
/// under specific names.
///
/// To start, we'll define a simple action handler in a module @LIT{/own/test}:
///
/// @code
/// arangosh> db._modules.save({
/// ........> path: "/own/test",
/// ........> content: "exports.do = function(req, res, options, next) { res.body = 'test'; res.responseCode = 200; res.contentType = 'text/html'; };",
/// ........> autoload: true });
/// @endcode
///
/// This does nothing but register a do action handler in a module @LIT{/own/test}.
/// The action handler is not yet callable, but must be mapped to a route first.
/// To map the action to the route @LIT{/ourtest}, execute the following command:
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/ourtest",
/// ........> action: { controller: "/own/test" } });
/// @endcode
///
/// In order to see the module in action, you must either restart the server
/// or call the internal reload function.
///
/// @code
/// arangosh> require("internal").reloadRouting()
/// @endcode
///
/// Now use the browser and access
///
/// @LIT{http://localhost:8529/ourtest}
///
/// You will see that the module's do function has been executed.
///
/// @section UserManualActionsAdvanced Advanced Usages
//////////////////////////////////////////////////////
///
/// For detailed information see the reference manual.
///
/// @subsection UserManualActionsAdvancedRedirects Redirects
///
/// Use the following for a permanent redirect:
///
/// @code
/// arangosh> db._routing.save({
/// ........> url: "/",
/// ........> action: {
/// ........> controller: "org/arangodb/actions",
/// ........> do: "redirectRequest",
/// ........> options: {
/// ........> permanently: true,
/// ........> destination: "http://somewhere.else/" } } });
/// @endcode
///
/// @subsection UserManualActionsAdvancedBundles Routing Bundles
///
/// Instead of adding all routes for package separately, you can
/// specify a bundle.
///
/// @code
/// {
/// routes: [
/// { url: "/url1", content: "..." },
/// { url: "/url2", content: "..." },
/// { url: "/url3", content: "..." },
/// ...
/// ]
/// }
/// @endcode
///
/// The advantage is, that you can put all your routes into one document
/// and use a common prefix.
///
/// @code
/// {
/// urlPrefix: "/test",
///
/// routes: [
/// { url: "/url1", content: "..." },
/// { url: "/url2", content: "..." },
/// { url: "/url3", content: "..." },
/// ...
/// ]
/// }
/// @endcode
///
/// will define the URL @LIT{/test/url1}, @LIT{/test/url2}, and
/// @LIT{/test/url3}.
///
/// @subsection UserManualActionsAdvancedMiddleware Writing Middleware
///
/// Assume, you want to log every request. In this case you can easily define
/// an action for the whole url-space @LIT{/}. This action simply logs
/// the requests, calls the next in line, and logs the response.
///
/// @code
/// exports.logRequest = function (req, res, options, next) {
/// console.log("received request: %s", JSON.stringify(req));
/// next();
/// console.log("produced response: %s", JSON.stringify(res));
/// };
/// @endcode
///
/// This function is available as @LIT{org/arangodb/actions/logRequest}.
/// You need to tell ArangoDB that it is should use a prefix match and
/// that the shortest match should win in this case:
///
/// @code
/// arangosh> db._routing.save({
/// ........> middleware: [
/// ........> { url: { match: "/*" }, action: { controller: "org/arangodb/actions", do: "logRequest" } }
/// ........> ]
/// ........> });
/// @endcode
////////////////////////////////////////////////////////////////////////////////
// -----------------------------------------------------------------------------
// --SECTION-- END-OF-FILE
// -----------------------------------------------------------------------------

1
arangod/PriorityQueue/pqueueindex.c
View File

@ -454,6 +454,7 @@ PQIndexElements* PQIndex_top(PQIndex* idx, uint64_t numElements) {
bool PQIndex_update(PQIndex* idx, const PQIndexElement* oldElement, const PQIndexElement* newElement) {
assert(false);
return false;
}

50
arangod/RestServer/ArangoServer.cpp
View File

@ -71,10 +71,7 @@
#include "RestHandler/RestImportHandler.h"
#include "Scheduler/ApplicationScheduler.h"
#ifndef _WIN32
#include "V8/V8LineEditor.h"
#endif
#include "V8/v8-conv.h"
#include "V8/v8-utils.h"
#include "V8Server/ApplicationV8.h"
@ -246,16 +243,23 @@ void ArangoServer::buildApplicationServer () {
_applicationServer->setUserConfigFile(string(".arango") + string(1,TRI_DIR_SEPARATOR_CHAR) + string("arangod.conf") );
// .............................................................................
// multi-threading scheduler and dispatcher
// multi-threading scheduler
// .............................................................................
_applicationScheduler = new ApplicationScheduler(_applicationServer);
_applicationScheduler->allowMultiScheduler(true);
_applicationServer->addFeature(_applicationScheduler);
// .............................................................................
// dispatcher
// .............................................................................
_applicationDispatcher = new ApplicationDispatcher(_applicationScheduler);
_applicationServer->addFeature(_applicationDispatcher);
// .............................................................................
// V8 engine
// .............................................................................
@ -304,11 +308,16 @@ void ArangoServer::buildApplicationServer () {
// daemon and supervisor mode
// .............................................................................
additional[ApplicationServer::OPTIONS_CMDLINE]
("console", "do not start as server, start a JavaScript emergency console instead")
("upgrade", "perform a database upgrade")
;
additional[ApplicationServer::OPTIONS_HIDDEN]
("no-upgrade", "skip a database upgrade")
;
#ifdef TRI_ENABLE_MRUBY
additional[ApplicationServer::OPTIONS_CMDLINE]
("ruby-console", "do not start as server, start a Ruby emergency console instead")
@ -380,6 +389,7 @@ void ArangoServer::buildApplicationServer () {
// endpoint server
// .............................................................................
_applicationEndpointServer = new ApplicationEndpointServer(_applicationServer,
_applicationScheduler,
_applicationDispatcher,
@ -396,6 +406,7 @@ void ArangoServer::buildApplicationServer () {
exit(EXIT_FAILURE);
}
#ifdef TRI_HAVE_ICU
// .............................................................................
// set language of default collator
@ -415,6 +426,7 @@ void ArangoServer::buildApplicationServer () {
LOGGER_INFO << "using default language '" << Utf8Helper::DefaultUtf8Helper.getCollatorLanguage() << "'" ;
}
#endif
// .............................................................................
// disable access to the HTML admin interface
@ -518,6 +530,7 @@ void ArangoServer::buildApplicationServer () {
exit(EXIT_FAILURE);
}
}
}
////////////////////////////////////////////////////////////////////////////////
@ -548,6 +561,11 @@ int ArangoServer::startupServer () {
_applicationV8->performUpgrade();
}
// skip an upgrade even if VERSION is missing
if (_applicationServer->programOptions().has("no-upgrade")) {
_applicationV8->skipUpgrade();
}
#if TRI_ENABLE_MRUBY
_applicationMR->setVocbase(_vocbase);
_applicationMR->setConcurrency(_dispatcherThreads);
@ -574,6 +592,7 @@ int ArangoServer::startupServer () {
httpOptions._contexts.insert("api");
httpOptions._contexts.insert("admin");
// create the server
_applicationEndpointServer->buildServers();
@ -603,13 +622,16 @@ int ArangoServer::startupServer () {
LOGGER_INFO << "ArangoDB (version " << TRIAGENS_VERSION << ") is ready for business";
LOGGER_INFO << "Have Fun!";
_applicationServer->wait();
// .............................................................................
// and cleanup
// .............................................................................
_applicationServer->stop();
closeDatabase();
return 0;
@ -646,9 +668,16 @@ int ArangoServer::executeConsole (OperationMode::server_operation_mode_e mode) {
// set-up V8 context
_applicationV8->setVocbase(_vocbase);
_applicationV8->setConcurrency(1);
if (_applicationServer->programOptions().has("upgrade")) {
_applicationV8->performUpgrade();
}
// skip an upgrade even if VERSION is missing
if (_applicationServer->programOptions().has("no-upgrade")) {
_applicationV8->skipUpgrade();
}
_applicationV8->disableActions();
ok = _applicationV8->prepare();
@ -763,7 +792,7 @@ int ArangoServer::executeConsole (OperationMode::server_operation_mode_e mode) {
v8::TryCatch tryCatch;
for (size_t i = 0; i < _scriptFile.size(); ++i) {
bool r = TRI_LoadJavaScriptFile(context->_context, _scriptFile[i].c_str());
bool r = TRI_ExecuteGlobalJavaScriptFile(_scriptFile[i].c_str());
if (! r) {
LOGGER_FATAL << "cannot load script '" << _scriptFile[i] << ", giving up";
@ -817,7 +846,6 @@ int ArangoServer::executeConsole (OperationMode::server_operation_mode_e mode) {
// .............................................................................
case OperationMode::MODE_CONSOLE: {
#ifndef _WIN32
V8LineEditor console(context->_context, ".arango");
console.open(true);
@ -851,7 +879,6 @@ int ArangoServer::executeConsole (OperationMode::server_operation_mode_e mode) {
}
}
#endif
break;
}
@ -874,7 +901,11 @@ int ArangoServer::executeConsole (OperationMode::server_operation_mode_e mode) {
closeDatabase();
Random::shutdown();
return ok ? EXIT_SUCCESS : EXIT_FAILURE;
if (!ok) {
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
////////////////////////////////////////////////////////////////////////////////
@ -1056,6 +1087,8 @@ int ArangoServer::executeRubyConsole () {
////////////////////////////////////////////////////////////////////////////////
void ArangoServer::openDatabase () {
TRI_InitialiseVocBase();
_vocbase = TRI_OpenVocBase(_databasePath.c_str());
if (! _vocbase) {
@ -1082,6 +1115,7 @@ void ArangoServer::closeDatabase () {
TRI_DestroyVocBase(_vocbase);
TRI_Free(TRI_UNKNOWN_MEM_ZONE, _vocbase);
_vocbase = 0;
TRI_ShutdownVocBase();
LOGGER_INFO << "ArangoDB has been shut down";
}

4
arangod/RestServer/ArangoServer.h
View File

@ -28,6 +28,10 @@
#ifndef TRIAGENS_REST_SERVER_ARANGO_SERVER_H
#define TRIAGENS_REST_SERVER_ARANGO_SERVER_H 1
#ifdef _WIN32
#include "BasicsC/win-utils.h"
#endif
#include "Rest/AnyServer.h"
#include "Rest/OperationMode.h"

40
arangod/RestServer/arango.cpp
View File

@ -50,20 +50,52 @@ using namespace triagens::arango;
int main (int argc, char* argv[]) {
int res;
#ifdef _WIN32
// ...........................................................................
// Call this function to do various initialistions for windows only
// ...........................................................................
// ...........................................................................
// Uncomment this to call this for extended debug information.
// If you familiar with valgrind ... then this is not like that, however
// you do get some similar functionality.
// ...........................................................................
//res = initialiseWindows(TRI_WIN_INITIAL_SET_DEBUG_FLAG, 0);
res = initialiseWindows(TRI_WIN_INITIAL_SET_INVALID_HANLE_HANDLER, 0);
if (res != 0) {
_exit(1);
}
res = initialiseWindows(TRI_WIN_INITIAL_WSASTARTUP_FUNCTION_CALL, 0);
if (res != 0) {
_exit(1);
}
#endif
TRIAGENS_RESULT_GENERATOR_INITIALISE(argc, argv);
TRI_InitialiseVocBase();
// create and start a ArangoDB server
ArangoServer server(argc, argv);
res = server.start();
// shutdown
TRI_ShutdownVocBase();
TRIAGENS_RESULT_GENERATOR_SHUTDOWN;
#ifdef _WIN32
// ...........................................................................
// TODO: need a terminate function for windows to be called and cleanup
// any windows specific stuff.
// TODO: find the memory deallocation/allocation error
// ...........................................................................
res = finaliseWindows(TRI_WIN_FINAL_WSASTARTUP_FUNCTION_CALL, 0);
#endif
return res;
}

7
arangod/SkipLists/skiplist.c
View File

@ -501,6 +501,7 @@ void* TRI_EndNodeSkipList(TRI_skiplist_t* skiplist) {
int TRI_InsertElementSkipList(TRI_skiplist_t* skiplist, void* element, bool overwrite) {
// Use TRI_InsertKeySkipList instead of calling this method
assert(false);
return 0;
}
@ -887,6 +888,7 @@ void* TRI_LeftLookupByKeySkipList (TRI_skiplist_t* skiplist, void* key) {
void* TRI_LookupByElementSkipList (TRI_skiplist_t* skiplist, void* element) {
assert(false); // there is no way we can be here
return NULL;
}
@ -1207,6 +1209,7 @@ int TRI_RemoveElementSkipList (TRI_skiplist_t* skiplist, void* element, void* ol
int TRI_RemoveKeySkipList (TRI_skiplist_t* skiplist, void* key, void* old) {
// Use the TRI_RemoveElementSkipList method instead.
assert(false);
return 0;
}
@ -1695,6 +1698,7 @@ void* TRI_LeftLookupByKeySkipListMulti(TRI_skiplist_multi_t* skiplist, void* key
void* TRI_LookupByElementSkipListMulti(TRI_skiplist_multi_t* skiplist, void* element) {
assert(false); // there is no way you should be here
return 0;
}
@ -1705,6 +1709,7 @@ void* TRI_LookupByElementSkipListMulti(TRI_skiplist_multi_t* skiplist, void* ele
void* TRI_LookupByKeySkipListMulti(TRI_skiplist_multi_t* skiplist, void* key) {
assert(false); // there is no way you should be here
return 0;
}
@ -1954,6 +1959,7 @@ int TRI_InsertElementSkipListMulti(TRI_skiplist_multi_t* skiplist, void* element
int TRI_InsertKeySkipListMulti(TRI_skiplist_multi_t* skiplist, void* key, void* element, bool overwrite) {
// Use TRI_InsertelementSkipList instead of calling this method
assert(false);
return 0;
}
@ -2183,6 +2189,7 @@ int TRI_RemoveElementSkipListMulti (TRI_skiplist_multi_t* skiplist, void* elemen
int TRI_RemoveKeySkipListMulti(TRI_skiplist_multi_t* skiplist, void* key, void* old) {
// Use the TRI_RemoveElementSkipListMulti method instead.
assert(false);
return 0;
}

2
arangod/SkipLists/skiplistIndex.c
View File

@ -987,6 +987,7 @@ bool SkiplistIndex_update(SkiplistIndex* skiplistIndex, const SkiplistIndexEleme
// then adds the afterElement -- should never be called here
// call SkiplistIndex_remove first and then SkiplistIndex_add
assert(false);
return false; // shuts the VC++ up
}
@ -1390,6 +1391,7 @@ int MultiSkiplistIndex_remove(SkiplistIndex* skiplistIndex, SkiplistIndexElement
bool MultiSkiplistIndex_update(SkiplistIndex* skiplistIndex, SkiplistIndexElement* beforeElement, SkiplistIndexElement* afterElement) {
assert(false); // should never be called directly
return false; // shuts the VC++ up
}

15
arangod/V8Server/ApplicationV8.cpp
View File

@ -180,6 +180,7 @@ ApplicationV8::ApplicationV8 (string const& binaryPath)
_actionPath(),
_useActions(true),
_performUpgrade(false),
_skipUpgrade(false),
_gcInterval(1000),
_gcFrequency(10.0),
_v8Options(""),
@ -239,6 +240,14 @@ void ApplicationV8::performUpgrade () {
_performUpgrade = true;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief skip a database upgrade
////////////////////////////////////////////////////////////////////////////////
void ApplicationV8::skipUpgrade () {
_skipUpgrade = true;
}
////////////////////////////////////////////////////////////////////////////////
/// @brief enters an context
////////////////////////////////////////////////////////////////////////////////
@ -511,6 +520,7 @@ void ApplicationV8::setupOptions (map<string, basics::ProgramOptionsDescription>
bool ApplicationV8::prepare () {
LOGGER_DEBUG << "V8 version: " << v8::V8::GetVersion();
// use a minimum of 1 second for GC
if (_gcFrequency < 1) {
_gcFrequency = 1;
@ -529,7 +539,6 @@ bool ApplicationV8::prepare () {
// set up the startup loader
if (_startupPath.empty()) {
LOGGER_INFO << "using built-in JavaScript startup files";
_startupLoader.defineScript("common/bootstrap/modules.js", JS_common_bootstrap_modules);
_startupLoader.defineScript("common/bootstrap/monkeypatches.js", JS_common_bootstrap_monkeypatches);
_startupLoader.defineScript("common/bootstrap/print.js", JS_common_bootstrap_print);
@ -557,6 +566,7 @@ bool ApplicationV8::prepare () {
}
}
if (_v8Options.size() > 0) {
LOGGER_INFO << "using V8 options '" << _v8Options << "'";
v8::V8::SetFlagsFromString(_v8Options.c_str(), _v8Options.size());
@ -669,6 +679,7 @@ bool ApplicationV8::prepareV8Instance (const size_t i) {
TRI_InitV8VocBridge(context->_context, _vocbase, i);
TRI_InitV8Queries(context->_context);
if (_useActions) {
TRI_InitV8Actions(context->_context, this);
}
@ -693,7 +704,7 @@ bool ApplicationV8::prepareV8Instance (const size_t i) {
}
if (i == 0) {
if (i == 0 && ! _skipUpgrade) {
LOGGER_DEBUG << "running database version check";
const string script = _startupLoader.buildScript(JS_server_version_check);

12
arangod/V8Server/ApplicationV8.h
View File

@ -206,6 +206,12 @@ namespace triagens {
void performUpgrade ();
////////////////////////////////////////////////////////////////////////////////
/// @brief disable the database version check
////////////////////////////////////////////////////////////////////////////////
void skipUpgrade ();
////////////////////////////////////////////////////////////////////////////////
/// @brief enters an context
////////////////////////////////////////////////////////////////////////////////
@ -401,6 +407,12 @@ namespace triagens {
bool _performUpgrade;
////////////////////////////////////////////////////////////////////////////////
/// @brief perform a database upgrade
////////////////////////////////////////////////////////////////////////////////
bool _skipUpgrade;
////////////////////////////////////////////////////////////////////////////////
/// @brief JavaScript garbage collection interval (each x requests)
///

1
arangod/V8Server/v8-actions.cpp
View File

@ -45,7 +45,6 @@ using namespace triagens::basics;
using namespace triagens::rest;
using namespace triagens::arango;
// -----------------------------------------------------------------------------
// --SECTION-- forward declarations
// -----------------------------------------------------------------------------

8
arangod/V8Server/v8-vocbase.cpp
View File

@ -2839,7 +2839,7 @@ static v8::Handle<v8::Value> JS_UpgradeVocbaseCol (v8::Arguments const& argv) {
break;
}
off_t paddedSize = ((marker._size + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
off_t paddedSize = TRI_DF_ALIGN_BLOCK(marker._size);
char payload[paddedSize];
char* p = (char*) &payload;
@ -2882,7 +2882,7 @@ static v8::Handle<v8::Value> JS_UpgradeVocbaseCol (v8::Arguments const& argv) {
sprintf(didBuffer,"%d", (unsigned int) oldMarker->_did);
keySize = strlen(didBuffer) + 1;
keyBodySize = ((keySize + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
keyBodySize = TRI_DF_ALIGN_BLOCK(keySize);
keyBody = (char*) TRI_Allocate(TRI_CORE_MEM_ZONE, keyBodySize, true);
TRI_CopyString(keyBody, didBuffer, keySize);
@ -2942,7 +2942,7 @@ static v8::Handle<v8::Value> JS_UpgradeVocbaseCol (v8::Arguments const& argv) {
toSize = strlen(toDidBuffer) + 1;
fromSize = strlen(fromDidBuffer) + 1;
keyBodySize = ((keySize + toSize + fromSize + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
keyBodySize = TRI_DF_ALIGN_BLOCK(keySize + toSize + fromSize);
keyBody = (char*) TRI_Allocate(TRI_CORE_MEM_ZONE, keyBodySize, true);
TRI_CopyString(keyBody, didBuffer, keySize);
@ -2996,7 +2996,7 @@ static v8::Handle<v8::Value> JS_UpgradeVocbaseCol (v8::Arguments const& argv) {
sprintf(didBuffer,"%d", (unsigned int) oldMarker->_did);
keySize = strlen(didBuffer) + 1;
keyBodySize = ((keySize + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
keyBodySize = TRI_DF_ALIGN_BLOCK(keySize);
keyBody = (char*) TRI_Allocate(TRI_CORE_MEM_ZONE, keyBodySize, true);
TRI_CopyString(keyBody, didBuffer, keySize);

1
arangod/VocBase/cleanup.c
View File

@ -271,6 +271,7 @@ void TRI_CleanupVocBase (void* data) {
// server shutdown
break;
}
}
TRI_DestroyVectorPointer(&collections);

12
arangod/VocBase/collection.c
View File

@ -242,9 +242,9 @@ static bool CheckCollection (TRI_collection_t* collection) {
TRI_PushBackVectorPointer(&all, datafile);
// check the document header
ptr = datafile->_data;
ptr += ((sizeof(TRI_df_header_marker_t) + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;;
cm = (TRI_col_header_marker_t*) ptr;
ptr = datafile->_data;
ptr += TRI_DF_ALIGN_BLOCK(sizeof(TRI_df_header_marker_t));
cm = (TRI_col_header_marker_t*) ptr;
if (cm->base._type != TRI_COL_MARKER_HEADER) {
LOG_ERROR("collection header mismatch in file '%s', expected TRI_COL_MARKER_HEADER, found %lu",
@ -346,9 +346,7 @@ static bool CheckCollection (TRI_collection_t* collection) {
else {
collection->_lastError = datafile->_lastError;
stop = true;
LOG_ERROR("cannot rename sealed log-file to %s, this should not happen: %s", filename, TRI_last_error());
break;
}
@ -677,7 +675,9 @@ TRI_collection_t* TRI_CreateCollection (TRI_vocbase_t* vocbase,
}
InitCollection(vocbase, collection, filename, parameter);
/* PANAIA: 1) the parameter file if it exists must be removed
2) if collection
*/
// return collection
return collection;
}

13
arangod/VocBase/collection.h
View File

@ -183,7 +183,7 @@ TRI_col_state_e;
typedef uint32_t TRI_col_version_t;
////////////////////////////////////////////////////////////////////////////////
/// @brief collection type
/// @brief collection enum
////////////////////////////////////////////////////////////////////////////////
typedef enum {
@ -198,10 +198,15 @@ TRI_col_type_e;
////////////////////////////////////////////////////////////////////////////////
typedef struct TRI_col_header_marker_s {
TRI_df_marker_t base;
TRI_df_marker_t base; // 24 bytes
TRI_col_type_e _type;
TRI_voc_cid_t _cid;
TRI_col_type_t _type; // 4 bytes
#ifdef TRI_PADDING_32
char _padding_col_header_marker[4];
#endif
TRI_voc_cid_t _cid; // 8 bytes
}
TRI_col_header_marker_t;

133
arangod/VocBase/datafile.c
View File

@ -25,6 +25,10 @@
/// @author Copyright 2011, triagens GmbH, Cologne, Germany
////////////////////////////////////////////////////////////////////////////////
#ifdef _WIN32
#include <BasicsC/win-utils.h>
#endif
#include "datafile.h"
#include <BasicsC/hashes.h>
@ -33,6 +37,7 @@
#include <BasicsC/strings.h>
#include <BasicsC/files.h>
// #define DEBUG_DATAFILE 1
// -----------------------------------------------------------------------------
@ -121,11 +126,11 @@ static int TruncateDatafile (TRI_datafile_t* datafile, TRI_voc_size_t vocSize) {
}
// create sparse file
offset = lseek(fd, maximalSize - 1, SEEK_SET);
offset = TRI_LSEEK(fd, maximalSize - 1, SEEK_SET);
if (offset == (off_t) -1) {
TRI_set_errno(TRI_ERROR_SYS_ERROR);
close(fd);
TRI_CLOSE(fd);
// remove empty file
TRI_UnlinkFile(filename);
@ -135,11 +140,11 @@ static int TruncateDatafile (TRI_datafile_t* datafile, TRI_voc_size_t vocSize) {
}
zero = 0;
res = write(fd, &zero, 1);
res = TRI_WRITE(fd, &zero, 1);
if (res < 0) {
TRI_set_errno(TRI_ERROR_SYS_ERROR);
close(fd);
TRI_CLOSE(fd);
// remove empty file
TRI_UnlinkFile(filename);
@ -149,11 +154,11 @@ static int TruncateDatafile (TRI_datafile_t* datafile, TRI_voc_size_t vocSize) {
}
// memory map the data
res = TRI_MMFile(0, maximalSize, PROT_WRITE | PROT_READ, MAP_SHARED, &fd, &mmHandle, 0, &data);
res = TRI_MMFile(0, maximalSize, PROT_WRITE | PROT_READ, MAP_SHARED, fd, &mmHandle, 0, &data);
if (res != TRI_ERROR_NO_ERROR) {
TRI_set_errno(res);
close(fd);
TRI_CLOSE(fd);
// remove empty file
TRI_UnlinkFile(filename);
@ -166,15 +171,21 @@ static int TruncateDatafile (TRI_datafile_t* datafile, TRI_voc_size_t vocSize) {
memcpy(data, datafile->_data, vocSize);
// patch the datafile structure
res = TRI_UNMMFile(datafile->_data, datafile->_maximalSize, &(datafile->_fd), &(datafile->_mmHandle));
res = TRI_UNMMFile(datafile->_data, datafile->_maximalSize, datafile->_fd, &(datafile->_mmHandle));
if (res < 0) {
LOG_ERROR("munmap failed with: %d", res);
return res;
}
close(datafile->_fd);
// the datafile->_mmHandle object has been closed in the underlying TRI_UNMMFile(...) call above
// .............................................................................................
// For windows: Mem mapped files use handles
// the datafile->_mmHandle handle object has been closed in the underlying
// TRI_UNMMFile(...) call above so we do not need to close it for the associated file below
// .............................................................................................
TRI_CLOSE(datafile->_fd);
datafile->_data = data;
datafile->_next = (char*)(data) + vocSize;
@ -287,7 +298,7 @@ static TRI_df_scan_t ScanDatafile (TRI_datafile_t const* datafile) {
TRI_PushBackVector(&scan._entries, &entry);
size = ((marker->_size + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
size = TRI_DF_ALIGN_BLOCK(marker->_size);
currentSize += size;
if (marker->_type == TRI_DF_MARKER_FOOTER) {
@ -385,7 +396,7 @@ static bool CheckDatafile (TRI_datafile_t* datafile) {
TRI_UpdateTickVocBase(marker->_tick);
size = ((marker->_size + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
size = TRI_DF_ALIGN_BLOCK(marker->_size);
currentSize += size;
if (marker->_type == TRI_DF_MARKER_FOOTER) {
@ -422,8 +433,11 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
struct stat status;
TRI_df_header_marker_t header;
void* mmHandle;
// open the file
// ..........................................................................
// attempt to open a datafile file
// ..........................................................................
fd = TRI_OPEN(filename, O_RDWR);
if (fd < 0) {
@ -434,12 +448,14 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
return NULL;
}
// compute the size of the file
res = fstat(fd, &status);
if (res < 0) {
TRI_set_errno(TRI_ERROR_SYS_ERROR);
close(fd);
TRI_CLOSE(fd);
LOG_ERROR("cannot get status of datafile '%s': %s", filename, TRI_last_error());
@ -451,7 +467,7 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
if (size < sizeof(TRI_df_header_marker_t) + sizeof(TRI_df_footer_marker_t)) {
TRI_set_errno(TRI_ERROR_ARANGO_CORRUPTED_DATAFILE);
close(fd);
TRI_CLOSE(fd);
LOG_ERROR("datafile '%s' is corrupted, size is only %u", filename, (unsigned int) size);
@ -467,7 +483,7 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
if (! ok) {
LOG_ERROR("cannot read datafile header from '%s': %s", filename, TRI_last_error());
close(fd);
TRI_CLOSE(fd);
return NULL;
}
@ -480,7 +496,7 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
LOG_ERROR("corrupted datafile header read from '%s'", filename);
if (! ignoreErrors) {
close(fd);
TRI_CLOSE(fd);
return NULL;
}
}
@ -495,7 +511,7 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
filename);
if (! ignoreErrors) {
close(fd);
TRI_CLOSE(fd);
return NULL;
}
}
@ -509,11 +525,11 @@ static TRI_datafile_t* OpenDatafile (char const* filename, bool ignoreErrors) {
}
// map datafile into memory
res = TRI_MMFile(0, size, PROT_READ, MAP_SHARED, &fd, &mmHandle, 0, &data);
res = TRI_MMFile(0, size, PROT_READ, MAP_SHARED, fd, &mmHandle, 0, &data);
if (res != TRI_ERROR_NO_ERROR) {
TRI_set_errno(res);
close(fd);
TRI_CLOSE(fd);
LOG_ERROR("cannot memory map file '%s': '%d'", filename, res);
return NULL;
}
@ -589,12 +605,12 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
}
// create sparse file
offset = lseek(fd, maximalSize - 1, SEEK_SET);
offset = TRI_LSEEK(fd, maximalSize - 1, SEEK_SET);
if (offset == (off_t) -1) {
TRI_set_errno(TRI_ERROR_SYS_ERROR);
close(fd);
TRI_CLOSE(fd);
// remove empty file
TRI_UnlinkFile(filename);
@ -603,11 +619,11 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
}
zero = 0;
res = write(fd, &zero, 1);
res = TRI_WRITE(fd, &zero, 1);
if (res < 0) {
TRI_set_errno(TRI_ERROR_SYS_ERROR);
close(fd);
TRI_CLOSE(fd);
// remove empty file
TRI_UnlinkFile(filename);
@ -617,11 +633,11 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
}
// memory map the data
res = TRI_MMFile(0, maximalSize, PROT_WRITE | PROT_READ, MAP_SHARED, &fd, &mmHandle, 0, &data);
res = TRI_MMFile(0, maximalSize, PROT_WRITE | PROT_READ, MAP_SHARED, fd, &mmHandle, 0, &data);
if (res != TRI_ERROR_NO_ERROR) {
TRI_set_errno(res);
close(fd);
TRI_CLOSE(fd);
// remove empty file
TRI_UnlinkFile(filename);
@ -630,20 +646,22 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
return NULL;
}
// get next tick
tick = TRI_NewTickVocBase();
// create datafile structure
datafile = TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, sizeof(TRI_datafile_t), false);
datafile = (TRI_datafile_t*)(TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, sizeof(TRI_datafile_t), false));
if (datafile == NULL) {
TRI_set_errno(TRI_ERROR_OUT_OF_MEMORY);
close(fd);
TRI_CLOSE(fd);
LOG_ERROR("out-of-memory");
return NULL;
}
InitDatafile(datafile,
TRI_DuplicateString(filename),
fd,
@ -653,6 +671,7 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
tick,
data);
datafile->_state = TRI_DF_STATE_WRITE;
// create the header
@ -666,6 +685,7 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
header._maximalSize = maximalSize;
header._fid = tick;
// create CRC
TRI_FillCrcMarkerDatafile(&header.base, sizeof(TRI_df_header_marker_t), 0, 0, 0, 0);
@ -677,10 +697,9 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
}
if (result != TRI_ERROR_NO_ERROR) {
LOG_ERROR("cannot write header to datafile '%s'",
filename);
TRI_UNMMFile(datafile->_data, datafile->_maximalSize, &fd, &mmHandle);
close(fd);
LOG_ERROR("cannot write header to datafile '%s'",filename);
TRI_UNMMFile(datafile->_data, datafile->_maximalSize, fd, &mmHandle);
TRI_CLOSE(fd);
TRI_FreeString(TRI_UNKNOWN_MEM_ZONE, datafile->_filename);
TRI_Free(TRI_UNKNOWN_MEM_ZONE, datafile);
@ -688,11 +707,13 @@ TRI_datafile_t* TRI_CreateDatafile (char const* filename, TRI_voc_size_t maximal
return NULL;
}
LOG_DEBUG("created datafile '%s' of size %u and page-size %u",
filename,
(unsigned int) maximalSize,
(unsigned int) PageSize);
return datafile;
}
@ -822,7 +843,7 @@ int TRI_ReserveElementDatafile (TRI_datafile_t* datafile,
TRI_voc_size_t size,
TRI_df_marker_t** position) {
*position = 0;
size = ((size + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
size = TRI_DF_ALIGN_BLOCK(size);
if (datafile->_state != TRI_DF_STATE_WRITE) {
if (datafile->_state == TRI_DF_STATE_READ) {
@ -961,7 +982,7 @@ bool TRI_IterateDatafile (TRI_datafile_t* datafile,
return false;
}
size = ((marker->_size + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
size = TRI_DF_ALIGN_BLOCK(marker->_size);
ptr += size;
}
@ -988,8 +1009,8 @@ TRI_datafile_t* TRI_OpenDatafile (char const* filename) {
ok = CheckDatafile(datafile);
if (!ok) {
TRI_UNMMFile(datafile->_data, datafile->_maximalSize, &(datafile->_fd), &(datafile->_mmHandle));
close(datafile->_fd);
TRI_UNMMFile(datafile->_data, datafile->_maximalSize, datafile->_fd, &(datafile->_mmHandle));
TRI_CLOSE(datafile->_fd);
LOG_ERROR("datafile '%s' is corrupt", datafile->_filename);
// must free datafile here
@ -1001,7 +1022,7 @@ TRI_datafile_t* TRI_OpenDatafile (char const* filename) {
// change to read-write if no footer has been found
if (! datafile->_isSealed) {
datafile->_state = TRI_DF_STATE_WRITE;
TRI_ProtectMMFile(datafile->_data, datafile->_maximalSize, PROT_READ | PROT_WRITE, &(datafile->_fd), &(datafile->_mmHandle));
TRI_ProtectMMFile(datafile->_data, datafile->_maximalSize, PROT_READ | PROT_WRITE, datafile->_fd, &(datafile->_mmHandle));
}
return datafile;
@ -1032,7 +1053,7 @@ TRI_datafile_t* TRI_ForcedOpenDatafile (char const* filename) {
else {
if (! datafile->_isSealed) {
datafile->_state = TRI_DF_STATE_WRITE;
TRI_ProtectMMFile(datafile->_data, datafile->_maximalSize, PROT_READ | PROT_WRITE, &(datafile->_fd), &(datafile->_mmHandle));
TRI_ProtectMMFile(datafile->_data, datafile->_maximalSize, PROT_READ | PROT_WRITE, datafile->_fd, &(datafile->_mmHandle));
}
}
@ -1046,8 +1067,7 @@ TRI_datafile_t* TRI_ForcedOpenDatafile (char const* filename) {
bool TRI_CloseDatafile (TRI_datafile_t* datafile) {
if (datafile->_state == TRI_DF_STATE_READ || datafile->_state == TRI_DF_STATE_WRITE) {
int res;
res = TRI_UNMMFile(datafile->_data, datafile->_maximalSize, &(datafile->_fd), &(datafile->_mmHandle));
res = TRI_UNMMFile(datafile->_data, datafile->_maximalSize, datafile->_fd, &(datafile->_mmHandle));
if (res != TRI_ERROR_NO_ERROR) {
LOG_ERROR("munmap failed with: %d", res);
@ -1057,7 +1077,7 @@ bool TRI_CloseDatafile (TRI_datafile_t* datafile) {
}
else {
close(datafile->_fd);
TRI_CLOSE(datafile->_fd);
datafile->_state = TRI_DF_STATE_CLOSED;
datafile->_data = 0;
@ -1155,6 +1175,7 @@ int TRI_SealDatafile (TRI_datafile_t* datafile) {
ok = TRI_msync(datafile->_fd, datafile->_mmHandle, datafile->_data, ((char*) datafile->_data) + datafile->_currentSize);
if (! ok) {
abort();
datafile->_state = TRI_DF_STATE_WRITE_ERROR;
if (errno == ENOSPC) {
@ -1171,14 +1192,28 @@ int TRI_SealDatafile (TRI_datafile_t* datafile) {
datafile->_synced = datafile->_written;
datafile->_nSynced = datafile->_nWritten;
// change the datafile to read-only
TRI_ProtectMMFile(datafile->_data, datafile->_maximalSize, PROT_READ, &(datafile->_fd), &(datafile->_mmHandle));
/*
TODO: do we have to unmap file? That is, release the memory which has been allocated for
this file? At the moment the windows of function TRI_ProtectMMFile does nothing.
*/
TRI_ProtectMMFile(datafile->_data, datafile->_maximalSize, PROT_READ, datafile->_fd, &(datafile->_mmHandle));
// truncate datafile
if (ok) {
res = ftruncate(datafile->_fd, datafile->_currentSize);
#ifdef _WIN32
res = 0;
/*
res = ftruncate(datafile->_fd, datafile->_currentSize);
Linux centric problems:
Under windows can not reduce size of the memory mapped file without unmappping it!
However, apparently we may have users
*/
#else
res = ftruncate(datafile->_fd, datafile->_currentSize);
#endif
if (res < 0) {
abort();
LOG_ERROR("cannot truncate datafile '%s': %s", datafile->_filename, TRI_last_error());
datafile->_lastError = TRI_set_errno(TRI_ERROR_SYS_ERROR);
ok = false;
@ -1188,7 +1223,11 @@ int TRI_SealDatafile (TRI_datafile_t* datafile) {
datafile->_state = TRI_DF_STATE_READ;
}
return ok ? TRI_ERROR_NO_ERROR : datafile->_lastError;
if (!ok) {
return datafile->_lastError;
}
return TRI_ERROR_NO_ERROR;
}
////////////////////////////////////////////////////////////////////////////////

29
arangod/VocBase/datafile.h
View File

@ -112,7 +112,7 @@ extern "C" {
/// @brief alignment in datafile blocks
////////////////////////////////////////////////////////////////////////////////
#define TRI_DF_BLOCK_ALIGN (8)
#define TRI_DF_BLOCK_ALIGNMENT (8)
////////////////////////////////////////////////////////////////////////////////
/// @}
@ -302,6 +302,9 @@ typedef struct TRI_df_marker_s {
TRI_voc_tick_t _tick; // 8 bytes, will be generated
#ifdef TRI_PADDING_32
char _padding_df_marker[4];
#endif
}
TRI_df_marker_t;
@ -336,11 +339,11 @@ TRI_df_marker_t;
////////////////////////////////////////////////////////////////////////////////
typedef struct TRI_df_header_marker_s {
TRI_df_marker_t base;
TRI_df_marker_t base; // 24 bytes
TRI_df_version_t _version;
TRI_voc_size_t _maximalSize;
TRI_voc_tick_t _fid;
TRI_df_version_t _version; // 4 bytes
TRI_voc_size_t _maximalSize; // 4 bytes
TRI_voc_tick_t _fid; // 8 bytes
}
TRI_df_header_marker_t;
@ -371,10 +374,10 @@ TRI_df_header_marker_t;
////////////////////////////////////////////////////////////////////////////////
typedef struct TRI_df_footer_marker_s {
TRI_df_marker_t base;
TRI_df_marker_t base; // 24 bytes
TRI_voc_size_t _maximalSize;
TRI_voc_size_t _totalSize;
TRI_voc_size_t _maximalSize; // 4 bytes
TRI_voc_size_t _totalSize; // 4 bytes
}
TRI_df_footer_marker_t;
@ -383,7 +386,7 @@ TRI_df_footer_marker_t;
////////////////////////////////////////////////////////////////////////////////
typedef struct TRI_df_document_marker_s {
TRI_df_marker_t base;
TRI_df_marker_t base; // 24 bytes
}
TRI_df_document_marker_t;
@ -392,7 +395,7 @@ TRI_df_document_marker_t;
////////////////////////////////////////////////////////////////////////////////
typedef struct TRI_df_skip_marker_s {
TRI_df_marker_t base;
TRI_df_marker_t base; // 24 bytes
}
TRI_df_skip_marker_t;
@ -448,6 +451,12 @@ void TRI_FreeDatafile (TRI_datafile_t* datafile);
/// @{
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @brief aligns in datafile blocks
////////////////////////////////////////////////////////////////////////////////
#define TRI_DF_ALIGN_BLOCK(a) ((((a) + TRI_DF_BLOCK_ALIGNMENT - 1) / TRI_DF_BLOCK_ALIGNMENT) * TRI_DF_BLOCK_ALIGNMENT)
////////////////////////////////////////////////////////////////////////////////
/// @brief checks a CRC of a marker
////////////////////////////////////////////////////////////////////////////////

4
arangod/VocBase/document-collection.c
View File

@ -1138,7 +1138,7 @@ static int CreateShapedJson (TRI_doc_operation_context_t* context,
keySize += 1;
keyBodySize = ((keySize + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
keyBodySize = TRI_DF_ALIGN_BLOCK(keySize);
keyBody = TRI_Allocate(TRI_CORE_MEM_ZONE, keyBodySize, true);
TRI_CopyString(keyBody, (char*) &keyBuffer, keySize);
@ -1190,7 +1190,7 @@ static int CreateShapedJson (TRI_doc_operation_context_t* context,
keySize += 1;
keyBodySize = ((keySize + fromSize + toSize + TRI_DF_BLOCK_ALIGN - 1) / TRI_DF_BLOCK_ALIGN) * TRI_DF_BLOCK_ALIGN;
keyBodySize = TRI_DF_ALIGN_BLOCK(keySize + fromSize + toSize);
keyBody = TRI_Allocate(TRI_CORE_MEM_ZONE, keyBodySize, true);
TRI_CopyString(keyBody, (char*) &keyBuffer, keySize);

5
arangod/VocBase/primary-collection.c
View File

@ -139,6 +139,7 @@ static TRI_datafile_t* CreateJournal (TRI_primary_collection_t* primary, bool co
// create journal file
journal = TRI_CreateDatafile(filename, collection->_info._maximalSize);
if (journal == NULL) {
if (TRI_errno() == TRI_ERROR_OUT_OF_MEMORY_MMAP) {
collection->_lastError = TRI_set_errno(TRI_ERROR_OUT_OF_MEMORY_MMAP);
@ -149,14 +150,14 @@ static TRI_datafile_t* CreateJournal (TRI_primary_collection_t* primary, bool co
collection->_state = TRI_COL_STATE_WRITE_ERROR;
}
LOG_ERROR("cannot create new journal in '%s'", filename);
LOG_ERROR("cannot create new primary journal in '%s'", filename);
TRI_FreeString(TRI_CORE_MEM_ZONE, filename);
return NULL;
}
TRI_FreeString(TRI_CORE_MEM_ZONE, filename);
LOG_TRACE("created a new journal '%s'", journal->_filename);
LOG_TRACE("created a new primary journal '%s'", journal->_filename);
// and use the correct name
number = TRI_StringUInt32(journal->_fid);

9
arangod/VocBase/shape-collection.c
View File

@ -62,7 +62,8 @@ static bool CreateJournal (TRI_shape_collection_t* collection) {
jname = TRI_Concatenate3String("journal-", number, ".db");
TRI_FreeString(TRI_CORE_MEM_ZONE, number);
filename = TRI_Concatenate2File(collection->base._directory, jname);
TRI_FreeString(TRI_CORE_MEM_ZONE, jname);
@ -79,14 +80,14 @@ static bool CreateJournal (TRI_shape_collection_t* collection) {
collection->base._state = TRI_COL_STATE_WRITE_ERROR;
}
LOG_ERROR("cannot create new journal '%s': %s", filename, TRI_last_error());
LOG_ERROR("cannot create new shape journal '%s': %s", filename, TRI_last_error());
TRI_FreeString(TRI_CORE_MEM_ZONE, filename);
return false;
}
TRI_FreeString(TRI_CORE_MEM_ZONE, filename);
LOG_TRACE("created a new journal '%s'", journal->_filename);
LOG_TRACE("created a new shape journal '%s'", journal->_filename);
// and use the correct name
number = TRI_StringUInt32(journal->_fid);
@ -100,6 +101,7 @@ static bool CreateJournal (TRI_shape_collection_t* collection) {
if (! ok) {
LOG_WARNING("failed to rename the journal to '%s': %s", filename, TRI_last_error());
exit(1);
}
else {
LOG_TRACE("renamed journal to '%s'", filename);
@ -399,6 +401,7 @@ int TRI_WriteShapeCollection (TRI_shape_collection_t* collection,
TRI_datafile_t* journal;
int res;
// generate a new tick
marker->_tick = TRI_NewTickVocBase();

2
arangod/VocBase/synchroniser.c
View File

@ -349,6 +349,7 @@ void TRI_SynchroniserVocBase (void* data) {
TRI_InitVectorPointer(&collections, TRI_UNKNOWN_MEM_ZONE);
while (true) {
size_t n;
size_t i;
@ -424,6 +425,7 @@ void TRI_SynchroniserVocBase (void* data) {
if (state == 2) {
break;
}
}
TRI_DestroyVectorPointer(&collections);

4
arangod/VocBase/voc-shaper.c
View File

@ -404,7 +404,6 @@ static TRI_shape_aid_t FindAttributeName (TRI_shaper_t* shaper, char const* name
weightedAttribute = TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, sizeof(attribute_weight_t), false);
if (weightedAttribute != NULL) {
weightedAttribute->_aid = markerResult->_aid;
weightedAttribute->_weight = TRI_VOC_UNDEFINED_ATTRIBUTE_WEIGHT;
weightedAttribute->_attribute = (char*)(markerResult) + sizeof(TRI_df_attribute_marker_t);
@ -569,6 +568,7 @@ static TRI_shape_t const* FindShape (TRI_shaper_t* shaper, TRI_shape_t* shape) {
voc_shaper_t* s;
void* f;
s = (voc_shaper_t*) shaper;
found = TRI_LookupByElementAssociativeSynced(&s->_shapeDictionary, shape);
@ -925,8 +925,10 @@ TRI_shaper_t* TRI_CreateVocShaper (TRI_vocbase_t* vocbase,
// override wait for sync for shapes
parameter._waitForSync = waitForSync;
collection = TRI_CreateShapeCollection(vocbase, path, &parameter);
if (collection == NULL) {
return NULL;
}

35
arangod/VocBase/vocbase.c
View File

@ -353,9 +353,21 @@ static bool DropCollectionCallback (TRI_collection_t* col, void* data) {
// .............................................................................
if (collection->_path != NULL) {
regcomp(&re, "^(.*)/collection-([0-9][0-9]*)$", REG_EXTENDED);
int regExpResult;
if (regexec(&re, collection->_path, sizeof(matches) / sizeof(matches[0]), matches, 0) == 0) {
#ifdef _WIN32
// .........................................................................
// Just thank your lucky stars that there are only 4 backslashes
// .........................................................................
regcomp(&re, "^(.*)\\\\collection-([0-9][0-9]*)$", REG_ICASE | REG_EXTENDED);
#else
regcomp(&re, "^(.*)/collection-([0-9][0-9]*)$", REG_ICASE | REG_EXTENDED);
#endif
regExpResult = regexec(&re, collection->_path, sizeof(matches) / sizeof(matches[0]), matches, 0);
if (regExpResult == 0) {
char const* first = collection->_path + matches[1].rm_so;
size_t firstLen = matches[1].rm_eo - matches[1].rm_so;
@ -1073,9 +1085,11 @@ bool TRI_msync (int fd, void* mmHandle, char const* begin, char const* end) {
uintptr_t g = (intptr_t) PageSize;
char* b = (char*)( (p / g) * g );
char* e = (char*)( ((q + g - 1) / g) * g );
char* e = (char*)( ((q + g - 1) / g) * g );
int result;
result = TRI_FlushMMFile(fd, &mmHandle, b, e - b, MS_SYNC);
int result = TRI_FlushMMFile(&fd, &mmHandle, b, e - b, MS_SYNC);
if (result != TRI_ERROR_NO_ERROR) {
TRI_set_errno(result);
return false;
@ -1223,6 +1237,7 @@ TRI_vocbase_t* TRI_OpenVocBase (char const* path) {
return NULL;
}
// .............................................................................
// vocbase is now active
// .............................................................................
@ -1329,6 +1344,7 @@ void TRI_DestroyVocBase (TRI_vocbase_t* vocbase) {
// free the filename path
TRI_Free(TRI_CORE_MEM_ZONE, vocbase->_path);
}
////////////////////////////////////////////////////////////////////////////////
@ -1432,6 +1448,7 @@ TRI_vocbase_col_t* TRI_CreateCollectionVocBase (TRI_vocbase_t* vocbase,
assert(parameter);
name = parameter->_name;
// check that the name does not contain any strange characters
if (! TRI_IsAllowedCollectionName(parameter->_isSystem, name)) {
TRI_set_errno(TRI_ERROR_ARANGO_ILLEGAL_NAME);
@ -1480,13 +1497,15 @@ TRI_vocbase_col_t* TRI_CreateCollectionVocBase (TRI_vocbase_t* vocbase,
col = &primary->base;
// add collection container
// add collection container -- however note that the compactor is added later which could fail!
collection = AddCollection(vocbase,
col->_info._type,
col->_info._name,
col->_info._cid,
col->_directory);
if (collection == NULL) {
if (TRI_IS_DOCUMENT_COLLECTION(type)) {
TRI_CloseDocumentCollection(document);
@ -1497,11 +1516,13 @@ TRI_vocbase_col_t* TRI_CreateCollectionVocBase (TRI_vocbase_t* vocbase,
return NULL;
}
collection->_status = TRI_VOC_COL_STATUS_LOADED;
collection->_collection = primary;
FreeCollectionPath(collection);
collection->_path = TRI_DuplicateString(primary->base._directory);
TRI_WRITE_UNLOCK_COLLECTIONS_VOCBASE(vocbase);
return collection;
}
@ -1919,7 +1940,9 @@ void TRI_InitialiseVocBase () {
#ifdef TRI_ICU_VERSION
LOG_TRACE("%s", "$Revision: ICU " TRI_ICU_VERSION " $");
#endif
LOG_TRACE("sizeof df_header: %d", (int) sizeof(TRI_df_marker_t));
LOG_TRACE("sizeof df_header_marker: %d", (int) sizeof(TRI_df_header_marker_t));
}
////////////////////////////////////////////////////////////////////////////////

4
arangosh/V8Client/ImportHelper.cpp
View File

@ -149,9 +149,9 @@ namespace triagens {
}
parser._dataAdd = this;
char buffer[10240];
char buffer[16384];
while (!_hasError) {
while (! _hasError) {
v8::HandleScope scope;
ssize_t n = read(fd, buffer, sizeof(buffer));

5
arangosh/V8Client/arangosh.cpp
View File

@ -1,4 +1,3 @@
////////////////////////////////////////////////////////////////////////////////
/// @brief V8 shell
///
/// @file
@ -985,10 +984,10 @@ static bool RunScripts (v8::Handle<v8::Context> context,
}
if (execute) {
TRI_ExecuteJavaScriptFile(context, scripts[i].c_str());
TRI_ExecuteLocalJavaScriptFile(scripts[i].c_str());
}
else {
TRI_LoadJavaScriptFile(context, scripts[i].c_str());
TRI_ParseJavaScriptFile(scripts[i].c_str());
}
if (tryCatch.HasCaught()) {

5
build.h Executable file
View File

@ -0,0 +1,5 @@
#ifdef _WIN32
#include "build_win.h"
#else
#include "build_posix.h"
#endif

1
build_posix.h Normal file
View File

@ -0,0 +1 @@
#define TRIAGENS_VERSION "1.1.1"

13
build_win.h Executable file
View File

@ -0,0 +1,13 @@
#ifdef _WIN64
#ifdef _DEBUG
#define TRIAGENS_VERSION "1.1.1 [WIN64-DEBUG BETA]"
#else
#define TRIAGENS_VERSION "1.1.1 [WIN64-RELEASE BETA]"
#endif
#else
#ifdef _DEBUG
#define TRIAGENS_VERSION "1.1.1 [WIN32-DEBUG BETA]"
#else
#define TRIAGENS_VERSION "1.1.1 [WIN32-RELEASE BETA]"
#endif
#endif

3
config/generateErrorfile.py
View File

@ -43,8 +43,9 @@ def genJsFile(errors):
msg = e[2].replace("\n", " ").replace("\\", "").replace("\"", "\\\"")
out = out\
+ " " + name.ljust(30) + " : { \"code\" : " + e[1] + ", \"message\" : \"" + msg + "\" }"
i = i + 1
if i < len(errors):
out = out + ", \n"
else:

2
configure.ac
View File

@ -193,7 +193,7 @@ dnl ============================================================================
dnl --SECTION-- GENERATE FILES
dnl ============================================================================
BUILD_H="\$(top_srcdir)/build.h"
BUILD_H="\$(top_srcdir)/build_posix.h"
AC_SUBST(BUILD_H)
AC_CONFIG_FILES([Makefile Documentation/arango.template])

39
js/common/tests/shell-edge.js
View File

@ -103,7 +103,6 @@ function CollectionEdgeSuiteErrorHandling () {
assertEqual(ERRORS.ERROR_ARANGO_DOCUMENT_HANDLE_BAD.code, err.errorNum);
}
try {
edge.save(v1, "1234/56/46", {});
fail();
@ -431,44 +430,6 @@ function CollectionEdgeSuite () {
assertEqual(ERRORS.ERROR_ARANGO_DOCUMENT_HANDLE_BAD.code, err.errorNum);
}
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test rollback of edge insert
////////////////////////////////////////////////////////////////////////////////
testRollbackEdgeInsert : function () {
var v1 = vertex.save({ "x" : 1 });
var v2 = vertex.save({ "x" : 2 });
edge.ensureUniqueConstraint("myid");
edge.save(v1, v2, { "myid" : 1 });
try {
edge.save(v1, v2, { "myid" : 1 });
}
catch (err) {
assertEqual(ERRORS.ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATED.code, err.errorNum);
}
},
////////////////////////////////////////////////////////////////////////////////
/// @brief test rollback of edge update
////////////////////////////////////////////////////////////////////////////////
testRollbackEdgeUpdate : function () {
var v1 = vertex.save({ "x" : 1 });
var v2 = vertex.save({ "x" : 2 });
edge.ensureUniqueConstraint("myid");
edge.save(v1, v2, { "myid" : 1 });
var e2 = edge.save(v1, v2, { "myid" : 2 });
try {
edge.update(e2, { "myid" : 1 });
}
catch (err) {
assertEqual(ERRORS.ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATED.code, err.errorNum);
}
},
////////////////////////////////////////////////////////////////////////////////
/// @brief invalid collection type for edges

6
js/server/arango-dfdb.js
View File

@ -331,6 +331,10 @@ function main (argv) {
while (true) {
var line = console.getline();
if (line == "") {
break;
}
if (line === "*") {
for (i = 0; i < collections.length; ++i) {
a.push(i);
@ -341,7 +345,7 @@ function main (argv) {
else {
var l = parseInt(line);
if (l < 0 || l >= collections.length) {
if (l < 0 || l >= collections.length || l === null || l === undefined || isNaN(l)) {
printf("Please select a number between 0 and %d: ", collections.length - 1);
}
else {

51
js/server/modules/org/arangodb/new 2.txt Normal file
View File

@ -0,0 +1,51 @@
/I"..\..\lib"
/I"..\"
/I"..\..\"
/I"..\..\VisualStudio"
/I"..\..\3rdParty\VisualStudio\openssl\x86\include"
/I"..\..\3rdParty\VisualStudio\mygetopt"
/I"..\..\3rdParty\VisualStudio\regex-2.7\src"
/I"..\..\3rdParty\VisualStudio\readline-5.0-1-lib\include"
/I"..\..\arangod"
/I"..\..\3rdParty\VisualStudio\V8\include"
/Zi (debug information format)
/nologo (suppress startup banner)
/W3 (warnings)
/WX- (warnings)
/O2 (optimise)
/Oi
/Oy-
/GL
/D "WIN32"
/D "NDEBUG"
/D "_CONSOLE"
/D "_WIN32"
/D "YY_NO_UNISTD_H"
/D "_UNICODE"
/D "UNICODE"
/D ")"
/Gm-
/EHsc
/MT
/GS !!!!!!!!!!!!!!
/Gy ?????????????
/fp:precise
/Zc:wchar_t ???????????????????
/Zc:forScope !!!!!!!!!!!!!!!!!!
/Gd ???????????????????
/analyze-
/errorReport:queue
/Fp"Release\arangod.pch"
/Fa"Release\"
/Fo"Release\"
/Fd"Release\vc100.pdb"

3
js/server/version-check.js
View File

@ -95,11 +95,14 @@
if (FS_EXISTS(versionFile)) {
// VERSION file exists, read its contents
var versionInfo = SYS_READ(versionFile);
if (versionInfo != '') {
var versionValues = JSON.parse(versionInfo);
if (versionValues && versionValues.version && ! isNaN(versionValues.version)) {
lastVersion = parseFloat(versionValues.version);
}
if (versionValues && versionValues.tasks && typeof(versionValues.tasks) === 'object') {
lastTasks = versionValues.tasks || { };
}

19
lib/ApplicationServer/ApplicationServer.cpp
View File

@ -284,7 +284,6 @@ void ApplicationServer::setupLogging () {
// the user specified a log file to use but it could not be created. bail out
std::cerr << "failed to create logfile '" << _logFile << "'. Please check the path and permissions." << std::endl;
exit(EXIT_FAILURE);
}
}
#ifdef TRI_ENABLE_SYSLOG
@ -384,6 +383,7 @@ bool ApplicationServer::parse (int argc,
// UID and GID
// .............................................................................
storeRealPrivileges();
extractPrivileges();
dropPrivileges();
@ -474,7 +474,10 @@ bool ApplicationServer::parse (int argc,
void ApplicationServer::prepare () {
// prepare all features
// prepare all features - why reverse?
// reason: features might depend on each other. by creating them in reverse order and shutting them
// down in forward order, we ensure that the features created last are destroyed first, i.e.: LIFO
for (vector<ApplicationFeature*>::reverse_iterator i = _features.rbegin(); i != _features.rend(); ++i) {
ApplicationFeature* feature = *i;
@ -575,13 +578,11 @@ void ApplicationServer::wait () {
// wait until we receive a stop signal
while (running && _stopping == 0) {
// check the parent and wait for a second
if (! checkParent()) {
running = false;
break;
}
sleep(1);
}
}
@ -594,7 +595,6 @@ void ApplicationServer::beginShutdown () {
if (_stopping != 0) {
return;
}
_stopping = 1;
}
@ -605,6 +605,7 @@ void ApplicationServer::beginShutdown () {
void ApplicationServer::stop () {
beginShutdown();
// close all features
for (vector<ApplicationFeature*>::iterator i = _features.begin(); i != _features.end(); ++i) {
ApplicationFeature* feature = *i;
@ -614,6 +615,7 @@ void ApplicationServer::stop () {
LOGGER_TRACE << "closed server feature '" << feature->getName() << "'";
}
// stop all features
for (vector<ApplicationFeature*>::reverse_iterator i = _features.rbegin(); i != _features.rend(); ++i) {
ApplicationFeature* feature = *i;
@ -624,6 +626,7 @@ void ApplicationServer::stop () {
LOGGER_TRACE << "shut down server feature '" << feature->getName() << "'";
}
}
////////////////////////////////////////////////////////////////////////////////
@ -901,7 +904,8 @@ bool ApplicationServer::readConfigurationFile () {
LOGGER_DEBUG << "no init file has been specified";
}
// nothing has been specified on the command line regarding configuration file
// nothing has been specified on the command line regarding the user's configuration file
if (! _userConfigFile.empty()) {
// .........................................................................
@ -975,7 +979,9 @@ bool ApplicationServer::readConfigurationFile () {
return ok;
}
else {
LOGGER_INFO << "no user init file '" << homeDir << "' found";
}
}
else {
@ -983,6 +989,7 @@ bool ApplicationServer::readConfigurationFile () {
}
}
if (_systemConfigPath.empty()) {
#ifdef _SYSCONFDIR_

4
lib/Basics/Common.h
View File

@ -30,6 +30,10 @@
#include "BasicsC/common.h"
#ifdef _WIN32
#include "BasicsC/win-utils.h"
#endif
////////////////////////////////////////////////////////////////////////////////
/// @addtogroup Configuration Configuration
/// @{

10
lib/Basics/FileUtils.cpp
View File

@ -328,8 +328,14 @@ namespace triagens {
bool isSymbolicLink (string const& path) {
#ifdef TRI_HAVE_WIN32_SYMBOLIC_LINK
return true;
// .........................................................................
// TODO: On the NTFS file system, there are the following file links:
// hard links -
// junctions -
// symbolic links -
// .........................................................................
return false;
#else

14
lib/Basics/RandomGenerator.cpp
View File

@ -245,8 +245,10 @@ namespace RandomHelper {
#else
public:
RandomDeviceWin32 () : cryptoHandle(0), pos(0) {
CryptAcquireContext(&cryptoHandle, NULL, NULL, PROV_RSA_FULL, 0);
if (cryptoHandle == 0) {
BOOL result;
result = CryptAcquireContext(&cryptoHandle, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT | CRYPT_SILENT);
if (cryptoHandle == 0 || result == FALSE) {
printf("%s:%s:%d:%d",__FILE__,__FUNCTION__,__LINE__,GetLastError());
THROW_INTERNAL_ERROR("cannot create cryptographic windows handle");
}
fillBuffer();
@ -255,8 +257,8 @@ namespace RandomHelper {
~RandomDeviceWin32 () {
if (cryptoHandle != 0) {
CryptReleaseContext(cryptoHandle, 0);
}
CryptReleaseContext(cryptoHandle, 0);
}
}
@ -274,8 +276,8 @@ namespace RandomHelper {
BYTE* ptr = reinterpret_cast<BYTE*>(&buffer);
// fill the buffer with random characters
int result = CryptGenRandom(cryptoHandle, n, ptr);
if (result == 0) {
int result = CryptGenRandom(cryptoHandle, n, ptr);
if (result == 0) {
LOGGER_FATAL << "read on random device failed: nothing read";
THROW_INTERNAL_ERROR("read on random device failed");
}

2
lib/Basics/ReadWriteLock.h
View File

@ -145,7 +145,7 @@ namespace triagens {
////////////////////////////////////////////////////////////////////////////////
private:
////////////////////////////////////////////////////////////////////////////////
/// @brief read-write lock variable
////////////////////////////////////////////////////////////////////////////////

8
lib/Basics/StringUtils.cpp
View File

@ -3197,6 +3197,14 @@ namespace triagens {
// ADDITIONAL STRING UTILITIES
// -----------------------------------------------------------------------------
string correctPath(const string& incorrectPath) {
#ifdef _WIN32
return replace (incorrectPath, "/", "\\");
#else
return replace (incorrectPath, "\\", "/");
#endif
}
/// In a list str = "xx,yy,zz ...", entry(n,str,',') returns the nth entry of the list delimited
/// by ','. E.g entry(2,str,',') = 'yy'

7
lib/Basics/StringUtils.h
View File

@ -740,6 +740,13 @@ namespace triagens {
// ADDITIONAL STRING UTILITIES
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
/// @ingroup Utilities
/// @brief replaces incorrect path delimiter character for window and linux
////////////////////////////////////////////////////////////////////////////////
string correctPath (const string& incorrectPath);
////////////////////////////////////////////////////////////////////////////////
/// @ingroup Utilities
/// @brief finds n.th entry

9
lib/BasicsC/common.h
View File

@ -105,15 +105,24 @@
#include <sys/time.h>
#endif
#ifdef TRI_HAVE_UNISTD_H
#include <unistd.h>
#endif
// .................................................................................................
// The problem we have for visual studio is that if we include WinSock2.h here it may conflict later
// in some other source file. The conflict arises when windows.h is included BEFORE WinSock2.h --
// this is a visual studio issue. For now be VERY careful to ensure that if you need windows.h, then
// you include this file AFTER common.h.
// .................................................................................................
#ifdef TRI_HAVE_WINSOCK2_H
#include <WinSock2.h>
typedef long suseconds_t;
#endif
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////

126
lib/BasicsC/files.c Normal file → Executable file
View File

@ -523,8 +523,20 @@ TRI_vector_string_t TRI_FilesDirectory (char const* path) {
int TRI_RenameFile (char const* old, char const* filename) {
int res;
#ifdef _WIN32
BOOL moveResult = 0;
moveResult = MoveFileExA(old, filename, MOVEFILE_COPY_ALLOWED | MOVEFILE_REPLACE_EXISTING);
if (!moveResult) {
DWORD errorCode = GetLastError();
res = -1;
}
else {
res = 0;
}
#else
res = rename(old, filename);
#endif
if (res != 0) {
LOG_TRACE("cannot rename file from '%s' to '%s': %s", old, filename, TRI_LAST_ERROR_STR);
@ -560,9 +572,11 @@ bool TRI_ReadPointer (int fd, void* buffer, size_t length) {
ptr = buffer;
while (0 < length) {
ssize_t n = TRI_READ(fd, ptr, length);
if (n < 0) {
TRI_set_errno(TRI_ERROR_SYS_ERROR);
LOG_ERROR("cannot read: %s", TRI_LAST_ERROR_STR);
@ -995,6 +1009,7 @@ int TRI_DestroyLockFile (char const* filename) {
return false;
}
fd = TRI_OPEN(filename, O_RDWR);
// ..........................................................................
@ -1063,20 +1078,110 @@ int TRI_DestroyLockFile (char const* filename) {
/// for those cases.
/// It is the caller's responsibility to free the string created by this
/// function
/// TODO: the implementation is naive and may be simplified greatly on Windows
/// by using GetFullPathName()
////////////////////////////////////////////////////////////////////////////////
#ifdef _WIN32
char* TRI_GetAbsolutePath (char const* fileName, char const* currentWorkingDirectory) {
char* result;
char* ptr;
size_t cwdLength;
size_t fileLength;
bool ok;
// ...........................................................................
// Check that fileName actually makes some sense
// ...........................................................................
if (fileName == NULL || *fileName == '\0') {
return NULL;
}
// ...........................................................................
// Under windows we can assume that fileName is absolute if fileName starts
// with a letter A-Z followed by a colon followed by either a forward or
// backslash.
// ...........................................................................
if ((fileName[0] > 64 && fileName[0] < 91) || (fileName[0] > 96 && fileName[0] < 123)) {
if ((fileName[1] != '\0') && (fileName[1] == ':')) {
if ((fileName[2] != '\0') && (fileName[2] == '/' || fileName[2] == '\\')) {
return TRI_DuplicateStringZ(TRI_UNKNOWN_MEM_ZONE, fileName);
}
}
}
// ...........................................................................
// The fileName itself was not absolute, so we attempt to amalagmate the
// currentWorkingDirectory with the fileName
// ...........................................................................
// ...........................................................................
// Check that the currentWorkingDirectory makes sense
// ...........................................................................
if (currentWorkingDirectory == NULL || *currentWorkingDirectory == '\0') {
return NULL;
}
// ...........................................................................
// Under windows the currentWorkingDirectory should start
// with a letter A-Z followed by a colon followed by either a forward or
// backslash.
// ...........................................................................
ok = false;
if ((currentWorkingDirectory[0] > 64 && currentWorkingDirectory[0] < 91) || (currentWorkingDirectory[0] > 96 && currentWorkingDirectory[0] < 123)) {
if ((currentWorkingDirectory[1] != '\0') && (currentWorkingDirectory[1] == ':')) {
if ((currentWorkingDirectory[2] != '\0') && (currentWorkingDirectory[2] == '/' || currentWorkingDirectory[2] == '\\')) {
ok = true;
}
}
}
if (!ok) {
return NULL;
}
// ...........................................................................
// Determine the total legnth of the new string
// ...........................................................................
cwdLength = strlen(currentWorkingDirectory);
fileLength = strlen(fileName);
if (currentWorkingDirectory[cwdLength - 1] != '\\' && currentWorkingDirectory[cwdLength - 1] != '/') {
// we do not require a backslash
result = TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, (cwdLength + fileLength + 1) * sizeof(char), false);
memcpy(result, currentWorkingDirectory, cwdLength);
memcpy(result + cwdLength, fileName, fileLength);
result[cwdLength + fileLength] = '\0';
}
else {
// we do require a backslash
result = TRI_Allocate(TRI_UNKNOWN_MEM_ZONE, (cwdLength + fileLength + 2) * sizeof(char), false);
memcpy(result, currentWorkingDirectory, cwdLength);
result[cwdLength] = '\\';
memcpy(result + cwdLength + 1, fileName, fileLength);
result[cwdLength + fileLength + 1] = '\0';
}
return result;
}
#else
char* TRI_GetAbsolutePath (char const* file, char const* cwd) {
char* result;
char* ptr;
size_t cwdLength;
bool isAbsolute;
#ifdef _WIN32
#error please validate if this works on Windows
#endif
if (file == NULL || *file == '\0') {
return NULL;
}
@ -1110,15 +1215,9 @@ char* TRI_GetAbsolutePath (char const* file, char const* cwd) {
memcpy(ptr, cwd, cwdLength);
ptr += cwdLength;
#ifdef _WIN32
if (cwd[cwdLength - 1] != '\\') {
*(ptr++) = '\\';
}
#else
if (cwd[cwdLength - 1] != '/') {
*(ptr++) = '/';
}
#endif
memcpy(ptr, file, strlen(file));
ptr += strlen(file);
*ptr = '\0';
@ -1126,6 +1225,7 @@ char* TRI_GetAbsolutePath (char const* file, char const* cwd) {
return result;
}
#endif
////////////////////////////////////////////////////////////////////////////////
/// @brief locates the directory containing the program

6
lib/BasicsC/files.h
View File

@ -28,6 +28,10 @@
#ifndef TRIAGENS_BASICS_C_FILES_H
#define TRIAGENS_BASICS_C_FILES_H 1
#ifdef _WIN32
#include "BasicsC/win-utils.h"
#endif
#include "BasicsC/common.h"
#include "BasicsC/vector.h"
@ -49,7 +53,7 @@ extern "C" {
/// @brief sets close-on-exit for a socket
////////////////////////////////////////////////////////////////////////////////
bool TRI_SetCloseOnExecFile (int fd);
bool TRI_SetCloseOnExecFile (socket_t fd);
////////////////////////////////////////////////////////////////////////////////
/// @brief returns the size of a file

1
lib/BasicsC/json.c
View File

@ -26,7 +26,6 @@
////////////////////////////////////////////////////////////////////////////////
#include "json.h"
#include "BasicsC/files.h"
#include "BasicsC/logging.h"
#include "BasicsC/string-buffer.h"

2
lib/BasicsC/locks-posix.c
View File

@ -355,7 +355,7 @@ void TRI_WriteLockReadWriteLock (TRI_read_write_lock_t* lock) {
rc = pthread_rwlock_wrlock(lock);
if (rc != 0) {
LOG_ERROR("could not read-lock the read-write lock: %s", strerror(rc));
LOG_ERROR("could not write-lock the read-write lock: %s", strerror(rc));
if (rc == EDEADLK) {
LOG_ERROR("rw-lock deadlock detected");
}

165
lib/BasicsC/locks-win32.c
View File

@ -186,18 +186,41 @@ void TRI_UnlockSpin (TRI_spin_t* spin) {
////////////////////////////////////////////////////////////////////////////////
void TRI_InitReadWriteLock (TRI_read_write_lock_t* lock) {
// ...........................................................................
// set the number of readers reading on the read_write lock to 0
// ...........................................................................
lock->_readers = 0;
// ...........................................................................
// Signaled: writer has no access
// Non-Signaled: writer has access, block readers
// Creates an event which allows a thread to wait on it (perhaps should use
// a mutux rather than an event here). The writer event is set to signalled
// when CreateEvent is called with these parameters.
// ...........................................................................
lock->_writerEvent = CreateEvent(0, TRUE, TRUE, 0);
// ...........................................................................
// Signaled: no readers
// Non-Signaled: some readers have access, block writer
// Same as the writer event above except this is the reader event
// ...........................................................................
lock->_readersEvent = CreateEvent(0, TRUE, TRUE, 0);
// ...........................................................................
// Creates critical sections for writer and readers.
// Waits for ownership of the specified critical section object.
// The function returns when the calling thread is granted ownership.
// ...........................................................................
InitializeCriticalSection(&lock->_lockWriter);
InitializeCriticalSection(&lock->_lockReaders);
}
@ -232,8 +255,18 @@ void TRI_DestroyReadWriteLock (TRI_read_write_lock_t* lock) {
////////////////////////////////////////////////////////////////////////////////
static void IncrementReaders (TRI_read_write_lock_t* lock) {
// ...........................................................................
// increment the number of readers we have on the read_write lock
// ...........................................................................
lock->_readers++;
// ...........................................................................
// Since the number of readers must be positive, set the readers event to
// non-signalled so that any write event will have to wait.
// ...........................................................................
ResetEvent(lock->_readersEvent);
}
@ -242,8 +275,19 @@ static void IncrementReaders (TRI_read_write_lock_t* lock) {
////////////////////////////////////////////////////////////////////////////////
static void DecrementReaders (TRI_read_write_lock_t* lock) {
// ...........................................................................
// reduce the number of readers using the read_write lock by 1
// ...........................................................................
lock->_readers--;
// ...........................................................................
// When the number of readers is 0, set the event to signalled which allows
// a writer to use the read_write lock.
// ...........................................................................
if (lock->_readers == 0) {
SetEvent(lock->_readersEvent);
}
@ -292,15 +336,35 @@ bool TRI_TryReadLockReadWriteLock (TRI_read_write_lock_t* lock) {
////////////////////////////////////////////////////////////////////////////////
void TRI_ReadLockReadWriteLock (TRI_read_write_lock_t* lock) {
while (true) {
// ........................................................................
// Waits for a writer to finish if there is one. This function only
// returns when the writer event is in a signalled state
// ........................................................................
WaitForSingleObject(lock->_writerEvent, INFINITE);
// .........................................................................
// This thread will wait here until this resource becomes excusively available
// .........................................................................
EnterCriticalSection(&lock->_lockReaders);
IncrementReaders(lock);
// .........................................................................
// allows some other thread to use this resource
// .........................................................................
LeaveCriticalSection(&lock->_lockReaders);
// it could have happened that the writer event is no longer in a signalled
// state. Between leaving the crtical section and here a writer sneaked in.
//
if (WaitForSingleObject(lock->_writerEvent, 0) != WAIT_OBJECT_0) {
exit(EXIT_FAILURE);
EnterCriticalSection(&lock->_lockReaders);
DecrementReaders(lock);
LeaveCriticalSection(&lock->_lockReaders);
@ -318,12 +382,13 @@ void TRI_ReadLockReadWriteLock (TRI_read_write_lock_t* lock) {
void TRI_ReadUnlockReadWriteLock (TRI_read_write_lock_t* lock) {
EnterCriticalSection(&lock->_lockReaders);
/* this is wrong since it is possible for the write locker to block this event
// a write lock eists
if (WaitForSingleObject(lock->_writerEvent, 0) != WAIT_OBJECT_0) {
LOG_FATAL("write lock, but trying to unlock read");
exit(EXIT_FAILURE);
}
// at least one reader exists
else if (0 < lock->_readers) {
DecrementReaders(lock);
@ -331,14 +396,28 @@ void TRI_ReadUnlockReadWriteLock (TRI_read_write_lock_t* lock) {
// ups, no writer and no reader
else {
LeaveCriticalSection(&lock->_lockWriter);
LeaveCriticalSection(&lock->_lockReaders);
LOG_FATAL("no reader and no writer, but trying to unlock");
exit(EXIT_FAILURE);
}
*/
if (0 < lock->_readers) {
DecrementReaders(lock);
}
// oops no reader
else {
LeaveCriticalSection(&lock->_lockReaders);
LOG_FATAL("no reader, but trying to unlock read lock");
exit(EXIT_FAILURE);
}
LeaveCriticalSection(&lock->_lockReaders);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief tries to write lock a read-write lock
/// TODO: not yet implemented
@ -353,44 +432,101 @@ bool TRI_TryWriteLockReadWriteLock (TRI_read_write_lock_t* lock) {
////////////////////////////////////////////////////////////////////////////////
void TRI_WriteLockReadWriteLock (TRI_read_write_lock_t* lock) {
// ...........................................................................
// Lock so no other thread can access this
// EnterCriticalSection(&lock->_lockWriter) will block this thread until
// it has been released by the other thread.
// ...........................................................................
EnterCriticalSection(&lock->_lockWriter);
// ...........................................................................
// Wait until the lock->_writerEvent is in a 'signalled' state
// ...........................................................................
WaitForSingleObject(lock->_writerEvent, INFINITE);
ResetEvent(lock->_writerEvent);
// ...........................................................................
// Set _writeEvent as nonsignalled -- this will block other read/write
// lockers
// ...........................................................................
ResetEvent(lock->_writerEvent);
// ...........................................................................
// If there are ANY read locks outstanding, then wait until these are cleared
// ...........................................................................
WaitForSingleObject(lock->_readersEvent, INFINITE);
// ...........................................................................
// Allow other threads to access this function
// ...........................................................................
LeaveCriticalSection(&lock->_lockWriter);
}
////////////////////////////////////////////////////////////////////////////////
/// @brief write unlocks read-write lock
////////////////////////////////////////////////////////////////////////////////
void TRI_WriteUnlockReadWriteLock (TRI_read_write_lock_t* lock) {
//printf("%s:%s:%s:%d:%s:%d\n","oreste",__FILE__,__FUNCTION__,__LINE__,"!!!!!!!!!!!!!!!!!",(uint64_t)(lock));
// ...........................................................................
// Write lock this _lockReader so no other threads can access this
// This will block this thread until it is released by the other thread
// We do not need to lock the _lockWriter SINCE the TRI_WriteLockReadWriteLock
// function above will lock (due to the ResetEvent(lock->_writerEvent); )
// ...........................................................................
EnterCriticalSection(&lock->_lockReaders);
// a write lock eists
// ...........................................................................
// In the function TRI_WriteLockReadWriteLock we set the _writerEvent to
// 'nonsignalled'. So if a write lock exists clear it by setting it to
// 'signalled'
// ...........................................................................
if (WaitForSingleObject(lock->_writerEvent, 0) != WAIT_OBJECT_0) {
SetEvent(lock->_writerEvent);
}
// at least one reader exists
// ...........................................................................
// Oops at least one reader exists - something terrible happened.
// ...........................................................................
else if (0 < lock->_readers) {
LeaveCriticalSection(&lock->_lockReaders);
LOG_FATAL("read lock, but trying to unlock write");
exit(EXIT_FAILURE);
}
// ups, no writer and no reader
// ...........................................................................
// Oops we are trying to unlock a write lock, but there isn't one! Something
// terrible happend.
// ...........................................................................
else {
LeaveCriticalSection(&lock->_lockWriter);
LeaveCriticalSection(&lock->_lockReaders);
LOG_FATAL("no reader and no writer, but trying to unlock");
exit(EXIT_FAILURE);
}
LeaveCriticalSection(&lock->_lockWriter);
// ...........................................................................
// Allow read locks to be applied now.
// ...........................................................................
LeaveCriticalSection(&lock->_lockReaders);
}
////////////////////////////////////////////////////////////////////////////////
@ -596,7 +732,7 @@ void TRI_WaitCondition (TRI_condition_t* cond) {
}
////////////////////////////////////////////////////////////////////////////////
/// @brief waits for a signal with a timeout in milli-seconds
/// @brief waits for a signal with a timeout in micro-seconds
///
/// Note that you must hold the lock.
////////////////////////////////////////////////////////////////////////////////
@ -605,6 +741,13 @@ bool TRI_TimedWaitCondition (TRI_condition_t* cond, uint64_t delay) {
bool lastWaiter;
DWORD res;
// ...........................................................................
// The POSIX threads function pthread_cond_timedwait accepts microseconds
// while the the function SignalObjectAndWait accepts milliseconds
// ...........................................................................
delay = delay / 1000;
// avoid race conditions
EnterCriticalSection(&cond->_lockWaiters);
cond->_waiters++;

5
lib/BasicsC/logging.c
View File

@ -393,10 +393,12 @@ static int GenerateMessage (char* buffer,
n = 0;
TRI_LockSpin(&OutputPrefixLock);
if (OutputPrefix && *OutputPrefix) {
n = snprintf(buffer, size, "%s ", OutputPrefix);
}
TRI_UnlockSpin(&OutputPrefixLock);
if (n < 0) {
@ -515,6 +517,7 @@ static void OutputMessage (TRI_log_level_e level,
size_t length,
bool copy) {
size_t i;
if (! LoggingActive) {
if (! copy) {
@ -689,10 +692,12 @@ static void LogThread (char const* func,
// write time in buffer
len = strftime(buffer, 32, "%Y-%m-%dT%H:%M:%SZ ", &tb);
va_copy(ap2, ap);
n = GenerateMessage(buffer + len, sizeof(buffer) - len, func, file, line, level, processId, threadId, fmt, ap2);
va_end(ap2);
if (n == -1) {
TRI_Log(func, file, line, TRI_LOG_LEVEL_WARNING, TRI_LOG_SEVERITY_HUMAN, "format string is corrupt");
return;

23
lib/BasicsC/memory-map-posix.c
View File

@ -48,7 +48,7 @@
// Flush memory mapped file to disk
////////////////////////////////////////////////////////////////////////////////
int TRI_FlushMMFile(void* fileHandle, void** mmHandle, void* startingAddress, size_t numOfBytesToFlush, int flags) {
int TRI_FlushMMFile(int fileDescriptor, void** mmHandle, void* startingAddress, size_t numOfBytesToFlush, int flags) {
// ...........................................................................
// Possible flags to send are (based upon the Ubuntu Linux ASM include files:
@ -69,12 +69,10 @@ int TRI_FlushMMFile(void* fileHandle, void** mmHandle, void* startingAddress, si
#ifdef __APPLE__
if (res == 0) {
if (fileHandle != NULL) {
int fd = *((int*)(fileHandle));
res = fcntl(fd, F_FULLFSYNC, 0);
}
res = fcntl(fileDescriptor, F_FULLFSYNC, 0);
}
#endif
if (res == 0) {
// msync was successful
return TRI_ERROR_NO_ERROR;
@ -88,7 +86,7 @@ int TRI_FlushMMFile(void* fileHandle, void** mmHandle, void* startingAddress, si
return TRI_ERROR_ARANGO_MSYNC_FAILED;
}
return TRI_ERROR_SYS_ERROR;
}
@ -96,21 +94,16 @@ int TRI_MMFile(void* memoryAddress,
size_t numOfBytesToInitialise,
int memoryProtection,
int flags,
void* fileHandle,
int fileDescriptor,
void** mmHandle,
int64_t offset,
void** result) {
int fd = -1;
off_t offsetRetyped = (off_t)(offset);
*mmHandle = NULL; // only useful for windows
if (fileHandle != NULL) {
fd = *((int*)(fileHandle));
}
*result = mmap(memoryAddress, numOfBytesToInitialise, memoryProtection, flags, fd, offsetRetyped);
*result = mmap(memoryAddress, numOfBytesToInitialise, memoryProtection, flags, fileDescriptor, offsetRetyped);
if (*result != MAP_FAILED) {
return TRI_ERROR_NO_ERROR;
@ -123,7 +116,7 @@ int TRI_MMFile(void* memoryAddress,
}
int TRI_UNMMFile(void* memoryAddress, size_t numOfBytesToUnMap, void* fileHandle, void** mmHandle) {
int TRI_UNMMFile(void* memoryAddress, size_t numOfBytesToUnMap, int fileDescriptor, void** mmHandle) {
int result;
assert(*mmHandle == NULL);
@ -142,7 +135,7 @@ int TRI_UNMMFile(void* memoryAddress, size_t numOfBytesToUnMap, void* fileHandle
}
int TRI_ProtectMMFile(void* memoryAddress, size_t numOfBytesToProtect, int flags, void* fileHandle, void** mmHandle) {
int TRI_ProtectMMFile(void* memoryAddress, size_t numOfBytesToProtect, int flags, int fileDescriptor, void** mmHandle) {
int result;
assert(*mmHandle == NULL);

132
lib/BasicsC/memory-map-win32.c
View File

@ -38,7 +38,7 @@
/// @{
////////////////////////////////////////////////////////////////////////////////
int TRI_FlushMMFile(void* fileHandle, void** mmHandle, void* startingAddress, size_t numOfBytesToFlush, int flags) {
int TRI_FlushMMFile(int fileDescriptor, void** mmHandle, void* startingAddress, size_t numOfBytesToFlush, int flags) {
// ...........................................................................
// Possible flags to send are (based upon the Ubuntu Linux ASM include files:
@ -51,11 +51,44 @@ int TRI_FlushMMFile(void* fileHandle, void** mmHandle, void* startingAddress, si
// FlushFileBuffers ensures file written to disk
// ...........................................................................
bool ok = FlushViewOfFile(startingAddress, numOfBytesToFlush);
if (ok && ((flags & MS_SYNC) == MS_SYNC)) {
ok = FlushFileBuffers(fileHandle);
// ...........................................................................
// Whenever we talk to the memory map functions, we require a file handle
// rather than a file descriptor. However, we only store file descriptors for
// now - this may change.
// ...........................................................................
HANDLE fileHandle;
BOOL result;
if (fileDescriptor < 0) { // an invalid file descriptor of course means an invalid handle
return TRI_ERROR_NO_ERROR;
}
if (ok) {
// ...........................................................................
// Attempt to convert file descriptor into an operating system file handle
// ...........................................................................
//printf("oreste:_get_osfhandle _get_osfhandle 1000:BEFORE:#############################:file=%d\n",fileDescriptor);
fileHandle = (HANDLE)_get_osfhandle(fileDescriptor);
//printf("oreste:_get_osfhandle _get_osfhandle 1000:AFTER:#############################:file=%d\n",fileDescriptor);
// ...........................................................................
// An invalid file system handle was returned.
// ...........................................................................
if (fileHandle == INVALID_HANDLE_VALUE ) {
return TRI_ERROR_SYS_ERROR;
}
result = FlushViewOfFile(startingAddress, numOfBytesToFlush);
if (result && ((flags & MS_SYNC) == MS_SYNC)) {
result = FlushFileBuffers(fileHandle);
}
if (result) {
return TRI_ERROR_NO_ERROR;
}
return TRI_ERROR_SYS_ERROR;
@ -63,13 +96,13 @@ int TRI_FlushMMFile(void* fileHandle, void** mmHandle, void* startingAddress, si
int TRI_MMFile(void* memoryAddress, size_t numOfBytesToInitialise, int memoryProtection,
int flags, void* fileHandle, void** mmHandle, int64_t offset, void** result) {
int flags, int fileDescriptor, void** mmHandle, int64_t offset, void** result) {
DWORD objectProtection = PAGE_READONLY;
DWORD viewProtection = FILE_MAP_READ;
LARGE_INTEGER mmLength;
HANDLE fileHandle;
// ...........................................................................
// Set the high and low order 32 bits for using a 64 bit integer
// ...........................................................................
@ -77,6 +110,35 @@ int TRI_MMFile(void* memoryAddress, size_t numOfBytesToInitialise, int memoryPro
mmLength.QuadPart = numOfBytesToInitialise;
// ...........................................................................
// Whenever we talk to the memory map functions, we require a file handle
// rather than a file descriptor. However, we only store file descriptors for
// now - this may change.
// ...........................................................................
if (fileDescriptor < 0) { // an invalid file descriptor of course means an invalid handle
fileHandle = INVALID_HANDLE_VALUE;
}
else {
// ...........................................................................
// Attempt to convert file descriptor into an operating system file handle
// ...........................................................................
fileHandle = (HANDLE)_get_osfhandle(fileDescriptor);
// ...........................................................................
// An invalid file system handle was returned.
// ...........................................................................
if (fileHandle == INVALID_HANDLE_VALUE ) {
LOG_DEBUG("File descriptor converted to an invalid handle");
LOG_TRACE("File descriptor converted to an invalid handle");
return TRI_ERROR_SYS_ERROR;
}
}
// ...........................................................................
// There are two steps for mapping a file:
// Create the handle and then bring the memory mapped file into 'view'
@ -87,9 +149,14 @@ int TRI_MMFile(void* memoryAddress, size_t numOfBytesToInitialise, int memoryPro
// so we assume no execution and only read access
// ...........................................................................
if (fileHandle == NULL) {
fileHandle = INVALID_HANDLE_VALUE; // lives in virtual memory rather than a real file
}
//res = TRI_MMFile(0, maximalSize, PROT_WRITE | PROT_READ, MAP_SHARED, &fd, &mmHandle, 0, &data);
// ...........................................................................
// If the fileHandle (or file descriptor) is set to NULL, then the are not
// memory mapping a real file, rather the file resides in virtual memory
// ...........................................................................
if ((flags & PROT_READ) == PROT_READ) {
@ -131,27 +198,54 @@ int TRI_MMFile(void* memoryAddress, size_t numOfBytesToInitialise, int memoryPro
objectProtection = PAGE_READWRITE;
viewProtection = FILE_MAP_ALL_ACCESS;
}
*mmHandle = CreateFileMapping(fileHandle, NULL, objectProtection, mmLength.HighPart, mmLength.LowPart, NULL);
// ...........................................................................
// TODO: determine the correct memory protection and then uncomment
// ...........................................................................
// *mmHandle = CreateFileMapping(fileHandle, NULL, objectProtection, mmLength.HighPart, mmLength.LowPart, NULL);
*mmHandle = CreateFileMapping(fileHandle, NULL, PAGE_READWRITE, mmLength.HighPart, mmLength.LowPart, NULL);
// ...........................................................................
// If we have failed for some reason return system error for now.
// TODO: map windows error codes to triagens.
// We do however output some trace information with the errorcode
// ...........................................................................
if (*mmHandle == NULL) {
// we have failure for some reason
// TODO: map the error codes of windows to the TRI_ERROR (see function DWORD WINAPI GetLastError(void) );
DWORD errorCode = GetLastError();
LOG_DEBUG("File descriptor converted to an invalid handle",errorCode);
LOG_TRACE("File descriptor converted to an invalid handle",errorCode);
return TRI_ERROR_SYS_ERROR;
}
// ........................................................................
// We have a valid handle, now map the view. We let the OS handle where the
// view is placed in memory.
// We have a valid mm handle, now map the view. We let the OS decide
// where this view is placed in memory.
// ........................................................................
*result = MapViewOfFile(*mmHandle, viewProtection, 0, 0, 0);
//TODO: fix the viewProtection above *result = MapViewOfFile(*mmHandle, viewProtection, 0, 0, 0);
*result = MapViewOfFile(*mmHandle, FILE_MAP_ALL_ACCESS, 0, 0, numOfBytesToInitialise);
// ........................................................................
// The map view of file has failed.
// ........................................................................
if (*result == NULL) {
DWORD errorCode = GetLastError();
CloseHandle(*mmHandle);
// we have failure for some reason
// TODO: map the error codes of windows to the TRI_ERROR (see function DWORD WINAPI GetLastError(void) );
if (errorCode == ERROR_NOT_ENOUGH_MEMORY) {
LOG_DEBUG("MapViewOfFile failed with out of memory error %d",errorCode);
LOG_TRACE("MapViewOfFile failed with out of memory error %d",errorCode);
return TRI_ERROR_OUT_OF_MEMORY;
}
LOG_DEBUG("MapViewOfFile failed with error code = %d",errorCode);
LOG_TRACE("MapViewOfFile failed with error code = %d",errorCode);
return TRI_ERROR_SYS_ERROR;
}
@ -159,7 +253,7 @@ int TRI_MMFile(void* memoryAddress, size_t numOfBytesToInitialise, int memoryPro
}
int TRI_UNMMFile(void* memoryAddress, size_t numOfBytesToUnMap, void* fileHandle, void** mmHandle) {
int TRI_UNMMFile(void* memoryAddress, size_t numOfBytesToUnMap, int fileDescriptor, void** mmHandle) {
bool ok = UnmapViewOfFile(memoryAddress);
ok = (CloseHandle(*mmHandle) && ok);
if (!ok) {
@ -169,7 +263,7 @@ int TRI_UNMMFile(void* memoryAddress, size_t numOfBytesToUnMap, void* fileHandl
}
int TRI_ProtectMMFile(void* memoryAddress, size_t numOfBytesToProtect, int flags, void* fileHandle, void** mmHandle) {
int TRI_ProtectMMFile(void* memoryAddress, size_t numOfBytesToProtect, int flags, int fileDescriptor, void** mmHandle) {
DWORD objectProtection = PAGE_READONLY;
DWORD viewProtection = FILE_MAP_READ;
LARGE_INTEGER mmLength;

10
lib/BasicsC/memory-map-win32.h
View File

@ -39,11 +39,11 @@
// Flags used when we create a memory map -- dummy flags for windows for now
////////////////////////////////////////////////////////////////////////////////
#define MAP_SHARED 0x01 /* Share changes */
#define MAP_PRIVATE 0x02 /* Changes are private */
#define MAP_TYPE 0x0f /* Mask for type of mapping */
#define MAP_FIXED 0x10 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x20 /* don't use a file */
#define MAP_SHARED 0x01 /* Share changes */
#define MAP_PRIVATE 0x02 /* Changes are private */
#define MAP_TYPE 0x0f /* Mask for type of mapping */
#define MAP_FIXED 0x10 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x20 /* don't use a file */
////////////////////////////////////////////////////////////////////////////////

8
lib/BasicsC/memory-map.h
View File

@ -67,7 +67,7 @@ extern "C" {
/// @brief flushes changes made in memory back to disk
////////////////////////////////////////////////////////////////////////////////
int TRI_FlushMMFile (void* fileHandle,
int TRI_FlushMMFile (int fileDescriptor,
void** mmHandle,
void* startingAddress,
size_t numOfBytesToFlush,
@ -82,7 +82,7 @@ int TRI_MMFile (void* memoryAddress,
size_t numOfBytesToInitialise,
int memoryProtection,
int flags,
void* fileHandle,
int fileDescriptor,
void** mmHandle,
int64_t offset,
void** result);
@ -94,7 +94,7 @@ int TRI_MMFile (void* memoryAddress,
int TRI_UNMMFile (void* memoryAddress,
size_t numOfBytesToUnMap,
void* fileHandle,
int fileDescriptor,
void** mmHandle);
@ -105,7 +105,7 @@ int TRI_UNMMFile (void* memoryAddress,
int TRI_ProtectMMFile (void* memoryAddress,
size_t numOfBytesToProtect,
int flags,
void* fileHandle,
int fileDescriptor,
void** mmHandle);
////////////////////////////////////////////////////////////////////////////////

74
lib/BasicsC/operating-system.h
View File

@ -33,7 +33,7 @@
#endif
// -----------------------------------------------------------------------------
// --SECTION-- apple
// --SECTION-- global
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
@ -51,12 +51,18 @@
#define __STDC_LIMIT_MACROS
#endif
#if defined(__amd64__) || defined(__amd64) || defined(__x86_64__) || defined(__x86_64) || defined(_M_X64) || defined(_M_AMD64)
#undef TRI_PADDING_32
#else
#define TRI_PADDING_32 1
#endif
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////
// -----------------------------------------------------------------------------
// --SECTION-- apple
// --Section-- apple
// -----------------------------------------------------------------------------
////////////////////////////////////////////////////////////////////////////////
@ -133,21 +139,29 @@
#define TRI_CHDIR chdir
#define TRI_CLOSE close
#define TRI_CLOSE_SOCKET close
#define TRI_CREATE(a,b,c) open((a), (b), (c))
#define TRI_GETCWD getcwd
#define TRI_LSEEK lseek
#define TRI_MKDIR(a,b) mkdir((a), (b))
#define TRI_OPEN(a,b) open((a), (b))
#define TRI_READ read
#define TRI_READ_SOCKET(a,b,c,d) read((a), (b), (c))
#define TRI_RMDIR rmdir
#define TRI_SLEEP sleep
#define TRI_UNLINK unlink
#define TRI_WRITE write
#define TRI_WRITE_SOCKET(a,b,c,d) write((a), (b), (c))
#define TRI_LAST_ERROR_STR strerror(errno)
#define TRI_uid_t uid_t
#define TRI_gid_t gid_t
typedef int socket_t;
#define INVALID_SOCKET -1
#define SOCKET_ERROR -1
#endif
////////////////////////////////////////////////////////////////////////////////
@ -211,21 +225,29 @@
#define TRI_CHDIR chdir
#define TRI_CLOSE close
#define TRI_CLOSE_SOCKET close
#define TRI_CREATE(a,b,c) open((a), (b), (c))
#define TRI_LSEEK lseek
#define TRI_GETCWD getcwd
#define TRI_MKDIR(a,b) mkdir((a), (b))
#define TRI_OPEN(a,b) open((a), (b))
#define TRI_READ read
#define TRI_READ_SOCKET(a,b,c,d) read((a), (b), (c))
#define TRI_RMDIR rmdir
#define TRI_SLEEP sleep
#define TRI_UNLINK unlink
#define TRI_WRITE write
#define TRI_WRITE_SOCKET(a,b,c,d) write((a), (b), (c))
#define TRI_LAST_ERROR_STR strerror(errno)
#define TRI_uid_t uid_t
#define TRI_gid_t gid_t
typedef int socket_t;
#define INVALID_SOCKET -1
#define SOCKET_ERROR -1
#endif
////////////////////////////////////////////////////////////////////////////////
@ -323,21 +345,29 @@
#define TRI_CHDIR chdir
#define TRI_CLOSE close
#define TRI_CLOSE_SOCKET close
#define TRI_CREATE(a,b,c) open((a), (b), (c))
#define TRI_LSEEK lseek
#define TRI_GETCWD getcwd
#define TRI_MKDIR(a,b) mkdir((a), (b))
#define TRI_OPEN(a,b) open((a), (b))
#define TRI_READ read
#define TRI_READ_SOCKET(a,b,c,d) read((a), (b), (c))
#define TRI_RMDIR rmdir
#define TRI_SLEEP sleep
#define TRI_UNLINK unlink
#define TRI_WRITE write
#define TRI_WRITE_SOCKET(a,b,c,d) write((a), (b), (c))
#define TRI_LAST_ERROR_STR strerror(errno)
#define TRI_uid_t uid_t
#define TRI_gid_t gid_t
typedef int socket_t;
#define INVALID_SOCKET -1
#define SOCKET_ERROR -1
#endif
////////////////////////////////////////////////////////////////////////////////
@ -358,11 +388,22 @@
#define TRI_DIR_SEPARATOR_CHAR '\\'
#define TRI_DIR_SEPARATOR_STR "\\"
// ..............................................................................
// This directive below suppresses warnings about using the 'new' more secure CRT
// functions.
// ..............................................................................
#define _CRT_SECURE_NO_WARNINGS 1
#define _CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES 1
// ..............................................................................
// This directive below provides a manner in which the 'new' more secure functions
// for example, strcpy is automatically converted to strcpy_s. This is enabled
// by default. We have disabled it here.
// ..............................................................................
//#define _CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES 1
#include <stdio.h>
#include <io.h>
#include <WinSock2.h>
#define TRI_WIN32_CONSOLE 1
#define TRI_WIN32_THREAD_LOCAL_STORAGE 1
@ -423,27 +464,46 @@ extern "C" {
}
#endif
#ifndef __BOOL_DEFINED
typedef unsigned int bool;
//typedef unsigned int bool; - this never ever going to work. Problem is sizeof(bool) in VS C++ is 1 byte and
// sizeof(bool) in VS C (C compiler) is -- whatever you want. However, when structures are interchanged between
// C & C++ (as in arango) all hell will break loose.
typedef unsigned char bool;
#define true 1
#define false 0
#endif
#define va_copy(d,s) ((d) = (s))
// we do not have owner read and owner write under windows
// so map these to global read, global write
// these are used when creating a file
#define S_IRUSR _S_IREAD
#define S_IWUSR _S_IWRITE
#define S_IRGRP _S_IREAD
#define S_IWGRP _S_IWRITE
#define O_RDONLY _O_RDONLY
#define TRI_CHDIR _chdir
#define TRI_CLOSE _close
#define TRI_CREATE(a,b,c) _open((a), (b))
#define TRI_CLOSE_SOCKET TRI_WIN_closesocket
/* #define TRI_CREATE(a,b,c) _open((a), (b), (c)) */
#define TRI_CREATE(a,b,c) TRI_createFile((a), (b), (c))
#define TRI_GETCWD _getcwd
#define TRI_LSEEK _lseek
#define TRI_MKDIR(a,b) _mkdir((a))
#define TRI_OPEN(a,b) _open((a), (b))
/* #define TRI_OPEN(a,b) _open((a), (b)) */
#define TRI_OPEN(a,b) TRI_openFile((a), (b))
#define TRI_READ _read
#define TRI_READ_SOCKET(a,b,c,d) recv((a), (b), (c), (d))
#define TRI_RMDIR _rmdir
#define TRI_SLEEP TRI_sleep
#define TRI_UNLINK _unlink
#define TRI_WRITE _write
#define TRI_WRITE_SOCKET(a,b,c,d) send((a), (b), (c), (d))
#define TRI_LAST_ERROR_STR strerror(errno)
@ -467,6 +527,7 @@ typedef unsigned int bool;
#define alloca _alloca
typedef SOCKET socket_t;
#endif
@ -483,7 +544,6 @@ typedef unsigned int bool;
/// @{
////////////////////////////////////////////////////////////////////////////////
typedef int socket_t;
////////////////////////////////////////////////////////////////////////////////

3
lib/BasicsC/strings.h
View File

@ -320,6 +320,9 @@ size_t TRI_CharLengthUtf8String (const char*);
char* TRI_PrefixUtf8String (const char*, const uint32_t);
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////

11
lib/BasicsC/threads-win32.c
View File

@ -1,5 +1,5 @@
////////////////////////////////////////////////////////////////////////////////
/// @brief threads in win32
/// @brief threads in win32 & win64
///
/// @file
///
@ -205,6 +205,7 @@ void TRI_JoinThread (TRI_thread_t* thread) {
bool TRI_SignalThread (TRI_thread_t* thread, int signum) {
// TODO: NO NATIVE implementation of signals
return false;
}
@ -213,7 +214,13 @@ bool TRI_SignalThread (TRI_thread_t* thread, int signum) {
////////////////////////////////////////////////////////////////////////////////
bool TRI_IsSelfThread (TRI_thread_t* thread) {
return ( GetCurrentThreadId() == GetThreadId(thread) );
return false;
// ...........................................................................
// The GetThreadID(...) function is only available in Windows Vista or Higher
// TODO: Change the TRI_thread_t into a structure which stores the thread id
// as well as the thread handle. This can then be passed around
// ...........................................................................
//return ( GetCurrentThreadId() == GetThreadId(thread) );
}

467
lib/BasicsC/win-utils.c
View File

@ -1,116 +1,359 @@
////////////////////////////////////////////////////////////////////////////////
/// @brief some utilities for windows
///
/// @file
///
/// DISCLAIMER
///
/// Copyright 2012 triagens GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is triAGENS GmbH, Cologne, Germany
///
/// @author Dr. O
/// @author Copyright 2011-2012, triAGENS GmbH, Cologne, Germany
////////////////////////////////////////////////////////////////////////////////
#include <windows.h>
#include <io.h>
#include "win-utils.h"
/*
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <share.h>
*/
////////////////////////////////////////////////////////////////////////////////
/// @addtogroup Windows_Utilties
/// @{
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
/// @brief some utilities for windows
///
/// @file
///
/// DISCLAIMER
///
/// Copyright 2012 triagens GmbH, Cologne, Germany
///
/// Licensed under the Apache License, Version 2.0 (the "License");
/// you may not use this file except in compliance with the License.
/// You may obtain a copy of the License at
///
/// http://www.apache.org/licenses/LICENSE-2.0
///
/// Unless required by applicable law or agreed to in writing, software
/// distributed under the License is distributed on an "AS IS" BASIS,
/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
/// See the License for the specific language governing permissions and
/// limitations under the License.
///
/// Copyright holder is triAGENS GmbH, Cologne, Germany
///
/// @author Dr. O
/// @author Copyright 2011-2012, triAGENS GmbH, Cologne, Germany
////////////////////////////////////////////////////////////////////////////////
#include <io.h>
#include "win-utils.h"
#include <BasicsC/logging.h>
#include <windows.h>
#include <string.h>
#include <malloc.h>
#include <crtdbg.h>
/*
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <share.h>
*/
// .............................................................................
// Some global variables which may be required throughtout the lifetime of the
// server
// .............................................................................
_invalid_parameter_handler oldInvalidHandleHandler;
_invalid_parameter_handler newInvalidHandleHandler;
////////////////////////////////////////////////////////////////////////////////
/// @addtogroup Windows_Utilties
/// @{
////////////////////////////////////////////////////////////////////////////////
int ftruncate(int fd, long newSize) {
int result = _chsize(fd, newSize);
return result;
}
int getpagesize(void) {
static int pageSize = 0; // only define it once
if (!pageSize) {
// first time, so call the system info function
SYSTEM_INFO systemInfo;
GetSystemInfo (&systemInfo);
pageSize = systemInfo.dwPageSize;
}
return pageSize;
}
////////////////////////////////////////////////////////////////////////////////
// Calls the windows Sleep function which always sleeps for milliseconds
////////////////////////////////////////////////////////////////////////////////
void TRI_sleep(unsigned long waitTime) {
Sleep(waitTime * 1000);
}
////////////////////////////////////////////////////////////////////////////////
// Calls a timer which waits for a signal after the elapsed time.
// The timer is acurate to 100nanoseconds
////////////////////////////////////////////////////////////////////////////////
void TRI_usleep(unsigned long waitTime) {
int result;
HANDLE hTimer = NULL; // stores the handle of the timer object
LARGE_INTEGER wTime; // essentially a 64bit number
wTime.QuadPart = waitTime * 10; // *10 to change to microseconds
wTime.QuadPart = -wTime.QuadPart; // negative indicates relative time elapsed,
// Create an unnamed waitable timer.
hTimer = CreateWaitableTimer(NULL, 1, NULL);
if (hTimer == NULL) {
// no much we can do at this low level
return;
}
// Set timer to wait for indicated micro seconds.
if (!SetWaitableTimer(hTimer, &wTime, 0, NULL, NULL, 0)) {
// no much we can do at this low level
return;
}
// Wait for the timer - but don't wait for ever.
result = WaitForSingleObject(hTimer, ((waitTime/1000) + 1)); // wait for a 1 millisecond at least
// todo: go through what the result is e.g. WAIT_OBJECT_0
return;
}
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////
// Local Variables:
// mode: outline-minor
// outline-regexp: "^\\(/// @brief\\|/// {@inheritDoc}\\|/// @addtogroup\\|// --SECTION--\\|/// @\\}\\)"
// End:
int getpagesize(void) {
static int pageSize = 0; // only define it once
if (!pageSize) {
// first time, so call the system info function
SYSTEM_INFO systemInfo;
GetSystemInfo (&systemInfo);
pageSize = systemInfo.dwPageSize;
}
return pageSize;
}
////////////////////////////////////////////////////////////////////////////////
// Calls the windows Sleep function which always sleeps for milliseconds
////////////////////////////////////////////////////////////////////////////////
void TRI_sleep(unsigned long waitTime) {
Sleep(waitTime * 1000);
}
////////////////////////////////////////////////////////////////////////////////
// Calls a timer which waits for a signal after the elapsed time.
// The timer is acurate to 100nanoseconds
////////////////////////////////////////////////////////////////////////////////
void TRI_usleep(unsigned long waitTime) {
int result;
HANDLE hTimer = NULL; // stores the handle of the timer object
LARGE_INTEGER wTime; // essentially a 64bit number
wTime.QuadPart = waitTime * 10; // *10 to change to microseconds
wTime.QuadPart = -wTime.QuadPart; // negative indicates relative time elapsed,
// Create an unnamed waitable timer.
hTimer = CreateWaitableTimer(NULL, 1, NULL);
if (hTimer == NULL) {
// no much we can do at this low level
return;
}
// Set timer to wait for indicated micro seconds.
if (!SetWaitableTimer(hTimer, &wTime, 0, NULL, NULL, 0)) {
// no much we can do at this low level
return;
}
// Wait for the timer - but don't wait for ever.
result = WaitForSingleObject(hTimer, ((waitTime/1000) + 1)); // wait for a 1 millisecond at least
// todo: go through what the result is e.g. WAIT_OBJECT_0
return;
}
////////////////////////////////////////////////////////////////////////////////
// Sets up a handler when invalid (win) handles are passed to a windows function.
// This is not of much use since no values can be returned. All we can do
// for now is to ignore error and hope it goes away!
////////////////////////////////////////////////////////////////////////////////
static void InvalidParameterHandler(const wchar_t* expression, // expression sent to function - NULL
const wchar_t* function, // name of function - NULL
const wchar_t* file, // file where code resides - NULL
unsigned int line, // line within file - NULL
uintptr_t pReserved) { // in case microsoft forget something
LOG_ERROR("Invalid handle parameter passed");
/* start oreste -debug */
if (expression != 0) {
wprintf(L"win-utils.c:InvalidParameterHandler:EXPRESSION = %s\n",expression);
}
else {
wprintf(L"win-utils.c:InvalidParameterHandler:EXPRESSION = NULL\n");
}
if (function != 0) {
wprintf(L"win-utils.c:InvalidParameterHandler:FUNCTION = %s\n",function);
}
else {
wprintf(L"win-utils.c:InvalidParameterHandler:FUNCTION = NULL\n");
}
if (file!= 0) {
wprintf(L"win-utils.c:InvalidParameterHandler:FILE = %s\n",file);
}
else {
wprintf(L"win-utils.c:InvalidParameterHandler:FILE = NULL\n");
}
printf("oreste:%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%:win-utils.c:InvalidParameterHandler:LINE = %ud\n",line);
/* end oreste -debug */
//abort();
// TODO: use the wcstombs_s function to convert wchar to char - since all the above
// wchar never will contain 2 byte chars
}
////////////////////////////////////////////////////////////////////////////////
// Called from the 'main' and performs any initialisation requirements which
// are specific to windows.
//
// If this function returns 0, then no errors encountered. If < 0, then the
// calling function should terminate the application. If > 0, then the
// calling function should decide what to do.
////////////////////////////////////////////////////////////////////////////////
int finaliseWindows(const TRI_win_finalise_e finaliseWhat, const char* data) {
// ............................................................................
// The data is used to transport information from the calling function to here
// it may be NULL (and will be in most cases)
// ............................................................................
switch (finaliseWhat) {
case TRI_WIN_FINAL_WSASTARTUP_FUNCTION_CALL: {
int errorCode;
errorCode = WSACleanup();
if (errorCode != 0) {
// can not use LOG_ etc here since the logging may have terminated
printf("ERROR: Could not perform a valid Winsock2 cleanup. WSACleanup returned error %d.",errorCode);
return -1;
}
return 0;
}
default: {
// can not use LOG_ etc here since the logging may have terminated
printf("ERROR: Invalid windows finalisation called");
return -1;
}
}
return -1;
}
int initialiseWindows(const TRI_win_initialise_e initialiseWhat, const char* data) {
// ............................................................................
// The data is used to transport information from the calling function to here
// it may be NULL (and will be in most cases)
// ............................................................................
switch (initialiseWhat) {
case TRI_WIN_INITIAL_SET_DEBUG_FLAG: {
_CrtSetDbgFlag(_CrtSetDbgFlag(_CRTDBG_LEAK_CHECK_DF)|_CRTDBG_CHECK_ALWAYS_DF);
return 0;
}
// ...........................................................................
// Assign a handler for invalid handles
// ...........................................................................
case TRI_WIN_INITIAL_SET_INVALID_HANLE_HANDLER: {
newInvalidHandleHandler = InvalidParameterHandler;
oldInvalidHandleHandler = _set_invalid_parameter_handler(newInvalidHandleHandler);
return 0;
}
case TRI_WIN_INITIAL_WSASTARTUP_FUNCTION_CALL: {
int errorCode;
WSADATA wsaData;
WORD wVersionRequested = MAKEWORD(2,2);
errorCode = WSAStartup(wVersionRequested, &wsaData);
if (errorCode != 0) {
LOG_ERROR("Could not find a usuable Winsock DLL. WSAStartup returned an error.");
return -1;
}
if (LOBYTE(wsaData.wVersion) != 2 || HIBYTE(wsaData.wVersion) != 2) {
LOG_ERROR("Could not find a usuable Winsock DLL. WSAStartup did not return version 2.2.");
WSACleanup();
return -1;
}
return 0;
}
default: {
LOG_ERROR("Invalid windows initialisation called");
return -1;
}
}
return -1;
}
int TRI_WIN_closesocket(SOCKET s) {
int res;
res = shutdown(s,2);
res = closesocket(s);
return res;
}
int TRI_createFile (const char* filename, int openFlags, int modeFlags) {
HANDLE fileHandle;
int fileDescriptor;
fileHandle = CreateFileA(filename,
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
CREATE_NEW,
0,
NULL);
if (fileHandle == INVALID_HANDLE_VALUE) {
return -1;
}
fileDescriptor = _open_osfhandle( (intptr_t)(fileHandle), O_RDWR| _O_BINARY);
return fileDescriptor;
/*
#define O_RDONLY _O_RDONLY
#define O_WRONLY _O_WRONLY
#define O_RDWR _O_RDWR
#define O_APPEND _O_APPEND
#define O_CREAT _O_CREAT
#define O_TRUNC _O_TRUNC
#define O_EXCL _O_EXCL
#define O_TEXT _O_TEXT
#define O_BINARY _O_BINARY
#define O_RAW _O_BINARY
#define O_TEMPORARY _O_TEMPORARY
#define O_NOINHERIT _O_NOINHERIT
#define O_SEQUENTIAL _O_SEQUENTIAL
#define O_RANDOM _O_RANDOM
//filename, O_CREAT | O_EXCL | O_RDWR, S_IRUSR | S_IWUSR
*/
}
////////////////////////////////////////////////////////////////////////////////////
// Creates or opens a file using the windows CreateFile method. Notice below we
// have used the method CreateFileA to avoid unicode characters - for now anyway
// TODO oreste: map the flags e.g. O_RDWR to the equivalents for CreateFileA
////////////////////////////////////////////////////////////////////////////////////
int TRI_openFile (const char* filename, int openFlags) {
HANDLE fileHandle;
int fileDescriptor;
fileHandle = CreateFileA(filename,
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
0,
NULL);
if (fileHandle == INVALID_HANDLE_VALUE) {
return -1;
}
fileDescriptor = _open_osfhandle( (intptr_t)(fileHandle), O_RDWR| _O_BINARY);
return fileDescriptor;
/*
#define O_RDONLY _O_RDONLY
#define O_WRONLY _O_WRONLY
#define O_RDWR _O_RDWR
#define O_APPEND _O_APPEND
#define O_CREAT _O_CREAT
#define O_TRUNC _O_TRUNC
#define O_EXCL _O_EXCL
#define O_TEXT _O_TEXT
#define O_BINARY _O_BINARY
#define O_RAW _O_BINARY
#define O_TEMPORARY _O_TEMPORARY
#define O_NOINHERIT _O_NOINHERIT
#define O_SEQUENTIAL _O_SEQUENTIAL
#define O_RANDOM _O_RANDOM
//filename, O_CREAT | O_EXCL | O_RDWR, S_IRUSR | S_IWUSR
*/
}
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////
// Local Variables:
// mode: outline-minor
// outline-regexp: "^\\(/// @brief\\|/// {@inheritDoc}\\|/// @addtogroup\\|// --SECTION--\\|/// @\\}\\)"
// End:

32
lib/BasicsC/win-utils.h
View File

@ -28,6 +28,7 @@
#ifndef TRIAGENS_BASICS_C_WIN_UTILS_H
#define TRIAGENS_BASICS_C_WIN_UTILS_H 1
#include <WinSock2.h>
/* Constants rounded for 21 decimals.
#define M_E 2.71828182845904523536
@ -55,6 +56,25 @@ extern "C" {
/// @{
////////////////////////////////////////////////////////////////////////////////
// .............................................................................
// Called before anything else starts - initialises whatever is required to be
// initalised.
// .............................................................................
typedef enum {
TRI_WIN_FINAL_SET_INVALID_HANLE_HANDLER,
TRI_WIN_FINAL_WSASTARTUP_FUNCTION_CALL
}
TRI_win_finalise_e;
typedef enum {
TRI_WIN_INITIAL_SET_DEBUG_FLAG,
TRI_WIN_INITIAL_SET_INVALID_HANLE_HANDLER,
TRI_WIN_INITIAL_WSASTARTUP_FUNCTION_CALL
}
TRI_win_initialise_e;
int finaliseWindows (const TRI_win_finalise_e, const char*);
int initialiseWindows (const TRI_win_initialise_e, const char*);
// .............................................................................
// windows equivalent of ftruncate (the truncation of an open file) is
@ -71,6 +91,18 @@ int ftruncate (int, long);
int getpagesize (void);
int TRI_WIN_closesocket (SOCKET);
// .............................................................................
// This function uses the CreateFile windows method rather than _open which
// then will allow the application to rename files on the fly.
// .............................................................................
int TRI_createFile (const char* filename, int openFlags, int modeFlags);
int TRI_openFile (const char* filename, int openFlags);
// .............................................................................
// the sleep function in windows is for milliseconds, on linux it is for seconds
// this provides a translation

14
lib/GeneralServer/GeneralAsyncCommTask.h
View File

@ -110,9 +110,17 @@ namespace triagens {
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
void setup (Scheduler* scheduler, EventLoop loop) {
SocketTask::setup(scheduler, loop);
AsyncTask::setup(scheduler, loop);
bool setup (Scheduler* scheduler, EventLoop loop) {
bool ok;
ok = SocketTask::setup(scheduler, loop);
if (!ok) {
return false;
}
ok = AsyncTask::setup(scheduler, loop);
if (!ok) {
return false;
}
return true;
}
////////////////////////////////////////////////////////////////////////////////

8
lib/GeneralServer/GeneralCommTask.h
View File

@ -29,6 +29,10 @@
#ifndef TRIAGENS_GENERAL_SERVER_GENERAL_COMM_TASK_H
#define TRIAGENS_GENERAL_SERVER_GENERAL_COMM_TASK_H 1
#ifdef _WIN32
#include "BasicsC/win-utils.h"
#endif
#include "Basics/Common.h"
#include "Basics/StringBuffer.h"
@ -100,6 +104,7 @@ namespace triagens {
<< ", client ip " << _connectionInfo.clientAddress
<< ", client port " << _connectionInfo.clientPort;
pair<size_t, size_t> p = server->getHandlerFactory()->sizeRestrictions();
_maximalHeaderSize = p.first;
@ -169,7 +174,8 @@ namespace triagens {
void beginShutdown () {
if (commSocket != -1) {
TRI_CLOSE(commSocket);
TRI_CLOSE_SOCKET(commSocket);
commSocket = -1;
}
}

9
lib/GeneralServer/GeneralServer.h
View File

@ -190,6 +190,7 @@ namespace triagens {
LOGGER_TRACE << "trying to bind to endpoint '" << (*i)->getSpecification() << "' for requests";
bool ok = openEndpoint(*i);
if (ok) {
LOGGER_DEBUG << "bound to endpoint '" << (*i)->getSpecification() << "'";
}
@ -418,11 +419,15 @@ namespace triagens {
bool openEndpoint (Endpoint* endpoint) {
ListenTask* task = new GeneralListenTask<S> (dynamic_cast<S*> (this), endpoint, true);
if (! task->isBound()) {
// ...................................................................
// For some reason we have failed in our endeavour to bind to the socket -
// this effectively terminates the server
// ...................................................................
if (! task->isBound()) {
deleteTask(task);
return false;
}
_scheduler->registerTask(task);
_listenTasks.push_back(task);

5
lib/HttpServer/HttpCommTask.h
View File

@ -127,11 +127,13 @@ namespace triagens {
bool processRead () {
if (this->_requestPending || this->_readBuffer->c_str() == 0) {
return true;
}
bool handleRequest = false;
if (! this->_readRequestBody) {
#ifdef TRI_ENABLE_FIGURES
@ -151,8 +153,10 @@ namespace triagens {
}
}
size_t headerLength = ptr - this->_readBuffer->c_str();
if (headerLength > this->_maximalHeaderSize) {
LOGGER_WARNING << "maximal header size is " << this->_maximalHeaderSize << ", request header size is " << headerLength;
// header is too large
@ -354,6 +358,7 @@ namespace triagens {
// authenticated
if (auth) {
HttpHandler* handler = this->_server->getHandlerFactory()->createHandler(this->_request);
bool ok = false;

3
lib/HttpServer/HttpHandlerFactory.cpp
View File

@ -238,11 +238,13 @@ HttpHandler* HttpHandlerFactory::createHandler (HttpRequest* request) {
}
#endif
map<string, create_fptr> const& ii = _constructors;
string path = request->requestPath();
map<string, create_fptr>::const_iterator i = ii.find(path);
void* data = 0;
// no direct match, check prefix matches
if (i == ii.end()) {
LOGGER_TRACE << "no direct handler found, trying prefixes";
@ -327,6 +329,7 @@ HttpHandler* HttpHandlerFactory::createHandler (HttpRequest* request) {
}
}
// look up data
map<string, void*> const& jj = _datas;
map<string, void*>::const_iterator j = jj.find(path);

3
lib/Logger/Logger.cpp
View File

@ -169,8 +169,7 @@ static void OutputMachine (string const& text, LoggerData::Info const& info) {
switch (info._category) {
case TRI_LOG_CATEGORY_FATAL: line.appendText("FATAL"); break;
case TRI_LOG_CATEGORY_ERROR: line.appendText("ERROR"); break;
case TRI_LOG_CATEGORY_WARNING: line.appendText("WARNING"); break;
case TRI_LOG_CATEGORY_WARNING: { line.appendText("WARNING"); break; }
case TRI_LOG_CATEGORY_REQUEST_IN_START: line.appendText("REQUEST-IN-START"); break;
case TRI_LOG_CATEGORY_REQUEST_IN_END: line.appendText("REQUEST-IN-END"); break;
case TRI_LOG_CATEGORY_REQUEST_OUT_START: line.appendText("REQUEST-OUT-START"); break;

21
lib/Logger/LoggerData.cpp
View File

@ -204,6 +204,27 @@ Info::Info ()
_prefix() {
}
Info::Info (const Info& originalInfo) {
_messageIdentifier._name = originalInfo._messageIdentifier._name;
_level = originalInfo._level;
_category = originalInfo._category;
_severity = originalInfo._severity;
_functional._name = originalInfo._functional._name;
_peg._name = originalInfo._peg._name;
_task._name = originalInfo._task._name;
_position._file = originalInfo._position._file;
_position._function = originalInfo._position._function;
_position._line = originalInfo._position._line;
_measure._unit = originalInfo._measure._unit;
_measure._value = originalInfo._measure._value;
_extras.clear();
for (vector<Extra>::const_iterator i = originalInfo._extras.begin(); i != originalInfo._extras.end(); ++i) {
_extras.push_back(Extra((*i)._position, (*i)._name));
}
_userIdentifier._user = originalInfo._userIdentifier._user;
_prefix = originalInfo._prefix;
}
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////

2
lib/Logger/LoggerData.h
View File

@ -226,6 +226,8 @@ namespace triagens {
struct Info {
Info();
Info(const Info& originalInfo);
static ApplicationName _applicationName;
static Facility _facility;
static HostName _hostName;

11
lib/Logger/LoggerStream.cpp
View File

@ -124,8 +124,7 @@ static void computeInfo (LoggerData::Info& info) {
////////////////////////////////////////////////////////////////////////////////
LoggerStream::LoggerStream () :
_stream(new stringstream()),
_info() {
_stream(new stringstream()), _info() {
}
////////////////////////////////////////////////////////////////////////////////
@ -141,8 +140,7 @@ LoggerStream::LoggerStream (LoggerData::Info const& info) :
////////////////////////////////////////////////////////////////////////////////
LoggerStream::LoggerStream (LoggerStream const& copy) :
_stream(new stringstream()), _info(copy._info) {
_stream->str(copy._stream->str());
_stream(copy._stream), _info(copy._info) {
}
////////////////////////////////////////////////////////////////////////////////
@ -154,7 +152,6 @@ LoggerStream::~LoggerStream () {
if (_stream != 0) {
Logger::output(static_cast<stringstream*> (_stream)->str(), _info);
delete _stream;
}
}
@ -190,7 +187,6 @@ LoggerStream& LoggerStream::operator<< (std::ostream& (*fptr) (std::ostream&)) {
LoggerStream& LoggerStream::operator<< (TRI_log_level_e level) {
_info._level = level;
return *this;
}
@ -200,7 +196,6 @@ LoggerStream& LoggerStream::operator<< (TRI_log_level_e level) {
LoggerStream& LoggerStream::operator<< (TRI_log_category_e category) {
_info._category = category;
return *this;
}
@ -210,7 +205,6 @@ LoggerStream& LoggerStream::operator<< (TRI_log_category_e category) {
LoggerStream& LoggerStream::operator<< (TRI_log_severity_e severity) {
_info._severity = severity;
return *this;
}
@ -306,6 +300,7 @@ LoggerStream& LoggerStream::operator<< (LoggerData::UserIdentifier const& userId
return *this;
}
////////////////////////////////////////////////////////////////////////////////
/// @}
////////////////////////////////////////////////////////////////////////////////

1
lib/Logger/LoggerStream.h
View File

@ -236,6 +236,7 @@ namespace triagens {
stringstream* _stream;
////////////////////////////////////////////////////////////////////////////////
/// @brief information
////////////////////////////////////////////////////////////////////////////////

2
lib/ProgramOptions/program-options.c
View File

@ -212,7 +212,7 @@ static void CreateDoubleOption (po_double_t * desc, void const * input, void * o
po = (TRI_program_options_t*) (output);
InitOptionStructure(&doubleOpt, desc->base._name, 1, 0, po->_longopts._length);
InitOptionStructure(&doubleOpt, desc->base._name, 1, 0, (int)(po->_longopts._length));
memset(&item, 0, sizeof(item));

1
lib/Rest/AnyServer.cpp
View File

@ -401,7 +401,6 @@ int AnyServer::start () {
WritePidFile(_pidFile, TRI_CurrentProcessId());
}
prepareServer();
int res = startupServer();
if (! _pidFile.empty()) {

2
lib/Rest/Endpoint.h
View File

@ -197,7 +197,7 @@ namespace triagens {
/// @brief connect the endpoint
////////////////////////////////////////////////////////////////////////////////
virtual int connect (double, double) = 0;
virtual socket_t connect (double, double) = 0;
////////////////////////////////////////////////////////////////////////////////
/// @brief disconnect the endpoint

36
lib/Rest/EndpointIp.cpp
View File

@ -21,6 +21,7 @@
///
/// Copyright holder is triAGENS GmbH, Cologne, Germany
///
/// @author Dr. O
/// @author Jan Steemann
/// @author Copyright 2012, triAGENS GmbH, Cologne, Germany
////////////////////////////////////////////////////////////////////////////////
@ -117,19 +118,18 @@ EndpointIp::~EndpointIp () {
socket_t EndpointIp::connectSocket (const struct addrinfo* aip, double connectTimeout, double requestTimeout) {
// set address and port
char host[NI_MAXHOST], serv[NI_MAXSERV];
if (getnameinfo(aip->ai_addr, aip->ai_addrlen,
if (::getnameinfo(aip->ai_addr, aip->ai_addrlen,
host, sizeof(host),
serv, sizeof(serv), NI_NUMERICHOST | NI_NUMERICSERV) == 0) {
LOGGER_TRACE << "bind to address '" << string(host) << "' port '" << _port << "'";
}
int listenSocket = socket(aip->ai_family, aip->ai_socktype, aip->ai_protocol);
if (listenSocket == -1) {
socket_t listenSocket = ::socket(aip->ai_family, aip->ai_socktype, aip->ai_protocol);
if (listenSocket == INVALID_SOCKET) {
return 0;
}
if (_type == ENDPOINT_SERVER) {
// try to reuse address
int opt = 1;
@ -143,21 +143,21 @@ socket_t EndpointIp::connectSocket (const struct addrinfo* aip, double connectTi
LOGGER_TRACE << "reuse address flag set";
// server needs to bind to socket
int result = bind(listenSocket, aip->ai_addr, aip->ai_addrlen);
int result = ::bind(listenSocket, aip->ai_addr, aip->ai_addrlen);
if (result != 0) {
// error
TRI_CLOSE(listenSocket);
TRI_CLOSE_SOCKET(listenSocket);
return 0;
}
// listen for new connection, executed for server endpoints only
LOGGER_TRACE << "using backlog size " << _listenBacklog;
result = listen(listenSocket, _listenBacklog);
if (result < 0) {
TRI_CLOSE(listenSocket);
result = ::listen(listenSocket, _listenBacklog);
if (result == SOCKET_ERROR) {
TRI_CLOSE_SOCKET(listenSocket);
// todo: get the correct error code using WSAGetLastError for windows
LOGGER_ERROR << "listen failed with " << errno << " (" << strerror(errno) << ")";
return 0;
@ -169,15 +169,16 @@ socket_t EndpointIp::connectSocket (const struct addrinfo* aip, double connectTi
// set timeout
setTimeout(listenSocket, connectTimeout);
if (::connect(listenSocket, (const struct sockaddr*) aip->ai_addr, aip->ai_addrlen) != 0) {
TRI_CLOSE(listenSocket);
int result = ::connect(listenSocket, (const struct sockaddr*) aip->ai_addr, aip->ai_addrlen);
if ( result == SOCKET_ERROR) {
TRI_CLOSE_SOCKET(listenSocket);
return 0;
}
}
if (!setSocketFlags(listenSocket)) {
TRI_CLOSE(listenSocket);
if (!setSocketFlags(listenSocket)) { // set some common socket flags for a server
TRI_CLOSE_SOCKET(listenSocket);
return 0;
}
@ -239,7 +240,7 @@ socket_t EndpointIp::connect (double connectTimeout, double requestTimeout) {
return 0;
}
int listenSocket = 0;
socket_t listenSocket = 0;
// Try all returned addresses until one works
for (aip = result; aip != NULL; aip = aip->ai_next) {
@ -265,7 +266,7 @@ void EndpointIp::disconnect () {
assert(_socket);
_connected = false;
TRI_CLOSE(_socket);
TRI_CLOSE_SOCKET(_socket);
_socket = 0;
}
@ -281,6 +282,7 @@ bool EndpointIp::initIncoming (socket_t incoming) {
int res = setsockopt(incoming, IPPROTO_TCP, TCP_NODELAY, (char*) &n, sizeof(n));
if (res != 0 ) {
// todo: get correct windows error code
LOGGER_WARNING << "setsockopt failed with " << errno << " (" << strerror(errno) << ")";
return false;

1
lib/Rest/EndpointIp.h
View File

@ -21,6 +21,7 @@
///
/// Copyright holder is triAGENS GmbH, Cologne, Germany
///
/// @author Dr. O
/// @author Jan Steemann
/// @author Copyright 2012, triAGENS GmbH, Cologne, Germany
////////////////////////////////////////////////////////////////////////////////

2
lib/Rest/InitialiseRest.cpp
View File

@ -125,8 +125,10 @@ namespace triagens {
OpenSSL_add_all_algorithms();
ERR_load_crypto_strings();
#ifdef TRI_OPENSSL_VERSION
revision = "$Revision: OPENSSL " TRI_OPENSSL_VERSION " $";
LOGGER_TRACE << revision;
#endif
#ifdef TRI_LIBEV_VERSION
revision = "$Revision: LIBEV " TRI_LIBEV_VERSION " $";

97
lib/Scheduler/ApplicationScheduler.cpp
View File

@ -193,37 +193,83 @@ static SignalTask* localSignalTask;
bool CtrlHandler(DWORD eventType) {
ControlCTask* ccTask = (ControlCTask*)(localSignalTask);
string msg = ccTask->_server->getName() + " [shutting down]";
bool shutdown = false;
string shutdownMessage;
// .........................................................................
// TODO: popup message
// .........................................................................
switch (eventType) {
case CTRL_C_EVENT:
case CTRL_CLOSE_EVENT:
case CTRL_LOGOFF_EVENT:
case CTRL_BREAK_EVENT: {
//TODO: windows does not appreciate changing the environment in this manner
//TRI_SetProcessTitle(msg.c_str());
shutdown = true;
shutdownMessage = "control-break received";
break;
}
case CTRL_C_EVENT: {
//TODO: windows does not appreciate changing the environment in this manner
//TRI_SetProcessTitle(msg.c_str());
shutdown = true;
shutdownMessage = "control-c received";
break;
}
case CTRL_CLOSE_EVENT: {
//TODO: windows does not appreciate changing the environment in this manner
//TRI_SetProcessTitle(msg.c_str());
shutdown = true;
shutdownMessage = "window-close received";
break;
}
case CTRL_LOGOFF_EVENT: {
//TODO: windows does not appreciate changing the environment in this manner
//TRI_SetProcessTitle(msg.c_str());
shutdown = true;
shutdownMessage = "user-logoff received";
break;
}
case CTRL_SHUTDOWN_EVENT: {
string msg = ccTask->_server->getName() + " [shutting down]";
TRI_SetProcessTitle(msg.c_str());
if (ccTask->_seen == 0) {
LOGGER_INFO << "control-c received, beginning shut down sequence";
ccTask->_server->beginShutdown();
}
else {
LOGGER_INFO << "control-c received, terminating";
exit(EXIT_FAILURE);
}
++ccTask->_seen;
return true;
//TODO: windows does not appreciate changing the environment in this manner
//TRI_SetProcessTitle(msg.c_str());
shutdown = true;
shutdownMessage = "system-shutdown received";
break;
}
default: {
return false;
shutdown = false;
break;
}
} // end of switch statement
if (shutdown == false) {
LOGGER_ERROR << "Invalid CTRL HANDLER event received - ignoring event";
return true;
}
return false;
if (ccTask->_seen == 0) {
LOGGER_INFO << shutdownMessage << ", beginning shut down sequence";
ccTask->_server->beginShutdown();
++ccTask->_seen;
return true;
}
// ............................................................................
// user is desperate to kill the server!
// ............................................................................
LOGGER_INFO << shutdownMessage << ", terminating";
_exit(EXIT_FAILURE); // quick exit for windows
return true;
}
#endif
@ -357,7 +403,16 @@ void ApplicationScheduler::setupOptions (map<string, ProgramOptionsDescription>&
// .............................................................................
options[ApplicationServer::OPTIONS_SERVER + ":help-extended"]
#ifdef _WIN32
// ...........................................................................
// since we are trying to use libev, then only select is available
// no point in allowing this to be configured at this stage. Perhaps if we
// eventually write a native libev we can offer something.
// ...........................................................................
//("scheduler.backend", &_backend, "1: select, 2: poll, 4: epoll")
#else
("scheduler.backend", &_backend, "1: select, 2: poll, 4: epoll")
#endif
("scheduler.report-interval", &_reportIntervall, "scheduler report interval")
("server.no-reuse-address", "do not try to reuse address")
#ifdef TRI_HAVE_GETRLIMIT

6
lib/Scheduler/AsyncTask.cpp
View File

@ -58,11 +58,15 @@ void AsyncTask::signal () {
// Task methods
// -----------------------------------------------------------------------------
void AsyncTask::setup (Scheduler* scheduler, EventLoop loop) {
bool AsyncTask::setup (Scheduler* scheduler, EventLoop loop) {
this->scheduler = scheduler;
this->loop = loop;
watcher = scheduler->installAsyncEvent(loop, this);
if (watcher == -1) {
return false;
}
return true;
}

2
lib/Scheduler/AsyncTask.h
View File

@ -79,7 +79,7 @@ namespace triagens {
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
void setup (Scheduler*, EventLoop);
bool setup (Scheduler*, EventLoop);
////////////////////////////////////////////////////////////////////////////////
/// {@inheritDoc}

15
lib/Scheduler/ListenTask.cpp
View File

@ -41,6 +41,7 @@
#ifdef TRI_HAVE_WINSOCK2_H
#include "BasicsC/win-utils.h"
#include <WinSock2.h>
#include <WS2tcpip.h>
#endif
@ -93,15 +94,20 @@ bool ListenTask::isBound () const {
// Task methods
// -----------------------------------------------------------------------------
void ListenTask::setup (Scheduler* scheduler, EventLoop loop) {
bool ListenTask::setup (Scheduler* scheduler, EventLoop loop) {
if (! isBound()) {
return;
return true;
}
this->scheduler = scheduler;
this->loop = loop;
readWatcher = scheduler->installSocketEvent(loop, EVENT_SOCKET_READ, this, listenSocket);
if (readWatcher == -1) {
return false;
}
return true;
}
@ -126,7 +132,6 @@ bool ListenTask::handleEvent (EventToken token, EventType revents) {
// accept connection
socket_t connfd = accept(listenSocket, (sockaddr*) &addr, &len);
if (connfd == INVALID_SOCKET) {
++acceptFailures;
@ -150,7 +155,7 @@ bool ListenTask::handleEvent (EventToken token, EventType revents) {
int res = getsockname(connfd, (sockaddr*) &addr_out, &len_out);
if (res != 0) {
TRI_CLOSE(connfd);
TRI_CLOSE_SOCKET(connfd);
LOGGER_WARNING << "getsockname failed with " << errno << " (" << strerror(errno) << ")";
@ -162,7 +167,7 @@ bool ListenTask::handleEvent (EventToken token, EventType revents) {
// disable nagle's algorithm, set to non-blocking and close-on-exec
bool result = _endpoint->initIncoming(connfd);
if (!result) {
TRI_CLOSE(connfd);
TRI_CLOSE_SOCKET(connfd);
// TODO GeneralFigures::incCounter<GeneralFigures::GeneralServerStatistics::connectErrorsAccessor>();

2
lib/Scheduler/ListenTask.h
View File

@ -103,7 +103,7 @@ namespace triagens {
/// Note that registerTask must only be called when the socket is bound.
////////////////////////////////////////////////////////////////////////////////
void setup (Scheduler*, EventLoop);
bool setup (Scheduler*, EventLoop);
////////////////////////////////////////////////////////////////////////////////
/// {@inheritDoc}

6
lib/Scheduler/PeriodicTask.cpp
View File

@ -63,11 +63,15 @@ void PeriodicTask::resetTimer (double offset, double intervall) {
// Task methods
// -----------------------------------------------------------------------------
void PeriodicTask::setup (Scheduler* scheduler, EventLoop loop) {
bool PeriodicTask::setup (Scheduler* scheduler, EventLoop loop) {
this->scheduler = scheduler;
this->loop = loop;
watcher = scheduler->installPeriodicEvent(loop, this, offset, intervall);
if (watcher == -1) {
return false;
}
return true;
}

2
lib/Scheduler/PeriodicTask.h
View File

@ -78,7 +78,7 @@ namespace triagens {
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
void setup (Scheduler*, EventLoop);
bool setup (Scheduler*, EventLoop);
////////////////////////////////////////////////////////////////////////////////
/// {@inheritDoc}

83
lib/Scheduler/SchedulerLibev.cpp
View File

@ -260,6 +260,7 @@ SchedulerLibev::SchedulerLibev (size_t concurrency, int backend)
: Scheduler(concurrency),
_backend(backend) {
//_backend = 1;
// setup lock
SCHEDULER_INIT(&_watcherLock);
@ -530,28 +531,72 @@ EventToken SchedulerLibev::installSignalEvent (EventLoop loop, Task* task, int s
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
EventToken SchedulerLibev::installSocketEvent (EventLoop loop, EventType type, Task* task, socket_t fd) {
SocketWatcher* watcher = new SocketWatcher;
watcher->loop = (struct ev_loop*) lookupLoop(loop);
watcher->task = task;
watcher->token = registerWatcher(watcher, EVENT_SOCKET_READ);
#ifdef _WIN32
int flags = 0;
// ..........................................................................
// Windows likes to operate on SOCKET types (sort of handles) while libev
// likes to operate on file descriptors
// ..........................................................................
EventToken SchedulerLibev::installSocketEvent (EventLoop loop, EventType type, Task* task, socket_t socket) {
SocketWatcher* watcher = new SocketWatcher;
watcher->loop = (struct ev_loop*) lookupLoop(loop);
watcher->task = task;
int flags = 0;
if (type & EVENT_SOCKET_READ) {
flags |= EV_READ;
if (type & EVENT_SOCKET_READ) {
flags |= EV_READ;
}
if (type & EVENT_SOCKET_WRITE) {
flags |= EV_WRITE;
}
// ..........................................................................
// The problem we have here is that this opening of the fs handle may fail.
// There is no mechanism to the calling function to report failure.
// ..........................................................................
LOGGER_TRACE << "attempting to convert socket handle to socket descriptor";
int fd = _open_osfhandle (socket, 0);
if (fd == -1) {
LOGGER_ERROR << "could not convert socket handle to socket descriptor";
delete watcher;
abort();
// Dr. O TODO: return to calling function
return -1;
}
watcher->token = registerWatcher(watcher, EVENT_SOCKET_READ);
ev_io* w = (ev_io*) watcher;
ev_io_init(w, socketCallback, fd, flags);
ev_io_start(watcher->loop, w);
return watcher->token;
}
#else
EventToken SchedulerLibev::installSocketEvent (EventLoop loop, EventType type, Task* task, socket_t fd) {
SocketWatcher* watcher = new SocketWatcher;
watcher->loop = (struct ev_loop*) lookupLoop(loop);
watcher->task = task;
watcher->token = registerWatcher(watcher, EVENT_SOCKET_READ);
int flags = 0;
if (type & EVENT_SOCKET_WRITE) {
flags |= EV_WRITE;
if (type & EVENT_SOCKET_READ) {
flags |= EV_READ;
}
if (type & EVENT_SOCKET_WRITE) {
flags |= EV_WRITE;
}
ev_io* w = (ev_io*) watcher;
ev_io_init(w, socketCallback, fd, flags);
ev_io_start(watcher->loop, w);
return watcher->token;
}
ev_io* w = (ev_io*) watcher;
ev_io_init(w, socketCallback, fd, flags);
ev_io_start(watcher->loop, w);
return watcher->token;
}
#endif
////////////////////////////////////////////////////////////////////////////////
/// {@inheritDoc}
@ -657,7 +702,7 @@ void SchedulerLibev::rearmTimer (EventToken token, double timeout) {
void* SchedulerLibev::lookupWatcher (EventToken token) {
SCHEDULER_LOCK(&_watcherLock);
if (token >= _watchers.size()) {
if (token >= (EventToken) _watchers.size()) {
SCHEDULER_UNLOCK(&_watcherLock);
return 0;
}
@ -675,7 +720,7 @@ void* SchedulerLibev::lookupWatcher (EventToken token) {
void* SchedulerLibev::lookupWatcher (EventToken token, EventType& type) {
SCHEDULER_LOCK(&_watcherLock);
if (token >= _watchers.size()) {
if (token >= (EventToken) _watchers.size()) {
SCHEDULER_UNLOCK(&_watcherLock);
return 0;
}

3
lib/Scheduler/SignalTask.cpp
View File

@ -82,7 +82,7 @@ bool SignalTask::addSignal (int signal) {
// Task methods
// -----------------------------------------------------------------------------
void SignalTask::setup (Scheduler* scheduler, EventLoop loop) {
bool SignalTask::setup (Scheduler* scheduler, EventLoop loop) {
this->scheduler = scheduler;
this->loop = loop;
@ -91,6 +91,7 @@ void SignalTask::setup (Scheduler* scheduler, EventLoop loop) {
for (set<int>::iterator i = signals.begin(); i != signals.end() && pos < MAX_SIGNALS; ++i, ++pos) {
watcher[pos] = scheduler->installSignalEvent(loop, this, *i);
}
return true;
}

2
lib/Scheduler/SignalTask.h
View File

@ -91,7 +91,7 @@ namespace triagens {
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
void setup (Scheduler*, EventLoop);
bool setup (Scheduler*, EventLoop);
////////////////////////////////////////////////////////////////////////////////
/// {@inheritDoc}

19
lib/Scheduler/SocketTask.cpp
View File

@ -80,8 +80,8 @@ SocketTask::SocketTask (socket_t fd, double keepAliveTimeout)
SocketTask::~SocketTask () {
if (commSocket != -1) {
TRI_CLOSE(commSocket);
//close(commSocket);
TRI_CLOSE_SOCKET(commSocket);
commSocket = -1;
}
if (_writeBuffer != 0) {
@ -149,11 +149,12 @@ void SocketTask::setKeepAliveTimeout (double timeout) {
bool SocketTask::fillReadBuffer (bool& closed) {
closed = false;
int nr = TRI_READ(commSocket, tmpReadBuffer, READ_BLOCK_SIZE);
int nr = TRI_READ_SOCKET(commSocket, tmpReadBuffer, READ_BLOCK_SIZE, 0);
if (nr > 0) {
_readBuffer->appendText(tmpReadBuffer, nr);
return true;
}
else if (nr == 0) {
@ -164,6 +165,7 @@ bool SocketTask::fillReadBuffer (bool& closed) {
return false;
}
else {
if (errno == EINTR) {
return fillReadBuffer(closed);
}
@ -204,7 +206,7 @@ bool SocketTask::handleWrite (bool& closed, bool noWrite) {
int nr = 0;
if (0 < len) {
nr = TRI_WRITE(commSocket, _writeBuffer->begin() + writeLength, (int) len);
nr = TRI_WRITE_SOCKET(commSocket, _writeBuffer->begin() + writeLength, (int) len, 0);
if (nr < 0) {
if (errno == EINTR) {
@ -406,20 +408,23 @@ bool SocketTask::hasWriteBuffer () const {
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
void SocketTask::setup (Scheduler* scheduler, EventLoop loop) {
bool SocketTask::setup (Scheduler* scheduler, EventLoop loop) {
this->scheduler = scheduler;
this->loop = loop;
watcher = scheduler->installAsyncEvent(loop, this);
readWatcher = scheduler->installSocketEvent(loop, EVENT_SOCKET_READ, this, commSocket);
writeWatcher = scheduler->installSocketEvent(loop, EVENT_SOCKET_WRITE, this, commSocket);
if (readWatcher == -1 || writeWatcher == -1) {
return false;
}
// install timer for keep-alive timeout with some high default value
keepAliveWatcher = scheduler->installTimerEvent(loop, this, 60.0);
// and stop it immediately so it's not actively at the start
scheduler->clearTimer(keepAliveWatcher);
tid = Thread::currentThreadId();
return true;
}
////////////////////////////////////////////////////////////////////////////////

9
lib/Scheduler/SocketTask.h
View File

@ -35,6 +35,10 @@
#include "Basics/Thread.h"
#include "Statistics/StatisticsAgent.h"
#ifdef _WIN32
#include "BasicsC/win-utils.h"
#endif
// -----------------------------------------------------------------------------
// --SECTION-- forward declarations
// -----------------------------------------------------------------------------
@ -237,7 +241,7 @@ namespace triagens {
/// {@inheritDoc}
////////////////////////////////////////////////////////////////////////////////
void setup (Scheduler*, EventLoop);
bool setup (Scheduler*, EventLoop);
////////////////////////////////////////////////////////////////////////////////
/// {@inheritDoc}
@ -295,7 +299,8 @@ namespace triagens {
////////////////////////////////////////////////////////////////////////////////
socket_t commSocket;
int fileDescriptor;
////////////////////////////////////////////////////////////////////////////////
/// @brief keep-alive timeout in seconds
////////////////////////////////////////////////////////////////////////////////

2
lib/Scheduler/Task.h
View File

@ -119,7 +119,7 @@ namespace triagens {
/// belongs to the loop parameter.
////////////////////////////////////////////////////////////////////////////////
virtual void setup (Scheduler*, EventLoop) = 0;
virtual bool setup (Scheduler*, EventLoop) = 0;
////////////////////////////////////////////////////////////////////////////////
/// @brief called to clear the callback information

5
lib/Scheduler/TaskManager.cpp
View File

@ -49,7 +49,10 @@ void TaskManager::deleteTask (Task* task) {
void TaskManager::setupTask (Task* task, Scheduler* scheduler, EventLoop loop) {
task->setup(scheduler, loop);
string name = task->getName();
bool ok = task->setup(scheduler, loop);
// TODO: respond when not ok
(void) ok;
}

Some files were not shown because too many files have changed in this diff Show More