C Bindings – Java

This is the second post in this serie. In this post I’m going to show how to create a C library for usage in Java.

Why would we want to do such a thing? Java is a platform-independent language, and C is platform dependent, so when we use a C library in Java, we’re binding ourselves to the platform this library is built for.
But when our project is intended to run on a dedicated platform, there is no such problem, and if this platform has some special hardware, or if we want to enhance our SW performance, we may have to use C. And if this platform runs Android OS, this becomes very useful, as we can develope the low-level stuff in C, and bind it to the application which is always written in Java.

We will make use of the “Java Native Interface” (JNI) framework in order to build our library. “Native” here stands for “platform-specific”. There’s an excellent Wikipedia page on JNI, I recommend reading it.

The code for this post can be found on github.

The header file

First thing we need to do is to include “jni.h”, this file contains all the definition we need for developing a Java module in C. Let’s look at the declaration of the function in our header file, and see some of these definitions:

JNIEXPORT jint JNICALL Java_LinuxSocket_open(JNIEnv *env, jobject obj,
                                             jstring jSocketName);
JNIEXPORT void JNICALL Java_LinuxSocket_close(JNIEnv *env, jobject obj,
                                              jint jSocket);
JNIEXPORT jstring JNICALL Java_LinuxSocket_read(JNIEnv *env, jobject obj,
                                                jint jSocket);
JNIEXPORT jint JNICALL Java_LinuxSocket_write(JNIEnv *env, jobject obj,
                                              jint jSocket, jstring jBuffer);
  • First thing we notice here is “JNIEXPORT” and “JNICALL” – This is how we make a “Java method” out of our C function.
  • Second thing is the function name. The underscores replace the dot operator, so we’re actually creating “Java.LinuxSocket.open”, where “LinuxSocket” is our library and “open” is a method in this library.
  • The first argument to each one of these function is a pointer to “JNIEnv”, this is our “link” to the Java runtime – We can use this parameter for creating and deleting Java objects (such as exceptions and strings), call Java objects’ methods, etc.
    The second parameter is a “jobject”, and this is the object which this method belongs to (we are not making use of it in our code)
  • We can see that we receive and return Java defined types: “jint”, “jstring” and so on.

Implementation

Let’s look at the “open” method:

#define MESSAGE_MAX_SIZE 256

JNIEXPORT jint JNICALL Java_UnixSocket_open(JNIEnv *env, jobject obj,
                                             jstring jSocketName)
{
    char message[MESSAGE_MAX_SIZE];

    const char *socket_name = (*env)->GetStringUTFChars(env, jSocketName, NULL);
    int s = socket_open(socket_name);
    
    if (s <= 0) {         jclass Exception = (*env)->FindClass(env, "java/lang/Exception");
        snprintf(message, MESSAGE_MAX_SIZE, "Couldn't open socket %s", socket_name);
        (*env)->ThrowNew(env, Exception, message);
    }

    (*env)->ReleaseStringUTFChars(env, jSocketName, socket_name);
    return (jint)s;
}

Right on the start we can see usage of the “JNIEnv” parameter. We use it for converting a Java string into a C-Style string. Notice that when we’re finished, we need to release this string, as we do right before we return.
We use “socket_open” which we get out of our marvelous unix-socket-client static library, and then we check for errors Java-style. In Java, a fully object-oriented language, the proper way of handling errors is by throwing an exception. So again, we’re using our “JNIEnv” parameter in order to create an exception object which we’re going to throw.
Even if an exception was thrown, the function is going to continue till its end (unlike the normal behaviour inside Java itself), so we have our chance to release the string before returning. In such case (exception thrown), there will be no meaning for the return value.

The rest of the functions are more or less the same, so we’ll leave them to the reader (as they say in math books), and go ahead to see how we’re going to use this library in Java.

By building our library as a shared object, and installing it at the proper place, we’re making our library available for Java applications.

The Java part

We create a class named “UnixSocket” – Same as our library functions (java.UnixSocket.*).

public class UnixSocket

In its “static” section we load our shared library.

static {
    try {
        System.loadLibrary("unixsockets");
    } catch (UnsatisfiedLinkError e) {
        System.err.println("Error: UnixSockets JNI Library failed to load\n" + e);
        System.exit(1);
    }
 }

The library functions are declared as “native”, so that Java knows that these are implemented in the JNI layer.

native private int open(String socketName);
native private void close(int socket);
native private String read(int socket);
native private int write(int socket, String out);

Our class saves the returned file descriptor as an int (fd). Here is the implementation of the class’s constructor, which calls “open”. Although it looks simple (as it actually is), you can see that it throws an exception. This is the exception we discussed in the C implementation above.

public UnixSocket(String socketName) throws Exception {
    fd = open(socketName);
 }

Testing

As I mentioned, the way I chose to implement this, “open” is called with the class constructor – We pass the socket name to the constructor, then “open” is called. Let’s see how this works out in the test program.

UnixSocket socket;
try {
    socket = new UnixSocket(args[0]);
} catch (Exception e) {
    System.out.println(e.getMessage());
    return;
}

If “open” succeeds, then socket is created successfully, otherwise, “open” (at the very bottom layer) will throw an exception, which is caught here. This exception holds an error message, and we can print it out

Now, the output of the Java test program – The program sends a (silly) string to the listener, and print out the message it got back from the listener:

$ java UnixSocketTest /tmp/socket1
Got response: Recieved 16 Byte
Got response: Recieved 15 Byte
Got response: Recieved 9 Byte
Got response: Recieved 8 Byte

And the output of the listener:

$ ./listenerd /tmp/socket1
[LISTENER] INFO  Starting "Unix Domain Sockets" demo listener
[LISTENER] INFO  * Starting communication with a client *
[LISTENER] INFO  Incoming message: Try this string
[LISTENER] INFO  Incoming message: Maybe this one
[LISTENER] INFO  Incoming message: Have fun
[LISTENER] INFO  Incoming message: Bye bye
[LISTENER] INFO  Client socket was closed by peer

Yes! Everything works!
Till next time,
Amnon.

C Bindings – Lua

This is the first post of a serie I’m going to write on binding C to other programing languages (or script languages – I’m disregarding the differences here).

High-level languages often offer an API for creating libraries written in C. Why would we want this?
One thing is performance – With C you can write performance-oriented code in order to do a specific calculation/operation much faster, and even incorporate assembly code and other machine-specific accelerations. With C we can also optimize the amount of RAM usage.
Another thing is that languages such as Java or Python are not bound to the machine type or OS, and sometimes you want to use features that are unique to the platform you’re working on, like a specific processor features, OS mechanisms or HW features, and using C you can have access to the system’s resources.

And the downside of this is, of course, that we bind our implementation to a specific platform. We can, obviously, compile our C library for every platform we intend our software to run on, but this can be very hard, up to impossible. For example, creating such a library for Java on an Android platform – If you don’t have administrator privileges, and most users don’t, you won’t be able to even install it.

I’m going to show examples for binding C with a few languages. I will use the client side library of the unix sockets example from a previous post, and I will build a client in the particular language, which can communicate with the unix sockets listener.

The code for this serie of posts can be found on github, and it will be updated as I go and add more implementation for other languages.

Lua

Introduction

Lua is a small-footprint scripting language, intended (mostly) for embedded platforms. Although it has a very compact syntax, it is quite powerful. It is developed by the “Pontifical Catholic University of Rio de Janeiro in Brazil”, and has a complete book, free online, for version 5.1 (currently the latest version is 5.3.3), you can find it in the link.
I’m sure, although I didn’t check, that there’s a Lua library for unix domain sockets, but I’m doing this for the exercise.

Including a dynamic library in Lua

In Lua, we include other libraries using “require”, like so:

local sockets = require('somelib')

“somelib” might be either a “.lua” file or a “.so” file. An so file is a dynamic library, which in Lua’s case, uses Lua’s infrastructure and adhere to some special conventions.
Lua’s interpreter looks for the file in a few predefined locations, depending on where Lua is installed (and also in the calling file’s path).

“Unix Sockets” (client) Library

We would like our API to implement these features:

  • “Open”: Function for opening a socket for communication
    Input: Socket “file” name
    Output: Socket Object when pass ; “nil” on fail
  • Socket Object
    Holds the socket’s file descriptor, and implements the following methods:

    • “Write”: Writes data to socket
      Input: Data
      Output: Pass/Fail
    • “Read”: Read data from socket:
      Input: None
      Output: Data/nil
    • “Close”: Close the socket.
      No input nor output

    I’m actually cheating a little here. Lua doesn’t really have objects in its full sense, so all the above methods will have to take the object itself as the first parameter. This object will be implemented using a Lua table (no Lua metatables here – keeping it simple)

Lua “main”

The function that gets called when you require a Lua dynamic library must be in the form of:

int luaopen_somelib(lua_State *L)

Where “somelib” is the exact name of the so (that is “somelib.so”).

“L” has an important role – It holds the arguments’ stack. So we need to “pop” our inputs from it, and “push” our outputs into it. The actual C return value (int) is the number of argument we’re returning Lua-wise, that is, the number of parameters we pushed into the stack.

Our main looks like this:

static const luaL_Reg unix_sockets_lib[] = {
    {"open",  unix_sockets_open },
    {NULL, NULL}
};

int luaopen_libunixsockets(lua_State *L)
{
    luaL_newlib(L, unix_sockets_lib);

    return 1;
}

“luaL_newlib” hides some functionality here. It creates a new Lua table, adds

the functions from”unix_socket_lib” to this table, and pushes this table into the stack.
We return “1” because there’s one return parameter on the stack.

The list of functions itself – “unix_sockets_lib”, contain one function which is the implementation for “open”, and as Lua defines, it is terminated by {NULL, NULL}.

Open

static int unix_sockets_open(lua_State *L)
{
    const char *name = lua_tostring(L, 1);
    int sock = socket_open(name);
    if (sock < 0) {
        lua_pushnil(L);
        return 1;
    }

    lua_newtable(L);
    luaL_setfuncs(L, socket_methods, 0);
    lua_pushinteger(L, sock);
    lua_setfield(L, 2, "fd");

    return 1;
}

The only input parameter for this function is a string, and “lua_tostring” is the function that brings us this parameter from the stack. Similar functions exist for every other Lua type.
We use “socket_open” from our unix_sockets static library in order to open the socket, and if we fail, we push “nil” to the stack, and return 1 (the number of return values).
If the operation passes, however, we create a new table, using “lua_newtable”. This table is automatically pushed into the stack, and the operation we are going to perform on it will be performed inside the stack.
This table is our “socket object”, so we need to attach its methods, and we do this by “luaL_setfuncs”.
Then we want to save the fd in this table. Inserting a new field to a table is done by pushing the field to the stack, and then “merging” it into the table. You see how it goes there – We push the integer, and then we set the field “fd” of the 2nd object in the stack (the table, the 1st is the integer) to be the pushed value.

Notice that Lua library functions has two different prefixes here. The ones starting with “lua” belong to the basic infrastructure included by “lua.h”. The ones starting with “luaL” belong to an extension library, included by “luaxlib.h”.

Write

static int unix_sockets_write(lua_State *L)
{
    int res;
    int sock = lua_getfield(L, 1, "fd");
    const char *buffer = lua_tostring(L, 2);

    res = socket_write(sock, buffer, strlen(buffer));
    if (res <= 0)
        lua_pushnil(L);
    else
        lua_pushboolean(L, 1);
    return 1;
}

As we defined, “write” takes two parameters. The first is the socket “object”, and the second is a string (data).
First, we take extract the fd from the table, “lua_getfield” will do this for us. Then we take out the string using “lua_tostring”.
Then, we use “socket_write” from our library, and push the proper result to the stack – nil upon failure, “true” upon success.

“read” and “close” are based on the same concepts, you can review the code if you want.

The Lua script – Using the C library

As you’ll see, this is quite straightforward. All the real operation are well hidden in the C layer. All we have to do now is call one of our API functions.

Including the library:

local sockets = require('libunixsockets')

Opening a new socket:

local s = sockets.open(sock_name)

Writing/Reading:

local e = sock:write(v)
    if not e then
    print('Failed sending ' .. v .. ' to listener')
    break -- This happens inside a loop
end

local resp = sock:read()
print('* Sent "' .. v .. '", listener reponse: "' .. tostring(resp) .. '"')

Closing:

s:close()

Testing

Here is the output test run of our script:

Opened socket "/tmp/sock1"
* Sent "Test string #1", listener response: "Recieved 14 Bytes"
* Sent "Another try", listener response: "Recieved 11 Bytes"
* Sent "This is the last message in this test!", listener response: "Recieved 38 Bytes"
Socket closed

On the server’s side we’ll see:

$ ./listenerd /tmp/sock1
[LISTENER] INFO  Starting "Unix Domain Sockets" demo listener
[LISTENER] INFO  * Starting communication with a client *
[LISTENER] INFO  Incoming message: Test string #1
[LISTENER] INFO  Incoming message: Another try
[LISTENER] INFO  Incoming message: This is the last message in this test!
[LISTENER] INFO  Client socket was closed by peer
^C[LISTENER] INFO  Terminating service

That’s it, I hope you liked it. In the next post I’ll demonstrate the same for Java.
Amnon.

Unix Domain Sockets

“Unix domain sockets” is a powerful socket-based IPC (inter-process communication) mechanism, it uses the same-old sockets API, and is simple to use… well, it has many features, so getting deep in it might be a bit complicated, but the basic use of Unix domain sockets it quite simple.

The man page is excellent, and it has a very good code example, almost as good as mine 🙂

All the code for this post can be found in github.

Unix-domain server

Basically, we create a “server” by opening a socket (either stream or datagram) and linking it to an address, which is actually a filename in our filesystem*. Then we listen for clients and wait for incoming data. Exactly as we do with network sockets.

* There are other binding options, but I’m going to review the filesystem one here.

In my example code, my listener waits for a client (using “accept”), and when a client connect, it handles the connection until the client closes (not optimal, because a client can starve other clients, but this is only an example).
In order to terminate the listener, it handles “Ctrl-C” – when this signal is received, it exits gracefully.

So, this is what we need to do:

  • Step 1: Create a socket and bind it to an address (a filename)
    1. Creating the socket:
       int sock = socket(AF_UNIX, SOCK_STREAM, 0);

      We are using the “AF_UNIX” family, which is the Unix-domain family, with SOCK_STREAM. We can also use SOCK_DGRAM if we want a datagram socket.

    2. Binding the socket:
       res = bind_address(sock, path);

      Where “sock” is the result from the call to “socket()”, and “path” is the filename that represents the socket. This filename mustn’t exist, therefore, we call “unlink(path)” before calling bind, to be sure that the filename is cleared.

    3. Listen:
       res = listen(sock, MAX_CLIENTS);

      “MAX_CLIENTS” is the length of the clients queue – We’re handling one client at a time.

  • Step 2: In a loop, wait for clients
     while (!quit) {
         int client_socket = accept(sock, NULL, NULL);
         if (client_socket < 0) {          if (errno != EINTR)              continue;          LOGE("Failed accepting client (%d: %s)", errno, strerror(errno));          break;      }      quit = handle_client(client_socket) > 0 ? 1 : 0;
         if (quit) 
             LOGI("Got termination command, exiting");
    
         close(client_socket);
    }
    

    Notes:
    Our socket is blocking, as we didn’t define it otherwise, so “accept()” will block until a client tries to connect.
    We exit only if accept fails on EINTR because we don’t want just any client connection error to kill our service.
    handle_client handles the input/output from/to the client. I defined it, arbitrarily, so that if it returns any positive value, then the listener exits.

    Regarding listener ending, I think it is important to define a SIGTERM handler, so that if the program needs to exit, it will do so gracefully (closing sockets and unlinking the address).

  • Step 3: When a client connects, receive data from it and send back result.
    This is really trivial, and it happens in “handle_client()”. We read the input data like so:

     len = read(sock, buff, MAX_BUFFER_LEN-1);

    Where “sock” is the client socket’s file descriptor that we received from “accept()”.
    Now we do all our processing on the input, and when we want to send data back to the client, we use “write”:

     res = write(sock, out_buff, outbuff_len);

    We can repeat reading/writing as long as the client is connected, and when we decide to terminate the connection, we close the client socket:

     close(client_socket);

Easy enough.

Unix-domain client

Well, this is even easier than creating a server.

  • Step 1: Create a new Unix-domain socket and connect to the server
    1. Creating a new client socket is exactly the same as for the server
    2. Connecting:
      struct sockaddr_un address;
      
      memset(&address, 0x00, sizeof(address));
      address.sun_family = AF_UNIX;
      strncpy(address.sun_path, socket_name, SOCKET_NAME_MAX_LEN);
      
      res = connect(sock, (struct sockaddr *)&address, sizeof(address));
  • Step 2: Read/Write data
    Again, exactly the same as explained above
  • Step 3: Disconnect
    Dito – Use close.

Execution example

Listener

$ ./listener /tmp/sock1
[LISTENER] INFO  Starting "Unix Domain Sockets" demo listener
[LISTENER] INFO  * Starting communication with a client *
[LISTENER] INFO  Incoming message: Test this string
[LISTENER] INFO  Client socket was closed by peer
[LISTENER] INFO  * Starting communication with a client *
[LISTENER] INFO  Incoming message: Take another one
[LISTENER] INFO  Client socket was closed by peer
^C[LISTENER] INFO  Terminating service

Client

$ ./test_client /tmp/sock1 "Test this string"
INFO  Starting "Unix Domain Sockets" demo client
INFO  Got back: Got your message
$ ./test_client /tmp/sock1 "Take another one"
INFO  Starting "Unix Domain Sockets" demo client
INFO  Got back: Got your message

Just what we wanted!
Until next time,
Amnon.