COROIPC_OVERVIEW(8)        Corosync Cluster Engine Programmer's Manual        COROIPC_OVERVIEW(8)



NAME
       coroipc_overview - Overview of coroipc libraries

OVERVIEW
       The  coroipcs  and coroipcc libraries provide a generically reusable very high performance
       shared memory IPC sytem for client and service applications.  It  supports  many  features
       including:

       65536 user services and 65536 command ids per service.

       Shared memory implementation for very high performance.

       A  synchronous  request/response channel and asynchronous response channel per ipc connec‐
       tion.

       User defined private data per IPC connection.

       Ability to call a function per service on ipc connection and disconnection.

       Authenticated IPC connection with ability for developer to define which UIDs and GIDs  are
       valid at connection time.

       Fully abstracted poll system so that any poll library may be used.

       User defined selector for determining the proper function to call per service and id.


Description of the libraries
       There are two shared libraries available for developing IPC client applications.

       The first library is coroipcs.so which is responsible for the server implementation.  This
       library should be linked with the server and then initialized with coroipcs_init(3).

       Once the library is initialized, it will provide service to coroipcc.so library users.

       The second library is coroipcc.so which is  responsible  for  the  client  implementation.
       This  library  should  be  linked  with  the  client and requires no initialization.  This
       library provides communication functions for sending and receiving  synchronous  requests,
       and also reading asynchronous message requests from the server.


Initializing the coroipcs library
       To  use  the  coroipcs  library, the developer creates a coroipcs_init_state structure and
       populates it with function names.   The  functions  do  various  operations  described  in
       coroipcs_init(3)  man  page.   Not all operations must be specified.  If some are missing,
       the corosync ipcs library will automatically populate the structure with internal versions
       which provide basic functionality.


Communicating with the coroipcc clients
       Every ipc connection is represented by a void * pointer which uniquely identifies the data
       set for the IPC connection.  Each IPC connection also contains user defined private  data.
       To  obtain  this  private data pointer, the function coroipcs_private_data_get(3) function
       can be called.

       There are two  channels  for  communication.   The  primary  channel  is  the  synchronous
       request/response  channel.   Another  channel  is  available  for out of band asynchronous
       responses from the server.

       To  send  a  response   on   the   syncronous   channel,    coroipcs_response_send(3)   or
       coroipcs_response_iov_send(3) should be used.  To send a message on the asynchronous chan‐
       nel, coroipcs_dispatch_send(3) or coroipc_dispatch_iov_send(3) should be used.


The abstracted poll system
       There are many different poll systems in use in applications.   They  are  usually  intri‐
       cately  tied  up  in the implementation of the application and each provide different APIs
       and  functionality.   To  manage  this,  the  coroipcs  library  provides   callbacks   in
       coroipcs_init(3)  which  should be called when a new connection should be added for accept
       system calls or to dispatch messages.

       These callbacks add the relevant fd to the application's poll system.  When  the  applica‐
       tion  poll  system  triggers the callback registered by the user defined poll adding func‐
       tions, they then call either coroipc_handler_accept(3) or coroipc_handler_dispatch(3).


Initializing the coroipcc library
       No initialization is required in the coroipcc library.


Lifecycle of an IPC connection.
       An IPC connection is made to the server with coroipcc_service_connect(3).   This  function
       connects to the server and requests channels be created for communication.  To disconnect,
       the client either exits or executes the function coroipcc_service_disconnect(3).


Synchronous communication
       There are two functions for sending a request and receiving a response.  The  first  func‐
       tion coroipcc_msg_send_reply_receive(3) sends an iovector request and receives a response.
       This function  copies  the  response  into  the  response  buffer.   the  second  function
       coroipcc_msg_end_reply_receive_in_buf(3)  does not copy the response buffer and allows for
       zero-copy reading of the response when the lifetime of the response buffer is known.


Asynchronous communication
       The  coroipcc_dispatch_recv(3)  function  receives  an  out-of-band  asyncronous  message.
       Unlike the synchronous communication channel, the asynchronous messages are queued and can
       provide very high out-of-band performance.

       To determine when to call coroipcc_dispatch_recv(3) the corosync_fd_get(3) call is used to
       obtain a file descriptor used in the poll(2) or select(2) system calls.

       Finally  the  current  dispatch flow control state can be obtained with coroipcc_flow_con‐
       trol_get(3).


Performance
       The ipc system is tuned for very high performance while also  being  comletely  abstracted
       from  the  underlying poll mechanism and any internalisms required by the server.  The ipc
       system achieves such high performance by using shared memory as oppossed to  slower  tech‐
       niques such as UNIX_PF sockets.

       We  intend to do further development to allow syncronous requests to return messages in an
       asyncronous way to avoid blocking involved in the syncronous request/response  model  used
       today for higher throughput in some use cases.


Security
       The  ipc system uses default operating system security mechanics to ensure ipc connections
       are validated.  A callback used with coroipcs_init(3) is called for every new ipc  connec‐
       tion  with the parameters of UID and GID.  The callback then determines if the UID and GID
       are authenticated for  communication.   More  about  this  topic  can  be  viewed  in  the
       coroipcs_init(3) man page.


SEE ALSO
       coroipcs_ipc_init(3),          coroipcs_ipc_exit(3),         coroipcs_private_data_get(3),
       coroipcs_respone_send(3),    coroipcs_response_iov_send(3),     coroipcs_dispatch_send(3),
       coroipcs_dispatch_iov_send(3),     coroipcs_refcount_inc(3),     coroipcs_refcount_dec(3),
       coroipcs_handler_accept(3), coroipcs_handler_dispatch(3),

       cooripcc_service_connect(3),                               coroipcc_service_disconnect(3),
       coroipcc_msg_send_reply_receive(3),             coroipcc_msg_send_reply_receive_in_buf(3),
       coroipcc_dispatch_recv(3), coroipcc_fd_get(3), coroipcc_dispatch_flow_control_get(3)



corosync Man Page                           2009-03-21                        COROIPC_OVERVIEW(8)