Difference between API and ABI

StackOverflow https://stackoverflow.com/questions/3784389

  •  05-10-2019
  •  | 
  •  

Question

I am new to linux system programming and I came across API and ABI while reading Linux System Programming.

Definition of API :

An API defines the interfaces by which one piece of software communicates with another at the source level.

Definition of ABI :

Whereas an API defines a source interface, an ABI defines the low-level binary interface between two or more pieces of software on a particular architecture. It defines how an application interacts with itself, how an application interacts with the kernel, and how an application interacts with libraries.

How can a program communicate at a source level ? What is a source level ? Is it related to source code in anyway? Or the source of the library gets included in the main program ?

The only difference I know is API is mostly used by programmers and ABI is mostly used by compiler.

Was it helpful?

Solution

The API is what humans use. We write source code. When we write a program and want to use some library function we write code like:

 long howManyDecibels = 123L;
 int ok = livenMyHills( howManyDecibels);

and we needed to know that there is a method livenMyHills(), which takes a long integer parameter. So as a Programming Interface it's all expressed in source code. The compiler turns this into executable instructions which conform to the implementation of this language on this particular operating system. And in this case result in some low level operations on an Audio unit. So particular bits and bytes are squirted at some hardware. So at runtime there's lots of Binary level action going on which we don't usually see.

OTHER TIPS

API: Application Program Interface

This is the set of public types/variables/functions that you expose from your application/library.

In C/C++ this is what you expose in the header files that you ship with the application.

ABI: Application Binary Interface

This is how the compiler builds an application.
It defines things (but is not limited to):

  • How parameters are passed to functions (registers/stack).
  • Who cleans parameters from the stack (caller/callee).
  • Where the return value is placed for return.
  • How exceptions propagate.

I mostly come across these terms in the sense of an API-incompatible change, or an ABI-incompatible change.

An API change is essentially where code that would have compiled with the previous version won't work anymore. This can happen because you added an argument to a function, or changed the name of something accessible outside of your local code. Any time you change a header, and it forces you to change something in a .c/.cpp file, you've made an API-change.

An ABI change is where code that has already been compiled against version 1 will no longer work with version 2 of a codebase (usually a library). This is generally trickier to keep track of than API-incompatible change since something as simple as adding a virtual method to a class can be ABI incompatible.

I've found two extremely useful resources for figuring out what ABI compatibility is and how to preserve it:

this is my layman explanations:

  • api - think include files. they provide programming interface
  • abi - think kernel module. when you run it on some kernel, they has to agree how to communicate without include files, ie as low-level binary interface

(Application Binary Interface) A specification for a specific hardware platform combined with the operating system. It is one step beyond the API (Application Program Interface), which defines the calls from the application to the operating system. The ABI defines the API plus the machine language for a particular CPU family. An API does not ensure runtime compatibility, but an ABI does, because it defines the machine language, or runtime, format.

enter image description here

Courtesy

Let me give a specific example how ABI and API differ in Java.

An ABI incompatible change is if I change a method A#m() from taking a String as an argument to String... argument. This is not ABI compatible because you have to recompile code that is calling that, but it is API compatible as you can resolve it by recompiling without any code changes in the caller.

Here is the example spelled out. I have my Java library with class A

// Version 1.0.0
public class A {
    public void m(String string) {
        System.out.println(string);
    }
}

And I have a class that uses this library

public class Main {
    public static void main(String[] args) {
        (new A()).m("string");
    }
}

Now, the library author compiled their class A, I compiled my class Main and it is all working nicely. Imagine a new version of A comes

// Version 2.0.0
public class A {
    public void m(String... string) {
        System.out.println(string[0]);
    }
}

If I just take the new compiled class A and drop it together with the previously compiled class Main, I get an exception on attempt to invoke the method

Exception in thread "main" java.lang.NoSuchMethodError: A.m(Ljava/lang/String;)V
        at Main.main(Main.java:5)

If I recompile Main, this is fixed and all is working again.

Linux shared library minimal runnable API vs ABI example

This answer has been extracted from my other answer: What is an application binary interface (ABI)? but I felt that it directly answers this one as well, and that the questions are not duplicates.

In the context of shared libraries, the most important implication of "having a stable ABI" is that you don't need to recompile your programs after the library changes.

As we will see in the example below, it is possible to modify the ABI, breaking programs, even though the API is unchanged.

main.c

#include <assert.h>
#include <stdlib.h>

#include "mylib.h"

int main(void) {
    mylib_mystrict *myobject = mylib_init(1);
    assert(myobject->old_field == 1);
    free(myobject);
    return EXIT_SUCCESS;
}

mylib.c

#include <stdlib.h>

#include "mylib.h"

mylib_mystruct* mylib_init(int old_field) {
    mylib_mystruct *myobject;
    myobject = malloc(sizeof(mylib_mystruct));
    myobject->old_field = old_field;
    return myobject;
}

mylib.h

#ifndef MYLIB_H
#define MYLIB_H

typedef struct {
    int old_field;
} mylib_mystruct;

mylib_mystruct* mylib_init(int old_field);

#endif

Compiles and runs fine with:

cc='gcc -pedantic-errors -std=c89 -Wall -Wextra'
$cc -fPIC -c -o mylib.o mylib.c
$cc -L . -shared -o libmylib.so mylib.o
$cc -L . -o main.out main.c -lmylib
LD_LIBRARY_PATH=. ./main.out

Now, suppose that for v2 of the library, we want to add a new field to mylib_mystrict called new_field.

If we added the field before old_field as in:

typedef struct {
    int new_field;
    int old_field;
} mylib_mystruct;

and rebuilt the library but not main.out, then the assert fails!

This is because the line:

myobject->old_field == 1

had generated assembly that is trying to access the very first int of the struct, which is now new_field instead of the expected old_field.

Therefore this change broke the ABI.

If, however, we add new_field after old_field:

typedef struct {
    int old_field;
    int new_field;
} mylib_mystruct;

then the old generated assembly still accesses the first int of the struct, and the program still works, because we kept the ABI stable.

Here is a fully automated version of this example on GitHub.

Another way to keep this ABI stable would have been to treat mylib_mystruct as an opaque struct, and only access its fields through method helpers. This makes it easier to keep the ABI stable, but would incur a performance overhead as we'd do more function calls.

API vs ABI

In the previous example, it is interesting to note that adding the new_field before old_field, only broke the ABI, but not the API.

What this means, is that if we had recompiled our main.c program against the library, it would have worked regardless.

We would also have broken the API however if we had changed for example the function signature:

mylib_mystruct* mylib_init(int old_field, int new_field);

since in that case, main.c would stop compiling altogether.

Semantic API vs programming API vs ABI

We can also classify API changes in a third type: semantic changes.

For example, if we had modified

myobject->old_field = old_field;

to:

myobject->old_field = old_field + 1;

then this would have broken neither API nor ABI, but main.c would still break!

This is because we changed the "human description" of what the function is supposed to do rather than a programmatically noticeable aspect.

I just had the philosophical insight that formal verification of software in a sense moves more of the "semantic API" into a more "programmatically verifiable API".

Semantic API vs Programming API

We can also classify API changes in a third type: semantic changes.

The semantic API, is usually a natural language description of what the API is supposed to do, usually included in the API documentation.

It is therefore possible to break the semantic API without breaking the program build itself.

For example, if we had modified

myobject->old_field = old_field;

to:

myobject->old_field = old_field + 1;

then this would have broken neither programming API, nor ABI, but main.c the semantic API would break.

There are two ways to programmatically check the contract API:

  • test a bunch of corner cases. Easy to do, but you might always miss one.
  • formal verification. Harder to do, but produces mathematical proof of correctness, essentially unifying documentation and tests into a "human" / machine verifiable manner! As long as there isn't a bug in your formal description of course ;-)

Tested in Ubuntu 18.10, GCC 8.2.0.

Your program (source code) can be compiled with modules who provide proper API.

Your program (binary) can run on platforms who provide proper ABI.

API restricts type definitions, function definitions, macros, sometimes global variables a library should expose.

ABI restricts what a "platform" should provide for you program to run on. I like to consider it in 3 levels:

  • processor level - the instruction set, the calling convention

  • kernel level - the system call convention, the special file path convention (e.g. the /proc and /sys files in Linux), etc.

  • OS level - the object format, the runtime libraries, etc.

Consider a cross-compiler named arm-linux-gnueabi-gcc. "arm" indicates the processor architecture, "linux" indicates the kernel, "gnu" indicates its target programs use GNU's libc as runtime library, different from arm-linux-androideabi-gcc which use Android's libc implementation.

enter image description here

Application programming interface (API) - is represented by the highest level of abstraction. This API interfaces applications to libraries and/or the core OS.

Application Binary Interface (ABI) covers facts such as low-level data types and call conventions and it also defines a format for many programs. Mainly, system calls are defined at this level. Moreover, this type of interface enables portability of various applications and libraries across OS which employ the same ABI.

Read more here

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top