SCHED_FIFO thread is preempted by SCHED_OTHER thread in Linux

Question

I have written test program to test out SCHED_FIFO. I have learnt that SCHED_FIFO cannot be preempted by SCHED_OTHER threads. But I couldn't explain the results obtained when same program is run multiple times.

/* Includes */
#include <unistd.h>     /* Symbolic Constants */
#include <sys/types.h>  /* Primitive System Data Types */ 
#include <errno.h>      /* Errors */
#include <stdio.h>      /* Input/Output */
#include <stdlib.h>     /* General Utilities */
#include <pthread.h>    /* POSIX Threads */
#include <string.h>     /* String handling */
#include <sched.h>
/* prototype for thread routine */
void print_message_function ( void *ptr );
void print_message_function1 ( void *ptr );

/* struct to hold data to be passed to a thread
this shows how multiple data items can be passed to a thread */
typedef struct str_thdata
{
int thread_no;
int thread_value;
char message[100];
  } thdata;

int main()
  {
pthread_t thread1, thread2;  /* thread variables */
thdata data1, data2;         /* structs to be passed to threads */

/* initialize data to pass to thread 1 */
data1.thread_no = 1;
data1.thread_value = 0;
strcpy(data1.message, "Hello!");

/* initialize data to pass to thread 2 */
data2.thread_no = 2;
data2.thread_value = 10000;
strcpy(data2.message, "Hi!");

/* create threads 1 and 2 */    
pthread_create (&thread1, NULL, (void *) &print_message_function, (void *) &data1);
pthread_create (&thread2, NULL, (void *) &print_message_function1, (void *) &data2);

/* Main block now waits for both threads to terminate, before it exits
   If main block exits, both threads exit, even if the threads have not
   finished their work */ 
pthread_join(thread1, NULL);
pthread_join(thread2, NULL);

/* exit */  
exit(0);
} /* main() */

/** 
* print_message_function is used as the start routine for the threads used
* it accepts a void pointer 
**/
void print_message_function ( void *ptr )
{

thdata *data;            
data = (thdata *) ptr;  /* type cast to a pointer to thdata */

struct sched_param param;
//int priority=10;
/* sched_priority will be the priority of the thread */
//param.sched_priority = priority;
/* only supported policy, others will result in ENOTSUP */

int policy = SCHED_OTHER;
/* scheduling parameters of target thread */
pthread_setschedparam(pthread_self(), policy, &param);
printf("Thread %d says sched policy  %d \n", data->thread_no, SCHED_OTHER);
pthread_getschedparam(pthread_self(),&policy,&param);

printf("Thread %d says %s  %d \n", data->thread_no, data->message,policy);

int i=0;
/* do the work */
printf("Thread %d says %s %d \n", data->thread_no, data->message,(int)pthread_self());
for(i=0;i<100;i++)

printf("Thread %d says %d \n", data->thread_no,data->thread_value++);
pthread_exit(0); /* exit */
} /* print_message_function ( void *ptr ) */



void print_message_function1 ( void *ptr )
{

thdata *data;            
data = (thdata *) ptr;  /* type cast to a pointer to thdata */

struct sched_param param;
int priority=10;
/* sched_priority will be the priority of the thread */
param.sched_priority = priority;
/* only supported policy, others will result in ENOTSUP * /

int policy = SCHED_FIFO;
/* scheduling parameters of target thread */
pthread_setschedparam(pthread_self(), policy, &param);
printf("Thread %d says sched policy %d \n", data->thread_no, SCHED_FIFO);

pthread_getschedparam(pthread_self(),&policy,&param);

printf("Thread %d says %s  %d \n", data->thread_no, data->message,policy);

int i=0;
/* do the work */
printf("Thread %d says %s  %d \n", data->thread_no, data->message,(int)pthread_self());
for(i=0;i<100;i++)

printf("Thread %d says %d \n", data->thread_no,data->thread_value++);
pthread_exit(0); /* exit */
} /* print_message_function ( void *ptr ) */

I have got unexpected results in multiple runs where I have seen SCHED_FIFO is preempted by SCHED_OTHER thread, i.e. as per program, thread 2 is in FIFO mode, while thread 1 is SCHED_OTHER mode. I have seen multiple times where thread2 is preempted by thread1.

Can someone help me out in finding the issue?

Answer 1

You probably have these sysctl settings in effect, which are default values:

kernel.sched_rt_period_us = 1000000
kernel.sched_rt_runtime_us = 950000

This means that real time threads are allowed to hog only 95% of every 1 second period.

Also see: Can't provoke Priority Inversion in C++

Answer 2

Also whenever your thread blocks on IO, perhaps from the printf statements, another thread may be scheduled.