How to log on file using threads

Question

I'm using a simulator (ns3), in my main application I allocate several applications named Consumer for download some files. In this class (Consumer), when I finish to download a file, I want to log on a file some statistics, so I implemented these functions (below showed) using threads, that if in my main I have a small number of consumers there are not problems, but with a very big number I can't open the log file when I call the function that create the thread.

//consumer.cc
void
Consumer::OnData (Ptr<const Data> contentObject)
{
  //do something 
  if (...)
  {
   int a = Consumer::create_thread();
   std::string s = "foo";
   Consumer::Log(s);
   }
}

int Consumer::create_thread(){   
    int rc = pthread_create(&logger, NULL, &Consumer::runFunc, (void*) this); //logger is declared in consumer.h as pthread_t logger;
    if(rc) printf("ERROR; return code from pthread_create() is %d\n", rc);
    sem_init(&queueSem, 0, 0);
    pthread_mutex_init(&mutex, NULL);
    //isLogActive = true;
    myfile.open ("ret.txt", std::ios::app);
    if(!myfile.is_open())
        std::cout << "Error opening the log file...";
return 1;
}

void Consumer::run_t(){
    for(;;)
    {
        sem_wait(&queueSem);
        //if(!isLogActive)
        //  break;
        pthread_mutex_lock (&mutex);
        myfile << queue.front(); 
        queue.pop();
        pthread_mutex_unlock (&mutex);
        myfile.flush(); 
    }
    while(!queue.empty())
    {
                std::cout<<queue.front()<<std::endl;
        myfile << queue.front(); 
        queue.pop();
    }            
    if(myfile.is_open())
    {
        myfile.close();
        std::cout << "File Ret closed.\n";
    }
    else
        std::cerr << "Calcolo error.\n";
    pthread_exit(NULL);
}

void * Consumer::runFunc(void * self){

    ((Consumer*)self)->run_t();
    return NULL;

}

void Consumer::Log(std::string toLog)
{

    ss.str("");
    ss.clear();
    ss << toLog << "\n"; 
    pthread_mutex_lock (&mutex);
    queue.push(ss.str());
    pthread_mutex_unlock (&mutex);
    sem_post(&queueSem);
}

What do you think is the problem? And how could I fix it?

Answer 1

Most likely the problem is that you have multiple threads trying to append to the file at the same time. Nothing good can come of this.

My suggestion would be to have a logging thread and a queue. When a thread wants to write to the log, it just puts the message on the logging queue. The logging thread reads from the queue and writes to the log file.

Of course, you'll have to add some synchronization to the queue so that it doesn't get corrupted by concurrent requests, or find a concurrent queue that already handles that synchronization for you.