spin_lock_irqsave vs spin_lock_irq

Question

On an SMP machine we must use spin_lock_irqsave and not spin_lock_irq from interrupt context.

Why would we want to save the flags (which contain the IF)?

Is there another interrupt routine that could interrupt us?

Answer 1

I am new to the kernel but from what I gather from Robert Love's book "Linux Kernel Development", if interrupts are already disabled on the processor before your code starts locking, when you call spin_unlock_irq you will release the lock in an erroneous manner. If you save the flags and release it with the flags, the function spin_lock_irqsave will just return the interrupt to its previous state.

Example with spin_lock_irqsave

spinlock_t mLock = SPIN_LOCK_UNLOCK;
unsigned long flags;

spin_lock_irqsave(&mLock, flags); // save the state, if locked already it is saved in flags
// Critical section
spin_unlock_irqrestore(&mLock, flags); // return to the formally state specified in flags

Example with spin_lock_irq( without irqsave ):

spinlock_t mLock = SPIN_LOCK_UNLOCK;
unsigned long flags;

spin_lock_irq(&mLock); // Does not know if already locked
// Critical section
spin_unlock_irq(&mLock); // Could result in an error unlock...

Answer 2

spin_lock_irqsave is basically used to save the interrupt state before taking the spin lock, this is because spin lock disables the interrupt, when the lock is taken in interrupt context, and re-enables it when while unlocking. The interrupt state is saved so that it should reinstate the interrupts again.

Example:

1. Lets say interrupt x was disabled before spin lock was acquired
2. spin_lock_irq will disable the interrupt x and take the the lock
3. spin_unlock_irq will enable the interrupt x.

So in the 3rd step above after releasing the lock we will have interrupt x enabled which was earlier disabled before the lock was acquired.

So only when you are sure that interrupts are not disabled only then you should spin_lock_irq otherwise you should always use spin_lock_irqsave.

Answer 3

The need for spin_lock_irqsave besides spin_lock_irq is quite similar to the reason local_irq_save(flags) is needed besides local_irq_disable. Here is a good explanation of this requirement taken from Linux Kernel Development Second Edition by Robert Love.

The local_irq_disable() routine is dangerous if interrupts were already disabled prior to its invocation. The corresponding call to local_irq_enable() unconditionally enables interrupts, despite the fact that they were off to begin with. Instead, a mechanism is needed to restore interrupts to a previous state. This is a common concern because a given code path in the kernel can be reached both with and without interrupts enabled, depending on the call chain. For example, imagine the previous code snippet is part of a larger function. Imagine that this function is called by two other functions, one which disables interrupts and one which does not. Because it is becoming harder as the kernel grows in size and complexity to know all the code paths leading up to a function, it is much safer to save the state of the interrupt system before disabling it. Then, when you are ready to reenable interrupts, you simply restore them to their original state:

unsigned long flags;

local_irq_save(flags);    /* interrupts are now disabled */ /* ... */
local_irq_restore(flags); /* interrupts are restored to their previous
state */

Note that these methods are implemented at least in part as macros, so the flags parameter (which must be defined as an unsigned long) is seemingly passed by value. This parameter contains architecture-specific data containing the state of the interrupt systems. Because at least one supported architecture incorporates stack information into the value (ahem, SPARC), flags cannot be passed to another function (specifically, it must remain on the same stack frame). For this reason, the call to save and the call to restore interrupts must occur in the same function.

All the previous functions can be called from both interrupt and process context.

Answer 4

Reading Why kernel code/thread executing in interrupt context cannot sleep? which links to Robert Loves article, I read this :

some interrupt handlers (known in Linux as fast interrupt handlers) run with all interrupts on the local processor disabled. This is done to ensure that the interrupt handler runs without interruption, as quickly as possible. More so, all interrupt handlers run with their current interrupt line disabled on all processors. This ensures that two interrupt handlers for the same interrupt line do not run concurrently. It also prevents device driver writers from having to handle recursive interrupts, which complicate programming.

Answer 5

Below is part of code in linux kernel 4.15.18, which shows that spiin_lock_irq() will call __raw_spin_lock_irq(). However, it will not save any flags as you can see below part of the code but disable the interrupt.

  static inline void __raw_spin_lock_irq(raw_spinlock_t *lock)
    {
        local_irq_disable();
        preempt_disable();
        spin_acquire(&lock->dep_map, 0, 0, _RET_IP_);
        LOCK_CONTENDED(lock, do_raw_spin_trylock, do_raw_spin_lock);
    }

Below code shows spin_lock_irqsave() which saves the current stage of flag and then preempt disable.

static inline unsigned long __raw_spin_lock_irqsave(raw_spinlock_t *lock)
{
    unsigned long flags;

    local_irq_save(flags);
    preempt_disable();
    spin_acquire(&lock->dep_map, 0, 0, _RET_IP_);
    /*
     * On lockdep we dont want the hand-coded irq-enable of
     * do_raw_spin_lock_flags() code, because lockdep assumes
     * that interrupts are not re-enabled during lock-acquire:
     */
#ifdef CONFIG_LOCKDEP
    LOCK_CONTENDED(lock, do_raw_spin_trylock, do_raw_spin_lock);
#else
    do_raw_spin_lock_flags(lock, &flags);
#endif
    return flags;
}

Answer 6

This question starts from the false assertion: On an SMP machine we must use spin_lock_irqsave and not spin_lock_irq from interrupt context.

Neither of these should be used from interrupt context, on SMP or on UP. That said, spin_lock_irqsave() may be used from interrupt context, as being more universal (it can be used in both interrupt and normal contexts), but you are supposed to use spin_lock() from interrupt context, and spin_lock_irq() or spin_lock_irqsave() from normal context. The use of spin_lock_irq() is almost always the wrong thing to do in interrupt context, being this SMP or UP. It may work because most interrupt handlers run with IRQs locally enabled, but you shouldn't try that.