particle swarm optimization inertia factor

Question

i am reading in soft computing algorithms ,currently in "Particle Swarm Optimization ",i understand the technique in general but ,i stopped at mathematical or physics part which i can't imagine or understand how it works or how it affect the flying,that part is the first part in the equation which update the velocity which is called the "Inertia Factor" the complete update velocity equation is : i read in one article in section 2.3 "Ineteria Factor" that:

"This variation of the algorithm aims to balance two possible PSO tendencies (de- pendent on parameterization) of either exploiting areas around known solutions or explore new areas of the search space. To do so this variation focuses on the momentum component of the particles' velocity equation 2. Notice that if you remove this component the movement of the particle has no memory of the pre- vious direction of movement and it will always explore close to a found solution. On the other hand if the velocity component is used, or even multiplied by a w (inertial weight, balances the importance of the momentum component) factor the particle will tend to explore new areas of the search space since it cannot easily change its velocity towards the best solutions. It must rst \counteract" the momentum previously gained, in doing so it enables the exploration of new areas with the time \spend counteracting" the previous momentum. This vari- ation is achieved by multiplying the previous velocity component with a weight value, w."

the full pdf at: https://www.google.com.eg/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CDIQFjAA&url=http%3A%2F%2Fweb.ist.utl.pt%2F~gdgp%2FVA%2Fdata%2Fpso.pdf&ei=0HwrUaHBOYrItQbwwIDoDw&usg=AFQjCNH8vChXHXWz_ydHxJKAY0cUa94n-g

but i can't also imagine how physicaly or numerically this is happend and how this factor affect going from exploration level to exploitative level ,so need a numerical example to see how it's work and imagine how it's work.

also ,in Genetic Algorithm there's a schema theorem which is a proof of GA success of finding optimum solution,is there's such athoerm for PSO.

Answer 1

It's not easy to explain PSO using mathematics (see Wikipedia article for example). But you can think like this: the equation has 3 parts:

particle speed = inertia + local memory + global memory

So you control the 'importance' of these components by varying the coefficientes in each part. There's no analytical way to see this, unless you make the stocastic part constant and ignore things like particle-particle interation.

Exploit: take advantage of the best know solutions (local and global).
Explore: search in new directions, but don't ignore the best know solutions.

In a nutshell, you can control how much importance to give for the particle current speed (inertia), the particle memory of the best know solution, and the particle memory of the swarm best know solution.

I hope it can help you! Br's

Answer 2

Inertia was not the part of the original PSO algorithm introduced by Kennedy and Eberhart in 1995. It's been three years until Shi and Eberhart published this extension and showed (to some extent) that it works better.

One can set that value to a constant (supposedly [0.8 to 1.2] is best). However, the point of the parameter is to balance exploitation and exploration of space, and authors got best results when they defined the parameter with a linear function, that decreases over time from [1.4 to 0]. Their rationale was that first one should exploit solutions to find a good seed and later exploit area around the seed.

My feeling about it is that the closer you are to 0, the more chaotic turns particles make.

For a detailed answer refer to Shi, Eberhart 1998 - "A modified Particle Swarm Optimizer".

Answer 3

Inertia controls the influence of the previous velocity.

When high, cognitive and social components are less relevant. (particle keeps going its way, exploring new portions of the space)

When low, particle explores better the space where the best-so-far optimum has been found

Inertia can change over time: Start high, later decrease