That is because you are overwriting the variable $cartOutput each time your foreach loop runs. So that means that only the last element of your cart is visible. You can remedy this by either appending to $cartOutput or by making it an array, and imploding it afterwards.
Like this (using implode):
<?php
$cartOutput=array();
if(!isset($_SESSION["cart_array"])|| count($_SESSION["cart_array"])<1){
$cartOutput="<h2 align='center'>Your shopping cart is empty</h2>";
}else{
$i=0;
foreach($_SESSION["cart_array"] as $each_item){
$i++;
$cartOutput[] = "<h2>Cart Item $i</h2>";
$cartOutput[] = "Item ID:" .$each_item['item_id']."<br>";
$cartOutput[] ="Item Quantity:" .$each_item['item_id']."<br>";
}
$cartOutput = implode('<br />', $cartOutput);
}
?>
Alternatively you can also just append all output to the already existant output in $cartOutput like so:
<?php
$cartOutput="";
if(!isset($_SESSION["cart_array"])|| count($_SESSION["cart_array"])<1){
$cartOutput="<h2 align='center'>Your shopping cart is empty</h2>";
}else{
$i=0;
foreach($_SESSION["cart_array"] as $each_item){
$i++;
// note the use of .= which is a combined assignment and concatenation operators (it concatenates all content after the .= to the string preceding it)
$cartOutput .= "<h2>Cart Item $i</h2>";
$cartOutput .= "Item ID:" .$each_item['item_id']."<br>";
$cartOutput .="Item Quantity:" .$each_item['item_id']."<br>";
}
}
?>