It's not clear from the post how many rows are in your TableLayoutPanel, but you have to make sure the rows exist in order to add those controls. I don't think an exception is thrown when trying to add controls to a non-existent cell:
If MainMenu.table_analysts.RowStyles.Count - 1 < rows Then
MainMenu.table_analysts.RowStyles.Add(New RowStyle(SizeType.Percent))
End If
For i As Integer = 0 to rows
MainMenu.table_analysts.RowStyles(i).Height = 1 / rows
Next
MainMenu.table_analysts.Controls.Add(p, 0, rows)
MainMenu.table_analysts.Controls.Add(p2, 1, rows)
rows = rows + 1
Also, make sure the cells are empty before adding the panels by disposing any existing controls. Only one control is allowed per cell.
As Idle_Mind commented, make sure you also create "new" controls when you call your routine:
Public Sub addCAI()
If Not String.IsNullOrWhiteSpace(txt_analyst.Text) Then
Dim p As New Panel()
Dim p2 As New Panel()
Dim tb As New Label()
Dim tb2 As New Label()