How to fast up databases queries, when thousands of entries [closed]

Question

Closed. This question needs to be more focused. It is not currently accepting answers.

---

Want to improve this question? Update the question so it focuses on one problem only by editing this post.

Closed 8 years ago.

Improve this question

For learning php and codeigniter, i've started to create test website... something like blog. ut that's not the point. I'm using codeigniter pagination to show all my blog's posts. but... it's starting to slow down when my database table (posts) has over 14k entries.... And I don't need answer like: "if it's blog and you'll never write so much posts then just don't think about that"... I need real solution of how to faster all this stuff.

Here's controller:

    function all()
{
    // Pagination $config
    $config = $this->m_posts->getPaginationConfig('all', $this->m_posts->getPostsCount());
    $this->pagination->initialize($config);
    $data['posts'] = $this->m_posts->getAllPosts($config['per_page'], $page);
    $data['pagination'] = $this->pagination->create_links();

    $this->template->build('posts/posts', $data);
}

Model:

    function getPaginationConfig($base_url, $total_rows)
{
    $config['base_url']         = base_url() . 'posts/'. $base_url;
    $config['total_rows']       = $total_rows;
    // ----
    $config['per_page']         = 10;
    $config['num_links']        = 5;
    $config['use_page_numbers'] = TRUE;
    $config['uri_segment']      = 3;
    return $config;
}
    function getPostsCount($limit)
{
    $this->db->order_by('id', 'desc');
    $q = $this->db->get('posts');
    return $q->num_rows();
}

function getAllPosts($limit = 0, $start = 0)
{
    $this->db->order_by('id', 'desc');
    $this->db->where('active', 1);
    // Get LATEST posts -> pagination
    $q = $this->db->get('posts', $limit, $start);
    $array = $q->result_array();
    $data = $this->createPostsArray($array);
    return $data;
}
    function createPostsArray($array)
{
    foreach ($array as $key => $row) 
    {
        $array[$key]['usr_info']   = $this->user->getUserData($row['usr_id'], 'nickname');
        $this->load->helper('cat_helper');
        $array[$key]['cat_ids'] = explodeCategories($row['cat_ids']);
        foreach ($array[$key]['cat_ids'] as $numb => $value)
        {
            $array[$key]['categories'][$numb] = $this->getCategoryName($value);
        }
    }

    return $array;
}

Answer 1

First of, change your getPostsCount function to this

function getPostsCount () {
      return $this->db->count_all('posts');
}

The way you're doing it now is wasting time/memory/cpu & code lines for nothing.

Second thing, use a left/inner join to get other data rather than throwing a bunch of queries in a foreach statement (this is WRONG).

If you still need help with joining things, show table structures to get more help.

I think this tiny change, will make big difference.

EDIT:

After Providing more info, here's your query with a join to users (as it's not clear how your categories works, I am not including it).

function getAllPosts($limit = 0, $start = 0) {
         $q = $this->db->select('p.*, u.nickname')
                        ->from('posts p')
                        ->join('users u', 'p.user_id = u.id', 'left')
                        ->limit($limit, $start)
                        ->get();
         return $q->result_array();
 }

This will return posts with the nickname of the user, as for the category it's not clear how you're storing them nor what explode categories is doing, If you're storing them in comma-separated field, you can use a single query to get all categories using ->where_in('id', array_of_ids);

You'll need to go through the manual to get more help on how to do things: http://ellislab.com/codeigniter/user-guide/