Python (1040, 'Too many connections')

Question

im newbie in Python. I have Item table with 3 attributes ( name, price, image) I want to get html from link and save to table. So i do that

from books.models import Book, Author, Publisher, Item
import urllib
import gzip
from StringIO import StringIO
from BeautifulSoup import BeautifulSoup
import MySQLdb as mdb

def updateItems(request): 
    con = mdb.connect('localhost', 'anhnt', '12', 'db_django');
    cur = con.cursor()
    link = "http://pandagift.vn/i263/price1/qua-tang-doc-dao.htm"
    request = urllib.urlopen(link)
    output1 = open('web.txt', 'wb')
    html = ""
    html = request.read()
    output1.write(html)
    output1.close()
    beautifulSoup = BeautifulSoup(html)
    list_item = beautifulSoup.findAll("div", {"class":"Item_ProcNews"})
    if list_item:
    for item in list_item :
        #lay anh san pham
        image = item.findAll("img")[0]      
        st = str(image)
        beginindex = st.find('src') + 5
            endindex = st.find('jpg') + 3
        if(endindex == 2):
                endindex = st.find('png') + 3
            if(endindex == 2):
            endindex = st.find('jpeg') + 4
            if(endindex == 3):
                endindex = st.find('gif') + 3
            if(endindex == 2):
                        endindex = st.find('bmp') + 3
                if(endindex == 2):
                        endindex = st.find('JPG') + 3
        itemimage = str(image)[beginindex:endindex].encode('utf-8')
        #lay ten san pham
            name = item.findAll("span", {"class":"item-name"})[0]
        temp = name.findAll("a")[0]
        #itemname = temp.get('alt').encode('utf-8')
        itemname = temp.string.encode('utf-8')
            #lay gia san pham
            price = item.findAll("span", {"class":"price"})[0]
        #temp1 = str(price)
        #beginindex1 = temp1.find('price') + 7
            #endindex1 = temp1.find('/span') -1
        #itemprice = str(temp1)[beginindex1:endindex1]
        itemprice = str(price.string)
        #luu vao csdl
        query = "INSERT INTO books_item(name, price, image) VALUES('"+itemname+"', '"+itemprice+"', '"+itemimage+"')"
        cur.execute(query)
        #print query
    else:
        cur.execute("INSERT INTO books_item(name, price) VALUES('Hat', '10000vnd')")
    updateItems(request)

Its return

  Exception Type:   OperationalError
  Exception Value:  (1040, 'Too many connections')

Please tell me why its happen and can i do fix it. Thanks many :)

Answer 1

http://dev.mysql.com/doc/refman/5.0/en/too-many-connections.html

If you get a Too many connections error when you try to connect to the mysqld server, this means that all available connections are in use by other clients.

The number of connections permitted is controlled by the max_connections system variable. Its default value is 100. If you need to support more connections, you should set a larger value for this variable.

This means that you have opened more connections to the database than permitted by the server without closing them. I never see calls to close the connection con in your code.

Answer 2

I solved this problem by increasing the maximum number of connections allowed by MySQL in my django app settings.py file. More information in MySQL docs

DATABASES = { 
  'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': '',
    'USER': '',
    'PASSWORD': '',
    'HOST': 'localhost',
    'PORT': '3306',
    'OPTIONS': {
           "init_command": "SET GLOBAL max_connections = 100000", #<-- The fix
    }
  }
}

Answer 3

you need: cur.close() and con.close().

Every connection should be closed after use it

Answer 4

My situation is not exactly related to OP, but first result on google for my problem, so...

I was getting the too many connections error in django as well. I have two servers deployed on AWS, one EC2 t2.micro (web server) and one RDS t2.micro (mysql database). The web server is running debian, with apt-get installs for apache2 (2.4.10-10+deb8u3), python (2.7.9), and django (1.7.7-1+deb8u3). I'm using django just for the ORM and website is hosted on apache using mod_wsgi.

mysql:

mysql> show variables like "max_connections";
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 66    |
+-----------------+-------+
1 row in set (0.01 sec)

mysql> SHOW VARIABLES LIKE "%version%";
+-------------------------+------------------------------+
| Variable_name           | Value                        |
+-------------------------+------------------------------+
| innodb_version          | 5.6.23                       |
| protocol_version        | 10                           |
| slave_type_conversions  |                              |
| version                 | 5.6.23-log                   |
| version_comment         | MySQL Community Server (GPL) |
| version_compile_machine | x86_64                       |
| version_compile_os      | Linux                        |
+-------------------------+------------------------------+
7 rows in set (0.02 sec)

I had to tune down some settings in the /etc/apache2/sites-enabled/000-default.conf to fix the too many connections error. These four settings were previously not in the config file (default values):

ServerLimit 2
MaxRequestWorkers 10
ThreadsPerChild 5

WSGIDaemonProcess aaa threads=6

<VirtualHost *:80>
    ....

mysql processlist (SELECT count(*) FROM INFORMATION_SCHEMA.PROCESSLIST WHERE user='www-user') now stays around 21.