Python, suds - dealing with rotten WSDL caches and responses

Question

For the past couple days, I've been banging my head against a stupid SOAP server.

In my error logs, I was getting an exception / tracelog that looked like this: "Exception: (Widget, None, ), must be qref" -- (the key phrase being "must be qref").

First, I figured out that we had a rotten cache. So I disabled caching, yet, problems continued.

Second, I figured out that it's really the server's fault for failing to give me a proper WSDL, randomly; especially when using multiple worker instances. News flash - my server operator contact confirms a "server too busy" error in his logs. "Wunderbar!" Apparently, the server redirects to an HTTP 200 page when it should have provided an HTTP 503.

I was wondering how to be more graceful in handling these amazingly bad behaviours, and I actually came up with a useful, fairly generic solution. Posting here for future reference.

Answer 1

Something like this:

from suds.client import Client
from suds.transport.https import HttpAuthenticated
from suds.xsd.doctor import ImportDoctor, Import
import os
import shutil
import time
from tempfile import gettempdir as tmp

class MyTransport(HttpAuthenticated):
    def __init__(self,*args,**kwargs):
        HttpAuthenticated.__init__(self,*args,**kwargs)
        self.last_headers = None
    def send(self,request):
        result = HttpAuthenticated.send(self,request)
        self.last_headers = result.headers
        return result

class TenaciousConnector():
    def init_client(self, wsdl_url):
        retries = 3
        services = None
        # Cached mode
        (client, services) = self._init_client(wsdl_url)
        while not services and retries > 0:
            nap_time = 6 - retries
            retries = retries - 1
            time.sleep(nap_time)
            # clear potentially rotten cache right before retrying
            shutil.rmtree(os.path.join(tmp(), 'suds'), True)
            (client, services) = self._init_client(wsdl_url)
        if not services:
            # No-cache mode
            retries = 3
            (client, services) = self._init_client(wsdl_url, False)
            while not services and retries > 0:
                nap_time = 6 - retries
                retries = retries - 1
                time.sleep(nap_time)
                (client, services) = self._init_client(wsdl_url, False)
        if not services:
            raise Exception("Failed at loading WSDL from {0}".format(wsdl_url))
        return client

    def _init_client(self, wsdl_url, cache=True):
        i_doc = ImportDoctor(Import('http://schemas.xmlsoap.org/soap/encoding/'))
        tx = MyTransport()
        if cache:
            client = Client(wsdl_url, doctor=i_doc, transport=tx)
        else:
            client = Client(wsdl_url, doctor=i_doc, transport=tx, cache=None)
        services = client.wsdl.services
        return (client, services)

Used kind of like this:

connector = TenaciousConnector()
client = connector.init_client("http://example.com/webservice/wsdl")
client.factory.create('Widget')