I think you're hitting the ServicePoint.ConnectionLimit
. This can be increased by tweaking ServicePointManager.DefaultConnectionLimit
before you start. Don't forget to .Dispose
of everything that can be disposed in the smallest possible scope... using
statements are good. If you don't close/dispose of WebRequest
related stuff, you can eat into your connection limit even if you believe that the request is finished.
C# program stops processing while making Web Request
-
18-07-2023 - |
Вопрос
I'm very new to C# and am playing around with writing a basic program. The goal is to grab an image and save it to the disk given a URL.
This is where my code essentially times out:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(m.Value.Trim()); // url string passed in from Regex function
HttpWebResponse resp = (HttpWebResponse) req.GetResponse(); // times out here
The code is in a class containing one function, which runs as such:
String url; // passed in as a parameter
String folder = @"C:\SMBC";
// create directory if not exists to save comic
if(!Directory.Exists(folder))
Directory.CreateDirectory(folder);
// visit the site and check for comics
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
while (reader.Peek() >= 0)
{
// if line contains "/comics/" then I'm in the right spot, we're at a URL now
String line = reader.ReadLine();
if (line.Contains("/comics/"))
{
// pull out the address of the image // example result: http://www.smbc-comics.com/comics/20020905-2.gif
Regex linkParser = new Regex(@"\b(?:https?://|www\.)\S+\b", RegexOptions.Compiled | RegexOptions.IgnoreCase);
foreach (Match m in linkParser.Matches(line))
{
// new local file in folder, use original file name
String name = @"" + folder +"\\" + m.Value.Substring(m.Value.LastIndexOf("/") + 1);
Console.WriteLine(m.Value.Trim()); //http://www.smbc-comics.com/comics/20020905-2.gif
Uri uri = new Uri(m.Value.Trim());
Console.WriteLine("Making request"); // works
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(m.Value.Trim());
Console.WriteLine("getting response"); // works
HttpWebResponse resp = (HttpWebResponse) req.GetResponse();
Console.WriteLine("opening stream"); // never shows
using (Stream inputStream = resp.GetResponseStream())
using (Stream outputStream = File.OpenWrite(name))
{
byte[] buffer = new byte[4096];
int bytesRead;
do
{
bytesRead = inputStream.Read(buffer, 0, buffer.Length);
outputStream.Write(buffer, 0, bytesRead);
} while (bytesRead != 0);
//outputStream.Close();
//inputStream.Close();
//resp.Close();
}
}
}
}
}
I am running this in the console and here's what I get:
After a while, the operation just times out, I know that the address is valid, because I visit it.
Is there something I am missing?
Решение
Другие советы
As a newbie, I was unaware that the framework let a limit on the number of connections allowed at any given time.
So you have to override this in case you have loops within loops of connections.
ServicePointManager.DefaultConnectionLimit = 4;
Не связан с StackOverflow