Ftp List directory - how to get only filenames

Question

I am struggling with a small piece of code which I am currently writing. The application is supposed to run once a day and download all files from an ftp server. My problem is:

Although in theory my routine to list the direcoty content runs fine, checks all files and saves them to a list, practical there are 2 errors:

1. The listing is html formatted
2. I only need the filename and extension

Code

string localPath = System.Reflection.Assembly.GetExecutingAssembly().Location;    
List<string> FtpListing = new List<string>();
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(Properties.Settings.Default.FtpUrl);
//request.Proxy = GlobalProxySelection.GetEmptyWebProxy();
request.Credentials = new NetworkCredential(Properties.Settings.Default.FtpUsername, Properties.Settings.Default.FtpPassword);
request.Method = WebRequestMethods.Ftp.ListDirectory;
using (StreamReader streamReader = new StreamReader(request.GetResponse().GetResponseStream()))
{
    string fileName = streamReader.ReadLine();
    while (fileName != null)
    {
         FtpListing.Add(fileName);
         fileName = streamReader.ReadLine();
    }
}

Without proxy it returns html, with the proxy statement uncommented I am getting a The remote server returned an error: (550) File unavailable (e.g., file not found, no access). error.

Where am I failing here?

/edit: here is a screenshot of the list, where all files should be listed, but instead a complete html file is saved:

Answer 1

I found a working solution by using the HtmlAgilityPack.

Since I can't change the html response, I changed my List<string> FtpListing to a simple string. With this string I parsed every <a> tag in the html code provided by the webrequest.

Code:

string HtmlResult = String.Empty;
Console.WriteLine("Starting listing of files....");
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(Properties.Settings.Default.FtpUrl);
request.Credentials = new NetworkCredential(Properties.Settings.Default.FtpUsername, Properties.Settings.Default.FtpPassword);
request.UsePassive = false;
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
using (Stream responsestream = response.GetResponseStream())
{
    using (StreamReader reader = new StreamReader(responsestream))
    {
        string _line = reader.ReadLine();
        while (_line != null && _line != String.Empty)
        {
            HtmlResult += _line;
            _line = reader.ReadLine();
        }
    }                    
}
//parse html output
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(HtmlResult);
foreach (HtmlAgilityPack.HtmlNode node in doc.DocumentNode.SelectNodes("//a[@href]"))
{
    if(node.InnerText.Contains(".txt")) FtpListing.Add(node.InnerText);
}
Console.WriteLine("{0} Files found", FtpListing.Count);

Answer 2

Use this code to return a list of file names in a certain ftp deirectory

    System.Net.FtpWebRequest ftpRequest =    System.Net.FtpWebRequest)System.Net.WebRequest.Create(SourceDirectory);
            ftpRequest.Credentials = new System.Net.NetworkCredential(SourceFTPUserName, SourceFTPPassword);
               ftpRequest.Method = WebRequestMethods.Ftp.ListDirectory;
              System.Net.FtpWebResponse response =        (System.Net.FtpWebResponse)ftpRequest.GetResponse();
        System.IO.StreamReader streamReader = new System.IO.StreamReader(response.GetResponseStream());

        List<string> directories = new List<string>();

        string line = streamReader.ReadLine();
        while (!string.IsNullOrEmpty(line))
        {
            directories.Add(line);
            line = streamReader.ReadLine();
        }

        streamReader.Close();
        return directories;

Hope it helps.