HiveBrain v1.2.0
Get Started
← Back to all entries
snippetcsharpMinor

How can I better my URL checker program?

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
canprogrambetterhowcheckerurl

Problem

I made a LinkChecker program which checks URLs from a text file and outputs status into another file like this:

http://valid-url.com;OK
http://invalid-url.com;The remote server returned an error: (404) Not Found.


I have about 1400 URLs in the input file and it takes about 12 minutes to run.

I made it work. In what ways can I achieve rightness and fastness?

public class LinkChecker
{
    static void Main()
    {
        string line = string.Empty;
        string inputFile = @"D:\tmp\links.txt";
        StreamReader file = new StreamReader(inputFile);
        string outputFile = @"D:\tmp\out.txt";

        ClearOutputFile(outputFile);

        while(!string.IsNullOrEmpty(line = file.ReadLine()))
        {
            string currentUrlAndStatus = CheckURL(line);
            if(!string.IsNullOrEmpty(currentUrlAndStatus))
            {
                WriteToFile(currentUrlStatus, outputFile);
            }
        }
    }

    private static string CheckURL(string url)
    {
        string status = string.Empty;

        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
        request.Method = "HEAD";
        request.Proxy = null;

        try
        {
            using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                status = response.StatusCode.ToString();
            }
        }
        catch (WebException ex)
        {
            status = ex.Status;
        }

        return url + ";" + status;
    }

    private static void ClearOutputFile(string file)
    {
        using (Filestream fileStream = File.Open(file, FileMode.Open)
        {
            fileStream.SetLength(0);
        }
    }

    private static void WriteToFile(string message, string filename)
    {
        using(var writer = new StreamWriter(filename, true))
        {
            writer.WriteLine(message);
        }
    }
}

Solution

You forgot using using in your StreamReader

using(StreamReader file = new StreamReader(inputFile)){
    string outputFile = @"D:\tmp\out.txt";

    ClearOutputFile(outputFile);

    while(!string.IsNullOrEmpty(line = file.ReadLine()))
    {
        string currentUrlAndStatus = CheckURL(line);
        if(!string.IsNullOrEmpty(currentUrlAndStatus))
        {
            WriteToFile(currentUrlStatus, outputFile);
        }
    }
  }


Use the var keyword for local variables, let the compiler figure out the type for you. Type inference helps in refactoring.

var line = string.Empty;
 var inputFile = @"D:\tmp\links.txt";
 using(var file = new StreamReader(inputFile)){
 //...
 }
 var outputFile = @"D:\tmp\out.txt";
 // etc..

Code Snippets

using(StreamReader file = new StreamReader(inputFile)){
    string outputFile = @"D:\tmp\out.txt";

    ClearOutputFile(outputFile);

    while(!string.IsNullOrEmpty(line = file.ReadLine()))
    {
        string currentUrlAndStatus = CheckURL(line);
        if(!string.IsNullOrEmpty(currentUrlAndStatus))
        {
            WriteToFile(currentUrlStatus, outputFile);
        }
    }
  }
var line = string.Empty;
 var inputFile = @"D:\tmp\links.txt";
 using(var file = new StreamReader(inputFile)){
 //...
 }
 var outputFile = @"D:\tmp\out.txt";
 // etc..

Context

StackExchange Code Review Q#55603, answer score: 5

Revisions (0)

No revisions yet.