Concurrent download in Go

I wrote a program for downloading files in concurrent / parallel (GOMAXPROCS > 1) manner.

This is my 2nd (non-toy) program written in Go. Please point out areas for improvement:

```
package main

import (
"errors"
"fmt"
"github.com/kennygrant/sanitize"
"github.com/nu7hatch/gouuid"
"github.com/op/go-logging"
"io"
"net/http"
"net/url"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"time"
)

type TransferMessage struct {
urlstr string
content_length int64
transferred int64
finished bool
filename string
}

const read_chunk = 65536

func MaybeWarn(err error, format string, args ...interface{}) {
if err != nil {
format += " (%s)"
args = append(args, err)
log.Warning(format, args...)
}
}

func MaybeExitErr(err error, format string, args ...interface{}) {
if err != nil {
format += " (%s)"
args = append(args, err)
log.Error(format, args...)
os.Exit(1)
}
}

func GetContLen(resp *http.Response) (content_length int64) {
clen := resp.Header.Get("Content-Length")
content_length, err := strconv.ParseInt(clen, 10, 64)
if err != nil {
content_length = -1
MaybeWarn(err, "Cannot read content length for url: %s", resp.Request.URL)
}
return
}

func GetUrlFilePath(urlstr string) string {
urlp, err := url.Parse(urlstr)
fname := sanitize.Name(urlp.Path)
if err != nil || fname == "" {
uid, err := uuid.NewV4()
MaybeExitErr(err, "Cannot generate uid for filename, problem url: %s, error: %s", urlstr, err)
fname = uid.String()
}
cwd, _ := filepath.Abs(filepath.Dir(os.Args[0]))
return filepath.Join(cwd, fname)

}

It's super fine to use globals (like log) in a small one-file program you've written. To make them more visible declare them at the very top of your file right after your types.

The global read_chunk doesn't follow Go naming convention. readChunk or readChunkSize will do it better.

SetupLog is used only once in the main function and it does unnecessarily job. Could be replaced with a single function call:

logging.SetLevel(logging.INFO, "")

I see how you've used MaybeWarn, MaybeExitErr and understand that it can be tedious to write if err != nil after most standard library functions. But it is just how it is. I suggest you to get used to it, handle possible errors in the right place and remove Maybe* functions.

ConsumeAllNonBlocking has several pitfalls:

• flag variable is useless. You can use break directly in switch body to break the for loop.

• The for loop is not used, it can be omited.

• receive operator has second parameter you might want to use

The function is used only once so I suggest you to write the select statement directly in place where you need it:

select {
case ret, ok := <- c:
default:
    // will block
}

There is no a single comment line across all your code. This can be perfectly valid for a simple program like FizzBuzz, but some complex one like concurrent HTTP downloader deserves some explanation. Same stands for user experience: it's wise to provide a usage string and command line arguments like quiet, read links from stdin and so on.

There is several places were the code is not formatted properly. go fmt is on the rescue.

StackExchange Code Review Q#51686, answer score: 2