HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Converting latitude and longitude coordinates from CSV using web service

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
coordinatescsvservicelongitudelatitudeusingwebandconvertingfrom

Problem

I am working on converting a mailing list that has longitude and latitude coordinates within the CSV file. This script I came up with does what I need, but this is my first real-world use of python. I want to know where I am making any mistakes, not using best-practices, and what can be optimized to make it faster.

The input.csv file has the following header:

"Email Address",MEMBER_RATING,OPTIN_TIME,OPTIN_IP,CONFIRM_TIME,CONFIRM_IP,LATITUDE,LONGITUDE,GMTOFF,DSTOFF,TIMEZONE,CC,REGION,LAST_CHANGED,LEID,EUID


And the script:

```
import sys
import os
import csv
import signal
import json
import urllib
import urllib2
import sqlite3
import codecs
import cStringIO

csv.field_size_limit(sys.maxsize)

class EmailList:

WEB_SERVICE_URL = 'http://open.mapquestapi.com/nominatim/v1/reverse.php?format=json'

def __init__(self, inputFile):
signal.signal(signal.SIGINT, self.signal_handler)
self.conn = None
self.initialize_database(inputFile)
self.convert_rows()
self.db_to_csv()

def signal_handler(self, signal, frame):
try:
self.conn.commit()
self.conn.close()
print '[DB changes committed and connection closed.]'
except sqlite3.ProgrammingError as e:
print '[script stopped]'
print e.message

sys.exit(0)

def initialize_database(self, file):

print 'checking for data.db...'

if not os.path.isfile('data.db'):

print 'data.db does not exist, converting csv to sqlite...'

with open(file) as inputFile:
reader = UnicodeReader(inputFile)

header = reader.next()

if self.conn is None:
self.conn = sqlite3.connect('data.db')

c = self.conn.cursor()

c.execute("DROP TABLE IF EXISTS email_list")

sql = """CREATE TABLE email_list (\n""" + \
",\n".join([("%s varchar" % name

Solution

Two things that pop out for me immediately:

  • You named a parameter "file", which is a built-in function in python.



  • In one of your except blocks, you incorrectly use a dictionary.



See below:

converted['city': '']
converted['state': '']
converted['country': '']
converted['postal_code': '']


Note, what that will do is attempt to use the slice operator on the dictionary. I just tried it now and it fails with a TypeError exception. What you want the code to look like is this:

converted['city'] = ''
converted['state'] = ''
converted['country'] = ''
converted['postal_code'] = ''


Another thing I've noticed. Dealing with dictionaries, you're not quite sure how to handle getting "optional" parameters from them. Have a look at the refactored code I wrote for your json dict handling:

city = json_result['address'].get('city', '')
state = json_result['address'].get('state', '')
cc = json_result['address'].get('country_code', '')
postal_code = json_result['address'].get('postcode', '')


The get method on dictionary objects takes an optional second parameter that specifies a default value to return if the key is not found.

Code Snippets

converted['city': '']
converted['state': '']
converted['country': '']
converted['postal_code': '']
converted['city'] = ''
converted['state'] = ''
converted['country'] = ''
converted['postal_code'] = ''
city = json_result['address'].get('city', '')
state = json_result['address'].get('state', '')
cc = json_result['address'].get('country_code', '')
postal_code = json_result['address'].get('postcode', '')

Context

StackExchange Code Review Q#32400, answer score: 2

Revisions (0)

No revisions yet.