HiveBrain v1.2.0
Get Started
← Back to all entries
patternpythonMinor

Watch a directory and insert new entries into database

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
directoryinsertnewintodatabasewatchandentries

Problem

This program's main function is to:

  • Watch a directory.



  • If there are new files, insert those as entries into the database.



  • Delete the files from the directory.



My below code runs well, but I want to optimize it and I think it needs better error handling.

import os
import time
import mysql.connector
import MySQLdb
import pooop

file_path = 'C:\\path\\to\\watch\\'

#send files to database
def insert_csv(file,filename):
try:
cnx = mysql.connector.connect(user='user',
password ='pass',host='192.168.1.1',
database='test')
cursor = cnx.cursor()
thedata = open(file, 'rb').read()
sql = "INSERT INTO testing(file_backup,file_name) VALUES (%s,%s)"
cursor.execute(sql,(thedata,filename))
cnx.commit()

except MySQLdb.OperationalError, e:
print e
except MySQLdb.ProgrammingError, e:
print e
finally:
cursor.close()

def walk():
try:
for dirpath, dirnames, files in os.walk(file_path):
for i in files:
file = dirpath+i
try:
insert_csv(file,i)
except Exception, e:
print e
finally:
os.remove(file)
print 'File successfully removed\n'+"-"*80
except Exception, e:
print e

#main loop that watches the directory
if __name__ == "__main__":
print "Application starts running\n"+"-"*80
while True:
walk()


The application will run on Windows XP. I'm using Python 2.7.

Solution

-
Naming

Nothing in the code suggests that csv files are special. Why a backup is called backup_csv?

-
Exception handling

The main code removes the file regardless of the insertion success. For example, if connect fails, the file is still removed. Is this an intended behaviour? I recommend to call os.remove() inside backup_csv.

-
misc

-
What is the purpose of time.sleep(1)?

-
I don't see a reason for if files clause. The for i in files works as expected if files is empty.

Context

StackExchange Code Review Q#108846, answer score: 2

Revisions (0)

No revisions yet.