HiveBrain v1.2.0
Get Started
← Back to all entries
patternMinor

Get RSS feeds and store them into database

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
rssintostoredatabasegetandfeedsthem

Problem

It is my first program in Clojure. It read RSS feed's list from text file, get each feed and store result into sqlite database.

project.clj:

(defproject cowl "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.8.0"],
                 [org.clojars.scsibug/feedparser-clj "0.4.0"]
                 [org.clojure/java.jdbc "0.3.7"]
                 [org.xerial/sqlite-jdbc "3.7.2"]]
  :main cowl.core/fetch
)


core.clj:

(ns cowl.core
  (:require [cowl.rss :as rss]    
            [cowl.db :as db]))

(defn fetch []

  "Fetch information from RSS and store it to DB"

  (with-open [r (clojure.java.io/reader "settings\\feed-list.txt")]
    (doseq [line (line-seq r)
           entry (rss/process-feed line)]      

      (db/insert-entity (:title entry) (:link entry) (:source entry)))))


entity.clj:

(ns cowl.entity)

(defn make-entity [title, link, source]
    {:title title, :link link, :source source})


rss.clj:

(ns cowl.rss
  (:require    
    [cowl.entity :as entity]   
    [cowl.db :as db] 
    [feedparser-clj.core :as rss]))

(defn construct-entry [source, entry]
    (entity/make-entity (:title entry) (:link entry) source))   

(defn process-feed [url]

    "Gets the feed URL and returns list of entites"
    (let [feed (rss/parse-feed url)
        construct-foo (partial construct-entry (:title feed))]

        (map construct-foo (:entries feed))))


db.clj:

```
(ns cowl.db
(:require
[clojure.java.jdbc :as sql]))

(def db {:classname "org.sqlite.JDBC", :subprotocol "sqlite", :subname "work.db"})

(defn create-db []
(sql/execute! db ["drop table if exists entities"])
(let [q (sql/create-table-ddl :entities
[:title :text]
[:link :text :primary :k

Solution

When using defn, the docstring goes before the arguments list, not after.

In order to prevent environment coupling, functions that use a db should accept it as an argument instead of accessing it via a Var. This may seem like an inconvenience, but it will yield much more maintainable code.

You should also improve the naming :) names likes construct-foo or process-feed are not very explicit.

Context

StackExchange Code Review Q#142998, answer score: 3

Revisions (0)

No revisions yet.