Recent Entries 10
- pattern tip 39d agoFree RSS sources for catching emergent trends without scrapingNeed to detect "what's just starting to trend" across the public internet, but: TikTok Creative Center can be taken offline (it was in early 2026 with a "back in Q2" notice), paid APIs like Apify add ~$30/mo, scraping TikTok directly invites bans, and Firecrawl refuses tiktok.com domains. Most trending sources only show currently-popular items, not freshly-emerged ones.
- pattern minor 112d agoGet RSS feeds and store them into databaseIt is my first program in Clojure. It read RSS feed's list from text file, get each feed and store result into sqlite database. project.clj: ``` (defproject cowl "0.1.0-SNAPSHOT" :description "FIXME: write description" :url "http://example.com/FIXME" :license {:name "Eclipse Public License" :url "http://www.eclipse.org/legal/epl-v10.html"} :dependencies [[org.clojure/clojure "1.8.0"], [org.clojars.scsibug/feedparser-clj "0.4.0"] [org.clojure/java.jdbc "0.3.7"] [org.xerial/sqlite-jdbc "3.7.2"]] :main cowl.core/fetch ) ``` core.clj: ``` (ns cowl.core (:require [cowl.rss :as rss] [cowl.db :as db])) (defn fetch [] "Fetch information from RSS and store it to DB" (with-open [r (clojure.java.io/reader "settings\\feed-list.txt")] (doseq [line (line-seq r) entry (rss/process-feed line)] (db/insert-entity (:title entry) (:link entry) (:source entry))))) ``` entity.clj: ``` (ns cowl.entity) (defn make-entity [title, link, source] {:title title, :link link, :source source}) ``` rss.clj: ``` (ns cowl.rss (:require [cowl.entity :as entity] [cowl.db :as db] [feedparser-clj.core :as rss])) (defn construct-entry [source, entry] (entity/make-entity (:title entry) (:link entry) source)) (defn process-feed [url] "Gets the feed URL and returns list of entites" (let [feed (rss/parse-feed url) construct-foo (partial construct-entry (:title feed))] (map construct-foo (:entries feed)))) ``` db.clj: ``` (ns cowl.db (:require [clojure.java.jdbc :as sql])) (def db {:classname "org.sqlite.JDBC", :subprotocol "sqlite", :subname "work.db"}) (defn create-db [] (sql/execute! db ["drop table if exists entities"]) (let [q (sql/create-table-ddl :entities [:title :text] [:link :text :primary :k
- pattern minor 112d agoSimple Java RSS readerAs a study project I was supposed to write an RSS reader in Java (I used Swing to make GUI) using MVC pattern. I finished it, but it's not quite polished yet (still gotta write these javadocs and add deleting categories/channels) and there are most likely a lot of dumb things as I wrote my first line of code in Java a week ago. I would really appreciate if you could check if I implemented MVC correctly (specifically my Controller which works as "delegate"), as I'm honestly a bit confused what should I put where and apparently there are A LOT of ways to implement the pattern. Right now I use the Controller as a "bridge" between Model and View, but I don't know for example if, for example, `elementInTreeFocusedEvent(JTree tree)` should be there. The interface (as in, error alerts, button labels and so on) is in Polish, but you should have no problem with it. GitHub Feeder.java ``` package feeder; import feeder.controller.Controller; import feeder.views.View; public class Feeder { public static void main(String[] args) { new View(new Controller()); } } ``` model/News.java ``` package feeder.model; import java.io.Serializable; /** * News is a container class for a single news item. * * @since 2016-06-10 * @version 1.0 */ public class News implements Serializable { private static final long serialVersionUID = 1L; private String title; private String link; private String date; private String channel; private String description; /** * @return the title */ public String getTitle() { return title; } /** * @param title the title to set */ public void setTitle(String title) { this.title = title; } /** * @return the link */ public String getLink() { return link; } /** * @param link the link to set */ public void setLink(String link) { this.link = link; } /** * @return the date */
- pattern minor 112d agoEmbedding an RSS feed into a page after it's already loadedI'm working on reworking our website from a WYSIWYG editor to MVC. I'm all right with server side code, but complete rubbish when it comes to client side Javascript, so I'd appreciate any/all feedback. The goal Embed this rss feed from our blog into a page on our main website. So that it looks like this: Design choices/rationale I had originally written this code to go get the RSS feed in the `News` controller, and bind it to the `News` view. This caused unacceptable load times for the page. Obviously, the page couldn't render anything until it had gone across the network to a page, that may or may not be available, and retrieve the RSS feed. It took upwards of 5-6 seconds for anything at all to render my browser. So, I decided to try my hand at retrieving this and loading it client side after allowing the "container" of the page to load. I extracted a `BlogFeed` partial view that is retrieved and inserted by an Ajax call after the page is ready. This allows initial rendering to happen quickly, only later filling in the content that takes time to retrieve. I considered either creating a separate `BlogFeedService` class, a separate `News` controller, or both, but it felt like over engineering a simple site that could almost be served up as static HTML. If either the `Home` controller, or `News` code grows any more, I will likely extract a few classes. I used MVC so that I could extend the site with more dynamic behavior later. I intend on adding a preview of some of our project's functionality through a webapp section of this site. Questions Is this a good approach? I'll admit that I basically copy/pasta'd the jquery code, then monkey patched it with a `Url.Action` to make sure I wasn't hardcoding Urls into it. Did I do the async/await stuff correctly? I'm never quite sure I've gotten that correct either. Secondarily, was using async await here overkill? It felt right, because reaching out over the network is potentially blocking, but there's not much for
- pattern minor 112d agoReading XML files from RSS of different websites in PHPI've created a working XML RSS reader. I wanted to know if it's a good one or not and if it's efficient for the server or not. I run this code every 12 hours or something give or take the amount of time that a new article will be published. This is the code itself with all the explanation of every part of the code. Tell me if it's efficient or if it needs some changes to do an even better a job. ``` prepare("insert INTO `articles` (PublishDate, ArticleTitle, ArticleBody) VALUES (?, ?, ?)"); $stmt->bind_param("sss", $PublishingDate, $Title, $Description); // making sure that all of them are string foreach ($RSSXMLURL as $URL) { // this for each loop will go to every xml file on the revious array $XML = simplexml_load_file($URL); // getting the xml file content foreach ($XML->channel->item as $item) { // this for each loop will get every item(article) which is //how it's written in every xml file $Title = $item->title; // getting the title $String = $item->description; // getting the article body in a temp variable for now so we can remove the ads // later on the code $Link = $item->link; // getting the article link $PublishingDate = $item->pubDate; // getting the publishing date of the article // this part of code will get all the ads from the articles // after some reseach i found out that most of them have one thing in commen // clear='all' which is written in a tag // so i check if there was any if (strpos($String,"clear='all'") !== false) { // if there was any $Pos = strpos($String, "clear='all'"); // i get it's position $String = str_replace(substr($String, $Pos), "", $String);// i remove it from the string through // replacing it with an empty space with that i completly remove the ads if (strlen($String) execute()){ $NumSa
- pattern minor 112d agoUnit-testing an RSS feed parserI'm working on a class to parse RSS feeds using `SyndicationFeed`. (related to Downloading data using HttpClient). I'm trying to write this so it can be unit-tested, but there's an awful lot of boring setup boilerplate before I get to the important stuff. My data source sometimes sends invalid RSS XML according to `XmlReader` (in the pattern of `This article ... keyword ...`), so I need to sanitize the input (for political reasons, I can't just say "this won't work until they fix their end"). I started with the following method, which does a few too many things and isn't very testable. My important logic starts after the `SyndicationFeed.Load()` call, but any tests require that all the XML handling works correctly. ``` public virtual FeedSummary ParseResponse(string data) { var response = new FeedSummary(); var doc = new XmlDocument(); doc.LoadXml(data); var builder = new StringBuilder(); var xmlSettings = new XmlWriterSettings(); xmlSettings.OmitXmlDeclaration = true; xmlSettings.ConformanceLevel = ConformanceLevel.Fragment; var htmlWriter = XmlWriter.Create(builder, xmlSettings); var transformationDocument = new XmlDocument(); transformationDocument.LoadXml(this.XsltSource); //The xslt source can be passed in via the class constructor so //I can test this using the identity transform var compiledTransform = new XslCompiledTransform(true); compiledTransform.Load(transformationDocument); compiledTransform.Transform(doc, htmlWriter); var reader = XmlReader.Create(new StringReader(builder.ToString())); var feed = SyndicationFeed.Load(reader); reader.Close(); var totalResults = feed.ElementExtensions.Single(x => x.OuterName == "totalResults"); response.TotalResults = Int32.Parse(totalResults.ToString()); foreach (var item in feed.Items) { var hits = new List(); var searchItem = new SearchHit() { Title = item.Title.Text, Item
- pattern minor 112d agoRSS parser for Node.JSI would like someone to review this code and tell if it can be done better, or if the code is elegant and simple enough for the task at hand. I used code climate and I got 4/4 and my test coverage is 96%, yet I would like a professional opinion about it. ``` 'use strict'; var cheerio = require('cheerio'); var FeedParser = require('feedparser'); var http = require('http'); var PromisePolyfill = require('promise'); var urlEncode = require('urlencode'); /** * Extracts a valid url from the RSS Feed Item, taking into account exception urls * @param {string} content - the item to be analyzed * @returns {string} - the valid url inside the item */ function getValidURL(content) { var $ = cheerio.load(content), responseUrl; $('a').each(function(i, e) { if ( $(e).attr('href').indexOf('reddit.com') === -1 && $(e).attr('href').indexOf('imgur.com') === -1 ) { responseUrl = $(e).attr('href'); } }); return responseUrl; } /** * Parses a RSS stream into an Object * @param {string} rssUrl - RSS url to fetch the stream * @returns {Object} - the Object with meta and item information */ exports.parseRss = function(rssUrl) { var responseObject = { title: '', link: '', image: '', items: [] }; return new PromisePolyfill(function(resolve, reject) { http.get(rssUrl, function(resGet) { resGet.pipe(new FeedParser({})) .on('error', function(error) { reject(error.message); }) .on('meta', function(meta) { responseObject.title = meta.title; responseObject.link = meta.link; }) .on('readable', function() { var stream = this, item, validUrl = ''; while ((item = stream.read())) { validUrl = getValidURL(item.description);
- pattern minor 112d agoRefreshing / Cycling through a parsed RSS feed every 5 minsI am working on a monitor signage display and have a "welcome to RSS" feed with just a title and description. I have code from feedEk that's been tweaked a bit to parse the feed and cycle it so I only have one title and desc. showing at a time. This feed could be added to or deleted info at any time so I need it to refresh every five minutes. I've tried several solutions on here and just can't seem to work it out. Here is the adjusted FeedEk code with comments on the adjustments: ``` (function (e) { e.fn.FeedEk = function (t) { var n = { FeedUrl: "http://myrss.com/", MaxCount: 1, ShowDesc: true, ShowPubDate: false, CharacterLimit: 100, TitleLinkTarget: "_self", iterate: false }; if (t) { e.extend(n, t) } var r = e(this).attr("id"); var i; processFeedData = function (t) { //This just makes it flash too much //e("#" + r).empty(); var s = ""; en = t.responseData.feed.entries; if (n.iterate == true) { //Setting a variable to store current item i = window.feedcur = typeof(window.feedcur) === 'undefined' ? 0 : window.feedcur; t = en[i]; s = makeString(t); //incrementing the current for the next time we loop through window.feedcur = ((i+1)%en.length); } else { for (i=0;i' + s + ""); } makeString = function (t) { s = '' + t.title + ""; if (n.ShowPubDate) { i = new Date(t.publishedDate); s += '' + i.toLocaleDateString() + "" } if (n.ShowDesc) { if (n.DescCharacterLimit > 0 && t.content.length > n.DescCharacterLimit) { s += '' + t.content.substr(0, n.DescCharacterLimit) + "..." } else { s += '' + t.content + "" } } return s; } if (typeof(window.feedContent) === 'undefined') {
- pattern minor 112d agoFeed API to display RSS feeds Ted talkI am trying to create a webpage that will read RSS feeds from the TED talk website and display it on a page. I am using Google's Feed API for this. Here is the link to view the code online. Could someone please tell me if there is a better way to display the data coming from the feed? HTML ``` TED Talks Google Feed API Welcome to the Ted talks Feed ``` Feed API (JavaScript) ``` google.load("feeds", "1"); // Our callback function, for when a feed is loaded. function feedLoaded(result) { if (!result.error) { for (var i = 0; i ' + 'View on Ted.com' + ''; var image = result.feed.entries[i].mediaGroups[0].contents[0].thumbnails[0].url; $(".inner").append(' '+ entry.title +' ' + entry.contentSnippet + ''+ link + ' '); } //On click of the thumbnail(Image), we will display more data with more metadata. $(".thumbnail").click(function() { //Removing all arrows before initializing $(".arrow-up").remove(); var parentClass = $(this).parent(); //Checking to see if the parent class already is expanded or not if ($(parentClass).hasClass( "expandedMain" ) ) { $("div").removeClass("expandedMain"); $(".expandedView").remove(); $(".arrow-up").remove(); } else{ $(".arrow-up").show(); $("div").removeClass("expandedMain"); $(this).parent().addClass("expandedMain"); var plant = $(this).parent(); var DataCount = plant[0].dataset.num; var entry = result.feed.entries[DataCount]; //Appending the cideo in a variable for use later var video = ' '; $(plant).append(' '+ entry.title +' ' + video + 'Published Date:'+ entry.publishedDate +' ' + entry.content + ''); $(".expandedView").show(1000); }
- pattern minor 112d agoRSS feed parserI've just started learning Swift. I started with an example of RSS feed parser. I have a wrapper class written in Objective-C which converted the XML response into `NSDictonary`. I'm using the `NSDictionary` returned from my wrapper class. Then I need to iterate through the `NSDictionary` and fetch all the title and display it in a `tableview`. Here is the code which I've written to iterate through `NSDictionary` and fetch all the value for the key "title": ``` let tempDict:NSDictionary = responseDict["rss"] as NSDictionary let tempDict1:NSDictionary = tempDict["channel"] as NSDictionary let contentArray:NSArray = tempDict1["item"] as NSArray; for (key, value : AnyObject) in enumerate(contentArray){ let titleDict = value["title"] as NSDictionary let title = titleDict["text"] as NSString println("Title \(title)") } ``` This feed that I'm trying to parse. How can I optimise the above code? Also, can you provide some hints on getting started with Swift basics? I've already tried the docs but I feel it's a bit hard.