{"id":1599,"date":"2010-01-23T17:50:15","date_gmt":"2010-01-23T22:50:15","guid":{"rendered":"http:\/\/judykat.com\/ken-judy\/?p=1599"},"modified":"2012-03-31T20:41:15","modified_gmt":"2012-04-01T00:41:15","slug":"rss-feed-site-syndicate","status":"publish","type":"post","link":"https:\/\/judykat.com\/ken-judy\/rss-feed-site-syndicate\/","title":{"rendered":"Consume a feed from a site that doesn&#8217;t syndicate using Yahoo Pipes"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" alt=\"clownmommy screenshot\" src=\"http:\/\/judykat.com\/ken-judy\/wp-content\/uploads\/public\/blogimages\/clown-mommy-screenshot.png\" title=\"clownmommy screenshot\" class=\"alignright\" width=\"267\" height=\"123\" \/>It&#8217;s surprising how hard it is to subscribe to sites intent on driving traffic to their main page, particularly if you want a subset of content by author or topic.<\/p>\n<p>My wife writes for the <span class=\"removed_link\" title=\"http:\/\/www.nycmomsblog.com\/kathieh\">nycmoms blog<\/span>. She wanted to show a list of her recent blogs on her <a href=\"http:\/\/clownmommy.com\">personal blog<\/a>.<\/p>\n<p>Nycmoms exposes a single feed aggregating recent posts across all authors.<\/p>\n<p><strong>Create a feed<\/strong><\/p>\n<p>My first iteration was to filter this feed through <a href=\"http:\/\/pipes.yahoo.com\">Yahoo Pipes<\/a>. Pipes provides a graphical interface for chaining rules for manipulating web content and exposing it in a feed.<\/p>\n<p>I used the &#8220;fetch feed&#8221; module pointed to the recent posts feed, connected it to the filter module set to only permit items where the author.name contained my wife&#8217;s handle. <\/p>\n<p>This produced a correct result. However, only one or two of my wife&#8217;s posts are in the &#8220;recent posts&#8221; feed at any given time.<\/p>\n<p>For the next iteration I found an alternative source outside the blog itself. Being a Yahoo tool, pipes provides a Yahoo search module. So, I searched for the phrase &#8220;Posted by KathieH&#8221; restricted to the site, nycmomsblog.com. <\/p>\n<p>This produced more but less precise results. It includes additional pages such as my wife&#8217;s bio and sorted by relevance not post date.<\/p>\n<p>I used the sort module on the item.updatedon field in the rss feed.<\/p>\n<p>Next I used the filter module to exclude anything that was not an actual post identified by the term: &#8220;NYC Moms: KathieH&#8221; and <a href=\"http:\/\/pipes.yahoo.com\/pipes\/pipe.run?_id=8a8edd5b1e5181a001892cee910c11fc&#038;_render=rss\">piped out to rss<\/a>.<\/p>\n<p>Then I used the RSS widget in wordpress to expose this feed on my wife&#8217;s blog. Done!<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Yahoo Pipes Screen Shot\" src=\"http:\/\/judykat.com\/ken-judy\/wp-content\/uploads\/public\/blogimages\/yahoo-pipes-screen-shot.png\" title=\"Yahoo Pipes Screen Shot\" class=\"alignnone\" width=\"588\" height=\"478\" \/><\/p>\n<p><strong>Consume the feed<\/strong><\/p>\n<p>Except, my wife&#8217;s blog began suffering from the dreaded &#8220;Fatal Error: Allowed Memory Size&#8221; error in the simplepie library used by the RSS widget.<\/p>\n<p><a href=\"http:\/\/wordpress.org\/support\/topic\/245958\">Well documented on the web<\/a>, this is caused by my service provider&#8217;s decision to restrict accounts to 32MB of memory for PHP script execution. This and the 50MB-200MB storage limits were reasonable when I first signed up with my ISP ten years ago but is about as appropriate as using a 80286 processor now.<\/p>\n<p>I decided to switch from server side processing to client side.<\/p>\n<p>Luckily, Yahoo Pipes exposes feeds in the JSONP format. JSON is easily parsed by Javascript in the browser. <a href=\"http:\/\/bob.pythonmac.org\/archives\/2005\/12\/05\/remote-json-jsonp\/\">JSONP is an established workaround (i.e. hack)<\/a> that allows the client to make requests to third party URL&#8217;s without triggering the cross site security restrictions built into current browsers. It simply wraps the data response in a prefix that the browser parses as a function (allowed) as opposed to pure data (not allowed).<\/p>\n<p>WordPress already uses the jQuery library which is <a href=\"http:\/\/api.jquery.com\/jQuery.getJSON\/\">well documented by examples<\/a> on how to request, parse and render a feed.<\/p>\n<p>I used the getJSON function to retrieve and parse the feed. It is exposed as a nested array containing at the top level, key\/value pairs associated with the entire feed and an array of items containing the actual posts.<\/p>\n<p>I loop through the value.items format them into HTML and append them into a div with the id nycmoms_posts. Probably overkill given how fast this all works but I display a loading indicator which I fade out when the loop is complete (from <a href=\"http:\/\/astheria.com\/\">Kyle Meyer<\/a>).<\/p>\n<div style=\"background-color:black;color:white;white-space:pre;width: 550px; overflow: auto; padding: 5px;font-size:11px;font-family:courier new, monospace;\">\n<p>&lt;div id=&#8221;nycmoms&#8221;&gt;<br \/>  &lt;h2&gt;Kathie on &lt;a href=&#8221;http:\/\/www.nycmomsblog.com\/kathieh\/&#8221;&gt;nycmomsblog.com&lt;\/a&gt;&lt;\/h2&gt;<br \/>  &lt;div id=&#8221;nycmoms_loading&#8221; &gt;&#8230;&lt;\/div&gt;<br \/>  &lt;ul id=&#8221;nycmoms_posts&#8221; \/&gt;<br \/>&lt;\/div&gt;<\/p>\n<p>&lt;style&gt;<br \/>  #nycmoms { width:250px;min-height:125px;}<br \/>  #nycmoms_loading { position:absolute;top:0;left:0;width:100%;height:100px;padding:100px 0 0 125px;}<br \/>  #nycmoms .description, #nycmoms .postdate { font-size:small;}<br \/>&lt;\/style&gt;<br \/>&lt;script src=&#8221;wp-includes\/js\/jquery\/jquery.js?ver=1.3.2&#8243;&gt;&lt;\/script&gt;<br \/>&lt;script&gt;<\/p>\n<p>var descriptionPattern = \/^(.*) &#8230; Posted by Kathie H. on (\\w+) (\\d+), (\\d+) .*$\/g<\/p>\n<p>window.onload = function() {<\/p>\n<p>  jQuery.getJSON(&#8220;http:\/\/pipes.yahoo.com\/pipes\/pipe.run?_id=8a8edd5b1e5181a001892cee910c11fc&#038;_render=json&#038;_callback=?&#8221;, <br \/>    function(data) {<br \/>      jQuery.each(data.value.items, function(i, post) {<br \/>        jQuery(&#8220;#nycmoms_posts&#8221;).append(<br \/>          &#8216;&lt;li&gt;&lt;a href=&#8221;&#8216; + post.link +'&#8221;&gt;&#8217; + formatTitle(post.title) + &#8220;&lt;\/a&gt;&#8221;<br \/>          + &#8220;&lt;!&#8211; &#8220;+formatDescription(post.description)+&#8221;&#8211;&gt;&#8221;<br \/>          + &#8220;&lt;\/li&gt;&#8221;<br \/>          ) , jQuery(&#8216;#nycmoms_loading&#8217;).fadeOut(500);<br \/>      });<br \/>  });<\/p>\n<p>}<\/p>\n<p>function formatDescription(description)<br \/>{<br \/>  return description.replace(descriptionPattern, &#8216;&lt;span=&#8221;description&gt;&#8221;$1 &#8230; &lt;\/span&gt;&lt;span id=&#8221;postdate&#8221;&gt;posted $2 $3, $4&lt;\/span&gt;&#8217;);<br \/>}<br \/>function formatTitle(title)<br \/>{<br \/>  return title.replace(\/^NYC Moms: \/, &#8221;);<br \/>}<\/p>\n<p>&lt;\/script&gt;\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>It&#8217;s surprising how hard it is to subscribe to sites intent on driving traffic to their main page, particularly if you want a subset of content by author or topic. My wife writes for the nycmoms blog. She wanted to &hellip; <a href=\"https:\/\/judykat.com\/ken-judy\/rss-feed-site-syndicate\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[30],"tags":[197,687,698,699,700,695,697,696],"class_list":["post-1599","post","type-post","status-publish","format-standard","hentry","category-software-development","tag-computing","tag-html","tag-html-element","tag-jquery","tag-json","tag-markup-languages","tag-span-and-div","tag-world-wide-web"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/posts\/1599","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/comments?post=1599"}],"version-history":[{"count":17,"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/posts\/1599\/revisions"}],"predecessor-version":[{"id":1615,"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/posts\/1599\/revisions\/1615"}],"wp:attachment":[{"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/media?parent=1599"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/categories?post=1599"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/judykat.com\/ken-judy\/wp-json\/wp\/v2\/tags?post=1599"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}