Archive for the ‘Javascript’ Category

REST, WEB API And CORS   2 comments

Introduction

Cross Domain AJAX calls (CORS) on desktop browsers require special processing on both the server side and in the way we call AJAX from within the browser. A general overview of CORS can be found hereASP.Net WEB API allows a couple of fairly straight forward ways to implement REST HTTP endpoints with CORS support.  Using the current release build of WEB API we can code our CORS handlers directly or if you want to use the nightly builds of the WEB API you can use an attribute approach.  This post will concentrate on how to write CORS handlers directly since this is the approach I have this in test right now and this approach allows you more flexibility in implementation and improved debugging options.  I will be concentrating on implementation details and assume you have read the background blogs listed above before we start.  I will also be looking at the browser side implementation of the CORS call and some issues with IE browsers (IE 9 in particular).  We are testing with Windows Server 2012 and are using Firefox, Chrome and IE as our test browsers.

Voice from the future: Brock Allen’s great work on CORS, CORS based CORS Attribute support has now been incorporated into Web API 2.  See here and here for details.

So What’s the Problem.

The W3C defines special procedures required if a browser is going to make an AJAX call to a server which is not in the domain of the page which served the page which is making the call (hence Cross Domain).  To enable CORS the server must implement CORS and the browser must make the AJAX call following some conventions.  In the WEB API framework CORS can be implemented on the method or site level.  We will focus on site level CORS in this post.  The WEB API pipeline allows us to hook in message handlers at several places.  The canonical CORS handler, given by the links listed above looks like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Security.Claims;
using System.Threading;
using System.Threading.Tasks;
using System.Web.Http;
using System.Web.Http.Tracing;

public class CorsHandler : DelegatingHandler
{
const string AccessControlRequestMethod = “Access-Control-Request-Method”;
const string AccessControlRequestHeaders = “Access-Control-Request-Headers”;
const string AccessControlAllowOrigin = “Access-Control-Allow-Origin”;
const string AccessControlAllowMethods = “Access-Control-Allow-Methods”;
const string AccessControlAllowHeaders = “Access-Control-Allow-Headers”;

       protected override Task<HttpResponseMessage> SendAsync( HttpRequestMessage request, CancellationToken cancellationToken )
{
const string Origin = “Origin”;
bool isCorsRequest = request.Headers.Contains( Origin );
bool isPreflightRequest = request.Method == HttpMethod.Options;
if ( isCorsRequest )
{
//HTTP CORS OPTIONS
if ( isPreflightRequest )
{
HttpResponseMessage response = new HttpResponseMessage( HttpStatusCode.OK );
response.Headers.Add( AccessControlAllowOrigin, request.Headers.GetValues( Origin ).First( ) );
                   string accessControlRequestMethod = request.Headers.GetValues( AccessControlRequestMethod ).FirstOrDefault( );
if ( accessControlRequestMethod != null )
{
response.Headers.Add( AccessControlAllowMethods, accessControlRequestMethod );
}

                   string requestedHeaders = string.Join( “, “, request.Headers.GetValues( AccessControlRequestHeaders ) );
if ( !string.IsNullOrEmpty( requestedHeaders ) )
{
response.Headers.Add( AccessControlAllowHeaders, requestedHeaders+”, AICJWT” );
}

                   TaskCompletionSource<HttpResponseMessage> tcs = new TaskCompletionSource<HttpResponseMessage>( );
tcs.SetResult( response );
return tcs.Task;
}
else
{
//HTTP CORS GET
return base.SendAsync( request, cancellationToken ).ContinueWith<HttpResponseMessage>( t =>
{
HttpResponseMessage resp = t.Result;
resp.Headers.Add( AccessControlAllowOrigin, request.Headers.GetValues( Origin ).First( ) );
return resp;
} );
}
}
else
{
//NOT A CORS CALL
return base.SendAsync( request, cancellationToken );
}
}
}

Lets break this down from the simplest part first.  We create a class derived from DelegatingHandler (since we are implementing at the site level).  We hook this handler into the system within the framework generated class WebApiConfig as

public static class WebApiConfig
{
public static void Register( HttpConfiguration config )
{
//your route code here

           );
config.MessageHandlers.Add( new WebAPI.Handler.CorsHandler( ) );

//other handlers are included here.
}
}

If you have other classes based on DelegatingHandler the order in which they are added in WebApiConfig matters.

In the simplest case where we are not making a CORS call we can simply return the handler without action as:

return base.SendAsync( request, cancellationToken );

When the CORS call is made by the browser the caller should include the standard HTTP header: Origin with a value of the calling pages domain.  The canonical code assumes this and uses the presence of this header to detect a CORS call. Hence the code:

const string Origin = “Origin”;
bool isCorsRequest = request.Headers.Contains( Origin );

If the CORS call is not an OPTIONS call (which the canonical code call preFlight) we see the code:

return base.SendAsync( request, cancellationToken ).ContinueWith<HttpResponseMessage>( t =>
{
HttpResponseMessage resp = t.Result;
resp.Headers.Add( AccessControlAllowOrigin, request.Headers.GetValues( Origin ).First( ) );
return resp;
} );

Here the code returns a required header for the Browser: Access-Control-Allow-Origin with the value taken from the Origin Header of the caller.

We could, if we choice to, have set the value to  the wild card value ( * ) but this openness may make your system administrator nervous.  Notice here that nothing in the W3C specification restricts what other Headers the sender can include in the CORS call.  However certain browsers (IE) and certain Javascript packages (jQuery) restrict the call to standard HTTP request Headers.  In our implementation this gave us some problems but more on this later. The browser code (User-Agent), not the user code, will refuse to accept the return if the Origin Header is missing or does not contain either the wild card or the calling page’s domain in the value for the Origin header.

So What is the Rest of the Handler Code Doing?

Following this document from Mozilla.org, the browser making the call may make an optional CORS OPTIONS call (see here for HTTP verbs if this one is new to you).  This preflight call (as the canonical code names it) has asks the server for details about what may be in the CORS request when it is actually call.  Following the Mozilla explanation here is what needs to happen:

    • 1a. The User-Agent, rather than doing the call directly, asks the server, the API, the permission to do the request. It does so with the following headers:
      • Access-Control-Request-Headers, contains the headers the User-Agent want to access.
      • Access-Control-Request-Method contains the method the User-Agent want to access.
    • 1b. The API answers what is authorized:
      • Access-Control-Allow-Origin the origin that’s accepted. Can be * or the domain name.
      • Access-Control-Allow-Methods a list of allowed methods. This can be cached. Note than the request asks permission for one method and the
        server should return a list of accepted methods.
      • Access-Allow-Headers a list of allowed headers, for all of the methods, since this can be cached as well.

In the canonical code given above here is what happens in the CORS OPTIONS call:

//( 0 )create a response object

HttpResponseMessage response = new HttpResponseMessage( HttpStatusCode.OK );
//( 1 ) build the value string for the Access-Control-Allow-Origin header from the ORIGIN header value of the request

response.Headers.Add( AccessControlAllowOrigin, request.Headers.GetValues( Origin ).First( ) );

//( 3 )build the value string for the Access-Control-Request-Headers from the values in the request
string accessControlRequestMethod = request.Headers.GetValues( AccessControlRequestMethod ).FirstOrDefault( );
if ( accessControlRequestMethod != null )
{
response.Headers.Add( AccessControlAllowMethods, accessControlRequestMethod );
}

//( 4 ) build the value string for the Access-Control-Allow-Headers header from the ORIGIN headers value of the request

string requestedHeaders = string.Join( “, “, request.Headers.GetValues( AccessControlRequestHeaders ) );
if ( !string.IsNullOrEmpty( requestedHeaders ) )
{
response.Headers.Add( AccessControlAllowHeaders, requestedHeaders);
}

//( 5 ) interrupt the pipeline and return the response object to the caller.

TaskCompletionSource<HttpResponseMessage> tcs = new TaskCompletionSource<HttpResponseMessage>( );
tcs.SetResult( response );
return tcs.Task;

Please note that we can put whatever we need into the Header values.  For example if we wanted to limit CORS calls to GET request only we could replace ( 3) with the simple:

response.Headers.Add( AccessControlAllowMethods, “GET” );

To allow a specific domain only to make the CORS call we could replace ( 1 ) with:

response.Headers.Add( AccessControlAllowOrigin, “www.special.com” );

In our case we wanted to allow a specific non-standard Header into the CORS request.  We called this Header AICJWT. So we expanded the key line in ( 4 ) to be:

response.Headers.Add( AccessControlAllowHeaders, requestedHeaders+”, AICJWT” );

The reason we added it explicitly here is due to problems in both JQuery and in IE.  Please note again that the CORS OPTIONS call is optional.  At this point in our development we were using the awesome async Framework 4.5  object: System.Net.Http.HttpClient.  This is a get object and very useful during development BUT there is no User-Agent (browser side code) involved.

The Trouble Begins: Browser Side Code

All seemed swell, till the JavaScript coders tried to call into the system.  JQuery forces a CORS OPTIONS call when it detects a Cross Domain AJAX call.  For reasons which remain unclear JQUERY does not include custom headers in the OPTIONS request.  Some people think this is in the W3C spec for CORS but I don’t see it there, do you?  Some folks out there indicate that the user-agent is forcing the OPTIONS request but this is not true.  If we use a direct AJAX call, not using JQUERY we can make our own CORS OPTIONS request or skip the OPTIONS call completely.  Here is the code to make the call using JavaScript in IE:

function callHttpReq() {
var invocation = new XMLHttpRequest();
var url = ‘
http://whereever/api/Concert?Year=1980′;
    var body = ”;

var token = “myspecialstuff”;
function callOtherDomain(){
if(invocation)
{
invocation.open(‘GET’, url, true);
invocation.setRequestHeader(‘AICJWT’, token);
invocation.setRequestHeader(‘Accept’, ‘application/json’);
invocation.onreadystatechange = httpHandler;
invocation.send(body);
}
}
callOtherDomain();

    function httpHandler() {
if(invocation.readyState == 4) {
var jsonObj = $.parseJSON(invocation.responseText);
if(jsonObj.length > 0) {
htmlStr = “<ul>”;
$.each(jsonObj, function(i,row) {
htmlStr += “<li>” + row.Date + ‘—-‘ + ” ” + row.Venue +”</li>”;
});
htmlStr += “</ul>”;
$(“#responeBody”).append(htmlStr);
}
}
}

}

Note we are skipping JQUERY because we require a custom header in our use case.  This step is not necessary if you are NOT using custom headers in the CORS call.  Note also that if you are not using JQUERY you need to use different AJAX object other than IE’s XMLHttpRequest.  If you can use JQUERY there is a masive amount of documentation about how to make AJAX calls and JQUERY will handle CORS and differences between the IE and other AJAX objects automatically.

IE Blues

OK all is good but when we test with IE 8 or 9 we get back the data from the CORS get BUT the user also gets the dialog box:

image

Microsoft tells us the USER can suppress this IN IE8 and IE9 by following this procedure:

You can check your Security Zone level as well as change this setting by going to Tools, Internet Options, and click Security tab. Your Security Zone level will be listed on this screen, by default this is set to Medium-high. To change the setting for your message click Custom Level , locate the Access data sources across domains under the Miscellaneous section and change the setting from Prompt to a desired setting.

image

We do not have this problem in Chrome or Firefox. Live Free or Die.

One Last Server Side Issue

During our testing, using Windows Server 2012 we ran into one additional problem.  Our CORS OPTIONS calls were not getting to our site but where being intercepted by the HTTP Module prior to the site Delegate Handler.  Without getting into it too deeply we needed to modify the web.config for our CORS site and disable WebDAV (don’t ask) and allow OPTIONS for the ExtensionlessUrlHandler.  See here for details.  As far as we know this is a pure Windows 2012 server issue.

My (Virtual) Year On Tour With the Grateful Dead   1 comment

 

From Tape to The Internet

Crimson flames tied through my ears
Rollin’ high and mighty traps
Pounced with fire on flaming roads
Using ideas as my maps
“We’ll meet on edges, soon,” said I
Proud ’neath heated brow
Ah, but I was so much older then
I’m younger than that now

My Back Pages by Bob Dylan

I first heard the Grateful Dead live in Albuquerque in 1971.  Like many, this was a seminal experience for me, changing my understanding of the meaning of performance and of Rock and Roll.  That over weight, black clad, Prankster with a halo of unmanageable black hair playing guitar was clearly the center of the band and the performance.  I was young and was unaware that this was Captain Trips.   Captain America was more like it.  Nor was I aware of the mythical aura that was even then growing around the band and Jerry and their fans.  Like many, my first experience of the Dead was prepared only by my youth and The Bear’s  purple haze of the night.  In the intervening years I was more attracted to Frank Zappa who produced a consistent recording experience that the Dead never would achieve. After the (limited) success of the Grateful Dead Movie in capturing what the Dead were, the world moved on. But the band played on.  Perhaps it is better that way.  The early taping of Dead shows from  the sound boards (thank you Owsley) and later by dedicated deadhead tapers left us with a rich vein of music and magic in the over 3000 individual live performances available in one form or another.  Beginning in the last decade of the last century, the Dead organization began to issue live sound board recordings from this corpus. 

Dick’s Picks and Me

Half-wracked prejudice leaped forth
“Rip down all hate,” I screamed
Lies that life is black and white
Spoke from my skull. I dreamed
Romantic facts of musketeers
Foundationed deep, somehow
Ah, but I was so much older then
I’m younger than that now

While a great resource for those of us not conversant in bit torrent, these releases where frequently expensive and (to my ears) over produced which subtracted from the raw energy of the original sound board tapes.  I am lucky to have access to a great music store with plentiful numbers of used Grateful Dead CD.  If you are in Maine visit a local Bull Moose Music store.   I was luck to be able to pick up many Live Dead concert CDs at a reasonable price.  Thank you Bull Moose.  Although the official Dead releases enhanced my life, there are some problems:

I) The list prices are quite high (try to get used copies)

II) Selections for the Dick’s Pick Series seem to be primarily based on best complete shows (The Dead were often hit and miss in the same night and limiting yourself to the best complete show skips a lot of great music. This problem has been reduced by the newer Road Trips Series and specialty releases like Ladies and Gentlemen… the Grateful Dead which cooks down the best of a four night stand at the Fillmore, 1971).

III) The processing of the raw tapes, IMHO, sometimes cook the life out of some the releases.

IV) Some of the specialty releases seem to be picked more for the historical importance than the quality of the performance (Closing the Winterland, for example).

Having said all that, if you can get the official releases used, some of them are great.

A Short Divergence in Our Story

Girls’ faces formed the forward path
From phony jealousy
To memorizing politics
Of ancient history
Flung down by corpse evangelists
Unthought of, though, somehow
Ah, but I was so much older then
I’m younger than that now

I started to cooked down my copies of the official live releases into playlist CDs (favorites of 1974, Dark Star releases, etc.).  Then I had open heart surgery, caught a post operative wound infection and almost died.  Stephen Gould wrote someplace that the greatest species in evolution are Bactria.  They are everywhere.  There are more Bactria in your body than body cells.  And I was in the three month war between the bugs and myself ( to be honest I had massive antibiotic infusions on my side).  My day was composed of pain medicine, James Joyce’s Ulysses, The Bible, and my CDs of live Dark Star performances.  Let’s just say that Dark Star and the Gospel of Mark were more significant than Tramadol in my recovery.   Rehab consisted of countless hours of treadmill work.  That, and an MP3 player packed with Scarlet Begonia and Fire on the Mountain.

BTW: Tom Constanten said somewhere that they didn’t play Dark Star, it was always going on,  they just joined in.  Although T.C. recommends ‘any East Coast Dark Star’ my favorites are early West Coast versions.

The Internet Archive Connection

In a soldier’s stance, I aimed my hand
At the mongrel dogs who teach
Fearing not that I’d become my enemy
In the instant that I preach
My pathway led by confusion boats
Mutiny from stern to bow
Ah, but I was so much older then
I’m younger than that now

The Internet Archive, in early 2000 began collecting, digitalizing and making available for re-distribution the large body of Grateful Dead concerts made by independent tapers and sound board recordings (SB) which were in circulation.  By policy SB recordings are available for playing on the web site and non-SB recordings are available for downloading.  There are multiple recordings available for most shows and these vary in quality from commercially releasable to barely audible.  There are over 8,000 individual recording of about 1,900 shows.  About 1,000 of these are SB. While vast the Internet Archive is not the most accessible site. Like most people I started with the feature of the Grateful Dead collection called:

Click Me:

 Grateful Dead Shows on This Day In History 

(If you have never been there – try the link right now).

For 08-30 (today while I am writing this) the Internet Archive will display 30 recordings (for shows of this date in 1985, 1983, 1981, 1980, 1970 and 1969).  There are limited sort options of these results.  Selecting a given recording brings one to a new web page containing an online player and (if the show is not an SB) download options.  I was hooked on the musical possibilities but trapped  by the limited user interface of the Internet Archive.   I wanted more.  Much more…

Hacking The Internet Archive

A self-ordained professor’s tongue Too serious to fool 
Spouted out that liberty
Is just equality in school
“Equality,” I spoke the word
As if a wedding vow
Ah, but I was so much older then
I’m younger than that now

My goal was to have the ability to listen to ALL of the Dead’s concerts but using only the best recordings and to be able to move through the collection using a better user interface which would allow me to decide where and when to go to any individual date. My goal was to spend a year and at least sample all 1,900 concerts and listen completely to all SB concert recordings.  I decided to complete this project in 12 months.  To do this I would first need to wrestle the Internet Archive (IA) to its knees.  Little did I know that this would take me on a programming journey involving three programing languages (Ruby, Javascript and C#), two data specifications (XML and JSON), two database engines (couchdb and SQL Server) as well as understanding the (somewhat loosely documented) search engine of IA, and more….  Readers interested in the technical details should see my series of postings on Ruby on Rails and CouchDB.  Please see Part 5 has the details of how to hack the Internet Archive to get at the data for the Grateful Dead recordings on IA.  Thus armed with the complete dataset from the Internet Archive of Grateful Dead recordings and a new front end I was ready to begin my listening  project.  There are over 8,000 recordings of over 2,000 concerts on the Internet Archive.  My first cut on the recordings in to use an algorithm to select ONE recording for each recording date for review.  This is a very simple selection based on the first of:

  • Was processed by Charlie Miller (IMHO the BEST processor of Grateful Dead Tapes)
  • Is a Sound Board Recording
  • Is  a Matrix Recording
  • Is the most recently posted tape for a given date.

Does this process miss some gems? Undoubtedly but it did give me 2,000 tapes to review rather than 8,000. With this criteria in place,  my local copy of the IA database and my own UI for IA I started listening in July, 2011.  I did not attempt to listen to all 2,000 recordings completely.  If a recording was of poor quality or the band was out of tune or Jerry was ‘uninspired’ I abandoned the tape have brief samples of my favorite tunes.  In the end I reviewed about 1,000 concerts by in thirteen months (I finished during the ‘days between’ period;[August 1 and  August 9]).  I ended up with about 475  concerts on my personal playlist of ‘greatest concerts’.  Along the way I wrote several reviews on this blog of concerts which I thought were particularly of note. and compiled hyperlinked list of shows by year (the series starts here) and hyperlinks to Dark Star concerts and Scarlet Begonia –> Fire on the Mountain concerts.  All of these blogs contain links to jump right into the concert within the Internet Archive (but you still need to use the IA music player however).  Do I have a favorite sequence of songs, a favorite concert, a favorite era.  Yes.  Am I going to tell you? No.  Dig in visit the Internet Archive and start listening.  It could save your life.

 

Days Between Grateful Dead

and there were days
and there were days I know
when all we ever wanted
was to learn and love and grow
Once we grew into our shoes
we told them where to go
walked halfway around the world
on promise of the glow
stood upon a mountain top

walked barefoot in the snow
gave the best we had to give
how much we’ll never know we’ll never know

Days Between by Garcia and Hunter

 

 

 

 

Was It Worth The Trip?

 

Yes!

 

                                                               To Bear and Captain Trips, we say Thank You and Rest In Peace.

All photos by cloud2013 except Bear and Captain Trips Credit: Rosi McGee 

 

PS: Stupid Grateful Dead Statistics From the Internet Archive Database

Top 12 Most Played By Era (excluding Space and Drums):

Title 1967-1971 1972-1978 1979-1990 1991-1995
Althea     *  
Big River   *    
Brown Eyed Women   *    
Casey Jones *      
Cassidy     *  
China Cat Sunflower *      
Corrina       *
Crazy Fingers       *
Cryptical Envelopment *      
Cumberland Blues *      
Dark Star *      
Deal   *    
El Paso   *    
Estimated Prophet     *  
Eyes Of ThWorld       *
Good Lovin *      
Hard to Handle *      
I Know You Rider *   *  
Jack Straw   *    
Lazy River Road       *
Little Red Rooster     *  
Looks Like Rain     *  
Me and My Uncle *      
Mexicali Blues   *    
         
Not Fade Away * * * *
Playing In ThBand   * *  
Sugar Magnolia   * * *
Sugaree   *    
Tennesse Jed   *    
Terrapin Station       *
The Other One     *  
Throwing Stones       *
Truckin   * *  
Turn On Your Lovelight *      
Uncle Johns Band *     *
Wang Dang Doodle       *
Way To Go Home       *
Wharf Rat     *  
When I Paint My Masterpiece       *

Internet Archive:  All Recordings and Sound Board Recordings

image

Concert Length

image

Song Counts By Year(Dark Star, Playin’ in the Band and Scarlet Begonia –> Fire On The Mountain)

ffimage

Microsoft MVC 3 and CouchDB – Low Level Get Calls   1 comment

I have written elsewhere on couchdb on Windows and using Ruby on Rails to interface to this system.  These posts can be found here:couchdb

Part 0 – REST, Ruby On Rails, CouchDB and Me

Part 1 – Ruby, The Command Line Version

Part 2 – Aptana IDE For Ruby

Part 3 CouchDB Up and Running on Windows

Part 4 – CouchDB, Curl and RUBY

Part 5 – Getting The Data Ready for CouchDB

Part 6 – Getting The Data Into And Out Of CouchDB

Part 7 – JQUERY,JPlayer and HTML5

In my work life I work in a Microsoft shop which for us means Microsoft servers for the back end and (mostly) pure HTML/AJAX frontends.  We are transitioning towards using Microsoft MVC 3 to provide HTTP end points for our AJAX calls.  Here are some notes from my POC work in this area.  My couch data consists of documents describing Grateful Dead concerts stored on the great site Internet Archive, if you have never visited the Internet Archive, please do so.  I back engineered the meta data of IA’s extensive collection of Dead concerts (over 2,000 concert recordings).  Visit the Grateful Dead Archive Home at the Internet Archive here.

CouchDB Documents and Views

I stored the meta data into a local couchdb (running on Windows XP).  The basic document I am storing is a master detail set for the ‘best’ recording for each Dead concert.  The Master part of the document contains the date, venue and other data of the concert and the detail set is an array of meta data on each song preformed during the concert.  As is traditional with couchdb, the documents are represented as JSON strings.  Here is what the document for the UR recording (1965-11-01) found on the IA:

{

“_id”: “1965-11-01”,tumblr_ld4jfoNw7F1qai6ym

“_rev”: “1-6ea272d20d7fc80e51c1ba53a5101ac1”,

“mx”: false,

“pubdate”: “2009-03-14”,

“sb”: true,

“venue”: “various”,

“tracks”: [

{

“uri”: “http://www.archive.org/download/gd1965-11- 01.sbd.bershaw.5417.sbeok.shnf/Acid4_01_vbr.mp3”,

“track”: “01”,

“title”: “Speed Limit”,

“time”: “09:48”

},

{

“uri”: “http://www.archive.org/download/gd1965-11-01.sbd.bershaw.5417.sbeok.shnf/Acid4_02_vbr.mp3&#8221;,

“track”: “02”,

“title”: “Neil Cassidy Raps”,

“time”: “02:19”

}

]

}

Couchdb allow the creation of views which are binary trees with user defined Keys and user defined sub sets of the document data.  If one wanted to return the venue and the tracks for each concert for a given Month and Day (across all years) the view created in couchdb would look like:

“MonthDay”: {

“map”: “function(doc){emit(doc._id.substr(5,2)+doc._id.substr(8,2),[doc.venue , doc.IAKey, doc.tracks ])}”

}

This view allows us to use and HTTP GET to pass in a monthday key (e.g. “1101”) and get back (as a JSON array)

the date (MMDDYY: doc._id.substr(5,2)+doc._id.substr(8,2))

the venue (doc.venue);

the AI URI of the concert (doc.IAKey); and

an array of track data (doc.tracks)

MVC URL Routing Maps

Although we could call couchdb directly from the browser, we normally work through a gateway system for security, so we will build a shim to sit between the browser and couchdb.  This allows us to flow the authentication / authorization stack separately from couchdb’s security system.  In MS MVC we can create a new HTTP endpoint for AJAX calls (our shim) is a very simple manner. Let’s create an endpoint which will look like:

http:\\{our server path}\DeadBase\MonthDay\{month}\{day}

where vacuum_routing

http:\\{our server path}\DeadBase\MonthDay\111

would request month:11 and day:01 concerts.  In MVC we can declare this routing as:

routes.MapRoute(

“MyMonthDay”,

“{controller}/{action}/{month}/{day}”, 

new { controller = “DeadBase”, action = “RestMonthDay”,null} );

Done.  Interestingly in MVC 3 this route definition will accept either the form:

http:\\{our server path}\DeadBase\MonthDay\{month}\{day} ; or

http:\\{our server path}\DeadBase\MonthDay?month=”??”&day=”??”

In the second form,  parameter order does not matter, but case does; quotation marks are optional and need to be dealt with internally by the action method.

either of these call will resolve to the same controller and method.

MVC Controller and Method HandlerMVC

We now need to create the shim which will be the target for the Http Endpoint.  In C# this looks like:

public class DeadBaseController : Controller

public string RestMonthDay( string month, string day )
{
//our shim code goes here

      }

    }

We able to use string as our return type because we will be calling couchdb which returns a string from of JSON by default.  As a side note if we wanted to use MVC 3 to return JSON from a native C# object our controller method takes a different form:

public JsonResult GetStateList()

{

List<ListItem> list = new List<ListItem>() {

new ListItem() { Value = “1”, Text = “VA” },

new ListItem() { Value = “2”, Text = “MD” },

new ListItem() { Value = “3”, Text = “DC” } };

return this.Json(list);

}

Our AJAX call from the browser does not need to know any of these details.  Here is one way to code the call in JavaScript using JQuery:

var url = urlBase + “?” + args;ajax

$.ajax({

url: url,

dataType: ‘json’,

success: okCallBack,

error: nookCallBack

});

function okCallBack(data) {

gdData = data;

//do something useful here

}

function nookCallBack(xhr, ajaxOptions, errorThrown) {

alert(“ErrorText:” + errorThrown + ” ” + “Error Code:” + xhr.status);

}

}

From Handler to CouchDB in C#

Here is the rest of the generic C# code to go from the Handler to CouchDB and back.

Clean the parameters and pass the call to a generic couchDB GET caller:mvc

image

Format the view name and parameter into couchdb format  and pass to the low level couchDB caller:

image

Classic Framework HTTP code to make the HTTP GET and return the results as a string back up the call stack:

image

We could (and did) take our Browser code from the Ruby on Rails project above and with minimum changes call our MVC shim.

Simple clean and fun.

Occupy your mind2

Ruby On Rails, CouchDB and Me – Part 7 – JQUERY,JPlayer and HTML5   Leave a comment

Part 0 – REST, Ruby On Rails, CouchDB and Me

Part 1 – Ruby, The Command Line Version

Part 2 – Aptana IDE For Ruby

Part 3 CouchDB Up and Running on Windows

Part 4 – CouchDB, Curl and RUBY

Part 5 – Getting The Data Ready for CouchDB  

Part 6 – Getting The Data Into And Out Of CouchDB

Part 7 – JQUERY,JPlayer and HTML5

We have two missions in the current post:

  • Getting Our browser side chops together: Using Javascript, DHTML, CSS Level 3, JQUERY against our JSON feed
  • Using JQuery UI and a JQuery UI Plugin: JPlayer to play songs from Internet Archive based on our JSON feed

Recall that our JSON feed provides concert and track data for concerts preformed on an arbitrary date. The top level data of the feed can be visualized like this:

clip_image001

The field total row refers to the total number of records in the database not the number of rows in this feed. The field offset indicates the entry point in the b-tree of the view used for this feed.  Beats me why these would be useful to the calling program!  Following this ‘header’ data we have each concert listed in Key order. The offsets and values are

  • 0: Venue
  • 1: IAKey
  • 2: Array of Track Data

We can visualize the expanded  track data array as:

image Within each offset of the array we have the fields:

  • uri     – The pointer into IA for the mp3 file
  • track – order in the concert of this track
  • title   – track name
  • time  – Track length as MM:SS

We clearly could iterate through these fields and list the concerts and tracks statically on the web page using standard ROR tools but lets be more dynamic.  Let’s first display the concert dates and venues and then display the tracks for a concert  when the user click on a concert without a round trip to the server (and ROR).

Do You Do JAVASCIPT?

Someone once said that Javascript is the only languare that people use without knowning how. Don’t be one of those people.  The cleanest approach to learning Javascript is Crockfords: Javascript:  The Good Parts – Simple, clean Javascript fun.  (Steal This Book Here) Read this even if you ‘know’ Javascript.  If you don’t like to read, try the movie:

JQUERY: It Puts the Dynamic into DHTML.

JQuery is my favorite Javascript library.  Not necessarily the best or the most common.  Just my favorite.  JQuery accomplishes two goals very well:

  • Eliminating (or at least simplifying) DHTML coding differences between all main stream browsers (and most non-mainstream ones);
  • Simplifying and abstracting the operations necessary to drive DHTML via Javascript.

The design of JQUERY leverages the CSS 3 selector syntax so you will need to understand modern CSS selectors.

DHTML was first introduced as a Microsoft extension.  Netscape (remember Netscape?) soon followed with a similar, but not exact DHTML API of its own.  Further each of these browsers also tended to render certain edge cases differently.  And the CSS Level 3 Selectors and HTML5 specifications were coming down the pike. Both CSS3 and HTML5 are now a reality on Chrome, Foxfire and Safari (and some day, real soon) on IE9.  What to do?  John Resiq had an idea and the idea was called JQUERY.  The BASIC idea is to use the CSS Level 3 selectors to selects sets of HTML Tags and then to preform actions on those tags using a common API which would mask the differences between Browsers (and differences between versions of browsers).  Along the way JQUERY attempts to provide features not available in some browsers as long as those features would appear in the (then emergent) HTML5 specification.  Learning JQUERY is difficult only because the API is abstract and their is no BEST text on JQUERY.  Here is how John explains JQUERY:

OK So Lets See Some Code Already!

Iterating The JSON Object In Javascript And Display Using JQUERY

Please refer to our prior post for a description of how the JSON object is delivered to the page via the Rails mark up in our rb file.  Basically we had a single line:

gdData=<%=  @parm  %> ;

Let’s work with this data to display the structure on the browser screen.

We start with two EMPTY HTML tags on our page:

<div id=”concertdiv“>

<lu id=’track’></lu>

We can iterate this object  using javascript as:

ConcertList2(gdData);

where ConcertList2 is defined as:

function ConcertList2(o){
var iaURL=”http://www.archive.org/details/”
for (ndx = o[“rows”].length – 1; ndx !=-1; ndx–) {
var cdate = o[“rows”][ndx].id;
var venue = o[“rows”][ndx][“value”][0];
var itemID=ndx.toString();
var uri= iaURL +o[“rows”][ndx][“value”][1]
var href=”<a id=”href” href=””+uri+”” target=”_BLANK”>”+” – IA -” +””;
var className=’normal’;
if (ndx==0){
className=’hilite’;
}
var item=”

” + cdate + ‘ – ‘ + venue + href + ‘

‘ ;
$(‘#concertdiv’).after(item);
}
}

The javascript variable “item” for a given concert would contain a string of HTML:

<p id=’0’ class=’concert normal’>1969-08-16 – Woodstock Music <a href=http://www.archive.org/details/gd1969-08-16.sbd.gmbfixed.95918.flac16’/>-IA-</p>

Note that this tag contains two class: ‘concert’ and ‘normal’.

The JQuery code line:

$(‘#concertdiv’).after(item);

consists of a selector:

#concertdiv

an action verb:

after

and an argument:

item

The selector uses CSS 3 syntax so it selects the SET of all tags with the ID of ‘concertdiv’.  In our page this is a set of one item.

Iterating through our JSON object will post-pend our items after the tag associated with concertdiv

The results  looks like this on the Browser Page:

1969-08-16 – Woodstock Music – IA –

1980-08-16 – Mississippi River Festival – IA –

1981-08-16 – MacArthur Court – University of Oregon – IA –

1987-08-16 – Town Park – IA –

1991-08-16 – Shoreline Amphitheatre – IA –

Simple, no?

We can iterate and display the tracks as:

TrackList(gdData,ndx);

where TrackList is defined as:

function TrackList(o,ndx){
$(‘li’).remove();
var ndx1=0;
for(ndx1=o[“rows”][ndx][“value”][2].length – 1;ndx1!=-1;ndx1–){
var title=o[“rows”][ndx][“value”][2][ndx1].title;
var time=o[“rows”][ndx][“value”][2][ndx1].time
var track=o[“rows”][ndx][“value”][2][ndx1].track
var uri=o[“rows”][ndx][“value”][2][ndx1].uri
var item=”<li>” + track + ‘ ‘+ time +’ ‘+title+ ‘</li>’;
$(‘#track’).after(item);
}

}

In this case our ‘item’ variable contains a simple HTML string like:

<li>01 03:08 Stage Announcements, Introduction</li?

The results on the browser page for a given concert will look like this:

  • 01 03:08 Stage Announcements, Introduction
  • 02 02:04 Saint Stephen >
  • 03 02:42 Mama Tried >
  • 04 00:38 High Time false start
  • 05 10:28 Stage Banter. Technical Difficulties
  • 06 19:05 Dark Star >
  • 07 06:10 High Time
  • 08 38:32 Turn On Your Lovelight
  • 09 01:52 Applause, Stage AnnouncementsWe can bind these two display routines together with two simple Javascript functions so that when we click on a concert name the page will refresh the track list without a visit back to the web server.First we will use JQUERY to BIND a function to a click event to the concert class:

    function bindClick(){
    $(‘.concert’).click(function() {
    removehilite();
    $(this).toggleClass(‘hilite’,true);
    TrackList(gdData,$(this).attr(‘id’));
    });
    }

  • This bound function uses the pre-defined function ‘removehilite’ to swap

  • function removehilite(){
    $(‘.concert’).toggleClass(‘hilite’,false);
    $(‘.concert’).toggleClass(‘normal’,true);
    }

    and a simple inline CCS definition:

    <style>
    .normal {color:#0B559B;}
    .hilite {color:#FF0000;}
    </style>

    We pull this all together into a simple driver as:

                gdData=<%=  @parm  %> ;
    $(document).ready(function(){
    ConcertList2(gdData);
    bindClick();
    TrackList(gdData,”0″);
    });

    Got it? Good.  Now let’s use a JQUERY UI plug in allow us to play concerts from our browser page.

    JQUERY UI and JQUERY UI Widgets

    As useful as JQUERY is for dynamic web pages let’s go further use the JQuery UI system and the UI Widget: JPlayer,  to allow us to play the mp3 files which reside on Internet Archive.  JQuery UI  is a system built on top of JQUERY to allow the systematic development of UI Widgets which page developers can deploy which minimize un wanted interactions between widgets.  Further the JQuery UI system (and widgets developed within that system) can use a systematic set of theme classes whose color scheme can be generated with a nice tool called ThemeRoller.  I will not have a lot to say about these products in general (except to say they are free and work great) and you will need to visit the links noted in this paragraph to learn more about these tools.

    HTML5 Audio Tag

    HTML5 has introduced a new tag to allow playing audio without using a plug in.  There are some issues still being worked out since there is NOT common agreement yet about whether the standard should universally support MP4 or OGG files universally.  Currently MP3 is supported by all browsers which support HTML5.  Nominally the new tag looks like this:

    <audio controls=”controls”>
    <source src=”horse.mp3″ type=”audio/mp3″ />
    Your browser does not support the audio element.
    </audio>

    Note that the line after the “source” tags is what is rendered if your browser does NOT support HTML5.  If we replace this line with appropriate code to support a plug-in like Flash we have a control which will play well in both HTML5 and HTM4 environments.  We could develop our own solution but I have been working with JPlayer a very nice JQUERY UI widget and will use that for this post.  I like this widget because JPlayer

    • Is a JQuery widget
    • Works with JQuery Themeroller
    • Has a very active user community
    • Displays graphics and video as well as audio tracks

    I developed my final browser page in this series using a modified version of  the  ‘demo 2’ code example which is downloaded along with JPlayer.  Here is the plan:

    Display the Concert list the same way as above (with a few extras for visual appeal).  Prepare the track list in a way similar to that used above but modified to put it in a form that JPlayer can both display the tracklist for us and load the track list into JPlayer (more on this below).  We are going to modify the RoR rb file but not the underlying R0R code.  We will let the browser do the work.  I follow this strategy since our next phase of the project will allow the user to select the date for which concert data is to be displayed and played using AJAX calls in a RESTful manner (more on this next time) rather than round tripping to the server when we want to load a new date (or date range).

    What changes?

    Two new Javascript files:  one for JPlayer and one to handle preparing the track list for JPlayer to consume; and a reference to the themeroller prepared CSS file:

    <link href=”/skin/jplayer.blue.monday.css” rel=”stylesheet” type=”text/css” />

    <script type=”text/// <![CDATA[
    javascript” src=”
    // ]]>http://ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js“></script>

    <script type=”text/// <![CDATA[
    javascript” src=”
    // ]]>/javascripts/jquery.jplayer.min.js“></script>

    <script type=”text/// <![CDATA[
    javascript” src=”
    // ]]>/javascripts/Playlist.js“></script>

    Playlist.js is the new file I am contributing to the mix the others are JQuery and JPlayer Javascript files.

    Using Themeroller styles I can generate a completely different style for the page and only chang the link reference to the CSS page to change how the page appears.  There are NO embedded style elements on the page.

    Rather than calling the TrackList method I am going to call a new method makePlayList when a concert is selected. This code  looks like this:

    function makePlayList(selected){

    var ndx1=0;

    var tList=new Array();

    for (ndx1 = gdData[“rows”][selected][“value”][2].length – 1; ndx1 != -1; ndx1–) {

    tList[ndx1] = buildTrack(gdData[“rows”][selected][“value”][2][ndx1].track, gdData[“rows”][selected][“value”][2][ndx1].time, gdData[“rows”][selected][“value”][2][ndx1].title, gdData[“rows”][selected][“value”][2][ndx1].uri, gdData[“rows”][selected].id.substring(0, 4));

    }

    return tList;

    }

    In turn, buildTrack looks like:

    var buildTrack=function(num,time,title,ref,cYear){
    var dwnldicon=’pic/download.png’;
    var    nameFMT=”$0     $1     $2<img src=’$4′>“;   
    var track=new Object();
    var name=nameFMT.replace(“$0”,num);
    name=name.replace(“$1”,time);
    name=name.replace(“$2”,title);
    name=name.replace(“$3”,ref);
    name=name.replace(“$4″,dwnldicon);
    track.name=name;
    track.mp3=ref;
    track.poster=”/pic/”+cYear+”.png”;
    return track;
    };

    All of which is returned to JPlayer as:

    mediaPlaylist.playlist = makePlayList(selected);

    Our core Javascript code now looks like:

    var mediaPlayer=null;

    $(document).ready(function(){

    ConcertList2(gdData);  //displays the concert list at the top of the page

    bindClick();  //binds a click event on a concert to loading a new playlist to JPlayer

    mediaPlaylist = new Playlist(“1”,makePlayList(0) , //jump start with the first concert item

    {

    ready: function() {

    mediaPlaylist.displayPlaylist();  //show the playlist

    mediaPlaylist.playlistInit(false);

    },

    ended: function() {

    mediaPlaylist.playlistNext();

    },

    swfPath: “javascripts”,  //jplayer option

    solution: “flash, html”, //jplayer option

    supplied: “mp3” //jplayer option

    });

    });

    Most of our HTML tags are stolen directly from the JPlayer ‘demo 2’ code and mostly deal with setting up the player controls (play, pause, stop, next, etc).

    OK.  The new browser page looks like this (in two Parts):

    Concert Listing Section:

    clip_image001[1]

    I am using icons for hyperlinks to the Internet Archive Grateful Dead Collection: and the smaller icons to link to all recordings for a given date:.   The bottom half of the screen contains the JPlayer and its user controls as well as a user selectable track list:

    image The image is associated with the track selected (there is a JPlayer bug  with the images.  If the same image is associated with two seccesive tracks the second picture will not be displayed – they are working on this). I use the selected Concert text (in this case “Madison Square Garden: 1987-09-16” as a hyperlink to the page containing the  concert recording on Internet Archive.

    These screen captures are from Chrome (Safari and FireFox look the same).  On IE 8 (and lower) HTML5 is not supported and the player reverts to Flash.  The track list on IE8 is not as pretty and is no longer selectable (although the player controls still work):

    image The Sad IE 8 Track Display

    What more can I say?

REST, Ruby On Rails, CouchDB and Me – Part 6 Getting The Data Into And Out Of CouchDB   1 comment

Part 0 – REST, Ruby On Rails, CouchDB and Me

Part 1 – Ruby, The Command Line Version

Part 2 – Aptana IDE For Ruby

Part 3 CouchDB Up and Running on Windows

Part 4 – CouchDB, Curl and RUBY

Part 5 – Getting The Data Ready for CouchDB

Part 7 – JQUERY,JPlayer and HTML5

 [This just in: couchDB and Microsoft MVC 3: here]

Our mission in today’s post: 

  • Serialize the Concert object to a JSON string (i.e. a JSON document);
  • Do POST requests to insert  a JSON document for each concert into the couchdb database;
  • Create couchDB views to allow optimized data retrieval; and
  • Create a couchDB view to optimize retrieval recordings for all years for an arbitrary Month and Day (this duplicates the data provided by the “Grateful Dead Shows on This Day In History” selection in the Internet Archive.

Our last post outlined the formal structure of our document and the JSON format of the document we intend to post to our couchdb database.  Also in that post we outlined the RUBY code used to collect and clean the data we accessed from the Internet Archive.  In order to place our concert information into our couchdb database we need to transform our RUBY objects into proper JSON format and then call an HTTP POST to place our resource (document) into the database.  Why a POST rather than a PUT command you ask?  Better to ask a true RESTafarian.  In REST PUT is used to create a resource (in couchdb terms this is the database) and POST is used to modify a resource (in our case create a document within the database – which modifies the database I guess).  Hmmmm.

Why JSON?  Ask These Guys:

dsc_0029

To transform a RUBY object into JSON format requires the use of a RUBY GEM: json.  This GEM comes in two flavors, a pure RUBY form (slower) and a C form (faster, but it requires that you have a working RUBY development stack, GNU C compiler etc. deployed on your local machine).  Since we are happy with the speed of the pure RUBY form.  The downside of this GEM is that it will not serialize pure RUBY objects as is.  To do that you need to code custom serialization handlers. By default, the JSON serializer will work directly on primitive (native) types (strings, ints, what have you), and the simple structures: arrays and hash tables.  So our first task will be to transform our data into hash table format.  Assume a RUBY array of concert objects (@selectList) and a parallel array of data for each track (tracks). In pseudo code we have:

@selectList.each do |recording|
tracks=GDConcertTracks.new(recording)
tracks.getTrackDetails

   jsonString=makeJSON(recording,tracks)
PostRecording(jsonString)
end

 

Our method makeJSON takes the RUBY objects and returns a JSON string as:

def makeJSONFull(recording,tracks)
tList=Hash.new()
tList[“_id”]=recording.date
tList[“IAKey”]=recording.uri
tList[“description”]=recording.description
tList[“venue”]=recording.title
tList[“pubdate”]=recording.pubdate
tList[“cm”]=recording.cm
tList[“sb”]=recording.sb
tList[“mx”]=recording.mx
tTracks=Array.new()
tracks.each do |t|
tItem=Hash.new()
tItem[“title”]=t.title
tItem[“time”]=t.time
tItem[“uri”]=t.trackURI
tItem[“track”]=t.track #this could be derived as offset – 1
tTracks[tTracks.count]=tItem
end
tList[“tracks”]=tTracks
return JSON.generate(tList)
end

where JSON.generate(tList) is the JSON GEM method generate (to serialize) and tList is a Hash containing only primitive types (strings) and a Hash list of track data.  The track data contains only primitive types (strings).  We pass the returned string (which contains our data in JSON format) to our routine PostRecording(jsonString).  This Routine looks like:

def PostRecording(jsonString)
uri=”
http://127.0.0.1:5984/deadbase/&#8221;
begin
responseBody=open(uri,:method=> :post, :body => jsonString,”Content-Type” => “application/json”).read
puts ‘POST Response Success: ‘ + responseBody
rescue OpenURI::HTTPError => the_error
puts ‘##########Post Response Error: ‘ + the_error.io.status[0]
end
end

HiREST-LowREST1

OK this routine depends on the RUBY GEM: rest-open-uri .  This GEM is a single file of RUBY code which depends on the RUBY components: uri and net/http (these are part of the base RUBY system).  “rest-open-uri” extends the base RUBY HTTP components for the HTTP GET verb  and extends them to include all of the HTTP verbs (PUT, POST, DELETE and HEAD)  necessary to implement any REST system.  The open method, as used here:

open(uri,:method=> :post, :body => jsonString,”Content-Type” => “application/json”).read

uri: the address of our couchdb database

Content-Type: our MIME type for this call (“application/json”)

body: our document

method: the HTTP verb POST
If this POST command fails we want to capture the error and (for this example) display the error:

rescue OpenURI::HTTPError => the_error
puts ‘##########Post Response Error: ‘ + the_error.io.status[0]

Actual error recovery is left as an exercise for the reader.

 

OK, we now have data in our database.  In our case this is is 2,014 documents (i.e. 2,014 separate Grateful Dead concerts).  Using Futon we see the database as:

clip_image001

In Futon, we can look at our data:

clip_image001[7]

And we can drill in on an individual document (for example the primordial recording on 1965-11-01):

clip_image001[9]

 

Of course what we need to able to return our document with a simple HTTP GET request.  Further we will want to return only those fields we will actually need from a document (i.e. we need to shape our data to minimize bandwidth).  We do that in couchdb with a view which is defined in a design document.  We can define multiple views in a single design document and can have multiple design documents in a given database. 

For our first view lets set the task to return all documents which occurs on a given month and day (regardless of year).  This is the filter. We want to shape our data to return only a subset of fields: the venue, the IAKey and the document tracks.  The key for this view will be the month+day (in MMDD format).  Our design document might look like:

 {
“_id”: “_design/basic”,
“_rev”: “19-fd2c9b34d2536ce1f187ab2d4e5413de”,
“views”: {
“MonthDay”: {
“map”: “function(doc){emit(doc._id.substr(5,2)+doc._id.substr(8,2),[doc.venue , doc.IAKey, doc.tracks ])}”
}
}
}

 

What this view (called MonthDay) does is maps each document (doc) into a key (doc._id.substr(5,2)+doc._id.substr(8,2)) and an array of return fields:

[doc.venue , doc.IAKey, doc.tracks ]

Note that doc.tracks itself returns an array of tracks.

Using Futon to test our work and find the results for all New Years Day Grateful Dead Concerts.  The first document looks like:

 

clip_image001[11]

(additional lines are omitted from this screen snap).

 

As an HTTP Get Verb we would write:

http://127.0.0.1:5984/deadbase/_design/basic/_view/MonthDay?startkey=%2201-01%22

This command users the

URI: http://127.0.0.1:5984/deadbase

The View: _design/basic/_view/MonthDay

startkey: this is a key word parameter field which tells couchdb to go to the b-tree which represents the MonthDay view and finds the first Key match and continues through the tree returning all Key matches.

If we wanted a range of Keys (say all holiday season Concerts) we can use the startkey and the endkey parameters:

http://127.0.0.1:5984/deadbase/_design/basic/_view/MonthDay?startkey=%221224%22&endkey=%221969-12-31%22

startkey” gives us a starting key in the b-tree and returns all documents until (and including) the b-tree key defined by endkey.

 

JSConf-eu-Alexander-Lang--Writing-apps-on-the-edge-with-CouchDB-e8451113

For more details see the Definitive Guide to CouchDB chapter: Finding Your Data With Views.  The key to simple view definition is defining an appropriate lookup key and defining what data to return.  We can define a simple listing view as:

"Jump": { "map": "function(doc){emit(doc._id,'['+doc.venue+'] ' +'http://www.archive.org/details/'+doc.IAKey)}" },
This returns a lookup key equal to the original key used in the database (doc._id) and the data as venue (doc.venue)
and a URL for the concert recording ('http://www.archive.org/details/'+doc.IAKey). Now if we want to return all
concerts in August, 1969 we can issue the GET:
http://127.0.0.1:5984/deadbase/_design/basic/_view/Jump?startkey=%221969-08-00%22&endkey=%221969-08-31%22

This will return the rows as:{“total_rows”:2013,”offset”:156,”rows”:[ {“id”:”1969-08-02″,”key”:”1969-08-02″,”value”:”[Family Dog at the Great Highway] http://www.archive.org/details/gd69-08-02.sbd.miller.30651.sbeok.flacf&#8221;}, {“id”:”1969-08-03″,”key”:”1969-08-03″,”value”:”[Family Dog at the Great Highway] http://www.archive.org/details/gd1969-08-03.sbd.miller.30652.sbeok.flac16&#8243;}, {“id”:”1969-08-16″,”key”:”1969-08-16″,”value”:”[Woodstock Music] http://www.archive.org/details/gd1969-08-16.sbd.gmb.fixed.95918.flac16&#8243;}, {“id”:”1969-08-21″,”key”:”1969-08-21″,”value”:”[Aqua Theater] http://www.archive.org/details/gd1969-08-21.sbd-part.tremblay.1170.sbeok.shnf&#8221;}, {“id”:”1969-08-23″,”key”:”1969-08-23″,”value”:”[Pelletier Farm] http://www.archive.org/details/gd1969-08-23.sbd.lai.6639.shnf&#8221;}, {“id”:”1969-08-28″,”key”:”1969-08-28″,”value”:”[Family Dog at the Great Highway] http://www.archive.org/details/gd69-08-28.sbd.lepley.4234.sbeok.shnf&#8221;}, {“id”:”1969-08-29″,”key”:”1969-08-29″,”value”:”[Family Dog at the Great Highway] http://www.archive.org/details/gd1969-08-29.sbd.lai.9179.sbefail.shnf&#8221;}, {“id”:”1969-08-30″,”key”:”1969-08-30″,”value”:”[Family Dog at the Great Highway]

]}

Get it? Good.

 
6011428946_f722b3cb71
What Would Dweelze Do?

Now let’s return to RUBY and write a web page which will display concert and track data for any given day in Grateful Dead History. Using the view:MonthView as defined above:

“MonthDay”: {
“map”: “function(doc){emit(doc._id.substr(5,2)+doc._id.substr(8,2),[doc.venue , doc.IAKey, doc.tracks ])}”
}

Lets return data using RUBY for concerts for all years for the Month and day of todays Date.  In Ruby we need to define a controller and a erb HTML file to accomplish this.  Lets say our page will be called player.  Our controller might look like:

require ‘rest-open-uri’

require ‘json’

class PlayerController < ApplicationController
def player
mmdd=Date.today.to_s[5..-1]
mmdd=mmdd.gsub(“-“,””)
base=’
http://127.0.0.1:5984/deadbase/
    view=’_design/basic/_view/MonthDay’
url=base+view+’?key=’
url=url+’%22’+mmdd+’%22′
jsonString=returnJSONString(url)
@parm=jsonString
end
def returnJSONString(url)
jsonString=”
open (url) do |x|
x.each_line do |y|
jsonString=jsonString+y
end
end
return jsonString;
end 

end

This returns the JSON returned from couchdb unaltered as a parameter to the erb page for player (player.html.erb).  If we wanted to work with the data within RUBY we would need to change the JSON back into a RUBY format by calling:

JSON.parse(jsonString)

For our purposes, however, we want to pass the data directly to the browser and will process the data using JavaScript this will in almost all cases be faster than processing the JSON back to RUBY and then formatting the data on the erb page.  In our first pass the route map for our page will be simple:

map.connect ‘player’, :controller => ‘player’, :action => ‘player’

and our first pass at an HTML page will look like this:

<!–DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01//EN” “http://www.w3.org/TR/html4/strict.dtd>
<html>
<head>
<meta http-equiv=”Content-Type” content=”text/html; charset=iso-8859-1″ />
<title>Player: On This Day In Grateful Dead History</title>
<script type=”text/javascript”>
gdData=<%=  @parm  %> ;
</script>
</head>
<body>
<h3>On This Day In Grateful Dead History</h3>
</body>
</html>

This page does absolutely NOTHING…  Except gets our JSON data to the Javascript on the browser page.  And that, dear reader is as far as we need to go with today’s post.

5719769521_0380972a03




	

REST, Ruby On Rails, CouchDB and Me – Part 5 Getting The Data Ready for CouchDB   3 comments

Part 0 – REST, Ruby On Rails, CouchDB and Me

Part 1 – Ruby, The Command Line Version

Part 2 – Aptana IDE For Ruby

Part 3 CouchDB Up and Running on Windows

Part 4 – CouchDB, Curl and RUBY

Part 5 – Getting The Data Ready for CouchDB

Part 6 – Getting The Data Into And Out Of CouchDB

Part 7 – JQUERY,JPlayer and HTML5

The Internet Archive and the Grateful Dead

The Internet Archive (IA)  is a 501( c ) 3 non-profit corporation dedicated to disseminating of public domain digital artifacts.  This includes such items as books, videos of imagesall kinds (TV Shows, shorts and feature films) audio recordings of all kinds (musical and spoken word).    One of their most popular projects is a truly huge collection of Grateful Dead concert recordings.  One 418910of the most visited pages of the Internet Archive’s web site is the “Grateful Dead Shows on This Day In History” page, which lists and allows play back of any Grateful Dead concerts they have in their collection of whatever day you visit the site.  Did I say that there IA has a large number of Grateful Dead concert recordings?   The are recordings of around 2,000 separate concert dates.  Any given concert may be represented by multiple recordings from a variety of sources: soundboard, audience recordings using professional and amateur equipment.  Original media ranges from cassette tapes, to 7 inch reel to reel and digital media.  If you are going to be working with the IA Grateful Dead collection please review the FAQ on the collections policy notes as well as the special notes here.

IA uses a very sophisticated data repository of meta data and an advanced query engine to allow retrieving both the meta data and the recordings.  Meta data can be retrieved directly using the “advanced search” engine.  On the day I started this post I visited IA and used the “Grateful Dead Shows on This Day In History”  The query returned data on 8  concerts (and 25 recordings of those 8 concerts).  A partial page image is given below:

image

Clicking on any of these entries moves us to a second screen in order to play the concert recording.  A screen shot of the playback screen looks like this:

image

Looking closer at the second screen we see the music player:

image

Can we design a faster, simpler and better looking interface into the Grateful Dead Archive?  Can couchDB help us? gratefuldead_20070108135140 The the first question will be addressed in a later post. This current post will look at how couchDB can  help us achieve a faster more efficient information system.  IA does a super job of serving up the music files on demand – there is no reason to duplicate their storage system.   However, IA is fairly slow to serve up meta data (such as the results of the “Grateful Dead Shows on This Day In History” query) Abstracting the IA metadata into a CouchDB database will allow us to serve up the meta data much faster than the IA query system.

Getting Data Into CouchDB

Our basic plan for using RUBY to get data from IA and into couchdb consists of:

  1. Prepare a URL query request to get the basic recording meta data (not the track meta data);
  2. Submit A GET request to IA using the URL query;
  3. Parse the XML returned to get at the individual Concert meta data fields;ruby
  4. Select the BEST recording for any given concert (more on this below);
  5. Prepare a URL to request track listing XML file based on the IA Primary Key of the selected concert recording;
  6. Submit a GET request to IA;
  7. Parse the XML returned to get at the individual track meta data fields;
  8. Create a ruby object which can be safely serialized to JSON;
  9. Serialize the object to a JSON string (i.e. a JSON document);
  10. Do a POST request to insert a JSON document for each concert into the couchdb database;
  11. Create couchDB views to allow optimized data retrieval; and
  12. Create a couchDB view to optimize retrieval recordings for all years for an arbitrary Month and Day (this duplicates the data provided by the “Grateful Dead Shows on This Day In History” selection in the Internet Archive.

Note we are not accessing nor storing the actual music files.  Before discussing how this plays out in practice lets define our JSON couchDB document.  We will cover items one through eight in this post.  We turn to items nine through twelve in the next post.

CouchDB Document Schema

CouchDB databases start with documents as the basic unit.  Typically a couchdb based application will have one database holding one or more variant document types.  There will be one or more design documents which provide multiple views, show functions and map functions as necessary to facilitate the application.  We will use a single document which will represent a abstract of the meta data contained in IA for individual recordings ( we are going to select the one ‘best’ recording per concert).  Our couchdb database will hold one document per concert.   The tracks (actually the track meta data will be stored as arrays within the  concert document).  We will populate the couchdb database in a single background session  pulling meta data (NOT THE MUSIC FILES) from IA and we will  include the IA publication date in the document so we can update our database when (if) new recordings are added to IA in the Grateful Dead collection.

Here are the document fields  which we will use:

Field

Notes

Typical Values

_id couchdb primary key.  We will use a natural key: a string representation of the concert date. 1969-07-04
_rev revision number provided by couchDB 1-6ea272d20d7fc80e51c1ba53a5101ac1
IAKey Internet Archive Key for this Recording gd1965-11-01.sbd.bershaw.5417.sbeok.shnf
pubdate Internet Archive Date When Recording was published to the web 2009-03-14
venue Wherethe concert took place Fillmore East Ballroom
description free text describing the concert – provided by the uploader Neal Cassady & The Warlocks 1965 1. Speed Limit studio recording/Prankster production tape circa late 1965
cm boolean – Recording by Charlie MIller  – used to select the ‘best’ recording true or false
sb boolean – Recording was made from a soundboard – used to select the ‘best’ recording true or false
mx boolean – A matrix style recording – used to select the ‘best’ recording true or false
tracks an array of meta data for each track of the recording see below

Each track in the tracks array  formally looks like:

Field Notes Typical value
IAKey The Internet Archive key for this track.  This key is unique within a given recording  (see the IAKey above) gd1965-11-01.sbd.bershaw.5417.sbeok.shnf/Acid4_01_vbr
track track number 02
title the song title Cold Rain and Snow
time the length of the track in minutes and seconds 09:48

Let call everything except the tracks our BASE data and the track data our TRACK data.

We insert documents to the database (using an HTTP post) as JSON so a typical document would look like this in JSON format:

{
“_id”: “1966-07-30”,
“IAKey”: “gd1966-07-30.sbd.GEMS.94631.flac16”,
“pubdate”: “2008-09-22”,
“venue”: “P.N.E. Garden Auditorium”,
“description”: “Set 1 Standing On The Corner I Know You Rider Next Time You See Me”,
“cm”: false,
“sb”: true,
“mx”: false,
“tracks”: [
{
“IAKey”: “gd1966-07-30.sbd.GEMS.94631.flac16/gd1966-07-30.d1t01_vbr”,
“track”: “01”,
“title”: “Standing On The Corner”,
“time”: “03:46”
},
{
“IAKey”: “gd1966-07-30.sbd.GEMS.94631.flac16/gd1966-07-30.d1t02_vbr”,
“track”: “02”,
“title”: “I Know You Rider”,
“time”: “03:18”
},
{
“IAKey”: “gd1966-07-30.sbd.GEMS.94631.flac16/gd1966-07-30.d1t03_vbr”,
“track”: “03”,
“title”: “Next Time You See Me”,
“time”: “04:00”
}
]
}

2654190796_c0a810ec44

Hacking The Internet Archive: Getting Data From IA and Into CouchDB:

Here is the URL to obtain the page for “Grateful Dead Shows on This Day In History”:

http://www.archive.org/search.php?query=collection:GratefulDead%20date:19??-08-04&sort=-/metadata/date

This is a simple GET request with a query for the IA “collection” of Grateful Dead items filtered on the date string: 19??-08-04 and sorted descending by the concert date.  This get returns an HTML page.  This type of interface is known as an HTTP RPC interface.  RPC (Remote Procedure Call) interfaces are not pure REST interfaces but they are somewhat RESTful inthat they allow us to make a data request using a late bound, loosely coupled HTTP call.  See here and here for more theoretic background on RPC calls.  IA provides an  “Advanced Search” function will allow us to return data for an arbitrarily complex query in one of several different data formats other than HTML.  We selected XML as the format  for our work here.  XML is the traditional format for HTTP RPC but other formats may be better for certain applications.  Unfortunaely IA does not directly document the format of the RPC data request but they do provide a QEB page to build the request.  The page looks like this:

image

Using this screen we can compose a HTTP RPC request which will mimic the URL produced by “Grateful Dead Shows on This Day In History” and with a little brain effort and experimentation we can  understand how to compose requests without using the QBE screen.  By feeding the RPC request query back into advanced search and selecting XML as an output format as shown here:

clip_image001

we produce both an example of the HTTP RPC request which will return our desired data in our desired format.  Thus we generate a HTMLEncoded RPC request like:

@uri=”http://www.archive.org/advancedsearch.php?q=collection%3AGratefulDead+date%3A#{_dateString}&fl%5B%5D=avg_rating&fl%5B%5D=date&fl%5B%5D=description&fl%5B%5D=downloads&fl%5B%5D=format&fl%5B%5D=identifier&fl%5B%5D=publicdate&fl%5B%5D=subject&fl%5B%5D=title&sort%5B%5D=date+desc&sort%5B%5D=&sort%5B%5D=&rows=2000&page=1&callback=callback&output=xml”

where we replace #{_dateString} with a date string like 19??-08-08.  Of course to one years worth of data we could use a data string like: 1968-??-??.  It is a simple extension of the query languge to replace the singular date request: date%3A#{_dateString} with a date range.

which returns Grateful Dead recording data for all years of the last century which were recorded on 08-08.  The XML output returned to the caller looks like:

clip_image001[4]

In a more graphic format the output looks like:

clip_image001[6]

Within Ruby we will need to make the HTTP Get request with a desired date range, transform the body of the return request into an XML document and use XPATH to parse the XML and retrieve the meta data values for each recording (see below).  The is NOTHING inherently wrong with this RPC interface.  It is flexible and allows us to select only the data fields we are interested in and return data only for the dates we wish.  Since RUBY supports neither native consumption of JSON nor XML. So the XML format of the data is as good as any other and numerous tools exist in RUBY to manipulate XML data.  I which RUBY had a more native interface for JSON but it does not.

At this point, we do not have meta-data about individual tracks in a given recording.  It turns out that we can get this data but not through an HTTP RPC request.  It turns our, dear reader, that if we have the IAKey for the recording we can obtain an xml file with track meta data by making the following call:

http://www.archive.org/download/{IAKEY}/{IAKEY}_files.xml.

This file contains assorted XML data, it varies by what formats IA makes available the individual tracks via a 309 (HTTP redirect).  This is not an RPC call so we are far from a RESTful interface here.  We do not have control over the fields or range of the data included in this call.  It is all or nothing.  But at least the XML format is simple to mainipulate.  With the IAKey in hand for an individual recording and making some reasonable guesses we can parse the XML file of track data and compose the TRACKS array for our couchDB document using XPATH. A single entry for the high bit rate mp3 track recording looks like:

<file name=”gd89-08-04d2t01_vbr.mp3″ source=”derivative”>
<creator>Grateful Dead</creator>
<title>Tuning</title>
<album>1989-08-04 – Cal Expo Amphitheatre</album>
<track>13</track>
<bitrate>194</bitrate>
<length>00:32</length>
<format>VBR MP3</format>
<original>gd89-08-04d2t01.flac</original>
<md5>91723df9ad8926180b855a0557fd9871</md5>
<mtime>1210562971</mtime>
<size>794943</size>
<crc32>2fb41312</crc32>
<sha1>80a1a78f909fedf2221fb281a11f89a250717e1d</sha1>
</file>

Note that we have the IAKey for the track (gd89-08-04d2t01 ) as part of the name attribute.

Garcia

Using a background Ruby Process to Read the Data

The following RUBY GEMS are required to complete this step:

rest-open-uri : This GEM extends open-uri to support POST, PUT and DELTE HTTP command

json : This GEM handles serialization and de-serialization of a limited subset of RUBY into JSON strings.

From the standard RUBY library we will also be using

rexml : This GEM creates XML documents from XML Strings and supports XPATH which we will use to read the XML documents from IA

Our first step is to extract the get the the data via HTTP and parse the XML file returned to find individual recordings.  There are  (in most cases) be multiple recordings per concert (per date) and we want to retain for the database only the “best”.

In pseudo Ruby code:

require ‘rest-open-uri’

require ‘rexml/document’

 def initialize(_dateString)

#HTTP GET, create a string of the response body and transform the string into an XML node tree
#mind the screen wrap and html Encoding:
@uri=http://www.archive.org/advancedsearch.php?q=collection%3AGratefulDead+date%3A#{_dateString}&fl%5B%5D=avg_rating&fl%5B%5D=date&fl%5B%5D=description&fl%5B%5D=downloads&fl%5B%5D=format&fl%5B%5D=identifier&fl%5B%5D=publicdate&fl%5B%5D=subject&fl%5B%5D=title&sort%5B%5D=date+desc&sort%5B%5D=&sort%5B%5D=&rows=2000&page=1&callback=callback&output=xml

  xmlString=”
open (@uri) do |x|       #build a representation of the response body as a string
x.each_line do |y|
xmlString=xmlString+y
end
if xmlString==”
puts ‘No String Returned From Internet Archive’
quit
end
end
@IAXMLDocument= REXML::Document.new(xmlString)  #turn the string into an XML document
end #open

Now we need  to loop through the XML document and pull out each ‘doc’ section using XPATH and read each doc section for the meta data for that recording.

#use XPATH and find each response/result/doc node and yield

def get_recordings(document)

document.elements.each(‘response/result/doc’)do |doc|

yield doc
end
end

#get the XML document and yield

def get_record(xmldoc)

get_recordings(xmldoc) do |doc|
yield doc
end
end

#general purpose XPATH method to extract element.text (the metadata values) for arbitrary XPATH expressions

def extract_ElmText(doc,xpath)

doc.elements.each(xpath) { |element|  return element.text }
end

def worker(xmldoc)

#main loop

_docCount=0

get_recordings(xmldoc) do |doc|
_docCount+=1
_date=extract_ElmText(doc,’date[@name=”date”]’)[0..9]
_description=extract_ElmText(doc,”str[@name=’description’]”)
_title=extract_ElmText(doc,”str[@name=’title’]”)
_title=clean_Title(_title)
_keylist=pull_keys(doc)
_pubdate=extract_ElmText(doc,’date[@name=”publicdate”]’)[0..9]  #there is a bug here , corrected by lines below

if (_pubdate.length==0)
_pubdate=’1999-01-01′
puts “#No Publication Date: {_date} #{_title}”
end
_uri=extract_ElmText(doc,’str[@name=”identifier”]’)

#make a RUBY class object to hold one recording
_record=GDRecording.new _date, _description, _tracklist, _title, _keylist, _pubdate,_uri

#save the recording class objects in an array

@list[@list.count]=_record

end

In this code the ‘worker’ method calls the helper methods to:

0) Do the HTTP  get to make the RPC request and read the response body one line at a time and

1) transform the lines into a single string and convert (REXML::Document.new) the string into an XML document for processing by XPATH

2) loop through the doc nodes of the xml tree and extract the values of the  meta data fields

3) the meta data values are passed to an RUBY class ( GDRecording) which holds this meta data for later processing,

4 finally we temporarily store the recordings in an array for the next processing step.

Note that these routines work  whether the query returns a single day (with multiple recordings) or multiple days or even the whole dataset!  What is essencial is that we process the file as N ‘doc’ sub trees (which represent single recordings) and have recording date (at least) to group our data and extract the ‘best’ recording within each date group.

Our next step will be group the recordings by day (i.e. concert) and provide our own filter to select a single ‘best’ recording for each concert.

Shake and Bake:  Finding A ‘Best’ Recording.

best

What is the best Grateful Dead concert.  Why the first one I went to of course.  Just ask any Deadhead and you will probably get the same answer.  But what is the best recording of any given GD concert? My approach is very simple.

  • Most recent posted recordings are better than older recordings. (least important criteria)
  • Soundboard recordings are better than audience recordings.
  • Matrix recordings are even better.
  • Recordings mixed by Charlie Miller are best of all. (most important criteria)

Well these are MY criteria.  What ever criteria as long as they are hieratical  you can code the select in a very trivial manner.  If we have a field in each recording for the concert date and a field for each selection criteria (we derive these from the keywords field in IA) we sort the recordings by date and then by each of the criteria from most important (Charlie Miller in may case) to least important (date posted) and then select the first recording in sort order within each date group. On Ruby the sort of our list of recordings is trivial to code and easy to maniuplate (add new criteria or change the priority of criteria). The sort statement looks like this:

@list.sort! { |a,b| (a.date+a.cm.to_s+a.sb.to_s+a.mx.to_s+a.pubdate ) <=> (b.date+b.cm.to_s+b.sb.to_s+b.mx.to_s+b.pubdate )   }

Once sorted we create a list of best recordings as:

def newSelect
_dateGroup=nil
_list=Array.new
if  @list==nil or @list.count==0
puts ‘No Recordings.’
return;
end
foreach do |rec|
if _dateGroup!=rec.date
if dateGroup!=nil
@selectList[@selectList.count]=list[0]
end
_dateGroup=rec.date
_list=Array.new
end
_list[_list.count]=rec
end
if dateGroup!=nil
@selectList[@selectList.count]=list[0]
end
end

Note that is code is not only simple but it is independent of the selection criteria we are using.

Now that we have a list of recordings we are interested in,  we can get the XML file of track meta data using the IAKey discussed above and making a simple GET call and parsing the XML file for the meta data for each.  Much of the code used duplicates the XML code presented above so we need not reproduce all the code except to show a short section which uses a slightly different XML XPATH syntax:

open (filesURI) do |x| x.each_line do |y| xmlString=xmlString+y end end
REXML::Document.new(xmlString).elements.each(‘files’) do |doc|

doc.elements.each(‘file’) {|file_elm|
obj=nil
title=nil
trackString=nil
lengthString=nil
obj=file_elm.attributes[“name”]
file_elm.elements.each(‘title’) { |element| title=element.text }
file_elm.elements.each(‘track’) { |element| trackString=element.text}
file_elm.elements.each(‘length’) { |element| lengthString=element.text}

{omitted code}

end

Okay now we have a (hash) list of recording meta data,  each item of which contains a (hash) list of track meta data for that recording.  In our next post we will leave this unRestful world behind and move into the RESTful world of couchDB when we:

  • Serialize the object to a JSON string (i.e. a JSON document);
  • Do POST requests to insert  a JSON document for each concert into the couchdb database;
  • Create couchDB views to allow optimized data retrieval; and
  • Create a couchDB view to optimize retrieval recordings for all years for an arbitrary Month and Day (this duplicates the data provided by the “Grateful Dead Shows on This Day In History” selection in the Internet Archive.

cat on fancy couch

REST, Ruby On Rails, CouchDB and Me – Part 0   4 comments

Part 0 – This Post

Part 1 – Ruby, The Command Line Version

Part 2 – Aptana IDE For Ruby

Part 3 – CouchDB Up and Running on Windows

Part 4 – CouchDB, Curl and RUBY

Part 5 – Getting The Data Ready for CouchDB

Part 7 – JQUERY,JPlayer and HTML5

The is the first in asmall series of blog posts concerning what is often known as as Open Source Stack for Web Development.  My interests here are in WEB 2.0 development.  We will be working towards using JQUERY and CouchDB to develop REST(ful) Web sites.  I will start with a focus on Rudy and Ruby on Rails (aka RadRails).   However, I will be using free and open source products not on UNIX but on Windows based systems.  Windows runs on fairly cheap hardware (compared to Apples UNIX based platforms) and Windows is much more accessible to new developers than Linux systems.  I will be working here with Windows XP and Windows Vista but you can play alone on Linux and MAC x86.  To get started let’s get together some software and supporting documentation and begin.

Ruby

Version 1.8.7 is the current stable release but since we are interested in using CouchDB I have installed the required ‘edge’ version 1.9.2.  Either of these version will support Ruby on Rails development. Ruby installations are available from several sites. Until you know exactly what you are doing stick with distributions from RubyInstaller.org since these are the baseline Ruby builds which everyone tests against first.  As you get deeper into it (or if  things get a little more sticky)  you try an integrated open stack like the one offered by BitNami.  At some point you may need to install the DevKit.  This is not as heavy as it sounds (but you will need to work with your system paths to get everything in place).  You can get the DevKit here and installation instructions are here.   Note:  the devkit download is now a Windows MSI file but the instructions are still based on a ZIP file extraction, but it all works trust me.  Don’t install either the BitNami Stack or the DevKit until you know you need it.   If you completely new to Ruby a nice introduction can be found in Why’s Poignant Guide To Ruby.

Ruby On Rails

There are a lot of different options to developing Ruby on Rails applications.  If I was not an IDE freak I could use  a simple text editor (I often use the non-open Primal Script) but being new to Ruby on Rails I wanted the support of and IDE.  For this option I selected the Aptana RadRails Development Environment.  This free development environment is based on the open Eclipse editor.  I downloaded the base Aptana IDE (Studio 2) and then added RADRails as a plug in.  These are available in Linux and Mac x86 installers in addition to windows.  You could install only RADRails but then you would have a crippled version of the Aptana product.  We will be noting Ruby on Rails training materials as we move along.

CouchDB

Although we will be using mySQL or SQLlite for some of our development, our real content manager will be the NOSQL couchDB.  This is our real goal in the project – testing the viability  of a REST(ful) HTTP addressable NOSQL database.  This project is from the Apache organization and is available for Linux, MAC x86 and Windows.  It runs as a service on the Windows OS.  The Windows installer is available here.  There is an excellent open source on line couchDB book available here.  For Ruby on Rails we will be using – CouchRest for our Ruby on Rails work with couchdb.  CouchRest is available as a ruby GEM and can be installed on Linux, MAC x86 and Windows versions of Ruby.

JQUERY

Web 2.0 is not possible with out modern, sophisticated JavaScript libraries.  One of the best of these is JQUERY and not surprisingly couchDB ships with a powerful jquery library (couchdb.js) to facilitate browser side manipulation of couchdb data.  For browser work you should be pulling your JQuery from CDN Host.  For use within Ruby for Rails project you will need to add JQUERY  support to your Aptana IDE.  JQuery works with all modern browsers (and even some which not).PacalII

%d bloggers like this: