Let me say write off that I do not pay for my own ticket to QCON, my boss picks up the tag. I love QCON. It is definitely not MIX. I go there to see what is happening in the world which 
is NOT Oracle and Not Microsoft. That’s the same reason I read their online Zine: InfoQ. QCon always provides a look at what is current and recent in the open stack world. This year we looked closely at REST, Mobile development, Web API and NOSQL. As they did last year QCON provides a nice look at what is open and emerging. Big metal with always be with us but the desk top is looking
very weak during the next few years while Mobile devices of all kinds and makers are exploding. The biggest fall out is that while HTML5 is only slowly emerging on desktops in place, all new Mobile devices (which is to say most new systems) will be fully HTML5 compliant. Not only that but with the exception of Windows Phones, the rendering engine for all mobile devices is based on WebKit. What this mean for those of us in the cubes is that worrying about how to bridge to pre-HTML5 browsers with HTML5 code is a non-issue. Mobile development is HTML5 development. The big metal end of the supply chain is being segmented into Web API servers (which service JSON XHR2 data calls) and the NOSQL engines which serve the WEB API farms. Remember a native mobile app ideally has pre-loaded all of its pages its interactions are solely over JSON XHR2 for data (be it documents, data or HTML fragments). The traditional JSP or ASPX web server is not really in play with native mobile apps and has and increasingly small role to play in “native like” or browser based mobile apps. Let’s move on.
“IPad Light by cloud2013”
Speaking of moving on: There is an occupation going on in this country. I visited occupations sites in San Francisco, UCal Berkeley and Berkeley “Tent City”. These are all very active and inspiring occupy sites. Now if we can only get to Occupy Silicon Valley!
I attended the REST in Practice tutorial this year and it was a very nice. The authors were well informed and the agenda comprehensive. I personally like the Richardson maturity model but think that people are not facing up to the fact that level three is rarely achieved in practice and the rules of web semantics necessary to interoperate at level 3 are almost non-existent. Remember the original REST model is client/server. The basic model is a finite state machine and the browser (and the user) are in this model required to be dumb as fish. Whether Javascript is a strong enough model and late binding semantics can be made clear enough to pull off level three is really an open question which no one has an answer to. If we forget about interoperability (except for OAuth) things start to fall into place but we thought OPENNESS was important to REST.
Workshop: REST In Practice by the Authors: Ian Robinson & Jim Webber
Why REST? The claims:
· Scalable
· Fault Tolerant
· Recoverable
· Secure
· Loosely coupled
Questions / Comment:
Do we agree with these goals?
Does REST achieve them?
Are there other ways to achieve the same goals?
REST design is important for serving AJAX requests and AJAX requests are becoming central to Mobile device development, as opposed to intra-corporate communication. See Web API section below.
Occupy Market Street (San Francisco)
The new basic Document for REST: Richardson Maturity Model (with DLR modifications)
Level 0:
One URI endpoint
One HTTP method [Get]
SOAP, RPC
Level 1:
Multiple URI,
One HTTP Method [Get]
Century Level HTTP Codes (200,300,400,500)
Level 2:
Multiple URI,
Multiple HTTP Methods
Fine Grain HTTP Codes (“Any code below 500 is not an error, it’s an event”)
URI Templates
Media Format Negotiation (Accept request-header)
Headers become major players in the interaction between client and server
Level 3: The Semantic Web
Level 2 plus
Links and Forms Tags (Hypermedia as the engine of state)
Plus emergent semantics
<shop xmlns=”http://schemas.restbucks.com/shop”
xmlns:rb=”http://relations.restbucks.com/”>
<items>
<item>…</item>
<item>…</item>
</items>
<link rel=”self” href=http://restbucks.com/quotes/1234 type=”application/restbucks+xml”/>
<link rel=”rb:order-form” href=”http://restbucks.com/order-forms/1234″ type=”application/restbucks+xml”/>
</shop>
Think of the browser (user) as a finite State Machine where the workflow is driven by link tags which direct the client as to which states it may transition to and the URI associated with each state transition.
The classic design paper on applied REST architecture is here: How To GET a Cup Of Coffee. Moving beyond level 1 requires fine grain usage of HTTP Status Codes, Link tags, the change headers and media type negotiation. Media formats beyond POX and JSON are required to use level 3 efficiently (OData and ATOM.PUB for example).
Dude, where’s my two phase commit? Not supported directly, use the change headers (if-modified, if-non-match, etag headers) or architectural redesign (redefine resources or workflow). Strategic choice is design of the finite state machine and defining resource granularity.
(Slide from Rest in Practice)
Architectural Choices:
The Bad Old Days: One resource many, many ‘verbs’.
The Happy Future: Many, many resources, few verbs.
The Hand Cuff Era: Few Resources, Few verbs.
The Greater Verbs:
GET: Retrieve a representation of a resource
POST: Create a new resource (Server sets the key)
PUT: Create new resource (Client sets the key); ( or Update an existing resource ?)
DELETE: Delete an existing resource
Comment: The proper use of PUT vs. POST is still subject to controversy and indicates (to me) that level 3 is still not well defined.
Typically they say POST to create a blog entry and PUT at append a comment to a blog. In Couchdb we POST to create a document and PUT to add a revision (not a delta) and get back a new version number. The difference here is how the resource is being defined, which is an architectural choice.
The Lesser Verbs:
OPTIONS: See which verbs a resource understands
HEAD: Return only the header (no response body)
PATCH: Does not exist in HTML5. This would be a delta Verb but no one could agree on a specification for the content. Microsoft did some early work on this with their XML Diffgram but no one else followed suit.
Security
Authentication (in order of increased security)
Basic Auth
Basic Auth + SSL
Digest
WSSE Authentication (ATOM uses this)
Message Security:
Message Level Encrypt (WS-SEC)
For the Microsoft coders I highly recommend
RESTful .Net (WCF For REST (Framework 3.5) Jon Flanders
There are significant advantages to building your RESTful services using .Net. Here is a comparison table to get you oriented:
DLR’s Cross Reference:
|
|
|
|
|
Web Service Standard |
REST Service |
WCF For REST (Framework 3.5) |
| 1 |
TCP/IP + others |
TCP/IP |
TCP/IP |
| 2 |
SOAP Wrapper |
HTTP |
HTTP |
| 3 |
SOAP Headers |
HTTP Headers |
HTTP Headers |
| 4 |
WS*Security |
Basic Auth/SSL |
Basic Auth/SSL or WS*Security |
| 5 |
Early Binding |
Late Binding |
Late Binding |
| 6 |
XSD |
WADL |
XSD, WADL |
| 7 |
XML |
Media Negotiation |
Media Negotiation |
| 8 |
SOAP FAULTS |
HTTP Response Codes |
HTTP Response Codes |
| 9 |
Single Endpoint |
Multiple Endpoints, URI Templates |
Multiple Endpoints, URI Templates |
| 10 |
Client Proxy |
Custom |
auto-generated Javascript proxy |
The REST of the Week
Wednesday is more or less vendor day at QCON and the sessions are a step down from the tutorials but the session quality
picked up again on Thursday and Friday. XXX XXXX who gave an excellent tutorial last year gave an informative talk on ‘good code’. The Mobile Development and HTML5 tracks were well attended and quite informative. The fie ld is wild open with many supporting systems being free to the developer (support will cost you extra) and the choices are broad: from browser ‘responsive design’ application to native appearing applications to native apps ( and someone threw in “hybrid app” into the mix). The Mobile panel of IBM DOJO, JQuery.Mobil and Sencha was hot. I am new (to say the least) to Mobile development but here are my (somewhat) random notes on these sessions:
MOBILE Development is HTML5 Development
HTML5 is the stack. Phone and Tablet applications use WebKit based rendering engines and HTML5 conformant browsers only (Windows Phone 7 is the exception here). HTML5 has its own new security concerns ( New Security Concerns)
Three major application development approaches are:
· Browser Applications;
· Native like Applications;
· Hybrid Applications; and
· Native Applications.
Browser applications may emulate the screens seen on the parallel desk top browser versions on the front end but in practice the major players (Facebook, YouTube, Gmail) make substantial modifications to at least the non-visual parts of the Mobile experience making extensive use of local storage and the HTML5 manifest standard for performance and to allow for a reasonable off line experience. Browser applications fall under the guidelines of Responsive Design (aka adaptive Design) and tend to be used when content will appear similarly between desktop and Mobile devices.
“Native like” applications use:
· The Browser in full screen Mode with no browser ‘chrome’; and
· Widgets are created using CSS, JS and HTML5 which simulate the ‘look and feel’ of a native application;
· No Access to Native Functionality (GPS, Camera, etc)
· Tend to use, but does not require use of HTML5 manifest or local storage but it is strongly encouraged. 
A Native application is still an HTML5 application with the following characteristics:
· All JS Libraries, CSS and HTML are packaged and pre-loaded using a vendor specific MSI/Setup package;
· AJAX type calls for data are allowed;
· Access to Native Widgets and/or Widgets are created using CSS, JS and HTML5
· Access to Native Functionality (GPS, Camera, etc)
· Standard HTTP GET or POST are NOT allowed
A Hybrid Application is a “Native Like” Application” placed within a wrapper which allows access to device hardware and software (like the camera) via a special JavaScript interface and, with additional special coding, can be packaged within a MSI/Setup and distributed as a pure Native application.
AJAX calls are made via XHR2 (aka XMLHttpRequest Level 2) which among other things relaxes the single domain requirement of XHR and processing Blob and File interfaces.
The following major vendors offer free libraries and IDE for development:
Native Apps: PhoneGap, Appcelerator
Native App Like: Sencha, PhoneGap, IBM Dojo
Browser App: JQuery.Mobile
PhoneGap does NOT require replacement of Sencha, JQuery.Mobil, Dojo.Mobile JQuery libraries.
PhoneGap allows JavaScript to call PhoneGap JavaScript libraries which abstract access to device hardware (camera, GPS, etc).
Sencha does not require replacement of the JQuery.Mobil, Dojo.Mobile JQuery libraries.
Although it is theoretically possible to create “Native like” applications with only JQuery.Mobile this is NOT encouraged.
Local Storage
This is a major area of performance efforts and is still very much open in terms of how best to approach the problem:
The major elements are:
App Cache (for pre-fetch. and Native App Approach)
DOM Storage (aka Web Storage)
IndexedDB (vs. Web SQL)
File API (this is really part of XHR2)
Storing Large Amounts of Data Locally
If you are looking to store many Megabytes – or more, beware that there are limits in place, which are handled in different ways depending on the browser and the particular API we’re talking about. In most cases, there is a magic number of 5MB. For Application Cache and the various offline stores, there will be no problem if your domain stores under 5MB. When you go above that, various things can happen: (a) it won’t work; (b) the browser will request the user for more space; (c) the browser will check for special configuration (as with the “unlimited_storage” permission in the Chrome extension manifest).
IndexedDB:
Web SQL Database is a web page API for storing data in databases that can be queried using a variant of SQL.
Storage Non-Support as of two weeks ago.
|
IE |
Chrome |
Safari |
Firefox |
iOS |
BBX[RIM] |
Android |
| IndexedDB |
Supported |
Supported |
No Support |
Supported |
No Support |
No Support |
No Support |
| WEB SQL |
No Support |
Supported |
Supported |
No Support |
Supported |
Supported |
Supported |
Doing HTML5 on non-HTML5 Browsers: If you are doing responsive design and need to work with Desktop and
Mobil using the same code base: JQuery.Mobile, DOJO and , Modernizr(strong Microsoft support for this JavaScript library).
WEB API
What is it? Just a name for breaking out the AJAX servers from the web server. This is an expansion of REST into just serving data for XHR. It is a helpful way to specialize our design discussions by separating serving pages (with MVC or whatever) from serving data calls from the web page. Except for security the two can be architecturally separated.
Web APIs Technology Stack
![clip_image002[6]](http://dlr2008.files.wordpress.com/2011/12/clip_image0026.gif?w=498&h=261 "clip_image002[6]")
Look familiarr? Looks like our old web server stack to me.
NOSQL
The CAP Theorem (and Here)
- Consistency: (all nodes have the same data at the same time)
- Availability: (every request receives a response – no timeouts, offline)
- Partition tolerance: (the system continues to operate despite arbitrary message loss)
Pick Any Two
If some of the data you are serving can tolerate Eventual Consistency then NOSQL is much faster.
If you need two phase commit, either use a SQL database OR redefine your resource to eliminate the need for the 2Phase Commit.
NoSQL databases come in two basic flavors:
Key/Value: This are popular with content management and where response time must be minimal. In general you define what btrees you want to use before the fact. There are no on the fly Joins or projects. MongoDB and CouchDB are typical leaders in this area.
Column Map: This is what Google calls Big Table. This is better for delivering groups of records based on criteria which may be defined ‘on the fly’. Cassandra is the leader in this group.
Web Sockets:
Sad to say this is still not standardized and preliminary support libraries are still a little rough. Things do not seem to have moved along much since the Microsoft sessions I attended at MIX 11.
Photos: All Photos by Cloud2013
Like this:
Like Loading...
if mercy’s in business
i wish it for you:
more than just ashes
when your dreams come true.
Dead Heads in the Twenty First Century
Back in the old daze, Dead Heads traded concert tapes (a primitive early non-digital recording technology). Some of these were recorded by fans and others where sound board recordings made by the Dead organization which found their way into general circulation (i.e. stolen). Eventually, the Dead organization began to publish selected concert tapes as LP records (another primitive non-digital recording technology) and later as CDs and internet downloads. For more information on the history of the Dead recordings see this article. Online you can buy recordings of Dead concerts at the official Dead $ite. Recording information is also available ( and in a more organized format) here. But what if: you don’t like the concerts released by the Dead Organization or if you just want MORE.
The Internet Archive
There is a unique organization and web site called The Internet Archive. The
Internet Archive isa truly vast catalog of public domain audio and video recording which are in the public domain. There is no internet collection which is larger. Their slogan is simply: Universal Access To All Knowledge. The Dead organization has released a vast number of Dead concert recordings to The Internet Archive, to which are added the large number of fan recordings. In all there are over 7,000 Grateful Dead Recordings in their collection. Let me write that again, 7,000 concerts recordings. Fan recordings are often available for direct download and most sound board recording are available in streaming (only) format. But see my note at the end of this post for a resent development. The only exception is that concerts which were the basis for Grateful Dead official releases or which concerts which have been commercially released on Dead.Net. These are not available on the archive. Sound quality ranges from ok to GREAT, mono to full stereo. The search facilities of the archive are well thought out. Search by song, venue and/or date. The only problem is that there are so many recordings over such a long period (1968 – 1995) that you may not know where to begin. You could focus on a particular year (when did you first attend a Dead concert) or a favorite song. The Internet archive does provide some help with a “Recently Reviewed Concert” link; and a “Dead Concerts on this Day in History” link. You can also look for postings by Chris Miller and “Matrix” or the key word: soundboard.
What’s on my IPod Right Now
Well, let me by honest here: I am obsessed with the song sequence:
Scarlet Begonias followed by Fire On The Mountain
The Dead played this combination over 250 times between 1977 and 1994. This sequence runs from 20 to 30 minuets in length (tending towards the high end in the 1990′s). From my samples the best tours for this sequence are 1977, 1978, and 1989-1992. There are some bad versions (typically during the 1982-1988 tours and the 1977 tour when Donna signs off key many nights!). Here is my very subjective list of good to great versions based on what I have listened to so far.
The Offical Releases:
Dick’s Picks #18 (Jerry and both Godchauxs are on this night).
Dick’s Picks #13 (This is a hidden track. Put on Disc 2, Track 4 “Saint of Circumstance” at the end of this track is 20 seconds of dead air after that comes a November 1, 1979 version of Scarlet Bagonias -> Fire on the Mountain. The rest of Dick’s Picks #13 was based on records made May 6, 1981. This is a kick ass version and is my reference track for the best version of this particular sequence).
Dick’s Pick #06 (skip this one, slow and lack luster)
From the Internet Archives (these are all streaming only version but see my notes at the end of this blog on how to make a local copy:
Great Versions:
1977-05-08
1988-03-31
1989-08-06
1990-03-22
1991-06-25
1992-05-21
Not So Great but worth a listen:
1978-04-11
1978-04-24
1978-09-02
1981-02-27
1981-10-21
1983-10-21
1988-03-31
1990-03-22
1990-10-19
1994-10-14
1994-06-19
Making Local Copies Of Streaming Media
There are several different free or inexpensive programs which will grab the audio stream and save it to an mp3 (or other) format. I have had good luck with Replay Music 3 ($19.95) combined with the Google Chrome Browser. Replay Music will allow saving to WAV format in addition to MP3. The downside of this approach is, of course, that this is real time recording only.
There is a Better Way
If you run the Chrome Browser and access the Internet Arhive, when you go to a stream only Dead concert the screen will look something like these before and after pictures:
With the Chrome (and maybe Safari) browser, Internet Archive will offer you the ability to convert to the HTML5 Audio tag in place of the embedded browser (which uses the Object tag). Say YES to this offer. Having done so follow these steps to allow SAVING of the audio files directly WITHOUT streaming.
- Start playing a track you wish to save
- Right click on the time slider bar at the top of the music player,
- select “Save Audio file”
- select a save location
- click ok
That’s it. Here is what the right click drop down list looks like:
Thank you HTML5 Audio Tag. Jerry would be proud.
Remember you must start playing a track for this technique to work.
Question:
Does the Internet Archive know that using the Audio tag defeats the intend of streaming media only?
Will they plug the hole?
Can they plug the hole or is this basic to the nature of the HTML5 audio tag?
Save now while you can!!
Like this:
Like Loading...
is run by the same folks you put out the web magazine InfoQ. For my money (and its not) in the past QCon schedule is a little long on Agile and Java and not strong enough on JavaScript libraries. The 2012 tracks rectify this problem somewhat. What QCon is strongest is for its embrace of open and free (or almost free) technologies to power the web next year. This years tracks include a reprise of “Cross Platform Mobile”, “Programmable Web”, “No SQL” (on all three days) and a bevy of tracks on hard core web development ( “Real Time Web” , “UX”, “Taming HTML5 and JavaScript” and “Dynamic Languages for the Web” plus the usual cast of characters: Java, Agile and what looks like a strong three day track of end to end “solutions”. Strangely JQuery is not featured anywhere in this conference.
that’s the basic facts, now the question is: Should YOU attend QCon 2012? The best part of QCon is that it is not a standard vendor
conference and (except for Agile) nobody is trying to sell you anything. The attendee’s tend to be working programmers from startups in the SF area and from Europe which helps the sessions to be very focused on what really works. for programmers and startup firms. There is no focus on mega systems like Oracle products or SharePoint. In this environment, couchdb (and this years bad boy Neo) are about as establishment as you get. QCon has traditionally been very friendly towards NO SQL databases and has been I consistent good sorce for information on this topic. This year promises to continue in this trend. I regularly avoid pure vendor sessions so of the three days of regular conference sessions I can look forward to about 2.5 days of good sessions. The tutorials that I have attended are top notch and honest. I can not evaluate the daily keynote speeches since I am normally sleep in past these. Attendees have told me the keynote party and mixer is nice but I am always in the Haight district when this happens on Wednesday night. If the boss is paying and you are coding the web and web devices you must attend this conference. If it’s your dime….
![clip_image002[6]](\Documents%20and%20Settings\dredfield\Local%20Settings\Temp\WindowsLiveWriter-429641856\supfiles14B27BFC\clip_image001%5b7%5d.png "clip_image002[6]")
