=>

We interrupt this series to take a side trip concerning application logging.  The series begins here. NLog is an excellent open source logging project available from NuGet and other sources.   The sample code for this blog post can be found HERE. Although this is a kitchen sink implementation (Log to files, event logs, database, SMTP whatever) I will be using it as a simple way to log text information to files.  Once you have created a Visual Studio Project open Tools / NuGet Package  Manager/Package Manager Console.  From Here you can add NLog to your object with the command:

PM> Install-Package NLog

This will install NLog, modify your project and add a project reference for NLog.  Although NLog targets and rules can be managed programmatically, I 

normally user the configuration file: 

NLog.Config

You can set this up using the Package Manager Console with the command:

PM> Install-Package NLog.Config

Configuration File Setup

The NLog config file is then modified to define “targets” and “rules”.  The former defines where log entries are written and the latter define which log 

levels are written to which targets.  A file based target section might look like:

<targets>

   DLR.Flickr/Debug.txt” archiveNumbering=”Rolling”   archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”/>

<target name=”logfile” xsi:type=”File” layout=”${message}”    fileName=”C:/temp/DLR.Flickr/Info.txt”  archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”/>

<target name=”Errorsfile” xsi:type=”File” layout=”${message}” fileName=”C:/temp/DLR.Flickr/Error.txt” archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”/>

<target name=”Fatalfile” xsi:type=”File” layout=”${message}”  fileName=”C:/temp/DLR.Flickr/Fatal.txt” archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”/>

</targets>

where name is the symbolic name of the target xsi:type defines this as a file target.  If you are controlling the layout of the log entry set layout to “${message}”.  Given that we are using xsi:type as File we can use fileName to set the physical location of the log file.  The value of fileName can be changed programmatically at runtime but I will not give examples here.

NLog defines five Log levels:  Debug, Info, Warn, Error and Fatal.  These levels are defined in an enum and have the names have no special significance except as you define them.  The Rules section of the config file defines which Log Levels are written to which targets. A given level can be written to zero to many targets.  My Rules section typically looks like:

<rules>

<logger name=”*” minlevel=”Debug” maxlevel=”Debug” writeTo=”debugfile” />

<logger name=”*” minlevel=”Info” maxlevel= “Info” writeTo=”logfile” />

<logger name=”*” minlevel=”Warn” maxlevel=”Warn” writeTo=”Warnfile” />

<logger name=”*” minlevel=”Error” maxlevel=”Error” writeTo=”Errorfile” />

<logger name=”*” minlevel=”Fatal” maxlevel=”Fatal” writeTo=”Fatalfile” />
  </rules>

More complex rules like the following are possible:

    <logger name=”*” minlevel=”Error” maxlevel=”Error” writeTo=”Errorfile” />

       <logger name=”*” minlevel=”Error” maxlevel=”Fatal” writeTo=”Fatalfile” />

NLog initialization at runtime is very simple.  Typically you can you a single line like:

using NLog;

static Logger _LogEngine = LogManager.GetLogger(“Log Name”);

this need only be called once.

The simplest NLog log call (given the definition layout=”${message}”  ) would look like:

_LogEngine.Log(NLog.LogLevel.Info, “Info Message”);

We can extend this quite simply.  I have a single class extension providing a simple extension of NLog on Git Hub.  You can find it here.  Specifically I have provided wrapper methods for each NLog.LogLevel and support for Exception Stack Dumps.  Include this file in your project (after installing NLog and NLog config) then you can write:

using DLR.Util;

namespace DLR.CCDB.ConsoleApp

{

    class Program

{

static void Main(string[] args)

{

string _CorrelationID=System.Guid.NewGuid().ToString();

CCDB cbase = new CCDB { CorrelationID = _CorrelationID };

cbase.Client = CouchbaseManager.Instance;

NLS.Info(_CorrelationID, “Helllo, CouchBase”);

try{

throw new ApplicationException(“My Exception”);

}catch(Exception x){

NLS.Error(_CorrelationID,”Error”,x.Message);

//OR

NLS.Error(_CorrelationID,”Error”,x);

}

_CorrelationID is supported here so in multiuser situations (like WebAPI) we can identify which messages where written by which task.  In a console app this is not strictly necessary.  The call to NLS.Info results in an output log line like:

DLR|20140909-152031037|2f8f89ce-51de-4269-9ae0-9313ad2a0243|Helllo, CouchBase|

where:

• DLR is the Log Engine name (more than one engine can write to a given log file);
• 20140909-152031037 is the terse timestamp of the form: YYYYMMDD-HHMMSSmmm; and
• Hello, CouchBase is our text message

My call:

NLS.Error(_CorrelationID,”Error”,x);

would result in a log line like:

DLR|20140909-152544801|46e656cd-4e17-4285-a5f3-e1484dad2995|Error|Error Data. Message: [My Exception]Stack Trace:  DLR.CCDB.ConsoleApp.Program.MainString args|

where Error is my message;

Error Data. Message: [My Exception] is the Message in ApplicationException; and

Stack Trace:  DLR.CCDB.ConsoleApp.Program.MainString args| is the stack dump.

NLS will handle nested exceptions and stack dumps but we are only showing a single un-nested exception in this example.

OK! That’s it for this post.  We will, hopefully return to couchBase and the Flickr API in the next post.

=>

This is the third post in this series on how to harvest statistical data from your (or a friend’s) Flickr Picture View
data.  The series begins here.  Today we are looking at CouchBase as a noSQL database to store our Flickr data.  This post will get as far as getting the shell of a console application up and will defer example code samples for the next blog post.

CouchBase  is a commercialized version of the public domain project Apache CouchDB.  CouchDB is open source and CouchBase is not.  Both support API libraries for .Net and Java.  Commercial development with CouchBase is NOT free.  The CouchDB wiki lists five active C# libraries for CouchDB.  CouchBase supports a many API libraries including .Net and Java.  I have written about CouchDB and Ruby in a prior series of posts which can be found here. Both systems support multi-server nodes and concurrency controls.  Neither of these features will be touched on in the post.  Our focus here will be on an introduction to the minimum necessary administration skills and API coding to help us with our goal of storing information about Users, Photos and View Counts through time.  Along the way we will also discuss JSON Serialization / Deserialization using Newtonsoft.JSON, open source application Logging with NLog.  I will defer the discussion of CouchBase views for a subsequent post.

Data Model Overview.

Ultimately we want to store information about each User. For each user we will store information for one or more Photo and for each Photo, information on one or more View Counts.  Users and Photos have their own Primary Key, supplied as an ID field from Flickr.  Our view counts will be collected each day and the Primary Key of the Counts is represented by the date the view count data was collected.  This could be modeled into a traditional RDBMS in third normal form, but this pattern is also most naturally represented as a nesting of lists of objects within container objects.  Rather than say we are storing nested objects it is more typical today to say that this data can be thought of as a structured Document.  The most natural way to store and access this data is by simple (or compounds of ) primary keys.  When we get to the point where we are doing manipulation of the data for statistical analysis and summary our most natural mode of access will be by a key composed of the User ID and Photo ID and iterating there view counts by Date ID (or Date ID range).  A very simple way to model this is with a Key / Value noSQL database based on document storage (aka a Document Store).  We could call this design an object oriented database model but that would be old fashion.  Here is the visual of the data model:

The full Document could be represented as a compound C# object:

   class CObject
{
public CUser User { get; set; }
public List<CPhoto> Photo { get; set; }
}

public class CUser
{
public string FullName { get; set; }
public string Username { get; set; }
public string UserId { get; set; }
public string APIKey { get; set; }
public string SharedSecret { get; set; }
public string Token { get; set; }
public string TokenSecret { get; set; }

}

public class CPhoto
{
public string ID { get; set; }
public string Title { get; set; }
public string ThumbnailURL { get; set; }
public List<CView> Views { get; set; }
}

public class CView
{
public string Date { get; set; }
public int Views { get; set; }
}

In this post we will setup a single server CouchBase instance and develop a single user application to manipulate documents in a CouchBase “bucket”.  We will not model the complete object in this post but deal with a simplified version of Photo Object while we get our feet wet on CouchBase CRUD operations and simple CouchBase server Administration.  To make things as simple as possible, for this post, we will be working only with a modified version of the CPhoto object (Document).

Getting The Stuff You Need.

Shopping List

• Visual Studio
• CouchBase Server for Windows (this is open source couchdb enhanced and built on top of the open source memcache project)
• CouchBase .Net API library
• NLog
• Newtonsoft.JSON

Setting up a single node Windows CouchBase Server simple and basic administration is easy and fun. Download and run the installation of CouchBase from here. Fred willing all will go well and you will be placed at local page in your default browser. Bookmark this page and note the Port Number that has been assigned to the default instance of CouchBase.  On first use you will need to create an administrator Username and Password. I left the defaults alone for the Cluster and Server Nodes. Select the Tab Data Buckets.  You will need to decrease the Quota Usage limits for the default Bucket.  With the space you freed up, create a new Bucket called “DLR.Flickr.Example1” .  Here is what mine looks like:

 

And Here is the Bucket Settings Page:

OK.  Now take some time and review the documentation for the .Net SDK here.  You can read through or code along with the examples given there. Done? Now let’s get to work.

Starting to Code CouchBase

Open Visual Studio and select Tools/NuGet Package Manager/Package Manager Console and enter the command:

Install-Package CouchbaseNetClient

Create a new Visual Studio Console application.  I called mine:

DLR.CCDB.ConsoleApp and set the default namespace to DLR.CCDB.  Add references to:

Couchbase

Enyim.Memcached

Newtonsoft.Json

[ If you can not resolve Newtonsoft.Json:  Right click on the root of the project and select: Manage NuGet Packages.  Search on Newtonsoft.Json.  Select Install on JSON.Net.  Now try adding the Newtonsoft reference again.]

Now is a good time to add the open source Logging solution to your project.  Select: Manage NuGet Packages.  Search on NLOG. Install both  NLog and NLog Configuration.

Open your App.Config project file.  You will need to make several changes.  Here is what mine looks like after the changes.

Red items are added manually by me (you) and the Blue entries are added by the NuGet Package manager during the sets you followed above.

<!–?xml version=”1.0″ encoding=”utf-8″?>
<configuration>
<configSections>
Couchbase.Configuration.CouchbaseClientSection, Couchbase” />
</configSections>
<couchbase>
<servers bucket=”DLR.Flickr.Example1″ bucketPassword=””>
uri=”http://127.0.0.1:8091/pools” />
</servers>
</couchbase>
    <startup>
<supportedRuntime version=”v4.0″ sku=”.NETFramework,Version=v4.5″ />
</startup>
<runtime>
<assemblyBinding xmlns=”urn:schemas-microsoft-com:asm.v1″>
<dependentAssembly>
<assemblyIdentity name=”Newtonsoft.Json” publicKeyToken=”30ad4fe6b2a6aeed” culture=”neutral” />
<bindingRedirect oldVersion=”0.0.0.0-6.0.0.0″ newVersion=”6.0.0.0″ />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name=”Enyim.Caching” publicKeyToken=”05e9c6b5a9ec94c2″ culture=”neutral” />
<bindingRedirect oldVersion=”0.0.0.0-1.3.7.0″ newVersion=”1.3.7.0″ />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name=”NLog” publicKeyToken=”5120e14c03d0593c” culture=”neutral” />
<bindingRedirect oldVersion=”0.0.0.0-3.1.0.0″ newVersion=”3.1.0.0″ />
</dependentAssembly>
</assemblyBinding>
</runtime>
</configuration>

We are most interested in this section:

<servers bucket=”DLR.Flickr.Example1″ bucketPassword=””>
uri=”http://127.0.0.1:8091/pools” />
</servers>

 

bucket=”DLR.Flickr.Example1″

This sets your default API calls to the bucket “DLR.Flickr.Example1” which you created above.  Although we will not develop the theme here you can override the default bucket during runtime to deal with calls to multiple buckets in the same program.

uri=”http://127.0.0.1:8091/pools

This sets your local node. the http://127.0.0.1 is a constant for development projects (localhost) and the 8091 is the port assigned to CouchBase during installation (double check this value on your system by navigating to the CouchBase Console page you added to your favorites list above.

While we are here let’s make some changes (without explanation why at this point) in NLog.Config (which was created when you installed NLog above).  Replace the entire contents of the file with (mind the wrap):

<!–?xml version=”1.0″ encoding=”utf-8″ ?>
<nlog xmlns=”http://www.nlog-project.org/schemas/NLog.xsd”
      xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
      throwExceptions=”true”
internalLogFile=”C:/temp/NLog/WEBAPI/Internal.txt”
internalLogLevel=”Info”
>
<targets>
<target name=”debugfile” xsi:type=”File” layout=”${message}”  fileName=”C:/temp/DLR.Flickr/Debug.txt” archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”                        />
<target name=”logfile” xsi:type=”File” layout=”${message}”    fileName=”C:/temp/DLR.Flickr/Info.txt”  archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”                      />
<target name=”Errorsfile” xsi:type=”File” layout=”${message}” fileName=”C:/temp/DLR.Flickr/Error.txt” archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”                   />
<target name=”Fatalfile” xsi:type=”File” layout=”${message}”  fileName=”C:/temp/DLR.Flickr/Fatal.txt” archiveNumbering=”Rolling”  archiveEvery=”Day” maxArchiveFiles=”7″ ConcurrentWrites=”true”                    />
</targets>
<rules>
<logger name=”*” minlevel=”Debug” maxlevel=”Debug” writeTo=”debugfile” />
<logger name=”*” minlevel=”Info” maxlevel= “Fatal” writeTo=”logfile” />
<logger name=”*” minlevel=”Error” maxlevel=”Fatal” writeTo=”Errorsfile” />
<logger name=”*” minlevel=”Fatal” maxlevel=”Fatal” writeTo=”Fatalfile” />
</rules>
</nlog>

We will get back to the details of this configuration file in the next post.

Write the minimum test code possible. 

Replace the contents of Program.cs with

using System;

using Couchbase;

using Enyim.Caching.Memcached;

using Newtonsoft.Json;

using NLog;

namespace DLR.CCDB.ConsoleApp
{

class Program
{
static void Main(string[] args)
{
CouchbaseClient client=new CouchbaseClient();

        }

    }

}

Build and run.  You should have no errors and client should not equal null after the call

CouchbaseClient client=new CouchbaseClient();

Success?  Take a break and we will continue next week.

 

=>

This is Part II of my series on the Flickr API and View Statistics.  You can find Part 1 here.

Assume a C# static class containing the critical information you need to identify yourself to Flickr:

public static class User{

public static string FullName {get;set;}

public static string Username {get;set;}

public static string UserId {get;set;}

public static string APIKey {get;set;}

public static string SharedSecret {get;set;}

public static string Token {get;set;}

public static string TokenSecret {get;set;}

}

Where APIKey and SharedSecret identify you (as the developer) to Flickr and Token and TokenSecret identify your relationship with the user which encodes what rights the user has granted to you.

 

FlickrNet defines a C# class OAuthAccessToken as:

 public class OAuthAccessToken : IFlickrParsable
{

        public string UserId {get;set;}

        public string USername {get;set;}
public string Token { get; set; }
public string TokenSecret { get; set; }
}
}

Given the above, to get things rolling we need to populate the FlickrNet OAuthAccessToken fields from our User class, and then instantiate FlickrNet’s FlickrManager:

OAuthAccessToken o = new OAuthAccessToken();

o.Token=User.Token;

o.TokenSecret=User.TokenSecret;

o.UserId=User.UserId;

o.Username=User.Username;

FlickrNet.Flickr _F = FlickrManager.GetAuthInstance(User.APIKey, User.SharedSecret);

Now we can loop through our photos getting the number of views for each photo on the datetime of the call.  We do these with paged calls to FlickrNet’s PeopleGetPhotos:

public PhotoCollection PeopleGetPhotos(PhotoSearchExtras extras, int page, int perPage);

Here is a simple loop to do so (without error checking):

int page=0;

while (true){

PhotoCollection photo = _F.PeopleGetPhotos(PhotoSearchExtras.Views, page++, 500);

     if (photo.Count == 0){ break; }

//Loop through each photo on the page:

foreach (FlickrNet.Photo p in photo)
{
//Save The Results

    SaveByID(User.UserID,p.PhotoID, p.Title, p.ThumbnailURL, p.Views); 

}

}

OK so far?  Note that in FlickrNet the Views field is defined as int? (which means that Views may contain a null) For our purposes here we will regularly convert those nulls to zeros.  The Views is not a delta but the total number of views (clicks since the photo was first posted up to the time in which the call is made).  Most of our next post will discuss one approach to collect and store views per day (or some other sample period).

ThumbnailURL can be used to retrieve the thumbnail of the image.  Note here that starting last month Flickr increased the default size of the thumbnails on new photos.  We will come back to this issue in a later post which deals with displays.  Speaking of display there is a field called LargeURL associated with each Flickr Photo object but we have had trouble working with.  Instead I will display full size images within Flickr using the formula:

https://www.flickr.com/photos/{full user name no spaces}/{photoID/in/photostream/

Example:

https://www.flickr.com/photos/dennisredfield/14588760519/

display the Flickr page for this photo.

 

Series continues here.

 

=>

In Yahoo’s misguided attempts to make Flickr a Phone photo sharing app like Instragram has radically redesigned the interfaces on both the desk top and on phones. Along the way they give you the option of uploading all of your phone photos directly to Flickr from your phone.  Which is nice, I guess, but Google and Microsoft and DropBox also offer the same service. I mean why would I want to dump my raw photos to Flickr when I, like most Flickr users, do our processing offline (in Photoshop for example) and post our best work to Flickr.  It would seem they are abandoning their core users (amateur photographers) to make way for the Instragram raw sewage stream of sunsets, meals and selfies.  Strange.  They are going to lose out to the Instragram / Facebook for that market and alienate the current group of core users. Is this what you want to see when you go to Flickr?  This:

  Or this? 

In order to accommodate the change Flickr, like all the big players offers a huge amount of cloud storage ‘for free’.  Something like two Gigabyte of photo storage.   Before the big changes at Flickr there was a special Flickr service “Flickr Pro” which for $29 a year got you, among other things, daily usage statistics (going back 30 days) of your photos.  Statistics were also available via the Flickr API (more on this in a moment). With the new Flickr statistics are gone and the Flickr API for statistics are also gone.

So I started digging into the existing, and still supported, Flickr API to see what was necessary to get usage statistics from Flickr for my photos using the Flickr APP.  I a C# programmer by profession so I started by looking at existing Flickr API libraries to leverage in this quest.  The godfather of C# Flickr API libraries is the FlickrNet API Library which is available in source under both the LGPL 2.1 license, and the Apache 2.0 license.

But First

This is a great and fully comprehensive library but to use it you need a couple of things to get started.  You must obtain a non-commercial personal API Key from Flickr (an application key). Apply for one here. But wait there is more.  Although the Flickr API is fairly straight forward you must, must, must, use Flickr’s implementation of OAuth with the application key to access Flickr data.  This post will be devoted to getting up and running with the Flickr API and Oauth.  Subsequent posts will discuss how to read photo view data and present a simple file database to store statistics.  Finally I will turn to using CouchDB and a Web front end to do the same work.  For this part we will switch over to Ruby and write our own simplified FLickrNet library.

OAuth

OAuth, in its many variations, is all over the place on the web these days.  Every time you see a “Sign In with…” Facebook, Google, whatever you are using OAuth in action.  You must use the Flickr OAuth implementation  to access your own data on Flickr.  It is really not all that hard although at first blush it looks impossible.  Here is Flickr’s own overview:

Simple and fun you say!

It’s not that bad when you get into it.

Using code samples provided by the folks at FlickrNet here is a simple walk through of how to setup OAuth to access your own data for a desktop application.  First we will look at it as a simple cook book approach (in C#) and then give a much simpler view of OAuth than the above diagram.  And that will be it for this post.

• Application Key

Get an application Key and a “shared” secret from Flickr (start here).

• Grant Access Writes for this Application Key

Get the user (that’s you to grant you access to their (your) photos.

Prepare a special HTTP  Request to Yahoo which looks like

http://www.flickr.com/services/oauth/authorize?oauth_token={hash}&perms={permsValue}

where:

{permsValue} is the access writes you are requesting to have granted to the caller

{hash} is a   Base64 encoded SHA-1 hash based on your Application Key and shared secret.  Not to worry FlickrNet handles the details for you.

Assuming you have created a FlickrManager class object (following the examples in the FlickrNet Samples Site) which includes your Application Key and shared secret the request URL is created as:

Flickr f = FlickrManager.GetInstance();  //you create this and encode your Application Key and Shared Secret into this class

requestToken = f.OAuthGetRequestToken(“oob”); //”oob” indicates that you are making a desktop request not a request from a web page

string url = f.OAuthCalculateAuthorizationUrl(requestToken.Token, AuthLevel.Read);//Request access to only what you need, in this case READ

opening this url will direct you the Yahoo login page and once logged in ask if you which to grant access for this application key to your data (photos and stuff) on Flickr.  When you answer YES Flickr will present to you on the screen (no call back as requested) an grant access key which is unique to your data and your application key.  This key is also called the“verifier” Write this down, we will use it below.

Here is what is happening in principle.  Flickr and you as the developer have a trust relationship where they grant you and application key for your use as a non-commercial developer.  The shared secret  is used as a key to encrypt messages between you (the developer) and Flickr.  This is used to request an access grant key between this application key (as a developer) and your data (as a Flickr user).  This is a one time process.

Get The OAuth Token

Each time we want to get  data back from Flickr we need to combine our Application Key, Shared secret and the access grant into a second specialized URL Get request to receive an OAUTH token which is time limited. In simple FlickrNet Code this looks like:

Flickr f = FlickrManager.GetInstance();
var accessToken = f.OAuthGetAccessToken(requestToken, verifier);
FlickrManager.OAuthToken = accessToken;

Here requestToken is a class object containing the Application Key and shared secret.  The field verifier contains the Grant Access Key we obtained above. The call to OAuthGetAccessToken builds a HTTP Get request to:

https://www.flickr.com/services/oauth/request_token?oauth_signiature={sig}

where:

seg is a SHA-1 Hash of the input parameters. Let FlickrNet handle the details for you.

Ok So Far? Here is a picture:

 

Each data call to Flickr for data is an HTTP call and should be signed with the Application Key and the Access Object Token. Responses are returned in any of several formats of which XML and JSON are the most important.  Here is what a simple call to get photo data for a specific user (and returning XML) looks like:

https://api.flickr.com/services/rest/?method=flickr.people.getPhotos&api_key=123456789&user_id=12345&format=rest&auth_token=12345-6789&api_sig=98765

where:

api_sig is a hash of this call based on our Token Secret.  Not to worry we will use the FlickrNet assembly to generate this value.

Ok.  That’s enough for one post.  Get your Application Key from Flickr, Run the samples for FlickrNet to get your access key and generate an Access Object (OAuth Token), and in the next post we will dig in with using the Flickr API.

 

Part 2 continues here.

 

 

 

 

 

=>

Although I have covered this topic in a technical blog some time ago, the programmatic approach is not for everyone.  As one correspondent wrote:

Cloud,

I just want to listen to Dark Star on my IPOD, not get a computer science degree.

Well, if you just want to listen to Dark Star (or Fire On the Mountain for that matter) just go here or here.  Right Click on any track listed, select SAVE AS and Bear’s your Uncle.  But if your needs go deeper and you don’t want to write code; here dear reader is a simple explanation involving nothing more technical than a browser (Chrome will do fine but use Internet Explorer will work also) and a text editor (notepad for example).

Know Your Rights

The first thing we do, let’s kill all the lawyers…

                                              Shakespeare  ‘Henry VI,” Part II

Please read the statement from the Internet Archive on use of the Grateful Dead recordings stored on the Internet Archive here.  Understanding and interpretation of same is up to you, not me. I am not a lawyer, I don’t even play one on television, thank you.

Doing a Single Track

Let’s say we like an early Dark Star, say:  Grateful Dead Live at Carousel Ballroom on 1968-01-17 for example.  Cursing to the Internet Archive we find the concert page for the Soundboard version we like the best:

Opps, No downloads on this concert. Let’s take a closer look at the browser screen, at the URL text box at the top:

the “Https://archive.org/details” is the same on all concerts but the other part is the unique key for this particular version of this concert:

gd1968-01-17.sbd.jeff.3927.shnf

I will call this the IAKEY.  We will use this key to delve into the Internet Archive a little more deeply.  Copy this text string from the URL text box and copy it into a Notepad text document.  Now we need a list of all the mp3 files associated with this recording of this concert. Internet Archive stores these in an xml file uniquely defined for each recording. Copy the following template string into a second Notepad text document:

http://www.archive.org/download/{IAKEY}/{IAKEY}_files.xml

Now using text replace the string “{IAKEY} with the IAKey we got perviously.  When you are done the string will look like:

http://www.archive.org/download/gd1968-01-17.sbd.jeff.3927.shnf/gd1968-01-17.sbd.jeff.3927.shnf_files.xml

Open a new browser window and copy this new string into the URL text box and press enter.  Here is what you get back:

Search for “Dark Star” on the screen and locate the mp3 file name I will call this the TrackMP3 (ignore any mp3 entries with the digits 64 in the mp3 file name).  In this case the mp3 for Dark Star is:

gd68-01-17d1t03_vbr.mp3

Open up your Notepad document as paste this string into it:

http://www.archive.org/download/{IAKEY}/{TrackMP3}

Now in this string replace {IAKEY} with gd1968-01-17.sbd.jeff.3927.shnf as we did before and replace {TrackMP3} with gd68-01-17d1t03_vbr.mp3

The final text string will now look like:

http://www.archive.org/download/gd1968-01-17.sbd.jeff.3927.shnf/gd68-01-17d1t03_vbr.mp3

Open up a new browser tab and paste this string into it and press enter.  There you are Dark Star will start playing and just RIGHT CLICK and Select “Save As” and Bear’s your uncle.

Doing a Group of Tracks

This gets tedious right away.  You can improve this process to skip the part where the file starts playing first by using a little HTML code.  I am not going to explain the whole process but it works like this:

follow the steps above and get a couple (or all) of the TrackMP3 strings.  “Turn on Your Love Light” has an TrackMP3 of gd68-01-17d1t01_vbr.mp3.  Cryptical Envelopment has a TrackMP3 of gd68-01-17d2t01_vbr.mp3 and  Dark Star we already know.

Open a text editor and enter the lines:

<html><body>

<a href=http://www.archive.org/download/{IAKEY}/{TrackMP3}> {title}</a>

<a href=http://www.archive.org/download/{IAKEY}/{TrackMP3}> {title}</a>

<a href=http://www.archive.org/download/{IAKEY}/{TrackMP3}> {title}</a> 

</body><html>

Now replace {IAKey} and {TrackMP3} and {title} in each line with the strings you got above and save the text file with the extension HTML (NOT as a txt file).  Load the HTML file into the browser (drag and drop will work) and you will see in the browser:

Turn on Your Love Light

Cryptical Envelopment

Dark Star

Right click on each of these in turn and click Save As… I hope you are getting the picture now.

=>

“Paint It Black, You Devil”

Angst fan on the Rolling Stones Get Your Ya Yas Out Album

 

It’s My Own Invention or The Name Of This Song Is Called

You are sad,” the Knight said in an anxious tone: “let me sing you a song to comfort you.”

“Is it very long?” Alice asked, for she had heard a good deal of poetry that day.

“It’s long,” said the Knight, “but it’s very, very beautiful. Everybody that hears me sing it——either it brings the tears into their eyes, or else——”

“Or else what?” said Alice, for the Knight had made a sudden pause.

“Or else it doesn’t, you know. The name of the song is called ‘Haddocks Eyes.'”

“Oh, that’s the name of the song, is it?” Alice said, trying to feel interested.

“No, you don’t understand,” the Knight said, looking a little vexed. “That’s what the name is called. The name really is ‘The Aged Aged Man.'”

“Then I ought to have said ‘That’s what the song is called’?” Alice corrected herself.

“No, you oughtn’t: that’s quite another thing! The song is called ‘Ways And Means‘: but that’s only what it’s called, you know!”

“Well, what is the song, then? ” said Alice, who was by this time completely bewildered.

“I was coming to that,” the Knight said. “The song really is ‘A-sitting On A Gate‘: and the tune’s my own invention.”

Rutherford Chang buys White Albums, just White Albums.  He currently has almost 1,000 first editions of

the Beatle’s classic “The Beatles” (The record is not actually titled The White Album).  At the  top of this

blog is a photograph of my wife’s first edition copy.  Unlike most limited edition art works the owners of

The White Album “prints” often adorned there copies with there own expressions (artistic or otherwise). 

Mary Jo felt free to put her dorm name and room number on hers so if it was lost someone could have

returned it to her!

 

 

 

 

 

 

 

                                                                                                                                                                 The cover was designed by Richard Hamilton, and the Beatles recorded the album in 1968.  So tense were

the recording sessions that at various times George Martin (producer), Geoff Emerick and Ringo all “quit” 

for at least some length of time.  Love it or hate it this revolutionary set of tracks is almost fifty years old. I

have purchased it four times (as media has changed).

 

Chang Rutherford made digital recording of 100 vinyl copies of the first side of the White Album and

mixed them down to a single track. Listen to it here. I think it’s great.  I took the last three minutes of

Chang’s track (Happiness Is a Warm Gun” X 100) created a mash-up with some other Beatles sounds

which you can view / listen to here:

White Album Side One X 100 : The Last Three Minutes

The Cloud2013 All Digital Sound and Media Studio Used To Create: The Last Three Minutes and this blog post:

=>

It a long way from the early John Lennon demo tape for Revolution 1 to the Nike Ad.

You say you want a revolution
Well you know
We all want to change the world
You tell me that it’s evolution
Well you know
We all want to change the world
But when you talk about destruction
Don’t you know you can count me out/in

We can start with the demo tapes done before the White Album.

We recorded the song twice. The Beatles were getting real tense with each other. I did the slow version and I wanted it out as a single: as a statement of The Beatles’ position on Vietnam and The Beatles’ position on revolution. For years, on The Beatles’ tours, Brian Epstein had stopped us from saying anything about Vietnam or the war. And he wouldn’t allow questions about it. But on one of the last tours, I said, ‘I am going to answer about the war. We can’t ignore it.’ I absolutely wanted The Beatles to say something about the war.

John Lennon
All We Are Saying, David Sheff

According to Wikipedia:

The White Album (Kinfauns) demos (1968)

In May 1968, The Beatles met at Kinfauns, the Esher home of George Harrison, to review and record demos of songs under consideration for their next album; twenty-seven songs, mostly acoustic, have become public from this session.^[40] Seven of the songs were released on Anthology 3, including “Junk“, a song McCartney would later record for his first solo album. Of the twenty demo songs not officially released, fifteen would be recorded and released on the White Album, while “Not Guilty” and “What’s the New Mary Jane” would be recorded for the album but not make the final line-up.

And here is The “Kinfauns Acoustic Demo” (I love this one):

Kinfauns Acoustic Demo

You say you got a real solution
Well you know
We don’t love to see the plan
You ask me for a contribution
Well you know
We’re doing what we can
But if you want money for people with minds that hate
All I can tell you is brother you have to wait

Then the Beatles as a group played the track many, many times.  Here is Take 20 with some post processing:

Revolution 1 Take 20 (Rejected Mix)

From Wikipedia again on Take 20:

Take 20

Low-quality monitor mixes of the full-length version of Revolution appeared on various bootlegs, such as From Kinfauns to Chaos, throughout the 90s.^[16] Then in 2009, a high-quality version labeled “Revolution Take 20” appeared on the bootleg CD Revolution: Take…Your Knickers Off!^[17][18] The release triggered great media fanfare and activity among fans. This version, RM1 (Remix in Mono #1) of Take 20, runs 10 minutes 46 seconds (at the correct speed) and was created at the end of the 4 June session, with a copy taken away by Lennon.^[19] It was an attempt by Lennon to augment the full-length version of “Revolution” in a way that satisfied him before he chose to split the piece between the edited “Revolution 1” and the musique concrete “Revolution 9”.

Finally we get the White Album mix which is pretty sweet:

White Album Revolution as Released.

George thought this was TOO SLOW for a single and John wanted the song out NOW.  And so on July 10, 1968 the fast, loose and out of control version (more in keeping with the times really) was recorded.

We got into distortion on that, which we had a lot of complaints from the technical people about. But that was the idea: it was John’s song and the idea was to push it right to the limit. Well, we went to the limit and beyond.

George Martin
Anthology

Here is “GetGBackEsto”’s restored version of a live studio performance of the fast version of Revolution (with backing vocals absent from the released single):

Live Take of Revolution Post July 10

The Video of this performance with the raw sound mix can be seen:  Beatles Revolution Fast

Finally he is John’s mix for the single Revolution (this was the “B” side to Hey Jude):

Fast Revolution as Released

This is the one I bought in High School and it was before the White Album and slow version of Revolution was released.  Listening to it had a big effect on me.

You say you’ll change the constitution
Well you know
We all want to change your head
You tell me it’s the institution
Well you know
You better free your mind instead
But if you go carrying pictures of Chairman Mao
You ain’t going to make it with anyone anyhow

The less said about Nike’s use of the fast version of Revolution for a shoe ad the better.

From 1983 through 1990 The Grateful Dead pulled out Revolution as an encore closer.  Hear them below:

19831012	Madison Square Garden	Revolution	05:11
19831017	Olympic Arena	Revolution	05:18
19831022	Carrier Dome, Syracuse U	Revolution	05:21
19831031	Marin Veterans Memorial Auditorium	Revolution	05:20
19841009	The Centrum	Revolution	04:57
19841020	Carrier Dome, Syracuse U	Revolution	05:30
19850408	The Spectrum	Revolution	05:08
19850702	Civic Arena	Revolution	04:51
19851108	Community War Memorial Auditorium	Revolution	06:02
19900328	Nassau Veterans Memorial Coliseum	Revolution	05:09

This just in

=>

Centuries of invaders couldn’t break the Great Wall of China, but a Chinese yuan can. Well, the “Great Firewall,” at least.

A series of one yuan banknotes became a whole lot more valuable after being stamped with a quick response (QR) code — a type of matrix barcode that, when scanned by a smartphone, sends a user to a website stored in the code — that circumvents the infamous firewall.

Beneath the stamped QR codes are the words: “Scan and download software to break the Internet firewall.” Doing so leads to an Amazon cloud link that hosts downloadable software to get around the firewall.

Read the rest of the story here http://mashable.com/2014/01/24/chinese-money-firewall/.

=>

.

=>

If I was an eagle I’d dress like a duck 
Crawl like a lizard and honk like a truck
If I get a notion I’ll climb this tree
or chop it down and you can’t stop me
Chop it down and you can’t stop me
Freedom
Liberty
Leave me alone
To find my own way home

Liberty.  (Giants Stadium 1993-06-06)

I have been spending some time with the 1993 Grateful Dead concert tapes on the Internet Archive.   The Modern Deadhead blog has a nice discussion of this year with the Dead here.  Jerry is drawing from deep within himself with both is playing and his singing during the summer tour.  There are some powerful performances during 1993 including:

• Days Between
• Liberty
• Lazy River Road. (1993-08-21)
• So Many Roads. (1993-12-19)
• Standing On The Moon. (1993-08-21), and
• Rain. (1993-09-16)

The 1993-08-22 show at the Autzen Stadium an the University of Oregon has been in pretty heavy rotation on my mp3 player.  There is a lot to recommend this show including  a stand away version of Days Between with a great vocal and a strong end solo.  The first set opens with Jack Straw.  About 60 seconds into the show the mikes die and Bob’s guitar set up goes into cardiac arrest.  It takes several minuets for his setup to reboot and come back on line.  Jerry, who is no stranger to equipment failures and guitar solos takes the weight and carries the song with some real pyrotechnics. The first set ends with a fresh sounding The Music Never Stopped.  The second set is action packed starting with Help On the Way –> Slipknot –> Franklin’s Tower.  The closer is really a great Space followed by The Wheel –> I Need A Miracle –> Days Between –> Not Fade Away. WoW.  Liberty takes us home as an encore. Listen NOW!

1993-08-22 Autzen Stadium  (University of Oregon)

Click Track to Play or Steal

01	crowd/tuning	05:07
02	Jack Straw	09:12
03	Bertha	08:04
04	Little Red Rooster	09:26
05	Broken Arrow	05:25
06	Althea	08:02
07	Masterpiece	05:27
08	Tennessee Jed	08:56
09	The Music Never Stopped	08:48
10	crowd/tuning)	02:46
11	Help On The Way–	04:26
12	Slipknot!–	05:28
13	Franklin’s Tower	11:55
14	Samson And Delilah	06:46
15	Ship Of Fools	07:51
16	Corinna–	09:23
17	Drums–	07:46
18	Spacey Drums–	12:38
19	Space–	13:03
20	The Wheel–	06:59
21	I Need A Miracle–	04:05
22	Days Between–	10:51
23	Not Fade Away	11:41
24	Liberty	06:28

With film footage from all over space and time.

And while we are visiting the past here is a little snip from 1981:

• New Years Eve Countdown With Ken Kesey and Iko Iko (1981-12-31 Oakland Auditorium)

Went to the well but the water was dry
Dipped my bucket in the clear blue sky
Looked in the bottom and what did I see?
The whole damned world looking back at me

If I was a bottle I’d spill for love
Sake of mercy I’d kill for love
If I was a liar I’d lie for love
Sake of my baby I’d die for love
Sake of my baby I’d die for love

Freedom
Liberty
Leave me alone
To find my own way home
To find my own way home
I’m gonna find my own way home

RIP, Jerry.

Chanting by The Guyoto Monks (1995-06-02)