Archive for the ‘REST’ Category

Sharepoint 2013 REST API: Retrieving List Item Data   1 comment

I first discussed retrieving List Item data here: Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data.  The simple REST API call:

Web/list(guid’850fae4-0cce-8c30-c2a85001e215’)/Items

functions like

select * from Table1

in a SQL data base.  However Sharepoint, although resting on SQL Server for everything, will only return the a limited number of rows from the underlying list.  The maximum number of items (rows) returned in a single call is governed by a system constraint.  On our out of the box Sharepoint 2013 installation the limit was 100 rows!  Supposable Sharepoint Sever Admins can change the maximum value but they can not eliminate the constraint.  The naïve coder might think that this can be gotten around using the $SKIP parameter in the ODATA extensions to the Sharepoint REST API.  Alas $SKIP is not implemented, instead M$ implemented it own undocumented skip parameters.  No to worry you don’t need to provide your own parameter values.  The initial call using

…/Items

returns X rows of the table (in primary key index order) where X is equal to the page constraint discussed above.  If there are additional rows the metadata in the returned data structure will return a “next” hypermedia link whose value is fully qualified Sharepoint Rest API call for the next X rows (i.e. the next page). 

google-barge-sinking

                               Google Barge Sinking Slowly in San Francisco Harbor after being infected with the Sharepoint 2013 Virus

We need then to setup a typical paging object to read items by page until all items are read.  I will be leveraging the code patterns given in the series referenced above the page calls (…/items) return results in XML format and is cast as an XDocument type.  Then we can probe for a “next” link as:

static string _GetLinkToNextPage(XDocument xDoc)
{
const string matchString = “/Web/Lists”;
var links = from g in xDoc.Descendants().Elements(CSPNamespace.baseNS + “link”)
select g;
const string NEXT = “next”;
foreach (XElement link in links)
{
string rel = string.Empty;
string href = string.Empty;
foreach (XAttribute att in link.Attributes())
{
if (att.Name.LocalName == “rel”)
{
rel = att.Value;
}
if (att.Name.LocalName == “href”)
{
href = att.Value;
}

               }
if (rel == NEXT)
{

                   //return just the relative path
return href.Substring(href.IndexOf(matchString));
}
}

           //return null if no link is found
return null;
}

The complete object template looks like:

public static class CItemPage
{
public static List<CEntry> _Worker(HttpClient client, string next)
public static List Exec(HttpClient client, CEntry listMetaData)
       static void _Accumulate(XDocument items, ref List allItems)
static string _GetLinkToNextPage(XDocument xDoc)
static XDocument _GetListMetaData(HttpClient client)

}

The entry point for the paging activity is CItemPage.Exec which turns around and calls worker where we loop through the data and (for my own purposes) I then accumulate the items from all pages in a single list<CEntry>  using my method _Accumlate.  calls Worker

       public static List _Worker(HttpClient client, string next)
{
try
{
List allItems = new List();
XDocument items = null;
while (next != null)
{
items = _GetItems(client, next);
next = _GetLinkToNextPage(items);
_Accumulate(items, ref allItems);
}
return allItems;
}
catch (Exception x)
{
var b = x.Message;
return null;
}
}

 

Easy and fun.  Should there be an easier, faster way? yes.

DLRSharePoint-On-Prem-isnt-Dead-Yet

This madness must stop.

Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends   4 comments

Since our last post was so long I left a few odds and ends for this post.  Specifically I will touch on the following in this posting:s_w19_1a35314u

  • Testing for the Presence of a Document within a Document Library
  • Testing for the presence a Folder within a Document Library
  • How to Create a Folder within a Document Library; and
  • How to Create a custom HTTP Exception class derived from the base Exception Class

Testing for the Presence of a Document within a Document Library

As with all things Sharepoint the most important part of this task is composing the correct uri fragment.  We need to have three pieces of information to preform the test:

  • The Document Library Name
  • The Folder Path (if any) within the Library
  • The Document Name (root plus extension)

We then compose a uri fragment as:

web/GetFolderByServerRelativeUrl(‘/{Document Library/FolderPath’)/Files(‘{Document Name}’)

So if we are testing for a Document myDocument.PDF in the folder path  AdminFolder\ClaimsFolder in a Document Library Call Accounting Documents our uri fragment becomes:

web/GetFolderByServerRelativeUrl(‘/Accounting Documents/AdminFolder\ClaimsFolder’)/Files(‘myDocument.PDF’)

One then makes an HTTP Get call against the Sharepoint REST API.  An Http Status Code of OK (numeric value: 200) indicates that the file exists.

An Http Status Code of NotFound (numeric value: 404 ) indicates that the file is not found at that location.

This get call does NOT return the document itself to the caller.

Testing for the presence a Folder within a Document Library

This test is even simpler.  We need two pieces of information:

  • The Document Library Name
  • The Folder Path (if any) within the Library

We compose the uri fragment as:

web/GetFolderByServerRelativeUrl(‘/{Document Library Name\Folder Path}’)

One then makes an HTTP Get call against the Sharepoint REST API.  An Http Status Code of OK (numeric value: 200) indicates that the folder path exists.  An Http Status Code of NotFound (numeric value: 404 ) indicates that the folder path is not found within the library.s_w04_1a35329u

How to Create a Folder within a Document Library

In order to create a folder we need to:

Compose a proper uri fragment;

Compose a JSON formatted content body (System.Net.Http.HttpContent);

Get a REST API Digest Value (See Part 3 of this series on this) and include it in the header; and

Make a HTTP POST call to the Sharepoint REST API

So here we go.  The uri fragment takes the simple fixed form of:

web/folders

The JSON HTTPContent format does the real work and takes the form of:

[ ‘__metadata’: [ ‘type’: ‘SP.Folder’ ], ‘ServerRelativeUrl’: ‘/{Document Library Name\Folder Path}’]

So if our Document Library is “Accounting Documents” and your folder name is “ClaimsFolder” our JSON looks like:

[ ‘__metadata’: [ ‘type’: ‘SP.Folder’ ], ‘ServerRelativeUrl’: ‘/Accounting Documents/ClaimsFolder}’]

Having placed this value into a string object as:

string data=”[ ‘__metadata’: [ ‘type’: ‘SP.Folder’ ], ‘ServerRelativeUrl’: ‘/Accounting Documents/ClaimsFolder}’]”;

we create a  HTTPContent object as

System.Net.Http.HttpContent reqContent = new StringContent(data);

After adding the correct Digest Header and ContentType Header our post looks like:

var resp = client.PostAsync(uri, reqContent).Result;

string respString = resp.Content.ReadAsStringAsync().Result;

If the folder is created successfully we will get back an HTTP Status of Created (numeric: 201 )

To create a nested folder just expand the path within ServerRelativeUrl.

New lets turn to a topic that does really fit any where else but I will just stuff it in here.

How to Create a custom HTTP Exception class derived from the base Exception Classs_w01_1a35360u

HTTP calls are parameterized calls and when they fail there is some information we may wish to attach to the Exception object created which we create on errors.  Specifically:  The URL which was called, the returned HTTP Status Code and the Response Body. Here is a simple derived HTTP exception class which I use:

public class HTTPException : ApplicationException
{
public string URL { get; set; }
public string Response { get; set; }
public HttpStatusCode Status { get; set; }
public HTTPException(string message, HttpStatusCode status, string uRLString, string respString)
: base(message)
{
URL = uRLString;
Response = respString;
Status = status;
}

public HTTPException(string message, HttpStatusCode status, string uRLString, Exception innerException)
: base(message, innerException)
{
URL = uRLString;
Response = string.Empty;
Status = status;
}

Assume a typical HTTP method call like:

public static HttpStatusCode RestGet(System.Net.Http.HttpClient client, string uri,List allowed, out string respString)
{
respString = string.Empty;
HttpResponseMessage resp=null;
try
{
resp = client.GetAsync(uri).Result;
respString = resp.Content.ReadAsStringAsync().Result;
_DisplayDebugInfo(client, resp, uri, null, null, respString);
}
catch (Exception x)
{
throw new HTTP.Exceptions.HTTPException(“RestGet”,  HttpStatusCode.ServiceUnavailable,                       client.BaseAddress.ToString() + “/” + uri, x);
}

if (statusCode != HttpStatusCode.OK)
{
throw new HTTP.Exceptions.HTTPException(“RestGet”, statusCode, client.BaseAddress.ToString() + “/” + uri,respString);

         }
return statusCode;
}

We can pick this up in a outer try/catch block like:

try{

//make your HTTP call here

}catch(HTTPException xo){

    Console.Writeline(xo.URL);

    Conosole.WriteLine(xo.Message);

}catch(Exception x1){

    Conosole.WriteLine(xo.Message);

}

Ok That’s it for the REST API and the Client HTTP object.  Next up:  Preparing a WEB API Endpoint to be called FROM Sharepoint 2013 Workflow.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files

VaranasiGaruda1

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data   5 comments

Now that the administrative details of Part 2 are over we can now address some useful issues. Recall that if we query a List we can obtain the uri segment needed to access the items in the list.  This for example would return all items the referenced list:

Web/list(guid’850fae4-0cce-8c30-c2a85001e215’)/Items

Please see this post for an important update about data pages when using the …/Items call

If we don’t want all the items we can apply an OData $filter verb to select a subset of items matching some criteria. If we want a subset of items with a specific Author we could create the following fragment:

Web/list(guid’850fae4-0cce-8c30-c2a85001e215’)/Items?$filter=Author eq ‘Cloud2013’

If you are concerned about download sizes of your calls into the REST API you can limit the property fields returned using the OData $select verb.

The feed returned from a GET call with an uri fragment like that above has an identical structure to the feed discussed in Part 2 of this series. Depending on the list and the $filter there may be zero, one or many entry nodes.  Each entry node contains information on one Item with its corresponding Links collection and properties collection.  In this case the Link collections refers to operations which can be made this single item, for example the “ContentType” Link (Web/Lists(guid’850f0ae4-0cce-47e9-8c30-c2a85001e216′)/Items(3)/ContentType) would return a feed with an entry for links and properties associated with the content item for this item.  The properties collection is, of course, the fields for an item and the values associated with that item.   Here is a Quick Watch screen snap of the data for an item:

QW1

This looks pretty normal.  The field “Title” appears with its value “Points To Approved Item”.  Some of the field names have been Sharepointerized. Our field “AIC Description” has become “AIC_x0020_Description”.  What is going on here?  According to this post:

When you create a column on a list, both its DisplayName and StaticName are set to the same value. However, the StaticName contains converted values for some characters, most notably a space ‘ ‘ is converted to '_x0020_'. So if the DisplayName is ‘Product Description’, then the StaticName will be 'Product_x0020_Description'.

There’s another little bugaboo: The StaticName is limited to 32 characters including the translations for special characters.

In translation, the DisplayName is the name you see on the Sharepoint UI screen.  The value of the display name can change.  When the column is created a fixed “StaticName” is created equal to the initial DisplayName with spaces converted to _x0020.  When working with the REST API we are seeing the StaticName. Ok so far?  We have another problem some fields are missing!  The REST API does not support all columns using the

Web/list(guid’850fae4-0cce-8c30-c2a85001e215’)/Items

format above.  This interesting post has more details. 

So we will need to do some more digging Sharepoint has to store the value somewhere.  Well, in the true spirit of Sharepoint we have to say: it depends.  In our case we us a Publishing Hyperlink field.  This is a field which allows a list item to contain a reference to some other object.  That object can be:

  • an external Web URL
  • a Sharepoint Page; or
  • A Document stored in a Sharepoint Document Library

BTW, in the first instance the whole URL is stored and in the later two instances only the RELATIVE link is stored since these are assumed to point to a SP page or document within the current site.  At our site we need to retrieve the true value of the Publishing Hyperlink.  And as you can see from the above the field (which we call “AIC Publishing Hyperlink”) does not appear in any form in the original REST call.  If we jump to the Links collection for an item we can find a link called:  FieldValuesForEdit. In our case this is:

Web/Lists(guid’cc1e1896-8df8-4f32-8f1d-2afab37329ed’)/Items(2)/FieldValuesForEdit

If we call that link we get back an entry which in Quick Watch looks like:

QW2

OK, there it is, hiding in Sharepointification form as “AIC_x005f_x0020_x005f_Publishing_x005f_x0020_x005f_Hyper” (where did the _X005f come from you might ask. I have no idea, some things in SP I just accept).  The value is a HTML anchor tag and it is pretty simple to parse these to recover the href value.  Which in this case is  (after decoding) “/DLR%20Document%20Library/1969.html” where ““/DLR%20Document%20Library/” points to a Sharepoint Document Library and 1969.html as the Document.  We are working on how Managed Metadata values are stored but our research is not yet complete on this.  In my next post I will discuss how to download and upload a document from a document library (which is how I fell into this problem of retrieving the value stored in a Publishing Hyperlink field.

Ok, we are almost done for today.  Let’s look at a security issue.  What values you see for a field on an item depends on the identity of user making the REST API call and the state of the item within Sharepoint when you look at it (it’s a Quantum thing).   Recall the publication settings for a list item in Sharepoint. This screen snap may help your memory:

settings

When we enable Content Approval and set “Who should see draft items” to Only users who can approve items (and the author of the item)” then an item can exist in two states at once.  If we have an approved item then everyone sees the same thing:

approved

No if an editor changes an item, say by changing the content of the “Description” field and the item is waiting for approval the item is in a Pending Approval Status like this:

pending

If you are a general user you see the values in the Approved item.  If you are an approver or the author you see the values in the Pending Item.  As in the UI so in the REST API.  The values returned to you in your REST API call depend on the Sharepoint security group of the identity you use to access the item and the list settings for approval and visibility. So as in all things, be careful what you ask for.  In some use cases you may want to see pending values and in others you may want to see only approved values.  There are use cases, we have one in workflow processing for a list where we want to see both.  In this case we use two identities to access the same list item and so can compare the pending and approved items.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

Coming up: Document processing!

The Google Barge In Portland Maine.  Don’t believe the cover story.  It’s a 3D printer which can only produce other 3D printer barges!

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data   4 comments

Let’s assume that we have an HTTPClient object as described in Part 1.  That is we have

  • created the HttpClient objected,
  • set the base URL to our Sharepoint site, 
  • set the Authentication Header; and
  • set the Accept Header for  “application/atom+xml”. 

Now we want to query the REST API for the metadata of a Sharepoint List.  Sharepoint List Titles are unique by design we can use the List Title (rather than the GUID) to locate the item and return its metadata. The format of this call as a uri fragment is:

web/lists/GetByTitle(‘listname’)

Now our code looks like:

string uri=”web/lists/GetByTitle(‘Master Document Library’)”;

//Note that spaces are allowed in most calls but there are other situations where spaces are escaped!

HttpResponseMessage resp = client.GetAsync(uri).Result;

string respString = resp.Content.ReadAsStringAsync().Result;

if (resp.StatusCode != HttpStatusCode.OK)

{

throw new ApplicationExcepiton(

                    “HTTP Error. Status: {0} Reason: {1}”,resp.StatusCode,    resp.ReasonPhrase );         

           }

This will put a string representation of the XML formatted metadata about the list into the HttpResponseMessage.  Please note that the call to extract the XML from the response body:

string respString = resp.Content.ReadAsStringAsync().Result;

is only appropriate for string results (XML or JSON as specified in our Accept Header) and is not correct if the return is binary data.  I will copy binary data in a subsequent blog when I discuss file upload and download.

Few things are less documented than the exact contents of the metadata feed returned by calls like this from the REST API.  On a high level it is an Atom feed which allows for a combination of collections of  Property nodes (key/value pairs) and collections of Link nodes. The Property Nodes are the meta data fields related to the list and the Link Nodes are uri segments to guide additional REST API calls concerning what ever item is currently being queried.  Neither the properties nor the links are a fixed are vary from Sharepoint object to object and even vary between objects of the same type if those object depending on the values of the property fields (for example a list item on a list with versioning will contain a Link node for access to the versions.  If the list items are not versioned then the link item of versions will not be emitted.

Rather than list XML directly I will use the visualization tool XMLSPY to display the XML in a “grid view”.  On a high level the entry for a list would look like:

Untitled picture

The Links are on the root of the entry node and the properties are nested as entry/content/properties.  Note that the XML makes heavy use of XML namespaces and any creative manipulation of the entry requires some knowledge of XML, namespaces,  XPath or LINQ for XML.  I use LINQ for XML at my desk so I will use that idiom rather than XPATH to manipulate these objects.  If we expand the properties node it will look something like this:

Untitled picture2

There is a lot of data here about the list most of it only lightly documented.  We can see however that the property key: d:title contains the name of our list which we queried on and d:iD contains the GUID for the list.  The later never changes but the former can be renamed.

If we expand the Links collection  it would look something like this:

Untitled picture3

Note item 10, the Items link. The href attribute contains the uri for the REST API to retrieve all the items in this list, while the Fields link (item 7) is an uri for the Fields currently defined for this list. If we know the d:Id of a particular item (item IDs are not GUIDS for simple integers), say 6, we can retrieve a single item with the uri of the Items uri and post pending in the form of:

Web/list(guid’850fae4-0cce-8c30-c2a85001e215’)/Items(6)

What about the link with a blank title? For historical reasons this is blank but it represents the EDIT link.  To make my life simpler I translate the XML property and link collections into C# Dictionary objects and place them in a C# class with two supporting methods:

public class CEntry

{

public Dictionary<string, string> Links;

public Dictionary<string, string> Properties;

      public string  GetLink(string key){

string value = string.Empty;

Links.TryGetValue(key, out value);

return value;

}

      public string GetProperty(string key)

{

string value = string.Empty;

Properties.TryGetValue(key, out value);

return value;

}

}

At this time I am not using any of the root nodes so I just discard them. I get to the Dictionary objects from the XML using LINQ for XML.  I learned what little I know about LINQ for XML from this book.  To brush up on your XML try this book.   For a XML tree containing entry node(s) my LINQ looks like this:

public static class CSPNamespace

{

public static XNamespace metaDataNS = @”http://schemas.microsoft.com/ado/2007/08/dataservices/metadata”;

       public static XNamespace baseNS = @”http://www.w3.org/2005/Atom”;

       public static XNamespace dataServicesNS = @”http://schemas.microsoft.com/ado/2007/08/dataservices”;

   }

XDocument xList =XDocument.Parse(respString);

IEnumerable<CEntry> group = from g in xList.Descendants(CSPNamespace.baseNS + “entry”)

select new CEntry

{

           Links = MakeLinkDictionary(g),

Properties = MakePropertyDictionary(g)

};

The IEnumerable collection needs special processing before it is accessed.  The following test can help:

To see if the collection contains one or more entries:

group != null && group.Any();

Having passed that test, we can then use the simple Count function to see how many entries are in the collection.

group.Count()

To get the first (or only) entry from the collection

group.First()

These last two test will fails if the collection fails the test above

        CEntry cell = group.First();  //Assumes one and one only

Where MakeLinkDicitionary and MakePropertyDictionary look like:

public  static Dictionary<string, string=””> MakePropertyDictionary(XElement xs)

{

Dictionary<string, string=””> pList = new Dictionary<string, string=””>();

var group = from g in xs.Elements(CSPNamespace.baseNS + “content”).Descendants(CSPNamespace.metaDataNS + “properties”)

select g;

foreach (XElement property in group.Elements())

{

pList.Add(property.Name.LocalName, property.Value);

}

return pList;

}

public  static Dictionary<string, string=””> MakeLinkDictionary(XElement xs)

{

       Dictionary<string, string=””> lList = new Dictionary<string, string=””>();    IEnumerable links = from g in

                      xs.Elements(CSPNamespace.baseNS + “link”)

select g;

foreach (XElement link in links)

{

string rel = string.Empty;

string href = string.Empty;

foreach (XAttribute att in link.Attributes())

{

if (att.Name.LocalName == “title”)

{

                   if (string.IsNulOrEmpty(att.Value)){

                      rel = “Edit”;

                   }else{

rel = att.Value;

                        }

}

if (att.Name.LocalName == “href”)

{

href = att.Value;

}

}

lList.Add(rel, href);

       }

return lList;

}

After this pre-processing the meta data can be accessed in a fairly straight forward manner.

var listMetaData=group.First();

string uri=listMetaData.GetLink(“Fields”);

string iD=listMetaData.GetProperty(“Title”);

We will turn to what to actual do with the meta data in the next post.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient   13 comments

This is the first of a multipart series on the Sharepoint REST API and will focus on using this API with C#.  At our shop we need to manipulate documents and list items between sites.  For our purposes we will be calling a custom WEB API HTTP Endpoint from Sharepoint workflows, these endpoints will then manipulatete Sharepoint objects using the Sharepoint REST API using C#.  If REST means nothing to you but a good sleep start here. The topics we will cover include:

  • HTTPClient Object (Framework 4.5)
  • Understanding and Manipulating the Result Sets returned by the REST API (Here)
  • Downloading and Uploading Documents from Document Libraries Using The REST API
  • Developing WEB API Endpoints To package our units of work (Web API 2.X)
  • Calling the WEB API Endpoints from Sharepoint Workflows using GET HTTP Web Service Action

I will not be looking at calling the REST API directly from within Sharepoint Workflows.  How many blog posts will this be?  I am not sure probably three posts in total.

Getting Our Feet Wet:  What an HTTP GET request to the Sharepoint REST API Looks like (10,000 foot view)

The new async HttpClient is a great advance over previous Microsoft Http clients.  You need to understand something about formal HTTP communications.  Specifically the roles of the different HTTP Verbs, Request Headers, HTTP Errors, and for POST commands, Response Headers and the Response Body.  So lets begin.  When we make AJAX calls in JavaScript all of our calls are passed through the machinery of the User Agent (Chrome, Firefox, etc.) and if you are using a helper library like JQuery, the helper library is doing a lot of the work for you.  When you switch to C# and the HTTPClient there is no User Agent and the client itself is quite thin so you have to a lot of work to get a well engineer solution.  So lets begin with some of the basics of an HTTP Get request.tumblr_ls7xrkHeuq1qchzcpo1_1280

There are four components of an HTTP Get request to be concerned about:

  • Target URL
    • What endpoint is the request headed for
  • Query String
    • What, if any, arguments are passed to the target
  • Accept Header
    • The format the results should be written back to the client
  • Authorization Header
    • What identity is the request being make under

For a Sharepoint REST API we want the site URL for the target URL and append to this \_api\.  So we start with something like:

http://myWeb.domain.com/_api/

In a non-RESTful interface the query string takes the form of:

?object=List&Title=MyList

In a RESTful interface this becomes appended to the base URL itself as something like

{baseURL}\List\ByTitle\MyList

Microsoft’s design for the REST API is a little eccentric and does not follow this pattern. A REST API call to get the meta-data for a list (not the list items) can look like this:

{myserver}/_api/web/lists/GetByTitle(‘MyList’)

where: {myserver}/_api/ is the baseURL and web/lists/GetByTitle(‘MyList’) are the arguments and values used in the call.

A GET call for a List, or any Sharepoint object returns a very complex objects whose structure is (basically) identical for different object types so once you understand the basic structure of the response it applies to many different objects.  We will not discuss that response object from a Sharepoint REST API Get request in this post. Note here that objects.  Anywhere the REST API allows an argument to be enclosed in single quote marks we may use an argument value with embedded spaces with out escaping the spaces.  Of concern to us at this point is the format of the response to the call. We have two choices XML format or JSON format.  When we are in the browser world and are using JavaScript to process the response, typically JSON is the preferred format, but XML can also be processed using JavaScript and assorted helper libraries.  When we are using a language like C# we can work most effectively with XML output.  In my case I prefer to receive XML and process that response using LINQ for XML.

Authentication for the Sharepoint REST API comes in three flavors:  OAuth token, Basic Authentication and (Windows) Integrated.  OAuth token only applies if you are calling from a Sharepoint APP (which means you are using JavaScript by definition) or from a Sharepoint Provider-Hosted High Trust App. (a very rare beast indeed and if you are using one you probably don’t need to be reading this post).  I will be concerned here with Basic and Integrated Authentication.

Now Lets Look at some Details Of A Sharepoint REST API call using C#

The HTTPClient object

Simple Anonymous HTTP GET call using HttpClient

        System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
_Client.BaseAddress = new Uri(baseURL);
HttpResponseMessage resp = client.GetAsync(uriString).Result;
string respString = resp.Content.ReadAsStringAsync().Result;
        if (resp.StatusCode != HttpStatusCode.OK)
{
throw new ApplicationException(“BAD”);
}

Where baseURL is a string representation of the site we are calling into for the REST API:

http://myServer.com/_api/

and

urrString is a string representation of the actual REST API call we which to make:

web/lists/GetByTitle(‘MyList’)

Since we are not setting an authorization method we are calling anonymously and this call in its current form would return a failure HTTP Status and the response body will contain information in the REST API default return format.  The HTTP status is returned as a Framework Enum System.Net.HttpStatusCode.  In formal terms HTTP status codes are numeric values.  If you want to recover the actual value you can cast HttpStatusCode to an int.  Thus  HttpStatusCode.OK cast as an integer is 200. This call as written wourld return HttpStatusCode.Unauthorized ( 401).

We need to associate an identity to the HttpClient object in order to get the REST API to return data to us.  I will discuss two ways to do this  (of the many options):

BASIC Authorization and Windows Integrated Authorization.  Basic Authorization depends on adding an Authorization Header to the client with  appropriate  user information (typically domain, user name and password) in a very specific format.

Header Name:  Authorization

Argument: BASIC domain\username\password

where domain\username\password are encoded as a Base 645 String:

static string _FormatBasicAuth(string domain, string user, string password)
{
const string format0 = @”{0}\{1}”;
const string format1 = @”{0}:{1}”;
string userName = string.Format(format0, domain, user);

return Convert.ToBase64String(Encoding.Default.GetBytes(

               string.Format(format1, userName, password)));

}

We then create the Authorization header on the HttpClient object:

System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();

_Client.DefaultRequestHeaders.Authorization =

         new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(Domain, User, Password));

Note that the password is NOT encrypted so BASIC Authorization should only be used over a secure transport layer like SSL.

An alternative is to use Microsoft’s Integrated Authorization (also know as challenge and response).  This takes  network credentials (either of the active thread or created from initialization values) and allows the server to confirm the identity and validity of the caller without transmitting the data over a secure transport layer (‘that password it self is never passed over the wire’).  Here is an example using Network Credentials created on the fly:

_Client = new System.Net.Http.HttpClient(new HttpClientHandler()
{
new NetworkCredential(User, Password, Domain)
});

Alternatively the NetworkCredential object can be created from the running thread of your C# program (you do not need to supply the password  and other parts of the credentials explicitly in this case).  When set up this way you do not use an Authentication Header.  The Windows challenge and response pattern (Http Status 401) are controlled on the client side by the HttpClientHandler.

Assuming your REST API call is well formed and your credentials  are valid the response from the server will contain a representation of a Sharepoint entity (or entities).  I will discuss the structure of this data in a subsequent post.  You can control how the response is formatted by adding an Accept Header to the HttpClient Object.  For the REST API you have two choices: XML or JSON.  If you do not supply a header the default format is XML.  For the REST api both the XML and JSON take a very specific format in the Accept header.  XML requires a Accept Header (if supplied) with an argument of:

application/atom+xml

while JSON requires an argument of:

application/json;odata=verbose

The Accept Header for XML can be added to the HTTPClient as:

  _Client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));

Currently there is a bug in the HTTPClient which prevents adding an Accept Header for REST API formated JSON.  For JSON you need to use a different format:

_Client.DefaultRequestHeaders.Add(“Accept”,@”application/json;odata=verbose”);

Normally I use the XML format of the response for C# programs (although using JSON.NET you could process the return in JSON format).  Javascript coders will normally us the JSON format (although they could process the XML format).

My typical HTTP Get calls looks something like this:

System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
_Client.BaseAddress = new Uri(baseURL);
_Client.DefaultRequestHeaders.Accept.Add(

             new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));

_Client.DefaultRequestHeaders.Authorization =

         new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(Domain, User, Password));

HttpResponseMessage resp = client.GetAsync(uriString).Result;
string respString = resp.Content.ReadAsStringAsync().Result;

if (resp.StatusCode==HttpStatusCode.OK){

        ProcessTheRESTAPIResults(respString)

}

switch (resp.StatusCode){

    //process Errors here

    //Note if the error is low level (you never reached the RESTAPI processor the

    //respString is a simple string message

    //if the error is returned by the RESTAPI processor the format is XML

}

OK, that is enough to get started, in the next post we will turn to more complex REST API GET calls and how to process the XML entities returned from successful Sharepoint REST API calls.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

Sharepoint 2013, The REST API and The C# HTTPClient   4 comments

This is a short post to highlight two issues with how the C# HTTPClient is implemented in Framework 4.0 and how to work around these issues.  The issues are the Content Header and the Accept Headers for JSON data.  This post is NOT a full discussion of using the HTTPClient and the Sharepoint 2013 REST API but is limited solely to properly code the Content Header and the Accept Header for this interface.

The new Asynchronous web client   System.Net.Http.HttpClient is a joy to work with.  Recently I was tasked with interfacing with the Sharepoint 2013 REST API using C# (don’t ask why). Most example code for the REST interface are written using JQUERY on the browser.  Since we needed to call the API from within a C# program we attempted to use the HttpClient for this purpose.  Sharepoint Data in the REST interface is in ODATA format.  To use the API we need to declare the Accept Headers for the format of the data we want the ODATA to be formatted by Sharepoint.  If we are using POSTS, PUTS or DELETE with the API we need to declare the Content Type Header to describe the format of the data we are sending to the Sharepoint REST API.  Our choices are:

Data Format Header Value
XML application/atom+xml
JSON application/json;odata=verbose

Setting up the HTTPClient, for all verbs looks like this:

1)System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
2)  _Client.BaseAddress = new Uri(baseURL);

// When the format we are using for incoming data is XML we add this line:

3) _Client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(“application/atom+xml”));

When the format we are using for incoming data is JSON if we replace, in line #3 “application/atom+xml” with the required “application/json;odata=verbose”, Line #3 will thrown an Exception.  The work around is to replace line #3 with:

_Client.DefaultRequestHeaders.Add(“Accept”, “application/json;odata=verbose”);

//and off we go

4) HttpResponseMessage resp = client.GetAsync(uri).Result;
5) string respString = resp.Content.ReadAsStringAsync().Result;

When we are using the HTTP verbs POST, PUT or DELETE we need to send a Request Body with the data we want to send to the server and set a Content Type Header to tell server what format data in the body contains.  The HTTPClient holds the request body in its own object (System.Net.Http.HttpContent):

1) string myData=”your data XML or JSON goes here”;

2) System.Net.Http.HttpContent reqContent = new StringContent(myData);

//We set the Content Type header on this object, NOT on the HttpClient object as:

3) reqContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(“application/atom+xml”);

When the format for HTTPContent is JSON, if we replace “application/atom+xml” with the required “application/json;odata=verbose”, Line #3 will thrown an Exception. The work around is to replace line #3 with:

reqContent.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse(“application/json;odata=verbose”);

//and off we go

4) var resp = _ClientAddItem.PostAsync(addItemURL, reqContent).Result;

5) string respStringOut = resp.Content.ReadAsStringAsync().Result;

Strange but true.  You are welcome.

Ramp Up for QCon 2012 San Francisco   Leave a comment

I am getting ready to attend QCON 2012 in San Francisco, this will be my third QCon SF. You can read my posts for the 2011 edition here and here.  I enjoy QCon and find the tutorials especially useful.  But let me say right off that I don’t attend on my own dime.  In the past I have attended QCon and Microsoft’s MIX conference.  Sadly, Microsoft has decided to kill MIX, this is bad for Microsoft and for the developer community.  I have written posts about MIX here.  Now that MVC 4 is open sourced, one wonders why Microsoft corporate has killed MIX.

QCon is run by the same folks you put out the web magazine InfoQ.  For my money (and its not) in the past QCon schedule is a little long on Agile and Java and not strong enough on JavaScript libraries. The 2012 tracks rectify this problem somewhat. What QCon is strongest is for its embrace of open and free (or almost free) technologies to power the web next year.  This years tracks include a reprise of “Cross Platform Mobile”, “Programmable Web”, “No SQL” (on all three days) and a bevy of tracks on hard core web development ( “Real Time Web” , “UX”, “Taming HTML5 and JavaScript” and “Dynamic Languages for the Web” plus the usual cast of characters: Java, Agile and what looks like a strong three day track of end to end “solutions”. Strangely JQuery is not featured anywhere in this conference.

The tutorials include a full day session on cross platform development  using Phonegap.  Robinson and Weber who presented an excellent and well attended session on RESTful Development will return with a Neo4j programming class.  With excellent timing Peter Bell will present a four hour tutorial on CoffeeScript.  If there was a MIX 2012, they would be talking about how MVC4 is supporting CoffeeScript (sigh).  Track and tutorials can be found here and here.  Note the venue has also changed this year from the centrally located, and very urban, Union Square area of SF to the more up scale Embarcadero area. For folks who like to explore the city, as opposed to pub crawling, this is a bummer.  But it is ocean view and close to the (now departed) Occupy SF site.

Well, that’s the basic facts, now the question is: Should YOU attend QCon 2012?  The best part of QCon is that it is not a standard vendor conference and (except for Agile) nobody is trying to sell you anything.  The attendee’s tend to be working programmers from startups in the SF area and from Europe which helps the sessions to be very focused on what really works. for programmers and startup firms.  There is no focus on mega systems like Oracle products or SharePoint.  In this environment, couchdb (and this years bad boy Neo) are about as establishment as you get. QCon has traditionally been very friendly towards NO SQL databases and has been I consistent good sorce for information on this topic.  This year promises to continue in this trend.   I regularly avoid pure vendor sessions so of the three days of regular conference sessions I can look forward to about 2.5 days of good sessions.  The tutorials that I have attended are top notch and honest.  I can not evaluate the daily keynote speeches since I am normally sleep in past these.  Attendees have told me the keynote party and mixer is nice but I am always in the Haight district when this happens on Wednesday night.  If the boss is paying and you are coding the web and web devices you must attend this conference.  If it’s your dime….

I will be bloging QCon 2012 on my Nexus 7
%d bloggers like this: