Archive for the ‘Sharepoint’ Category

Sharepoint 2013 REST API: Retrieving List Item Data   1 comment

I first discussed retrieving List Item data here: Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data.  The simple REST API call:


functions like

select * from Table1

in a SQL data base.  However Sharepoint, although resting on SQL Server for everything, will only return the a limited number of rows from the underlying list.  The maximum number of items (rows) returned in a single call is governed by a system constraint.  On our out of the box Sharepoint 2013 installation the limit was 100 rows!  Supposable Sharepoint Sever Admins can change the maximum value but they can not eliminate the constraint.  The naïve coder might think that this can be gotten around using the $SKIP parameter in the ODATA extensions to the Sharepoint REST API.  Alas $SKIP is not implemented, instead M$ implemented it own undocumented skip parameters.  No to worry you don’t need to provide your own parameter values.  The initial call using


returns X rows of the table (in primary key index order) where X is equal to the page constraint discussed above.  If there are additional rows the metadata in the returned data structure will return a “next” hypermedia link whose value is fully qualified Sharepoint Rest API call for the next X rows (i.e. the next page). 


                               Google Barge Sinking Slowly in San Francisco Harbor after being infected with the Sharepoint 2013 Virus

We need then to setup a typical paging object to read items by page until all items are read.  I will be leveraging the code patterns given in the series referenced above the page calls (…/items) return results in XML format and is cast as an XDocument type.  Then we can probe for a “next” link as:

static string _GetLinkToNextPage(XDocument xDoc)
const string matchString = “/Web/Lists”;
var links = from g in xDoc.Descendants().Elements(CSPNamespace.baseNS + “link”)
select g;
const string NEXT = “next”;
foreach (XElement link in links)
string rel = string.Empty;
string href = string.Empty;
foreach (XAttribute att in link.Attributes())
if (att.Name.LocalName == “rel”)
rel = att.Value;
if (att.Name.LocalName == “href”)
href = att.Value;

if (rel == NEXT)

                   //return just the relative path
return href.Substring(href.IndexOf(matchString));

           //return null if no link is found
return null;

The complete object template looks like:

public static class CItemPage
public static List<CEntry> _Worker(HttpClient client, string next)
public static List Exec(HttpClient client, CEntry listMetaData)
       static void _Accumulate(XDocument items, ref List allItems)
static string _GetLinkToNextPage(XDocument xDoc)
static XDocument _GetListMetaData(HttpClient client)


The entry point for the paging activity is CItemPage.Exec which turns around and calls worker where we loop through the data and (for my own purposes) I then accumulate the items from all pages in a single list<CEntry>  using my method _Accumlate.  calls Worker

       public static List _Worker(HttpClient client, string next)
List allItems = new List();
XDocument items = null;
while (next != null)
items = _GetItems(client, next);
next = _GetLinkToNextPage(items);
_Accumulate(items, ref allItems);
return allItems;
catch (Exception x)
var b = x.Message;
return null;


Easy and fun.  Should there be an easier, faster way? yes.


This madness must stop.

Sharepoint 2013 REST API: Testing for the Existence of a Document In a Document Library   Leave a comment

This is just a short program note concerning the REST API for Sharepoint 2013.  If you want to test for the existence of a Document within a Document Library in Sharepoint you might be tempted to use this uri fragment:

web/getfilebyserverrelativeurl(‘{document library name and folder path }\{document name}’)”

and indeed if the document exists this call will return an HTTP Status 200 code and the standard ATOM feed with one entry node with the meta data for the document.

<!–?xml version=”1.0″ encoding=”utf-8″?>
xml:base=”” xmlns=”” xmlns:d=”” xmlns:m=”” xmlns:georss=”” xmlns:gml=””>
[many other links]


<d:ServerRelativeUrl>/DLR Document Library/_f37-06_tc_big.svg</d:ServerRelativeUrl>

                [many other properties]


Now here is todays puzzle.  If the file does not exist, what does Sharepoint return for this call:

  1. Http Status Code 200 and an Atom Feed with no entry nodes;
  2. HTTP Status Code 404 (Not Found) and no Atom Feed; or
  3. HTTP Status Code 500 (Internal Server Error)
If you selected answer 1 you have been programming Sharepoint 2013 awhile but are still quite naïve.

If you selected answer 2 you are a RESTafarian and have no business programming Microsoft products.

If you selected answer 3 you have been programming this interface for way to long and should find another line of work.

The correct answer is 3!  Don’t ask me why.

To be consistent we would like to query Sharepoint 2013 and get an Atom Feed if the file exists or not and expect that the feed will have zero entry nodes when the Document does not exist.  How to do this?  Query the Document Library and folder path using a completely different uri fragment  for all files and use an OData filter to limit the returned Atom Feed to zero entry nodes with the document does not exist. Here is an example:

web/GetFolderByServerRelativeUrl(‘{document library name and folder path }’)/Files?$filter=Name eq ‘{document name}’

Now you will get an HTTP Status code of 200 whether the document exists or not.  The Atom Feed will be returned in all cases but  will differ depending if the Document exists (one entry node ) or if the Document does not exist (no entry nodes).

Strange but true.  Don’t let this error happen to you.


Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends   4 comments

Since our last post was so long I left a few odds and ends for this post.  Specifically I will touch on the following in this posting:s_w19_1a35314u

  • Testing for the Presence of a Document within a Document Library
  • Testing for the presence a Folder within a Document Library
  • How to Create a Folder within a Document Library; and
  • How to Create a custom HTTP Exception class derived from the base Exception Class

Testing for the Presence of a Document within a Document Library

As with all things Sharepoint the most important part of this task is composing the correct uri fragment.  We need to have three pieces of information to preform the test:

  • The Document Library Name
  • The Folder Path (if any) within the Library
  • The Document Name (root plus extension)

We then compose a uri fragment as:

web/GetFolderByServerRelativeUrl(‘/{Document Library/FolderPath’)/Files(‘{Document Name}’)

So if we are testing for a Document myDocument.PDF in the folder path  AdminFolder\ClaimsFolder in a Document Library Call Accounting Documents our uri fragment becomes:

web/GetFolderByServerRelativeUrl(‘/Accounting Documents/AdminFolder\ClaimsFolder’)/Files(‘myDocument.PDF’)

One then makes an HTTP Get call against the Sharepoint REST API.  An Http Status Code of OK (numeric value: 200) indicates that the file exists.

An Http Status Code of NotFound (numeric value: 404 ) indicates that the file is not found at that location.

This get call does NOT return the document itself to the caller.

Testing for the presence a Folder within a Document Library

This test is even simpler.  We need two pieces of information:

  • The Document Library Name
  • The Folder Path (if any) within the Library

We compose the uri fragment as:

web/GetFolderByServerRelativeUrl(‘/{Document Library Name\Folder Path}’)

One then makes an HTTP Get call against the Sharepoint REST API.  An Http Status Code of OK (numeric value: 200) indicates that the folder path exists.  An Http Status Code of NotFound (numeric value: 404 ) indicates that the folder path is not found within the library.s_w04_1a35329u

How to Create a Folder within a Document Library

In order to create a folder we need to:

Compose a proper uri fragment;

Compose a JSON formatted content body (System.Net.Http.HttpContent);

Get a REST API Digest Value (See Part 3 of this series on this) and include it in the header; and

Make a HTTP POST call to the Sharepoint REST API

So here we go.  The uri fragment takes the simple fixed form of:


The JSON HTTPContent format does the real work and takes the form of:

[ ‘__metadata’: [ ‘type’: ‘SP.Folder’ ], ‘ServerRelativeUrl’: ‘/{Document Library Name\Folder Path}’]

So if our Document Library is “Accounting Documents” and your folder name is “ClaimsFolder” our JSON looks like:

[ ‘__metadata’: [ ‘type’: ‘SP.Folder’ ], ‘ServerRelativeUrl’: ‘/Accounting Documents/ClaimsFolder}’]

Having placed this value into a string object as:

string data=”[ ‘__metadata’: [ ‘type’: ‘SP.Folder’ ], ‘ServerRelativeUrl’: ‘/Accounting Documents/ClaimsFolder}’]”;

we create a  HTTPContent object as

System.Net.Http.HttpContent reqContent = new StringContent(data);

After adding the correct Digest Header and ContentType Header our post looks like:

var resp = client.PostAsync(uri, reqContent).Result;

string respString = resp.Content.ReadAsStringAsync().Result;

If the folder is created successfully we will get back an HTTP Status of Created (numeric: 201 )

To create a nested folder just expand the path within ServerRelativeUrl.

New lets turn to a topic that does really fit any where else but I will just stuff it in here.

How to Create a custom HTTP Exception class derived from the base Exception Classs_w01_1a35360u

HTTP calls are parameterized calls and when they fail there is some information we may wish to attach to the Exception object created which we create on errors.  Specifically:  The URL which was called, the returned HTTP Status Code and the Response Body. Here is a simple derived HTTP exception class which I use:

public class HTTPException : ApplicationException
public string URL { get; set; }
public string Response { get; set; }
public HttpStatusCode Status { get; set; }
public HTTPException(string message, HttpStatusCode status, string uRLString, string respString)
: base(message)
URL = uRLString;
Response = respString;
Status = status;

public HTTPException(string message, HttpStatusCode status, string uRLString, Exception innerException)
: base(message, innerException)
URL = uRLString;
Response = string.Empty;
Status = status;

Assume a typical HTTP method call like:

public static HttpStatusCode RestGet(System.Net.Http.HttpClient client, string uri,List allowed, out string respString)
respString = string.Empty;
HttpResponseMessage resp=null;
resp = client.GetAsync(uri).Result;
respString = resp.Content.ReadAsStringAsync().Result;
_DisplayDebugInfo(client, resp, uri, null, null, respString);
catch (Exception x)
throw new HTTP.Exceptions.HTTPException(“RestGet”,  HttpStatusCode.ServiceUnavailable,                       client.BaseAddress.ToString() + “/” + uri, x);

if (statusCode != HttpStatusCode.OK)
throw new HTTP.Exceptions.HTTPException(“RestGet”, statusCode, client.BaseAddress.ToString() + “/” + uri,respString);

return statusCode;

We can pick this up in a outer try/catch block like:


//make your HTTP call here

}catch(HTTPException xo){



}catch(Exception x1){



Ok That’s it for the REST API and the Client HTTP object.  Next up:  Preparing a WEB API Endpoint to be called FROM Sharepoint 2013 Workflow.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files


Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files   4 comments

The Document Library is a list like all things within Sharepoint, but it is a list with a difference. Each list item within a document library is associated with one Document.  To access the document through the user agent (the browser) we can address the item “directly” using the simple URI:

https://{server name}/{document library name}\{folder name(s)}\{document name}.{extension}

Note: I will use the convention of using the brackets {} to indicate parameters which are supplied when you instantiate a line of code.  Do NOT include the brackets in your code.

This will conjure up the document and launch an appropriate viewer application (Adobe Adobe, MS Word or what have you).  This type of addressing will not work when we are using the REST API for uploading or downloading a document. Although we ignored the issue of folders within lists in our discussions so far we need to address the issue here.  Keep in mind however that folders can appear in normal (non-Document Library) lists also.  I will defer didscussion of creating folders and deletes of folders and documents for a subsequent post.  In this post let’s work on the downloading documents  and then look at how to upload documents.  As with all things Sharepoint, it doesn’t work how you might think.

Document Library Item Details

If we use the REST API and retrieve a list all items within a document library in the normal manner (see Part II for details) our collection of  entry elements looks normal each entry has a collection of link nodes and (within the content node) a collection of property nodes.  Out of the box the properties seem straight forward.

Document Library Item Node

Note that the Document associated with this item does NOT appear in the list of properties.  But we can get that with a second REST API call.  If we expand the d:FileSystemObjectType properties for the item pictured above (which is a normal item) we see that its value is 0.  If the Library has folders then the folder is also listed as an entry in the entry collection along with document items.  Here is how one such entry item for a folder looks:

Folder2 Item

Looks the same right?  If we expand the d:FileSystemObjectType its value is 1. The “documentation” for FileSystemObjectType is obsure and indicates that this property is “An enumeration value that indicates the type: file, folder, Web, or invalid”.  So apparently 1 indicates a folder item while our observed value of “0” indicates a file item. Or something.  Don’t be confused by the title on this item, the name I gave to this folder is Folder0.0. Moving on, having an item which references a file (Document) we can make a call using the link “FieldValuesAsText” we can get a second item entry with the actual document reference:


Here we have the file name (d:fileLeafRef) the Document Library (d:fileDirRef) and the relative path for this item (d:FileRef).  It is the later value with we get when we use the Publication Hyperlink field in a List Item.  Note that d:FileRef will include any folders that the document is nested within. In this case the PDF file called “Screen clipping taken 9152013.pdf” is in the root of the Document Library “DLR Document Library”.

Document Down Load

In order to download a file we need to retrieve the binary byte stream for the document using the REST API.  Here is the basic idea.

Build a  basic uri fragment on this format:

web/GetFolderByServerRelativeUrl(‘/{path}’)/Files/$value?$filter=Name eq ‘{document name}’


{path} is the Document Library and path found in d:fileRef excluding the document and extension.

{document} is the value found in d:fileLeafRef (document Name plus extension)

The $value directs Sharepoint to prove the binary bytes for this document.

As an alternative we can form a uri fragment in this form:


Which is the value of the “EDIT” link from the links collection of the document library item and appending


Now call a REST API get against this uri fragment using a HTTPClient prepared as discussed in Part I with the addition of a header as follows:

client.DefaultRequestHeaders.Add(“binaryStringResponseBody”, “true”);

//The header tells Sharepoint to give us an uncontaminated binary stream

HttpResponseMessage resp = client.GetAsync(uri).Result;

Now we need to read the response body (which is binary format) into a binary MemoryStream:

MemoryStream ms=new MemoryStream();
ms.Position = 0;

This memory stream can be passed to a file write routine to persist the document to disk or in our case can be used to upload a document to a different Sharepoint site.

Document Upload

To upload a document into a Document Library the procedure is the same but different.  The Tasks are:

  • Get the document into a  MemoryStream object;
  • Prepare an HTTP Client Object for an HTTP Post
  • Get A Digest Object From Sharepoint using a REST API Post
    • Code Method: RetrieveDigest & RESTPost
    • The LINQ code to extract the Digest value from the response is given in the ctor method of the class:  CDigest
  • Prepare a Posting URI fragment
  • Prepare A uri fragment to use to POST the binary data to the Document Library
  • Prepare an HTTP Client Object for an second HTTP Post
    • Code Method: GetHTTPClient
    • Code Method: _PutBytes & RESTPostBinary
      • Set a Header containing the Digest Object
      • Prepare a Response Content  with the binary data from the memory stream
      • Set a Response Header with the proper Media Type on the Response Content Object
      • Set a Response Header telling Sharepoint you are sending a binary object in the ‘body’
      • Attach the Content Body to the HttpClient Object
      • Call the Post Command

and “like magic”, your document is created and attached to the Sharepoint like and a file Item is created in the Document library as a side effect of the upload.  Wow.  Lets fill in the details for this process.  I will assume you have created and populated the Memory Stream object with the contents of your binary file.  I will show a full code sample at the end of this post.

Get a Sharepoint Digest Object

In order to modify data within Sharepoint using the REST API you must request a Digest object from Sharepoint and attach this Digest item to your post command.  To get the digest value we must make an HTTP POST call to a special Sharepoint REST API Endpoint:


The actual Digest value is stored in the property field: FormDigestValue and its duration is given in the field: FormDigestTimeoutSeconds.  If your program is long running you may need to get a fresh copy of the Digest to use occasionally.  My code to retrieve a Digest looks something like this:

public static SPObject.CDigest RetrieveDigest(HttpClient client)
string respString;
statusCode = RESTPost(client,”contextinfo”);
if (statusCode == HttpStatusCode.OK)
return new SPObject.CDigest(respString);
throw ApplicationException(“Something Bad Happened”);
public static string RESTPost(System.Net.Http.HttpClient client, string uri)
var resp = client.PostAsync(uri, new StringContent(string.Empty)).Result;
respString = resp.Content.ReadAsStringAsync().Result;

            if (resp.StatusCode!=HttpStatus.OK){

                 throw new ApplicationException(“Something Bad Happened”);

return respString;

public class SPObject
public class CDigest
public string Value { get; set; }
public string TimeOutSeconds { get; set; }
public System.DateTime GetTime { get; set; }
public CDigest() { }
public CDigest(string respString)
XDocument xResponse = XDocument.Parse(respString);
IEnumerable digestList = from g in

                     xResponse.Descendants(CSPNamespace.dataServicesNS + “GetContextWebInformation”)
select new CDigest
Value = g.Elements(CSPNamespace.dataServicesNS + 

TimeOutSeconds = g.Elements(CSPNamespace.dataServicesNS +

GetTime = System.DateTime.Now

CDigest digest = digestList.First();
Value = digest.Value;
TimeOutSeconds = digest.TimeOutSeconds;
GetTime = digest.GetTime;

Uploading The Binary Data with an HTTP POST

Now that we have a Digest object we need to compose our uri to post the data as a binary byte stream into the document library.  According to this Microsoft Documentation  (11/1/2013) the basic uri fragment is composed from this form:

web/GetFolderByServerRelativeUrl(‘{path}’/Files/add(url='{document}’,overwrite={true | false})


{path} is the Document Library and any folders.

{document} is the document and its extension.

For example for a document yourClaims.PDF within the Claims folder within the document library called My Library and we allow file overwrites,  the uri fragment becomes:

web/GetFolderByServerRelativeUrl(‘My Library/Claims/’/Files/add(url=’yourClaims.PDF’,overwrite=true)

Now we need to add the Digest Value to a Header of the HTTPClient object:

client.DefaultRequestHeaders.Add(“X-RequestDigest”, {digest values as an unqouted string});

Ok so far so good.  You can see my code example below in the method _PutBytes.  In order to POST data we need to prepare a  response body.  In C# we do this with the special object: HttpContent as:

HttpContent reqContent = new StreamContent(ms);

where ms is the open memorystream of the binary bytes of our document to post. We need to add two specialized headers to this object before we post:

reqContent.Headers.Add(“binaryStringRequestBody”, “true”);
reqContent.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse(“application/json;odata=verbose”);

Now we are ready to use the HTTPClient PostAsync method to POST our document to the server.  See my method RESTPOSTBinary below for a coding example.

void _PutBytes

( CENTRYTranslate data, MemoryStream ms, HttpClient clientWriter,string fullpath, string documentName)
SPObject.CDigest digest = AIC.Http.Client.CHttpObject.RetrieveDigest(clientWriter, out statusCode);
string URLCreateDocument =

“web/GetFolderByServerRelativeUrl(‘{path}’/Files/add(url='{document}’,overwrite={true | false})”

           clientWriter.DefaultRequestHeaders.Add(“X-RequestDigest”, digest.Value);
string respString = CHttpObject.RESTPostBinary(clientWriter, URLCreateDocument, ms);

            if (statusCode != HttpStatusCode.OK)
throw ApplicationException(“Something Bad Happened”);


public static string RESTPostBinary(System.Net.Http.HttpClient client, string uri,MemoryStream ms)

         HttpContent reqContent = new StreamContent(ms);
reqContent.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse(CStaticClass.CONTENTTYPEHEADERJSON);
var resp = client.PostAsync(uri, reqContent).Result;
If (resp.status!=HTTPStatus.OK){

              throw new ApplicationException(“SOMETHING BAD HAPPENDED”);


         return resp.Content.ReadAsStringAsync().Result;

Easy and fun, no?

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data   5 comments

Now that the administrative details of Part 2 are over we can now address some useful issues. Recall that if we query a List we can obtain the uri segment needed to access the items in the list.  This for example would return all items the referenced list:


Please see this post for an important update about data pages when using the …/Items call

If we don’t want all the items we can apply an OData $filter verb to select a subset of items matching some criteria. If we want a subset of items with a specific Author we could create the following fragment:

Web/list(guid’850fae4-0cce-8c30-c2a85001e215’)/Items?$filter=Author eq ‘Cloud2013’

If you are concerned about download sizes of your calls into the REST API you can limit the property fields returned using the OData $select verb.

The feed returned from a GET call with an uri fragment like that above has an identical structure to the feed discussed in Part 2 of this series. Depending on the list and the $filter there may be zero, one or many entry nodes.  Each entry node contains information on one Item with its corresponding Links collection and properties collection.  In this case the Link collections refers to operations which can be made this single item, for example the “ContentType” Link (Web/Lists(guid’850f0ae4-0cce-47e9-8c30-c2a85001e216′)/Items(3)/ContentType) would return a feed with an entry for links and properties associated with the content item for this item.  The properties collection is, of course, the fields for an item and the values associated with that item.   Here is a Quick Watch screen snap of the data for an item:


This looks pretty normal.  The field “Title” appears with its value “Points To Approved Item”.  Some of the field names have been Sharepointerized. Our field “AIC Description” has become “AIC_x0020_Description”.  What is going on here?  According to this post:

When you create a column on a list, both its DisplayName and StaticName are set to the same value. However, the StaticName contains converted values for some characters, most notably a space ‘ ‘ is converted to '_x0020_'. So if the DisplayName is ‘Product Description’, then the StaticName will be 'Product_x0020_Description'.

There’s another little bugaboo: The StaticName is limited to 32 characters including the translations for special characters.

In translation, the DisplayName is the name you see on the Sharepoint UI screen.  The value of the display name can change.  When the column is created a fixed “StaticName” is created equal to the initial DisplayName with spaces converted to _x0020.  When working with the REST API we are seeing the StaticName. Ok so far?  We have another problem some fields are missing!  The REST API does not support all columns using the


format above.  This interesting post has more details. 

So we will need to do some more digging Sharepoint has to store the value somewhere.  Well, in the true spirit of Sharepoint we have to say: it depends.  In our case we us a Publishing Hyperlink field.  This is a field which allows a list item to contain a reference to some other object.  That object can be:

  • an external Web URL
  • a Sharepoint Page; or
  • A Document stored in a Sharepoint Document Library

BTW, in the first instance the whole URL is stored and in the later two instances only the RELATIVE link is stored since these are assumed to point to a SP page or document within the current site.  At our site we need to retrieve the true value of the Publishing Hyperlink.  And as you can see from the above the field (which we call “AIC Publishing Hyperlink”) does not appear in any form in the original REST call.  If we jump to the Links collection for an item we can find a link called:  FieldValuesForEdit. In our case this is:


If we call that link we get back an entry which in Quick Watch looks like:


OK, there it is, hiding in Sharepointification form as “AIC_x005f_x0020_x005f_Publishing_x005f_x0020_x005f_Hyper” (where did the _X005f come from you might ask. I have no idea, some things in SP I just accept).  The value is a HTML anchor tag and it is pretty simple to parse these to recover the href value.  Which in this case is  (after decoding) “/DLR%20Document%20Library/1969.html” where ““/DLR%20Document%20Library/” points to a Sharepoint Document Library and 1969.html as the Document.  We are working on how Managed Metadata values are stored but our research is not yet complete on this.  In my next post I will discuss how to download and upload a document from a document library (which is how I fell into this problem of retrieving the value stored in a Publishing Hyperlink field.

Ok, we are almost done for today.  Let’s look at a security issue.  What values you see for a field on an item depends on the identity of user making the REST API call and the state of the item within Sharepoint when you look at it (it’s a Quantum thing).   Recall the publication settings for a list item in Sharepoint. This screen snap may help your memory:


When we enable Content Approval and set “Who should see draft items” to Only users who can approve items (and the author of the item)” then an item can exist in two states at once.  If we have an approved item then everyone sees the same thing:


No if an editor changes an item, say by changing the content of the “Description” field and the item is waiting for approval the item is in a Pending Approval Status like this:


If you are a general user you see the values in the Approved item.  If you are an approver or the author you see the values in the Pending Item.  As in the UI so in the REST API.  The values returned to you in your REST API call depend on the Sharepoint security group of the identity you use to access the item and the list settings for approval and visibility. So as in all things, be careful what you ask for.  In some use cases you may want to see pending values and in others you may want to see only approved values.  There are use cases, we have one in workflow processing for a list where we want to see both.  In this case we use two identities to access the same list item and so can compare the pending and approved items.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

Coming up: Document processing!

The Google Barge In Portland Maine.  Don’t believe the cover story.  It’s a 3D printer which can only produce other 3D printer barges!

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data   4 comments

Let’s assume that we have an HTTPClient object as described in Part 1.  That is we have

  • created the HttpClient objected,
  • set the base URL to our Sharepoint site, 
  • set the Authentication Header; and
  • set the Accept Header for  “application/atom+xml”. 

Now we want to query the REST API for the metadata of a Sharepoint List.  Sharepoint List Titles are unique by design we can use the List Title (rather than the GUID) to locate the item and return its metadata. The format of this call as a uri fragment is:


Now our code looks like:

string uri=”web/lists/GetByTitle(‘Master Document Library’)”;

//Note that spaces are allowed in most calls but there are other situations where spaces are escaped!

HttpResponseMessage resp = client.GetAsync(uri).Result;

string respString = resp.Content.ReadAsStringAsync().Result;

if (resp.StatusCode != HttpStatusCode.OK)


throw new ApplicationExcepiton(

                    “HTTP Error. Status: {0} Reason: {1}”,resp.StatusCode,    resp.ReasonPhrase );         


This will put a string representation of the XML formatted metadata about the list into the HttpResponseMessage.  Please note that the call to extract the XML from the response body:

string respString = resp.Content.ReadAsStringAsync().Result;

is only appropriate for string results (XML or JSON as specified in our Accept Header) and is not correct if the return is binary data.  I will copy binary data in a subsequent blog when I discuss file upload and download.

Few things are less documented than the exact contents of the metadata feed returned by calls like this from the REST API.  On a high level it is an Atom feed which allows for a combination of collections of  Property nodes (key/value pairs) and collections of Link nodes. The Property Nodes are the meta data fields related to the list and the Link Nodes are uri segments to guide additional REST API calls concerning what ever item is currently being queried.  Neither the properties nor the links are a fixed are vary from Sharepoint object to object and even vary between objects of the same type if those object depending on the values of the property fields (for example a list item on a list with versioning will contain a Link node for access to the versions.  If the list items are not versioned then the link item of versions will not be emitted.

Rather than list XML directly I will use the visualization tool XMLSPY to display the XML in a “grid view”.  On a high level the entry for a list would look like:

Untitled picture

The Links are on the root of the entry node and the properties are nested as entry/content/properties.  Note that the XML makes heavy use of XML namespaces and any creative manipulation of the entry requires some knowledge of XML, namespaces,  XPath or LINQ for XML.  I use LINQ for XML at my desk so I will use that idiom rather than XPATH to manipulate these objects.  If we expand the properties node it will look something like this:

Untitled picture2

There is a lot of data here about the list most of it only lightly documented.  We can see however that the property key: d:title contains the name of our list which we queried on and d:iD contains the GUID for the list.  The later never changes but the former can be renamed.

If we expand the Links collection  it would look something like this:

Untitled picture3

Note item 10, the Items link. The href attribute contains the uri for the REST API to retrieve all the items in this list, while the Fields link (item 7) is an uri for the Fields currently defined for this list. If we know the d:Id of a particular item (item IDs are not GUIDS for simple integers), say 6, we can retrieve a single item with the uri of the Items uri and post pending in the form of:


What about the link with a blank title? For historical reasons this is blank but it represents the EDIT link.  To make my life simpler I translate the XML property and link collections into C# Dictionary objects and place them in a C# class with two supporting methods:

public class CEntry


public Dictionary<string, string> Links;

public Dictionary<string, string> Properties;

      public string  GetLink(string key){

string value = string.Empty;

Links.TryGetValue(key, out value);

return value;


      public string GetProperty(string key)


string value = string.Empty;

Properties.TryGetValue(key, out value);

return value;



At this time I am not using any of the root nodes so I just discard them. I get to the Dictionary objects from the XML using LINQ for XML.  I learned what little I know about LINQ for XML from this book.  To brush up on your XML try this book.   For a XML tree containing entry node(s) my LINQ looks like this:

public static class CSPNamespace


public static XNamespace metaDataNS = @””;

       public static XNamespace baseNS = @””;

       public static XNamespace dataServicesNS = @””;


XDocument xList =XDocument.Parse(respString);

IEnumerable<CEntry> group = from g in xList.Descendants(CSPNamespace.baseNS + “entry”)

select new CEntry


           Links = MakeLinkDictionary(g),

Properties = MakePropertyDictionary(g)


The IEnumerable collection needs special processing before it is accessed.  The following test can help:

To see if the collection contains one or more entries:

group != null && group.Any();

Having passed that test, we can then use the simple Count function to see how many entries are in the collection.


To get the first (or only) entry from the collection


These last two test will fails if the collection fails the test above

        CEntry cell = group.First();  //Assumes one and one only

Where MakeLinkDicitionary and MakePropertyDictionary look like:

public  static Dictionary<string, string=””> MakePropertyDictionary(XElement xs)


Dictionary<string, string=””> pList = new Dictionary<string, string=””>();

var group = from g in xs.Elements(CSPNamespace.baseNS + “content”).Descendants(CSPNamespace.metaDataNS + “properties”)

select g;

foreach (XElement property in group.Elements())


pList.Add(property.Name.LocalName, property.Value);


return pList;


public  static Dictionary<string, string=””> MakeLinkDictionary(XElement xs)


       Dictionary<string, string=””> lList = new Dictionary<string, string=””>();    IEnumerable links = from g in

                      xs.Elements(CSPNamespace.baseNS + “link”)

select g;

foreach (XElement link in links)


string rel = string.Empty;

string href = string.Empty;

foreach (XAttribute att in link.Attributes())


if (att.Name.LocalName == “title”)


                   if (string.IsNulOrEmpty(att.Value)){

                      rel = “Edit”;


rel = att.Value;



if (att.Name.LocalName == “href”)


href = att.Value;



lList.Add(rel, href);


return lList;


After this pre-processing the meta data can be accessed in a fairly straight forward manner.

var listMetaData=group.First();

string uri=listMetaData.GetLink(“Fields”);

string iD=listMetaData.GetProperty(“Title”);

We will turn to what to actual do with the meta data in the next post.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient   13 comments

This is the first of a multipart series on the Sharepoint REST API and will focus on using this API with C#.  At our shop we need to manipulate documents and list items between sites.  For our purposes we will be calling a custom WEB API HTTP Endpoint from Sharepoint workflows, these endpoints will then manipulatete Sharepoint objects using the Sharepoint REST API using C#.  If REST means nothing to you but a good sleep start here. The topics we will cover include:

  • HTTPClient Object (Framework 4.5)
  • Understanding and Manipulating the Result Sets returned by the REST API (Here)
  • Downloading and Uploading Documents from Document Libraries Using The REST API
  • Developing WEB API Endpoints To package our units of work (Web API 2.X)
  • Calling the WEB API Endpoints from Sharepoint Workflows using GET HTTP Web Service Action

I will not be looking at calling the REST API directly from within Sharepoint Workflows.  How many blog posts will this be?  I am not sure probably three posts in total.

Getting Our Feet Wet:  What an HTTP GET request to the Sharepoint REST API Looks like (10,000 foot view)

The new async HttpClient is a great advance over previous Microsoft Http clients.  You need to understand something about formal HTTP communications.  Specifically the roles of the different HTTP Verbs, Request Headers, HTTP Errors, and for POST commands, Response Headers and the Response Body.  So lets begin.  When we make AJAX calls in JavaScript all of our calls are passed through the machinery of the User Agent (Chrome, Firefox, etc.) and if you are using a helper library like JQuery, the helper library is doing a lot of the work for you.  When you switch to C# and the HTTPClient there is no User Agent and the client itself is quite thin so you have to a lot of work to get a well engineer solution.  So lets begin with some of the basics of an HTTP Get request.tumblr_ls7xrkHeuq1qchzcpo1_1280

There are four components of an HTTP Get request to be concerned about:

  • Target URL
    • What endpoint is the request headed for
  • Query String
    • What, if any, arguments are passed to the target
  • Accept Header
    • The format the results should be written back to the client
  • Authorization Header
    • What identity is the request being make under

For a Sharepoint REST API we want the site URL for the target URL and append to this \_api\.  So we start with something like:

In a non-RESTful interface the query string takes the form of:


In a RESTful interface this becomes appended to the base URL itself as something like


Microsoft’s design for the REST API is a little eccentric and does not follow this pattern. A REST API call to get the meta-data for a list (not the list items) can look like this:


where: {myserver}/_api/ is the baseURL and web/lists/GetByTitle(‘MyList’) are the arguments and values used in the call.

A GET call for a List, or any Sharepoint object returns a very complex objects whose structure is (basically) identical for different object types so once you understand the basic structure of the response it applies to many different objects.  We will not discuss that response object from a Sharepoint REST API Get request in this post. Note here that objects.  Anywhere the REST API allows an argument to be enclosed in single quote marks we may use an argument value with embedded spaces with out escaping the spaces.  Of concern to us at this point is the format of the response to the call. We have two choices XML format or JSON format.  When we are in the browser world and are using JavaScript to process the response, typically JSON is the preferred format, but XML can also be processed using JavaScript and assorted helper libraries.  When we are using a language like C# we can work most effectively with XML output.  In my case I prefer to receive XML and process that response using LINQ for XML.

Authentication for the Sharepoint REST API comes in three flavors:  OAuth token, Basic Authentication and (Windows) Integrated.  OAuth token only applies if you are calling from a Sharepoint APP (which means you are using JavaScript by definition) or from a Sharepoint Provider-Hosted High Trust App. (a very rare beast indeed and if you are using one you probably don’t need to be reading this post).  I will be concerned here with Basic and Integrated Authentication.

Now Lets Look at some Details Of A Sharepoint REST API call using C#

The HTTPClient object

Simple Anonymous HTTP GET call using HttpClient

        System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
_Client.BaseAddress = new Uri(baseURL);
HttpResponseMessage resp = client.GetAsync(uriString).Result;
string respString = resp.Content.ReadAsStringAsync().Result;
        if (resp.StatusCode != HttpStatusCode.OK)
throw new ApplicationException(“BAD”);

Where baseURL is a string representation of the site we are calling into for the REST API:


urrString is a string representation of the actual REST API call we which to make:


Since we are not setting an authorization method we are calling anonymously and this call in its current form would return a failure HTTP Status and the response body will contain information in the REST API default return format.  The HTTP status is returned as a Framework Enum System.Net.HttpStatusCode.  In formal terms HTTP status codes are numeric values.  If you want to recover the actual value you can cast HttpStatusCode to an int.  Thus  HttpStatusCode.OK cast as an integer is 200. This call as written wourld return HttpStatusCode.Unauthorized ( 401).

We need to associate an identity to the HttpClient object in order to get the REST API to return data to us.  I will discuss two ways to do this  (of the many options):

BASIC Authorization and Windows Integrated Authorization.  Basic Authorization depends on adding an Authorization Header to the client with  appropriate  user information (typically domain, user name and password) in a very specific format.

Header Name:  Authorization

Argument: BASIC domain\username\password

where domain\username\password are encoded as a Base 645 String:

static string _FormatBasicAuth(string domain, string user, string password)
const string format0 = @”{0}\{1}”;
const string format1 = @”{0}:{1}”;
string userName = string.Format(format0, domain, user);

return Convert.ToBase64String(Encoding.Default.GetBytes(

               string.Format(format1, userName, password)));


We then create the Authorization header on the HttpClient object:

System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();

_Client.DefaultRequestHeaders.Authorization =

         new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(Domain, User, Password));

Note that the password is NOT encrypted so BASIC Authorization should only be used over a secure transport layer like SSL.

An alternative is to use Microsoft’s Integrated Authorization (also know as challenge and response).  This takes  network credentials (either of the active thread or created from initialization values) and allows the server to confirm the identity and validity of the caller without transmitting the data over a secure transport layer (‘that password it self is never passed over the wire’).  Here is an example using Network Credentials created on the fly:

_Client = new System.Net.Http.HttpClient(new HttpClientHandler()
new NetworkCredential(User, Password, Domain)

Alternatively the NetworkCredential object can be created from the running thread of your C# program (you do not need to supply the password  and other parts of the credentials explicitly in this case).  When set up this way you do not use an Authentication Header.  The Windows challenge and response pattern (Http Status 401) are controlled on the client side by the HttpClientHandler.

Assuming your REST API call is well formed and your credentials  are valid the response from the server will contain a representation of a Sharepoint entity (or entities).  I will discuss the structure of this data in a subsequent post.  You can control how the response is formatted by adding an Accept Header to the HttpClient Object.  For the REST API you have two choices: XML or JSON.  If you do not supply a header the default format is XML.  For the REST api both the XML and JSON take a very specific format in the Accept header.  XML requires a Accept Header (if supplied) with an argument of:


while JSON requires an argument of:


The Accept Header for XML can be added to the HTTPClient as:

  _Client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));

Currently there is a bug in the HTTPClient which prevents adding an Accept Header for REST API formated JSON.  For JSON you need to use a different format:


Normally I use the XML format of the response for C# programs (although using JSON.NET you could process the return in JSON format).  Javascript coders will normally us the JSON format (although they could process the XML format).

My typical HTTP Get calls looks something like this:

System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
_Client.BaseAddress = new Uri(baseURL);

             new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));

_Client.DefaultRequestHeaders.Authorization =

         new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(Domain, User, Password));

HttpResponseMessage resp = client.GetAsync(uriString).Result;
string respString = resp.Content.ReadAsStringAsync().Result;

if (resp.StatusCode==HttpStatusCode.OK){



switch (resp.StatusCode){

    //process Errors here

    //Note if the error is low level (you never reached the RESTAPI processor the

    //respString is a simple string message

    //if the error is returned by the RESTAPI processor the format is XML


OK, that is enough to get started, in the next post we will turn to more complex REST API GET calls and how to process the XML entities returned from successful Sharepoint REST API calls.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

%d bloggers like this: